Principal Data Engineer
Principal Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
$ 160000 - 200000
1 Vacancy
Job Description
The Principal Software Engineer willbe part of the Data Engineering team under the Data & Analytics group. Data Engineering manages the transition to production for advance machine learning and geolocation algorithms developed by both the Processing Algorithms and Data Science teams. This team also develops and manages scalable data processing platforms for exploratory data analysis and realtime analytics to support our analysts in their geospatial data exploration needs. As a Software Engineer you will be working closely with our scientists to optimize algorithms for lowlatency highly scalable production environments that directly support our customers.
We work in small teams to rapidly prototype and productize new ideas based on handson intheweeds engineering. Youll be responsible for designing and implementing distributed backend software systems. We support a broad range of software applications to accomplish our mission especially favoring Python and C languages for batch processing within cloud deployments (Kubernetes Docker).
Location:This position is hybrid with work from home flexibility.
- Lead to the architecture design and implementation and maintenance of processing and data science algorithms optimizing for scalable lowlatency deployment to a batchprocessing cloud environment
- Write clean efficient and welldocumented Python code to implement data extraction transformation and loading processes
- Work closely with Processing Algorithms & Data Science teams to integrate optimize and deploy stateoftheart algorithms to productionready applications
- Develop maintain and optimize AWSbased ETL solutions leveraging AWS services like Lambda S3 EC2 RDS and others.
- Apply analytical debugging problem solving skills to support and debug dataheavy applications in production to achieve long term product goals in terms of performance and reliability
- Participate in collaborative software development practices particularly performing merge request reviews providing design feedback etc.
- Guide and mentor other individual contributors providing technical leadership code reviews and guidance on best practices.
- Work in a fastpaced agile environment effectively communicate and track development activities using agile tools like JIRA/Confluence.
- Ability to work independently and within a team environment with geographically distributed team members.
Requirements
- Bachelors or Masters degree in Computer Science Electrical/Computer Engineering or comparable experience
- 7 years of professional software development experience using Python
- Strong background in designing and developing Extract Transform and Load (ETL) processes particularly within a cloudnative architecture.
- Extensive experience working in an AWS environment including knowledge of AWS services and solutions (Amazon S3 Amazon EC2 AWS Lambda)
- Experience with modern data orchestration tools (e.g. Apache Airflow AWS Step Functions)
- Experience developing and supporting DevOps bestpractices (e.g. GitLabbased CI/CD)
- Demonstrated experience developing software in a Linux environment
- Experience with Infrastructure as Code (IaC) tools (e.g. Terraform)
- Experience deploying code using serverless cloud technologies such as AWS Lambda
- Experience deploying software and infrastructure into a Docker and/or Kubernetes environment using an industrystandard cloud provider (AWS etc.
- Experience with standard Python tools & framework (e.g. NumPy pandas SciPy scikit)
- Knowledge of data warehousing data lakes and modern data engineering patterns in the cloud
- Experience in developing and managing interactive web applications using technologies such as JavaScript React and Angular. Skilled in backend frameworks like Node.js or Django and familiar with databases such as MySQL and PostgreSQL.
Benefits
Required Experience:
Staff IC
Employment Type
Full-Time
