Lead Data Engineer
Lead Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Company Overview
Incedo is a US-based consulting data science and technology services firm with over 3000 people helping clients
from our six offices across US Mexico and India. We help our clients achieve competitive advantage through
end-to-end digital transformation. Our uniqueness lies in bringing together strong engineering data science and
design capabilities coupled with deep domain understanding. We combine services and products to maximize
business impact for our clients in telecom Banking Wealth Management product engineering and life science
& healthcare industries.
Working at Incedo will provide you an opportunity to work with industry leading client organizations deep
technology and domain experts and global teams. Incedo University our learning platform provides ample
learning opportunities starting with a structured onboarding program and carrying throughout various stages of
your career. A variety of fun activities is also an integral part of our friendly work environment. Our flexible
career paths allow you to grow into a program manager a technical architect or a domain expert based on your
skills and interests.
Our Mission is to enable our clients to maximize business impact from technology by
- Harnessing the transformational impact of emerging technologies
- Bridging the gap between business and technology
Role Description
As an AWS Data Engineer your role will be to design develop and maintain scalable data pipelines on AWS. You will work closely with technical analysts client stakeholders data scientists and other team members to ensure data quality and integrity while optimizing data storage solutions for performance and cost-efficiency. This role requires leveraging AWS native technologies and Databricks for data transformations and scalable data processing.
Technical Skills
Responsibilities
Lead and support the delivery of data platform modernization projects.
Design and develop robust and scalable data pipelines leveraging AWS native services.
Optimize ETL processes ensuring efficient data transformation.
Migrate workflows from on-premise to AWS cloud ensuring data quality and consistency.
Design automations and integrations to resolve data inconsistencies and quality issues
Perform system testing and validation to ensure successful integration and functionality.
Implement security and compliance controls in the cloud environment.
Ensure data quality pre- and post-migration through validation checks and addressing issues regarding completeness consistency and accuracy of data sets.
Collaborate with data architects and lead developers to identify and document manual data movement workflows and design automation strategies.
Nice-to-have skills
Qualifications
7 years experience with a core data engineering skillset leveraging AWS native technologies (AWS Glue Python Snowflake S3 Redshift).
Experience in the design and development of robust and scalable data pipelines leveraging AWS native services.
Proficiency in leveraging Snowflake for data transformations optimization of ETL pipelines and scalable data processing.
Experience with streaming and batch data pipeline/engineering architectures.
Familiarity with DataOps concepts and tooling for source control and setting up CI/CD pipelines on AWS.
Hands-on experience with Databricks and a willingness to grow capabilities.
Experience with data engineering and storage solutions (AWS Glue EMR Lambda Redshift S3).
Strong problem-solving and analytical skills.
Knowledge of Dataiku is needed
Graduate/Post-Graduate degree in Computer Science or a related field.
Company Value
We value diversity at Incedo. We do not discriminate based on race religion color national origin gender sexual orientation age marital status veteran status or disability status.
Employment Type
Full Time
