Lead Data Engineer

Charlotte, VT - USA

Monthly Salary: Not Disclosed

Posted on: 5 hours ago

Vacancies: 1 Vacancy

Job Summary

Were seeking an experienced data engineer to analyze large volume of different data set identify the data gaps and address the same. Build automated future ready data reconciliation process in-place.

Technical Skills:

Lead the Analysis of Data Sources ETL Processes and Databases. Identify data and process gaps and recommend data remediation and long-term process improvements.
Utilize deep expertise in various ETL tools e.g. IBM DataStage) RDBMS (e.g. DB2 PostgreSQL etc. and different data formats e.g. Complex XML JSON CSV to drive analysis.
Leverage expertise in understanding and testing REST/SOAP Integrations.
Apply Python expertise to build tools for comparing data between heterogeneous sources CSV XML JSON RDBMS etc. and automating the data validation process.
Demonstrate Strong Business Acumen and Stakeholder Communication Skills.
Translate complex business challenges into clear data-driven solutions effectively communicating technical concepts and project progress to both technical and non-technical stakeholders including senior leadership.
Act as a key liaison between the data team and business units providing data-backed insights and recommendations that directly influence business strategy.
Manage concurrent projects in a dynamic research-oriented environment ensuring timely delivery and high-quality outcomes.
Experience in the insurance domain is preferable.
AWS Data Engineer certification is good to have.

Key Responsibilities

Collaborate with business units and stakeholders to document business needs processes reporting requirements and challenges.
Lead the design and development of robust data pipelines utilizing deep understanding of existing ETL frameworks to build scalable and efficient solutions.
Analyze and understand diverse source systems and data formats to create and validate synthetic datasets for testing and development.
Develop automated data validation tools using Python to ensure data integrity across different datasets and generate comparison output reports.
Ensure industry best practices are followed in all aspects of the project.
Develop a robust development testing approach to minimize quality integration and user testing issues.
Leverage generative AI to enhance data analysis accelerate development streamline unit testing and improve documentation.
Apply statistical analysis to large datasets uncovering key patterns identifying potential challenges and generating actionable insights.
Partner with business leaders to understand their challenges and provide data-driven recommendations that improve processes and inform strategic decisions.

Qualification:

Somebody who has at least 12 years of data engineering experience has played Lead Data Engineer role.
Bachelors degree (or equivalent) in computer science information technology engineering or related discipline
Education qualification: Any degree from a reputed college

Were seeking an experienced data engineer to analyze large volume of different data set identify the data gaps and address the same. Build automated future ready data reconciliation process in-place. Technical Skills: Lead the Analysis of Data Sources ETL Processes and Databases. Identify data and ...

Were seeking an experienced data engineer to analyze large volume of different data set identify the data gaps and address the same. Build automated future ready data reconciliation process in-place.

Technical Skills:

Lead the Analysis of Data Sources ETL Processes and Databases. Identify data and process gaps and recommend data remediation and long-term process improvements.
Utilize deep expertise in various ETL tools e.g. IBM DataStage) RDBMS (e.g. DB2 PostgreSQL etc. and different data formats e.g. Complex XML JSON CSV to drive analysis.
Leverage expertise in understanding and testing REST/SOAP Integrations.
Apply Python expertise to build tools for comparing data between heterogeneous sources CSV XML JSON RDBMS etc. and automating the data validation process.
Demonstrate Strong Business Acumen and Stakeholder Communication Skills.
Translate complex business challenges into clear data-driven solutions effectively communicating technical concepts and project progress to both technical and non-technical stakeholders including senior leadership.
Act as a key liaison between the data team and business units providing data-backed insights and recommendations that directly influence business strategy.
Manage concurrent projects in a dynamic research-oriented environment ensuring timely delivery and high-quality outcomes.
Experience in the insurance domain is preferable.
AWS Data Engineer certification is good to have.

Key Responsibilities

Collaborate with business units and stakeholders to document business needs processes reporting requirements and challenges.
Lead the design and development of robust data pipelines utilizing deep understanding of existing ETL frameworks to build scalable and efficient solutions.
Analyze and understand diverse source systems and data formats to create and validate synthetic datasets for testing and development.
Develop automated data validation tools using Python to ensure data integrity across different datasets and generate comparison output reports.
Ensure industry best practices are followed in all aspects of the project.
Develop a robust development testing approach to minimize quality integration and user testing issues.
Leverage generative AI to enhance data analysis accelerate development streamline unit testing and improve documentation.
Apply statistical analysis to large datasets uncovering key patterns identifying potential challenges and generating actionable insights.
Partner with business leaders to understand their challenges and provide data-driven recommendations that improve processes and inform strategic decisions.

Qualification:

Somebody who has at least 12 years of data engineering experience has played Lead Data Engineer role.
Bachelors degree (or equivalent) in computer science information technology engineering or related discipline
Education qualification: Any degree from a reputed college