Lead Data Engineer
Share
Job Location:
Newark, DE - USA
Monthly Salary:
Not Disclosed
Posted on:
18 hours ago
Vacancies:
1 Vacancy
Job Summary
Lead Data Engineer(Microsoft Fabric)
Location NewarkNJ
This role requires core experience and expertise on - Microsoft Fabric PySpark SQL Data modeling (dimensional & transactional)ETL/ELT pipeline design Data validation & data quality frameworks Azure Integration
Job purpose
This position is part of the Enterprise Data & Analytics Capability team under the Global Technology this role you will lead the design development and optimization of large-scale data solutions on the Databricks platform
Key responsibilities
Design build and maintain scalable data pipelines on Databricks (using Spark Delta Lake etc.)
Write clean efficient and maintainable PySpark or SQL code for data transformation
Design robust data models for analytics and reporting
Ensure data quality consistency and governance
Handle batch and streaming data workflows
Provide architectural guidance and support in platform usage
Drive best practices in data engineering across the team
Monitor and optimize performance of Spark jobs and cluster usage
Ensure compliance with security and data privacy standards
Key competencies
Essential skills
Bachelors degree in Computer Science Engineering or a related field
Minimum of 5 years programming experience which includes at least one year working with a big data platform; experience in data engineering domain Python SQL and cloud platforms such as Azure
Familiarity with relevant systems tools languages and business domain which includes Data Lakehouse principles relational and Kimball data models (required)
Experience with CI/CD pipelines and version control tools (required)
Knowledge of data visualization tools and BI platforms (preferred)
Certification in Databricks or relevant cloud platforms (preferred)
Good communication (verbal and written)
Experience in managing client stakeholders
