Lead Data Engineer

Job Responsibilities

Responsible for leading a team of talented data engineers responsible for designing building and maintaining scalable data pipelines and infrastructure Work closely with cross-functional teams to ensure client data systems meet the highest standards of quality and performance Lead mentor and develop a team of data engineers fostering a collaborative and inclusive team environment Conduct regular performance reviews provide feedback and set goals for team members Identify and address skill gaps and provide opportunities for professional development Plan execute and deliver data engineering projects on time and within scope Coordinate with stakeholders to gather requirements set priorities and define project timelines. Ensure projects align with overall business objectives and data strategy Oversee the design development and maintenance of data pipelines ETL processes and data warehouse Ensure data quality integrity and security across all data engineering projects. Identify opportunities for process improvements and drive initiatives to enhance the efficiency and effectiveness of data operations. Has strong conceptual understanding of Data Warehousing and ETL Data Governance and Security Cloud Computing and Batch & Real Time data processing Ability to build/drive reusable frameworks that can drive efficiency of the overall data system Has executed and lead multiple projects including on - streaming batch large data pipelines etc. Manages conversation with the client stakeholders to understand the Internal Use Only requirement and translate it into technical outcomes.

Required Tech Stack

Strong experience with Databricks Spark and cloud platforms (Azure AWS GCP). Architect and deploy cloud-based data solutions (Azure AWS GCP). Define CI/CD strategies for data pipelines using Terraform Azure DevOps or GitHub Actions. Implement data cataloging lineage tracking and access control (Unity Catalog Collibra Alation). Ensure compliance with GDPR CCPA and industry-specific data security policies. Develop strategies for distributed computing parallel processing and caching mechanisms.

Required Experience:

Senior IC