Data Engineer Lead
Data Engineer Lead
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Not Disclosed
Salary Not Disclosed
1 Vacancy
Job Description
Title: Data Engineer Lead
Location: San Jose CA (local resource needed)
Overview:
We are looking for a skilled Technical Lead to oversee the design and implementation of large-scale data engineering and migration projects. The ideal candidate will have hands-on expertise in Databricks Azure Data Factory and related technologies with a proven track record of leading at least 3 successful data projects.
Experience Required:
Minimum of 12 years of experience in data engineering and architecture roles.
Must have led 3 or more end-to-end enterprise data projects Databricks and Azure technologies.
Primary Skills:
Deep knowledge of architecture and complex data models
Extensive experience with Databricks PySpark Spark SQL and Delta Lake.
Strong command over ETL tools such as Data Services and Azure Data Factory.
Proficient in programming languages: Python Java or Scala.
Hands-on with Azure cloud services including networking and security configurations.
Key Responsibilities:
1. Lead the design and implementation of ETL solutions using Azure Data Factory.
2. Guide and mentor team members on data engineering best practices.
3. Develop and maintain complex ETL pipelines using Python Java or Scala.
4. Ensure high-quality secure and performant data migrations using Azure Blob Storage and Spark JDBC.
5. Drive collaboration across teams to implement advanced analytics and ML models in Databricks.
6. Identify and resolve performance bottlenecks and network-related issues.
7. Ensure adherence to data governance and compliance standards across all data flows.
Secondary Skills:
Understanding of Databricks ML and analytics tools.
Experience resolving networking/VPN issues related to data flow.
Familiarity with data governance security and compliance frameworks.
Experience in setting up CI/CD pipelines and using DevOps tools is a plus.
Employment Type
Full-time
