Lead Data Engineer
Share
Job Location:
Monthly Salary:
Experience Required:
Posted on:
4 days ago
Vacancies:
1 Vacancy
Job Summary
We are looking for a highly skilled Lead Data Engineer / Architect to design build and optimize scalable data pipelines and modern data platforms. This role is critical in transforming raw data into meaningful insights and enabling enterprise-level analytics.
Key Responsibilities
- Design develop and optimize ETL/ELT pipelines using Python PySpark Scala and SQL
- Build and manage data workflows using Informatica Databricks AWS Glue Snowflake
- Work across multi-cloud platforms (AWS Azure GCP Microsoft Fabric)
- Manage and optimize databases such as Redshift SQL Server AWS RDS
- Orchestrate workflows using Apache Airflow
- Implement real-time streaming solutions using Apache Kafka
- Deploy and manage containerized applications using Kubernetes
- Automate processes using Unix shell scripting
- Implement CI/CD pipelines and ensure data governance
- Collaborate with BI teams for Tableau Looker Power BI
- Troubleshoot and optimize performance of pipelines and queries
- Maintain technical documentation
Requirements
- Bachelors/Masters degree in Computer Science or related field
- 7 years of experience in Data Engineering
- Strong skills in Python PySpark Scala SQL
- Experience with Informatica Databricks AWS Glue
- Expertise in Snowflake & Redshift
- Experience in AWS / Azure / GCP
- Experience with Apache Airflow
- Knowledge of Apache Kafka & Kubernetes
- Experience with CI/CD pipelines
- Exposure to BI tools (Tableau Looker Power BI)
Required Skills:
Python PySpark Scala SQL AWS Azure GCP Microsoft Fabric Snowflake & Redshift Kafka
