Lead AI Data Engineer

Bangalore - India

Monthly Salary: Not Disclosed

Posted on: 30+ days ago

Vacancies: 1 Vacancy

Department:

Engineering

Job Summary

Lead AI Data Engineer

Location: Bengaluru Karnataka India

About the Role:

Were looking for an experienced AI Data Engineer (4-8 years) to join our data this role youll build and maintain our data infrastructure on AWS enabling analytics and AI teams to extract actionable insights. Youll design and manage end-to-end data pipelines ensuring high-quality reliable and real-time data while also contributing to ML/GenAI workflows and model deployment pipelines.

What Youll Do:

Design and build scalable data pipelines/transformations using Spark / PySpark / Scala.
Manage and optimize Airflow DAGs for complex data workflows.
Clean transform and prepare data for analytics AI and ML use cases.
Use Python for automation data processing and internal tooling.
Work with AWS services (S3 Redshift EMR Glue Athena) to maintain robust data infrastructure.
Collaborate with Analytics and AI teams to design pipelines for ML/GenAI projects.
Contribute to (TypeScript) backend development for data services.
Automate deployments using CI/CD pipelines (GitHub Actions).
Monitor troubleshoot and ensure data quality consistency and reliability across systems.
Build and maintain data warehouses/lakes and handle real-time streaming data using Kafka or similar technologies.

What Youll Need:

Bachelors or Masters in Computer Science Engineering or related field.
4-8 years of hands-on experience in data engineering.
Strong expertise in Spark / Scala for large-scale data processing.
Proficient in Airflow for managing and optimizing complex DAGs.
Advanced Python skills for data manipulation automation and tool development.
Proven experience with AWS related cloud services (S3 Redshift EMR Glue Athena IAM EC2).
Solid understanding of ETL/ELT data preparation and analytics workflows.
Familiar with and TypeScript for backend data services.
Experience with automated CI/CD (GitHub Actions).
Familiarity with CDC Tools like Debezium.
Strong SQL knowledge of data warehousing and streaming (Kafka Flink Kinesis) and excellent communication skills.

Bonus Points:

Experience with data lake technologies (Delta Lake Apache Iceberg).
Knowledge of ML/GenAI model deployment pipelines.
Familiarity with data governance quality frameworks and statistics.
Experience with infrastructure as code (Terraform).
Familiarity with containers (Docker Kubernetes).
Experience with monitoring and logging tools (Prometheus Grafana).

Lead AI Data EngineerLocation: Bengaluru Karnataka IndiaAbout the Role:Were looking for an experienced AI Data Engineer (4-8 years) to join our data this role youll build and maintain our data infrastructure on AWS enabling analytics and AI teams to extract actionable insights. Youll design and man...

Lead AI Data Engineer

Location: Bengaluru Karnataka India

About the Role:

Were looking for an experienced AI Data Engineer (4-8 years) to join our data this role youll build and maintain our data infrastructure on AWS enabling analytics and AI teams to extract actionable insights. Youll design and manage end-to-end data pipelines ensuring high-quality reliable and real-time data while also contributing to ML/GenAI workflows and model deployment pipelines.

What Youll Do:

Design and build scalable data pipelines/transformations using Spark / PySpark / Scala.
Manage and optimize Airflow DAGs for complex data workflows.
Clean transform and prepare data for analytics AI and ML use cases.
Use Python for automation data processing and internal tooling.
Work with AWS services (S3 Redshift EMR Glue Athena) to maintain robust data infrastructure.
Collaborate with Analytics and AI teams to design pipelines for ML/GenAI projects.
Contribute to (TypeScript) backend development for data services.
Automate deployments using CI/CD pipelines (GitHub Actions).
Monitor troubleshoot and ensure data quality consistency and reliability across systems.
Build and maintain data warehouses/lakes and handle real-time streaming data using Kafka or similar technologies.

What Youll Need:

Bachelors or Masters in Computer Science Engineering or related field.
4-8 years of hands-on experience in data engineering.
Strong expertise in Spark / Scala for large-scale data processing.
Proficient in Airflow for managing and optimizing complex DAGs.
Advanced Python skills for data manipulation automation and tool development.
Proven experience with AWS related cloud services (S3 Redshift EMR Glue Athena IAM EC2).
Solid understanding of ETL/ELT data preparation and analytics workflows.
Familiar with and TypeScript for backend data services.
Experience with automated CI/CD (GitHub Actions).
Familiarity with CDC Tools like Debezium.
Strong SQL knowledge of data warehousing and streaming (Kafka Flink Kinesis) and excellent communication skills.

Bonus Points: