Java Lead Software Engineer

Chennai - India

Monthly Salary: Not Disclosed

Posted on: 30+ days ago

Vacancies: 1 Vacancy

Job Summary

Exp: 6 - 9 years

Design develop and maintain efficient and reliable data pipelines using Java Scala Apache Spark and Confluent Cloud (Kafka KStreams kSQLDB Schema Registry)
Leverage Apache Spark (Java/Scala) for large-scale data processing and transformation.
Experience with building maintaining and debugging applications and data pipelines using Confluent Cloud (Kafka KStreams kSQLDB Schema Registry).
Build and optimize data storage solutions using NoSQL databases such as ScyllaDB and/or Cassandra.
Experienced with AWS services required for Data Engineering such as EMR ServerlessEMR AWS Glue CodeCommit EC2 S3 etc.
Familiarity with workflow orchestration tools such as Airflow
Experience with building and deploying applications using Docker or AWS ECS or AWS EKS
Well versed with code management using tools like GitHub and CI/CD pipelines and deployment of data pipelines on AWS cloud.
Implement and manage search and analytics capabilities using AWS OpenSearch and/or Elasticsearch.
Collaborate with data scientists analysts and other engineers to understand data requirements and deliver effective solutions.
Monitor and troubleshoot data pipelines to ensure data quality and performance.
Implement data governance and data quality best practices.
Automate data ingestion processing and deployment processes.
Stay up-to-date with the latest data engineering trends and technologies.
Contribute to the design and architecture of our data platform on AWS.

Exp: 6 - 9 yearsDesign develop and maintain efficient and reliable data pipelines using Java Scala Apache Spark and Confluent Cloud (Kafka KStreams kSQLDB Schema Registry)Leverage Apache Spark (Java/Scala) for large-scale data processing and transformation.Experience with building maintaining and de...