Job Opening: Data Engineer (Scala Spark Java Kafka SQL)
About the Client:
Our client is a leading global Fortune 500 IT solutions company that specializes in providing straightforward and scalable solutions to solve intricate business challenges. With a team of over 1500 professionals they offer technical and domain expertise across various platforms and industries to assist enterprise companies in enhancing productivity efficiency and optimizing their technology investments.
Responsibilities:
1. Design and develop scalable data pipelines using Scala Java and Spark
2. Build and manage realtime data streaming solutions with Apache Kafka
3. Write optimized SQL queries for data transformation and reporting
4. Implement and maintain ETL workflows for batch and streaming data
5. Ensure data quality integrity and performance in all engineering processes
6. Collaborate with data scientists analysts and DevOps teams for solution delivery
7. Work in agile teams to support ongoing data platform enhancements
8. Optimize and troubleshoot data processes for speed and efficiency
Location:
Bangalore/Hyderabad
Notice Period:
Immediate to 45 days only
Requirements
Requirements:
1. Strong experience in big data engineering with handson expertise in Scala and Java.
2. Proficient in developing data pipelines and distributed processing using Apache Spark.
3. Solid experience in streaming data platforms particularly Apache Kafka.
4. Expertise in SQL for querying and transforming largescale datasets.
5. Ability to design build and maintain ETL pipelines and batch/streaming workflows.
6. Familiarity with data lake warehouse and cloud platforms (AWS Azure or GCP).
7. Strong understanding of data modeling partitioning and performance tuning.
8. Experience in working with Agile methodologies and collaborative team environments.
9. Excellent problemsolving skills and ability to optimize data processes for scalability and reliability.
1. Strong experience in big data engineering, with hands-on expertise in Scala and Java. 2. Proficient in developing data pipelines and distributed processing using Apache Spark. 3. Solid experience in streaming data platforms, particularly Apache Kafka. 4. Expertise in SQL for querying and transforming large-scale datasets. 5. Ability to design, build, and maintain ETL pipelines and batch/streaming workflows. 6. Familiarity with data lake, warehouse, and cloud platforms (AWS, Azure, or GCP). 7. Strong understanding of data modeling, partitioning, and performance tuning. 8. Experience in working with Agile methodologies and collaborative team environments. 9. Excellent problem-solving skills and ability to optimize data processes for scalability and reliability
Education
Any graduate or post graduate