Senior Data Engineer
Share
Job Location:
Monthly Salary:
Posted on:
3 hours ago
Vacancies:
1 Vacancy
Job Summary
Role: Senior Data Engineer
Experience: 5-8 Years
Location: Chennai Hyderabad Bangalore Pune Noida Gurugram Mumbai
Work Mode: Hybrid
Job Description:We are seeking a Senior Data Engineer who combines technical engineering prowess with a strong analytical this role you will be responsible for building automating and validating scalable data pipelines while performing deep-dive data analysis to ensure the reliability and business value of our data platform.
Key Skills:
- Big Data - Data Engineer
- Spark (Scala/Python)
- Azure
- Kafka/Streaming
- IaC (Terraform)/CI CD
Key Responsibilities:
Pipeline Engineering:Design and optimize high-performance data processing solutions using Azure Databricks (Spark Delta Lake Unity Catalog).
Data Analysis:Perform thorough data analysis to identify trends ensure data integrity and support stakeholder reporting needs.
CI/CD & IaC:Implement and manage automated deployment pipelines using GitHub Actions and Jenkins while provisioning Azure resources via Terraform.
Testing & Validation:Own the reliability of the codebase by writing comprehensive unit tests(ScalaTest/Pytest) and conducting end-to-end (E2E) validations.
Development:Write clean production-grade code in both Scala and Python.
Operations: Support the DevOps/Operationslifecycle including production monitoring performance tuning and incident resolution.
Required Qualifications:
Experience:At least 5 years in a Data Engineering Data Analysis or DevOps role.
Tools:Expert knowledge of Azure Databricks Terraform Jenkins and GitHub Actions.
Languages:Strong hands-on experience in Scalaand Python.
Soft Skills: Excellent verbal and written communication; ability to translate technical tasks into business insights.
Testing: Demonstrated ability to build automated testing frameworks for ETL/ELT pipelines
Required Skills:
ScalaPythonAzureKafka
