Senior Data Engineer
Senior Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
We are seeking a highly skilled Senior Data Engineer to design develop and maintain robust data pipelines that extract transform and load data from disparate applications into our Azurebased data platform. The ideal candidate will have a strong foundation in data engineering principles a deep understanding of cloud technologies and a passion for solving complex data challenges.
Roles and Responsibilities
Design and develop scalable and efficient data pipelines using Azure Data Factory Azure Databricks Kafka and other relevant tools.
Extract data from various applications using APIs webhooks or other methods.
Ingest and process realtime data streams using Kafka.
Transform and clean data to ensure data quality and consistency.
Load data into ADLS Gen2 in optimized formats (e.g. Parquet Delta Lake).
Build and maintain data models and schemas.
Collaborate with data analysts and business stakeholders to understand data requirements and deliver actionable insights.
Optimize data pipelines for performance and costefficiency.
Implement data governance and security best practices.
Mentor junior data engineers and provide technical guidance.
Skills and Qualifications
Strong proficiency in Python Scala or Java programming languages.
Expertise in data engineering tools and technologies such as Azure Data Factory Azure Databricks ADLS Gen2 Kafka and Delta Lake.
Experience with cloud computing platforms preferably Azure.
Knowledge of data modelling ETL/ELT processes data warehousing and realtime data processing concepts.
Strong problemsolving and analytical skills.
Ability to work independently and as part of a team.
Excellent communication and interpersonal skills.
Bachelors degree in computer science engineering or a related field.
Preferred Qualifications
Knowledge of machine learning and AI concepts.
Certifications in Azure or related technologies.
Experience with Node.js Java or .NET Core for developing connector APIs.
MustHave Technologies
Python
Scala or Java
Azure Data Factory
Azure Databricks
ADLS Gen2
Kafka
Delta Lake
SQL
This role offers a unique opportunity to work on challenging data engineering projects and contribute to the growth of our datadriven organization.
Per Month INR
Employment Type
Full Time
