Data Engineer Data Platforms
Data Engineer Data Platforms
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Location: Client Office Work from Office (Day 1 Mandatory)
Total Experience: 6 8 Years
Relevant Experience: 6 8 Years
Mode of Interview: Face-to-Face (Mandatory)
Employment Type: Full-time
-
Work from office at client location is mandatory from Day 1.
-
Ensure basic validation and candidate interest check is completed before profile submission.
-
Relevant experience will determine candidate band; profiles with lesser experience will not be considered for higher bands.
-
Full BGV (Background Verification) must be completed before onboarding.
-
Candidate must provide a 6-month NOC from DOJ as regularization may be considered post 6 months.
-
Airflow (v2.x)
-
Python & Bash scripting
-
Cloudera CDP
-
Spark or Hive-based data pipelines
-
Kafka REST APIs and database integration
-
CI/CD implementation for Airflow DAGs
-
Lead the design development and maintenance of ETL/ELT workflows using Apache Airflow.
-
Schedule orchestrate and monitor DAGs ensuring performance reliability and recoverability.
-
Integrate Airflow with Cloudera services Kafka REST APIs and enterprise databases.
-
Implement robust error handling logging retry policies and alerting mechanisms.
-
Collaborate with cross-functional teams including data engineers DevOps and analytics stakeholders.
-
Ensure all pipelines adhere to performance and data governance standards.
-
6 8 years of relevant experience with strong hands-on expertise in Airflow.
-
Excellent Python and Bash scripting skills.
-
Proven ability to integrate Airflow in a Cloudera-based data ecosystem.
-
Experience working with big data platforms and handling streaming and batch pipelines.
-
Strong problem-solving and collaboration abilities.
-
Ready for immediate on-premises onboarding and face-to-face evaluation process.
Employment Type
Full Time
