Principal Consultant ADF Airflow Engineer

Ready to build the future with AI
At Genpact we dont just keep up with technologywe set the pace. AI and digital innovation are redefining industries and were leading the charge. Genpacts AI Gigafactory our industry-first accelerator is an example of how were scaling advanced technology solutions to help global enterprises work smarter grow faster and transform at scale. From large-scale models to agentic AI our breakthrough solutions tackle companies most complex challenges.
If you thrive in a fast-moving innovation-driven environment love building and deploying cutting-edge AI solutions and want to push the boundaries of whats possible this is your moment.
Genpact (NYSE: G) is an advanced technology services and solutions company that delivers lasting value for leading enterprises globally. Through our deep business knowledge operational excellence and cutting-edge solutions we help companies across industries get ahead and stay ahead. Powered by curiosity courage and innovation our teams implement data technology and AI to create tomorrow today. Get to know us at and on LinkedIn X YouTube and Facebook.
Inviting applications for the role of PrincipalConsultant ADF and Airflow Engineer
In this role 2 lines about the role

Responsibilities
Data Pipeline Architecture & Development:
Design build and maintain scalable and reliable ETL/ELT pipelines using Azure Data Factory (ADF) to ingest process and transform large volumes of structured and unstructured data.
Develop and implement complex data transformation logic and workflows using Python PySpark and SQL within the Azure ecosystem (e.g. Azure Databricks Azure Synapse Analytics).
Utilize Apache Airflow for the robust orchestration scheduling and monitoring of these complex data pipelines and workflows.
Performance Optimization & Tuning:
Optimize data processing jobs and queries for performance scalability and cost-effectiveness in Azure environments.
Monitor pipeline performance troubleshoot failures and implement improvements to ensure high efficiency and reliability of data systems.
Perform advanced SQL tuning and work with big data technologies to manage and process large datasets efficiently.
Data Governance & Quality Assurance:
Implement and enforce data quality checks governance standards and security best practices (e.g. encryption access control) across all data pipelines and storage solutions.
Manage and optimize data storage solutions including Azure Data Lake Storage (ADLS) Azure SQL Database and Azure Synapse.
Collaboration & Participation:
Collaborate with data scientists analysts and business stakeholders to understand data requirements and translate them into effective technical solutions.
Actively participate in solution design and architecture discussions taking end-to-end ownership of data engineering initiatives.
DevOps & Automation:
Implement CI/CD pipelines for deploying ADF pipelines and Airflow DAGs (Directed Acyclic Graphs) using tools like Azure DevOps or Jenkins.
Automate data workflows error handling and alerting mechanisms for production systems.
Qualifications we seek in you!
Minimum Qualifications
ohands-on/project experience in data engineering specifically in data ingestion integration transformation and pipeline orchestration. Used ADF and Airflow at a greater scale in multiple projects.
oA bachelors degree or above in computer science Information Technology Engineering or equivalent
oIn-depth knowledge of Azure Data Factory (ADF) capabilities including designing developing and deploying complex ETL/ELT pipelines linked services datasets and triggers.
oStrong Proficiency in Apache Airflow for designing developing and maintaining scalable data pipelines and Directed Acyclic Graphs (DAGs).
oStrong proficiency in Python Pyspark for data processing automation and general programming tasks along with expert-level SQL knowledge (including query optimization for large datasets).
oSignificant experience with the Azure cloud ecosystem including services such as Azure Data Lake Storage (ADLS) Azure Synapse Analytics Azure Databricks Azure SQL Database and Cosmos DB.
oSolid understanding of data warehousing data modelling database management and ETL/ELT methodologies.
oStrong working knowledge of code management (Git/GitHub) and continuous integration/continuous deployment (CI/CD) pipelines using tools like Azure DevOps
oAbility to work independently or cross-functionally and communicate effectively with technical and non-technical stakeholders (e.g. analysts business teams) is highly valued.
oOwning and designing end-to-end scalable data architectures and complex pipelines.
oMonitoring troubleshooting and optimizing data pipelines and platform performance.
oCollaborating with cross-functional teams (analysts data scientists business teams) to gather requirements and deliver solutions.
oEnforcing data governance standards and best practices.

Preferred Qualifications/ Skills
oExperience with other languages like Scala or Java is also beneficial for big data environments.
oHands-on experience with Databricks for data processing and running data pipelines is a recurring requirement.
oExperience with ADLS and Azure Synapse Analytics is crucial for comprehensive data solutions.
oFamiliarity with Azure Key Vault Azure Logic Apps Event Grid and Azure Purview
oKnowledge of data governance standards security protocols data quality implementation lineage and access control is a preferred qualification
oGood understanding of the insurance domain and its functionality.
oGood to have MS Azure DP-203 certified

Why join Genpact
Lead AI-first transformation Build and scale AI solutions that redefine industries
Make an impact Drive change for global enterprises and solve business challenges that matter
Accelerate your careerGain hands-on experience world-class training mentorship and AI certifications to advance your skills
Grow with the best Learn from top engineers data scientists and AI experts in a dynamic fast-moving workplace
Committed to ethical AI Work in an environment where governance transparency and security are at the core of everything we build
Thrive in a values-driven culture Our courage curiosity and incisiveness - built on a foundation of integrity and inclusion - allow your ideas to fuel progress
Come join the 140000 coders tech shapers and growth makers at Genpact and take your career in the only direction that matters: Up.
Lets build tomorrow together.

Genpact is an Equal Opportunity Employer and considers applicants for all positions without regard to race color religion or belief sex age national origin citizenship status marital status military/veteran status genetic information sexual orientation gender identity physical or mental disability or any other characteristic protected by applicable laws. Genpact is committed to creating a dynamic work environment that values respect and integrity customer focus and innovation.
Furthermore please do note that Genpact does not charge fees to process job applications and applicants are not required to pay to participate in our hiring process in any other way. Examples of such scams include purchasing a starter kit paying to apply or purchasing equipment or training.