Senior Data Engineer, AI Centre of Excellence

The opportunity

As a Senior Data Engineer (ML Ops & Pipelines) at the AI Centre of Excellence (COE) within the Office of the Chief Technology Officer at EY you will play a pivotal role in architecting developing and optimizing scalable data pipelines and MLOps solutions to power advanced AI and machine learning applications. Your expertise will be critical in managing complex data flows orchestrating ETL processes and ensuring the delivery of AI-ready data to support innovative products. You will collaborate closely with AI/ML scientists cloud architects product managers and fullstack engineering teams to deliver high-impact robust and secure data solutions. If you have a passion for building production-grade pipelines using state-of-the-art Azure tools and open-source frameworks we invite you to join our dynamic team.

Your key responsibilities

• Pipeline Architecture & Development: Design build and maintain scalable reliable and high-performance data pipelines using Azure Data Factory Databricks and Spark to support machine learning and analytics workloads.
• ETL & Data Integration: Lead the creation and optimization of ETL processes for structured and unstructured data ensuring data quality lineage and compliance with enterprise standards.
• Feature Store Management: Develop and manage feature stores to streamline the reusability and governance of engineered features for AI and ML models.
• ML Ops and Automation: Implement robust MLOps workflows that enable continuous integration deployment monitoring and retraining of ML models in production environments.
• Collaboration: Partner with data scientists ML engineers and business stakeholders to translate analytic requirements into scalable data solutions and provide guidance on best practices for data engineering in an AI context.
• Communication: Excellent communication skills to collaborate effectively with cross-functional teams including data scientists analysts and business stakeholders.
• Cloud and Security: Ensure secure data movement and storage by applying security best practices and compliance protocols within Azure cloud environments.
• Continuous Improvement: Stay current with advancements in data engineering MLOps and cloud technologies and proactively apply new techniques to enhance existing solutions.

Skills and attributes for success

• Education:Bachelors or Masters degree in Computer Science Information Management Data Engineering or a closely related technical field.
• Experience: 4-6 years of professional experience in data engineering with a strong emphasis on ML Ops data pipelines and large-scale ETL/ELT processes.
• Technical Skills: Advanced proficiency in Azure Data Factory Databricks and Apache Spark for designing and implementing complex data pipelines and distributed data processing jobs; expertise in
• ETL/ELT processes data wrangling and integrating diverse data sources into unified high-quality datasets suitable for AI and analytics applications.
• MLOps and Workflow Management: Hands-on experience with feature store management and supporting ML and AI workflows in production environments; solid understanding of MLOps principles including CI/CD for machine learning automation of model deployment and monitoring practices.
• Programming and Scripting: Proficient in SQL and Python for data transformation orchestration and automation tasks; familiarity with additional programming languages (e.g. Scala R) is a plus.
• Cloud Technologies: Experience with cloud-native tools and services (e.g. Azure AWS Google Cloud) for data storage processing and orchestration.
• Data Security and Compliance: Working knowledge of data security privacy and compliance regulations (e.g. GDPR HIPAA) in cloud-based solutions ensuring data governance and protection.
• Analytical and Problem-Solving Skills: Strong analytical skills to troubleshoot data-related issues and optimize data workflows for performance and efficiency.

What we offer you

The EY benefits package is designed to support your physical emotional financial and social wellbeing. Our extensive benefits include comprehensive medical dental and prescription drug coverage as well as mental health benefits a robust Employee Assistance Program and group savings plans to promote your overall wellbeing. We offer generous time off including personal days vacation days and additional firm-wide holidays along with the option to purchase extra vacation days. Employees can take advantage of EYs exclusive learning programs tailored just for them. We also provide internal opportunities for career development and advancement enabling you to grow within the firm. Get involved in meaningful volunteering through EY Ripples and make a positive impact in the community.

Learn about our commitment to DE&I at you ready to shape your future with confidence Apply today.

To help create the best experience during the recruitment process please describe any accommodations you may need.

Diversity and Inclusion at EY

Diversity and inclusiveness are at the heart of who we are and how we work. Were committed to fostering an environment where differences are valued policies and practices are equitable and our people feel a sense of belonging. We embrace diversity and are committed to combating systemic racism advancing gender equity and women in leadership advocating for the 2SLGBTQIA community promoting our neuroinclusion and accessibility initiatives and are dedicated to amplifying the voices of Indigenous peoples (First Nations Inuit and Mtis) nationally as we strive towards reconciliation. Our diverse experiences abilities backgrounds and perspectives make our people unique and help guide us. Because when people feel free to be their authentic selves at work they bring their best and are empowered to build a better working world.