AI Data Engineer 3
Share
Job Location:
Monthly Salary:
Posted on:
2 hours ago
Vacancies:
1 Vacancy
Job Summary
Summary:
The main function of an AI Data Engineer is to construct and maintain data pipelines ETL processes and data storage systems to facilitate efficient data collection processing and analysis. They ensure data quality and accessibility for data-driven decision-making collaborating closely with data scientists and analysts.
Job Responsibilities:
As a Data Engineer in MDO you will drive critical initiatives to help our data platform scale to the needs of business priorities and technology advancements
Build scalable data models and data pipelines to extract load and transform data ensure secure data storage with focus on data quality and compliance on Azure using services (ADF HDI Databricks Synapse and Fabric etc.)
You will play a critical role in developing and building datasets and integrate with AI/ML and Co-Pilot applications
Collaborate with data scientists analysts and other stakeholders to understand data requirements and deliver solutions
Implement and manage CI/CD(YAML Classic ) pipelines for data engineering projects leveraging tools like Azure DevOps
You will anticipate data governance needs designing data modeling and handling procedures to ensure compliance with all applicable laws and policies. Youll also govern data accessibility within your assigned pipelines.
Acts as a Designated Responsible Individual (DRI) and guides other engineers by developing and following the playbook working on call to monitor system/product/service for degradation downtime or interruptions alerting stakeholders about status and initiates actions to restore system/product/service for simple and complex problems when appropriate.
Qualifications:
Bachelors degree in computer science Engineering or a related field
OR Masters Degree in Computer Science Math Software Engineering Computer Engineering or related field
OR equivalent experience
Experience with building data pipelines and data stores in Azure PySpark Fabric
5 years of experience in data engineering or a similar role.
Proficiency in programming languages such as Python Scala or Java.
Strong experience with SQL and database technologies
Familiarity with big data technologies (e.g. Big Data Spark Python Azure tech Stack Fabric).
The main function of an AI Data Engineer is to construct and maintain data pipelines ETL processes and data storage systems to facilitate efficient data collection processing and analysis. They ensure data quality and accessibility for data-driven decision-making collaborating closely with data scientists and analysts.
Job Responsibilities:
As a Data Engineer in MDO you will drive critical initiatives to help our data platform scale to the needs of business priorities and technology advancements
Build scalable data models and data pipelines to extract load and transform data ensure secure data storage with focus on data quality and compliance on Azure using services (ADF HDI Databricks Synapse and Fabric etc.)
You will play a critical role in developing and building datasets and integrate with AI/ML and Co-Pilot applications
Collaborate with data scientists analysts and other stakeholders to understand data requirements and deliver solutions
Implement and manage CI/CD(YAML Classic ) pipelines for data engineering projects leveraging tools like Azure DevOps
You will anticipate data governance needs designing data modeling and handling procedures to ensure compliance with all applicable laws and policies. Youll also govern data accessibility within your assigned pipelines.
Acts as a Designated Responsible Individual (DRI) and guides other engineers by developing and following the playbook working on call to monitor system/product/service for degradation downtime or interruptions alerting stakeholders about status and initiates actions to restore system/product/service for simple and complex problems when appropriate.
Qualifications:
Bachelors degree in computer science Engineering or a related field
OR Masters Degree in Computer Science Math Software Engineering Computer Engineering or related field
OR equivalent experience
Experience with building data pipelines and data stores in Azure PySpark Fabric
5 years of experience in data engineering or a similar role.
Proficiency in programming languages such as Python Scala or Java.
Strong experience with SQL and database technologies
Familiarity with big data technologies (e.g. Big Data Spark Python Azure tech Stack Fabric).
