The Role
We are looking for a Data Engineer with 610 years of experience. Proficient in ETL and Data Modeling responsible for developing databases and provisioning data to support organizational data analytics and insights initiatives. Daytoday management of data pipelines loading and provisioning. Duties include designing building and maintaining batch or realtime data pipelines in production optimizing data infrastructure developing ETL processes for data extraction and preparing raw data for consumption. Experience with onpremises and Cloudbased solutions data modeling and workflow management tools. Advanced SQL skills and expertise in building and optimizing big data data pipelines. ails working closely with business users gathering feedback and providing technical support as required. The ideal person will:
Designing constructing and ensuring the ongoing operation of batch or realtime data pipelines in a production environment.
Managing and finetuning the essential data infrastructure to facilitate accurate data extraction transformation and loading from a wide range of data sources.
Crafting ETL (Extract Transform Load) processes to facilitate data extraction and manipulation from multiple sources including the automation of data workflows like data ingestion aggregation and ETL processing.
Converting raw data stored in Data Warehouses into accessible datasets suitable for both technical and nontechnical stakeholders. Collaborating with data scientists and key functional leaders in sales marketing and product to implement machine learning models for production use.
Possessing a comprehensive understanding of both onpremise and Cloudbased solutions (e.g. AWS Azure GCP) and demonstrating strong proficiency in the creation of data pipelines data modeling and the management of workflow tools such as Airflow and Azkaban.
Demonstrating a proven ability to work effectively both as an independent contributor and as a team player.
Showcasing advanced knowledge of SQL and extensive experience in working with relational databases including query authoring (SQL) along with familiarity with various database systems.
Displaying a track record of building and optimizing data pipelines architectural structures and datasets associated with big data environments.
The Person
Bachelors degree in a relevant field preferably in Computer Science or Information Technology.
A solid track record of 610 years of professional experience in data engineering and data modeling.
Proficiency in the design construction and maintenance of data pipelines for batch and realtime processing in production.
Proven expertise in managing and optimizing data infrastructure to enable accurate data extraction transformation and loading from a diverse range of data sources.
Demonstrated ability to design and implement ETL (Extract Transform Load) processes including the automation of data workflows such as data ingestion aggregation and ETL processing.
Strong data transformation skills and the capacity to convert raw data in Data Warehouses into consumable datasets for technical and nontechnical stakeholders.
Collaborative skills to partner with data scientists and functional leaders in sales marketing and product to deploy machine learning models in production.
A comprehensive understanding of both onpremise and Cloudbased solutions (e.g. AWS Azure GCP).
Proficiency in creating data pipelines data modeling and managing workflow tools like Airflow and Azkaban.
Proven ability to work effectively both as an independent contributor and as part of a team.
Advanced SQL knowledge and extensive experience working with relational databases including query authoring (SQL) and familiarity with a variety of database systems.
A strong track record in building and optimizing big data data pipelines architectural structures and datasets.
Note: Preferred candidates will be in Doha or currently working in Qatar
Our client is an equal opportunity employer and we celebrate diversity and are committed to creating an inclusive environment for all.
Note: This job description is intended to convey information essential to understanding the scope of the position and is not an exhaustive list of skills efforts duties responsibilities or working conditions associated with it.
gcp,azkaban,aws,sql,etl,data extraction,azure,data warehouses,data modeling,airflow,machine learning