Data Engineer Azure Data Marketplace Platform

Data Engineer - Azure Data Marketplace Platform

Location:Warsaw

Contract Type:3 Months
Job type:Hybrid

• Work oncutting-edgeAI data marketplace technology at Manager level

• Build world-class data products using Azure Data Factory and Databricks

• Join cross-functional teams focused on stabilityoptimisationand innovation

• Leverage 8 years of experience in a role that values resilience and scalability

Company Overview

Our client is a global professional servicesorganisationoperatingat the forefront of data engineering and AI technology. They are building an advanced data marketplace platform designed to deliver robust scalable data assets to consumers across the business. With a focus on innovationcollaborationand technical excellence they provide an environment where experienced engineers can contribute to meaningful projects that shape the future of data-driven decision making.

They are seeking an experienced Data Engineer at Manager level to join their team in Warsaw. This contract role offers the opportunity to work on sophisticated data ingestion pipelines within their AI Data Marketplace collaborating with talented professionals to build resilient fault-tolerant systems.

Position Overview

As a Data Engineer you will playa central rolein developing andmaintainingthe data ingestion pipelines that power their AI Data Marketplace platform. You will work closely with cross-functional teams to design build andoptimisedata assets ensuring stabilitytraceabilityand high availability. Yourexpertisein Azure technologies big dataprocessingand ETL will directly contribute to delivering world-class data products and services to internal consumers.

Responsibilities

• Identifyopportunities to improve data assets and services within the AI Data Marketplace

• Design and develop data ingestion pipelines with focus on stability andoptimisation

• Buildtestandmaintainrobust data pipelines and processes

• Write resilient fault-tolerantand modular code for high availability systems

• Research and implement new methods to enhance data products

• Collaborate across teams to ensure smooth transitions through ingestion and consumption phases

• Manage Azure Data Factory pipelines activitiestriggersand data flows

• Process large datasets using Azure Databricks ApacheSparkandPySpark

• Transform and integrate data from various sources using ETL processes

Requirements

• 8-10 years of experience as a Data Engineer

• Expert knowledge of Azure Data Factory including pipeline development and management

• Proficiencyin Azure Databricks for data processing with notebooksjobsand clusters

• Strong programming skills in PythonPySparkand SQL

• Experience with Apache Spark for big data processing

• Solid understanding of ETL processes and data transformation

• Familiarity with Azure storage solutions including Azure SQL Database Blob Storage and Data Lake Storage

• Knowledge of data warehousing principles and scalable data modelling techniques

• Experience integrating multiple data sources in cloud environments

• Ability to write modular testable code for production systems

Benefits

While specific benefits have not beendisclosed you will join a collaborative environment where technical excellence is valued and alongside skilled professionals on meaningful projects that advance data engineering capabilities with opportunities to research and implement innovative solutions that shape the future of their data marketplace platform.

How to Apply

To apply for this role pleasesubmityour CV using the form below or email