Build and maintain ETL/ELT pipelines in Microsoft Fabric Notebooks (Python/PySpark) ingesting data from multiple source systems via REST and SOAP APIs
Implement Bronze layer landing (raw data no transformation) Silver layer cleansing/typing/deduplication and Gold layer aggression for business analytics
Design and build custom API connectors with OAuth 2.0/Bearer token authentication incremental sync pagination handling rate-limit/retry logic and error recovery
Configure and manage Fabric Workspaces across Dev/Test/Production environments using Fabric Deployment Pipelines and Git integration for CI/CD
Build and maintain Power BI semantic models (star/snowflake schemas) supporting operational reporting and analytics dashboards
Implement row-level security (RLS) using Azure AD RBAC warehouse-level DAX filters and dynamic RLS platforms
Set up monitoring alerting and telemetry using Azure Monitor Log Analytics and Application insights to track pipeline health and data freshness
Manage API credentials and secrets via Azure Key Vault with automated rotation policies
Embed within the clients development process working directly with business stakeholders to gather requirements and co-develop solutions
Contribute to data governance: data dictionary creation lineage tracking and documentation of transformation logic across all pipeline stages
Requirements
5 years experience in data engineering or a similar role in a commercial environment
Hands-on experience with Microsoft Fabric (Lakehouses Notebooks Data Pipelines OneLake SQL Analytics Endpoints) or equivalent depth in Azure Synapse Analytics
Advanced SQL for data transformation performance tuning and Delta Lake table management
Proficient in Python/PySpark for data processing API integration and pipeline automation
Proven experience building custom API connectors (REST SOAP/XML) with OAuth 2.0 pagination rate limiting and incremental sync patterns
Strong understanding of Medallion Architecture data lakehouse concepts and Delta Lake (merge upsert soft-delete handling time travel)
Experience with dimensional modelling (star/snowflake schemas) and Power BI semantic model development
Working knowledge of Azure services: Key Vault Azure Monitor Log Analytics Azure AD/Entra ID
Experience translating business requirements into dimensional models iteratively (not just building pre-designed schemas)
Comfort operating autonomously with minimal supervision in a client-embedded model
Experience implementing row-level security in Power BI (DAX-based RLS dynamic security models)
Familiarity with CI/CD in a Fabric context: Deployment Pipelines Git integration environment promotion workflows
Experience with semi-structured data formats (JSON XML) and handling schema evolution across pipeline layers
Preferred
Prior work in retail wholesale distribution or fresh produce / supply chain data environments
Experience with ERP data extraction and replication tools in a Fabric or Azure context
Knowledge of dbt Power Query or similar transformation frameworks
Relevant Azure or Microsoft certifications (DP-600 DP-203 PL-300)
Openness to AI coding assistants (GitHub Copilot Cursor Claude) as part of your development workflow
What We Offer
Competitive salary with mentorship and career growth opportunities
Hands-on work with cloud-first technologies (Microsoft Fabric Azure Power BI)
Long-term embedded role with a single client providing deep domain expertise and continuity
Fast-paced innovative environment where AI-augmented development is the norm
90000 - 120000 a month
We may use artificial intelligence (AI) tools to support parts of the hiring process such as reviewing applications analyzing resumes or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed please contact us.
Required Experience:
IC
Responsibilities Build and maintain ETL/ELT pipelines in Microsoft Fabric Notebooks (Python/PySpark) ingesting data from multiple source systems via REST and SOAP APIsImplement Bronze layer landing (raw data no transformation) Silver layer cleansing/typing/deduplication and Gold layer aggression for...
Responsibilities
Build and maintain ETL/ELT pipelines in Microsoft Fabric Notebooks (Python/PySpark) ingesting data from multiple source systems via REST and SOAP APIs
Implement Bronze layer landing (raw data no transformation) Silver layer cleansing/typing/deduplication and Gold layer aggression for business analytics
Design and build custom API connectors with OAuth 2.0/Bearer token authentication incremental sync pagination handling rate-limit/retry logic and error recovery
Configure and manage Fabric Workspaces across Dev/Test/Production environments using Fabric Deployment Pipelines and Git integration for CI/CD
Build and maintain Power BI semantic models (star/snowflake schemas) supporting operational reporting and analytics dashboards
Implement row-level security (RLS) using Azure AD RBAC warehouse-level DAX filters and dynamic RLS platforms
Set up monitoring alerting and telemetry using Azure Monitor Log Analytics and Application insights to track pipeline health and data freshness
Manage API credentials and secrets via Azure Key Vault with automated rotation policies
Embed within the clients development process working directly with business stakeholders to gather requirements and co-develop solutions
Contribute to data governance: data dictionary creation lineage tracking and documentation of transformation logic across all pipeline stages
Requirements
5 years experience in data engineering or a similar role in a commercial environment
Hands-on experience with Microsoft Fabric (Lakehouses Notebooks Data Pipelines OneLake SQL Analytics Endpoints) or equivalent depth in Azure Synapse Analytics
Advanced SQL for data transformation performance tuning and Delta Lake table management
Proficient in Python/PySpark for data processing API integration and pipeline automation
Proven experience building custom API connectors (REST SOAP/XML) with OAuth 2.0 pagination rate limiting and incremental sync patterns
Strong understanding of Medallion Architecture data lakehouse concepts and Delta Lake (merge upsert soft-delete handling time travel)
Experience with dimensional modelling (star/snowflake schemas) and Power BI semantic model development
Working knowledge of Azure services: Key Vault Azure Monitor Log Analytics Azure AD/Entra ID
Experience translating business requirements into dimensional models iteratively (not just building pre-designed schemas)
Comfort operating autonomously with minimal supervision in a client-embedded model
Experience implementing row-level security in Power BI (DAX-based RLS dynamic security models)
Familiarity with CI/CD in a Fabric context: Deployment Pipelines Git integration environment promotion workflows
Experience with semi-structured data formats (JSON XML) and handling schema evolution across pipeline layers
Preferred
Prior work in retail wholesale distribution or fresh produce / supply chain data environments
Experience with ERP data extraction and replication tools in a Fabric or Azure context
Knowledge of dbt Power Query or similar transformation frameworks
Relevant Azure or Microsoft certifications (DP-600 DP-203 PL-300)
Openness to AI coding assistants (GitHub Copilot Cursor Claude) as part of your development workflow
What We Offer
Competitive salary with mentorship and career growth opportunities
Hands-on work with cloud-first technologies (Microsoft Fabric Azure Power BI)
Long-term embedded role with a single client providing deep domain expertise and continuity
Fast-paced innovative environment where AI-augmented development is the norm
90000 - 120000 a month
We may use artificial intelligence (AI) tools to support parts of the hiring process such as reviewing applications analyzing resumes or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed please contact us.