Data & AI Engineer – Microsoft Fabric, Experimentation Data & Agent Development

Job Title: Data & AI Engineer - Microsoft Fabric Experimentation Data & Agent Development

Location: Bellevue WA/Remote
Experience: 6-8 Years

Job Summary

We are seeking a highly skilled Data & AI Engineer with strong expertise in Microsoft Fabric Azure AI Services data engineering and AI-powered analytics solutions. The ideal candidate will design and manage scalable experimentation data platforms build curated Silver/Gold data layers develop AI-powered data agents and support enterprise analytics and experimentation workflows.

This role requires hands-on expertise in Microsoft Fabric PySpark clickstream analytics Adobe Customer Journey Analytics (CJA) Azure AI Services and modern data engineering best practices.

Required Skills

• Strong expertise in:
  • Microsoft Fabric
  • Data Engineering
  • Data Modeling
  • Big Data Platforms
  • Data Pipeline Design
• Hands-on experience with:
  • PySpark
  • Azure AI Services
  • AI/ML Integration
  • Microsoft Fabric Data Agents
  • Azure Data Factory (ADF)
• Strong understanding of:
  • Clickstream data
  • Customer Analytics
  • Adobe Analytics
  • Adobe Customer Journey Analytics (CJA)
  • Experimentation data platforms
• Experience with:
  • Data validation and reconciliation
  • Performance tuning and monitoring
  • Telemetry pipelines
  • Data governance and semantic modeling
• Strong analytical and problem-solving skills
• Ecommerce domain experience preferred

Key Responsibilities

Experimentation Data Enablement

• Design build and maintain curated Silver-layer datasets in Microsoft Fabric for experimentation analytics and reporting
• Develop reusable data products including standardized tables and views for dashboards scorecards and ad hoc analysis
• Ensure datasets are analysis-ready cleaned deduplicated and semantically aligned

Data Gap Analysis & Modeling

• Conduct gap assessments across experimentation KPIs telemetry systems and Silver-layer datasets
• Identify data quality latency schema and join-key issues and recommend remediation strategies
• Improve data models using:
  • Facts and dimensions
  • Conformance rules
  • Surrogate keys
  • Grain optimization

Gold Layer & Stakeholder Management

• Conduct workshops with BI reporting experimentation and engineering teams
• Define KPI calculations attribution logic governance standards and refresh SLAs
• Create source-to-target mappings data dictionaries validation rules and technical specifications

Pipeline Engineering

• Build and maintain robust data pipelines using:
  • Microsoft Fabric Pipelines
  • Azure Data Factory (ADF)
  • Telemetry/1DS pipelines
• Implement orchestration incremental loads monitoring and error handling mechanisms

Data Validation & Reconciliation

• Validate and reconcile datasets with Adobe Customer Journey Analytics (CJA)
• Build automated validation routines for:
  • Missing data
  • Duplicate detection
  • Schema drift
  • Metric anomaly detection
• Coordinate issue resolution with telemetry and engineering teams

AI Agent Development

• Design and build AI-powered data agents using:
  • Fabric Data Agents
  • Copilot
  • Azure OpenAI
• Enable use cases such as:
  • Automated scorecards
  • Narrative summaries
  • Self-service analytics Q&A
  • Metric definition assistants
  • Data quality monitoring assistants
• Define AI agent architecture RBAC security grounding sources and evaluation metrics

Governance & Operational Excellence

• Maintain documentation for:
  • Data transformations
  • Pipeline dependencies
  • Metric definitions
  • Validation rules
• Implement best practices for:
  • Naming conventions
  • Semantic consistency
  • Versioning
  • Cost optimization
• Support operational monitoring troubleshooting and continuous improvement initiatives

Preferred Qualifications

• Experience supporting experimentation and customer analytics platforms
• Strong knowledge of ecommerce and digital analytics ecosystems
• Exposure to AI-driven analytics and self-service data platforms
• Strong collaboration and stakeholder communication skills