Data Architect Tech Lead | P&SC Data and Intelligence

Job Description:

• The Data Architect / Tech Lead is the technical authority for P&SC Data and Intelligences data engineering foundation.
• This role sets and owns the architectural standards platform patterns and engineering quality bar that every squad builds against.
• The Architect is a hands-on technical leader who designs the systems reviews the implementations resolves the hardest engineering problems and ensures that what gets built across squads is consistent scalable and built to last.
• This is not a governance role it is a delivery-adjacent engineering leadership role that moves fast makes real decisions and directly shapes the quality of what ships.

Core Responsibilities

• Own the end-to-end technical architecture for the P&SC data platform including the Omni Databricks lakehouse design medallion layer standards Fabric IQ semantic layer patterns Unity Catalog governance and real-time streaming infrastructure.
• Define and enforce engineering standards across all P&SC squads including pipeline design patterns naming conventions data quality frameworks DevOps practices and documentation requirements.
• Serve as the primary technical decision-maker for platform-level choices including ingestion patterns streaming architecture compute optimization and data product certification criteria.
• Lead architecture design reviews for major squad deliverables providing hands-on technical guidance to senior data engineers and identifying design risks before they reach production. Operate as the technical lead for the Data Engineering squad directly contributing to complex pipeline builds.
• Partner with the Real-Time Integration Engineering squad to define event-driven architecture standards Kafka and Azure Event Hub streaming patterns and the integration contracts that govern source system data feeds into the SC Control Tower.
• Resolve cross-squad technical dependencies unblocking data engineers on complex ingestion transformation or platform issues that fall outside squad-level expertise.
• IT architecture and cloud engineering teams to ensure P&SC data products are built in alignment with enterprise-wide platform standards and licensing constraints.
• Mentor senior and mid-level engineers across squads raising the technical ceiling of the entire P&SC data engineering practice.

Required Qualifications

• 10 years of data engineering or data platform engineering experience with at least 3 years in an architect principal engineer or technical lead capacity across large-scale enterprise data platforms.
• Deep expertise in Databricks including Delta Lake Unity Catalog Databricks Workflows Structured Streaming and performance tuning at scale.
• Hands-on experience with Microsoft Fabric including OneLake Fabric Data Engineering and Fabric IQ semantic layer design.
• Expert-level experience designing and implementing real-time streaming architectures using Kafka Azure Event Hub Delta Live Tables or comparable technologies.
• Proven ability to design and enforce engineering standards across multi-squad data engineering organizations including pattern libraries code review practices and architectural governance. Experience architecting ERP or enterprise system ingestion pipelines at scale with strong familiarity with CDC (Change Data Capture) patterns and tools.
• Strong proficiency in PySpark and Spark SQL for distributed data transformation and pipeline architecture.
• Experience leading complex legacy platform migrations including extraction validation rebuild and decommission programs.
• Fluency operating across technical product and business stakeholder conversations including the ability to translate architecture decisions into business impact terms.
• Experience building and mentor