Solution Architect- Data and AI

Job Description:

Own architectural decisions (ADRs) reference blueprints and target-state designs; arbitrate trade-offs on latency cost and scalability.

Architect end-to-end data and AI solutions: medallion Lakehouse (Bronze/Silver/Gold) on ADLS Gen2 Delta Lake streaming ingestion ML deployment pipelines.

Optimize large-scale PySpark/Databricks workloads partitioning AQE skew handling Photon cluster tuning.

Design data models across PostgreSQL nowflake (Kimball Data Vault 2.0 denormalized serving).

Productionize ML/DL models via Azure ML AKS Functions; expose FastAPI endpoints for batch and real-time inference.

Enforce security and compliance Entra ID Key Vault PII Unity Catalog / Purview GDPR.

Build CI/CD and MLOps pipelines (Azure DevOps Terraform/Bicep MLflow); instrument observability.

Data Modeling Architecture/Design must also be done by Data and AI Solution Architect

This typically includes STAR SCHEMA Design Normalization/Denormalization Data Patterns etc.

Ability to evaluate and recommend which platform component would be best fit for given requirement.

Example: Decide between Postgres Db/Mongo Db Decide between ODAP/One System Platform etc.

Implementation of Continuous Architecture

Know how about BI domain AI domains (NLP GEN A RAG Forecasting Computer Vision Agentic AI LLM) average or above average knowledge of concepts and internal technicalities.

Coach delivery teams align with Data Science Product QA and Platform leads and contribute to the CDOs data policy.

Operate hands-on when needed ingestion cleansing validation repository design root-cause analysis.

Must-Have

Advanced Python PySpark SQL

Expert in Databricks Delta Lake Spark Structured Streaming

Strong Azure stack: ADLS Gen2 ADF Synapse Azure ML AKS Event Hubs Key Vault

Proven design of medallion Lakehouse architectures at TBPB scale

Deep DB expertise: PostgreSQL Synapse Snowflake ADLS

ML/DL fluency: scikit-learn TensorFlow PyTorch MLflow

Hands-on PII/GDPR data handling and cloud security

CI/CD & MLOps: Azure DevOps or GitHub Actions Terraform/Bicep Docker Kubernetes