Solution Architect
Solution Architect
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Solution Architect – Data & Machine Learning Platform (MLP)
Experience: 12 years
Position Overview
We are looking for a highly skilled Solution Architect to lead the design and implementation of a Data and Machine Learning Platform spanning edge cloud and on-prem components. The ideal candidate will have deep experience in Azure cloud services data engineering edge computing and ML lifecycle management.
Technical Expertise
Azure Cloud Stack & DevOps
Azure Databricks (including ML workspace for Feature Store and Model Store)
Azure Data Factory (ADF) for orchestration and compute
Azure Data Lake Storage (ADLS) implementing medallion architecture (raw bronze silver gold)
Azure Event Hub: Experience in defining topics managing consumer groups and integrating ETL events
Azure Streaming Analytics: Real-time data processing for telemetry and operational data
Azure Key Vault
Azure App Service
Azure Container Registry (ACR)
Azure IoT Hub for connecting edge devices
Azure DevOps & GitHub Actions (for CI/CD pipelines)
GitHub self-hosted runners for ML workflow automation
Edge and On-Prem Integration
Strong experience in OT-IT integration and data extraction from industrial systems
Edge VM deployment using:
- Docker and Portainer for container orchestration
- RabbitMQ for messaging (read/write services from edge)
- OPC UA for interfacing with PLCs (e.g. FX Filter NH3 Compressor)
- IDMZ deployment practices and edge-to-cloud data service integration
Machine Learning Platform (MLP) and MLOps
End-to-end ML lifecycle implementation: Feature Engineering Model Training & Validation Model Export Versioning and Deployment
Hands-on with ADB ML workspace Feature Store Model Store
Monitoring deployed models at 1-minute intervals
Understanding of training vs inference cloud vs edge deployment
Cadence for ML models (Weekly Refresh Monthly Retrain Quarterly Revamp)
Use of GitHub monorepo structure for managing model code
Data Architecture & Integration
Implementation of medallion architecture in the data platform
Integration with Unity Catalog (UC) for governance data sharing and cataloging
Experience with CDC tools (e.g. Aecorsoft) for real-time SAP data ingestion
Consumption layer design for BI ML and operational workloads
Familiarity with streaming and API-based ingestion from external environments
Template-driven ingestion and mapping using configurations
Governance and Data Modeling
Define and enforce data governance standards using Unity Catalog and enterprise frameworks. Design scalable data models to support operational analytics and ML features. Implement policies for access control quality and metadata tagging across DLZ/zones.
Key Responsibilities
1. Architect Integrated Solutions: Lead architectural design across edge cloud and ML across zones
2. Build and Govern Data Platform: Oversee ingestion transformation and cataloging across Raw Gold layers aligned to UC.
3. Enable Scalable ML Platform: Support ML teams with infrastructure for feature storage model ops and deployment pipelines.
4. Edge Integration and Automation: Design robust and secure OT-IT interfaces with RabbitMQ OPC UA and container orchestration tools.
5. Monitor and Optimize Pipelines: Set up real-time monitoring for ML and ETL pipelines; optimize for performance and cost.
6. Governance and Security Compliance: Ensure enterprise compliance tagging and secure access across all zones and services.
7. Lead CI/CD Automation: Use GitHub Actions and Azure DevOps to streamline deployment of ML workflows and platform components.
Skills
Devops, Azure, Rabbitmq, Data Modeling, Scala, Compliance, Machine Learning, Vault, Workflow, Docker, Cadence, Cloud Services
Employment Type
Full Time
