Data Architect With AI

We are seeking a 12 experienced Data Architect with strong expertise in Microsoft Fabric Azure Databricks Azure Data Factory Azure Data Lake SQL Data Warehouse and modern cloud data platforms to define and implement scalable secure high-performance data architectures.

The ideal candidate will be responsible for translating business and technical requirements into robust enterprise data architecture and solution designs defining data platforms integration patterns data models governance frameworks and analytics solutions. The role will involve working closely with business stakeholders data engineers BI teams application teams and cloud/infrastructure teams to deliver scalable and cost-effective data solutions.

The Data Architect will provide technical leadership across batch and real-time data processing Lakehouse/Data Warehouse architecture data integration data modeling data governance security Power BI and cloud data platforms.

Key Responsibilities:

Data Architecture & Solution Design

• Design and define end-to-end enterprise data architectures using Microsoft Fabric Azure Databricks Azure Data Factory Azure Data Lake Synapse and related cloud services.

• Translate business requirements into scalable secure and maintainable data architecture and solution designs.

• Define architecture patterns for data ingestion transformation storage processing serving analytics and reporting.

• Design Lakehouse Data Warehouse and Medallion Architecture solutions based on business and technical requirements.

• Design and define AI/ML-ready data architectures to support Machine Learning Generative AI and advanced analytics use cases.

• Define data architecture patterns for Generative AI LLM RAG embeddings and vector search solutions

• Evaluate and recommend appropriate technologies platforms and architectural patterns based on performance scalability security availability and cost.

• Create architecture diagrams data flow diagrams solution designs and technical documentation.

• Design secure and scalable integration of enterprise data platforms with AI/ML and Generative AI applications. 

• Integrate AI agents with APIs databases cloud services (AWS Azure) and enterprise applications.

• Good exposure to work on RFPs sales proposal technical and estimate.

Microsoft Fabric & Databricks Architecture

• Define enterprise architecture using Microsoft Fabric including OneLake Lakehouse Warehouse Data Factory Dataflows Power BI and related Fabric capabilities.

• Design scalable Azure Databricks and Delta Lake architectures for large-scale data processing and analytics.

• Define Bronze Silver and Gold data-layer strategies and standards.

• Design Databricks solutions using Apache Spark Structured Streaming Delta Lake Unity Catalog and optimized data processing patterns.

• Define standards for partitioning file sizes schema evolution incremental processing data retention and performance optimization.

• Guide teams in optimizing Databricks workloads for performance and cloud cost efficiency.

Data Integration & ETL/ELT Architecture

• Design data integration architectures for data originating from databases APIs files SaaS applications Dynamics 365 and streaming sources.

• Define appropriate batch micro-batch and real-time/streaming ingestion patterns.

• Establish ETL/ELT patterns using ADF Fabric Data Factory Databricks and other cloud-native services.

• Define frameworks for incremental loads CDC SCD Type 1/Type 2 reconciliation error handling and data recovery.

• Design reusable and scalable data ingestion frameworks.

Data Modelling & Data Architecture

• Define conceptual logical and physical data models based on business requirements.

• Design dimensional models star schemas fact and dimension tables and analytical data models.

• Define enterprise data standards naming conventions data domains and master/reference data structures.

• Design models optimized for Power BI analytics reporting and downstream applications.

• Define strategies for managing historical data slowly changing dimensions data retention and archival.

Data Governance Security & Compliance

• Define enterprise data governance and security architecture across the data platform.

• Implement and govern Microsoft Purview Unity Catalog RBAC data classification lineage auditing and access-control strategies.

• Define security patterns for row-level security column-level security masking encryption secrets management and secure data access.

• Establish data ownership data stewardship metadata lineage and data quality standards.

Performance Scalability & Cost Optimization

• Define architecture strategies to support high-volume and high-performance data workloads.

• Identify and resolve architectural bottlenecks across ingestion storage transformation database and reporting layers.

• Optimize Databricks/Spark Delta Lake Fabric SQL and Power BI workloads.

• Define appropriate compute sizing workload isolation partitioning caching indexing and optimization strategies.

• Perform capacity planning scalability assessment and cost optimization for cloud data platforms.

Cloud & Technology Architecture

• Design cloud-based data solutions primarily on Microsoft Azure with knowledge of AWS/GCP data platforms.

• Define integration patterns across Azure AWS APIs SaaS applications and on-premises systems.

• Evaluate new technologies and recommend solutions based on business requirements and architectural standards.

• Define cloud architecture covering availability scalability networking security disaster recovery and operational support.

• Strong understanding of data governance security metadata lineage and data quality.

• Experience with cloud architecture scalability performance optimization and cost management.

• Good understanding of CI/CD DevOps Infrastructure as Code monitoring and deployment strategies.

Key Competencies

• Enterprise Data Architecture

• Solution Architecture

• Cloud Data Architecture

• Databricks & Lakehouse Architecture

• Data Modeling

• Data Governance & Security

• Performance & Cost Optimization

• Technical Leadership

• Stakeholder Management

• Architecture Documentation & Design

Qualifications :

Educational qualification:

B.E/ or equivalent Qualification

Experience :

• 12 experienced Data Architect with strong expertise in Microsoft Fabric Azure Databricks Azure Data Factory Azure Data Lake SQL Data Warehouse and modern cloud data platforms to define and implement scalable secure high-performance data architectures.

Mandatory/requires Skills :

• Design and define end-to-end enterprise data architectures using Microsoft Fabric Azure Databricks Azure Data Factory Azure Data Lake Synapse and related cloud services.

• Translate business requirements into scalable secure and maintainable data architecture and solution designs.

• Define architecture patterns for data ingestion transformation storage processing serving analytics and reporting.

• Design Lakehouse Data Warehouse and Medallion Architecture solutions based on business and technical requirements.

• Design and define AI/ML-ready data architectures to support Machine Learning Generative AI and advanced analytics use cases.

• Define data architecture patterns for Generative AI LLM RAG embeddings and vector search solutions

• Evaluate and recommend appropriate technologies platforms and architectural patterns based on performance scalability security availability and cost.

• Create architecture diagrams data flow diagrams solution designs and technical documentation.

• Design secure and scalable integration of enterprise data platforms with AI/ML and Generative AI applications. 

• Integrate AI agents with APIs databases cloud services (AWS Azure) and enterprise applications.

• Good exposure to work on RFPs sales proposal technical and estimate.

Preferred Skills :

Additional Information :

Remote Work :

No

Employment Type :

Full-time