AI & Data Platform Architect

Principal AI & Data Architect Enterprise Data Platform (AI Analytics & GenAI)
Role Summary: We are seeking a Senior AI & Data Platform Architect with deep expertise in Azure Databricks Snowflake cloud-scale data engineering Lakehouse architecture and enterprise analytics this role you will define design and govern enterprise-scale data processing analytics and AI-ready platform solutions across Azure Databricks and Snowflake. You will lead architectural decisions across multiple initiatives ensuring alignment with business objectives cloud strategy security performance scalability reliability governance and cost optimization standards.
The ideal candidate will combine strong hands-on architecture experience platform engineering mindset stakeholder leadership and the ability to guide delivery teams on modern data platform implementation patterns. This role will be accountable for designing end-to-end Databricks and Snowflake solutions establishing architecture standards reviewing solution designs mentoring senior engineers and improving overall data platform maturity across the organization.
Roles & Responsibilities
Act as the architecture lead and subject matter expert for Azure Databricks Snowflake and enterprise data platform initiatives across projects and programs.
Analyze business data functional and non-functional requirements and translate them into scalable end-to-end architecture designs.
Design modern data lakehouse and data warehouse architectures using Azure Databricks Snowflake Azure Data Lake Storage Gen2 Delta Lake and Azure Data Factory.
Define and govern medallion architecture standards across raw curated and serving layers including bronze silver and gold data patterns.
Architect scalable ingestion transformation orchestration and consumption patterns across batch incremental near-real-time and streaming workloads.
Define Databricks cluster architecture job design workflow orchestration notebook standards and workload isolation strategies.
Design Snowflake warehouse strategies data modelling patterns workload management query optimization cost controls and secure data sharing approaches.
Establish reusable reference architectures solution patterns guardrails and engineering standards for Databricks Snowflake and AI-ready data products.
Drive performance optimization scalability resiliency and cost efficiency through cluster tuning SQL optimization resource management and platform monitoring.
Define data security and governance practices including RBAC ABAC secrets management encryption row-level and column-level access masking lineage and compliance controls.
Review solution designs notebooks SQL assets pipelines infrastructure templates and deployment approaches for architectural compliance.
Collaborate with enterprise architects cybersecurity cloud infrastructure data governance platform operations analytics and business stakeholders on key design decisions.
Guide CI/CD DevOps and release management strategies for Databricks and Snowflake deployments using Git-based workflows and automated promotion patterns.
Provide architectural guidance during production incidents root cause analysis platform optimization capacity planning and operational maturity initiatives.
Mentor technical leads senior data engineers and platform engineers through architecture reviews design walkthroughs and best-practice enablement.
Professional & Technical Skills Must Have
Strong hands-on and architectural experience with Azure Databricks including cluster configuration cluster policies job scheduling workflow orchestration notebook design and workload optimization.
Advanced expertise in Apache Spark architecture PySpark Spark SQL Spark performance tuning partitioning caching shuffle optimization and scalable data processing patterns.
Deep understanding of Delta Lake Delta tables schema evolution time travel optimization vacuuming and lakehouse reliability patterns.
Strong proficiency in SQL for data transformations data modeling analytics workloads and performance tuning.
Strong experience with Snowflake architecture including virtual warehouses resource monitors workload isolation Snowpipe Streams Tasks Snowpark secure data sharing cloning and Time Travel.
Experience designing scalable data models across lakehouse data warehouse dimensional Data Vault semantic and consumption-oriented architectures.
Deep understanding of data engineering data warehousing ELT/ETL metadata management orchestration and data quality frameworks.
Experience implementing secure scalable governed and AI-ready data platforms using Azure Databricks Snowflake Azure Data Lake Storage Gen2 and modern cloud-native services.
Strong knowledge of Azure cloud security identity access control networking private endpoints secrets management and governance controls.
Experience defining CI/CD and deployment strategies for Databricks notebooks jobs workflows libraries infrastructure-as-code and Snowflake database objects.
Ability to evaluate architectural trade-offs across performance cost scalability maintainability security reliability and delivery speed.
Strong communication skills with the ability to influence enterprise architects security teams platform teams engineering teams and senior stakeholders.
Preferred Qualifications
Experience designing enterprise-scale data platforms that support analytics reporting machine learning Generative AI and agentic AI use cases.
Exposure to Unity Catalog catalog design data ownership models metadata management lineage and governance operating models.
Experience with Snowflake Cortex Snowpark external functions vector search semantic search or AI-enabled analytics capabilities is preferred.
Experience integrating Databricks and Snowflake with BI ML MLOps LLMOps data catalog observability and enterprise monitoring tools.
Knowledge of streaming and event-driven architectures using Kafka Event Hubs structured streaming Snowpipe Streaming or equivalent technologies.
Experience with platform modernization migration from legacy data warehouses cloud data lake implementation or large-scale data product enablement.
Familiarity with cost optimization practices across Databricks compute Snowflake warehouses storage tiers orchestration and consumption workloads.
Relevant certifications such as Databricks Data Engineer Professional Databricks Solutions Architect SnowPro Core SnowPro Advanced Architect or Microsoft Azure Solutions Architect are desirable.
Additional Information
The candidate should have 912 years of experience in data engineering analytics big data platforms or cloud data architecture with clear architecture ownership across enterprise initiatives.
This position is based at Bengaluru Chennai or Hyderabad office locations.
Minimum 15 years of full-time education or equivalent qualification is required.
The role is responsible for driving architecture governance technical standards platform scalability platform stability cost optimization and engineering maturity across multiple teams.
The architect will be accountable for enabling robust secure high-performance and governed Databricks and Snowflake solutions aligned to enterprise data and AI strategy.
Key Success Measures
Enterprise architecture standards for Databricks and Snowflake are defined communicated and adopted across delivery teams.
Data pipelines and analytics workloads are scalable secure performant cost-efficient and aligned with governance requirements.
Reusable platform patterns medallion architecture standards CI/CD practices and deployment guardrails improve delivery consistency.
Production incidents performance bottlenecks and platform risks are reduced through proactive design reviews and operational governance.
Senior engineers and technical leads are mentored to improve architecture quality platform maturity and engineering excellence.