Senior Data Architect

Descripción del puesto

Were looking for a Senior Data Architect to join our Data & AI Team. Candidates must be located in Guadalajara Mexico due to travel requirements.

Responsibilities:

• Design develop and optimize enterprise-scale ETL/ELT pipelines using Azure Data Factory (ADF) and Azure Databricks.
• Build robust data ingestion frameworks supporting batch and real-time data from APIs databases SaaS applications and external sources.
• Implement and maintain Medallion Architecture (Bronze Silver Gold) to support scalable and governed data processing.
• Establish CI/CD practices automation frameworks and deployment standards for data engineering workflows.
• Architect implement and manage enterprise data platforms leveraging Azure Data Lake Storage (ADLS) Azure SQL and Azure Databricks.
• Lead platform performance tuning cluster optimization workload management and cloud cost governance initiatives.
• Drive DataOps best practices including automated testing version control monitoring observability and operational excellence.
• Own and evolve the enterprise Databricks Lakehouse architecture including Delta Lake design patterns and implementation standards.
• Establish and enforce data engineering standards naming conventions security frameworks and governance practices through Unity Catalog.
• Lead workspace organization job orchestration strategies cluster governance and platform scalability initiatives.
• Design and implement enterprise data quality frameworks validation rules auditing processes and error-handling mechanisms.
• Maintain metadata management practices data lineage technical documentation and governance standards.
• Ensure compliance with security privacy and regulatory requirements.
• Partner with business analysts BI teams data scientists and application teams to deliver trusted analytics-ready datasets.
• Design reusable data models that support reporting self-service analytics AI/ML initiatives and predictive modeling.
• Enable modern data and AI capabilities through scalable and governed data architecture.

Requisitos

Requirements:

• Advance English Communication skills are required
• 8 years of hands-on experience in Data Architecture Data Engineering or related roles.
• Deep expertise with Azure Data Factory (ADF) Azure Databricks Delta Lake Spark and Azure Data Lake Storage (ADLS).
• Strong proficiency in SQL including complex query development performance tuning optimization and stored procedures.
• Proven experience designing implementing and supporting enterprise-scale Data Lakehouse platforms.
• Strong understanding of data governance security metadata management and Unity Catalog.
• Excellent communication and stakeholder management skills with the ability to translate business requirements into technical solutions.

Preferred Qualifications:

• Experience working with healthcare data including EHR/EMR HL7 FHIR Claims and Revenue Cycle Management (RCM).
• Knowledge of AI/ML workflows feature engineering and MLOps practices.
• Experience implementing CI/CD pipelines using Azure DevOps GitHub Actions or Databricks Repos.
• Proficiency in Python for ETL development automation and AI/ML support.
• Experience with real-time and streaming data processing using Structured Streaming.
• Familiarity with Azure Synapse Analytics or comparable cloud data warehousing platforms.

Must-have skills:

• Azure Databricks architecture.
• Azure Data Factory pipelines.
• Delta Lake and Medallion architecture.
• Spark PySpark and SQL.
• ADLS and Azure SQL.
• Unity Catalog access controls governance and lineage.
• Cluster design workload isolation performance tuning and cost optimization.
• CI/CD with Azure DevOps GitHub Actions or Databricks Repos.
• DataOps: testing monitoring version control documentation.
• Batch and streaming ingestion from APIs databases SaaS platforms and internal systems.
• Data validation auditing error handling and reusable data models.

Strong plus:

• Healthcare data experience: EHR/EMR HL7 FHIR claims RCM.
• AI/ML enablement feature engineering or model support.
• Dev/Test/Prod Databricks workspace strategy.
• Azure Synapse or similar warehousing experience.