Azure Data Architect (Databricks & Lakehouse Platform)

Melville, NY - USA

Monthly Salary: Not Disclosed

Posted on: 9 days ago

Vacancies: 1 Vacancy

Job Summary

Job Title: Hands-on Data Architect (Databricks Lakehouse) Role Overview

Hands-on Data Architect responsible for designing and implementing Databricks Lakehouse architecture while building scalable data pipelines. Own end-to-end data flow from ingestion to transformation with a focus on performance governance and security.

Key Responsibilities

Design Databricks Lakehouse (Bronze/Silver/Gold) architecture
Build data ingestion pipelines for API & FTP sources
Develop cleansing transformation and enrichment frameworks
Convert SQL Server logic & stored procedures into PySpark/Spark SQL
Implement and optimize Delta Lake (performance tuning partitioning)
Establish data governance security and PHI controls
Build and manage CI/CD pipelines for data workflows
Ensure scalability reliability and performance of data pipelines
Mentor and guide engineering teams

Required Skills

Strong experience in Databricks & Delta Lake
Advanced PySpark and Spark SQL
End-to-end data pipeline design & development
Expertise in SQL Server (queries stored procedures)
Hands-on Azure cloud (ADLS ADF Synapse)
Knowledge of ETL/ELT data modeling and optimization
Experience in data governance security and compliance
Familiarity with CI/CD and DevOps practices

Preferred

Healthcare domain experience (PHI HIPAA)
Experience with real-time/streaming pipelines
Strong mentoring and leadership skills

Job Title: Hands-on Data Architect (Databricks Lakehouse) Role Overview Hands-on Data Architect responsible for designing and implementing Databricks Lakehouse architecture while building scalable data pipelines. Own end-to-end data flow from ingestion to transformation with a focus on performance g...