Databricks LeadArchitect

Boston, NH - USA

Monthly Salary: Not Disclosed

Posted on: 2 hours ago

Vacancies: 1 Vacancy

Job Summary

Job Title : Sr. Databricks Lead/Architect

Location: Boston MA (5 days onsite)

Duration: 12 Months

Key Responsibilities

Stakeholder Engagement & Solution Leadership

Act as the primary technical liaison between business stakeholders and engineering teams.
Lead requirement discovery sessions and translate business needs into scalable data architecture solutions.
Present architecture approaches trade-offs and roadmaps to leadership and decision-makers.
Provide technical leadership across design implementation and major delivery milestones.
Collaborate with distributed and offshore teams to ensure alignment and quality execution.

Data & Platform Architecture

Design and implement end-to-end Azure-based data platforms leveraging Azure Databricks and Azure Data Lake.
Define and implement lakehouse architecture patterns supporting batch streaming and near real-time processing.
Establish data modeling standards including layered architecture (bronze/silver/gold) and curated data models.
Design integration patterns with upstream and downstream systems including APIs event streaming platforms and analytics tools.
Support cloud modernization and legacy platform transformation initiatives.
Ensure data governance security scalability and cost optimization best practices are applied.

Technical Governance & Delivery Oversight

Review and validate development deliverables to ensure adherence to architecture standards.
Provide guidance on Spark performance optimization data engineering best practices and reusable framework design.
Establish CI/CD processes and DevOps enablement using Azure DevOps and Git.
Implement monitoring logging and performance optimization strategies across the data platform.
Drive continuous improvement in architecture performance and cost efficiency.

Required Technical Skills

Strong hands-on expertise in Azure Databricks (Spark Delta Lake Unity Catalog).
Deep experience with:
- Azure Data Lake Storage (ADLS Gen2)
- Azure Data Factory or Synapse Pipelines
- Event-driven or streaming architectures (Kafka Event Hub Structured Streaming)
- Python SQL Spark optimization techniques
Solid understanding of modern data warehousing and lakehouse architectures.
Experience integrating BI tools such as Power BI or similar analytics platforms.
Exposure to ML/AI integration within Databricks environments is a plus.

Experience & Qualifications

10 years of experience in data engineering or data architecture roles.
5 years designing and implementing solutions on Azure data platforms.
3 years in a technical lead or solution architect capacity.
Azure and/or Databricks certifications preferred.

Job Title : Sr. Databricks Lead/Architect Location: Boston MA (5 days onsite) Duration: 12 Months Key Responsibilities Stakeholder Engagement & Solution Leadership Act as the primary technical liaison between business stakeholders and engineering teams. Lead requirement discovery sessions...