Lead Databricks Data Architect
Lead Databricks Data Architect
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Not Disclosed
Salary Not Disclosed
1 Vacancy
Job Description
Job Title: Lead Databricks Data Architect
Location: Atlanta GA (Onsite/Hybrid/Remote as per requirement)
Experience: 8 10 Years
Job Type: Full Time
Job Overview
We are seeking an experienced Lead Databricks Data Architect to design and implement robust scalable and secure cloud-based data solutions. This role involves working with Azure cloud services leading development of data ingestion pipelines and optimizing data architecture to meet evolving business needs.
Key Responsibilities
- Architect and build end-to-end data solutions using Azure Databricks Azure Data Factory and Azure SQL Database.
- Lead the development of real-time and batch data pipelines using Delta Live Tables (DLT) and Structured Streaming.
- Collaborate with stakeholders to translate business requirements into scalable data warehouse solutions on Azure.
- Provide strategic leadership for large-scale data programs and ensure alignment with organizational goals.
- Drive performance optimization of Spark jobs and Databricks workflows through advanced tuning techniques.
- Build and maintain metadata-driven ingestion frameworks to support automation and scalability.
- Work cross-functionally with data scientists analysts and other stakeholders to deliver business-aligned data solutions.
- Implement data governance quality checks and security measures to maintain data integrity and compliance.
- Monitor and enhance cloud-based data infrastructure for performance and cost-efficiency.
- Lead or support migration and upgrade efforts of legacy applications (e.g. .NET Framework) to cloud platforms.
- Utilize Azure DevOps tools including Git pipelines and work item management.
- Troubleshoot issues on Windows Server environments (e.g. IIS Event Viewer Performance Monitor).
- Participate in Agile/Scrum ceremonies including sprint planning retrospectives and daily standups.
Technical Skills Required
- Strong proficiency in:
- Azure Databricks ADF Azure SQL
- Apache Spark (job tuning and internals)
- Structured Streaming Delta Live Tables
- Experience in:
- .NET Frameworks 4.8.2 and Core 6/8
- MVC Blazor WebForms
- Web services & API integrations
- Microsoft SQL Server (DDL DML stored procedures)
- Nota Fiscal integration (Preferred in finance/logistics/related domains)
- Familiarity with:
- Azure authentication and key vault
- Agile/Lean methodologies
Preferred Qualifications
- Proven experience designing and delivering enterprise-level cloud data platforms.
- Ability to lead cross-functional teams and mentor junior developers.
- Excellent communication documentation and stakeholder management skills.
Familiarity with CI/CD and infrastructure-as-code in cloud environments.
Employment Type
Full Time
