Senior Data Engineer – Azure & Databricks
Share
Job Location:
Alpharetta, GA - USA
Yearly Salary:
$ 82 - 123
Posted on:
9 hours ago
Vacancies:
1 Vacancy
Job Summary
Senior Data Engineer Azure & Databricks
Industry: Technology Consulting / Digital Transformation
Role Level: Lead II Software Engineering
Location: Alpharetta Georgia (USA)
Employment Type: Full-Time
About the Role:
We are seeking an experienced Senior Data Engineer Azure & Databricks with a proven track record (8 years) of designing building and scaling modern data platforms within the Azure ecosystem.
This role is ideal for someone who thrives in complex enterprise environments is deeply hands-on with Databricks and Spark and can lead end-to-end data engineering initiatives from architecture to implementation.
You will be responsible for building robust high-performance data pipelines establishing engineering best practices and collaborating with cross-functional teams to deliver reliable scalable data solutions across both batch and real-time streams.
Key Responsibilities:
Architect design and implement scalable data platforms and pipelines using Azure and Databricks.
Build and optimise ingestion transformation and processing workflows for batch and real-time (streaming) use cases.
Work extensively with ADLS Delta Lake and Spark (Python/PySpark) to enable large-scale data engineering capabilities.
Lead the development of complex ETL/ELT pipelines ensuring performance reliability and code quality.
Design conceptual logical and physical data models to support analytics and operational workloads.
Work with relational and lakehouse systems including PostgreSQL and Delta Lake.
Define implement and enforce best practices for data governance security quality and architecture standards.
Partner closely with architects data scientists analysts and business teams to translate requirements into scalable technical solutions.
Troubleshoot production issues drive performance optimisation and support continuous platform improvements.
Mentor junior engineers and contribute to the creation of reusable components and engineering standards.
Required Qualifications:
8 years of hands-on data engineering experience in enterprise environments.
Strong expertise in Azure services particularly Azure Databricks Azure Functions and (preferably) Azure Data Factory.
Advanced proficiency in Apache Spark with Python (PySpark).
Strong SQL skills including query optimisation and performance tuning.
Deep experience with ETL/ELT methodologies scheduling and data orchestration.
Hands-on expertise with Delta Lake (ACID schema evolution performance tuning).
Strong understanding of data modelling (normalised dimensional lakehouse).
Proven experience with batch and streaming technologies such as Kafka or Event Hub.
Solid grasp of data architecture distributed systems and cloud-native design patterns.
Ability to design and evaluate end-to-end technical solutions and recommend best-fit architectures.
Excellent analytical problem-solving and communication skills.
Ability to collaborate across teams and lead technical discussions.
Preferred Skills:
Experience with CI/CD tools such as Azure DevOps and Git.
Familiarity with Infrastructure-as-Code (Terraform ARM).
Exposure to data governance and metadata cataloguing tools (e.g. Azure Purview).
Experience supporting machine learning or BI workloads on Databricks.
Benefits:
For Full-Time Regular Employees
Minimum 10 days paid vacation annually
6 days paid sick leave (prorated for new hires)
10 paid holidays
Paid bereavement and jury duty leave
Eligibility for 401(k) Retirement Plan with employer match
Medical dental and vision insurance eligibility (employee dependents)
Company-paid:
Basic life insurance
Accidental death & disability coverage
Short- and long-term disability
Access to HSA and FSA programs (healthcare dependent care commuting)
Application Process:
To apply please submit your CV and a cover letter detailing your relevant experience and qualifications for the role. Applications will be reviewed on a rolling basis so early submissions are encouraged.
Equal Opportunity Employer:
We value diversity and are committed to creating an inclusive work environment. We encourage applicants from all backgrounds to apply.
Contact: Elvis Eckardt
Required Experience:
Senior IC
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
About Company
Position: IT Support Technician Location: Onsite in Redhill, Surrey with 6 Remote Days per monthDepartment: EnterpriseSection: Service Management TeamReports to: Head of Technology & Networks About the Role: As a Support Technician, you will provide technical support for customer sys ... View more
