Lead Data Engineer

Cincinnati, OH - USA

Monthly Salary: Not Disclosed

Posted on: 17 hours ago

Vacancies: 1 Vacancy

Job Summary

Location: onsite 5x in Cincinnati 45241 (possibly flexible to hybrid Tues-Thurs onsite)

MUST HAVES:

10 years of experience as a Data Engineer

5 hears of experience using Databricks

1 year of experience as a Tech Lead or Team Lead

1 year of experience with Unity Catalogue

1 year of experience with DAG for job optimization

5 years of experience with Python and Spark (PySpark)

Experience deploying to the cloud preferably Azure

Plusses:

Devops and CI/CD experience

Azure cloud

AI and ML

DAY 2 DAY:

A large fortune 500 organization is seeking a Lead Data Engineer that will sit onsite in Cincinnati Ohio for a long term contract. The team is seeking a Lead Data Engineer experienced in implementing modern data solutions in Azure with strong hands-on skills in Databricks Spark Python and cloud-based DataOps practices. The Lead Data Engineer will analyze design and develop data products pipelines and information architecture deliverables focusing on data as an enterprise asset. This role also supports cloud infrastructure automation and CI/CD using Terraform GitHub and GitHub Actions to deliver scalable reliable and secure data solutions.

Analyze design and develop enterprise data solutions with a focus on Azure Databricks Spark Python and SQL

Develop optimize and maintain Spark/PySpark data pipelines including managing performance issues such as data skew partitioning caching and shuffle optimization

Build and support Delta Lake tables and data models for analytical and operational use cases

Apply reusable design patterns data standards and architecture guidelines across the enterprise including collaboration with 84.51 when needed

Use Terraform to provision and manage cloud and Databricks resources supporting Infrastructure as Code (IaC) practices

Implement and maintain CI/CD workflows using GitHub and GitHub Actions for source control testing and pipeline deployment

Manage Git-based workflows for Databricks notebooks jobs and data engineering artifacts

Troubleshoot failures and improve reliability across Databricks jobs clusters and data pipelines

Apply cloud computing skills to deploy fixes upgrades and enhancements in Azure environments

Work closely with engineering teams to enhance tools systems development processes and data security

Participate in the development and communication of data strategy standards and roadmaps

Draft architectural diagrams interface specifications and other design documents

Promote the reuse of data assets and contribute to enterprise data catalog practices

Deliver timely and effective support and communication to stakeholders and end users

Mentor team members on data engineering principles best practices and emerging technologies

Location: onsite 5x in Cincinnati 45241 (possibly flexible to hybrid Tues-Thurs onsite) MUST HAVES: 10 years of experience as a Data Engineer 5 hears of experience using Databricks 1 year of experience as a Tech Lead or Team Lead 1 year of experience with Unity Catalogue 1 year of experience wit...