Data DevOps Engineer

Job Description:

DevOps -

Azure

Databricks

Terraform

Github

We are seeking a highly skilled Data DevOps Engineer to join our operations team. You will be responsible for building automating and maintaining scalable data pipelines on the Azure platform. This role bridges the gap between data engineering and IT operations with a heavy focus on high-quality code automated deployments and system reliability.

Key Responsibilities:

Data Pipeline Engineering:Design and optimize scalable processing solutions using Azure Databricks with a focus on Scala and Python.

CI/CD & Infrastructure:Build and manage automated deployment pipelines using GitHub Actions and Jenkins while provisioning infrastructure via Terraform.

Quality & Testing:Own the reliability of the codebase by writing comprehensive unit tests(using frameworks like pytest or ScalaTest) and performing thorough end-to-end (E2E) validations.

Operations & Monitoring:Participate in the DevOps/Operations lifecycle including monitoring production pipelines troubleshooting performance issues and ensuring data integrity.

Collaboration:Communicate complex technical requirements clearly to both technical and non-technical stakeholders.

Required Qualifications:

Experience:3 years in a Data Engineering or DevOps role.

Languages:Strong proficiency in Scala and Python for big data applications.

Cloud Platform:Hands-on expertise in Azure Databricks (Spark Delta Lake Unity Catalog).

Automation Tools:Proven experience with Git GitHub Actions Jenkins and Terraform.

Testing Mindset:Demonstrated experience in building testable code and performing system-wide validation checks.

Soft Skills:Excellent verbal and written communication; ability to work effectively in a collaborative operations-focused team.

Required Skills:

DevOpsAzureData BricksTerraformGithub