DataBricks Data Engineer

Boston, NH - USA

Monthly Salary: Not Disclosed

Posted on: 7 hours ago

Vacancies: 1 Vacancy

Job Summary

Job Description

Job Description:

Design develop and maintain scalable data pipelines and ETL processes using Python Databricks and AWS Glue.

Build and optimize data warehouses and data lakes on PySpark and AWS platforms.

Implement data ingestion transformation and orchestration workflows using AWS Step Functions Lambda and Glue Jobs.

Monitor troubleshoot and optimize pipeline performance using AWS CloudWatch and other monitoring tools.

Collaborate with data analysts data scientists and business stakeholders to define data requirements and deliver reliable datasets.

Ensure data quality governance and security across all data systems and workflows.

Automate repetitive data engineering tasks and contribute to building reusable frameworks and templates.

Support continuous improvement by adopting best practices for CI/CD version control and infrastructure-as-code (IaC).

Job Description Job Description: Design develop and maintain scalable data pipelines and ETL processes using Python Databricks and AWS Glue. Build and optimize data warehouses and data lakes on PySpark and AWS platforms. Implement data ingestion transformation and orchestration workflows using AW...