Data Engineer w Databricks

New York City, NY - USA

Monthly Salary: Not Disclosed

Posted on: 19 hours ago

Vacancies: 1 Vacancy

Job Summary

Solution IT Inc. is looking for Data Engineer w/ Databricks for one of its clients at NYC NY/ HYBRID

Job Title: Data Engineer w/ Databricks

Responsibilities:

Develop optimize and maintain Databricks notebooks for ETL and data transformation.
Design and implement scalable data pipelines using Databricks and Azure Data Factory.
Apply strong knowledge of distributed computing concepts to large-scale data processing.
Work with data lake architectures and Delta Lake for reliable and performant data storage.
Automate data quality checks monitoring and logging.
Integrate data from multiple cloud sources such as Azure Blob Data Lake Storage and SQL/NoSQL databases.
Perform data modeling and schema design for analytical workloads.
Implement CI/CD pipelines for Databricks notebooks and jobs.
Manage workspace administration cluster management and job orchestration in Databricks.

Required Skills

Proven experience with Databricks for large-scale data processing and analytics..
Familiarity with cloud data platforms preferably Azure.
Understanding of data engineering best practices and distributed systems.
Experience with data pipeline orchestration and automation

Work Site: NYC NY/ (Hybrid)

Duration: 6 Months

Expected Start Date: Immediate

Number of Positions: 1

Please send your responses back to

About Solution IT

Solution IT is a national IT consulting company specializing in: Technology Staffing and Oracle E-Business Solutions based in Boston Massachusetts.

Solution IT Inc. is looking for Data Engineer w/ Databricks for one of its clients at NYC NY/ HYBRID Job Title: Data Engineer w/ Databricks Responsibilities: Develop optimize and maintain Databricks notebooks for ETL and data transformation. Design and implement scalable data pipelines using Databr...