Data Engineer
Share
Job Location:
Pittsburgh, PA - USA
Monthly Salary:
Not Disclosed
Posted on:
9 hours ago
Vacancies:
1 Vacancy
Job Summary
Data Engineer
Location Onsite Warrendale PA / Pittsburgh PA (U.S.)
This role requires core experience and expertise on Databricks (advanced hands-on) Python ETL/ELT pipeline development Spark (SQL/PySpark).
Job purpose
- The Data Engineer will be responsible for designing developing and optimizing large-scale data pipelines using Databricks Spark and Python. This is a short-term high-priority onsite engagement (6 9 months) supporting critical client data engineering initiatives. The engineer will work directly at the client location in Pennsylvania and collaborate closely with technical teams to deliver scalable and high-performing data solutions.
Key responsibilities:
-
- Design build and maintain Databricks-based ETL/ELT pipelines.
- Develop high-performance Spark (PySpark) workflows for data processing.
- Work with large-scale data in Lakehouse/Data Lake environments.
- Optimize and troubleshoot existing Databricks jobs and clusters.
- Collaborate with business and technical stakeholders to understand data requirements.
- Implement data quality checks validation rules and monitoring processes.
- Work with orchestration tools (ADF or equivalent) to schedule and automate workflows.
- Ensure best practices in version control CI/CD and documentation.
- Support production pipelines and resolve data-related issues proactively
Key competencies
-
- Strong analytical and problem-solving mindset.
- Ability to work independently in a fast-paced onsite environment.
- Excellent communication and cross-functional collaboration skills.
- Strong ownership and accountability for deliverables.
- Adaptability to dynamic project needs.
