Hadoop PySpark Developer
Hadoop PySpark Developer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Not Disclosed
Salary Not Disclosed
1 Vacancy
Job Description
Job Title: Hadoop PySpark Developer
Location: Pittsburgh PA (Onsite)
Experience Level: 8 years preferred
Were hiring a Hadoop PySpark Developer to work onsite in Pittsburgh. Youll be using tools like Spark Python Hive and Kafka to build and manage data pipelines on the Cloudera Data Platform. The role involves working with large datasets building ETL solutions and making sure everything runs smoothly and efficiently. Youll also work with other teams to understand data needs and help deliver reliable solutions.
Roles & Responsibilities
- Build ETL pipelines using Spark Python Hive and Kafka
- Understand business requirements and turn them into technical tasks
- Work with data analysts and business users to gather data needs
- Create design documents and source-to-target mappings
- Make sure data pipelines run on time and handle large volumes
- Improve performance of Spark jobs and existing pipelines
- Set up data governance and ingestion frameworks
- Help with production deployments and fix issues when needed
- Support data migration and rebuilding history data
- Review code and test cases and help solve technical problems
- Work independently and coordinate with global teams
- Bring in ideas and best practices to improve solutions
Key Skills
- Apache Spark (PySpark)
- Python
- Hive
- Kafka
- Cloudera Data Platform
- Linux/Unix
- Impala
- Big Data tools
- Cloud technologies
- Performance tuning
- Data governance
- Good communication and teamwork
Employment Type
Full Time
