PySpark Data Engineer
Job Location:
Monthly Salary:
Experience Required:
Posted on:
Vacancies:
Job Summary
- Design develop and maintain scalable data pipelines using PySpark on Databricks
- Build and optimize data processing workflows using Python
- Implement workflow orchestration and scheduling (preferably using Airflow if applicable)
- Work in an Agile delivery environment with cross-functional teams
- Ensure data quality performance and reliability of data solutions
- Support integration of data from multiple sources into analytics-ready structures
- 5 years of hands-on experience in PySpark (Databricks) and Python
- Strong experience in building and maintaining data engineering pipelines
- Exposure to Airflow (preferred not mandatory)
- Overall 68 years of professional experience in data engineering or related roles
- Strong communication skills and ability to work with distributed teams
- Good understanding of Agile development practices
- Develop efficient and scalable big data processing solutions using PySpark
- Debug optimize and enhance existing data workflows and pipelines
- Work independently as well as collaboratively in Agile teams
- Translate business requirements into technical data solutions
- Manage multiple tasks and deliver within deadlines in a fast-paced environment
- Opportunity to work on modern data engineering stack including Databricks and Python
- 6-month engagement duration with potential for extension based on performance
- Exposure to large-scale data engineering projects in an international environment
- Agile-driven collaborative working culture
As a PySpark Data Engineer you will: Design develop and maintain scalable data pipelines using PySpark on Databricks Build and optimize data processing workflows using Python Implement workflow orchestration and scheduling (preferably using Airflow if applicable) Work in an Agile delivery environment with cross-functional teams Ensure data quality performance and reliability of data solutions Support integration of data from multiple sources into analytics-ready structures What You Bring to the Table: 5 years of hands-on experience in PySpark (Databricks) and Python Strong experience in building and maintaining data engineering pipelines Exposure to Airflow (preferred not mandatory) Overall 68 years of professional experience in data engineering or related roles Strong communication skills and ability to work with distributed teams Good understanding of Agile development practices You should possess the ability to: Develop efficient and scalable big data processing solutions using PySpark Debug optimize and enhance existing data workflows and pipelines Work independently as well as collaboratively in Agile teams Translate business requirements into technical data solutions Manage multiple tasks and deliver within deadlines in a fast-paced environment What we bring to the table: Opportunity to work on modern data engineering stack including Databricks and Python 6-month engagement duration with potential for extension based on performance Exposure to large-scale data engineering projects in an international environment Agile-driven collaborative working culture Lets Connect: Want to discuss this opportunity in more detail Feel free to reach out. Recruiter: Giftson Paul Davidson Phone:; Extn : 151 E-mail: LinkedIn:
