Job Description
More than just a recruitment company. At PRTR we have been a part of our customers success for 30 years as their total HR solutions partner. With 550 dedicated professionals and over 15000 outsourced staff we will continue to carry out our mission to develop a better career a better life and a better society and thrive to become the No.1 people solutions organization in Southeast Asia.
Roles and Responsibilities
- Designing and implementing scalable reliable and efficient data pipelines for ingesting processing and storing large amounts of data from a variety of sources using cloud-based technologies Python and PySpark.
- Building and maintaining data lakes data warehouses and other data storage and processing systems on the cloud.
- Writing and maintaining ETL/ELT jobs and data integration scripts to ensure smooth and accurate data flow.
- Implementing data security and compliance measures to protect data privacy and ensure regulatory compliance.
- Collaborating with data scientists and analysts to understand their data needs and provide them with access to the required data.
- Staying up-to-date on the latest developments in cloud-based data engineering particularly in the context of Azure AWS and GCP and proactively bringing new ideas and technologies to the team.
- Monitoring and optimising the performance of data pipelines and systems identifying and resolving any issues or bottlenecks that may arise.
Qualifications
- Bachelors or Masters degree in Computer Science Data Science or a related field.
- Minimum of 5 years of experience as a Data Engineer with a strong focus on cloud-based data infrastructure.
- Proficient programming skills in Python Java or a similar language with an emphasis on Python.
- Extensive experience with cloud-based data storage and processing technologies particularly Azure AWS and GCP.
- Familiarity with ETL/ELT tools and frameworks such as Apache Beam Apache Spark or Apache Flink.
- Knowledge of data modelling principles and experience working with SQL databases.
- Strong problem-solving skills and the ability to troubleshoot and resolve issues efficiently.
- Excellent communication and collaboration skills to work effectively with cross-functional teams.