Data Engineer
Job Location:
Job Location:
Kuala Lumpur - Malaysia
Monthly Salary:
Not Disclosed
Monthly Salary:
Posted on:
9 hours ago
Posted on:
Vacancies:
1 Vacancy
Vacancies:
Job Summary
Job Description:
- Create and manage data pipelines for diverse sources in our data lake/warehouse.
- Collaborate with analysts to implement necessary data transformations. Research relevant data sources for property development.
- Develop web scraping scripts for data acquisition.
- Automate deployment and testing of data pipelines using CI/CD.
- Utilize Azure AWS and other technologies for data solutions.
- Implement data integration between systems.
- Monitor and optimize pipeline performance.
- Ensure data security compliance and quality.
- Stay updated on data engineering trends.
Requirements:
- Bachelors or Masters degree in Computer Science Engineering or related field.
- Proven experience as a Data Engineer with strong understanding of data engineering concepts.
- Hands-on experience with Azure (Azure Function Azure Data Factory Azure Databricks) and/or AWS (AWS Glue Lambda Amazon Redshift).
- Proficiency in Python Java or Scala and experience with data processing frameworks (e.g. Spark Hadoop).
- Knowledge of data modelling warehousing and database technologies.
