Data Engineer Contract MelbourneSydney
Job Location:
Salary:
Experience Required:
Posted on:
Vacancies:
Job Summary
Requirements
- 5 years of Data Engineering experience in enterprise environments.
- 3 years of hands-on experience working with Azure Databricks and Apache Spark.
- Proven experience delivering data solutions on Azure cloud platforms.
- Experience designing and implementing scalable data pipelines and data lake architectures.
- Strong experience with Agile delivery methodologies.
- Ability to work independently and contribute immediately within a project-based environment.
- Strong stakeholder management and communication skills.
Key Responsibilities :
- Design develop and optimize data pipelines using Azure Databricks and Apache Spark.
- Build and maintain robust ETL/ELT frameworks to ingest transform and load data from multiple source systems.
- Develop scalable data solutions leveraging Azure Data Factory (ADF) Azure Data Lake Storage (ADLS) and Azure Synapse Analytics.
- Implement data quality validation and monitoring frameworks to ensure data integrity and reliability.
- Collaborate with solution architects business analysts data scientists and stakeholders to deliver project outcomes.
- Optimize Spark workloads cluster configurations and data processing performance.
- Support deployment activities through CI/CD pipelines and DevOps practices.
- Troubleshoot production issues and provide timely resolution of data platform incidents.
- Contribute to technical documentation knowledge transfer and operational handover activities.
- Ensure adherence to security governance compliance and enterprise data standards.
Required Skills:
Essential Experience : 5 years of Data Engineering experience in enterprise environments. 3 years of hands-on experience working with Azure Databricks and Apache Spark. Proven experience delivering data solutions on Azure cloud platforms. Experience designing and implementing scalable data pipelines and data lake architectures. Strong experience with Agile delivery methodologies. Ability to work independently and contribute immediately within a project-based environment. Strong stakeholder management and communication skills. Key Responsibilities : Design develop and optimize data pipelines using Azure Databricks and Apache Spark. Build and maintain robust ETL/ELT frameworks to ingest transform and load data from multiple source systems. Develop scalable data solutions leveraging Azure Data Factory (ADF) Azure Data Lake Storage (ADLS) and Azure Synapse Analytics. Implement data quality validation and monitoring frameworks to ensure data integrity and reliability. Collaborate with solution architects business analysts data scientists and stakeholders to deliver project outcomes. Optimize Spark workloads cluster configurations and data processing performance. Support deployment activities through CI/CD pipelines and DevOps practices. Troubleshoot production issues and provide timely resolution of data platform incidents. Contribute to technical documentation knowledge transfer and operational handover activities. Ensure adherence to security governance compliance and enterprise data standards. Duration: 6 Months and possible extension Eligibility: Australian/NZ Citizens/PR Holders only Email:
