Urgent requirement of Databricks Architect - Contract - Brisbane
Requirements
- Seeking a highly skilled Databricks Architect to lead the migration of enterprise data workloads from Greenplum to Databricks.
- This role requires deep expertise in distributed data systems cloud architecture and data engineering best practices.
- The architect will be responsible for designing scalable secure and high-performance data pipelines and lake house solutions on Databricks.
Key Responsibilities:
- Lead the end-to-end migration strategy from Greenplum to Databricks.
- Analyze existing Greenplum schemas workloads and performance metrics.
- Design target architecture using Databricks Lakehouse Delta Lake and Spark.
- Develop scalable ETL/ELT pipelines using PySpark SQL and Databricks Workflows.
- Ensure data quality integrity and consistency throughout the migration process.
- Collaborate with data engineers DBAs and business stakeholders to validate and optimize migrated data.
- Implement best practices for data governance security and compliance in Databricks.
- Provide technical leadership and mentorship to the migration team.
- Document architecture decisions migration plans and operational procedures.
Required Skills and Qualifications:
- Bachelor s or master s degree in computer science Information Systems or related field.
- 8 years of experience in data engineering or architecture roles.
- Proven experience with Greenplum and MPP systems.
- Experience with data migration tools and frameworks.
- Strong hands-on experience with Databricks Apache Spark and Delta Lake.
- Proficiency in SQL Python and PySpark.
- Experience with cloud platforms (preferably AWS).
- Knowledge of data modeling performance tuning and distributed computing.
- Certifications in Databricks or cloud platforms is preferred
- Excellent communication leadership and problem-solving skills.
Duration: 06 Months and possible extension
Eligibility: Australian/NZ Citizens/PR Holders only