Senior Azure Data Engineer – Databricks Sydney Only

Experience:8Years

Technology Focus: Azure Databricks PySpark Python SQL Data Engineering

We are looking for an experienced Senior Azure Data Engineer / Lead with 8 years of software/data engineering experience and strong expertise in Azure Databricks Python SQL and Apache Spark.

The ideal candidate will have hands-on experience designing and developing complex data pipelines distributed data processing solutions ETL frameworks and large-scale data platforms within Agile environments.

• 8 years of experience in software/data engineering and development preferably within Agile environments.
• Strong hands-on expertise in Python and SQL; Java experience is an advantage.
• Design develop and optimize complex data pipelines and ETL workflows.
• Strong experience with Apache PySpark Databricks Azure Synapse Azure Data Factory (ADF) and Azure Functions.
• Hands-on experience with data modelling data warehousing and large-scale ETL architectures.
• Experience working with MS SQL ADLS Gen2 and NoSQL/data storage technologies.
• Good understanding of object storage and Big Data architectures particularly Azure Data Lake Storage Gen2.
• Experience managing and optimizing distributed systems and clusters for large-scale/batch data processing.
• Knowledge of MLOps architecture and practices.
• Strong understanding of modern data modelling concepts and enterprise data warehouse architecture.
• Familiarity with Azure DevOps Git and modern CI/CD practices.
• Good knowledge of ETL and data warehousing methodologies.
• Informatica experience is an advantage.
• Ability to work closely with architects developers data teams and business stakeholders in an Agile environment.