Head of Data Engineering

Who We Are

iSelect and Compare the Market are trusted well-known brands dedicated to helping millions of Australians compare and buy personal finance and household products such as insurance energy and loans. Together we provide a seamless customer-focused experience that makes it easier for Australians to find the right products to suit their needs.

What Youll be doing

As Head of Data Engineering you will lead the development scalability and operational excellence of our core data platforms. Youll drive best-in-class engineering practices ensure data integrity and availability and oversee automation monitoring and optimisation of data workflows. You will also shape modern data architectures implement cloud-based big data solutions and champion DataOps methodologies to deliver efficient reliable and innovative data capabilities.

• Define and deliver the data engineering roadmap ensuring governance security and data quality.
• Architect build and optimise scalable data solutions utilising Databricks as our Enterprise Date Platform.
• Implement DataOps practices for automation CI/CD and testing.
• Build and lead high-performing teams while fostering collaboration and innovation.
• Stay ahead of emerging trends and adopt tools to enhance automation and cost efficiency.

Whats in it for you

At Compare the Market and iSelect we offer a dynamic and collaborative work environment where your talents will be recognised and rewarded. Youll have the opportunity to make a meaningful impact and contribute to our continued success we also provide:

• Vibrant and social community who have annual celebrations family fun days and regular events.
• Enjoy flexible work arrangements including the option of one day working from home each week or a 9-day fortnight.
• Enjoy additional ME leave days
• Employee discounts on Car Home Travel insurance.
• Income protection insurance provided to support you in the event of non-work-related illness or injury.

What Sets You Apart

• Demonstrated leadership and team-building skills with experience managing high-performing engineering teams.
• Ability to develop strategy influence senior stakeholders and align data solutions with business objectives.
• Extensive experience with cloud-based data platforms (AWS Azure GCP) and data lake/lakehouse architectures.
• Expertise in big data frameworks (Apache Spark Hadoop Snowflake Redshift BigQuery Delta Lake) and strong skills in Python PySpark SQL.
• Proven ability to design and optimise ETL/ELT pipelines for structured and unstructured data.
• Hands-on experience with DataOps practices including CI/CD (Terraform dbt GitHub Actions Azure DevOps) and orchestration tools (Airflow Dagster Prefect).
• Knowledge of real-time streaming technologies (Kafka Kinesis Delta Live Tables).
• Strong understanding of data governance security and compliance (Unity Catalog GDPR CCPA HIPAA) including RBAC encryption and data masking.

Sound like a good fit Lets talk.

At Compare the Market and iSelect were proud to be an Equal Opportunity Employer and we embrace diversity.

This role is available to candidates with unlimited Australian working rights only. Sponsorship is not available.

Compare the Market and iSelect are committed to protecting your privacy. All applications and personal information are kept confidential and securely stored in line with Australian privacy laws used only for recruitment purposes and not shared with third parties outside the Auto and General Holdings Group without your consent.