Lead – Data Devops

About us:

We are proudly owned by Auto & General Insurance Company. The A&G group is made up of trusted and well-known brands such as iSelect Compare the Market and Budget Direct. In late 2024 iSelect and Compare the Market formed a new Aggregation business within the A&G group and now support millions of Australians to compare and buy their personal finance and household products such as insurance energy and loans.

iSelect

At iSelect were passionate about making Australians lives easier by saving them time money and effort. With over 20 years experience we are a one-stop destination for comparison and purchasing across insurance utilities and personal finance products. We compare a wide range of brands and our trained comparison experts help customers to compare select and save more putting them in control.

Compare the Market

At CTM our Noble Purpose is to make a difference in the lives of Australians by simplifying the process of making sound financial decisions. This Noble Purpose is our cultural north star guiding our direction prioritisation and decision making.

How you fit:

The Lead - Data DevOps is responsible for designing implementing and maintaining highly available scalable and secure data pipelines and platforms. This role ensures efficient CI/CD processes infrastructure automation and monitoring to support the organizations data ecosystem.

The role requires a strong background in DataOps DevOps cloud data platforms and infrastructure automation. They will work closely with Data Engineering Security IT Operations and Analytics teams to streamline data workflows enhance system reliability and optimize cloud-based data environments.

The primary focus areas include infrastructure as code (IaC) CI/CD for data pipelines monitoring & observability cost optimization and cloud security.

What you do:

Data Platform & Pipeline Automation

• Design implement and maintain CI/CD pipelines for data engineering workflows.
• Automate data pipeline deployments monitoring and scaling to ensure reliability and performance.
• Develop and enforce infrastructure-as-code (IaC) practices using tools like Terraform AWS CloudFormation or Azure Bicep.
• Work with Data Engineers and IT teams to optimize ETL/ELT processes and cloud data platforms.

Cloud Infrastructure & Security

• Oversee data infrastructure in cloud environments (AWS Azure GCP) ensuring high availability and security.
• Implement role-based access control (RBAC) encryption and network security best practices within data environments.
• Collaborate with Security and Compliance teams to ensure regulatory compliance
• Optimize cloud storage compute and networking costs while maintaining performance

Monitoring Observability & Incident Response

• Implement monitoring solutions for real-time alerting anomaly detection and performance optimization.
• Utilize logging and observability tools such as Prometheus Grafana Datadog Splunk or CloudWatch.
• Establish incident response processes including automated rollback strategies and disaster recovery plans.
• Work with Site Reliability Engineering (SRE) teams to ensure high system availability and minimal downtime.

Data Ops & DevOps Best Practices

• Advocate for DataOps methodologies to improve collaboration between Data Engineering IT and Analytics teams.
• Implement version control automated testing and release management for data pipelines.
• Continuously improve CI/CD workflows to enhance deployment frequency and reduce failures.
• Enforce best practices for data governance lineage tracking and auditability.

Collaboration & Stakeholder Engagement

• Work closely with Data Engineers Analytics Security and IT Operations to streamline data infrastructure.
• Act as the subject matter expert (SME) on Data DevOps providing training and mentorship to team members.
• Engage with business leaders and data users to align infrastructure improvements with business goals.

What you need:

• Bachelors or Masters degree in Computer Science Data Engineering Cloud Computing or a related field/relevant experience.
• Experience & Skills:
• 3-5 years of experience in DevOps Data Engineering or Cloud Infrastructure roles.
• Hands-on experience with CI/CD tools (e.g. GitHub Actions Jenkins Azure DevOps).
• Proficiency in containerization and orchestration (Docker Kubernetes ECS EKS AKS) as plus.
• Experience implementing infrastructure as code (IaC) using Terraform
• Expertise in cloud data platforms (AWS Azure GCP) with services like AWS Lambda S3 RDS Redshift Glue EMR Databricks BigQuery etc.
• Strong understanding of cloud networking security policies and identity management (IAM VPC KMS Secrets Manager etc.).
• Experience setting up monitoring and alerting using tools like Prometheus Grafana CloudWatch Datadog etc.
• Ability to troubleshoot performance issues and optimize system reliability.
• Strong experience in Python Bash or PowerShell for automation and scripting.
• Ability to write automation scripts for infrastructure provisioning deployments and data pipeline management.
• Ability to lead a small team and mentor junior engineers and promote DevOps best practices across teams.
• Strong communication skills to work with both technical and non-technical stakeholders

Whats in it for you:

• Career Opportunities
• A competitive salary plus generous uncapped commission scheme
• The opportunity to work with a well-known household brand and public listed company
• Work with a team of highly energised motivated and driven professionals
• 3 recharge days per year (yes in addition to your annual leave!)
• A free coffee each day a funky caf and a state of the art work space
• An impressive Employee Benefits Programs

If you are someone who fosters team spirit has resilience and aspires to continue learning and developing their skills to drive commercial outcomes you will have tremendous success in helping us achieve our progressive growth agendas. High energy levels and a love of having fun are also important to working in this energetic business.