Urgent requirement of Senior Data Engineer - Perm/Contract - Sydney
Requirements
- Embedded within one of our line-of-business data and reporting teams you will collaborate with stakeholders to understand their data objectives and champion the use and evolution of the central data platform.
- You will work within a domain-aligned data product team consisting of line-of-business data owners and SMEs working alongside engineering resources provided by Technology.
- You will be hands-on in the ingestion of core data assets by the product teams their curation into source-aligned data products and the creation of consumable products in the form of files reports dashboards and APIs.
- You will form part of a data engineering practice and community that will design and drive the adoption of standards frameworks and best practices.
- You will ensure data pipelines and products are designed and built to be run within a critical market infrastructure environment by the persistent product teams who build them.
- A hands-on engineer with deep experience working with a variety of AWS services for ingestion curation storage orchestration data management analytics and distribution.
- Good knowledge of data modelling and how to structure data within a data warehousing environment while following best practice data management standards.
- Solid data product architecture and solution design experience with a good working knowledge of DevSecOps practices to achieve high levels of automation across complex data pipelines.
- Experience in real-time streaming using Confluent Kafka and AWS services.
- Experience with Airflow AWS Glue Apache Iceberg Athena RedShift and DataZone.
- Experience in Data visualization tools such as Tableau/Power BI/QuickSight.
- Solid experience in Python Spark and Terraform/Terragrunt. Nice to have
- Experience working within a platform/product operating model where frameworks and development kits are leveraged to achieve standardisation re-use and scale.
- A working knowledge of data mesh/data product concepts with federated data ownership at the domain-level but operating on centralised platforms and governance.
- Experience working in the Capital Markets domain and understanding of derivative products.
Duration: Permanent / 06 Months and possible extension
Eligibility: Australian/NZ Citizens/PR Holders only
Email: