Senior Data Engineer
Share
Job Location:
Monthly Salary:
Posted on:
30+ days ago
Vacancies:
1 Vacancy
Job Summary
As a Senior Data Engineer you will design build and maintain reliable scalable data pipelines and infrastructure that underpin analytics AI and operational systems across Nearmap. Youll ensure timely accurate and secure delivery of data from multiple sources into the warehouse enabling downstream teams to create trusted actionable insights. The role partners closely with Analytics Engineers Engineering and technology teams and other stakeholders to provide a robust cost-efficient and well-governed data platform that supports the companys strategic growth.
Key Responsibilities
- Design develop and maintain scalable pipelines to move and transform data from various batch and streaming sources into our data warehouse.
- Collaborate with Analytics Engineers to provide clean reliable foundational data for transformation and modelling.
- Build and manage reverse-ETL capabilties to sync data from the platform to operational systems and business tools.
- Partner with data source owners (application teams product teams third-party vendors) to establish schema expectations and SLAs ensure smooth data evolution and drive best practices on data quality.
- Monitor data pipelines and troubleshoot issues to ensure data accuracy and availability.
- Document data flows transformations and data catalogs/dictionaries.
- Optimize data workflows for performance reliability and cost efficiency.
- Develop Snowflake data warehouse infrastructure manage warehouse resources maintain data assets and apply security and access controls.
- Architect deploy and maintain highly available and scalable data systems and infrastructure in AWS (Snowpipe and Airbyte).
- Manage integrations and workflow across data warehouse AWS and other systems to ensure integrity and availability of assets.
- Establish best practices for data architecture security governance and engineering standards.
- Prototype infrastructure and evaluate new data technologies that enable the team to leverage advanced analytics and intelligent automation.
Qualifications :
- 5 years experience as data engineer or data platform engineer
- 3 years hands on experience with Snowflake management
- End-to-end data pipeline design development and maintenance
- Experience with Infrastructure as Code using Terraform and CI/CD
- Proficiency with data pipeline tools (dbt Airbyte Fivetran Hightouch Airflow Dagster etc)
- Experience with building data platform in AWS (Glue Kinesis S3 API Gateway etc).
- Building integration between data warehouse and external systems
Good to Have:
- Modern data warehouse modelling practices
- Hands-on experience with Kubernetes Docker and container orchestration
- Experience with platform and pipeline observability and monitoring
- Data lake and lakehouse management experience
Tech Stack at Nearmap - must have skills required:
- SQL
- Python
- Terraform
- Snowflake
- AWS Services (Glue Firehose S3 API Gateway ECS ASG etc)
- CI/CD
Good to have:
- dbt
- Data lake file formats (Parquet Avro Iceberg) and partitioning strategies
- Other cloud providers
Additional Information :
Some of our benefits
Nearmap takes a holistic approach to our employees emotional physical and financial wellness. Some of our current benefits include:
- Quarterly wellbeing day off - Four additional days off annually for your YOU Days
- Access to LinkedIn Learning
- Wellbeing and technology allowance
- Annual flu vaccinations
- Hybrid flexibility for this role
- Nearmap subscription (of course!)
- Stocked kitchen with access to all the snacks you need
- In-office lunch every Tuesday and Thursday at our Sydney CBD office
- Showers available for anyone cycling to work or lunchtime gym-goers!
Working at Nearmap
We move fast and work smart; often wearing multiple hats. We adapted to remote working with ease and are continually looking at ways to improve. Were proud of our inclusive supportive culture and maintain a safe environment where everyone feels a sense of belonging and can be themselves.
If you can see yourself working at Nearmap and feel you have the right level of experience we invite you to get in touch.
Read the product documentation for Nearmap AI:
a deep dive into Nearmap AI listen to AI Systems Senior Director Mike Bewley on the Mapscaping podcast to agencies: Thanks but we got this! Nearmap does not accept unsolicited resumes from recruitment agencies and search firms. Please do not email or send unsolicited resumes to any Nearmap employee location or address. Nearmap is not responsible for any fees related to unsolicited resumes.
Remote Work :
No
Employment Type :
Full-time
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
About Company
The sky's not the limit at Nearmap. Nearmap is the Australian-founded, global tech pioneer innovating the location intelligence game. Customers rely on Nearmap for consistent, reliable, high-resolution imagery, insights, and answers to create meaningful change in the world and propel ... View more
