Lead Data Engineer
Job Location:
Monthly Salary:
Posted:
Application Deadline:
Vacancies:
Job Summary
Our federal client is looking for a Data Engineer to join its Enterprise Analytics and Technology Services (EATS) section to work across its data and analytics platforms. We are seeking candidates with strong experience in developing Extract Transform Load (ETL) and Extract Load Transform (ELT) processes and /or the development of data products and complex data visualisations.
The role will be responsible for design development and testing activities across several data movement data transformation and data visualisation processes within DAFF. The data movement and transformation processes focus on the preparation of data for use in decision making processes across the department utilising modern cloud technology (Azure & Databricks) to enable operational analytics use cases.
To be successful in the role you must have a strong work ethic including taking ownership providing leadership having a high level of productivity and working with a multi-disciplinary team in an Agile development environment. You must have suitable qualifications or training in data engineering techniques and at least 5 years experience working as a data engineer.
Responsibilities include but not limited to:
- Be responsive flexible and work collaboratively as part of an agile team.
- Strong relationship building and negotiation skills.
- Strong written and oral communication skills.
- Creating and maintaining automated ingest and transformation patterns and frameworks.
- Designing building and maintaining data ingest and transformation solutions to meet current and emerging needs.
- Assisting project teams achieve objectives that align with departmental divisional and program priorities.
- Supporting data engineers in delivery teams through regular quality reviews and constructive feedback on utilising data assets to produce quality data products.
The successful candidate will require experience with the following techniques and technologies:
- Azure hosted services including:
- Data Integration - Data Factory SQL Server Integration Services and/or Databricks
- Data Store - SQL Server and/or Data Lake Storage
- Analytics - Azure Databricks Azure Machine Learning ArcGIS Enterprise
- Development tools DevOps Visual Studio
- Data technology solutions sourcing (Oracle Ingres Azure SQL Server) automated ingestion
- Data Preparation
- Transformation of data into formats tailored for advanced analytics and AI use cases Parquet and/or Delta
- Data Visualisation
- Analyse and interpret complex data sets and to identify trends and patterns.
- Design principles to create appropriate visualisations for target audience.
- Visualisation tools - Power BI.
Required Skills:
Key duties and responsibilities We are seeking a Senior Storage Engineer who has experience working in large complex ICT environments. The successful candidate will be responsible for deploying configuring managing and tuning the departments storage systems at physical and virtual layers. The Senior Storage Engineer will also be responsible to deploy configure and manage Storage Area Network Switches as required. The successful candidate should have knowledge of new and emerging technologies to shape the future direction of the departments storage and data platforms. The role requires producing design documents build documents operations document handover and sharing of knowledge with other team members and mentoring junior team members. Essential criteria experience and knowledge in the design administration tuning and troubleshooting of one or more of the following storage systems: HPE 3PAR Primera Alletra Storage Systems NetApp E-Series Dell Data Domain Brocade Fabric Switches SANnav Scality Artesca to produce design build and operations documents for processes and procedures. communication (verbal and written) and collaboration skills to work with skilled technical people including making presentations to staff and stakeholders and coaching staff. / Knowledge of following technology: Veeam Backup and Replication
About Company
35 employees
Indigitise is a leading professional services enterprise with a strong presence across Australia, specializing in IT consulting, IT services, and digital transformation solutions. Our team has extensive experience providing innovative and effective IT solutions to the Commonwealth Gov ... View more
