Senior Data Engineer

We are seeking an experienced Senior Data Engineer to design build and optimize scalable data pipelines on Azure-based cloud platforms. This role is highly hands-on and focuses on delivering robust production-grade data engineering solutions using Azure Databricks PySpark Delta Lake Python and SQL. You will work closely with analytics data science and business teams to enable reliable high-quality data at scale.

Design build and optimize scalable reliable data pipelines using Azure Databricks PySpark and Delta Lake to support large-scale data transformation and analytics needs. Develop and maintain complex SQL and Python code for data ingestion transformation validation and quality checks across cloud data platforms. Implement endtoend pipelines leveraging Azure services such as ADLS and Azure Data Factory while configuring and managing Databricks notebooks workflows jobs and clusters for performance and cost efficiency. Apply CI/CD best practices to deliver productiongrade data engineering solutions monitor and troubleshoot pipeline performance and collaborate closely with stakeholders to translate business requirements into robust secure and highquality data solutions.

Qualifications :

• Strong hands-on experience with Azure Databricks including notebooks workflows cluster configuration and job orchestration.
• Proven expertise in PySpark and Spark-based data processing for large-scale datasets.
• Solid experience with Delta Lake for transactional data management and optimisation.
• Proficiency in Python and SQL for building production-grade data transformation and validation logic.
• Experience working with Azure data services particularly ADLS and Azure Data Factory (ADF).
• Hands-on experience implementing CI/CD pipelines for data engineering workloads.
• Strong understanding of data engineering best practices including performance tuning partitioning and schema design.
• Experience working in agile cloud-first enterprise environments.

Additional Information :

Why join us:

At NCS Australia youll be part of a team that values your growth and development. Our comprehensive global learning and development program will equip you with the skills and knowledge needed to succeed. Plus youll be mentored by some of the industrys best and have access to a worldwide network of career opportunities. Join us now and discover just how far your career can go!

We are in the process of refreshing our employee benefits so dont forget to ask our Talent Acquisition Team about whats on offer. 

We take pride in our commitment to valuing and supporting our people and the communities we serve. Diversity inclusivity and being different are not just buzzwords for us - they are integral to our culture. We are dedicated to upholding Equal Employment Opportunity Principles and providing reasonable accommodations to all our applicants. If you require any assistance or modifications to navigate our careers site or complete your application please dont hesitate to reach out to us at .

Other important stuff: 

To be eligible for a position with us applicants will need to have valid work rights for Australia and be willing to undergo a comprehensive background checking process including probity and police checks.

As we navigate through our business integration process we will continue to issue contracts under our existing entities of ARQ Dialog CatapultBI ICEMEDIA Eighty20 Riley or AccessHQ.

Visit our website  more information.

Agencies: Weve got this. We request that you do not contact NCS employees outside of the Talent Acquisition team. NCS exclusively accepts resumes from agencies on our preferred supplier panel through the NCS Agency Portal. Agencies that submit resumes must have a valid fee agreement and be assigned to the particular requisition by the Talent Acquisition team. Any resumes that are submitted outside of this process will become the sole property of NCS. If a candidate is hired outside of this process no fee or payment will be given.

Why join us: 

NCS Australia is where you can feel at home nurturing your talents and skills as we make tomorrow together one day at a time. Our benefits include paid parental leave initiatives focused on your well-being and discounted health insurance. You will also enjoy discounts on various products and services and be regularly recognised and rewarded for high performance. We are committed to your career development through our Capability Fingerprint industry and partner training programs special interest groups and an AI-driven learning platform. No matter where you are in your career we offer meaningful work and opportunities for growth.

NCS Australia is an equal-opportunity employer and we take pride in our commitment to valuing and supporting our people and the communities we are dedicated to attracting retaining and developing our people regardless of gender identity ethnicity sexual orientation disability and age. Applications are encouraged from all sectors of the community and we strongly encourage applications from the Veterans Aboriginal and/or Torres Strait Islander community.

At NCS Australia we are committed to supporting adjustments throughout the recruitment and selection process as well as during employment. We actively support and encourage people with disability to apply. 

Agencies:

Weve got this. We request that you do not contact NCS employees outside of the Talent Acquisition team. NCS exclusively accepts resumes from agencies on our preferred supplier panel through the NCS Agency Portal. Agencies that submit resumes must have a valid fee agreement and be assigned to the particular requisition by the Talent Acquisition team. Any resumes that are submitted outside of this process will become the sole property of NCS. If a candidate is hired outside of this process no fee or payment will be given.

Why join us: 

NCS Australia is where you can feel at home nurturing your talents and skills as we make tomorrow together one day at a time. Our benefits include paid parental leave initiatives focused on your well-being and discounted health insurance. You will also enjoy discounts on various products and services and be regularly recognised and rewarded for high performance. We are committed to your career development through our Capability Fingerprint industry and partner training programs special interest groups and an AI-driven learning platform. No matter where you are in your career we offer meaningful work and opportunities for growth.

NCS Australia is an equal-opportunity employer and we take pride in our commitment to valuing and supporting our people and the communities we are dedicated to attracting retaining and developing our people regardless of gender identity ethnicity sexual orientation disability and age. Applications are encouraged from all sectors of the community and we strongly encourage applications from the Veterans Aboriginal and/or Torres Strait Islander community.

At NCS Australia we are committed to supporting adjustments throughout the recruitment and selection process as well as during employment. We actively support and encourage people with disability to apply. 

Agencies:

Weve got this. We request that you do not contact NCS employees outside of the Talent Acquisition team. NCS exclusively accepts resumes from agencies on our preferred supplier panel through the NCS Agency Portal. Agencies that submit resumes must have a valid fee agreement and be assigned to the particular requisition by the Talent Acquisition team. Any resumes that are submitted outside of this process will become the sole property of NCS. If a candidate is hired outside of this process no fee or payment will be given.

Work rights and background checks: 

To be eligible for a position with us applicants will need to have valid work rights for Australia and be willing to undergo a comprehensive background checking process including probity and police checks

Remote Work :

No

Employment Type :

Full-time