Senior Test Analyst in AI & Automation

We are seeking a Senior Test Analyst to design automate and execute testing for LLMpowered applications and agentic systems operating across multiple platforms and services. You will lead Responsible AI testing (safety guardrails fairness explainability) redteaming/adversarial evaluation and multi-system integration testing. You will also coordinate releases across DEV SIT UAT and PROD producing auditready test evidence for Cyber and Change boards.

Youll partner closely with AI engineering platform SecOps and Network Operations teams to assure safety reliability and compliance of AI capabilities at scale.

As a Senior Test Analyst you will design and execute comprehensive test strategies for AIdriven and multisystem solutions ensuring quality across integration API functional and Responsible AI domains. You will develop automated test suites using Python or shell scripting validate LLM behaviour and agent workflows and assess system reliability safety and explainability through structured test scenarios. You will contribute to redteam and adversarial testing to identify vulnerabilities such as prompt injection misuse of agent tools and guardrail weaknesses. The role requires leading testing across multiple environments (DEV SIT UAT PROD) preparing highquality test evidence and documentation for Cyber and Change Board reviews and coordinating with crossfunctional teams to ensure smooth releases prompt defect resolution and adherence to governance standards.

Qualifications :

• Strong API & integration testing across distributed systems; experience with multisystem orchestration and asynchronous workflows.
• Demonstrated understanding of LLM behavior and agent architectures (function/tool calling planners RAG memory guardrails).
• Handson automation in Python and shell
• Responsible AI testing knowledge: safety categories content moderation hallucination detection explainability (e.g. traceable RAG) data/privacy considerations evaluation metrics.
• Experience preparing and presenting test evidence and signoff documentation to Cyber/Security and Change boards.
• Proven track record coordinating release testing across DEV/SIT/UAT/PROD including cutovers and postdeployment checks.

Additional Information :

Why join us: 

NCS Australia is where you can feel at home nurturing your talents and skills as we make tomorrow together one day at a time. Our benefits include paid parental leave initiatives focused on your well-being and discounted health insurance. You will also enjoy discounts on various products and services and be regularly recognised and rewarded for high performance. We are committed to your career development through our Capability Fingerprint industry and partner training programs special interest groups and an AI-driven learning platform. No matter where you are in your career we offer meaningful work and opportunities for growth.

NCS Australia is an equal-opportunity employer and we take pride in our commitment to valuing and supporting our people and the communities we are dedicated to attracting retaining and developing our people regardless of gender identity ethnicity sexual orientation disability and age. Applications are encouraged from all sectors of the community and we strongly encourage applications from the Veterans Aboriginal and/or Torres Strait Islander community.

At NCS Australia we are committed to supporting adjustments throughout the recruitment and selection process as well as during employment. We actively support and encourage people with disability to apply. 

Agencies:

Weve got this. We request that you do not contact NCS employees outside of the Talent Acquisition team. NCS exclusively accepts resumes from agencies on our preferred supplier panel through the NCS Agency Portal. Agencies that submit resumes must have a valid fee agreement and be assigned to the particular requisition by the Talent Acquisition team. Any resumes that are submitted outside of this process will become the sole property of NCS. If a candidate is hired outside of this process no fee or payment will be given.

Work rights and background checks: 

To be eligible for a position with us applicants will need to have valid work rights for Australia and be willing to undergo a comprehensive background checking process including probity and police checks

Remote Work :

No

Employment Type :

Full-time