2025-0309 Application Test Engineer (MS Windows Web) (NS) TUE 14 Oct

Deadline Date: Tuesday 14 October 2025

Requirement: Application Test Engineer (MS Windows / Web)

Location: The Hague. NL

Full Time On-Site: No

Time On-Site: 60% (3 days/week) ON SITE NCI Agency The Hague

40% (2 days/week) Off Site working from home either from duty-location or from their NATO home country

Period of Performance: 2025 BASE: As soon as possible and not later than 10 November until 31 December 2025 with possibility to exercise sprints from the following options:

2026 Options: 1st January 2026 until 31st December 2026

2027 Options: 1st January 2027 until 31st December 2027

2028 Options: 1st January 2028 until 31st December 2028

Required Security Clearance: NATO SECRET

1 INTRODUCTION

The NATO Information and Communication Agency (NCIA) located in The Hague The Netherlands is involved in the maintenance and development of command and control capabilities for NATO as well as in the support of these capabilities.

In the light of these activities the C2 Business Area is looking for an Application Test Engineer with experience in MS Windows and web applications testing.

2 OBJECTIVES

The main objective of this statement of work is to provide application-testing expertise and ensure the quality of the various applications within the domains of Force Management and Operations Planning CBRN and Land C2 application services.

Delivery of the services under this statement of work requires skills and experience as application test engineer for MS Windows and web applications

3 SCOPE OF WORK

The contractor will be work under the direction and guidance of a NCIA technical lead and product manager and will provide the service using an agile and iterative approach using multiple sprints. Each sprint is planned for an approximate duration of 2 weeks.

The work shall be executed mainly on-site at the NCIA The exceptional cases sprints might require the visit to other NATO sites and for such visits the travel cost will be reimbursed separately. The work will be carried out accessing the Azure DevOps NATO Software Factory development environment as well as other Agency testbeds.

The key responsibilities of an Application Test Engineer to be performed are:

• Test Planning and Design: Creating detailed test plans test cases and test scripts to cover various aspects of the application including functionality performance security and usability.
• Test Execution: Running tests both manually and through automation to identify bugs and ensure the application meets requirements.
• Defect Management: Logging and tracking defects using bug-tracking tools and working with developers to resolve them.
• Test Automation: Developing and implementing automated test scripts to improve testing efficiency and coverage.
• Collaboration: Working closely with developers business analysts and project managers to ensure a smooth and efficient testing process.
• Test Environment Management: Setting up and maintaining test environments to ensure accurate and reliable testing.
• Reporting: Generating reports on test results defect status and overall application quality.
• Continuous Improvement: Identifying areas for improvement in the testing process and implementing solutions to enhance efficiency and effectiveness.

The scope of the assignment will be covering the following deliverables:

• Test Plan (to be delivered periodically as required to support the activity under test;
• Test Case / Procedure (to be delivered as required to support the specific activity under test for the period needed;
• Test Run / Logs (Test logs generated are archived and provided to support the event/activity; defects are recorded);
• Test Report (to be delivered for each event supported);
• Issue/Observation Reports (these reports will be manually entered and their specific counts will be further reflected in the test reporting);
• Test scripts (test scripts developed and maintained in a master repository to support the various test setups and executions including automated testing);
• Technical reports (generated as required and directed).

The services underpinning the deliverables are required to be delivered within NCIA business hours as per paragraph 9 of this statement of work and their related acceptance criteria. The content and scope of each sprint will be agreed with the technical lead and product manager during the sprint-planning meeting in writing.

The contractor shall participate in workshops events and conferences related to the supported services as requested by the NCIA project team.

4 COORDINATION AND REPORTING

The contractor shall report to the Agencys designated technical lead and product manager.

The contractor shall participate in daily status update meetings sprint planning sprint retrospectives and other meetings in person or via electronic means using conference call capabilities according to the technical leads instructions. When using conference call capabilities the camera feature should be used at all times.

For each sprint to be considered as complete and payable the contractor:

• Must report the progress of his/her service and any impediments during the sprint daily meetings;
• Should report verbally the service completed during the sprint review;
• Should prepare a demo of the service completed and present it to the team during the sprint demo meeting.

Due to the agile and iterative approach there is a need to define a set of specific arrangements between the NCIA and the contractor that specifically defines the deliverables to be provided for each sprint as well as their associated acceptance criteria. This includes sprint planning execution and review processes which are detailed below:

Sprint Planning

• Objective: Plan the objectives for the upcoming sprint.
• Kick-off meeting: Conduct a bi-weekly (every two weeks) meetings with the contractor to plan the objectives of upcoming sprint and review contractors effort to meet the agreed deliverables.
• Set sprint goals: Define clear achievable goals for the sprint and associated acceptance criteria including specific delivery targets and required level of effort to be recorded in the sprint meeting minutes.
• Backlog Review: Review and prioritise the backlog of previous sprints.
• Assess each payment milestone cycle duration of two sprints. State of completion and validation of each sprint status and sign off sprints to be submitted for payment as covered in section 5.

Sprint Execution

• Objective: Contractor to execute the agreed sprint plans with continuous monitoring and adjustments.
• Regular meetings between NCIA and the contractor to review sprint progress address issues and make necessary adjustments to the plan.
• Progress Tracking: Contractor to use a shared dashboard or tool to track the status of the sprint deliveries and any issues.
• Quality Assurance: Contractor shall ensure that the quality standards agreed for the sprint deliverables are maintained throughout the sprint.
• Quality Control: NCIA to perform the final quality control of the agreed deliverables and provide feedback on any issues.

Sprint Review

• Objective: Review the sprint performance and identify areas for improvement.
• At the end of each sprint there will be a meeting between the NCIA and the contractor to review the outcomes against the acceptance criteria comprising sprint goals.
• Define specific actions to address issues and enhance the next sprint.

Sprint Payment

• For each sprint to be considered as complete and payable the contractor must report the outcome of their service during the sprint first verbally during the sprint review meeting and then in writing within three (3) days after the sprints end date. A report must be sent by email to the NCIA service or project manager listing the achievements and services held against the agreed tasking list set for the sprint.
• The contractors payment for each sprint will be depending upon the achievement of agreed acceptance criteria for each task defined at the sprint planning stage. This will include specific delivery targets and quality standards.
• The payment shall be dependent upon successful acceptance as set in the above planning/review meetings. This will follow the payment milestones that shall include a completed Delivery Acceptance Sheet (DAS) (Annex B).
• Invoices shall be accompanied with a Delivery Acceptance Sheet (DAS) (Annex B) signed by the contractor and service or project manager.
• If the contractor fails to meet the agreed acceptance criteria for any task the NCIA reserves the right to withhold payment for that task/sprint.

At the end of the contract but as a minimum the end of each calendar year the contractor shall provide a project closure report that is summarizing the activities during the period of performance at high level.

5 DELIVERABLES AND PAYMENT MILESTONES

The following is expected from this statement of work:

1) Completion of the activities underpinning the deliverables agreed in each sprint meeting as per sections 3 and 4 above;

2) Produce sprint completion reports (format: e-mail update) which include details of activities performed and resulting deliverables. Specifically the resolved Product

Backlog Items will be broken down into number of tasks traceable through Azure DevOps NATO Software Factory development environment.

Payment schedule will be at the end of each two sprints following the acceptance of the deliverables and the sprint report.

The NCIA reserves the possibility to exercise a number of sprints at a later time and or additional options based on the same sprint deliverable timeframe and cost depending on project priorities and requirements.

The payment shall be dependent upon successful acceptance of the deliverables the sprint report and the Delivery Acceptance Sheet (DAS) (Annex B) signed by the contractor and the NCIA service or project manager.

Invoices shall be accompanied with a signed Delivery Acceptance Sheet (Annex B) The following deliverables are expected within the performance period.

2025 BASE: 10 November 2025 31 December 2025

Deliverable: 3 sprints (Number of sprints is estimated and will be adjusted based on actual starting date.)

Payment Milestones: Upon completion of each two sprints following acceptance

2026 OPTIONS: 01 January 2026 to 31 December 2026

Deliverable: Up to 26 sprints

Cost Ceiling: Price will be determined by applying the price adjustment formula as outlined in CO115786 AAS Special Provisions article 6.5.

Payment Milestones: Upon completion of each two sprints following acceptance

2027 OPTIONS: 01 January 2027 to 31 December 2027

Deliverable: Up to 26 sprints

Cost Ceiling: Price will be determined by applying the price adjustment formula as outlined in CO115786 AAS Special Provisions article 6.5.

Payment Milestones: Upon completion of each two sprints following acceptance

2028 OPTIONS: 01 January 2028 to 31 December 2028

Deliverable: Up to 26 sprints

Cost Ceiling: Price will be determined by applying the price adjustment formula as outlined in CO115786 AAS Special Provisions article 6.5.

Payment Milestones: Upon completion of each two sprints following acceptance

6 SCHEDULE

This task order will be active immediately after signing of the contract by both parties.

The BASE period of performance is as soon as possible but not later 10 November 2025 and will end no later than 31 December 2025.

If the 2026 option is exercised the period of performance is 01 January 2026 to 31 December 2026.

If the 2027 option is exercised the period of performance is 01 January 2027 to 31 December 2027.

If the 2028 option is exercised the period of performance is 01 January 2028 to 31 December 2028.

7 SECURITY

Performance of the services described in this statement of work requires a valid NATO SECRET security clearance prior to the start of the engagement.

The contractor shall be aware of all security rules pertaining to the handling of NATO classified information.

Personnel Security Clearance (PSC). Individuals who require access or may have access to information classified NATO Confidential or above during the course of their duties shall have a PSC at the appropriate level which is valid for the duration of the authorized addition such individuals are required to: Have a need-to-know; Have been briefed on their security obligations in respect to the protection of NATO classified information; And have acknowledged their responsibilities either in writing or an equivalent method which ensures non-repudiation.

8 CONSTRAINTS

All the documentation provided under this statement of work will be based on NCI Agency templates or otherwise agreed with service or project manager.

All scripts documentation required code test case will be stored under configuration management and/or in the provided NCIA tools.

All developed solutions tools and code under this statement of work will be property of the NCIA.

9 PRACTICAL ARRANGEMENTS

This is a deliverables-based contract.

The contractor will be required to work 60% (3 days/week) on site NCI Agency The Hague and up to 40% (2 days/week) off-site working from home either from duty-location or from their NATO home country.

NCIA core business hours: Monday to Thursdayand Friday(CET)

Contractor Furnished Services: Contractor shall furnish everything required to perform the contract.

Access to relevant environments and licenses to commercial third-party libraries required to perform the contract will be provided by NCI Agency.

The work depicted in this statement of work is expected to be carried by a SINGLE RESOURCE.

10 TRAVEL

There may be requirements to travel to other sites within NATO for completing these tasks.

Travel expenses are out of scope and will be borne by the NCI Agency separately in accordance with Article 5.5 of AAS Framework Contract and within the limits of the NCIA Travel Directive.

11 QUALIFICATIONS

See Requirements

7 SECURITY

• Performance of the services described in this statement of work requires a valid NATO SECRET security clearance prior to the start of the engagement.

11 QUALIFICATIONS

The consultancy support for this service requires an Application Test Engineer with the following qualifications:

Required minimum qualifications are:

• Bachelors degree in computer science/engineering or related field;
• 3-5 years of extensive and progressive expertise as application test engineer of MS Windows and web applications;
• Proven knowledge of and practical experience in modern test engineering and test management methods and paradigms;
• Experience Microsoft Azure DevOps and Test Manager;
• Proficiency in scripting like Java Python C# and Typescript or programming experience.

Preferable qualifications are:

• Masters degree in computer science/engineering or related field;
• Knowledge Web API or similar;
• Knowledge of MS SQL Server;
• Experience with Git source control;
• Recent experience in planning and execution of validation activities within large-scale C2 exercises;
• Certifications like ISTQB or CSTE.

Relevant soft skills are:

• Strong problem-solving and analytical skills;
• Strong attention to detail;
• Good communication skills both spoken and written in English;
• Good presentation skills taking in consideration the target audience;
• Team player friendly and always willing to help others and to share knowledge;
• Good collaboration skills with the ability to work in a multinational and diverse team;
• Growth mind set always wanting to improve and to learn;
• Proactivity and taking ownership.