2025 - 0309 Application Test Engineer (MS Windows / Web) (NS) - TUE 14 Oct

Application Test Engineer (MS Windows / Web)

The NATO Information and Communication Agency (NCIA) located in The Hague, The Netherlands, is involved in the maintenance and development of command and control capabilities for NATO as well as in the support of these capabilities.

In the light of these activities, the C2 Business Area is looking for an Application Test Engineer with experience in MS Windows and web applications testing.

The main objective of this statement of work is to provide application-testing expertise and ensure the quality of the various applications within the domains of Force Management and Operations Planning, CBRN and Land C2 application services.

Delivery of the services under this statement of work requires skills and experience as an application test engineer for MS Windows and web applications.

The contractor will be working under the direction and guidance of a NCIA technical lead and product manager, and will provide the services using an agile and iterative approach using multiple sprints. Each sprint is planned for an approximate duration of 2 weeks.

The work shall be executed mainly on-site at the NCIA The Hague. In exceptional cases, sprints might require the visit to other NATO sites and for such visits, the travel cost will be reimbursed separately. The work will be carried out accessing the Azure DevOps NATO Software Factory development environment as well as other Agency testbeds.

The key responsibilities of an Application Test Engineer to be performed are:

• Test Planning and Design: Creating detailed test plans, test cases, and test scripts to cover various aspects of the application, including functionality, performance, security, and usability.
• Test Execution: Running tests, both manually and through automation, to identify bugs and ensure the application meets requirements.
• Defect Management: Logging and tracking defects using bug-tracking tools, and working with developers to resolve them.
• Test Automation: Developing and implementing automated test scripts to improve testing efficiency and coverage.
• Collaboration: Working closely with developers, business analysts, and project managers to ensure a smooth and efficient testing process.
• Test Environment Management: Setting up and maintaining test environments to ensure accurate and reliable testing.
• Reporting: Generating reports on test results, defect status, and overall application quality.
• Continuous Improvement: Identifying areas for improvement in the testing process and implementing solutions to enhance efficiency and effectiveness.

The scope of the assignment will be covering the following deliverables:

• Test Plan (to be delivered periodically as required to support the activity under test;
• Test Case / Procedure (to be delivered as required to support the specific activity under test for the period needed;
• Test Run / Logs (Test logs generated are archived and provided to support the event/activity; defects are recorded);
• Test Report (to be delivered for each event supported);
• Issue/Observation Reports (these reports will be manually entered and their specific counts will be further reflected in the test reporting);
• Test scripts (test scripts developed and maintained in a master repository to support the various test setups and executions, including automated testing);
• Technical reports (generated as required and directed).

The services underpinning the deliverables are required to be delivered within NCIA business hours as per paragraph 9 of this statement of work and their related acceptance criteria. The content and scope of each sprint will be agreed with the technical lead and product manager during the sprint-planning meeting in writing.

The contractor shall participate in workshops, events and conferences related to the supported services, as requested by the NCIA project team.

The contractor shall report to the Agency's designated technical lead and product manager.

The contractor shall participate in daily status update meetings, sprint planning, sprint retrospectives and other meetings, in person or via electronic means using conference call capabilities, according to the technical lead's instructions. When using conference call capabilities, the camera feature should be used at all times.

For each sprint to be considered as complete and payable, the contractor:

• Must report the progress of his/her service and any impediments during the sprint daily meetings;
• Should report verbally the service completed during the sprint review;
• Should prepare a demo of the service completed and present it to the team during the sprint demo meeting.

Due to the agile and iterative approach, there is a need to define a set of specific arrangements between the NCIA and the contractor that specifically defines the deliverables to be provided for each sprint as well as their associated acceptance criteria. This includes sprint planning, execution and review processes, which are detailed below:

Sprint Planning:

• Objective: Plan the objectives for the upcoming sprint.
• Kick-off meeting: Conduct a bi-weekly (every two weeks) meetings with the contractor to plan the objectives of upcoming sprint and review contractor's effort to meet the agreed deliverables.
• Set sprint goals: Define clear, achievable goals for the sprint and associated acceptance criteria, including specific delivery targets and required level of effort, to be recorded in the sprint meeting minutes.
• Backlog Review: Review and prioritise the backlog of previous sprints.
• Assess each payment milestone cycle duration of two sprints. State of completion and validation of each sprint status and sign off sprints to be submitted for payment as covered in section 5.

Sprint Execution:

• Objective: Contractor to execute the agreed "sprint plans" with continuous monitoring and adjustments.
• Regular meetings between NCIA and the contractor to review sprint progress, address issues, and make necessary adjustments to the plan.
• Progress Tracking: Contractor to use a shared dashboard or tool to track the status of the sprint deliveries and any issues.
• Quality Assurance: Contractor shall ensure that the quality standards agreed for the sprint deliverables are maintained throughout the sprint.
• Quality Control: NCIA to perform the final quality control of the agreed deliverables and provide feedback on any issues.

Sprint Review:

• Objective: Review the sprint performance and identify areas for improvement.
• At the end of each sprint, there will be a meeting between the NCIA and the contractor to review the outcomes against the acceptance criteria comprising sprint goals.
• Define specific actions to address issues and enhance the next sprint.

Sprint Payment:

• For each sprint to be considered as complete and payable, the contractor must report the outcome of their service during the sprint, first verbally during the sprint review meeting and then in writing within three (3) days after the sprint's end date. A report must be sent by email to the NCIA service or project manager, listing the achievements and services held against the agreed tasking list set for the sprint.
• The contractor's payment for each sprint will be depending upon the achievement of agreed acceptance criteria for each task, defined at the sprint planning stage. This will include specific delivery targets and quality standards.
• The payment shall be dependent upon successful acceptance as set in the above planning/review meetings. This will follow the payment milestones that shall include a completed Delivery Acceptance Sheet (DAS) – (Annex B).
• Invoices shall be accompanied with a Delivery Acceptance Sheet (DAS) – (Annex B) signed by the contractor and service or project manager.
• If the contractor fails to meet the agreed acceptance criteria for any task, the NCIA reserves the right to withhold payment for that task/sprint.

At the end of the contract, but as a minimum the end of each calendar year, the contractor shall provide a project closure report that is summarizing the activities during the period of performance at high level.

The following is expected from this statement of work:

1) Completion of the activities underpinning the deliverables agreed in each sprint meeting as per sections 3 and 4 above;

2) Produce sprint completion reports (format: e-mail update), which include details of activities performed and resulting deliverables. Specifically, the resolved Product

Backlog Items will be broken down into number of tasks, traceable through Azure DevOps NATO Software Factory development environment.

Payment schedule will be at the end of each two sprints, following the acceptance of the deliverables and the sprint report.

The NCIA reserves the possibility to exercise a number of sprints at a later time and or additional options, based on the same sprint deliverable timeframe and cost, depending on project priorities and requirements.

The payment shall be dependent upon successful