Consultancy for Final Project Evaluation

2. Purpose and objectives of the evaluation

Mandate for conducting evaluation: SNV has implemented SWaSSH4A since 2022 and is as a partner of the Austrian Development Agency mandated and interested in conducting a final project evaluation. As per the project document (section 7.2) the end-of-project evaluation will be outsourced. The TOR at hand describe the corresponding assignment for which SNV seeks to hire a consulting team.

Purpose of the evaluation: The main purpose of this evaluation is learning and accountability for better project design and implementation in the future.

Users of the evaluation: SNV will use the evaluation results for learning and designing future projects. ADA is the second main user of the evaluation results.

The key objectives of the final evaluation are:

1. To assess the projects Relevance Effectiveness and prospects for Sustainability
2. To identify facilitating and hindering factors for results achievement
3. To document lessons learned and good practices and good practices and
4. To provide practical recommendations based on the findings.

3. Scope

This evaluation will assess the relevance effectiveness and prospects for sustainability of the SWaSSH4A from the project design phase (end 2020 to October 2022) to its implementation up to the end of data collection of this evaluation (November 2022 to early October 2025). The evaluation will cover all output areas of the project as summarised under section 1.3 (project background).

Geographical scope: The evaluation will generally cover all 4 districts where the project is being implemented with a deeper analysis in a sample of the 34 project sub-counties (Table 1).

Project districts and Subcounties

Alebtong:    Omoro Adwir Angetta Amugu Abako Awei Aloi Akura Abia and Apala

Dokolo:       Kangai Adeknino Kwera Okwongodul Dokolo Okwalongwen Bata Amwoma   Agwata & Adok

Kole:           Akalo Bala Ayer Aboke Okwerodot and Alito

Lira:            Awiodyek Agali Itek Bar Ogur Agweng Aromo and Ayami

Applicants for this assignment are asked to suggest in their technical offer a sampling strategy that ensures sampled Subcounties and villages are equally spread over the 4 project districts. The sampling strategy should start from the 51 villages with rehabilitated boreholes under the project combined with other criteria such as flood-prone areas hard to reach areas and high demand water sources. Beyond the villages with rehabilitated boreholes the sampling strategy should also include villages with water sources that were repaired or have benefited from Operation and Maintenance Systems. The sampling strategy will be refined during the inception phase. Data collection in the field includes a minimum of 10 villages.

In terms of evaluation criteria assessed the evaluation will be limited to relevance effectiveness and prospects for sustainability of the SWaSSH4A project. As part of the assessment of relevance effectiveness and sustainability the evaluation will also assess how the project contributed to gender equality (OECD-DAC policy marker on Gender equality (Marker 1) and how and with which results (and against which challenges) it applied the Gender Equity and Social Inclusion (GESI) The OECD-DAC evaluation standards as well as the ADA PP Evaluation Guidelines are to be used as framework for this evaluation.

4. Evaluation Questions

Relevance:

1. To what extent does the project address the specific WASH needs of vulnerable groups (women persons with disabilities poor households) and the affordability of services for the poorest wealth quintiles in the project districts
2. How well has the project aligned with Ugandas National Development Plan III national WASH targets and the climate resilience of WASH infrastructure by local governments and how could the contribution/relevance to national WASH priorities and goals have been enhanced/be enhanced in future

Effectiveness:

1. To what extent have the projects intended results been achieved and what factors influenced (non-) achievements and how Can unintended results be identified. and if so which ones
2. To what extent and how has the project enhanced women and girls access to water sanitation and hygiene promoted their participation in decision-making processes and enabled them to equally exercise their rights
3. To what extent and how has the project strengthened the capacity of the district and subcounty local governments and line agencies (MWE Rural Water and Sanitation Regional Centre 1) to lead and institutionalise quality sanitation demand creation and rural water supply development including Behaviour Change Communication (BCC) for WASH
4. How effective are the Water Source Committees and other supportive structures (Handpump Mechanics/ Association) in ensuring proper Operation and maintenance of water facilities as well as ensuring water safety and service continuity What good practice and challenges can be identified in this context

Prospects for sustainability:

1. What is the likelihood that the responsible governance structures (DWSCCs SWSSBs WSCs HPMA) and established multi-stakeholder coordination mechanisms will continue to sustain achieved results after project closure and why
2. To what extent are water supply services financially sustainable at the WSC level and are water tariffs affordable and accessible to the poorest households (short- mid- and long term and how can challenges in this context be addressed
3.  How resilient are the WASH systems and infrastructure to climate-related risks such as flooding and drought and what measures are in place to ensure their durability and continued service delivery

5. Design and Approach

The evaluation will follow ADA evaluation guidelines and OECD/DAC norms and standards as well as common ethical criteria for evaluations ensuring the highest levels of integrity and accountability. The evaluation should be gender sensitive participatory and respect the rights and dignity of all involved and promote a learning approach. Stakeholder and community engagement should be done by applying the principles of informed consent confidentiality transparency and a focus on the common good. The consultant/s will propose a methodology and are expected to deploy multiple methods drawing on different sources and triangulating information to successfully deliver this assignment. A human rights-based approach (HRBA) ADCs cross-cutting issues as well as the basic principles and quality standards applying to ADAs programme and project design should be incorporated in the methodology.

The following data collection methods are suggested and deemed sufficiently rigorous to allow for a complete fair and unbiased the technical proposal the evaluators will suggest how to use these (and others) to conclude the assessment with the utmost quality.

1. Document Review (secondary Data): The evaluators will review: the baseline and midline reports project proposal (budget log frame) workplans project reports (technical and financial) programmatic monitoring data collected and analysed using a data management platform project publications (conference contributions technical briefs and case studies) and any other documents deemed critically important for the evaluation
2. Onsite visual observation: In conjunction with other methods the evaluator will observe community water use conditions and O&M-related activities household sanitation and hygiene facilities ask questions visit communities water sources sub-counties and districts and make sketches take photographs or videos etc)
3. Key informant interviews: Semi-structured interviews will be conducted with key informants (groups or individuals) including project staff SNVs local partners staff sub-county and district officials Water User Committee (WUC) members Sub-county Water and Sanitation Board (SWSSB) members ADA staff Ministry of Water and Environment (MWE) officials at RWSC 1 local leaders of the target communities beneficiaries Hand Pump Mechanics (HPMs) and representatives of their associations spare parts dealers other development partners and organisations as relevant.
4. Focus Group Discussions (max. of 4-8 people each group): including community members across wealth and social groups women and men people with disabilities the elderly and youth.

In their technical proposal the evaluation team may suggest which methods are used to assess which areas of analysis and gather information from groups of informants. Specifically the evaluators should review the evaluation questions and add from their experience as well as indicate how the data and information to answer the questions will be obtained and analysed. The Evaluation report (in an annex) should capture all details of the evaluators own data and information collection (questionnaires group compositions). All documents gathered and generated including photos will be handed over to SNV at the end of the assignment in a well-structured manner.

For the assessment of outputs the evaluation can use the projects programmatic monitoring data and the results and data of the baseline and midline surveys. Surveys to collect WASH coverage data at individual households are thus not foreseen under the evaluation. It should be mentioned that an endline survey is planned following parallel with the evaluation and drawing on its draft results might be possible.

6. Timeline and Deliverables

6.1 Timeline

The evaluation is expected to be conducted between end of August to end of October 2025.

The following deliverables are to be produced and submitted as part of this assignment:

• Draft and final inception report with a work plan refined and based on technical proposal (maximum 10 pages excluding annexes) in line with the structure and content requirements of Annex 5 of the ADA Guidelines for Programme and Project evaluations.
• Presentation of preliminary findings (virtual meeting)
• Draft and final evaluation report (not more than 30 pages excluding annexes) in line with the structure and content requirements of Annex 6 of the ADA Guidelines for Programme and Project evaluations.
• Completed Result-Assessment Form (RAF) submitted together with the draft evaluation report
• Physical Presentation of final evaluation results at regional and national dissemination workshops

The inception report evaluation report and RAF need to fulfil ADA standards for program and project evaluations and will be quality checked by SNV and ADA before approval.

The estimated number of working days needed for this assignment are 100. For details see below:

Milestones/deliverables and timeline

1. Evaluation kick-off/Inception meeting (1) by Mid September

2. Document Review Preliminary Interviews and Draft and Present Inception Report (12) by Mid-September

3. Finalise Inception Report addressing feedback by SNV and ADA (during meeting and in written on the report) (3) by End September

4. Data collection in the field (District Subcounty Village Levels) and Virtual (60) by October

5. Data processing and analysis (7) by October

6. Presentation of Preliminary Findings and submission of Draft evaluation report with Results Assessment

Form (RAF) (12) by End October

7. Finalise evaluation report with RAF addressing Feedback by SNV and ADA (5) End October

The evaluation team is responsible for preparing a comprehensive report which presents findings to address the evaluation questions highlight key changes as a result of the project intervention draw out lessons learned present findings and recommendations. The report should reflect comments and feedback received from stakeholders. The evaluation team should refer to the ADA guidelines for the structure of the evaluation reports (Annex 5). The language of the report should be English with no jargon and with specialist terms explained.

The recommendations must be related to the conclusions without replicating them. A recommendation derives directly from one or more conclusions. Recommendations should be as realistic operational and pragmatic as possible; that is they should take careful account of the circumstances currently prevailing in the context of the project and of the resources available to implement them. They could concern policy organizational and operational aspects for both SNV and ADA.

The evaluation deliverables to be generated by the evaluation team include:

1. Draft and Final Inception report

The evaluators will prepare an inception report which details the evaluators understanding of the evaluation and how the evaluation questions will be addressed. This is to ensure that the evaluators and the stakeholders SNV and ADA have a shared understanding of the evaluation. The inception report will include the evaluation matrix summarising the evaluation design methodology evaluation questions data sources and collection analysis tool for each data source and the measure by which each question will be evaluated. (for structure and content requirements see Annex 5 on quality checklist for Inception Report). The draft inception report will be reviewed by SNV and ADA and the Final Inception Report should be submitted two (2) weeks from the date of contract signing.

2. Draft and final evaluation report

The evaluation team will prepare a draft Evaluation Report for the project cognizant of the proposed format of the report and checklist used for the assessment of evaluation reports (for structure and content requirements see Annex 6 of ADA guidelines). The draft report (max. 45 pages excluding Annexes) will be reviewed by SNV and ADA to ensure that the evaluation meets the required quality criteria. The report will be produced in English. The consultant needs to prepare a presentation of the preliminary findings and share it with SNV and major stakeholders.  A completed Results- Assessment Form (RAF) must be submitted together with the draft evaluation report (Annex 9 of the guidelines). Report to be coherently structured with a logical flow       

3. The final report (30-50 pages) for the project will include comments from SNV and ADA and will be submitted 5 days after receiving all the comments. This will be submitted to SNV through the SWaSSH4A Project Manager for validation. It will include recommendations (for future programming and to inform policy) and conclusions. (See Annex 6 of ADA guidelines for Quality Checklist for Evaluation Report). The end of project evaluation report should be by 23rd October 2025.

Payment Schedule

The consultant will be paid in three instalments as indicated below.

Payment                           Deliverable to trigger payment                    % of contract amount payable

1st Instalment                   Inception report for the assignment                                      30%

2nd Instalment                  Draft evaluation report                                                          40%

3rd Instalment                   Final evaluation report                                                          30%

The contract price will be fixed regardless of changes in the cost components.

7. Evaluation Management Arrangements

The evaluation will be managed by SNV in collaboration with a review panel that includes representatives from SNV and ADA with the SWaSSH4A project manager serving as the main focal point supported by the project team to ensure that all deliverables meet the Terms of Reference. The evaluation management team will uphold the OECD/DAC principles particularly impartiality and independence throughout the process.

It is SNVs responsibility to avail data and documents as listed above and assist the evaluators in getting in touch with the relevant key informants and provide contacts and information to organise group discussions. It is the responsibility of the evaluators to follow up plan and schedule for the activities. The evaluators are responsible for logistics arrangements (travel accommodation). SNV shall not bear any cost.

Responsibilities of the Team Leader

1. Finalise the evaluation questions methodology and work plan in consultation with the evaluation manager.
2. Review background materials and conduct initial research and analysis.
3. Develop a detailed schedule for fieldwork and other activities including time for unexpected delays.
4. Assign roles and responsibilities within the evaluation team ensuring gender balance and efficient use of time.
5. Coordinate with SNV and other partners to ensure timely and high-quality fieldwork.
6. Lead the fieldwork including interviews focus group discussions and other data collection methods.
7. Keep the evaluation manager and stakeholders informed especially if there are changes or challenges as well as a list of people to be interviewed planned visits and team member responsibilities.
8. Analyse the data collected and develop findings conclusions and practical recommendations.
9. Present preliminary findings to the evaluation manager SNV and other relevant teams.
10. Prepare and revise the final report executive summary and annexes based on feedback from SNV and ADA.
11. Ensure the evaluation process is objective inclusive and meets quality standards.
12. Lead the writing of all reports and ensure they are delivered within the agreed timeframe.
13. Participate in regional and national-level dissemination

Responsibilities of SNV (Evaluation Manager)

1. Participate in the kick-off meeting to ensure everyone understands the purpose scope of the evaluation and ensure standard templates are used.
2. Provide all relevant background documents including previous evaluations reports studies existing data and information that will support the evaluation.
3. Provide information on SNVs approach to using score cards to assess progress in governance through facilitated discussion and scoring in groups and with key informants
4. Write an official letter to introduce the evaluation expert or team to key stakeholders.
5. Review and give feedback on the draft inception report and formally approve it before data collection starts.
6. Participate in meetings with the evaluation team to monitor progress and provide technical guidance.
7. Ensure the evaluation remains independent and objective throughout the process.
8. Facilitates dissemination workshops at the regional and national level (outside this assignment expert/team participates only).

Qualifications :

8. Requirements for the Evaluators

Eligibility: the evaluators must not have been involved in the design or implementation of the programme or project being evaluated.

A team of consultants or a consulting company can apply for this consultancy. The evaluation team should consist of a minimum of 3 and a maximum of 5 members and must be gender diverse.

The team leader has to be nominated with full responsibility for conducting the evaluation.

All team members must contribute to all phases of the evaluation ensuring deliverables are availed within the time frame and with quality.

Key specific requirements to be fulfilled by the evaluation team:

Team Leader:

• Demonstrated experience in leading and conducting independent external evaluations of similar size and thematic area of development cooperation projects (leading at least 1 contributing to at least 3)
• Relevant International working experience in countries with comparable setting.

All team Members:

• Demonstrated experience in contributing to independent external evaluations of similar size and thematic area of development cooperation projects

• Demonstrated experience in designing and conducting qualitative data collection and analysis in the context of WASH and including in rural communities.
• Demonstrated expertise in WASH and related environmental and climate change issues.
• Project design implementation and monitoring experience in development cooperation.
• Expertise and experience in human rights-based WASH and gender-responsive approaches
• Familiarity with Uganda and its national WASH sector
• High level of professionalism and ability to work independently and within deadlines
• Strong communication and meeting/workshop facilitation skills
• Excellent computer skills ability to draft well-formatted presentations and documents including compelling tables and graphs that support the presentation of results
• Language skills: excellent spoken and written English

Requisite documentation for the consultant

• Company Legal registration documents (Articles of Association and Powers of Attorney)
• Tax Compliance Certificate
• Valid trading license
• Company profile
• CVs of all the proposed staff for the assignment
• List of References (with valid contacts) for earlier similar tasks undertaken

Important to Note:

• At least one team member must be fluent in the Local Languages

Familiarity with ADA and SNV is an asset.

Additional Information :

9. Specifications for the Submission of Offers

Offers of interested bidders need to consist of a technical and a financial offer.

The technical offer should have a maximum of 10 pages presenting the understanding of the assignment and how the herein proposed scope evaluation questions and targets design approach including the sampling strategy will be put into action as well as a preliminary work addition the offer should include the CV(s) of the consultants team as annexes preferably no longer than 5 pages. CVs of all team members need to be submitted and a clear attribution of roles must be included in the technical proposal. All the CVs submitted must be signed by the staff proposed for the team.

The financial offer in EUR has to include the fee rate per expert and the estimated number of working days per expert as well as travel and other expenses broken down in detail and directly related to the conduct of data collection activities. Other reimbursable costs or lump sums are not eligible. It is assumed that data collection can be concluded within max. of 60 working days for the entire team. The cost for dissemination event(s) will be borne by SNV. The net costs for this evaluation are estimated between EUR 35000 and 40000.

The detailed CVs should clearly respond to the above-described requirements and include three reference persons (names phone and email contact). To allow objective assessment of the consultants match with the requirements proofs such as reference review or evaluation reports and other written documents such as scientific articles papers (working version also possible) can be submitted (max. three relevant documents). Teams of individual consultants need to specify who the team leader is (responsible for timely delivery at the highest quality) and the sole liable and contractual partner of SNV. All team members must have a significant role in each of the evaluation phases (inception data collection and analysis reporting).

Administrative requirements for Firms/companies: For companies the following shall also be required: Registration documents in Uganda; Valid trading licenses /NGO permit; Memo & articles of association/ constitution; Powers of attorney; CVs of the entire proposed team in response to above requirements and reference assignments. All team members must have a significant role in each of the evaluation phases (inception data collection and analysis reporting).

Email submissions of all required documents (Administrative Technical and financial proposals in compressed folders) clearly indicating Final evaluation of SWaSSH4A project in the subject line should be submitted to the email address: not later than 10:00 am (Nairobi Time) on 5th September  2025. Any questions requiring clarification shall be sent to the email address above with Clarification on Final Evaluation of SWaSSH4A project in the subject line of the mail no later than 29th August 2025.

The Contractor will have to agree that:

• The final product (or excerpts of it or parts of it) will be made public as per the ADAs Guidelines for Programme and Projects Evaluations
• Photos and videos need to be taken in line with SNVs guidelines (the required consent form and guidance can be availed from SNV)

Note on value added tax: With reference to Article 24.3 of the Austrian Development Agency General Terms and Conditions of Contract for Consultant Services and Similar Intellectual Services (hereinafter General Terms) the Contractor shall only be entitled to charge to the CA value added taxes incurred during the implementation of the Service Contract in the event that at the time of the submission of the final financial statement the Contractor can prove that such value added taxes are not recoverable by any means and it is established that they are effectively borne by him/her.

10. Bid Evaluation/Evaluation Criteria

Bids shall be evaluated through evaluation stages: Preliminary examination technical evaluation and financial evaluation.

Preliminary examination

Preliminary examination will be based on a pass/fail criteria. It is aimed at checking compliance with the bid requirements such as submission of bid within the stipulated time frame and required format signing of the bid documents submission letter legal status of the company (registration certificate certificate of registration articles of association) powers of attorney valid income tax clearance certificate valid trading license among others.

Technical evaluation

The technical proposal shall be scored out of 80%. Table 2: Technical evaluation criteria

Financial evaluation

The financial proposal will be scored out of 20% using the formula below.

FsLowest Price considering all proposals (LP) Quated price of the proposal   being evaluated (QP)*20%

Determination of the best bidder

The best evaluated bidder is one that obtains the highest combined scores (CS) for technical score and financial scores determined as follows:

CsT sFs

Where:  CS Combined score Ts Technical score and Fs Financial score.

11. Annexes

These (and other) documents relevant to the project will be shared with the selected team after signing the consultancy agreement and during the inception meeting. The list below just gives an indication as to the type of documents available for the desk review:

Project documents and data

1. Project description (project document including annexes)
2. Grant Agreement
3. Approved work plans
4. Project progress reports
5. Baseline and midline reports
6. Project publications

ADA guiding documents.

• Evaluation Policy of the Austrian Development Cooperation1
• ADA Guidelines for Programme and Project Evaluation2
• Result-Assessment Form (RAF) for Programme and Project Evaluations3

1

3