Terms of Reference

Final Project Evaluation

Integrated support programming for vulnerable communities in Sudan

Program Title: Integrated support programming for vulnerable communities in Sudan

Donor: USAID/BHA

Implementing Agency: Relief International

Location: North Darfur and Blue Nile States

Grant implementation period: Aug 1, 2020 to July 31, 2022

Evaluation time period: The evaluation is scheduled to take place from 20th July 2022 to 8th August 2022

BACKGROUND:

Relief International has been working to improve access to health and nutrition services through an integrated health and nutrition program with a BHA project*:*implemented in 17 health facilities in North Darfur; and 5 health facilities in Blue Nile State.

SUMMARY

RI implements PHC and nutrition activities targeting <5 children, adolescents, adult and elderly. The health supports the SMOH’s community Health Model for RMNCH working closely Midwives, CHWs, to carry out basic curative services based on Community Integrated Management of Childhood Illness. While through the nutrition intervention RI targets SAM and MAM children <5 years of age and, PLWs and caregivers of infants through improved access to CMMA and IYCF; and WASH program provides hygiene promotion, water supply, and sanitation services to communities in the same localities. Through the health, nutrition, and WASH services the project targeted to reach a total of 680, 256 people in both States.

Through the health, nutrition, and WASH intervention, RI supports the provision of essential and lifesaving services for communities in Zamzam IDP camp and Al-Malha localities in North Darfur; and Damazine and Rosaries localities in Blue Nile over a course of two years. Additionally, the program also builds the capacity of MOH seconded staff in health services and management of acute malnutrition.

PROGRAM OBJECTIVE

Improved access to quality primary health, nutrition, and WASH services

PURPOSE OF EVALUATION:

A final evaluation will be conducted by an external consultant to assess the overall performance of the project with a focus on the project’s relevance, effectiveness, efficiency, and sustainability. The final evaluation will also identify best practices and key lessons in all technical aspects and program management approaches. Findings of the evaluation will be shared with USAID/BHA, relevant government bureaus, and Clusters. The evaluation primarily answers the following questions:

Appropriateness of Design and Monitoring systems

To determine how well the program was designed and monitored, against the objectives, benchmarks and processes established in the program design document

Coherence

How was the project coordinated with the activities of other agencies (including national bodies) and how coherent are RI’s activities with national policy, RI country strategy and other humanitarian actors?

Coverage

Were activities coverage met as planned and did project respond to needs of the target group?

Target versus actual

Assess the progress made towards achieving each Project Goal (or technical sector) based on the current log frame, design and monitoring data.

Results

Determine the changes/results and/or potential impact of the program.

Strengths/ Weaknesses

Determine strengths (including successful innovations and promising practices) and weaknesses (factors impeding progress) of the program planning, design, implementation M&E, and ongoing community management in the program.

Efficiency and Effectiveness

To determine whether the resources (financial, human & materials) have been used economically and wisely for the well-being of the community.

Equity

To determine how the program involved and benefited different genders, ethnic minorities, children and the disabled throughout the planning, design, implementation, monitoring and evaluation and ongoing management of the program.

Capacity

Assess RI structure and staff capacity to undertake program implementation.

Assess the status of stakeholders (primarily HFs and Community leaders) to design, implement, monitor and evaluate activities.

Ownership and Sustainability

To assess the overall management and structure of the program, particularly focusing on the potential for the program to be owned by local people and for benefits to continue after the RI may close.

Participation

Assess the level and quality of participation by women, children, people with disabilities, and the poor.

Cross cutting issues

What measures have RI put in place in order to ensure gender equality and sensitivity to other gender issues throughout the program cycle; and security to employees?

Beneficiary accountability

What effective and feasible system is in place to gather beneficiary feedback?

Lessons learned

What are the lessons we learned in terms of overall program management , activity implementation, monitoring , reporting,

Recommendations

Provide specific, actionable, and practical recommendations for focus on key sectors during the next phase.

EVALUATION METHODS:

The consultant is expected to propose and an appropriate study design involving both quantitative and qualitative data collection entailing beneficiary household survey; Health Facility document reviews, focused group discussions, Key informant interview, and stakeholder interviews. The evaluation should combine both quantitative and qualitative methods.

The study design should entail a holistic evaluation, comprising both the qualitative and quantitative methodologies designed to capture the performance of health and nutrition sectors. The consultant and his/her team are expected to use different data sources, comprising of primary data (through qualitative and quantitative household survey questionnaires) and secondary sources including project document, progress reports, assessments, etc. Furthermore, the study will be undertaken through a participatory and collective manner based on active involvement of project beneficiaries and other stakeholders. The following data collection and analysis methods will also be used to address the evaluation questions. The following data collection and analysis methods will be used to address the evaluation questions:

• Review of documentation and secondary data – The evaluator will assess project records and reports as well as the results of prior surveys carried out with beneficiaries.
• Structured and Semi-structured interviews and focus groups with beneficiaries – The evaluator will prepare household questionnaire to measure progress on indicators (specific to WASH KAP indicators and Health KAP indicators), and FGD/KII checklists to facilitate focus group discussions with beneficiaries, and semi-structured interviews with key stakeholders, such as the representatives from the local authorities, SMoH and selected beneficiaries
• Site visits and observations – The evaluator will visit various sites where program activities have been implemented.
• Staff interviews – Evaluator will hold discussion with program, finance, HR, and Logistic staffs

The evaluator will be responsible for analysis, interpretation and compilation of data.

SCHEDULE:

The duration period for the evaluation to be 25 days. The data collection will be conducted in 8 days, while the remaining days will be used for data entry, analysis and report write up.

REPORTING

Relief International requests the following sections in the report:

• Background
• List of Acronyms
• Table of Contents which identifies page numbers for the major content areas of the report.
• Executive Summary (2 to 3 pages) should be a clear and concise stand-alone document that gives readers the essential contents of the evaluation report in 2 or 3 pages, previewing the main points in order to enable readers to build a mental framework for organizing and understanding the detailed information within the report. In addition, the Executive Summary helps readers determine the key results and recommendations of the report. Thus, the Executive Summary should include: major lessons learned; maximum of two paragraphs describing the program, summary of targets and intended outcomes; areas of meaningful under or over achievement.
• Methodology: sampling method including strengths and weaknesses of method used inclusion of stakeholders and staff, rough schedule of activities, description of any statistical analysis undertaken, including justification and software package used. The discussion of any random sampling used should include details on how the random respondents were identified and invited to participate. This section should also address constraints and limitations of the evaluation process and rigor. The methodology section should also include a detailed description of data collection techniques used throughout the evaluation.
• Discussion of findings: The main body of the report shall elaborate the points listed in the Executive Summary. It will include references to the methodology used for the evaluation and the context of the action. In particular, for each key conclusion there should be a corresponding recommendation.
• Recommendations: should be as realistic, operational and pragmatic as possible; that is, they should take careful account of the circumstances currently prevailing in the context of the action
• Limitations of the study: this should also include areas for further research.

In addition to the report, Relief International requests the following outputs:

• Copies of the raw data
• A presentation to RI field team, and Khartoum staff to discuss on preliminary findings.

RI’S RESPONSIBILITIES:

• RI will facilitate logistical arrangement including international and in country travel (flight ticket). The consultant will be response to cover all accommodation costs.
• At the field level, RI will be responsible to make the logistical arrangements for data collection.This entails recruitment of enumerators, supervisors and mobilize the target groups for interviews.
• RI will provide input to the inception report, tools and KAP draft report.
• While RI undertakes the responsibility of facilitating travel and organizing meetings for the evaluator, it is the evaluator’s ultimate responsibility to follow through and ensure that all relevant parties are interviewed and relevant project sites visited.
• Travel to and within the project locations involves some risk. Other than providing standard security arrangements and information, RI takes no responsibility for the safety of the evaluator and / or his property during this exercise.
• Early in the course of the evaluation, the evaluator must submit a list of people / groups that s/he will wish to talk with, so that the program staff can arrange meetings. Examples include the local authorities, local partners involved in the program, and select beneficiaries (the evaluator must indicate the general demographics of the beneficiaries s/he would like to meet – for example, farmers, women, performance level of the beneficiaries, etc).
• Upon completion of the evaluation, a soft copy must be sent to the team for review and feedback

CONSULTANT’S RESPONSIBILITIES:

• Develop and submit inception report which will include an interpretation of the tasks and design methodologies, which will be used in the KAP study.
• Develop appropriate tools and instruments for data gathering
• Conduct a desk review (Survey reports, Baseline report and other secondary sources) relevant to the survey.
• Train enumerators
• Undertake KII and FGD with key stakeholders in the field.
• Process data entry, cleaning, and analyse the data
• Revise the report and based on the RI internal technical feedback, submit a final report
• Develop context based actionable recommendations which include specific recommendations based on evidence at state level.

REQUIREMENTS

• Fluency in English and
• Knowledge of Arabic languages is desirable
• Technical knowledge of health, nutrition, and WASH programs
• Previous evaluation experience
• Previous experience of BAH program evaluation will be desirable
• Cultural sensitivity

SURVEY TIMELINE

Date

Activity

28th July 2022

Consultants travel to the field

29h July 2022

Meeting with RI ND field staff and prepare field work

31st July 2022

Training of enumerators in North Darfur

1st Aug – 5th August 2022

Data collection in North Darfur

6th August 2022

Meeting with local partners and line ministries

8th August 2022

Meeting with RI Blue Nile technical team in the field

9th August 2022

Training of enumerators in Blue Nile

10th– 15th August 2022

Data collection in Blue Nile

25th August 2022

Share first draft report

30th August 2022

Presentation of key findings to staff

5th September 2022

Final report submission

Interested applicants should present an application, as follows:

• A technical proposal highlighting their understanding of the ToR and how they propose to respond to the ToR -requirements including the evaluation design, tools, and technologies in place for them to perform the assignment, and implementation plan.
• A financial proposal with a detailed budget breakdown considering the details in the ToR and timelines therein. Preparation of questionnaire and report submissions can be done remotely
• Sample work: at least two (2) samples of similar work (evaluation reports) performed for peer organization in Sudan or in the Horn of Africa region in the last 3 years.