|Location||South and Central Darfur, Sudan|
|Date Posted||March 8, 2021|
Norwegian Church Aid (NCA) was part of the international relief programme launched by international community in 2004. The programme until December 2020 was supported by Action by Churches Together International (ACT) and Caritas Internationalis (Caritas). ACT International is the network of Protestant and Orthodox Aid and development agencies worldwide. Caritas Internationalis is the international confederation of Roman Catholic relief and development agencies. ACT and Caritas networks work together in a joint response to the Darfur crisis through NCA which implements the programme on behalf of the two networks. NCA is a member of ACT and provides the legal basis for the operation in Sudan. NCA’s Darfur programme has been a multi-sector operation managed by NCA working jointly with national implementing and contractual partners.
2020 was the final year of the combined appeal, and as of 2021, NCA manages the programmes in Darfur as independently.
The targeted overall outcome of the project is that Conflict affected communities of Darfur have enhanced resilience to the recurrent crises in South and Central Darfur States. The project primarily targeted internally displaced persons, South Sudanese and Central African refuges and host communities under Four Results:
1) Improved access to WASH related supply and services among South Sudanese refugees, IDPs, Returnees and host communities
2) Improved access and utilisation of health services and reduced morbidity/mortality associated with illness among South Sudanese refugees, IDPs and host communities.
3) Reduce malnutrition cases among children <5, and Pregnant and Lactating women (PLWs) within IDPs, South Sudanese Refugees and host communities in Bilel IDP areas.
4) Support IDPs and returnees affected by recent conflict in North Jebal Marra locality, Central Darfur state with ES/NFIs
The project aimed to reach a total of 240,121 persons (108,054 male / 132,067 female) in the targeted settlements.
The programmes have received funding from DG ECHO since 2009. The previous evaluation of DG ECHO funded support was conducted at the end of 2019. The current grant support period under evaluation is from 01-08-2019 to 31-03-2021.
In 2020 the WASH, and Health and Nutrition sectors through DG ECHO funding aimed at implementing projects in Central Darfur (Hassa Hissa, Hamedia, Khamsadageig, Funga, Foko Deko, Rokero) and in South Darfur (Bilel, Al Radom and Um Dafug settlements). Target beneficiaries included South Sudanese and Central African Republic refugees, internally displaced persons, and host communities.
The purpose of this evaluation is to assess the DG ECHO funded project results versus DAC criteria for evaluation as well as towards crosscutting issues of gender, conflict sensitivity and environment. In addition, the evaluation will assess to what extent the core dimension of participation of beneficiaries has been included in project design, planning and implementation and to what extent the NCA systems and procedures for complaints handling have been implemented by the project. Finally, the evaluation will seek to identify and assess key traits of NCA added value in implementation of the DG ECHO funded project in Darfur.
The evaluation will feed into the NCA Sudan Country Strategy 2020-2024. It is expected that lessons learned and recommendations from the evaluation will empower NCA to improve its response in Darfur and to enable increased alignment towards described needs and challenges facing the intended beneficiaries of this response. It will in addition document best practices and methodologies of the respective sector interventions there by enabling adjustment of responses to the changing dynamics of the challenges and concerns of the Darfur region. The evaluation will form the basis for external and internal stakeholder discussions and reflections determining the future architecture and mechanisms of NCA interventions in Darfur.
III. DESIGN AND APPROACH
The evaluation will use pre-post performance evaluation design, will employ industry standard mixed-methods of both quantitative and qualitative data collection and include as a minimum the following approaches:
- A clear evaluation approach and rigorous design to examine programme performance and delivery process, including justification of why the chosen approach is appropriate.
- Examination of the programme theory of change as an overarching framework for the evaluation, including anticipated outputs and outcomes, and linkages and assumptions between them.
- A clear evaluation framework, setting out the data collection methods and data sources that will be used to answer each of the evaluation questions.
- Triangulation of different data sources, perspectives (including beneficiaries and other stakeholders) and time points when addressing each of the questions.
- Consideration of how beneficiary populations and communities will be involved, including how their perspectives will be captured, and how feedback on outcomes / findings will be provided to them.
Sources of Data and Data Collection Methods
The following broader methods are suggested which could be revised and finalised with the selected external consultant in line with the design and approach recommended above:
(i) Quantitative beneficiary household survey to estimate outcome and output performance indicators and to compare with baseline benchmarks. Inclusive of Knowledge, Attitude and Practices survey and SMART Nutrition survey modalities.
(ii) Document review: Programme proposal; baseline and interim survey reports; donor reports; reports of assessments or standard operating guidelines; routine monitoring system tools; and updated Log frame which precent achievements against milestones targets of sector performance indicators of impact, outcome and outputs.
(iii) Interviews, discussions and post-evaluation workshop: with stakeholders including DG ECHO; national and state government (WES, Ministry of Health, and Humanitarian Aid Commission)
(iv) Field Observations: WASH installation, project formed/facilitated WASH committees, Bilel primary health care and nutrition facility.
IV. EVALUATION QUESTIONS
1. Project results
(i) Assess the Relevance of the project and the extent to which the project has included vulnerable groups and responded to the expressed needs of the intended beneficiaries. How did the project adapt to feedback from beneficiaries and changing needs?
(ii) Assess the Effectiveness of the project in terms of delivering benefits to intended beneficiaries and achieving outcomes within agreed timeframes and budget. Which key factors have been important in supporting and/or limiting achievement of results? Which unintended results, both positive and negative were produced; how did these occur?
(iii) Assess the Efficiency of the project in terms of to which extent project delivery options and models have ensured efficient use of funds and added value, including procurement, logistics, management structures; integration of and synergy between project activities; delivery at scale.
(iv) Assess the Impact of the project towards enhancing resilience and empowering beneficiaries towards achieving lasting change in livelihoods.
(v) Assess the Sustainability of results achieved by the project in terms of participation and enhanced capacities of beneficiaries as well as local government and local leadership structures. To what extent has the project aligned and responded to policies and strategies of national and state government.
2. Cross-cutting issues
(i) To what extent has gender been considered in project design and implementation? How has the programme demonstrated good practice in ensuring issues such as protection and UNSCR 1325 on women, peace and security have been considered in project design and implementation
(ii) To what has extent has the project included and addressed conflict sensitivity and “Do No Harm” in its planning, implementation and monitoring
(iii) To what extent has the project included and addressed environmental challenges and concerns.
3. Participation and complaints handling
(iv) Assess to which extent NCA guidelines for participation of beneficiaries as well as systems and procedures for complaint handling have been incorporated and implemented in the project
4. NCA Added Value
(v) Identify and assess key factors of NCA added value in implementation of the DG ECHO project.
V. TIMING AND RESPONSIBILITIES
1. NCA Responsibilities
NCA shall establish an Evaluation Steering Group that will be responsible for the evaluation. The Steering group composition will be comprised of
- Representatives at management level from both NCA Sudan and NCA Darfur representations
- Representative from NCA Oslo head office (Country Advisor)
- Led by a person appointed by the Country Office Management team.
Both men and women must be represented.
If possible, a partner representative should participate.
The steering group assigns one focal point that is the one to manage the relations and contact with the evaluation team.
Specifically, NCA will be responsible for the following:
- Provide all relevant documentation for the evaluation team.
- Provide security updates and orientation.
- Provide all logistical arrangements including internal travel and accommodation in Sudan, organise meetings and field visits as required.
- Inputs to the design and methodology of the inception report, including providing informational material and feedback.
- Comments and inputs to the draft report.
- Management response to the report.
- Dissemination of the report.
2. Time frame and site visits
The time frame and site visits for the evaluation will be determined by the Evaluation Steering Group. Upon the commencement of the evaluation the timeframe for the respective content/deliverables is stipulated as 3. Evaluation team
The composition and necessary qualifications of the evaluation team will be determined by the Evaluation Steering Group. The evaluation will be led by an external consultant with experience in the thematic areas of WASH, and Health and Nutrition, in particular in emergency settings. The consultant will lead the overall management of the evaluation design, implementation including capacity building for survey teams, data management and analysis, and report writing.
Stakeholders involvement: In coordination and collaboration with NCA, the evaluation team shall work with the different stakeholders (community-based committees, local administrators, etc) in their respective geographic areas of implementation. In addition, government line ministries particularly WES and the Ministry of Health will have significant contribution to the project implementation and the consultant would be expected to have discussions and consider them as part of the project stakeholders.
The consultant will be responsible for the key deliverables as itemised below:
VI. DELIVERABLES AND BUDGET
The following deliverables are expected from the evaluation team:
(i) Budget within existing frames
(ii) Inception Report, Draft Report and Final Report
(iii) Feedback from stakeholders to draft report
(iv) Evaluation report within the requirements: 1-3-25: One page: Recommendations. Three pages: Executive Summary. 25 pages: Presentation of the findings
(v) Presentation of evaluation report including recommendations
The Evaluation report and all documents pertaining to the report are considered as NCA property and under NCA ownership.