SOAR II Program Final Evaluation Program

Request for Proposal (RFP)

                                            For SOAR -II Program Final Evaluation

.    Number of people receiving direct hygiene promotion (excluding mass media campaigns and without double-counting)

·        Percent of people targeted by the hygiene promotion program who know at least three (3) of the five (5) critical times to wash hands

·        Percent of households targeted by the hygiene promotion program who store their drinking water safely in clean containers

·        Percent of children under 5 years with diarrhea in the past two weeks

WASH NFI

·        Total number of people receiving WASH NFIs assistance through all modalities (without double-counting)

·        Percent of households reporting satisfaction with the contents of the WASH NFIs received through direct distribution (i.e., kits) or vouchers

·        Percent of households reporting satisfaction with the quality of WASH NFIs received through direct distribution (i.e., kits), vouchers, or cash

·        Percent of people targeted by the WASH NFI program who report feeling safe during the NFI distribution

 Livestock 

·        Number of people benefiting from livestock activities

·        Number of animals benefiting from livestock activities

·        Number of individuals accessing products or services from livestock market actor

·        Number of animals owned per individual

·        Total USD value of vouchers redeemed by beneficiaries

Improving agricultural production/food security

·        Number of months of household food self-sufficiency as a result of improved agricultural production programming

·        Number of people directly benefiting from improving agricultural production and/or food security activities

·        Number of hectares under improved agricultural methods

·        Percentage of households with access to sufficient seed to plant

·        Number of individuals accessing products or services from agriculture market actors

Pest and pesticides

·        Number of people trained in appropriate crop protection practices

·        Number and percentage of hectares protected against disease or pest attacks

·        Number and percentage of people practicing appropriate crop protection procedures

Management of acute malnutrition 

·        Number of health care staff trained in the prevention and management of acute malnutrition

·        Number of supported sites managing acute malnutrition

·        Number of people admitted, rates of recovery, default, death, relapse, and the average length of stay for people admitted to Management of Acute Malnutrition sites

·        Number of people screened for malnutrition by community outreach workers

·        Number of cooking demonstration conducted at each food distribution center or OTP clinic

Infant and young child feeding in emergencies 

·        The proportion of infants 0–5 months of age who are fed exclusively with breast milk

·        The proportion of children 6–23 months of age who receive foods from 4 or more food groups

·        Number of people receiving behavior change interventions to improve infant and young child feeding practices

·        The proportion of caregivers who can mention at least 3 appropriate IYCF practices

Multipurpose Cash

·        Total number of people assisted through multipurpose cash activities

·        Percentage of households who report being able to meet the basic needs of their households (all/most/some/none), according to their priorities

·        Percentage of beneficiaries reporting that humanitarian assistance is delivered in a safe, accessible, accountable, and participatory manner

·        Percentage of households by the Food Consumption Score (FCS) phase (Poor, Borderline, and Acceptable)

·        Percentage of households using an unsafe water source because they cannot afford to use a safer water source

·        Percentage of households reporting adequate access to household non-food items

·        Total USD amount of cash transferred to beneficiaries

·        A total number of people who opened and accessed electronic accounts in the Banks/MFIs due to the MPCA.

3. Background & Relevant Documents                                    

The goal of the program is to support conflict and drought affected vulnerable individuals in four target woredas in the Somali and Oromia regions are able to respond to their immediate WASH, nutrition requirements and basic needs, while also restoring their capacity to meet their food security needs and navigate future shocks and stresses. The program targeted 84,843 individuals of which 39,182 individuals (46% ) are Internally Displaced People (IDPs), 4,353 individuals (5%) are returnees and 41,308 (49%)are host communities.  From the total targeted IDPs and returnees (43,535 individuals); 15,198 individuals are men, 26,488 individuals are women, 925 individuals are boys, 924 girls).

The SOAR II program is based on four outcome areas:

1.      Water, Sanitation & Hygiene – Improved health and wellbeing of target communities through adequate water, sanitation and hygiene infrastructure and practices.

2.      Agriculture & Food Security – Households are engaged in lucrative, market driven and climate-resilient agricultural livelihood opportunities.

3. Nutrition – Treatment of acute malnutrition contributes to the health of children under five (CU5).Multi-purpose Cash Assistance – Vulnerable households are able to access and consume basic goods, and are better able to withstand shocks and stresses

Existing Program Information Sources:

·        Project proposal narrative and workplan

·        Project MEL plan including table and narratives

Project baseline data and report

·        Physical progress of the program

·        Program post distribution monitoring reports

·        Program Semi-annual report

·        Organizational chart, MOUs and partnership agreements

·        External midterm evaluation reports from government

4)          Evaluation Design

A mix of qualitative and quantitative approaches will be preferred for undertaking this final evaluation assessment to ensure the triangulation and validation of information in the areas of interest. The team combined Focus Group Discussions (FGDs) with Key Informant Interviews (KIIs) and Individual Household interviews to gather data from the sampled respondents in each of the targeted woredas. This will be implemented as follows:

I.           FGDs – The targeted localities will have both male and female FGD for the sampled villages where one will be randomly selected and FGD conducted.

II.           KIIs – In the targeted localities attempt to interview a sample of community leaders, existing WASH committee representatives, other NGO representatives, or government officials). The FGDs and KIIs will feed into evaluation questions relating to relevance, efficiency, effectiveness and sustainability of the program activities.

III.           Household Interviews – The baseline survey adopted households as a survey unit. When possible, the head of household will be interviewed, although this criterion will not strictly enforce to account for gender balanced information.

A.     Evaluation Questions

The key questions of the evaluation include

RELEVANCE: The extent to which the intervention objectives and design respond to beneficiaries’, global, country, donor and partner/institution needs, policies, and priorities, and continue to do so if circumstances change.

·        Were the project outcomes aligned with the relevant sectors/goals and targets in Ethiopia development plan USAID resilience pillar.

EFFECTIVENESS: The extent to which the intervention achieved, or is expected to achieve, its objectives, and its outcome.

·        What factors have contributed to the success and failures of meeting project objectives?

·        Whether the SOAR II program improved and expanded on the livelihood assets and opportunities for the target households or not.

·        How can our interventions strengthen local coping systems at household and community levels?

·        What can we learn from the household and communities that cope better than others?

·        Whether the SOAR II program has contributed to improve household food security status of the households in targeted woredas measured with food consumption score and livelihood coping strategy of the targeted community or not?

EFFICIENCY: The extent to which the intervention delivers, or is likely to deliver, outcomes in an economic and timely way. Focusing on the four sectors; WASH, agriculture and food security, nutrition and multi-purpose cash assistance.

·        Was the project budget used appropriately according to original plans and narratives? Was the financial control system in place?

·        What systems were placed to ensure project inputs provided were of the highest possible quality and were acceptable to beneficiaries?

·        Were other methodologies of delivery considered and would these have been more cost efficient?

·        How do our interventions affect the power dynamics between market actors? How does this affect our target group?

·        Was the intervention undertaken on the required scale in order to have realistic outcomes?

SUSTAINABILITY:

·        The extent to which the benefits of the intervention continue or are likely to continue. Throw light on how Mercy Corps SOAR II Program interventions strengthen local coping systems;

·        what can Mercy Corps learn from communities that cope better than others;  how did market-based interventions help Mercy Corps SOAR program target group prepare for, cope with, and recover from shocks?

COHERENCE:

·        How well does the intervention fit? To what extent did the activity consider gender equity, protection, of the participants,

·        how has management adapted the activity design or implementation based on monitoring information and feedback from the target population

ii.  Evaluation Methodology

 To compare with baseline values, the SOAR II program final evaluation will follow a similar sampling strategy used during the baseline survey that is sector-based sampling strategy should be employed given the nature of the program. A mix of qualitative and quantitative approaches will be used in this study, to ensure the triangulation and validation of information in the areas of interest. The data will be gathered through individual household interviews, focus group discussions and key informant interviews.   The survey populations are: i) internally displaced people  ii) returnees and; iii) host communities. The survey location is the SOAR program intervention in Somali and Oromia regions, in 4 woredas including Tulligulid, Ararso woredas of Somali region and, Liban and Gumi Eldalo Woredas of Oromia region).

Given the large number of beneficiaries, the consultant is expected to apply standard sampling approaches and data collection methodology to ensure that we collect data from a statistically significant proportion of the target population that will be adequately representative of the total households. Using Mercy Corps standard sample size calculation tool. The total targeted population of SOAR II project is 84,843 and the sample size is approximately 406 of which 187 from IDPs, 197 host community and 21returnees. The collected data would be disaggregated by sex, age and type of community (host, IDP and returnee).

The below table indicates the total number of SAOR II program participants by woreda

For the qualitative data, the consultant is expected to conduct key informant interviews with  elders/opinion leaders, village leaders and camp leaders. Focus group discussions will be conducted separately for men and women in target communities.

Data Collection Tools and Process – Similar to the baseline survey, three different types of tools will be developed for data collection including an FGD guide, a KII question guide, and an individual household questionnaire that will be developed and piloted for easy administration in the field. The consultant will hire, train and supervise a team of enumerators. The data collection exercise would take approximately fourteen days for all target locations including preparation of tools, piloting and finalizing tools, recruitment and training of enumerators, field mobilization and field data collection process.

The recommended data analysis is SPSS or STATA.SPSS.

Team Composition, Roles & Responsibilities

Qualified applicants will have the following minimum requirements:

• Strong statistics background – acquired through academic studies, practical experience or both.
• Demonstrated experience with a mixed-methods data collection approach.
• Knowledge of Targeting Sampling strategies. Previous experience in studies in BHA funded humanitarian assistance and recovery projects.
• Demonstrated experience in adaptive management and in supporting teams that work in challenging operational environments, where security constraints impose limits to research and require research plans to be constantly adapted.
• Experience in USAID/BHA (previously OFDA) mandate, evaluation policy, procedures, and program management.
• Demonstrated, strong ethical approach to data protection.
• Strong understanding of resilience frameworks.
• Knowledge of the Somali and Oromia regions context.
• Good analytical and critical thinking skills.
• Ability to work both independently and as part of a team.
• The team should have members who are fluent in English Amharic, Afan Oromo and Somali languages written and oral.

i. Chart of Responsibilities

6)   Description of Deliverables

Consultant Deliverables: The Consultant will provide the following deliverables:

• Submit technical and financial proposal for this final evaluation including details and due dates for any draft reports planned for review
• Review the secondary data and analyze qualitative data from KII and Focus Group
• Refine the quantitative and qualitative data collection templates and send to the Mercy Corps, MEL manager, by email.
• An accessible dataset covering all the analyzed data will be developed and handed over to the Mercy Corps M&E team. The analyzed data will consolidate findings from all localities.
• An evaluation brief of 3 pages maximum, with findings, highlights and graphics that illustrate them.
• A comprehensive final evaluation report including (where relevant) tables, maps and graphs shared for review by Mercy Corps. The report will highlight considerations on ethnic affiliation, gender, age and disability aspects and will propose relevant recommendations to increase inclusion of vulnerable groups.
• Provide Mercy Corps staff with the opportunity to respond to the findings of the report. Up to three rounds of comments by Mercy Corps will be provided with the consultant expected to provide a revised report within five working days after each review. Both parties commit to wrap up this phase of back and forth within a maximum of 3 weeks.
• Amend the report following feedback from Mercy Corps and send it back to Mercy Corps for final validation.
• Conduct a presentation of the study findings and recommendations to the Mercy Corps team

Report Structure & Content: Define the structure and content requirements of the report, using or adapting the following outline:

●       Cover Page, List of Acronyms

●       Table of Contents

●       Executive Summary:

●       Methodology:

●       Limitations:

●       Results:

●       Synthesis, Recommendations and Lessons Learned:

●       Conflicts of Interest:

●       Annexes: