TERMS OF REFERENCE FOR CONDUCTING TERMINAL EVALUATION FOR THE PROJECT: MEETING THE RIGHTS OF PEOPLE WITH DISABILITIES (MRPWDs) PROJECT

Description of Programme Location and Context

Meeting the Rights of People with Disabilities (MRPWDs) project is a project implemented in 7 program locations in Ethiopia: Addis Ababa, Bahirdar, Hawassa, Jimma, Harar, Gode, and Mekelle since January 2021. The project is dedicated to contribute towards the realization of rights of people with disabilities. The project has been embarked on 5 outcomes. These were improved ability of people with disabilities to claim their rights, improved ability of people with disabilities to diversify and improve their livelihoods, developmental and protection needs of people with disabilities are fulfilled, and improved access to quality inclusive education for boys and girls with disabilities.

The terminal evaluation will focus on assessing the overall outcomes, impact, achievements, and lessons learned from the project. The evaluation will examine whether the project achieved its intended objectives, its overall effectiveness, sustainability, relevance, and its potential for replication or scaling up. It is also expected that the findings and recommendations of the Final Project Evaluation will help to design better quality projects in the future. Hence, this Terms of Reference (ToR) is developed to serve as a request for proposal (Technical and financial) from consultancy firms who are interested in conducting a terminal evaluation on targets of MRPWDs project such as children with disabilities (CwDs) and their caregivers, students with disabilities (SwDs) and Organizations of people with disabilities (OPDs) benefited from the SOS Children’s Village. While the project was conducted in 7 program locations, the terminal evaluation will be conducted only on 4 selected locations: Addis Ababa, Bahirdar, Mekelle, and Harar.

Rationale of the Evaluation

The terminal evaluation of MRPWDs project is being conducted to assess the overall performance, achievements, and sustainability of the project implemented over the past five years across Addis Abeba, Bahirdar, Mekelle and Harar cities. As the project approaches completion, this evaluation will serve as a key accountability, learning, and decision-making tool for all key stakeholders. Specifically, the rationale for this final evaluation includes:

• Accountability: The evaluation will assess the extent to which the project has achieved its stated objectives and delivered on its planned outputs and outcomes, as defined in the provided Result Framework. This is essential for accountability to donors, beneficiaries, and other stakeholders, ensuring that resources have been used effectively and as intended.
• Learning and Program Improvement: The evaluation will serve as a critical learning exercise. It will provide an opportunity to identify what worked well, what did not, and why. The findings will be used to improve the design and implementation of similar future projects.
• Sustainability and Impact: The evaluation will assess the likelihood of the project’s results and benefits being sustained after the project ends. It will also seek to identify any unintended positive or negative impacts, contributing to a deeper understanding of the project’s true effect on the target population and the broader community. The findings will provide strategic recommendations for scaling up or replicating the successful components of the project in the future.

Objectives of the final Evaluation

General Objective:

The overall objective of this terminal evaluation is to analyze the results of SOS Children’s Villages MRPWDs Project. The terminal evaluation will also assess the achievement of the project results against targets, planned outputs and activities, and draw lessons learned that can both improve the project’s sustainability and provide input to the enhancement of projects and programs related to Person with Disabilities. The evaluation should also analyze best practices, specific lessons learned and come up with recommendations on the strategies to be used and how to implement them.

Specific Objectives:

The specific objectives of the terminal evaluation are to:

• Assess the relevance, efficiency, effectiveness, impact, and sustainability of SOS Children’s Village MRPWDs Project interventions on the four outcomes in the four target areas using DAC criteria.
• Review of the project’s logical framework and theory of change to determine if the design was sound and realistic.
• To determine the extent to which the project interventions addressed the gaps /challenges identified thus meeting the target beneficiaries’ and stakeholder’s expectations.
• Study the organizational, functional, and financial capacity of SOS-supported DPOs
• Identify key lessons learned, good practices, challenges including DPOs, Schools, and SACCO practices, challenges and contribute to organizational learning and recommend improvements for future project interventions.

Project Impact, Outcomes and Output

Goal: Contribute towards the realization of rights of people with disabilities.

• Outcome 1: Improved the ability of people with disabilities to claim their rights and participate in decision making at family and local level

  Output 1.1. Improved awareness of people with disabilities and stakeholders on the rights of people with disabilities

• Outcome 2: Ability of people with disabilities to diversify and improve their livelihood

  Output 1121: Improved entrepreneurship and business literacy of people with Disabilities

• Outcome 3: Developmental and protection needs of people with disabilities are fulfilled

  Output 3.1. Improved food and shelter of target children

  Output 3.2. Health status of people with disabilities improved

  Output 3.3. Improved capacities of caregivers to provide care and support for their children

• Outcome 4: Improved access to quality inclusive education for girls and boys with disabilities Improved awareness of families and communities on inclusive education

  Output 4.1. Boys and Girls with disabilities are supported with material and other services to access education

  Output 4.2. Improved quality of inclusive education

• Outcome 5: Increased capacity of DPOs and networks to support person’s with disability and influence local decision making

  Output 5.1. Supporting the development of organizational capacity of DPOs

3.4 Major evaluation /research questions:

The commissioned consultant should list relevant and possible evaluation questions by referring to the project document and the project result framework and show the terminal evaluation questions during the inception report presentation.

Scope of the Final Evaluation:

The project has been implemented in SOS Children’s Village Addis Ababa, Bahir Dar, Harar, and Mekelle program locations. Geographically, the terminal evaluation will focus on the intervention areas of the MRPWDs project, which are in:

• Addis Ababa – Nifas Silk Sub-City, Woreda 02
• Bahirdar – Belay Zeleke, Fascilo, Tana, Atse Tewodros, Dagmawi Menelik, and Gish Abay sub-cities
• Harar – Hakim Woreda (Hamaressa Kebele) and Sofi Woreda (Gelmeshira Kebele)
• Mekelle – All sub-cities (Semien, Ayder, Adihaki, Hawelti, Hadinet, Kuha, and Kedamay Weyane), as well as nearby communities in Enderta. The terminal evaluation will cover the projects intervention periods (January 2021-December 2025). The project has 1000 children with disabilities and 500 caregivers of children with disabilities as direct participants and 8000 children and 4000 caregivers as indirect participants.

The rights of people with disability project has the following direct project participants

Methodology

The consultancy firm to be recruited is expected to develop appropriate and standard quantitative and qualitative methodologies that can generate the highest quality and most credible evidence. The commissioned consultant should use a mixed method approaches (quantitative and qualitative methods) to answer the terminal evaluation questions. As far as possible, the consultant should disaggregate data by sex, age, disability while collecting and analyzing data. The consultant should also clearly explain which questions will be answered using which methods. Data collection methods proposed by the consultant should be linked to the specific target group question(s). In addition to the data collection methodology, the consultant should refer to the relevant desk review of the project agreement documents. Moreover, the consultant is expected to explain the design and process of data collection tools, data collection plans and data analysis instruments. The consultant is also expected to test data collection tools before the actual utilization.

The Consultant should also ensure that the survey and qualitative methods (such as focus group discussion (FGD), key informant interview (KII) and group discussion) with participants are representative of the project’s target groups and key stakeholders (Community members, Children, youth, caregivers (men & women), KIP CBOs, SACCOs, Schools.)The evaluation team is tasked with developing a suitable methodology, including all required data collection instruments, sampling methods, and data analysis techniques, as per the requirements to answer all evaluation questions. A disability gender-sensitive approach to the evaluation process is expected.

Sampling

The consultant is required to propose statistically sound sampling strategies to ensure representativeness. He/she is required to clearly state the sample size and the acceptable margin of error. The consultant will develop a detailed data analysis plan, outlining the statistical methods to be used for the quantitative data and the thematic analysis approach for the qualitative data. All data collection activities should be conducted following the highest ethical standards, ensuring informed consent, confidentiality, and the safety of all participants.

Work plan and expected deliverables:

Workplan/timetable

The terminal evaluation task is expected to be finalized within 45 days after the contractual agreement is signed. The consultant is expected to develop her/his detailed work plan based on the following table.

Expected deliverables

The commissioned external consultant must deliver the terminal evaluation findings within 45 days of the contract signed. Based on the work plan, SOS Children’s Villages in Ethiopia expect the following deliverables:

• An inception report of 8 pages on the evaluation design, methodology, sampling frame, sampling technique, sample size, assessment tools and work plan
• Draft evaluation report.
• The consultant is expected to present findings (for both draft and final findings) by intervention areas or programme locations for TOFI project.
• Final report – The findings of evaluation report including an executive summary and data collection tools in both electronic and hardcopy formats should be submitted.
• Raw data, which has been cleaned (both qualitative and quantitative, including original field notes for in-depth interviews and focus group discussions, and recorded audio material), should be submitted with the report. SOS CVE will have sole ownership of all final data, and any findings shall only be shared or reproduced with the permission of SOS Children’s Village Ethiopia.

Report Criteria

The reporting criteria for disability mainstreaming project terminal evaluation shall be in line with the SOS Children’s Villages Ethiopia result based management (RBM) toolkit and should be shared with the winner consultant along with the data review process and/or for the preparation of the inception report.

Child safeguarding and ethical issues

SOS Children’s Villages is committed to ensuring that all research, evaluation and data collection processes (i.e. evidence-generating activities) undertaken by SOS Children’s Villages and its partners are ethical and respect child safeguarding policy and procedure.

The consultant must respect the rights, dignity and protection of children and other vulnerable population groups and should ensure special protection for children and other vulnerable groups during any data-generating activities to minimize any potential risks. Any research, evaluation and data collection SOS Children’s Villages is directly carried out or is involved in as a partner.

Ethical practices need to be ensured in the following circumstances:

• Any research, baseline, midterm or final evaluations and data collection SOS Children’s Villages has commissioned for ethical oversight of these processes.
• Any research, evaluation and data collection carried out by researchers/consultants on SOS Children’s Villages programmes and participants.

Hence, relevant staff in SOS Children’s Village of the four program locations will ensure that any researchers, evaluators and data collectors should receive awareness training on, sign and adhere to SOS Children’s Villages core policies:

Obtaining consent from research participants is central to the research relationship and signals respect for the research participant’s dignity, their capability to express their views and their right to have these heard in matters that affect them. Informed consent is an explicit agreement which requires participants to be informed about and understand the research/assessment. This must be given voluntarily and be renegotiable, so that participants may withdraw at any stage of the research process.

Logistical arrangements

The awarded consultant shall show feasible logistical arrangements for the assignment as part of the technical proposal. National or location-level staff (SOS CVE) will be available to help organize the interviews including contacting SOS CVE, announcing and local preparation of evaluation, and linking to community duty bearers and national authorities if required.

Duration of the contract and terms of payment

Payment will be made only upon SOS Children’s Villages’ acceptance of the work performed in accordance with the above-described deliverables. Payment will be effected by bank transfer in the currency of birr.

Funding and Payment: The consultancy firm will be paid by SOS Children’s Villages as follows:

• 30% on the submission and approval of the inception report/Technical Proposal.
• 30% on completion of the draft report.
• 40% on completion of the final report.

Duration of contract: the contract is effective from the moment it was signed until the acceptance of work by the SOS Children’s Villages in Ethiopia management team.

Notice of Delay

Shall the successful bidder encounter a delay in the performance of the contract which may be excusable under unavoidable circumstances; the contractor shall notify SOS Children’s Villages in writing about the causes of any such delays within one (1) week from the beginning of the delay.

After receipt of the Contractor’s notice of delay, SOS Children’s Villages in Ethiopia shall analyze the facts and extent of the delay and extend the time for performance when in its judgment the facts justify such an extension.

Copyright and other proprietary rights

SOS Children’s Villages shall be entitled to all intellectual property and other proprietary rights including, but not limited to, copyrights, and trademarks, with regard to products, processes, inventions, ideas, know-how, or documents and other materials which the Contractor has developed for SOS Children’s Villages under the Contract and which bear a direct relation to or are produced or prepared or collected in consequence of, or during the course of, the performance of the Contract. The Contractor acknowledges and agrees that such products, documents, and other materials constitute works made for hire for SOS Children’s Villages.

All materials: interviews, reports, recommendations, and all other data compiled by or received by the Contractor under the Contract shall be the property of SOS Children’s Villages and shall be treated as confidential and shall be delivered only to SOS Children’s Villages authorized officials on completion of work under the Contract. The external consultant is obliged to hand over all raw data collected during the assessment to SOS Children’s Villages in Ethiopia.

Termination

The termination of the service agreement for the assignment will be in accordance with the contractual agreement to be included at the formal agreement’s actual signing.

Annex: The winning consultancy firm will be shared with the following documents.

• Project result framework.
• Project documents
• Relevant reports
• SOS Children’s Villages’ terminal evaluation report rewriting template.

• The applicant consultancy firm shall have at least a master’s degree in a relevant field of study such Psychology, Sociology, Social Work, development studies, Economics, Project Management, Monitoring & Evaluation and other related fields per the required assignment

Professional Experience:

• At least 5 years’ work experience & proven competency in assessments, monitoring, research, terminal evaluations, baseline assessments or organizational/program evaluation (baseline, midterm, and final evaluations). The applicant consultancy firm shall attach at least one sample final evaluation report produced on a similar topic.

Technical Skills:

• Strong analytical and report-writing skills, with the ability to present complex data and findings in a clear, concise, and accessible manner.
• Proficiency in data analysis software (e.g., SPSS, Stata, R, or Excel) and familiarity with mobile data collection tools (e.g., Kobo Toolbox, ODK Collect).
• Familiarity with the project management and evaluation standards of major donors and international organizations.

Contextual Knowledge:

• A solid understanding of the socioeconomic and environmental context of areas
• Knowledge of and experience with SOS Children’s Villages’ mission and values, particularly regarding child safeguarding and protection, is essential.

Soft Skills:

• Excellent communication and interpersonal skills to effectively engage with a diverse range of stakeholders, including project beneficiaries (youth and caregivers), community leaders, project staff, and government officials.
• Ability to work independently, manage time effectively, and deliver high-quality outputs within a strict deadline.
• A good understanding of disability inclusion, family strengthening, community empowerment, livelihoods and development progr work.
• A good understanding of child/youth rights/safeguarding & other issues affecting vulnerable children in the Ethiopian context.
• Proven experience in working with conceptual frameworks and data collection methods (including age-appropriate data collection methods)
• Proven experience in participatory processes and data collection methods

Language:

• Fluency in written and spoken English is mandatory.
• Proven fluency in local languages relevant to the project areas is a significant advantage and may be required for fieldwork and stakeholder consultations.

Legality:

• Legally registered firms with renewed license, VAT registration and TIN number