ToR: Conducting mid-term Evaluation of the Jimma Standard project

Description of program location

SOS Children’s Village Jimma, as an umbrella of SOS Children’s Villages International, is one of eight programs under SOS Children’s Villages in Ethiopia and has been operating various interventions primarily focusing on children, young people, families, and communities in the towns of Jimma, Agaro, Bedelle, Nekemte and Woliso towns in Oromia regional state. SOS Children’s Village Jimma currently operates alternative childcare programs (SOS Family-like Care, Supervised Independent living, Kinship Care and Foster Family Care), Family and community development programs (Grow Equal and EduCare projects), humanitarian emergencies, advocacy, and the SOS Hermann Gmeiner Schools and kindergarten. The programme location has been providing alternative childcare for the children’s who already lost their parental care and under, family strengthening program also supports children who already lost their parental care.

Rational for conducting midterm evaluation:

SOS Children’s Villages in Ethiopia has planned to conduct a midterm evaluation for its Jimma standard project. The evaluation is intended to evaluate the process and performance at the midpoint of its implementation period. It helps determine whether the project is on track to achieve its intended results and provides recommendations for improvement during the remaining implementation period.

The project already has baseline values and indicator targets incorporated in the results framework, along with established means of verification to assurance that data is reliable, accessible, and measurable. Conducting a midterm evaluation is therefore essential to assess the progress made toward achieving planned results and to verify whether implementation is on track. The mid-term evaluation also helps determine the quality, consistency, and reliability of data sources and verification mechanisms used during monitoring and reporting processes. Additionally, the midterm evaluation provides evidence-based findings that support informed decision-making, identify implementation gaps and challenges, and generate recommendations for improving project performance during the remaining implementation period.

Objectives of the Midterm Evaluation

General Objective

The overall objective of a midterm evaluation is to assess the progress, performance, relevance, efficiency, and effectiveness of a project at the midpoint of its implementation period. It helps determine whether the project is on track to achieve its intended results and provides recommendations for improvement during the remaining implementation period of Jimma Standard project.

Specific Objective

The specific objectives of the midterm evaluation are:

• To measure the extent to which planned outcomes and output level targets have been achieved so far.
• To determine the effectiveness and efficiency of the project in terms of resources utilization, activities accomplishment, and strategies are being utilized to achieve expected results.
• To identify implementation gaps, risks, constraints, good practices, and lessons learned during project execution.
• To generate practical recommendations for strengthening implementation, improving performance, and enhancing the likelihood of achieving project goals by the end of the project period.
• To provide evidence-based information to guide strategic decisions and future planning.

Project goal, outcomes and outputs:

The project has the following goal, outcomes and outputs.

Goal:  Children and Youn persons grow up with the bonds they need to become their strongest selves.

Project Outcome & Output

Outcome 1: Children are cared for and protected.

• Output 1.1. Children have appropriate living arrangements.
• Output 1.2: Children understand and exercise their rights.
• Output 1.3. Children have access to basic essential services.
• Output 1.4. Children have access to social & emotional support.
• Output 1.5. Caregivers’ parenting skills improved.
• Output 1.6. Families have sufficient resources.

Outcome 2. Young people are competent and employed

• Output 2.1. Young people have appropriate living arrangements.
• Output 2.2. Young people’s academic achievement improved
• Output 2.3. Young people’s MHPSS and physical development improved
• Output 2.4. Young people possess marketable job skills
• Output 2.5. Caregivers and staff have the skills and knowledge on youth care & development.
• Output 2.5. Youth are linked to partners

Outcome 3: System and structures respond to the needs and rights of vulnerable children and young people.

• Output 3.1: Program’s capacity on advocacy work improved
• Output 3.1: Communities have improved awareness on child right and protection
• Output 3.1: Community support structure and service providers strengthened.

Major Midterm Evaluation Questions

Child Care and Protection

• To what extent are children being adequately cared for and protected in all existing care options (Family Like care (FLC), Foster Family care (FFC), Other alternative care/Kinship care (OAC/KC), Supervised Independent living arrangement (SIL)
• What major risks and protection issues are children facing in Family Like Care (FLC) especially SOS Families in the community and physically reintegrated families?
• What community-based child protection mechanisms currently exist, and how effective are they?

Young people competency & employment

• To what extent are young people equipped with the competencies required for employment and sustainable livelihoods?
• To what extent have young people acquired relevant competencies through the project?
• How has the project contributed to improving the skills, competencies, and employment opportunities of young people?
• What factors facilitate or hinder young people’s access to employment and livelihood opportunities?

System and structures respond to the needs and rights of vulnerable children and young people.

• To what extent do systems and structures effectively respond to the needs and uphold the rights of vulnerable children and young people?
• To what extent have the capacities of government institutions, community-based organizations, and child protection structures been strengthened?
• How effective are referral pathways and coordination mechanisms in addressing the needs of vulnerable children and young people? Top of Form

Baseline for Monitoring and Evaluation

• What are the midterm evaluation status for the outcome and output indicators?

The commissioned consultant should further list relevant, tailored and possible mid-term evaluation questions for both quantitative and qualitative methodologies. He/she is expected to refer to the project document in detail, log frame and in addition to developing evaluation questions, and data collection tools. SOS CVE Ethiopia will share the project document, log frame/result framework for winner consultancy firm.

Scope of the Midterm Evaluation

Geographically, midterm evaluation is limited to the project participant children, youth, families, the communities and key implementation partners (SaCCO) in Jimma town in Oromia reginal state under SOS children’s Village Jimma programme location. The mid-term evaluation will take place in Jimma town. The Project has the following direct participants/beneficiaries.

Mid-term Evaluation Methodology

The consultant is expected to develop appropriate and standard quantitative and qualitative methodologies that can generate the highest quality and most credible evidence. The commissioned consultant should use mixed method approaches (quantitative and qualitative methods) to answer the evaluation questions. As far as possible, the consultant should disaggregate data by sex, age, disability disaggregated data (SADD) while collecting and analyzing data. The consultant should also clearly explain which questions will be answered using which methods. Data collection methods proposed by the consultant should be linked to the specific target group question(s). In addition to the data collection methodology, the consultant should refer to the relevant desk review of the project agreement documents. Moreover, the consultant is expected to explain the design and process of data collection tools, data collection plans and data analysis instruments. The consultant is also expected to use tools like SWOT (strengths, weaknesses, opportunities & threats) analysis while facilitating meetings with partners and stakeholders.

The Consultant should also ensure that the survey and qualitative methods (such as focus group discussion (FGD), key informant interview (KII) and group discussion) with participants are representative of the project’s target groups and key stakeholders (Community members, Children, youth, care givers (men & women), CBOs. schools and SaCCo)

Sampling

The Commissioned consultant is expected to use appropriate sampling techniques and set a representative sample size for both qualitative and quantitative data collection methods. Based on the project objectives and results framework. The consultant should produce appropriate sampling methods and ensure representativeness of the sampled households or respondents, and the sampling frame is from target children, youth, care givers /care professionals.

Workplan and expected deliverable

Workplan/timetable

The mid-term evaluation task is expected to be finalized within 40 days after the contractual agreement is signed.

The consultant is expected to develop her/his detailed work plan based on the following table.

Responsibility of the consultant:

1. • The awarded consultant shall show feasible logistical arrangements for the assignment as part of the technical proposal
   • Refer the project document, result framework and indicators and develop baseline study questions and detailed tools for both quantitative and qualitative data collection methods.
   • Recruit and train data collectors/ enumerators
   • Covers transport and per diem costs for data collectors
   • Develop inception report with data collection tools, data collection plans and present for SOS CVE team and data collection plans
   • Compiled draft and final baseline report based on scheduled time

Note that SOS CVE programme location-level staff (SOSCVE) will be available to help organize the interviews including contacting SOSCV, announcing and local preparation of evaluation, and linking to community duty bearers and national authorities. SOSCVE team will facilitate while the consultant collects data from prgramme participants, partners and key government stakeholders.

Deliverables

The commissioned external consultant must deliver the midterm evaluation findings within 40 days of the contract signing. Based on the work plan, SOS Children’s Villages of Jimma programme location, Programme MEAL and the national office MEAL and FS teams expect the following deliverables:

• An inception report of 8 pages on the midterm evaluation design, methodology, sampling frame, sampling technique, sample size, assessment tools and work plan
• Draft midterm evaluation report of maximum 20 pages.
• The consultant is expected to present findings (for both draft and final findings) by intervention areas under Jimma programme location, Jimma town.
• Final midterm evaluation report – The findings of midterm evaluation report including an executive summary and data collection tools in both electronic and hardcopy formats should be submitted.
• Raw data, which has been cleaned (both qualitative and quantitative, including original field notes for in-depth interviews and focus group discussions, and recorded audio material), should be submitted with the report. SOS CVE will have sole ownership of all final data, and any findings shall only be shared or reproduced with the permission of SOS Children’s Village Ethiopia.

Reporting Criteria

The reporting criteria for Jimma Standard project midterm evaluation shall be in line with the SOS Children’s Villages Ethiopia result based project management (RBPM) toolkit and should be shared with the winner consultant along with the data review process and/or for the preparation of the inception report.

Child Safeguarding and ethical issues.

SOS Children’s Villages is committed to ensuring that all research, evaluation and data collection processes (i.e. evidence-generating activities) undertaken by SOS Children’s Villages and its partners are ethical and respect child safeguarding policy and procedure.

The consultant must respect the rights, dignity and protection of children and other vulnerable population groups and should ensure special protection for children and other vulnerable groups during any data-generating activities to minimize any potential risks. Any research, evaluation and data collection SOS Children’s Villages is directly carrying out or is involved in as a partner.

Ethical practices need to be ensured in the following circumstances: 

• Any research, baseline, midterm or final evaluations and data collection SOS Children’s Villages has commissioned for ethical oversight of these processes.
• Any research, evaluation and data collection carried out by researchers/consultants on SOS Children’s Villages programmes and participants.

Hence, relevant managers in Jimma and Awassa will ensure that any researchers, evaluators and data collectors should receive awareness training on, sign and adhere to SOS Children’s Villages core policies:

Obtaining consent from research participants is central to the research relationship and signals respect for the research participant’s dignity, their capability to express their views and their right to have these heard in matters that affect them. Informed consent is an explicit agreement which requires participants to be informed about and understand the research/assessment. This must be given voluntarily and be renegotiable, so that participants may withdraw at any stage of the research process.

Logistic Arrangements

The awarded consultant should show feasible logistical arrangements for the assignment as part of the technical proposal. National or location-level staff (SOSCV) will be available to help organize the interviews including contacting SOSCV, announcing and local preparation of evaluation, and linking to community duty bearers and national authorities if required.

Duration of the the contract and terms of payment

Payment will be made only upon SOS Children’s Villages’ acceptance of the work performed in accordance with the above-described deliverables. Financial proposals should include proposed stage payments. Payment will be affected by bank transfer in the currency of birr.

Funding and Payment: The consultant will be paid by SOS Children’s Villages as follows:

30% on the submission and approval of the inception report.

30% on completion of the draft report.

40% on completion of the final report.

Duration of contract: the contract is effective from the moment it was signed until the acceptance of work by the SOS Children’s Villages in Ethiopia management team.

Notice of Delay

Shall the successful bidder encounter a delay in the performance of the contract which may be excusable under unavoidable circumstances; the contractor shall notify SOS Children’s Villages in writing about the causes of any such delays within one (1) week from the beginning of the delay.

After receipt of the Contractor’s notice of delay, SOS Children’s Villages in Ethiopia shall analyze the facts and extent of the delay and extend the time for performance when in its judgment the facts justify such an extension.

Copyright and other proprietary rights

SOS Children’s Villages shall be entitled to all intellectual property and other proprietary rights including, but not limited to, copyrights, and trademarks, with regard to products, processes, inventions, ideas, know-how, or documents and other materials which the Contractor has developed for SOS Children’s Villages under the Contract and which bear a direct relation to or are produced or prepared or collected in consequence of, or during the course of, the performance of the Contract. The Contractor acknowledges and agrees that such products, documents, and other materials constitute works made for hire for SOS Children’s Villages.

All materials: interviews, reports, recommendations, and all other data compiled by or received by the Contractor under the Contract shall be the property of SOS Children’s Villages and shall be treated as confidential and shall be delivered only to SOS Children’s Villages authorized officials on completion of work under the Contract. The external consultant is obliged to hand over all raw data collected during the assessment to SOS Children’s Villages in Ethiopia.

Termination

The termination of the service agreement for the assignment will be in accordance with the contractual agreement to be included at the formal agreement’s actual signing.

Qualification of the researcher / research team

• The applicant consultant shall have PHD degree (at least the team leader) in relevant fields of study such Psychology, Sociology, Social Work, development studies, Economics, Project Management, Monitoring & Evaluation and other related fields per the required assignment
• At least 5 years’ work experience in international NGOs & proven competency in assessments, monitoring, and evaluation, including baseline assessments or organizational/program evaluation (baseline, midterm, and final evaluations). The incumbent shall attach at least one sample baseline or final evaluation report produced by the candidate on a similar topic.
• A good understanding of family strengthening, community empowerment, livelihoods and development programme work.
• A good understanding of child/youth rights/safeguarding & other issues affecting vulnerable children in the Ethiopian context.
• Proven experience in working with conceptual frameworks and data collection methods (including age-appropriate data collection methods)
• Proven experience in participatory processes and data collection methods
• Strong skills in coordination, good facilitation, organizational, and interpersonal skills
• Proven experience in participatory processes
• Strong skills in coordinating teamwork
• Analytical and conceptual skills on midterm and final evaluations.
• Excellent written and spoken communication skills in English.
• Ability to transfer complex concepts and ideas into practical and simple language.
• Experience in organizing research processes with SOS Children’s Villages or similar child focused organizations.
• Experience in managing baselines, midterm and final evaluations in respect of the participating communities’ culture, social norms, values, and behaviour; and maintain appropriate relationships with participants of this evaluation.
• Legally registered firms with renewed license, VAT registration and TIN number