Terms of Reference for Consultancy Service to Conduct End-Term Evaluation for “Empower young people to promote their own economic, civic and social development” project

Amref Background

Amref Health Africa is the largest African-based and led international health development organization serving over 30 million people annually across 35 countries in sub-Saharan Africa. Strategically engaging with communities, governments, and local partners for the past 65 years, we work through our European and North American network of sister organizations on program implementation, advocacy, fundraising and partnership. Our subsidiaries include Amref Flying Doctors, Amref Enterprises Limited, and the Amref International University.

Our organization is driven by its vision to bring lasting health change in Africa and its mission to catalyze and drive community-led and people-centered health systems while addressing social determinants of health. Amref believes that the power to transform Africa’s health lies within its communities and therefore strives to ensure that health systems are functioning at an optimal level and communities are empowered to hold these systems accountable for the delivery of high-quality and affordable health care.

Amref has been active in Ethiopia since the 1960s and became fully operational as of 2002. We have been partnering with the Ethiopian government and local communities to co-create solutions and implement diverse health and development project portfolios throughout the country for over twenty years. Aligning with the national Health Sector Transformation Plan II (HSTP II), we work to ensure health equity by serving women and children, reaching the most disadvantaged, inaccessible communities including pastoralist communities and placing an emphasis on youth development.

Our programs are designed to support community level interventions while strengthening health systems at the regional and national levels. With an integrated and multi-sectoral approach, Amref strategically addresses cross-cutting issues through these programmatic focus areas.

Project Background

AFD Kefeta project works towards empowering youth to advance their own economic, civic, and social development. This project approach applies integrated skill building, financial access, technical skills training, youth-friendly health services, and opportunities to participate in and lead social and community issues. Kefeta Plus plans to reach more than 500,000 young people (15-29 years old, 55% of female) by designing youth centered interventions in youth centers and career development centers at Higher Education Institutions, health centers, public universities and TVET colleges.

The project activities are thus divided into three strata (civic, economic and social) which correspond to the three specific objectives of the project, namely: (i) SO1: Strengthening the capacity of young people to defend their rights, (ii) SO2: The creation of new economic opportunities for young people and (iii) SO3: Young people’s access to more appropriate health services. The project applies to a transformative and inclusive approach in addressing gender equality and inclusion issues and achieving and promoting positive and sustainable change.

Major project Indicators

Project objective verifiable indicators as stated in the project document

INDICATORS

R1.1 : Number of individuals receiving civic education through training

A. 1.1.1 : Percentage of youth who participate in civil society activities as a result of soft skills and life skills training or project initiatives

R.1.2. : Number of young people who declare having improved their personal effectiveness at the end of the training

A.1.2.1 : Number of civil society organizations (CSOs) benefiting from the project engaged in advocacy interventions.

A.1.2.2: Rate of increase in the level of civic engagement and/or political participation of participants following participation in a project activity

R2.1 : Number of people having a new job following participation in the KEFETA project

A.2.1.1 : Percentage of people who improved their soft skills after participating in the KEFETA project.

A.2.1.2. : Number of host country higher education institutions benefiting from capacity building support under the project

A.2.1.3 : Number of partnerships concluded to support youth employment and create youth-friendly services

R2.2 Number of women having access to economic opportunities after the training

Number of people who improved their hard skills and work-related skills

A.2.2.1 : Total number of clients benefiting from financial services provided by financial intermediaries

R3.1 The prevalence rate of modern contraceptives among young people has increased

A3.1.1 : Percentage of AFD-assisted service delivery site providing family planning (FP) counseling and/or services

A.3.1.2 : Number of health care providers trained on SRH/FP, including SGBV

A 3.2 The proportion of young people declaring that they have basic knowledge on contraceptive methods

A3.2.1 Number of young people who received SRH services health and family planning services and/or counseling

A.3.2.2 :Number of vulnerable young people connected to services adapted to their needs

Objectives of the project

Overall objective: Empower Ethiopian youth to promote their own economic, civic and social development.

Specific objectives:

1. Strengthen the capacity of young people to advocate for their rights and take leadership roles

2. Creating new economic opportunities for young people

3. Improve young people’s access to more appropriate health services

Purpose of this end-term evaluation

The purpose of the end term evaluation is to assess the extent to which the KEFTA Plus project has achieved its objectives and its overall performance. The objective of this evaluation includes:

General objective:

The overall objective of the evaluation is to assess the extent to which the project has achieved its intended results at long term outcome and intermediate outcome against predetermined targets. And the baseline findings will be used as a reference point to compare changes.

Specific objective:

1. To assess the degree to which project deliverable mentioned above was achieved
2. To examine the changes that resulted from project intervention comparing with baseline findings
3. To assess the relevance, efficiency, effectiveness and sustainability and impact of the project.
4. To identify and document best practices/success stories and model facilities renovated.
5. To provide recommendation to partners for scale up and donors for future funding

Results from this evaluation will be used to assess the significance of the intervention and to come up with evidence-based recommendations for scaling up or not. To highlight the project’s achievements/results, the consultant will focus on the results and outcome indicators targeted during project design.

Geographic coverage

This end term evaluation will be conducted in two project implementation areas (Debre Birhan town and Addis Ababa- Lemi Kura sub city). Selection of representative study units will be the responsibility of the consultant based on statistically sound criteria.

Proposed Methodology

The consultant is expected to propose a scientifically sound methodology that best suits the evaluation objective. Amref is open to considering the consultant’s suggestions. The methodology should include an appropriate sampling approach with robust data collection and quality assurance processes. The methodology should not only inform program decision-makers from a programmatic and scientific perspective but also generate findings that are suitable for publication in a peer-reviewed journal. The goal is to develop an evaluation approach that is both rigorous and practical, providing Amref with the information needed to make evidence-based decisions, while also producing outputs that contribute to the broader scientific literature in this domain.

Specific tasks to be done by the consultant

The list below will include main activities to follow :

• Meet Amref Health Africa in Ethiopia head office to present the work plan as per the tasks required as per the terms of reference (ToR).
• Review key databases and researches and other relevant documents like project baseline report, reports, documented data …etc including but not limited to develop the introduction, frameworks and methods will used.
• All reviewed databases should be included in standard referencing software’s preferably endnote program (also share will shared the databases used to Amref for reviewing)
• The firm should get approvals from Amref M&E department on the body of the content, introduction, framework/s and methods used before starting the final tool development.
• Develop the final tool as per the agreed introductions, frameworks and methods.
• Meet Amref Health Africa in Ethiopia head office and project staff at the beginning and at the end of the field work for inception report and debriefing.
• Discuss with Amref program development/program/project managers and M&E staffs
• Identify key partner for interviews from the government and private sectors at all levels
• Develop protocol that clearly defines the methodologies for sampling/ disaggregated by district/urban, data collection, entry, cleaning, compilation, analysis of data and report writing (Approvals required from Amref M&E department) and so that Amref health Africa will have a mandate to follow the process directly or indirectly.
• Recruit qualified data collectors and train evaluation team and pre-test data collection tools.
• The team compositions for the data collectors have to be very well qualified and have experience in both quantitative and qualitative research.
• Coordinate data collection, processing, analysis, management and reporting.
• Conduct a final debriefing at an Amref health Africa Ethiopia Office
• Develop and submit final report incorporating all comments given by Amref Health Africa Ethiopia, datasets and Do-files.

Period for Submission and deliverables

• The consultancy firm or team should follow the overall time frame of 60 calendar days starting from signing the agreement to the finalization of report (incorporating all changes and requests from Amref).
• The consultancy firm will submit and present the inception report that clearly defines the methodologies for sampling and data collection tools in the first week following signing the contract. Final data collection tools will be annexed to the protocol and address the feedback in the second week of agreement.
• Training of data collectors and supervisors will take place in the second week following singing the contract.
• By end of fifth week, data should have been analyzed and first draft of report submitted. The actual data collection will take place in third and fourth week following signing of the contacts.
• The consultant will update Amref Health Africa in Ethiopia on the progress of the work regularly.

Expected Key Outputs

• Detailed Final proposal: including the consultant’s understanding of the assignment, proposed methodology to be used (for sampling, data collection, and entry, cleaning and data analysis), report structure, deliverables, timelines, budget and CV.
• Final Data Collection Tools (Use Amref data warehouse for data collection)
• Analysis plan and steps
• 1ts draft of the end term evaluation report
• 2nd draft of the end term evaluation Report
• Final report and presentations as required.
• Final Evaluation Report: hard copy, complete data set and analysis syntax / preferably STATA dataset and DO-file.

Required Experience

• The team will consist of professionals in the fields of public health, economics and Social Science with a minimum of second-Degree. Very good professional diversity is an advantage in related areas. In addition, to the above requirement, consultancy firm or team is required to meet the following minimum requirements in order to qualify for the submission of its proposal:
• Significant Previous experience in similar tasks(evaluations) or consultancies specifically in project integrations and outcome-based financing; previous experience in similar tasks or consultancy. In particular, the applicant must have experience in conducting quality evaluation and performing impact data collection methods associated with service integration setting.
• Previous outstanding research and publication in reputable journals.
• Ability to effectively coordinate with government, NGO, and local stakeholders’ essential, an understanding of sectors coordination mechanisms and asset.
• Analytical and conceptual ability, demonstrated understanding of evaluations, research design and social research methods to understand the impact of the project.
• The consultant should have experience of using digital data collection tools /technologies (ODK, big data analysis and others).
• High quality English report writing and proofreading.
• The consultancy firm should be a registered firm and whose license has been re-registered and renewed for the current Ethiopian fiscal year.

Evaluation Criteria

Evaluation Criteria of Applicants

Any interested bidder shall provide, in its submission in response to this TOR, sufficient documentary evidence of adherence to eligibility and minimum qualification criteria detailed below. Amref Health Africa will assess the documents submitted by the applicants in response to this TOR.

A. Preliminary Evaluation Criteria

Bids will be evaluated based on the criteria below.

Bids lacking any of the documents below will be considered as non-responsive and therefore will be eliminated at this stage.

 

No.	Particulars	Marks	Compliant	Non-compliant
1	Provide a copy of valid business Trading License.	1 or 0
2	Provide VAT Certificate	1 or 0
3	Provide a copy of Valid TIN/Tax Compliance Certificate	1 or 0

Note: Bids missing any of the 3 mandatory requirements above will be considered as non-responsive bid and therefore will be eliminated at this stage.

Requirement	Criteria	Max Score	Actual Score	Remarks
Introduction & frameworks, & method used	[54 marks] total Marks	54 Marks
	Description of local area contexts and population- relating to the study nature (2) Clarity of the study population description- How specifically explain (2) Determined justifiable sampling procedure and sample size Method selected for the study -3 Sample size and determination -2 How the sampling frame mentioned-5 Options presented for the methods-2
	Does the consultant understand the ToR and show an approach for the approaches ( 10) Stated appropriate and relevant data collection approaches as per the ToR ( 10) Clarity of Methods used to address objectives and target groups for this evaluation ( 5) Does the consultant explain clarify the analysis for each objectives- 4 Method to capture the targets/study units -5 How the ethical issues addressed mentioned-1 Feasibility of the study with stated method Vs objective- 3
	Letter of interest addressing track record. [5 marks]
Work Experience	Provide appropriate references to demonstrate having the technical experience specified in the TOR. Track record in similar assignments, (Attach relevant documents as proof of previous works handled) Provided 4 reference & contact details [6 Marks] Provided 2 reference & contact details [4 Marks] Provided 1 reference & contact detail [2 Marks] Iv, Did not provide reference [0 Marks]	6 marks
Organizational Capacity, Team Composition and Qualification	Team Qualifications (Provide CVs and academic certificates of the team which includes educational background, professional certifications, ) [10 marks) Years of relevant experience of the team [5 marks] Attach company profile. – [5 Marks]	10 Marks
	Total Score	70
Note that: The technical proposal should have met the high quality has to get a score of 75% or 52.5/70 of the technical marks.

The remaining 30% accounts for the finance offer of the consultants. Only technically qualified consultants are invited to the financial proposal.

• Proposals will be evaluated based on experience and qualification of the candidates, price/cost effectiveness, and conceptual understanding and suggested methodology/approach. The evaluation mainly considers the three blocks namely methodological proposal/technical proposal/, the profile of the evaluators/consultant team members/ and the economic/financial/ offer.
• The weight for technical and financial proposals shall be 70 % and 30% respectively. The financial proposal will be considered when only the technical assessment is above 60 out of 100%.
• Failure to respect the statements in the TOR by the consulting firm will result in the termination of the contract agreement by Amref Health Africa.
• The consultant is obliged to avail all the experts mentioned in the technical proposal for the assignment. If the consultant wants to change any personnel, he/she should formally communicate to Amref Health Africa and get approval

Technical evaluation Criteria

Logistics

Amref Health Africa in Ethiopia will not provide office space, computers, copying, printing and telephone services to the Consultant, the consultant is expected to cover the costs related to vehicle for the fieldwork and thus, advised to include all those costs that Amref Health Africa will not cover including vehicle costs for the field work within prices to be quoted by his/her firm.

Others

• The consultant will work closely with Amref health Africa Ethiopia Program Manager, Regional Manager and M & E unit and project staff
• All relevant documents should be submitted to Amref health Africa upon completion of tasks both in hard and soft versions.
• The consultant should not share any of the evaluation results to any other third party without the consent and approval of Amref Health Africa in Ethiopia

Payment Modalities

The consultant will be issued consultancy contracts and paid in accordance with Amref Health Africa’s financial rules and regulations. Payment needs to be correlated to deliverables – two installments are typically planned for:

• The first payment of 30% in advance of the total agreed contractual amount will be made immediately after the signing of the contract agreement based on Amref Health Africa’s financial regulations and procedures.
• The second payment of 70% of the total agreed contractual amount shall be made effective to the consultant upon the approval and acceptance of the final evaluation report.

Annexes

Any documents such as curriculum vitae, publications, work experiences, renewed license and other information, which the applicants feel, will assist the proposal review team in evaluating the proposal may be attached as annexes.