Invitation to bid for Provision of consultancy service Consultancy Service to Conduct End-Term Evaluation for the project: Women and girls lead change for gender equality in Dubti district in the Afar Region

Background

Ethiopia ranks 79th out of 156 countries in terms of gender gap[1], with women facing severe disadvantages in literacy, health, livelihoods, and basic rights.

The Afar region exemplifies these inequalities, showing some of the worst health indicators in the country. It has one of the highest adolescent birth rates (20%), the lowest average age at marriage (16.4 years), and first sexual intercourse (16.2 years). Modern contraceptive use is only 13%, while just 28.3% of births occur in medical facilities, and under-five mortality remains high at 58 deaths per 1,000 live births[2]. These figures reflect a regional reality shaped by interrelated socio-cultural practices such as early marriage, forced marriage, and female genital mutilation (FGM).

Afar’s customary system, Adda, prescribes rigid gender norms, designating men as heads of households with decision-making power and ownership of assets, while relegating women to caregiving roles and household chores. Women are excluded from social decision-making, which is dominated by older men, and must comply with decisions, regardless of their well-being. These deep-rooted structures perpetuate harmful gender stereotypes and reinforce systemic inequalities.

Baseline data from this project[3] underscores these challenges: 42% of women and 4.1% of schoolgirls have experienced early marriage, and 75% of women aged 15–49 have undergone FGM, with prevalence among schoolgirls at 33%. Menstrual health issues further compound disparities, with 15% of girls missing school during menstruation due to lack of awareness (33%) and inadequate privacy (90%). Knowledge and attitudes towards gender-based violence (GBV) are alarmingly low among both household heads (51.4% and 52.1%) and women (54.8% and 70.2%), while GBV remains prevalent in all forms—sexual (48.9%), physical (45.2%), and psychological (41.2%).

These issues are rooted in power imbalances, harmful cultural norms, and systemic barriers to care, such as inadequate facilities, insufficient legal protection, and lack of psychosocial support. The normalization of GBV in Afari society, fueled by entrenched social structures, highlights the urgent need for targeted interventions to address gender inequalities and dismantle discriminatory practices.

Background of the project: Amref Health Africa is the beneficiary of the grant from the Spanish Agency for International Development Cooperation (AECID) for the implementation of the project entitled “Women and girls lead change for gender equality in Dubti district in the Afar Region” in Semera-Logiya and Dubti towns. Health Africa in Ethiopia, is the project implementer on the field. Amref Health Africa Spain provides technical support for its implementation and is the interlocutor with AECID at the level of the central offices in Madrid and the Technical Cooperation Office in Ethiopia. The project started on February 14, 2023 and will end on 14 February 2025.

The goal of the project is to contribute to the improvement of women’s and girls’ rights in the Dubti district by advancing gender equality through their social and economic empowerment.

Project objectives and results:

Overall objective: Contributing to the improvement of women’s and girls’ rights in the Afar Region

Specific objective: Advancing gender equality through the social and economic empowerment of women and girls

Results:

• Result 1: Improved community engagement in preventing and responding to gender-based violence.
• Result 2: Increased school attendance of girls through improved menstrual hygiene and health.
• Result 3: Sexual and reproductive health and gender-based violence response services improved.
• Result 4: Improved access to economic opportunities for vulnerable women and youth.

Major project Deliverables

At this time, we expect to have the following major outcomes / project Objectively Verifiable Indicators (OVIs) to be achieved:

Specific objective

• 15% increase in the number of users attending SRH and GBV services.
• 10% reduction in prevalence of GBV in the 2 districts.
• 15% reduction in absenteeism of girls/adolescents in intervention schools
• 15% reduction in absenteeism of girls/adolescents in intervention schools
• 75% of the created entrepreneurship groups have a continuous and sufficient income 6 months after the end of the project.

Result 1:

• 10% increase in the number of reported cases of gender violence
• 10% increase in the number of people who declare themselves against violent attitudes of husbands towards wives.
• 50% increase in the number of people who would support a woman who has been abused by her partner.
• 30% reduction in child marriages (girls married before the age of 18) and FGM in children under the age of 1.

Result 2:

• 50% of the school community have improved their knowledge and attitude towards menstrual hygiene management
• 20% increase in the number of women and girls using affordable and hygienic menstrual pads.

Result 3:

• 10% increase in the number of women receiving GBV and SRH counselling services
• 30% increase in the number of women who are aware of the availability of comprehensive gender-based violence services in their community

Result 4

• At least 12 business entrepreneurship groups with continuous income within 6 months at the end of the project
• At least 70% of the entrepreneurship groups get credit from micro-finance offices.

Project Activities

Result 1:

• A.1.1. Community training on human rights, gender equality, SRH and VAW
• A.1.2. Establishment of a community learning and experience-sharing platform
• A.1.3. Involvement of community and faith-based organisations in the prevention of gender-based violence A.1.4.
• A.1.4. Workshops and community dialogue sessions at district level on GBV
• A.1.5.: Baseline study (including KAP on youth perception of GBV and menstruation)
• A.1.6. Final external evaluation

Result 2:

• A.2.1. Establish, strengthen and support an Advisory Committee, made up of girls and adolescents, in schools.
• A.2.2. Training students in menstrual pad production and provision of materials for menstrual pad manufacture.
• A.2.3. Training in Menstrual Hygiene Management for the school community (students and teachers).
• A.2.4. Construction of Safe Spaces for menstrual hygiene management in 2 secondary schools.

Result 3:

• A.3.1. Training for health, legal and law enforcement professionals on gender-based violence
• A.3.2. Training for health personnel on youth-friendly sexual and reproductive health services
• A.3.3. Equip 6 medical centres to provide youth and women-friendly SRH and GBV services A.3.4.
• A.3.4. Development of a quality improvement manual for GBV centres
• A.3.5. Strengthening of the referral system between GBV and SRH points of care

Result 4:

• A.4.1. Entrepreneurship skills training for women and young people in vulnerable situations
• A.4.2. Vocational training for women and young people in vulnerable situations.
• A.4.3. Financial support to entrepreneurship groups and micro-finance credit facilitation
• A.4.4. Joint supervision of support and monitoring of entrepreneurial activities
• A.4.5. Mentoring and counselling of young women and recent graduates with successful business people

Beneficiaries of the project

The direct rights holders are 3,500 women, 2,900 female primary and secondary school pupils and 5,250 youth in Dubti district (this includes the towns of Semera, Logia and Dubti and Dubti kebeles), who will be empowered through education, awareness raising and support to undertake economic activities provided by the project.

The project has worked in 6 public schools: 3 primary schools – Migilikebo, Halodora and A/Sheleko – and 3 secondary schools – Logiya, Semara and Dubti and 6 medical centres – the OSC of Dubti hospital and 5 district health centres.

All members of the community, couples, parents, community and religious leaders, will benefit indirectly form the project in both intervention woredas, which is estimated 134,797 pupils.

Table: Direct and Indirect beneficiaries reached. Expected to be measured by the Consultant during the evaluation

 

Result	Direct beneficiaries	Indirect beneficiaries
Result 1: Improved community engagement in preventing and responding to gender-based violence.	Women and girls in vulnerable situations benefited/participated from the project activity interventions stipulated under project Result1   Community and religious, women leaders, WDT, HEW, school community, CBOs, elderly people, health workers, local authorites and administrators of the implementing kebeles involved and who has participated in the activities of Result1. Need to be Sex and age disaggregation	People/community members indirectly benefited from the project interventions stipulated under project result 1 ( Ex- families, community members) Need to be Sex and age disaggregation   See Annex-2: guideline for used for estimating beneficiary reach
Result 2: Increased school attendance of girls through improved menstrual hygiene and health.	School community ( students, teachers, school clubs, parents’ school association, etc.) in primary and secondary schools directly benefited/participated from the project activity interventions stipulated under project Result 2   Need to be Sex and age disaggregation	People/community members indirectly benefited from the project interventions stipulated under project result 2 ( Ex- families, of students) Need to be Sex and age disaggregation   See Annex-2: guideline for used for estimating beneficiary reach
Result 3: Sexual and reproductive health and gender-based violence response services improved.	Victims of gender-based violence, women and girls directly benefited from the project activity interventions stipulated under project Result 3   People served by justice, health and law enforcement professionals trained in activities under Result 3.   Need to be Sex and age disaggregation	People/community members indirectly benefited from the project interventions stipulated under project result 3 ( ex. patients in the 6 medical centers equipped by Amref under Result 3)   Need to be Sex and age disaggregation   See Annex-2: guideline for used for estimating beneficiary reach
Result 4: Improved access to economic opportunities for vulnerable women and youth.	Women members of the entrepreneurship groups formed and supported with the activities of Result 4.   Need age disaggregation.	Family members of the women members of the entrepreneurship group. Need to be Sex and age disaggregation   See Annex-2: guideline for used for estimating beneficiary reach

Purpose of this end term evaluation

The purpose of the end term evaluation is to assess the extent to which the project has achieved its objectives and the overall performance. The objective of this evaluation includes:

General objective:

The overall objective of the evaluation is to assess the extent to which the project has achieved its intended results at outcome and output levels against predetermined targets. And the baseline findings will be used as a reference point to compare changes.

Specific objective:

1. To assess the degree to which project deliverable mentioned above were achieved
2. To examine the changes that resulted from project intervention comparing with baseline findings
3. To assess the relevance, efficiency, effectiveness and sustainability and impact of the project;
4. To identify best practices/success stories
5. Track record of the scaling up impact of the previous program and supplement the recommendation
6. To provide recommendation to partners for scale up and donors for future funding

Results from this evaluation will be used to assess the significance of the intervention and to come up with evidence-based recommendation for scaling up or not. To highlight the project’s achievements/results, the consultant will focus on the results and outcome indicators targeted during project design.

Scope of the evaluation

This end term evaluation will be conducted in Dubti district, in the towns of Semera, Logia and Dubti town. Both Dubti and Semera-Logiya town belongs to Administrative Zone 1 (Awsi resu) of Afar region. Selection of representative study units will be the responsibility of the consultant based on statistically sound criteria.

Actors Involved in the Evaluation Process

The evaluation will be undertaken by External Independent consultant recruited through competitive bid process using Amref procurement procedure. Amref M & E and research unit will lead and monitor the overall evaluation process together with the project staff and Amref Spain.

The evaluation team will work closely with Amref’s monitoring and evaluation (M&E) unit and research unit as well as project staff (project manager, program manager and project assistant). Government partners at regional and district level (Health, Women & Children’s Affairs, Education, Regional Labour & Training and Justice offices and departments) and kebele/woreda administrations are expected to provide the necessary support for the field evaluation process.

The different units of Amref Health Africa, such as Procurement and Finance, also have an important role in the evaluation process, especially in the recruitment of the consultant and in making the stipulated payments.

Agents involved and collaborating entities in the project and their relevance and participation in the evaluation are shown below:

Agents involved and entities collaborating in the intervention	Relevance for the evaluation	Participation in the evaluation	Key informants
Office of Women’s & Children’s Affairs and its district office	Ensures implementation of women’s policy, enforcement of laws in relation to gender-based violence and women’s equal participation in social, economic and political affairs.   They have participated in the project by providing community training on human rights, gender equality, SRH and violence against women. Supported the implementation of activities.	They will participate in key informant interviews (KIIs) where they will provide in-depth information on the achievements, successes, challenges and impacts of the project implementation   Support the evaluation process	The project focal/contact/ person at regional Women and Child Afairs bureau and respective district offices.
Afar Regional Justice Office and courts	Ensure implementation of laws/policies specifically in relation to preventing and responding to gender-based violence and harmful traditional practices (HTP), such as FGM or child marriage.   They have put in place mechanisms to develop monitoring, tracking and reporting of GBV and HTP cases in collaboration with community members and local administration.	They will participate KIIs and support the evaluation process	Relevant officials and experts for the project intervention
Regional Health Office and district heath offices with their facilities (Health centers and hospitals)	Regional decision-makers in health matters. Made recommendations during the project registration process. Supported the implementation of activities. Ensure the selection and availability of experts for training. Provide necessary support (health and psychosocial) to survivors of sexual and gender-based violence and referral linkage from community to health facilities through HEWs. Provide youth friendly sexual and reproductive health for vulnerable girls and youths Assign roles in response to sexual and gender-based violence within health centres and strengthen the referral system. Participated in the rapid assessment of medical equipment and supplies needs in the 6 intervention medical centres. (the management team of health centers and hospitals)	They will participate KIIs and support the evaluation process	The project focal/contact/ person at regional health bureau and respective district health offices. The management team of health centers and hospitals
District education offices and schools	Selection of the 6 public intervention schools. Support and coordinate activities in schools. Ensure the selection and availability of experts for training. Prevention activities through the advisory committees and liaise with the local police to report suspicious situations. Create awareness for students and school community on gender equality, VAW including FGM, early marriage, & GBV, and MHM for girls and work with the community Closely monitor truancy cases.	They will participate KIIs and support the evaluation process	Relevant official to the project intervention
Regional and district police	Investigate cases of HTP and gender-based violence and refer them to the responsible government agency for prosecution.	They will participate KIIs and support the evaluation process	The focal persons for the regional and district police offices.
Regional labour and training office	Advice, consultation and participation in the process of creating economic opportunities for women.	They will participate KIIs and support the evaluation process	Focal person for the regional office
District microfinance offices	Support training and training and lending to women in the economic groups formed and will network with other institutions	They will participate KIIs and support the evaluation process	Relevant officials to the project intervention
Government vocational training institutions (TVT):	To provide courses in entrepreneurial skills and vocational training for women and youth in vulnerable situations.	They will participate in key informant interviews (KIIs) and support the evaluation process	Relevant focal person to the project intervention
Kebele/woreda administration:	Facilitate the overall implementation of project activities.	They will support the evaluation process
Amref Project Team	Field project implementers and programmatic support	They will provide the evaluation team with all the information on the project, from the formulation phase, implementation, monitoring and evaluation.	Program Manager Project Manager Project Officer M&E Officer
Consultant Team		Develop detail evaluation methodology and tool; conduct the field data collection; and submit the report	To be recruited through Amref competitive bid process

The participation and responsibilities of each stakeholders are shown below:

S.N	Organization/Unit	Responsibility/Involvement
1	Evaluation management unit: Amref Spain ( desk officer), Amref ET ( PM, PO, M&E research unit, finance unit)	It acts as the mandated unit of the evaluation, in charge of validating and supervising the quality of the process, drafting the ToR, hiring the evaluation team/consultant/, publishing and disseminating the results. • Develop draft TOR • Finalize the TOR incorporating comments and feedbacks • Share the final TOR and Request the procurement unit for advertisement • Participate in the technical evaluation of the consultants applied for recruitment • Follow the recruitment of the consultant and signing of the agreement • Receive, review/comment/ the detail inception report and data tool submitted by the consultant • Receive the final approved data collection tool and inception report with agreed work plan • Provide necessary supportive documents for the consultant (project documents, reports, etc.) . Provide necessary documents and information for the consultant that will be useful for the evaluation . Facilitate and provide necessary support for the consultant at field level ( such as contacting/introducing the consultant team with government partner offices, arrange meetings if necessary, arranging beneficiaries for interviews and focus group discussions, showing project sites, etc. . Monitor the data collection and evaluation process at field level . Early communicate the regional government partners and community members to be aware about the evaluation and to provide necessary support and information for the evaluation team. Receive, review/comment the evaluation report and ensure the project evaluation is done as planned, with the expected quality and standard . Publishing and disseminating the results Make payments for the consultant as per the agreement and agreed schedule
	Evaluation Monitoring Committee: One representative from the OTC, Amref Spain ( desk officer), Amref ET (M&E research unit)	Responsibilities: Receive, review/comment/ the detail inception report and data tool submitted by the consultant • Receive the final approved data collection tool and inception report with agreed work plan • Receive, review/comment the evaluation report and ensure the project evaluation is done as planned, with the expected quality and standard validate the products that are obtained identify information needs Monitor the quality of the process . Publishing and disseminating the results   It will be formed by the following with the specifics roles: M&E Unit: Mihret Ayele will organize the following up and meetings with the evaluation team and the Evaluation Monitoring Committee. Amref Spain: Ana Gallego, she will participate in the meetings with the M&E and evaluation team to provide Amref Spain feedback OTC: Thais González Capella will participate in the meetings with the M&E and evaluation team. She will be invited to participate in the field to provide the donor feedback/when ever necessary   The M&E team will receive the documents (inception report, tools, etc) from the evaluator, which will be shared with Amref Spain and the OTC for review. In joint telematics meetings the evaluator will present his reports and all parties (Amref Et, Amref SP and OTC) will give their comments and will validate the documents.
	External consultant	Develop, share the detail evaluation proposal/inception report/ that includes the detail data collection tool, methodology, and time frame Revise the tools and inception report as per the feedbacks given and share the final approved one Conduct the data collection, as per the agreed inception report and time frame Avail all professionals and data collectors indicated in the technical proposal Submit the draft evaluation report Submit the final evaluation report both in hard and softcopy after approval Submit the raw data ( SPSS, or Stata)

Documents and sources of information

The evaluation team is expected to review the project and other relevant documents, including but not limited to those shown in the following documentation matrix:

Documents	Authorship	Location
Project documents
Project formulation document	Amerf	Google Drive
Partnership Agreement between Amerf Health Africa Ethiopia and Amref Salud Africa Spain	Amerf	Google Drive
MoU between Amref Health Africa Ethiopia and Amhara Government	Amerf	Google Drive
Desk study on ‘Policies and practices on gender equality and gender-based violence (GBV) in Ethiopia’.		Google Drive
Baseline of the project	HORN Alliance Consulting and Trading PLC	Google Drive
Monitoring & Evaluation Plan	Amerf	Google Drive
Annual project monitoring report (including means of verification)	Amerf	Google Drive
Regulatory Framework
Guide for Evaluations of Development Cooperation Agreements, Projects and Actions	AECID	Google Drive
Spanish Development Cooperation Evaluation Management Handbook	SECI – MAEC	Google Drive
Rules for the management, monitoring and justification of NGDO agreements, projects and development cooperation actions. Application Guide	AECID	Google Drive
Studies on the sector/region
Policy and Practices towards Gender Equality and Sexual and Gender Based Violence (SGBV) in Ethiopia: A Desk Review	The HORN Alliance Consulting and Trading PLC	Google Drive
Spanish cooperation planning documents
Country Partnership Framework Ethiopia – Spain 2011-2015	OTC Ethiopia	Google Drive
V Master Plan for Spanish Cooperation	AECID	Google Drive
Plans, regulations or other documents from the country of intervention
National Costed Roadmap to End Child Marriage and FGM/C 2020–2024	Ministry of Women, Children and Youth (MoWCYA)	Google Drive
National Strategy and Action Plan on Harmful Traditional Practices (HTPs) against Women and Children in Ethiopia. Ministry of Women, Children and Youth (MoWCYA)	Ministry of Women, Children and Youth (MoWCYA)	Google Drive
National gender mainstreaming guidelines- Ministry of Women’s affairs	Ministry of Women’s affairs	Google Drive

These documents will be available to the Evaluation Monitoring Committee and the Evaluation Team in a Google Drive folder shared with all parties.

Evaluation focus

The consultant will develop a detail data collection methodology and data collection tool that enables to captures all project achievements against all indicators indicated in the log frame and information as per standard five evaluation dimensions ( DAC evaluation criteria) comparing the data collected with the data obtained in the baseline attached in order to measure the progress.

One for each of the evaluation criteria. A battery of specific questions should be presented to analyses the expected/actual impacts, as well as the implementation strategy implemented and to respond to the different aspects of each of the evaluation criteria.

DAC criteria to be used:

Relevance: The degree to which the aid activity is appropriate to the priorities and policies of the target group, the recipient and the donor. In assessing the relevance of the project, it is useful to consider the following questions.

• What is the relevance of carrying out this project in the region, and specifically in the project target districts
• To what extent are the project objectives still valid in order to meet the overall objective?
• Has the project intervention logic (activities, outputs and objectives) been relevant to achieve the overall objective?

Efficiency: Efficiency measures qualitative and quantitative results in relation to inputs. It is an economic term that means that the assistance uses the least costly resources possible to achieve the desired results.

In assessing project efficiency, it is useful to consider the following questions

• To what extent have the external factors faced by the project (COVID-19 political instability, conflict, etc) conditioned the efficiency of the project?
• Were the activities cost-effective and were the funds managed in an adaptive manner?
• Was the program or project implemented in the most efficient way compared to other alternatives?
• Was the strategy implemented by Amref efficient and effective?

Effectiveness: The extent to which an aid activity achieves its objectives. When assessing project effectiveness it is useful to consider the following questions:

• To what extent have the objectives been/are likely to be achieved?
• What were the main factors and risks that influenced the achievement or non-achievement of the objectives?

Impact: The positive and negative changes produced by a development intervention, directly or indirectly, intended or unintended. This refers to the main impacts and effects resulting from the activity on local social, economic, environmental and development indicators. The review should address both intended and unintended outcomes and should also include the positive and negative impact of external factors. When assessing the impact of the project it is useful to consider the following questions

• What direct positive effects has the project intervention brought?
• What indirect positive effects did the project intervention have?
• What negative positive effects has the project directly or indirectly caused?
• What positive and negative impact did the external constraints have on the project beneficiaries?
• Has the project made a real difference to the beneficiaries? In how many beneficiaries?
• Highlight aspects of positive impact from a social, economic and environmental point of view.

Sustainability: Sustainability is concerned with measuring whether the benefits of an activity are likely to continue after donor funding has been withdrawn. Projects must be both environmentally and financially sustainable. When assessing project sustainability it is useful to consider the following questions:

• To what extent do the benefits of the project continue after donor funding has ended?
• What have been the main factors that have played a role in achieving or failing to achieve project sustainability?
• What mechanisms has the project put in place to ensure project sustainability?

The evaluation will also review and identify substantial and/or accidental modifications during project execution, and will Identify foreseen and unforeseen risks (political instability and conflict mainly) that may have affected the achievement of the project results and the above the evaluation criteria.

1. Methodology and work plan

The final evaluation has to employ both quantitative and qualitative survey methods including KAP survey. The detail evaluation questions, Sampling frame/Sampling unit/, Sampling procedure and sample size determination, method of data collection and analysis, method of qualitative data analysis are among the major ones that need to be included in the methodology part. The consultant will prepare or adopt appropriate tools, which covers the key areas identified in the key tasks of the survey. The tools should be seen and discussed with Amref Health Africa in Ethiopia M & E unit and project staff. The consultant also expected to develop a tool to capture case stories / success stories/ best practices/form the project implementation

Work plan

The consultant is expected to complete the assignment within 6 weeks of the agreement signed with Amref. The detail time schedule need to be submitted by the consultant as part of the technical proposal and inception report subject for review and revision by Amref in discussion with the consultant.

1. Structure and Presentation of the final evaluation report

The consultant will produce a report not more than 50 pages plus appendices, in Microsoft Word using Arial font 12. The report should include:

I. Basic information such as:

• Project title
• Agency name
• Project Reference Number
• Country
• Name of local partner(s)
• Name of person who compiled the evaluation report, including summary of role/contribution of others in the team
• Period during which the evaluation undertaken

II. Executive Summary (1 page)

III. Full Evaluation of Project containing:

• Contents page
• Abbreviations/acronyms page
• Executive summary
• Achievement Rating Scale
• A short introduction to the project
• The evaluation methodology
• Findings from the assessment
• A summary of recommendations
• Lessons learned indicating with whom and how lessons should be shared

Annexes (ToR, names and contact details of the evaluators along with a signed declaration of their independence from the project team; the evaluation schedule, people met, documents consulted statistical data on baselines and end of project surveys; the logical framework, and Data collection tools.

Possible list of Annexes

• Case/success stories
• The ToR.
• The work plan, composition and description of the mission.
• Proposed methodology, techniques and sources used to collect information
• Documentary review: list of secondary sources used.
• Interviews: list of informants, interview script, transcripts and notes.
• Surveys: models, raw data compiled and statistical analysis.
• Participatory workshops: rapporteurship and products.
• Allegations and comments from different actors on the draft report if
• Considers it relevant, especially if there are disagreements and have not been
• Reflected in the body of the report. (See responsibility premise).
• Evaluation summary sheet (model CAD24).

The final evaluation report all attached documents which are part of the evaluation report will also be translated in Spanish language

IV. A table detailing Actual Cumulative Results of the project, corresponding with all indicators included in the PMF

Reporting requirement

• Inception report containing a detailed work plan for the entire duration of the consultancy service will be submitted within a week of signing of the agreement;
• Weekly progress report should be made by the consultant to the Amref Health Africa Ethiopia (Project Manager, Programme manager and M&E officer);
• Evaluation protocol (including data collection tools); and
• Draft and final evaluation reports.

[1] Global Gender Gap Report 2024, World Economic Forum

[2] CSA, 2016 and Mini Demographic and Health Survey 2019

[3] Amref Health Africa, Women and girls Lead change for gender equality in Afar Region, Ethiopia: project baseline assessment, November 2023

 

The Evaluation team

Extensive skills, experiences and qualification in applying qualitative and quantitative evaluation methods, data analysis skills and facilitation skills and other relevant competencies – such as language proficiency for locally spoken languages required to conduct this evaluation. Specifically;

• At least Master’s Degree in public health, epidemiology, sociology and demography or related field and demonstrated experience in household data collection related to SRH,
• A renewed professional license for consultancy in similar tasks,
• At least five years of experience in undertaking similar researches, surveys of health, women empowerment and gender related projects.
• Good knowledge with demonstrated practical working experience or research,
• Experience of effective interaction with local non-government organizations, government departments, and international organizations,
• Excellent spoken and written communication skills in English,
• Evidence of similar work in the recent past will be a key requirement. And Attachment of all supportive documents with the technical and financial proposal is required ( such as that shows the consultant’s previous experience )
• Local Language skills of data collectors,
• Minimum75% of the evaluation team composition is expected to be part of data collection
• Expertise in statistical software, SPSS, STATA, R etc
• The consultant should be a registered firm and whose license has been renewed for the current Ethiopian fiscal year.

Other remarks

• The Consultant has to guarantee its independence of the evaluated intervention, not being linked with its management or with any element that composes it. Likewise, it must not have worked for the executing agency/Amref Health Africa/ of the subsidy or for its local partners at least during the period of identification and execution of the intervention to be evaluated.
• Amref Health Africa will provide the information and support that the evaluation team needs and may be offered to accompany it, but it will be the one that determines if it is pertinent that the entity or its local partners be present during the development of the work of the Evaluation, since this could condition the results of the same.
• It will be an asset if the evaluation consultancy team is a mixed team. Taking account of the focus of the intervention on women empowerment and leadership the absence of women integrating the Evaluation team could introduce a negative bias on opinions/impressions obtained from women interviewed
• The evaluation teams/consultant/ will have the responsibility to show issues not specifically mentioned in ToR, if this were necessary to obtain a more complete analysis of the intervention. Although the draft report may be agreed upon in the follow-up committee with the other stakeholders, the evaluation team should record its assessment even though the grantor or other actors disagree. You can always clarify where there are discrepancies.

1. Premises of Evaluation, Authorship and Publication

• Anonymity and confidentiality. – The evaluation must respect the right of individuals to provide information ensuring their anonymity and confidentiality.
• Responsibility. – Any disagreement or difference of opinion that may arise between the team members of the consultant or between the team members and those responsible for the intervention, in relation to the conclusions and / or recommendations, should be mentioned in the report. Any statement must be supported by the team or record the disagreement about it.
• Integrity. – The Evaluation team/consultant/ will be responsible for highlighting issues not specifically mentioned in ToR if this is necessary to obtain a more complete analysis of the intervention
• Independence. – The evaluation team/consultant/ must guarantee its independence from the evaluated intervention, not being linked to its management or any element that composes it.
• Validation of information. – It is the responsibility of the evaluation team/consultant/ to ensure the accuracy of the information collected for the preparation of reports, and ultimately to be responsible for the information presented in the Evaluation Report.
• Incidents.- In the event of problems occurring during fieldwork or at any other stage of the evaluation, they must be immediately notified Amref Health Africa, and the latter, if it considers it necessary, To the AECID. Otherwise, the existence of such problems may not be used to justify the failure to obtain the results established by the entity in the present TOR.
• Copyright and Disclosure. – Clarifies that all copyrights fall on the contracting entity of the evaluation. The disclosure of the information collected and the Final Report is the prerogative of the entity executing the grant/Amref Health Africa/. However, the AECID reserves the right to reproduce, distribute or publicly communicate the evaluation report without prior agreement with said entity, when required by the correct development of administrative procedures and will do so with prior authorization of the same, when is required for other types of reasons.
• Sanctioning regime. – In case of delayed delivery of reports or in the event that the quality of the reports delivered is manifestly lower than agreed, the penalties and arbitrations established by the contracting entity will be applicable according to the established private contract with the evaluation team/consultant/.

2. Presentation/submission/ of the Technical and financial Proposal and valuation criteria

Competitive bidders are expected to submit their bid proposal as follows:

• Cover letter: – Signed one page cover letter containing, the name, mailing, address, telephone number and brief relevant information of the applicant.
• Detail Technical Proposal In addition to detailed technical matters, the technical proposal shall provide a description of the consultant including an outline of the consultant’s recent experience in similar undertakings and a detailed plan for accomplishing the tasks described in the specific task section. This proposal should be submitted with a separate sealed envelope.
• Detailed Financial Proposal: – The financial proposal should list itemized details of costs associated with the Evaluation. All proposed costs should be expressed in Ethiopian Birr and this proposal should be submitted with a separate sealed envelope.
• References: – A list of three former or current clients for whom similar work has been done together with their addresses and telephone numbers.
• Annexes– Any documents such as curriculum vitae or other information which the applicants feel will assist the proposal review team in evaluating the proposal may be attached as annexes.

3. Evaluation Criteria of Applicants

Any interested bidder, shall provide, in its submission in response to this TOR, sufficient documentary evidence of adherence to the eligibility and minimum qualification criteria detailed below. Amref Health Africa will assess the documents submitted by the applicants in response to this TOR;

A. Preliminary Evaluation Criteria

Bids will be evaluated based on the below criteria.

Bids lacking any of the documents below will be considered as non-responsive and therefore will be eliminated at this stage.

No.	Particulars	Marks	Compliant	Non-compliant
1	Provide a copy of valid business Trading License.	1 or 0
2	Provide VAT Certificate	1 or 0
3	Provide a copy of Valid TIN/Tax Compliance Certificate	1 or 0

Note: Bids missing any of the 3 mandatory requirements above will be considered as non-responsive bid and therefore will be eliminated at this stage.

Requirement	Criteria	Max Score	Actual Score	Remarks
Introduction & frameworks & method used	[54 marks] total Marks	54 Marks
	Description of local area contexts and population- relating to the study nature (2) Clarity of the study population description- How specifically explain (2) Determined justifiable sampling procedure and sample size Method selected for the study -3 Sample size and determination -2 How the sampling frame mentioned-5 Options presented for the methods-2
	Does the consultant understand the ToR and show an approach for the approaches ( 10) Stated appropriate and relevant data collection approaches as per the ToR ( 10) Clarity of Methods used to address objectives and target groups for this evaluation ( 5) Does the consultant explain clarify the analysis for each objectives- 4 Method to capture the targets/study units -5 How the ethical issues addressed mentioned-1 Feasibility of the study with stated method Vs objective- 3
	Letter of interest addressing track record. [5 marks]
Work Experience	Provide appropriate references to demonstrate having the technical experience specified in the TOR. Track record in similar assignments, (Attach relevant documents as proof of previous works handled) Provided 4 reference & contact details [6 Marks] Provided 2 reference & contact details [4 Marks] Provided 1 reference & contact detail [2 Marks] Iv, Did not provide reference [0 Marks]	6 marks
Organizational Capacity, Team Composition and Qualification	Team Qualifications (Provide CVs and academic certificates of the team which includes Educational background, professional certifications, ) [10 marks) Years of relevant experience of the team [5 marks] Attach company profile. – [5 Marks]	10 Marks
	Total Score	70
Note that: The technical proposal should have met the high quality has to get a score of 75% or 52.5/70 of the technical marks.

The remaining 30% accounts for the finance offer of the consultants. Only technically qualified consultants are invited to the financial proposal.

• Proposals will be evaluated based on experience and qualification of the candidates, price/cost effectiveness, and conceptual understanding and suggested methodology/approach. The evaluation mainly consider the three blocks namely methodological proposal/technical proposal/, the profile of the evaluators/consultant team members/ and the economic/financial/ offer.
• The weight for technical and financial proposals shall be 70 % and 30% respectively. The financial proposal will be considered when only the technical assessment is above 60 out of 100%.
• Failure to respect the statements in the TOR by the consulting firm will result in the termination of the contract agreement by Amref Health Africa.
• The consultant is obliged to avail all the experts mentioned in the technical proposal for the assignment. If the consultant wants to change any personnel, he/she should formally communicate to Amref Health Africa and get approval

Technical evaluation Criteria

1. Logistics

Amref Health Africa in Ethiopia will not provide office space, computers, copying, printing and telephone services to the Consultant, The consultant is expected to cover the costs related to vehicle for the fieldwork and thus, advised to include all those costs that Amref Health Africa will not cover including vehicle costs for the field work within prices to be quoted by his/her firm.

Others

• The consultant will work closely with Amref health Africa Ethiopia Program Manager, Regional Manager and M & E unit and project staff
• All relevant documents should be submitted to Amref health Africa upon completion of tasks both in hard and soft versions.
• The consultant should not share any of the evaluation results to any other third party without the consent and approval of Amref Health Africa in Ethiopia

1. Payment Modalities

The consultant will be issued consultancy contracts and paid in accordance with Amref Health Africa’s financial rules and regulations. Payment needs to be correlated to deliverables – two installments are typically planned for:

• The first payment of 30% advance of the total agreed contractual amount will be made immediately after the signing of the contract agreement based on Amref Health Africa’s financial regulations and procedures.
• The second payment of 70% of the total agreed contractual amount shall be made effective to the consultant upon the approval and acceptance of the final evaluation report.

1. Annexes

EVALUATION SHEET ( CAD)

Title		Place
Sector		Sub-sector
Type of evaluation		Cost (€)
Date of implementation	Executing agent		Target group
Evaluation date	Executing agent
Background and objective of the intervention
Principles and objectives of the evaluation
Methodology and tools
Analysis of achievement of indicators stated in the proposal
Conclusions and recommendations according to criteria for the evaluation of the intervention (include those evaluated)	Effectiveness
	Efficiency
	Relevance
	Impact
	Sustainability
Specific conclusions
Specific recommendations
Agents that have intervened

Annex-2: Beneficiary Reach Estimation Guideline

Amref Health Africa

Guidelines for estimating the beneficiaries reached

Background

Amref Health Africa uses ‘beneficiaries reached’ as one of the global indicators to measure programme/or project performance. Implementing and M&E teams are required to report on the number of beneficiaries reached on regular basis. Beneficiaries could be defined as a sub-population in a defined catchment area who have benefited either directly or indirectly from the services provided by a programme/or project. The existing literature is silent about the methods for computing ‘beneficiaries reached’ by a project, especially those reached indirectly. While it is easier to compute/or estimate the beneficiaries who were reached directly with service, it has proven difficult to estimate those reached indirectly. Estimation of those reached indirectly is influenced by the context in which it is done and the type of project. For instance, in estimating the number of beneficiaries reached indirectly in a nomadic setup, a different approach from what works in ordinary rural settings or urban informal settlements is needed.

Implementing and M&E teams should familiarize themselves with the project design, documents (log frame, proposal and budget) and context before estimating the beneficiaries reached. They should be conversant with fundamental aspects of the project as indicated herein.

1. Understand the project context (nomadic, rural or informal settlements)
2. Establish the total catchment population, which the project was intended to serve as stipulated in the plan/proposal.
3. What does the proposal document say about the number of people to be reached directly and indirectly (disaggregated by sex and age, socio-economic status, etc)?
4. What activity results (outputs) is the project designed to produce?
5. What are the verifiable indicators for measuring the activity results (outputs) and short and medium term changes (outcomes)?
6. How many people does the project have to reach in order to achieve each of the planned project results- activity results (outputs), short and medium term changes (outcomes)?

A common pitfall in estimating the number of the beneficiaries reached, especially those reached indirectly is double counting. For instance, household members who benefited from health talks provided by a CHW could as well be beneficiaries of a village water borehole. Implementing teams should be careful not to count same beneficiaries more than once.

Estimating the beneficiaries reached

Following are examples of how to estimate the beneficiaries reached directly;

• number of community health workers trained can be obtained from the training workshop/seminar reports, financial records, attendance list or records;
• number of people served at health facility level(dispensary, health centre or hospital) can be extracted from the service provider report, health facility records or health management information system (HMIS);
• number of people attending awareness-raising and sensitization meetings on any thematic area e.g. NCDs, can be estimated directly during the meeting (journalist approach!) or by direct counting.

To estimate the number of beneficiaries reached (directly and indirectly) in a specific catchment area, a population census or demographic health survey (DHS) data should be used.

The steps involved are as follows.

1. Obtain the population of the project catchment area from local authorities-(census data). Extract information on the number of households and the average household size from the census data.
2. Determine the number of households (H) in the project area. Consult with the local authorities e.g. sub-county/sub-district statistics office or administration office, or baseline survey reports where available.
3. Obtain the DHS mean household size (S) statistic for the country/region/district or county/sub-county, where available[1].
4. Compute the target population, (K). That is, K=H x S.
5. Establish the total number of households served by the project, (L).
6. Compute the estimated beneficiaries reached (Y). That is, Y= L x S.
7. Use the DHS age-sex structure statistic for the country where project is based. DHS data on sex-age structure gives proportions of males, females and different ages. DHS provides population proportions in terms of males (M) and females (F) as percentages (%) of the total population. Use DHS male-female statistic to disaggregate the beneficiaries reached. DHS age-proportion statistic(s) should as well be used to disaggregate the beneficiaries reached by age.
8. Establish the Ministry of Health guideline or norm on how many households (N) each CHW is required to serve.
9. Use the DHS mean household size (S) statistic to compute the beneficiaries reached/or to be reached. That is, number of households (H) multiplied by DHS mean household size (S). Hence, if the project has trained (P) community health workers (CHWs), the beneficiaries (Y) could be computed as follows:
10. Extract data from health facility records e.g. maternity ward, outpatient department, FP, outreach/mobile clinics, public health office records, etc. Alternatively, use the accepted average number of people per population that a nurse or a doctor is required to serve, (WHO human resources for health guidelines).

Alternatively, where census data is not available, use the following steps to estimate the number of beneficiaries reached.

To disaggregate the number of the beneficiaries reached by sex,

To estimate the number of beneficiaries reached by community own resource persons e.g. CHWs,

Y= P x H x S.

To estimate the number of beneficiaries reached by a healthcare provider e.g. nurse, midwife, environmental health officer, etc.

Reporting

Report the number of directly reached beneficiaries separately from those reached indirectly. Disaggregate data under each of the two categories (indirect and indirect) by sex (male, female) and age (below 18 years, 18 years and above). Training takes a huge budget in health and development programmes. It is important to report on the number of people trained, being a subcategory of the beneficiaries reached directly.

	Below 18 years (<)					18 years and above (>)
	Male	%	Female	%	Total	male	%	female	%	Total	Grand total
# of beneficiaries reached directly
#of people trained (this is a subcategory of beneficiaries reached directly)
# of beneficiaries reached indirectly

[1] Where DHS data is not available, use the rule of thumb of 5 people per household as the last available option!