Terms of Reference for End-line evaluation of Project Evaluation of MA 2022 Health Programme Improved MNCH and MDR-TB Services across 9 Facilities in Southern Somalia

Terms of Reference for End-line evaluation of Project Evaluation of MA 2022 Health Programme Improved MNCH and MDR-TB Services across 9 Facilities in Southern Somalia

2022-04 Health Programme Improved MNCH and MDR-TB Services across 9 Facilities in Southern Somalia

1. BACKGROUND

Muslim Aid supports 9 static health facilities (8 MNCHs and 1 MDR-TB) since 2022, with the projects based in Banadir, Bakool, Lower Shebelle, Hiran, and Lower Juba regions to provide quality primary health care services targeting women and children. Services planned include treatment of common morbidities, first aid and treatment of minor injuries, treatment of communicable diseases, maternal and new-born health (including comprehensive ANC, skilled delivery services, management of intrapartum complications for mothers and new-born, timely PNC), child health services (including the treatment of malaria, pneumonia, and diarrhea), health promotion and education, and immunization services. These facilities are further supported to monitor, track, and respond adequately to the trends in morbidities and potential outbreaks of communicable diseases such as measles and AWD caused by the protracted drought using a real-time health management information system (HMIS).

MA supports the health facilities with equipment, pharmaceuticals, medical supplies, staff, and infrastructure to be able to provide preventive, diagnostic, treatment, and referral services to ensure a comprehensive package is available to all patients. In addition, MA continues to ensure global standards for infection prevention and control procedures at all service delivery sites. Data generated in health facilities are collected in MOH-approved HMIS forms, transmitted through the district, regional to the national level where MA supports the analysis and use for decision-making. All health services are provided free of charge with no discrimination by gender, age, sex, status, or ethnicity.

Health: As a result of the drought, population displacement, and overcrowding in IDP camps, outbreaks of infectious diseases are common. MA’s response for drought is building on the recently ended MNCH/MDR health response with detection, response to potential outbreaks and provision of health services to affected populations as described in the Humanitarian Response Plan (HRP). Through an integrated approach, MA supports equitable access to emergency and essential health care services. By integrating services, especially with nutrition and WASH, actions will prevent individuals slipping back into malnourished and fragile health conditions. With the ongoing drought, increase in AWD/cholera outbreaks and deterioration of the nutritional status of children younger than 5 years is frequent. A scale up for detection and case management services for AWD/cholera and those severely malnourished with complications, is being done to save lives. Measles vaccination coverage, estimated below 50% in rural villages, is not enough to prevent measles outbreaks. Susceptibility to communicable diseases such as these is compounded by poor nutrition, low immunization coverage, and limited vector control interventions (Partners SCI Baseline Report 2020).

MA recognizes the need to sustain and further consolidate the humanitarian health assistance for IDPs and underserved rural populations and supports the Maternal Newborn and Child Health (MNCH) facilities and Multi-Drug Resistant Tuberculosis (MDR-TB) hospital through the provision of incentive

salary payments of health workers and providing running costs for the MNCH facilities and MDR-TB hospital. Additionally, Muslim Aid provides capacity building and quality assurance through support supervision and monitoring of health facilities on quarterly bases. Additionally, the MNCH facilities are provided with a monthly essential drug for the treatment of common illness at the facility. The targeted facilities are listed below: –

1. PURPOSE

The purpose of the end line evaluation is to assess the overall performance, outcome and impact of the project with an aim of providing realistic and feasible recommendations for improving similar programs in Somalia context. The findings of the end line evaluation will be used as a post-test measurement of changes in the indicators attributable to the program interventions.

2a. SPECIFIC OBJECTIVES

The evaluation objectives include the following;

1. Evaluate to what extent the response has fulfilled its objectives to save lives and alleviate suffering through improved access to primary healthcare and the technical strength of the program, as the projects is a continuation of existing health project which is supporting 8 MNCHs and 1 MDR-TB hospital.
2. Undertake a comparative assessment on the progress achieved in delivering the program results and to identify key successes, gaps, and constraints that need to be addressed.
3. Assess Relevance, efficiency, effectiveness, sustainability, coherence and impact criteria used; and also the protection component of the intervention
4. Assess the impact of the actions on the targeted population and document the lessons learnt, digital documentary of case studies in emergency health, education and drought projects and provide recommendations for future interventions.
5. EVALUATION CRITERIA

Following evaluation questions should be at least included in the evaluation criteria. These evaluation criteria are based on OECD DAC.

1. Relevance

• To what extent are the objectives of the programme being valid for country programme and target community?

• Are the activities and outputs of the programme consistent with the overall goal and the attainment of its objectives?

• Are the activities and outputs of the programme consistent with the intended impacts and effects of its Strategy?

• Was the design of the project the most appropriate and relevant to promote the strategy?

1. Effectiveness

• To what extent were the project objectives demonstrating progress on Core indicators?

• To what extent project achieved planned objectives and results?

• What were the major underlying factors (internal and external) influencing the achievement or non-achievement of the results within the one years of project implementation?

• To what extent technical input provided to the project team in leading the implementation of plans and proposed activities of the Programme?

1. Efficiency

• Assess the activities are cost-efficient within the programmes supported by project?

• Does the Project offer better value for money within the rural and urban context in terms of its impact?

• Identify the cost factors spent to deliver interventions in the urban and rural community and identify the reasons behind budget underspent

1. Impact

• What are the key short term and long term changes produced by project, positive or negative and what are the key factors behind these changes?

• What real difference has the project activities made for the direct, and indirect stakeholders of promoting HEALTH/interventions?

1. Sustainability

S1) to what extent did the benefits of the projects continue after the fund support is ceased?

S2) what were the major factors which influenced the achievement or non-achievement of sustainability of the project?

1. Coordination

• How have the project activities been coordinated within different stakeholders to achieve overall objective?

1. Scalability/Replicability

• Is there any likely ability of the project or its components to be scaled or replicated in other programme areas?

• Who are the main actors in the scale-up/replication of the approach in other urban and rural locations and how has the project engaged with them to date?

4.1 METHODOLOGY

The lead consultant with the other team members will develop the evaluation methodology in collaboration with Health team and Senior Programs Manager. The lead evaluator or team is expected to write an inception report following review of literature and reference documents using the guidance format provided by MA Somalia.

Such a methodology should define an appropriate sample size and specify mechanisms that will be adopted to avoid selection bias. The lead evaluator will be expected to utilize various data collection tools like questionnaires, key informant interviews guides, observation to obtain data from the various respondents and key stakeholders. As a minimum, the evaluation process will include the following key steps:

1. Gender balance in respondents or sample size, with a margin of 10% flexibility.

1. Review of relevant literature related to the project (c.f. list of reference materials) and drafting an inception report;( proposal, quarterly reports, monitoring reports, budgets among others)

1. Data collection to apply-appropriate data collection tools (e.g. questionnaire, checklist etc.) for Interviews and focus group discussion;

1. Data analysis and Evaluation Report writing; and
2. Presentation of key evaluation findings;
3. Program Indicators to be evaluated

The program intends to assess the following principal indicators from Project;

1. Percentage of pregnant women who have attended at least two comprehensive antenatal clinics
2. Percentage of newborns that received postnatal care within three days after delivery
3. Percentage of births assisted by a skilled attendant at birth
4. Percentage of pregnant women in their third trimester who received a clean delivery kit
5. Number and percentage of community members who can recall target health education messages
6. Number of people admitted, rates of recovery, default, death, relapse, and average length of stay for people admitted to Management of Acute Malnutrition site
7. Proportion of infants 0-5 months of age who are fed exclusively with breast milk
8. Percentage of targeted population whose health status improved as a result of the intervention
9. Percentage of people adopting safe health behaviour, attitude, and practices after participating on trainings and awareness sessions
10. Number of established activities or practices to ensure inclusion of PWD participation during the project cycle
11. Number of established activities or practices to ensure inclusive of women participation during the project cycle
12. Percentage of individuals in targeted areas with a positive perception regarding the intervention

4.2. REFERENCE MATERIAL

• Project Technical Proposal(year) 2022
• Monitoring and Evaluation Plan for (year) 2022
• Performance indicators – log frame
• Quarterly reports
• Monitoring reports