Save the Childreno Mwayi wa Moyo ("A Chance to Live")

244 Pages·2016·15.41 MB·English

Save the Childreno
Mwayi wa Moyo ("A Chance to Live")
Blantyre District, Malawi
Final Evaluation Report
Cooperative Agreement: Al D-OAA-A- 11-00058
Project Dates: 1 October 2011 - 31 March 2016
Category: Innovation
Submitted by:
Save the Children Federation, Inc.
501 Kings Highway East, Suite 400, Fairfield, CT 06825
Telephone: [phone] - Fax: [phone]
Authors:
This publication was produced at the request of the United States Agency for International
Development and prepared independently by: John Murray, External Consultant and Final
Evaluation Team Leader; Karen Z. Waltensperger, Senior Advisor, Community
and Child Health, Save the Children; Steve Macheso, Project Manager,
Mwayi wa Moyo, Save the Children Malawi; Sharon Lake-Post,
Editorial Consultant, Save the Children.
Contact Persons:
Eric Swedberg, Senior Director, Child Health
Carmen Weder, Associate Director, Department of Global Health
Submitted to USAID/GH/HIDN/CSHGP
August 15, 2016
AID
FROM THE AMERICAN PEOPLE
This report is made possible by the generous support of the American people through the United
States Agency for International Development (USAID). The contents are the responsibility of Save the
Children and do not necessarily reflect the views of USAID or the United States Government
Table of Contents
Page
Acronyms
Executive Summary
Evaluation Purpose and Evaluation Questions 4
A. Project Background 4
B. Evaluation Methods and Limitations 10
C. Findings, Conclusions and Recommendation Highlights
D. Conclusions and Recommendations 25
Annexes
Annex I: List of Publications and Presentations Related to the Project..... ....... 27
Annex 2: Project Management Evaluation 28
Annex 3: Mwayi wa Moyo 4.5-Year Work Plan Table 3 I
Annex 4: Rapid CATCH Indicator Table 38
Annex 5: Final KPC Report 40
Annex 6: Community Health Worker Training Matrix 99
Annex 7: Evaluation Scope of Work 106
Annex 8: Evaluation Methods and Limitations 117
Annex 9: Data Collection Instruments 120
Annex 10: Information Sources 128
Annex I I: Disclosure of Any Conflicts of Interest 134
Annex 12: Statement of Differences 150
Annex 13: Evaluation Team Members, Roles and Titles 151
Annex 14: Operations Research Final Report 154
Annex 15: Stakeholder Debrief PowerPoint Presentation 199
Annex 16: Project Data Form 215
Annex 17: Presentation: Mwayi wa Mayo: Streamlining and Integrating the
Community Package 222
Annex 18: Mwayi wa Moyo Project Indicator M&E Table 230
Annex 19: Community Mobilization Highlights 235
Mwayi wa Moyo (CS-27) Malawi, Final Evaluation
Save the Children, August 15, 2016
Acronyms
ACSD Accelerated Child Survival and Development
ACT Artemisinin Combination Therapy
ADCs Area Development Committees
AEHO Assistant Environmental Health Officer
ANC Antenatal Care
BLM Banjo la Mtsogolo- a local family planning NGO
C-EHP Community Essential Health Package
CAC Community Action Cycle
CAG Community Action Group
CBD Community-Based Distributor
CBMNC Community-Based Maternal and Newborn Care (national package of
interventions delivered by HSAs)
CBMNH Community-Based Maternal and Newborn Health
CBO Community-Based Organization
CDD Control of Diarrheal Diseases
CHAM Churches Health Association of Malawi
CIDA Canadian International Development Agency
CM Community Mobilization
COM College of Medicine
CSHGP Child Survival and Child Health Program
CYP Couples Years of Protection
DAPP Development Aid from People to People
DEHO District Environmental Health Officer
DHMT District Health Management Team
DHO District Health Office(r)
DIP Detailed Implementation Plan
st rd
DPT I /DPT3 Diphtheria/Pertussis ImmunizationI Li Dose
EHO Environmental Health Officer
EHP Essential Health Package
Mwayi wa Moyo (CS-27) Malawi, Final Evaluation iii
Save the Children, August 15, 2016
ENC Essential Newborn Care
FANC Focused Antenatal Care
FE Final Evaluation
FP Family Planning
GVH Group Village Headman/Headmen
HBB Helping Babies Breathe
HC Health Center
HF Health Facility
HIV Human lmmuno-deficiency Virus
HSA Health Surveillance Assistant
HVV Health Worker
iCCM Integrated Community Case Management
IMNC Integrated Maternal and Newborn Care (curriculum for facility-based health
workers)
IMNCI Integrated Management of Newborn and Childhood Illnesses
IR Intermediate Result
KPC Knowledge, Practices, Coverage
LA Lumefantrine and Artemether (ACT used in Malawi)
LAM Lactational Amenorrhea Method
LBW Low Birthweight
M&E Monitoring and Evaluation
MCHIP Maternal and Child Health Integrated Program
MICS Maternal Infant Child Survival Project, Save the Children Malawi project
MN Maternal and Newborn
MNCH Maternal Newborn and Child Health
MoH Ministry of Health
MOI Missed Opportunity Index
MUAC Mid-upper Arm Circumference
NEP National Evaluation Platform
NGO Nongovernmental Organization
NSO National Statistics Office
Mwayi wa Moyo (CS-27) Malawi, Final Evaluation iv
Save the Children, August 15, 2016
OR Operations Research
ORS Oral Rehydration Sachets/Salts
ORT Oral Rehydration Therapy
PCM Pneumonia Case Management
PHCU Primary Health Care Unit
PMTCT Prevention of Mother-to-Child Transmissions
PNC Postnatal Care
PPFP Postpartum Family Planning
PSI Population Services International
PTM Prevention and Treatment of Malaria
RHD Reproductive Health Directorate
QECH Queen Elizabeth Central Hospital
SBA Skilled Birth Attendant
SBCC Socio-Behavioral Change Communication
SDA Seventh Day Adventist
SHSA Senior Health Surveillance Assistant
SRH Sexual and Reproductive Health
TA Traditional Authority
TB Tuberculosis
TBA Traditional Birth Attendant
TT2+ Tetanus Toxoid (at least 2 doses)
TWG Technical Working Group
USAID United States Agency for International Development
VC Village Clinic
VDC Village Development Committee
VHC Village Health Committee
WHO World Health Organization
Mwayi wa Moyo (CS-27) Malawi, Final Evaluation
Save the Children, August 15, 2016
EXECUTIVE SUMMARY: Mwayi wa Moyo Integrated Family Planning, Maternal,
Neonatal and Child Health Project in Malawi: Final Evaluation Report
Evaluation purpose
The purpose of the final evaluation (FE) was to determine whether the Mwayi wa Mayo
Integrated Family Planning, Maternal, Newborn and Child Health Project increased use of
evidence-based, life-saving interventions by women, caregivers and children in the Blantyre
District of Malawi. The evaluation was conducted between February 15 and 24, 2016.
Evaluation questions
The FE drew upon existing data collected or compiled during the project cycle and additional
data collected during the evaluation for the following purposes: I) To provide an overview of
project goals, objectives, and key intervention strategies implemented; 2) To determine the
extent to which the project accomplished the results outlined in the Detailed Implementation
Plan (DIP) and to present evidence of these accomplishments; 3) To describe key factors that
contributed to what worked or did not work regarding some or all aspects of the program,
with a focus on the integrated approach to programming; 4) To determine which elements of
the integrated community-based (family planning (FP), integrated community case management
(iCCM), maternal, newborn and child health (MNCH) and community mobilization (CM)
approaches used in Blantyre District are likely to be sustained or expanded (through
institutionalization or policies); 5) To determine whether the operations research (OR) design
was adequate to answer the key questions; and whether OR findings influenced policy, practice
or capacity development; 6) To describe how the health surveillance assistant (HSA) workforce
and Community Essential Health Package (C-EHP) content issues affected HSA performance;
and 7) To describe project contributions to improving the effectiveness and sustainability of CM
in the current context of Malawi; and to identify strategies that should be taken forward.
Evaluation methods
Seven principal methods were used for the evaluation: I) Review of knowledge, practice and
coverage (KPC) household surveys conducted at project baseline and endline; 2) Review of OR
data on integrated training, supervision and service delivery; 3) Review of routine data from
Village Clinic (VC) registers; 4) Document review — including policy documents, program
reports, technical reports, reports of training activities, health facility (HF) registers, and
training and health education materials; 5) Interviews with district staff and managers, and a
review meeting with the District Health Office (DHO), District Environmental Health Officer
(DEHO), Coordinators, Senior Health Surveillance Assistants (SHSAs), HSAs, members of
community groups — catchments of two facilities; 6) Field visits — site visits were made to eight
randomly selected health centers (HC) and in-depth interviews conducted with staff and
community members; and 7) A final review and dissemination meeting with district and national
stakeholders.
Project background
Mwayi wa Mayo was a five-year Innovation Project (CS-27 cycle) running between I October
2011-31 March 2016. The project was funded by USAID's Child Survival and Health Grant
Program (CSHGP), with matching funding from Save the Children, Towers Watson, and the
Pfizer Foundation. The project targeted hard-to-reach communities in Blantyre District with
Mwayi wa Moyo (CS-27) Malawi, Final Evaluation 1
Save the Children, August 15, 2016
limited access to health care services. Malaria, pneumonia, diarrhea and under-nutrition are the
primary contributors to morbidity and mortality of children 1-59 months old globally and in
Malawi, with newborns dying of asphyxia, prematurity/low birthweight (LBW) and sepsis. The
project's strategic objective was increased use of key MNCH + Postpartum Family Planning (PPFP)
services and practices. All project activities were implemented in close collaboration with the
Blantyre District Health Management Team (DHMT) using routine district systems. The project
had four main components: 1) Increased access to and availability of high impact MNCH and FP
interventions; 2) Improved quality of high impact MNCH and FP interventions; 3) Increased demand for
MNCH and FP care services and healthy practices in the home and community; and 4) An enabled
environment at all levels to support effective delivery of MNCH and FP interventions. The project
innovation was the delivery of high impact interventions using an integrated approach to
training, supervision and clinical mentoring; and the project's OR component aimed to test
whether the integrated approach reduced missed opportunities and affected quality of HSA
case management.
The principal conclusions of the FE are:
I. Improved coverage of several high impact interventions along the lifecycle of women, pregnant
women and children is noted in project areas, although not all targets were met. Data show
that more women use modern methods of contraception, seek antenatal care (ANC) and
deliver with a skilled birth attendant (SBA); and that sick children with fever are more likely to
be treated early, and those with suspected pneumonia are more likely to be taken for care.
Exclusive breastfeeding shows improvement, and coverage with key preventive interventions
also show significant improvements during the project period.
2. Some improvement was seen in management of diarrhea. Declines in sick children receiving
increased fluids and continued feeding were noted, and zinc was introduced by the project; but
there were no significant changes in management of diarrhea with oral rehydration therapy
(ORT).
3. HSAs in project areas are able to provide integrated community case-management for sick
children at reasonable standards for most interventions — referral remains a challenge.
Observation-based data on HSA case-management practices show that HSAs perform 70-80
percent of case management tasks correctly, scoring highly on classification and home
treatment tasks.
4. Integrated training, supervision and service delivery resulted in limited improvements in quality
of care by HSAs. Those HSAs using integrated approaches had fewer missed opportunities to
provide services to women and children, although missed opportunities remained very high and
benefits largely disappeared by 12 months. No significant difference was noted in quality of
clinical case-management by HSAs between areas using integrated and vertical approaches.
5. More attention to quality of care provided by facility-based health workers (HWs) is needed.
No data were available about the quality of delivery care or care of sick newborns and sick
children provided at first level and referral HFs. Population-based data suggest that early
breastfeeding practices at the time of delivery and diarrhea management need improvement.
6. Integrated training, supervision and service delivery have a number of benefits. A number of
benefits to integrated approaches to training, supervision and service delivery were noted
including: a reduction by two days in total training days; improved efficiency of supervision and
follow-up (reducing the number and costs of visits); and high demand for FP interventions. For
these reasons, continuing integrated approaches may be warranted.
Mwayi wa Moyo (CS-27) Malawi, Final Evaluation 2
Save the Children, August 15, 2016
7. Implementation through the routine district system has strengthened capacity. District
engagement has strengthened planning, training skills support for supervision, and management
using data.
8. HSA coverage and deployment is an important problem that will limit program effectiveness in
the long term — and needs urgent review. Inadequate numbers of HSAs are available in project
areas; and their and effectiveness is limited because many do not reside in their assigned, hard-
to-reach areas and have to spend two to three days a week working at fixed-site HCs.
9. Strengthening availability of essential medicines is a country-wide challenge and needs continued
attention. The project used match funds to provide essential HSA medicines during the project
period to cover district shortfalls. Continued attention at national, district and HC levels is
needed to ensure that HSAs have adequate supplies of medicines.
I 0. Limitations to sustainability remain and will require long-term support. Sustainability will be
limited by lack of district capacity (human and financial) to cover recurrent activities previously
supported by the project such as: supply of essential medicines and supplies; regular
supervision; planning meetings; printing of data registers; and data management.
The principal recommendations of the FE are:
I. Provide continued support to the district to strengthen DHMT capacity for managing and
overseeing iCCM, Community-Based Maternal and Newborn Care (CBMNC) and FP activities
— in collaboration with local partners (medicine supply, regular supervision, monitoring HSA
coverage and re-training, collection and use of data). (Responsible: Save the Children local and
national, DHMT, district development partners).
2. Continue to integrate approaches to training, supervision and service delivery for women,
mothers and children — with a focus on ensuring that FP activities are integrated into all
approaches. (Responsible: DHMT, Save the Children Malawi).
3. Write-up and disseminate findings
• Complete analysis of OR findings and publish or disseminate results. (Responsible: College
of Medicine [COM], Save the Children).
• Document project findings, approaches, methods and materials and OR results - ensure that
findings are shared with the Ministry of Health (MoH) and other stakeholders including
provinces and districts. (Responsible: Save the Children, DHMT).
• Use findings to inform continuation of programming in Blantyre under a new multidistrict
initiative and development of other community-based MNCH initiatives in Malawi.
(Responsible: Save the Children, nationally and globally).
4. Use field experience to inform the national rollout of iCCM and implementation of the
CBMNC package — through national technical working groups, emphasizing use of integrated
approaches to community programming. (Responsible: Save the Children)
5. The recent MoH decision to adopt the WHO Care for Newborns and Children in the Community
manuals for newborn, sick child, and well child means that the Mwayi wa Mayo integrated
materials will not be taken forward.
Mwayi wa Moyo (CS-27) Malawi, Final Evaluation 3
Save the Children, August 15, 2016
EVALUATION PURPOSE AND EVALUATION QUESTIONS
Evaluation Purpose
The purpose of the FE was to determine whether the Mwayi wa Moyo (Integrated Family
Planning, Maternal, Newborn and Child Health Project) increased use of evidence-based, life-
saving interventions by women, caregivers and children in Blantyre District, Malawi. The aim of
the FE was to use data to identify effective, integrated, community-based approaches used by
the project and to document mechanisms by which these approaches worked; while also
identifying approaches that had been less successful. As a part of this process, the evaluation
aimed to identify the extent to which project activities strengthened the capacity and
sustainability of district MoH systems, used and documented innovative community-based
program approaches, and informed national programming. Evaluation findings are intended to
provide evidence-based recommendations to inform local and national planning in Malawi and in
other countries implementing community-based FP, MNCH programs.
Evaluation Questions
The FE drew upon existing data collected or compiled during the project cycle and additional
data collected during the evaluation for the following purposes:
1) To provide an overview of project goals, objectives, and key intervention strategies
implemented;
2) To determine the extent to which the project accomplished the results outlined in the DIP
and to present evidence of these accomplishments;
3) To describe key factors that contributed to what worked or did not work regarding some
or all aspects of the program, with a focus on the integrated approach to programming;
4) To determine which elements of the integrated community-based (FP, iCCM, CBMNC,
HIV/TB) and CM approaches used in Blantyre District are likely to be sustained or
expanded (through institutionalization or policies);
5) To determine whether the OR design was adequate to answer the key questions; and
whether OR findings influenced policy, practice or capacity development;
6) To describe how HSA workforce and C-EHP content issues affected HSA performance; and
7) To describe project contributions to improving the effectiveness and sustainability of CM in
the current context of Malawi; and to identify strategies that should be taken forward.
A. PROJECT BACKGROUND
A.I. Setting
Blantyre District is a rural district located in the Shire Highlands, in Malawi's Southern Region.
Much of the population of rural Blantyre District has limited access to health care services.
Malaria, pneumonia, diarrhea and under nutrition are the primary contributors to morbidity and
mortality of children 1-59 months old, with asphyxia, pre-maturity/LBW and sepsis the most
important causes of newborn deaths. A baseline, under-five mortality rate of 121/1000, and a
newborn mortality rate of 30/1000 live births, were estimated in the district'. At baseline, 44%
of mothers made four ANC visits during pregnancy, 85% of deliveries were supported by a
skilled attendant, 79% of newborns were put to the breast within an hour of birth, and 23% of
mothers reported a postnatal care (PNC) visit within two days of delivery. About half of the
Ministry of Health, UNICEF. MICS 2006 Report, March 2009
Mwayi wa Moyo (CS-27) Malawi, Final Evaluation 4
Save the Children, August 15, 2016
women (56%) reported using a modern method of contraception. Sixty-five percent of children
with suspected pneumonia were taken to an appropriate provider — 56% within 24 hours of
onset of illness; 19% of children with fever received an antimalarial within 24 hours of the onset
of fever; 65% of children with diarrhea received oral rehydration sachets/salts (ORS); and less
than half (44%) of children 0-23 months old slept under an insecticide-treated bednet. A
retrospective review of pediatric hospital records at the Queen Elizabeth Central Hospital
(QECH) in Blantyre from 1998 to 2008, showed that the burden of malaria during the six first
months of life may be substantial.' The limited health systems in the district's rural areas
presented unique challenges, making it suitable for a project that focused on strengthening the
existing community health worker network (HSAs) and building community-based approaches.
A.2. Goals and objectives
Goal
Under-5 mortality reduced
Strategic USE of key MNCH+PPFP services
Objective and practices increased
Intermediate Access to and
Demand for
Results availability of
interventions
high-impact
improved
interventions
• HSAs provide • HSAs and health • Integt • Enhanced
Illustrative integrated package of workers trained SBCC for community action
Strategies high-impact and delivering MNCH+PPFP for improved use of
community-based integrated implemented MNCH+PPFP
interventions for package through • Program learning
MNCH+PPFP • Reliable supply locally on integration in
(including CM/SBCC) of drugs and appropriate Blantyre transferred
• Workers in first-line equipment channels to other districts
health facilities
• Supervisors apply • HSAs • National policies
provide integrated
integrated clinical facilitate and strategies
package of high-
mentoring and community favoring integration
impact MNCH+PPFP
supervision mobilization of community-
(including HBB,
practices for for based
KMC, management of
HSAs MNCH+PPFP MNCH+PPFP
NNS)
A.3. Project location
Located in the Shire Highlands, in Malawi's Southern Region, Blantyre District is situated on the
eastern edge of the Great Rift Valley. The DHMT oversees health programming for the district,
which has 17 rural and semi-rural facilities, staffed by clinical officers, nurses, nurse-midwives,
medical assistants, environmental health officers (EHOs) and HSAs. The Churches Health
Association of Malawi (CHAM) operates two of the facilities under agreements with the MoH
and district. There is one referral hospital in Blantyre District (Queen Elizabeth Hospital).
2 Larru B, Molyneux E, Kuile FO, Taylor T, Molyneux M, Terlouw DJ Malaria in infants below six months of age:
retrospective surveillance of hospital admission records in Blantyre, Malawi, Malaria Journal 2009, 8:310
Mwayi wa Moyo (CS-27) Malawi, Final Evaluation 5
Save the Children, August 15, 2016