Evaluation of Child Friendly Spaces: Tools and guidance for monitoring and evaluating CFS

35 Pages·2015·0.7 MB·English

Evaluation of Child
Friendly Spaces
Tools and guidance for monitoring
and evaluating CFS
Contents
1. Introduction 2
2. Setting up a good-quality monitoring system for CFS 3
2.1 Defining the outputs that we need to monitor 3
2.2 Tips for successful monitoring of outputs 3
2.3 Tools to measure outputs 4
2.3.1 Registration record 4
2.3.2 Attendance record 5
2.3.3 Activity record 7
2.3.4 Referral tracking system 8
2.3.5 CFS quality standards checklist 9
3. Designing an impact evaluation of CFS 11
3.1 Defining the outcomes you want to measure 11
3.2 Defining the outcomes to measure: Who? What? How? activity 12
3.3 Selecting the tools to measure outcomes 13
3.4 Designing an impact evaluation framework: Planning template 15
3.5 Understanding more about the ‘How?’: Survey design in brief 16
3.6 Adapting existing tools to measure outcomes 17
4. Data collection 19
4.1 Selection, training and supervision 19
4.2 Tips for successful data collection and evaluation 20
5. Data analysis 21
6. Sample tools for evaluation of CFS outcomes 23
6.1 Decision-making guide for the selection of measures related
to mental health and psychosocial well-being outcomes 23
6.2 Emergency Developmental Assets Profile (EmDAP) 25
6.3 Middle East Psychosocial Questionnaire 25
6.4 Arab Youth Mental Health Scale 26
6.5 Psychosocial Well-being (Uganda/DRC) 26
6.6 Strengths and Difficulties Questionnaire (SDQ) 27
6.7 Child Protection Rapid Assessment (CPRA) 27
6.8 Participatory Ranking Methodology (PRM) 28
6.9 Adapted Functional Literacy Assessment Tool (FLAT) 29
7. References 30
© World Vision International 2015
All rights reserved. Produced by Humanitarian & Emergency Affairs (HEA)
on behalf of World Vision International.
Managed on behalf of HEA by: Kevin Savage, Marisa Vojta, Franz Böttcher.
Authors: Sarah Lilley, Janna Metzler, Alastair Ager. Copyediting: Joan Laflamme.
Design: Broadley Design
1
1. Introduction
Child friendly spaces (CFS) are safe spaces where This document draws on the learning and experience
communities create nurturing environments in which of the project to provide practical guidance to child
children can access free and structured play and protection and MHPSS practitioners for monitoring and
learning activities. CFS, also commonly referred to evaluating child friendly spaces. It presents tools for
as Child Centred Spaces or Safe Spaces for Children, planning and implementing monitoring and evaluation
may provide educational and psychosocial support of CFS. For each tool, the objectives are explained,
and other activities that restore a sense of normality along with insights and lessons on the usefulness of
and continuity for children whose lives have been the tool based on the learning and experiences of the
disrupted by war, natural disaster or other emergency evaluation teams.
situations. They are generally designed and operated
There are two sections:
in a participatory manner, often using existing physical
1. Setting up a good-quality monitoring system for
spaces and seeking to connect to local community
resources and activities. They may serve a specific age CFS: This section presents tools that should be used
group of children or a variety of age ranges. on a regular basis to monitor the quality of CFS
implementation.
Even though they are one of the most widely used
interventions in emergencies for child protection 2. Designing an impact evaluation of CFS: This
and psychosocial support, little evidence documents section describes the methodology and process
their outcomes and impacts. There is widespread that were used to conduct the multi-country impact
commitment among humanitarian agencies to evaluation, and shares tools, practical tips and
strengthen the evidence base of programming. learning from the project.
Recognizing this, the Child Protection Working Group
For further support on using the tools included here,
(CPWG) of the Global Protection Cluster and the
please contact the CFS Task Force within the Child
Inter-Agency Standing Committee (IASC) Reference
Protection Working Group.4
Group on Mental Health and Psychosocial Support in
Emergency Settings have identified research in this area
as a high priority.
In response to the commitment to strengthen the
evidence base for humanitarian practice and the
prioritisation of CFS as a key area for research, World
Vision and Columbia University, working with Save
the Children, UNICEF and others, engaged in a
collaborative project to document the outcomes and
impacts of CFS and develop capacity for rigorous
evaluation. A series of impact evaluations were carried
out over three years in multiple countries. The findings
were published in a research report.
1 Definition of CFS taken from Minimum Standards of Child 3 Metzler, Kaijuka, Vojta, Savage, Yamano, Schafer, Yu, Ebulu and
Protection in Humanitarian Action. Child Protection Working Ager, (2013); Metzler, Savage, Vojta, Yamano, Schafer and Ager,
Group (CPWG), (2012). (2013); Metzler, Atrooshi, Khudeda, Ali and Ager, (2014); Lilley,
2 Ager, Metzler, Vojta, and Savage, (2013). Atrooshi, Metzler and Ager, (2014); Metzler, Ishaq, Hermosilla,
Mumba and Ager, (2015); Eyber, Bermudez, Vojta, Savage and
Bengehya, (2014).
4 For more information: http://cpwg.net/what-we-do/taskforces/
2
2. Setting up a good-quality monitoring system for CFS
Regular monitoring of CFS is important to ensure that 2.2 Tips for successful
implementation is on track and to make real-time monitoring of outputs
adjustments to improve the quality of activities. A CFS
monitoring system should be simple and practical Simple data management system
so that it can be implemented in even the most
Regular monitoring of the outputs mentioned above
challenging of emergency contexts. It is also important
will generate a lot of information that needs to be
that it meets the requirements specified in general
recorded and safely stored. It is therefore essential
guidance provided by your agency with respect to
to have a simple and organised system for storing,
monitoring and evaluation. These are generally focused
analysing and using the data. Typically, we use paper-
not only on maintaining consistency in procedures but
based forms to collect the monitoring information in
also on ensuring that monitoring activity actively feeds
the field. To ensure that the data are not lost, the paper
into programming.
forms should be carefully stored in a lockable filing
2.1 Defining the outputs that cabinet. It is also essential to set up a simple Excel-
based system (or equivalent software) to record the
we need to monitor
data in electronic form so that it can be easily accessed
A basic monitoring system for CFS should include at any time and can be for used future monitoring and
tools and processes for regularly tracking the following evaluation needs.
outputs (immediate results):
Standardised tools across agencies
• Registration – the number of children registered
Many agencies implement similar CFS models in
by the CFS.
humanitarian contexts. To ensure that information
• Attendance – the number of children attending the can be aggregated and analysed across agencies
CFS each day. Disaggregated information should be it is useful to use standardised tools for monitoring
collected on attendance, listing sex and age of the where possible. Standardised tools for monitoring CFS
children at a minimum. If feasible, data can also be attendance and quality, for example, can be developed
disaggregated by disability status. at the onset of the emergency through the child
• Activities – the type of activities conducted in the protection cluster or equivalent coordination group.
CFS each day. This should be captured in a timetable
that is updated as the activity schedule changes.
• Referrals – the number of children who attend the
CFS who are referred to other services. As well as
providing a direct service to children, a CFS can
be used as an entry point to assess the needs of
children and identify particularly vulnerable children
who require additional support services. Therefore in
some cases, children attending a CFS will be referred
to receive other, additional support. A monitoring
system should be in place to track the referral and
follow-up process.
• CFS quality standards – the extent to which
a CFS is meeting minimum quality standards.
Regular monitoring of the quality of CFS should be
undertaken so as to enable rapid changes to be
made where necessary to maintain or improve quality
to ensure the best service for children.
3
2.3 Tools to measure outputs 2.3.1 Registration record Comments on the usefulness of the toolKey stepsKey objectives • • It is an essential tool to inform implementation Register all children attending the CFS at first To track number of children enrolled in the CFS of CFS.arrival to programme.and their characteristics (sex, age, disability status) • • Enter the registration data into information To support appropriate targeting of CFS activities management system (below). Time needed: Approximately 10 minutes for each • Registration records can be kept and used for child at time of enrolment programme reports, attendance verification Frequency: Once per childand tracking referrals of children. Example of a registration database in Excel Confidentiality Statement Review the following guidance with lawyer in country to ensure that it is appropriate. The data on this form will be used by staff at the child friendly space to support the implementation of the programme. Data to support ongoing research and development of children’s programmes may be collected here, but no identifying information (such as the caregiver name, the child’s name, phone number or place of residence) will be given to outside organisations without the caregiver’s consent. UNHCR Reg # Adults at home Medical Special Number of other Registration Child Family SexAgeDate of Current Place Caregiver Phone No. Date of Arrival Date of Arrival (if applicable)(e.g. father, aunt, ConditionsNeedschildren in family No.NameNameregistrationof Residence(if comfortable)to (country)to (camp/CFS grandmother)home (under 18)location) A0001 A0002 A0003 A0004 A0005 A0006 A0007 A0008 A0009 A00010 A00011
4
n
Comments on the usefulness of the tool It is an essential tool to inform implementation of CFS. Strong attendance is likely indicative that childre and caregivers value the activities offered. Attendance of children is, of course, voluntary, but irregular attendance of a child may indicate circumstances with which agency staff can assist. Regular and accurate completion of the attendance record is also a potentially valuable indicator of programme quality.
s
u
ey steps Register all children attending the CFS each session using a daily attendance sheet. At the end of the day, enter the attendance data into a database. On a weekly or twice-weekly basis, review attendance and look for patterns of attendance by sex, age group and disability status (if feasible to collect this data). Use analysis of attendance data to target hard-to- reach groups and tailor activities appropriate to characteristics of those attending. AgeDisability stat
K • • • •
x
s e
e et S
ti e
vi h
2.3.2 Attendance record Key objectives • To track how many children attend the CFS each day and their characteristics (sex, age, disability status) • To support appropriate targeting of CFS acti Time needed: Approximately 15 minutes at start of each CFS session Frequency: Daily Example of a paper-based daily attendance s Name of CFS Date Session name/number Name of staff member completing form Registration numberName of child
5
A1 Total Days Attended in January0 0 0 A1 Total Days Attended in February 0 0 A1 Total Days Attended in March 0 0
30 M 31 T 30 Sa 0 0
29 S 30 W 29 F 0 0
28 Sa 0 0 29 T 28 T
27 F 0 0 28 M 27 W
26 T 27 S 26 T
25 W 26 Sa 0 0 25 M
24 T 25 F 0 0 24 S
23 M 24 T 23 Sa 0 0
22 S 23 w 22 F 0 0
21 Sa 0 0 22 T 21 T
20 F 0 0 21 M 20 w
19 T 20 S 19 T
18 W 19 Sa 0 0 18 M
17 T 18 F 0 0 17 S
16 M 17 T 16 Sa 0 0
15 S 16 w 15 F 0 0
14 Sa 0 0 15 T 14 T
13 F 0 0 14 M 13 w
12 T 13 S 12 T
11 W 12 Sa 0 0 11 M
10 T 11 F 0 0 10 S
0
9 M 1 T 9 Sa 0 0
8 S 9 w 8 F 0 0
7 Sa 0 0 8 T 7 T
6 F 0 0 7 M 6 w
5 T 6 S 5 T
4 W 5 Sa 0 0 4 M
el
xc 3 T 4 F 0 0 3 S
n E me) 15 2 M me) 3 T me) 15 2 Sa
ase i S Na Jan- 1 S S Na 2 w S Na Mar- 1 F
nce datab m - (Insert CF SexAge 00 00 m - (Insert CF Feb-14 1 SMT 00 00 m - (Insert CF SexAge 00 00
da or or ge or
n F d F A 0 0 F d
ple of an atte ndance Tracking Name of the chil ndance Tracking Name of Sexthe child 0 0 ndance Tracking Name of the chil
am Atte Reg. No Atte Reg. No Atte Reg. No
x
E
6
e.
et n training Children gested her
Youth Registration/meet and gre Service learning Snack Life skills/conflict resolutio Lunch Reading circle Exam preparation Outside time/ Sports Parent/guardian pick up s: A Handbook for Save the y vary widely from those sug
ea
School-age Registration/meet and greet Psychosocial games and activities Snack Puzzle, games, centers Lunch Reading circle Exam preparation Outside time/ Sports Parent/guardian pick up Child Friendly Spaces in Emergencie Children. The focus of activities m
d friendly spaces activity plan6 Pre-School Registration/meet and greet Psychosocial games and activities Snack Puzzle, games, centers Lunch Story time Arts and Crafts Outside time/ parachute game Parent/guardian pick up 6 at reflect Example taken from Staff, (2008). Save th
xample of chil Time 09:00-09:15 09:15-10:30 10:30-10:45 10:45-12:00 12:00-12:30 12:30-13:00 13:00-15:00 15:00-16:00 16:00 ultural activities th
E c
d
n
a
2.3.3 Activity record Key objectives • that are To record and track the activities5 implemented in the CFS • To inform changes to the CFS activity schedule Time needed: As the CFS is being set up it may take a few days to establish the activity plan, with changes being made as the CFS is developed. Frequency: The activity record should be updated as regularly as the schedule changes. Copies of old schedules should be stored for monitoring and reviewing changes and to assist in the compilation of project reports. Key steps • Prepare an activity schedule, disaggregated by age group, as shown in the example below. • Update the schedule and keep copies of old schedules on file for review. 5 Activities should include a range of participative, recreational the agreed goals of the CFS programme.
7
2.3.4 Referral tracking system
Key objectives Frequency: The referral tracking form should be
completed every time a referral of a child is made.
• To record the number of children who attend
the CFS who are referred to other services Key steps
• To record information on the type of services • Immediately after a meeting or decision to refer
to which referrals are being made a child to another service, complete the referral
• To record information on follow-up activities form (example below).
carried out by CFS staff • Keep copies of referral forms on file as part of case
Time needed: 15–30 minutes to complete form management system and for periodic review.
(depending on nature of referral and level of detail • On a bi-weekly or monthly basis, conduct a periodic
required) monitoring review of the number of referrals made,
which services the referrals are to and status
of follow-up actions.
Example of a referral form
Referral No.
CFS REFERRAL FORM FOR SERVICES
Date: Time: am/pm
Child Name: Age:
CFS Facilitator:
CFS Location ID:
CFS001 CFS005 CFS009
CFS002 CFS006 CFS0010
CFS003 CFS007 CFS0011
CFS004 CFS008 CFS0012
Reason for Referral (summary)
Action taken by CFS Facilitator
Contacted caregiver/parent on
Held meeting with caregiver, child and suprvisor
Referred to (service provider name)
Submitted to National/Regional Child Protection Advisor
Planned follow-up
CFS Facilitator signature: Date:
8
2.3.5 CFS Quality standards checklist
Key objectives Comments on the usefulness of the tool
• To assess the extent to which the CFS is meeting The Quality Standards for Children’s Activities and CFS
quality standards Programmes Assessment – specified within the World
• To support improvement of quality of the CFS Vision Children in Emergencies Manual7 – provide
quality standards for 17 areas of CFS programming
Time needed: Approximately 30 minutes
including (but not limited to) awareness of protection
Frequency: Monthly issues, activities content, playground equipment,
record keeping and planning, and visitor information.
Key steps
This assessment together with Save the Children’s CFS
• Visit each CFS and, where available, interview Handbook8 and UNICEF’s CFS Guidelines9 were the
the lead facilitator or animator. basis for the CFS Quality Standards Checklist used in
• Look around the CFS and observe activities. the multi-country evaluation. The checklist is shown
below and comprises 13 items to guide observational
• On the basis of talking to facilitator and observing
assessment during site visits. In programme monitoring,
activities and CFS environment, code the following
the checklist should be used in coordination with the
quality standards with the appropriate letter:
full-length quality assessment to gauge the quality of
• Y (Yes) = CFS meets the quality standard
programming accurately. The checklist itself focuses
• P (Partly) = CFS is close to meeting the quality
only on directly observable indicators; however, these
standard
are potentially useful ‘proxies’ for some of the broader
• N (No) = CFS does not meet quality standard quality standards that are less easily measurable.
• * = Not enough information available to make
determination.
CFS Quality standards checklist
Name of CFS:
Date Assessed:
Quality Standards Yes/Partly/No Action Points
1. Children’s work displayed in space
2. Equipment in good condition (e.g. toys not broken)
3. No toy guns or military-type toys
4. Code of conduct displayed in picture or word form
5. Activities available two hours per day, three days per week
6. One person responsible for daily inspection of equipment and keeps record, register
7. Supervisor-to-child ratio is adequate
8. Record kept of all visitors
9. Drinking water available (and staff knowledge of proper hygiene)
10. First aid kit available (and stocked appropriately)
11. Attendance records kept (and are up to date/current)
12. Emergency protocol clearly outlined and documents (i.e. in word or picture form)
13. Activity schedule prepared in advance of use
Total Yes:
Total Partly:
Total No:
7 World Vision International, (2006). 9 Global Protection Cluster, Global Education Cluster, INEE
8 Save the Children, (2008). and IASC, (2011).
9