Global Climate Change Alliance: Pacific Small Island States Evaluation Report

262 Pages·2016·3.17 MB·English

by  Damien Sweeney

Global Climate Change Alliance:
Pacific Small Island States
Evaluation Report

PREPARED FOR

Pacific Community

23 May 2016
REPORT PREPARED BY
PACIFIC RESEARCH AND EVALUATION ASSOCIATES
www.prea.com.au
CORRESPONDENCE RELATED TO THIS REPORT SHOULD BE ADDRESSED TO:
NAME PHONE EMAIL
Martin Pritchard [phone] [email protected]
Damien Sweeney [phone] [email protected]

VERSION CONTROL
VERSION STATUS DATE
1.0 Draft- submitted to SPC 19/03/2015
1.1 Revisions from SPC- Dr. Gillian Cambers 28/03/2015
1.2 Second Draft – submitted to SPC and EU 01/04/2016
1.3 SPC feedback from GC, JU and EU from presentation 12/04/16
1.4 Third draft – circulated to countries, SPC and EU 14/04/16
1.5 Final version incorporating feedback from countries, 10/05/16
SPC and EU
1.6 New SPC logo. Integrate final feedback from SPC. 23/05/16

ACKNOWLEDGEMENTS
PREA acknowledges all the contributors to this report: the national coordinators and
country-level project teams that were interviewed, other interviewees, and particularly
the SPC GCCA: PSIS project team that gave their time to provide input and feedback
to the report. Specific thanks to Dr. Gillian Cambers for her valuable comments on draft
reports.

DISCLAIMER
Whilst care has been taken in the preparation of the material in this document to
ensure its accuracy, Pacific Research And Evaluation Associates and other contributors
do not warrant that the information contained in this document is error–free and, to
the extent permissible under law, it will not be liable for any claim by any party acting
on such information.

COVER IMAGE CREDITS from left to right: New rainwater tank installed in Palau, D. Sweeney; Sedi-tunnel
groynes in Tongatapu, M. Pritchard; Water quality monitoring at a community well in Kiribati, M. Pritchard.
Acronyms and Abbreviations
ACSE Adapting to Climate Change and Sustainable Energy project
ADB Asian Development Bank
AusAID Australian Agency for International Development
CBA Cost-Benefit Analysis
C-CAP Coastal Community Adaptation Project
CCCA Climate Change Coordination Adviser
CCCI Climate Change Cook Islands
CCCPIR Coping with Climate Change in the Pacific Islands Region
CCM Climate Change Matters
CePaCT Centre for Pacific Crops and Trees
CRGA Committee of Representatives of Governments and Administrations
CROP Council for Regional Organisations in the Pacific
DAC Development Assistance Committee
DPCC Development Partners for Climate Change
EDF European Development Fund
EHU Kiribati Environmental Health Unit
EIA Environmental Impact Assessment
EMP Environmental Management Plan
EPA Environmental Protection Agency
EQPB Palau Environmental Quality Protection Board
EU European Union
EUD EU Delegation
FFD First flush device
FNU Fiji National University
FSM Federated States of Micronesia
GCCA Global Climate Change Alliance
GCCA: PSIS Global Climate Change Alliance: Pacific Small Island States project
GCF Green Climate Fund
GIS Geographic Information System
GIZ Deutsche Gesellschaft für Internationale Zusammenarbeit
GSD Geo Sciences Division (SPC)
HDPE High-density polyethylene
IWRM Integrated Water Resources Management
JNAP Joint National Action Plan for Climate Change Adaptation and
Disaster Risk Management
KAP Kiribati Adaptation Program
KRA Key Result Area
LFA Logical Framework Approach
LRD Tuvalu Land Resource Division
M&E monitoring and evaluation
MFEM Ministry of Finance and Economic Management, Cook Islands
MMR Ministry of Marine Resources Cook Islands
MOPW Ministry of Public Works
MTR Mid-term review
NAPA National Adaptation Programmes of Action
NDPB National Development Bank of Palau
NGO Non-governmental organisation
NIE National Implementing Entity to Adaptation Fund
NISP Niue National Integrated Strategic Plan
NUC Nauru Utilities Corporation
NZAID New Zealand Agency for International Development
OECD Organisation for Economic Co-operation and Development
OERC Palau’s Office of Environmental Response and Coordination
PACC Pacific Adaptation to Climate Change project
PCCP Pacific Climate Change Portal
PCCR Pacific Climate Change Roundtable
PDD Project Design Document
PECF Pacific Environment Community Fund
PET polyethylene terephthalate
PET Polyethylene terephthalate
PHD Public Health Division
PICS Pacific Island countries
PIFACC Pacific Islands Framework for Action on Climate Change
PIFS Pacific Islands Forum Secretariat
PMU Project Management Unit
PPUC Palau Public Utilities Corporation
PREL Pacific Resources for Education and Learning
PREL Pacific Resources for Education and Learning
PWCI Palau Water Conservation Initiative
RFP Request for Proposal
RMI Republic of Marshall Islands
ROM Results Orientated Monitoring
RONAdapt Republic of Nauru Adaptation to Climate Change and Disaster Risk
Reduction Action Plan
SCMs Steering Committee Meetings
SCUBA Self-contained underwater breathing apparatus
SIDS Small Island Developing States
SODIS Solar disinfection of water
SOP Standard Operating Procedures
SPC Secretariat of the Pacific Community
SPREP Secretariat of the Pacific Regional Environment Programme
SRDP Strategy for Climate and Disaster Resilient Development in the
Pacific
SRIC-CC Strengthening Resilience of our Islands and Communities
SRIC-CC Strengthening the Resilience of our Islands and Our Communities to
Climate Change Project (Cook Islands)
STSISP South Tarawa Sanitation Improvement Sector Project
TOR terms of reference
TVET Technical Vocational Education and Training
UNDP United Nations Development Programme
UNICEF United Nations Children's Emergency Fund
USAID United States Agency for International Development
USP University of the South Pacific
WARD CROP-CEO Working Arm on Climate Change and Disaster Resilient
Development
WASH Water Supply, Sanitation and Hygiene
WHO World Health Organization
Table of Contents
1. Executive summary ................................................................................................................... 1
1.1 Best practices arising from the GCCA: PSIS project ........................................................... 6
1.2 Recommendations ............................................................................................................ 9
1.2.1 POST-PROJECT ............................................................................................................. 9
1.2.2 MOVING FORWARD ................................................................................................ 10
2. Introduction ............................................................................................................................. 13
2.1 Project purpose and objective .......................................................................................... 13
2.2 Purpose of the evaluation ................................................................................................. 13
2.3 Evaluation objectives ........................................................................................................ 13
2.4 Scope of the evaluation ................................................................................................... 14
2.5 Evaluation methodology ................................................................................................. 14
2.5.1 Limitations................................................................................................................... 16
3. Findings .................................................................................................................................... 17
3.1 RELEVANCE: To what extent have the objectives of the GCCA: PSIS project been
consistent with beneficiaries' requirements, country’ needs, global priorities and partners' EU's
policies, and SPC’s Climate Change Engagement Strategy? ........................................................ 17
3.1.1 Mutual reinforcement (coherence) .............................................................................. 20
3.1.2 EC value added ............................................................................................................ 22
3.2 EFFECTIVENESS: To what extent has the GCCA: PSIS project attained, or is expected to
attain, its specific objective(s)? .................................................................................................... 22
3.2.1 Factors that facilitated and hindered the achievement of project objectives ............... 29
3.3 IMPACT: To what extent has the GCCA: PSIS project supported the Governments of Cook
Islands, Kiribati, Marshall Islands, Federated States of Micronesia, Nauru, Niue, Palau, Tonga and
Tuvalu, in their efforts to tackle the adverse effects of climate change?....................................... 33
3.3.1 Country level impacts .................................................................................................. 34
3.4 SUSTAINABILITY: To what extent will the project benefits in the targeted sectors be
sustainable over the longer term (5 years)? ................................................................................. 36
3.5 EFFICIENCY: To what extent does the GCCA: PSIS project demonstrate timely
implementation, sound financial management and value for money? ........................................ 41
3.5.1 Work planning and narrative reporting ........................................................................ 41
3.5.2 Funding allocation ....................................................................................................... 42
3.5.3 Financial management ................................................................................................ 43
3.5.4 Funding expenditure .................................................................................................... 44
3.5.5 Efficiency of project approaches selected to achieve project purpose .......................... 44
3.5.6 Quality of outputs delivered. ....................................................................................... 45
3.5.7 Assets .......................................................................................................................... 46
3.5.8 Personnel ..................................................................................................................... 46
3.5.9 Risk Management ........................................................................................................ 47
3.5.10 Responding to risk ................................................................................................... 48
3.5.11 Timeliness ................................................................................................................ 48
3.6 CROSS-CUTTING: To what extent has the project demonstrated sensitivity to
environmental and gender issues? .............................................................................................. 50
3.6.1 Gender ......................................................................................................................... 50
3.6.2 Environment ............................................................................................................ 53
3.7 VISIBILITY: To what extent have project activities and results been made visible in both
the beneficiary countries and the European Union countries? ..................................................... 55
3.7.1 Organisation and regional level communications plans ............................................... 56
3.7.2 National level communications plans ........................................................................... 56
3.7.3 Customisation of communications tools ...................................................................... 58
4. Overall assessment and conclusions ........................................................................................ 61
4.1 Relevance / Coherence / EC-value added ......................................................................... 61
4.2 Effectiveness ................................................................................................................... 62
4.3 Impact ............................................................................................................................. 63
4.4 Sustainability ................................................................................................................... 64
4.5 Efficiency ......................................................................................................................... 64
4.6 Gender and environment ................................................................................................. 65
4.7 Visibility ........................................................................................................................... 66
5. Best practices .......................................................................................................................... 66
6. Recommendations .................................................................................................................. 69
6.1.1 POST-PROJECT ........................................................................................................... 69
6.1.2 MOVING FORWARD ................................................................................................ 70
7. ANNEX ..................................................................................................................................... 73
7.1 Annex 1. Terms of Reference ............................................................................................ 73
7.2 Annex 2.Independent evaluation of the ‘proposal preparation using the Logical
Framework Approach’ activities .................................................................................................. 79
7.3 Annex 3. Evaluation Methodology ................................................................................... 80
7.3.1 Evaluation Consultant CVs ........................................................................................... 98
7.4 Annex 4. Documents reviewed to inform the evaluation ............................................... 100
7.5 Annex 5. Stakeholders interviewed to inform the evaluation ......................................... 106
7.6 Annex 6. Country Evaluation Reports ............................................................................. 111
7.6.1 Cook Islands Evaluation Report ................................................................................... 111
7.6.2 FSM Evaluation Report .............................................................................................. 122
7.6.3 Kiribati Evaluation Report ........................................................................................... 134
7.6.4 Marshall Islands Evaluation Report ............................................................................. 145
7.6.5 Nauru Evaluation Report ............................................................................................. 155
7.6.6 Niue Evaluation Report .......................................................................................... 166
7.6.7 Palau Evaluation Report .............................................................................................. 177
7.6.8 Tonga Evaluation Report ............................................................................................ 193
7.6.9 Tuvalu Evaluation Report ....................................................................................... 202
7.7 Annex 7. Case studies .................................................................................................... 212
7.7.1 Kiribati SODIS CAMPAIGN - What it takes to change behaviour ................................ 212
7.7.2 Palau’s Climate Change Policy – The importance of teamwork .................................. 216
7.7.3 Best practice coastal protection in Tonga .................................................................. 220
7.8 Annex 8. Project alignment ........................................................................................... 223
7.9 Annex 9. Logframe Matrix ............................................................................................. 225
7.9.1 Annex 9.1. Original Logframe Matrix ......................................................................... 225
7.9.2 Annex 9.2. Revised Regional Logframe Matrix ............................................................227
7.9.3 Annex 9.3. Regional Logframe Matrix with results ...................................................... 231
7.10 Annex 10. Cascading logframes ..................................................................................... 240
7.11 Annex 11. Summary financial data ................................................................................. 243
7.12 Annex 12. Gender representation .................................................................................. 246
7.13 Annex 13. Communications Tools Summary ...................................................................247
7.14 Annex 14 DAC Evaluation Report Summaries ............................................................... 252
Pacific Research & Evaluation Associates
1. EXECUTIVE SUMMARY
This is an end-of-project evaluation of the Global Climate Change Alliance: Pacific Small Islands
States (GCCA: PSIS) project. It provides an independent assessment of the project’s performance
against key evaluation criteria, and identified lessons and practical recommendations.
Donor European Union (EU) Funding €11.4m (On target to
spend all funds)
Implementing Pacific Community Duration 5 years (July 2011 –
agency (SPC) November 2016)
Participating Cook Islands, Federated States of Micronesia (FSM), Kiribati, Republic of
countries Marshall Islands (RMI), Nauru, Niue, Palau, Tonga and Tuvalu
Focus Climate change adaptation
Target sectors Coastal protection, marine resources, health, agriculture, freshwater

The GCCA: PSIS project covers a vast geographical expanse, with approximately 8,000 km separating
Palau in the northwest to the Cook Islands in the southeast1. Some countries are spread across large
areas, with the islands of Kiribati dispersed across 3.5 million square kilometres2. The nine
participating countries have a total population of approximately 431,000, with the smallest country
being Niue (1,479 people) and three countries (Kiribati, FSM and Tonga) with populations above
100,000. A large percentage of the populations reside on low-lying atolls or islands that face
increasing threat from sea-level rise. Food and water security are key challenges across many of the
countries. The region’s characteristics and issues are shared with other Small Island Developing
States (SIDS)3.
There are significant challenges in implementing projects within the region, including:
 Transport logistics - unreliable, infrequent and expensive inter and intra-country transport
 Capacity constraints across the government and private sector
 Capability constraints across the government and private sector
Overall, the project was highly effective and successful in increasing the capacity of the nine target
countries to adapt to climate change. The on-ground climate change adaptation measures have
made a real and immediate increase in country’s climate change resilience. They also serve as models
that can be replicated or in some cases scaled up. Mainstreaming and capacity building activities
complimented these measures and form the foundation for future work to address future climate
change adaptation needs. Advancements in the ability for countries to access new modalities of
climate change financings was also achieved and this will help provide the resources required to
support future projects. Measured against the revised regional logframe matrix, the project achieved
all but one of its targets which further demonstrates the successful implementation of the project4.
The overall assessment of the GCCA: PSIS project is that it has been a valuable investment of EU-
funding to support Pacific countries to adapt to climate change.

1 Approximately a similar distance separating Brussels (Belgium) and Beijing (China).
2 In comparison, the European Union’s land area is 4,325 million square kilometres, with a population of 508
million people.
3 See UNEP 2014. Emerging issues for Small Island Developing States. Results of the UNEP Foresight Process.
United Nations Environment Programme (UNEP), Nairobi, Kenya.
4 Indicator not achieved related to national coordinators uploading documents to the Pacific Climate Change
Portal.
1
Pacific Research & Evaluation Associates
Logframe results summary
Overall objective  Over ten new activities addressed country requests
To support the governments of nine small for climate change adaptation
island states of the Pacific in their efforts  200 national sector specialists trained to integrating
to tackle the adverse effects of climate climate change adaptation into key sectors
change (Training provided in media, finance, Pacific Climate
Change Portal)
 396 people trained in proposal preparation using the
Logical Framework Approach (LFA)
 Additional 992 people trained on various subject
(Food security, water security, vector borne disease,
agroforestry and home vegetable gardening, tractor
maintenance, media, epidemiology, WASH, GIS,
food safety, behaviour change etc.)
 1,407 community members in Tarawa, Kiribati
trained in the use of SODIS
Purpose  New formal mechanisms in SPC to coordinate four
To promote long term strategies and different donors/partners engaged in delivery of
approaches to adaptation planning and to climate change resilience
pave the way for more effective and  GCCA: PSIS officer embedded in SPREP
coordinated aid delivery on climate  National climate change policy that integrates
change at the national and regional level disaster risk management in three countries
KRA1. Climate change mainstreamed  Three countries (Nauru, Palau, Tonga) with
into national /sector policies and plans new/revised national climate change policy
 Four countries (Kiribati, Nauru, Tuvalu, FSM) with
new/revised sector plans incorporating climate
change resilience.
KRA2. Progress access to climate change  Review of mainstreaming of climate change into
funding by new modalities national plans and policies completed for all nine
countries
 Review and assessment of national and sector
policies in relation to budget support modalities in
the Pacific smaller island states completed for nine
countries
 Two countries (Cook Islands, Tonga) assisted with
accessing/developing new climate finance
modalities.
KRA3. National climate change Eight of nine national adaptation projects completed by
adaptation projects March 2016, whilst the remaining one project which has
to be significantly downsized and is on schedule to
complete by June 2016:
 Four water sector projects (Fais in FSM; Tobi,
Sonsorol, Kayangel and Angaur in Palau; Niue,
Nauru,)
 Two coastal sector projects (Tonga, RMI)
 One agriculture sector project (Tuvalu)
 One health sector project (Kiribati)
 One marine resources sector project (Cook Islands)
 Nine national lessons learnt workshops and one
regional lessons learnt workshop
2
Pacific Research & Evaluation Associates
 15 videos created reaching over 39,000 people
KRA4. Streamlined technical assistance  Two new regional coordination tools available
through regional collaboration (Pacific Climate Change Portal (PCCP), SPC climate
change project matrix)
 200 SPC GCCA project documents uploaded to
PCCP attracting 37,900 views. SPC GCCA website
attracted 3,8000 hits in 2012.

On-ground climate change adaptation project highlights

% country population
benefiting from on-
ground projects
Cook Islands 1.37
FSM 0.27
Kiribati 56.36 146,285 direct
RMI 3.25 and indirect
beneficiaries
Nauru 100
Niue 100 1,588
participants in
Palau 4.06
regional and
Tonga 3.26
national
Tuvalu 70.9
trainings

3