(2018) Economic evaluation of population health interventions aimed at children and delivered at sc

336 Pages·2017·4.39 MB·English

Boyer, Nicole Renée Soldner (2018) Economic evaluation of population
health interventions aimed at children and delivered at school. PhD thesis.

http://theses.gla.ac.uk/9012/

Copyright and moral rights for this work are retained by the author
A copy can be downloaded for personal non-commercial research or study, without prior
permission or charge
This work cannot be reproduced or quoted extensively from without first obtaining
permission in writing from the author
The content must not be changed in any way or sold commercially in any format or
medium without the formal permission of the author
When referring to this work, full bibliographic details including the author, title,
awarding institution and date of the thesis must be given

Enlighten:Theses
http://theses.gla.ac.uk/
[email protected]
Economic Evaluation of Population
Health Interventions Aimed at Children
and Delivered at School

Nicole Renée Soldner Boyer

MSc Public Health, BSc Community Health

Submitted in fulfilment of the requirements for the degree of

Doctor of Philosophy

Health Economics and Health Technology Assessment (HEHTA)

Institute of Health and Wellbeing

College of Medical, Veterinary and Life Science

University of Glasgow

October 2017
ii

Abstract
Background: Population health interventions by their nature affect an entire population
and are typically delivered outwith of health services and within the community, such as
in schools. An example of such interventions are those that aim to improve children’s
social and emotional wellbeing, which have demonstrated effectiveness in the short-term
and potentially the long-term. However, challenges arise when conducting economic
evaluations of population health interventions, most notably the difficulties of identifying,
measuring, and valuing broader intersectoral costs, health, and non-health outcomes.
Economic evaluation in an education context is relatively novel, but could provide
decision-makers with information to help them make transparent and consistent
decisions about how to allocate limited funds. This thesis examined the role for economic
evaluation in school-based interventions and sought to determine appropriate methods
for its implementation in addition to examining appropriate child-focused outcome
measures. Thus, the overarching research question asked, ‘How should the cost-
effectiveness of school-based, population health interventions aimed at children be
determined?’
Methods: A mixed methods approach to this thesis was used:
(i) a systematic literature review and narrative synthesis to determine which
evaluation methods (economic and non-economic) are currently being used in
school-based population health interventions;
(ii) a case study to illustrate an economic evaluation (including cost-utility and
cost-effectiveness analysis) of a school-based intervention to reflect on the
advantages and disadvantages for decision making in this context; and
(iii) an exploration of outcome measures (through mapping validation) for valuing
child health and social and emotional wellbeing in school-based programmes
to support future evaluation work in this context.
iii

Data for the economic evaluation and mapping validation study were available from a
cluster randomised controlled trial of the Roots of Empathy programme in Northern
Ireland (Ref: 10/3006/02).
Results: The systematic review found that the methods currently being utilised to
evaluate school programmes are varied (including economic evaluation, cost only, and
effectiveness only studies), with poor quality reporting for the economic evaluations. Of
the few cost-utility analyses in school-based settings identified, none had directly
measured health-related quality of life using child measures or values. The case study
cost-utility analysis using Child Health Utility 9D of a school-based intervention was found
to be cost-effective from the National Health Service perspective with an incremental
cost-effectiveness ratio of £11,000 per quality-adjusted life year (confidence interval: -
£95,500 to £147,000), however the wide confidence interval demonstrates considerable
uncertainty. This uncertainty is likely due to a lack of statistically significant effect that
remained at the 36-month follow-up. Cost-effectiveness analysis using child behavioural
descriptive measure, the Strengths and Difficulties Questionnaire, resulted in an
incremental cost-effectiveness ratio of £197 per unit decrease in total difficulties score
(confidence interval: £77 to £471). The Strengths and Difficulties Questionnaire is suitable
for measuring social and emotional wellbeing, but is less advantageous for cost-
effectiveness decision-making as no consensus has been reached as to what a clinically
meaningful change in score represents, nor has a cost-effectiveness threshold been
defined. It remains uncertain how these cost-effectiveness results will be interpreted in
an education decision-making context where cost-effectiveness thresholds have not been
set up. The mapping validation study validated a mapping algorithm to convert the
Strengths and Difficulties Questionnaire into child health utility. Using this algorithm
provides an option for valuing incremental changes in health-related quality of life against
a generally accepted cost-effectiveness threshold from a health service perspective.
Conclusions: Given the findings from the various aspects of work undertaken for this
thesis to address population health issues, this thesis identified cost-benefit analysis as
currently the most comprehensive method for determining the value for money of
school-based public health interventions. Cost-benefit analysis incorporates monetary
valuation of multisector outcomes in a final net benefit/loss result allowing clear,
consistent, decision-making criteria to be set. Other methods such as cost-consequence
iv

analysis, cost-utility analysis, and multi-criteria decision analysis may also be suitable
depending on the decision-making context and problem. This thesis demonstrates a lack
of clear decision-making criteria in place for funding allocation decisions in education (e.g.
education specific cost-effectiveness thresholds). Furthermore, there is no equitable
method currently in place for apportioning the cost of funding public health interventions
that generate benefits for multiple sectors. From a health service perspective, directly
measuring child health utility using the Child Health Utility 9D is preferred as it is the only
preference-based measure developed specifically for children and valued by young
people. Mean child health utility can be predicted by mapping from the Strengths and
Difficulties Questionnaire. This affords the opportunity to estimate longer-term utility by
utilising long-term cohort data that routinely collects the Strengths and Difficulties
Questionnaire, as long-term cost-effectiveness of school-based preventive programmes is
an area in need of further research. The school setting plays an important role in shaping
our young people’s futures. Economic evaluation of school-based population health
interventions is justified, as schools need to maximise their existing resources in order to
give children the best start in life.
v

Table of Contents
Abstract .................................................................................................................................. ii
List of Tables ........................................................................................................................ viii
List of Figures ........................................................................................................................ ix
Acknowledgement ................................................................................................................. x
Author’s Declaration ............................................................................................................. xi
Publications, Working Papers and Presentations ................................................................ xii
Abbreviations ...................................................................................................................... xiii
1 Introduction ................................................................................................................... 1
1.1 Roots of Empathy .................................................................................................... 6
1.2 Research question and aims .................................................................................... 9
1.3 Thesis outline ......................................................................................................... 10
2 Economic Evaluation and Population Health Economic Evaluation ............................ 13
2.1 Economic evaluation methods .............................................................................. 14
2.1.1 Introduction and definitions .......................................................................... 14
2.1.2 A brief history of economic evaluation .......................................................... 18
2.1.3 Cost-effectiveness analysis (CEA) ................................................................... 20
2.1.4 Cost-utility analysis (CUA) .............................................................................. 24
2.1.5 Cost-benefit analysis (CBA) ............................................................................ 30
2.1.6 Cost-minimization analysis (CMA) ................................................................. 32
2.1.7 Cost-consequence analysis (CCA) .................................................................. 33
2.1.8 The role of economic evaluation in healthcare decision making in the UK and
internationally .............................................................................................................. 34
2.1.9 Summary ........................................................................................................ 42
2.2 Economic evaluation of population health interventions ..................................... 42
2.2.1 Cost-effectiveness of PHIs .............................................................................. 43
2.2.2 Economics of prevention ............................................................................... 54
2.2.3 Challenges of economic evaluation of PHIs ................................................... 57
2.2.4 Economic evaluation in school settings ......................................................... 62
2.2.5 Summary ........................................................................................................ 64
2.3 Conclusion ............................................................................................................. 65
3 School-based intervention evaluation methodologies: a systematic review .............. 67
3.1 Introduction ........................................................................................................... 67
3.1.1 Aim ................................................................................................................. 68
3.1.2 Research Question ......................................................................................... 69
3.2 Systematic Review Methods ................................................................................. 69
3.2.1 Inclusion Criteria ............................................................................................ 70
vi

3.2.2 Database and Search Strategies ..................................................................... 72
3.2.3 Mitigating bias ................................................................................................ 74
3.2.4 Data Extraction and Study Appraisal ............................................................. 75
3.2.5 Data Synthesis ................................................................................................ 79
3.3 Results ................................................................................................................... 82
3.3.1 Preliminary synthesis ..................................................................................... 84
3.3.2 Exploring relationships within groups and between studies ......................... 86
3.3.3 Assessing the robustness of the synthesis. .................................................... 93
3.4 Discussion .............................................................................................................. 98
3.4.1 Summary of findings ...................................................................................... 98
3.4.2 Limitations ....................................................................................................101
3.5 Conclusion ...........................................................................................................102
4 RoE Economic Evaluation Methods: a Case Study .....................................................104
4.1 Introduction .........................................................................................................104
4.2 Economic evaluation in child health ...................................................................105
4.2.1 Paediatric Outcome Measures .....................................................................106
4.3 Roots of Empathy ................................................................................................115
4.3.1 The RoE trial .................................................................................................116
4.3.2 RoE trial aims ................................................................................................117
4.3.3 Data Collection .............................................................................................118
4.4 RoE main within-trial analysis methods ..............................................................120
4.4.1 Overview ......................................................................................................121
4.4.2 Costs .............................................................................................................122
4.4.3 Outcomes .....................................................................................................131
4.4.4 Missing Data .................................................................................................132
4.4.5 Analyses........................................................................................................136
4.4.6 Sensitivity Analyses ......................................................................................138
4.4.7 Summary ......................................................................................................141
5 RoE Main Trial Results: a Case Study .........................................................................142
5.1 Introduction .........................................................................................................142
5.2 RoE Main Trial Descriptive Results ......................................................................142
5.3 Costs ....................................................................................................................144
5.3.1 RoE intervention costs .................................................................................145
5.3.2 Resource Use ................................................................................................146
5.4 Outcomes ............................................................................................................148
5.5 Missing Data ........................................................................................................148
5.5.1 Logistic Regression .......................................................................................151
vii

5.6 Cost-effectiveness ...............................................................................................152
5.6.1 Sensitivity Analysis .......................................................................................155
5.7 Discussion ............................................................................................................159
5.8 Limitations ...........................................................................................................166
5.9 Conclusion ...........................................................................................................168
6 Appropriate Outcome Measurement: a Mapping Validation Study .........................170
6.1 Study aims ...........................................................................................................173
6.2 Methods ..............................................................................................................173
6.2.1 Strengths and Difficulties Questionnaire .....................................................173
6.2.2 Child Health Utility 9D ..................................................................................175
6.2.3 Analysis .........................................................................................................176
6.3 Results .................................................................................................................176
6.4 Discussion ............................................................................................................181
6.4.1 Reflection of the overall aims and research question .................................185
6.4.2 Limitations ....................................................................................................188
6.5 Conclusion ...........................................................................................................190
7 Chapter summary and discussion ..............................................................................191
7.1 Introduction .........................................................................................................191
7.2 Chapter Summaries .............................................................................................191
7.2.1 Chapter 1 ......................................................................................................191
7.2.2 Chapter 2 ......................................................................................................192
7.2.3 Chapter 3 ......................................................................................................198
7.2.4 Chapter 4 ......................................................................................................200
7.2.5 Chapter 5 ......................................................................................................202
7.2.6 Chapter 6 ......................................................................................................204
7.3 Limitations and Strengths ....................................................................................206
7.3.1 Critique of methods .....................................................................................206
7.3.2 Strengths of study ........................................................................................215
8 Recommendations and Conclusion ............................................................................219
8.1 Implications for policy and practice ....................................................................219
8.2 Recommendations ...............................................................................................221
8.3 Areas for further research ...................................................................................223
8.3.1 Practical application of appropriate methodology to evaluate PHIs ...........223
8.3.2 Defining resource allocation decision criteria in education ........................225
8.3.3 Determining long-term cost-effectiveness of SEL programmes ..................226
8.4 Conclusion ...........................................................................................................227
Appendices .........................................................................................................................229
viii

List of References ...............................................................................................................299

List of Tables
Table 1: NICE reference case side-by-side comparison summary. Replicated from
Developing NICE guidelines: the manual (PMG 20)150 ......................................................... 48
Table 2: Number of records identified by each database searched .................................... 82
Table 3: Study characteristics .............................................................................................. 85
Table 4: CHEERS checklist totals by study ............................................................................ 95
Table 5: CHEERS checklist totals by item ............................................................................. 96
Table 6: CHU9D descriptive system ...................................................................................110
Table 7: Bandings for interpretation of Teacher Completed SDQ .....................................113
Table 8: Data from the RoE trial collected for the economic evaluation ..........................122
Table 9: Hospital and Community Health Services Index ..................................................124
Table 10: OECD Purchasing Power Parities (PPP) ..............................................................125
Table 11: Component costs of the RoE programme ..........................................................125
Table 12: Discount table for Annuitization ........................................................................126
Table 13: RoE resource use, unit costs, and sources for unit costs ...................................130
Table 14: ICER for cost-effectiveness analyses on SDQ .....................................................138
Table 15: List of sensitivity analyses ..................................................................................140
Table 16: Baseline sample characteristics .........................................................................144
Table 17: Summarised cost of the Roots of Empathy Intervention ...................................145
Table 18: Detailed cost breakdown of intervention costs .................................................145
Table 19: Mean resource use costs by group and differences between groups ...............147
Table 20: Variable descriptions and missing data percentages .........................................149
Table 21: Association between missing cost and baseline variables .................................151
Table 22: Association between missing QALY and baseline variables ...............................151
Table 23: Example of regression output for association between missing and observed
values .................................................................................................................................152
Table 24: Cost-effectiveness results (adolescent values) ..................................................153
Table 25: QALY gain over population demonstrating the prevention paradox .................157
Table 26: Cost-effectiveness results (adult values) ...........................................................158
Table 27: SDQ domain score four band categorisation* ...................................................174
Table 28: Characteristics of participants ............................................................................180
Table 29: Differences in utility values ................................................................................181
ix

List of Figures
Figure 1: Cost-effectiveness plane ....................................................................................... 23
Figure 2: Simple example of individual QALY gain ............................................................... 28
Figure 3: QALY gains from a comparison of two alternatives .............................................. 29
Figure 4: List of steps to perform MCDA from Thokala et al.160 ISPOR task force ............... 52
Figure 5: MCDA checklist from Marsh et al.161 ISPOR task force ......................................... 53
Figure 6: Shifting the distribution of risk in the population approach versus a targeted
high-risk approach ................................................................................................................ 55
Figure 7: CHEERS Checklist From Husereau et al. Value in Health 16 (2013) 231-250........ 78
Figure 8: Flow chart of study selection process ................................................................... 83
Figure 9: Flow chart of study selection process ................................................................... 83
Figure 10: Concept map representing relationships and findings from the narrative
synthesis ............................................................................................................................... 92
Figure 11: Marginal increase in investment at different stages of the life cycle - adapted
from Heckman, 2008299 ......................................................................................................105
Figure 12: Resource use questionnaire ..............................................................................129
Figure 13: Flow diagram of recruitment and testing of children .......................................143
Figure 14: Pattern of missing data in a. costs and b. QALYs. Black shading represents
missing data grey represents observed data. ....................................................................150
Figure 15: Cost-effectiveness plane representing 1000 bootstrapped cost and QALY pairs
............................................................................................................................................154
Figure 16: CEAC showing probability of RoE being cost-effective compared to usual
classroom activities. The dashed lines indicate the probability of RoE being cost-effective
at the defined threshold. ...................................................................................................154
Figure 17: SDQ response frequency ...................................................................................178
Figure 18: CHU9D response frequency by level .................................................................179