Table Of ContentJitetal.BMCMedicine2013,11:23
http://www.biomedcentral.com/1741-7015/11/23
CORRESPONDENCE Open Access
Economic analyses to support decisions about
HPV vaccination in low- and middle-income
countries: a consensus report and guide for
analysts
Mark Jit1,2, Carol Levin3, Marc Brisson4,5, Ann Levin6, Stephen Resch7, Johannes Berkhof8, Jane Kim7 and
Raymond Hutubessy9*
Background
Abstract
The World Health Organization (WHO) recommends
Low- and middle-income countries need to consider thatcost-effectivenessbeconsideredbeforehumanpapil-
economic issues such as cost-effectiveness, lomavirus (HPV) vaccination is introduced in national
affordability and sustainability before introducing a programs [1]. However, many low- and middle-income
program for human papillomavirus (HPV) vaccination. countries(LMICs)lackthetechnicalcapacityandaccurate
However, many such countries lack the technical empiricaldatatodevelopandparameterizedenovomod-
capacity and data to conduct their own analyses. els of complex interventions such as HPV vaccination
Analysts informing policy decisions should address [2-5].Giventheseconstraints,LMICdecision-makersmay
the following questions: 1) Is an economic analysis want to apply or adapt a previous economic evaluation
needed? 2) Should analyses address costs, conductedinadifferentcountry.Thepracticeofadapting
epidemiological outcomes, or both? 3) If costs are existingmodelsiscommon:onesystematicreviewofeco-
considered, what sort of analysis is needed? 4) If nomicevaluationsofHPVvaccinationfoundthat35of58
outcomes are considered, what sort of model should relevantarticleswereadaptationsofpreviousmodels[6].
be used? 5) How complex should the analysis be? 6) However, most existing analyses (44/58) were set exclu-
How should uncertainty be captured? 7) How should sivelyinhigh-incomecountries(HICs).Hence,analystsin
model results be communicated? Selecting the LMICs who want to understand whether they should
appropriate analysis is essential to ensure that all the adapt a previous model, develop a new one, or not con-
important features of the decision problem are duct an economic evaluation at all have fewer examples
correctly represented, but that the analyses are not andguidanceonwhichtorely.
more complex than necessary. This report describes Furthermore,analystsoftenfaceadilemmainchoosing
the consensus of an expert group convened by the whichmodelstodeveloporadapt.Ontheonehand,sim-
World Health Organization, prioritizing key issues to plermodelsare easiertoparameterize,adapt, andinter-
be addressed when considering economic analyses to pret; however, such models are designed to answer a
support HPV vaccine introduction in these countries. limitedrangeofquestions,andcanbemisleadingifused
Keywords: Human papillomavirus, vaccination, low- toaddressmorecomplexissues[7,8].Ontheotherhand,
and middle-income countries, economic evaluation. modelsthatareequippedtoaddressmorecomplexissues
mayrequireexpertiseordatathatmaynotbeavailablein
thatcountry.
Inadditiontocost-effectivenessresults,decision-makers
mayneedtoknowthefinancingrequirementsandafford-
ability of vaccine introduction to support financial plan-
ningandforecasting.Suchconsiderationsareinformedby
*Correspondence:[email protected]
9InitiativeforVaccineResearch,WorldHealthOrganization,20AvenueAppia, models that comprehensively capture the programmatic
CH-1211Geneva,Switzerland costs and logistic considerations involved in national
Fulllistofauthorinformationisavailableattheendofthearticle
©2013Jitetal;licenseeBioMedCentralLtd.ThisisanOpenAccessarticledistributedunderthetermsoftheCreativeCommons
AttributionLicense(http://creativecommons.org/licenses/by/2.0),whichpermitsunrestricteduse,distribution,andreproductionin
anymedium,providedtheoriginalworkisproperlycited.
Jitetal.BMCMedicine2013,11:23 Page2of9
http://www.biomedcentral.com/1741-7015/11/23
scale-upof vaccine introduction[9], butare less focused extent to which these analyses are suited for their own
on reproducing the natural history and epidemiology of country. Ideally, such a process should be transparent
HPV-relateddiseases. and unbiased, using best-practice methodology for evi-
Afurtherchallengeistranslatingeconomicresultsinto dence review and synthesis [11].
conclusionsthat are useful to decision-makers. In parti-
cular, decision-makers need to understand the type of Question 2: Should analyses address costs,
policy questionsthateconomic models canaddress,and epidemiological outcomes or both?
theappropriatecaveatsaroundmodelconclusions(such Once a decision is made to conduct an economic eva-
asdatashortcomingsandmodeluncertainty). luation, the type of analysis needs to be chosen. First, a
To address these questions, the WHO convened a clear, answerable policy question is defined [12]. This
panel of modelers and economists to develop guidance will determine 1) which costs and/or health outcomes
foranalystsbasedinLMICswhoadvisepolicy-makerson (for example, cervical-cancer incidence or disability-
HPV vaccination (but based on principles applicable to adjusted life years (DALYs) lost) are considered, 2) what
all countries). Panel members held several meetings to the intervention and comparator under evaluation are,
outline issues,thenagreedon asummary.Eachmember 3) who the target audience is, and 4) what methodologi-
then drafted sections of the guidelines, then received cal options (such as time horizon and economic per-
comments from all other participants. Comments were spective) are chosen.
collated,harmonized and furtherreviewed byallpartici- Analyses involving costs alone usually use primary
pantsuntilcompleteconsensuswasreached.Fulldetails data from demonstration projects and other studies
ofguidelinedevelopmentareavailableintheAppendix. exploring different HPV vaccine delivery programs [9].
These are useful for several reasons. Firstly, vaccine pur-
Question 1: Is an economic analysis needed? chase costs are often borne by external donors who
Threewaystouseeconomicanalysestoinformdecisions make decisions based on their own analyses. However,
aroundHPVvaccinationare:1)conductingnostructured national decision-makers still need to investigate
analysis,2)borrowinginsightsfromanalysesinotherset- whether vaccine delivery is affordable in the short term
tings,and3)conductingadenovocountry-specificanaly- and sustainable in the long term. Secondly, if vaccina-
sisusinganeconomicmodel. tion is introduced, planners need to allocate resources
The option of not conducting any analysis is particu- for the financial costs of introducing the vaccine.
larlyattractiveinLMICswithlimitedanalyticalcapacity. Thirdly, these data inform cost minimization studies
However,suchcountriesmayalsohavethegreatestneed that address questions such as optimal delivery strategy
for evidence, because poor allocation of funding at the (school-based, health facility-based, using national
margin hasgreaterhealthconsequences when funds are immunization days, or a combination). They also inform
limited. budget impact analyses, and are essential inputs into full
Policy-makersmayalsoconsiderusingexistinganalyses economic evaluations such as cost-effectiveness analyses.
andinsightsfrom othersettings. Whenanalytical results Alternatively, models may project the effects of vacci-
arestableacrosssettings,conductingnewsetting-specific nation on disease burden [13], using measures such as
studiesmay addlittle value. Forexample, model conclu- cervical-cancer morbidity, mortality, or DALYs. These
sionsaboutvaccinatinggirlsbeforeonsetofsexualactivity studies are essential inputs for cost-effectiveness ana-
have been similar across HICs [7,8]. There have been lyses, but are also useful in their own right to inform
fewer results from LMICs, but existing analyses suggest surveillance and to understand emerging post-vaccina-
thatHPVvaccinesneedtobepricedmuchmorecompeti- tion trends such as changes in the average age of infec-
tively in LMICs than in HICs for vaccination to be cost- tion, waning vaccine effectiveness, herd immunity, and
effective[6,10]. potential type replacement. They can also show if vacci-
Vaccine price, HPV prevalence, and uptake of cancer nation is equitable in terms of how its effects are dis-
screening and treatment are key drivers of the cost- tributed by income strata, geographical location, ethnic
effectiveness of HPV vaccination in LMICs [6,10]. group, and gender.
Hence, studies investigating similar policy questions in For setting priorities, full economic analyses consider-
settings where such parameters are similar can be ing both costs and outcomes are needed. Such analyses
adapted to a new setting. Many parameters that vary address whether HPV vaccination is worth the invest-
across settings (such as unit costs) are usually easy to ment compared with either current practice such as
adjust for in economic models. Hence, before deciding existing cervical-cancer screening and treatment [14] or
whether to conduct a de novo analysis, decision-makers doing nothing [15]. They can also address more detailed
should consider existing analyses, investigate the main questions such as the choice of vaccine (bivalent or
drivers of results in similar countries, and appraise the quadrivalent), target ages for routine and catch-up
Jitetal.BMCMedicine2013,11:23 Page3of9
http://www.biomedcentral.com/1741-7015/11/23
vaccination, inclusion of boys in the HPV vaccination (cid:129)Aningredientsapproachestimatesthequantityand
program [16], and revising screening policies to opti- cost of each individual resource required by a pro-
mize synergies with vaccination [8]. gram [19].Thisisthepreferredapproachtoevaluate
In HICs, economic models often inform evidence- the value of resources used to introduce HPV vacci-
based priority-setting discussions through National nationbecausetheservice-deliverystrategiesusedare
Immunization Technical Advisory Groups (NITAGs) differentfromthoseusedforinfantvaccines.
[17]. In LMICs, priority-setting discussions are still rele- (cid:129)Averagecostsbasedonadministrativereturnsortar-
vant, but may not be held because of lack of capacity to iffsareoftenusedineconomicevaluationsinHICs,on
perform local evaluations and/or the absence of advisory thebasisthatdeliverycostsaresmallcomparedtothe
groups to interpret results. In addition, decisions are cost of vaccine purchase, and hence inaccuracies in
often driven by donors such as international funding their estimation are less important. However, such
agencies or individual donor countries, so spending assumptionsmaynotholdinmanyLMICs.
priorities are set by the international community.
Therefore, more pragmatic analyses such as budget 4) Choose between incremental costs (the additional
impact analyses focusing on cost savings, affordability, costs that occur with the HPV vaccination (or other
and sustainability may be more relevant in these coun- intervention)) or all costs, including shared costs. For
tries [18]. example, the transport costs may be shared with other
vaccines, so would not be fully included in an incremen-
Question 3: If costs are considered, what sort of tal analysis. Incremental cost analyses are typically used
analysis is needed? for cost-effectiveness and cost-benefit analyses. How-
An economic analysis involving costs involves the fol- ever, for partial economic analyses, total cost analyses
lowing steps. can inform comparisons between different strategies.
1) Plan a data-collection study, which can be either 5) Ifvaccinationisbeingcompared with screeningand
retrospective or prospective. treatment,estimatethevalueofresourcesusedforscreen-
2) Choose between calculating financial or economic ing,diagnosis,andtreatmentofneoplasias.Eveninhealth
costs (or both). systems without organized screening, there is a need to
account for the cost of providing cancer treatment and
(cid:129) Financial costs are the value of resources to the palliativecare.
payer (such as the Ministry of Health), and are used 6) Distinguish between recurrent or operational costs
to assess affordability. They include actual resources (those lasting less than a year) and capital costs (lasting
purchased such as injection supplies, outreach and longer than a year), so that the value of capital goods
daily allowances, and funds for training and develop- can be spread over a longer period [19].
ing new communication materials. 7) Identify start-up costs needed in the first year of
(cid:129)Economiccostscomprisetheopportunitycostsofall vaccination, such as introduction costs (microplanning,
outlays,includingresourcesalreadypaidfororowned initial training, and initial social mobilization material
bytheMinistryofHealth,andothersourcesoffinan- development) and capital goods (additional cold chain
cing,suchasthesalariesofhealthpersonnel,vaccines equipment, vehicle requirements, and incinerators).
paid for by partners, and donated equipment or ser-
vices such as volunteer time. They give a more com- Question 4: If outcomes are considered, what sort
plete picture of resources that are tied up in the of model should be used?
provision of the new vaccine and their opportunity Modelchoiceshouldbedrivenbythepolicyquestionsto
costs, and should be used if a cost-effectiveness or beanswered.Thesimplestmodels(termed‘proportionate
cost-benefitanalysisistobeconducted.Theyarealso outcomesstaticmodels’;Table1),donotcaptureindirect
used to evaluate sources of required financing for benefitsofvaccination(herdimmunity),butcanstillpro-
introductionandscale-up,andtoassessthefeasibility videconservativeestimatesofthecost-effectivenessofvac-
ofprovidingthenecessaryhealthsystemrequirements cinatinggirlsbeforetheystartsexualactivity.Theyalsodo
ona sustainable basis. They also give a betterassess- notcapturechangesinsexualactivity,screeningbehavior,
ment ofthe share ofcosts financedby different part- ordemographics.
nersincludingexternalpartners,becausetheyinclude Proportionate outcomes models assume that cervical-
thevalueofdonatedgoods. cancerincidencedoesnotchangeintheabsenceofvacci-
nation,soshouldnotbeusedincountriesinwhichcervical
3) Choose between calculating costs using an ingredi- screeninghasrecentlyorwillshortlybeintroduced.Ignor-
ents approach, or using average costs. ing the potential reduction in cervical-cancer incidence
Jitetal.BMCMedicine2013,11:23 Page4of9
http://www.biomedcentral.com/1741-7015/11/23
Table 1Typesofmodels involving health outcomes, from leastto most complex.
Modeltype Features Datarequirements
Demography Epidemiology Clinical
Static Reductionincervicalcancerdueto Population Cervical-cancerincidence,mortality None
proportionate vaccinationistheproductofeffective structure,birth andproportioncausedbyHPV16/18,
outcomes vaccinecoverageandcervical-cancer anddeathrates byage
model incidencecausedbyvaccine-typeHPV
Static Representsthenaturalhistoryofdisease Population Cervical-cancerincidence,mortality Uptakeandefficacyof
progression fromHPVinfectiontocervicalcancer, structure,birth andproportioncausedbyHPV16/18, screening,diagnosisand
model usuallyinasinglebirthcohort anddeathrates byage;femaleHPVprevalenceand treatmentofcancerand
clearancebyage pre-cancer
Transmission CapturesHPVtransmissionfrominfectedto Population Cervical-cancerincidence,mortality Uptakeandefficacyof
dynamic susceptibleindividualsintheentire structure,birth andproportioncausedbyHPV16/18, screening,diagnosisand
model population anddeathrates, byage;HPVprevalenceandclearance treatmentofcancerand
sexualbehavior byageandsex pre-cancer
Hybridmodel Usesastaticprogressionmodeltocapture Population Cervical-cancerincidence,mortality Uptakeandefficacyof
diseasenaturalhistory,andadynamic structure,birth andproportioncausedbyHPV16/18, screening,diagnosisand
modeltocaptureinfectiontransmission anddeathrates, byage;HPVprevalenceandclearance treatmentofcancerand
sexualbehavior byageandsex pre-cancer
resultingfromscreeningwouldbiastheresultsinfavorof but can be incorporated as add-on modules to static
HPVvaccination.Suchsituationsrequiretheuseof‘static models. Thus, end users can begin with the simpler
progression models’ (Table 1), which model the natural model, and then use the hybrid module for more com-
history of disease from HPV infection to cervical cancer, plexquestions.Thedisadvantageofthisapproachisthat
andtheeffectofscreeningondiseaseprogression. the wayinwhichdynamic model predictionsinformthe
Questions such as the incremental cost-effectiveness staticmodelcanlacktransparency.
of vaccinating boys, the optimal age of routine and Countries or donors may need to choose between the
catch-up vaccination, and the optimal combination of competing HPV vaccines. Few modeling studies have
screening and vaccination, should be examined using directly compared bivalent and quadrivalent vaccines,
‘transmission dynamic models’ (Table 1) which capture and none of these studies have been carried out in
indirect effects of vaccination [8,20]. Such models are LMICs.StudiesinHICssuggestthatquadrivalentvacci-
also required if evidence is needed on the magnitude nationismorecost-effectivethan bivalentvaccination if
and timing of the benefits of vaccination. theyareequallypriced,becauseoftheadditionalbenefits
Static models overestimate the incremental benefit of ofpreventinggenitalwarts[22-24].InmanyLMICshow-
vaccinating boysinadditiontogirls[20]. Thisisbecause ever, preventingcancermay be themainorsole priority
they do not capture the herd protection males already forHPVvaccination,sobivalentvaccinationmayhavean
receive from female-only HPV vaccination programs. advantage because of its better cross-protection against
Hence,adynamicmodelisneededtoshowthatmalevac- non-vaccine oncogenic HPV types [25]. Such questions
cination iscost-effective, althoughastatic model may be should be addressed using multi-type transmission
usedtoconfirmthatvaccinatingboysisnotcost-effective dynamicmodelsbecause1)staticmodelsdonotcapture
atall,astheresultsunderthedynamicmodelwillbeless thepictureofwartpreventioninmalesasaresultofvac-
favorable than under the static model. Furthermore, it is cinatingfemales,and2)multi-typemodelsareneededto
unclearwhetherusingastaticmodelwillproduceconser- accuratelycapturethedifferentcross-protectiveeffectsof
vativeestimatesforcatch-upstrategies.Thisisbecausethe thetwovaccines.
potential for incremental gains in the catch-up cohorts
may be smallerunder adynamicframeworkwheresome Question 5: How complex should the analysis be?
vaccinated individuals would have been indirectly pro- Modeltypesrangefrom simple‘proportionateoutcomes’
tectedthroughherdimmunity[8]. spreadsheet models to complex dynamic transmission
Simple static cohort models are usually difficult to models(seeTable1).Complexmodelsmaygeneratemore
adapt into the dynamic models. Hence, some studies accurate results and address more questions, butrequire
have captured herd immunity using hybrid models, additional data, time, effort, and expertise. Using them
which use dynamic models to represent transmission, may require dependenceonexternalconsultants orcon-
and static models to represent disease progression and sumerarelocalexpertise.However,theymayalsoengage
screening (Table 1) [15,16,21]. Hybrid models can local experts with strong analytical skills in the public
address the same policy questions as dynamic models, healthpolicy-makingprocess.
Jitetal.BMCMedicine2013,11:23 Page5of9
http://www.biomedcentral.com/1741-7015/11/23
The first consideration must be to ensure that model certainaspectsofthediseaseorinterventionarenotexpli-
resultsincreasetheprobabilityofadecisionalignedwith citly incorporated. Hence, policy-makers short on both
thedecision-maker’spreferences. Oversimplifiedmodels data and technical expertise could focus on more basic
may make poor predictions, leading to worse policy questions that can be robustly answered with simpler
choices than would have occurred in the absence of models,andshouldconsiderwhethertheirpolicydecision
model-basedevidence.Hence,sometimesonlyasophisti- can also be informed by adapting insights from other
catedmodelisappropriate.Forexample,ifastaticmodel settings in which sophisticated modeling has been per-
(whichignoresherdimmunity)suggeststhatvaccination formedwithricherdata.Forexample,countriesthathave
is not cost-effective, then vaccination should not be yet to introduce HPV vaccination should first assess the
rejected without first testing the conclusion using a cost-effectiveness of routine vaccination of girls in early
dynamicmodelthatwillcapturemorebenefits.Similarly, adolescence,whichoftencanbeaddressedusingrelatively
investigatingthe relative value ofbivalentand quadriva- simplemodels(seeQuestion4above).Assessingthecost-
lent vaccines,orthepreciseupper agelimitforcatch-up effectivenessofmalevaccinationrequiresmoretechnically
vaccination,requiresophisticatedmodelscalibratedwith demanding analyses, so countries may need to draw
gooddata. insights from existing studies in high-income [8,28] and
A second consideration is the trade-off between the middle-income[16]countries,whichsuggestthatinitially
incremental informational value and additional time/ focusing onincreasing vaccine uptake in girls islikely to
effort needed for a more sophisticated modeling be a more efficient way to reduce cervical-cancer inci-
approach. A model should be as parsimonious as possi- dence. In some cases, however, conclusions drawn in
ble; it should capture effects vital to understanding the other settings may not apply to low-income countries
policy questions (such as herd protection when examin- becauseofdifferencesinsexualbehavior(suchaspartner-
ing the effects of vaccinating boys on the incidence of shipconcurrency),demographicstructure,HPVtypedis-
disease in girls), but equally should not have unneces- tribution, availability of screening and treatment, and
sary detail [26]. Determining whether particular effects co-factorssuchasHIVinfection.
are important for decision-making may require a more A fourth consideration is the objective, the intended
complex model to investigate whether the results of the audience,andtheultimateuseofresultsofthemodeling
simpler model are biased. Often, analysts can draw on a exercise.Forinstance,usingasimplemodeldevelopedby
repertoire of simple model structures that have been in-countryanalystsmaybuildinternalcapacityforpolicy
validated against more complex models and peer- modeling and cost-effectiveness analysis. If capacity to
reviewed. For instance, a range of HPV models were developevenasimplemodeldenovoislacking,itmaybe
recently compared using a standardized input dataset possible for local analysts to be advised by external
[6]. Analysts can also draw on methodological work experts or to adapt a model developed overseas. For
investigating the effect of different simplifications to example, models of HPV vaccination in Thailand [14]
model structure [20]. However, methodological investi- andSouthAfrica[29]weredevelopedbyindependentin-
gations in middle- and high-income settings may not countryanalystsadaptinganearliermodelfromtheUSA
represent behavioral determinants and disease co-factors [30]. Many modeling groups are open to adapting their
in low-income settings. existingmodelstonew settingsincollaborationwithin-
The third consideration is the data requirement. Even country analysts [6]. Another alternative is to develop a
simplemodelsrequiredemographic,epidemiological,clin- regionalnetworkofexpertiseinusingaparticularmodel,
ical,andeconomicdata(Table1).Ideally,theseshouldbe such as the ProVac initiative in Latin America [4] and
local data from large population-based cohorts or ade- the PREHDICT (Prevention Strategies for HPV-related
quatelypoweredtrials.Whenunavailable,lessreliabledata DiseasesinEuropeanCountries)initiativeinEurope[31].
sourcesmustbeused,suchasdatafromcancerregistries Overtime,developingin-country capacity may facilitate
andhealthcareutilization reports(whichmaybeincom- more informed policy analyses and greater use of evi-
plete and hence biased), or data extrapolated from other denceindecision-making.Evenamodeldevelopedover-
countries.Complexmodelsmayhavegreaterdatarequire- seas may strengthen capacity if it is parameterized and
ments,sodatashortcomingsmaycompromisetheirbene- interpretedbyin-countryanalystsinsuchawaythatthey
fits by introducing additional uncertainty. For example, gain knowledge of its design. As these analysts grow
transmissionmodelsmayrequiredataonsexualbehavior, moreexperienced,morecomplexfeaturessuchassexual
althoughsuchdatamayhavebeenpreviouslycollectedto partnerships, demographic change, and co-factors such
investigateHIVcontrolstrategies[27]. asHIVinfectioncanbeadded.Previousexperiencefrom
However, simpler models do not guarantee reduced HIV modeling suggests that it may be efficient to build
uncertainty,andmayindeedincreaseuncertainty,because simple models with sufficient flexibility to incorporate
Jitetal.BMCMedicine2013,11:23 Page6of9
http://www.biomedcentral.com/1741-7015/11/23
added levels of complexity as new questions, evidence, testsfordetectingglandularlesions)andpopulationbeha-
andcapabilitiesemerge[32]. vior (such as screening attendance, vaccine uptake, and
In addition, models constructed locally (or in partner- sexualbehavior).Instaticmodels,theeffectofherdimmu-
shipwithforeignexpertise)helptoengagepolicy-makers nityisuncertainbecauseitisnotincludedinthemodel.If
and program managers throughout the analytic process, there are no data to inform suchassumptions, structural
potentiallyleadingtoadeeperunderstandingofthelocal uncertaintyisexploredbyconstructingdifferentscenarios
driversofhealthimpactandcost,highlightingthevalueof representingalternativeassumptions.Ideally,thescenarios
data, and increasing awareness of data gaps. Similar should be informed by eliciting expert opinions, and in
engagement might be reached when a complex model is some cases, this may allow explicit quantification of the
accompaniedbyaneasy-to-useinterfacethatlocalstake- uncertaintyaroundparticularassumptions.
holders can parameterize themselves. However, results Methodologicaluncertaintyisideallyaddressedbyhav-
from simple models may be met with less skepticism ing a ‘reference case’ specifying the methodological
becausetheirstructure,inputsandassumptionsaregener- choicesthatshouldbemadeforallhealtheconomiceva-
ally more transparent to endusers. Lastly, policy-makers luationsin a country. If thisis unavailable,international
andprogrammanagersmay feel more ownershipofana- guidelines(suchasWHOrecommendations[12])canbe
lysesconductedlocally,sotheinfluenceoftheresultsmay used to ensure comparability with evaluations in other
weighmoreheavilyinthedecision-makingprocess. countries, but analysts should also show the conse-
quences of making different methodological choices.
Question 6: How should uncertainty be captured? Somemethodologicalchoices(suchasdiscountrates)are
Modelsmustconveymethodological,structural,andpara- basedonnationalconsiderationssuchasincomegrowth
meteruncertainty[33].Theserespectivelyarisefromdif- rates,sointernationalreferencecases maynotalwaysbe
ferentchoicesmadeabouttheanalyticmethodsused(for appropriate.
example,payerorsocietalperspective,discountingrates),
underlyingmodelassumptions,andmodelparameters. Question 7: How should model results be
Parameter uncertainty arises when the available input communicated?
datagiveonlylimitedinformationaboutthe‘true’valueof Whereas simpler models are likely to have more uncer-
the model parameters. This is particularly important in tainty,complex models may be more difficultto explain.
settings where available data are limited. Some health The most important outcome of a decision analysis is
technology authorities such as the National Institute for identifying the choice most likely to bring the greatest
HealthandClinicalExcellence (NICE)intheUKrecom- benefit (the expected utility-maximizing choice) under
mend theuse ofprobabilistic sensitivityanalysisto study base-case assumptions [34]. However, assumptions and
parameteruncertainty.Thisinvolvesspecifyingaprobabil- uncertaintiesaroundtheoutcomemustbecommunicated
itydistributiontorepresenttheuncertaintyofeachmodel sothatoptionstoimprovedataoranalysisqualitycanbe
parameter, and then propagating this uncertainty to the considered, and the credibility of such analyses is not
outcomes of the analysis (such as the cost-effectiveness threatenedifthemostprobableoutcomesdonotmateria-
ratio). However, even in developed countries, only a few lize. Unfortunately, when results are presented to deci-
HPVmodelershavetakenthisapproach[8].Thismaybe sion-makers, the assumptions, caveats, and sensitivity
because HPV models usually contain many parameters analysesare often greatly simplified. To avoid this,com-
whose uncertaintydistributionsarehardtoquantify,and municationshouldbeinitiatedearlyasanongoingprocess
whose values are dependent on each other in ways that inwhichkeymodel assumptionsare discussed,and their
require computationally demandingmethodsto capture. facevalidityevaluatedtoinformlatermodeliterations.
Instead,manycost-effectivenessanalysesrelyonunivariate Notalloutcomesusedineconomicevaluations(suchas
sensitivityanalysesinstead,whereeachparameterisvaried cost per DALY prevented) are fully appreciated by deci-
withinafixedrangeandtheotherskeptattheirbase-case sion-makers.Outcomesinclinicalunitssuchasmorbidity
values. Such an approach may still give insight, but does and mortality averted may be most appreciated by clini-
notconvey the total amount ofuncertainty causedby all cians, whereas financial planners may want intervention
parameters. costs,amountsofavertedspending,andtheeffectsontax
Structuraluncertaintyincervical-cancermodelscanarise receipts and public sector budgets. However, for many
frommodelrepresentationsofthedisease’snaturalhistory such metrics, there is no standard way of comparing
(suchasnaturalimmunitymechanisms,progressionfrom betweendifferenttypesofinterventions(suchasvaccina-
neoplasia to cancer, interaction betweenHPV types, and tionandspendingonacutecare).Hence,theopportunity
reactivationoflatentinfections),effectivenessofinterven- costofaninvestmentshouldbemade clear;forexample,
tions (such as the long-term efficacy of vaccination, the thevalueofotherinvestmentsthatmaybedisplacedasa
effectofvaccineboosting,andthesensitivityofscreening resultofvaccineintroduction.
Jitetal.BMCMedicine2013,11:23 Page7of9
http://www.biomedcentral.com/1741-7015/11/23
Otheranalystscomprise the second audience forcom- Reaching recommendations
municating model assumptions and results, so that they The WHO initiated discussions during the Interna-
canprovidetechnicalpeerreviewinassessingitsvalidity. tional Papillomavirus Conference, and asked this group
Sufficientdetailshouldbeprovidedtoallowfullreprodu- of experts to discuss and outline the issues to be
cibilitybycapablemodelers,andagoodunderstandingof addressed with respect to economic evaluations of
how the model works without having to reproduce it. HPV vaccination in LMICs, and the key principles to
Althoughthisshouldinprinciplebemorestraightforward use in addressing them. These were developed further
for simpler models, a simple model is not necessarily during a teleconference and a face-to-face expert meet-
‘transparent’ if it is ambiguously documented. A recent ing hosted by the WHO in December 2011. An outline
discussionpaperbytheinternationalHPVmodelingcom- of the discussions was drawn up and agreed on by all
munitysuggestedthatpublicationsofmodelresultsshould participants. Participants were then assigned sections
showa variety ofmeasures (such as intermediate disease of the paper to draft. Each section then received com-
endpoints, sources of parameters, and methods used to ments from all the other participants. The comments
calibratethem)eveniftheyarenotdirectlyrelevanttothe were collated and harmonized by MJ, and sent out for
policydecision,tofacilitatetheirappraisalbyothermode- further comments until complete consensus was
lers[35].Inmanycases,thiswillrequiretheuseofonline reached. In total, four iterations of this editing cycle
technicalappendicesduetoprintjournals’wordlimits. were made.
Summary Incorporating differing views
Economicanalysesareimportanttoolsforfacilitatingevi- The two areas in which participants had differing views
dence-based decision-making about HPV vaccination. were 1) the need for simple versus complex models and
Selecting the appropriate analysis is essential to ensure 2) the need for a partial analysis (costs or epidemiologi-
that all the important features of the decision problem cal outcomes only) versus a full economic analysis. To
are correctly represented. However, analyses should not reconcile these views, a summary of different views was
bemorecomplexthannecessary,assimpleranalysescan drawn up by MJ and agreed on by all participants. Two
enable model development by local analysts and greater participants (SR and RH) were asked to write sections
ownership of the results by local stakeholders. On a on the complexity and scope of models, respectively. CL
research level, a priority for future work is investigating and AL were also asked to expand upon when an incre-
theeffectthatkeybehavioraldeterminants,demographic mental analysis would be required versus a full eco-
changes,anddiseaseco-factorsimportantinlow-income nomic analysis. These sections then went through the
settings have on model results, so that insights from four editing iterations described above until complete
models of HPV vaccination in HICs can be made more consensus was reached. Hence, the final versions incor-
widelyapplicable. porated the views of all participants. At the end of the
process, there were no further unresolved issues.
Appendix: how the guidelines were formulated
Panel selection Strengths and limitations of the process
The guideline development process was initiated by the A key strength of this process is bringing together
WHO Initiative for Vaccine Research (IVR) of the analysts with different skills (in epidemiology, costing,
Department of Immunization, Vaccines and Biologicals engaging with policy-makers, and methodological devel-
(IVB). In his capacity as senior lead economist for the opment) who are not often involved in working
IVR,RHinvitedMJ,CL,MB,AL,JBandJKtoparticipate together. By harmonizing different views and opinions,
in a technical experts group that convened at the 27th we have been able to produce more nuanced guidelines.
International Papillomavirus Conference on September One limitation of the process was that participant selec-
17to22,2011.Theseindividualshaveanextensivepubli- tion and discussion of topics was not performed using a
cationrecord,technicalexpertiseand/orworkexperience systematic framework, because the purpose was to pro-
in 1) modeling and economic analysis of HPV vaccina- duce practical guidelines for field use rather than to pro-
tioninlow/middle-incomecountries,and2)contributing pose a comprehensive set of principles to be adopted
to methodological advances in HPV modeling. SR was universally. The way in which consensus was reached
selected on recommendation of the initial participants, (via iterative discussions by a nominated panel of
because of his involvement in the CerviVac initiative, a experts) is similar to the methods used by other WHO
largeconsortiumledby thePanAmerican HealthOrga- technical advisory groups such as its Immunization and
nization (PAHO) developing HPV models for LMICs Vaccines related Implementation Research (IVIR) advi-
thatwasnototherwiserepresentedinthepanel. sory committee.
Jitetal.BMCMedicine2013,11:23 Page8of9
http://www.biomedcentral.com/1741-7015/11/23
Abbreviations 10. GoldieSJ,O’SheaM,CamposNG,DiazM,SweetS,KimSY:Healthand
HPV:Humanpapillomavirus;HICs:Highincomecountries;LMICs:Lowand economicoutcomesofHPV16,18vaccinationin72GAVI-eligible
middle-incomecountries;WHO:WorldHealthOrganization. countries.Vaccine2008,26:4080-4093.
11. MoherD,LiberatiA,TetzlaffJ,AltmanDG:Preferredreportingitemsfor
Authors’contributions systematicreviewsandmeta-analyses:thePRISMAstatement.PLoSMed
RHandMJconceivedthemanuscript.Allauthorscontributedtowards 2009,6:e1000097.
writingthemanuscriptandallauthorshavereadandapprovedthe 12. WalkerDG,HutubessyR,BeutelsP:WHOGuideforstandardisationof
manuscriptforpublication. economicevaluationsofimmunizationprogrammes.Vaccine2010,
28:2356-2359.
Competinginterests 13. TracyL,GaffHD,BurgessC,SowS,GravittPE,TracyJK:Estimatingthe
MBhasconsultedandreceivedreimbursementfortravelexpensesfrom impactofhumanpapillomavirus(HPV)vaccinationonHPVprevalence
MerckFrosstandGlaxoSmithKline.JB’sinstitutereceivedresearchgrants andcervicalcancerincidenceinMali.ClinInfectDis2011,52:641-645.
fromGlaxoSmithKlinein2007-2010andincidentalconsultancyfeesfrom 14. PraditsitthikornN,TeerawattananonY,TantivessS,LimwattananonS,
SanofiPasteur.Noneoftheotherauthorshasanyknowncompeting RiewpaiboonA’,ChichareonS,IeumwananonthachaiN,
interests. TangcharoensathienV:Economicevaluationofpolicyoptionsfor
preventionandcontrolofcervicalcancerinThailand.
Acknowledgements Pharmacoeconomics2011,29:781-806.
RHisastaffmemberoftheWorldHealthOrganization.Theviewsexpressed 15. GinsbergGM,EdejerTT,LauerJA,SepulvedaC:Screening,preventionand
isthatoftheauthoranddonotnecessarilyrepresenttheviewsoftheWorld treatmentofcervicalcancer–aglobalandregionalgeneralizedcost-
HealthOrganization. effectivenessanalysis.Vaccine2009,27:6060-6079.
16. KimJJ,Andres-BeckB,GoldieSJ:ThevalueofincludingboysinanHPV
Authordetails vaccinationprogramme:acost-effectivenessanalysisinalow-resource
1ModellingandEconomicsUnit,HealthProtectionAgency,61Colindale setting.BrJCancer2007,97:1322-1328.
Avenue,LondonNW95EQ,UK.2DepartmentofInfectiousDisease 17. SenouciK,BlauJ,NyambatB,CoumbaFP,GautierL,DaSilvaA,
Epidemiology,LondonSchoolofHygieneandTropicalMedicine,Keppel FavorovMO,ClemensJD,StoeckelP,GessnerBD:TheSupporting
Street,LondonWC1E7HT,UK.3PATH,2201WestlakeAvenue,Suite200 IndependentImmunizationandVaccineAdvisoryCommittees(SIVAC)
Seattle,WA98121USA.4CentrederechercheduCHUdeQuébec,Hôpital initiative:acountry-driven,multi-partnerprogramtosupportevidence-
Saint-Sacrement,1050CheminSainte-Foy,Québec,G1S4L8,Canada. baseddecisionmaking.Vaccine2010,28(Suppl1):A26-A30.
5DepartmentofInfectiousDiseaseEpidemiology,ImperialCollege,London, 18. MauskopfJA,SullivanSD,AnnemansL,CaroJ,MullinsCD,NuijtenM,
UK.6IndependentConsultant,6414HollinsDr.,Bethesda,MD20817USA. OrlewskaE,WatkinsJ,TruemanP:Principlesofgoodpracticeforbudget
7CenterforHealthDecisionScience,HarvardSchoolofPublicHealth,718 impactanalysis:reportoftheISPORTaskForceongoodresearch
HuntingtonAve,Boston,MA02130,USA.8DepartmentofEpidemiologyand practices–budgetimpactanalysis.ValueHealth2007,10:336-347.
Biostatistics,VUUniversityMedicalCentre,PObox7057,1007MB 19. Tan-TorresEdejerT,BaltussenR,AdamT,HutubessyR,AcharyaA,EvansDB,
Amsterdam,TheNetherlands.9InitiativeforVaccineResearch,WorldHealth MurrayCJLe:Makingchoicesinhealth:WHOguidetocost-effectiveness
Organization,20AvenueAppia,CH-1211Geneva,Switzerland. analysis.Geneva:WorldHealthOrganisation;2003.Geneva,WorldHealth
Organisation;2003.
Received:2November2012 Accepted:30January2013 20. VandV,BrissonM,BoilyMC:Understandingdifferencesinpredictionsof
Published:30January2013 HPVvaccineeffectiveness:Acomparativemodel-basedanalysis.Vaccine
2010,28:5473-5484.
21. BogaardsJA,CoupeVM,XiridouM,MeijerCJ,WallingaJ,BerkhofJ:Long-
References
termimpactofhumanpapillomavirusvaccinationoninfectionrates,
1. WorldHealthOrganization:Humanpapillomavirusvaccines.WHO
cervicalabnormalities,andcancerincidence.Epidemiology2011,
positionpaper.WklyEpidemiolRec2009,84:118-131.
22:505-515.
2. HutubessyR,HenaoAM,NamgyalP,MoorthyV,HombachJ:Resultsfrom
22. JitM,ChapmanR,HughesO,ChoiYH:Comparingbivalentand
evaluationsofmodelsandcost-effectivenesstoolstosupport
quadrivalenthumanpapillomavirusvaccines:economicevaluation
introductiondecisionsfornewvaccinesneedcriticalappraisal.BMCMed
basedontransmissionmodel.BMJ2011,343:d5775.
2011,9:55.
23. DeeA,HowellF:Acost-utilityanalysisofaddingabivalentor
3. YothasamutJ,TantivessS,TeerawattananonY:Usingeconomicevaluation
quadrivalentHPVvaccinetotheIrishcervicalscreeningprogramme.
inpolicydecision-makinginAsiancountries:missionimpossibleor
EurJPublicHealth2010,20:213-219.
missionprobable?ValueHealth2009,12(Suppl3):S26-S30.
24. BrissonM,VandV,DeWalsP,BoilyMC:Thepotentialcost-effectiveness
4. AndrusJK,ToscanoCM,LewisM,OliveiraL,RoperoAM,DavilaM,
ofprophylactichumanpapillomavirusvaccinesinCanada.Vaccine2007,
FitzsimmonsJW:Amodelforenhancingevidence-basedcapacitytomake
25:5399-5408.
informedpolicydecisionsontheintroductionofnewvaccinesinthe
Americas:PAHO’sProVacinitiative.PublicHealthRep2007,122:811-816. 25. MalagonT,DroletM,BoilyMC,FrancoEL,JitM,BrissonJ,BrissonM:Cross-
protectiveefficacyoftwohumanpapillomavirusvaccines:asystematic
5. IglesiasCP,DrummondMF,RoviraJ:Health-caredecision-making
reviewandmeta-analysis.LancetInfectDis2012.
processesinLatinAmerica:problemsandprospectsfortheuseof
26. BrennanA,ChickSE,DaviesR:Ataxonomyofmodelstructuresfor
economicevaluation.IntJTechnolAssessHealthCare2005,21:1-14.
economicevaluationofhealthtechnologies.HealthEcon2006,
6. JitM,DemarteauN,ElbashaE,GinsbergG,KimJ,PraditsitthikornN,
15:1295-1310.
SinanovicE,HutubessyR:Humanpapillomavirusvaccineintroductionin
27. WellingsK,NanchahalK,MacdowallW,McManusS,ErensB,MercerCH,
low-incomeandmiddle-incomecountries:guidanceontheuseofcost-
JohnsonAM,CopasAJ,KorovessisC,FentonKA,etal:Sexualbehaviourin
effectivenessmodels.BMCMed2011,9:54.
Britain:earlyheterosexualexperience.Lancet2001,358:1843-1850.
7. KimJJ,BrissonM,EdmundsWJ,GoldieSJ:Modelingcervicalcancer
28. BogaardsJA,KretzschmarM,XiridouM,MeijerCJ,BerkhofJ,WallingaJ:
preventionindevelopedcountries.Vaccine2008,26(Suppl10):K76-K86.
Sex-specificimmunizationforsexuallytransmittedinfectionssuchas
8. BrissonM,VandV,BoilyMC:Economicevaluationofhuman
humanpapillomavirus:insightsfrommathematicalmodels.PLoSMed
papillomavirusvaccinationindevelopedcountries.PublicHealth
2011,8:e1001147.
Genomics2009,12:343-351.
29. SinanovicE,MoodleyJ,BaroneMA,MallS,ClearyS,HarriesJ:Thepotential
9. LevinCE,VanMinhH,OdagaJ,SaritRoutS,NguyenThiNgocD,
cost-effectivenessofaddingahumanpapillomavirusvaccinetothe
MenezesL,MendozaAraujoM,LaMontagneDS:Incrementalcostsof
cervicalcancerscreeningprogrammeinSouthAfrica.Vaccine2009,
strategiestodeliverhumanpapillomavirusvaccinetoyoungadolescent
27:6196-6202.
girlsinIndia,Peru,UgandaandVietNam.BullWorldHealthOrgan2012.
Jitetal.BMCMedicine2013,11:23 Page9of9
http://www.biomedcentral.com/1741-7015/11/23
30. KulasingamSL,MyersER:Potentialhealthandeconomicimpactof
addingahumanpapillomavirusvaccinetoscreeningprograms.JAMA
2003,290:781-789.
31. health-economicModellingofPreventionSTrategiesforHpv-related
DiseasesinEuropeanCountries.[http://cordis.europa.eu/search/index.cfm?
fuseaction=proj.document&PJ_RCN=11314364],Accessedon29November
2012.
32. LasryA,RichterA,LutscherF:Recommendationsforincreasingtheuseof
HIV/AIDSresourceallocationmodels.BMCPublicHealth2009,9(Suppl1):S8.
33. BilckeJ,BeutelsP,BrissonM,JitM:Accountingformethodological,
structural,andparameteruncertaintyindecision-analyticmodels:a
practicalguide.MedDecisMaking2011,31:675-692.
34. ClaxtonK:Theirrelevanceofinference:adecision-makingapproachto
thestochasticevaluationofhealthcaretechnologies.JHealthEcon1999,
18:341-364.
35. CraigBM,BrissonM,ChessonH,GiulianoAR,JitM:Proceedingsofthe
ModelingEvidenceinHPVPre-ConferenceWorkshopinMalmo,
Sweden,May9-10,2009.ClinTher2010,32:1546-1564.
Pre-publicationhistory
Thepre-publicationhistoryforthispapercanbeaccessedhere:
http://www.biomedcentral.com/1741-7015/11/23/prepub
doi:10.1186/1741-7015-11-23
Citethisarticleas:Jitetal.:Economicanalysestosupportdecisions
aboutHPVvaccinationinlow-andmiddle-incomecountries:a
consensusreportandguideforanalysts.BMCMedicine201311:23.
Submit your next manuscript to BioMed Central
and take full advantage of:
• Convenient online submission
• Thorough peer review
• No space constraints or color figure charges
• Immediate publication on acceptance
• Inclusion in PubMed, CAS, Scopus and Google Scholar
• Research which is freely available for redistribution
Submit your manuscript at
www.biomedcentral.com/submit
