Bayesian statistical modelling

598 Pages·2006·3.219 MB·English

by  Peter Congdon

J0470018755pre October17,2006 20:27 CharCount=0
Bayesian Statistical Modelling
SecondEdition
PETERCONGDON
QueenMary,UniversityofLondon,UK
iii
J0470018755pre October17,2006 20:27 CharCount=0
iii
J0470018755pre October17,2006 20:27 CharCount=0
Bayesian Statistical Modelling
i
J0470018755pre October17,2006 20:27 CharCount=0
WILEYSERIESINPROBABILITYANDSTATISTICS
establishedbyWalterA.ShewhartandSamuelS.Wilks
Editors
DavidJ.Balding,PeterBloomfield,NoelA.C.Cressie,NicholasI.Fisher,
IainM.Johnstone,J.B.Kadane,GeertMolenberghs,LouiseM.Ryan,
DavidW.Scott,AdrianF.M.Smith,JozefL.Teugels
EditorsEmeriti
VicBarnett,J.StuartHunter,DavidG.Kendall
Acompletelistofthetitlesinthisseriesappearsattheendofthisvolume.
ii
J0470018755pre October17,2006 20:27 CharCount=0
Bayesian Statistical Modelling
SecondEdition
PETERCONGDON
QueenMary,UniversityofLondon,UK
iii
J0470018755pre October17,2006 20:27 CharCount=0
Copyright(cid:2)C 2006 JohnWiley&SonsLtd,TheAtrium,SouthernGate,Chichester,
WestSussexPO198SQ,England
Telephone [phone]
Email(forordersandcustomerserviceenquiries):[email protected]
Visit our Home Page on www.wiley.com
AllRightsReserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystemortransmittedin
anyformorbyanymeans,electronic,mechanical,photocopying,recording,scanningorotherwise,exceptunder
thetermsoftheCopyright,DesignsandPatentsAct1988orunderthetermsofalicenceissuedbytheCopyright
LicensingAgencyLtd,90TottenhamCourtRoad,LondonW1T4LP,UK,withoutthepermissioninwritingofthe
Publisher.RequeststothePublishershouldbeaddressedtothePermissionsDepartment,JohnWiley&SonsLtd,
TheAtrium,SouthernGate,Chichester,WestSussexPO198SQ,England,[email protected],or
faxedto([phone].
Designationsusedbycompaniestodistinguishtheirproductsareoftenclaimedastrademarks.Allbrandnamesand
productnamesusedinthisbookaretradenames,servicemarks,trademarksorregisteredtrademarksoftheir
respectiveowners.ThePublisherisnotassociatedwithanyproductorvendormentionedinthisbook.
Thispublicationisdesignedtoprovideaccurateandauthoritativeinformationinregardtothesubjectmatter
covered.ItissoldontheunderstandingthatthePublisherisnotengagedinrenderingprofessionalservices.If
professionaladviceorotherexpertassistanceisrequired,theservicesofacompetentprofessionalshouldbesought.
OtherWileyEditorialOffices
JohnWiley&SonsInc.,111RiverStreet,Hoboken,NJ07030,USA
Jossey-Bass,989MarketStreet,SanFrancisco,CA94103-1741,USA
Wiley-VCHVerlagGmbH,Boschstr.12,D-69469Weinheim,Germany
JohnWiley&SonsAustraliaLtd,42McDougallStreet,Milton,Queensland4064,Australia
JohnWiley&Sons(Asia)PteLtd,2ClementiLoop#02-01,JinXingDistripark,Singapore129809
JohnWiley&SonsCanadaLtd,6045FreemontBlvd,Mississauga,Ontario,L5R4J3,Canada
Wileyalsopublishesitsbooksinavarietyofelectronicformats.Somecontentthatappearsinprintmaynotbe
availableinelectronicbooks.
BritishLibraryCataloguinginPublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary
ISBN-13 978-0-470-01875-0(HB)
ISBN-10 0-470-01875-5(HB)
Typesetin10/12ptTimesbyTechBooks,NewDelhi,India
PrintedandboundinGreatBritainbyAntonyRoweLtd,Chippenham,Wiltshire
Thisbookisprintedonacid-freepaperresponsiblymanufacturedfromsustainableforestry
inwhichatleasttwotreesareplantedforeachoneusedforpaperproduction.
iv
J0470018755pre October17,2006 20:27 CharCount=0
Contents
Preface xiii
Chapter1 Introduction:TheBayesianMethod,itsBenefitsandImplementation 1
1.1 TheBayesapproachanditspotentialadvantages 1
1.2 ExpressingprioruncertaintyaboutparametersandBayesianupdating 2
1.3 MCMCsamplingandinferencesfromposteriordensities 5
1.4 ThemainMCMCsamplingalgorithms 9
1.4.1 Gibbssampling 12
1.5 ConvergenceofMCMCsamples 14
1.6 Predictionsfromsampling:usingtheposteriorpredictivedensity 18
1.7 Thepresentbook 18
References 19
Chapter2 BayesianModelChoice,ComparisonandChecking 25
2.1 Introduction:theformalapproachtoBayesmodelchoiceand
averaging 25
2.2 AnalyticmarginallikelihoodapproximationsandtheBayes
informationcriterion 28
2.3 MarginallikelihoodapproximationsfromtheMCMCoutput 30
2.4 ApproximatingBayesfactorsormodelprobabilities 36
2.5 Jointspacesearchmethods 38
2.6 Directmodelaveragingbybinaryandcontinuousselectionindicators 41
2.7 Predictivemodelcomparisonviacross-validation 43
2.8 Predictivefitcriteriaandposteriorpredictivemodelchecks 46
2.9 TheDICcriterion 48
2.10 Posterioranditeration-specificcomparisonsoflikelihoodsand
penalisedlikelihoods 50
2.11 Montecarloestimatesofmodelprobabilities 52
References 57
Chapter3 TheMajorDensitiesandtheirApplication 63
3.1 Introduction 63
3.2 Univariatenormalwithknownvariance 64
3.2.1 Testinghypothesesonnormalparameters 66
J0470018755pre October17,2006 20:27 CharCount=0
vi CONTENTS
3.3 Inferenceonunivariatenormalparameters,meanandvariance
unknown 69
3.4 Heavytailedandskewdensityalternativestothenormal 71
3.5 Categoricaldistributions:binomialandbinarydata 74
3.5.1 Simulatingcontrolsthroughhistoricalexposure 76
3.6 Poissondistributionforeventcounts 79
3.7 Themultinomialanddirichletdensitiesforcategoricaland
proportionaldata 82
3.8 Multivariatecontinuousdata:multivariatenormalandtdensities 85
3.8.1 Partitioningmultivariatepriors 87
3.8.2 Themultivariatetdensity 88
3.9 Applicationsofstandarddensities:classificationrules 91
3.10 Applicationsofstandarddensities:multivariatediscrimination 98
Exercises 100
References 102
Chapter4 NormalLinearRegression,GeneralLinearModels
andLog-LinearModels 109
4.1 ThecontextforBayesianregressionmethods 109
4.2 Thenormallinearregressionmodel 111
4.2.1 Unknownregressionvariance 112
4.3 Normallinearregression:variableandmodelselection,outlier
detectionanderrorform 116
4.3.1 Otherpredictorandmodelsearchmethods 118
4.4 Bayesianridgepriorsformulticollinearity 121
4.5 Generallinearmodels 123
4.6 Binaryandbinomialregression 123
4.6.1 Priorsonregressioncoefficients 124
4.6.2 Modelchecks 126
4.7 Latentdatasamplingforbinaryregression 129
4.8 Poissonregression 132
4.8.1 Poissonregressionforcontingencytables 134
4.8.2 Log-linearmodelselection 139
4.9 Multivariateresponses 140
Exercises 143
References 146
Chapter5 HierarchicalPriorsforPoolingStrengthandOverdispersed
RegressionModelling 151
5.1 Hierarchicalpriorsforpoolingstrengthandingenerallinear
modelregression 151
5.2 Hierarchicalpriors:conjugateandnon-conjugatemixing 152
5.3 Hierarchicalpriorsfornormaldatawithapplicationsin
meta-analysis 153
5.3.1 Priorforsecond-stagevariance 155
J0470018755pre October17,2006 20:27 CharCount=0
CONTENTS vii
5.4 Poolingstrengthunderexchangeablemodelsforpoissonoutcomes 157
5.4.1 Hierarchicalpriorchoices 158
5.4.2 Parametersampling 159
5.5 Combininginformationforbinomialoutcomes 162
5.6 Randomeffectsregressionforoverdispersedcountand
binomialdata 165
5.7 Overdispersednormalregression:thescale-mixturestudentt
model 169
5.8 Thenormalmeta-analysismodelallowingforheterogeneityin
studydesignorpatientrisk 173
5.9 Hierarchicalpriorsformultinomialdata 176
5.9.1 Histogramsmoothing 177
Exercises 179
References 183
Chapter6 DiscreteMixturePriors 187
6.1 Introduction:therelevanceandapplicabilityofdiscretemixtures 187
6.2 Discretemixturesofparametricdensities 188
6.2.1 Modelchoice 190
6.3 Identifiabilityconstraints 191
6.4 Hurdleandzero-inflatedmodelsfordiscretedata 195
6.5 Regressionmixturesforheterogeneoussubpopulations 197
6.6 Discretemixturescombinedwithparametricrandomeffects 200
6.7 Non-parametricmixturemodellingviadirichletprocesspriors 201
6.8 Othernon-parametricpriors 207
Exercises 212
References 216
Chapter7 MultinomialandOrdinalRegressionModels 219
7.1 Introduction:applicationswithcategoricandordinaldata 219
7.2 Multinomiallogitchoicemodels 221
7.3 Themultinomialprobitrepresentationofinterdependentchoices 224
7.4 Mixedmultinomiallogitmodels 228
7.5 Individuallevelordinalregression 230
7.6 Scoresfororderedfactorsincontingencytables 235
Exercises 237
References 238
Chapter8 TimeSeriesModels 241
8.1 Introduction:alternativeapproachestotimeseriesmodels 241
8.2 Autoregressivemodelsintheobservations 242
8.2.1 Priorsonautoregressivecoefficients 244
8.2.2 Initialconditionsaslatentdata 246
8.3 TrendstationarityintheAR1model 248
8.4 Autoregressivemovingaveragemodels 250