Bayesian Statistical Modelling, Second Edition

582 Pages·2006·6.285 MB·English

by  Peter Congdon(auth.), Walter A. Shewhart, Samuel S. Wilks(eds.)

J0470018755pre October17,2006 20:27 CharCount=0
Bayesian Statistical Modelling
Bayesian Statistical Modelling, Second Edition P. Congdon
(cid:2)C 2006 John Wiley & Sons, Ltd. ISBN: 0-470-01875-5
i
J0470018755pre October17,2006 20:27 CharCount=0
WILEYSERIESINPROBABILITYANDSTATISTICS
establishedbyWalterA.ShewhartandSamuelS.Wilks
Editors
DavidJ.Balding,PeterBloomfield,NoelA.C.Cressie,NicholasI.Fisher,
IainM.Johnstone,J.B.Kadane,GeertMolenberghs,LouiseM.Ryan,
DavidW.Scott,AdrianF.M.Smith,JozefL.Teugels
EditorsEmeriti
VicBarnett,J.StuartHunter,DavidG.Kendall
Acompletelistofthetitlesinthisseriesappearsattheendofthisvolume.
ii
J0470018755pre October17,2006 20:27 CharCount=0
Bayesian Statistical Modelling
SecondEdition
PETERCONGDON
QueenMary,UniversityofLondon,UK
iii
J0470018755pre October17,2006 20:27 CharCount=0
Copyright(cid:2)C 2006 JohnWiley&SonsLtd,TheAtrium,SouthernGate,Chichester,
WestSussexPO198SQ,England
Telephone [phone]
Email(forordersandcustomerserviceenquiries):[email protected]
VisitourHomePageonwww.wiley.com
AllRightsReserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystemortransmittedin
anyformorbyanymeans,electronic,mechanical,photocopying,recording,scanningorotherwise,exceptunder
thetermsoftheCopyright,DesignsandPatentsAct1988orunderthetermsofalicenceissuedbytheCopyright
LicensingAgencyLtd,90TottenhamCourtRoad,LondonW1T4LP,UK,withoutthepermissioninwritingofthe
Publisher.RequeststothePublishershouldbeaddressedtothePermissionsDepartment,JohnWiley&SonsLtd,
TheAtrium,SouthernGate,Chichester,WestSussexPO198SQ,England,[email protected],or
faxedto([phone].
Designationsusedbycompaniestodistinguishtheirproductsareoftenclaimedastrademarks.Allbrandnamesand
productnamesusedinthisbookaretradenames,servicemarks,trademarksorregisteredtrademarksoftheir
respectiveowners.ThePublisherisnotassociatedwithanyproductorvendormentionedinthisbook.
Thispublicationisdesignedtoprovideaccurateandauthoritativeinformationinregardtothesubjectmatter
covered.ItissoldontheunderstandingthatthePublisherisnotengagedinrenderingprofessionalservices.If
professionaladviceorotherexpertassistanceisrequired,theservicesofacompetentprofessionalshouldbesought.
OtherWileyEditorialOffices
JohnWiley&SonsInc.,111RiverStreet,Hoboken,NJ07030,USA
Jossey-Bass,989MarketStreet,SanFrancisco,CA94103-1741,USA
Wiley-VCHVerlagGmbH,Boschstr.12,D-69469Weinheim,Germany
JohnWiley&SonsAustraliaLtd,42McDougallStreet,Milton,Queensland4064,Australia
JohnWiley&Sons(Asia)PteLtd,2ClementiLoop#02-01,JinXingDistripark,Singapore129809
JohnWiley&SonsCanadaLtd,6045FreemontBlvd,Mississauga,Ontario,L5R4J3,Canada
Wileyalsopublishesitsbooksinavarietyofelectronicformats.Somecontentthatappearsinprintmaynotbe
availableinelectronicbooks.
BritishLibraryCataloguinginPublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary
ISBN-13 978-0-470-01875-0(HB)
ISBN-10 0-470-01875-5(HB)
Typesetin10/12ptTimesbyTechBooks,NewDelhi,India
PrintedandboundinGreatBritainbyAntonyRoweLtd,Chippenham,Wiltshire
Thisbookisprintedonacid-freepaperresponsiblymanufacturedfromsustainableforestry
inwhichatleasttwotreesareplantedforeachoneusedforpaperproduction.
iv
J0470018755pre October17,2006 20:27 CharCount=0
Contents
Preface xiii
Chapter1 Introduction:TheBayesianMethod,itsBenefitsandImplementation 1
1.1 TheBayesapproachanditspotentialadvantages 1
1.2 ExpressingprioruncertaintyaboutparametersandBayesianupdating 2
1.3 MCMCsamplingandinferencesfromposteriordensities 5
1.4 ThemainMCMCsamplingalgorithms 9
1.4.1 Gibbssampling 12
1.5 ConvergenceofMCMCsamples 14
1.6 Predictionsfromsampling:usingtheposteriorpredictivedensity 18
1.7 Thepresentbook 18
References 19
Chapter2 BayesianModelChoice,ComparisonandChecking 25
2.1 Introduction:theformalapproachtoBayesmodelchoiceand
averaging 25
2.2 AnalyticmarginallikelihoodapproximationsandtheBayes
informationcriterion 28
2.3 MarginallikelihoodapproximationsfromtheMCMCoutput 30
2.4 ApproximatingBayesfactorsormodelprobabilities 36
2.5 Jointspacesearchmethods 38
2.6 Directmodelaveragingbybinaryandcontinuousselectionindicators 41
2.7 Predictivemodelcomparisonviacross-validation 43
2.8 Predictivefitcriteriaandposteriorpredictivemodelchecks 46
2.9 TheDICcriterion 48
2.10 Posterioranditeration-specificcomparisonsoflikelihoodsand
penalisedlikelihoods 50
2.11 Montecarloestimatesofmodelprobabilities 52
References 57
Chapter3 TheMajorDensitiesandtheirApplication 63
3.1 Introduction 63
3.2 Univariatenormalwithknownvariance 64
3.2.1 Testinghypothesesonnormalparameters 66
J0470018755pre October17,2006 20:27 CharCount=0
vi CONTENTS
3.3 Inferenceonunivariatenormalparameters,meanandvariance
unknown 69
3.4 Heavytailedandskewdensityalternativestothenormal 71
3.5 Categoricaldistributions:binomialandbinarydata 74
3.5.1 Simulatingcontrolsthroughhistoricalexposure 76
3.6 Poissondistributionforeventcounts 79
3.7 Themultinomialanddirichletdensitiesforcategoricaland
proportionaldata 82
3.8 Multivariatecontinuousdata:multivariatenormalandtdensities 85
3.8.1 Partitioningmultivariatepriors 87
3.8.2 Themultivariatetdensity 88
3.9 Applicationsofstandarddensities:classificationrules 91
3.10 Applicationsofstandarddensities:multivariatediscrimination 98
Exercises 100
References 102
Chapter4 NormalLinearRegression,GeneralLinearModels
andLog-LinearModels 109
4.1 ThecontextforBayesianregressionmethods 109
4.2 Thenormallinearregressionmodel 111
4.2.1 Unknownregressionvariance 112
4.3 Normallinearregression:variableandmodelselection,outlier
detectionanderrorform 116
4.3.1 Otherpredictorandmodelsearchmethods 118
4.4 Bayesianridgepriorsformulticollinearity 121
4.5 Generallinearmodels 123
4.6 Binaryandbinomialregression 123
4.6.1 Priorsonregressioncoefficients 124
4.6.2 Modelchecks 126
4.7 Latentdatasamplingforbinaryregression 129
4.8 Poissonregression 132
4.8.1 Poissonregressionforcontingencytables 134
4.8.2 Log-linearmodelselection 139
4.9 Multivariateresponses 140
Exercises 143
References 146
Chapter5 HierarchicalPriorsforPoolingStrengthandOverdispersed
RegressionModelling 151
5.1 Hierarchicalpriorsforpoolingstrengthandingenerallinear
modelregression 151
5.2 Hierarchicalpriors:conjugateandnon-conjugatemixing 152
5.3 Hierarchicalpriorsfornormaldatawithapplicationsin
meta-analysis 153
5.3.1 Priorforsecond-stagevariance 155
J0470018755pre October17,2006 20:27 CharCount=0
CONTENTS vii
5.4 Poolingstrengthunderexchangeablemodelsforpoissonoutcomes 157
5.4.1 Hierarchicalpriorchoices 158
5.4.2 Parametersampling 159
5.5 Combininginformationforbinomialoutcomes 162
5.6 Randomeffectsregressionforoverdispersedcountand
binomialdata 165
5.7 Overdispersednormalregression:thescale-mixturestudentt
model 169
5.8 Thenormalmeta-analysismodelallowingforheterogeneityin
studydesignorpatientrisk 173
5.9 Hierarchicalpriorsformultinomialdata 176
5.9.1 Histogramsmoothing 177
Exercises 179
References 183
Chapter6 DiscreteMixturePriors 187
6.1 Introduction:therelevanceandapplicabilityofdiscretemixtures 187
6.2 Discretemixturesofparametricdensities 188
6.2.1 Modelchoice 190
6.3 Identifiabilityconstraints 191
6.4 Hurdleandzero-inflatedmodelsfordiscretedata 195
6.5 Regressionmixturesforheterogeneoussubpopulations 197
6.6 Discretemixturescombinedwithparametricrandomeffects 200
6.7 Non-parametricmixturemodellingviadirichletprocesspriors 201
6.8 Othernon-parametricpriors 207
Exercises 212
References 216
Chapter7 MultinomialandOrdinalRegressionModels 219
7.1 Introduction:applicationswithcategoricandordinaldata 219
7.2 Multinomiallogitchoicemodels 221
7.3 Themultinomialprobitrepresentationofinterdependentchoices 224
7.4 Mixedmultinomiallogitmodels 228
7.5 Individuallevelordinalregression 230
7.6 Scoresfororderedfactorsincontingencytables 235
Exercises 237
References 238
Chapter8 TimeSeriesModels 241
8.1 Introduction:alternativeapproachestotimeseriesmodels 241
8.2 Autoregressivemodelsintheobservations 242
8.2.1 Priorsonautoregressivecoefficients 244
8.2.2 Initialconditionsaslatentdata 246
8.3 TrendstationarityintheAR1model 248
8.4 Autoregressivemovingaveragemodels 250
J0470018755pre October17,2006 20:27 CharCount=0
viii CONTENTS
8.5 Autoregressiveerrors 253
8.6 Multivariateseries 255
8.7 Timeseriesmodelsfordiscreteoutcomes 257
8.7.1 Observation-drivenautodependence 257
8.7.2 INARmodels 258
8.7.3 Errorautocorrelation 259
8.8 Dynamiclinearmodelsandtimevaryingcoefficients 261
8.8.1 SomecommonformsofDLM 264
8.8.2 Priorsfortime-specificvariancesorinterventions 267
8.8.3 Nonlinearandnon-Gaussianstate-spacemodels 268
8.9 Modelsforvarianceevolution 273
8.9.1 ARCHandGARCHmodels 274
8.9.2 Stochasticvolatilitymodels 275
8.10 Modellingstructuralshiftsandoutliers 277
8.10.1 Markovmixturesandtransitionfunctions 279
8.11 Othernonlinearmodels 282
Exercises 285
References 288
Chapter9 ModellingSpatialDependencies 297
9.1 Introduction:implicationsofspatialdependence 297
9.2 Discretespaceregressionsformetricdata 298
9.3 Discretespatialregressionwithstructuredandunstructured
randomeffects 303
9.3.1 ProperCARpriors 306
9.4 Movingaveragepriors 311
9.5 Multivariatespatialpriorsandspatiallyvaryingregressioneffects 313
9.6 Robustmodelsfordiscontinuitiesandnon-standarderrors 317
9.7 Continuousspacemodellinginregressionandinterpolation 321
Exercises 325
References 329
Chapter10 NonlinearandNonparametricRegression 333
10.1 Approachestomodellingnonlinearity 333
10.2 Nonlinearmetricdatamodelswithknownfunctionalform 335
10.3 Box–Coxtransformationsandfractionalpolynomials 338
10.4 Nonlinearregressionthroughsplineandradialbasisfunctions 342
10.4.1 Shrinkagemodelsforsplinecoefficients 345
10.4.2 Modellinginteractioneffects 346
10.5 Applicationofstate-spacepriorsingeneraladditive
nonparametricregression 350
10.5.1 Continuouspredictorspaceprior 351
10.5.2 Discretepredictorspacepriors 353
Exercises 359
References 362
J0470018755pre October17,2006 20:27 CharCount=0
CONTENTS ix
Chapter11 MultilevelandPanelDataModels 367
11.1 Introduction:nesteddatastructures 367
11.2 Multilevelstructures 369
11.2.1 Themultilevelnormallinearmodel 369
11.2.2 Generallinearmixedmodelsfordiscreteoutcomes 370
11.2.3 Multinomialandordinalmultilevelmodels 372
11.2.4 Robustnessregardingclustereffects 373
11.2.5 Conjugateapproachesfordiscretedata 374
11.3 Heteroscedasticityinmultilevelmodels 379
11.4 Randomeffectsforcrossedfactors 381
11.5 Paneldatamodels:thenormalmixedmodelandextensions 387
11.5.1 Autocorrelatederrors 390
11.5.2 Autoregressionin y 391
11.6 Modelsforpaneldiscrete(binary,countandcategorical)
observations 393
11.6.1 Binarypaneldata 393
11.6.2 Repeatedcounts 395
11.6.3 Panelcategoricaldata 397
11.7 Growthcurvemodels 400
11.8 Dynamicmodelsforlongitudinaldata:poolingstrengthover
unitsandtimes 403
11.9 Areaapcandspatiotemporalmodels 407
11.9.1 Age–perioddata 408
11.9.2 Area–timedata 409
11.9.3 Age–area–perioddata 409
11.9.4 Interactionpriors 410
Exercises 413
References 418
Chapter12 LatentVariableandStructuralEquationModels
forMultivariateData 425
12.1 Introduction:latenttraitsandlatentclasses 425
12.2 FactoranalysisandSEMSforcontinuousdata 427
12.2.1 Identifiabilityconstraintsinlatenttrait(factor
analysis)models 429
12.3 Latentclassmodels 433
12.3.1 Localdependence 437
12.4 FactoranalysisandSEMSformultivariatediscretedata 441
12.5 Nonlinearfactormodels 447
Exercises 450
References 452
Chapter13 SurvivalandEventHistoryAnalysis 457
13.1 Introduction 457
13.2 Parametricsurvivalanalysisincontinuoustime 458
J0470018755pre October17,2006 20:27 CharCount=0
x CONTENTS
13.2.1 Censoredobservations 459
13.2.2 Formsofparametrichazardandsurvivalcurves 460
13.2.3 Modellingcovariateimpactsandtimedependencein
thehazardrate 461
13.3 Acceleratedhazardparametricmodels 464
13.4 Countingprocessmodels 466
13.5 Semiparametrichazardmodels 469
13.5.1 Priorsforthebaselinehazard 470
13.5.2 Gammaprocessprioroncumulativehazard 472
13.6 Competingrisk-continuoustimemodels 475
13.7 Variationsinproneness:modelsforfrailty 477
13.8 Discretetimesurvivalmodels 482
Exercises 486
References 487
Chapter14 MissingDataModels 493
14.1 Introduction:typesofmissingness 493
14.2 Selectionandpatternmixturemodelsforthejoint
data-missingnessdensity 494
14.3 Sharedrandomeffectandcommonfactormodels 498
14.4 Missingpredictordata 500
14.5 Multipleimputation 503
14.6 Categoricalresponsedatawithpossiblenon-random
missingness:hierarchicalandregressionmodels 506
14.6.1 Hierarchicalmodelsforresponseandnon-response
bystrata 506
14.6.2 Regressionframeworks 510
14.7 Missingnesswithmixturesofcontinuousandcategorical
data 516
14.8 Missingcellsincontingencytables 518
14.8.1 Ecologicalinference 519
Exercises 526
References 529
Chapter15 MeasurementError,SeeminglyUnrelatedRegressions,and
SimultaneousEquations 533
15.1 Introduction 533
15.2 Measurementerrorinbothpredictorsandresponseinnormal
linearregression 533
15.2.1 PriorinformationonXoritsdensity 535
15.2.2 Measurementerroringenerallinearmodels 537
15.3 Misclassificationofcategoricalvariables 541
15.4 Simultaneousequationsandinstrumentsforendogenous
variables 546