Table Of ContentThis page intentionally left blank
Bayesian Nonparametrics
Bayesiannonparametricsworks–theoretically,computationally.Thetheoryprovides
highlyflexiblemodelswhosecomplexitygrowsappropriatelywiththeamountofdata.
Computationalissues,thoughchallenging,arenolongerintractable.Allthatisneeded
isanentrypoint:thisintelligentbookistheperfectguidetowhatcanseemaforbidding
landscape.
TutorialchaptersbyGhosal,LijoiandPru¨nster,TehandJordan,andDunsonadvance
fromtheory,tobasicmodelsandhierarchicalmodeling,toapplicationsandimplemen-
tation,particularlyincomputerscienceandbiostatistics.Thesearecomplementedby
companionchaptersbytheeditorsandGriffinandQuintana,providingadditionalmod-
els,examiningcomputationalissues,identifyingfuturegrowthareas,andgivinglinks
torelatedtopics.
This coherent text gives ready access both to underlying principles and to state-
of-the-art practice. Specific examples are drawn from information retrieval, neuro-
linguisticprogramming,machinevision,computationalbiology,biostatistics,andbio-
informatics.
Nils Lid Hjort isProfessorofMathematicalStatisticsintheDepartmentofMath-
ematicsattheUniversityofOslo.
Chris Holmes is Professor of Biostatistics in the Department of Statistics at the
UniversityofOxford.
Peter Mu¨ ller isProfessorintheDepartmentofBiostatisticsattheUniversityof
TexasM.D.AndersonCancerCenter.
Stephen G. Walker is Professor of Statistics in the Institute of Mathematics,
StatisticsandActuarialScienceattheUniversityofKent,Canterbury.
CAMBRIDGE SERIES IN STATISTICAL AND
PROBABILISTIC MATHEMATICS
EditorialBoard
Z.Ghahramani(DepartmentofEngineering,UniversityofCambridge)
R.Gill(MathematicalInstitute,LeidenUniversity)
F.P.Kelly(DepartmentofPureMathematicsandMathematicalStatistics,
UniversityofCambridge)
B.D.Ripley(DepartmentofStatistics,UniversityofOxford)
S.Ross(DepartmentofIndustrialandSystemsEngineering,UniversityofSouthernCalifornia)
B.W.Silverman(StPeter’sCollege,Oxford)
M.Stein(DepartmentofStatistics,UniversityofChicago)
Thisseriesofhigh-qualityupper-divisiontextbooksandexpositorymonographscoversallaspects
ofstochasticapplicablemathematics.Thetopicsrangefrompureandappliedstatisticstoprob-
ability theory, operations research, optimization, and mathematical programming. The books
contain clear presentations of new developments in the field and also of the state of the art in
classicalmethods.Whileemphasizingrigoroustreatmentoftheoreticalmethods,thebooksalso
containapplicationsanddiscussionsofnewtechniquesmadepossiblebyadvancesincomputa-
tionalpractice.
Acompletelistofbooksintheseriescanbefoundat
http://www.cambridge.org/uk/series/sSeries.asp?code=CSPM
Recenttitlesincludethefollowing:
6. EmpiricalProcessesinM-Estimation,bySaravandeGeer
7. NumericalMethodsofStatistics,byJohnF.Monahan
8. AUser’sGuidetoMeasureTheoreticProbability,byDavidPollard
9. TheEstimationandTrackingofFrequency,byB.G.QuinnandE.J.Hannan
10. DataAnalysisandGraphicsusingR(2ndEdition),byJohnMaindonaldandJohnBraun
11. StatisticalModels,byA.C.Davison
12. SemiparametricRegression,byDavidRuppert,M.P.WandandR.J.Carroll
13. ExercisesinProbability,byLo¨ıcChaumontandMarcYor
14. StatisticalAnalysisofStochasticProcessesinTime,byJ.K.Lindsey
15. MeasureTheoryandFiltering,byLakhdarAggounandRobertElliott
16. EssentialsofStatisticalInference,byG.A.YoungandR.L.Smith
17. ElementsofDistributionTheory,byThomasA.Severini
18. StatisticalMechanicsofDisorderedSystems,byAntonBovier
19. TheCoordinate-FreeApproachtoLinearModels,byMichaelJ.Wichura
20. RandomGraphDynamics,byRickDurrett
21. Networks,byPeterWhittle
22. SaddlepointApproximationswithApplications,byRonaldW.Butler
23. AppliedAsymptotics,byA.R.Brazzale,A.C.DavisonandN.Reid
24. RandomNetworksforCommunication,byMassimoFranceschettiandRonaldMeester
25. DesignofComparativeExperiments,byR.A.Bailey
26. SymmetryStudies,byMarlosA.G.Viana
27. ModelSelectionandModelAveraging,byGerdaClaeskensandNilsLidHjort
28. BayesianNonparametrics,editedbyNilsLidHjortetal.
29. FiniteandLargeSampleStatisticalTheory(2ndEdition),byPranabK.Sen,
JulioM.SingerandAntonioC.Pedrosa-de-Lima
30. BrownianMotion,byPeterMo¨rtersandYuvalPeres
Bayesian Nonparametrics
Editedby
Nils Lid Hjort
UniversityofOslo
Chris Holmes
UniversityofOxford
Peter Mu¨ller
UniversityofTexas
M.D.AndersonCancerCenter
Stephen G. Walker
UniversityofKent
CAMBRIDGE UNIVERSITY PRESS
Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore,
São Paulo, Delhi, Dubai, Tokyo
Cambridge University Press
The Edinburgh Building, Cambridge CB2 8RU, UK
Published in the United States of America by Cambridge University Press, New York
www.cambridge.org
Information on this title: www.cambridge.org/9780521513463
© Cambridge University Press 2010
This publication is in copyright. Subject to statutory exception and to the
provision of relevant collective licensing agreements, no reproduction of any part
may take place without the written permission of Cambridge University Press.
First published in print format 2010
ISBN-13 978-0-511-67536-2 eBook (NetLibrary)
ISBN-13 978-0-521-51346-3 Hardback
Cambridge University Press has no responsibility for the persistence or accuracy
of urls for external or third-party internet websites referred to in this publication,
and does not guarantee that any content on such websites is, or will remain,
accurate or appropriate.
Contents
Listofcontributors pageviii
AninvitationtoBayesiannonparametrics
NilsLidHjort,ChrisHolmes,PeterMu¨ller
andStephenG.Walker 1
1 Bayesiannonparametricmethods: motivationandideas
StephenG.Walker 22
1.1 Introduction 22
1.2 Bayesianchoices 24
1.3 Decisiontheory 26
1.4 Asymptotics 27
1.5 Generalposteriorinference 29
1.6 Discussion 33
References 33
2 TheDirichletprocess,relatedpriorsandposterior
asymptotics
SubhashisGhosal 35
2.1 Introduction 35
2.2 TheDirichletprocess 36
2.3 PriorsrelatedtotheDirichletprocess 46
2.4 Posteriorconsistency 49
2.5 Convergenceratesofposteriordistributions 60
2.6 Adaptationandmodelselection 67
2.7 Bernshteˇın–vonMisestheorems 71
2.8 Concludingremarks 74
References 76
v
vi Contents
3 ModelsbeyondtheDirichletprocess
AntonioLijoiandIgorPru¨nster 80
3.1 Introduction 80
3.2 Modelsforsurvivalanalysis 86
3.3 Generalclassesofdiscretenonparametricpriors 99
3.4 Modelsfordensityestimation 114
3.5 Randommeans 126
3.6 Concludingremarks 129
References 130
4 Furthermodelsandapplications
NilsLidHjort 137
4.1 Betaprocessesforsurvivalandeventhistorymodels 137
4.2 Quantileinference 144
4.3 Shapeanalysis 148
4.4 Timeserieswithnonparametriccorrelationfunction 150
4.5 Concludingremarks 152
References 155
5 HierarchicalBayesiannonparametricmodelswith
applications
YeeWhyeTehandMichaelI.Jordan 158
5.1 Introduction 158
5.2 HierarchicalDirichletprocesses 160
5.3 HiddenMarkovmodelswithinfinitestatespaces 171
5.4 HierarchicalPitman–Yorprocesses 177
5.5 ThebetaprocessandtheIndianbuffetprocess 184
5.6 Semiparametricmodels 193
5.7 InferenceforhierarchicalBayesiannonparametricmodels 195
5.8 Discussion 202
References 203
6 ComputationalissuesarisinginBayesiannonparametric
hierarchicalmodels
JimGriffinandChrisHolmes 208
6.1 Introduction 208
6.2 Constructionoffinite-dimensionalmeasuresonobservables 209
6.3 Recent advances in computation for Dirichlet processmixture
models 211
References 221
Contents vii
7 NonparametricBayesapplicationstobiostatistics
DavidB.Dunson 223
7.1 Introduction 223
7.2 HierarchicalmodelingwithDirichletprocesspriors 224
7.3 NonparametricBayesfunctionaldataanalysis 236
7.4 Localborrowingofinformationandclustering 245
7.5 Borrowinginformationacrossstudiesandcenters 248
7.6 Flexiblemodelingofconditionaldistributions 250
7.7 Bioinformatics 260
7.8 Nonparametrichypothesistesting 265
7.9 Discussion 267
References 268
8 MorenonparametricBayesianmodelsforbiostatistics
PeterMu¨llerandFernandoQuintana 274
8.1 Introduction 274
8.2 Randompartitions 275
8.3 Po´lyatrees 277
8.4 MoreDDPmodels 279
8.5 Otherdataformats 283
8.6 AnRpackagefornonparametricBayesianinference 286
8.7 Discussion 289
References 290
Authorindex 292
Subjectindex 297
Contributors
DavidB.Dunson AntonioLijoi
InstituteofStatistics DepartmentofEconomics
andDecisionSciences andQuantitativeMethods
DukeUniversity UniversityofPavia
Durham,NC27708-0251,USA 27100Pavia,Italy
PeterMu¨ller
SubhashisGhosal
DepartmentofBiostatistics
DepartmentofStatistics
M.D.AndersonCancerCenter
NorthCarolinaStateUniversity
UniversityofTexas
Raleigh,NC27695,USA
Houston,TX77030-4009,USA
JimGriffin
IgorPru¨nster
InstituteofMathematics,Statistics
DepartmentofStatistics
andActuarialScience
andAppliedMathematics
UniversityofKent
UniversityofTurin
CanterburyCT27NZ,UK
10122Turin,Italy
NilsLidHjort FernandoQuintana
DepartmentofMathematics DepartmentofStatistics
UniversityofOslo PontificalCatholicUniversityofChile
N-0316Oslo,Norway 3542000Santiago,Chile
YeeWhyeTeh
ChrisHolmes
GatsbyComputational
OxfordCentreforGeneFunction
NeuroscienceUnit
UniversityofOxford
UniversityCollegeLondon
OxfordOX13QB,UK
LondonWC1N3AR,UK
MichaelI.Jordan StephenG.Walker
DepartmentofElectricalEngineering InstituteofMathematics,Statistics
andComputerScience andActuarialScience
UniversityofCalifornia,Berkeley UniversityofKent
Berkeley,CA94720-1776,USA CanterburyCT27NZ,UK
viii
