Computer Age Statistical Inference: Algorithms, Evidence, and Data Science

495 Pages·2016·4.14 MB·English

by  Bradley Efron

“Howandwhyiscomputationalstatisticstakingovertheworld?Inthisseriouswork
ofsynthesisthatisalsofuntoread,EfronandHastie,twopioneersintheintegrationof
parametricandnonparametricstatisticalideas,givetheirtakeontheunreasonableeffec-
tivenessofstatisticsandmachinelearninginthecontextofaseriesofclear,historically
informedexamples.”
—AndrewGelman,ColumbiaUniversity
“Thisunusualbookdescribesthenatureofstatisticsbydisplayingmultipleexamples
ofthewaythefieldhasevolvedoverthepast60years,asithasadaptedtotherapid
increaseinavailablecomputingpower.Theauthors’perspectiveissummarizednicely
when they say, ‘Very roughly speaking, algorithms are what statisticians do, while
inference says why they do them.’ The book explains this ‘why’; that is, it explains
the purpose and progress of statistical research, through a close look at many major
methods,methodstheauthorsthemselveshaveadvancedandstudiedatgreatlength.
Bothenjoyableandenlightening,ComputerAgeStatisticalInferenceiswrittenespe-
ciallyforthosewhowanttohearthebigideas,andseetheminstantiatedthroughthe
essentialmathematicsthatdefinesstatisticalanalysis.Itmakesagreatsupplementto
thetraditionalcurriculaforbeginninggraduatestudents.”
—RobKass,CarnegieMellonUniversity
“This is a terrific book. It gives a clear, accessible, and entertaining account of the
interplaybetweentheoryandmethodologicaldevelopmentthathasdrivenstatisticsin
thecomputerage.Theauthorssucceedbrilliantlyinlocatingcontemporaryalgorithmic
methodologiesforanalysisof‘bigdata’withintheframeworkofestablishedstatistical
theory.”
—AlastairYoung,ImperialCollegeLondon
“Thisisaguidedtourofmodernstatisticsthatemphasizestheconceptualandcompu-
tationaladvancesofthelastcentury.Authoredbytwomastersofthefield,itoffersjust
therightmixofmathematicalanalysisandinsightfulcommentary.”
—HalVarian,Google
“EfronandHastieguideusthroughthemazeofbreakthroughstatisticalmethodologies
followingthecomputingevolution:whytheyweredeveloped,theirproperties,andhow
theyareused.Highlightingtheirorigins,thebookhelpsusunderstandeachmethod’s
roles in inference and/or prediction. The inference–prediction distinction maintained
throughoutthebookisawelcomeandimportantnoveltyinthelandscapeofstatistics
books.”
—GalitShmueli,NationalTsingHuaUniversity
“A masterful guide to how the inferential bases of classical statistics can provide a
principleddisciplinaryframeforthedatascienceofthetwenty-firstcentury.”
—StephenStigler,UniversityofChicago,authorof
SevenPillarsofStatisticalWisdom
“ComputerAgeStatisticalInferenceoffersarefreshingviewofmodernstatistics.Algo-
rithmicsareputonequalfootingwithintuition,properties,andtheabstractarguments
behindthem.Themethodscoveredareindispensabletopracticingstatisticalanalysts
intoday’sbigdataandbigcomputinglandscape.”
—RobertGramacy,TheUniversityofChicagoBoothSchoolofBusiness
“Everyaspiringdatascientistshouldcarefullystudythisbook,useitasareference,and
carry it with them everywhere. The presentation through the two-and-a-half-century
historyofstatisticalinferenceprovidesinsightintothedevelopmentofthediscipline,
puttingdatascienceinitshistoricalplace.”
—MarkGirolami,ImperialCollegeLondon
“Efron and Hastie are two immensely talented and accomplished scholars who have
managedtobrilliantlyweavethefiberof250yearsofstatisticalinferenceintothemore
recent historical mechanization of computing. This book provides the reader with a
mid-leveloverviewofthelast60-someyearsbydetailingthenuancesofastatistical
communitythat,historically,hasbeenself-segregatedintocampsofBayes,frequentist,
andFisheryetinmorerecentyearshasbeenunifiedbyadvancesincomputing.Whatis
lefttobeexploredistheemergenceof,androlethat,bigdatatheorywillhaveinbridg-
ingthegapbetweendatascienceandstatisticalmethodology.Whatevertheoutcome,
theauthorsprovideavisionofhigh-speedcomputinghavingtremendouspotentialto
enablethecontributionsofstatisticalinferencetowardmethodologiesthataddressboth
globalandsocietalissues.”
—RebeccaDoerge,CarnegieMellonUniversity
“Inthisbook,twomastersofmodernstatisticsgiveaninsightfultouroftheintertwined
worldsofstatisticsandcomputation.Throughaseriesofimportanttopics,Efronand
Hastieilluminatehowmodernmethodsforpredictingandunderstandingdataarerooted
inbothstatisticalandcomputationalthinking.Theyshowhowtheriseofcomputational
powerhastransformedtraditionalmethodsandquestions,andhowithaspointedusto
newwaysofthinkingaboutstatistics.”
—DavidBlei,ColumbiaUniversity
Absolutelybrilliant.Thisbeautifullywrittencompendiumreviewsmanybigstatistical
ideas,includingtheauthors’own.Amustforanyoneengagedcreativelyinstatistics
andthedatasciences,forrepeateduse.EfronandHastiedemonstratetheever-growing
powerofstatisticalreasoning,past,present,andfuture.
—CarlMorris,HarvardUniversity
ComputerAgeStatisticalInference
Thetwenty-firstcenturyhasseenabreathtakingexpansionofstatisticalmethodology,
bothinscopeandininfluence.“Bigdata,”“datascience,”and“machinelearning”have
becomefamiliartermsinthenews,asstatisticalmethodsarebroughttobearuponthe
enormousdatasetsofmodernscienceandcommerce.Howdidwegethere?Andwhere
arewegoing?
Thisbooktakesusonanexhilaratingjourneythroughtherevolutionindataanaly-
sisfollowingtheintroductionofelectroniccomputationinthe1950s.Beginningwith
classical inferential theories – Bayesian, frequentist, Fisherian – individual chapters
take up a series of influential topics: survival analysis, logistic regression, empirical
Bayes, the jackknife and bootstrap, random forests, neural networks, Markov chain
MonteCarlo,inferenceaftermodelselection,anddozensmore.Thedistinctlymodern
approach integrates methodology and algorithms with statistical inference. The book
endswithspeculationonthefuturedirectionofstatisticsanddatascience.
BRADLEY EFRONisMaxH.SteinProfessor,ProfessorofStatistics,andProfessorof
BiomedicalDataScienceatStanfordUniversity.Hehasheldvisitingfacultyappoint-
ments at Harvard, UC Berkeley, and Imperial College London. Efron has worked
extensivelyontheoriesofstatisticalinference,andistheinventorofthebootstrapsam-
plingtechnique.HereceivedtheNationalMedalofSciencein2005andtheGuyMedal
inGoldoftheRoyalStatisticalSocietyin2014.
TREVOR HASTIEisJohnA.OverdeckProfessor,ProfessorofStatistics,andProfes-
sorofBiomedicalDataScienceatStanfordUniversity.HeiscoauthorofElementsof
StatisticalLearning,akeytextinthefieldofmoderndataanalysis.Heisalsoknownfor
hisworkongeneralizedadditivemodelsandprincipalcurves,andforhiscontributions
totheRcomputingenvironment.HastiewasawardedtheEmmanuelandCarolParzen
prizeforStatisticalInnovationin2014.
INSTITUTE OF MATHEMATICAL STATISTICS
MONOGRAPHS
EditorialBoard
D.R.Cox(UniversityofOxford)
B.Hambly(UniversityofOxford)
S.Holmes(StanfordUniversity)
J.Wellner(UniversityofWashington)
IMSMonographsareconciseresearchmonographsofhighqualityonanybranchof
statistics or probability of sufficient interest to warrant publication as books. Some
concern relatively traditional topics in need of up-to-date assessment. Others are on
emergingthemes.Inallcasestheobjectiveistoprovideabalancedviewofthefield.
OtherBooksintheSeries
1. Large-ScaleInference,byBradleyEfron
2. NonparametricInferenceonManifolds,byAbhishekBhattacharyaandRabi
Battacharya
3. TheSkew-NormalandRelatedFamilies,byAdelchiAzzalini
4. Case-ControlStudies,byRuthH.KeoghandD.R.Cox
5. ComputerAgeStatisticalInference,byBradleyEfronandTrevorHastie
Computer Age Statistical Inference
Algorithms, Evidence, and Data Science
BRADLEY EFRON
StanfordUniversity,California
TREVOR HASTIE
StanfordUniversity,California
32AvenueoftheAmericas,NewYorkNY10013-2473,USA
CambridgeUniversityPressispartoftheUniversityofCambridge.
ItfurtherstheUniversity’smissionbydisseminatingknowledgeinthepursuitof
education,learningandresearchatthehighestinternationallevelsofexcellence.
www.cambridge.org
Informationonthistitle:www.cambridge.org/9781107149892
(cid:2)c BradleyEfronandTrevorHastie2016
Thispublicationisincopyright.Subjecttostatutoryexception
andtotheprovisionsofrelevantcollectivelicensingagreements,
noreproductionofanypartmaytakeplacewithoutthewritten
permissionofCambridgeUniversityPress.
Firstpublished2016
PrintedintheUnitedKingdombyClays,StIvesplc
AcataloguerecordforthispublicationisavailablefromtheBritishLibrary
ISBN978-1-107-14989-2Hardback
CambridgeUniversityPresshasnoresponsibilityforthepersistenceoraccuracyof
URLsforexternalorthird-partyinternetwebsitesreferredtointhispublication,
anddoesnotguaranteethatanycontentonsuchwebsitesis,orwillremain,
accurateorappropriate.
ToDonnaandLynda

Contents
Preface xv
Acknowledgments xviii
Notation xix
PartI ClassicStatisticalInference 1
1 AlgorithmsandInference 3
1.1 ARegressionExample 4
1.2 HypothesisTesting 8
1.3 Notes 11
2 FrequentistInference 12
2.1 FrequentisminPractice 14
2.2 FrequentistOptimality 18
2.3 NotesandDetails 20
3 BayesianInference 22
3.1 TwoExamples 24
3.2 UninformativePriorDistributions 28
3.3 FlawsinFrequentistInference 30
3.4 ABayesian/FrequentistComparisonList 33
3.5 NotesandDetails 36
4 FisherianInferenceandMaximumLikelihoodEstimation 38
4.1 LikelihoodandMaximumLikelihood 38
4.2 FisherInformationandtheMLE 41
4.3 ConditionalInference 45
4.4 PermutationandRandomization 49
4.5 NotesandDetails 51
5 ParametricModelsandExponentialFamilies 53
ix