Regression Modeling and Data Analysis with Applications in R

363 Pages·2020·7.649 MB·English

by  Samprit Chatterjee, Jeffrey S. Simonoff

(cid:2)
Handbook of Regression Analysis
With Applications in R
(cid:2) (cid:2)
(cid:2)
(cid:2)
WILEYSERIESINPROBABILITYANDSTATISTICS
EstablishedbyWALTERA.SHEWHARTandSAMUELS.WILKS
Editors
DavidJ.Balding,NoelA.C.Cressie,GarrettM.Fitzmaurice,Harvey
Goldstein,GeertMolenberghs,DavidW.Scott,AdrianF.M.Smith,and
RueyS.Tsay
EditorsEmeriti
VicBarnett,RalphA.Bradley,J.StuartHunter,J.B.Kadane,DavidG.
Kendall,andJozefL.Teugels
Acompletelistofthetitlesinthisseriesappearsattheendofthisvolume.
(cid:2) (cid:2)
(cid:2)
(cid:2)
Handbook of Regression
Analysis With Applications
in R
Second Edition
Samprit Chatterjee
NewYorkUniversity,NewYork,USA
(cid:2) Jeffrey S. Simonoff (cid:2)
NewYorkUniversity,NewYork,USA
(cid:2)
(cid:2)
Thissecondeditionfirstpublished2020
©2020JohnWiley&Sons,Inc
EditionHistory
Wiley-Blackwell(1e,2013)
Allrightsreserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,ortransmitted,in
anyformorbyanymeans,electronic,mechanical,photocopying,recordingorotherwise,exceptaspermittedby
law.Adviceonhowtoobtainpermissiontoreusematerialfromthistitleisavailableathttp://www.wiley.com/go/
permissions.
TherightofSampritChatterjeeandJefferyS.Simonofftobeidentifiedastheauthorsofthisworkhasbeen
assertedinaccordancewithlaw.
RegisteredOffice
JohnWiley&Sons,Inc.,111RiverStreet,Hoboken,NJ07030,USA
EditorialOffice
111RiverStreet,Hoboken,NJ07030,USA
Fordetailsofourglobaleditorialoffices,customerservices,andmoreinformationaboutWileyproductsvisitus
atwww.wiley.com.
Wileyalsopublishesitsbooksinavarietyofelectronicformatsandbyprint-on-demand.Somecontentthat
appearsinstandardprintversionsofthisbookmaynotbeavailableinotherformats.
LimitofLiability/DisclaimerofWarranty
Whilethepublisherandauthorshaveusedtheirbesteffortsinpreparingthiswork,theymakenorepresentations
orwarrantieswithrespecttotheaccuracyorcompletenessofthecontentsofthisworkandspecificallydisclaimall
warranties,includingwithoutlimitationanyimpliedwarrantiesofmerchantabilityorfitnessforaparticular
purpose.Nowarrantymaybecreatedorextendedbysalesrepresentatives,writtensalesmaterialsorpromotional
statementsforthiswork.Thefactthatanorganization,website,orproductisreferredtointhisworkasacitation
and/orpotentialsourceoffurtherinformationdoesnotmeanthatthepublisherandauthorsendorsethe
informationorservicestheorganization,website,orproductmayprovideorrecommendationsitmaymake.This
(cid:2) workissoldwiththeunderstandingthatthepublisherisnotengagedinrenderingprofessionalservices.The (cid:2)
adviceandstrategiescontainedhereinmaynotbesuitableforyoursituation.Youshouldconsultwithaspecialist
whereappropriate.Further,readersshouldbeawarethatwebsiteslistedinthisworkmayhavechangedor
disappearedbetweenwhenthisworkwaswrittenandwhenitisread.Neitherthepublishernorauthorsshallbe
liableforanylossofprofitoranyothercommercialdamages,includingbutnotlimitedtospecial,incidental,
consequential,orotherdamages.
LibraryofCongressCataloging-in-PublicationData
Names:Chatterjee,Samprit,1938-author.|Simonoff,JeffreyS.,author.
Title:HandbookofregressionanalysiswithapplicationsinR/Professor
SampritChatterjee,NewYorkUniversity,ProfessorJeffreyS.Simonoff,
NewYorkUniversity.
Othertitles:Handbookofregressionanalysis
Description:Secondedition.|Hoboken,NJ:Wiley,2020.|Series:Wiley
seriesinprobabilityandstatistics|Revisededitionof:Handbookof
regressionanalysis.2013.|Includesbibliographicalreferencesand
index.
Identifiers:LCCN2020006580(print)|LCCN2020006581(ebook)|ISBN
9781119392378(hardback)|ISBN9781119392477(adobepdf)|ISBN
9781119392484(epub)
Subjects:LCSH:Regressionanalysis--Handbooks,manuals,etc.|R(Computer
programlanguage)
Classification:LCCQA278.2.C4982020(print)|LCCQA278.2(ebook)|
DDC519.5/36--dc23
LCrecordavailableathttps://lccn.loc.gov/2020006580
LCebookrecordavailableathttps://lccn.loc.gov/2020006581
CoverDesign:Wiley
CoverImage:©DmitriyRybin/Shutterstock
Setin10.82/12ptAGaramondProbySPiGlobal,Chennai,India
PrintedintheUnitedStatesofAmerica
10 9 8 7 6 5 4 3 2 1
(cid:2)
(cid:2)
Dedicated to everyone who labors in the field
of statistics, whether they are students,
teachers, researchers, or data analysts.
(cid:2) (cid:2)
(cid:2)
(cid:2)
Contents
PrefacetotheSecondEdition xv
PrefacetotheFirstEdition xix
Part I
The Multiple Linear Regression Model
1 MultipleLinearRegression 3
1.1 Introduction 3
1.2 ConceptsandBackgroundMaterial 4
1.2.1 TheLinearRegressionModel 4
1.2.2 EstimationUsingLeastSquares 5
1.2.3 Assumptions 8
1.3 Methodology 9
(cid:2) (cid:2)
1.3.1 InterpretingRegressionCoefficients 9
1.3.2 MeasuringtheStrengthoftheRegression
Relationship 10
1.3.3 HypothesisTestsandConfidenceIntervals
forβ 12
1.3.4 FittedValuesandPredictions 13
1.3.5 CheckingAssumptionsUsingResidualPlots 14
1.4 Example—EstimatingHomePrices 15
1.5 Summary 19
2 ModelBuilding 23
2.1 Introduction 23
2.2 ConceptsandBackgroundMaterial 24
2.2.1 UsingHypothesisTeststoCompareModels 24
2.2.2 Collinearity 26
2.3 Methodology 29
2.3.1 ModelSelection 29
2.3.2 Example—EstimatingHomePrices
(continued) 31
2.4 IndicatorVariablesandModelingInteractions 38
2.4.1 Example—ElectronicVotingandthe2004
PresidentialElection 40
2.5 Summary 46
vii
(cid:2)
(cid:2)
viii CONTENTS
Part II
Addressing Violations of Assumptions
3 DiagnosticsforUnusualObservations 53
3.1 Introduction 53
3.2 ConceptsandBackgroundMaterial 54
3.3 Methodology 56
3.3.1 ResidualsandOutliers 56
3.3.2 LeveragePoints 57
3.3.3 InfluentialPointsandCook’sDistance 58
3.4 Example—EstimatingHomePrices(continued) 60
3.5 Summary 63
4 TransformationsandLinearizableModels 67
4.1 Introduction 67
4.2 ConceptsandBackgroundMaterial:TheLog-LogModel 69
4.3 ConceptsandBackgroundMaterial:SemilogModels 69
4.3.1 LoggedResponseVariable 70
4.3.2 LoggedPredictorVariable 70
4.4 Example—PredictingMovieGrossesAfterOneWeek 71
4.5 Summary 77
(cid:2) (cid:2)
5 TimeSeriesDataandAutocorrelation 79
5.1 Introduction 79
5.2 ConceptsandBackgroundMaterial 81
5.3 Methodology:IdentifyingAutocorrelation 83
5.3.1 TheDurbin-WatsonStatistic 83
5.3.2 TheAutocorrelationFunction(ACF) 84
5.3.3 ResidualPlotsandtheRunsTest 85
5.4 Methodology:AddressingAutocorrelation 86
5.4.1 DetrendingandDeseasonalizing 86
5.4.2 Example—e-CommerceRetailSales 87
5.4.3 LaggingandDifferencing 93
5.4.4 Example—StockIndexes 94
5.4.5 GeneralizedLeastSquares(GLS):
TheCochrane-OrcuttProcedure 99
5.4.6 Example—TimeIntervalsBetweenOldFaithful
GeyserEruptions 100
5.5 Summary 104
(cid:2)
(cid:2)
CONTENTS ix
Part III
Categorical Predictors
6 AnalysisofVariance 109
6.1 Introduction 109
6.2 ConceptsandBackgroundMaterial 110
6.2.1 One-WayANOVA 110
6.2.2 Two-WayANOVA 111
6.3 Methodology 113
6.3.1 CodingsforCategoricalPredictors 113
6.3.2 MultipleComparisons 118
6.3.3 Levene’sTestandWeightedLeastSquares 120
6.3.4 MembershipinMultipleGroups 123
6.4 Example—DVDSalesofMovies 125
6.5 Higher-WayANOVA 130
6.6 Summary 132
7 AnalysisofCovariance 135
7.1 Introduction 135
7.2 Methodology 136
7.2.1 ConstantShiftModels 136
(cid:2) 7.2.2 VaryingSlopeModels 137 (cid:2)
7.3 Example—InternationalGrossesofMovies 137
7.4 Summary 142
Part IV
Non-Gaussian Regression Models
8 LogisticRegression 145
8.1 Introduction 145
8.2 ConceptsandBackgroundMaterial 147
8.2.1 TheLogitResponseFunction 148
8.2.2 BernoulliandBinomialRandomVariables 149
8.2.3 ProspectiveandRetrospectiveDesigns 149
8.3 Methodology 152
8.3.1 MaximumLikelihoodEstimation 152
8.3.2 Inference,ModelComparison,andModel
Selection 153
(cid:2)
(cid:2)
x CONTENTS
8.3.3 Goodness-of-Fit 155
8.3.4 MeasuresofAssociationandClassification
Accuracy 157
8.3.5 Diagnostics 159
8.4 Example—SmokingandMortality 159
8.5 Example—ModelingBankruptcy 163
8.6 Summary 168
9 MultinomialRegression 173
9.1 Introduction 173
9.2 ConceptsandBackgroundMaterial 174
9.2.1 NominalResponseVariable 174
9.2.2 OrdinalResponseVariable 176
9.3 Methodology 178
9.3.1 Estimation 178
9.3.2 Inference,ModelComparisons,andStrengthof
Fit 178
9.3.3 LackofFitandViolationsof
Assumptions 180
9.4 Example—CityBondRatings 180
9.5 Summary 184
(cid:2) (cid:2)
10 CountRegression 187
10.1 Introduction 187
10.2 ConceptsandBackgroundMaterial 188
10.2.1 ThePoissonRandomVariable 188
10.2.2 GeneralizedLinearModels 189
10.3 Methodology 190
10.3.1 EstimationandInference 190
10.3.2 Offsets 191
10.4 OverdispersionandNegativeBinomialRegression 192
10.4.1 Quasi-likelihood 192
10.4.2 NegativeBinomialRegression 193
10.5 Example—UnprovokedSharkAttacksinFlorida 194
10.6 OtherCountRegressionModels 201
10.7 PoissonRegressionandWeightedLeastSquares 203
10.7.1 Example—InternationalGrossesofMovies
(continued) 204
10.8 Summary 206
11 ModelsforTime-to-Event(Survival)Data 209
11.1 Introduction 210
11.2 ConceptsandBackgroundMaterial 211
11.2.1 TheNatureofSurvivalData 211
11.2.2 AcceleratedFailureTimeModels 212
11.2.3 TheProportionalHazardsModel 214
(cid:2)
(cid:2)
CONTENTS xi
11.3 Methodology 214
11.3.1 TheKaplan-MeierEstimatorandtheLog-Rank
Test 214
11.3.2 Parametric(Likelihood)Estimation 219
11.3.3 Semiparametric(PartialLikelihood)
Estimation 221
11.3.4 TheBuckley-JamesEstimator 223
11.4 Example—TheSurvivalofBroadwayShows
(continued) 223
11.5 Left-Truncated/Right-CensoredDataandTime-Varying
Covariates 230
11.5.1 Left-Truncated/Right-CensoredData 230
11.5.2 Example—TheSurvivalofBroadwayShows
(continued) 233
11.5.3 Time-VaryingCovariates 233
11.5.4 Example—FemaleHeadsofGovernment 235
11.6 Summary 238
Part V
Other Regression Models
(cid:2) (cid:2)
12 NonlinearRegression 243
12.1 Introduction 243
12.2 ConceptsandBackgroundMaterial 244
12.3 Methodology 246
12.3.1 NonlinearLeastSquaresEstimation 246
12.3.2 InferenceforNonlinearRegressionModels 247
12.4 Example—Michaelis-MentenEnzymeKinetics 248
12.5 Summary 252
13 ModelsforLongitudinalandNestedData 255
13.1 Introduction 255
13.2 ConceptsandBackgroundMaterial 257
13.2.1 NestedDataandANOVA 257
13.2.2 LongitudinalDataandTimeSeries 258
13.2.3 FixedEffectsVersusRandomEffects 259
13.3 Methodology 260
13.3.1 TheLinearMixedEffectsModel 260
13.3.2 TheGeneralizedLinearMixedEffectsModel 262
13.3.3 GeneralizedEstimatingEquations 262
13.3.4 NonlinearMixedEffectsModels 263
13.4 Example—TumorGrowthinaCancerStudy 264
13.5 Example—UnprovokedSharkAttacksintheUnited
States 269
13.6 Summary 275
(cid:2)