Semiparametric regression

402 Pages·2003·3.06 MB·English

by  David Ruppert

Semiparametric Regression
Semiparametricregressionisconcernedwiththeflexibleincorporationofnonlinear
functional relationships in regression analyses. Any application area that uses re-
gressionanalysiscanbenefitfromsemiparametricregression.Assumingonlyabasic
familiaritywithordinaryparametricregression,thisuser-friendlybookexplainsthe
techniquesandbenefitsofsemiparametricregressioninaconciseandmodularfash-
ion. Theauthorsmakeliberaluseofgraphicsandexamplespluscasestudiestaken
fromenvironmental,financial,andotherapplications. Theyincludepracticaladvice
onimplementationandpointerstorelevantsoftware.
Thisbookissuitableasatextbookforstudentswithlittlebackgroundinregression
as well as a reference book for statistically oriented scientists – such as biostatisti-
cians, econometricians, quantitative social scientists, and epidemiologists – with a
good working knowledge of regression and the desire to begin using more flexi-
blesemiparametricmodels. Evenexpertsonsemiparametricregressionshouldfind
somethingnewhere.
DavidRuppertistheAndrewSchultz,Jr.,ProfessorofEngineering(SchoolofOp-
erationsResearchandIndustrialEngineering)andProfessorofStatisticalScienceat
Cornell University. He has served as editor for a number of prestigious series and
journalsandhaspublishedsome80articlesofhisownaswellasco-authoringtwo
popularbooks,TransformationandWeightinginRegressionandMeasurementError
inNonlinearModels. HeisalsowinneroftheWilcoxonPrizeforbestpracticalap-
plicationspaperintechnometricsandanelectedFellowoftheAmericanStatistical
AssociationandtheInstituteofMathematicalStatistics.
M.P.WandisProfessorofStatisticsattheUniversityofNewSouthWalesinSydney,
Australia. HehasheldfacultyappointmentsatHarvardUniversity,RiceUniversity,
andTexasA&MUniversity. Dr.WandisaFellowoftheAmericanStatisticalAssoci-
ationandhasservedasanassociateeditorfortheJournaloftheAmericanStatistical
AssociationandBiometrika. HeiswinneroftheP.A.P.MoranMedalforstatistical
research.
R. J. Carroll is Distinguished Professor of Statistics, Nutrition and Toxicology at
TexasA&MUniversity. AmonghismanyhonorsaretheCOPSSPresidents’Award,
the Fisher Lecture, the SnedecorAward, and theWilcoxon Prize. He is an elected
FellowoftheAmericanStatisticalAssociationandtheInstituteofMathematicalSta-
tisticsaswellasanelectedmemberoftheInternationalStatisticalInstitute.
CAMBRIDGE SERIES IN STATISTICAL AND PROBABILISTIC MATHEMATICS
EditorialBoard
R.Gill(Departmentof Mathematics,UtrechtUniversity)
B.D.Ripley(DepartmentofStatistics,UniversityofOxford)
S.Ross(DepartmentofIndustrialEngineering,UniversityofCalifornia,Berkeley)
M.Stein(DepartmentofStatistics,UniversityofChicago)
D.Williams(Schoolof MathematicalSciences,Universityof Bath)
Thisseriesofhigh-qualityupper-divisiontextbooksandexpositorymonographscov-
ersallaspectsofstochasticapplicablemathematics. Thetopicsrangefrompureand
appliedstatisticstoprobabilitytheory,operationsresearch,optimization,andmath-
ematicalprogramming. Thebookscontainclearpresentationsofnewdevelopments
inthefieldandalsoofthestateoftheartinclassicalmethods. Whileemphasizing
rigorous treatment of theoretical methods, the books also contain applications and
discussionsofnewtechniquesmadepossiblebyadvancesincomputationalpractice.
Alreadypublished
1. BootstrapMethodsandTheirApplication,byA.C.DavisonandD.V.Hinkley
2. MarkovChains,byJ.Norris
3. AsymptoticStatistics,byA.W.vanderVaart
4. WaveletMethodsforTimeSeriesAnalysis,byDonaldB.Percivaland
AndrewT.Walden
5. BayesianMethods,byThomasLeonardandJohnS.J.Hsu
6. EmpiricalProcessesinM-Estimation,bySaravandeGeer
7. NumericalMethodsofStatistics,byJohnF.Monahan
8. AUser’sGuidetoMeasureTheoreticProbability,byDavidPollard
9. TheEstimationandTrackingofFrequency,byB.G.QuinnandE.J.Hannan
10. DataAnalysisandGraphicsusingR,byJohnMaindonaldandJohnBraun
11. StatisticalModels,byA.C.Davison
Semiparametric Regression
DAVID RUPPERT
CornellUniversity
M. P.WAND
HarvardUniversity
R. J. CARROLL
TexasA&MUniversity
  
Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, São Paulo
Cambridge University Press
 
The Edinburgh Building, Cambridge , United Kingdom
Published in the United States of America by Cambridge University Press, New York
www.cambridge.org
Information on this title: www.cambridge.org/9780521780506
© David Ruppert, M. P.Wand, R. J. Carroll 2003
This book is in copyright. Subject to statutory exception and to the provision of
relevant collective licensing agreements, no reproduction of any part may take place
without the written permission of Cambridge University Press.
First published in print format 2003
- 
isbn-13 978-0-511-06683-2 eBook (NetLibrary)
- 
isbn-10 0-511-06683-X eBook (NetLibrary)
- 
isbn-13 978-0-521-78050-6 hardback
- 
isbn-10 0-521-78050-0 hardback
- 
isbn-13 978-0-521-78516-7 paperback
- 
isbn-10 0-521-78516-2 paperback
Cambridge University Press has no responsibility for the persistence or accuracy of

s for external or third-party internet websites referred to in this book, and does not
guarantee that any content on such websites is, or will remain, accurate or appropriate.
ToAnne,withlove
—David
Tomywife’sparents,AyhanandRecep
—Matt
ToBrettandJeb
—Raymond
Contents
Preface page xiii
GuidetoNotation xv
1 Introduction 1
1.1 AssessingtheCarcinogenicityofPhenolphthalein 3
1.2 SalinityandFishinginNorthCarolina 4
1.3 ManagementofaRetirementFund 5
1.4 BiomonitoringofAirborneMercury 7
1.5 TermStructureofInterestRates 7
1.6 AirPollutionandMortalityinMilan:TheHarvestingEffect 11
2 ParametricRegression 15
2.1 Introduction 15
2.2 LinearRegressionModels 15
2.3 RegressionDiagnostics 20
2.4 Inference 28
2.5 ParametricAdditiveModels 36
2.6 ModelSelection 44
2.7 PolynomialRegressionModels 46
2.8 NonlinearRegression 48
2.9 TransformationsinRegression 51
2.10 BibliographicNotes 55
2.11 SummaryofFormulas 55
3 ScatterplotSmoothing 57
3.1 Introduction 57
3.2 PreliminaryIdeas 58
3.3 PracticalImplementation 62
3.4 AutomaticKnotSelection 64
3.5 PenalizedSplineRegression 65
3.6 QuadraticSplineBases 67
3.7 OtherSplineModelsandBases 69
3.8 OtherPenalties 74
3.9 GeneralDefinitionofaPenalizedSpline 75
3.10 LinearSmoothers 76
3.11 ErrorofaSmoother 76
vii
viii Contents
3.12 RankofaSmoother 78
3.13 DegreesofFreedomofaSmoother 80
3.14 ResidualDegreesofFreedom 82
3.15 OtherApproachestoScatterplotSmoothing 84
3.16 ChoosingaScatterplotSmoother 87
3.17 BibliographicalNotes 88
3.18 SummaryofFormulas 89
4 MixedModels 91
4.1 Introduction 91
4.2 MixedModels 91
4.3 Prediction 95
4.4 TheLinearMixedModel(LMM) 98
4.5 EstimationandPredictioninLMM 98
4.6 EstimatedBLUP(EBLUP) 101
4.7 StandardErrorEstimation 102
4.8 HypothesisTesting 104
4.9 PenalizedSplinesasBLUPs 108
4.10 BibliographicalNotes 110
4.11 SummaryofFormulas 110
5 AutomaticScatterplotSmoothing 112
5.1 Introduction 112
5.2 TheLikelihoodApproach 113
5.3 TheModelSelectionApproach 114
5.4 CaveatsofAutomaticParameterSelection 120
5.5 ChoosingtheKnotsandBasisFunctions 123
5.6 AutomaticSelectionoftheNumberofKnots 127
5.7 BibliographicalNotes 131
5.8 SummaryofFormulas 131
6 Inference 133
6.1 Introduction 133
6.2 VariabilityBands 133
6.3 ConfidenceandPredictionIntervals 135
6.4 InferenceforPenalizedSplines 137
6.5 SimultaneousConfidenceBands 142
6.6 TestingtheAdequacyofParametricModels 145
6.7 TestingforNoEffect 149
6.8 InferenceUsingFirstDerivatives 151
6.9 TestingforExistenceofaFeature 156
6.10 BibliographicalNotes 158
6.11 SummaryofFormulas 159
7 SimpleSemiparametricModels 161
7.1 Introduction 161
7.2 BeyondScatterplotSmoothing 161
Contents ix
7.3 SemiparametricBinaryOffsetModel 162
7.4 AdditivityandInteractions 164
7.5 GeneralParametricComponent 164
7.6 Inference 167
7.7 BibliographicalNotes 168
8 AdditiveModels 170
8.1 Introduction 170
8.2 FittinganAdditiveModel 171
8.3 DegreesofFreedom 174
8.4 SmoothingParameterSelection 176
8.5 HypothesisTesting 181
8.6 ModelSelection 183
8.7 BibliographicalNotes 185
9 SemiparametricMixedModels 186
9.1 Introduction 186
9.2 AdditiveMixedModels 186
9.3 Subject-SpecificCurves 191
9.4 BibliographicalNotes 192
10 GeneralizedParametricRegression 194
10.1 Introduction 194
10.2 BinaryResponseData 194
10.3 LogisticRegression 195
10.4 OtherGeneralizedLinearModels 197
10.5 IterativelyReweightedLeastSquares 200
10.6 HatMatrix,DegreesofFreedom,andStandardErrors 201
10.7 OverdispersionandVarianceFunctions:Pseudolikelihood 201
10.8 GeneralizedLinearMixedModels 203
10.9 Deviance 209
10.10 TechnicalDetails 210
10.11 BibliographicalNotes 213
11 GeneralizedAdditiveModels 214
11.1 Introduction 214
11.2 GeneralizedScatterplotSmoothing 215
11.3 GeneralizedAdditiveMixedModels 217
11.4 Degrees-of-FreedomApproximations 219
11.5 AutomaticSmoothingParameterSelection 220
11.6 HypothesisTesting 220
11.7 ModelSelection 221
11.8 DensityEstimation 221
11.9 BibliographicalNotes 222
12 InteractionModels 223
12.1 Introduction 223
12.2 Binary-by-ContinuousInteractionModels 224
x Contents
12.3 Factor-by-CurveInteractionsinAdditiveModels 226
12.4 VaryingCoefficientModels 234
12.5 Continuous-by-ContinuousInteractions 235
12.6 BibliographicalNotes 237
13 BivariateSmoothing 238
13.1 Introduction 238
13.2 ChoiceofBivariateBasisFunctions 240
13.3 Kriging 242
13.4 GeneralRadialSmoothing 248
13.5 DefaultAutomaticBivariateSmoother 256
13.6 GeoadditiveModels 258
13.7 AdditivePlusInteractionModels 259
13.8 GeneralizedBivariateSmoothing 259
13.9 Appendix:EquivalenceofBLUPusingZ andZ 259
R P
13.10 BibliographicalNotes 260
14 VarianceFunctionEstimation 261
14.1 Introduction 261
14.2 Formulation 263
14.3 ApplicationtotheLIDARData 264
14.4 QuasilikelihoodandVarianceFunctions 266
14.5 BibliographicalNotes 267
15 MeasurementError 268
15.1 Introduction 268
15.2 Formulation 269
15.3 TheExpectationMaximization(EM)Algorithm 270
15.4 SimulatedExampleRevisited 273
15.5 SensitivityAnalysisExample 273
15.6 BibliographicalNotes 275
16 BayesianSemiparametricRegression 276
16.1 Introduction 276
16.2 GeneralFramework 277
16.3 ScatterplotSmoothing 279
16.4 LinearMixedModels 285
16.5 GeneralizedLinearMixedModels 288
16.6 Rao–Blackwellization 291
16.7 BibliographicalNotes 292
17 SpatiallyAdaptiveSmoothing 293
17.1 Introduction 293
17.2 ALocalPenaltyMethod 294
17.3 CompletelyAutomaticAlgorithm 295
17.4 BayesianInference 296