Table Of ContentSpringer Texts in Statistics
Ronald Christensen
Advanced
Linear
Modeling
Statistical Learning and Dependent Data
Third Edition
Springer Texts in Statistics
SeriesEditors
G.Allen,DepartmentofStatistics,Houston,TX,USA
R.DeVeaux,DepartmentofMathematicsandStatistics,WilliamsCollege,
Williamstown,MA,USA
R.Nugent,DepartmentofStatistics,CarnegieMellonUniversity,Pittsburgh,
PA,USA
SpringerTextsinStatistics(STS)includesadvancedtextbooksfrom3rd-to4th-year
undergraduatecoursesto1st-to2nd-yeargraduatecourses.Exercisesetsshouldbe
included.TheserieseditorsarecurrentlyGeneveraI.Allen,RichardD.DeVeaux,
and Rebecca Nugent. Stephen Fienberg, George Casella, and Ingram Olkin were
editorsoftheseriesformanyyears.
Moreinformationaboutthisseriesathttp://www.springer.com/series/417
Ronald Christensen
Advanced Linear Modeling
Statistical Learning and Dependent Data
Third Edition
RonaldChristensen
DepartmentofMathematicsandStatistics
UniversityofNewMexico
Albuquerque,NM,USA
ISSN1431-875X ISSN2197-4136 (electronic)
SpringerTextsinStatistics
ISBN978-3-030-29163-1 ISBN978-3-030-29164-8 (eBook)
https://doi.org/10.1007/978-3-030-29164-8
©SpringerNatureSwitzerlandAG1991,2001,2019
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthors,andtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressedorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional
claimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG.
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
ToPete,Russ,andScot,
mypalsfromhighschool;
also
Wes,Ed,andeveryonefromgraduateschool.
Preface to the Third Edition
This is the third edition of Advanced Linear Modeling (ALM). It is roughly 50%
longer than the previous edition. It discusses the extension of linear models into
areasbeyondthoseusuallyaddressedinregressionandanalysisofvariance.Asin
previouseditions,itsprimaryemphasisisonmodelsinwhichthedatadisplaysome
formofdependenceandmanyofthechangesfromthepreviouseditionweremade
tosystematizethisemphasisondependentdata.Nonetheless,itbeginswithtopics
in modern regression analysis related to nonparametric regression and penalized
estimation(regularization).Rcodefortheanalysesinthebookisavailableathttp://
www.stat.unm.edu/∼fletcher/R-ALMIII.pdf.
MathematicalbackgroundiscontainedinAppendixAondifferentiationandKro-
neckerproducts.AlsosomenotationusedthroughoutthebookissetinSect.1.1.
ThiseditionhasbeenwritteninconjunctionwiththefiftheditionofChristensen
(2011),oftenhereafterreferredtoasPA.Somediscussionsthatpreviouslyappeared
in PA have been moved here. Obviously, you cannot do advanced linear modeling
without previously learning about linear modeling. I have tried to make this book
readable to people who have studied linear model theory from sources other than
PA,butIneedtocitesomesourceforbasicresultsonlinearmodels,soobviouslyI
citePA.IncaseswhereIneedtociteresultsforwhichthenewversionofPAisdif-
ferentfromthepreviousedition(s),thecitationsaregivenasPA-V.Ihaverearranged
thetopicsfromthepreviouseditionofALMsothatthematerialrelatedtoindepen-
dent data comes first followed by the material on dependent data. The chapter on
responsesurfaceshasbeendroppedbutisavailableinanewvolumedownloadable
frommywebsite:http://www.stat.unm.edu/∼fletcher/TopicsInDesign.Somefamil-
iarity with inner products is assumed, especially in Chaps.1 and 3. The required
familiaritycanbeacquiredfromPA.
Chapter 1 expands the previous introduction to nonparametric regression. The
discussionfollowswhatiscommonlyknownasthebasisfunctionapproach,despite
thefactthatmanyofthetechniquesdonotactuallyinvolvetheuseofbasisfunctions
per se. In fact, when dealing with spaces of functions the very idea of a basis is
subject to competing definitions. Tim Hanson pointed out to me the obvious fact
that if a group of functions are linearly independent, they always form a basis for
vii
viii PrefacetotheThirdEdition
thespacethattheyspan,butIthinkthatinnonparametricregressiontheideaisto
approximatewidercollectionsoffunctionsthanjustthesespanningsets.Chapter1
now also includes a short introduction to models involving an entire function of
predictorvariables.
Chapter2isanexpandedversionofthediscussionofpenalizedregressionfrom
Christensen(2011).AnewChap.3extendsthisbyintroducingreproducingkernel
Hilbertspaces.
Chapter4isnewexceptforthelastsection.Itgivesresultsonanextremelygen-
erallinearmodelfordependentorheteroscedasticdata.Itowesanobviousdebtto
Christensen(2011,Chapter12).Itcontainsseveralparticularlyusefulexercises.In
a standard course on linear model theory, the theory of estimation and testing for
dependent data is typically introduced but not developed, see for example Chris-
tensen(2011,Sections2.7and3.8).Section4.1ofthisbookreviews,butdoesnot
re-prove,thoseresults.Thisbookthenappliesthosefundamentalresultstodevelop
theoryforawidevarietyofpracticalmodels.
Ifinallyfiguredouthow,withoutoverwhelmingtheideasinabstrusenotation,to
presentMINQUEaslinearmodeling,soIhavedonethatinChap.4.Inatechnical
subsection,IgiveintotheabstrusenotationsoastoderivetheMINQUEequations.
Previously,IjustreferredthereadertoRaoforthederivation.
Chapter5onmixed modelsoriginallyappeared inPA.Ithas beenshortenedin
places due of overlap with Chap.4 but includes several new examples and exer-
cises.Itcontainsanewemphasisonlinearcovariancestructuresthatleadsnotonly
to variance component models but the new Sect.5.6 that examines a quite general
longitudinaldatamodel.Thedetailsoftherecoveryofinterblockinformationfora
balanced incomplete block design from PA no longer seem relevant, so they were
relegated,alongwiththeresponsesurfacematerial,tothevolumeonmywebsite.
Chapters 6 and 7 introduce time series: first the frequency domain which uses
modelsfromChap.1butwithrandomeffectsasinChap.5andthenthetimedomain
approachwhichcanbeviewedasapplicationsofideasfromthefrequencydomain.
Chapter8onspatialdataislittlechangedfromthepreviousedition.Mostly,the
referenceshavebeenupdated.
Theformerchapteronmultivariatemodelshasbeensplitintothree:Chap.9on
generaltheorywithanewsectionrelatingmultivariatemodelstospatialandtimese-
riesmodelsandanewdiscussionofmultiplecomparisons,Chap.10onapplications
tospecific models, and Chap.11 with an expanded discussion of generalized mul-
tivariatelinearmodels(alsoknownasgeneralizedmultivariateanalysisofvariance
(GMANOVA)andgrowthcurvemodels).
Chapters12and14areupdatedversionsofthepreviouschaptersondiscriminant
analysis and principal components. Chapter 13 is a new chapter on binary regres-
sionanddiscrimination.Itsraisond’eˆtreisthatitdevotesconsiderableattentionto
support vector machines. Chapter 14 contains a new section on classical multidi-
mensionalscaling.
From time to time, I mention the virtues of Bayesian approaches to problems
discussedinthebook.OneplacetolookformoreinformationisBIDA,i.e.,Chris-
tensen,Johnson,Branscum,andHanson(2010).
PrefacetotheThirdEdition ix
Thanks to my son Fletcher who is always the first person I ask when I have
doubts. Joe Cavanaugh and Mohammad Hattab have been particularly helpful as
haveTimHanson,WesJohnson,andEdBedrick.Finally,mythankstoAlNosedal-
Sanchez, Curt Storlie, and Thomas Lee for letting me modify our joint paper into
Chap.3.
AsIhavementionedelsewhere,thelargenumberofreferencestomyotherworks
isasmuchaboutslothasitisego.Insomesense,withtheexceptionofBIDA,allof
mybooksarevariationsonatheme.
Albuquerque,NM,USA RonaldChristensen
February2019
Preface to the Second Edition
ThisisthesecondeditionofLinearModelsforMultivariate,TimeSeriesandSpatial
Data.Ithasanewtitletoindicatethatitcontainsmuchnewmaterial.Theprimary
changesaretheadditionoftwonewchapters:oneonnonparametricregressionand
the other on response surface maximization. As before, thepresentations focus on
thelinearmodelaspectsofthesubject.Forexample,inthenonparametricregression
chapter there is very little about kernel regression estimation but quite a bit about
seriesapproximations,splines,andregressiontrees,allofwhichcanbeviewedas
linearmodeling.
Theneweditionalsoincludesvarioussmallerchanges.Ofparticularnotearea
subsectioninChap.1onmodelinglongitudinal(repeatedmeasures)dataandasec-
tioninChap.6oncovariancestructuresforspatiallatticedata.Iwouldliketothank
Dale Zimmerman for the suggestion of incorporating material on spatial lattices.
Anotherchangeisthatthesubjectindexisnowentirelyalphabetical.
Albuquerque,NM,USA RonaldChristensen
May9,2000
xi
