An introduction to envelopes: dimension reduction for efficient estimation in multivariate statistics

306 Pages·2018·2.293 MB·English

by  Cook, R. Dennis

AnIntroductiontoEnvelopes
WILEYSERIESINPROBABILITYANDSTATISTICS
EstablishedbyWalterA.ShewhartandSamuelS.Wilks
Editors:DavidJ.Balding,NoelA.C.Cressie,GarrettM.Fitzmaurice,
GeofH.Givens,HarveyGoldstein,GeertMolenberghs,DavidW.Scott,
AdrianF.M.Smith,RueyS.Tsay
EditorsEmeriti:J.StuartHunter,IainM.Johnstone,JosephB.Kadane,
JozefL.Teugels
TheWileySeriesinProbabilityandStatisticsiswellestablishedandauthor-
itative. It covers many topics of current research interest in both pure and
applied statistics and probability theory. Written by leading statisticians and
institutions, the titles span both state-of-the-art developments in the field
andclassicalmethods.
Reflecting the wide range of current research in statistics, the series encom-
passesapplied,methodologicalandtheoreticalstatistics,rangingfromapplica-
tionsandnewtechniquesmadepossiblebyadvancesincomputerizedpractice
torigoroustreatmentoftheoreticalapproaches.
This series provides essential and invaluable reading for all statisticians,
whetherinacademia,industry,government,orresearch.
A complete list of titles in this series can be found at http://www.wiley.com/
go/wsps
An Introduction to Envelopes
DimensionReductionforEfficientEstimationin
MultivariateStatistics
R.DennisCook
SchoolofStatistics
UniversityofMinnesota
U.S.A.
Thiseditionfirstpublished2018
©2018JohnWiley&Sons,Inc
EditionHistory
Allrightsreserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,ortransmitted,
inanyformorbyanymeans,electronic,mechanical,photocopying,recordingorotherwise,exceptas
permittedbylaw.Adviceonhowtoobtainpermissiontoreusematerialfromthistitleisavailableathttp://www
.wiley.com/go/permissions.
TherightofR.DennisCookbeidentifiedastheauthorofthematerialinthisworkhasbeenassertedin
accordancewithlaw.
RegisteredOffice(s)
JohnWiley&Sons,Inc.,111RiverStreet,Hoboken,NJ07030,USA
EditorialOffice
111RiverStreet,Hoboken,NJ07030,USA
Fordetailsofourglobaleditorialoffices,customerservices,andmoreinformationaboutWileyproductsvisit
usatwww.wiley.com.
Wileyalsopublishesitsbooksinavarietyofelectronicformatsandbyprint-on-demand.Somecontentthat
appearsinstandardprintversionsofthisbookmaynotbeavailableinotherformats.
LimitofLiability/DisclaimerofWarranty
Inviewofongoingresearch,equipmentmodifications,changesingovernmentalregulations,andtheconstant
flowofinformationrelatingtotheuseofexperimentalreagents,equipment,anddevices,thereaderisurgedto
reviewandevaluatetheinformationprovidedinthepackageinsertorinstructionsforeachchemical,pieceof
equipment,reagent,ordevicefor,amongotherthings,anychangesintheinstructionsorindicationofusage
andforaddedwarningsandprecautions.Whilethepublisherandauthorshaveusedtheirbesteffortsin
preparingthiswork,theymakenorepresentationsorwarrantieswithrespecttotheaccuracyorcompleteness
ofthecontentsofthisworkandspecificallydisclaimallwarranties,includingwithoutlimitationanyimplied
warrantiesofmerchantabilityorfitnessforaparticularpurpose.Nowarrantymaybecreatedorextendedby
salesrepresentatives,writtensalesmaterialsorpromotionalstatementsforthiswork.Thefactthatan
organization,website,orproductisreferredtointhisworkasacitationand/orpotentialsourceoffurther
informationdoesnotmeanthatthepublisherandauthorsendorsetheinformationorservicestheorganization,
website,orproductmayprovideorrecommendationsitmaymake.Thisworkissoldwiththeunderstanding
thatthepublisherisnotengagedinrenderingprofessionalservices.Theadviceandstrategiescontainedherein
maynotbesuitableforyoursituation.Youshouldconsultwithaspecialistwhereappropriate.Further,readers
shouldbeawarethatwebsiteslistedinthisworkmayhavechangedordisappearedbetweenwhenthisworkwas
writtenandwhenitisread.Neitherthepublishernorauthorsshallbeliableforanylossofprofitoranyother
commercialdamages,includingbutnotlimitedtospecial,incidental,consequential,orotherdamages.
LibraryofCongressCataloging-in-PublicationData
Names:Cook,R.Dennis,author.
Title:Anintroductiontoenvelopes:dimensionreductionforefficient
estimationinmultivariatestatistics/R.DennisCook.
Description:1stedition.|Hoboken,NJ:JohnWiley&Sons,2018.|Series:
Wileyseriesinprobabilityandstatistics|
Identifiers:LCCN2018023695(print)|LCCN2018036057(ebook)|ISBN
9781119422952(AdobePDF)|ISBN9781119422969(ePub)|ISBN9781119422938
(hardcover)
Subjects:LCSH:Multivariateanalysis.|Dimensionreduction(Statistics)
Classification:LCCQA278(ebook)|LCCQA278.C6482018(print)|DDC
519.5/35–dc23
LCrecordavailableathttps://lccn.loc.gov/2018023695
CoverDesign:Wiley
CoverImage:©teekid/iStockphotoandCourtesyofR.DennisCook
Setin10/12ptWarnockProbySPiGlobal,Chennai,India
PrintedintheUnitedStatesofAmerica
10 9 8 7 6 5 4 3 2 1
ForSandra
SisterandArtist
1949–2017
vii
Contents
Preface xv
NotationandDefinitions xix
1 ResponseEnvelopes 1
1.1 TheMultivariateLinearModel 2
1.1.1 PartitionedModelsandAddedVariablePlots 5
1.1.2 AlternativeModelForms 6
1.2 EnvelopeModelforResponseReduction 6
1.3 Illustrations 10
1.3.1 ASchematicExample 10
1.3.2 CompoundSymmetry 13
1.3.3 WheatProtein:IntroductoryIllustration 13
1.3.4 CattleWeights:InitialFit 14
1.4 MoreontheEnvelopeModel 19
1.4.1 RelationshipwithSufficiency 19
1.4.2 ParameterCount 19
1.4.3 PotentialGains 20
1.5 MaximumLikelihoodEstimation 21
1.5.1 Derivation 21
1.5.2 CattleWeights:VariationoftheX-VariantPartsofY 23
1.5.3 Insightsinto̂() 24
Σ
1.5.4 ScalingtheResponses 25
1.6 AsymptoticDistributions 25
1.7 FittedValuesandPredictions 28
1.8 TestingtheResponses 29
1.8.1 TestDevelopment 29
1.8.2 TestingIndividualResponses 32
1.8.3 TestingContainmentOnly 34
1.9 NonnormalErrors 34
1.10 SelectingtheEnvelopeDimension,u 36
1.10.1 SelectionMethods 36
viii Contents
1.10.1.1 LikelihoodRatioTesting 36
1.10.1.2 InformationCriteria 37
1.10.1.3 Cross-validation 37
1.10.2 InferringAboutrank(𝛽) 38
1.10.3 AsymptoticConsiderations 38
1.10.4 OverestimationVersusUnderestimationofu 41
1.10.5 CattleWeights:Influenceofu 43
1.11 BootstrapandUncertaintyintheEnvelopeDimension 45
1.11.1 BootstrapforEnvelopeModels 45
1.11.2 WheatProtein:BootstrapandAsymptoticStandard
Errors,uFixed 46
1.11.3 CattleWeights:Bootstrappingu 47
1.11.4 BootstrapSmoothing 48
1.11.5 CattleData:BootstrapSmoothing 49
2 IllustrativeAnalysesUsingResponseEnvelopes 51
2.1 WheatProtein:FullData 51
2.2 BerkeleyGuidanceStudy 51
2.3 Banknotes 54
2.4 EgyptianSkulls 55
2.5 AustralianInstituteofSport:ResponseEnvelopes 58
2.6 AirPollution 59
2.7 MultivariateBioassay 63
2.8 BrainVolumes 65
2.9 ReducingLeadLevelsinChildren 67
3 PartialResponseEnvelopes 69
3.1 PartialEnvelopeModel 69
3.2 Estimation 71
3.2.1 AsymptoticDistributionof𝛽̂ 72
1
3.2.2 Selectingu 73
1
3.3 Illustrations 74
3.3.1 CattleWeight:IncorporatingBasalWeight 74
3.3.2 Mens’Urine 74
3.4 PartialEnvelopesforPrediction 77
3.4.1 Rationale 77
3.4.2 PulpFibers:PartialEnvelopesandPrediction 78
3.5 ReducingPartoftheResponse 79
4 PredictorEnvelopes 81
4.1 ModelFormulations 81
4.1.1 LinearPredictorReduction 81
4.1.1.1 PredictorEnvelopeModel 83
Contents ix
4.1.1.2 ExpositoryExample 83
4.1.2 LatentVariableFormulationofPartialLeastSquares
Regression 84
4.1.3 PotentialAdvantages 86
4.2 SIMPLS 88
4.2.1 SIMPLSAlgorithm 88
4.2.2 SIMPLSWhenn<p 90
4.2.2.1 BehavioroftheSIMPLSAlgorithm 90
4.2.2.2 AsymptoticPropertiesofSIMPLS 91
4.3 Likelihood-BasedPredictorEnvelopes 94
4.3.1 Estimation 95
4.3.2 ComparisionswithSIMPLSandPrincipalComponent
Regression 97
4.3.2.1 PrincipalComponentRegression 98
4.3.2.2 SIMPLS 98
4.3.3 AsymptoticProperties 98
4.3.4 FittedValuesandPrediction 100
4.3.5 ChoiceofDimension 101
4.3.6 RelevantComponents 101
4.4 Illustrations 102
4.4.1 ExpositoryExample,Continued 102
4.4.2 AustralianInstituteofSport:PredictorEnvelopes 103
4.4.3 WheatProtein:PredictingProteinContent 105
4.4.4 Mussels’Muscles:PredictorEnvelopes 106
4.4.5 MeatProperties 109
4.5 SimultaneousPredictor–ResponseEnvelopes 109
4.5.1 ModelFormulation 109
4.5.2 PotentialGain 110
4.5.3 Estimation 113
5 EnvelopingMultivariateMeans 117
5.1 EnvelopingaSingleMean 117
5.1.1 EnvelopeStructure 117
5.1.2 EnvelopeModel 119
5.1.3 Estimation 120
5.1.3.1 MaximumLikelihoodEstimation 120
5.1.3.2 AsymptoticVarianceof𝜇̂ 121
5.1.3.3 Selectingu=dim( ()) 122
Σ
5.1.4 MinneapolisSchools 122
5.1.4.1 TwoTransformedResponses 123
5.1.4.2 FourUntransformedResponses 124
5.1.5 FunctionalData 126
5.2 EnvelopingMultipleMeanswithHeteroscedasticErrors 126
x Contents
5.2.1 HeteroscedasticEnvelopes 126
5.2.2 Estimation 128
5.2.3 CattleWeights:HeteroscedasticEnvelopeFit 129
5.3 ExtensiontoHeteroscedasticRegressions 130
6 EnvelopeAlgorithms 133
6.1 Likelihood-BasedEnvelopeEstimation 133
6.2 StartingValues 135
6.2.1 ChoosingtheStartingValuefromtheEigenvectors
̂
ofM 135
6.2.2 ChoosingtheStartingValuefromtheEigenvectors
̂ ̂
ofM+U 137
6.2.3 Summary 138
6.3 ANon-GrassmannAlgorithmforEstimating () 139
M
6.4 SequentialLikelihood-BasedEnvelopeEstimation 141
6.4.1 The1DAlgorithm 141
6.4.2 EnvelopeComponentScreening 142
6.4.2.1 ECSAlgorithm 143
6.4.2.2 AlternativeECSAlgorithm 144
6.5 SequentialMoment-BasedEnvelopeEstimation 145
6.5.1 BasicAlgorithm 145
6.5.2 KrylovMatricesanddim()=1 147
6.5.3 VariationsontheBasicAlgorithm 147
7 EnvelopeExtensions 149
7.1 EnvelopesforVector-ValuedParameters 149
7.1.1 Illustrations 151
7.1.2 EstimationBasedonaCompleteLikelihood 154
7.1.2.1 LikelihoodConstruction 154
7.1.2.2 AsterModels 156
7.2 EnvelopesforMatrix-ValuedParameters 157
7.3 EnvelopesforMatrix-ValuedResponses 160
7.3.1 InitialModeling 161
7.3.2 ModelswithKroneckerStructure 163
7.3.3 EnvelopeModelswithKroneckerStructure 164
7.4 SpatialEnvelopes 166
7.5 SparseResponseEnvelopes 168
7.5.1 SparseResponseEnvelopeswhenr ≪n 168
7.5.2 CattleWeightsandBrainVolumes:SparseFits 169
7.5.3 SparseEnvelopeswhenr >n 170
7.6 BayesianResponseEnvelopes 171