Richly Parameterized Linear Models: Additive, Time Series, and Spatial Models Using Random Effects

464 Pages·2013·15.36 MB·English

by  James S Hodges

Statistics
Texts in Statistical Science
“In a very personal writing style, Hodges explores what we do and do not
understand about mixed linear models. … By also discussing mysterious, R
Richly Parameterized
inconvenient, or plainly wrong results, we simply gain more insight and i
c
understanding. This works for me; I have never gained so much (hard to
h
get) insight in so short a time from any other book I have read. I highly
l
recommend it!” y Linear Models

—Håvard Rue, Norwegian University of Science and Technology P
a
“This book is a masterpiece, destined to become a classic. … There is
r Additive, Time Series,
not presently a unified theory, like that for linear regression, to explain
a
how, why, and when our numerical routines give results that should be m
questioned, or at least examined further. Even so, this book does the best
and Spatial Models Using
e
job I have seen of explaining what can go wrong and what the state of the
t
art is. The subject is important; the writing is excellent; and the examples e
are compelling. I am excited by the prospect of teaching a course from this r Random Effects
i
book. Its clarity of thought and presentation are exemplary. I recommend it z
e
for anyone who fits complicated models.”
d
—Michael Lavine, University of Massachusetts Amherst

L
Richly Parameterized Linear Models: Additive, Time Series, and Spatial
i
n
Models Using Random Effects takes a first step in developing a full theory
e
of richly parameterized models, which would allow statisticians to better
a
understand their analysis results. The author examines what is known and
r
unknown about mixed linear models and identifies research opportunities.
M
Features
o
• Develops a unified theory for a large class of popular models
d
• Explains how to overcome some common difficulties in using these
e
models to analyze data
l
• Covers both Bayesian and non-Bayesian methods s
• Includes detailed analyses of many real datasets, with the data and
other materials available on the author’s website
• Provides standard exercises and open questions at the end of each
H
chapter o
d
g
e
s James S. Hodges
K12996
K12996_Cover.indd 1 10/3/13 10:57 AM
Richly Parameterized
Linear Models
Additive, Time Series,
and Spatial Models Using
Random Effects
CHAPMAN & HALL/CRC
Texts in Statistical Science Series
Series Editors
Francesca Dominici, Harvard School of Public Health, USA
Julian J. Faraway, University of Bath, UK
Martin Tanner, Northwestern University, USA
Jim Zidek, University of British Columbia, Canada
Analysis of Failure and Survival Data Data Driven Statistical Methods
P. J. Smith P. Sprent
The Analysis of Time Series: An Introduction, Decision Analysis: A Bayesian Approach
Sixth Edition J.Q. Smith
C. Chatfield
Design and Analysis of Experiments with SAS
Applied Bayesian Forecasting and Time Series J. Lawson
Analysis
Elementary Applications of Probability Theory,
A. Pole, M. West, and J. Harrison
Second Edition
Applied Categorical and Count Data Analysis H.C. Tuckwell
W. Tang, H. He, and X.M. Tu
Elements of Simulation
Applied Nonparametric Statistical Methods, B.J.T. Morgan
Fourth Edition
Epidemiology: Study Design and
P. Sprent and N.C. Smeeton
Data Analysis, Third Edition
Applied Statistics: Handbook of GENSTAT M. Woodward
Analyses
Essential Statistics, Fourth Edition
E.J. Snell and H. Simpson
D.A.G. Rees
Applied Statistics: Principles and Examples
Exercises and Solutions in Statistical Theory
D.R. Cox and E.J. Snell L.L. Kupper, B.H. Neelon, and S.M. O’Brien
Applied Stochastic Modelling, Second Edition Exercises and Solutions in Biostatistical Theory
B.J.T. Morgan L.L. Kupper, B.H. Neelon, and S.M. O’Brien
Bayesian Data Analysis, Third Edition Extending the Linear Model with R:
A. Gelman, J.B. Carlin, H.S. Stern, D.B. Dunson, Generalized Linear, Mixed Effects and
A. Vehtari, and D.B. Rubin Nonparametric Regression Models
J.J. Faraway
Bayesian Ideas and Data Analysis: An
Introduction for Scientists and Statisticians A First Course in Linear Model Theory
N. Ravishanker and D.K. Dey
R. Christensen, W. Johnson, A. Branscum,
and T.E. Hanson Generalized Additive Models:
An Introduction with R
Bayesian Methods for Data Analysis,
S. Wood
Third Edition
B.P. Carlin and T.A. Louis Generalized Linear Mixed Models:
Modern Concepts, Methods and Applications
Beyond ANOVA: Basics of Applied Statistics
W. W. Stroup
R.G. Miller, Jr.
Graphics for Statistics and Data Analysis with R
The BUGS Book: A Practical Introduction to
K.J. Keen
Bayesian Analysis
Interpreting Data: A First Course
D. Lunn, C. Jackson, N. Best, A. Thomas, and
in Statistics
D. Spiegelhalter
A.J.B. Anderson
A Course in Categorical Data Analysis
Introduction to General and Generalized
T. Leonard
Linear Models
A Course in Large Sample Theory
H. Madsen and P. Thyregod
T.S. Ferguson
An Introduction to Generalized Multivariate Analysis of Variance and
Linear Models, Third Edition Repeated Measures: A Practical Approach for
A.J. Dobson and A.G. Barnett Behavioural Scientists
D.J. Hand and C.C. Taylor
Introduction to Multivariate Analysis
C. Chatfield and A.J. Collins Multivariate Statistics: A Practical Approach
B. Flury and H. Riedwyl
Introduction to Optimization Methods and
Their Applications in Statistics Multivariate Survival Analysis and Competing
B.S. Everitt Risks
M. Crowder
Introduction to Probability with R
K. Baclawski Nonparametric Methods in Statistics with SAS
Applications
Introduction to Randomized Controlled
O. Korosteleva
Clinical Trials, Second Edition
J.N.S. Matthews Pólya Urn Models
H. Mahmoud
Introduction to Statistical Inference and Its
Applications with R Practical Data Analysis for Designed
M.W. Trosset Experiments
B.S. Yandell
Introduction to Statistical Limit Theory
A.M. Polansky Practical Longitudinal Data Analysis
D.J. Hand and M. Crowder
Introduction to Statistical Methods for
Clinical Trials Practical Multivariate Analysis, Fifth Edition
T.D. Cook and D.L. DeMets A. Afifi, S. May, and V.A. Clark
Introduction to Statistical Process Control Practical Statistics for Medical Research
P. Qiu D.G. Altman
Introduction to the Theory of Statistical A Primer on Linear Models
Inference J.F. Monahan
H. Liero and S. Zwanzig
Principles of Uncertainty
Large Sample Methods in Statistics J.B. Kadane
P.K. Sen and J. da Motta Singer
Probability: Methods and Measurement
Linear Algebra and Matrix Analysis for A. O’Hagan
Statistics
Problem Solving: A Statistician’s Guide,
S. Banerjee and A. Roy
Second Edition
Logistic Regression Models C. Chatfield
J.M. Hilbe
Randomization, Bootstrap and Monte Carlo
Markov Chain Monte Carlo: Methods in Biology, Third Edition
Stochastic Simulation for Bayesian Inference, B.F.J. Manly
Second Edition
Readings in Decision Analysis
D. Gamerman and H.F. Lopes
S. French
Mathematical Statistics
Richly Parameterized Linear Models:
K. Knight
Additive, Time Series, and Spatial Models
Modeling and Analysis of Stochastic Systems, Using Random Effects
Second Edition J.S. Hodges
V.G. Kulkarni
Sampling Methodologies with Applications
Modelling Binary Data, Second Edition P.S.R.S. Rao
D. Collett
Stationary Stochastic Processes: Theory and
Modelling Survival Data in Medical Research, Applications
Second Edition G. Lindgren
D. Collett
Statistical Analysis of Reliability Data Statistics in Engineering: A Practical Approach
M.J. Crowder, A.C. Kimber, A.V. Metcalfe
T.J. Sweeting, and R.L. Smith
Statistics in Research and Development,
Statistical Methods for Spatial Data Analysis Second Edition
O. Schabenberger and C.A. Gotway R. Caulcutt
Statistical Methods for SPC and TQM Stochastic Processes: An Introduction,
D. Bissell Second Edition
P.W. Jones and P. Smith
Statistical Methods in Agriculture and
Experimental Biology, Second Edition Survival Analysis Using S: Analysis of
R. Mead, R.N. Curnow, and A.M. Hasted Time-to-Event Data
M. Tableman and J.S. Kim
Statistical Process Control: Theory and
Practice, Third Edition The Theory of Linear Models
G.B. Wetherill and D.W. Brown B. Jørgensen
Statistical Theory: A Concise Introduction Time Series Analysis
F. Abramovich and Y. Ritov H. Madsen
Statistical Theory, Fourth Edition Time Series: Modeling, Computation, and
B.W. Lindgren Inference
R. Prado and M. West
Statistics for Accountants
S. Letchford Understanding Advanced Statistical Methods
P.H. Westfall and K.S.S. Henning
Statistics for Epidemiology
N.P. Jewell
Statistics for Technology: A Course in Applied
Statistics, Third Edition
C. Chatfield
Texts in Statistical Science
Richly Parameterized
Linear Models
Additive, Time Series,
and Spatial Models Using
Random Effects
James S. Hodges
University of Minnesota
Minneapolis, USA
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2014 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Version Date: 20130730
International Standard Book Number-13: 978-1-4398-6684-9 (eBook - PDF)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts
have been made to publish reliable data and information, but the author and publisher cannot assume
responsibility for the validity of all materials or the consequences of their use. The authors and publishers
have attempted to trace the copyright holders of all material reproduced in this publication and apologize to
copyright holders if permission to publish in this form has not been obtained. If any copyright material has
not been acknowledged please write and let us know so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmit-
ted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented,
including photocopying, microfilming, and recording, or in any information storage or retrieval system,
without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.
com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood
Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and
registration for a variety of users. For organizations that have been granted a photocopy license by the CCC,
a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used
only for identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
Contents
ListofExamples xv
ListofFigures xvii
ListofTables xxvii
Preface xxxi
Acknowledgments xxxvii
I MixedLinearModels:Syntax,Theory,andMethods 1
1 AnOpinionatedSurveyofMethodsforMixedLinearModels 5
1.1 MixedLinearModelsintheStandardFormulation 5
1.2 ConventionalAnalysisoftheMixedLinearModel 16
1.2.1 Overview 16
1.2.2 MeanStructureEstimates 17
1.2.3 Estimatingφ,theUnknownsinGandR 19
1.2.3.1 Maximizingthe,orRathera,Likelihood 19
1.2.3.2 MaximizingtheRestricted(Residual)Likelihood 20
1.2.4 OtherMachineryofConventionalStatisticalInference 21
1.2.4.1 StandardErrorsforFixedandRandomEffects 22
1.2.4.2 Testing and Intervals for Fixed and Random
Effects 23
1.2.4.3 TestingandIntervalsforφ 24
1.3 BayesianAnalysisoftheMixedLinearModel 26
1.3.1 AVeryBriefReviewofBayesianAnalysisinGeneral 26
1.3.2 BayesianAnalysisofMixedLinearModels 29
1.3.2.1 TestsandIntervalsforUnknowns 29
1.3.2.2 SomeCommentsontheBayesianAnalysis 31
1.3.2.3 PriorDistributionsforVariances 33
1.3.2.4 PriorDistributionsforCovarianceMatrices 35
1.3.3 ComputingforBayesianAnalyses 37
1.4 ConventionalandBayesianApproachesCompared 40
1.4.1 AdvantagesandDisadvantagesoftheTwoApproaches 40
vii
viii Contents
1.4.2 ConventionalandBayesianAnalysesfortheViral-Structure
Example 43
1.5 AFewWordsaboutComputing 46
Exercises 48
2 TwoMoreTools:AlternativeFormulation,MeasuresofComplexity 51
2.1 AlternativeFormulation:The“Constraint-Case”Formulation 51
2.1.1 TheConstraint-CaseFormulation 51
2.1.2 ABriefHistoryoftheConstraint-CaseFormulation 55
2.2 MeasuringtheComplexityofaMixedLinearModelFit 55
2.2.1 DFintheWholeFit 56
2.2.2 PartitioningaFit’sDFintoComponents 58
2.2.2.1 Notation,MotivationforthePartitionofDF 58
2.2.2.2 PartitioningDF 59
2.2.2.3 PropertiesoftheDefinition 59
2.2.2.4 ExampleofPartitioningDF 60
2.2.3 UsesofDFforMixedLinearModels 63
2.2.3.1 UsingDFtoSpecifyF-Tests 63
2.2.3.2 Using DF to Describe Model Size, for Model-
SelectionCriteria 65
2.2.3.3 PriorDistributionsonDF 65
2.2.4 DFComparedto pDofSpiegelhalteretal.(2002) 68
2.2.5 SomeIntuitionforFractionalDegreesofFreedom 70
2.2.5.1 UsingtheConstraint-CaseFormulation 70
2.2.5.2 UsingtheMixedModelFormulation 72
Exercises 73
II RichlyParameterizedModelsasMixedLinearModels 75
3 PenalizedSplinesasMixedLinearModels 79
3.1 PenalizedSplines:Basis,Knots,andPenalty 79
3.2 MoreonBasis,Knots,andPenalty 83
3.2.1 AFewMoreBases 83
3.2.2 ABitMoreonPenaltyFunctions 85
3.2.3 BriefCommentsonSomeOperationalMatters 87
3.3 MixedLinearModelRepresentation 88
3.3.1 ApplyingtheMixed-Linear-ModelApproach 90
3.3.2 Brief Comments on Other Aspects of the Mixed-Linear-
ModelAnalysis 98
Exercises 99
Contents ix
4 AdditiveModelsandModelswithInteractions 101
4.1 AdditiveModelsasMixedLinearModels 102
4.1.1 ThePigJawboneExample 102
4.1.2 AdditiveModelsDefined 103
4.1.3 AdditiveModelsFittothePigJawboneData 104
4.2 ModelswithInteractions 109
4.2.1 Categorical-by-ContinuousInteractions 110
4.2.2 Categorical-by-Categorical Interactions (Smoothed
ANOVA) 114
4.2.3 Smoothed ANOVA, Balanced Design with a Single Error
Term 115
4.2.3.1 NotationandOtherMachinery 116
4.2.3.2 DFinEffects,PriorDistributions,SANOVATable 119
4.2.3.3 SANOVAAppliedtotheDenture-LinerExample 121
4.2.4 SmoothedANOVAforMoreGeneralDesigns 126
Exercises 126
5 SpatialModelsasMixedLinearModels 129
5.1 GeostatisticalModels 130
5.2 ModelsforArealData 132
5.2.1 CommonArealModels:SAR,CAR,andICAR 133
5.2.2 MoreontheICARModel/Prior 136
5.2.3 SmoothedANOVAwithSpatialSmoothing 141
5.3 Two-DimensionalPenalizedSplines 144
5.3.1 Tensor-ProductBasis 144
5.3.2 RadialBasis 145
5.3.3 ACommentonTensor-Productvs.RadialBasis 149
Exercises 149
6 Time-SeriesModelsasMixedLinearModels 151
6.1 Example:LinearGrowthModel 151
6.2 DynamicLinearModelsinSomeGenerality 154
6.3 ExampleofaMulti-componentDLM 156
Exercises 160
7 TwoOtherSyntaxesforRichlyParameterizedModels 163
7.1 SchematicComparisonoftheSyntaxes 163
7.2 GaussianMarkovRandomFields(Rue&Held2005) 164
7.3 Likelihood Inference for Models with Unobservables (Lee et al.
2006) 168
Exercises 171