Applied Linear Statistical Models

1415 Pages·2004·49.826 MB·English

by  Michael H. Kutner, William Li, Christopher J. Nachtsheim, John Neter

Applied Linear
Statistical Models
Fifth Edition
Michael H. Kutner
EmOlY University
Christopher J. Nachtsheim
University ofM innesota
John Neter
University of Georgia
William Li
Universlty ofM innesota
wa
McGraw-Hili
t:a Irwin
Boston Burr RIdge, IL Dubuque, IA MadIson, WI New York San FrancIsco St LoUIs
Bangkok Bogota Caracas Kuala Lumpur LIsbon London Madnd MexIco CIty
MIlan Montreal New Deihl SantIago Seoul Smgapore Sydney TaIpeI Toronto
The McGraw·HiII Companies
McGraw-Hili
~
t:a Irwin
APPUED UNEAR STATISTICAL MODELS
Published by McGraw-Hill!Irwin, a business unit of The McGraw-Hill Companies, Inc., 1221 Avenue of the
Americas, New York, NY, 10020. Copyright © 2005, 1996, 1990, 1983, 1974 by The McGraw-Hill Compan
Inc. All rights reserved. No part ofthis publication may be reproduced or distributed in any form or by any
means, or stored in a database or retrieval system, without the prior written consent of The McGraw-Hill
Companies, Inc., including, but not limited to, in any networl< or other electronic storage or transmission, or
broadcast for distance learning.
Some ancillaries, including electronic and print components, may not be available to customers outside the
United States.
This book is printed on acid-free paper.
1234567890DocmOC0987654
ISBN 0-07-238688-6
Editorial director: Brent Gordon
Executive editor: Richard T. Hercher, lr.
Editorial assistant: Lee Stone
Senior marketing manager: Douglas Reiner
Media producer: Elizabeth Mavetz
Project manager: lim Labeots
Production supervisor: Gina Hangos
Lead designer: Pam Verros
Supplement producer: Matthew Peny
Senior digital content specialist: Brian Nacik
Cover design: Kiera Pohl
"!ypeface: 10/12 Times Roman
Compositor: Interactive Composition Corporation
Printer: R R Donnelley
Library of Congress Cataloging-in-Publication Data
Kutner, Michael H.
Applied linear statistical models.-5th ed.! Michael H Kutner ... let al].
p. cm. - (McGraw-HillfIrwin series Operations and decision sciences)
Rev. ed. of: Applied linear regression models. 4th ed. c2004.
Includes bibliographical references and index.
ISBN 0-07-238688-6 (acid-free paper)
1. Regression analysis. 2. Mathematical statistics. I. Kutner, Michael H. Applied linear
regression models. II. Title. III. Series.
QA278.2.K87 2005
519.5'36-dc22 2004052447
www.mhhe.com
~ ....... .
To
Nancy, Michelle, Allison,
, Maureen, Abigael, Andrew, Henry G.,
ies,
Dorothy, Ron, David,
Dezhong, Chenghua, Xu
;(\)
:«)
i\j
"
.......
~
........
Preface
Linear statistical models for regression, analysis of variance, and experimental design are
widely used today in business administration, economics, engineering, and the social, health,
and biological sciences. Successful applications of these models require a sound understand
ing of both the underlying theory and the practical problems that are encountered in using
the models in real-life situations. While Applied linear Statistical Models, Fifth Edition, is
basically an applied book, it seeks to blend theory and applications effectively, avoiding the
extremes of presenting theory in isolation and of giving elements of applications without
the needed understanding of the theoretical foundations.
The fifth edition differs from the fourth in a number of important respects.
In the area of regression analysis (Parts I-III):
1. We have reorganized the chapters for better clarity and flow of topics. Material from
the old Chapter 15 on normal correlation models has been integrated throughout the
text where appropriate. Much of the material is now found in an expanded Chapter
2, which focuses on inference in regression analysis. Material from the old Chapter 7
pertaining to polynomial and interaction regression models and from old Chapter 11
on quantitative predictors has been integrated into a new Chapter 8 called, "Models
for Quantitative and Qualitative Predictors." Material on model validation from old
Chapter lOis now fully integrated with updated material on model selection in a new
Chapter 9 entitled, "Building the Regression Model I: Model Selection and Validation."
2. We have added material on important techniques for data mining, including regression
trees and neural network models in Chapters 11 and 13, respectively.
3. The chapter on logistic regression (Chapter 14) has been extensively revised and
expanded to include a more thorough treatment of logistic, probit, and complemen
tary log-log models, logistic regression residuals, model selection, model assessment,
logistic regression diagnostics, and goodness of fit tests. We have also developed new
material on polytomous (multicategory) nominal logistic regression models and poly
tomous ordinal logistic regression models.
4. We have expanded the discussion of model selection methods and criteria. The Akaike
information criterion and Schwarz Bayesian criterion have been added, and a greater
emphasis is placed on the use of cross-validation for model selection and validation.
In the areas pertaining to the design and analysis of experimental and observational studies
(Parts IV-VI):
5. In the previous edition, Chapters 16 through 25 emphasized the analysis of variance,
and the design of experiments was not encountered formally until Chapter 26. We
have completely reorganized Parts IV-VI, emphasizing the design of experimental and
observational studies from the start. In a new Chapter 15, we provide an overview of
the basic concepts and planning approaches used in the design of experimental and
observational studies, drawing in part from material from old Chapters 16, 26, and
27. Fundamental concepts of experimental design, including the basic types of factors,
vi
Preface vii
treatments, experimental units, randomization, and blocking are described in detail.
This is followed by an overview of standard experimental designs, as well as the basic
types of observational studies, including cross-sectional, retrospective, and prospective
studies. Each of the design topics introduced in Chapter 15 is then covered in greater
detail in the chapters that follow. We emphasize the importance of good statistical
design of scientific studies, and make the point that proper design often leads to a
simple analYSIS. We note that the statistical analysis techniques used for observational
and experimental studies are often the same, but the ability to "prove" cause-and-effect
requires a carefully designed experimental study.
6. Previously, the planning of sample sizes was covered -in Chapter 26. We now present
material on planning of sample sizes in the relevant chapter, rather than devoting a
single, general discussion to this issue.
7. We have expanded and updated our coverage (Section 24.2) on the interpretation of
interaction plots for multi-factor studies.
8. We have reorganized and expanded the material on repeated measures designs in Chap
ter 27. In particular, we introduce methods for handling the analysis of factor effects
when interactions between subjects and treatments are important, and when interactions
between factors are important.
9. We have added material on the design and analysis of balanced incomplete block
experiments in Section 28.1, including the planning of sample sizes. A new appendix
(B.15) has been added that provides standard balanced incomplete block designs.
10. We have added new material on robust product and process design experiments in
Chapter 29, and illustrate its use with a case study from the automotive industry. These
experiments are frequently used in industrial studies to identify product or process
designs that exhibit low levels of variation.
The remaining changes pertain to both regression analysis (Parts I-III) and the design and
analysis of experimental and observational studies (Parts IV-VI):
11. We have made extensive revisions to the problem material. Problem data sets are
generally larger and more challenging, and we have included a large number of new
case data sets in Appendix C. In addition, we have added a new category of chapter
exercises, called Case Studies. These are open-ended problems that require students,
given an overall objective, to carry out complete analyses of the various case data sets in
Appendix C. They are distinct from the material in the Problems and Projects sections,
which frequently ask students to simply carry out specific .analytical procedures.
12. We have substantially expanded the amount of graphic presentation, including much
greater use of scatter plot matrices, three-dimensional rotating plots, three-dimensional
response surface and contour plots, conditional effects plots, and main effects and
interaction plots.
13. Throughout the text, we have made extensive revisions in the exposition on the basis
of classroom experience to improve the clarity of the presentation.
We have included in this book not only the more conventional topics in regression and
design, but also topics that are frequently slighted, though important in practice. We devote
three chapters (Chapters 9-11) to the model-building process for regression, including
computer-assisted selection procedures for identifying good subsets of predictor variables
X Preface
The Student Solutions Manual and all of the data files on the compact disk can also be
downloaded from the book's website at: www.mhhe.com/kutnerALSM5e.Alist of errata
for the book as well as some useful, related links will also be maintained at this address.
A book such as this cannot be written without substantial assistance from numerous
persons. We are indebted to the many contributors who have developed the theory and
practice discussed in this book. We also would like to acknowledge appreciation to our stu
dents, who helped us in a variety of ways to fashion the method of presentation contained
herein. We are grateful to the many users of Applied Linear Statistical Models and Applied
Linear Regression Models, who have provided us with comments and suggestions based
on their teaching with these texts. We are also indebted to Professors James E. Holstein,
University of Missouri, and David L. Sherry, University of West Florida, for their review of
Applied Linear Statistical Models, First Edition; to Professors Samuel Kotz, University of
Maryland at College Park, Ralph P. Russo, University ofIowa, and Peter F. Thall, The George
Washington University, for theirreview of Applied Linear Regression Models, First Edition;
to Professors John S. Y Chiu, University of Washington, James A. Calvin, University of
Iowa, and Michael F. Driscoll, Arizona State University, for their review of Applied Linear
Statistical Models, Second Edition; to Professor Richard Anderson-Sprecher, University
of Wyoming, for his review of Applied Linear Regression Models, Second Edition; and to
Professors Alexander von Eye, The Pennsylvania State University, Samuel Kotz, University
of Maryland at College Park, and John B. Willett, Harvard University, for their review of
Applied Linear Statistical Models, Third Edition; to Professors Jason Abrevaya, Univer
sity of Chicago, Frank Alt, University of Maryland, Vitoria Chen, Georgia Tech, Rebecca
Doerge, Purdue University, Mark Henry, Clemson University, Jim Hobert, University of
Florida, Ken Koehler, Iowa State University, Chii-Dean Lin, University of Massachussets
Amherst, Mark Reiser, Arizona State University, Lawrence Ries, University of Missouri
Columbia, and Ehsan Soofi, University of Wisconsin Milwaukee, for their reviews of
Applied Linear Regression Models, Third Edition, or Applied Linear Statistical Models,
Fourth Edition. These reviews provided many important suggestions, for which we are
most grateful.
In addition, valuable assistance was provided by Professors Richard K. Burdick,
Arizona State University, R. Dennis Cook, University of Minnesota. W. J. Conover, Texas
Tech University, Mark E. Johnson, University of Central Florida. Dick DeVeaux, Williams
College, and by Drs. Richard I. Beckman, Los Alamos National Laboratory, Ronald L.
Iman, Sandia National Laboratories, Lexin Li, University of California Davis, and Brad
Jones, SAS Institute. We are most appreciative of their willing help. We are also indebted
to the 88 participants in a survey concerning Applied Linear Regression Models, Second
Edition, the 76 participants in a survey concerning Applied Linear Statistical Models, Third
Edition, and the 73 participants in a survey concerning Applied Linear Regression Models,
Third Edition, or Applied Linear Statistical Models, Fourth Edition. Helpful suggestions
were received in these surveys, for which we are thankful.
Weiyong Zhang and Vincent Agboto assisted us diligently in the development of new
problem material, and Lexin Li and Yingwen Dong helped prepare the revised Instructor
Solutions Manual and Student Solutions Manual under considerable time pressure. Amy
Hendrickson provided much-needed LaTeX expertise. George Cotsonis assisted us dili
gently in preparing computer-generated plots and in checking analysis results. We are most
Preface xi
grateful to these persons for their invaluable help and assistance. We also wish to thank
the various members of the Carlson Executive MBA Program classes of 2003 and 2004;
notably Mike Ohmes, Trevor Bynum, Baxter Stephenson, Zakir Salyani, Sanders Marvin,
Trent Spurgeon, Nate Ogzawalla, David Mott, Preston McKenzie, Bruce Dejong, and TIm
Kensok, for their contributions of interesting and relevant case study data and materials.
Finally, our families bore patiently the pressures caused by our commitment to complete
this revision. We are appreciative of their understanding.
Michael H. Kutner
Christopher J. Nachtsheim
John Neter
Williamli
Contents
PART ONE Cited References 33
SIMPLE LINEAR REGRESSION 1 Problems 33
Exercises 37
Chapter 1 Projects 38
Linear Regression with One Predictor
Chapter 2
Variable 2
Inferences in Regression and Correlation
1.1 Relations between Variables 2 Analysis 40
Functional Relation between Two
2 1 Inferences Concerning f31 40
Variables 2
Statistical Relation between Tho Variables 3 Sampling Distribution of b I 41
1.2 Regression Models and Their Uses 5 Sampling Distribution of(bl - ,8d/s{bd 44
Confidence Interval for ,81 45
Historical Origins 5
Tests Concerning ,81 47
Basic Concepts 5
2.2 Inferences Concerning f30 48
Construction ofR egression Models 7
Uses ofR egression Analysis 8 Sampling Distribution ofbo 48
Regression and Causality 8 Sampling Distribution of(bo - ,8o)/s{bo} 49
Confidence Interval for f30 49
Use of Computers 9
23 Some Considerations on Making Inferences
1.3 Simple Linear Regression Model
Concerning f30 and f31 50
with Distribution of Error Tenns
Effects of Departures from Normality 50
Unspecified 9
Interpretation of Confidence Coefficient
Formal Statement ofM odel 9
and Risks of Errors 50
Important Features ofM odel 9
Spacing of the X Levels 50
Meaning ofR egression Parameters 11
Power of Tests 50
Alternative Versions ofR egression Model 12
1.4 Data for Regression Analysis 12 2.4 Interval Estimation of E{Yh} 52
Observational Data 12 Sampling Distribution ofYh 52
Sampling Distribution of
Experimental Data 13
Completely Randomized Design 13 (Yh - E{Yh})/s{Y,J 54
1.5 Overview of Steps in Regression Confidence Interval for E {Yh} 54
2.5 Prediction of New Observation 55
Analysis 13
1.6 Estimation of Regression Function 15 Prediction Interval for Yh(new) when
Parameters Known 56
Method ofL east Squares 15
Prediction Interval for Yh(new) when
Point Estimation ofM ean Response 21
Parameters Unknown 57
Residuals 22
Prediction ofM ean of m New Observations
Properties ofF ined Regression Line 23
1.7 Estimation of Error Tenns Variance 0-2 24 for Given Xh 60
2.6 Confidence Band for Regression Line 61
Point Estimator of0 -2 24
2.7 Analysis of Variance Approach
1.B Normal Error Regression Model 26
to Regression Analysis 63
Model 26
Partitioning of Total Sum of Squares 63
Estimation of Parameters by Method
Breakdown of Degrees ofF reedom 66
ofM aximum Likelihood 27
xii
Contents xiii
Mean Squares 66 3.4 Overview of Tests Involving
Analysis of Variance Table 67 Residuals 114
Expected Mean Squares 68 Tests for Randomness lI4
F Test of f31 = 0 verSUS f31 =1= 0 69 Tests for Constancy of Variance lI5
2.8 General Linear Test Approach 72 Tests for Outliers 115
Full Model 72 ~ests for Normality 115
Reduced Model 72 3.5 Correlation Test for Nonnality 115
Test Statistic 73 3.6 Tests for Constancy of Error
Summary 73 Variance 116
2.9 Descriptive Measures of Linear Association Brown-Forsythe Test 116
between X and Y 74 Breusch-Pagan Test lI8
Coefficient ofD etermination 74 3.7 F Test for Lack of Fit 119
Limitations of R2 75 Assumptions 119
Coefficient of Correlation 76 Notation 121
2.10 Considerations in Applying Regression Full Model 121
Analysis 77 Reduced Model 123
211 Nonnal Correlation Models 78 Test Statistic 123
Distinction between Regression and ANOVA Table 124
Correlation Model 78 3.8 Overview of Remedial Measures 127
Bivariate Normal Distribution 78 Nonlinearity ofR egression
Conditional Inferences 80 Function 128
Inferences on Correlation Coefficients 83 Nonconstancy of Error Variance 128
Spearman Rank Correlation Coefficient 87 Nonindependence of Error Terms 128
Cited References 89 Nonnormality of Error Terms 128
Problems 89 Omission of Important Predictor
Exercises 97 Variables 129
Projects 98 Outlying Observations 129
3.9 Transfonnations 129
Chapter 3 Transformations for Nonlinear
Diagnostics and Remedial Measures 100 Relation Only 129
Transformations for Nonnormality
3.1 Diagnostics for Predictor Variable 100
and Unequal Error Variances 132
3.2 Residuals 102
Box-Cox Transformations 134
Properties ofR esiduals 102 3.10 Exploration of Shape of Regression
Semistudentized Residuals 103
Function 137
Departures from Model to Be Studied by
Lowess Method 138
Residuals 103
Use of Smoothed Curves to Confirm Hued
3.3 Diagnostics for Residuals 103
Regression Function 139
Nonlinearity ofR egression Function 104
3.11 Case Example-Plutonium
Nonconstancy of Error Variance 107 - Measurement 141
Presence of Outliers 108 Cited References 146
Nonindependence ofE rror Terms 108
Problems 1.46
Nonnormality of Error Terms 110 Exercises 151
Omission ofI mportant Predictor
Projects 152
Variables 112
Case Studies 153 ',,-
Some Final Comments 114
xiv Contents
Chapter 4 Vector and Matrix with All Elements
Unity 187
Simultaneous Inferences and Other
Zero Vector 187
Topics in Regression Analysis 154
5.5 Linear Dependence and Rank
4.1 Joint Estimation of f30 and f31 154 of Matrix 188
Need for Joint Estimation 154 Linear Dependence 188
Bonferroni Joint Confidence Intervals 155 Rank ofM atrix 188
4.2 Simultaneous Estimation of Mean 5.6 Inverse of a Matrix 189
Responses 157 Finding the Inverse 190
Working-Hotelling Procedure 158 Uses of Inverse Matrix 192
Bonferroni Procedure 159 5.7 Some Basic Results for Matrices 193
4.3 Simultaneous Prediction Intervals 5.B Random Vectors and Matrices 193
for New Observations 160 Expectation ofR andom Vector or Matrix
4.4 Regression through Origin 161 Variance-Covariance Matrix
Model 161 ofR andom Vector 194
Inferences 161 Some Basic Results 196
Important Cautionsfor Using Regression Multivariate Normal Distribution 196
through Origin 164 5.9 Simple Linear Regression Model
4.5 Effects of Measurement Errors 165 in Matrix Terms 197
Measurement Errors in Y 165 5.10 Least Squares Estimation
Measurement Errors in X 165 of Regression Parameters 199
Berkson Model 167 Normal Equations 199
4.6 Inverse Predictions 168 Estimated Regression Coefficients 200
4.7 Choice of X Levels 170 5.11 Fitted Values and Residuals 202
Cited References 172 Fitted Values 202
Problems 172 Residuals 203
Exercises 175 5.12 Analysis of Variance Results 204
Projects 175 Sums of Squares 204
Sums of Squares as Quadratic
II! Chapter 5 Forms 205
i
In Matrix Approach to Simple 5.13 Inferences in Regression Analysis 206
0'
II Linear Regression Analysis 176 Regression Coefficients 207
I!/
Mean Response 208
5.1 Matrices 176
Prediction ofN ew Observation 209
II Definition ofM atrix 176 Cited Reference 209
,I Square Matrix 178 Problems 209
I"I Vector 178 Exercises 212
Transpose 178
Il
Ii Equality ofM atrices 179 PART TWO
5.2 Matrix Addition and Subtraction 180
MULTIPLE LINEAR
5.3 Matrix MUltiplication 182
II REGRESSION 213
Multiplication of a Matrix by a Scalar 182
Multiplication of a Matrix by a Matrix 182
Chapter 6
II 5.4 Special Types of Matrices 185
Multiple Regression I 214
II Symmetric Matrix 185
Diagonal Matrix 185 6.1 Multiple Regression Models 214
!!