Applied Linear Regression Models

728 Pages·2004·4.113 MB·English

by  Michael H. Kutner, Christopher J Nachtsheim, John Neter

Applied Linear
Regression Models
The McGraw-Hill/Irwin Series: Operations and Decision Sciences
BUSINESSSTATISTICS Doane,Mathieson,andTracy Wilson,Keating,andJohnGalt
Alwan VisualStatistics Solutions,Inc.
StatisticalProcessAnalysis SecondEdition,2.0 BusinessForecasting
FirstEdition Gitlow,Oppenheim,andOppenheim FourthEdition
AczelandSounderpandian QualityManagement:Toolsand Zagorsky
CompleteBusinessStatistics MethodsforImprovement BusinessInformation
FifthEdition SecondEdition FirstEdition
BowermanandO’Connell Lind,Marchal,andWathen
BusinessStatisticsinPractice BasicStatisticsforBusiness
ThirdEdition andEconomics QUANTITATIVEMETHODS
FourthEdition ANDMANAGEMENT
BryantandSmith SCIENCE
PracticalDataAnalysis: Lind,Marchal,andMason
CaseStudiesinBusinessStatistics, StatisticalTechniquesinBusiness Bodily,Carraway,Frey,Pfeifer
VolumesI,II,III andEconomics QuantitativeBusinessAnalysis:
SecondEdition EleventhEdition TextandCases
FirstEdition
CooperandSchindler Merchant,Goffinet,andKoehler
BusinessResearchMethods BasicStatisticsUsingExcel Bonini,Hausman,andBierman
EighthEdition forOffice2000 QuantitativeAnalysisforBusiness
ThirdEdition Decisions
Delurgio NinthEdition
ForecastingPrinciples SahaiandKhurshid
andApplications PocketDictionaryofStatistics HillierandHillier
FirstEdition FirstEdition IntroductiontoManagementScience:
AModelingandCaseStudiesApproach
Doane Siegel withSpreadsheets
LearningStats PracticalBusinessStatistics SecondEdition
FirstEdition FifthEdition
Applied Linear
Regression Models
Fourth Edition
Michael H. Kutner
EmoryUniversity
Christopher J. Nachtsheim
UniversityofMinnesota
John Neter
UniversityofGeorgia
Boston Burr Ridge, IL Dubuque, IA Madison, WI New York San Francisco St. Louis
Bangkok Bogotá Caracas Kuala Lumpur Lisbon London Madrid Mexico City
Milan Montreal New Delhi Santiago Seoul Singapore Sydney Taipei Toronto
APPLIEDLINEARREGRESSIONMODELS
PublishedbyMcGraw-Hill/Irwin,abusinessunitofTheMcGraw-HillCompanies,Inc.,1221Avenueofthe
Americas,NewYork,NY,10020.Copyright(cid:1)c 2004,1996,1989,1983byTheMcGraw-HillCompanies,Inc.
Allrightsreserved.Nopartofthispublicationmaybereproducedordistributedinanyformorbyanymeans,
orstoredinadatabaseorretrievalsystem,withoutthepriorwrittenconsentofTheMcGraw-HillCompanies,
Inc.,including,butnotlimitedto,inanynetworkorotherelectronicstorageortransmission,orbroadcastfor
distancelearning.Someancillaries,includingelectronicandprintcomponents,maynotbeavailableto
customersoutsidetheUnitedStates.
Thisbookisprintedonacid-freepaper.
1234567890DOC/DOC 0987654
ISBN0-07-301344-7
Editorialdirector: BrentGordon
Executiveeditor: RichardT.Hercher,Jr.
Editorialassistant: LeeStone
Producer,Mediatechnology: AnthonySherman
Projectmanager: JimLabeots
Productionsupervisor: GinaHangos
Leaddesigner: PamVerros
Supplementproducer: MatthewPerry
Seniordigitalcontentspecialist: BrianNacik
Coverdesign: KieraPohl
Typeface: 10/12TimesRoman
Compositor: InteractiveCompositionCorporation
Printer: R.R.Donnelley
LibraryofCongressCataloging-in-PublicationData
Kutner,MichaelH.
Appliedlinearregressionmodels.—4thed./MichaelH.Kutner,ChristopherJ.Nachtsheim,JohnNeter.
p. cm.
Earliered.enteredundertitleofJohnNeter.
Includesbibliographicalreferencesandindex.
ISBN0-07-238691-6(acid-freepaper)
1.Regressionanalysis. I.Nachtsheim,ChristopherII.Neter,John.III.Title.
QA278.2.N462004
519.5(cid:1)36—dc21 2003044224
www.mhhe.com
To
Nancy, Michelle, Allison,
Maureen, Abigael, Andrew, Henry G.,
Dorothy, Ron, David
Preface
Linear regression models are widely used today in business administration, economics,
engineering, and the social, health, and biological sciences. Successful applications of
thesemodelsrequireasoundunderstandingofboththeunderlyingtheoryandthepractical
problems that are encountered in using the models in real-life situations. While Applied
LinearRegressionModels,FourthEdition,isbasicallyanappliedbook,itseekstoblend
theoryandapplicationseffectively,avoidingtheextremesofpresentingtheoryinisolation
andofgivingelementsofapplicationswithouttheneededunderstandingofthetheoretical
foundations.
Thefourtheditiondiffersfromthethirdinanumberofimportantrespects.
1. We have reorganized the chapters for better clarity and flow of topics. Material from
the old Chapter 15 on normal correlation models has been integrated throughout the
textwhereappropriate.MuchofthematerialisnowfoundinanexpandedChapter2,
which focuses on inference in regression analysis. Material from the old Chapter 7
pertaining to polynomial and interaction regression models and from old Chapter 11
onqualitativepredictorshasbeenintegratedintoanewChapter8called,“Regression
ModelsforQuantitativeandQualitativePredictors.”Materialonmodelvalidationfrom
oldChapter10isnowfullyintegratedwithupdatedmaterialonmodelselectioninanew
Chapter9entitled,“BuildingtheRegressionModelI:ModelSelectionandValidation.”
2. Wehaveaddedmaterialonimportanttechniquesfordatamining,includingregression
treesandneuralnetworkmodels,inChapters11and13,respectively.
3. Wehavemadeextensiverevisionstotheproblemmaterial.Problemdatasetsaregen-
erallylargerandmorechallenging,andwehaveincludedalargenumberofnewcase
data sets in Appendix C. In addition, we have added a new category of chapter exer-
cises,calledCaseStudies.Theseareopen-endedproblemsthatrequirestudents,given
an overall objective, to carry out complete analyses of the various case data sets in
AppendixC.TheyaredistinctfromthematerialintheProblemsandProjectssections,
whichfrequentlyaskstudentstosimplycarryoutspecificanalyticalprocedures.
4. The chapter on logistic regression (Chapter 14) has been extensively revised and ex-
panded to include a more thorough treatment of logistic, probit, and complementary
log-logmodels,logisticregressionresiduals,modelselection,modelassessment,logis-
ticregressiondiagnostics,andgoodnessoffittests.Wehavealsodevelopednewmate-
rialonpolytomous(multicategory)nominallogisticregressionmodelsandpolytomous
ordinallogisticregressionmodels.
5. Wehaveexpandedthediscussionofmodelselectionmethodsandcriteria.TheAkaike
information criterion and Schwarz Bayesian criterion have been added, and a greater
emphasisisplacedontheuseofcross-validationformodelselectionandvalidation.
6. We have substantially expanded the amount of graphic presentation, including much
greateruseofscatterplotmatrices,three-dimensionalrotatingplots,three-dimensional
responsesurfaceandcontourplots,andconditionaleffectsplots.
7. Throughoutthetext,wehavemadeextensiverevisionsintheexpositiononthebasisof
classroomexperiencetoimprovetheclarityofthepresentation.
vi
Preface vii
Wehaveincludedinthisbooknotonlythemoreconventionaltopicsinregression,butalso
topicsthatarefrequentlyslighted,thoughimportantinpractice.Wedevotethreechapters
(Chapter9–11)tothemodel-buildingprocessforregression,includingcomputer-assisted
selection procedures for identifying good subsets of predictor variables and validation of
the chosen regression model. Two chapters (Chapters 8 and 14) are devoted to indicator
variables, covering both response and predictor indicator variables. The use of residual
analysisandotherdiagnosticsforexaminingtheappropriatenessofaregressionmodelis
a recurring theme throughout this book. So is the use of remedial measures that may be
helpfulwhenthemodelisnotappropriate.Intheanalysisoftheresultsofastudy,wegive
greater emphasis to the use of estimation procedures than to significance tests, because
estimationisoftenmoremeaningfulinpractice.Also,sincepracticalproblemsseldomare
concernedwithasingleinference,westresstheuseofsimultaneousinferenceprocedures.
Theoreticalideasarepresentedtothedegreeneededforgoodunderstandinginmaking
soundapplications.Proofsaregiveninthoseinstanceswherewefeeltheyservetodemon-
strateanimportantmethodofapproach.Emphasisisplacedonathoroughunderstandingof
theregressionmodels,particularlythemeaningofthemodelparameters,sincesuchunder-
standingisbasictoproperapplications.Awidevarietyofexamplesandcasesispresented
toillustratetheuseofthetheoreticalprinciples,toshowthegreatdiversityofapplicationsof
regressionmodels,andtodemonstratehowanalysesarecarriedoutfordifferentproblems.
Weuse“Comments”sectionsineachchaptertopresentadditionaldiscussionandmatters
related to the mainstream of development. In this way, the basic ideas in a chapter are
presentedconciselyandwithoutdistraction.
Applicationsofregressionmodelsfrequentlyrequireextensivecomputations.Wetake
thepositionthatacomputerisavailableinmostappliedworkandthatalmosteverycomputer
userhasaccesstoprogrampackagesforregressionanalysis.Hence,weexplainthebasic
mathematicalstepsinfittingaregressionmodelbutdonotdwelloncomputationaldetails.
This approach permits us to avoid many complex formulas and enables us to focus on
basicprinciples.Wemakeextensiveuseinthistextofcomputercapabilitiesforperforming
computationsandpreparinggraphicplots,andweillustrateavarietyofcomputerprintouts
andplotsandexplainhowtheyareusedforanalysis.
Aselectionofproblemsisprovidedattheendofeachchapter.Herereaderscanreinforce
theirunderstandingofthemethodologyandusetheconceptslearnedtoanalyzedata.We
havebeencarefultosupplydata-analysisproblemsthattypifygenuineapplications.Inmost
problemsthecalculationsarebesthandledonacomputer.Tofacilitatedataentry,acompact
diskisprovidedwiththetextthatincludesthedatasetsforallexamples,problems,exercises,
projects,andcasestudies,aswellasforthedatasetsinAppendixC.TheREADME.TXT
fileonthecompactdiskprovidesinformationabouttheidentificationofthedatasets.
WeassumethatthereaderofAppliedLinearRegressionModels,FourthEdition,hashad
anintroductorycourseinstatisticalinference,coveringthematerialoutlinedinAppendixA.
Shouldsomegapsinthereader’sbackgroundexist,therelevantportionsofanintroductory
textcanbestudied,ortheinstructoroftheclassmayusesupplementalmaterialsforcovering
themissingsegments.AppendixAisprimarilyintendedasareferenceofbasicstatistical
resultsforcontinuinguseasthereaderprogressesthroughthebook.
CalculusisnotrequiredforreadingAppliedLinearRegressionModels,FourthEdition.
In a number of instances, we use calculus to demonstrate how some important results
are obtained, but these demonstrations are confined to supplementary comments and can
viii Preface
be omitted without any loss of continuity. Readers who do know calculus will find these
comments in natural sequence so that the benefits of the mathematical developments are
obtainedintheirimmediatecontext.Somebasicelementsofmatrixalgebraareneededfor
multipleregression.Chapter5introducestheseelementsofmatrixalgebrainthecontext
ofsimpleregressionforeasylearning.
AppliedLinearRegressionModels,FourthEdition,isintendedforuseinundergraduate
andgraduatecoursesinregressionanalysisandinsecondcoursesinappliedstatistics.The
extenttowhichmaterialpresentedinthistextisusedinaparticularcoursedependsuponthe
amountoftimeavailableandtheobjectivesofthecourse.Thebasicelementsofregression
arecoveredinChapters1,2,3,4(Sections4.1–4.3),5,6,7,8,9,and10.Chapters11,12,
13,and14canbecoveredastimepermitsandinterestsdictate.
Thisbookcanalsobeusedforself-studybypersonsengagedinthefieldsofbusiness
administration,economics,engineering,andthesocial,health,andbiologicalscienceswho
desiretoobtaincompetenceintheapplicationofregressionmodels.
AnInstructorSolutionsManual,containingdetailedsolutionstoallnumericalproblems
and analytical exercises, is available from the publisher, McGraw-Hill/Irwin, for use by
instructors.
A Student Solutions Manual is included on the compact disk for use by students. The
StudentSolutionsManualprovidesintermediateandfinalnumericalresultsforeasyself-
checkingofsolutionsforselectedproblems.Weuseanasterisk(*)infrontoftheproblem
numbertodesignatetheproblemsforwhichthesolutionsappearintheStudentSolutions
Manual.
Available new for this Fourth Edition is a SAS and SPSS Program Solution Manual
forexamplescontainedinthetext.ThemanualwaspreparedbyDr.WilliamReplogleand
Dr.WilliamJohnson,bothoftheUniversityofMississippiMedicalSchool.Themanualuses
datafilestopresentsolutions,output,anddiscussionofnecessarystepsforstudentstosolve
examplesselectedfromthebookusingSASandSPSS.
TheStudentSolutionsManualandallofthedatafilesonthecompactdiskcanalsobe
downloadedfromthebook’swebsiteat:www.mhhe.com/KutnerALRM4e.Alistoferrata
forthebookaswellassomeuseful,relatedlinkswillalsobemaintainedatthisaddress.
A book such as this cannot be written without substantial assistance from numerous
persons. We are indebted to the many contributors who have developed the theory and
practicediscussedinthisbook.Wealsowouldliketoacknowledgeappreciationtoourstu-
dents,whohelpedusinavarietyofwaystofashionthemethodofpresentationcontained
herein.WearegratefultothemanyusersofAppliedLinearStatisticalModelsandApplied
Linear Regression Models, who have provided us with comments and suggestions based
on their teaching with these texts. We are also indebted to Professors James E. Holstein,
UniversityofMissouri,andDavidL.Sherry,UniversityofWestFlorida,fortheirreviewof
AppliedLinearStatisticalModels,FirstEdition;toProfessorsSamuelKotz,Universityof
MarylandatCollegePark,RalphP.Russo,UniversityofIowa,andPeterF.Thall,TheGeorge
WashingtonUniversity,fortheirreviewofAppliedLinearRegressionModels,FirstEdition;
to Professors John S. Y Chiu, University of Washington, James A. Calvin, University of
Iowa,andMichaelF.Driscoll,ArizonaStateUniversity,fortheirreviewofAppliedLinear
StatisticalModels,SecondEdition;toProfessorRichardAnderson-Sprecher,Universityof
Wyoming,forhisreviewofAppliedLinearRegressionModels,SecondEdition;andtoPro-
fessorsAlexandervonEye,ThePennsylvaniaStateUniversity,SamuelKotz,Universityof
Preface ix
MarylandatCollegePark,andJohnB.Willett,HarvardUniversity,fortheirreviewofAp-
pliedLinearStatisticalModels,ThirdEdition;toProfessorsJasonAbrevaya,Universityof
Chicago,FrankAlt,UniversityofMaryland,VitoriaChen,GeorgiaTech,RebeccaDoerge,
Purdue University, Mark Henry, Clemson University, Jim Hobert, University of Florida,
KenKoehler,IowaStateUniversity,Chii-DeanLin,UniversityofMassachusettsAmherst,
MarkReiser,ArizonaStateUniversity,LawrenceRies,UniversityofMissouriColumbia,
andEhsanSoofi,UniversityofWisconsinMilwaukee,fortheirreviewsofAppliedLinear
Regression Models, Third Edition, or Applied Linear Statistical Models, Fourth Edition.
Thesereviewsprovidedmanyimportantsuggestions,forwhichwearemostgrateful.
Inaddition,valuableassistancewasprovidedbyProfessorsRichardK.Burdick,Arizona
StateUniversity,R.DennisCook,UniversityofMinnesota,W.J.Conover,TexasTechUni-
versity,MarkE.Johnson,UniversityofCentralFlorida,DickDeVeaux,WilliamsCollege,
andbyDrs.RichardI.Beckman,LosAlamosNationalLaboratory,RonaldL.Iman,Sandia
NationalLaboratories,andBradJones,SASInstitute.Wearemostappreciativeoftheirwill-
inghelp.Wearealsoindebtedtothe88participantsinasurveyconcerningAppliedLinear
Regression Models, Second Edition, the 76 participants in a survey concerning Applied
LinearStatisticalModels,ThirdEdition,andthe73participantsinasurveyconcerningAp-
pliedLinearRegressionModels,ThirdEdition,orAppliedLinearStatisticalModels,Fourth
Edition.Helpfulsuggestionswerereceivedinthesesurveys,forwhichwearethankful.
Weiyong Zhang and Vincent Agboto assisted us in the development of new prob-
lem material, and Lexin Li and Yingwen Dong prepared the revised Instructor Solutions
ManualandStudentSolutionsManualunderconsiderabletimepressure.AmyHendrickson
providedmuch-neededLaTeXexpertise.Wearemostgratefultotheseindividualsfortheir
invaluablehelpandassistance.WealsowishtothankthevariousmembersoftheCarlson
ExecutiveMBAProgramclassesof2003and2004,notablyMikeOhmes,TrevorBynum,
BaxterStephenson,ZakirSalyani,SandersMarvin,TrentSpurgeon,NateOgzawalla,David
Mott,PrestonMcKenzie,BruceDeJong,andTimKensok,fortheircontributionsofinter-
estingandrelevantcasestudydataandmaterials.
Finally,ourfamiliesborepatientlythepressurescausedbyourcommitmenttocomplete
thisrevision.Weareappreciativeoftheirunderstanding.
MichaelH.Kutner
ChristopherJ.Nachtsheim
JohnNeter
Contents
PART ONE CitedReferences 33
Problems 33
SIMPLELINEARREGRESSION 1
Exercises 37
Chapter 1 Projects 38
LinearRegressionwithOnePredictor
Chapter 2
Variable 2
InferencesinRegressionandCorrelation
1.1 RelationsbetweenVariables 2 Analysis 40
FunctionalRelationbetweenTwo
2.1 InferencesConcerningβ 40
Variables 2 1
SamplingDistributionofb 41
StatisticalRelationbetweenTwoVariables 3 1
SamplingDistributionof(b −β )/s{b } 44
1.2 RegressionModelsandTheirUses 5 1 1 1
ConfidenceIntervalforβ 45
HistoricalOrigins 5 1
TestsConcerningβ 47
BasicConcepts 5 1
2.2 InferencesConcerningβ 48
ConstructionofRegressionModels 7 0
SamplingDistributionofb 48
UsesofRegressionAnalysis 8 0
SamplingDistributionof(b −β )/s{b } 49
RegressionandCausality 8 0 0 0
ConfidenceIntervalforβ 49
UseofComputers 9 0
2.3 SomeConsiderationsonMakingInferences
1.3 SimpleLinearRegressionModel
Concerningβ andβ 50
withDistributionofErrorTerms 0 1
EffectsofDeparturesfromNormality 50
Unspecified 9
InterpretationofConfidenceCoefficient
FormalStatementofModel 9
andRisksofErrors 50
ImportantFeaturesofModel 9
SpacingoftheXLevels 50
MeaningofRegressionParameters 11
PowerofTests 50
AlternativeVersionsofRegressionModel 12
2.4 IntervalEstimationof E{Y } 52
1.4 DataforRegressionAnalysis 12 h
SamplingDistributionofYˆ 52
ObservationalData 12 h
SamplingDistributionof
ExperimentalData 13
(Yˆ −E{Y })/s{Yˆ } 54
CompletelyRandomizedDesign 13 h h h
ConfidenceIntervalforE{Y } 54
1.5 OverviewofStepsinRegression h
2.5 PredictionofNewObservation 55
Analysis 13
1.6 EstimationofRegressionFunction 15 PredictionIntervalforYh(new)when
ParametersKnown 56
MethodofLeastSquares 15
PointEstimationofMeanResponse 21 PredictionIntervalforYh(new)when
ParametersUnknown 57
Residuals 22
PredictionofMeanofmNewObservations
PropertiesofFittedRegressionLine 23
1.7 EstimationofErrorTermsVarianceσ2 24 forGivenXh 60
PointEstimatorofσ2 24 2.6 ConfidenceBandforRegressionLine 61
2.7 AnalysisofVarianceApproach
1.8 NormalErrorRegressionModel 26
toRegressionAnalysis 63
Model 26
PartitioningofTotalSumofSquares 63
EstimationofParametersbyMethod
BreakdownofDegreesofFreedom 66
ofMaximumLikelihood 27
x