Table Of Contentebook
THE GUILFORD PRESS
Regression Analysis and Linear Models
Methodology in the Social Sciences
David A. Kenny, Founding Editor
Todd D. Little, Series Editor
www.guilford.com/MSS
This series provides applied researchers and students with analysis and research design
books that emphasize the use of methods to answer research questions. Rather than
emphasizing statistical theory, each volume in the series illustrates when a technique
should (and should not) be used and how the output from available software programs
should (and should not) be interpreted. Common pitfalls as well as areas of further
development are clearly articulated.
RECENT VOLUMES
DOING STATISTICAL MEDIATION AND MODERATION
Paul E. Jose
LONGITUDINAL STRUCTURAL EQUATION MODELING
Todd D. Little
INTRODUCTION TO MEDIATION, MODERATION, AND CONDITIONAL
PROCESS ANALYSIS: A REGRESSION-BASED APPROACH
Andrew F. Hayes
BAYESIAN STATISTICS FOR THE SOCIAL SCIENCES
David Kaplan
CONFIRMATORY FACTOR ANALYSIS FOR APPLIED RESEARCH,
SECOND EDITION
Timothy A. Brown
PRINCIPLES AND PRACTICE OF STRUCTURAL EQUATION MODELING,
FOURTH EDITION
Rex B. Kline
HYPOTHESIS TESTING AND MODEL SELECTION IN THE SOCIAL SCIENCES
David L. Weakliem
REGRESSION ANALYSIS AND LINEAR MODELS:
CONCEPTS, APPLICATIONS, AND IMPLEMENTATION
Richard B. Darlington and Andrew F. Hayes
GROWTH MODELING: STRUCTURAL EQUATION
AND MULTILEVEL MODELING APPROACHES
Kevin J. Grimm, Nilam Ram, and Ryne Estabrook
PSYCHOMETRIC METHODS: THEORY INTO PRACTICE
Larry R. Price
Regression Analysis
and Linear Models
Concepts, Applications, and Implementation
Richard B. Darlington
Andrew F. Hayes
Series Editor’s Note by Todd D. Little
THE GUILFORD PRESS
New York London
Copyright © 2017 The Guilford Press
A Division of Guilford Publications, Inc.
370 Seventh Avenue, Suite 1200, New York, NY 10001
www.guilford.com
All rights reserved
No part of this book may be reproduced, translated, stored in a retrieval
system, or transmitted, in any form or by any means, electronic,
mechanical, photocopying, microfilming, recording, or otherwise,
without written permission from the publisher.
Printed in the United States of America
This book is printed on acid-free paper.
Last digit is print number: 9 8 7 6 5 4 3 2 1
Library of Congress Cataloging-in-Publication Data is available
from the publisher.
ISBN 978-1-4625-2113-5 (hardcover)
Series Editor’s Note
What a partnership: Darlington and Hayes. Richard Darlington is an icon
of regression and linear modeling. His contributions to understanding the
general linear model have educated social and behavioral science research-
ers for nearly half a century. Andrew Hayes is an icon of applied regression
techniques, particularly in the context of mediation and moderation. His
contributions to conditional process modeling have shaped how we think
about and test processes of mediation and moderation. Bringing these two
icons together in collaboration gives us a work that any researcher should
use to learn and understand all aspects of linear modeling. The didactic
elements are thorough, conversational, and highly accessible. You’ll enjoy
Regression Analysis and Linear Models, not as a statistics book but rather as
a Hitchhiker’s Guide to the world of linear modeling. Linear modeling is
the bedrock material you need to know in order to grow into the more
advanced procedures, such as multilevel regression, structural equation
modeling, longitudinal modeling, and the like. The combination of clarity,
easy-to-digest “bite-sized” chapters, and comprehensive breadth of cover-
age is just wonderful. And the software coverage is equally comprehensive,
with examples in SAS, STATA, and SPSS (and some nice exposure to R)—
giving every discipline’s dominant software platform a thorough coverage.
In addition to the software coverage, the various examples that are used
span many disciplines and offer an engaging panorama of research ques-
tions and topics to stimulate the intellectually curious (a remedy for “aca-
demic attention deficit disorder”).
This book is not just about linear regression as a technique, but also
about research practice and the origins of scientific knowledge. The
v
vi Series Editor’s Note
thoughtful discussion of statistical control versus experimental control, for
example, provides the basis to understand when causal conclusions are suf-
ficiently implicated. As such, policy and practice can, in fact, rely on well-
crafted nonexperimental analyses. Practical guidance is also a hallmark
of this work, from detecting and managing irregularities, to collinearity
issues, to probing interactions, and so on. I particularly appreciate that they
take linear modeling all the way up through path analysis, an essential
starting point for many advanced latent variable modeling procedures.
This book will be well worn, dog-eared, highlighted, shared, re-read,
and simply cherished. It will now be required reading for all of my first-
year students and a recommended primer for all of my courses. And if you
are planning to come to one of my Stats Camp courses, brush up by review-
ing Darlington and Hayes.
As always, “Enjoy!” Oh, and to paraphrase the catch phrase from the
Hitchhiker’s Guide to the Galaxy: “Don’t forget your Darlington and Hayes.”
TODD D. LITTLE
Kicking off my Stats Camp
in Albuquerque, New Mexico
Preface
Linear regression analysis is by far the most popular analytical method in
the social and behavioral sciences, not to mention other fields like medi-
cine and public health. Everyone is exposed to regression analysis in some
form early on who undertakes scientific training, although sometimes that
exposure takes a disguised form. Even the most basic statistical proce-
dures taught to students in the sciences—the t-test and analysis of variance
(ANOVA), for instance—are really just forms of regression analysis. After
mastering these topics, students are often introduced to multiple regression
analysis as if it is something new and designed for a wholly different type
of problem than what they were exposed to in their first course. This book
shows how regression analysis, ANOVA, and the independent groups t-test
are one and the same. But we go far beyond drawing the parallels between
these methods, knowing that in order for you to advance your own study
in more advanced statistical methods, you need a solid background in the
fundamentals of linear modeling. This book attempts to give you that back-
ground, while facilitating your understanding using a conversational writ-
ing tone, minimizing the mathematics as much as possible, and focusing
on application and implementation using statistical software.
Although our intention was to deliver an introductory treatment of
regression analysis theory and application, we think even the seasoned
researcher and user of regression analysis will find him- or herself learn-
ing something new in each chapter. Indeed, with repeated readings of this
book we predict you will come to appreciate the glory of linear modeling
just as we have, and maybe even develop the kind of passion for the topic
that we developed and hope we have successfully conveyed to you.
vii
viii Preface
Regression analysis is conducted with computer software, and you
have many good programs to choose from. We emphasize three commer-
cial packages that are heavily used in the social and behavioral sciences:
IBM SPSS Statistics (referred to throughout the book simply as “SPSS”),
SAS, and STATA. A fourth program, R, is given some treatment in one of the
appendices. But this book is about the concepts and application of regres-
sion analysis and is not written as a how-to guide to using your software.
We assume that you already have at least some exposure to one of these
programs, some working experience entering and manipulating data, and
perhaps a book on your program available or a local expert to guide you
as needed. That said, we do provide relevant commands for each of these
programs for the key analyses and uses of regression analysis presented
in these pages, using different fonts and shades of gray to most clearly dis-
tinguish them from each other. Your program’s reference manual or user’s
guide, or your course instructor, can help you fine-tune and tailor the com-
mands we provide to extract other information from the analysis that you
may need one day.
In this rest of this preface, we provide a nonexhaustive summary of the
contents of the book, chapter by chapter, to give you a sense of what you
can expect to learn about in the pages that follow.
Overview of the Book
Chapter 1 introduces the book by focusing on the concept of “accounting
for something” when interpreting research results, and how a failure to
account for various explanations for an association between two variables
renders that association ambiguous in meaning and interpretation. Two
examples are offered in this first chapter, where the relationship between
two variables changes after accounting for the relationship between these
two variables and a third—a covariate. These examples are used to intro-
duce the concept of statistical control, which is a major theme of the book.
We discuss how the linear model, as a general analytic framework, can be
used to account for covariates in a flexible, versatile manner for many types
of data problems that a researcher confronts.
Chapters 2 and 3 are perhaps the core of the book, and everything that
follows builds on the material in these two chapters. Chapter 2 introduces
the concept of a conditional mean and how the ordinary least squares crite-
rion used in regression analysis for defining the best-fitting model yields a
model of conditional means by minimizing the sum of the squared resid-
uals. After illustrating some simple computations, which are then repli-
cated using regression routines in SPSS, SAS, and STATA, distinctions are
drawn between the correlation coefficient and the regression coefficient as
