Design and Analysis: A Researcher's Handbook

622 Pages·2004·40.432 MB·English

by  Geoffrey Keppel, Thomas D. Wickens

Design and Analysis
A Researcher's Handbook
Fourth Edition
Geoffrey Keppel
Thomas D. Wickens
University of California, Berkeley
•
/).
JI\.·
._._,
Upper Saddle River, New Jersey 07458
Library of Congress Cataloging-in-Publication Data
Keppel, Geoffrey.
Design and analysis: a researcher's handbook.-4th ed. / Geoffrey Keppel, Thomas D. Wickens.
p. cm.
Includes bibliographical references and index.
ISBN 0-13-515941-5 (alk. paper)
1. Social sciences-Statistical methods. 2. Factorial experiment designs. 3. Social
sciences-Research. I. Wickens, Thomas D., 1942. II. Title.
HA29.K44 2004
300'.72'4-dc22 2003064774
Senior Acquisitions Editor: Jayme Heffler
Editor-in-Chief: Leah Jewell
Editorial Assistant: Jennifer M. Conklin
Marketing Manager: Sheryl Adams
Assistant Managing Editor (Production): Maureen Richardson
Manufacturing Buyer: Tricia Kenny
Cover Design: Karen Salzbach
Composition/Full-Service Project Management: Fran Daniele, Prepare
Cr(ldits and acknowledgments borrowed from other sources and reproduced, with permission, in this
textbook appear on appropriate page within text.
Copyright © 2004, 1991, 1982, 1973 by Pearson Education, Inc., Upper Saddle River,
New Jersey, 07458.
Pearson Prentice Hall. All rights reserved. Printed in the United States of America. This publica
tion is protected by Copyright and permission should be obtained from the publisher prior to any
prohibited reproduction, storage in a retrieval system, or transmission in any form or by any means,
electronic, mechanical, photocopying, recording, or likewise. For information regarding permission(s),
write to: Rights and Permissions Department.
Pearson Prentice Hall™ is a trademark of Pearson Education, Inc.
Pearson® is a registered trademark of Pearson pie
Prentice Hall® is a registered trademark of Pearson Education, Inc.
Pearson Education LTD., London
Pearson Education Australia PTY, Limited, Sydney
Pearson Education Singapore, Pte. Ltd
Pearson Education North Asia Ltd, Hong Kong
Pearson Education Canada, Ltd., Toronto
Pearson Educaci6n de Mexico, S.A. de C.V.
Pearson Education-Japan, Tokyo
Pearson Education Malaysia, Pte. Ltd
•
10 9 8 7 6 5 4 3 2
ISBN 0-13-515941-5
To Sheila and Lucia
Contents
I INTRODUCTION 1
1 Experimental Design 2
1.1 Variables in Experimental Research 2
1.2 Control in Experimentation . . . 7
1.3 Populations and Generalizing . . 9
1.4 The Basic Experimental Designs 10
II SINGLE-FACTOR EXPERIMENTS 13
2 Sources of Variability and Sums of Squares 15
2.1 The Logic of Hypothesis Testing .. 18
2.2 The Component Deviations . . . . . . . . . 22
2.3 Sums of Squares: Defining Formulas . . . . 24
2.4 Sums of Squares: Computational Formulas 26
Exercises ................... . 31
3 Variance Estimates and the F Ratio 32
3.1 Completing the Analysis ... 32
3.2 Evaluating the F Ratio . . . . . 37
3.3 Errors in Hypothesis Testing . . 46
3.4 A Complete Numerical Example 50
3.5 Unequal Sample Sizes 54
Exercises ............... . 58
4 Analytical Comparisons Among Means 60
4.1 The Need for Analytical Comparisons . 60
4.2 An Example of Planned Comparisons . 62
4.3 Comparisons Among Treatment Means . 64
4.4 Evaluating Contrasts with a t Test . . . 71
4.5 Orthogonal Contrasts . . . . . . . . . . 76
4.6 Composite Contrasts Derived from Theory 79
4. 7 Comparing Three or More Means . 83
Exercises ..................... . 84
CONTENTS V
5 Analysis of Trend 88
5.1 Analysis of Linear Trend. 88
5.2 Analysis of Quadratic Trend . . . 95
5.3 Higher-Order Trend Components 98
5.4 Theoretical Prediction of Trend Components 100
5.5 Planning a Trend Analysis. 103
5.6 Monotonic Trend Analysis . 108
Exercises ............ . 109
6 Simultaneous Comparisons 111
6.1 Research Questions and Type I Error 112
6.2 Planned Comparisons .... 115
6.3 Restricted Sets of Contrasts . 117
6.4 Pairwise Comparisons ... 120
6.5 Post-Hoc Error Correction. 128
Exercises ............ . 130
7 The Linear Model and It~ Assumptions 132
7.1 The Statistical Model .......... . 132
7.2 Sampling Bias and the Loss of Subjects . 137
7.3 Violations of Distributional Assumptions 141
7.4 Dealing with Heterogeneity of Variance 150
7.5 Contrasts with Heterogeneous Variance 156
Exercises ................... . 158
8 Effect Size, Power, and Sample Size 159
8.1 Descriptive Measures of Effect Size 159
8.2 Effect Sizes in the Population 163
8.3 Power and Sample Size . 167
8.4 Determining Sample Size 169
8.5 Determining Power . 177
Exercises ........... . 179
9 Using Statistical Software 181
9.1 Using the Programs .... 182
9.2 An Example . . . . . . . . . 185
9.3 Hints, Cautions, and Advice . 189
Exercises ............. . 191
III TWO-WAY FACTORIAL EXPERIMENTS 193
10 Introduction to Factorial Designs 195
10.1 Basic Information from a Factorial Design 195
10.2 The Concept of Interaction ... 198
10.3 The Definition of an Interaction . . . . . . 201
vi CONTENTS
10.4 Further Examples of Interaction . . . . 204
10.5 Measurement of the Dependent Variable 206
Exercises ................. . 209
11 The Overall Two-Factor Analysis 211
11.1 Component Deviations ....... . 211
11.2 Computations in the Two-Way Analysis 214
11.3 A Numerical Example ..... 220
11.4 The Statistical Model . . . . . 224
11.5 Designs with a Blocking Factor 228
11.6 Measuring Effect Size . 232
11.7 Sample Size and Power 235
Exercises .......... . 239
12 Main Effects and Simple Effects 242
12.1 Interpreting a Two-Way Design .. 242
12.2 Comparisons for the Marginal Means . 244
12.3 Interpreting the Interaction 246
12.4 Testing the Simple Effects . . . . . . . 249
12.5 Simple Comparisons . . . . . . . . . . 256
12.6 Effect Sizes and Power for Simple Effects 258
12. 7 Controlling Familywise Type I Error 262
Exercises .................... . 264
13 The Analysis of Interaction Components 266
13.1 Types of Interaction Components . 266
13.2 Analyzing Interaction Contrasts . . . . 271
13.3 Orthogonal Interaction Contrasts . . . . 276
13.4 Testing Contrast-by-Factor Interactions 277
13.5 Contrasts Outside the Factorial Structure 282
13.6 Multiple Tests and Type I Error 285
Exercises .................... . 286
IV THE GENERAL LINEAR MODEL 289
14 The General Linear Model 291
14.1 The General Linear Model ...... . 292
14.2 The Two-Factor Analysis ...... . 298
14.3 Averaging of Groups and Individuals . 304
14.4 Contrasts and Other Analytical Analyses 307
14.5 Sensitivity to Assumptions 309
Exercises .................... . 310
15 The Analysis of Covariance 311
15.1 Covariance and Linear Regression 312
15.2 The Analysis of Covariance .... 318
CONTENTS vii
15.3 Adjusted Means . . . . . . . . . . . . . . . 324
15.4 Extensions of the Design .......... . 327
15.5 Assumptions of the Analysis of Covariance 330
15.6 Blocking and the Analysis of Covariance . 336
15. 7 Preexisting Covariate Differences . . 337
15.8 Effect Sizes, Power, and Sample Size 341
Exercises ................. . 344
V WITHIN-SUBJECT DESIGNS 347
16 The Single-Factor Within-Subject Design 350
16.1 The Analysis of Variance 350
16.2 Analytical Comparisons 356
16.3 Effect Size and Power 361
16.4 Computer Analysis . 365
~xercises ......... . 366
17 Further Within-Subject Topics 369
17.1 Advantages and Limitations . 369
17.2 The Statistical Model . . . 372
17.3 The Sphericity Assumption . 376
17.4 Incidental Effects ...... . 379
17.5 Analyzing a Counterbalanced Design . 386
17.6 Missing Data in Within-Subject Designs 393
Exercises ................... . 399
18 The Two-Factor Within-Subject Design 401
18.1 The Overall Analysis ........... . 401
18.2 Contrasts and Other Analytical Analyses 408
18.3 Assumptions and the Statistical Model . 417
18.4 Counterbalancing of Nuisance Variables 419
18.5 Effect Size and Sample Sizes 426
Exercises ................... . 429
19 The Mixed Design: Overall Analysis 432
19.1 The Overall Analysis of Variance . 432
19.2 Statistical Model and Assumptions .. . 441
19.3 The Multivariate Alternative ..... . 444
19.4 Missing Data and Unequal Sample Sizes 446
19.5 Effect Sizes and Sample-Size Calculations 446
Exercises .................... . 448
20 The Mixed Design: Analytical Analyses 449
20.1 Analysis of the Between-Subjects Factor 450
20.2 Analysis of the Within-Subject Factor 453
20.3 Analyses Involving the Interaction 458
Exercises .................. . 461
viii CONTENTS
VI HIGHER FACTORIAL DESIGNS
AND OTHER EXTENSIONS 463
21 The Overall Three-Factor Design 465
21.1 Components of the Three-Way Design 465
21.2 The Three-Way Interaction . . . . . 470
21.3 Computational Procedures . . . . . 476
21.4 Effect Size, Sample Size, and Power 480
Exercises ................. . 484
22 The Three-Way Analytical Analysis 486
22.1 Overview of the Analytical Analysis 486
22.2 Analyses Involving the Cell Means . . 490
22.3 Effects Based on Marginal Means ... 495
22.4 Three-Factor Interaction Components 499
22.5 Contrast-by-Factor Interactions ... 504
22.6 Extension to Higher-Order Designs . 506
Exercises ................. . 507
23 Within-Subject and Mixed Designs 510
23.1 Varieties of Three-Factor Designs . 510
23.2 The Overall Analysis . . . . . . . . . . . 512
23.3 Two Examples of Mixed Designs . . . . 514
23.4 Analytical Analyses in the A x B x C x S Design 516
23.5 Analytical Analysis in Mixed Designs 518
Exercises ......................... . 528
24 Random Factors and Generalization 530
24.1 Statistical Generalization over Design Factors . 530
24.2 Random Factors and the F Ratio . . . . . . . 534
24.3 Error Terms in Random-Factor Designs ... 537
24.4 Design Considerations with Random Factors 546
Exercises ...................... . 549
25 Nested Factors 550
25.1 Nested Factors 550
25.2 Analysis of the Nested Designs ........ . 553
25.3 Crossing a Nested Factor with Another Factor 562
25.4 Planning a Study with Random Factors 563
Exercises ....................... . 564
26 Higher-Order Designs 566
26.1 Multifactor Experiments in the Behavioral Sciences . 566
26.2 Analyzing Higher-Order Designs 567
Exercises· .......................... . 569
CONTENTS ix
A Statistical Tables 571
A.1 Critical Values of the F distribution 571
A.2 Critical Values of the t Distribution 576
A.3 Coefficients of Orthogonal Polynomials . 577
A.4 Critical Values of the Sidak-Bonferroni t Statistic . 578
A.5 Critical Values for Dunnett's Test ......... . 582
A.6 Critical Values of the Studentized Range Statistic . 586
A. 7 Power Functions . . . . . . . . . . . . . . . . . . . 590
B Abbreviated Answers to the Exercises 596
References 602
Subject Index 607
Author Index 612
Preface
This fourth edition of Design and Analysis is the result of a collaboration that began
informally about 25 years ago, when TDW was spending a sabbatical year at Berkeley
and attended GK's seminar on experimental design. We have continued our discus
sions of research design and statistical analysis over the years. TDW was intimately
familiar with the earlier editions of this book, using it as a text in his graduate courses
at UCLA and contributing substantially to the second and third editions. Our col
laboration on this new revision is the natural extension of our discussions. With our
equal participation, we have created an essentially new book. We have rewritten it
throughout, reorganized the chapters, and introduced much new material.
Our topic is the analysis of variance, the major statistical method used to analyze
experimental data in psychology and the behavioral sciences. It gives the researcher
a way to compare the means from a series of sets of scores and to determine whether
they differ. More than just identifying overall differences, however, the analysis of
variance provides a very specific set of procedures that allow a researcher to express
particular questions about the study and to measure and test them. Much of the
power and the subtlety of the analysis of variance lies in these analytical techniques,
which are necessary both when planning a study and when interpreting its results.
Accordingly, we devote over a third of the book to them.
The early part of the book contains a detailed discussion of the foundation block of
the analysis of variance-the experimental design with independent groups of subjects,
each of whom contributes a single observation to the analysis. We then elaborate this
structure by expanding it in several ways-to studies in which the groups are classified
in two or more ways, to studies with multiple observations, and to studies with several
sources of random variability. By the end of this book, a student or researcher will be
able to put these parts together and apply them to the analysis of any complex design.
We cover the major designs commonly used in psychological research and present,
with numerical examples, the analyses appropriate to them. Where possible, we cover
statistical arguments in the context of data analysis and interpretation. For example,
when we discuss the statistical assumptions that underlie an analysis, we both give
their mathematical form and describe how they relate to experimental procedures.
To the extent possible while keeping the size of our book within bounds, we describe
and illustrate how the analytical techniques are applied in both simple and complex
designs-too many books, in our opinion, fail the reader at this point.
We wrote the book with a particular reader in mind: a student at the advanced
undergraduate or beginning graduate level who is about to engage in experimental
research, but who has little or no formal mathematical or statistical background. We
tbfirefore include a discussion of statistical inference, and we limit our assumption of
mathematical skills to basic algebra. We give enough detail in our numerical examples