Beginning Behavioral Research: A Conceptual Primer

405 Pages·2012·5.109 MB·English

by  Ralph L. Rosnow, Robert Rosenthal

Beginning Behavioral
S
EVENTH
Research
E
DITION
A Conceptual Primer
RALPH L. ROSNOW
Emeritus, Temple University
ROBERT ROSENTHAL
University of California, Riverside
Boston Columbus Indianapolis New York San Francisco Upper Saddle River
Amsterdam Cape Town Dubai London Madrid Milan Munich Paris Montreal Toronto
Delhi Mexico City São Paulo Sydney Hong Kong Seoul Singapore Taipei Tokyo
Editorial Director: Craig Campanella
Editor in Chief: Jessica Mosher
Executive Editor: Stephen Frail
Editorial Assistant: Madelyn Schricker
Marketing Manager: Brigeth Rivera
Marketing Assistant: Jessica Warren
Production Manager: Holly Shufeldt
Full-Service Project Management/Composition: Integra Software Services, Ltd.
Cover, Creative Director: Jayne Conte
Cover Designer: Suzanne Behnke
Cover Art: Fotolia
Cover Printer: Lehigh-Phoenix Color/Hagerstown
Printer/Bindery: Edwards Brothers
Copyright © 2013, 2008, 2005, 2002 by Pearson Education, Inc. All rights reserved. Printed in the United States of
America. This publication is protected by Copyright and permission should be obtained from the publisher prior to any
prohibited reproduction, storage in a retrieval system, or transmission in any form or by any means, electronic, m echanical,
photocopying, recording, or likewise. To obtain permission(s) to use material from this work, please submit a written
request to Pearson Education, Inc., Permissions Department, One Lake Street, Upper Saddle River, New Jersey 07458 or you
may fax your request to 201-236-3290.
Library of Congress Cataloging-in-Publication Data
Rosnow, Ralph L.
Beginning behavioral research: a conceptual primer / Ralph L. Rosnow.—7th ed.
p. cm.
ISBN-13: 978-0-205-81031-4 (alk. paper)
ISBN-10: 0-205-81031-4 (alk. paper)
1. Psychology—Research—Methodology—Textbooks. 2. Social sciences—Research—Methodology—Textbooks.
I. Title.
BF76.5.R64 2013
300.72—dc23 2011046807
10 9 8 7 6 5 4 3 2 1
ISBN 10: 0-205-81031-4
ISBN 13: 978-0-205-81031-4
Contents
Preface ix What Are Hypothesis-Generating
Heuristics? 22
About the Authors xiii
What Is the Potential Role of
Serendipity? 25
P I
ART GETTING STARTED How Can I Do a Literature Search? 27
How Should I Go About Defining
1 Variables? 29
Behavioral Research and the
What Identifies “Good” Theories and
Scientific Method 1 Working Hypotheses? 30
What Is the Distinction between an
Preview Questions 1
Independent Variable and a Dependent
Why Study Research Methods and Data
Variable? 31
Analysis? 1
What Belongs in My Research Proposal? 33
What Alternatives Are There to the Scientific
Summary of Ideas 38
Method? 3
Key Terms 38
How Do Scientists Use Empirical
Multiple-Choice Questions for Review 38
Reasoning? 5
Discussion Questions for Review 39
How Is Empirical Reasoning Used in
Behavioral Research? 7 Answers to Review Questions 40
How Do Extraempirical Factors Come into
Play? 9
3
What Does Behavioral Research Cover? 11 Ethical Considerations and
How Does Research Go From Descriptive Guidelines 41
to Relational to Experimental? 12
Preview Questions 41
What Are the Characteristics of Good
Researchers? 14 How Do Ethical Guidelines in Research
Function? 41
Summary of Ideas 17
What Is Informed Consent, and When Is It
Key Terms 17
Used? 43
Multiple-Choice Questions for Review 17
How Are Ethics Reviews Done
Discussion Questions for Review 18
and Acted On? 45
Answers to Review Questions 19
What Are Obstacles to the Rendering of
“Full Justice”? 48
2 How Can a “Relationship of Trust” Be
From Hunches to Testable
Established? 49
Hypotheses 20
How Do Scientific Quality and Ethical
Quality Intertwine? 50
Preview Questions 20
Is Deception in Research Ever Justified? 51
What Is Meant by a Cycle of Discovery and
What Is the Purpose of Debriefing, and
Justification? 20
How Is It done? 55
iii
IV Contents
How Is Animal Research Governed by Ethical What Are Open-Ended and Fixed-Choice
Rules? 56 Items? 85
What Ethical Responsibilities Are There How Are Personality and Projective Tests
When Writing Up Research? 57 Used? 86
Summary of Ideas 59 What Is Meant by Measuring Implicit
Key Terms 60 Attitudes? 88
Multiple-Choice Questions for Review 60 What Are Numerical, Forced-Choice, and
Graphic Ratings? 89
Discussion Questions for Review 61
What Are Rating Errors, and How Are They
Answers to Review Questions 61
Controlled? 91
What Is the Semantic Differential
P II
ART O Method? 93
BSERVATION AND
What Are Likert Scales and Thurstone
M Scales? 95
EASUREMENT
How Are Items Prepared for a Questionnaire
4 or an Interview? 98
Methods of Systematic
How Are Face-to-Face and Telephone
Observation 63 Interviews Done? 99
How Are Behavioral Diaries Used in
Preview Questions 63
Research? 101
What Is Meant By Systematic
Summary of Ideas 102
Observation? 63
Key Terms 103
How Do Researchers Simultaneously
Multiple-Choice Questions for Review 103
Participate and Observe? 64
Discussion Questions for Review 104
What Can Be Learned from Quantifying
Observations? 66 Answers to Review Questions 105
How Are Judgment Studies Done? 69
How Does Content Analysis Work? 71 6
Reliability and Validity in Measurement
How Are Situations Simulated in Controlled
Settings? 73 and Research 106
What Are Plausible Rival Hypotheses and the
Preview Questions 106
Third-Variable Problem? 75
What Is the Difference Between Validity and
What Is the Distinction Between Reactive
Reliability? 106
and Nonreactive Observation? 77
What Are Random and Systematic
Summary of Ideas 79
Errors? 108
Key Terms 79
What Is the Purpose of Retest and
Multiple-Choice Questions for Review 79 Alternate-Form Reliability? 110
Discussion Questions for Review 80 What Is Internal-Consistency Reliability, and
Answers to Review Questions 80 How Is It Increased? 111
What Are Acceptable Test-Retest and
Internal-Consistency Reliabilities? 113
5
Methods for Looking Within How Is the Reliability of Judges
Measured? 114
Ourselves 82
How Is Reliability Related to Replication and
Preview Questions 82 External Validity? 116
What Are the Uses and Limitations of How Are Content and Criterion Validity
Self-Report Measures? 82 Defined? 118
Contents v
How Is Construct Validity Assessed in Test 8
Nonrandomized Research and Causal
Development? 119
How Is Construct Validity Relevant to Reasoning 149
Experimental Design? 122
Preview Questions 149
What Is the Importance of Statistical-
How Is Causal Reasoning Attempted in the
Conclusion Validity and Internal
Absence of Randomization? 149
Validity? 123
How Is the Third-Variable Problem
Summary of Ideas 124
Relevant? 151
Key Terms 125
What Is Meant By Subclassification on
Multiple-Choice Questions for Review 125
Propensity Scores? 152
Discussion Questions for Review 125
What Are Time-Series Designs and “Found
Answers to Review Questions 126 Experiments”? 154
What Within-Subjects Designs Are Used in
P III Single-Case Experiments? 155
ART D
ESIGN AND How Are Correlations Interpreted in
Cross-Lagged Panel Designs? 158
I
MPLEMENTATION
What Is the Difference Between Longitudinal
and Cross-Sectional Research? 160
7
Randomized Experiments and Summary of Ideas 161
Key Terms 162
Causal Inference 128
Multiple-Choice Questions for Review 162
Preview Questions 128
Discussion Questions for Review 163
What Is the Purpose of Randomized
Answers to Review Questions 163
Experiments? 128
How Is Random Assignment
Accomplished? 130 9
Survey Research and Subject
What Are Between-Subjects Designs? 131
Recruitment 164
What Is the Formative Logic of Experimental
Control? 132
Preview Questions 164
What Are Within-Subjects Designs? 134
What Are Opportunity and Probability
What Are Factorial Designs? 135
Samples? 164
What Is Meant By Counterbalancing the
What Is Meant By Bias and Instability in
Conditions? 136
Survey Research? 165
Why Is Causality Said to Be “Shrouded in
Why Do We Not Know “For Sure” the Bias in
Mystery”? 137
Sampling? 167
How Do Scientists Logically Puzzle Out
How Is Simple Random Sampling
Efficient Causality? 138
Done? 168
What Conditions Pose a Threat to Internal
What Are Stratified Random Sampling and
Validity? 139
Area Probability Sampling? 170
What Are Artifacts in Research? 141
What Did the Literary Digest Case Teach
Summary of Ideas 145 Pollsters? 171
Key Terms 146 What Are Point Estimates and Interval
Multiple-Choice Questions for Estimates? 173
Review 146 What Are the Benefits of Stratification? 174
Discussion Questions for Review 147 How Is Nonresponse Bias Handled in Survey
Answers to Review Questions 147 Research? 176
VI Contents
What Are the Typical Characteristics of 11
Correlating Variables 204
Volunteer Subjects? 177
How Is Volunteer Bias in Opportunity Preview Questions 204
Samples Managed? 179
What Are Different Forms of
Summary of Ideas 181 Correlations? 204
Key Terms 181 How Are Correlations Visualized in Scatter
Multiple-Choice Questions for Review 182 Plots? 206
Discussion Questions for Review 182 How Is a Product-Moment Correlation
Answers to Review Questions 183 Calculated? 207
How Is Dummy Coding Used in
Correlation? 208
P IV
ART D D When Is the Phi Coefficient Used? 210
ESCRIBING ATA AND
How Is a Correlation Calculated on
D I Ranks? 212
RAWING NFERENCES
Summary of Ideas 215
10 Key Terms 215
Summarizing the Data 184
Multiple-Choice Questions for Review 215
Preview Questions 184 Discussion Questions for Review 216
How Is Visual Integrity Ensured When Answers to Review Questions 217
Results Are Graphed? 184
How Are Frequencies Displayed in 12
Tables, Bar Graphs, and Line Understanding p Values and Effect
Graphs? 185 Size Indicators 219
How Do Stem-and-Leaf Charts Work? 187
Preview Questions 219
How Are Percentiles Used to Summarize
Part of a Batch? 189 Why Is It Important to Focus Not Just on
Statistical Significance? 219
How Is an Exploratory Data Analysis
Done? 189 What Is the Reasoning Behind Null
Hypothesis Significance Testing? 220
How Does Asymmetry Affect Measures of
Central Tendency? 191 What Is the Distinction Between Type I
Error and Type II Error? 222
How Do I Measure How “Spread Out” a Set
of Scores Is? 193 What Are One-Tailed and Two-Tailed
p Values? 224
What Are Descriptive and Inferential
Measures? 195 What Is the Counternull Statistic? 226
How Do I Estimate a Confidence Interval What Is the Purpose of Doing a Power
Around a Population Mean? 196 Analysis? 227
What Is Distinctive About the Normal How Do I Estimate a Confidence Interval
Distribution? 197 for an Effect Size Correlation? 229
Why Are z Scores Called Standard Scores, What Can Effect Sizes Tell Us of Practical
and How Are They Used? 198 Importance? 230
Summary of Ideas 200 What Does Killeen’s prep Tell Me? 234
Key Terms 201 Summary of Ideas 235
Multiple-Choice Questions for Key Terms 235
Review 201 Multiple-Choice Questions for Review 236
Discussion Questions for Review 202 Discussion Questions for Review 236
Answers to Review Questions 202 Answers to Review Questions 237
Contents vii
P V
ART S T How Do I Interpret Main and Interaction
TATISTICAL ESTS
Effects? 269
How Do I Compute a Two-Way
13
The Comparison of Two ANOVA and Set Up a Summary
Table? 270
Conditions 238
What Are Contrasts, and How Do I
Preview Questions 238 Compute Them on More Than Two
What Do Signal-to-Noise Ratios Have to Do Groups? 273
With t Tests? 238 What Do r , r , and r
effect size alerting contrast
How Do I Compute an Independent-Sample Tell Me? 276
t Test? 240 How Are Contrasts on Multiple Repeated
What Can a Table of p Values for t Teach Measures Computed? 277
Me? 242 How Are Latin Square Designs
What Is an Effect Size Index for an Analyzed? 280
Independent-Sample t? 244 Summary of Ideas 283
How Do I Interpret Cohen’s d for Key Terms 284
Independent Groups? 246
Multiple-Choice Questions for Review 284
How Do I Compute Interval Estimates for
Discussion Questions for Review 284
Cohen’s d? 248
Answers to Review Questions 285
How Can I Maximize the Independent-
Sample t? 250
How Does a Paired t Test Differ From an
15
Independent-Sample t Test? 251 The Analysis of Frequency
What Is an Effect Size Index for a
Tables 288
Paired t? 252
Summary of Ideas 254 Preview Questions 288
Key Terms 254 What Is the Purpose of Chi-Square
((cid:2)2)? 288
Multiple-Choice Questions for
Review 255 How Do I Compute 1-df
Chi-Squares? 289
Discussion Questions for Review 255
How Do I Obtain the p Value,
Answers to Review Questions 256
Effect Size, and Confidence
Interval? 291
14 What Is the Relationship Between
Comparisons of More Than Two 1-df (cid:2)2 and Phi? 293
Conditions 258
How Do I Deal With Tables Larger Than
2 (cid:3) 2? 293
Preview Questions 258
How Is Standardizing the Margins Done,
What Is Analysis of Variance (ANOVA), and
and What Can It Tell Me? 295
How Are F and t Related? 258
What Is a Binomial Effect-Size Display
How Is Variability Apportioned in a
Used For? 296
One-Way ANOVA? 261
A Journey Begun 298
How Are ANOVA Summary Tables Set Up
Summary of Ideas 299
and Interpreted? 263
Key Terms 299
How Can I Test for Simple Effects After an
Omnibus F? 266 Multiple-Choice Questions for Review 299
How Is Variability Apportioned in a Two- Discussion Questions for Review 300
Way ANOVA? 267 Answers to Review Questions 301
VIII Contents
Appendix A Reporting Your Research B.5. r Values and Their Associated
p Values 335
Results 303
B.6. Transformations of r to Fisher z 336
r
Research Reports in APA Style 303 B.7. Transformations of Fisher z
r
Getting Started 304 to r 337
Title Page 305
Abstract 305
Appendix C Introduction to
Introduction 305
Meta-Analysis 338
Method 320
Results 321 The Purpose of Meta-Analysis 338
Discussion 322 Some Pro and Con Arguments 338
References 322 Comparing Two Effect Sizes 339
Footnotes 323 Combining Two Effect Sizes 340
Tables and Figures 323 Obtaining an Overall Significance
Level 341
Appendix 323
Detective-Like Probing of Reported
Writing and Revising 324
Data 342
The File Drawer Problem 342
Appendix B Statistical Tables 325
B.1. z Values and Their Associated Glossary of Terms 344
One-Tailed p Values 326
B.2. t Values and Their Associated
References 357
One-Tailed and Two-Tailed
p Values 327
B.3. F Values and Their Associated Name Index 375
p Values 329
B.4. (cid:2) 2 Values and Their Associated
Subject Index 381
p Values 334
Preface
Welcome to the seventh edition of Beginning Behavioral Research: A Conceptual Primer. This book
was conceived as a text for students who, as part of a course in research methods, are expected
to plan an empirical study, to analyze and interpret the data, and to present their findings and
conclusions in a written report. The word Primer in the subtitle is intended to communicate the
idea that BBR 7th edition is focused on the rudiments, or first principles, of the application of the
scientific method in behavioral research. For philosophers of science as well as many p hilosophically
oriented psychologists, the scientific method has long been a fascinating and enigmatic part of
the puzzle of how scientists arrive at a rational, logically consistent picture of the psychological
world in which we live. As one psychologist put it, any metaphor used to characterize the scientific
method would have to be a mixed one:
It is not an explanatory web, a predictive network, a descriptive grammar, an experiential map, a
technological abacus, a practical almanac, or a moral calculus. It is not an arsenal of methods—l ogical,
mathematical, or instrumentative. (Koch, 1959, p. 2)
As we show in this book, the scientific method is all of these things—and much more. Applying
it to the wide range of questions of interest to behavioral researchers can be challenging, but also
great fun.
Although Beginning Behavioral Research was conceived as an undergraduate text for students
planning to do research, it has also been used in ways that go beyond that original purpose.
For example, it has been used in methods courses in which the production of a research project
was not a major goal. It has been used by undergraduate honors students and master’s degree
students in different disciplines or programs as a primary text and by doctoral students to ease
themselves into our advanced text, Essentials of Behavioral Research (Rosenthal & Rosnow, 1991,
2008). We have tried to anticipate and confront questions and uncertainties from the student’s
perspective not necessarily as a potential professional producer of empirical research, but as an
intelligent consumer of scientific findings. Our hope is that BBR 7th edition will continue to teach
students to understand not only the first principles of behavioral research, but also the difference
between good science and pseudoscience and the exacting standards of sound research.
Our Approach
Although much is new in this seventh edition, we have not tinkered with the overall organization of
the book, so that instructors who have used a previous edition will have no difficulty integrating this
edition into their course outlines and lectures. The chapters follow a linear sequence in which five
major parts (Getting Started, Observation and Measurement, Design and Implementation, Describing
Data and Drawing Inferences, and Statistical Tests) are subdivided into three chapters each. Once
again, there is a sample research proposal (in Chapter 2); instructions on writing a final research
report, along with a sample report, in Appendix A; an introduction to meta-analysis in Appendix C;
and a glossary of terms. The appendices and glossary are again tabbed so that they are easy to
find, and there is a summary list of statistical equations inside the front and back covers. As before,
preview questions open each chapter and serve as section headings in the material that follows.
Each chapter ends with a summary of ideas, a list of key terms, and multiple-choice and discussion
questions and answers for review.
Those familiar with our other work will know that we are not wedded to any single scientific
method, theory, or unit of analysis (Rosenthal & Rosnow, 2008; Rosnow, 1981). As the range of
ix