Statistics as Principled Argument

238 Pages·1995·4.01 MB·English

by  Robert P. Abelson

Statistics as Principled Argument

Statistics as Principled Argument
Robert P. Abelson
Yale University
Psychology Press
Taylor & Francis Group
New York London
Psychology Press Psychology Press
Taylor & Francis Group Taylor & Francis Group
711 Third Avenue 27 Church Road
New York, NY 10017 Hove, East Sussex BN3 2FA
© 1995 by Taylor & Francis Group, LLC
Originally published by Lawrence Erlbaum Associates
Reprinted in 2009 by Psychology Press, an imprint of Taylor & Francis Group
Cover design by Mairav Salomon-Dekel
International Standard Book Number-13: 978-0-8058-0528-4 (Softcover)
Except as permitted by U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or
utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including
photocopying, microfilming, and recording, or in any information storage or retrieval system, without written
permission from the publishers.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only
for identification and explanation without intent to infringe.
Library of Congress Cataloging-in-Publication Data
Catalog record is available from the Library of Congress
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the Psychology Press Web site at
http://www.psypress.com
This book is dedicated to Willa Dinwoodie Abelson and to
John Wilder Tukey, who in their special ways have
encouraged me in seeking wisdom’s direction.

Contents
Preface xi
Abelson’s Laws xv
1 Making Claims With Statistics 1
Misunderstandings of Statistics 1
Claims Made With Statistics: Comparison and Explanation 2
Language and Limitations of Null Hypothesis Tests 8
Persuasive Arguments: The MAGIC Criteria 11
Style and Convention 15
The Bottom Line 16
2 Elementary Arguments and the Role of Chance 17
Random Generating Processes 18
Random Sampling Processes 27
Summary 36
3 Magnitude of Effects 39
Probability Measures 39
Effect Sizes 45
Confidence Limits 52
vii
viii CONTENTS
4 Styles of Rhetoric 54
Brash, Stuffy, Liberal, and Conservative Styles 54
One-Tailed, Two-Tailed, and Lopsided Tests 57
Alternative Tests Applied to the Same Data Set 59
Defective Observations 68
Multiple Tests Within the Same Data Set 70
Stating and Interpreting p Values 74
In the Last Analysis 77
5 On Suspecting Fishiness 78
Strange Distributions of Observations 78
The Occurrence of Impossible Scores 88
Strange Test Statistics 89
Inconsistencies Between Parallel Statistics 97
Too Much Consistency in Parallel Statistics 100
A Cautionary Note on the Detection of Fishiness 103
6 Articulation of Results: Ticks and Buts 104
Ticks and Buts 104
Ticks and the Evolution of Knowledge 105
Comparisons of Pairs of Means 108
Refraining Results for Better Articulation 114
Multiple Comparisons 118
Contrasts 125
More Than One Dependent Variable 127
Further Remarks 130
7 Generality of Effects 132
The Nature of Generality 132
Treatment-by-Context Interactions Within Studies 135
Generality Across Studies: Meta-Analysis 149
Generality Across and Within Studies: A Parallel 153
Final Caveat 155
8 Interestingness of Argument 156
Can Statistics Be Interesting? 156
Theoretical Interest 158
Surprisingness 160
Importance 168
CONTENTS ix
9 Credibility of Argument 170
Why Research Claims Are Disbelieved 170
The Structure of Debates on Disbelieved Claims 172
Methodological Artifacts 180
The Influence of Criticism on Methodology 197
References 199
Author Index 212
Subject Index 216