Single-Case and Small-N Experimental Designs: A Practical Guide to Randomization Tests

305 Pages·2011·5.617 MB·English

by  Pat Dugard, Portia File, Jonathan Todman

Single-case and
Small-n Experimental
Designs
A Practical Guide to
Randomization Tests
Second Edition

Single-case and
Small-n Experimental
Designs
A Practical Guide to
Randomization Tests
Second Edition
Pat Dugard
Dundee University, UK
Portia File
University of Abertay Dundee, UK
Jonathan Todman
NHS Scotland
Routledge Routledge
Taylor & Francis Group Taylor & Francis Group
711 Third Avenue 27 Church Road
New York, NY 10017 Hove, East Sussex BN3 2FA
© 2012 by Taylor & Francis Group, LLC
Routledge is an imprint of Taylor & Francis Group, an Informa business
Printed in the United States of America on acid-free paper
Version Date: 20110805
International Standard Book Number: 978-0-415-88622-2 (Hardback) 978-0-415-88693-2 (Paperback)
For permission to photocopy or use material electronically from this work, please access www.copyright.
com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood
Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and
registration for a variety of users. For organizations that have been granted a photocopy license by the
CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are
used only for identification and explanation without intent to infringe.
Library of Congress Cataloging-in-Publication Data
Dugard, Pat.
Single-case and small-n experimental designs : a practical guide to randomization
tests, second edition / Pat Dugard, Portia File, Jonathan Todman. -- 2nd ed.
p. cm.
Rev. ed. of: Single-case and small-n experimental designs / John B. Todman, Pat
Dugard. 2001.
Summary: “Randomization tests are not a new idea, but they only became really
useful after the advent of fast computing. Making randomization tests accessible
to many more potential users by providing the means to use them within familiar
statistical software, this book serves as an introduction and provides macros to
perform in the familiar environments of SPSS and Excel. Though we expect that
the book will still appeal to researchers, we believe the changes in the new edition
will make the book an essential aid for graduate and senior undergraduate courses
in statistics, data analysis, and/or research methods, taught in departments of
psychology (especially clinical or counseling psychology), medicine, nursing, and
other health and social sciences”-- Provided by publisher.
Includes bibliographical references and index.
ISBN 978-0-415-88693-2 (pbk.) -- ISBN 978-0-415-88622-2 ()
1. Statistical hypothesis testing. 2. Experimental design. I. File, Portia. II. Todman,
John B. III. Todman, John B. Single-case and small-n experimental designs. IV. Title.
QA277.T63 2011
519.5’6--dc23 2011028087
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the Psychology Press Web site at
http://www.psypress.com
Dedication
To the memory of John Todman

Contents
Preface .................................................................................................................ix
Chapter 1 Single-case and small-n designs in context ...........................1
Introduction ........................................................................................................1
Single-case and small-n designs in the research process .............................4
Randomization tests’ time has come ...............................................................7
Using this book...................................................................................................7
Chapter 2 Understanding randomization tests .....................................11
An introductory example: testing for extrasensory perception (ESP) .....11
Classes of randomization tests .......................................................................14
Hypothesis tests for randomization designs ...............................................16
How randomization tests differ from more familiar tests .........................31
Why randomization tests are not widely used ............................................32
Two ways to obtain the reference set ............................................................33
The number of arrangements needed for significance to be achievable ....34
Exercises ............................................................................................................35
Chapter 3 Obtaining the data: Choosing the design ............................37
Introduction ......................................................................................................37
Designs analogous to ANOVA .......................................................................41
Phase designs....................................................................................................45
A design to investigate order effects .............................................................50
Exercises ............................................................................................................51
Chapter 4 Obtaining the data: Implementing the design ...................53
Introduction ......................................................................................................53
Designs analogous to ANOVA .......................................................................54
Phase designs....................................................................................................78
A design to test order effects ..........................................................................90
Exercises ............................................................................................................92
vii
viii Contents
Chapter 5 Analyzing the data: Using the macros .................................93
Introduction ......................................................................................................93
Analyses using the SPSS macros ...................................................................94
Analyses using the Excel macros ..................................................................110
Exercises ..........................................................................................................130
Chapter 6 Analyzing the data: Wider considerations ........................131
Introduction: the myth of the random sample ..........................................131
Randomization ...............................................................................................132
Phase designs: response guided intervention ...........................................136
Incomplete randomization............................................................................142
Exercises ..........................................................................................................144
Chapter 7 Size and power ........................................................................147
Introduction ....................................................................................................147
Estimating α by sampling the reference set ...............................................147
Size and power: some terminology .............................................................148
Maximizing power ........................................................................................150
Power of randomization tests .......................................................................154
Exercises ..........................................................................................................157
Chapter 8 Going further ...........................................................................159
Introduction ....................................................................................................159
Other sources of software for randomization tests...................................159
Writing your own macros: some general advice ........................................162
Using parts of our macros to write one of your own: an example .........163
Further reading ..............................................................................................170
Solutions to even numbered exercises ........................................................173
Appendix 1 ......................................................................................................177
Appendix 2 ......................................................................................................187
Appendix 3 ......................................................................................................229
Glossary ...........................................................................................................275
References .......................................................................................................281
Author Index ...................................................................................................283
Subject Index ...................................................................................................285
Preface
This book is intended as a practical guide for students and researchers
who are interested in the statistical analysis of data from single-case or
very small-n experiments. We said this in the preface to the first edition,
and it is true of this new edition as well. Randomization tests are not a
new idea, but they became really useful only after the advent of fast com-
puting. John Todman, lead author of the first edition of this book, wanted
to make randomization tests accessible to many more potential users by
providing the means to use them within familiar statistical software. As
such, this book serves as an introduction to randomization tests and pro-
vides macros to perform some of them in the familiar environments of
SPSS and Excel, which are already being used for data analysis by thou-
sands of researchers. Minitab was dropped from this edition, because we
believe fewer people are using it now, but the macros (plus a new one)
for use with SPSS and Excel are still a central feature of the book. As well
as the macros, we provide all the information you need to apply them to
your own data.
Because most researchers are not familiar with randomization designs
and tests, even if they are quite confident with data analysis and statistical
inference, we have described the basic ideas of randomization tests using
several extended examples. We also use examples to show how to choose
a suitable design and, having made a choice, how to implement it and
analyze the results. The emphasis throughout is on practical application
and understanding when and how to use randomization tests. There is
a book Web site (http://www.researchmethodsarena.com/9780415886932)
from which you can take the macros and example data. There is a separate
site for instructors only that contains the solutions to the exercises that are
not included in the text.
If you are among that majority of researchers and students who are
unfamiliar with randomization tests, studying this book will enable you
to understand how randomization tests work, why they can be used in
ix