Table Of ContentRelated Titles
Einstein THE COMMUNITY'S RESPONSE TO DRUG USE
Einstein DRUGS IN RELATION TO THE DRUG USER
Gottheil, McLeUan & Druley SUBSTANCE ABUSE AND PSYCHIATRIC ILLNESS
Nahas KEEP OFF THE GRASS
Nahas & Paton MARIHUANA: BIOLOGICAL EFFECTS
Einstein BEYOND DRUGS
Gottheil, McLeUan, Druley & Alterman ADDICTION RESEARCH AND TREATMENT
StUlman & Willette THE PSYCHOPHARMACOLOGY OF HALLUCINOGENS
Deniker, Radouco-Thomas & ViUeneuve NEURO-PSYCHOPHARMACOLOGY
Evaluating Alcohol and
Drug Abuse Treatment
Effectiveness
Recent Advances
Edited by
Linda Carter Sobell
Mark B. Sobell
Elliott Ward
Pergamon Press
New York · Oxford · Toronto · Sydney · Frankfurt · Paris
Pergamon Press Offices:
U.S.A Pergamon Press Inc., Maxwell House, Fairview Park,
Elmsford, New York 10523, U.S.A.
U.K. Pergamon Press Ltd., Headington Hill Hall,
Oxford 0X3 OBW, England
CANADA Pergamon of Canada Ltd., 150 Consumers Road,
Willowdale, Ontario M2J 1P9, Canada
AUSTRALIA Pergamon Press (Aust) Pty. Ltd., P.O. Box 544,
Potts Point, NSW 2011, Australia
FRANCE Pergamon Press SARL, 24 rue des Ecoles,
75240 Paris, Cedex 05, France
FEDERAL REPUBLIC Pergamon Press GmbH, 6242 Kronberg/Taunus,
OF GERMANY Pferdstrasse 1, Federal Republic of Germany
Copyright © 1980 Pergamon Press Inc.
Library of Congress Cataloging in Publication Data
Main entry under title:
Evaluating alcohol and drug abuse treatment effective
ness.
Bibliograpy: p.
Includes index.
1. Drug abuse—Treatment—Evaluation. 2. Alcohol
ism—Treatment—Evaluation. I. Sobell, Linda C.
II. Sobell, Mark B. III. Ward, Elliott, 1941-
RC564.E8 1980 616.8'6'06 79-20715
ISBN 0-08-022^97-2
All Rights reserved. No part of this publication may be reproduced,
stored in a retrieval system or transmitted in any form or by any means:
electronic, electrostatic, magnetic tape, mechanical, photocopying,
recording or otherwise, without permission in writing from the
publishers.
Printed in the United States of America
To
All Those Individuals Who Have Voluntarily Participated in
Alcohol and Drug Abuse Follow-up Studies
CONTRIBUTORS
David Adkins, Ph.D., Research Associate, Center for Behavioral Medicine,
Department of Psychiatry, University of Pennsylvania, Philadelphia,
Pennsylvania.
Henry M. Boudin, Ph.D., Hôpital St-Charles de Joliette, Joliette, P.Q., Canada.
Glenn R. Caddy, Ph.D., Associate Professor and Director of the Addiction Research
and Treatment Center, Department of Psychology, Old Dominion University,
Norfolk, Virginia.
Edward J. Callahan, Ph.D., Assistant Professor, Department of Psychology, West
Virginia University, Morgantown, West Virginia.
A. Mitch Cooper, Ph.D., Alcohol Programs, Community Mental Health Center of
Escambia County, Pensacola, Florida.
Terri Cooper, M.S.S.W., Geriatric Research Center, Santa Rose Rehabilitation
and Guidance Center, Milton, Florida.
Janice Boon McCollam, M.A., Department of Psychology, Vanderbilt University,
Nashville, Tennessee.
Stephen A. Maisto, Ph.D., Research Associate, Research Triangle Institute,
Research Triangle Park, North Carolina.
Albert A. Pawlowski, Ph.D., Chief, National Research Centers Branch, National
Institute on Alcohol Abuse and Alcoholism, Rockville, Maryland.
Ovide Pomerleau, Ph.D., Director, Center for Behavioral Medicine, Department of
Psychiatry, University of Pennsylvania, Philadelphia, Pennsylvania.
Richard A. Rawson, Ph.D., Director, HALT Project and University of California,
Los Angeles, California.
Barbara Sanders, M.S.S.W., Research Assistant II, Alcohol Programs, Dede Wallace
Center, Nashville, Tennessee.
IX
Linda Carter Sobell, Ph.D., Director, Alcohol Programs, Dede Wallace Center and
Adjunct Assistant Professor, Department of Psychology, Vanderbilt University,
Nashville, Tennessee.
Mark B. Sobell, Ph.D., Associate Professor and Director of Graduate Studies on
Alcohol Dependence, Department of Psychology, Vanderbilt University, Nashville,
Tennessee.
Elliott Ward, Ph.D., Private Practice, Nashville, Tennessee.
FOREWORD
This book presents a coherent set of analyses regarding problems associated
with treatment evaluation measurement and its methodology. The need for such an
analysis in the alcohol and drug addiction fields is obvious and long recognized.
It is certainly timely. On one hand, treatment outcome evaluation is virtually
absent in most alcohol and drug abuse programs; while on the other, the increased
visibility and support given to such treatment programs across the nation demand
an assessment of their effectiveness.
The book starts with a historical overview of treatment assessment research
focusing on méthodologie issues and methods for improving the quality of such
evaluations. It proceeds to specific reviews of problems associated with evalua
tion of alcoholism and drug addiction programs. Although there are many similar
ities between alcoholism and drug treatment programs, sufficient divergence exists
to warrant a separate discussion focusing on their unique differences. The prob
lems of follow-up evaluation studies, double-blind designs, and the reliability
and validity of measures of addiction are reviewed. The identification and selec
tion of outcome measures is analyzed and suggestions are made regarding appropri
ate statistical methods. The groundwork is laid for designing evaluation studies
of treatment efficacy and a prototype model evaluation is offered. Finally,
treatment service providers are encouraged to perform treatment evaluations. De
spite assessment difficulties, program staff should expand their interest in
evaluating the outcome of their treatment efforts by developing their skills in
conducting such studies. Suggestions are offered in this regard.
In summary, this book analyzes issues having national import for assessment of
alcohol and drug addiction treatment programs, it makes a number of suggestions
for improving the quality of evaluation studies, and application of the ideas pre
sented should have impact on improving the quality of the treatment provided.
Albert A. Pawlowski, Ph.D.
Chief, National Research Centers Branch
National Institute on Alcohol Abuse and
Alcoholism
XI
INTRODUCTION
MARK B. SOBELL LINDA CARTER SOBELL
Vanderbilt University Dede Wallace Center and
Vanderbilt University
As a result of recent state and federal mandates requiring health services
accountability, increasing attention has been given to evaluating the effective
ness of clinical treatment in mental health programs. In view of these mandates,
it seems clear that evaluation will soon become an integral part of clinical
treatment programs. Despite the increasing importance of treatment outcome evalu
ations, however, adequate evaluation components are presently lacking in most
alcohol and drug abuse treatment programs.
In the alcohol and drug abuse fields, attempts to conduct treatment evaluations
have often met with resistance. While the reasons for such resistance are plenti
ful, they often lack plausibility. For example, some claim that treatment evalua
tion is a costly diversion of resources from direct clinical services. This argu
ment, however, is premised on the assumption that clinical services are indeed
effective, or at least cannot be improved. When one considers the total costs of
providing alcohol and drug treatment services, coupled with their purportedly low
success rates, the increasing local, state and federal government emphasis on ob
taining comprehensive outcome data is easily understood, especially in the pre
vailing climate of fiscal austerity. Brash assertions, such as MOur program works
best for those who stay with the program," or "Our program only works for those
who are motivated to change," are no longer convincing or justifiable alternatives
to determining whether treatment has positive, negative or no measurable effects
upon clients. Rather, the present state of the art in alcohol and drug abuse
treatment necessitates that we vigorously pursue answers to questions concerning
which specific treatments work best with which particular types of clients. Only
in this way can we hope to meet the ethical obligation of knowledgeably providing
the most appropriate treatment for each client.
While alcohol and drug abuse treatment outcome evaluations have by no means
been totally lacking, the extant studies have typically been either highly spe
cialized research projects or large scale federally funded evaluations of federal
ly supported treatment programs. Various problems limit the value of both sorts
of studies. In the research studies, the outcome procedures and measures are fre
quently only, briefly described, and little if any attention is afforded to the more
subtle aspects of treatment outcome evaluation. These subtle considerations are
often critical determinants of whether evaluations can be effectively implemented.
Furthermore, the evaluation conducted in research studies often constitutes only a
xiii
minor or ancillary component of a larger project, rather than a focus of research.
Finally, even when-alcohol and drug treatment outcome studies have been conducted
within the rigid constraints of research requirements, they have been plagued by
multiple errors diminishing their internal and external validity (for a review of
these problems, see Chapter 1 by Maisto and Cooper).
In contrast to the research studies, the large scale federally sponsored evalu
ation efforts have focused on utilizing a single system to assess the effects of a
variety of treatment programs developed for a broad array of alcohol and drug
abusers. These broad brush studies, which influence national policies, also have
their shortcomings. For example, a basic requirement of any treatment outcome
evaluation is that the measures and procedures used be demonstrated to have rela
tively high reliability and validity. The drug studies, conducted primarily by
the Institute of Behavioral Research (IBR) at Texas Christian University, have
shown greater methodological sophistication than the alcohol studies in this re
gard. For example, Simpson, Lloyd and Gent (1976) reported that cross-checks of
incarceration and treatment reentry records supported the reliability and validity
of the IBR interview data. Unfortunately, these cross-checks only indirectly
address the question of whether drug abusers provide accurate reports of their
drug use, a question of paramount consideration in the interpretation of treatment
outcome data. Despite limited evidence that the self-reports of drug abusers
about their drug use have a respectable degree of validity (e.g., Ball, 1967;
O'Donnell, 1969; Robins, 1973; Stephens, 1972), such findings have resulted from
research studies that did not utilize the federal Drug Abuse Reporting Program
(DARP) for data collection. Thus, generalizations to the IBR data should be made
cautiously.
In contrast, the large scale alcohol treatment outcome investigations, pio
neered by investigators at the Stanford Research Institute and later at the Rand
Corporation, have totally neglected to investigate the reliability and validity of
their data. Only recently, and after several years of data collection, was an
investigation undertaken to examine the reliability of data obtained in the
national studies (Stallings, Brilliant £ Maginniss, 1978); presently, that invest
igation has not been completed.
Other problems have also hampered interpretation of the large scale studies.
Since many of the problems which have pervaded the national alcohol treatment out
come studies are discussed in some detail in Chapter 6 of this book (Sobell,
Maisto, Sobell, Cooper, Cooper ξ Sanders, 1979) they will not be reiterated here.
Several of these problems have similarly affected the drug studies. For example,
problems of subject attrition in these studies may create subject selection bias
(Campbell Ç Stanley, 1966). Thus, in a recent evaluation of 12-month outcome
results using the DARP system, data were collected on 77% of an original sample of
more than 4,000 clients (an additional 6% of the original sample had died;
Simpson, Savage, Lloyd £ Sells, 1978). Regrettably, it is unclear how that
study1s findings would be affected were data available on the 17% of subjects who
were not located. In this regard, several studies have suggested that alcoholics
not located for follow-up typically are faring much worse than those who are
located (see Chapter 1, by Maisto $ Cooper, and Chapter 6 by Sobell et al., for
reviews). Presently, similar data are not available for drug abusers who are not
located for follow-up or refuse to be interviewed.
While the national studies are not without their share of problems, they are
not worthless. Within certain constraints (e.g., the collapsing of data across a
number of treatment programs), these studies have yielded useful findings about
the probable overall effectiveness of alcohol and drug treatment programs. These
studies are not included in this book, however, for two reasons. First, they have
been amply reported elsewhere (e.g., Armor, Polich £ Stambul, 1978; Ruggels,
Armor, Polich, Mothershead $ Stephen, 1975; Sells, 1974a, b; Sells $ Simpson,
1976a, b, c; Towle, Bosely, Borgeson, Gray, Mothershead $ Norwood, 1973). Second,
the large scale studies, while generating a "big picture" analysis of treatment.
efficacy, are less applicable and useful for individual clinicians or treatment
xiv
programs.
In contrast to previous books on treatment evaluation, this book integrates
material from three focal areas within the rapidly developing field of alcohol and
drug abuse treatment outcome evaluation. Collectively, the chapters stress (a) an
exposition and understanding of the major practical problems involved in design
ing, conducting and interpreting treatment outcome evaluations, (b) the use of
innovative measures and procedures, and (c) the presentation of exemplary studies.
Because treatment outcome evaluation (follow-up) has been purported to be
extremely difficult to implement in clinical alcohol and drug abuse treatment set
tings, it is extremely important that clinical service providers and directors of
alcohol and drug abuse programs understand the necessity and basis for clinical
treatment evaluations, the problems inherent in such evaluations, ways to minimize
difficulties involved in conducting treatment outcome evaluations, and, of course,
the benefits which can derive from conducting such evaluations. Most of the
chapters in this book are elaborations of presentations given at a conference on
alcohol and drug abuse treatment outcome evaluation; the conference was directed
toward treatment providers and program administrators. Thus, many of the con
tributions have been oriented to present the methods and results of a variety of
alcohol and drug abuse treatment outcome evaluation projects in such a way as to
demonstrate that evaluation systems are feasible and can fulfill worthwhile pract
ical objectives for alcohol and drug treatment programs. Other chapters address
more basic issues, such as the design of evaluation studies and the interpreta
tion of outcome data. All of the chapters were written with an emphasis on
combining sound scientific method with practical clinical applications.
In preparing this book, our intention was for the reader to come away with a
better understanding of the importance and value of conducting clinical treatment
evaluations. Furthermore, we hope that the reader, after reading this book, will
recognize that (a) alcohol and drug treatment outcome evaluations do not have to
be mysterious and complicated enterprises restricted to research projects, but
rather are feasible and necessary undertakings within clinical treatment settings,
and (b) that the implementation of treatment outcome evaluations can be success
fully incorporated into the overall treatment delivery process.
Finally, the preparation of this volume was supported by numerous individuals
and agencies. First, we would, like to express our gratitude to the contributors,
whose manuscripts reflect some of the newest developments and trends in the fields
of alcohol and drug abuse treatment outcome evaluation, materials which heretofore
have been unpublished. Second, the conference and the preparation of this volume
was supported, in part, by three "major agencies: The Alcohol and Drug Section of
the Tennessee Department of Mental Health and Mental Retardation, the Alcohol
Programs of the Dede Wallace Center, and the Program of Graduate Studies on Alco
hol Dependence, Department of Psychology, Vanderbilt University (this latter pro
gram is supported, in part, by PHS Grant No. AA07072). Third, we would like to
thank all those staff members who were involved either directly or indirectly in
the actual evaluation project (see Chapter 6 by Sobell et al., in this book) which
was largely responsible for the realization of the conference and the publication
of its proceedings: Lonnie Cane, A. Mitch Cooper, Terri Cooper, Brian Garrett,
Ken Guestto, Christine Maisto, Stephen A. Maisto, Wayne McCauley, David Saltman,
Barbara Sanders, Carol Stein, Robert VanderSpek and Debi Zimmerman. Fourth, we
would like to acknowledge the cooperation and support of the Staff of the
Tennessee Department of Mental Health and Mental Retardation. Lastly, we owe
special gratitude to Nancy Crow, whose typing talents and toleration of frustra
tion have been vital to the preparation of this book.
References
Armor, D. J., Polich, J. M. ξ Stambul, H. B. Alcoholism and treatment. New York:
John Wiley £ Sons, 1978.
xv
