Large-scale Assessment Programs for All Students: Validity, Technical Adequacy, and Implementation

536 Pages·2002·27.03 MB·English

by  Gerald Tindal

Large-Scale Assessment
Programs for All Students:
Validity, Technical Adequacy,
and Implementation
This Page Intentionally Left Blank
Large-Scale Assessment
Programs for All Students:
hhdity, Technical Adequacy,
and Implementation
Gerald Tindal
University of Oregon
Thomas M. Haladyna
Arizona State University
LAWRENCE ERIBAUM ASSOCIATES, PUBLISHERS
2002 Mahwah, New Jersey 1,ondon
Copyright 0 2002 by LawrenceErlbaumAssociates,Inc.
All rights resewed. No part of this book may be reproduced in
any form, by photostat, microform, retrieval system, or any other
means, without the prior written permission of the publisher.
Lawrence Erlbaum Associates, Inc., Publishers
10 Industrial Avenue
Mahwah, New Jersey 07430
(:over design by Kathlyn Horlghtaling Lacey
Library of Congress Cataloging-in-Publication Data
Large-scale assessnlent programs for all students : validity.technicaladequacy, and
illlplelllentatio,l / edirctl by (;el.altl Tind;d, 'lhomas M. I lalatlyna.
p. cnl.
Incl~~debsibliographical references and index.
ISBN 0-8058-3709-4
I. Exalninations-Validity. 2. Educational tests and Ineasurenlents. 1. Tinclal,
(;eraltl. 11. Haladyna, Xllonlas M.
20010:3:3453
(:IP
Hooks published by 1,awrence Erlbauln Associates are printed on acid-frec paper,
and their bindings are chosen for strength and durability.
I'rintcd in the United States of Anlcrica
1 0 9 8 7 6 5 4 3 2 1
Contents
Preface ix

1 Large-ScaleAssessmentsfor All Students:

Issues and Options 1

Gerald Tidal

Part I: Validity Issues
2 Validation of the Uses and Interpretations of Results

of AStsastees sment and AccountabSiylsittye
ms 27
Robelf L. Linn
3 The Relevance of Messick’sFourFaces
for Understanding the Validity of High-Stakes
Assessments 49

Russell Gersten and Scott Baker

4 VariationinAchievementScoresRelated to Gender,

ForImteamt,
and Content Area Tested 67
Josefih Ryun and SarahDeMark
M.
V
vi <:ONTENTS
5 Supporting Documentation: Assuring More Valid
Score Test Interapnrdetations Uses 89
Thomas M. Haladpa
6 Legal Issues Affecting Special Populations
in Large-Scale Testing Programs 109
S. E. Phillips
7 Consequences of Assessment: What Is the Evidenc1e4?9
William A. Mehens
Part11: Technical Issues
8 Test Dimensionality 181
Richnrd Tale
9
Choosing an Item Format 213
Michael C. Rodriguez
10 Incorporating Classroom-Based Assessnlents
Into Large-Scale Assessment Programs 233
Catilelirre S. Taylor
11 Monitoring Raters in Performance Assessments 26 1
George Engelhard, .Jr.
12 Issues, Strategies, and Procedures for Applying
Standards When Multiple Measures Ase Employed 289
Joseph M. Ryan
13 Linking Bilingual Mathematics Assessments:

A Monolingual IRT Approach
317
Ser~lrgW. Choi aud Marly McCall

Part111: Implementation Issues
14 Participation in Large-Scale State Assessment
and Accountability Systems 34 1
Patricia J. Almond, Camilla IAr, Marlha L. T ~ L L ~ o ~ u ,
and Rachel Que)remoen
vii
15 Assessmentof English-1,anguageLearners
in the Oregon Statewide Assessment System:
NSattiaotne al and Perspectives 37 1
Richard P. Durd'n, Catherine Brow!, mrd Marty McCall
16 DeterminingWhenTest Alterations Are
Valid Accommodations or Modifications
for Large-Scale Assessment 395
Keith Hollenbeck
17 A Methodology for Creating an Alternative Assessment
427 MeasMuSryoesdste ifmie dUsing
Robert Helzoig
18 Out-of-Level Testing Revisited: New Concerns
in the Era of Standards-Based Reform 453
Martha I,. Tlrurlozo, .John Bielitrski, .Jane Mitrtmla,
atrd,Jumes Scoff
19 Keporting Results of StudentPerformance
on Large-Scale Assessments 467
Jiw I'sseldyke and.J. Ruth Nelsotl
Part IV: Epilogue
20 Research to Improve Large-Scale Testing 483
Thonus M. Haladytin
Author Index 499
Subject Index 509
This Page Intentionally Left Blank
Preface
As our title suggests, this book deals with three issues we believe are critical
to large-scale assessments of student achievement.Validity is the most im­
portant consideration in achievement testing. Technicaladeqnacy contin­
ues to challenge us. We constantly face technical challenges in designing,
administering, scoring, and reportingtest results. Implementation of'best
practices and adherence to testing standards are much needed in all as­
sessments of student achievement.
The emphasis on all s/ude,r/s in our title addresses the need for large-
scale assessments to be inclusive withregard to populations who are often
underserved both in terms of their educational programs and assewnent
of their achievement.Specifically,we refer to students with disabilities ant1
English-language learners.
'l'his book includes five perspectives regarding large-scale assessnlents:
1. The Standards for Educntioml and I'.sychologicnl Testitrg published by
the American Educational Research Association, the American Psycholog­
ical Association, and the National Council on Measurement in Education
in 1999 provide a contextfor the many topics discussed in this volume. We
believe these standards are vely important in guiding 11sin the develop­
ment of tests and the interpretation and use of test scores.
2. Validity has been cast as part ofthe decision-making process and not
simply a function of the measures. 'rhis viewis articulatecl clearly in all
chapters throughout the book.The authors have used ualidily consistently
ix