Assessing for Learning: Building a Sustainable Commitment Across the Institution

215 Pages·2004·1.73 MB·English

by  Peggy L. Maki

ASSESSING FOR LEARNING

ASSESSING FOR LEARNING
Building a Sustainable Commitment
Across the Institution
.
PEGGY L MAKI
STERLING, VIRGINIA
First published in 2004 by
Stylus Publishing, LLC
22883 Quicksilver Drive
Sterling, Virginia 20166
Copyright © 2004 by the American Association
forHigher Education
All rights reserved. No part of this book may be
reprinted or reproduced in any form or by any
electronic, mechanical or other means, now known
or hereafter invented, including photocopying,
recording and information storage and retrieval,
without permission in writing from the publishers.
Library of Congress Cataloging-in-Publication Data
Maki, Peggy.
Assessing for learning : building a sustainable
commitment across the institution / Peggy L.
Maki.—1st ed.
p. cm.
“Published in association with the American
Association for Higher Education.”
Includes bibliographical references.
ISBN 1-57922-087-8 (hard : alk. paper)
ISBN 1-57922-088-6 (pbk. : alk. paper)
1. Universities and colleges—United States—
Examinations. 2. Education, Higher—United
States—Evaluation. I. Title.
LB2366.2.M35 2004
378.1'67'1—dc22 2004001285
First edition, © 2004
ISBN: hardcover 1-57922-087-8
paperback 1-57922-088-6
Printed in the United States of America
All first editions printed on acid-free paper
CONTENTS
Acknowledgments xiii
Preface xvii
11
DEVELOPING A COLLECTIVE INSTITUTIONAL COMMITMENT 1
Overview 1
A Culture of Inquiry 1
Dialogue about Teaching and Learning across the Institution 3
Anatomy of the Collaborative Process 3
A Shared Commitment: Roles and Responsibilities 6
Presidents, Chancellors, and System Heads 6
Boards of Trustees 6
Campus Leaders 7
Faculty, Administrators, Staff, and Other Contributors
to Student Learning, Including Students 7
A Collaborative Beginning: Principles of Commitment 8
Anchors for Developing Institutional Principles of Commitment 8
The Science of Learning 8
The Scholarship of Teaching and Learning 8
Disciplinary and Professional Organizations’ Focus on Student Learning 9
Institutional Focus on Learning-Centeredness 10
Institutional Focus on Organizational Learning 11
Accountability 12
Meaningful Beginnings 13
Box 1.1 Institutional Example: University of Portland 14
Higher Education’s Ownership 14
Works Cited 15
Additional Resources 16
Worksheets, Guides, and Exercises 22
Box 1.2 Institutional Example: North Carolina State University 27
Box 1.3 Institutional Example: Rochester Community
and Technical College 27
vii
viii CONTENTS
22
BEGINNING WITH DIALOGUE ABOUT TEACHING AND LEARNING 31
Overview 31
The Continuum of Learning: Beyond an Aggregation of Courses,
Credits, and Seat Time 31
A Focus on Integration 32
Coordinating Committees 33
A Campus-Wide Assessment Committee 33
Program-Level Assessment Committees 34
Dialogue Focused on Expectations for Student Learning 35
Dialogue Focused on Verifying Expectations 36
Maps and Inventories 37
Maps 37
Box 2.1 Institutional Example: New Jersey City University 38
Box 2.2 Institutional Example: Rose-Hulman Institute of Technology 38
Inventories 38
Figure 2.3 Rose-Hulman Institute of Technology Curriculum Map 39
The Design of Our Work 40
Works Cited 40
Additional Resources 41
Worksheets, Guides, and Exercises 44
Appendix 2.1 Business Administration Program’s Curriculum Map 52
33
MAKING CLAIMS ABOUT STUDENT LEARNING WITHIN CONTEXTS FOR LEARNING 59
Overview 59
Learning Outcome Statements 60
Characteristics of Institution- and Program-Level Learning Outcome Statements 60
Box 3.1 Institutional Example: California State University Monterey Bay 60
The Difference between Program or Institutional Objectives
and Learning Outcomes or Learning Objectives 61
Box 3.2 Institutional Example: University of Washington 61
Levels of Learning Outcome Statements 62
Taxonomies 63
Collaboration to Develop and Review Outcome Statements 63
Strategies for Developing Outcome Statements 64
Strategy 1: Mission Statements 64
Strategy 2: Professional Organizations 64
CONTENTS ix
Box 3.3 Institutional Example: Stonehill College 64
Strategy 3: Student Work 65
Strategy 4: An Epistemological and Ethnographic Process 66
Strategy 5: Deep and Surface Approaches to Learning 67
Community Consensus about Learning Outcome Statements 67
Situating Students to Take Responsibility 68
Works Cited 68
Additional Resources 69
Worksheets, Guides, and Exercises 71
Appendix 3.1 Institutional Example: University of Washington 77
Appendix 3.2 Institutional Example: North Carolina State University 79
Appendix 3.3 Example of Leveled Outcome Statements 81
44
IDENTIFYING OR DESIGNING TASKS TO ASSESS THE DIMENSIONS
OF LEARNING 85
Overview 85
The Range of Texts that Demonstrate or Represent Learning 85
Multiple Methods of Assessment 86
Box 4.1 Institutional Example: Stanford University, California 87
Box 4.2 Institutional Example: University of Michigan 88
Box 4.3 Institutional Example: North Carolina State University 88
Direct and Indirect Methods of Assessment 88
Methods along the Continuum of Learning: Formative and Summative 89
Box 4.4 Institutional Example: Alverno College 89
Positions of Inquiry 90
Issues of Alignment: Outcomes, Inferences, and Students’ Learning Histories 90
Box 4.5 Institutional Example: Marian College 91
Box 4.6 Institutional Example: Keystone College 91
Properties of a Method: Validity and Reliability 92
Validity 93
Reliability 93
Standardized Tests 93
Designed Methods 94
Box 4.7 Institutional Example: Alverno College 95
Works Cited 97
Additional Resources 98
Worksheets, Guides, and Exercises 104
x CONTENTS
Box 4.8 Institutional Example: Academic Librarians and Faculty 108
Box 4.9 Institutional Example: Student Affairs and Academic Affairs 110
Appendix 4.1 Strategies for Reviewing and Selecting Standardized Instruments 113
Appendix 4.2 Inventory of Direct and Indirect Assessment Methods 114
55
REACHING CONSENSUS ABOUT CRITERIA AND STANDARDS OF JUDGMENT 119
Overview 119
Interpretations of Student Achievement 120
Norm Referencing 120
Criterion Referencing 120
Scoring Rubrics 121
Box 5.1 Institutional Example: Azusa Pacific University 122
Analytic and Holistic Rubrics 123
Strategies to Develop Scoring Rubrics 124
Box 5.2 Institutional Example: North Carolina State University 125
Development of Scoring Rubrics 125
Strategies to Assure Interrater Reliability 126
Threaded Opportunities for Institutional and Student Learning 127
Works Cited 128
Additional Resources 128
Worksheets, Guides, and Exercises 129
Appendix 5.1 Institutional Example: New Jersey City University 135
Appendix 5.2 Institutional Example: New Jersey City University 138
Appendix 5.3 Institutional Example: Rocky Mountain College of Art & Design 144
Appendix 5.4 Institutional Example: Hampden-Sydney College,
Department of Psychology 147
66
DESIGNING A CYCLE OF INQUIRY 153
Overview 153
A Design for Institutional Learning 153
Methods of Sampling 154
Some Key Institutional Contributors 154
Sampling 155
Box 6.1 Institutional Example: Florida Community College at Jacksonville 155
Times and Contexts for Collecting Evidence 157
Scoring 158
CONTENTS xi
Box 6.2 Institutional Example: University of South Florida 159
Analysis and Presentation of Results 160
Collective Interpretation of Analyzed Results 160
A Narrated Cycle 161
Box 6.3 Institutional Example: Mesa Community College 161
Beyond a Cycle 163
Box 6.4 Institutional Example: Implementation of the Assessment
Process at Providence College, 1999–2005 164
Box 6.5 Institutional Example: Providence College 164
Works Cited 165
Additional Resources 165
Worksheets, Guides, and Exercises 167
77
BUILDING A CORE INSTITUTIONAL PROCESS OF INQUIRY OVER TIME 171
Overview 171
A View of the Whole 171
Some Representative Structures, Processes, Decisions, and Channels and
Forms of Communication 172
Some Representative Structures 172
Box 7.1 Institutional Example: Portland State University 173
Box 7.2 Institutional Example: United States Naval Academy 175
Offices of Institutional Research and Planning 175
Box 7.3 Institutional Example: Rose-Hulman Institute of Technology 176
Processes and Decisions 176
Channels and Forms of Communication 176
Resources and Support: Human, Financial, and Technological 177
Box 7.4 Institutional Example: Indiana University–Purdue University Indianapolis 178
Campus Practices that Manifest an Institutional Commitment 181
Signs of Maturation 181
Box 7.5 Institutional Example: University of Wisconsin–River Falls 182
Works Cited 183
Additional Resources 183
Worksheets, Guides, and Exercises 185
Appendix 7.1 Institutional Example: Guam Community College’s Institutional
Commitment to Assessment: An Evolving Story of Dialogue and Ritual 190
Appendix 7.2 Consent Form 196
Index 199