University Rankings: Theoretical Basis, Methodology and Impacts on Global Higher Education

276 Pages·2011·3.659 MB·English

by  Jung Cheol Shin, Robert K. Toutkoushian (auth.), Jung Cheol Shin, Robert K. Toutkoushian, Ulrich Teichler (eds.)

University Rankings
The Changing Academy – The Changing Academic Profession
in International Comparative Perspective 3
Series Editors
William K. Cummings, The George Washington University, Washington, USA
Akira Arimoto, Kurashiki Sakuyo University, Kurashiki City, Okayama, Japan
Editorial Board
Jürgen Enders, University of Twente, Enschede, the Netherlands
Amy Metcalfe, the University of British Columbia, Vancouver, Canada
Christine Musselin, CSO Research Interests Higher Education and Research, Paris, France
Rui Santiago, University of Aveiro, Portugal
Simon Schwartzman, Institute for Studies and Labour and Society, Rio de Janeiro, Brazil
Ulrich Teichler, University of Kassel, Germany
Charles Wohluter, Northwest University, South Africa
Scope of the series
As the landscape of higher education has in recent years undergone significant changes, so
correspondingly have the backgrounds, specializations, expectations and work roles of academic
staff. The Academy is expected to be more professional in teaching, more productive in research
and more entrepreneurial in everything. Some of the changes involved have raised questions
about the attractiveness of an academic career for today’s graduates. At the same time, know­
ledge has come to be identified as the most vital resource of contemporary societies.
The Changing Academy series examines the nature and extent of the changes experienced by
the academic profession in recent years. It explores both the reasons for and the consequences
of these changes. It considers the implications of the changes for the attractiveness of the
academic profession as a career and for the ability of the academic community to contribute
to the further development of knowledge societies and the attainment of national goals. It
makes comparisons on these matters between different national higher education systems,
institutional types, disciplines and generations of academics, drawing initially on available
data­sets and qualitative research studies with special emphasis on the recent twenty nation
survey of the Changing Academic Profession. Among the themes featured will be:
1. Relevance of the Academy’s Work
2. Internationalization of the Academy
3. Current Governance and Management, particularly as perceived by the Academy
4. Commitment of the Academy
The audience includes researchers in higher education, sociology of education and political
science studies; university managers and administrators; national and institutional policy­
makers; officials and staff at governments and organizations, e.g. the World Bank.
For further volumes:
http://www.springer.com/series/8668
Jung Cheol Shin • Robert K. Toutkoushian
Ulrich Teichler
Editors
University Rankings
Theoretical Basis, Methodology
and Impacts on Global Higher Education
Editors
Jung Cheol Shin Robert K. Toutkoushian
Seoul National University University of Georgia
Department of Education Institute of Higher Education
Gwanak­ro 599 Meigs Hall
151­742 Seoul, Gwanak­gu 30602 Athens Georgia
Korea, Republic of (South Korea) USA
[email protected] [email protected]
Ulrich Teichler
Universität Kassel
Internationales Zentrum für
Hochschulforschung Kassel
Mönchebergstr. 17
34109 Kassel
Germany
[email protected]­kassel.de
ISBN 978­94­007­1115­0 e­ISBN 978­94­007­1116­7
DOI 10.1007/978­94­007­1116­7
Springer Dordrecht Heidelberg London New York
Library of Congress Control Number: 2011928551
© Springer Science+Business Media B.V. 2011
No part of this work may be reproduced, stored in a retrieval system, or transmitted in any form or by any
means, electronic, mechanical, photocopying, microfilming, recording or otherwise, without written
permission from the Publisher, with the exception of any material supplied specifically for the purpose
of being entered and executed on a computer system, for exclusive use by the purchaser of the work.
Printed on acid­free paper
Springer is part of Springer Science+Business Media (www.springer.com)
Preface
The book is designed to provide the reader with a comprehensive understanding on
university ranking schemes – its theoretical basis, methodological issues, and
impacts on society. Although rankings have been used in policy and academic
discussions, there are rare books in the market which provide the comprehensive
theoretical and methodological issues on rankings itself. Recently, policymakers
and the media often criticize their universities based on their ranking status. Further,
many of institutional leaders set rankings as their benchmark in their vision or
master plan for the university. Nevertheless, academics are quite skeptical of rank­
ings because they believe that they mislead higher education institutions as well as
have enormous methodological limitations.
The book is not about how to enhance the ranking status of an institution, nor
how to devise better ranking systems. Instead, we focused on how to widen the
understanding of university rankings for different audiences – academics, rankers,
and general people mostly parents and students. Academics, rankers, and their
audiences have different views on university rankings and use them for different
purposes. A group of academics have been developing precise technical methods of
measuring institutional performance, but are less interested generally in its social
impacts. Rankers do not pay much attention to the impacts that ranking games bring
on higher education society. Instead, they are interested in how to update their ranking
schemes better than their competing rankers so that they can sell the rankings
through their publications. Audiences of rankings only pay attention to the ranking
status of the university they are interested in, but not to the details behind the rankings.
This book is designed to help bridge this gap between ranking specialists (academics),
rankers, and ranking audiences.
The book has three major parts. Part I reviews the theoretical and practical basis
of university rankings, Part II focuses on the methodology used to derive rankings,
and Part III discusses the social impacts of university rankings.
In Part I, Jung Cheol Shin begins Chap. 2 with a discussion of organization
effectiveness, which is the theoretical base of ranking survey, neglecting academic
or policy discussions in many cases. Although a university’s ranking is a dimension
of measuring institutional effectiveness, rankers and its audiences often disregard
the fundamental issue that there are other aspects of effectiveness that should also
be considered. Since the 1980s, the organizational effectiveness of higher education
v
vi Preface
institutions has been evaluated by governments or their agencies to directly or
indirectly link institutional performance with budget allocation in some countries
(e.g., the USA, Australia, the UK, etc.). In Chap. 3, Grant Harman discusses
university rankings as well as public accountability and quality assurance as an
aspect of organizational effectiveness. Finally, in Chap. 4, Ulrich Teichler introduces
and discusses ranking literature focusing on theoretical and practical agendas of
rankings and its impacts on higher education.
In Part II, we discuss the methodologies used to derive university rankings. In
Chap. 5, Bernard Longden shows how rankers easily shift rankings by changing
weights or by inclusion or exclusion of indicators. In Chap. 6, Karen Webber
provides a comprehensive overview of how to measure institutional research, teaching,
and service performance. In the next chapter, Robert K. Toutkoushian, Karen
Webber, and Keith Trigwell discuss the details of measuring research and teaching
performance. In Chap. 8, Lutz Bornmann leads further discussions on measuring
research performance focusing on indicators, peer review, and reputation. This chapter
provides contemporary theoretical and practical issues of measuring research perfor­
mance, such as how to count citations in the real world. The last topic we cover in
Chap. 9 of Part II is about teaching quality.
In Part III, the respective contributors discuss the social impacts of ranking
survey. Although rankings are a dimension of organizational effectiveness, the social
impacts of rankings are enormous. Part III begins with social influences of ranking
survey at institution level. In Chap. 10, Christopher Morphew and Christopher
Swanson discuss how higher education institutions respond to rankings. They have
focused on how universities try to enhance their ranking status. In the Chap. 11,
William Locke discusses how universities institutionalize ranking schemes into
their internal systems and cultures. In Chap. 12, Akira Arimoto goes further on the
ranking impacts on higher education institutions and discusses how the rankings
impact faculty life on campus.
The contributors of the book are well­known researchers in higher education
worldwide and have many years of teaching and research experiences at higher
education institutions. We are confident that the issues we highlight and the theory
and practice issues we discuss will contribute to academic society of ranking
studies and also to the development of higher education. The work on this book can
be traced back to papers that were prepared for and presented at the International
Conference on Education Research (ICER­11), which was hosted by Education
Research Institute, Seoul National University in October 2009. We are grateful to
Education Research Institute for the financial supports and organizing the conference.
We thank Jung­Eun Lee and Hyun­Ju Park graduate students at Seoul National
University for their contribution to the editorial works.
Seoul National University, South Korea Jung Cheol Shin
University of Georgia, USA Robert K. Toutkoushian
University of Kassel, Germany Ulrich Teichler
Acknowledgment
Partial fulfillment of this work was supported by the National Research Foundation
of Korea Grant funded by the Korean Government (NRF­2010­330­B00232).
vii

Contents
1 The Past, Present, and Future of University Rankings ....................... 1
Jung Cheol Shin and Robert K. Toutkoushian
Part I Organizational Effectiveness, Quality, and Rankings
2 Organizational Effectiveness and University Rankings ...................... 19
Jung Cheol Shin
3 Competitors of Rankings: New Directions in Quality
Assurance and Accountability ............................................................... 35
Grant Harman
4 Social Contexts and Systemic Consequence of University
Rankings: A Meta-Analysis of the Ranking Literature ...................... 55
Ulrich Teichler
Part II Methodological Issues of University Rankings
5 Ranking Indicators and Weights ........................................................... 73
Bernard Longden
6 Measuring Faculty Productivity ............................................................ 105
Karen L. Webber
7 Measuring the Research Performance of Postsecondary
Institutions ............................................................................................... 123
Robert K. Toutkoushian and Karen Webber
8 Peer Review and Bibliometric:
Potentials and Problems ......................................................................... 145
Lutz Bornmann
ix
x Contents
9 Measuring Teaching Performance ......................................................... 165
Keith Trigwell
Part III Social Impacts of University Rankings
10 On the Efficacy of Raising Your University’s Rankings ...................... 185
Christopher C. Morphew and Christopher Swanson
11 The Institutionalization of Rankings: Managing Status Anxiety
in an Increasingly Marketized Environment ........................................ 201
William Locke
12 Reaction to Academic Ranking: Knowledge Production,
Faculty Productivity from an International Perspective ..................... 229
Akira Arimoto
13 The Future of University Rankings ....................................................... 259
Ulrich Teichler
Index ................................................................................................................. 267