The Changing Governance of the Sciences: The Advent of Research Evaluation Systems

272 Pages·2007·3.57 MB·English

by  Richard Whitley (auth.)

THE CHANGING GOVERNANCE OF THE SCIENCES
Sociology of the Sciences Yearbook
VOLUME XXVI
ManagingEditor:
Peter Weingart, Universität Bielefeld, Germany
EditorialBoard:
Yaron Ezrahi, The Israel Democracy Institute, Jerusalem, Israel
Ulrike Felt, Institute fu¨r Wissenschaftstheorie und Wissenschaftsforschung,
Vienna, Austria
Michael Hagner, Max-Planck-Institut fu¨r Wissenschaftsgeschichte,
Berlin, Germany
Stephen H. Hilgartner, Cornell University, Ithaca, U.S.A.
Sheila Jasanoff, Harvard University, Cambridge, MA, U.S.A.
Sabine Maasen, Wissenschaftsforschung/Wissenschaftssoziologie,
Basel, Switzerland
Everett Mendelsohn, Harvard University, Cambridge, MA, U.S.A.
Helga Nowotny, ETH Zu¨rich, Zu¨rich, Switzerland
Hans-Jo¨rg Rheinberger, Max-Planck Institut fu¨r Wissenschaftsgeschichte,
Berlin, Germany
Terry Shinn, GEMAS Maison des Sciences de l’Homme, Paris, France
Richard D.Whitley, Manchester Business School, University of Manchester,
United Kingdom
Bjo¨rn Wittrock, SCASSS, Uppsala, Sweden
Thetitlespublishedinthisseriesarelistedattheendofthisvolume.
THE CHANGING
GOVERNANCE OF THE
SCIENCES
The Advent of Research Evaluation Systems
Edited by
RICHARD WHITLEY
UniversityofManchester,UK
and
JOCHEN GLÄSER
University of Lancaster, UK
LibraryofCongressControlNumber:2008920619
ISBN978-1-4020-6745-7(HB)
ISBN978-1-4020-6746-4(e-book)
PublishedbySpringer,
P.O.Box17,3300AADordrecht,TheNetherlands.
www.springer.com
Printedonacid-freepaper
AllRightsReserved
© 2007Springer Science+Business Media B.V.
Nopartofthisworkmaybereproduced,storedinaretrievalsystem,ortransmitted
inanyformorbyanymeans,electronic,mechanical,photocopying,microfilming,recording
orotherwise,withoutwrittenpermissionfromthePublisher,withtheexception
ofanymaterialsuppliedspecificallyforthepurposeofbeingentered
andexecutedonacomputersystem,forexclusiveusebythepurchaserofthework.
PREFACE
This book arose from a conversation between Lars Engwall and Richard Whitley
during the 2003 European Group for Organization Studies Colloquium in Copenhagen
about important topics for future research. We agreed then, and this has been
amply confirmed by subsequent events, that the proliferation of research evalua-
tion schemes, especially in Europe, was an important feature of the changing rela-
tionships between the state, universities and scientific research more generally,
which needed systematic and comparative analysis. With the support of the Bank of
Sweden Tercentenary Foundation, this discussion led to workshops in Uppsala and
Manchester involving colleagues from Australia, Germany and the Netherlands at
which the main framework for such an analysis was developed. This framework
then formed the basis for the conference at Bielefeld in 2005 at which earlier
versions of most of the papers in this volume were presented and extensively
discussed.
We are very grateful for the support of the PRIME network of the European
Commission and the Max Planck Institute for the History of Science for this con-
ference, as well as the invaluable efforts of Peter Weingart of the University of
Bielefeld. We are also greatly indebted to the members of the Editorial Board who
commented on various drafts of these papers as well as to Aant Elzinga, Stefan
Kuhlmann, Philippe Laredo, Arie Rip and Nic Vonortas who acted as referees. In
preparing the volume for publication we have been greatly assisted by Kathryn
Morrison and would also like to acknowledge the help of Kate Barker in liaising
with the PRIME network, authors, and referees.

Richard Whitley
Jochen Gläser
v
TABLE OF CONTENTS

List of Contributors.................................................................................................xiii
Introduction

Chapter 1 Changing Governance of the Public Sciences: The Consequences
of Establishing Research Evaluation Systems for Knowledge
Production in Different Countries and Scientific Fields......................3
Richard Whitley

Introduction..........................................................................................3
Characteristics of Research Evaluation Systems and their
Consequences...........................................................................6
The Consequences of Strong Research Evaluation Systems
in Different Public Science Systems.......................................10
The Effects of Strong Research Evaluation Systems
on Different Kinds of Scientific Fields...................................19
Conclusions........................................................................................24
References .........................................................................................25

Universities as Strategic Actors: State Delegation and Changing
Patterns of Evaluation

Chapter 2 The Visible Hand versus the Invisible Hand: The Allocation
of Research Resources in Swedish Universities.................................31
Lars Engwall and Thorsten Nybom

Introduction........................................................................................31
Institutional Control...........................................................................32
Entry Control..........................................................................32
Institutional Rules...................................................................34
Conclusions............................................................................37
Input Control......................................................................................37
Screening of Candidates.........................................................37
Allocation Procedures.............................................................38
Conclusions............................................................................42
Output Control...................................................................................43
Conclusions........................................................................................45
References .........................................................................................47

vii
viii TABLE OF CONTENTS
Chapter 3 Prospective and Retrospective Evaluation Systems
in Context: Insights from Japan: .......................................................51
Robert Kneller

Introduction........................................................................................51
University Research and Prospective Peer Review ...........................52
The University Research System............................................52
The Organisation of University Research and the
Academic Reward Systems.........................................54
Research Funding and Prospective Peer Review....................55
The Advent of Retrospective Research Evaluation............................66
Final Remarks....................................................................................69
References..........................................................................................70
Appendix............................................................................................72

Chapter 4 Elite Through Rankings – The Emergence of the Enterprising
University...........................................................................................75
Peter Weingart and Sabine Maasen

Enterprising University Ahead?.........................................................75
Universities: Elements of Organisational Actorhood.........................77
Becoming an Entrepreneurial Actor: Elite Universities.....................80
Displaying the Entrepreneurial Actor: University Rankings..............83
Enterprising University in the Audit Society.....................................94
References..........................................................................................98

Chapter 5 The Social Construction of Bibliometric Evaluations......................101
Jochen Gläser and Grit Laudel

A Constructivist Approach to Bibliometric Evaluation Tools.........101
The Modalities of Bibliometric Evaluation Methods.......................102
Marketing by Dropping Modalities..................................................105
The Demand for Modality-free Bibliometric Evaluations...............108
Working on Modalities....................................................................112
Amateur Bibliometrics.....................................................................116
Conclusions......................................................................................118
References........................................................................................120

Weak and Strong Research Evaluation Systems

Chapter 6 Evaluation Without Evaluators: The Impact of Funding
Formulae on Australian University Research..................................127
Jochen Gläser and Grit Laudel

The Elusive Effects of Research Evaluation Systems......................127
Approach..........................................................................................129
TABLE OF CONTENTS ix
Research Strategy – the Challenge of Causal Attribution.....129
Methodology and Methods – the Challenge
of Empirical Identification.........................................132
Adaptations to a Strong Research Evaluation System.....................136
The Situation of Universities................................................136
Adaptation to the Research Funding Environment...............138
Consequences for Academics...............................................140
Adaptation by Academics.....................................................142
Changes in Knowledge Production.......................................145
Intended and Unintended Effects of Strong RES ............................146
References........................................................................................149

Chapter 7 The Basic State of Research in Germany: Conditions
of Knowledge Production Pre-Evaluation .......................................153
Stefan Lange

Introduction......................................................................................153
The Changing Governance of Academic Research in Germany .......154
University and Faculty Evaluation and Steering of Research..........157
Academic Responses to the Changing Context of Research............160
Time......................................................................................160
Funding.................................................................................161
Evaluations and Incentive Schemes......................................163
Profile-building.....................................................................164
Adaptation Patterns...............................................................165
Concluding Remarks........................................................................168
References........................................................................................169

Chapter 8 Research Evaluation as Organisational Development: The Work
of the Academic Advisory Council in Lower Saxony (FRG) .........171
Christof Schiene and Uwe Schimank

Introduction......................................................................................171
Procedure.........................................................................................173
Research Ideals................................................................................177
Effects..............................................................................................184
References........................................................................................188

Chapter 9 Interfering Governance and Emerging Centres of Control:
University Research Evaluation in the Netherlands ........................191
Barend van der Meulen

Introduction......................................................................................191
Episode 1: The Construction of an Evaluation.................................193
Episode 2: Construction of an Evaluation Object and Centres
of Control..............................................................................195
x TABLE OF CONTENTS
Episode 3: Standardisation, Comparability and Second Order
Dynamics..............................................................................197
Conclusions......................................................................................200
References........................................................................................202
Appendix..........................................................................................203

Chapter 10 Research Evaluation in Transition: Individual versus
Organisational Assessment in Spain................................................205
Laura Cruz-Castro and Luis Sanz-Menéndez

Introduction......................................................................................205
The Academic System: Governance and Funding...........................207
Size and Governance............................................................207
Funding.................................................................................209
Institutionalisation Processes and Characteristics of the
Spanish RES.........................................................................211
Institutional Arrangements of the Research Evaluation
System (RES)............................................................212
The National Agency for Evaluation and Foresight.............212
The National Commission for the Evaluation of Research
Activity......................................................................214
New Developments in the Spanish RES...............................215
Feedback Effects Between Evaluation and Research Systems........218
References........................................................................................221

Chapter 11 Death by Peer Review?: The Impact of Results-Oriented
Management in U.S. Research ........................................................225
Susan E. Cozzens

Analytic Frameworks.......................................................................226
The U.S. Public Research System....................................................229
Forms of Funding.............................................................................231
Types of Research Evaluation Systems...........................................232
Mission Agency Example.....................................................233
Basic Research Agency Changes .........................................235
Consequences...................................................................................236
The Benefits of Competition ................................................236
Interdisciplinarity..................................................................237
Short-termism ......................................................................238
Strategic Alignment .............................................................238
Open Windows ....................................................................239
Conclusions......................................................................................240
References........................................................................................241
TABLE OF CONTENTS xi
Concluding Reflections

Chapter 12 The Social Orders of Research Evaluation Systems........................245
Jochen Gläser

A Sociological Approach to Research Evaluation...........................245
The Problem of Governing Science.................................................246
The Place of RES in the Governance by Social Orders...................249
Three Rationales for RES.....................................................249
Intrusive RES........................................................................252
Competitive RES..................................................................254
Responses by Universities....................................................256
The Interactions of Hierarchies and Competitions
with Scientific Communities................................................258
Success and Possible Failures of RES..................................258
‘Competition Failure’ of Quasi-market RES........................259
‘Hierarchy Failure’ of Intrusive RES....................................262
Incentives for Future Research.........................................................263
References........................................................................................264

Author Index...........................................................................................................267