The Accountability for Quality Agenda in Higher Education

235 Pages·2010·3.83 MB·English

October 2010
Michael Gallagher
Executive Director
The Group of Eight1
Canberra, Australia

T1. HTEh Ae CvCieOwUs NeTxAprBeIsLsIeTdY iFnO tRh iQs pUaApLeITr Yd oA nGoEtN nDeAce IsNs aHriIlGy HreEpRr eEsDeUntC tAhTeI OviNe ws of Go8 Vice-Chancellors. PAGE 1
Making sense of a new agenda
in higher education policy
Preface for the dialogue with the Oxford University Centre for Educational Assessment
Governments in many countries exert pressures on universities to be more accountable for the results
they manage to achieve with the resources available to them. A recently added twist, ironically within
the context of falling government investment and rising student demand, is that governments are
intruding into areas which have long been regarded as prerogatives of autonomous universities.
The accompanying paper has been prepared as a draft to focus discussion between Australian and
British policy analysts in the first instance, and later with US counterparts. The main reason for a bilateral
discussion initially is that there are three significant factors in common between the Australian and
British higher education contexts.
First, there are commonalities in the structure and culture of universities in the UK and Australia.
Universities in both countries are primarily public institutions, which are assisted by the nation
state through normative financing mechanisms, but rely increasingly on self-earned income from
commercial operations and competitive sources. They have traditionally enjoyed, by comparison with
public universities in continental Europe, the Americas and Asia, a relatively high level of autonomy,
notably in respect of governance, revenue and expenditure, student admissions, academic and general
staff appointments, curriculum, teaching, research, student assessment, and the award of qualifications.
Public universities in North America have many similar characteristics but they function within mixed
public-private systems and where states or provinces, as distinct from national government, have the
major financing responsibilities for teaching-related purposes.2
The Australian degree structure is (with some recent institution-specific exceptions) closer to British (in
particular the Scottish undergraduate specialisation model) than North American or European models.
There are also similarities in the skills-based approach to vocational education and training, as distinct
from the broader capability approach of the German and other European VET systems.
Additionally, both Australia and Britain have reduced supply diversity by collapsing former polytechnics
into the university model, whereas structural differentiation continues to be a feature of European
systems while there is great diversity among North American universities as well as between them and
community colleges.
Second, in both countries there has been a longstanding convention of parity of esteem of higher
education awards.
This convention is underpinned by normative supply-side funding, whereby all courses in similar
disciplines are funded at the same rate across all institutions at least for domestic undergraduate
and research higher degree students. The convention extends to equivalence in expectations of the
capabilities of graduates of different institutions.
Both countries share the ambiguities and anomalies arising from the legacy policies of the elite era
2. In Australia, the federal government has assumed funding responsibility for higher education.
THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE i
which continue to operate in the context of diversification of post-mass higher education supply
and demand.
Third, in both countries there has been an increasing interest on the part of government in matters
of higher education quality assurance and research quality verification. Indeed there are some shared
policy and program features, reflecting regular interactions among policy makers and academic and
professional staff between the two countries.
In continental Europe a strong rationale for focusing on higher education outputs and results has
been the change in the approach to system steering from direct to indirect mechanisms, with greater
institutional discretion over the use of resources.
In Australia and Britain the agenda is driven by other forces, viz. the enlargement and diversification of
higher education demand and supply, and the changing nature of labour markets.
In Australia, the agenda is being driven by a number of different bodies and, apparently, from a
number of different points of view. It is not clear whether the dots are joined between ‘strengthening’
the national qualifications framework and its descriptors, standards references, provider accreditation
and re-accreditation, and academic performance reporting. There is no single authority with whom
to converse.
In Britain, the new Coalition Government has indicated an interest in revisiting the higher education
policy landscape, with an initial focus on graduate employability but its longer-term approach, in the
context of fiscal parsimony, has yet to be clarified.
In both countries, the language as well as the agenda is unclear. Why is this agenda on the table? Who
has put it there? What is it intended to achieve? How is it to be implemented? What actual impact
might it have, positively and negatively?
This paper is intended to help make sense of what is happening and inform public debate to promote
balanced outcomes.
THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE ii
Contents
Introduction 1
1. Dimensions of the Accountability for Quality Agenda 2
1.1 The current architecture for quality assurance in higher education 2
1.2 Proposed redesign of the quality assurance architecture 7
1.3 Different national interests 8
1.4 Internationally converging reform agenda 10
1.5 Common governmental objectives 13
1.6 A shared purpose? 14
1.7 The problem of conflation 18
1.8 The problem of confusion over standards-based approaches to learning outcomes 19
1.9 Why it is an ‘accountability’ agenda 21
2. Drivers of policy change 22
2.1 Wider expectations of higher education and university research 22
2.2 The changing nature of higher education demand and supply 25
2.3 Indicators of quality erosion in higher education 29
2.4 Concerns about graduate preparedness for changing job requirements 30
2.4.1 Employability skills 31
2.4.2 The changing nature of work 33
2.5 Disaffection with conventional quality assurance and performance reporting 37
2.5.1 The failure of quality assurance mechanisms to rid the system of rogue providers 38
2.5.2 The process orientation of quality assurance in higher education 39
2.5.3 The tendency of quality assurance to reduce diversity and quality 40
2.6 The inadequacy of information available to students and others 41
2.7 Increasing the responsiveness of higher education 45
2.7.1 The ‘democratistion of access’ agenda 45
2.7.3 The ‘seamlessness’ agenda 46
2.7.4 The ‘comparability’ agenda 48
THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE iii
3. Developments relating to accountability for quality agenda 50
3.1 Developments in the European Higher Education Area 50
3.1.1 The Bologna Process 50
3.1.2 Qualifications frameworks 50
3.1.3 The Diploma Supplement 52
3.1.4 Institutional classifications 53
3.1.5 Tuning 54
3.2 Developments in Britain 57
3.2.1 The ‘Academic Infrastructure’ 57
3.2.2 Framework for higher education qualifications in England, Wales and Northern Ireland 59
3.2.3 Programme Specifications 60
3.2.4 Subject benchmark statements 62
3.2.5 External examiners 67
3.2.6 Recent developments 69
3.3 Developments in the OECD 71
3.3.1 AHELO feasibility study 72
3.3.2 IMHE: Supporting Quality Teaching in Higher Education 73
3.4 Developments in the United States of America 74
3.4.1 The Spellings Commission 75
3.4.3 The Voluntary System of Accountability (VSA) 83
3.4.4 Dissent within the VSA ranks: opposition to standardised testing 86
3.4.5 The new Leadership Alliance for Student Learning and Accountability 88
3.4.6 Opposition to the VSA: revisiting the Spellings agenda 89
3.4.7 Tuning USA 90
3.5 Developments in Australia 92
3.5.1 Background 92
3.5.2 Vocational Education and Training Reform 96
3.5.2.1 Recent developments 96
3.5.2.2 Post-Fordism and the quest for ‘adaptable skills’ and continuous learning 97
3.5.2.3 New questions on the VET policy agenda 101
3.5.3 The new Quality and Regulatory Arrangements for Australian Higher Education 104
3.5.4 The Tertiary Education Quality and Standards Agency (TEQSA) 107
3.5.5 Higher Education Standards Framework 107
3.5.6 Benchmarks for teaching and learning quality assurance 109
3.5.7 Teaching Standards Framework 111
3.5.8 MyUni website 111
THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE iv
3.5.9 An Indicator Framework for Higher Education Performance Funding 112
3.5.10 Strengthening the Australian Qualifications Framework 113
3.5.10.1 University concerns 115
3.5.10.2 The intended use of the AQF as a regulatory instrument 117
4. Shifts, supplements, definitions and debates 121
4.1. Shifts 121
4.2 Supplements 124
4.3 Definitions 126
4.3.1 Inputs, processes, outputs and outcomes 126
4.3.2 Quality, quality assurance, quality enhancement and quality evaluation 129
4.3.3 Qualifications and Qualifications Frameworks 131
4.3.4 Standards 132
4.3.5 Comparability or consistency? 142
4.3.6 Fitness for purpose, fitness of purpose, and a standards-based approach 146
4.4 National Qualifications Frameworks 149
4.4.1 The role of NQFs 149
4.4.2 Assumptions about knowledge in NQFs 149
4.4.3 Learning about change management in the development of NQFs 152
5. Possible ways forward 154
5.1 Far-reaching changes in higher education fundamentals 154
5.2 Compelling grounds for government concern 156
5.3 New imperatives for universities alongside the need to safeguard important values 157
5.4 Areas of agreement and disagreement 158
5.4.1 Perspectives on the nature of the problems 158
5.4.2 Perspectives on solutions to the problems 159
5.4.2.1 Retreat from meta-regulation 160
5.4.2.2 Tendency to sameness 161
5.4.2.3 Intrusiveness 161
5.5 Tactical considerations for universities 162
5.6 What are the prospects for reducing the confusion and conflict? 164
5.7 Agreement on respective roles and responsibilities 166
5.7.1 Responsibilities of government 166
5.7.2 Responsibilities of universities 167
THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE v
5.7.3 Mutual responsibility rather than principal-agent relationship 168
5.7.4 Common and customised institutional accountabilities 169
5.8 Threshold Requirements for Licensing Higher Education Providers 171
5.8.1 Standards for initial provider registration 171
5.8.2 Monitoring of provider compliance with registration conditions 171
5.8.3 Re-registration of providers 171
5.8.4 Re-regulation proportional to risk 172
5.9 Beyond the Threshold: Enhancing Student Learning and Graduate Attainment 173
5.9.1 Setting expectations 173
5.9.2 Clarifying missions and goals 174
5.9.3 Professionalising educational assessment 174
5.9.4 Reporting performance 175
5.10 Conclusions 177
References 181
Attachments 193
A. Australian Higher Education Graduate Statement (Sample) 193
B. US College Portrait (Example) 199
C. University of South Australia Teaching and Learning Academic Standards Framework 2009 205
THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE vi
Introduction
Concurrently across several countries there is an interest in accounting for the quality of higher
education3 through the exploration of a “standards-based approach” and a “focus on outcomes”. This
relatively new “accountability for quality agenda” is ambitious and ambiguous. It brings together a
concern for relevance, performance and transparency. It represents both a continuation of previous
developments and a departure from some prior core tenets, in seeking to address changes in the role
and character of higher education. It is being pursued primarily by governments and in ways that cut
across the conventional responsibilities of higher education institutions themselves to validate the
educational qualifications they award. Thus it involves tensions between political and professional
perspectives, in an area where “by and large, academic standards are a matter of professional trust”
(James, 2003) but the bases of trust have been eroded (van Vught et al., 2010).
This paper explores the forces driving this agenda, the range of
This relatively new
public policy purposes related to it, and the options for universities
“accountability for
in responding to it in ways that do not demean higher education
and safeguard their substantive and operational autonomies. quality agenda”
The paper has particular regard to the Australian Government’s is ambitious and
2009 response to the Report of the Review of Australian Higher
ambiguous. It brings
Education and the subsequent policy debate.4 It considers the
together a concern for
Australian proposals for change in the context of policy directions
identified by OECD Education Ministers at their 2006 meeting relevance, performance
in Athens, the 2006 report of the Spellings Commission in the
and transparency.
USA,5 and the 2009 report of the UK House of Commons Select
Committee on Innovation, Universities, Science and Skills.6
The paper is organised into five parts. The first part outlines the
dimensions of the emerging accountability for quality agenda: its
scope and apparent purposes. The second part considers the declared and underlying drivers of policy
change. The third part looks at various manifestations of the agenda in Europe, Britain, the USA and
Australia. Part 4 discusses definitions of key concepts and debates about the issues, with reference
to the scholarly literature. The final part explores ways forward by establishing common ground
and reconciling differences between the higher education sector and government in designing an
appropriate and workable framework for the advancement of quality in higher education.
3. Higher Education in this paper refers to all programs leading to diploma and degree qualifications at ISCED 5B, 5A and 6,
regardless of the institutions or providers that offer them. There is contested ground between the Vocational Education and
Training, and Higher Education sectors at the diploma levels.
4. Australian Government (2009), Transforming Australia’s Higher Education System. Canberra.
5. US Department of Education (2006). A test of leadership: Charting the future of American higher. (Report of the Commission
appointed by Secretary of Education Margaret Spellings). Washington. DC.
6. House of Commons, (2009) Students and universities, report of the Select Committee for Innovation, Universities, Science and
Skills Committee. London.
THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 1
1. Dimensions of the Accountability
for Quality Agenda
This part describes existing frameworks and arrangements for higher education quality
validation in several countries, the range of reforms being proposed, and the given
rationales for the reforms. It explores the specific and shared circumstances giving rise
to the need for and direction of reform. It also explores the scope for common ground
between governments and higher education institutions in responding to new challenges.
1.1 The current architecture for quality
assurance in higher education
Over the last decade, and in some cases earlier, many countries have put in place quite comprehensive
arrangements for quality assurance in higher education.
In the US, higher education quality is addressed by three main means: accreditation; program review;
and assessment. Accreditation is a function of various non-governmental accrediting associations,
each of which is responsible for establishing the criteria and procedures for evaluating the quality of
educational institutions or programs. Accreditation serves to validate that an institution or program
meets minimum quality standards, but “does not provide an indication of the level of program quality
relative to other programs” (Gibeling, 2010). Program review is an internal function, (although as well
as internal reviewers it may involve external reviewers from peer institutions), undertaken periodically
every 5-10 years, and capturing a wide range of qualitative and quantitative indicators, designed to
provide feedback for program improvement. Program assessment is a function of public organisations
and agencies, each of which may require reporting against a specified set of indicators (typically
progression rates and times, completion rates, and graduate destinations). Assessment generally
“focuses on outcomes, involves a narrower set of measures (than program review) and is usually
continuous rather than periodic” (Gibeling, 2010). Accreditation, program review and assessment
can be seen to provide distinctive but complementary perspectives:
“Done properly, program review provides a perspective on the future, not a statement of the present or
a description of the past. In contrast, program assessment tends to focus on past success in achieving
specific outcomes and accreditation generally represents an evaluation of the current state of a
program or institution. Thus the three types of evaluation provide distinct perspectives and add value in
different ways. Furthermore, this difference in perspectives means that one form of evaluation cannot
easily substitute for another” (Gibeling, 2010).
Box 1 outlines the main features of the US Accreditation and Quality Assurance System (Eaton, 2008).
In Australian terminology, ‘accreditation’ means licensing of providers and registration of professional
qualifications. Recognition is the term given to the process of assuring the quality of the accrediting
bodies. The US concept of ‘quality assurance’ is much wider than the Australian notion of quality
auditing. Accreditation in the US is about both quality assurance: assuring threshold quality in higher
education; and quality improvement: assuring that institutions and programs have processes to try to do
what they do better (Eaton, 2008). The quality improvement agenda focuses on the assessment criteria
that each institution or program sets for itself. That is, US accreditation and quality assurance have a
standards-referenced dimension and a fitness-to-mission dimension. The US does not have a national
qualifications framework.
THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 2
In 2008 there were 19 accrediting bodies for the accreditation
…accreditation and
of institutions and 61 bodies for the accreditation of programs
quality assurance have a
(e.g. in law, medicine, engineering and health professions). The
accrediting bodies are autonomous, non-governmental, not-for- standards-referenced
profit organisations funded mainly from annual subscriptions
dimension and a fitness-
from institutions and programs. The accrediting bodies develop
to-mission dimension.
minimum standards that must be met in order to be accredited.
Institutions and programs undertake self studies based on those The US does not have a
standards, and then are subject to review by peers in the profession,
national qualifications
including through site visits and team reports. Accrediting bodies
framework.
make standards-referenced judgements through their decision-
making commissions and award or do not award accredited status.
Institutions and programs undergo periodic review to maintain
accredited status, which is required for access to federal and state
funding (Eaton, 2008).
Importantly, US Accreditation and Recognition are grounded in a set of values:
• That higher education institutions have primary responsibility for academic quality: They are the
leaders and the primary sources of authority in academic matters.
• That institutional mission is central to all judgements of academic quality.
• That institutional autonomy is essential to sustaining and enhancing academic quality.
• That the higher education enterprise—and the society—thrives on decentralisation and diversity
of institutional purpose and mission.
• That academic freedom flourishes only in an environment of academic leadership of institutions
(Eaton, 2008).
Box 1. Accreditation and recognition in the US
Relationship among Institutions Accredited by Recognised Accrediting Organisations,
Recognised Accrediting Organisations and Recognition Bodies*
Institutions Accrediting Recognition
and Programs Organisations** Bodies

Create, fund,
19,400 Government Regulation
participate in 61 Programmatic
Accredited Review and US Department
Accrediting Organisations
Programs accredit of Education
Review and recognise
19 Institutional
7,000 Create, fund, Accrediting Organisations Self Regulation
participate in
Accredited • Regional (8) Council for Higher
Review and
Institutions • Faith-Related (4) Education Accreditation
accredit
• Career-Related (7)
Create, fund, all members***
*2007 Council for Higher Education Accreditation (CHEA) Almanac of External Quality Review.
**Some accrediting organisations are recognised only by CHEA, some only by the US Department of Education, some by both.
***Not all accrediting institutions are members of CHEA.
THE ACCOUNTABILITY FOR QUALITY AGENDA IN HIGHER EDUCATION PAGE 3