Modeling and Measuring Competencies in Higher Education: Approaches to Challenges in Higher Education Policy and Practice

129 Pages·2017·3.24 MB·English

by  Olga Zlatkin-Troitschanskaia

Olga Zlatkin-Troitschanskaia · Hans Anand Pant
Corinna Lautenbach · Dimitar Molerov
Miriam Toepper · Sebastian Brückner
Modeling and Measuring
Competencies in Higher
Education
Approaches to Challenges in Higher
Education Policy and Practice
Modeling and Measuring Competencies
in Higher Education
Olga Zlatkin-Troitschanskaia
Hans Anand Pant · Corinna Lautenbach
Dimitar Molerov · Miriam Toepper
Sebastian Brückner
Modeling and Measuring
Competencies in Higher
Education
Approaches to Challenges in Higher
Education Policy and Practice
Olga Zlatkin-Troitschanskaia Dimitar Molerov
Johannes Gutenberg University Humboldt Universitaet Berlin
Mainz, Germany Germany
Hans Anand Pant Miriam Toepper
Humboldt Universitaet Berlin Johannes Gutenberg University
Germany Mainz, Germany
Corinna Lautenbach Sebastian Brückner
Humboldt Universitaet Berlin Johannes Gutenberg University
Germany Mainz, Germany
ISBN 978-3-658-15485-1 ISBN 978-3-658-15486-8 (eBook)
DOI 10.1007/978-3-658-15486-8
Library of Congress Control Number: 2016956286
Springer VS
© Springer Fachmedien Wiesbaden 2017
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
or information storage and retrieval, electronic adaptation, computer software, or by similar or
dissimilar methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are exempt
from the relevant protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authors or the editors give a warranty, express or implied, with respect to the material contained
herein or for any errors or omissions that may have been made.
Editorial: Stefanie Laux, Daniel Hawig
Printed on acid-free paper
This Springer VS imprint is published by Springer Nature
The registered company is Springer Fachmedien Wiesbaden GmbH
The registered company address is: Abraham-Lincoln-Str. 46, 65189 Wiesbaden, Germany
Table of Contents
Table of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IX
1 Competency Orientation in Higher Education . . . . . . . . . . . . . . . . . . . 1
2 An Overview of Assessment Practices . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.1 International and national frameworks and initiatives defi ning
learning outcomes in higher education . . . . . . . . . . . . . . . . . . . . . . . 7
2.2 Assessing competencies in higher education . . . . . . . . . . . . . . . . . . . 13
2.2.1 Assessment practices in various countries . . . . . . . . . . . . . . . . 13
2.2.2 Assessment practices in higher education institutions . . . . . . . 16
2.2.3 International assessments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3 Modeling and Measuring Competencies in Germany . . . . . . . . . . . . 21
The KoKoHs Program
3.1 Objectives and tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.2 Scope and projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.3 Models and instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.4 Validation and application in higher education practices . . . . . . . . . 25
V
VI Table of Contents
4 Demands in Higher Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Results of the KoKoHs Projects
4.1 Challenges in different study phases . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.2 Diversity orientation – Implications for assessment
at the beginning of studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.2.1 Integration of students . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.2.2 Diagnostic assessment – Findings from KoKoHs . . . . . . . . . . . 33
4.2.3 Application potential for practice . . . . . . . . . . . . . . . . . . . . . . . 37
4.3 Equity in higher education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.3.1 Access, opportunities, and success for diverse groups
of students . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.3.2 Process diagnostics and other applications
of assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.4 Promoting acquisition of domain-specifi c
and generic competencies in higher education . . . . . . . . . . . . . . . . . . 48
4.4.1 Strengthening employability and social participation . . . . . . . 48
4.4.2 Competency orientation in teaching and examination . . . . . . . 52
5 Development Perspectives for Higher Education Research
and Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
6 Conclusion and Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
7 An Overview of KoKoHs Projects, Models, and Instruments . . . . . . 65
8 Validation and Applications of KoKoHs Assessments . . . . . . . . . . . . . 79
9 Publication of KoKoHs Projects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Online . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
Table of Figures
Figure 1: Curriculum—instruction—assessment triad ……………………… 3
Figure 2: Constructive alignment triad …………………………………… 4
Figure 3: Clusters of competencies modeled and assessed
in the KoKoHs program ………………………………………… 22
Figure 4: Item for pre-service teachers of economics ……………………… 24
Figure 5: Competency model in mechanical engineering ………………… 35
Figure 6: Facets of business expertise ……………………………………… 43
Figure 7: Structure model of self-regulated learning ……………………… 44
Figure 8: Example of an online feedback system in the KoKoHs program … 47
Figure 9: Companies’ expectations towards bachelor and master graduates … 49
Figure 10: Companies’ expectations of competencies and experience
of bachelor graduates …………………………………………… 50
Figure 11: Model on pedagogical content knowledge
and content knowledge in physics………………………………… 54
Figure 12: Competency model for teaching computer sciences …………… 65
VII
1
Competency Orientation
in Higher Education
In recent years, higher education, particularly in Germany, has been marked by
fundamental structural and programmatic changes after the implementation of the
Bologna reform and the orientation towards competencies in the national stand-
ards for education, for instance, set forth by the Standing Conference of the Min-
isters of Education and Cultural Affairs in Germany. The quality of academic
education and its social returns have received increasing attention, and endeavors
to develop and implement fair assessments of students’ learning outcomes have
revealed diverse challenges. In Germany and internationally, it is becoming in-
creasingly important to defi ne and assess in an objective, reliable, and valid way
students’ learning outcomes in higher education, for which suitable instruments
are urgently needed. Current frameworks describing the competencies of higher
education students and graduates, such as the German and European Qualifi cation
Frameworks (DQR and EQF), provide a general orientation of goals for competen-
cy acquisition (see section 2). For practical application in higher education, they
need to be complemented with assessments that operationalize and differentiate
students’ knowledge and skills in a more detailed and exact way.
What knowledge and skills do students need in the 21st century?
In various studies stakeholders in politics and business in different countries have
been surveyed to determine the most important skills and knowledge students
should acquire in higher education to strengthen their employability (e.g., ACT
2012; O’Hara 2012; for Germany see DIHK 2015). Apart from expert knowledge
© Springer Fachmedien Wiesbaden 2017 1
O. Zlatkin-Troitschanskaia et al., Modeling and Measuring
Competencies in Higher Education,
DOI 10.1007/978-3-658-15486-8_1
2 1 Competency Orientation in Higher Education
in a domain, which is presupposed, cross-domain “21st century skills” (e.g., Lai
& Viering 2012, p. 2), such as critical thinking and problem solving, are expect-
ed from higher education graduates in nearly all OECD countries (OECD 2012).
Many studies emphasize the importance of such generic competencies, showing
that they are related and necessary to the acquisition of domain-specifi c compe-
tencies over the course of studies in higher education (e.g., Badcock, Pattison &
Harris 2010; Lai & Viering 2012; Mehralizadeh, Salehi & Marashi 2008).
At the same time, research has revealed systematic differences in the acqui-
sition of domain-specifi c and generic competencies among students in different
study domains, types of higher education institutions, and countries (e.g., Trem-
blay 2012; OECD 2013a). Some domains, institutions, and countries have been
more successful than others in promoting such competencies in their students
(e.g., OECD 2014; Zabal, Martin, Klaukien, Rammstedt, Baumert & Klieme 2013).
However, factors explaining successful competency-oriented teaching and learn-
ing have not been researched in Germany or internationally.1 The central question
arising from current research and practice in higher education is: How and to what
extent can domain-specifi c and generic competencies be taught and acquired in
higher education?
Competencies to be acquired, or, more broadly, expected learning outcomes of
higher education, such as domain knowledge, problem solving abilities, analytical
and critical thinking, academic self-concept, and epistemological beliefs, are in-
cluded in nearly all module and program descriptions in higher education, particu-
larly in Germany. Job requirements for graduates of higher education institutions
more and more frequently include these competencies, illustrating their impor-
tance for graduates’ employability (see DIHK, 2015); domain-specifi c and generic
competencies are key to graduates’ ability to establish and assert themselves in
continuously changing professional fi elds. Recent research fi ndings indicate that
generic competencies in particular enable students to apply knowledge acquired
over the course of studies in higher education successfully and fl exibly in pro-
fessional practice. Hence, these learning outcomes can be considered the “value
added” of higher education, essential for every modern knowledge-based society.
Current practices and potentials of competency-oriented teaching and learning in
higher education remain to be explored.
1 For a comprehensive overview of the state of international research, see Zlat-
kin-Troitschanskaia, Pant, Kuhn, Toepper & Lautenbach (2016c).
1 Competency Orientation in Higher Education 3
How are learning outcomes related to examination
and teaching-and-learning methods?
In fundamental teaching-and-learning theories such as the theory of learning and
knowing (Pellegrino, Chudowsky & Glaser 2001), competency assessments are
regarded as one of the cornerstones of the curriculum—instruction—assessment
triad (see Figure 1). According to the model of constructive alignment (see Figure
2, Biggs & Tang 2011), intended learning outcomes, teaching/learning activities,
and assessment tasks should be aligned in the best way possible. Only through fair
examination and assessment procedures can educational institutions and teachers
determine whether and to what extent curricular and instructional goals are being
met; only in connection with suffi ciently differentiated professional feedback sys-
tems for teachers, students, and other stakeholders can assessment results serve
to enhance learning opportunities and promote competency acquisition. Fair ex-
amination procedures with suitable feedback systems are essential for assuring
successful teaching-and-learning practice in higher education, based on objective,
reliable, and valid assessments. Students’ competency levels can be analyzed at
the beginning, over the course, and at the end of their studies to gain reliable in-
formation on students’ preconditions, competency acquisition, and learning out-
comes in order to address two key questions: How do students’ domain-specifi c
and generic competencies develop over the course of their studies? Which types of
teaching-and-learning opportunities should be prioritized in higher education?
Figure 1 Curriculum—instruction—assessment triad (Pellegrino et al. 2001).