Evaluation of Online Higher Education: Learning, Interaction and Technology

106 Pages·2014·3.33 MB·English

by  Ana Balula

SPRINGER BRIEFS IN EDUCATION
Ana Balula
António Moreira
Evaluation of
Online Higher
Education
Learning,
Interaction and
Technology
123
SpringerBriefs in Education
For further volumes:
http://www.springer.com/series/8914

Ana Balula (cid:129) António Moreira
Evaluation of Online
Higher Education
Learning, Interaction and Technology
Ana Balula António Moreira
Águeda School of Technology University of Aveiro
and Management - University of Aveiro Aveiro , Portugal
Águeda , Portugal
ISSN 2211-1921 ISSN 2211-193X (electronic)
ISBN 978-3-319-05424-7 ISBN 978-3-319-05425-4 (eBook)
DOI 10.1007/978-3-319-05425-4
Springer Cham Heidelberg New York Dordrecht London
Library of Congress Control Number: 2014938984
© Springer International Publishing Switzerland 2014
T his work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifi cally the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfi lms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology
now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection
with reviews or scholarly analysis or material supplied specifi cally for the purpose of being entered and
executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this
publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s
location, in its current version, and permission for use must always be obtained from Springer.
Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations
are liable to prosecution under the respective Copyright Law.
T he use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specifi c statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
While the advice and information in this book are believed to be true and accurate at the date of
publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for
any errors or omissions that may be made. The publisher makes no warranty, express or implied, with
respect to the material contained herein.
Printed on acid-free paper
Springer is part of Springer Science+Business Media (www.springer.com)
Preface
The book entitled “Evaluation of Online Higher Education—Learning, Interaction
and Technology” is, from my point of view, an important contribution to the chal-
lenges faced today by Pedagogy in Higher Education (HE), both at national and
international level, namely because it submits a proposal supported by the literature,
that has already been validated in real contexts, and that is highly innovative to the
process of improving e-teaching. This proposal is substantiated in the development
of an evaluation model of e-teaching activities and in how it can be put into practice,
using a tool called SCAI—an acronym that combines the initials of the poles defi ned
for the three dimensions (Learning, Interaction and Technology) considered in the
model: S ocial, C onceptual, A uthentic, I ndividual, I nstrumental and I ntegrated.
Although there are several studies in the literature that include models for the
evaluation of e-teaching, I do not know any that focuses on the central dimension of
any teaching and learning process—the e-teaching activities. In addition to the
robustness of the theoretical framework underlying the proposed model, I further
highlight the functional design of its application, as well as the fact that the SCAI
tool has been tested and validated in two contexts—two Ph.D. courses offered by
two prestigious Portuguese public universities, the University of Aveiro (UA) and
the University of Coimbra (UC), the courses “Multimedia in Education” and
“Ancient World”, respectively. The prestige of these universities was recently
recognized at the international level, and the UA was considered, by the ranking of
the British magazine Times Higher Education (2013), as the 66th best university in
the world with less than 50 years and one of the best among the Portuguese, and UC
was granted the UNESCO’s World Heritage status.
B eyond the specifi c contribution that was mentioned above, I consider that the
present work provides a comprehensive and up-to-date review of the literature on
the topic of Information and Communication Technologies (ICT) as a means of
enhancing the quality of teaching and learning, as well as the models that have been
developed for their evaluation. In this sense, I would strongly recommend its read-
ing to researchers who are interested in the subject of ICT, and in particular in HE,
as well as to HE teachers. They may fi nd, along its four sections, up-to-date teach-
ings that are likely to subsidize further research and underpin good practices in
v
vi Preface
e-teaching, specially as to the type of activities to offer students. In more specifi c
terms, the work includes:
(cid:129) In Chap. 1 , a literature review that focuses on the role of e-learning in HE and on
the most valued levels and facets within the different models for teaching evalu-
ation that were identifi ed.
(cid:129) In Chap. 2 , the proposal of an evaluation model for e-teaching in HE, in particu-
lar as to the considered dimensions (learning, interaction and technology) and
how to put the model (as a whole) into practice, taking into account the reading
matrix for the evaluation results, underpinned by the established theoretical
framework.
(cid:129) In Chap. 3 , the methodology used to test and validate the model in the two con-
texts mentioned above.
In the conclusion of this work, some fi nal remarks on the contribution of the
model for the improvement of e-learning in HE are put forward. Finally, a word
regarding the context of the study reported in the book. This book was developed
based on an innovative doctoral project conducted by the fi rst author and supervised
by the second author. Although the authors belong to different generations, the
research work of both has had national and international projection recognized
through the publication of scientifi c papers, the participation in research projects
and scientifi c supervision of post-graduate students.
Nilza Costa
Introd uction
N owadays, it is generally accepted that knowledge construction and subsequent
transmission of the generated information cannot be dissociated from the use (and
evolution) of Information and Communication Technologies (ICT). As a matter of
fact, in 2003, the Commission of the European Communities had already stressed
that the knowledge society emerges from an effective and effi cient coordination of
four elements, i.e. “the production of knowledge, mainly through scientifi c research;
its transmission through education and training; its dissemination through informa-
tion and communication technologies; its use in technological innovation”
(Commission of the European Communities, 2003, p. 5).
Based on the idea that these four aspects are, increasingly, the foundations of the
society we live in, the key role that Higher Education (HE) can (directly or indi-
rectly) have on people’s everyday life becomes clear. In other words, what we are
dealing with is an educational context in which scientifi c research, par excellence ,
is being developed (potentially generating new knowledge), results are validated/
questioned through discussion triggered by its transmission and dissemination and
so on. Thus, in this scenario, it is of utmost importance to understand how ICT can
be enhanced to broadcast and develop new knowledge.
Furthermore, the guidelines set by the Commission of the European Communities
(2003) emphasize the need to implement policy measures that allow the conver-
gence of the manifold HE systems in Europe into one “European Higher Education
Area” (EHEA). Among other things, these measures led several European Higher
Education Institutions (HEIs) to rethink their educational offer in terms of concepts
such as “quality” (in education), “mobility” (of students, teachers and researchers)
and “lifelong learning”. Consequently, offering courses with online components
seems to be a very appealing hypothesis to overcome/eliminate geographic and time
constraints in the access to education for both national and international students.
Besides, as it is referred by the European Economic and Social Committee (2006,
p. 24), “E-distance learning and training can enable the systematic, faster and
cheaper transfer of knowledge”.
Notwithstanding the above and in the words of the European Economic and
Social Committee (2006, p. 22), “Some of the initiatives taken by the European
vii
viii Introduction
Union in the last decade in the fi eld of ICT-assisted learning produced exceptional
results, while others demonstrated a lack of consistency and did not have the
intended outcome in terms of numbers and quality”. Therefore, in order to guaran-
tee the quality of these courses, the creation of a system for the evaluation of
e-teaching, with clearly defi ned criteria and as objective as possible (and that should
be applied systematically) is becoming increasingly important. It is so, because this
way HEIs can develop their work according to their own characteristics, but also in
terms of the objectives defi ned for a wider group—the EHEA. In fact, the interna-
tional and national literature consulted points out that the evaluation of teaching in
HE is already being done at several levels, especially aiming at improving its qual-
ity. However, most of the work focuses on specifi c cases, and in which a just fi t
evaluation methodology is followed, inasmuch as that, most times, it cannot be
generalized to other online courses and/or does not refl ect a model of teaching eval-
uation recognized by the scientifi c community or grounded on a theoretical frame-
work. In this respect, the University of Exeter, in its Teaching Quality Assurance
Manual, underlines that the “Evaluation of teaching involves collecting evidence,
from various stakeholders, for the purpose of improving the effectiveness of the
teaching–learning process. A successful evaluation generates outcomes that are
valid, reliable and indicate directions and action for improvement. There is much
more to an evaluation than putting out the questionnaire” (Exeter University 2010,
p. 1). In fact, the evaluation of teaching (as a whole) is considered a very complex
process, since the facets to be taken into account can be manifold and always ques-
tionable. In other words, when you want to evaluate teaching practices rating them
hierarchically, in terms of quality, the risk of performing an evaluation that is too
partial is always there, once it often comes to realities that cannot be compared for
several reasons: different scientifi c areas, different education subsystems (polytech-
nic vs. university), different resources available, etc.
In this work, a proposal for a model for the evaluation of e-teaching in HE is
presented, which was tested and validated by experts in the area, and that is meant
to be useful for HEIs and their teachers. It is important to underline, though, that
with this model, the objective is not to establish hierarchies of education quality in
aprioristic or absolute terms, i.e. the results will only be valid depending on the
goals that are institutionally defi ned for the e-teaching they intend to offer. It is fur-
ther added that, since the process of e-teaching is largely considered to be refl ected
in the e-teaching activities, which also contemplate learning and even assessment
activities, these constitute the primary object of analysis of this work. Note that, in
this work, the term “evaluation” will always be used as to the teaching process
within a certain subject or course degree and the term “assessment” as to the
students’ learning process and/or results.
B ecause the education level that is being dealt with (HE) has its own character-
istics, the proposed evaluation model will take into account some of the European
guidelines for HE, in particular regarding: (1) knowledge transfer to real contexts,
(2) the interaction underpinning the construction and application of knowledge, (3)
the centrality of the student in the teaching and learning process, and (4) the level of
ICT integration into the curriculum. Therefore, this work is structured in three parts:
Introduction ix
the fi rst (Chap. 1 ), in which the literature review is presented, in terms of the context
of this study, particularly with respect to (1) some European guidelines regarding
HE evaluation and quality, (2) ICT in the educational setting, and (3) some of the
most relevant theoretical models for the evaluation of teaching.
Based on these assumptions, a proposal of a model for the evaluation of e- teaching
in HE (Chap. 2 ) is put forward, in which three evaluation dimensions are defi ned—
Learning, Interaction and Technology. In fact, in terms of teaching evaluation, quite
a few works focusing on interaction were found, of which some addressed the way
ICT integration into the curriculum could be foreseen, but none of them addressed
the Learning Dimension (a crucial aspect in any teaching process) as it is presented
in this work. Besides, still as to the Learning Dimension, the most frequently used
indicator in the literature is based on the products (or results) of learning. In this
study, the option is different, once it refl ects how teachers and students develop the
teaching and learning process based on the e-activities defi ned. Grounded on the
theoretical conceptualization of each evaluation dimension (Learning, Interaction
and Technology), the structure of SCAI1 tool is presented and explained—that, for
now, assumes the form of questionnaire.
Having defi ned the theoretical principles underlying the evaluation model and
the instruments for collecting data, criteria and evaluation indicators are also
presented, as well as the way these can be put into practice. In terms of actors, the
focus lies on the direct agents in the teaching and learning process—students and
teachers. Besides, from a more holistic point of view and based on the assumption
that there will be an institutional agent defi ning the goals to be achieved in terms of
online teaching/learning, the methodology to be followed to obtain a more compre-
hensive evaluation, at the course level, is depicted, namely as to data reduction
techniques. The natural conclusion of this section consists of the presentation of the
confi gurations, as well as the respective reading matrices, for each dimension that
may be obtained through the use of the SCAI tool.
In Chap. 3 , the use of the SCAI tool in two real cases is portrayed, i.e. two
b-learning HE third cycle degree courses in Portugal (2009/2010), which was
undertaken to verify the applicability and validity of the proposed evaluation model.
This work fi nishes with a refl ection on the main potentialities and constraints of
the proposed model for the evaluation of e-learning activities in the HE context and
a summary of the most important conclusions that have been drawn as to its appli-
cability/usefulness for HEIs, for teachers and for students.
1 The SCAI acronym represents a combination of the initials of the poles defi ned for each of the
three dimensions considered, namely So cial, Co nceptual, A uthentic, I ndividual, In strumental and
In tegrated.