Formative Assessment, Learning Data Analytics and Gamification. In ICT Education

358 Pages·2016·18.08 MB·English

by  Santi Caballé

Formative Assessment,
Learning Data Analytics
and Gamification
Formative Assessment,
Learning Data Analytics
and Gamification
In ICT Education
Edited by
Santi Caballe´ and Robert Clariso´
Universitat Obertade Catalunya, Barcelona,Spain
Series Editor Fatos Xhafa
AMSTERDAM (cid:129) BOSTON (cid:129) HEIDELBERG (cid:129) LONDON
NEW YORK (cid:129) OXFORD (cid:129) PARIS (cid:129) SAN DIEGO
SAN FRANCISCO (cid:129) SINGAPORE (cid:129) SYDNEY (cid:129) TOKYO
Academic Press is an imprint of Elsevier
AcademicPressisanimprintofElsevier
125LondonWall,LondonEC2Y5AS,UK
525BStreet,Suite1800,SanDiego,CA92101-4495,USA
50HampshireStreet,5thFloor,Cambridge,MA02139,USA
TheBoulevard,LangfordLane,Kidlington,OxfordOX51GB,UK
Copyright#2016ElsevierInc.Allrightsreserved.
Nopartofthispublicationmaybereproducedortransmittedinanyformorbyanymeans,electronicormechanical,including
photocopying,recording,oranyinformationstorageandretrievalsystem,withoutpermissioninwritingfromthepublisher.
Detailsonhowtoseekpermission,furtherinformationaboutthePublisher’spermissionspoliciesandourarrangementswith
organizationssuchastheCopyrightClearanceCenterandtheCopyrightLicensingAgency,canbefoundatourwebsite:www.
elsevier.com/permissions.
ThisbookandtheindividualcontributionscontainedinitareprotectedundercopyrightbythePublisher(otherthanasmaybe
notedherein).
Notices
Knowledgeandbestpracticeinthisfieldareconstantlychanging.Asnewresearchandexperiencebroadenourunderstanding,
changesinresearchmethods,professionalpractices,ormedicaltreatmentmaybecomenecessary.
Practitionersandresearchersmustalwaysrelyontheirownexperienceandknowledgeinevaluatingandusinganyinformation,
methods,compounds,orexperimentsdescribedherein.Inusingsuchinformationormethodstheyshouldbemindfulof
theirownsafetyandthesafetyofothers,includingpartiesforwhomtheyhaveaprofessionalresponsibility.
Tothefullestextentofthelaw,neitherthePublishernortheauthors,contributors,oreditors,assumeanyliabilityforany
injuryand/ordamagetopersonsorpropertyasamatterofproductsliability,negligenceorotherwise,orfromanyuseoroperation
ofanymethods,products,instructions,orideascontainedinthematerialherein.
LibraryofCongressCataloging-in-PublicationData
AcatalogrecordforthisbookisavailablefromtheLibraryofCongress
BritishLibraryCataloguing-in-PublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary
ISBN:978-0-12-803637-2
ForinformationonallAcademicPresspublications
visitourwebsiteathttps://www.elsevier.com/
Publisher:ToddGreen
AcquisitionEditor:BrianRomer
EditorialProjectManager:AmyInvernizzi
ProductionProjectManager:PunithavathyGovindaradjane
Designer:VictoriaPearson
TypesetbySPiGlobal,India
To our families
List of Contributors
K. Aoki
The Open University of Japan, Chiba, Japan
J. Arnedo-Moreno
Open University ofCatalonia,Barcelona,Spain
I. Balasooriya
Universitat Obertade Catalunya, Barcelona,Spain
D. Baneres
Open University ofCatalonia,Barcelona,Spain
X. Baro´
Open University ofCatalonia,Barcelona,Spain
Z. Callejas
University ofGranada, Granada, Spain
M.M. Chan
GalileoUniversity, Guatemala, Guatemala
P. Compan˜-Rosique
Universitat d’Alacant, Alicante,Spain
J. Conesa
Open University ofCatalonia,Barcelona,Spain
M. Drlı´k
Constantinethe PhilosopherUniversity in Nitra, Nitra,Slovakia
A.A. Economides
University ofMacedonia, Thessaloniki, Greece
M. Elena Rodr´ıguez
Open University ofCatalonia,Barcelona,Spain
N. Escudero-Viladoms
Autonomous University of Barcelona,Barcelona,Spain
M.F. Farghally
Virginia Tech,Blacksburg, VA,UnitedStates
M. Feidakis
University ofAegean, Mytilene, Greece
E. Fouh
Virginia Tech,Blacksburg, VA,UnitedStates
F.J. Gallego-Dura´n
Universitat d’Alacant, Alicante,Spain
D. Griol
CarlosIII University ofMadrid, Madrid, Spain
xi
xii List of Contributors
A.-E. Guerrero-Rolda´n
Open University ofCatalonia,Barcelona, Spain
C.Guetl
Graz University ofTechnology, Graz, Austria
I. Guitart
Open University ofCatalonia,Barcelona, Spain
S.Hamouda
VirginiaTech,Blacksburg, VA, United States
E.Hettiarachchi
University of Colombo SchoolofComputing, Colombo,Sri Lanka
M.A. Huertas
Universitat Oberta de Catalunya, Barcelona, Spain
F. Llorens-Largo
Universitat d’Alacant, Alicante,Spain
V.I.Marı´n
University of theBalearicIslands, Palma, Spain
R.Molina-Carmona
Universitat d’Alacant, Alicante,Spain
E.Mor
Universitat Oberta de Catalunya, Barcelona, Spain
M. Munk
Constantine the Philosopher University in Nitra, Nitra, Slovakia
Z.Papamitsiou
University of Macedonia, Thessaloniki, Greece
(cid:1)
A.Perez Garcias
University of theBalearicIslands, Palma, Spain
D.Riera
Open University ofCatalonia,Barcelona, Spain
R.H. Rizzardini
Galileo University, Guatemala,Guatemala
T.Sancho-Vinuesa
Open University ofCatalonia,Barcelona, Spain
R.Satorre-Cuerda
Universitat d’Alacant, Alicante,Spain
C.A. Shaffer
VirginiaTech,Blacksburg, VA, United States
U.Tudevdagva
MongolianUniversity of Science and Technology, Ulaanbaatar, Mongolia
C.J. Villagra´-Arnedo
Universitat d’Alacant, Alicante,Spain
Foreword
D. Whitelock
ProfessorofTechnologyEnhancedAssessmentandLearning,
TheOpenUniversity,InstituteofEducation,
WaltonHall,MiltonKeynes,MK74AA,U.K.
In recent years a number of changes to the design and delivery of ICT Education have taken place.
These include the growth of e-assessment, together with an emerging focus on learning design and
gamification,agreaterinterestinastudent-centredapproachandaninstitutionaldesiretofindefficient
andeffectivewaystodeliverfeedbackandassessmentsupport.Infact,assessmentbothimpactsonand
isinfluencedbybroadercurriculumandsocietalchanges.Rowntree(1987)drawsourattentiontothe
roleof encompassing assessment bystating:
if we wish to discover the truth about an educational system, we must look into its assessment
procedures
Rowntree(1987,p.1)
Thepaceofchangeintheassessmentareahasbeenslow,yetthereisincreasingagreementacrossthe
sector that changes are necessary. Chris Rust (2013) in a recent review of the international state of
research on assessment and examinations in higher education concludes that there appears “a fairly
high degree of consistency and homogeneity […] both in terms of its criticisms of current practices
andoutcomes,andintheproposedsolutions[…].Andtheneedtodosomethingappearstobegaining
recognitionandperceivedimportance.Sodespitetheenormityoftheculture-changeneeded,wemay
actuallybeclosetoapproachingthepointwhenthecollectivewilltoengage[willreach]criticalmass.”
Manycommentators,includingWhitelock(2009),havearguedthatsuchchangemustbeaccompanied
byagreateremphasisonpedagogicallydrivenmodelsfore-assessment.Thesemodulesshouldfacil-
itate the blending of e-assessment and e-feedback into a holistic dialogic learning framework.
Chapter15ofthisbooksupportsthisargumentwithadiscussionoftheuseofconversationalagents
in educational games.
In order to effect meaningful changes in assessment and curriculum design with ICT institutions
mustunderstandtheroleofinnovationinthisareaand,moreimportantly,elicithowwellthisiswork-
ingforstudents.ThisbookillustratesthebenefitsandsomeoftheproblemsinmovingtheICTagenda
forwardwiththeassistanceofLearningAnalyticstopromotea“StudentsFirst”approachtolearning.
Formative assessment is stressed in Chapters 8 and 12 with illustrations of its use for cognitive
support as well as skill development. Chapter 3’s authors highlight its use for online engineering
education.Formativeassessmenthasgainedsubstantialgroundinthelast10yearswithitspotential
to promote student learning. Hattie and Yates (2014) emphasize the role of feedback for learning,
illustrating that technologies have shown a positive effect on learning achievements. Tamin et al.
(2011)reportanaverageeffectsizeof0.33whileHatie’smeta-analysisofcomputer-assistededucation
overthepast30yearsrevealedapositiveeffectsizeof0.37.Thesefindingsallpointtotheneedfor
xiii
xiv Foreword
vigorous evaluations oftechnological innovation ineducation and Chapter 4 points the way with its
structureoriented evaluationmodel(SURE).
Oneofthequestionsraisedinmanyofthechaptersinthisbookis,howcanICTassistwithstudent
retention and progression? Chapter 5 offers some insights from the teaching of mathematics, while
Chapter6suggeststhatmovingtowardsamorepersonalizedlearningmodelcanassistwiththispro-
cess.Chapter7’sauthorsemphasizetheroleoftrackingandmeasuringstudentprogresswithLearning
Analytics, which againcan lead toa morepersonalizedapproach to teaching and learning.
Rienties and Toetenel (2016) have employed Learning Analytics to analyze students’ online be-
havior linked to the learning designs of a substantial number of courses at the OUUK. They linked
151modules,whichcontained111,256studentswiththedifferentlearningdesignsforthesemodules.
Thenusingregressionmodelstheyfoundtheprimarypredictorforsuccesswasthetimethatstudents
spentoncommunicationactivities,ie,wherestudentswereworkingtogetherandlearningfromeach
other. They suggest that well-designed communication tasks that align with the learning objectives
of the course may be a way forward to enhance academic retention. This work illustrates how large
data sets are influencing recent evaluation studies and Chapter 7 proposes an Assessment Analytics
Framework (AAF) for enhancing students’ progress, while Chapter 9 stresses the importance of
Learning Analytics and Chapter 10 presents a methodology for the predictive modeling of students’
behavior. We should not, however, neglect the increasing interest in emotion-aware systems for
e-learning in Virtual Environments as described in Chapter 11. They describe the challenges and
benefitsof self-awareness.
MassiveOpenandOnlinecourses(MOOCs)arealsoreferredtoinChapter14.MOOCshavebeen
growinginpopularitysince2008.ThemainrecentdrivershavecomefromtheOpenEducationalRe-
sourcesmovementinCanadaandthe2008eventonConnectivismandConnectiveKnowledge,ledby
Stephen Davies and George Siemens through the University of Manitoba. Through these processes
MOOCswereconstructedsothatparticipantscouldusethetechnologiesoftheirchoiceandinteract
freely with the content and each other.The theoretical premises for the construction and delivery of
MOOCswerethoseofconnectivism(Siemens,2004),describedasaLearningTheoryfortheDigital
Age.Thismoderntheorytakesintoaccounttheuseofsocialnetworkingandtheavailabilityofapleth-
ora of informationon the web, which can be shared aroundthe worldalmost instantaneously.
MOOCs,however,havebeentermedasan“uncourse”byHirst(2009)sinceanyonecancontribute
anywhereand,infact,somelearnersdonotusethecoursetoolsbutprefertocollaborateusingwikis,
blogs,FacebookorTwitter(Beetham,2008;GuldbergandMackness,2009).Thispresentsanumberof
challenges,especiallywiththedesignofappropriatelearningtasks.Theuseofeducationalgamescan
supportlearnersinMOOCsandChapter14compareslearninginaMOOCwithandwithoutagamified
learningmethod. The latter was shown tomotivate andimprove student participation inthe course.
This book illustrates how technology can enrich student learning with its focus on information
technologiesandcomputationalmethods.Ahugerangeofapplicationsarebeingusedinhigheredu-
cation;however,awideextentofchoicebringsitsownchallengesforstudentsandteachersalike.Feed-
back is highlighted in many chapters and providing electronic feedback when no other support is
available for example on draft essays (Whitelock et al., 2015) can support learning when it is most
needed.Moreimportantly,thesetypesoffeedbacksystemscanalsoencouragestudentstobelievethat
they can improve their academic attainment. This is no small feat for learners who may often feel
isolatedandstretchedtryingtosqueezestudyintoadayfilledwithothercommitmentsanddemands
on their time.
Foreword xv
The chapters in this book support the idea that there is a need for a “second-look” at the whole
assessmentprocesswhile,moreimportantly,“closing-the-loop”througheffectiveinstitutionallearn-
inganddissemination.Theydescribesomerecentinnovationsandpointthewaytoevaluationframe-
worksthat assist with the development ofnew pedagogies for the 21st Century.
REFERENCES
Beetham, H., 2008. Learners’ experiences of e-learning: research from the UK. http://www.
networkedlearningconference.org.uk/past/nlc2008/abstracts/PDFs/BeethamIntro_464-466.pdf (accessed
27.02.16).
Guldberg,K.,Mackness,J.,2009.Foundationsofcommunitiesofpractice:enablersandbarrierstoparticipation.J.
Comput.Assist.Learn.25(6),528–538.
Hattie,J.,Yates,G.,2014.VisibleLearningandtheScienceofHowWeLearn.Routledge,NewYork,NY.
Hirst, T., 2009. Non-Linear Uncourses—Time for Linked Ed? http://ouseful.wordpress.com/2009/01/30/non-
linear-uncourses-time-for-linked-ed/(accessed27.02.16).
Rienties,B.,Toetenel,L.,2016.Theimpactoflearningdesignonstudentbehaviour,satisfactionandperformance:
across-institutionalcomparisonacross151modules.Comput.HumanBehav.60,333–341.
Rowntree,D.,1987.AssessingStudents:Howshallweknowthem?KoganPage,London.
Rust,C.,2013.TheInternationalStateofResearchonAssessmentandExaminationinHigherEducation,Sym-
posiumofAssessmentResearch,UniversityofHamburg,Germany,20August2013.
Siemens,G.,2004.Connectivism:alearningtheoryforthedigitalage.elearnspace.http://www.elearnspace.org/
Articles/connectivism.htm(accessed27.02.16).
Tamin,R.,Bernard,R.,Borokhovski,E.,Abrami,P.,Schmid,R.,2011.Whatfortyyearsofresearchsaysabout
theimpact of technology onlearning: Asecond ordermeta-analysis and validationstudy.Rev.Educ. Res.
81(1),4–28.http://dx.doi.org/10.3102/0034654310393361.(accessed27.02.16).
Whitelock,D.,2009.e-Assessment:developingnewdialoguesforthedigitalage.Br.J.Educ.Technol.40(2),
199–202.
Whitelock,D.,Gilbert,L.,Wills,G.,2013.Feedbackgenerators:providingfeedbackinMOOCS.In:International
Computer Assisted Assessment (CAA) Conference, Research into e-Assessment, DeVere Grand Harbour
Hotel,Southampton,July9–10,2013.
Whitelock,D.,Twiner,A.,Richardson,J.T.E.,Field,D.,Pulman,S.,2015.OpenEssayist:asupplyanddemand
learninganalyticstoolfordraftingacademicessays.In:The5thInternationalLearningAnalyticsandKnowl-
edge(LAK)Conference,Poughkeepsie,NewYork,USA,March16–20,2015.ISBN:978-1-4503-3417-4.
Preface
S.Caball(cid:1)e, R.Clariso´
UniversitatObertadeCatalunya,Barcelona,Spain
EducationinthefieldofInformationandCommunicationTechnologies(ICT)includesverypractical
competencies,whichcanonlybeacquiredbymeansofexperience,exercises,designing,projects,etc.
In addition to the challenge of motivating students to solve activities, lecturers face the problem of
assessingandprovidingsuitablefeedbackforeachsubmission.Receivingimmediateandcontinuous
feedbackcanfacilitatetheacquisitionofthecompetencies,althoughthisrequiressupportintheform
of automatic tools. The automation of the assessment process may be simple in some activities
(eg,practicalactivitiesonprogramming)butitmaybecomplexinactivitiesaboutdesignormodeling.
Monitoring the useof these tools can reveal very valuableinformationfor the tracking,manage-
mentandcontinuousimprovementofthecoursebytheteachingteam.However,inordertoleverageall
itspotential,thisinformationshouldbecomplementedwithdatafromothersources(eg,thestudent’s
academicfile) and historical informationfrom previous editions ofthe same course.
To this end,formative assessment (e-assessment)isanappropriate teachingstrategyfor learning
proceduralcompetencies, suchasthoseinthescope ofICT.Previousstudiescombineformativeas-
sessmenttoolswithLearningAnalyticsandgamification,inordertodesignalgorithmsforautomatic
feedback as well as improve the formative assessment fed back to students. However, these studies
explorealltheseaxesseparatelyandthushavelessimpactonthelearningprocess.Therefore,thein-
tegrationoftheseapproaches will create beneficial synergieswith relevante-learning scenarios.
Thecontributionsofthisbookhavearelevantimpactonstudents,lecturers,managersandacademic
coordinators. This impact results in students’ greater participation and performance while lowering
drop-outratesandimprovingsatisfactionandretentionlevels.Inaddition,tutors,academiccoordina-
torsandmanagersareprovidedwithtoolsthatfacilitatetheformativeassessment andfeedbackpro-
cesses. As a result, educational institutions can benefit from an improvement of their academic
outcomes,improvedstudentsatisfactionwiththeICTcoursesandprovisionofcompleteinformation
on the students’ activities for decision making (from prediction to estimation). Society at large will
leveragethegeneralizationofthisapproachinthelongtermthroughtheimprovementoftheperception
of ICTcoursesand amore efficient ICT higher education, resulting inabetter response tothe great
demands of ICT professionals.
Theultimateaimofthisbookistostimulateresearchfrombothatheoreticalandpracticalpointof
view into resources such as open source tools,which allows other educational institutions and orga-
nizationstoapply,evaluateandreproducethebook’scontributions.Inthisway,industryandacademic
researchers,professionalsandpractitionerscanleveragetheexperiencesandideasfoundinthebook.
Thisbookconsists of15 chaptersorganized into three major areas:
(cid:129) Formative e-assessment. The chapters in this area are concerned with the provision of immediate
feedback by means of automatic assessment. The scope of the research focuses on knowledge
areaswithhighcognitiveormodelinglevels,suchasthedesignormodelingofsoftwareandhardware.
xvii