ERIC EJ1116800: Hypertext or Textbook: Effects on Motivation and Gain in Knowledge

2016·1.5 MB·English

by  ERIC

education
sciences
Article
Hypertext or Textbook: Effects on Motivation and
Gain in Knowledge
CathérineConradty*andFranzX.Bogner
CentreofMathsandScienceEducation,UniversityofBayreuth,UniversityCampus,
95447Bayreuth,Germany;[email protected]
* Correspondence:[email protected];Tel.:[phone]
AcademicEditor:JamesAlbright
Received:4April2016;Accepted:16August2016;Published:24August2016
Abstract: Computers are considered innovative in classrooms, raising expectations of increased
cognitivelearningoutcomesormotivationwitheffectsonDeeperLearning(DL).The“newmedium”,
however, may cause cognitive overloads. Combined with gender-related variations in ability,
self-efficacy or self-confidence, computers may even diminish learning effects. Our empirical
study used a quasi-experimental design and the Intrinsic Motivation Inventory (IMI) to monitor
efficacy in knowledge gain and motivation when using computer-aided versus textbook-based
educationalunits.Oursampleconsistedof393eighthgraders.Oneobjectivefocusedongendereffects
associated with autonomous teacher-assisted learning via interactive software or an appropriate
textbook. Both groups finished with a recapitulation with the teacher. A third group concluded
acomputer-aidedlessonwithacomputerquiz. ToprovideevidenceforDLwetestedlong-term
memoryaftersixweeksandexamineditscorrelationwithintrinsicmotivationfactors. Ingeneral,
ourinterventionaffectedthegirls’butnottheboys’intrinsicmotivation. Werecordedsignificantly
higherpost-testscoresinthetextbook-basedlesson,butthedifferencesvanishedintheretentiontest.
Theteacher-assistedconsolidationphaseincreasedlong-termknowledgeandpositivelyintervened
withthestudents’interest. Thus,wefoundevidenceforDL.
Keywords: Deeper Learning; e-Learning effectiveness; gender issues; intermedia comparison;
learningmotivation;hypertext;evaluationofe-Learning;cognitiveload
1. Introduction
Comparedtotextbooks,computersareregardedasinnovativenewmediainclassroomsleading
toasubstantialincreaseinmotivationscoresandcognitivelearningoutcomes.Studentsseemtoregard
computersasanewtoolwhichmotivatesthemandpromotescognitivelearning[1],althoughseveral
problemscouldinterferewithsuccessinlearning[2]bycausingcognitiveloadproblems[3].
Almost 30 years ago, Salomon assumed that students might see computers as toys and thus
underestimatetheimportanceofthesubjectmatter[4]. Ontheotherhand,studentsmayregardprint
mediaasdifficult-to-handlelearningtoolsneedingmuchconcentration,whileincontrastaudiosand
filmsmayofferrelaxationratherthanencouraginghardwork;thismayreducetheattentionlevelfor
thesubjectmatterandconsequentlyreducecognitivelearningachievements. Meanwhile,schoolsare
reachingthedigitalage. Forthelasttenyearse-learninghasbeendiscussednotonlyasanenrichment
ofteachingmaterialsbutalsoasleadingtoimprovementinpersonalskills,andtoDeeperLearning
(DL),aconceptdescribingamoremeaningfulandself-dependentwayoflearning[5–7]. Therefore,
teacherschangetheirrole-perceptionfromabroadcasterofknowledgetoatutorandchangetheir
viewoftheirteaching: studentshavetoseethemselvesastheresponsiblepersonsfortheirlearning[6].
Educ.Sci.2016,6,29;doi:10.3390/educsci6030029 www.mdpi.com/journal/education
Educ.Sci.2016,6,29 2of16
The present study analyses learning success and intrinsic motivation in the context of
a self-regulated learning unit involving either textbooks or computers with varying amounts of
teachersupervision.
1.1. WhatisHypermediaandHypertext?
The terms “hypermedia” and “hypertext” were originally defined in 1965 by Ted Nelson;
“Hypertextisamethodtocreateandaccessnonlineartext”[8](p.4).Hypermediadescribesaversionof
hypertextbyemphasizingmultimedialaspects.Thus,hypermediacanbeunderstoodasacombination
ofthewords“hypertext”and“multimedia.”
The primary characteristic of hypertext is the information unit labelled as a “node”. A node
is a basic unit used in computer science. From a semiotic point of view the type of the node is
aconcretesymbol. Thusanodeisacentralelementindividinginformationintochunksofknowledge.
Thismaybepuretext(singlecharacter,roworapassage)inthebasichypertextoragraphic,audio
orvideo-elementinhypermedia[9,10]. Thegreatestandmostcommonusageofhypermediaisthe
WorldWideWeb(www): linksconnectinformationunits(nodes),andorganisetheinformationvia
therelationshipsamongthenodes[10].
Oftenthetermshypertextandhypermediaareusedsynonymously(e.g.,[11]). Sincewecompare
the effects of a text, presented in a book or in computer-aided multimedia, we decided to use the
originaltermhypertext,althoughthemultimediaaspectsofthecomputer-aidedlearningenvironment
couldjustifytheuseofthebroadertermhypermedia.
1.2. AdvantagesofLearningwithHypertext
1.2.1. LearningMotivationandDeeperLearning
Computer-aidedlearningisdeemedtosupportinterestandlearningsuccessbecauseofalearner’s
(inter-)activity(e.g.,[12]). Dale(2008)reported“therelativenewnessandcoolness”ofcomputers[7].
SincetheiPodassignmentsinhisstudydidnotfeellikeworktothestudents,itwaseasiertomotivate
studentsanddrawthemintotheinstructionalprocess.
Therearecomplexcomputerprogramsthatenableinteractivity, suchascommunicationwith
otherlearnersoradaptivelearningprograms: commonexamplesare“Wiki”s,collaborativewebsites
liketheencyclopaedia“Wikipedia”[13]or“LEO”(LinkEverythingOnline)[14],aninternet-based
electronicdictionarywithintegratedtrainingfunctions. Incontrasttothis,inconventionallearning
software, learners still have passive roles by mostly interacting with navigation features [15,16].
Justbyitself,thedynamicofhypertextspromotesinteractivitysincetheopportunityofself-regulated
learningencouragesexplorationoftheindividual’sinterests.Thushypertextisaconstructivistlearning
environment,offeringasuitablelearningsettingwhichspecificallysupportsconstructingknowledge
and preventing boredom [17]. Information offered in nodes is not essential for the present text,
althoughtheinformationreachedvianodesoffersthepossibilitytodeepenone’sknowledgeofatopic.
Simplyhavingfreedomofchoicemayhelptosatisfycuriosityandthusfosterjoyofstudying[18]
(p. 195). Anyhypertextofferingtheoptiontodealinmoredetailwiththesubjectrequiresadditional
involvement with it, so that the outcome is less likely to be just linear learning but a networked
thinkingprocess. Studentshavetoprocessthelearningmatter,resultinginmeaningfulknowledge
unachievablebyrote-learningalone[12].
Evenifactivityisreduced,awiderangeofinformationofvariablecomplexityrelevanttoone’s
interestisoffered,thusprovidingindividualisedlearning[12,15]. Theneedfornavigationandthe
didacticinteractionwithfeedbackmaysupportindividualmotivationandactivatethinkingabilities,
intensifyelaboration,permitcreativityandcultivatelearningbydiscovery[12].
Moreincidentaladvantagesofonlinelearningcouldbetheincreaseofpersonalresponsibility
forlearningandpersonaldevelopment,especiallywithinthesocialcommunitiesoftheplatforms[6]
(p. 311). Thesearevaluableskills,summarisedintheconceptofDeeperLearning(DL):Learnersare
Educ.Sci.2016,6,29 3of16
notpassiverecipients. Piaget[19]putsemphasisontheimportanceofactionandproblem-solving
in learning [20]. Learning does not occur by listening to the teacher in silence but is constructed
through active participation. Similarly, Papert describes such cognitive autonomy as “intellectual
self-determinism,” as children “act as creators rather than consumers of knowledge” [21] (p. 12).
Knowledgethatiscreatedinthiswayiscrosslinkedandmeaningful,whichiswhylearnersforgetless.
SeveralfactorsenhancingDLhavebeenidentified: degreeofinterestin,therelevanceofand
the challenge provided by the subject content; a workload which is perceived as comfortable by
students; provisionofaframeworkwhichdemonstratesinterrelationshipse.g. throughtheuseof
conceptmaps—orhypertexts;assessmentinstrumentswhichrewardDeeperLearning(e.g.,aquiz),
andstudentinvolvementintheirownlearningthroughtheuseofstrategiessuchasgroupworkor
negotiationoftopicsforsubjectassessmenttasks[22].
AlearningenvironmentfosteringDLgivesstudentstheabilitytopractisetheirLearningLab
skillsinthecontextofmeaningfulprojectsthatfostercriticalthinking,problemsolving,collaboration
andlearningperse[23]. AccordingtoDale,thehypertextenvironmentisparticularlysuitableforDL,
meetingallthesedemandswiththeopportunityofself-regulatedexplorativelearning[7].
Thuswithself-regulatedcomputer-aidedlearningweexpecthighergaininintrinsicmotivation
factorssuchasinterest,inturnassociatedwithhigherknowledgegain.
1.2.2. LearningSuccess
The developers Nelson and Engelbart, the fathers of hypertext, were inspired to evolve
the hypertext technology by Bush’s (1945) essay “As We May Think” [24], which describes
a sort of collective memory machine, called “Memex” (Memory Extender). This precursor of
hypertext-organised information in a manner comparable to the organisation structure of human
memoryasasemanticnetworkofinter-connectedconcepts[24].Itwasthuscapableofchangingaflood
ofinformationintoknowledge. Similarly,CollinsandQuillian[25]describeda“semanticmemory”as
“thenet-structureofhypertextreflectingthementalpresentationofknowledge,”asAusubeldidwith
his“assimilationtheoryofcognitivelearning”[26].Thisanalogyofhypertextandhumanmemorymay
supportthelearningprocess,asstudentsneednetworkedthinkingformeaningfulunderstanding[27],
whichiswhyhypertextsareregardedasaneffectiveeducationaltoolwiththeadditionalbenefitof
allowing conceptual learning. Similar to concept maps, hypertexts may provide a perfect tool for
visualpresentationofinterrelations[28].
Computersalsoprovideanopportunitytoaccessmultimedialandmulticodalinformationfor
learners using various learning styles [29,30]. Hypermedial learning environments are therefore
considered helpful tools to teach even complex, interdisciplinary subject matters with complex
structured concepts, as in science education. The multimodality and multicodality may help the
studenttounderstandastructure,andmaysupportcognitiveprocessing[31]. Learningvianodes
in a hypertext reveals relationships between different concepts: this helps learners in the active
meta-cognitiveprogressoflong-termknowledge,anessentialpartofmeaningfullearning—astep
toDL.
DL is “not memorising only to forget and it is not reciting or regurgitating what really is
not understood and cannot be applied” [32]. DL is thought to lead to long-term learning because
youforgetless;meaningfulknowledgeislearnedwhenintrinsicallymotivated,probablywithfun.
Computer-aidedlearningenvironmentsareconsideredtosupportDL,asstudentsacquireknowledge
byexploringaccordingtotheirinterest[6]. InaccordancewithPISAprogressiveteachingmethods,
like problem-based, inquiry-based learning, an individual and group project work can be used to
fosterself-regulatedlearninganddeepunderstandingandpreparestudentstoapplytheirknowledge
innovelsituations[33].
Educ.Sci.2016,6,29 4of16
1.3. Disadvantages
Asearlyas1987Conklinfoundtwomaindisadvantagesofhypertext: “disorientation”(“lostin
hyperspace”)and“cognitiveoverhead”[34](p. 40),whichmaycoincidewiththetheoryof“cognitive
(over-)load”[35]. Nowadayswithweb2.0theproblemhasbecomemorecomplexthanever!
Disorientation may occur with malfunctioning navigation design, supporting associative but
nottarget-orientedexploration. Thiscanleadtoproblemsespeciallywhenhypertextdataarehighly
complex, shallowlystructuredand/orauserhaslittlepre-knowledgeofhandlingcomputersand
hypertext[36,37]: “Navigationinitsnarrowsensemeanstomovethroughspace;initsbroadersense,
navigationalsoincludesvirtualmovementthroughcognitivespacemadeupofdataandofknowledge
emerging from those data” [38] (p. 63). With inappropriate navigation design, a learner has no
mentalmapofthehypertext,andmayrapidlylosetrackwherehe/sheismanoeuvringwithinthe
hypertext and where nodes interlink. Some learners, named “apathetic hypertext users” [39] will
alwaysgetlostinhyperspace. “Knowledge-seekers”or“booklovers”navigateinaconsequential,
logical,systemicandstrategicmanner[39]. Asthenamesuggests,thesestudentscouldlearnequally
wellwithtextbooksaswithhypertext. “Featureexplorers”or“resourcejunkies”aremoreinterestedin
thestyleofthehypertextandwhatkindofscreensitcontains,thanintheimportantsubstanceofthe
writtentext. Thoselearnerswillusethehypertextexploratively. Workbookusagemightremindsuch
learnerstotakeininformation.
Since the iPod assignments were not seen as work by the students, it was easier to motivate
studentsanddrawthemintotheinstructionalprocess[7]. Ontheotherhand,thisverypointcouldbe
adisadvantage: Salomondescribesanunderestimationofthematterasstudentsthink“printishard,
televisioniseasy”[4].
Another crucial problem hindering learning success may lie in cognitive overhead/overload.
The“GenerativeTheoryofMultimediaLearning”[40]maintainsthatlearnersarelimited-capacity
dualencoderswhoactivelyprocessinformationinordertointegrateitmeaningfullyintotheirexisting
knowledge. Workingmemoryisneededtointegratenewinformationandhandlethelearningaid.
Problematicalatthisjunctureisthelimitedinformationcapacityofthementalactivityofworking
memory,labelledcognitiveload(CL)(e.g.,[35,41]). Swelleretal.[3]assumethreeCLcomponents:
(CL1)intrinsicloadcausedbycontentcomplexity,(CL2)extraneousloadcausedbytheinstructional
mode, and (CL3) germane load necessary for individually processing information in preparation
for long-term memory. As all three components are assumed to be additive [42], an increase of
component(CL1)and/or(CL2)withoutadecreaseoftheotherwouldleadtothelossof(CL3),causing
Cognitive Overload. Reducing the capacity for (CL3) germane load would consequently reduce
cognitivelearningofthesubjectmatter. AppliedtotheCLtheory(CL2)extraneousloadreferstothe
instructiondesign. ThuscomputersandtextbookscouldcausedifferentCLs. Ahypermediadesign
followingtheprinciplesofmultimedialearningcanenhancealearner’sachievementbyavoiding
cognitiveoverload[40,43]. Theadvantagesofself-regulatedandinquiry-basedlearningcomplement
eachother[43]. Manyissuesfundamentaltotheuseofcomputersandhypermediahavebeenstudied;
preeminent are the principles of helpful hypermedia use, identified by Mayer [40]; for instance,
avoiding split attention which might increase cognitive load [35]; additionally, the modality of
information has been shown to have an effect on learning owing to its limitation of the working
memory capacity [41,44]. Probably computer handling is not a relevant cognitive load factor for
“digitalnatives”[45]. Dalereferstotheyouthoftodayasthe“iPodgeneration,”onewellpreparedfor
e-learning[7].
1.4. Gender
A number of studies of gender effects in hypermedia instruction have yielded inconsistent
results;somereportnodifferencesatall[46],othersreportlargedifferenceswithregardtointerest
in computers [47]. This may be an effect of the participants’ age: children monitored within the
project“Hole-in-the-WallEducationLtd”(NewDelhi,India)(HIWEL;www.hole-in-the-wall.com)
Educ.Sci.2016,6,29 5of16
learnedcomputerhandlingasarequiredskill(e.g.,theinternetlanguageEnglish)throughpractice.
Adolescentsandadultsseemtohaveanatural“timidity”,ascepticismtowardsnewmethods[48].
Previousstudieshaveshownthateducationalsoftwaremotivatesgirlslessthanboys[2]. Thismight
beexplainedbythefactthatdesignersofthesoftwaretendtobemale.
Inthe1990s, studiesinGermanyreportedboystobemoreexperiencedwithcomputersthan
girls [49]. Decker [50] drew the conclusion that boys have higher competence in this context,
a conclusion that could have been premature. Furthermore, the rapid development of children’s
computerhandlingneedsconsideration.
Ourresearchquestionswerethefollowing:
• Dostudentslearnwithhypertext?
• Dostudentsgainmorelong-termknowledgewithhypertext?
• Doeslackofteacher’stutoringeffectlong-termlearningsuccess?
• Doesautonomouslearningwithoutteacherassistanceaffectintrinsicmotivation?
• Doeshigherintrinsicmotivationcorrelatewithhigherknowledgetestresults?
• Aretheregenderdifferencesinlearningemotionsandsuccessbetweenthelearningenvironments?
2. MaterialsandMethods
2.1. TheTeachingUnit
Ourstudyconsistedofaquasi-experimentalbefore-after-control-impact(BACI)design[51](read
more2.4)withtheinstructionalmodeastheindependentvariable:
(1) Computer-aidedlessonwithcomputer-aidedconsolidationphase(CALC)(n=51);
(2) Computer-aidedlessonwithteacher-assistedconsolidationphase(CALT)(n=114);
(3) Textbook-basedlessonwithteacher-assistedconsolidationphase(TAL)(n=172);
(4) Controlgroup,notparticipatingintheunit(n=55).
Wedesignedacomputer-aidedlessonabouthoneybeesconsistentwiththeBavariansyllabusfor
theagegroup. Themodule“Honeybee—SocialInsects”highlightedthefollowingsubjectsandpaid
particularattentiontoevolutionarycontexts:
(cid:3)
Superorganism“beecolony”(e.g.,socialorganisation,beelanguage);
(cid:3)
Timelines(e.g.,lifehistoryofaworkingbee);
(cid:3)
Products(e.g.,beeswax);
(cid:3)
Matterforhumans(e.g.,economy).
2.2. TheHypertextDesign
Weusedacommonnavigationbarusedinweb2.0today(Figure1). Nodesalreadydisplayedon
pagesaremarkedbychangedfontcolour. Anadditionalsitemapcomparabletothetableofcontents
inthetextbookhelpscreateamentalmapofthehypertext[37]. Aglossaryisimplementedasatabular
overviewofthemostimportantterms,eachlinkingtothecorrespondingscreen. Theglossaryisnotso
muchanavigationalaidthroughthehypertextasapointertothelearningtarget. Aworkingbook
wasaccompaniedbyanoptionalguidedtourthroughthehypertext,asrecommendedbyJonassen[9].
Theworkingbooksupportstheself-regulatedlearnerstoreachtheeducationalobjective.
Educ.Sci.2016,6,29 6of16
Educ. Sci. 2016, 6, 29 6 of 15

Figure 1. Screenshot of the hypertext.
Figure1.Screenshotofthehypertext.
2.3. S2a.3m.pSlaem apnlde aDndesDigensi gn
We sWeelescetleedct eedigehigthht hggraraddeerrss ((NN == 339933))a tatth tehteo ptolepv elelv(“eGl y(m“Gnaysmiunma”s)iuinmB”a)v ainri aB,aGvearmriaa,n yG.eArmtoatanly. A
total ooff2 020b iboiloolgoygcyl acslsaesssepsa rptiacriptiactiepdatinedt hien stthued ys.tuTdhiys. cTohhiosr tcaolhreoardt yalhraedadsuyf fihcaiden stuefxfpiceireinent ceexwpeitrhience
withc coommppuutetresrssi nsicnecien fionrfmoramtioantiotenc htencohlongoyloisgpya irst pofartht eorf etghuel arregsyulllaabr ussy.llTahbeups.a rTthiceip paanrtsti’cmipeaanntas’g emean
age wwaass 1133.6.6( S(SDD= =0 .603.6)3y)e ayresa.rTsh. eTgheen dgeernddiestrr idbuisttiroinburetvioenal erdevaehailgehde ran huimghbeerr onfubmoybse(r5 4o%f btoo4y6s% ()5.4% to
Asingleteacherunknowntoallstudentstutoredalllessons. Afteraten-minuteintroductory
46%).
phasestudentsworkedcooperativelyingroupsoftwo, orthreeatmost, for60min. Thestudents
A single teacher unknown to all students tutored all lessons. After a ten-minute introductory
couldchoosetheirteam. StudentswithCALCandCALTusedaninformativeinternethomepagewith
phase students worked cooperatively in groups of two, or three at most, for 60 min. The students
pictures,texts,videosandanimations. StudentswithTALusedanequivalenttextbook,withthesame
could choose their team. Students with CALC and CALT used an informative internet homepage
textsandpicturesasthehomepagebutwiththeobviouslackofmultimodality.
with pictures, texts, videos and animations. Students with TAL used an equivalent textbook, with
Asaconsolidationphase,theclassreviewedthesubjectmatterfortenminutes. InCALTandTAL,
the ssatmudee tnetxstrse caanpdit upliactteudroens athse thbaes hisoomfaepqaugizep brueste wntietdh wthiteh oabPvoiowuesrP loaicnkt opfr emseunlttaitmioondwailtihtyt.e acher’s
tAutso ari ncgo.nTsooleixdcalutidoent epahchaeser’,s tahses icstlaanssce rienvCieAwLeCd, stthued esnutbsjeucste dmaantteeqru fivoarl etennt cmominpuutteesr-. aIind eCdAquLiTz and
TAL(, csf.tuTdabelnet1s) .recapitulated on the basis of a quiz presented with a PowerPoint presentation with
teacher’s tutoring. To exclude teacher’s assistance in CALC, students used an equivalent
Table1.Quasi-Experimental(BACI)DesignaoftheStudy.
computer-aided quiz (cf. Table 1).
Group TimeSpan Instruction1—CALC Instruction2—CALT Instruction3—TAL Control
Table 1. Quasi-Experimental (BACI) Design a of the Study.
Pre-test K1 K1 K1 K1
Delay 2weeks
Group Time span Instruction 1—CALC Instruction 2—CALT Instruction 3—TAL Control
Instructionb 70min CAL CAL TAL
Pre-Cteosnts olidation 10min CompKut1e r-Aided Teacher-AKs1s isted Teacher-AsKsi1st ed K1
DelayP ost-test 2 wee1k0sm in K2+ IMI K2+I MI K2+IMI K2
Delay 6weeks
Instruction b 70 min CAL CAL TAL
Retentiontest K3 K3 K3 K3
Consolidation 10 min Computer-Aided Teacher-Assisted Teacher-Assisted
Note:aThisdesignimpliedaone-waymultivariateanalysisofvariancewithrepeatedmeasureswhichhadto
Post-test 10 min K2 + IMI K2 + IMI K2 + IMI K2
berejectedonthegroundsofthenon-normaldistributionofvariables;bincludinga10-minintroduction.
Delay 6 weeks
RetentionA tewsto rkbookp rovidedaguideliKn3e throughthelessonsK.3Q uestionsinthewoKr3k bookhadtobKe3
aNnostwe:e ra eTdhaius tdoensoigmno uimslpylibeyd tha eotneea-mwauys inmgurletisveaarricahtei nanthaelyhsiosm oefp vaagreiaanncde twheithte xretbpoeoakterde smpeecatsiuvreelsy .
which had to be rejected on the grounds of the non-normal distribution of variables; b including a
10-min introduction.
A workbook provided a guideline through the lessons. Questions in the workbook had to be
answered autonomously by the team using research in the homepage and the textbook respectively.
Students were requested to complete their own workbook. There was no teacher support needed,
except with regard to technical problems with the in-school computer network.
Educ.Sci.2016,6,29 7of16
Studentswererequestedtocompletetheirownworkbook. Therewasnoteachersupportneeded,
exceptwithregardtotechnicalproblemswiththein-schoolcomputernetwork.
2.4. Quasi-ExperimentalBefore-After-Control-ImpactDesign(BACI)Design
Astudywithstudentsatschoolsineverydayconditionsiscomparabletoanecological“openfield
experiment”: howeverstrictthedesign,ineducationalsettingstherearealwaysmanyuncontrollable
factors. Thisiswhywecallthisstudy“quasi-experimental”. Weuseabefore-after-control-impact
design(BACI)(Smith[51]): Wemeasurecognitiveknowledgebeforeandafterthetreatment.
Inourstudywefocusedontheeffectofmedia: atextwasprovidedinahypermediaenvironment
usingcomputersorwithanequivalenttextbook. Inathirdtreatmentgroupweexcludedtheteacher’s
tutoring completely. The consolidation phase was a quiz with computer feedback instead of the
teacher-assistedrecapitulation. Wecollecteddataonstudents’cognitivelearningachievementand
intrinsicmotivation. Weexaminedtheeffectsofcomputer-aidedandtextbook-guidedlearningon
cognitiveknowledgeandlearningemotions,andtookgenderintoaccount.
2.5. DependentVariable
2.5.1. KnowledgeTest
Previousknowledgeandchangesinthelevelofknowledgeweremeasuredbymeansofapre-test
(K1)twoweeksbefore,apost-test(K2)immediatelyafterthelessonandaretentiontest(K3)6weeks
after that (Table 1). The questionnaire covered the content of the learning targets of the lesson.
The16itemswereinmultiple-choiceformatwiththreedistractorsandonecorrectanswer(examples
Table2). Althoughmultiplechoiceknowledgetestsdonottestmeaningfullearningthistoolcantest
long-termlearning,whichisconsideredasanindicatorforDeeperLearning[32].
Table2.Examplesformultiple-choicequestionnaires.
SubjectMatter Question MultipleChoiceDistractors
(cid:3) Byperformingthe“beedance”
Howdoesaworkingbee (cid:3) Thebeesmellslikethefood
(a) superorganism
beecolony tellitscolleaguewhatfood (cid:3) Thisisnottold,onlythelocationofthefood
ithasfound? (cid:3) Withspecificbuzzingsounds
(cid:3) Cleaning
(cid:3) Doingsentryduty
Whatisthefirstjobof
(b) timelines anew-bornworkingbee? (cid:3) Collectingfood
(cid:3) Buildingcomb
(cid:3) Nectar
(cid:3) Honeydew
Whichmaterialisnot
(c) products collectedbybees? (cid:3) Pollen
(cid:3) Wax
As students had hardly any knowledge (sum score M = 5.4 of 16, Figure 2), K1 had a low
Cronbach’salpha=0.05. Reliabilitywas,however,acceptablyhighinK2(Cronbach’salpha=0.73)
andquitegoodforthelearningtestsinK3(Cronbach’salpha=0.59). Generally,knowledgetestsare
difficulttotestforreliabilityalthoughreliabilitycoefficientslessthan0.6areusedfordifferentiating
groups[52](p. 213).
Educ. Sci. 2016, 6, 29 8 of 15
that the different sample sizes and the quasi-experimental design of the study affected the
non-normal distributed data (Shapiro-Wilk p = 0.000).
2.5.2. Intrinsic Motivation Inventory (IMI)
The “Intrinsic Motivation Inventory” (IMI) [54] is a multidimensional measurement device to
assess participants’ subjective experience related to a target activity (e.g., a lesson). IMI has been
applied effectively in many experiments related to intrinsic motivation and self-regulation (e.g., [55–
58]). We applied four of the six subscales of IMI: The subscale interest/enjoyment (IMI-I) is deemed
to be the self-report measure of intrinsic motivation (seven items; e.g., “I enjoyed doing this activity
very much.”). The concept of perceived competence (IMI-C) is considered to be a positive predictor
of both self-report and behavioural measures of intrinsic motivation (6 items; e.g., “I think I am
pretty good at this activity.”). The subscale effort (IMI-E) is relevant to some motivation questions
and is thought to measure activity’s relevance (five items; e.g., “I put a lot of effort into this.”). The
felt tension/pressure (IMI-T) is theorized to be a negative predictor of intrinsic motivation (five
items; e.g., “I was very relaxed in doing this.”). We used the German version of IMI, which has
previously been successfully applied, for instance, by Girwidz et al. [1]. Students scored their answer
on a five-digit Likert scale ranking from 1 (not true at all) to 5 (very true). The test was integrated in
the post-test K2 after the knowledge questionnaire, and applied immediately after the intervention
(Table 1).
2.6. Analysis
For statistical analyses we used SPSS 21.0 (IBM SPSS Statistics, IBM Deutschland GmbH,
Ehningen, Germany). Due to the partially non-normally distributed variables (Kolmogorov-Smirnov
test: p > 0.05) we used non-parametric analysis. Knowledge items were scored as correct (1 point) or
incorrect (0 point) and summed to yield a score. In the case of IMI, we calculated mean scores for
each subscale. We calculated inter-group differences with the Friedman test (FT) and between-group
differences with Mann-Whitney U test (MWU) for two independent samples, and the Kruskal-Wallis
test (KW) to analyse k independent samples. The significance of learning success was calculated
using the Wilcoxon test (WT).
3. Results
3.1. Cognitive Achievement
There was no test effect as the control group did not gain knowledge (FT: Chi-Square = 2.995, df
= 2, p = 0.224). The K1 showed neither differences between treatment groups (KW Chi-Square = 2.769,
Educ.Sci.2016,6,29 8of16
df = 3, p = 0.429) nor between the genders (MWU: Z = −0.864, p = 0.388).

FFiigguurree 22.. KKnnoowwlleeddggee ssuumm ssccoorreess ooff bbooyyss aanndd ggiirrllss ffoorr aallll iinnssttrruuccttiioonnss wwiitthhoouutt ccoonnttrrooll ((KK11:: pprree--tteesstt;;
KK22:: ppoosstt--tteesstt;; KK33:: rreetteennttiioonn tteesstt;; ggeennddeerr ddiiffffeerreenncceess:: ******:: pp ≤≤ 00.0.00011; ;****: :pp ≤≤ 0.001.0)1. ).
Itemdifficulty,definedasthepercentageofcorrectanswers[53]wasnormallydistributedfor
theK1,K2andK3. Contentvalidityoftheknowledgetestisgivenduetothesyllabus-basedsubject.
Furthermore,allknowledgeitemswereconstructedaccordingtothelearninggoalsoftheintervention.
Since the knowledge sum scores were not normally distributed (Kolmogorov–Smirnov with
Lillieforssignificancecorrection,p<0.001),weusednon-parametrictestsandboxplots. Wepresume
thatthedifferentsamplesizesandthequasi-experimentaldesignofthestudyaffectedthenon-normal
distributeddata(Shapiro-Wilkp=0.000).
2.5.2. IntrinsicMotivationInventory(IMI)
The“IntrinsicMotivationInventory”(IMI)[54]isamultidimensionalmeasurementdeviceto
assessparticipants’subjectiveexperiencerelatedtoatargetactivity(e.g.,alesson).IMIhasbeenapplied
effectively in many experiments related to intrinsic motivation and self-regulation (e.g., [55–58]).
WeappliedfourofthesixsubscalesofIMI:Thesubscaleinterest/enjoyment(IMI-I)isdeemedtobe
theself-reportmeasureofintrinsicmotivation(sevenitems;e.g.,“Ienjoyeddoingthisactivityvery
much.”). Theconceptofperceivedcompetence(IMI-C)isconsideredtobeapositivepredictorofboth
self-reportandbehaviouralmeasuresofintrinsicmotivation(6items;e.g.,“IthinkIamprettygoodat
thisactivity.”). Thesubscaleeffort(IMI-E)isrelevanttosomemotivationquestionsandisthoughtto
measureactivity’srelevance(fiveitems;e.g.,“Iputalotofeffortintothis.”). Thefelttension/pressure
(IMI-T) is theorized to be a negative predictor of intrinsic motivation (five items; e.g., “I was very
relaxedindoingthis.”). WeusedtheGermanversionofIMI,whichhaspreviouslybeensuccessfully
applied,forinstance,byGirwidzetal.[1]. Studentsscoredtheiransweronafive-digitLikertscale
rankingfrom1(nottrueatall)to5(verytrue). Thetestwasintegratedinthepost-testK2afterthe
knowledgequestionnaire,andappliedimmediatelyaftertheintervention(Table1).
2.6. Analysis
ForstatisticalanalysesweusedSPSS21.0(IBMSPSSStatistics,IBMDeutschlandGmbH,Ehningen,
Germany).Duetothepartiallynon-normallydistributedvariables(Kolmogorov-Smirnovtest:p>0.05)
we used non-parametric analysis. Knowledge items were scored as correct (1 point) or incorrect
(0point)andsummedtoyieldascore. InthecaseofIMI,wecalculatedmeanscoresforeachsubscale.
We calculated inter-group differences with the Friedman test (FT) and between-group differences
withMann-WhitneyUtest(MWU)fortwoindependentsamples,andtheKruskal-Wallistest(KW)to
analysekindependentsamples.ThesignificanceoflearningsuccesswascalculatedusingtheWilcoxon
test(WT).
Educ.Sci.2016,6,29 9of16
3. Results
3.1. CognitiveAchievement
Therewasnotesteffectasthecontrolgroupdidnotgainknowledge(FT:Chi-Square=2.995,df=2,
p=0.224). The K1showed neither differences between treatment groups (KWChi-Square =2.769,
Educ. Sci. 2016, 6, 29 9 of 15
df=3,p=0.429)norbetweenthegenders(MWU:Z=−0.864,p=0.388).
Students of TAL gained significantly higher sum scores than students of CALT immediately
Students of TAL gained significantly higher sum scores than students of CALT immediately
afterthelesson(MWU:K2Z=−4.430,p<0.001;Figure3—K2)andyieldedthehighestknowledge
after the lesson (MWU: K2 Z = −4.430, p < 0.001; Figure 3—K2) and yielded the highest knowledge
scoresamongallstudents. Aftersixweeksbothgroupsachievedequivalentcognitivesumscores
scores among all students. After six weeks both groups achieved equivalent cognitive sum scores
(MWU:K3Z=−1.098,p=0.272;Figure3—K3). InCALTaswellasinTALgirlsgainedsignificantly
(MWU: K3 Z = −1.098, p = 0.272; Figure 3—K3). In CALT as well as in TAL girls gained significantly
highersumscoresthanboysinK2(MWU:CALTZ=−2.385,p=0.017;TALZ=−3.784,p<0.001;
higher sum scores than boys in K2 (MWU: CALT Z = −2.385, p = 0.017; TAL Z = −3.784, p < 0.001;
Figure 3—K2). This was still found after six weeks in K3 (MWU: CALT Z = −3.777, p < 0.001;
Figure 3-K2). This was still found after six weeks in K3 (MWU: CALT Z = −3.777, p < 0.001;
TTAALLZ Z= =− −22..333399,, pp == 00..001199;; FFiigguurree 33——KK33)).. GGirirlsls ffoorrggoot tlelesss skknnoowwleldedgeg eafatfetre rsisxi xwweeekesk swwithit hCACLATL,T ,
wwhhiicchh ccaauusseedd ssiiggnniiffiiccaanntt ddiiffffeerreenncceess inin oobblilvivioionn rartaetse s(K(K3–3K–2K)2 ()g(irglisr lMs WMUW: UZ: =Z −=3.3−933.,3 p9 3=, 0p.0=010; .a0l0l1 ;
aMllWMUW: UZ: =Z −=3.0−336.,0 p3 6=, 0p.0=020;. 0F0i2g;uFreig 4u)r. e4).

FFiigguurree 33.. KKnnoowwlleeddggee ssuumm ssccoorreess aafftteerr ggeennddeerr.. CCAALLCC: :ccoommppuuteterr-a-aidideedd leleaarnrniningg wwitihth cocmomppuutetre-ra-iadiedde d
ccoonnssoolliiddaattiioonn;; CCAALLTT:: ccoommppuutteerr--aaiiddeedd lleeaarrnniinngg wwiitthh tteeaacchheerr--aassssisistetedd ccoonnsosolildidataitoinon; ;TTAALL: :tetxetxbtobooko k
lleeaarrnniinnggw witihthte atecahcehre-ars-assisstiesdtecdo ncosonlsiodlaitdioatni;oKn;1 :Kp1r:e -pterest-;teKs2t;: pKo2s: t-pteosstt;-Kte3s:t;r eKte3n: trioetnentetsiot;nb ltaecskt;a sbtlearciksk :
tarestaetrmisekn: ttreefafetmctse;nbt leufefeacstste; rbilsuke: agsetnedriesrk:e gffeencdtse;r* *e*f:fepc≤ts; 0*.*0*0: 1p; ≤* *0:.0p0≤1; 0**.0: 0p6 ≤; *0:.0p0≤6; *0:. 0p1 ≤5 )0..015).
All students learned with computers in both CALC and CALT. Cognitive achievement
All students learned with computers in both CALC and CALT. Cognitive achievement
immediately after the lesson was equivalent independent of the consolidation phase of CALC or
immediately after the lesson was equivalent independent of the consolidation phase of CALC or
CALT (MWU: K2 Z = −1.12, p = 0.265). In respect to long-term learning, students of CALT gained
CALT(MWU:K2Z=−1.12,p=0.265). Inrespecttolong-termlearning,studentsofCALTgained
significantly higher knowledge scores than students of CALC (MWU: Z = −2.23, p = 0.026).
significantlyhigherknowledgescoresthanstudentsofCALC(MWU:Z=−2.23,p=0.026).
The lack of teacher guidance is disadvantageous notably for girls: in CALC—without
The lack of teacher guidance is disadvantageous notably for girls: in CALC—without
teacher-guided consolidation phase—all students, even the girls, yielded lower knowledge scores in
teacher-guidedconsolidationphase—allstudents,eventhegirls,yieldedlowerknowledgescoresin
K3 than in CALT or TAL. The treatment effect is significant for girls with TAL (K2 MWU: Z = −2.858,
K3thaninCALTorTAL.ThetreatmenteffectissignificantforgirlswithTAL(K2MWU:Z=−2.858,
p = 0.004). But there is no treatment effect with the boys. Accordingly, gender effects were reduced in
p=0.004). Butthereisnotreatmenteffectwiththeboys. Accordingly,gendereffectswerereduced
CALC: there is no gender effect in K3 (MWU: Z = −0.949, p = 0.343) and hardly any in K2 (MWU: K2
inCALC:thereisnogendereffectinK3(MWU:Z=−0.949,p=0.343)andhardlyanyinK2(MWU:
Z = −2.065, p = 0.039).
K2ZT=h−is2 f.i0n6d5i,npg= is0 r.0em39a).rkable as girls seemed to learn more in general as both genders have similar
pre-kTnhoiswfilenddgine gsciosrreesm (MarWkaUbl eKa1s: Zg i=rl −s0s.e6e6m, pe d> 0to.5)le, ayrent gmirolsre gianing ehnigehraelr assumbo tshcogreens dthearsn hbaovyes sinim Kil2a r
p(rMe-WknUo wK2le: dZg =e −sc2o.6r8e5s,( pM =W 0.U00K7)1 a:nZd= K−3 0(.M66W,pU> K03.5: )Z, y=e −t3g.9ir7l4s,g pa =in 0h.0i0g0h;e Frisguumre s2c)o. resthanboysinK2
(MWUFoKr 2t:hZe i=nd−i2ca.6t8io5n, po=f D0.L00 w7)ea anrde Kin3te(MreWsteUd Kin3 :thZe= o−bl3iv.9io74n, rpa=te0, .w00h0i;cFhi gisu rceal2c)u.lated from the
diffeFreonrcteh oef iKnd3 iacnadti oKn2.o TfhDerLe wis ea asrigeniinfitcearnest tterdeaitnmtehnet eofbfelicvt iwonithra ltoew, ewsth oicbhlivisiocna lrcautelas tfeodunfrdo mwitthh e
dCifAfeLrTen (cTeabolfeK 33; FaingdurKe 24.)T. Gheirrles ifsoragsoitg mniofiscta wntitthre TaAtmLe, nwtheefrfeecats wboityhs lfoowrgeostt mobolriev iwonithra CteAsLfoCu. ndwith
CALT(Table3;Figure4). GirlsforgotmostwithTAL,whereasboysforgotmorewithCALC.
Educ.Sci.2016,6,29 10of16
Educ. Sci. 2016, 6, 29 10 of 15

FigurFeig u4.r eO4b.liOvibolniv i(oKn3(–KK32–)K 2a)ftaefrt egregnednedre.r .CCAALLCC:: ccoommppuutteerr-a-aididededl elaeranrinnginwg itwhitcho mcpoumtepru-atiedre-adided
consocolindsaotliiodna;t iConA;LCTA: LcTo:mcopmupteurt-ear-idaiedde dlelaearnrniningg wwiitthh tteeaacchheerr-a-asssissitsetdedc ocnosnosliodlaidtiaotni;oTnA; LT:AteLx: tbteoxotkbook
learnlienagrn winigthw tiethactheaecrh-aesr-saisssteisdte cdocnosnosloidlidaatitoionn))..
TaTbalbe l3e. 3S.tSattaitsitsitcic DDaattaa ooff tthhee KKnnoowwleleddggeeT eTsetss.ts.
Effect Effect TreTartematemnetn t TTeesstt StSattaisttiisctsics
Test EffecTt estEffect ConCtornotlr ol KK11,, KK22,,K K33 FTF:TC: hCi-hSiq-uSaqruea=r2e. 9=9 52,.9d9f5=, 2d,fp == 20,. 2p2 4= 0.224
K1 KW Chi-Square = 2.769, df = 3, p = 0.429
K1 KWChi-Square=2.769,df=3,p=0.429
TreatmentT rGeartomuepn tGroup KK22 KKWWC hCi-hSiq-uSaqruea=r2e1 =.1 4211,.1d4f1=, 2d,fp =< 20,. 0p0 0< 0.000
TATLA—LC—ACLATLT——CCAALLCC
DifferenceDs ifferences KK33 KKWWC hCi-hSiq-uSaqruea=r1e1 =.2 9181,.2d9f8=, 2d,fp == 20,. 0p0 4= 0.004
Oblivion KWChi-Square=17.456,df=2,p<0.000
Oblivion KW Chi-Square = 17.456, df = 2, p < 0.000
K2 MWU:Z=−4.430,p<0.000
K2 MWU: Z = −4.430, p < 0.000
TAL—CALT K3 MWU:Z=−1.098,p=0.272
TAL—CALT KO3b livion MMWWU:UZ: =Z− =3 −.013.60,9p8=, p0 .=0 002.272
Oblivion MWU: Z = −3.036, p = 0.002
K2 MWU:K2Z=−1.12,p=0.265
CALT—CALC KK32 MMWWU:UZ: =K−2 2Z.2 3=, −p1=.102.0, 2p6 = 0.265
CALT—CALC KO3b livion MMWWU:UZ: =Z− =3 −.926.32,3p, <p 0=. 000.0026
OK2blivion MMWWU:UZ: =Z− =2 −1.31.4916,3p, <p 0<. 000.0000
TAL—CALC KK32 MMWWU:UZ: =Z− =1 −1,2219.81,4p1=, p0 .<00 04.000
Oblivion MWU:Z=−17.456,p<0.000
TAL—CALC K3 MWU: Z = −11,298, p = 0.004
K2 KWChi-Square=24.070,df=2,p<0.000
Girls‘Treatment Oblivion MWU: Z = −17.456, p < 0.000
TAL—CALT—CALC K3 KWChi-Square=9.802,df=2,p=0.007
Effects K2 KW Chi-Square = 24.070, df = 2, p < 0.000
Girls‘ Treatment Oblivion KWChi-Square=12.213,df=2,p=0.002
TAL—CALT—CALC K3 KW Chi-Square = 9.802, df = 2, p = 0.007
Effects K2 MWU:Z=−3.915,p<0.000
TAL—CALT OObblliivviioonn MKWWU :CZh=i-−Sq3.u3a93r,ep == 102.0.20113, df = 2, p = 0.002
K2 MWU: Z = −3.915, p < 0.000
TACLA—LCT—ACLAT LC K3 MWU:Z=−2.858,p=0.004
Oblivion MWU: Z = −3.393, p = 0.001
K2 MWU:Z=−4.024,p<0.000
CATLATL——CCAALLCC KK33 MMWWU:UZ: =Z− =3 −.120.68,5p8=, p0 .=0 002.004
K2 MWU: Z = −4.024, p < 0.000
Boys’Treatment TATLA—LC—ACLACLT —CALC K2 KWChi-Square=12.890,df=2,p=0.002
Effects K3 MWU: Z = −3.106, p = 0.002
Boys’ Treatment K2 MWU:Z=−3.418,p=0.001
TATLA—LC—ACLATLT—CALC K2 KW Chi-Square = 12.890, df = 2, p = 0.002
Effects K3 MWU:Z=−2.457,p=0.014
KK22 MMWWU:UZ: =Z− =2 −.737.44,1p8=, p0 .=0 006.001
TACLA—LCT—ACLAT LC
KO3b livion MMWWU:UZ: =Z− =2 −.428.84,5p7=, p0 .=0 103.014
TAL—CALC K2 NMosWigU.d: iZffe =re −n2ce.7s7!4, p = 0.006
CALT—CALC
Oblivion MWU: Z = −2.488, p = 0.013
TAL—CALC No sig. differences!
K1 MWU: Z = −0.864, p = 0.388
Gender Effects all treatments K2 MWU: Z = −2.685, p = 0.007
K3 MWU: Z = −3.974, p < 0.000
K2 MWU: Z = −3.784, p < 0.001
TAL
K3 MWU: Z = −2.339, p = 0.019
K2 MWU: Z = −2.385, p = 0.017
CALT
K3 MWU: Z = −3.777, p < .001;
K2 MWU: Z = −2.065, p = 0.039
CALC
K3 MWU: Z = −0.949, p = 0.343