The effect of apologetic error messages and mood states on computer usersв€™ self-appraisal of ...

19 Pages·2010·1.1 MB·English

JournalofPragmatics42(2010)2430–2448
ContentslistsavailableatScienceDirect
Journal of Pragmatics
journal homepage: www.elsevier.com/locate/pragma
The effect of apologetic error messages and mood states on computer
users’ self-appraisal of performance
Mahir Akguna,*, Kursat Cagiltaya, Deniz Zeyrekb
aComputerEducationandInstructionalTechnologyDepartment,MiddleEastTechnicalUniversity,06351,Ankara,Turkey
bDepartmentofForeignLanguageEducation&CognitiveScience,MiddleEastTechnicalUniversity,06351,Ankara,Turkey
ARTI CLE INFO ABS TRA CT
Articlehistory: This study,inwhich310 universitystudents participated, wasdesignedtoinvestigate
Received31August2009 whethercomputerinterfacesthatofferhuman-likeapologeticerrormessagesinfluence
users’self-appraisalsofperformanceinthecomputerizedenvironment.Thestudyconsists
Keywords: ofthreephases.Inthefirstphase,usingtheCCSARP(cross-culturalstudyofspeechact
Self-appraisalofperformance
realization patterns) coding manual, apology strategy sequences were elicited from
Apologeticerrormessage
Turkish participants. Two of these apology strategy sequences were selected for the
Moodstate
secondphase,whichisatestincludingexperimentalandcontrolgroups.Theexperimental
Human-likecomputerinterface
groupswerepresentedwiththetwoapologysequences,andthecontrolgroupwasgivena
plain computer message. The second phase investigated whether any of these three
messageswereperceivedasapologies.Theresultsindicatethatthetwoapologymessages
wereperceivedasapologies,buttheplaincomputermessagewasnotperceivedasone.
Thethirdphaseinvestigatedtherelationshipbetweentheusers’moodsandtheirself-
appraisalsofperformanceafterthetransmissionoftheapologeticerrormessages.The
findings show that the influence of apology messages on the users’ self-appraisals of
performance depends on the participants’ mood state and the content of the apology
messages.
(cid:1)2010PublishedbyElsevierB.V.
1. Introduction
Oneofthemajorgoalsinhuman–computerinteraction(HCI)istheemploymentofdesignsthatallowmorehuman-like
communication(Burgoonetal.,2000;RitterandYoung,2001).Suchman(1987)mentionedtheterm‘‘sociabilityofcomputers’’
andadvisedthatpropertiesofhuman–humaninteraction(HHI)(e.g.,dialogue,conversation,etc.)shouldbeconsideredwhen
describingwhatgoesonbetweenpeopleandmachines.LisettiandSchiano(2000)morerecentlynotedaconspicuousparadigm
shiftinHCIfromthedesign-centeredapproach,i.e.,adaptingpeopletocomputers,totheuser-centeredapproach,i.e.,adapting
computerstopeople.Intheliterature,therehasbeenlittlediscussionconcerningwhichusers’needsshouldbeconsideredin
thedevelopmentofmorehuman-centeredinterfacesthatinteractwithuserswhentheirintendedtaskscannotbeexecuted
becauseoftheincapacityoftheproduct.Oneexampleofthissituationisacomputer’spresentationofanerrormessagewhena
problemoccursintheinterface.Mostoftheerrormessagescommonlyutilizedincomputerinterfacesareshortandplainly
technical;thesemessagesstraight-forwardlyrevealthenatureofcomputer-centereddesignstotheuserswhentheyencounter
aproblemduringtheirinteractionswiththeinterface(Tzeng,2004).Withregardtoerrormessages,analternativetothe
* Correspondingauthor.
E-mailaddresses:[email protected](M.Akgun),[email protected](K.Cagiltay),[email protected](D.Zeyrek).
0378-2166/$–seefrontmatter(cid:1)2010PublishedbyElsevierB.V.
doi:10.1016/j.pragma.2009.12.011
M.Akgunetal./JournalofPragmatics42(2010)2430–2448 2431
computer-centered interface is the human-centered interface, which presents error messages that include emotional
expressions,suchasapologies(Neilsen,1998;Tzeng,2004).
InHHI,apologiesaregenerallyusedtoexpressregret(Leech,1983;SchlenkerandDarby,1981)ortoalleviateindividuals’
anger caused from their disapproval of others’ action. In other words, apologies mitigate frustration and anger when
attemptedinteractionsfail.Similarly,Neilsen(1998)arguesthaterrormessagesrespondingtothecomputeruser’saction
shouldincludeasimpleapologeticstatementwhenthereasonfortheerroristhelimitationofthecomputerinterfaceto
performtheintendedtask.Tzeng(2006)conductedastudyinvestigatingusers’perceptionsofonlinesystemscontaining
threedifferenterrormessages,eachofwhichincludesdifferentpolitenessstrategies.Inthestudy,firstlyusers’politeness
orientationswereelicitedandthenparticipantswereaskedtointeractwithwebsitesincludingpre-determinedproblems.
When users encountered problems, the system provided certain error messages representing one positive politeness
strategy(i.e.,joke),onenegativepolitenessstrategy(i.e.,asimpleapology),andamechanicalmessagefortheerror(i.e.,the
page is temporarily unavailable). The findings of the study showed that users who deal with social events with polite
expressions preferred to receive apologetic messages significantly more than mechanical or joke messages, and they
preferredapologeticmessagessignificantlymorethanthosemessagesthatarelessorientedtopoliteexpressions.Tzeng
(2004)examinedwhetherapologeticfeedbackaffectsusers’performanceperceptioninthecomputerizedenvironment.A
computer-basedword-guessinggamewasdesignedforthestudyinwhichparticipantswereexpectedtoguessthecorrect
word after the computer provided a key word. Tzeng reported that the subjects in apologetic feedback groups did not
perceivetheirperformanceortheirabilitytoplaythegamemorepositivelythanthoseinnon-apologeticgroups.Thereason
fortheinsignificantresultsinthestudybyTzeng(2004)investigatingtheeffectofpoliteexpressions(i.e.,apologies)inthe
computerized environment might be the negligence of certain factors influencing the need for polite expressions. For
example,moodcanbesuchafactor.Forgas(1999)indicatedtheimportanceofmoodinthelevelofpolitenesspeopleuse.In
thisstudyitwasfoundthatindividualsinnegativeaffectproducedgreaterpolitenessthanthoseinpositiveaffectwhile
producingrequests.Inadditiontothis,anumberofdifferenttheorieshaveexplainedthemood-cognitionrelations(e.g.,
Bower,1981;Forgas,1995).Thefindingsofthestudiesexaminingthisrelationindicatethatmoodcaninfluencejudgments,
decisionsandevaluationsinatleasttwoways.First,moodcaninfluencewhatindividualsthink(thecontentofcognition).In
hisAssociativeNetworkTheory,Bower(1981)claimsthatmoodstateinfluencescognitiveprocessingrelatedtotheretrieval
ofinformationfromlong-termmemory.Accordingtothistheory,memoryconsistsofnetworksofassociatedconceptsand
eachcognitivenetworkincludesemotion-relatedmemoriesandcognitions.Withineachcognitivenetwork,certainemotion
nodes,whicharedefinedasthegroupsofassociatedconcepts,standforspecificemotionssuchassadness,angerandfear.
Theactivationofaparticularnodetriggerstherelatednetworkofconnectionstoretrieveemotion-relatedmemoriesand
cognitions.Bower(1981)foundthatindividualsrecalledmostoftheexperiencesthatwereaffectivelycongruentwiththe
mood they were in during recall. Stated differently, an individual in a positive mood retrieves positive thoughts and
associations more easily from memory, whereas recalling negative thoughts and associations is more effortless for an
individualinanegativemood.Inadditiontoassociativenetworktheory,Isen(1985)suggestedthatmostindividualsretrieve
positivethoughtsandassociationsmoreeasilythannegativethoughtsandassociationsbecausepositivematerialsarebetter
integratedandmoreextensive.
Second,moodcanhaveanimpactonhowindividualsthink(theprocessofcognition).Fielder(2000)indicatessome
differencesbetweenpositiveandnegativemoodsintermsoftheprocessofthinking.Internallydriven,top-down,flexible
andgenerativeprocessingstylesaremorerelatedtopositivemood,whileexternallyoriented,bottom-upandsystematic
thinkingstylesaremorerelatedtonegativemood.Forgas(1994,1995)developedtheAffectInfusionModelinorderto
contribute to the discussion concerning the process of cognition. According to the model, four alternative processing
strategiesmightbeusedwhenmakingjudgments:Thedirectaccessofapreexistingevaluation(1)andmotivatedprocessing
towardapreexistinggoal(2)involvelittlegenerativeandconstructiveprocessing;duringtheseprocessingstrategies,mood-
incongruentprocessingmayoccur.Ontheotherhand,theAffectInfusionModelsuggeststhatmood-congruentprocessing
occurs during (3) heuristic processing and (4) substantive processing. These processing strategies involve constructive
processing of information. Theories of mood-congruent processing support the idea that positive moods lead to more
favorablejudgments,whereasnegativemoodsleadtolessfavorablejudgments.
Previousstudiesaimingtounderstandtheeffectofhuman-likemessagesoncomputerusersgenerallyconcentratedon
theeffectofhuman-likemessagesintermsofactualperformance,performanceperception,self-appraisalandinteraction
time.Theactivitiesrelatedtothevariablesmeasuredinpreviousstudiesrequirecognitiveeffortandsothefactors(i.e.,mood
state)influencingcognitiveprocessesalsoaffectthelevelofsuccessinthesekindsofactivities.Therefore,moodstateasa
factorinfluencingbothcognitiveprocessesandpreferencesofpoliteexpressionsshouldbeconsideredwhentheeffectof
politeexpressions(i.e.,apologies)oncognitivevariables(i.e.,self-appraisal)isthefocusofinvestigation.
Consistentwiththepurposeofcreatinghuman-likecomputerinterfacesinHCI,thepropertiesofHHIhavebeenexpected
tobevalidforcomputerinterfaces.Asstatedbefore,oneofthecrucialaspectsofHHIistouseapologyformaintainingthe
relationship between the interactants by alleviating frustration and anger of the frustrated interactant whenever the
interaction fails for any of the various reasons. Therefore, HCI researchers have widely accepted the practice of using
apologeticerrormessagesinthecomputerizedenvironmentwhenusersencounteraproblemcausedbythecomputer’s
inabilitytocarryoutatask.
Basedonthemodelsexplainingtheeffectofmoodstateoncognitiveprocesses(e.g.,Bower,1981;Forgas,1995)andthe
findingsindicatingtheroleofmoodstateinthepreferencesofpoliteexpressions(e.g.,Forgas,1999),itcanbeproposedthat
2432 M.Akgunetal./JournalofPragmatics42(2010)2430–2448
moodstate shouldbetakenintoaccountwhentheeffectof politeexpressionsoncognitivevariablesisinvestigated.In
particular,theuseofapologeticerrormessagesinrelationwithusers’moodstatesinthecomputerizedenvironmentmay
influenceusers’self-appraisalsofperformancebymitigatingtheusers’frustrationandangerwhentheirintendedtasks
cannotbeexecuted.Inordertoachieveabetterunderstandingofthis,thepresentstudyinvestigateswhetherapologetic
errormessagesandtheusers’moodstatessignificantlyaffectcomputerusers’self-appraisalsofperformance.
2. Significanceofthestudy
Most studies that have focused on computer interfaces that present human-like messages investigated the effect of
human-likemessagesoncomputeruserswithoutconsideringotherimportantfactors,suchastheusers’moodstatesandthe
waythemessagesareformulated.Thereisasignificantneedforempiricalresearchontheeffectsofhuman-likemessageson
computerusersinrelationwiththeirmoodstates.Thefindingsofthisstudywillhopefullyrevealthesignificanceofusers’
moodstatesandhelptodeterminewhichformsofhuman-likeapologeticerrormessagesaremosteffectiveincomputer
interfaces.
The main goal of research concerning human-like interfaces is to develop interactive computer systems which have
certaincharacteristicsthatresemblethoseofhumanbeings.AccordingtoNassetal.(1994),thesocialrulesanddynamics
which guide HHI can be applied equally well to HCI. This idea has become one of the main areas of focus in research
concerninghuman-likeinterfaces.However,onlyafewstudieshaveconcentratedonthedynamicswithinHCI.Thisstudy
hasbeendesigned tofill thisgap withthehelpofthefieldof Pragmaticswhichis definedasa science focusingon the
language-usinghuman(Mey,1993).Theresearchersconductedapragmaticanalysistodeterminewhichapologystrategies
arepreferredinthecomputerizedenvironmentandhowtheseselectedapologystrategysequencesinteractwiththeusers’
moodstates.Theresultsyieldseveralsignificantfindingswhichtheauthorshopewillbeanimportantcontributiontofuture
researchinthefieldofthePragmaticsofHuman–ComputerInteraction.
3. Apologies
OlshtainandCohen(1983)proposedanapologytaxonomyconsistingoffivemainstrategiesandmanysub-strategies
(notmentionedinthispaper)1:
A. Illocutionaryforceindicatingdevices(IFIDs)
E.g.,Iapologize,Iamsorry.
B. Anexplanationoraccount(Exp)
E.g.,Thetrafficwasterrible,thebuswaslate.
C. Takeonresponsibility(ToR)
E.g.,Itismymistake,Ididn’tmeanto.
D. Offerofrepair(OoR)
E.g.,Iwillpayforthedamage.
E. Promiseofforbearance(PoF)
E.g.,Thiswon’thappenagain.
Whiletwooftheseapologystrategiesaregeneral,namelytheIFIDandtheToR,theotherthreestrategiesaresituation
specific(Olshtain,1989).Ithasbeenproposedthatthesetwostrategiescanbeusedinallsituationsandinalllanguages
(Olshtain,1989).IFIDsare‘‘formulaic,routinizedexpressionsinwhichthespeaker’sapologyismadeexplicit’’(Blum-Kulka
etal.,1989,p.290).TheToRstrategyreferstoacceptingtheresponsibilityofanactinordertoplacatethehearer.
Theremainingthreestrategies(theExp,theOoR,andthePoF)aresituationspecific,whichmeansthattheirusedepends
onthecontext.WhenthespeakerpreferstousethestrategyExp,s/hedescribesthesituationwhichcausedhimorherto
committheoffense.TheOoRmaybechosenbythespeakerforsituationsinwhichtheinconvenienceinfluencingthehearer
canberepaired.Finally,thePoFisemployedbythespeakerinordertoassurethehearerthattheactionwhichcreatedthe
needtoapologizewillnothappeninthefuture.
Olshtain(1989)presentedsevensituationsinadiscoursecompletionquestionnaire,andthenanalyzedtheresponses
givenbytheparticipantsinHebrew,AustralianEnglishandCanadianFrench.Thefindingsofthestudyshowedthatthe
HebrewspeakerspreferredtousetheIFIDandtheToRinallsituations.Therewasaconsiderablevariationinthelevelof
preferenceinrelationtothesituations.Olshtainalsofoundthatinthethreelanguagesstudied,therespondentsemployed
IFIDsbetween60%and75%ofthetimeandtheToRstrategybetween65%and70%ofthetime.Thesefindingsshowedthat
thesetwostrategiesarehighlyusedinallthreelanguages.Theotherthreestrategieshavemuchlowerusecomparedtothe
IFIDandtheToR.
1 Throughoutthepaper,theabbreviationsshownintheparentheseswillbeusedtorefertotheseapologystrategies.
M.Akgunetal./JournalofPragmatics42(2010)2430–2448 2433
VollmerandOlshtain(1989)usedthediscoursecompletiontesttoinvestigateapologypreferencesinGerman.Intheir
study,eightdifferentapologysituationsweredevelopedandparticipants’choicesofapologystrategiesacrosssituationswere
investigated.TheirfindingsshowedthattheparticipantsemployedtheIFIDandtheToRinallsituations:TheuseoftheIFID
acrosssituationsrangedfrom37.5%to84%.Ontheotherhand,theuseoftheToRacrosssituationswasbetween30%and94%.
Hatipoğlu(2003)investigatedtheapologystrategiesutilizedbyBritishandTurkishpeople.Thefindingsofthatstudy
revealedthathalfofthecollectedapologieswereexplainedbytheIFIDandtheToRinbothTurkishandBritishculture.The
percentagesofthestrategiesforeachculturewereasfollows:TheIFID=33%forBritishand24%forTurkish,theExp=10%for
Britishand9%forTurkish,theToR=17%forBritishand26%forTurkish,theOoR=24%forBritishand22%forTurkish,andthe
PoF=1%forBritishand1%forTurkish.
Inanotherstudy,Hatipoğlu(2004)investigatedtheformandtypeofapologiesusedine-mailmessages.Shefoundthat
e-mail apologies have some distinctive properties that distinguish them from those utilized in spoken and written
languages.Forexample,intheone-to-onedialoguegroupincludingpersonale-mailsintendedforonlyonereceiver,11.1%of
the total apologies were the apologies employed for ‘an offer of apology’ (i.e., I apologize) and 85.7% of them were the
apologies utilized for ‘an expression of regret’ (i.e., I am sorry). Since ‘an offer of apology’ is generally used for formal
expressionsand‘anexpressionofregret’ismostlyemployedforlessformalexpressions,thisfindingcouldbeutilizedasa
support for the idea that the language in e-mails is less formal than the one in traditional writing (Hatipoğlu, 2004).
Furthermore,ine-mailcommunications,apologiesareusedifthereisapossibilityofmultipleposting(e.g.,‘‘apologiesfor
anyduplicatee-mails/multipleposting’’,‘‘apologiesifNhasalreadysentthis’’and‘‘(with)apologiesfor(any)cross-posting’’).
Aswillbeexplainedinthesection5.2.4,thetaxonomyproposedbyOlshtainandCohen(1983)wasusedtoanalyze
apologystrategysequenceselicitedfromcomputerusers.Inthefollowingsection,adefinitionof‘‘human-likecomputer
interface’’andtheresultsofstudieswhichinvestigatedtheeffectivenessofthiskindofinterfaceonusersarepresented.
4. Human-likeinterfaces
Ahuman-likecomputerinterfaceisakindofinteractivecomputersystemwhichhascertaincharacteristicsthatresemble
characteristicsfoundinhumanbeings.Basedontheideathathumanemotionalexpressionsareimportantininterpersonal
communication,interactivecomputersystemshavebeenbuilttorespondtouserswhoexperiencefrustrationandother
negativeemotionswithemotionallysupportiveresponseinteractions.
Many studies have shown that people can apply the social rules which regulate HHI to HCI. As an exampleof these
studies,FoggandNass(1997)reportedthatuserswhoreceivedflatteryfromacomputerviewedtheinteractionasmore
enjoyablethanthosewhoreceivedplaincomputerfeedback.Thisstudyalsoshowedthatuserswhoreceivedflatteryfrom
thecomputerdisplayedamuchgreaterinterestincontinuingtoworkwiththecomputerthanthosewhoreceivedplain
computerfeedback.Kleinetal.(1999)reportedthesameresultsasFoggandNass.
Oneoftheimportantconceptsthatshapehuman-likeinterfacesisthemediaequation(i.e.,‘‘mediaequalsreallife’’).
Mediaequationresearchconcentratesonsocialrulesandnorms,suchaspoliteness,reciprocity,flattery,assignmentofroles,
andcriticism(ReevesandNass,1996).Nassetal.’s(1994)reportedthatpeople’sinteractionswithcomputersaresocial,and
thatthisisnottheresultofaconsciousbeliefthatcomputersarehuman-like.Inotherwords,userstreatcomputersasifthey
are human during their interactions with the computers, even though they know that the computers have no human
motivations,suchasfeelings.NassandMoon(2000)reportedthatpeopletendtorelyonsocialcategoriesandmindlessly
applysocialrulestotheircomputerinteractions.ResnikandLammers(1985)suggestedthathighself-esteemsubjectswho
receivedhuman-likeerrormessages(e.g.,‘Idon’tunderstandtheseletters’)performedsignificantlybetteroncomputerized
tasksthanhighself-esteemsubjectswhoreceivedlesspersonal,computer-likefeedback.
Nass et al. (1994) stated that people tend to treat computers as social actors; they call this the CASA paradigm:
‘‘Computers Are Social Actors.’’ CASA studies demonstrate that the social rules and dynamics guiding human-human
interactionareappliedequallywelltoHCI.Manystudieshaveconfirmedpeople’stendencytoviewcomputersassocial
actors. Picard(2000)demonstrated thatparticipants whointeractedwithanemotion-supportagentplayed a computer
gamesignificantlylongerthanthosewhointeractedwithsimilaragentsthatignoredtheiremotions.Johnsonetal.(2004)
indicatedthathighlycomputer-literateparticipantswhoreceivedfeedbackthatincludedflatteryfromthecomputertended
totreatthecomputerinamannerresemblingpeople’sreactionstoflatteryfromotherpeople.
Userfrustrationwithcomputer-providedinformationandtechnologyisapervasiveproblemcausedbyfactorssuchas
thecrashingofcomputersandpooruserinterfaces(Lazaretal.,2005).Lazaretal.(2005)investigatedwhetherthereare
commonalitiesbetweenstudentandworkplaceuserfrustrationduringinteractionswithcomputers.Thestudyreportedthat
therearethreeimportantfactorswhichinfluencefrustrationlevels:timelost,timeneededtorepair,andtheimportanceof
thetask.Anotherimportantfindingofthestudywasthatwhentheparticipantswereaskedtowritedownthespecificcauses
oftheirfrustration,themostcitedcausewasthewaycomputererrormessageswerepresented.
Kleinetal.(2002)reportedthatusersintheirstudywhointeractedwithanaffect-supportagent(thisagenthelpsusersto
recoverfromfrustrationbyusingactivelistening,empathy,andsympathy)engagedwiththesystemsignificantlylonger
thanusersintheirtwocontrolstudies.Similarly,Hone(2006)reportedthattext-basedagentscanbeeffectiveinreducing
userfrustration.Inadditiontothisfinding,Honeexaminedtheeffectsofembodiedagents.Theresultsofthisinvestigation
revealedthatembodiedagentsthatprovideemotionalfeedbackcanbemoreeffectivethantext-basedagentsforreducing
userfrustrationcausedbythecomputer.
2434 M.Akgunetal./JournalofPragmatics42(2010)2430–2448
In an empirical study similar to the present one, De Laere et al. (1998) compared human-like versus machine-like
interactionstylesofcomputerinterfaces.Theydidnotobserveasignificantdifferencebetweenthedifferentstylesinterms
ofusers’self-appraisals.Thefindingsofthestudyalsodemonstratedthatnegativefeedbackaffectstheself-perceptionsof
theparticipantsdifferentlythanpositivefeedback.
Tzeng (2004) examined whether apologetic feedback affects users’ performance perception in the computerized
environment.Thisstudysuggestedthatusersmaynotexpectcomputerstobepolite,butapologeticstatementsmadethe
subjectsfeelbetterabouttheirinteractionswiththeprogram.
Thepresentedfindingsofsections3and4supporttheideathatapologiespreferredinHHImightbedifferentfromthose
inthemediumsofHCI.Therefore,resultsinHCIresearchexaminingcomputerapologiesmustbecomparedtoresultsinHHI
researchinordertodevelopmoreeffectivehuman-likecomputerinterfaces.Weshallcontextualizeourfindingsonthe
backgroundofthesestudiesbelowinsection6.
5. Method
5.1. Overviewofthepresentstudy
Thisstudyconsistsofthreephases(seeFig.1).Thefirstphaseofthisstudywasconductedtoidentifywhatkindsof
apologystrategysequences(APSSs)arepreferredbyusersduringtheirinteractionswithacomputer.OftheAPSSselicitedin
thefirstphase,twowereselectedforuseinthesecondphase(WenamedtheseAPSSsApology1andApology2).Thepurpose
ofthesecondphasewastoinvestigatewhetherthesetwomessagesareactuallyperceivedasapologies;ifso,theywouldbe
utilizedinthethirdphase.Thethirdphasewasconductedinaccordancewiththefollowingquestion:Dothetwoapologetic
errormessagesinfluenceusers’self-appraisalsofperformanceinthecomputerizedenvironment?
5.2. Phase1
5.2.1. Participants
Eighty-sixuniversitystudentsfromMETU(43maleand43female,aged20–24years)participatedinthefirstphaseofthe
study.Alloftheparticipantshadtakenthecourse‘‘EducationalTechnologyandMaterialDevelopment,’’onthebasisof
which online instructional materials were designed (see section 5.2.2). Among the participants, 16 were from Foreign
LanguageEducation,15werefromElementaryScienceEducation,18werefromElementaryMathematicsEducation,17
werefromEarlyChildhoodEducation,and20werefromComputerEducationandInstructionalTechnology.
5.2.2. Materials
5.2.2.1. The online instructional materials. In the procedural part, a web environment was used. In order to simulate a
human–computerinteractionenvironmentinwhichaproblemoccursbecauseofthecomputer’sinabilitytocarryoutatask,
aMacromediaFlash-basede-learningenvironmentwasdeveloped.Theenvironmentconsistedoftwoparts:Thelessonpart
(seeFig.2)andthetestpart(seeFig.3).Thesystemfirstpresentedthelesson,inwhichbriefinformationabouteducational
technologyandinstructionalmaterialdevelopmentwaspresented.ForexampleFig.2presentsthreeapproachestolearning,
namelybehaviorism,cognitivismandconstructivismandhowtheydefinelearning,thelearningprocess,andtheroleof
learning.Afterthelesson,thesystemactivatedamultiple-choicetestincludingsevenquestionsaboutthetopicpresentedin
thelesson.Forexample,inFig.3,atestitemaboutthecognitivistlearningprocessispresentedalongwiththealternatives.
Thetestmaterialincorporatedadeliberatedesignproblem,intendedtocreatefrustrationfortheusers(seesection5.2.3).
Fig.1.Threephasesofthestudy.
M.Akgunetal./JournalofPragmatics42(2010)2430–2448 2435
Fig.2.Thelessonpartoftheonlineinstructionalmaterials.
Fig.3.Thetestpartoftheonlineinstructionalmaterials.
5.2.2.2. DiscourseCompletionTest. IntheDiscourseCompletionTest(DCT)usedforthepresentstudy,theproblemwhichthe
participantshadencounteredduringtheirinteractionwiththecomputerwasretold.Theparticipantswereaskedtowrite
which kind of apology message they would prefer if they encountered the same problem again in a computerized
environment.Ourreasonforusingthismethodinthestudyisthatsimulatingtheproblemoccurringinacomputerized
environmentismorerealisticthanwritingascenariothatexplainstheproblem.
5.2.3. Procedure
Theinterfacewasdesignedsothattheuserswouldencounteranimportantproblemcausedbythecomputer’sinabilityto
carryoutatask.Inthisstudyareal-lifelikeproblemwascreated.SincespecialTurkishcharactersneedspecialhandlingin
someauthoringprograms,theappearanceofproblematicTurkishcharactersisafrequentlyobservedproblem.That’swhy
thiscasewasusedinthisstudy.
Theparticipantsdidnotknowthattheywouldencounteraproblemwhileinteractingwiththecomputerinterface.Theyfirst
completedthelessonpartofthee-learningmaterials,andthentheystartedtodothetestconsistingofsevenquestions.Afterthe
2ndquestion,theyencounteredaproblemandreceivedanerrormessagesimplysayingthattherewasaprobleminthesystem.
Theproblemhadtodowithcharacterencoding,i.e.,Turkishalphabeticalcharacters(e.g.,c¸[theletterfortheEnglishletter
combinationch],s¸[theletterfortheEnglishlettercombinationsh])werenotpresentedproperly(seeFig.4).Morespecifically,
theletters¸ appearedinanunintelligibleformsuchas=FE.Thisproblemwasnotfixedbutthesystemallowedtheusersto
continuetheirinteractionwiththecomputer.Attheendoftheinteraction,theDCTwasdistributedtotheparticipants.
5.2.4. Codingandfrequencyanalysis
ThestatementselicitedbytheDCTwereanalyzedaccordingtotheCSSARPcodingmanual(Blum-KulkaandOlshtain,
1984).Allthesentenceswereindependentlyanalyzedandcodedbytwoindividuals(onefromthefieldofHCI,andtheother
fromthefieldofPragmatics)andtheanalyseswerecompared.Incaseswheretherewasnoagreementbetweenthecoders,
2436 M.Akgunetal./JournalofPragmatics42(2010)2430–2448
Fig.4.Thetestpartoftheonlineinstructionalmaterialsaftertheproblem.
theyadjudicatedeitherbychoosingoneofthecodingsorbyjointlyproposinganewone.Thefollowingexampleshowshow
anapologywasanalyzed:
‘‘Unfortunately,becauseofthesystem,therehasbeenanerror.Restassuredthatthiserrorisnotgoingtoberepeatedagain.
We’resorry’’
becauseofthesystem ToR
isnotgoingtoberepeatedagain PoF
We’resorry IFID
TheAPSSoftheapologeticmessageisthereforeToR-PoF-IFID.
At the next step, the frequency of stated preference for each of the APSSs was calculated. In order to quantify the
percentageforeachAPSS,thefrequencyvalue wasdividedby thenumberrepresentingthenumberof individualswho
participatedinthefirstphaseofthestudy.Forexample,iftheToR-PoF-IFIDsequenceispreferredby20of80individuals,
thenthepercentageforthisAPSSiscalculatedbydividing20by80(20/80=.25).
5.2.5. Results
Table1showsthat17APSSswereelicitedfromthedataobtainedinphase1.Theresultsshowedthatingeneralgender
hadaneffectonthepreferencesforapologymessages.Forexample,theAPSSconsistingoftheIFIDandtheToRstrategies
Table1
TheAPSSsderivedfromphase1.
APSSs Gender(%) Total(%)
Male Female
1.IFID–ToR 16.28 23.26 19.77
2.IFID–ToR–OoR 23.26 9.30 16.28
3.ToR–OoR 16.28 6.98 11.63
4.IFID 4.65 13.95 9.30
5.ToR 13.95 4.65 9.30
6.IFID–ToR–EXP 6.98 4.65 5.81
7.IFID–OoR .00 11.63 5.81
8.OoR 4.65 4.65 4.65
9.IFID–EXP 2.33 6.98 4.65
10.ToR–EXP–OoR 2.33 4.65 3.49
11.IFID–EXP–OoR 2.33 2.33 2.33
12.IFID–ToR–EXP–OoR 2.33 .00 1.16
13.ToR–EXP 2.33 .00 1.16
14.EXP–OoR .00 2.33 1.16
16.IFID–ToR–PoF 2.33 .00 1.16
15.IFID–INTENSa .00 2.33 1.16
17.IFID–ToR–PoF–INTENS .00 2.33 1.16
a INTENSrepresentsanintensifiedapology,suchas,‘‘Iamverysorry,’’or‘‘Iamterriblysorry.’’
M.Akgunetal./JournalofPragmatics42(2010)2430–2448 2437
Table2
Theapologystrategies.
Strategy Percentage Total(%)
Male Female
IFID 60.47 76.74 68.60
Exp 18.60 20.93 19.77
ToR 86.05 55.81 70.93
OoR 51.16 41.86 46.51
PoF 2.33 2.33 2.33
(no.1)waspreferredby16.28%ofthemaleparticipants(Ms),andby23.26%ofthefemaleparticipants(Fs).Thisdifference,
however,wasnotsignificant,t(83)=(cid:1).861,p=.392.
TheonlysignificantdifferencefoundbetweenthemaleandfemaleparticipantsisfortheAPSSincludingtheIFIDandthe
OoRstrategies(no.7)(Ms=0%<Fs=11.63%,t(41)=(cid:1)2.354,p<.05).
Thepercentagevalueofstatedpreferenceforeachindividualapologystrategywasalsoanalyzed(seeTable2).We
foundthattheToRstrategywasthemostwidelypreferredapologystrategy(70.93%oftherespondentspreferreditin
theirresponses).ThesecondmostfrequentlypreferredstrategywastheIFID(68.60%oftherespondentspreferredthis
strategy in their responses). The third most frequently preferred strategy was the OoR, preferred by 46.51% of the
respondents.
The remaining strategies, namely the Exp and the PoF, were preferred by 19.77% and 2.33% of the participants
respectively.Inaddition,theresultsindicatedthat66.10%oftherespondentswhopreferredtheIFIDalsopreferredtheToR
strategy.Similarly,63.93%oftheparticipantswhopreferredtoreceivetheToRstrategyalsopreferredtheIFIDstrategy.
Intermsofthefrequencyofpreferenceforindividualstrategies,wefoundthatthepercentageofthemales’selectionof
the ToR apology strategy was much higher than that of the females (males=86.05%>females=55.81%), and that this
difference was significant, t(60)=4.392, p<.01. A similar result was found for the IFID strategy; 60.47% of the males
preferredtheIFIDstrategy,whereas76.74%ofthefemalespreferreditintheirresponses.Thisdifferencewasnotsignificant,
t(81,43)=(cid:1)1.605, p=.112. The percentage of the OoR apology strategy preferred by the males was 51.16%, while the
percentage of the strategy preferred by the females was 41.86%. This difference was not significant either, t(83)=.761,
p=.449.
TheresultsshowedthatthepercentagesoftheExpstrategypreferredbybothsexesinthestudywerealmostidentical
(males=18.60%<females=20.93%).ThepercentagesofthePoFapologystrategypreferredbybothsexesinthestudywere
alsoidentical(males=2.33%=females=2.33%).
ThesefindingsareconsistentwiththoseofHatipoğlu(2003)inthatmostofthepreferredapologiesbyparticipantswho
werenativesoftheTurkishculturecontaintheToRandtheIFIDstrategies.Moreover,thefindingsofthisstudyarealso
consistent with those of Olshtain (1989), and Vollmer and Olshtain (1989) in that the IFID and the ToR were mainly
preferredinallsituations.ThepresentstudyalsoindicatedthatthePoFwastheleastpreferredstrategybyusersintheHCI
context.Similarly,Hatipoğlu(2003)indicatedthatitwastheleastpreferredstrategyinTurkishculture.Overall,itseems
clearthattheIFIDandtheToRstrategiesaremainlypreferredbothinHHIandHCI.ThisfindingalsosupportsCohenand
Olshtain’ssuggestionthattheIFIDandtheToRcanbeusedinallsituations,whereastheotherthreestrategiesaresituation
specific.
ItcanbeinferredthattheapologiesthatoneexpectstogetfromacomputeraresimilartotheapologiesutilizedinHHI.
Therefore,itcanbesaidthatthefindingsofthisstudysupportNassetal.’s(1994)argumentsthatpeopletendtotreat
computersassocialactors,andthatthesocialdynamicsforHHIcanbeappliedequallywelltoHCI.Itisthereforepossibleto
saythatcomputerinteractionsmaytriggerusers’socialschemathatarerelevanttosituationsinwhichtheuseofapologiesis
normallythoughttoberequired.
5.2.6. TherationaletochoosetwoAPSSstobeusedinthethirdphase
InordertodeterminetwoAPSSsforthesecondphase,thefollowingcriteriawereconsidered.
1. AccordingtoOlshtain(1989),IFIDsaredirectapologiesandtheycanbeusedinanysituationforeverylanguage.
Therefore,APSSsshouldbeselectedamongthoseincludingtheIFIDstrategysothatparticipantscouldeasily
perceivethemessagesasapologies.
2. The APSSs to be chosen should be preferred by more than one individual. Stated differently, the percentage
valuesoftheAPSSsshouldbemorethan1.16.
3. ThetwoAPSSstobechosenshouldbemaximallydifferentintermsofthestrategiestheyinclude.Inotherwords,
theAPSSsshouldnotconsistofthesameapologystrategiesexcepttheIFID.
4. Whenthethreecriteriawereconsideredtogether,itwasdeterminedthattwosetsofAPSSswerereachedin
ordertoutilizeinthesecondphase.ThefirstsetofAPSSsincludedIFID–ToRandIFID–Exp–OoR,whilethesecond
setconsistedofIFID–ToR–OoRandIFID–Exp.
2438 M.Akgunetal./JournalofPragmatics42(2010)2430–2448
5.3. Phase2
5.3.1. Participants
Thirty-twostudentswhodidnotparticipateinphaseI(17maleand15female,agedfrom20to24years)participatedin
the second phase of the study. All of the participants were chosen from a pool of students who had taken the course
‘‘EducationalTechnologyandMaterialDevelopment.’’
5.3.2. Apologyperceptionrating
Aratingscalewasdesignedtoseewhethertheparticipantsperceivedeitherofthetwomessagesasapologies.Inthistest,
thetwoapologymessagesandoneplainmessageformulatedbytheresearcherswerepresentedtoadifferentgroupof
participantswhowereaskedtojudgethevalueofthemessagesintermsoftheireffectivenessasapologies.Table3showsthe
selectedapologymessagesandthestrategiestheycontain.
5.3.3. Procedure
The procedure of the second phase was basically the same as that of the first phase. Additionally, the rating scale
explainedinsection5.3.2wasemployedinordertoinvestigatewhetherthemessageslistedinTable3wereperceivedas
apologiesornot.
5.3.4. Analysis
Todeterminewhetherthereisasignificantdifferencebetweentheapologymessagesandtheplaincomputermessage,a
repeatedANOVAanalysiswasconducted.Intheanalysis,threedifferentmessageswereconsideredasiftheywerethree
differentconditionsinwhichthesameparticipantswereinvolved.
5.3.5. Results
The repeatedANOVAresults indicatedthat thereis a significantdifference amongthemessages intermsof apology
perception,F(2.62)=70.541,p<.05.Whenweinvestigatedthepairedcomparisons,thetwoapologymessages(Mapology
1=4.750 and M apology 2=4.125) were found to be significantly different from the plain computer message
(Mplain=1.594),buttherewasnosignificantdifferencebetweenthetwoapologymessages(seeTable4).Theseresults
indicatedthattheplaincomputermessagewasnotperceivedasanapologymessage,whereasthetwoapologymessages
wereperceivedasapologies.
5.4. Phase3
In the third phase, answers to the main research question of the study were sought (i.e., what are the effect of the
participants’moodstatesandtheeffectoftheapologymessagetypeontheirself-appraisalsofperformance?).
5.4.1. Participants
Twohundredandtwentyuniversitystudentswereinvolvedinthethirdphaseofthestudy.Fortyofthemdroppedout
(12ofthemwereeliminatedbecauseofthemissingdataand28ofthemwereignoredbecausetheirresponsetimeswere
outsideoftheconfidenceintervaldesignatedforanacceptableresponsetime).Thereasontosetanacceptableresponse
timewas to eliminatetheparticipants whoresponded veryquicklyor veryslowly. Of theremaining 180students, 105
Table3
APSSsandmessages.
Messagegroup Apologystrategysequence Message(translatedintoEnglish)
Apology1 IFID–ToR–OoR Theproblemcausedbythesystemcouldnotbefixed.Weapologizeforthis.
Anynegativeeffectsoftheproblemonyourperformancewillbecompensated
duringtheevaluationofthetest.
Apology2 IFID–EXP Turkishcharacterscouldnotbeprintedappropriatelyduetoanerror.
Weapologizeforthis.
Controlmessage (Plainmessagewithnoapologyinit) Theproblemcouldnotbefixed.
Table4
Apologymessagecomparisons.
Meandifference Std.error Sig.
Comparisons
Apology1–Apology2 .625 .276 .092
Apology1–plainmessage 3.156 .330 .000
Apology2–plainmessage 2.531 .229 .000
M.Akgunetal./JournalofPragmatics42(2010)2430–2448 2439
werefemaleand75weremale.Theparticipants’agerangedfrom20to24years.Alloftheparticipantshadtakenthecourse
‘‘EducationalTechnologiesandMaterialDevelopment.’’Amongtheparticipants,21werefromForeignLanguageEducation,
29 were from Elementary Science Education, 36 were from Elementary Mathematics Education, 32 were from Early
ChildhoodEducation,and62werefromComputerEducationandInstructionalTechnology.
5.4.2. Materials
5.4.2.1. Mood questionnaire. Watson and Tellegen (1985) indicated thestability and robustness of positive and negative
affect in self-reports. Depending upon this finding, classifying participants into positive and negative affect groups was
chosenasawaytounderstandtheroleofmoodstateinstudents’self-appraisalsofperformance.Wechoseaquestionnaire
capableofrevealingpositiveandnegativeaffect.ThequestionnaireselectedwastheoneusedinEfklidesandPetkaki’sstudy
(2005), which includes 10 adjectives and comprises two main factors:one for negative affect (sad, melancholic, anxious,
pessimistic,anddisappointed)andoneforpositiveaffect(good,calm,happy,excited,andpleased).Inourstudy,anexplanatory
factoranalysiswasconductedtorevealtheunderlyingfactors.OurfindingswerethesameasthoseinEfklidesandPetkaki’s
study(2005).Accordingtoourresults,twofactorswerefound,positiveandnegativeaffects.Good,calm,happy,excited,and
pleasedloadedonpositiveaffect.Sad,melancholic,anxious,pessimistic,anddisappointedloadedonnegativeaffect.Thescoring
ofthequestionnairewasconductedseparatelyforthenegativeandpositivemoods.Ifaparticipant’spositivemoodwas
greater than his/her negative mood, s/he was considered to be in positive mood. For the reverse situation, s/he was
consideredtobeinnegativemoodbeforethetask.
5.4.2.2. Theonlineinstructionalmaterials. Inphase3,onlineinstructionalmaterialssimilartothoseusedinthefirstphase
were utilized. Recall that these materials consisted of two parts: a lesson part, in which the brief information about
educational technology and material development was presented, and a test part, which included seven questions. The
differenceinthematerialsusedinthethirdphasewasthatthisincludedthemoodstatequestionnairepresentedbeforethe
lesson,andtherewere14questionsinthistestinsteadofsevenquestions.Thirdly,theonlineenvironmenthadadatabase
connection that allowed the researchers to record the collected data in the database. In other words, the participants’
answerstothequestionsandtheirresponsestothemoodquestionnairewererecorded.Inaddition,theamountoftime
spentforeachquestionbyeachparticipantwasrecordedinordertoeliminatethoseparticipantswhoseresponsetimeswere
outsideoftheacceptableconfidenceinterval.
5.4.2.3. Test questions. Multiple-choice questions were utilized in the test part of this phase because this type of
questionnaireincreasesobjectivityanddecreasestherequiredefforttobeexertedbytheresearchersduringtheevaluation
oftheresponses.Sevendifferentpairsofmultiple-choicequestions(14questionsintotal)werepreparedbyanexpertofthe
subject.Ineachpair,thequestionshadthesameeducationalobjective,andtheywereapproximatelyofthesamelevelof
difficulty.Thenthequestionsineachpairwereseparatedandre-pairedinadifferentsetofmatchesinordertoformtwosets
ofquestions.Afterpreparingthetwosets,thequestionswerecheckedbytwoothersubject-matterexpertsintermsofthe
questions’ difficulty level, intended educational objectives, and correctness of the questions. Based on these experts’
suggestions,correctionsweremadetofinalizethequestionsforuseinthestudy.
5.4.3. Experimentaldesign
Thereweretwoexperimentalgroupsandonecontrolgroup.TheexperimentalgroupswereApology1groupandApology
2group.TheparticipantsreceivederrormessagesincludingtheapologeticstatementsshowninTable3.Theparticipants
wererandomlyassignedtothegroupsbythesystem.TheoutcomeoftherandomassignmentislistedinTable5.
5.4.4. Procedure
Intheproceduralpartofthestudy,awebenvironmentwasused.Afterthelog-onprocedure,thequestionnaireabout
moodstateswaspresented.Aftertheparticipantshadfilledinthequestionnaire,thesystempresentedtheshortlesson
about educational technologies. After completing the lesson, the participants took an exam related to the lesson. The
participantswereinstructedtoanswereachquestionbychoosingoneoutoffivealternatives,whereonewasalwayscorrect.
Table5
Randomassignmentoftheparticipants.
Apology1group Apology2group Controlgroup
Gender
Female 40 36 29
Male 26 21 28
Moodstate
Positiveaffect 41 34 28
Negativeaffect 25 23 29
Total 66 57 57