Bodker & Andersen

50 Pages·2005·0.95 MB·English

HUMAN–COMPUTER INTERACTION, 2005, Volume 20, pp.353–402
Copyright © 2005, Lawrence Erlbaum Associates, Inc.
Complex Mediation
Susanne Bødker and Peter Bøgh Andersen
University of Aarhus
ABSTRACT
Thisarticlehasitsstartingpointinalargenumberofempiricalfindingsregard-
ingcomputer-mediatedwork.Theseempiricalfindingshavechallengedourun-
derstandingoftheroleofmediationinsuchwork;ontheonehandasanaspectof
communicationandcooperationatworkandontheotherhandasanaspectof
humanengagementwithinstrumentsofwork.Onthebasisofpreviousworkin
activity-theoretical and semiotic human–computer interaction, we propose a
modeltoencompassbothoftheseaspects.Inadialoguewithourempiricalfind-
ingswemoveontoproposeanumberoftypesofmediationthathavehelpedto
enrichourunderstandingofmediatedworkandthedesignofcomputermedia-
tion for such work.
1. INTRODUCTION
Wehavebothworkedformanyyearstoprovideatheoreticalbasisforhu-
man–computerinteraction(HCI):oneofuswithsemiotics,theotherwithac-
tivitytheory.Wehavefoundthatthenotionofmediationisusefulfrombothof
Susanne Bødker is a computer scientist with an interest in HCI and activity
theory;sheisaprofessorintheDepartmentofComputerScienceatUniversity
of Aarhus. Peter Bøgh Andersen is a linguist with an interest in HCI and
semiotics; he is a professor in the Department of Information and Media
Studies at University of Aarhus.
354 BØDKER AND ANDERSEN
CONTENTS
1. INTRODUCTION
2. BASIC CONCEPTS
3. THE MODEL
3.1. Mediation
3.2. Slots and Fillers
3.3. Instrumental and Semiotic Actions
3.4. Expectations
Functional Categories
Meaningfulness
Morphing
Mixed Forms
4. CO-OCCURRING MEDIATORS
4.1. Juxtaposition of Mediators
4.2. Language as Mediator
5. DIFFERENT KINDS OF OBJECTS AND THE DIVISION OF LABOR
4.1. Metonymical Mediation
4.2. The Division of Labor
6. LEVELS
7. CHAINS
8. MULTI-MEDIATION
9. CONCLUSION AND FUTURE RESEARCH
9.1. Design Methods
9.2. Relations Between Instrumental and Semiotic Processes
9.3. Developmental Issues
these theoretical perspectives in understanding HCI (computer-mediated
work).Inrecentyears,ithasbecomemoreandmoreclear,however,thatthe
concept of mediation needs to be nuanced and extended to deal with the
complexworldsofmultiplemediationthatsurroundhumanbeings.Ourob-
servationssuggestthatreallifemediationisheterogeneous(comprisingnotonly
traditionalcontrolsanddisplays,aswellaspicturesandtexts,butalsoconver-
sations and entire activities); dynamic (mediators change through time); and
consistsofwebsofmediators,eitherusedsimultaneously,connectedinchains,or
organizedinlevels,eitherduetodifferentlevelsofautomationortodifferent
purposes of the activity. In addition, most activities involve instrumental
(tools, machinery) as well as semiotic mediation (displays, conversation). In
thisarticleweapproachamoredetailedunderstandingofhowwemaycon-
ceptualizesuchmultitudesofmediators,purposes,andactors.Throughthis,
we provide a richer set of concepts to understand mediation, based on
semiotics and activity theory.
COMPLEX MEDIATION 355
Weparticularlyfocusontheinterplaybetweeninstrumentalandsemiotic
activities.Svanæs(n.d.)recentlyclaimedthatHCIhaslackedintegrationof
communicationandphysicaldoing,ofsymbolicandnonsymbolicintegration.
Thisisevenwhenconsideringthemostrecentpostcognitivisticframeworks
(activitytheory,computersemiotics)andsuggestionsbySuchman(1987)and
WinogradandFlores(1986),havelackedintegrationofcommunicationand
physicaldoing,ofsymbolicandnonsymbolicinteraction.Althoughweonly
partiallyagreewithSvanæs,wedothinkthereisaproblem.Theproblemisnot
oneofdata,becausethereareseveralworkstudiesthatinfactdoprovideac-
countsofinstrumental,communicative,andcognitiveaspectsofwork(agood
exampleisHutchins,1996).Theprobleminsteadconsistsinintegratingthese
aspectsinaconsistentmodelthatmaintainssecurelifelinestoitscomponent
disciplinesandthusretainsaccesstotheirempiricalmethodsandresults.Inthis
articlewecontributetoasolutiontotheproblembyofferingasimplemodel
thatcombinesessentialconceptsfromactivitytheoryandsemiotics.
Thisarticlereflectsadiscussionthatwehavehadduringanextendedtime
period,basedonourempiricalstudiesofcomputer-mediatedworkinthejoint
Center for Human–Machine Interaction. Hence, our empirical basis comes
fromtwoprojectsinthiscenter:(a)thestudyofshipbridges(e.g.,Andersen,
2001a,2001b,2003;Andersen,Carstensen,&Nielsen,2002;Petersen,2002;
reportsinhttp://www.imv.au.dk/~pba/ElasticSystems)and(b)thestudyof
wastewaterplants(Bertelsen&Bødker,2001,2002;Bertelsen&Nielsen,1999;
Buur&Bødker,2000;Nielsen&Søndergaard,2000).Inthemaritimestudy,we
visitedanumberofshipsandmadeanin-depthexaminationofonelargecon-
tainership,basedon60hrofvideo.Inaddition,webuiltacoupleofprototypes
totestideasinspiredbythefieldwork.Thewastewatertreatmentplantresearch
includedworkplacestudies.Init,weappliedaninterventionistapproachthat
includedtheconstructionofprototypesfornewcomputersupportfortherun-
ningandoptimizationoftheplant.
Inthisarticle,theempiricalmaterialisthebeginningandtheend.Ourthe-
oreticalmodelswereformedandrefinedonthebasisofourempiricalstudies.
The models are not ends in themselves; instead, they are the means with
which we keep scrutinizing the empirical material, hoping that the models
willbreak.Itisinterestingthat,givenourdifferentbackgrounds,wehaveal-
wayssharedthisPopperianwayofusingtheoriesandmodels(Popper,1982)
and we have seen no reason to abandon it here.
Thisarticleisstructuredasfollows.Itsmaintheoreticalpurposeistofinda
wayofunderstandinganddescribinginstrumentalandcommunicativemedi-
ation of human action. This purpose is motivated in our fieldwork, where
bothtypesofmediationareprofoundlyintertwined.InSection2,wepresent
ourtheoreticalbackground:semioticsandactivitytheory.Themainpointis
thatonetheorycanofferwhattheotherlacksandthatacombinationthere-
fore is useful. In Section 3, we outline our model on the basis of the shared
356 BØDKER AND ANDERSEN
conceptofmediation.Intherestofthearticleweaddresstheimplicationof
thismodelfor(computer)mediatedwork.Section4dealswithcasesinwhich
multiple mediators (tools, controls, displays, communication) are used, par-
ticularlyincooperativearrangements.InSection5,we presenttheconceptof
metonymical mediation, in which the work object immediately amenable for
manipulationisonlyapartoftheobjectonereallywantstochange.Section6
deals with levels of mediation. We show how the focus of work regularly
changeslevel.InSection7,wetryoutthemodelintheveryfrequentcasesin
whichmediatorsarecombinedinchainsand,inSection8,wesummarizethe
preceding by suggesting that one should design for multi-mediation, that is,
recognize that more than one mediator normally is used in an activity, and
that individual mediators should be designed as a part of an assemblage of
mediators, either co-occurring, organized in levels, or connected in chains.
It is important to point out what we do not do.
Wedonotofferanewtheoreticalsynthesisoftwocomplicatedtheories.In-
stead, we offer a simple, systematical, and sound way of combining the in-
sightsofbothtraditionsinfieldworkanddesign.Ourmanyauthenticexam-
ples are intended to demonstrate the usefulness of the combination. The
proofofthepuddingis,sotospeak,intheeating,andthisiswhywealternate
between describing examples and refining the theoretical model.
Wedonotgivedetaileddesignguidelinesbut,onthebasisofempiricalfind-
ingsandtheory,wedoargueforadesignstrategythatisclearlycontroversialin
sometraditions.Forexample,theconceptofleveleddesigncallsDonaldNor-
man’sideaofinvisiblecomputersintoquestion(Norman1998;seealsoBergman
2000,p.10ff.),towhichwereturnattheconclusionofthisarticle.
2. BASIC CONCEPTS
Asimplereasonfortryingtocombineactivitytheoryandsemioticsisthat
instrumentalandsemioticactionsarenormallyintertwinedinworkactivities.
Philosophyhasbeenawareofthisforalongtime(see,e.g.,Svanæs,n.d.),and
Hutchins (1996), and others have discussed the consequences of the
intertwinement in HCI. Because activity-theoretical HCI has traditionally
beenconcernedwiththeformerandsemioticswiththelatter,wewilltryto
combine the approaches and see what happens.
Semioticsisthesciencethatstudiessignsandtheirfunctionanduseinsoci-
etyandasignissimplyanythingweusetostandforsomethingelse.Wecan
distinguish between two main traditions: (a) the American (C. S. Peirce,
UnitedStates,1839–1914)and(b)theEuropean(FerdinanddeSaussure,Swit-
zerland,1857–1913).Whatunitesthetwotraditionsistheconceptofsigns,but
whereastheEuropeantraditionemphasizessignsasformingasocialsystem,
the American tradition focuses on sign usage. Saussurean theory is a struc-
COMPLEX MEDIATION 357
tural theory, whereas Peirce saw processes (called semiosis: the process of
meaning-making)asthebasicsignphenomenon.Becauseofthisdiversity,the
choice of a particular semiotic framework requires some thought.
InconnectionwithHCI,semioticsandcommunicationtheoryhasmainly
beenusedforinterfacedesign(Nadin1988,Souza1993),informationsystem
analysis and design, and computer art (Nake 1994). In information system
analysisanddesigntherearetwomaintraditionsorganizedinseparatecon-
ferences: (a) the language action perspective (LAP), based on Winograd and
Flores(1986),and(b)theorganizationalsemiotics(OS)community,foundedby
Ronald Stamper (see, e.g., Stamper 1996, Liu 2000, and the Kluwer series
Studies in Organisational Semiotics[Liu, 2004]).
Themaincharacteristicoftheseapproachesistheiremphasisoncommu-
nicationandmeaning.Althoughtheydealwiththetechnicalaspectsoftech-
nology,theyfocusonthewayusersinterpretandusetechnology.However,a
mainprobleminbothLAPandorganizationalsemioticsisthattheylackcon-
ceptsformaterialprocesses.Thisproblemhasrecentlybeenaddressedinthe
LAPcommunity(Goldkuhl,2001)andhasresultedinanewseriesofconfer-
encescalled“ActioninLanguage,Organizations,andInformationSystems.”
Thus, there are many different variants to consider when deciding on a
semiotic framework. However, one condition is that the framework must
have a place for nonsemiotic, instrumental phenomena.
TheSaussureantraditionisolatedlanguageasaseparatestudyobject;ac-
cordingly, it is not immediately useful for us. However, the influential con-
temporaryschooloflinguisticsandsemiotics,calledsystemicfunctionallinguis-
tics, founded by M. A. K. Halliday (e.g., Halliday 1985, 1994), and its more
generaloffspring,socialsemiotics(e.g.Kress&vanLeeuwen1996),presentsa
promisingsolution.Itretainsmanyoftheinsightsofclassicallinguisticshav-
ingtodowiththefactthatlanguageisprimarilyasocialentityandmustmain-
tainacertainautonomy,standardization,andsystematicity,ifitsspeakersare
tomakethemselvesunderstood(thesystemicpart),butitaddsabodyoftheory
abouttherelationbetweenutterancesandtheirdifferentcontextsofuse(the
functionalpart).TheHallidaytraditionis,however,asdifficulttolearnaspure
linguistics,andthereforeitisnotsuitedforotherprofessions.Accordingly,we
have chosen to use it as background knowledge in the following.
The Peircean framework has the advantage that it incorporates language
usage into its foundation (the well-known distinction among syntax, seman-
tics,andpragmaticsisinspiredbyPeirce).Moreover,itisbasedonaverysim-
plefigureofthoughtthatisreusedinmanyconnections,itisrelativelyeasyto
learn and use, and may be able to function as a boundary object (Star &
Greisemer1989)betweendifferentprofessions.AlthoughthePeirceantradi-
tionlacksaprecisevocabularytodescribethedetailedfeaturesoflanguage,
webasethefollowingonthePeirceantradition,addingtoitbyusinginsights
from the Halliday tradition.
358 BØDKER AND ANDERSEN
Figure 1. The semiotic triangle.
ThebasicconceptinPeirceansemioticsisthesign;1aRepresentationstands
foranObjectunderacertainInterpretation:“Thephoto(Representation)isPe-
ter(Object)asayoungman(Interpretation;seeFigure1).”TheInterpretation
inPeirceansemioticsmediatesbetweenRepresentationandObject:TheInter-
pretationisthementalorbehavioralreactioncalledforthbythesignoritisthe
habit(rule)accordingtowhichtheRepresentationcanbesaidtorepresentits
Object.TheideaisthatweseldomhaveasimpledenotationwhereaRepresen-
tationdenotestheObject.Wecannothelpreferringtoobjectsunderacertain
aspect,expressingacertainpointofview,orreferringtoacertainruleorhabit
byvirtueofwhichthedenotationoccurs.Thus,although(a)“PersonAboughta
carfromPersonB”and(b)“PersonBsoldacartoPersonAmayrefertoexactly
thesameevent,(a)takestheperspectiveofthecustomer,assignstheinitiativeto
him,andevokesadifferentcommercialpatternofbehaviorthan(b).Thisisthe
senseinwhichtheInterpretationmediatesbetweenRepresentationandOb-
ject:Itplacesthesimpledenotationinalargercontextofviewpointsandhabits.
ThedashedlineinFigure1symbolizethattherelationbetweenRepresentation
andObjectisnotdirectbutmediatedbytheInterpretation.
Activitytheory,however,originateshistoricallyfromadialecticalmaterialist
psychology developed by Vygotsky and his students in the Soviet Union in
thebeginningofthe20thcentury.Activitytheoryrejectstheisolatedhuman
beingasanadequateunitofanalysis,insistingonculturalandtechnicalmedi-
ation of human activity. Accordingly, the unit of analysis includes technical
artifacts and cultural organization, and the focus of activity theory is much
1. Peircedistinguishedamongthreeaspectsofasign:(a)representamen,(b)object,
and(c)interpretatant.Weusethemorecommontermsrepresentation,object,andinterpre-
tationfor ease of reading. The common terms are used in Peirce’s sense.
COMPLEX MEDIATION 359
widerthanwhathasbeenthecoreconcernofpastHCIresearch(Bertelsen&
Bødker, 2001).
AccordingtoVygotsky(1962),humanactivityhasthreefundamentalchar-
acteristics:First,itisdirectedtowardamaterialoridealObject;second,itis
mediated by artifacts; and third, it is socially constituted within a culture.
Leontjev(1978,1981)broughtthisthinkingintothecontextofacommunity
by focusing on human joint activity.
According to Leontjev (1978), human activity can be analyzed into a
three-levelhierarchyofactivity,action,andoperation,eachofwhichreflectsthe
objective world. Activity is directed to satisfy a need through a material or
idealObject.TheSubject’sreflectionof(includingexpectationsto)thisOb-
jectisthemotiveoftheactivity.Humanactivityiscarriedoutthroughactions,
realizingobjectiveresults.Theseactionsaregovernedbytheconsciousgoals
oftheSubject.Goalsreflecttheobjectiveresultsofaction.Actionsarerealized
throughseriesofoperations,eachtriggeredbytheconditionsandstructureof
theaction.Theyareperformedwithoutconsciousthinkingbutareorientedin
the world on a nonconscious orienting basis.
Thethreelevelsofactivityarenotfixed;anactioncanbecomeanopera-
tionthroughautomation–internalizationandanoperationcanbecomeanac-
tion through conceptualization in breakdown situations (Bødker, 1991). A
separatelymotivated activityinonecontextcanbeanoperation inanother
activity. The focus in activity theory, in short, is ondevelopment.
This is also true of Peirce’s theory. One aspect that needs to concern us
here is Peirce’s observation that the Interpretation is regularly turned into a
newRepresentation,Interpretation →Representation withthesameObject.
i i+1
Thisiswhathappensduringconversations:Ireacttowhatissaidandusethis
reactiontoproducemynextlineandmyconversationpartnerdoesthesame.
Themodel,whichhasbeencalledunlimitedsemiosis,isusedinSection4.2to
account for conversations on a ship bridge.
Changes between actions and operations imply a focus shift (Bødker,
1991). When we write an article on a computer, we are normally focused
on the article, not the keys on the keyboard. We interpret our activity as “I
am writing an article by means of my computer.” However, if one of the
keys gets stuck, our attention shifts to the faulty key and the typing opera-
tion. What was an operation before now becomes an activity: What is
wrong with the key? Focus shifts are reflected in our Interpretation: When
askedwhatwearedoing,wecouldanswer“Iamfixingthekeytocontinue
working on my article.” The change is reflected in the syntactical structure
of the verbal interpretations: In the former case, the article-writing is de-
scribed in the main sentence, the work Object is the grammatical object,
and the Mediator is expressed in a subordinate instrumental adverbial (“by
means of my computer”), whereas in the latter case the situation is re-
360 BØDKER AND ANDERSEN
versed:Herethekey-fixingisdescribedinthemainsentenceandthepaper
writing is subordinated (“ to continue working on my article”). Physically
similar processes are taking place (pressing keys) but the relationship be-
tween tool and work Object is different and this difference shows up in our
verbal Interpretation. In semiotic terms, there is a different Interpretation
relatingMediatorandworkObject.Weelaborateonthisinthenextsection
but let us get on with activity theory here.
Humanactivityisconstantlydevelopingasaresultofcontradictionsand
instabilityandbecauseoftheconstructionofnewneeds.Inactivitytheoryitis
understoodthathumanbeingsdialecticallyre-createtheirownenvironment.
Thehistoricaldevelopmentofactivityisadevelopmentofartifactsanden-
vironments.Modesofactingwithinanactivitysystemarehistoricallycrystal-
lizedintoartifacts;inthissense,thehistoricaldevelopmentofactivitycanbe
readofffromthedevelopmentofartifactsmediatingthepraxis(Bærentsen,
1989; Bannon & Bødker, 1991).
Inactivitytheory,asithasbeenintroducedinHCI(byBødkerandothers),
theparadigmcaseisamaterialsituationinwhichaSubjectappliesaMedia-
tor-tooltoanObjecttochangeit(seeFigure2):“Thecarpenter(Subject)hit
thenail(Object)withahammer(Instrument).”TheworkObjectalwayshas
twosides:(a)theMaterialand(b)theOutcome.ItistheshapingoftheOut-
come from the Material that instrumental mediation is about: Material—“I
mashed the potatoes”; Outcome—“I baked a potato cake.”
InparticularthroughtheworkofEngeström(1987),thisinstrumentalme-
diation has been embedded in other mediations where, for example, lan-
guage plays a role as a Mediator between the Subject and a community of
practiceorbetweentwoSubjects.ArtifactsmediatenotonlybetweenSubject
andObjectbutalsobetweenSubjects,likelanguagedoes.Forexample,the
maritimemachinetelegraphmediatesnotonlybetweenthecaptainandthe
speedoftheshipbutalsobetweenthecaptainandthechiefengineerbyas-
signingrolestothem:Asachiefengineeroncesaid,theengineerprovidesthe
engine power, and the captain spends it.
However,aspointedoutby,forexample,Wells(2002),itwasnotuntilre-
centlythatspokenandwrittendiscoursebegantofigureasmediatorsinactiv-
ity-theoreticalanalysesofwork,andtheworktoexploretheirroleasmedia-
tors has been limited. Activity theory makes rather unclear distinctions
betweentheroleofinstrumentalmediationandthatofcommunicativemedi-
ation(exceptasanalyticalperspectivesdeterminedbyidentifyingwhetherwe
aredealingwithaSubject–ObjectrelationshiporaSubject–Subjectone).In
activity theory, according to Wells, we need a more fine-grained analysis of
human–humanmediationthanthatindicatedbyEngeström’s(1987)general
human–mediator–community triangle, in which the Mediator is “language”
or rules/procedures (see Figure 3).
COMPLEX MEDIATION 361
Figure 2. The activity-theoretic triangle.
Figure 3. Rules and language as mediators between Subject and community.
Still,incontrasttotheHCI-orientedwork,activitytheoryitselftookinter-
estinmeaningandsemiosisfromitsverybeginning,asthefollowingquota-
tion illustrates:
Thetool’sfunctionistoserveastheconductorofhumaninfluenceontheobject
of activity; it is externally oriented; it must lead to changes in objects. It is a
meansbywhichhumanexternalactivityisaimedatmasteringandtriumphing
over,nature.Thesign,however,changesnothingintheobjectofapsychologi-
caloperation.Itisameansofinternalactivityaimedatmasteringoneself;the
sign is internally oriented. (Vygotsky, 1978, p. 55)
Inthisarticlewepresentamorefinelygrainedtakeonthissuggestion.Our
datacontradictthesharpdistinctionbetweentoolsandsignssuggestedinthe
quotation;instead,puretoolsandsigns,cleaninstrumentalandsemioticbe-
havior,areonlytheoreticalendpointsofascale.Realtoolsandsignsarelo-
cated not at these endpoints but somewhere in between.
362 BØDKER AND ANDERSEN
Insum,onecansaythatboththeoriessharetheconceptofmediation,the
idea that humans always put something else between themselves and their
Objectofwork.Inactivitytheory,theparadigmisoftenmaterialmediation,
wheretoolsormachineryareplacedbetweentheSubjectanditsObjectbutit
canalsobesignsreferringtotheObjector,inabroadersense,normsandreg-
ulations,culture,ortheprevailingdivisionoflabor.InPeirceansemiotics,the
mediator between tool–Representation and Object is conceptual or behav-
ioralandreferstoageneralpattern:apointofview,acodeofbehaviororan
intention (which can be taken as a loose rule that partially allows us to guess
future behavior).
What exactly all this means is the topic of the rest of this article.
3. THE MODEL
Inthissection,wesuggestaprincipledwayofcouplingactivitytheoryand
semiotics.Wetakeadvantageofthefactthatmediationplaysacrucialrolein
both theories.
3.1. Mediation
Wehaveseenthatthesameword,mediation,isusedinbothsemioticsand
activity theory but it does not follow that they mean the same. One simple
suggestionistoclaimthatwhereastoolsmediatebetweenSubjectsandmate-
rial Objects, signs mediate between Subjects (Bødker, 1991; Wells, 2002).
However, communication is much more indirect than instrumental actions:
Youcanforcethenailintothewoodbutyoucannotsimilarlyforceanopinion
into your conversation partner; a change of opinion is a choice the listener
makes on his or her own (Wells, 2002).
Conversation is more like a kind of coconstruction in which the partici-
pantstaketurnstoconstructasharedtext.Moreover,thesemioticObjectof
conversationisnottheotherparticipantbutratherthetopicoftheconversa-
tion (but of course the other participant may incidentally happen to be the
topic).Thisperspectiveisnotuniquetosemioticsbutissharedwithotherap-
proaches, such as interaction analysis.
Aspeakercan,ofcourse,makecleartothelistenerthatheintendshimto
believeordosomethingbut,intheend,theoutcomesolelydependsonthe
goodwillofthelistener(Posner,1993).Whereasinstrumentalactivitiescanbe
individualorshared,dependingonthecircumstances,communicativeactivi-
tiesareconstitutedassocialphenomenaandaredestroyediftheparticipants
fail to respond.
Thebasicmappingwehereattempttomakebetweenactivitytheoryand
semioticsisverysimple:WeidentifytheMediatorofactivitytheorywiththe