Learning Contextual Inquiry and Distributed Cognition

19 Pages·2015·1.12 MB·English

by  Erik Berndt

CognTechWork(2015)17:431–449
DOI10.1007/s10111-014-0314-y
ORIGINAL ARTICLE
Learning Contextual Inquiry and Distributed Cognition: a case
study on technology use in anaesthesia
Erik Berndt • Dominic Furniss • Ann Blandford
Received:11April2014/Accepted:21October2014/Publishedonline:6November2014
(cid:2)TheAuthor(s)2014.ThisarticleispublishedwithopenaccessatSpringerlink.com
Abstract There have been few studies on how analysts 1 Introduction
learn or use frameworks to support gathering and analysis
offielddata.DistributedCognitionforTeamwork(DiCoT) Tounderstandthecomplexityofinteractivetechnologyuse
is a framework that has been developed to facilitate the inpractice,fieldstudiesareessential.However,enteringan
learning of Distributed Cognition (DCog), focusing on unfamiliar,complexenvironmentcanbeoverwhelmingfor
analysingsmall team interactions. DiCoT,inturn, exploits an analyst without relevant background knowledge and
representations from Contextual Inquiry (CI). The present experience (Wong and Blandford 2003). The analyst may
study is a reflective account of the experience of learning experience difficulties in seeing a clearly defined structure
first CI and then DiCoT for studying the use of infusion in these systems (Norman 2011) and may direct their
devices in operating theatres. We report on how each attention to unimportant aspects of the environment while
framework supported a novice analyst (the first author) in neglecting essential ones.
structuring his data gathering and analysis, and the chal- Theoriesandframeworkshavebeenproposedtoprovide
lenges that he faced. There are three contributions of this a structure for data gathering and to offer scaffolding for
work: (1) an example of learning CI and DCog in a semi- the consolidation of results. Distributed Cognition (DCog:
structuredway;(2)anaccountoftheprocessandoutcomes Hollan et al. 2000) is a theory that has been shown to
of learning and using CI and DiCoT in a complex setting; deliver valuable insights into the design and use of tech-
and (3) an outline account of information flow in anaes- nology (e.g. Hazlehurst et al. 2007, 2008; Xiao 2005).
thesia.WhileCIwaseasiertolearnandconsequentlygave However, most accounts of the application of DCog omit
better initial support to the novice analyst entering a details of the process by which data were gathered, struc-
complexworksetting,DiCoTgaveaddedvaluethroughits tured,oranalysed,makingitdifficulttofolloworreplicate
focus on information propagation and transformation as the details. Halverson (2002) argues that the lack of
well as the roles of people and artefacts in supporting explicitly named constructs or methods in DCog has
communication and situation awareness. This study makes impededtake-upanduseofthistheory;whetherornotthis
visible many of the challenges of learning to apply a is true, use has historically required substantial craft skill.
framework that are commonly encountered but rarely Indeed,Benyon(2002,p.191)explicitlyomitsDCogfrom
reported. his review of representations for human-centred system
development ‘‘because real methods have not appeared
Keywords Distributed Cognition (cid:2) Contextual Inquiry (cid:2) based upon it’’. In response to this, methods for applying
Healthcare (cid:2) Anaesthesia (cid:2) DiCoT (cid:2) Anaesthesiology DCog in practice have been developed, to facilitate learn-
ing and more clearly articulate principles of DCog. For
example, Wright et al. (2000) propose the Resources
Model for analysing an interface; Galliers et al. (2007)
E.Berndt(cid:2)D.Furniss(cid:2)A.Blandford(&)
propose Determining Information flow Breakdown (DIB);
UCLIC,UniversityCollegeLondon,MaletPlaceEngineering
and Furniss and Blandford (2006) propose Distributed
Building,GowerStreet,LondonWC1E6BT,UK
e-mail:[email protected] Cognition for Teamwork (DiCoT) for analysing devices
123
432 CognTechWork(2015)17:431–449
and small team working. All of these frameworks aim to infusion devices, whose primary purpose is for managing
make DCog more accessible to researchers and practitio- patient care within the context of an ongoing operation.
ners by adding structural support to the application of
DCog. DiCoT presents concepts and principles of DCog 2.1 Learning a method: attributes and process
linked with models proposed for Contextual Inquiry (CI:
Beyer and Holtzblatt 1998). First, a word on terminology: we use the term ‘‘theory’’ to
The aim of the study reported here was to better refertoDCog,recognisingthattheapplicationoftheoryto
understandhowanoviceanalystlearnsandappliesCIand anyparticularsituationwillrequireaprocess.Processesfor
DiCoT. A complex healthcare setting (the work of anaes- analysis—whether or not grounded in theory—are widely
thetists in an operating theatre) was taken as a fieldwork referred to in the literature as ‘‘methods,’’ so where
setting for the study. Within that setting, the aim of the appropriate we refer to CI and DiCoT as ‘‘methods’’.
study was to understand the interactions of the anaesthe- However,neitherishighlyproceduralised,soatothertimes
tists,howthedesignofproceduresandoftheenvironment we refer to them as ‘‘frameworks’’ (for structuring data
supports work, and particularly how they used infusion gathering and analysis), to disambiguate them from the
devices. The study had multiple aims: tobetter understand research method of this study.
the interactions of anaesthetists with infusion devices and The focus of this study is on learning to apply CI and
with each other; to identify requirements for improved DiCoT.Therehavebeenfewstructuredstudiesoflearning
learningresourcesforDiCoT;andtobetterunderstandand any methods in HCI, and we are not aware of any studies
make visible the process of learning and using CI and thatpresentthedetailsofhowpeoplehavelearnedtoapply
DiCoT in a complex setting. The focus of this paper is on CI or conduct a DCog analysis: the closest is the work of
the last of these aims. Sellberg and Lindblom (2014), described below. The pre-
viousworkthatmoststronglyinfluencedthedesignofthis
study was that of John and Packer (1995). They provide
2 Background details of how one relative novice learned and used the
cognitive walkthrough method using a case study
Distributed Cognition is a theoretical perspective that approach. The study participant kept a detailed diary of
views cognition as being distributed between people and actions, difficulties, insights, and discussions held while
artefacts within a work system. It focuses particularly on learning the method and applying it. Importantly, this
how information is represented, communicated, and details the process, which is absent from many studies.
transformed within the system (Hutchins 1995; Hollan They argue that detailing process: (1) allows one to
et al. 2000). DCog analysis relies on ethnographically understandhow amethod was used andhowinsightswere
gathered data (Halverson 2002)—typically involving both gained; (2) gives a better understanding of what to expect
observation and interviews. Halverson (2002, p. 249) whensettingouttouseamethod;and(3)givesmeaningful
argues that the utility of DCog lies in ‘‘its theoretical feedback to method developers to improve their tech-
commitment to examine [the] broader socio-cultural- niques. All of these considerations are of interest in this
technical system, which is necessary for the collaboration study, although the focus of reporting is on the first two.
between individuals mediated by artifacts’’. This use of
DCog in the operating theatre is exemplified by the work 2.2 Contextual Inquiry (CI)
of Hazlehurstet al. (2007), who studied team coordination
in the cardiac operating theatre, focusing particularly on Inordertostructurehislearning, thenoviceanalystinthis
how situation awareness is maintained between surgeons study went through an explicit stage of learning and
and perfusionist (but paying relatively little attention to applying CI (Beyer and Holtzblatt 1998).
the anaesthetists who are the focus in the study reported Contextual Inquiry is part of the Contextual Design
here). methodology proposed by Beyer and Holtzblatt (1998). CI
As well as focusing attention on the broader system, uses contextual interviews: interviews with people within
DCog has also been applied to the details of individual their working context. During an interview session, the
user–device interactions. For example, Nemeth et al. analyst observes the interviewee and asks questions for
(2004)analysehowthedesignofartefactsintheoperating clarification. Beyer and Holtzblatt (1998) assert that inte-
theatre supports or hinders communication and coordina- gration of observation and interview gives access to unar-
tion between team members. Their focus is on artefacts, ticulated knowledge and allows the analyst to develop a
such as schedules and display boards, whose primary deeper understanding of the domain. Four interaction
functionisincoordinatingworkoveranextendedperiodof principles are proposed to guide the interviews: partner-
time, rather than those, such as patient monitors and ship, focus, interpretation, and context. The principle of
123
CognTechWork(2015)17:431–449 433
‘‘partnership’’ states that the interviewer should try to frameworks, which have better descriptive power. They
establish a master–apprenticeship relationship with the subsequently offer a comparison based on an empirical
interviewee, making it clear that s/he has inferior knowl- study; in their study, all three frameworks were applied to
edgeaboutthedomainbutahighinterestinunderstanding the same dataset based on 30 h of observations in three
it.Atthesame time,s/heshouldtrytomaintainaconstant dentalclinics,focusingonworkatthefrontdesk.Itappears
‘‘focus’’ on her/his unit of analysis, avoiding shifts. Fol- thatthemainanalystsweretheauthors,althoughthisisnot
lowing every contextual interview, the collected data are clearly stated. The CI analysis is one of the clearest
‘‘interpreted’’ and a pictorial representation is drawn. available accounts of the experience and outcomes of
Through this, the results are more accessible for discus- applyingthisframework.TheauthorsnotethatlearningCI
sions within the design team to build an understanding of wasmuchmoredifficultthantheliteraturehadledthemto
‘‘context’’. expect, e.g. in determining the appropriate level of detail
Five work models are created for every interviewee, for a flow model and in translating abstract guidance into
each highlighting a specific aspect of work. These indi- details of the practice of modelling.
vidual models are unified to five general models after all
interviews have been conducted: 2.3 Distributed Cognition for Teamwork (DiCoT)
• A flow model shows the overall organisation and
Analyses using DCog have traditionally been performed
coordination of the workflow;
using cognitive ethnography (e.g. Hutchins 1995). DiCoT
• A sequence model gives a description of tasks;
(Furniss and Blandford 2006) was developed as a semi-
• An artefact model represents objects used for work;
structured framework to help apply DCog, focusing on
• A cultural model provides a schematic overview of
small team interactions. The original version of DiCoT,
informal and broad influences on actors; and
which formed the basis for this study, did not specify the
• A physical model illustrates the workplace’s physical
datagatheringtechnique,butassumedamixofobservation
structure.
and situated interviews very similar to CI. A DiCoT ana-
ContextualInquiryhasbeenappliedinvariouscontexts, lysis involves constructing five models (Blandford and
e.g.tostudytheintroductionofATMsinIndia(DeAngeli Furniss 2006) to describe activities as observed:
et al. 2004) and the design of mobile exhibition services
• The information flow model shows how information
(Fouskas et al. 2002). In healthcare, CI has been used to
flows in tasks and between actors.
inform the design of a clinical trials screening tool (Gen-
• Artefact models focus on the structure of tools and
nari and Reddy 2000) and in the area of telecardiology
representations, and how they affect cognitive work in
(Gil-Rodr´ıguezetal.2007),butnodetailsofthemodelling
practice.
are presented in either of these cases. In contrast, Coble
• Physical models focus on the layout of the environ-
et al. (1995) report a detailed account of contextual inter-
ment, e.g. a desk or a room, and how this impacts the
views, model building, and consolidation for requirements
propagation of information.
capture for a clinical information system. They present a
• The social model focuses on the social relationships,
clearaccountofthevalueofgatheringuserrequirementsin
responsibilities, knowledge, and goal sharing between
this way, including identifying some unexpected clinician
individuals,andhowthisinfluencesthecomputationof
behaviours and the value of getting clinicians involved
the system.
such that they felt like partners in the development pro-
• The evolutionary model shows how the system has
gramme; however,they present fewdetails ofhow models
changed over time.
were constructed, focusing more on the identification of
user requirements. Each model is associated with a set of DCog concepts
Sellberg and Lindblom (2014) present a comparative andprinciples,whichserveasachecklistorvocabularyfor
study of CI (set within its broader context of Contextual analysingthemodelintermsofDCogtheory.Theseenable
Design)andtwoframeworksthatarebasedonDCog(DIB, the analyst to describe how the system ‘‘works’’ in DCog
whichfocusesoninformationcommunicationbreakdowns, terms and to identify points of vulnerability, ambiguity,
and CASADEMA, which focuses more on DCog theory and strength in system performance. The full list of prin-
andlessonstructuredmethod).Theyfirstofferareviewof ciples is included as ‘‘Appendix 1’’ to this paper.
the three frameworks based on prior literature on each, Principles associated with information flow are con-
concluding that CI’s lack of theoretical basis may make it cerned with how information moves and is transformed in
‘‘uninformed and fraught with bias’’ (p. 471), but that it is the system (and any obstacles to effective information
better framed to have application power (i.e. relevance to flow)—both formally and informally; whether any agent
design practice) than either of the DCog-based acts as an information hub (and if so, how that role is
123
434 CognTechWork(2015)17:431–449
Table1 ComparisonofCIandDiCoT
Feature CI DiCoT
Referencesource BeyerandHoltzblatt(1998) FurnissandBlandford(2006)andvariousotherMastersdissertations
andresearchpapers
Background CIwasdevelopedfromyearsofconsultingin DiCoTwasdevelopedinanacademiccontext
industry
Purpose Understandingworkforgatheringdesign Understandingthepropagationandtransformationofinformationat
requirements work
Relatedtheory Norelatedtheory UsedtofacilitatetheapplicationofDCog
Datagathering Contextualinterviews Observations,interviews,andotherethnographicmethods
Models Flow Informationflow
developedin Physical Physical
analysis
Artefact Artefact
Cultural (typicallydevelopedintheordershown,withsubsequentcyclesof
Sequence iteration)
(nosuggestedorderformodeldevelopment) Optionalmodels:
Social
Evolutionary
Systemactivitiesmeta-model
Subsequent Constructingaccountofworkandfuturedesign ConstructingaccountofworkintermsofDCogconceptsand
analysis requirements(forcontextualdesign) assessmentagainstDCogprinciples(see‘‘Appendix1’’)
Analysts Multipleanalystsareencouragedtogatherand Typically,asingleanalystconductsthestudy
bringtogetherdifferentperspectives
facilitated, and consequences of the hub being unavailable management of home haemodialysis (Rajkomar et al.
for any reason); and how interruptions are managed (in 2013). These case studies were all resources used by the
terms of triggering new activity or disrupting existing novice analyst to learn DiCoT. They exemplify DiCoT’s
activity). use, but there is, as yet, no single authoritative source for
Principles associated with the physical model are con- its practice.
cerned with how space, layout, and movements support
cognition and action and how the design of the system 2.4 Summary comparison of CI and DiCoT
supports situation awareness (Fioratou et al. 2010; Hazle-
hurst et al. 2007). Table 1 summarises the key features of CI and DiCoT.
Similar principles are associated with artefact models; Inthestudyreportedhere,theCIanalysisstoppedatthe
artefacts can also be moved around and used in various point where the models had been used to describe work as
ways, so further principles are concerned with how they performed, rather than progressing to any new design
mediatecommunicationandcanbeappropriatedtosupport generation.
cognition in other ways.
The social structures and evolutionary models are less
well developed, but associated principles are concerned 3 Study method
with understanding how cognition is distributed across
social groups and how both artefacts and expertise have As noted above, the aim of this study was to better
evolved into their current forms. understand the process of learning to apply CI and DiCoT
DistributedCognitionforTeamworkhasbeenappliedto inacomplexsetting.Thisisasinglecasestudy.AsGerring
analysevariouscomplexsettings, including:anambulance (2007,p.1)argues,‘‘Sometimes,in-depthknowledgeofan
dispatch control room (Blandford and Furniss 2006; Fur- individualexampleismorehelpfulthanfleetingknowledge
nissandBlandford2006);thepracticesofanagilesoftware about a larger number of examples’’. Similarly, discussing
development team (Sharp et al. 2006); mobile healthcare idiographic research (which focuses on the details of par-
work (McKnight and Doherty 2008); a medical equipment ticular cases rather than generalising across populations),
library (Werth and Furniss 2012); infusion pump use inan Tsoukas (1989, p. 551) notes that ‘‘Idiographic research
Oncology Day Care Unit (Furniss et al. 2011); infusion conceptualizes the causal capability of structures, while at
practiceinanICU(RajkomarandBlandford2012);andthe the same time it sheds light on the contingent manner
123
CognTechWork(2015)17:431–449 435
through which a set of postulated causal powers interact The ‘‘object of study’’ was the process and outcome of
and gives rise to the flux of the phenomena under study’’. completing CI and DiCoT analyses. To complete the
The work reported here is in this idiographic tradition, analyses,itwasnecessarytogatherdatainthechosenwork
seeking to gain understanding through an in-depth case context; this was operating theatres of a busy hospital in
study rather than by generalising across cases. As noted London. The focus of data gathering was on how anaes-
above, it replicates as closely as possible the study design thetistscoordinatetheirworkandparticularlyhowinfusion
of John and Packer (1995), which was also a single in- devices were used by anaesthetists, and how their design
depth case study. influenced their use. Eleven observations were made: ten
This work was made possible by the arrival of the first elective liver operations and one eye surgery, totalling
author, EB, as an intern in the research laboratory of the 40 h. Sessions were conducted during the morning (the
other authors. EB had a background in Psychology, and normal time for operations), and each took 3–5 h. In total,
1 year of education in HCI, including in ethnography, but 11 different anaesthetists were interviewed and observed.
no prior exposure to either CI or DCog. He also had an AllobservationswereconductedbyEB,butmeetingswith
interest in health technology, but no prior experience of DF and AB took place regularly.
observationalstudiesinoperatingtheatresorknowledgeof Diary entries and the CI and DiCoT outcomes were
anaesthesiapractice.Asaconsequence,hewaswellplaced analysedusingThematicAnalysis(BraunandClarke2006)
to conduct an in-depth diary study of learning and using based around questions about the challenges of learning
DCog in an unfamiliar, complex setting. each framework, and what insights were achieved at dif-
The three authors of this paper, who we will refer to as ferentpointsinthestudy.Theinitialreflexiveanalysiswas
EB, DF, and AB, took different roles in this work. EB’s completed by EB, in discussion with DF; this was subse-
rolewastosystematicallylearnandapplyCIandDiCoTin quently reviewed and extended by AB in discussion with
thechosencontextofstudy,maintainingadetaileddiaryof both EB and DF.
activities, difficulties, insights, and findings as he did so.
DF took a mentoring role, advising as needed on the
3.1 Stages of analysis
applicationofbothCIandDiCoT.ExampleentriesinEB’s
diary illustrate the relationship:
The study took place in three parts: Part 1 applied CI,
In the talk with [DF] I discussed these questions and Part 2 applied DiCoT, and Part 3 was a reflexive ana-
it helped me a lot to talk about it. Furthermore [DF] lysis of the processes and outcomes of learning and
had some really good ideas. We concluded that CI is applying CI and DiCoT to the work of anaesthetists. CI
an orientation, and should not be used dogmatically. was learnt and applied first because DiCoT builds on
CI’s techniques and models. The two phases of learning
Diary Entry, Friday, week 2 of part 1
were separated so as to make it possible to identify
AfterwardsIdiscussedwith[DF]aboutthedifficulties insights that were derived directly from the initial
I had during interviewing and modeling. I am very modelling and those that were facilitated by the appli-
unhappywithsomeofthemodelsandthinkthattheydo cation of the DCog principles embodied in DiCoT. In the
not map the context properly. However, [DF] argued early phases of the study, we were regarding it as a
thatalotofthecontextinformationisinmyhead,and comparison study, to identify what kinds of issues each
thatthemodelsareonlyarepresentationofit. framework helped to identify [in a similar style to the
study of Sellberg and Lindblom (2014)]. During Part 3
Diary Entry, Wednesday, week 5 of part 1
(the reflexive analysis), we realised that this was an
The substance of discussions between EB and DF was inappropriate way to think of the frameworks, since
recorded and subsequently included in the analysis for DiCoT makes use of some of the CI models, so our
reflection. Our strategy was that EB should work as far as analysis shifted focus from how the frameworks compare
possiblefrompublishedsources,soastomakethelearning with each other to better understand their complemen-
process as inspectable as possible. He received guidance tarity and the overall process of learning CI and DiCoT
fromDFwhenneeded,butmuchofthisconcernedgaining as steps of conducting a DCog analysis.
access to, and working in, the study setting, rather than The study working with each framework involved three
details of how to gather and analyse data. AB oversaw the sub-parts: learning the framework; data gathering and
project, negotiated access to the study setting, provided analysis using that framework (and maintaining a diary of
strategic direction, and conducted a retrospective inde- learning); and reflection on what had been learned about
pendent analysis of all the data generated by EB. the framework. In each case, EB’s learning was largely
123
436 CognTechWork(2015)17:431–449
self-directed, based on reading existing resources pre- prepare for future sessions. Final consolidation of results,
senting the framework and seeking clarification where adjustments to the models and evaluation of the DiCoT
needed from DF. Since data gathering and analysis were diary, took place in the last week of part 2.
‘‘owned’’ by EB, with mentoring guidance from DF and
general from AB, we focus in the following sections on
EB’s experience. 3.1.3 Part3—3 weeksintensive,followedbyfurthercycles
of reflection and iteration
3.1.1 Part 1—5 weeks on CI
All data were reanalysed to review the process and out-
ContextualInquirywaslearntoverthefirstweek,followed comes of EB learning and applying CI and DiCoT, as
by 3 weeks of data gathering. Contrary to prior planning, phases of learning to do a DCog analysis.
the first week of data gathering was used for general con-
textorientation,ratherthantheapplicationofCI,becauseit
provedtobetoochallengingtobothmakesenseofthenew 4 Results
setting and apply CI in that context.
Here, we present EB’s experiences of learning and apply-
until now I have mainly unstructured, chaotic data
ing the frameworks sequentially. The first section reviews
Diary Entry, Wednesday, week 2 of part 1 EB’s experiences of learning CI and DiCoT; in the fol-
lowingsection,wereflectonhisuseoftheframeworksand
I think because I was so overwhelmed by all the
the insights they afforded.
different information I sometimes took some infor-
mationasgranted,didn’twriteitdown.Maybeitisa
4.1 Learning CI and DiCoT
good idea to create some kind of standardized
observation sheet to steer the observation
When reviewing EB’s diary of learning CI, a theme that
Diary Entry, Friday, week 2 of part 1 emerges repeatedly is the challenges that he faced in rep-
resenting and reasoning about the complexity of the study
The difficulties experienced were not attributed to CI
setting,andinexpressingthatcomplexityinratherrigidCI
itself, but to the complexity of the context of study.
models. When he moved on to learning DiCoT, his per-
During the two subsequent weeks, CI was applied in six
ception was that it was clearer how to use the models and
sessions.Extensivenotesweretakenduringeachsession.
principlestoreasonaboutthecomplexityofthesetting.For
Consolidation of the gathered data took place directly
DiCoT, the biggest challenge lay in the loose definition of
after the session. Two days each week were taken to
the framework and the fact that the underpinning DCog
create models, identify gaps in the data and prepare for
theorywasnotcoveredindepthinthereferenceresources.
subsequent observations. After data gathering was com-
Three main themes emerged in the analysis: how to rep-
pleted,resultswere consolidatedand finaladjustmentsto
resentandreasonaboutcomplexity;theroleofresourcesin
the models were made. The diary was evaluated in the
supporting learning; and the process of learning to work
last week of part 1 to reflect on what had been learned
withthemodels(andletthemguidedatagatheringaswell
about CI.
as analysis).
3.1.2 Part 2—5 weeks on DiCoT
4.1.1 Representing and reasoning about complexity
After the CI analysis was completed, attention switched to
DiCoT. The learning took 2 weeks, a week longer than
Beyer and Holtzblatt (1998, p. 21) describe Contextual
planned.
Design (of which CI is a key component) as being ‘‘opti-
The learning of DiCoT is really cumbersome. So mized for large, complex projects’’ and yet, as noted also
many different sources and the abstract principles by Sellberg and Lindblom (2014), the text book did not
make it really hard to learn. offerstrategiesformanagingthevarietyandcomplexityof
activities observed in the anaesthetist’s work space.
Diary Entry, Wednesday, week 2 of part 2
When I firstly used CI I felt very stupid during the
In the following 2 weeks, five data gathering sessions
first observation days, because reading the book
were conducted. As before, consolidation of results took
implied it to be a very easy straightforward task.
placeimmediatelyafterthesession.Twodaysaweekwere
taken to create models, identify gaps in the data, and Diaryentry,Wednesday,Week1ofpart2.
123
CognTechWork(2015)17:431–449 437
Early on, the complexity of the work being studied felt the scope of one’s analysis. […] the ICU is an area
overwhelming. Everything is interrelated, and EB wanted with a lot of independent activities. You can never
to capture that complexity: observe all of them, so you have to take decisions
which activities to observe and which to let aside.
It is just too much information for creating models. I
Meta models help you to justify your decision. They
need a focus. Everything is interrelated (if I only
show that there ARE other activities, but only
focus on drugs that are given for anaesthesia, I still
describes them very broadly and their influence on
have to focus on ventilation and monitoring).
the primary task.
Diary entry, Tuesday, Week 3 of part 1.
Diaryentry,Wednesday,Week2ofpart2.
It was difficult to identify an appropriate level of
The greater support that the DiCoT resources provided
abstraction to avoid being overwhelmed by details:
for dealing with complexity did not, however, make the
Mightitmaybegoodideatoputsomeprocessingjust DiCoTphaseoflearninganymorestraightforwardoverall.
in a black box, to be able to focus on the relevant
things. However, […] tasks are so much interrelated 4.1.2 Working with inadequate information about DiCoT
it might steal a lot of the complexity of the actual
task. As noted above, there is no single source of DiCoT guid-
ance, so learning it involved reading several research
Diaryentry,Wednesday,Week3ofpart1.
papers. EB discerned inconsistencies between reports and
The issue of level of abstraction and managing com- yet felt that they reinforced each other.
plexity emerged repeatedly. For example, when trying to
The papers were mostly revision and did not add
create a first sequence model, EB resorted to creating an
substantialinformation.Ihavethefeelingthatthereis
overview model showing all relevant tasks, which he then
no consistent view on DiCoT yet, which makes it
broke down into three sequence models. This was an
very flexible on the one hand, but also more difficult
improvised approach because the CI text does not provide
to learn and understand on the other.
any guidelines on the process of model creation. The
assumption appears to be that complexity can be handled Diary entry, Tuesday, Week 1 of part 2.
through decomposition, which was not readily achieved in
Reflecting the relative lack of maturity (and arguably
the anaesthesia context.
greater ambition to model interactions) of DiCoT com-
Otherquestionsemerged,forwhichnoclearanswerwas
pared with CI, EB quickly became aware of some of the
found, such as whether there was an optimal order for
tensions and challenges of creating, in particular, the
developing models; how to model interruptions to work;
DiCoTartefactmodel,andthewaysthatithadbeenshaped
where to make clear distinctions between different situa-
by need and circumstance:
tions or merge models; and how to model contingent
aspectsofsituations,giventhatthereisnotimecomponent Itisinterestingtoseehowtheartefactmodelevolved
tosomekeymodels: theyprovideastatic perspectiveona to a very complex model in DiCoT. In CI it was
continuously evolving context. relativelyconstrainedandeasytocreate.InDiCoTit
EB felt that DiCoT gave better guidance and resources getsacompletedifferentmeaning[…].Modelsaren’t
formanagingthecomplexityofthesituationbeingstudied: as rigid as they are in CI and could be adjusted to
one’s own requirements.
[DiCoTpapers]givebetterindicationshowtohandle
all the complexity one is facing at the beginning. Diaryentry,Thursday,Week1ofpart2.
Furthermore they […] propose a sequence of models
ThisviewofCIasbeingrigidisprobablyaconsequence
[…] What I also really appreciated was the men-
of relying on one (well written, clear) reference source for
tioningoflikelyproblemssomeonemightface(being
the framework. One issue that is obvious with hindsight,
overwhelmed, don’t know where to start)
but was not at the time, is that EB focused on reading
Diaryentry,Wednesday,Week1ofpart2. DiCoT literature rather than the broader DCog literature.
All the DiCoT models were consistent with the broader
One particular strategy for managing complexity that
DCog literature, simply emphasising or de-emphasising
had been developed by Rajkomar and Blandford (2012)
differentelementsofDCogtheorywhilealsorespondingto
was the introduction of a system activities ‘‘meta-model’’:
the demands of the particular situation being represented.
it isn’t really part of distributed cognition, but To the developers of different DiCoT models, the differ-
important to support the decisions one has made on ences were not perceived as significant (because all were
123
438 CognTechWork(2015)17:431–449
consistent with DCog), but to EB, the differences were he uses here, one might argue that the analyst’s prior
perplexing and hard to deal with: assumptions are not relevant to analysis):
Today I tried to prepare for the first observation Imissapossibilitytofillinreallybroadobservations,
tomorrow and figured out, that it is much more dif- like‘‘Theworkofananaesthetistislessstressfulthan
ficult than I thought. The DiCoTs applied by [dif- assumed’’.
ferent authors] are quite different. They interpret the
Diary entry, Tuesday, Week 3 of part 1.
modelsinverydifferentways,whichmakesithardto
identify one single DiCoT framework. In the early days, EB clearly had not internalised the
semantics (purpose or meaning) of the models he was
Diary entry, Tuesday, Week 2 of part 2.
working with:
Distributed Cognition for Teamwork is based on the
Iwrotedownallmyobservationagain,categorizedit
DCog literature and was developed as a guide to DCog
(which still isn’t easy, because nearly every model
analysis.Atitsheartareasetofabstractprinciplesthatare
fits every observation). I also find it very hard to be
intendedtoguidetheanalyst:intheconstructionofDiCoT
consistent. I think sometimes it happens that I mark
models;indrawinginferencesfromthosemodelstoinform
something as ‘‘out of scope’’ whereas in other occa-
future design; and in identifying the strengths and weak-
sions it is still part of the ‘‘relevant’’ information.
nessesofexistinginteractive(teambased)systems(Furniss
and Blandford 2006). However, the relationships between Diaryentry,Wednesday,Week3ofpart1.
principlesandmodelsthatwereself-evidenttoDFandAB
Initially, EB was trying to follow Beyer and Holtzblatt
were not so to EB:
(1998) ‘‘to the letter’’. When presenting a method, it is
I still do not completely understand the abstract always difficult for authors to articulate how it can be
principles, and their links to the models. legitimately adapted. Methods are, essentially, tools and
are only useful insofar as they help with achieving the
Diaryentry,Thursday,Week2ofpart2.
desiredoutcome.Expertanalystsdevelopasenseofwhere
Further, the DiCoT concepts, which are drawn from the methods and models can be adapted (Blandford et al.
DCog literature, were not adequately explained in the 1998), and where it is appropriate to take short cuts, but
DiCoT literature. this was not apparent to a novice:
it is really hard to apply the abstract concepts of the IreadtwopapersonresearchthatalsoappliedtheCI
DiCoT to the concrete situation. What is an infor- intheirstudies,inordertoseehowstrictlytheystuck
mation transformation, who is a buffer? tothe‘‘rules’’.Itbecameclearthattheydonoteither.
Both studies left out some of the five models and
Diary entry, Thursday, Week 3 of part 2.
adjusted the approach in a manner fitting to their
Although not devised as a study of the existing DiCoT focus.
literature, the study did, in practice, serve as an enlight-
Diary entry, Thursday, Week 3 of part 1.
ening formative evaluation of that literature, highlighting
tacit assumptions, unclear explanations and, in particular, Reflecting back over the first 3 weeks of conducting
the extent to which an understanding of DiCoT relied on observationsandconstructingmodels,EBcoulddiscernthe
establishedunderstandingofDCog.Thiswillinformfuture influence of the modelling on his data gathering, showing
workonclarifyingandtestingDiCoT,butisnotafocusfor thatthemodellingwasshapinghislaterdatagathering:
this paper.
Icouldobserveapatterninmyobservations,becauseI
have used colour coding for irrelevant observations.
4.1.3 Learning to work with models (any models) […]Isometimesfocusedonthingsthatweren’trele-
vantafterinspectionafterwards.WhenIfirststartedto
An additional theme that emerged through the diary was create models, the information content of my obser-
the challenge of learning any model-based technique, and vationincreasedsignificantly,probablybecauseInow
the way that it initially seems to present obstacles to hadanewfocusandfeelingwhatmightbeimportant.
overcome, but gradually becomes a powerful tool to aid
Diary entry, Thursday, Week 5 of part 1.
thinking.
Any model-based technique focuses on some aspects MovingontotheDiCoTpartofthestudy,aswiththeCI
while downplaying others. The novice analyst found this models, it gradually became evident that the requirements
focusfrustratingattimes(eventhough,totaketheexample of constructing models were shaping the data gathering:
123
CognTechWork(2015)17:431–449 439
DiCoT highlights some aspects I haven’t looked at described in the available models. As discussed below,
before—hierarchy, evolution, representation of model construction helped to expose many issues with the
resources, which I now try to implement in my system, but it did not expose all the issues (such as the
models. challenges of setting up one of the infusion devices) that
emerged through observations and interviews:
Diaryentry,Monday,Week4ofpart2.
I had some problems in using the existing models to
By the latter stages of the study, EB was becoming
show what I have observed. Most difficult was the
masterful at using models to generate questions to focus
modellingof the setup and adjustment process of the
furtherdatagathering,sothatdatagatheringwasbecoming
[named infusion device].
increasingly shaped by the analysis activity:
Diary entry,Sunday, Week 5ofpart2.
Yesterday evening I reviewed my observations, the
templates I created and the DCog principles and
4.1.4 Summary
created questions for today. That works great. Every
day I get a lot out of only a few hours.
In summary, the approach of structuring learning into two
Diary entry, Tuesday, Week 4 of part 2. stages helped to highlight contrasts between two elements
of the DiCoT framework (modelling and principles) that
Models focus attention on particular aspects of the
are needed to completing a DCog analysis. It exposed
system.Theirvaluegraduallyemergedoverthetimeofthe
manytacitassumptionsintheextantDiCoTmaterials,e.g.
study, and EB gradually felt empowered by them. For
that the link between DiCoT and DCog was self-evident;
example:
that the relationship between models and principles was
the social structure model made the organisation of self-evident; that the existing materials were clear enough
work clearer to me. to support a novice analyst; and that the variations in
modelling approaches between different case studies were
Diaryentry,Wednesday,Week4ofpart2.
negligible. While we were aware that the DiCoT frame-
Ihave gradually built up areal understanding for the workhadbeenshapedbythecasestudiesthroughwhichit
models. I often think ‘Oh that fits into the… model’. had been developed, the fact that it had been applied in
FurthermoreIcanaskmuchbetterquestions,becauseI threecomplexworksettingspreviouslygaveusreasonable
know what information I want to have. Last but not confidence that it could be applied in anaesthesia—a con-
least, I come to a point where I can ask the same fidence that was eventually borne out in practice, but only
questionstodifferentpeopleandrevealcontradictions. after overcoming significant conceptual hurdles.
Diaryentry,Thursday,Week4ofpart2.
4.2 The outcomes of applying CI and DiCoT
It would be wrong to conclude that modelling became
straightforwardinthefinalfewdaysoftheproject.Further
Complete sets of models for each framework were devel-
difficulties were still encountered. Reference to worked
opedoverthecourseofthestudy.Intheearlystages,theCI
examples helped; for example, it showed how flexibly
cultural model helped EB gain an overview of very broad
models could be used. However, it did not resolve all
influences on the unit of analysis. For example, it became
issues:
apparent that there were issues with the purchase of
I started with the artefact model of the [infusion equipment across the whole hospital, which resulted in
device]. […] First I thought it was very clear, but there being different makes and models of devices, with
after working longer on it I got unsurer and unsurer inconsistent interfaces, being available in the department.
[…] I read the paper of [DF], wherein he applied Although this theme was not developed in the DCog ana-
somekindofinformationflowmodeltothesequence lysis,earlyexposuretosuchbroadercontextualfactorswas
ofadministration.We[…]agreedthatitwouldmake helpful in better understanding the broad work system and
more sense for me to put this part in the artefact outside influences that shape it.
model and then discuss the sequence in terms of The overall work of anaesthesia can be understood as
resources. involving three phases: inducing, maintaining, and ending
anaesthesia. The focus for data gathering and analysis in
Diaryentry,Monday,Week5ofpart2.
this study was on maintaining anaesthesia. This, in turn,
In particular, there were issues that emerged through comprises three primary aspects: hypnosis (keeping the
observations that were not readily amenable to being patient asleep), analgesia (keeping the patient free from
123
440 CognTechWork(2015)17:431–449
Fig.1 Flowmodel(ContextualInquiry),focusingontheanaesthetistteam.Numberedissuesarediscussedbelow
pain), and keeping muscles relaxed for surgery. The prin- howinteractionsbetweenactorsandartefactstakeplacein
cipal technologies used by anaesthetists are infusion devi- the system, but they have differences in their level of
ces for administering drugs and monitors to maintain abstraction and vocabulary. This had not been apparent in
awareness of the state of the patient (these are shown in earlierwork,whereDCoghadbeenlearnedbeforeCI,and
Fig. 3below).EB’sanalysescoveredtheflowofworkand so the CI models had been implicitly interpreted in DCog
ofinformation;theroleofthephysicallayoutinsupporting terms.
or disrupting work; the social structures; and the detailed The elements of workflow within CI can be people
design and use of an epidural pump and a syringe driver (ovals), groups (dotted rectangle), places, and artefacts
(for pain management). The details of these analyses are (shaded ovals), as illustrated in Fig. 1. Connections
beyond the scope of this paper. between elements show how the elements interact to build
For illustrative purposes, we focus on the flow models up the workflow. For every person, their most important
developed in the two stages of the study. We have chosen responsibilities are described. Small lightning bolts repre-
to focus on this model because DCog expands the infor- sent breakdowns within the flow.
mation processing metaphor of the mind to broader so- Based on CI, EB’s flow model description was as
ciotechnicalsystems(Hollanetal.2000),whichmakesthe follows:
information flow model central to DiCoT. Flow is also a
Twoanaesthetistsformtheanaestheticteam.Bothare
key model in CI.
responsible for maintaining the homeostasis of the
EB interpreted the use of the flow model subtly differ-
patient and for supervision of the three subsequent
ently in the two frameworks. In CI, the flow model is
steps of anaesthesia (inducement, maintenance and
concerned with the flow of work and the broad kinds of
ending of anaesthesia). Both anaesthetists assist each
communication between agents, whereas DiCoT talks
otherduringtheir workandcommunicate frequently.
about the flow of information. Both are concerned with
123