Table Of ContentWhat Is It Like to Be a Bot?
What Is It Like to Be a Bot?
D. E. Wittkower
The Oxford Handbook of Philosophy of Technology
Edited by Shannon Vallor
Subject: Philosophy, Philosophy of Science Online Publication Date: Jan 2021
DOI: 10.1093/oxfordhb/9780190851187.013.23
Abstract and Keywords
This chapter seeks to further develop, define, and differentiate human-technics alterity
relations within postphenomenological philosophy of technology. A central case study of
the Alexa digital assistant establishes that digital assistants require the adoption of the
intentional stance, and illustrates that this structural requirement is different from an
thropomorphic projection of mindedness onto technical objects. Human-technics alterity
relations based on projection are then more generally differentiated from human-technics
alterity relations based on actual encoded pseudo-mental contents, where there are mat
ters of fact that directly correspond to user conceptualizations of “intentions” or “knowl
edge” in technical systems or objects. Finally, functions and user benefits to different al
terity relations are explored, establishing that there is a meaningful set of cases where
the projection of a mind in human-technics alterity relations positively impacts technical
functions and user experiences.
Keywords: phenomenology, postphenomenology, digital assistants, philosophy of mind, alterity relations, inten
tional stance, infosphere, care
1. Introduction
Thomas Nagel’s “What Is it Like to Be a Bat?” (Nagel 1974) gathered together and re
framed numerous issues in philosophy of mind, and launched renewed and reformulated
inquiry into how we can know other minds and the experiences of others. This chapter
outlines a branching-off from this scholarly conversation in a novel direction—instead of
asking about the extent to which we can know the experiences of other minds, I seek to
ask in what ways technologies require us to know the non-experiences of non-minds. This
rather paradoxical formulation will be unpacked as we go forward, but put briefly: We
sometimes treat some technologies as if they have minds, and some technologies are de
signed with interfaces that encourage or require that users treat them as if they have
minds. This chapter seeks to outline what we are doing when we develop and use a
pseudo-“theory of mind” for mindless things.
Page 1 of 16
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). © Oxford University Press, 2018. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in
Oxford Handbooks Online for personal use (for details see Privacy Policy and Legal Notice).
Subscriber: New York University Libraries; date: 14 February 2021
What Is It Like to Be a Bot?
Nagel’s article used the case of the bat to focus and motivate his argument, but took aim
at issues falling outside of human-bat understanding. Similarly, this chapter seeks to get
at larger issues that pervade human-technology understanding, but will use a bot as a fo
cusing and motivating example: in particular, Alexa, the digital assistant implemented on
Amazon’s devices, most distinctively on the Amazon Echo. Interacting with Alexa through
the Echo presents a clear and dramatic need for users to act as if they are adopting a the
ory of mind in technology use—other technologies may encourage or require this
pseudo-“theory of mind” in more subtle or incomplete ways and, I suspect, will increas
ingly do so in future technological development.
We will begin with a microphenomenology of user interaction with Alexa and a heterophe
nomenology of Alexa that emerges in use, making clear what sort of fictitious theory of
mind the user is required to adopt. This will be followed by a wider consideration of rela
tions with technological “others,” outlining a central distinction between a merely pro
jected “other” and those technological “others” the function of which requires that the
user treat them as an “other,” rather than a mere technical artifact or system. Finally, we
will turn to the user experience itself to ask what affordances and effects follow from
adopting a fictitious theory of mind toward technical systems and objects.
2. Notes on Methodology and Terminology
In phenomenology, there is a risk that we take our introspective experience as evidence
of universal facts of consciousness. Phenomenology differs from mere introspection in
that, reflecting its Kantian foundations, it recognizes that experiences—the phenomena
studied by phenomenology—are not bare facts of sense-data, but are the product of
sense-data as encountered through the conditions for the possibility of experience, and
are further shaped by our ideas about ourselves and the world. Phenomenology seeks to
isolate experiences in their internal structure, in some versions even “bracketing off”
questions about the correspondences of our experiences to elements of the world that
they are experiences of (Husserl [1931] 1960). When done carefully, this allows us to
speak to the structure of experience, and to take note of where our experiences do not ac
tually contain what we expect, allowing us to isolate and describe elements of Weltan
schauung that we use to construct experience. For example, in “The Age of the World Pic
ture” Martin Heidegger argues that place, not space, is phenomenally present in our ex
perience, and that space as a three-dimensional existing nothingness in which external
experiences occur is a kind of retroactive interpretation of the world as inherently mea
surable which emerges with the development of experimental science in the modern peri
od in European history (Heidegger 1977a). As we begin to equate knowledge of the ob
jects of external experience with their measurement, we begin to hold that only that
which can be measured is real, and this eventually leads to the uncritical adoption of the
metaphysical position that reality is always already articulated in the forms of human
measurement.
Page 2 of 16
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). © Oxford University Press, 2018. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in
Oxford Handbooks Online for personal use (for details see Privacy Policy and Legal Notice).
Subscriber: New York University Libraries; date: 14 February 2021
What Is It Like to Be a Bot?
Heterophenomenology, as articulated by Daniel Dennett (1991), similarly brackets ques
tions of correspondence to reality in order to isolate the structure of experience. Here,
though, the question is not whether and to what extent experiences correspond to that of
which they are experiences, but whether and to what extent experiences of the experi
ences of others correspond to the experiences of others. Dennett uses this heterophenom
enological approach in order to avoid the problem of other minds when making claims
about consciousness; to address what we can know about consciousnesses outside of our
own, given that we cannot possibly have access to the qualia (the “what it’s like”) of the
consciousness of others.
Dennett argues that uncontroversial assumptions built into any human subject experi
mental design, for example, the assumption that subjects can be given instructions for the
experimental process, require this kind of bracketing insofar as they must adopt an inten
tional stance—as assumption that the subject has a set of intentions and reasons that mo
tivate, contextualize, and lie behind the data gathered. “[U]ttered noises,” he says, “are
to be interpreted as things the subjects wanted to say, of propositions they meant to as
sert, for instance, for various reasons” (Dennett 1991: 76). Dennett claims that without
adopting such an intentional stance, empirical study of the minds and experiences of oth
ers is impossible.
We intuitively adopt an intentional stance toward many others, including non-humans,
based on strong evidence. It is difficult to account for the actions of dogs and cats, for ex
ample, without attributing to them intentions and desires. In other cases, we use inten
tional language metaphorically as a kind of shorthand, as when we say that “water wants
to find its own level.” There are many messy in-betweens as well, such as when we speak
of the intentionality of insects, or that a spindly seedling growing too tall to support itself
is “trying to get out of the shade to get more sun.” In many in-between cases, such as
“the sunflower tries to turn to face the sun,” the best account of what we mean is neither
pure metaphor (as in “heavy things try to fall”) or a real theory of mind (as in “the cat
must be hungry”). Instead, we refer to the pseudo-intentionality of a biological proper
function as defined by Ruth Millikan (1984): a way that mindless things, without con
scious intention, react to their environment that has an evolutionarily established func
tion, constituting a set of actions that have an “aboutness” regarding elements of its envi
ronment that is embedded within the way that causal structures have been established,
but that doesn’t really exist as an intention within individual members of the species.
In using heterophenomenology to articulate our experience of bots as “others,” we are
departing entirely from Dennett’s purpose of studying presumptively conscious others
and articulating an adoption of an intentional stance distinct from any of those mentioned
in the above examples. Alexa’s interface directs us to use an intentional stance both in
our interactions and our intentions toward her—we find ourselves saying things to our
selves like “she thought I said [x/y/z]” or “she doesn’t know how to do that.” This is, how
ever, not because we actually have a theory of mind about her. We know she is not the
kind of thing, like a person or a dog or a cat, that can have experiences. Instead, we are
directed to adopt an intentional stance because, first, the voice commands programmed
Page 3 of 16
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). © Oxford University Press, 2018. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in
Oxford Handbooks Online for personal use (for details see Privacy Policy and Legal Notice).
Subscriber: New York University Libraries; date: 14 February 2021
What Is It Like to Be a Bot?
into the device include phrasing that implies a theory of mind, second, because there is a
representation relation that holds between the audio input she receives and the com
mands she parses from that input which is best and most easily understood through in
tentional language, and third, because a second-order understanding of how she “under
stands” what we say gives us reason not to use a second-order understanding in our actu
al use of Alexa, but to return to a first-order intentional stance. The first of these reasons,
that she is programmed to recognize phrasing that implies she can listen, hear, under
stand, etc. should already be clear enough, but the other two factors require explanation.
We use language that implies Alexa’s mindedness as required by the commands she is
programmed to receive, but this language reflects a very concrete reality: there is an
“aboutness” of her listening, and she has an “understanding” of what we have said to her
that is distinct from our intentions or projection, as is clear from how she can (and does)
“get things wrong” and can (and does) “think” that we said something different from
what we think we said. If we wished to articulate that intentionality objectively and accu
rately, something like Millikan’s account would work quite well—her responses are dictat
ed by proper functions established through voice recognition software trained on large
data sets—but this second-order understanding of the “intentionality” of her actions is not
the one that we must adopt as users. We are required in practice to adopt a first-order in
tentional stance in order to use devices with Alexa, even though we have no (second or
der) theory of mind about them.
When we do engage in second-order reasoning about Alexa, thinking about how she
processes sound (“listens”) and parses commands (“does things”) according to her ontol
ogy (“understanding”), we are usually routed back to the first-order intentional stance.
We have little window into the way that Alexa processes input, and have little access to
her code other than as interactant. The imbalance between the user’s knowledge of how
Alexa is programmed and the programmer’s knowledge of how users are likely to talk to
her makes second-order reasoning ineffectual: even a tech-savvy user is often better off
thinking through how to communicate with Alexa by adopting an intentional stance than
by thinking of her as programming.
This is what I meant at the outset of this chapter by saying that our goal is to understand
how the use of bots like Alexa requires us to understand the non-experiences of non-
minds. To use Alexa, we must adopt the intentional stance toward a non-subject that has
neither experiences nor mindedness, and we must interact with her in a way that ad
dresses specific, factual experiences that she is not having and specific, factual intentions
and interpretations that she does not have. These “non-experiences” are not a simple lack
of experiences and Alexa’s “non-mind” is not a simple lack of mind—when Alexa incor
rectly “thinks” I asked her to do X rather than Y, and I try to say it so she’ll “understand”
this time, her actually existing “non-experience” of my intention is an object of my
thought and action. This is quite distinct from a microwave oven’s entire lack of experi
ence of my intention when it overheats my food, or a toaster’s entire lack of understand
ing of my intention when the middle setting doesn’t brown the bread to my preference. In
these cases, there is nothing at all in the device to refer to as an “understanding” or an
Page 4 of 16
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). © Oxford University Press, 2018. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in
Oxford Handbooks Online for personal use (for details see Privacy Policy and Legal Notice).
Subscriber: New York University Libraries; date: 14 February 2021
What Is It Like to Be a Bot?
“interpretation,” only my own, sadly disconnected intention. Alexa, though no more sub
ject to experiences and no more conscious than these or any number of other kitchen
tools, functions in a way in which there are concrete, real, objecting “interpretations” and
“understandings” that she “has” that are outside of both my mind and the direct interface
present to the senses.
I will use strikethrough text to identify these “intentions” or experiences-which-are-not-
one in order to recognize that they have an objective content and aboutness with which
we interact, despite the fact that they are neither intentions nor experiences. Hence, I
will say that, for example, Alexa thinks that I asked her X rather than Y, and thus she mis
understood or misinterpreted my request. This typographical convention allows us to ar
ticulate that the user is adopting an intentional stance when trying to transmit meaning
and intention to a technical system. Compare, for example, with the video editor’s rela
tionship to their software (Irwin 2005), or the familiar case of moving a table or image
within a Microsoft Word document. In this case, we have an intention which we are trying
to realize within the document, and which is frequently frustrated by a system that often
responds with unpredictable repagination or unexpected, drastic, unintended reformat
ting. But here, our attempts to realize our intentions in the document take the form of try
ing to figure out how to get it to do what we intended. With Alexa, although the underly
ing causal structure is not much different, our attempt is not to do something to get it to
respond as intended, but instead to figure out how to phrase or pronounce our request so
that she interprets or understands what we mean—to communicate rather than just to en
act our intention, so that the semantic content present within the technological other cor
responds to the semantic content within our own conscious intention.
Having clarified this point about the manner in which we adopt an intentional stance to
ward at least some technical systems or objects, such as Alexa, in the absence of a theory
of mind, we are ready to engage in a heterophenomenology of Alexa. We will do so in the
mode of microphenomenology (Ihde 1990)—a phenomenology of a particular set of experi
ences rather than a wider existential phenomenology of our worldedness more generally.
So, our question will be “what is our experience of Alexa’s experience like” rather than
“what is it like to be in a world inhabited by smart devices that have experiences.” Once
we have finished this microphenomenology of the heterophenomenology of Alexa, we will
use it to engage in a more general analysis of human-technics alterity relations.
3. Opening the Black Box of Alexa’s Echo
We began the last section by noting that, in phenomenology, there is a risk that we take
our own introspective experience as evidence of universal facts of consciousness. In het
erophenomenology—outlining the experience of other minds—there is a risk that we mis
take our projections for observations. Dennett, when outlining heterophenomenology
(1991), made a very strong case that heterophenomenology can be done responsibly if we
take care to stick close to evidence and to take note of when and how we are adopting the
intentional stance when making judgments about other minds.
Page 5 of 16
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). © Oxford University Press, 2018. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in
Oxford Handbooks Online for personal use (for details see Privacy Policy and Legal Notice).
Subscriber: New York University Libraries; date: 14 February 2021
What Is It Like to Be a Bot?
This danger is even more pronounced here, though, since we are addressing the minded
ness of technical systems that clearly do not actually have minds. While this pseudo-mind
edness is no mere metaphor or projection, since there is a fact of the matter about what
Alexa thinks we said or meant, there is obviously an element of metaphor or analogy in
our understanding of her experiences, and this is bound to lead to some amount of falla
cious projection of thoughts and understanding and even personality. Observer bias is an
other danger: it must be considered that the ways I’ve interacted with her may not be
representative of the range of use, or even of typical use.
But other factors count in our favor, here. First, our goal is not an accurate theory of
Alexa, or a sociology of Alexa use, but only an articulation of the kind of stance her inter
face demands, so an incomplete or somewhat biased sample should present no serious is
sues in our analysis. Second, you may have your own experiences that can provide verifi
cation and nuance to those outlined here. Third, I have research assistants of a very valu
able kind: my kids. They take Alexa at interface value (Turkle 1995) with less resistance
than most adults, and interact with her without a strong understanding of what technical
systems can do, and without preconceived ideas about what sort of programming and
databases Alexa has or has access to. This puts them in a position to ask Alexa questions I
never would (“Alexa, when’s my mom’s birthday?”) and to ask questions about Alexa I
never would (“Why doesn’t Alexa work [on my Amazon Fire tablet] in the car?”).
We’ve lived with Alexa for a little over a year, mostly through an Amazon Echo that’s lo
cated in our primary living space—a countertop in the center of a large open-plan room
that includes both our kitchen, our den, and a table for crafts and homework. Several
months ago, we placed a Google Home Mini alongside the Echo in order to experiment
with their differing worlds and minds. Neither has been connected to any “smart home”
features, so our interaction with both has taken place entirely in informational rather
than mixed informational-physical spaces.
Alexa has a strong social presence in our household. She is always listening for her name,
and we regularly have to tell her we aren’t talking to her, especially since she sometimes
mishears my son’s name, “Elijah,” as her own name. We’ve tried changing her “wake
word” from “Alexa” to “Echo”—and, even so, she regularly mishears things as queries di
rected to her, even from television shows. In the mornings, we ask her for the weather,
and then the news, and in the afternoons we ask her to play music as we cook, clean, and
do homework.
Although her interface is audio only, she has a physical location in the kitchen in the
Echo, and when she hears her wake word, a blue light moves around the circumference of
the top of the echo to point toward the person speaking. This light serves as a face in that
it indicates “an entry point that hides interiority” (Wellner 2014, 308); a receptive “quasi-
face” (2014, 311) of an interface, like the cellphone’s screen (2014, 313). This directional
intentionality is met in kind: we have the habit of turning to face her, in her Echo, even
though the audio interface does not require that we make “eye contact” with her (Botten
berg 2015, 178–179).
Page 6 of 16
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). © Oxford University Press, 2018. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in
Oxford Handbooks Online for personal use (for details see Privacy Policy and Legal Notice).
Subscriber: New York University Libraries; date: 14 February 2021
What Is It Like to Be a Bot?
In these interactions, we experience Alexa as separate from her device and from her
device’s actions. We ask her to play something, but do not mistake her for the thing play
ing or the thing played. In radio listening, there is the physical radio and the radio station
“playing,” which we elide when we say we are “listening to the radio,” but Alexa main
tains a separation as an intermediary; she will play something on the Echo, but we can in
terrupt to talk to her. She is experienced as being in the object, and as controlling it, but
separate from it and always tarrying alongside its actions.
In using the Echo, we have been disciplined by Alexa’s ontology and programming. We
have learned specific phrases—I’ve learned to say “Alexa, ask NPR One to play the latest
hourly newscast,” since other phrases don’t seem to get her to do the right thing. My
daughter has learned that she must append “original motion picture soundtrack,” an oth
erwise unlikely phrase for a six-year-old, to her requests for Sing or My Little Pony. Using
Alexa requires adopting an intentional stance and a fictitious theory of mind, and also re
quires detailed understanding of how her mind works; how she categorizes and accesses
things. Using Alexa requires us to think about how she thinks about things; we must think
about what it’s like to be a bot.
Talking with Alexa is, of course, often frustrating, most of all for my daughter, whose
high-pitched voice and (understandably) child-like diction is not easily recognized by
Alexa. After my daughter asks Alexa something several times, I must often intervene and
ask again on her behalf. To be sure, part of this is that, having a better understanding of
the underlying mindless processes of the technical system, I am better able to move to a
second-order perspective and speak to Alexa qua voice-recognition software, sharpening
my tone and diction and carefully separating words. Part of this is surely also a reflection
of how my speech patterns, unlike hers, are firmly within the range of voices and speech
patterns on which Alexa has been trained—YouTube searches return numerous examples
of people with less common accents, especially Scottish accents, who are unable to get
Alexa to understand them unless they use impersonations of normative English or Ameri
can accents.
The Echo’s audio-only interface projects an informational space, dualistically separate
from physical reality. Alexa’s “skills” are accessed through voice commands only, and
bring her into different patterns of recognition and response—the work normally done
through conversational implicature must take place explicitly. Skills appear as conversa
tional modes or topics, where queries are understood differently when “in Spotify” is
added to the end of a question or after, for example, beginning a game of “20 questions.”
This produces a shared, shifting modulation of the intentional stance, where it is under
stood that Alexa knows that we are talking about shopping or music or a trivia game, de
pending on which skill we have accessed or which “conversation” we are “having.” The
user must learn to navigate Alexa’s informational ontology, getting to know topics she rec
ognizes and knows what to do with—“weather,” “news,” “shopping list,” or links with par
ticular apps that must be installed—and also different modes of interaction, such as
games or socialbot chat mode.
Page 7 of 16
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). © Oxford University Press, 2018. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in
Oxford Handbooks Online for personal use (for details see Privacy Policy and Legal Notice).
Subscriber: New York University Libraries; date: 14 February 2021
What Is It Like to Be a Bot?
All this speaks to the ways in which we must conceive of Alexa through the intentional
stance in order to accomplish tasks with her; how we must not only understand her as
having a mind, a mind that is not one, but we must also get to know her, how she thinks,
and how to speak to her in a way she understands. We may not ever explicitly think about
what it is like to be a bot, but we must get a sense of her world, the way she is worlded, in
order to ask her to navigate her information ontology on our behalf.
With this microphenomenology of the user’s heterophenomenology of Alexa in place, we
can now turn to the microphenomenology of alterity relations in human-technics interac
tion more generally. In doing so, we will be able to distinguish the kind of interaction with
technology that takes place through an intentional stance from other related forms of in
teracting with technology that participates in different but related kinds of “otherness.”
4. Opening the Black Box of Human-Technics
Alterity Relations
Don Ihde’s influential taxonomy of human-technics relations (1990) provides some basic
ways that human relations with worlds can be mediated by technology:
Embodiment: (I –> technology) –> world
Hermeneutic: I –> (technology –> world)
Alterity: I –> technology -(- world)
In embodiment relations, the technology disappears into the user in the user’s experience
of the world as mediated by the technology. Glasses are a clear example—when they are
well fitted to and the proper prescription for the user, the user primarily experiences
their technologically-modified field of vision as if it were not technologically mediated,
with the technology becoming an extension of the self rather than an object of experi
ence. In hermeneutic relations it is the technology and the world that merge, so that, for
example, a fluent reader experiences ideas and claims rather than printed words so much
so that that even a repeated word in a sentence may go unnoticed even by an attentive
reader.
In alterity relations, by contrast, the user’s experience is directly an experience of the
technology, and its revealing of a world may or may not be important to or present in the
user’s experience. Ihde provides several different kinds of examples. In one, he asks us to
consider driving a sports car, just for the fun of it. We might enjoy the responsiveness of
the vehicle and its power and handling, quite separately from our enjoyment of the
scenery or the utilitarian function of getting where we’re going. In another example, he
considers playing an arcade game, in which we are in a contest against fictional agents,
Space Invaders perhaps, who we seek to beat. In alterity relations, technologies present
what he calls “technological intentionalities” in a “quasi-otherness” that is rich enough
for us to experience and interact with them as others, standing on their own in their
Page 8 of 16
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). © Oxford University Press, 2018. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in
Oxford Handbooks Online for personal use (for details see Privacy Policy and Legal Notice).
Subscriber: New York University Libraries; date: 14 February 2021
What Is It Like to Be a Bot?
world rather than acting as a window to or translation of a “true” world that lies beyond
them.
If we are knowledgeable enough to “read” them, we can certainly adopt a stance which
erases this intentionality—for example, feeling the particular responsiveness of the car to
find out more about its internal mechanics, or figuring out the rules by which a computer
program moves the sprites that our spaceship-avatar-sprite “shoots”—but normal use
adopts the intentional stance; a stance in which we treat a person, object, or avatar as
having intentions and therefore adopt some kind of theory of mind.
These cases are not so different from one another in Ihde’s analysis, but they are differ
ent in a way that has become increasingly pressing in the decades since he wrote this
analysis. In the case of the sports car, we experience the technology as having a charac
ter and an intentionality based on how it makes us feel in our use of it; in the case of the
arcade game, our use of it is premised on a world and an ontology, internal to the technol
ogy, which we navigate through our perception of intentionality in its elements.
We name cars and project personalities upon them based on their brand and appearance
and ways of working or not working—or, we don’t, according to our preference. Regard
less, this layer of quasi-alterity is overlaid upon an existing world that is already com
plete, and does not require this projection. It is adopted as a kind of shorthand to under
stand a real world to which alterity bears only a metaphorical relation (“she doesn’t like
to start on cold mornings”), or as an enjoyable humanization of technologies which we de
pend upon and regularly interact with, and which might otherwise be experienced as for
eign or uncaring. These functions are often collapsed and oversimplified as “anthropo
morphism”—a vague and overbroad term which I find it easier and clearer to simply
avoid.
By contrast, it is impossible to interact with many computer games without adoption of an
intentional stance toward their elements, which we interact with through a world quite
separate from our existing world.1 If we consider more complicated games, like role-play
ing games (RPGs), we see cases where consideration of the thoughts and motivations of
non-player characters (NPCs) is necessary to game play, and we are required to “read”
these others as people, not as mere sprites and in-game instructions, in order to appropri
ately interact with an intentionality that has a programmed, dynamic, responsive struc
ture. This intentional stance is not merely projection and also is no metaphor: the facts of
a character’s name and motivations are written into the fictional world, much like facts
about fictional characters in books or films, rather than being a metaphorical or purely
fictional overlay as in the case of the car. But, unlike facts about fictional persons (insofar
as such things exist),2 NPCs interests, concerns, and character are objects of the player’s
intentional actions. A book gives us the opportunity for hermeneutic interaction with a
fictional world in which we get to know about fictional others, but RPGs can put us in an
alterity relation with minds that we must think about as actively present others in order
to successfully interact with them, and with which we engage through an embodiment re
lation with our avatar.
Page 9 of 16
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). © Oxford University Press, 2018. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in
Oxford Handbooks Online for personal use (for details see Privacy Policy and Legal Notice).
Subscriber: New York University Libraries; date: 14 February 2021
What Is It Like to Be a Bot?
In both the case of the car and the case of the game we adopt a fictitious theory of mind,
but in the former case this is merely metaphorical or make-believe, while in the latter, it
is necessary and functional. For clarity, we can refer to the pseudo-mindedness of things
in the former case as “projected minds,” and will refer to the pseudo-mindedness of
things in the latter case as minds, as above. This locution is intended to reflect that it is
non-optional to interact with these things through the category of minds, despite that
they are clearly without minds. They are “non-minds” in that they are “minds that are not
one”; they are not merely things without minds, but are minds (interactionally) that are
not minds (really).
Even in cases where it is interactionally necessary to treat unminded things as minds, we
regularly retreat into second-order cognition in which they appear as clearly unminded.
The early chatbot ELIZA provides a nice example. To interact with her and have a fun
conversation, it is necessary to talk to her as if she’s actually a psychotherapist, but her
ability to respond well to us is so limited that we have to think about her as a mere pro
gram in order to formulate and reformulate our replies to her in order to maintain the il
lusion. In RPGs, similarly, we have to adopt an intentional stance to figure out what an
NPC wants for a quest, but we may have to leave that stance in favor of a technical/pro
gramming stance in order to figure out how to complete a task by phrasing a reply in the
right way, or by having a certain item equipped rather than in our inventory, or finding a
“give” command in the interface, or so on. We can even fail to make these shifts in the
right way. Sherry Turkle (1995) has documented people taking things at “interface value”
to the extent that they found real personal insights in conversations with ELIZA, moving
into a space that seems to simultaneously approach ELIZA as a projected mind, as a non-
mind, and as a mere computer program. In massively multiplayer online role-playing
games (MMORPGs) it is sometimes possible to mistake an NPC for another player, or an
other player for an NPC.
Similarly, outside of explicitly fictional worlds, Sarah Nyberg programmed a bot to argue
with members of the alt-right on Twitter, which turned out to be highly effective, even in
spite of giving away the game a bit by naming it “@arguetron.” In what Nyberg described
as her “favorite interaction,” a (since suspended) Twitter user repeatedly sexually ha
rassed the bot, eventually asking “@arguetron so what are you wearing?” to which @ar
guetron replied “[@suspended username redacted] how are all these Julian Assange fans
finding me” (Nyberg 2016). As Leonard Foner said about a similar case, a chatbot named
Julia fending off sexual advances in the days of multi-user dungeons (MUDs), “it’s not en
tirely clear to me whether Julia passed a Turing test here or [the human interactant]
failed one” (quoted in Turkle 1995, 93).
By opening up the black box of “alterity” in alterity relations, we have seen that people
adopt the intentional stance towards unminded things for a variety of reasons: as a game,
as required by an interface, to humanize technical systems, for fun, or simply by mistake.
We have also identified two kinds of ways of adopting a fictitious theory of mind in our re
Page 10 of 16
PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). © Oxford University Press, 2018. All Rights
Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in
Oxford Handbooks Online for personal use (for details see Privacy Policy and Legal Notice).
Subscriber: New York University Libraries; date: 14 February 2021
