Table Of ContentLecture Notes in Artificial Intelligence 1195
Subseries of Lecture Notes in Computer Science
Edited by J. G. Carbonell and J. Siekmann
Lecture Notes in Computer Science
Edited by G. Goos, J. Hartmanis and J. van Leeuwen
Robert Trappl Paolo Petta (Eds.)
Creating Personalities
for Synthetic Actors
Towards Autonomous
Personality Agents
Springer
Series Editors
Jaime G. Carbonell, Carnegie Mellon University, Pittsburgh, PA, USA
J6rg Siekmann, University of Saarland, Saarbrticken, Germany
Volume Editors
Robert Trappl
Paolo Petta
Austrian Research Institute forArtificial Intelligence
S chottengasse 3, A- 1010 Vienna, Austria
E-mail: robert @a i.univie.ac.at
P r e f a c e
Progress in computer animation has attained such a speed that computer-
generated human faces and figures on ubiquitous screens wilt soon be indistin-
guishable from those of real humans. The potential both for scripted films and
real-time interaction with users is enormous. However, in order to cope with this
potential, these faces and figures must be guided by autonomous "personality
agents". But what is the current state of the art in this far less visible domain
of research?
To answer this question, we decided to organise a two-day workshop which
took place at the premises of the Austrian Research Institute for Artificial Intel-
ligence in Vienna. To this workshop we invited scientists known for their work
in this area, ranging from computer animation specialists, computer scientists,
and specialists in AI, to psychologists and philosophers.
This book is the collection of the papers which were presented as first drafts
at that occasion. The lively discussion was taped, transcribed, and sent to the
participants. We are extremely grateful to them for the pains they took to con-
sider the comments, to enhance their papers with new material, and to submit
them on time. At least one author of each of the papers presented in this book
was present at our workshop, with the sole exception of Barbara Hayes-Roth,
who regrettably had already accepted other duties when the date of the event
was set.
We want to thank all our colleagues at the Austrian Research Institute for
Artificial Intelligence, especially Dr. Bernhard Pfahringer, who was not only very
instrumental in the scientific preparation and organization of the workshop but
also proposed its name, and thus the title of this book, and to Gerda Helscher,
MA, for preparing the transcripts of the tape recordings and for her help in the
finalization of this volume. Both Prof. JSrg Siekmann, one of the LNAI series
editors, and Mr. Alfred Hofmann of Springer-Verlag were ideal partners in our
endeavor.
We are grateful to Dr. Erhard Busek, then Federal Minister for Science and
Research, for his support of an agent project of which the workshop was an
integral part, and to Dr. Ren~ Fries from the same Ministry, who took great care
to keep the project going. We hope we will get to be as grateful to Dr. Caspar
Einem, now Federal Minister for Science and Transport, since we have applied
to his Ministry for its financial support of a research project on the development
of personality agents.
It is our hope that this book will serve as a useful guide to the different
approaches to create personalities for synthetic actors and to stimulate future
research in this promising field.
Vienna, Austria, January 1997 Robert Trappl and Paolo Pet ta
C o n t e n t s
W h y t o C r e a t e P e r s o n a l i t i e s fo r S y n t h e t i c A c t o r s . . . . . . . . . . . . . . . . . . . . 1
Paolo Pe t ta and Robert Trappl
D r e s s i n g V i r t u a l H u m a n s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Nadia Magnenat Tha lmann and Pascal Volino
A u t o n o m o u s V i r t u a l A c t o r s B a s e d o n V i r t u a l S e n s o r s . . . . . . . . . . . . . 25
Daniel Thalmann, Hansrudi Noser, and Zhiyong Huang
T o w a r d s P e r s o n a l i t i e s fo r A n i m a t e d A g e n t s
w i t h R e a c t i v e a n d P l a n n i n g B e h a v i o r s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
Norman I. Badler, Barry D. Reich, and Bonnie L. Webber
I M P R O V : A S y s t e m fo r R e a l - T i m e A n i m a t i o n
o f B e h a v i o r - B a s e d I n t e r a c t i v e S y n t h e t i c A c t o r s . . . . . . . . . . . . . . . . . . . . . 58
Athomas Goldberg
M u l t i - l e v e l C o n t r o l fo r A n i m a t e d A u t o n o m o u s A g e n t s :
D o t h e R i g h t T h i n g . . . Oh , N o t T h a t . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
Bruce Blumberg and Tinsley Galyean
Too l s fo r a n I n t e r a c t i v e V i r t u a l C i n e m a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
David Blair and Tom Meyer
A c t i n g in C h a r a c t e r . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
Barbara Hayes-Roth, Robert van Gent, and Daniel Huber
S o m e R e q u i r e m e n t s a n d A p p r o a c h e s
fo r N a t u r a l L a n g u a g e in a B e l i e v a b l e A g e n t . . . . . . . . . . . . . . . . . . . . . . . . 113
A. Bryan Loyall
P e r s o n a l i t y P a r a m e t e r s a n d P r o g r a m s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
Dave Moffat
W h a t S o r t o f C o n t r o l S y s t e m Is A b l e t o H a v e a P e r s o n a l i t y ? . . . . . 166
Aaron Sloman
P e r s o n a l i t i e s fo r S y n t h e t i c A c t o r s :
C u r r e n t I s s u e s a n d S o m e P e r s p e c t i v e s . . . . . . . . . . . . . . . . . . . ~. . . . . . . . . . . 209
Paolo Pet ta and Robert Trappl
P e r s o n a l i t i e s fo r S y n t h e t i c A c t o r s : A B i b l i o g r a p h y . . . . . . . . . . . . . . . . 219
Paolo Pet ta
I n d e x
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Why to Create Personalit ies
for Synthetic Actors
Paolo Petta, Robert Trappl
Austrian Research Institute for Artificial Intelligence*, Vienna, and
Department of Medical Cybernetics and Artificial Intelligence, University of Vienna
{paolo, robert }@ ai.univie.ac. at
1 I n t r o d u c t i o n
The last decade has seen not only the impressive developments from the first brief
appearance of a computer-generated character in a feature film 2, over the first
digital performer in a movie 3, all the way to the first fully computer-generated
theatrical motion picture4; synthetic actors have also conquered demanding real-
t ime domains from the (co-)moderation of live television shows to the evaluation
of complex real-world scenarios modded in detail in advanced simulators.
Over this timespan the rendering of the external appearance has progressed
from the basic geometrical and physical modeling and animation over the inclu-
sion of dependencies on the simulated environment to event- and object-based
behaviours. As synthetic actors were placed in simulated worlds of growing com-
plexity, an obvious requirement that came about was to make them perform
in these worlds: for convincing realistic behaviour they ought to perform as if
they were gathering information as it became available to them and they should
interact with objects - - e.g., tracking, avoiding, manipula t ing , . . . them - - in
plausible ways. It soon became evident that the first approach of setting out
from an omniscient position where all the data of the simulated world was made
available to programs driving the actors was very inefficient in a number of re-
spects. For instance, those pieces of information that could actually be accessed
given the actor's current state in the world had to be laboriously identified and
progressively singled out. Similarly, all sequences of actions making up basic
behaviours of an agent had to be explicitly prescribed: take as an example an
object "catching the eye" of the actor which would then turn around to inspect
it more closely. Instead, taking the step forward and equipping actors with vir-
tual sensors, turning their bodies into virtual effectors, and using these devices
to guide the execution of behaviours proved to be a superior alternative, with
* The Austrian Research Institute for Artificial Intelligence is supported by the Aus-
trian Federal Ministry of Science, Transport, and the Arts.
Young Sherlock Holmes, directed by Barry Levinson, visual effects by Industrial
Light & Magic.
3 Casper, directed by Brad Silberling, visual effects by Industrial Light & Magic.
4 Toy Story, directed by John Lasseter, visual effects by Pixar.
the additional benefit of introducing a well defined interface between the actor
and its environment, allowing to deploy actors in different worlds 5.
However, all of these external aspects still cover only part of what makes
up an actor: as entities tha t "act a part or take part in an affair", synthetic
actors also have to be endowed with internal capabilities, both affective and
cognitive--i .e. , in concordance with the following definition by OshersonlO
Cognitive Science is the study of human intelligence in all of its forms,
from perception and action to language and reasoning. The exercise of
intelligence is called cognition. Under the rubric of cognition fall such di-
verse human activities as recognizing a friend's voice over the telephone,
reading a novel, jumping from stone to stone in a creek, explaining an
idea to classmates, remembering the way home from work, and choosing
a profession. Cognitive processes are essential to each of these activities;
indeed, they are essential to everything we do. p.ix
From the point of view of the different applications, the actual require-
ments calling for these internal faculties can vary widely: feature film animators
might just want to be able to instruct their puppets using high level commands
(e.g., "walk leisurely across the room") or be assisted in the more delicate and
ephemeral task of ensuring consistent pat terns of behaviour of a vir tual actor
over a longer t ime span; in other settings, achieving e.g. various degrees of agent
au tonomy can play an essential role in providing effective assistance to users or
in achieving credible and interesting performance in virtual environments. Espe-
cially in this latter context, the inevitable limitations in terms of incompleteness
of information about the world tha t can be perceived by the actor via its vir-
tual sensors as well as restrictions in terms of available resources 6 become of
particular relevance. Different approaches have been taken to tackle this prob-
lem. Some favour runtime performance and acceptance by a broad public even
though adhering to scientifically disproved preconceptions of folk psychology - -
as in the Oz project directed by Joseph Bates at Carnegie Mellon University.
Others place the emphasis on the soundness of the implementat ion of a cognitive
theory resting on plausible first principles, as illustrated in the chapters by Dave
Moffat and Aaron Sloman in this volume. Also, it must not be overlooked tha t
as the exterior architecture of agents and their environments become more com-
plex and structured, control of these worlds necessarily also becomes of a more
indirect, higher-level nature, and consequently has to rely on the availability of
interior mechanisms of corresponding functionality.
All of these circumstances thus give rise to concerns regarding the characteris-
tics of the high-level behaviour of the synthetic actors, both in terms of reliability
5 A related effort which gained a certain publicity was the work carried out by an
undergraduate student at the MIT Media Lab who interfaced their ALIVE system
to the game world of DOOM. See 12, 3 for a more detailed discussion of these
topics.
Computational resources and timing constraints, especially for characters '%hrown"
into worlds where they have to act in real time.
and predictabili ty--ensuring that the performance remains within anticipated or
prescribable l imits--and interestingness--the manifestation of individual distin-
guishing marks due to which the actors become recognizable individuals: in other
words, the design of tailored personalities. The papers included in the present
collection span the range from theoretical to applied and application-specific as-
pect.s, laying a foundation for the creation of multifarious and more believable
synthetic actors. From its very beginning the event at which these works were
presented was characterized by an informal atmosphere which engendered an-
imated exchanges of opinions. The results of these discussions are reflected in
the contents of the revised versions of the workshop papers, tn the following
we will briefly summarize some key points presented in the single contributions,
deferring a principled discussion of the subject domain to the final chapter of
this volume.
2 C o n t e n t s
2.1 V i r t u a l H u m a n s
An essential point of emphasis of the research supervised by Nadia Magnenat-
Thalmann and Daniel Thalmann at the MIRALab in Geneva and the EPFL in
Lausanne is the pursuit of the highest qualitative standards (i.e., photorealism)
of the appearance of their creations: achievement of flexible real-time behaviour is
subordinated to this primary goal. Consequently, special consideration is given
e.g. to the simulation of different surfaces, such as clothes made of different
fabrics and materiMs, or human hair and skin 7. Similarly, single courses of
actions--such as the grasping of objects or the facial expressions--are modeled
in accurate detail 12. These latter procedures also rely on the last focus of re-
search to be mentioned here, namely the synthetic sensing of the environment. In
particular this includes the virtual vision, tactile, and audition, which contribute
decisively to a "natural" behaviour of the virtual actors. Most Of this research is
strongly application-oriented, with telecooperation--following the maxim that
the realistic representation of Avatars 7 has a crucial impact of the acceptance of
the sys tem--and the entertainment industries (e.g., the co-moderation of prime
time shows) as important application areas.
The research efforts presented by Bonnie Webber and Norman Badler 1
also direct their attention on realism: the work centered on
behaviour, making Jack capable of e.g., "stumbling" over obstacles s. At the
highest level, the behaviour of all instances of Jack in one running simulation
are centrolied by parallel transition net~orks, in which all possible courses of
action for a given scene are defined in an abstract manner (roughly comparable
to Roger Schank's scripts): to solve occurring partial problems, specialized prob-
lem solving methods can be activated as required (e.g., heuristics and planning
procedures to chose a hiding place in the context of a game of hide-and-seek).
At New York University's Media Research Lab (NYU MgL) an alternative
approach is taken for a similar subject 4. In the framework of a procedural
animation technique, single movements of human figures are defined along with
parameter settings for all possible transitions between these actions. As eval-
uation criterion for this latter step the assessment of the "naturalness" of the
resulting impression as given by human experts was drawn up. This exact mod-
eling is subsequently softened by the superposition of stochastic noise functions,
so that each repetition of a single movement is performed with slight variations;
but also when at a standstill, the joints of the figures thereby feature a slight
"unrest", which contributes to the realistic appearance.
Given that in this way the single movements can be recalled in an efficient
manner, it is possibie to control the whole animation at ~ higher level in real
time. This functionality is provided by behavioural scripts, which define the
"vocabulary" of each object (actors as well as any other entities comprised in
the scene). Instead of equipping each actor e.g. with comprehensive knowledge
about all possible actions, the respective relevant information is thereby stored
in a distributed fashion (a tea cup "knows", that it can be sipped from, a ve-
hicle knows that it can be mounted and driven, etc.). Among the opportunities
arising from this approach, it is also easily possible to achieve the effect of a
stepwise acquisition of skills in mastering recurrent tasks by instantiating and
updating variables stored with the given actor. The considered application sce-
narios for this technology include the interactive design of virtual choreographies
and virtual sociM environments akin to graphicM Internet MUD (Multi-User Di-
mensions) and their more recent, commercial variants.
2.2 V i r t u a l E t h o l o g y
Bruce Blumberg and Tinsley Galyean 3 present further developments of the
Hamsterdam architecture which is being applied in the cont, ext of the ALIVE
(ArtificiM Life Interactive Video Environment) project at the MIT Media Lab.
ALIVE implements the metaphor of a "magic mirror" which allows users to
immerse in a virtual environment without requiring them to wear any special
equipment: this mirror is realized as a wide-area screen on which the image of
the user as acquired by video cameras is displayed after being composited with
As ~n interesting side note, the use of reinforcement learning Mgorithms was found
to be of indispensable help in the adjustment of the numerous p~r~meters of this
complex numerical system.
