Table Of ContentSituated Communication
Edited by
Gert Rickheit
Ipke Wachsmuth
Mouton de Gruyter
Situated Communication
≥
Trends in Linguistics
Studies and Monographs 166
Editors
Walter Bisang
Hans Henrich Hock
Werner Winter
Mouton de Gruyter
Berlin · New York
Situated Communication
edited by
Gert Rickheit
Ipke Wachsmuth
Mouton de Gruyter
Berlin · New York
MoutondeGruyter(formerlyMouton,TheHague)
isaDivisionofWalterdeGruyterGmbH&Co.KG,Berlin.
(cid:2)(cid:2) Printedonacid-freepaperwhichfallswithintheguidelines
oftheANSItoensurepermanenceanddurability.
LibraryofCongressCataloging-in-PublicationData
Situatedcommunication/editedbyGertRickheit,IpkeWachsmuth.
p.cm.(cid:2)(Trendsinlinguistics.Studiesandmonographs;166)
Includesbibliographicalreferencesandindex.
ISBN-13:978-3-11-018897-4(hardcover:alk.paper)
ISBN-10:3-11-018897-X(hardcover:alk.paper)
1.Context(Linguistics) 2.Cohesion(Linguistics) 3.Reference
(Linguistics) 4. Psycholinguistics. 5. Computational linguistics.
I.Rickheit,Gert. II.Wachsmuth,Ipke. III.Series.
P325.5.C65S57 2006
410(cid:2)dc22
2005034470
ISBN-13: 978-3-11-018897-4
ISBN-10: 3-11-018897-X
ISSN 1861-4302
BibliographicinformationpublishedbyDieDeutscheBibliothek
DieDeutscheBibliothekliststhispublicationintheDeutscheNationalbibliografie;
detailedbibliographicdataisavailableintheInternetat(cid:3)http://dnb.ddb.de(cid:4).
”Copyright2006byWalterdeGruyterGmbH&Co.KG,D-10785Berlin
All rights reserved, including those of translation into foreign languages. No part of this
bookmaybereproducedortransmittedinanyformorbyanymeans,electronicormechan-
ical,includingphotocopy,recordingoranyinformationstorageandretrievalsystem,with-
outpermissioninwritingfromthepublisher.
PrintedinGermany.
Contents
Introduction
Gert Rickheit and Ipke Wachsmuth . . . . . . . . . . . . . . . . . . . . 1
The constitution of meaning in situated communication
Gert Rickheit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Processing instructions
Petra Weiß, Thies Pfeiffer, Hans-Jürgen Eikmeyer,
and Gert Rickheit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Visually grounded language processing in object reference
Constanze Vorwerg, Sven Wachsmuth, and Gudrun Socher . . . . . . 77
Psycholinguistic experiments on spatial relations
using stereoscopic presentation
Helmut Flitter, Thies Pfeiffer, and Gert Rickheit. . . . . . . . . . . . . 127
Deictic object reference in task-oriented dialogue
Alfred Kranstedt, Andy Lücking, Thies Pfeiffer,
Hannes Rieser, and Ipke Wachsmuth . . . . . . . . . . . . . . . . . . 155
Computational models of visual tagging
Marc Pomplun, Elena Carbone, Hendrik Koesling,
Lorenz Sichelschmidt, and Helge Ritter. . . . . . . . . . . . . . . . . . 209
Neurobiological aspects of meaning constitution
during language processing
Horst M. Müller. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
Neuroinformatic techniques in cognitive neuroscience
of language
Matthias Kaper, Peter Meinicke, Horst M. Müller,
Sabine Weiss, Holger Bekel, Thomas Herrmann,
Axel Saalbach, and Helge Ritter. . . . . . . . . . . . . . . . . . . . . . 265
vi Contents
Situated interaction with a virtual human – perception,
action, and cognition
Nadine Leßmann, Stefan Kopp, and Ipke Wachsmuth . . . . . . . . . 287
Integrated perception for cooperative human-machine
interaction
Christian Bauckhage, Gernot A. Fink, Jannik Fritsch,
Nils Jungclaus, Susanne Kronenberg, Franz Kummert,
Frank Lömker, Gerhard Sagerer, and Sven Wachsmuth. . . . . . . . . 325
Architectures of situated communicators: From perception
to cognition to learning
Gernot A. Fink, Jannik Fritsch, Nadine Leßmann,
Helge Ritter, Gerhard Sagerer, Jochen J. Steil,
and Ipke Wachsmuth . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357
A systems framework of communicative understanding
Hans-Jürgen Eikmeyer, Walther Kindt,
Yvonne Rittgeroth, and Hans Strohner . . . . . . . . . . . . . . . . . . 377
System theoretical modeling on situated communication
Hans-Jürgen Eikmeyer, Walther Kindt,
and Hans Strohner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409
Index of names . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435
Subject index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443
Introduction
Gert Rickheit and Ipke Wachsmuth
This volume contains a selection of studies from the Collaborative Research
Center (CRC) “Situated Artificial Communicators”. The Research Center
(Sonderforschungsbereich 360) was established in 1993 and has been funded
by grants from the German Research Foundation (Deutsche Forschungsge-
meinschaft) for more than twelve years. The research initiative has brought
together computer scientists, linguists, psycholinguists, and psychologists in
an endeavor to investigate human-human and human-machine interaction in
situations which closely model everyday workplace demands.
The Collaborative Research Center comprises a number of basic research
projects and several projects of a more applied character. While the basic
research projects have dealt with the study and explanation of linguistic and
cognitive characteristics of communication, the applied projects have pur-
sued the transmission and implementation of cognitive principles and their
utilization in artificial information processing systems. In the course of the
scientific collaboration, a strong interaction between projects in these two
strands of research has evolved. The starting point of the empirical research
has been the attempt to corroborate or refute theoretically derived hypothe-
ses by means of systematic experimentation. Results from the experiments
have then been validated in computer simulations of cognitive processing.
These models have led to more specific hypotheses which have been put to
test in further experiments, and so on. In this way – on the basis of what we
have termed the experimental-simulative method – we have successfully
developed information processing systems with cognitively adequate appli-
cation functionalities.
In the CRC, phenomena such as flexibility and robustness, the situation
and knowledge dependence of language and image processing, and their
integration in communication have been examined in much detail. In doing
so, questions like the following have been pursued:
– How can irregular, fragmented, and distorted information be so easily and
adequately assigned to definite states?
– What are the cognitive criteria in the organization and selection of do-
main knowledge relevant to the situation in question?
2 Gert Rickheit, Ipke Wachsmuth
– What is the role of the situational and cognitive context in the processing
of information?
– How can the interaction between perception and knowledge be organized
in order to facilitate adequate understanding?
– Which recommendations for the design of integrated communication
systems can be derived from experimentation and simulation?
Within the framework of the Collaborative Research Center, these questions
have been investigated in an interdisciplinary fashion in thematically repre-
sentative projects which support and complement each other in terms of
theory and methodology. In recent years, diverse research efforts in cogni-
tive science have made clear that, while interdisciplinary cooperation is vital
for the development of theoretical and formal models; scientific progress has
as prerequisites a close coupling to a particular problem area and a solid
empirical foundation. Drawing on this experience, all the projects in the
CRC are tied to a common scenario and subscribe to a joint methodological
basis. Such a research strategy does not only foster the establishment of a
common empirical basis for the participating projects; it also allows for the
results to be systematically related to one another.
The innovation potential of cognitive science arises from the cooperation
of the various disciplines. A truly interdisciplinary approach will bring about
synergy, and thus give rise to highly creative solutions in terms of theory
formation and methodology. In contemporary cognitive science, theoretical
conceptualizations which yield progress in several disciplines are being pro-
posed with increasing frequency. Within the scope of these developing theo-
ries, empirical and formal methods are increasingly interlinked. Building on
this, computer simulation can be applied for the purpose of validating theory.
Such an interweaving of experimental and simulative work is the hallmark of
the CRC: Here, the experimental-simulative method is systematically ap-
plied and further developed with respect to situated communication.
The technological relevance of research in knowledge-based language
and image processing stems from its application potential in the areas of
information, communication, and education. In fact, natural language sys-
tems – to some extent – have already transcended the experimental stage.
The mission behind this is the advancement of human-machine communica-
tion in natural language. Since most of the information in mass media is
language coded, adequate design will make available various computer ser-
vices to society. The information overflow which we are likely to encounter
in the near future brings along a growing need to handle, as adequately and
efficiently as possible, the knowledge accrued and to be conveyed.
Introduction 3
The volume in hand comprises thirteen contributions which focus on special
aspects of situated communication. The first chapter, “The constitution of
meaning in situated communication” by Gert Rickheit, the CRC coordinator,
introduces the concept of situated artificial communicators as jointly devel-
oped by the members of the research initiative. In the chapter, the basic sce-
nario of the CRC is outlined: During the cooperative accomplishment of
assembly tasks, different participants with different competences use verbal
and non-verbal means in order to coordinate their sensorimotor activities.
Since situated artificial communicators are a reconstruction of natural com-
municators interacting in specific situations, the characteristics of situated-
ness, integration, and robustness of the artificial systems have been system-
atically examined.
The chapter “Processing instructions”, authored by Petra Weiß, Thies
Pfeiffer, Hans-Jürgen Eikmeyer, and Gert Rickheit, presents a closer look at
the processing of instructions in the context of an assembly task domain. The
results of pertinent experiments show that and how the interpretation of in-
structions is determined by the intricate interplay of linguistic information
and the visual context. Comparing the performance of both systems, human
and machine, they found that the performance depends partly on the struc-
ture of the problem domain and partly on the structure of the conceptual
knowledge and the processes working therein.
In their account of “Visually grounded language processing in object ref-
erence”, Constanze Vorwerg, Sven Wachsmuth, and Gudrun Socher describe
how representations of verbal and visual information are mapped onto each
other in human communicators and how such a mapping can be achieved by
artificial systems communicating with humans. At that, categorization prin-
ciples play a major role since categorization constitutes the connecting link
between vision and language. Categorization includes the assignment of a
given perceptual input to a category that is associated with a linguistic term.
So, when referring to visual object attributes, a speaker deploys at least two
kinds of processes within conceptualization: the selection of an attribute
dimension, and the categorization of dimension values on the basis of a ref-
erence system.
Helmut Flitter, Thies Pfeiffer, and Gert Rickheit show that „Psycholingu-
istic experiments on spatial relations using stereoscopic presentation“ yield
results that differ from those of 2½D experiments which make use of per-
spectival pictures. Proceeding from the assumption that the processing of 3D
pictures requires different and possibly higher cognitive demands than the
processing of 2½D pictures, the authors have observed a wider variety of
answers for descriptions of spatial locations given under 3D presentation
