Table Of ContentLecture Notes in Computer Science 5237
CommencedPublicationin1973
FoundingandFormerSeriesEditors:
GerhardGoos,JurisHartmanis,andJanvanLeeuwen
EditorialBoard
DavidHutchison
LancasterUniversity,UK
TakeoKanade
CarnegieMellonUniversity,Pittsburgh,PA,USA
JosefKittler
UniversityofSurrey,Guildford,UK
JonM.Kleinberg
CornellUniversity,Ithaca,NY,USA
AlfredKobsa
UniversityofCalifornia,Irvine,CA,USA
FriedemannMattern
ETHZurich,Switzerland
JohnC.Mitchell
StanfordUniversity,CA,USA
MoniNaor
WeizmannInstituteofScience,Rehovot,Israel
OscarNierstrasz
UniversityofBern,Switzerland
C.PanduRangan
IndianInstituteofTechnology,Madras,India
BernhardSteffen
UniversityofDortmund,Germany
MadhuSudan
MassachusettsInstituteofTechnology,MA,USA
DemetriTerzopoulos
UniversityofCalifornia,LosAngeles,CA,USA
DougTygar
UniversityofCalifornia,Berkeley,CA,USA
GerhardWeikum
Max-PlanckInstituteofComputerScience,Saarbruecken,Germany
Andrei Popescu-Belis
Rainer Stiefelhagen (Eds.)
Machine Learning for
Multimodal Interaction
5th International Workshop, MLMI 2008
Utrecht, The Netherlands, September 8-10, 2008
Proceedings
1 3
VolumeEditors
AndreiPopescu-Belis
IdiapResearchInstitute,CentreduParc
RueMarconi19,CasePostale592,1920,Martigny,Switzerland
E-mail:[email protected]
RainerStiefelhagen
UniversitätKarlsruhe(TH),InteractiveSystemsLabs
AmFasanengarten5,76131Karlsruhe,Germany
E-mail:[email protected]
LibraryofCongressControlNumber:2008933844
CRSubjectClassification(1998):H.5.2-3,H.5,I.2.6,I.2.10,I.2,D.2,K.4,I.4
LNCSSublibrary:SL3–InformationSystemsandApplication,incl.Internet/Web
andHCI
ISSN 0302-9743
ISBN-10 3-540-85852-0SpringerBerlinHeidelbergNewYork
ISBN-13 978-3-540-85852-2SpringerBerlinHeidelbergNewYork
Thisworkissubjecttocopyright.Allrightsarereserved,whetherthewholeorpartofthematerialis
concerned,specificallytherightsoftranslation,reprinting,re-useofillustrations,recitation,broadcasting,
reproductiononmicrofilmsorinanyotherway,andstorageindatabanks.Duplicationofthispublication
orpartsthereofispermittedonlyundertheprovisionsoftheGermanCopyrightLawofSeptember9,1965,
initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.Violationsareliable
toprosecutionundertheGermanCopyrightLaw.
SpringerisapartofSpringerScience+BusinessMedia
springer.com
©Springer-VerlagBerlinHeidelberg2008
PrintedinGermany
Typesetting:Camera-readybyauthor,dataconversionbyScientificPublishingServices,Chennai,India
Printedonacid-freepaper SPIN:12513789 06/3180 543210
Preface
TheseriesofworkshopsonMachineLearningforMultimodalInteraction(MLMI)
celebratesthisyearitsfifthanniversary.Onthisoccasion,anumberofinnovations
havebeenintroducedinthereviewingandpublicationprocedures,whilekeeping
thefocusonthesamescientifictopics.
For the first time, the reviewing process has been adapted in order to pre-
paretheproceedingsintimefortheworkshop,heldonSeptember8–10,2008,in
Utrecht, The Netherlands. The 47 submissions received by the Program Com-
mittee were first reviewed by three PC members each, and then advocated by
an Area Chair. Overall, 12 oral presentations (ca. 25% of all submissions) and
15 poster presentations were selected. Authors were given one month to revise
theirpapersaccordingtothe reviews,andthefinalversionswerebrieflychecked
by the two ProgramCo-chairs.Both types of presentation have been give equal
space in the present proceedings.
The 32 papers gathered in this volume cover a wide range of topics re-
lated to human-human communication modeling and processing, as well as to
human-computer interaction, using severalcommunication modalities. A signif-
icantnumber ofpapersfocus onthe analysisofnon-verbalcommunicationcues,
such as the expression of emotions, laughter, face turning, or gestures, which
demonstrates a growing interest for social signal processing. Yet, another large
set ofpapers targets the analysisof communicativecontent, with a focus onthe
abstractionofinformationfrommeetingsintheformofsummaries,actionitems,
ordialogueacts.OthertopicspresentedatMLMI2008includeaudio-visualscene
analysis, speech processing, interactive systems and applications.
Aspecialsessiononuserrequirementsandevaluationofmultimodalmeeting
browsers/assistants was organized, with a separate submission track, resulting
in the five papers gatheredin the lastpartofthese proceedings.The goalofthe
session was to put together the lessons learned from several large projects in-
volvingmeeting technology– partly the same projectsthat launchedthe MLMI
series. The session included short presentations of user requirements and evalu-
ation studies, introduced by a related keynote talk, and followed by a plenary
discussion.
MLMI2008featuredfourkeynotetalks,focusingontheory,applications,and
evaluation. The organizers would like to express their gratitude to the invited
speakers:Maja Pantic, from the Imperial College London and the University of
Twente;Catherine Pelachaud,fromthe MontreuilUniversity ofTechnologyand
INRIA Rocquencourt; Mark Sanderson, from the University of Sheffield; and
Stephen Von Rump, CEO of HeadThere, Inc., San Francisco, CA.
The number of MLMI satellite events keeps growing every year. The fol-
lowing events were associated to MLMI 2008: the AMI Career Day, an op-
portunity for young scientists to talk to representatives of companies working
VI Preface
on meeting technology and prepare the next steps of their careers; the above-
mentioned special session on user requirements and evaluation of multimodal
meeting browsers/assistants;a workshopon the evaluation of automatic speech
recognition systems for Dutch; an interproject meeting on the evaluation of
space-time audio processing; and a student poster session.
Toconclude,wewouldliketo warmlythankallthe members ofthe Program
Committee,the AreaChairs,andthe people involvedinthe workshoporganiza-
tion,inparticularDavidvanLeeuwen(TNO).WearegratefultoHervéBourlard
(Idiap Research Institute) and Steve Renals (University of Edinburgh) for their
constant support. MLMI 2008 acknowledges sponsoring from the AMIDA Inte-
grated Project, which is funded by the European Commission under the Infor-
mationSocietyTechnologiespriorityofthesixthFrameworkProgram,andfrom
the IM2 National Center of Competence in Research, which is funded by the
Swiss National Science Foundation.
June 2008 Andrei Popescu-Belis
Rainer Stiefelhagen
Organization
Organizing Committee
Andrei Popescu-Belis Idiap Research Institute (Program Co-chair)
Rainer Stiefelhagen University of Karlsruhe (Program Co-chair)
David van Leeuwen TNO (OrganizationChair)
Anton Nijholt University of Twente (Special Sessions Chair)
Workshop Organization
Songfang Huang University of Edinburgh (Proceedings)
Jonathan Kilgour University of Edinburgh (Webmaster)
Marjolein Klootwijk TNO (Secretary)
Suzanne Schrijvers-van der Klip TNO (Secretary)
Josef Žižka Brno University of Technology (Webmaster)
Program Committee
Jan Alexandersson DFKI
Tilman Becker DFKI
Samy Bengio Google Inc.
Hervé Bourlard Idiap Research Institute
Lukas Burget Brno University of Technology
Nick Campbell ATR
Jean Carletta University of Edinburgh
Rolf Carlson KTH
Jan “Honza” Černocký Brno University of Technology
Trevor Darrell MIT
John Dines Idiap Research Institute
Gerald Friedland ICSI
Sadaoki Furui Tokyo Institute of Technology
John Garofolo NIST
Daniel Gatica-Perez Idiap Research Institute
Thomas Hain University of Sheffield
Mary Harper University of Maryland
James Henderson University of Geneva
Hynek Hermansky Idiap Research Institute
Simon King University of Edinburgh
Michael Kipp DFKI
Denis Lalanne University of Fribourg
David van Leeuwen TNO (Organization Chair)
VIII Organization
Yang Liu University of Texas at Dallas (Area Chair)
Stéphane Marchand-Maillet University of Geneva
Jean-Claude Martin LIMSI-CNRS
Helen Meng Chinese University of Hong Kong
Luděk Müller University of West Bohemia
Anton Nijholt University of Twente (Area Chair)
Fabio Pianesi FBK-IRST (Area Chair)
Andrei Popescu-Belis Idiap Research Institute (Program Co-chair)
Wilfried Post TNO
Gerasimos Potamianos IBM T.J. Watson (Area Chair)
Steve Renals University of Edinburgh
Gerhard Sagerer University of Bielefeld (Area Chair)
Nicu Sebe University of Amsterdam
Elizabeth Shriberg SRI and ICSI
Pavel Smrž Brno University of Technology
Rainer Stiefelhagen University of Karlsruhe (Program Co-chair)
Jean-Philippe Thiran EPFL
Matthew Turk UCSB
Enrique Vidal Universidad Politécnica de Valencia
Steve Whittaker University of Sheffield
Jie Yang CMU
Sponsoring Programs, Projects and Institutions
Programs:
– European Commission, through the Multimodal Interfaces objective of the
Information Society Technologies (IST) priority of the sixth Framework
Program.
– Swiss NationalScience Foundation,throughthe NationalCenter ofCompe-
tence in Research (NCCR) Program.
Projects:
– AMIDA, Augmented Multiparty Interaction with Distance Access,
http://www.amiproject.org
– IM2, Interactive Multimodal Information Management, http://www.im2.ch
Institutions:
– TNO, Netherlands Organisationfor Applied Scientific Research,
http://www.tno.nl
– SIKS, Dutch Research School for Information and Knowledge Systems,
http://www.siks.nl
– FIT BUT, Faculty of Information Technology, Brno University of Technol-
ogy, http://www.fit.vutbr.cz
Table of Contents
I Face, Gesture and Nonverbal Communication
Visual Focus of Attention in Dynamic Meeting Scenarios.............. 1
Michael Voit and Rainer Stiefelhagen
Fast and Robust Face Tracking for Analyzing Multiparty Face-to-Face
Meetings........................................................ 14
Kazuhiro Otsuka and Junji Yamato
What Does the Face-Turning Action Imply in Consensus Building
Communication?................................................. 26
Tetsuro Onishi, Takatsugu Hirayama, and Takashi Matsuyama
Distinguishing the Communicative Functions of Gestures: An
Experiment with Annotated Gesture Data .......................... 38
Kristiina Jokinen, Costanza Navarretta, and Patrizia Paggio
Optimised Meeting Recording and Annotation Using Real-Time Video
Analysis ........................................................ 50
Paul Chippendale and Oswald Lanz
Ambiguity Modeling in Latent Spaces .............................. 62
Carl Henrik Ek, Jon Rihan, Philip H.S. Torr, Gr´egory Rogez, and
Neil D. Lawrence
II Audio-Visual Scene Analysis and Speech Processing
Inclusion of Video Information for Detection of Acoustic Events Using
the Fuzzy Integral................................................ 74
Taras Butko, Andrey Temko, Climent Nadeu, and Cristian Canton
Audio-Visual Clustering for 3D Speaker Localization ................. 86
Vasil Khalidov, Florence Forbes, Miles Hansard, Elise Arnaud, and
Radu Horaud
A Hybrid Generative-Discriminative Approach to Speaker
Diarization...................................................... 98
Athanasios K. Noulas, Tim van Kasteren, and Ben J.A. Kro¨se
X Table of Contents
A Neural Network Based Regression Approach for Recognizing
Simultaneous Speech ............................................. 110
Weifeng Li, Kenichi Kumatani, John Dines,
Mathew Magimai-Doss, and Herv´e Bourlard
Hilbert Envelope Based Features for Far-Field Speech Recognition ..... 119
Samuel Thomas, Sriram Ganapathy, and Hynek Hermansky
Multimodal Unit Selection for 2D Audiovisual Text-to-Speech
Synthesis ....................................................... 125
Wesley Mattheyses, Lukas Latacz, Werner Verhelst, and
Hichem Sahli
III Social Signal Processing
Decision-Level Fusion for Audio-Visual Laughter Detection............ 137
Boris Reuderink, Mannes Poel, Khiet Truong, Ronald Poppe, and
Maja Pantic
Detection of Laughter-in-Interaction in Multichannel Close-Talk
Microphone Recordings of Meetings ................................ 149
Kornel Laskowski and Tanja Schultz
Automatic Recognition of Spontaneous Emotions in Speech Using
Acoustic and Lexical Features ..................................... 161
Khiet P. Truong and Stephan A. Raaijmakers
Daily Routine Classification from Mobile Phone Data................. 173
Katayoun Farrahi and Daniel Gatica-Perez
IV Human-Human Spoken Dialogue Processing
Hybrid Multi-step Disfluency Detection............................. 185
Sebastian Germesin, Tilman Becker, and Peter Poller
Exploring Features and Classifiers for Dialogue Act Segmentation ...... 196
Harm op den Akker and Christian Schulz
Detecting Action Items in Meetings ................................ 208
Gabriel Murray and Steve Renals
Modeling Topic and Role Information in Meetings Using the
Hierarchical Dirichlet Process...................................... 214
Songfang Huang and Steve Renals
Time-Compressing Speech: ASR Transcripts Are an Effective Way to
Support Gist Extraction .......................................... 226
Simon Tucker, Nicos Kyprianou, and Steve Whittaker
