Table Of ContentResearch Reports ESPRIT
Project 5345 . DIMUS . Volume 1
Edited in cooperation with
the Commission of the European Communities
S. Pfleger J. Goncalves D. Vernon (Eds.)
Data Fusion Applications
Workshop Proceedings
Brussels, November 25, 1992
Springer-Verlag
Berlin Heidelberg New York
London Paris Tokyo
Hong Kong Barcelona
Budapest
Volume Editors
S. Pfleger
Technical University of Munich
Orleansstr. 34, D-81667 Munich, Germany
J. Gonc;:alves
Commission of the European Communities
Joint Research Center, 1-21020 Ispra, Italy
D. Vernon
Commission of the European Communities
Rue de la Loi 200, B-1049 Brussels, Belgium
ESPRITProject 5345 "Data Integration in MultisensorSystems (DIMUS)" belongs
to the Subprogramme "Information Processing Systems and Software" of
ESPRIT, the European Strategic Programme for Research und Development in
Information Technology supported by the Comission of the European Communi
ties.
The project investigates the advanced technology of multisensoring in order to
obtain correct and complete scene information in monitoring and control appli
cations with safety-critical constraints. The objective is to design and develop a
system supporting a human operator in the interpretation of situations occurring
in a metro (subway) station enviroment. A prototype system is being developed
containing a set of tools for the integration and fusion of visual and non-visual
data provided by different kind of sensors.
CR Subject Classification (1991): H.4, 1.0, K.6
ISBN-13: 978-3-540-56973-2 e-ISBN-13: 978-3-642-84990-9
DOl: 10.1007/978-3-642-84990-9
Publication No. EUR 15247 EN of the
Commission of the European Communities, Dissemination of Scientific and Technical Knowhow Unit,
Directorate General Information Technologies and Industries, and Telecommunications,
Luxembourg
LEGAL NOTICE
Neither the Commission of the European Communities nor any person acting on behalf of the
Commission is responsible for the use which might be made of the following information.
© ECSC - EEC - EAEC, Brussels - Luxembourg, 1993
Typesetting: Camera-ready by authors
45/3140 -543210 - Printed on acid-free paper
Foreword
The 1992 ESPRIT Conference and Exhibition which was held in Brussels during
the week of November 23rd to 27th was remarkable for the number of technical
workshops which it hosted. One of these workshops was devoted to the topic of
"Data Fusion Applications" and its success is indicative of the healthy rapport
which now exists between the industrial practitioners on the one hand and the
research community on the other. ESPRIT - the European Strategic Programme
for Research and Development in Information Technology -has played no small
part in fostering this relationship. Data Fusion itself is, and will remain, an
important topic in information technology. The reason is straightforward: as we
attempt to expand the application domains of information technology to
encompass ever more sophisticated systems and environments, the data which is
derived from a single source cannot be assumed to be adequate to characterize
fully the application domain. The ability to combine data derived from several
sources, data fusion, to provide a coherent, informative and useful
characterization then assumes a key role in the development of robust IT
systems. This volume brings together a valuable collection of experiences in
building systems which depend on successful data fusion and it constitutes a
useful point of reference in the development of advanced IT systems in ESPRIT.
June 1993 D. E. Talbot
Preface
One is faced with many problems when designing computer-based systems
which have to interpret information as it arises in natural environments. Chief
among these problems is the apparent inadequacy of the available data which
characterizes the application at hand, and often we can only form a partial
picture of what the data is supposed to convey. On the other hand, we have the
self-evident ability of humans to work extremely well in equally testing
circumstances. A striking example of this ability concerns the detection of Coal
Workers Pneumoconiosis (CWP), an occupational disease affecting coal miners.
The early detection of CWP is effected by identifying small round opacities in a
sequence of chest radiographs which have been generated at regular intervals
over time. In spite of the absence of an independent reference model, CWP
readers achieve remarkable performance and consistency in the classification of
radiographs into one of twelve categories in the system approved by the
International Labour Office, exhibiting an inter- and intra-reader standard
variation which is better than one category.
The surprising factor is that CWP readers substantially disagree in pin
pointing the cues, i.e. small round opacities, which form the basis for the final
classification. In other words, there is strong agreement in the overall
classification in spite of the fact that the same radiograph manifests different
features to different readers. It can be argued that the difference in the
identification of these features is a consequence of different learning sets and
procedures. Whatever the case, it appears that the successful interpretation
depends at least as much on the way the reader collates - or fuses - the features
as much as it does upon the individual features -the data - themselves.
So how do we deal with such problems in computer-based systems? There
are two obvious, and popular, solutions. The first is to improve significantly the
characteristics of the sensors which are responsible for gathering the data in the
first place, in the belief that with better, more reliable, and less noisy data, the
problem will solve itself. While this is a valid, and frequently a necessary
approach, it is by no means a panacea for these types of applications. The
second solution is to try to glean as much data as possible from as many sources
as possible and then to 'combine' this data in order to reconstruct and
characterize the complete application domain. Unfortunately, this data fusion is
not at all a trivial task and there is no single unified and proven solution which
is applicable in all circumstances. Nonetheless, there are many plausible and
useful approaches which can be used, and are being used, to solve particular
applications in data fusion. At the 1992 ESPRIT Conference and Exhibition, the
DIMUS project organized a workshop on Data Fusion Applications which
emphasized the solutions to practical problems. The papers in this book describe
the projects and the work presented at this workshop. A number of other papers
were included later, in view of their relevance.
VIII
As one reads through this volume, one cannot but be struck by the
extraordinarily wide variety of application domains in which data fusion plays a
role: from the increasingly-important medical area, through industrial control, to
surveillance applications in the nuclear and the transport industries. Without
doubt, the scientific aspects of data fusion, i.e. the formal modelling of different
information sources and their commonality which underpins successful fusion,
is maturing. Equally, the validation of these theories and techniques through
application has an important role to play in this maturation process. Together,
they will see the development of more robust data fusion capabilities and, con
sequently, we will increasingly find complex problems in poorly-characterized
domains yielding to solution, to the benefit of industry and the Research and
Development community alike.
This book represents a unique snap-shot of the current state of play in the
development and validation of technique for data fusion in a wide variety of
applications, ranging from the fusion of visual and non-visual data to the
integration of multi-sensor systems, and embracing many of the essential issues
of, e.g., architectures, real-time responses, and safety-critical requirements. It
forms a valuable reference point against which we can measure our progress in
data fusion research and development, and it represents a useful foundation
upon which we can build.
s.
June 1993 Pfleger
J. Gon~alves
D. Vernon
Contents
Combining Two Imaging Modalities for Neuroradiological
Diagnosis: 3D Representation of Cerebral Blood Vessels .... ........................ 1
M. Bahner, J. Dick, B. Kardatzki, H. Ruder, M. Schmidt,
A. Steitz, C. Bertram, D. Hentschel, T. Hildebrand, E. Hundt,
R. Kutka, S. Stier, G. Gerig, T. Koller, O. Kiibler, G. Szekely
Hybrid Inference Components for Monitoring
of Artificial Respiration ................................................................................. 17
K. Giirtner, S. Fuchs, H. Jauch
Information Fusion in Monitoring Applications using the
Category Model ............................................................................................. 27
W. Steinke
A Flexible Real-Time System for Vessel Traffic Control
and Monitoring ..... .................. .................. .................. .......................... ......... 38
L. Stefanelli
ESPRIT Project AZZURRO: Data Fusion for
Marine Protection ........... .... .................... .................... ................ .... ........ .... ... 44
S. Ghelfo
ESPRIT Project DIMUS: Data Integration in Multisensor Systems ............. 50
F. Benvenuto, M. Ferrettino, M. Pasquali, F. Perotti, P. Verrecchia
A Reflex-Based Approach to Fusion of Visual Data ..................................... 61
A. Bozzoli, M. Rossi, R. BarbO, B. Caprile, G. Carlevaro
Sensor Fusion in a Peg-In-Hole Operation with a
Fuzzy Control Approach ............... .............. ........ ...... ................ ................ .... 71
J. Zhang, J. Raczkowsky
Fuzzy Logic Techniques for Sensor Fusion in
Real-Time Expert Systems ............................................................................ 79
JA Aguilar-Crespo, J. M. Dominguez, E. de Pablo, X. Alanuin
Task-Directed Sensor-Planning 87
G. Grunwald, G.D. Hager
x
Incremental Map-making in Indoor Environments ................... ................ ... 102
A. Ekman, D. Stromberg
Image Segmentation Improvement with a 3-D Microwave Radar ............... 115
A. Siebert, J. Ostertag, B. Radig,
M. Rozman, J. Detlefsen, J. Bernasch
A Vectorial Definition of Conceptual Distance for Prototype
Acquisition and Refinement ......................................................................... 123
C. Moneta, G. Vernazza, R. Zunino
Distributed Knowledge-based Systems for Integration
of Image Processing Modules ...................................................................... 133
C. Regazzoni
Spatial Fusion of Multisensor Visual Information for
Crowding Evaluation .................................................................................... 155
M. Peri, C. Regazzoni, A. Tesei, G. Vernazza
Robust Multisensor Fusion in Underground Stations ................................... 172
S. Pfleger, A. Milano
On Tracking Edges 183
M. Tistarelli
Force and Position Sensing Resistors: An Emerging Technology............... 196
J. Hagen, M. Witte
Fusing Two Views using Object Motion .... ........................ ...... .................... 204
D. Hogg, A. Baumberg
Mobile Robotics for the Surveillance of Fissile Materials
Storage Areas: Sensors and Data Fusion ...................................................... 214
J. G. M. Gom;alves, G. Campos, V. Santos
V. Sequeira, F. Silva
Data Fusion for Environmental Monitoring ...... .......... .... ........................ ..... 246
T. Wittig, H.-N. Pham
Concluding Remarks: Advanced European Research
on Data Fusion ... .................. .... .......... ............ .... ............ .... ...................... ..... 262
K.c. Varghese, S. Pfleger
Contributors .... .................. .................. ...................... ...................... .... .......... 266
Combining Two Imaging Modalities for
Neuroradiological Diagnosis:
3D Representation of Cerebral Blood Vessels
Michael Bahner, Jiirgen Dick, Bernd Kardatzki, Hanns Ruder, Matthias
Schmidt, Amo Steitz
Theoretical Astrophysics, University of TUbingen
Carsten Bertram, Dietmar Hentschel, Thomas Hildebrand
Siemens Medical Systems, Erlangen
Eckart Hundt, Robert Kutka, Sebastian Stier
Siemens Corporate Research, MUnchen
Guido Gerig, Thomas Koller, Olaf KUbler, Gabor Szekely
Communication Technology Laboratory, Image Science Division, ETH ZUrich
Abstract
Today the integration of information from different imaging modalities in
medicine such as Computer Tomography or Magnetic Resonance Imaging
(MRI) is left to the physician and gets little support from computers. In the case
of neuroradiological diagnosis, information about cerebral blood vessels is
available from 3D volume data from Magnetic Resonance Angiography (MRA)
and from 2D images generated by Digital Subtraction Angiography (DSA). The
DSA images have a higher resolution than MRA data, and therefore neuroradi
ologists are highly interested in a 3D reconstruction of cerebral blood vessels
from different DSA projections. On the other hand, MRA contains important
functional information, the velocity of blood flow. This paper describes work in
progress to make available to the physician the full 3D information from both
imaging modalities including an approach to 3D reconstruction from DSA im
ages which makes use of the MRA data. The 3D DSA reconstruction also opens
the way to an integration of information from DSA with completely different
types of information, for example information on anatomical structure or soft
tissue from MRI. An integral part of this work is a pilot system for clinical vali
dation.
As a typical case the neuroradiologist is interested in a 3D representation of
the blood vessels surrounding an aneurysm which would significantly support a
subsequent operation. To this end MRA data and DSA images are analyzed to
extract information about the blood vessels such as position, orientation, width,
and branchings. These items of information are input to the 3D reconstruction
based on both imaging modalities. With these results physicians are able to in
spect the relevant volume in three dimensions using various visualization tools
including a combined display of MRA and the 3D DSA reconstruction.
2
The envisaged benefits of this system are reduced patient risk due to
shorter DSA examinations involving less X-ray and contrast agent and improved
representations of the pathology leading to a better diagnosis and treatment.
This work is part of COVIRA (Computer Vision in Radiology), project
A2003 of the AIM (Advanced Informatics in Medicine) programme of the
European Commission.l This project started in 1992 and will last until 1994.
1. Introduction
For many decades the radiologist's working tools were conventional X-ray and
film. This has changed completely during the last decade as other imaging
modalities such as Computer Tomography (CT), Magnetic Resonance Imaging
(MRI), or Digital Subtraction Angiography (DSA) entered clinical practice. The
number of images stemming from a single patient has enormously increased and
makes the use of computers indispensable. This holds especially for visualiza
tion where the display of three-dimensional data sets requires both sophisticated
computer graphics and large computing resources. Far from being fully suffi
cient the visualization tools supplied with an imaging system at least provide a
basic functionality.
But the radiologist wants more than just a separate display of the available
imaging modalities. The latter provide him with different types of information
about the same location in the human body, for example structural (anatomical)
information from MRI and CT with functional information from Positron
Emission Tomography (PET), or soft tissue anatomy from MRI with vessel
anatomy from Magnetic Resonance Angiography (MRA) or DSA. This integra
tion of information is still mainly left to the imagination of the radiologist who
has to reduce a large amount of information to a concise diagnosis which he
communicates to the physician re"ponsible for the patient's treatment. An inte-
1 Participants in the COVIRA consortium are:
Philips; Medical Systems, Best (NL) (prime contractor) and Madrid (E);
Corporate Research, Hamburg (D)
Siemens AG, Erlangen (D) and Munich (D)
IBM UK Scientific Centre, Winchester (UK)
Gregorio Maranon General Hospital, Madrid (E)
University of TUbingen, Neuroradiology and Theoretical Astrophysics (D)
German Cancer Research Centre, Heidelberg (D)
University of Leuven, Neurosurgery, Radiology and Electrical Engineering (B)
University of Utrecht, Neurosurgery and Computer Vision (NL)
Royal Marsden Hospital/Institute of Cancer Research, Sutton (UK)
National Hospital for Neurology and Neurosurgery, London (UK)
Foundation of Research and Technology, Crete (GR)
University of Sheffield (UK)
University of Genoa (I)
University of Aachen (D)
University of Hamburg (D)
Federal Institute of Technology, ZUrich (CH)
