Table Of ContentMachine Intelligence and
Pattem Recognition
Volume 16
Series Editors
L.N. KANAL
and
A. ROSENFELD
University of Maryland
College Park, Maryland, U.S.A.
ELSEVIER
Amsterdam - Lausanne - New York - Oxford - Shannon - Tokyo
Pattem Recognition in Practice IV:
Multiple Paradigms,
Comparative Studies and
Hybrid Systems
Proceedings of an Intemational Workshop held on Vlieland,
The Netherlands, 1-3 June 1994
^^A;ME 1-3 1994 I
Edited by
Edzard S. GELSEMA
Department of Medical Informatics
Erasmus University
Rotterdam, The Netherlands
Laveen S. KANAL
Department of Computer Science
University of Maryland
College Park, MB, USA.
1994
ELSEVIER
Amsterdam - Lausanne - New York - Oxford - Shannon - Tokyo
ELSEVIER SCIENCE B.V.
Sara Burgerhartstraat 25
RO. Box 211, 1000 AE Amsterdam, The Netherlands
ISBN: 0 444 81892 8
© 1994 Elsevier Science B.V. All rights reserved.
No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form
or by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior written
permission of the publisher, Elsevier Science B.V, Copyright & Permissions Department, RO. Box
521, 1000 AM Amsterdam, The Netheriands.
Special regulations for readers in the U.S.A. - This publication has been registered with the Copyright
Clearance Center Inc. (CCC), Salem, Massachusetts. Information can be obtained from the CCC about
conditions under which photocopies of parts of this publication may be made in the U.S.A. All other
copyright questions, including photocopying outside of the U.S.A., should be referred to the copyright
owner, Elsevier Science B.V, unless otherwise specified.
No responsibility is assumed by the publisher for any injury and/or damage to persons or property as a
matter of products liability, negligence or otherwise, or from any use or operation of any methods, pro
ducts, instructions or ideas contained in the material herein.
This book is printed on acid-free paper.
Printed in The Netheriands
PREFACE
The series of conferences "Pattern Recognition in Practice" started in 1980.
Subsequently, the conferences in 1985 and in 1988 firmly established these conferences
as a well appreciated workshop-type series, quite different in character from the much
larger international conferences on pattern recognition (ICPR's). As in the previous
conferences, the main aim of PRP-IV was to stimulate exchanges between professionals
developing pattern recognition methodologies and those who use pattern recognition
techniques in their professional work. As in PRP-III, the scope included artificial
intelligence techniques. Due to the fact that ICPR-11 was organized in The Hague in
1992, PRP-IV did not materialize until 1994. The venue was changed to a more
interesting spot in The Netheriands: the island Vlieland. Judging from the many
enthousiastic comments received during and after the event, the conference was a
success, not in the last place because of the scientific content. The 42 papers presented
have been organized in tiiis book into six parts:
1. Pattern Recognition
2. Signal- and Image Processing
3. Probabilistic Reasoning
4. Neural Networks
5. Comparative Studies
6. Hybrid Systems
Already in Part I, the integration of techniques from artificial mtelligence and from
pattern recognition becomes apparent. The opening paper by Vamos is not only a
philosophical introduction to the theme but also describes large scale projects an which
the representation of knowledge is the central issue. The two following papers describe
two non-parametric methods of classification: the k-NN nearest neighbour rule,
supplemented with evidence theory and a decision tree building scheme, incorporating
domain knowledge such as feature cost and allowing incremental learning. Homegger et
al. compare the use of several types of Hidden Markov Models to 2-D object recognition
in computer vision. Extensions to 3-D object recognition are briefly discussed. Part I
continues wdth four papers describing an extension of grammatical inference to grammars
for point sets. The process of UpWrite, an iterative sequence of mappings of a set of
objects at one level of description to one single object at some higher level is introduced.
The process is derived from properties of assemblies of neurons and might therefore be a
model of the functioning of the nervous system. Applications to the recognition of cubes
in 2-D images, the classification of moving objects and Gaussian mixture modelling are
described. The final paper in Part I deals vsdth syntactic modelling of sleep-EEG's. A
system aiming at predicting the entrance into REM sleep on the basis of its grammatical
structure is described.
The papers in Part II describe various methodological issues and applications of signal
and image processing. The opening paper in this part, by Brailovsky, deals with the
suppression of noise in signals and images. It is argued that in order to recover the
original signal or image, the global structure must be taken into account. Results with
piecewise linear signals are presented. The second paper addresses the problem of
reflectance ratios. Region reflectance ratios are invariant to illumination and other
acquisition parameters. This property is exploited to recognize objects from a single
brightness image. The next paper describes an image segmentation algorithm based on a
possible model of the human perceptive process and incorporating techniques from
artificial intelligence. The segmentation is intended as a preprocessing step, aiming at
facilitating human interpretation. The next three papers deal with various forms of graph
matching in image processing. Wilson and Hancock describe a procedure using discrete
relaxation in which the probability distributions of null matches and erroneous matches
are modelled by a uniform and memoryless probability ftmction. The next paper
describes structural inexact matching using neural networks as an altemative to tree
search methods. Results on an example of a synthetic image are presented. This paper is
followed by one dealing with inexact and elastic matching of magnetic resonance brain
images with an anatomical atias for the identification of cortical sulci. The method is
validated on the basis of 55 pairs of sulci. The final two papers in Part II both deal with
remote sensing images. Mulder et al, describe a system in which knowledge about
agricultural fields, including crop development is represented in models. Costs of
misclassification are defined in terms of economic costs for the user of the information.
The last paper deals vsdth the application of various filtering techniques for noise
suppression while preserving texture information. Results on SAR images in an
application of natural resource management are presented.
The papers in Part III discuss various methods of probabilistic reasoning. The three first
use the Bayesian network paradigm. The paper by Lemmer deals with the problem of
incoφorating space and time restrictions into the Bayesian network formalism. Methods
for both prediction and inference are presented. First a model is introduced which,
however, is computationally intractable. Several assumptions are shown to lead to a
model which is computationally feasible. Van de Stadt describes extensions to a
Bayesian belief network to provide a framework for the plaiming of actions under
uncertainty. The scheme is dynamical, adapting the planning process to the available
information. In the third paper, probabilistic Bayesian reasoning is used in an object
recognition system using a set of volumetric primitives called geons. The probabilities
are used to focus attention of the vision system. It is concluded that the method is
superior to deterministic reasoning. The last paper, by Bhatnagar, describes a decision
tree reasoning scheme in which the primitives in terms of which the target concepts are
leamt are pre-specified by the agent performing the partitioning and leaming task.
In Part IV, several aspects of neural networks are reviewed. Whereas the literature in the
last decades has abundantly described the constmction and application of neural
networks in various domains, time has now come to reflect on the properties of such
networks and on the question of the interpretation of weights in a trained network. Part
IV opens vsdth a thorough analysis by Raudys, giving at least three explanations for the
vil
intriguing properties which neural networks exhibit in the presence of small training sets.
In the paper by Kappen, a general framework for neural network leaming is presented.
In this framework, supervised feed-forward leaming, unsupervised leaming and
clustering may be viewed as special cases. The next four papers all try to counter the
common view of neiu*al networks as magic black boxes. The first paper of this quartet,
by Sethi and Yoo, describes a backtracking tree search to convert the weights in a neural
network into symbolic representations which may be used to understand the knowledge
embodied in a trained network. Christmas, Kittler and Petrou present an analysis of
object labelling and classification allowing to derive interpretations of the weights,
response fimctions and the nodes in mutilayer perceptrons. This may guide the choice of
such parameters in the design phase of neursd networks. The next paper presents an
altemative feedforward scheme to train neural network classifiers. It can improve
convergence and generalization properties when applied to binary problems. It uses
digital circuit fault test theory to trace sensitive paths from the input to the output layer.
The next paper presents a method of feature selection in the multi-layer perceptron
framework. The partial derivatives of the outputs vdth respect to the inputs are used to
determine each input attribute's contribution to the classification. The method is
illustrated by an example of radiological bone lesion diagnosis. The last two papers
mainly concentrate on applications of neural networks. Oja describes the use of various
hardware and software implementations of practical production quality in various
architectures. In addition, the main advantages and disadvantages of neural networks are
summarized. Greenhill and Davies describe an application of neural networks to the
problem of noise suppression in digital images. They indicate that the network can be
adapted, by training, to various types of noise. This gives the ANN an advantage over
more conventional types of filters.
Part V collects various comparative studies, which are becoming increasingly important
in a field which can still be described as "a bag of tools for a bag of problems". The
availability of numerous well known, standard data sets facilitates such comparisons. The
first paper in this part, by Schmidt and Duin, presents a thorough comparison of the
performance of neural networks and traditional classifiers, i.e., the nearest mean and the
nearest neighbour classifier. It describes a duplication of the NETtalk experiment and
discusses a comparative study on three other traditional data sets. The conclusion is that
in most cases, the two traditional methods considered are superior. The next paper
discusses the relative merits of classical sequential methods and genetic algorithm search.
The purpose is to determine whether properties established for such methods m medium
scale experiments extend to problems of much higher dimensions. Talmon et al. describe
a comparison of the performance of a neural net classifier with that of classification trees
in a medical domam. Also, the knowledge encoded in the various classifiers is compared.
Mishra and Raghavan compare a number of clustering algorithms with respect to
computation cost and quality of the solution. The results on several data sets of various
sizes and dimensionalities are presented. The problem of comparison is shown to be
complicated, due to the fact that most of the techniques require design parameter
settings, influencmg their behaviour. Mohiuddin and Mao compare four different
classifiers for isolated handprinted character recognition, using the NIST database. A
hybrid system in which the top three solutions obtained by a neural network are re-
evaluated by a nearest template classifier is shown to have superior recognition accuracy.
The fmal paper in Part V, by Procter and Illingworth, describes a comparison of two
techniques, i.e.. Randomised Hough Transform and Genetic Algorithm applied to the
problem of ellipse extraction. The paper aims at forming the basis for making a reasoned
choice between the two methods. Differences exist mainly in computation cost, rather
than in performance.
The fmal part of these Proceedings show the extent to which the integration of pattem
recognition and artificial intelligence, alluded to in the subtitle of the Proceedings of
"Pattern Recognition in Practice III" has materialized. The first paper on Hybrid
Systems, by Holz and Loew, introduces the concept of relative feature importance (RFI),
as a measure of discriminative power. The measure is based on the stmcture of the data.
Since directly computing RFI requires exhaustive search, they introduce a hybrid genetic,
neural technique to estimate RFI. The following two papers deal with applications in
robotics. The first, by Jiang and Bunke, describes a multisensory system intended to
support the vision requirements of an intelligent robot. The system comprises a vision
planner which may transform a vision request into concrete vision operations or which
may develop altemative strategies. The second paper on robotics deals with the
development of a multi-level architecture for real-time reasoning in the domain of mobile
robots. Data and knowledge exist in the system in multiple representations. The hybrid
knowledge base is a suitable representation to allow reasoning accross levels in the
hierarchy. The paper by Parikh discusses the application of the technique of constraint
satisfaction to solve configuration and location problems in computer vision. The
constraint problem is transformed into an energy minimizing problem which is then
solved by a hybrid neural/genetic technique. Smyth describes a hybrid scheme based on
discriminative and generative models for the real-time detection of faults in complex
dynamic systems. The scheme combines a neural network and a Hidden Markov Model
and is applied to the monitoring of antenna pointing systems. A hybrid system for the
on-line recognition of cursive script is described by Bramall and Higgins. The system
consists of various knowledge sources, which act on the information available on a
blackboard, using various artificial intelligence and pattem recognition techniques. The
final paper describes a hybrid system combining a neural- and a semantic network
intended for the recognition of a hand and its 3-D orientation. The system is evaluated
on a set of 300 stereo colour images of real world scenes.
The papers in this volume indicate that the era of detailed comparisons of the merits of
techniques of pattem recognition and artificial intelligence and of the integration of such
techniques into flexible and powerfiil systems has begun. These papers may give
prospective users a feeling for the applicability of the various methods in their field of
specialization. The rapid development in the pattem recognition field has motivated the
conference chairmen to announce "Pattem Recognition in Practice V" to be held in 1997.
July, 1994, Rotterdam, Edzard S. Gelsema
College Park, Laveen N. Kanal
Editors
ACKNOWLEDGEMENTS
This book is a collection of papers presented at the international conference "Pattern
Recognition in Practice IV", held in Vlieland, The Netherlands from June 1 to June 3,
1994.
The conference was financially sponsored by the following companies, who are grateful
ly acknowledged for their support:
Hewlett-Packard S.A., Medical Products Group Europe
Océ-van der Grinten N.V.
IBM Nederland N.V.
The conference was organized at a somewhat remote spot. The logistics of the transpor
tation of the participants from Amsterdam Schiphol airport to Vlieland was coordinated
by Guus Beckers and Jifke Veenland, who also helped in maintaining the participant
registration fdes. Teun Timmers coordinated the communications with hotel Seeduyn in
Vlieland, where the conference was held. The active support from these three members
of the Organizing Committee was much appreciated and is gratefully acknowledged.
The conference program was composed of invited contributions and of reactions to the
call for papers. The tasks of speaker invitation and paper selection were carried out by
the Program Committee consisting of, besides the editors, Eric Backer, Bob Duin and
Anil Jain. Their efforts resuhed in an attractive scientific program of high quality
content.
During the conference, all discussions were recorded on tape and transcribed in the
secretariat. The recording process was expertly carried out by Frits Vogels, assisted by
Ben van den Boom. The process of transcription from tape to paper was done by Annett
Bosch and Andre Redert and supervised by Guus Beckers, Jifke Veenland and Teun
Timmers. Thanks to the dedicated efforts of all these individuals, these Proceedings
contain an edited rendition of these discussions as a valuable addition to the original
papers.
Many large and small duties which come with running a conference were diligently
carried out by the head of the secretariat. Loes de Langen. Her continuous, enthousiastic
dedication had an infectuous effect on all in and around the secretariat. With her talent
of anticipation and improvisation she prevented or solved many small or larger problems,
bringing order in the nerve center of the conference.
Finally, as it was expressed in the opening address, whereas the Organizing Committee
had provided the extemal conditions for a successful conference, the attendees' contribu
tions and active participation gave the conference its intrinsic scientific value. The draft
manuscripts and revised versions were almost always delivered within the prescribed
deadlines. We want to thank all participants for their cooperation.
The conference chairmen
Edzard S. Gelsema
Laveen N. Kanal
Pattern Recognition in Practice IV
E.S. Gelsema and L.N. Kanal
© 1994 Elsevier Science B.V. All rights reserved.
Patterns in the role of knowledge representation
T. Vamos
Computer and Automation Institute, Hungarian Academy of Sciences*
1. INTRODUCEN
Representation of knowledge in the brain is the focus of cognitive science. Several
highly acknowledged schools share the view that this representation is close to the
concept of patterns. We emphasize the word close, i.e. pattern is used partly in a
metaphoric sense; the real processes are the subject of research and speculation. Though
relevant achievements and well proved facts are available, it is still argued if it is
possible to have a fmal understanding of the mental processes, because of the extreme
complexity of the brain and due to the philosophical paradoxes of self-reflexion.
The evidences are rather clear. The entire phylogenetic evolution of the neural
system, even long before it developed the specially dedicated central organism of the
brain, had been characterized by some memorizing abilities of coherent effects, relevant
for survival, and similarly a storage of coherent responses, partly inherited, partly learnt.
These coherent memory items are those which are the archetypes of knowledge patterns,
and the consequent responses have a pattern organization, too. All mobile creatures of
nature owe some motion pattern primitives which are characteristic for the species and
have well-defined organizational patterns (muscle and nerve patterns in the higher level
animals) in their biological constitution. In this sense, patterns of life have a rather
closed, predetermined character on one hand and are open-ended for variational
complexity on the other. The pattern reality and metaphor are especially vision-related in
the human, vision being our richest source for knowledge acquisition.
The concept of patterns labelled by different names, accompanies the whole history
of literature, philosophy and psychology, asserting the evidence of the validity for the
idea. It was also proved that the creative human thinking preserves the way of preverbal
periods. Patterns - fragmental impressions - come together in a mystic procedure before
any formulation in words and further, in strong disciplinary rules of scientific reasoning.
We could refer to volumes of bibliographies containing only titles on the subject,
but for our practical considerations this short allusion should be enough to clarify why
we tried to apply the pattern concept for knowledge representation particularly for
representation of weakly structured or unstructured, i.e. soft knowledge.
• Sponsoring agency: Hungarian National Science Foundation; research grant number: 2584
