Table Of ContentFrom Synapses to Rules
Discovering Symbolic Rules
from Neural Processed Data
From Synapses to Rules
Discovering Symbolic Rules
from Neural Processed Data
Edited by
Bruno Apolloni
Department of Information Science
University of Milan
Milan, Italy
and
Franz Kurfess
Department of Computer Science
California Polytechnic State University
San Luis Obispo, California
Springer-Science+Business Media, LLC
Proceedings of the International School on Neural Nets "E.R. Caianiello" Fifth Course: From Synapses to
Rules: Discovering Symbolic Rules From Neural Processed Data, held February 25-March 7, 2002, in Erice,
Sicily, Italy
ISBN 978-1-4613-5204-4 ISBN 978-1-4615-0705-5 (eBook)
DOI 10.1007/978-1-4615-0705-5
©2002 Springer Science+Business Media New York
Originally published by Kluwer Academic/Plenum Publishers, New York in 2002
Softcover reprint of the hardcover 1st edition 2002
http ://www. wkap.nl/
10 9 8 7 6 5 4 3 21
A CLP. record for this book is available from the Library of Congress
All rights reserved
No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any
means, electronic, mechanical, photocopying, microfilming, recording, or otherwise, without written
permission from the Publisher, with the exception of any material supplied specifically for the purpose of
being entered and executed on a computer system, for exclusive use by the purchaser of the work.
Preface
One high-level ability of the human brain is to understand what it has
learned. This seems to be the crucial advantage in comparison to the brain
activity of other primates. At present we are technologically almost ready to
artificially reproduce human brain tissue, but we still do not fully understand
the information processing and the related biological mechanisms underlying
this ability. Thus an electronic clone of the human brain is still far from
being realizable.
At the same time, around twenty years after the revival of the
connectionist paradigm, we are not yet satisfied with the typical subsymbolic
attitude of devices like neural networks: we can make them learn to solve
even difficult problems, but without a clear explanation of why a solution
works. Indeed, to widely use these devices in a reliable and non elementary
way we need formal and understandable expressions of the learnt functions.
These must be susceptible of being tested, manipulated and composed with
other similar expressions to build more structured functions as a solution of
complex problems via the usual deductive methods of the Artificial
Intelligence.
Many effort have been steered in this directions in the last years,
constructing artificial hybrid systems where a cooperation between the
sub symbolic processing of the neural networks merges in various modes
with symbolic algorithms. In parallel, neurobiology research keeps on
supplying more and more detailed explanations of the low-level phenomena
responsible for mental processes.
This book aims at collecting recent contributions in these fields with the
ambitious goal of providing some guidelines for examining the whole
process, from the acquisition of sensory data to their symbolic explanation.
We identify the early processing of these data with the typical connectionist
v
VI Preface
computations implemented by either biological or artificial neural networks
and expect as final output of the process a set of formal rules. That is what
we evocatively call a trip from synapses to rules.
We organized the contributions in three tracks. A theoretical track
examines the mathematical foundations of learning, allowing us a unitary
approach for both subsymbolic and symbolic attitudes. The task of the
former can be a little relaxed, as it is responsible for extracting elementary
properties from data, whose mutual relations will be discovered and further
manipulated at the symbolic level. This manipulation is framed in a
backtracking procedure that is the symbolic companion of the
backpropagation methods. In one direction formulas are composed to
describe higher level properties of the data; in the back one formulas are
remeditated to be simplified enough to be understandable to the users. This
phase is variously managed in the frame of fuzzy set theory under different
entropic criteria, and the unifying approach of utilizing an approximation of
a formula in order to increase its understandability.
A nature driven track relates learning algorithms to the physics of both
the training set generation and of the brain processing. Though mapping
brain activities to mind functionalities is a hard task far from getting an
exhaustive conclusion, in the book we state some links between the two sets.
Mostly, we examine some evidence concerning the physics of the brain
activity in relation to some competencies of the mind. We also investigate
information processing architectures and procedures that are based on this
evidence, and may explain some cognitive processes at various complexity
levels. An interesting aspect of these models is that they show a good fit with
the mathematical models discussed in the previous track. We close the loop
of information generation by also considering the processing of the signals
generated by a human brain and communicated in a verbal or non verbal
communication mode to the sensors of another human brain who tries
learning something from them.
The third track is systemic. Facing logical and physiological hints
coming from the previous tracks, here we want to implement the trip from
synapses to rules. This track has a twofold profile. On one hand we clarify
the terms of the problem, discussing benefits and draw-backs of the
sub symbolic and symbolic approaches for knowledge representation, and
trying to clarify misconceptions and myths about connectionism and neural
netwrok. On the other hand, systems are presented that realize an integration
between procedures dealing with immediate signals and those managing
symbols. We provide both an overview of these systems and proposals for
new approaches, where integration stands for either a tight cooperation
between the two different kinds of procedures, or an inherent mixture of
Preface vii
conceptual tools representative of the two approaches into a single
procedure.
This book is actually a collection of contributions that we tried to
integrate into a unified body. We do not assume that it provides a complete
theory for supporting complete applications. Rather, we aim at outlining a
solid framework for further developments in the field. With this aim we
group papers of the same track into specific part of the book, each starting
with a preface trying to supply a common frame of reference for the various
articles. Analogously we end the part with a section that resumes concluding
remarks from the respective papers. These sections were written by the
Editors, with a special contribution of Gabriele Biella in the premise of the
second track. Whether these developments are grounded in the field of
cognitive science or oriented towards computational applications, we hope
that this book helps narrowing the divide between the two perspectives. Our
goal here actually is twofold: understanding how information processing
works in a human brain is a powerful way for improving our computational
capabilities, and more sophisticated computational models lead to a better
comprehension of cognitive aspects of the human mind. Despite its
deficiencies and crutches, an electronic clone of some functions of our brain
seems to be the most powerful computing device for solving some highly
complex computational problems.
Bruno Apolloni and Franz Kurfess
Acknowledgments
This volume is the result of interactions and cooperation among many
people. We thank the authors of the chapters first for their enthusiastic and
constructive lectures at the Course "From synapses to rules: discovering
symbolic rules from neural processed data", organized in Erice (Italy) by the
International School on Neural Nets "E. R. Caianiello". These lectures make
up in fact the basis of the book. We also thank the school's students for their
intellectual contribution, some of which fonn chapters in the book. A special
mention goes to the young assistants of laren lab (http://laren.dsi.unimi.it):
beside co-authoring some of the chapters, they made a major technical
contribution in editing the volume.
ix
Contributors
• Luigi F. Agnati, Department of BioMedical Sciences, University
of Modena, Italy, [email protected]
• Bruno Apolloni, Dipartimento di Scienze dell'Informazione,
Universita degli Studi di Milano, Via Comelico 39/41 20135
Milano Italy, [email protected]
• Stefano Baraghini, Dipartimento di Scienze dell'Informazione,
Universita degli Studi di Milano Via Comelico 39/41 20135
Milano Italy, [email protected]
• Simone Bassis, Dipartimento di Scienze dell'Informazione,
Universita degli Studi di Milano, Via Come1ico 39/41 20135
Milano Italy, [email protected]
• F. Benfenati, Department of Experimental Medicine, Section of
Human Physiology, University of Genova, Italy
• Gabriele E. M. Biella, Istituto di Neuroscienze e Bioimmagini,
Consiglio Nazionale delle Ricerche, Via Fratelli Cervi 93 20090
Segrate (MI) Italy, [email protected]
• Anna Esposito, Department of Computer Science and
Engineering, Wright State University, 3640 Col. Glenn Hwy.,
Dayton, Ohio, USA, [email protected]
• Massimo Ferri, Department of Mathematics, University of
Bologna, Italy, [email protected]
• Kjell Fuxe, Department of Neuroscience, Karolinska Institutet,
Sweden, [email protected]
• Sabrina Gaito, Dipartimento di Matematica "F. Enriques",
Universita degli Studi di Milano, Via Saldini 50 20133 Milano
Italy, [email protected]
Xl
xu Contributors
• Stefania Gentili, Department of Mathematics and Computer
Science, DIMI, University of Udine, Via delle Scienze 208,
Udine Italy, [email protected]
• Marco Gori, Dipartimento di Ingegneria dell'Infonnazione, Via
Roma, 5653100 Siena Italy
• Domenico Iannizzi, Dipartimento di Scienze dell'Infonnazione,
Universita degli Studi di Milano, Via Comelico 39/41 20135
Milano Italy, [email protected]
• Franz J. Kurfess, Computer Science Department, California
Polytechnic State University, San Luis Obispo CA USA,
[email protected]
• Dario Malchiodi, Dipartimento di Scienze dell'Infonnazione,
Universita degli Studi di Milano, Via Comelico 39/41 20135
Milano Italy, [email protected]
• Maria Marinaro, Dipartimento di Fisica "E. R. Caianiello"
Universita di Salerno Baronissi (SA); INFM Sezione di Salerno;
and IIASS Vietri suI Mare (SA), Italy, [email protected]
• Corrado Mencar, Dipartimento di Infonnatica Universita degli
Studi di Bari Via Orabona 4 70125 Bari Italy,
[email protected]
• Anna Morpurgo, Dipartimento di Scienze dell'Infonnazione,
Universita degli Studi di Milano, Via Comelico 39/41 20135
Milano Italy, [email protected]
• Daniele Mundici, Dipartimento di Scienze dell'Infonnazione,
Universita degli Studi di Milano, Via Comelico 39/41 20135
Milano Italy, [email protected]
• Christos Orovas, Dipartimento di Scienze dell'Infonnazione,
Universita degli Studi di Milano, Via Comelico 39/41 20135
Milano Italy, [email protected]
• Giorgio Palmas, ST Microelectronics, Agrate Brianza (MI) Italy,
[email protected]
• Asim Roy, School of Infonnation Systems, Arizona State
University, Tempe, AZ, USA, [email protected]
• L. M. Santarossa, Department of BioMedical Sciences,
University of Modena, Italy
• Silvia Scarpetta, Dipartimento di Fisica "E. R. Caianiello"
Universita di Salerno Baronissi (SA); INFM Sezione di Salerno;
and IIASS Vietri suI Mare (SA) Italy, [email protected]
• Ron Sun, CECS Department, University of Missouri-Columbia,
[email protected]
Contributors Xlll
• Anna Maria Zanaboni, Dipartimento di Scienze
dell'Infonnazione, Universita degli Studi di Milano Via
Comelico 39/41 20135 Milano Italy, [email protected]
