Table Of ContentNEURAL NETWORKS
for
PERCEFnon
Volume 1
Human and Machine Perception
Edited by
Hany Wechsler
George Mason university
Fairfax, Virginia
ACADEMIC PRESS, INC.
Harcourt Brace Jovanovich, Publishers
Boston San Diego New York
London Sydney Tokyo Toronto
This book is printed on acid-free paper. Θ
Copyright © 1992 by Academic Press, Inc.
All rights reserved.
No part of this publication may be reproduced or
transmitted in any form or by any means, electronic
or mechanical, including photocopy, recording, or
any information storage and retrieval system, without
permission in writing from the publisher.
Cover design by Elizabeth E. Tustian
ACADEMIC PRESS, INC.
1250 Sixth Avenue, San Diego, CA 92101
United Kingdom Edition published by
ACADEMIC PRESS LIMITED
24-28 Oval Road, London NWl 7DX
Library of Congress Cataloging-in-Publication Data
Neural Networks for perception / edited by Harry Wechsler,
p. cm.
Includes bibliographical references and index.
Contents: v. 1. Human and machine perception — v. 2. Computation,
learning, and architecture.
ISBN 0-12-741251-4 (v. 1). — ISBN 0-12-741252-2 (v. 2)
1. Neural networks (Computer science) 2. Perception.
I. Wechsler, Harry.
QA76.87.N485 1991
006.3—dc20 91-24207
CIP
Printed in the United States of America
91 92 93 94 9 8 7 6 5 4 3 2 1
To my daughter, Gabriela Anya
Contents of Volume 2
PART III
Computation and Learning
IILIntroduction
H. Wechsler
ULI Learning Visual Behaviors
D.H. Ballard and S. D. Whitehead
111.2 Nonparametric Regression Analysis Using Self-Organizing
Topological Maps
V. Cherkassky and H. Lari-Najafi
111.3 Theory of the Backpropagation Neural Network
R. Hecht-Nielsen
111.4 Hopfield Model and Optimization Problems
Behrooz Kamgar-Parsi and Behzad Kamgar-Parsi
ΙΠ.5 DAM, Regression Analysis, and Attentive Recognition
W. Pölzleitner
ΙΠ.6 Intelligence Code Machine
V.M. Stern
III.7 Cycling Logarithmically Converging Networks That Flow Information
to Behave (Perceive) and Learn
L. Uhr
ΙΠ.8 Computation and Learning in the Context of Neural
Network Capacity
S.S. Venkatesh
PART IV
Arcliitectures
IV. Introduction
H. Wechsler
XI
xii Contents of Volume 2
IV. 1 Competitive and Cooperative Multimode Dynamics in Photorefractive
Ring Circuits
D.Z. Anderson, C. Benkert, and D.D. Crouch
IV.2 Hybrid Neural Networks and Algorithms
D. Casasent
IV.3 The Use of Fixed Holograms for Massively-Interconnected, Low-
Power Neural Networks
H. Jeon, J. Shamir, R.B. Johnson, H.J. Caulfield, J. Kinser, C. Hester,
and M. Temmen
IV.4 Electronic Circuits for Adaptive Synapses
J. Mann and J. Raffel
IV.5 Neural Network Computations on a Fine Grain Array
Processor
S.S. Wilson
Contributors
Numbers in parentheses indicate pages on which the authors' contributions
begin.
Igor Aleksander (202), Department of Electrical Engineering, Imperial Col
lege of Science, Technology, and Medicine, Exhibition Road, London
SW7 2BT, England
Erhardt Earth (234), Department of Computer Science, School of Informa
tion Technology and Electrical Engineering, The University of Mel
bourne, Parkville, Victoria 3052, Australia
Jezekiel Ben-Arie (214), Department of Electrical and Computer Engineer
ing, Illinois Institute of Technology, Chicago, Illinois 60616
Terry Caelli (234), Department of Computer Science, School of Information
Technology and Electrical Engineering, The University of Melbourne,
Parkville, Victoria 3052, Australia
Gail A. Carpenter (248), Center for Adaptive Systems and Graduate Pro
gram in Cognitive and Neural Systems, Boston University, 111 Cumming-
ton Street, Boston, Massachusetts 02215
R. Chellappa (492), Electrical Engineering Department, University of Mary
land, College Park, Maryland 20742
Leon N. Cooper (8), Department of Physics and Center for Neural Science,
Brown University, Providence, Rhode Island 02912
Gerald M. Edelman (41), Neurosciences Institute and Rockefeller Univer
sity, New York, New York 10021
Shimon Edelman (25), Applied Mathematics, Weizmann Institute of Sci
ence, Rehovot, Israel
Mario Ferraro (234), Dipartimento di Fisica Sperimentale, Universita' di To
rino, Italy
Leif H. Finkel (41), Department of Bioengineering, 220 S. 33rd Street, Uni
versity of Pennsylvania, Philadelphia, Pennsylvania 19104
Stephen Grossberg (64, 248), Center for Adaptive Systems and Graduate
Program in Cognitive and Neural Systems, Boston University, 111 Cum-
mington Street, Boston, Massachusetts 02215
xni
Χίν Contributors
Anya C. Hurlbert (265), Department of Physiology, Oxford University, Ox
ford, England
Bela Julesz (145), Laboratory of Vision Research, 41 Gordon Road, Kilmer
Campus, Rutgers University, New Brunswick, New Jersey 08903
Michael Kuperstein (285), Neurogen Laboratories, Inc., 325 Harvard Street,
Suite 202, BrookHne, Massachusetts 02146
Jitendra Malik (315), Electrical Engineering and Computer Science, Univer
sity of California, Berkeley, California 94720
M. Manohar (345), Universities Space Research Association, Greenbelt,
Maryland 20771
Erkki Oja (368), Information Technology, Lappeenranta University of Tech
nology, P.O. Box 20, SF-53851 Lappeenranta, Finland
Lance M. Optican (104), Laboratory of Sensorimotor Research, National
Eye Institute, Bethesda, Maryland 20892
Pietro Perona (315), California Institute of Technology, Department of Elec
trical Engineering, Pasadena, California 91125
Todd R. Reed (386), Department of Electrical and Computer Engineering,
University of California, Davis, California 95616
George N. Reeke, Jr. (41), Neurosciences Institute and Rockefeller Univer
sity, New York, New York 10021
Daniel Reisfeld (477), Department of Computer Science, Tel Aviv Univer
sity, Tel Aviv, Israel
John Reynolds (248), Center for Adaptive Systems and Graduate Program
in Cognitive and Neural Systems, Boston University, 111 Cummington
Street, Boston, Massachusetts 02215
Barry J. Richmond (104), Laboratory of Neuropsychology, National Insti
tute of Mental Health, Building 9, Room IN-107, 9000 Rockville Pike,
Bethesda, Maryland 20892
G. Sandini (398), Dipartimento di Informatica, Sistemistica e Telematica,
University of Genoa, Genoa, Italy
Michael Seibert (426), Machine Intelligence Technology Group, MIT Lin
coln Laboratory, Lexington, Massachusetts 02173
Jude W. Shavlik (445), Computer Sciences Department, University of Wis
consin, Madison, Wisconsin 53706
Shigeru Tanaka (120), Fundamental Research Lab, NEC Corporation, Mi-
yukigaoka 34, Tsukuba, Ibaraki 305, Japan
J.C. Tilton (345), NASA Goddard Space Flight Center, Greenbelt, Mary
land 20071
M. Tistarelli (398), Dipartimento di Informatica, Sistemistica e Telematica,
University of Genoa, Genoa, Italy
Geoffrey G. Towell (445), Computer Sciences Department, University of
Wisconsin, Madison, Wisconsin 53706
Contributors XV
Allen Μ. Waxman (426), Machine Intelligence Technology Group, MIT Lin
coln Laboratory, Lexington, Massachusetts 02173
Harry Wechsler (462), Department of Computer Science, George Mason
University, Fairfax, Virginia 22030
Douglas Williams (145), Laboratory of Vision Research, 41 Gordon Road,
Kilmer Campus, Rutgers University, New Brunswick, New Jersey
08903
Haim Wolfson (477), Department of Computer Science, Tel Aviv University,
Tel Aviv, Israel
Yehezkel Yeshurun (477), Department of Computer Science, Tel Aviv Uni
versity, Tel Aviv, Israel
Y.T. Zhou (492), HNC, Inc., 5501 Oberlin Drive, San Diego, Cali
fornia 92121
G. Lee Zimmerman (176), Department of Electrical Engineering, Tulane
University, New Orleans, Louisiana 70118
Foreword
Neural Networks for Perception explores perception and the recent research
in neural networks that has advanced our understanding of both human and
machine perception. Perception is a major facet of our senses and provides
us with the essential information needed to broaden our horizons and to
connect us to the surrounding world, enabling safe movement and advanta
geous manipulation. Far beyond being merely a scientific challenge, the pos
sibility of emulating the human sense of perception would revolutionize
countless technologies, such as visual tracking and object recognition, ro
botics and flexible manufacturing, automation and control, and autonomous
navigation for future space missions. As Aristotle noted,
"All men, by nature, desire to know. An indication of this is the delight we
take in our senses, for even apart from their usefulness they are loved for them
selves and above all others the sense of sight. For not only with a view to
action, but even when we are not going to do anything we prefer seeing to
everything else. The reason is that this, most of all senses, makes us know and
brings to light many differences between things."
Indeed, reflecting the intricate connection between perception and purpose
ful activity, many of the papers in this book deal with meaningful tasks.
Meanwhile, we are witnessing the rapid growth of neural networks re
search as a novel and viable approach to emulating intelligence in general
and to achieving the recognition and perceptual learning functions of vision.
Neural network research is a synergetic endeavor that draws from cognitive
and neuro-sciences, physics, signal processing, and pattern recognition.
Neural networks (NN), also known as artificial neural systems (ANS), are
implemented as parallel and distributed processing (PDP) models of com
putation consisting of dense interconnections among computational process
ing elements (PE or "neuron"). The competitive processes that take place
among the PEs enable neural networks to display fault-tolerance and ro
bustness with respect to noisy and/or incomplete sensory inputs, while al
lowing graceful degradation with respect to faulty memory storage and in
ternal processing.
xvn
xviü Foreword
Neural Networks for Perception showcases the work of preeminent prac
titioners in the field of neural networks and enhances our understanding of
what neural networks are and how they can be gainfully employed. It is
organized into two volumes: The first, subtitled Human and Machine Per
ception, focuses on models for understanding human perception in terms of
distributed computation and examples of PDP models for machine percep
tion. The second, subtitled Computation, Learning, and Architectures, ex
amines computational and adaptation problems related to the use of neu
ronal systems, and the corresponding hardware architectures capable of
implementing neural networks for perception and of coping with the com
plexity inherent in massively distributed computation.
Perception is just one of the capabilities needed to implement machine
intelligence. The discussion on perception involves, by default, the full range
of dialectics on the fundamentals of both human and machine intelligence.
Normal science and technological development are always conducted within
some predefined paradigm and this work is no exception. The paradigms
attempt to model the everlasting dichotomy of brain and matter using spe
cific metaphors. One of the metaphors for neural networks is statistical
physics and thermodynamics; nonetheless, some thoughts on the feasibility
and future use of evolution and quantum mechanics are contemplated as
well. NN advancements parallel those underway in artificial intelligence to
ward the development of perceptual systems. Consequently, the possibility
of hybrid systems, consisting of NN and AI components, is also considered.
Many have postulated possible arguments about what intelligence is and
how it impinges on perception. Apparently, recognition is a basic biological
function crucial to biological systems in recognizing specific patterns and
responding appropriately: antibodies attack foreign intruders; our ears cap
ture sound and speech; animals have to locate edible plants; and sensory-
motor interactions involved in navigation and manipulation are predicated
on adequate recognition capabilities. Failure to recognize can be fatal; rec
ognition should therefore be the ultimate goal of the perceptual system, and
indeed, it probably underlies much of what is intelligence.
Albert Szent-Gyorgi said that
"The brain is not an organ of thinking but an organ of survival, like claws and
fangs. It is made in such a way as to make us accept as truth that which is only
advantage. It is an exceptional, almost pathological constitution one has, if one
follows thoughts logically through, regardless of consequences. Such people
make martyrs, apostles, or scientists, and mostly end on the stake, or in a
chair, electric or academic."
The concepts of recognition and reasoning by analogy underlie recent views
on both planning and learning as espoused by the case-based reasoning
methodology.
