Table Of ContentSILICON IMPLEMENTATION OF
PULSE CODED NEURAL NETWORKS
THE KLUWER INTERNATIONAL SERIES
IN ENGINEERING AND COMPUTER SCIENCE
SILICON IMPLEMENTATI ON OF
PULSE CODED NEURAL NETWORKS
edited by
Mona E. Zaghloul
The George Washington University
Jack L. Meador
Washington State University
Robert W. Newcomb
University of Maryland
~.
"
SPRINGER SCIENCE+BUSINESS MEDIA, LLC
Library of Congress Cataloging-in-Publication Data
Silicon implementation of pulse coded neural networks / edited by Mona
E. Zaghloul, Jack L. Meador, Robert W. Newcomb.
p. cm. -- (The Kluwer international series in engineering and
computer science)
Includes bibliographical references and index.
ISBN 978-1-4613-6152-7 ISBN 978-1-4615-2680-3 (eBook)
DOI 10.1007/978-1-4615-2680-3
1. Neural networks (Computer science) 2. Semiconductors.
1. Zaghloul, M. E. (Mona Elwakkad) II. Meador, Jack L.
III. Newcomb, Robert W. IV. Series.
QA76.87.S55 1994
006.3'3--dc20 93-48181
CIP
Copyright @ 1994 by Springer Science+Business Media New York
Originally published by Kluwer Academic Publishers in 1994
Softcover ceprint of the hardcover 1s t edition 1994
AH rights reserved. No part of this publication may be reproduced, stored in
a retrieval system or transmitted in any form or by any means, mechanical,
photo-copying, recording, or otherwise, without the prior written permission of
the publisher, Springer Science+Business Media, LLC.
Printed on acid-free paper.
TABLE OF CONTENTS
PREFACE vii
1. Some Historical Perspectives on Early
Pulse Coded Neural Network Circuits
R. W. Newcomb 1
2. Pulse Techniques in Neural VLSI:
AReview
A. F. Murray 9
3. Silicon Dendritic Trees
J. G. Elias 39
4. Silicon Neurons for Phase and Frequency
Detection and Pattern Generation
M. DeYong and C. Fields 65
5. Pulse Coded Winner-Take-AII Networks
J. L. Meador and P. D. Rylander 79
6. Realization ofBoolean Functions Using
a Pulse Coded Neuron
M. de Savigny andR. W. Newcomb 101
7. Design ofPulse Coded Neural Processing
Element Using Modified Neural Type Cells
G. Moon andM. E. Zaghloul 113
8. Low-Power Silicon Neurons, Axons
and Synapses
J. Lazzaro and J. Wawrzynek 153
9. Synchronous Pulse Density Modulation
in Neural Network Implementation
J. Tomberg 165
vi
10. CMOS Analog Neural Network Systems Based
on Oscillatory Neurons
B. Linares-Barranco, E. Sanchez-Sinencio,
A. Rodriguez-Vazquez andJ. L. Huertas 199
11. ADigital Neural Network Architecture
Using Random Pulse Trains
G. R. Salam andR. M. Goodman 249
12. An Unsupervised Neural Processor
1. Donald andL. A. Akers 263
INDEX 291
Preface
When confronted with the hows and whys of nature's computational
engines, some prefer to focus upon neural function: addressing issues of
neural system behavior and its relation to natural intelligence. Then there are
those who preferthe study ofthe "mechanics" ofneural systems: the nuts and
bolts of the "wetware": the neurons and synapses. Those who investigate
pulse coded implementations ofartificial neural networks know whatit means
to stand at the boundary which lies between these two worlds: notjust asking
why natural neural systems behaveas they do, but also how they achieve their
marvelous feats. The research results presented in this book not only address
moreconventional abstractnotionsofneural-likeprocessing,butalsothemore
specific details ofneural-likeprocessors.
It has been established for some time that natural neural systems perform
a great deal of information processing via electrochemical pulses.
Accordingly, pulse coded neural network concepts are receiving increased
attention in artificial neural network research. This increased interest is
compounded by continuing advances in the field ofVLSIcircuitdesign. This
is the first time in history in which it is practical to construct networks of
neuron-like circuits of reasonable complexity that can be applied to real
problems. We believe that the pioneering work in artificial neural systems
presented in this book will lead to furtheradvances that will notonly be useful
in some practical sense, but may also provide some additional insight into the
operation oftheirnatural counterparts.
The idea ofcreating abookdedicatedto pulsecodingin neural VLSI was
first conceived at the International Conference on Circuits and Systems by
Monazaghloul in the springof 1992. Aspecial session had been organizedby
Jack Meador and Robert Newcomb for the conference which was the first of
its kind and had attracted a number of researchers from both Europe and the
US who are working in the area. A poll ofauthors presenting at that session
showed unanimous support for the development of a book which would
showcase recent progress in pulse coded implementations. This book
represents the combined contributions of both those who were present at that
first special session as well as several others who are currently conducting
research in this fascinating area.
This volume seeks to cover many of the relevant contemporary studies
coming out of this newly emerging area. It is essentially a selection,
reorganization,and expansion ofrecentpublications. In addition, Prof. Robert
viii
Newcomb provides a historical perspective on early pulse coded neural
network circuits in Chapter 1. This is followed by a review of more recent
pulsed neural VLSI techniques given by Alan Murray in Chapter 2. In that
chapter, Dr. Murray reviews the variety of pulse coding strategies available
and discusses pulse implementation techniques being investigated at the
University ofEdinburgh. Silicondendritictrees which areintended toemulate
the behavior of spatially extensive biological neurons are then discussed by
John Elias in Chapter 3. A model for a typical spiking neuron is described by
Mark DeYong and Chris Fields in Chapter 4 and is shown to be useful for
phase and frequency detection, filtering, and pattern encoding. In Chapter 5,
Jack Meadorand PaulHylander presentapulsecodedwinner-take-all network
which encodes decision strength as a variable rate pulse train generated by the
winning unit. The Neural Type Cell which is a special circuit that accepts
analog inputs and produces pulse coded outputs is used in Chapter 6 by Marc
De Savigny and Robert Newcomb to implement Boolean functions as well as
by Gyu Moon and Mona Zaghloul in Chapter7 as part ofa unique processing
element that can be used as a basic cell for artificial neural network
implementation. Low power silicon implementation of Neurons, Axons and
Synapses are introduced by John Lazzaro, and John Wawrzynek in Chapter 8.
These new implementations improve upon the power dissipation
characteristics of the self-resetting neuron and silicon axon described by
Carver Mead in his popular 1989 book. Synchronous pulse density
modulation techniques are presented in Chapter 9 by Jouni Tomberg. Both
switched capacitor and digital implementations of synchronous pulse coded
arithmetic are discussed in the context of established network architectures
here. The transconductance mode technique is presented in Chapter 10 by
Bernabe Linares-Barranco, Edgar Sanchez-Sinencio, Angel Rodriquez
Vazquez,andJose Huertas to implementanequivalentcircuitforan oscillatory
neuron based on the simplified operation ofthe living neuron. This oscillatory
neuron is used to implement and test established neural architectures. In
Chapter II a digital architecture that utilizes random bit sequence properties is
described by Gamze Salam and Rodney Goodman. The extension of
previously established stochastic pulse coded implementation techniques are
presented and demonstrated. Finally in Chapter 12, James Donald and Lex
Akers present experimental results obtained with two adaptive pulse coded
signal processing chips for real-time control. These chips use pulse coded
signals for inter-chip communication and analog weights for information
storage. In all of this work, novel circuits are presented and innovative
techniques for implementing conventional neural network architectures are
introduced.
ix
By way of acknowledgment we would first like to thank all of the
contributing authors for their prompt assistance with the details, some
requiring their attention on very short notice. We would also like to express
special thanks to PaulHylanderand IoanaPirvulescufor theirsuggestions and
assistance with theearly drafts ofthe book. We would also like toexpress our
gratitude to Holly Snyder for her assistance with the details of final draft
preparation.
Mona Zaghloul
lackMeador
RobertNewcomb
1
SOME HISTORICAL
PERSPECTIVES ON EARLY PULSE
CODED NEURAL NETWORK
CIRCUITS
Robert W. Newcomb
Electrical Engineering Department
University ofMaryland, College Park MD, 20742
PERSPECTIVES
From the beginnings of mankind the means of brain activity must have
fascinated man. And although Galvani had shown in the late 1700s that
muscles were excited by electrical activity of the nerves [Galvani 1791,
Brazier61] it wasnotknown through mostofthe 1800'swhatwas thebasisfor
activity of the brain - indeed it is still unknown how a person thinks. In any
event the publication by the Polish neurophysiologist Adolf Beck in the
Centralblatt fur Physiologie [Beck 1890], concerning his measurements of
electrical activity in thebrain [Beck 1888],causedconsiderablecontroversy as
towhomwas the firstoneto achievesuchanaccomplishment. Afteralmostall
sides were heard from, the controversy was settled by a further letter to the
Centralblatt by Richard Caton calling attention to the measurements he had
reported to the August 24, 1875, meeting of the British Medical Association
and recorded in the report ofthe meeting [Caton 1875]. Among statements in
Caton'soriginalreportis thefollowing: "Whenany partofthegrey matterisin
a state of functional activity, its electric current usually exhibits negative
variation" [Brazier 61] where by "negative variation" at the time was meant
action potentials. Thus, we see that measurements were made on the pulse
coded electricalactivity in the brain asearly as 1875.
