Cellular Neural Networks and Analog VLSI

104 Pages·1998·5.13 MB·English

by  J. M. Cruz

CELLULAR NEURAL NETWORKS
AND ANALOG VLSI
edited by
Leon O. Chua
University 0/ California, Berkeley
Glenn Gulak
University a/Toronto
Edmund Pierzchala
Analogy, Inc.
Angel Rodriguez-Vazquez
Universidad de Sevilla
A Special Issue of
ANALOG INTEGRAT ED CIRCUITS
AND SIGNAL PROCESSING
An International Journal
Volume 15, No.3 (1998)
SPRINGER SCIENCE+BUSINESS MEDIA, LLC
ANALOG INTEGRATED
CIRCUITS AND
SIGNAL PROCESSING
An International Jo umal
Volume 15, No.3, March 1998
Special Issue: Cellular Neural Networks and Analog VLSI
Guest Editors: Leon O. Chua, Glenn Gulak, Edmund Pierzchall and Angel Rodriguez-Vazquez
Guest Editorial ......................... L. Chua. E. Pierzchala. G. Gulak and A. Rodriguez-Vazquez
A 16 x 16 Cellular Neural Network Universal Chip: The First Complete Single-Chip Dynamic Computer
Array with Distributed Memory and with Gray-Scale Input-Output ........... J. M. Cruz and L. O. Chua 3
A 6 x 6 Cells Interconnection-Oriented Programmable Chip for CNN ............................ .
· . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. M. Salerno. F. Sargeni and Vincenzo Bonaiuto 15
Analog VLSI Design Constraints of Programmable Cellular Neural Networks ... P. Kinget and M. Steyaert 27
Focal-Plane and Multiple Chip VLSI Approaches to CNNs ..................................... .
· . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. M. Anguita. F. J. Pelayo. E. Ros. D. Palomar and A. Prieto 39
Architecture and Design of I-D Enhanced Cellular Neural Network Processors for Signal Detection .....
· .................................. M. Y. Wang. B. J. Sheu. T. W. Berger; W. C. Young and A. K. Cho 53
Analog VLSI Circuits for Competitive Learning Networks ..................................... .
· ................................................ H. C. Card, D. K. McNeill and C. R. Schneider 67
Design of Neural Networks Based on Wave-Parallel Computing Technique ........................ .
· ................................................ Y. Yuminaka. Y. Sasaki. T. Aoki and T. Higuchi 91
Library of Congress Cataloging-in-Publication Data
Cellular neural networks and analog VLSI / edited by Leon O. Chua ...
[et al.].
p. cm.
HA special issue of Analog integrated circuits and signal
processing, an international journal, volume 15, no. 3 (1998)."
Includes bibliographical references.
ISBN 978-1-4419-5030-7 ISBN 978-1-4757-4730-0 (eBook)
DOI 10.1007/978-1-4757-4730-0
1. Neural networks (Computer science) 2. Integrated circuits-
-Very large scale integration. 1. Chua, Leon O., 1936-
II. Analog integrated circuits and signal processing. Special
issue.
QA76.87.C44 1998
621.39'9--dc21 97-53185
CIP
Copyright © 1998 by Springer Science+Business Media New York
Originally published by Kluwer Academic Publishers in 1998
Softcover reprint ofthe hardcover Ist edition 1998
Ali rights reserved. No part of this publication may be reproduced, stored in a
retrieval system or transmitted in any farm or by any means, mechanical, photo
copying, recarding, or otherwise, without the priar written permission of the
publisher, Springer Science+Business Media, LLC.
Printed an acid-free paper.
it.
Analog Integrated Circuits and Signal Processing, 15,225-226 (1998)
..." © 1998 Kluwer Academic Publishers, Boston.
Editorial
Hardly anyone would question the importance of dig has resulted in a dramatic three-orders-of-magnitude
ital systems in information procesing. The "digital" improvement in speed, power, and area over the equiv
revolution is both a result and a cause ot technological alent conventional DSP implementations.
changes, especially in semiconductor manufacturing. In this issue, CNN analog-circuit implementations
A few wonder if, or when, analog systems will be are of primary interest.
come extinct. So long as "digital" is merely a way of A companion issue of Field-Programmable Analog
coding continuous information, a convenient abstrac Arrays (FPAAs) is in preparation. CNNs (at least their
tion which allows the ignoring of intermediate states, analog implementations) and FPAAs share the "ana
the question whether "analog" is more important than log" nature, though both address rather different appli
"digital" or vice-versa, is irrelevant. cations. Also, some FPAAs have cellular architectures,
In a great many situations it is convenient or ben which can be used to implement CNNs, but can also be
eficial to ignore intermediate states in a system. It is utilized for entirely different applications.
simplistic, to say the least, to assume that the only inter
esting or practically useful perspective on information Leon O. Chua
processing, is one that originates that way. Stating it Edmund Pierzchala
differently, there is no justification for the belief, not Glenn Gulak
unheard of today, that only digital information process Angel Rodriguez-Vazquez
ing has a bright future.
The present issue contains some recent results of re
search in a broad class of analog systems called "Cel Notes
lular Neural Networks" (CNNs), which consist of an
array of locally-interconnected analog circuit units, I. See the Proceedings of the bi-annuallEEE International Work
shop on Cellular Neural Networks and their Applications (held in
called "cells."
1990, 1992, 1994, and 1996), as well as several special issues on
Although the CNN paradigm allows the cells to be Cellular Neural Networks of the IEEE Transactions on Circuits
of arbitrary complexity, the current generation of CNN and Systems (March 1993) and the International Journal of Cir
chips uses only simple first-order cells (one capacitor cuit Theory and Applications (1992 and 1996). See also: L. O.
Chua "CNN: A Paradigm for Complexity," Int. J. of Bifurration
per cell) in order to maximize the cell density (array
and CIUlOS, Vol. 7, no. 9, September 1997.
size). Although the 8-bit accuracy of current analog
CNN chips is adequate for most CNN-based image
processing applications, much higher accuracy for spe
cialized applications can be achieved via digital emu
lations of the CNN's nonlinear differential equations,
at the cost of a reduction in speed and cell density.
CNNs, introduced in 1987 by Prof. Leon O. Chua
and his collaborators, have sparked worldwide interest
Leon Chua is currently a Professor of electrical en
and have become a very fruitful and active area of re
gineering and computer sciences at the University of
search. The current intense activityl on CNN research
California, Berkeley. His research interests are in the
was triggered by the recent generalization of the CNN
areas of general nonlinear network and system theory.
paradigm by Prof. T. Roska and Prof. L. Chua, called
He has been a consultant to various electronic indus
the CNN Universal Machine, which is a fully pro
tries in the areas of nonlinear network analysis, mod
grammable von Neumann stored program supercom
eling, and computer aided design. He is the author of
puter capable of executing tera (1012) XPS (analog in Introduction to Nonlinear Network Theory (McGraw
structions per second) on a single chip. This invention
Hill, 1969), and a coauthor of the books Computer
226 Chua, Pierzchala, Gulak and Rodrfguez-Vazquez
Aided Analysis of Electronic Circuits: Algorithms and
Computational Techniques (Prentice-Hall, 1975), Lin
ear and Nonlinear Circuits (McGraw-Hill, 1987), and
Practical Numerical Algorithms for Chaotic Systems
(Springer-Verlag, 1989). He has published many pa
pers in the area of nonlinear networks and systems. He
served as editor of the IEEE Transactions on Circuits
and Systems from 1973 to 1975 and as President of
the IEEE Society on Circuits and Systems in 1976. He Glenn Gulak is a professor in the Department of Elec
is presently the editor of the International Journal of trical and Computer Engineering at the University of
Bifurcation and Chaos and a deputy editor of the Inter Toronto. He is a senior member of the IEEE and a
national Journal of Circuit Theory and Applications. registered professional engineer in the province of On
Professor Chua is the holder of six U.S. patents. He tario. His research interests are in the areas of circuits,
is also the recipient of several awards and prizes, in algorithms and VLSI architectures for digital com
cluding the 1967 IEEE Browder J. Thompson Memo munications and signal processing applications. He
rial Prize Award, the 1973 IEEE W,R.G. Baker Prize has received several teaching awards for undergradu
Award, the 1974 Frederick Emmons Terman Award, ate courses taught in both the Department of Computer
the 1976 Miller Research Professorship, the 1982 Se Science and the Department of Electrical in both the
nior Visiting Fellowship at Cambridge University, the Department of Computer Science and the Department
1982/83 Alexander von Humbolt Senior U.S. Scien of Electrical and Computer Engineering at the Univer
tist Award at the Technical University of Munich, the sity of Toronto. Dr. Gulak received his Ph.D. from the
1983/84 Visiting U.S. Scientist Award at Waseda Uni University of Manitoba while holding a Natural Sci
versity, Tokyo, the IEEE Centennial Medal in 1985, ences and Engineering Research Council of Canada
the 1985 Myril B. Reed Best Paper Prize, the 1985 Postgraduate Scholarship. From 1985 to 1988 he was a
and 1989 IEEE Guillemin-Cauer Prizes, and the 1995 research associate in the Information Systems Labora
M.E. Van Valkenburg Award. In 1986 he was awarded a tory and the Computer Systems Laboratory at Stanford
Professeur InvitiInternational Award at the University University. He has served on the ISSCC Signal Pro
of Paris-Sud from the French Ministry of Education. cessing Technical Subcommittee since 1990 and cur
He was also awarded a Doctor Honoris Causa from the rently serves as Program Secretary for ISSCC.
Ecole Poly technique Federale-Lausanne, Switzerland,
in 1983, an Honorary Doctorate from the University
bio and
of 1992, a Doctor Honoris Causa from the Techni
photo
cal University of Budapest, Hungary, in 1994, a Doc
not available
tor Honoris Causa from the University of Santiago de
at time of
Compostela, Spain, in 1995, a Doctor Honoris Causa
print
from the University of Frankfurt, Germany, in 1996,
and a Doctor Honoris Causa from the Technical Uni Angel Rodriguez-Vazquez
versity ofIasa, Romania, in 1997.
bio and
photo
not available
at time of
print
Edmund Pierzchala
2
,~., Analog Integrated Circuits and Signal Processing. 15.227-237 (1998)
© 1998 Kluwer Academic Publishers. Boston,
A 16 16 Cellular Neural Network Universal Chip:
X
The First Complete Single-Chip Dynamic Computer Array with Distributed Memory
and with Gray-Scale Input-Output
J. M. CRUZ I AND L. O. CHUA2
I Department of Electrical Engineering and Computer Sciences. University oj California., Berkeley; Current ,address: Sun Microsystems. Inc,
2Department of Electrical Engineering and Computer Sciences. University of California. Berkeley
Abstract. This paper presents a 16 x 16 Cellular Neural Network Univ~rsal ChiP. with a~alog input and output
ports, which can read in and process gray-scale images in the analog domain, The chip contains about 5,000 analog
multipliers and has been fabricated in a 0.8 /Lm CMOS process.
Key Words: CNN, CNN universal chip, computer array, CMOS, analog, nonlinear dynamics
1. Introduction time samples and discrete amplitude values [10]. These
implementations vary in their area and speed efficiency,
The Cellular Neural Network (CNN) architecture was
with reported data showing higher efficiency in both
presented in [1] and [2]. This architecture allows par
respects for circuits implementing the CNN dynamics
allel analog processing of images using an array of
directly in the analog domain. These reported chips
locally interconnected cells with fixed weights. The
also vary in other aspects of their functionality, includ
local interconnection feature allows efficient VLSI im
ing their capability to locally store and logically ma
plementations [3][4], and many CNN chips have been
nipulate binary images, and their capability to input,
reported since the first operational CNN chip was pre
store, process, and output not only binary images but
sented in 1991 [5].
gray-scale images.
In 1993 the CNN architecture was augmented, in
In this paper we present the first CNN Universal Chip
corporating programmable weights, local storage, and
with the capability to input, store, process, and output
local logic. The new augmented architecture is called
gray-scale images in the analog domain combined with
Cellular Neural Network Universal Machine [6], as it
the capability to locally store and logically process bi
has proven to be able to solve all the algorithms that
nary images. The chip has been fabricated in a 0.8um
can be executed by a Turing Machine [7].
3-metal I-poly technology of HP.
Different types of chip implementations of the CNN
Each cell implements the dynamics of the CNN
Universal Machine or parts of it have been reported
equation with a state resistance by state capacitance
recently [8]-[12], and they are usually referred to as
product below 90 ns. For typical applications the dy
CNN Universal Chips. These implementations in
namics of the entire array settles to a constant state in
clude: (a) designs which operate according to the orig
200 to 250 ns. Each cell has a computing power, ex
inal CNN continuous-time analog-amplitude dynam
cluding input- output operations, of 4 to 5 million pixels
ical equations given in [1], as in the design reported
per second and the entire chip has a peak computing
in [12] and in a programmable chip design without
capability of 1 billion pixels per second. The maxi
logic memory reported in [11]; (b) a design which op
mum electrical power consumption of the entire chip
erates according to a modified continuous-time analog
when operating at 5 Volts is 0.3 Watts.
amplitude CNN equation in which the state variable is
clipped to unity and the final chip output is binary [8];
2. Chip Architecture
(c) a design which emulates the CNN dynamics by
using discrete time samples but maintaining analog Figure 1 shows a photograph of the chip, packaged in
amplitude [9]; and (d) a digital implementation which a 132 PGA. The main components of the chip are as
emulates the original CNN dynamics by using discrete follows.
3
228 1. M. Cruz and L. O. Chua
Fig. I. Photograph of packaged chip.
• Array of 16 x 16 CNN cells for simultaneous pro part of the chip is the 16 x 16 array of cells, which is
cessing of image blocks of 14 x 14 pixels. divided in an inner array of 14 x 14 processing cells
• Analog input port bus with 14 lines to load exter and in 60 boundary cells located at the outer edges
nal binary or gray-scale images column by col and corners. This structure can be used to perform
umn. parallel processing of blocks 14 x 14 pixels at a time
• Serial analog output port to read out gray-scale with different boundary conditions.
processed images or to trace in real time the tran At the left side of the cell array is a 14-line analog
sient evolution of the output variable of any se input bus for the external loading of analog data into
lected cell during the analog processing. the two internal analog memories of each one of the
• Digital output port bus with 14 lines to fast read processing cells. The loading of these memories is
out binary processed images. done column by column. At the bottom of the chip
• Analog control port with 19 analog lines to set the diagram is the column selector decoder, which is used
analog CNN template coefficients. to indicate which column is being selected for input
• Digital control port to set the logic operations and or output. At the right side of the array are: (1) a set
to start and stop analog process. of 14 logic buffers which drive a 14-line logic parallel
Figure 2 shows a diagram of the internal placement output and (2) a 14: 1 analog MUX connected to an
of the main components in the chip. In the central analog buffer providing an analog serial output.
4
A 16 X 16 Cellular Neural Network Universal Chip 229
M.ln.rn~ ,0\001"1 buIT. ..
\
\
'oJ A 001"1
~~ , rli
OUlpUl
~= I ofcdb
-!- ofmoln
I~ •m y
Ii.0
--
-t-
l:I~UI
I' 1
I
InpUI ~ ~0
IIn.log ~ t
bu ~ b u.
I
f-- l-
~
I~
I I'
t -I~ t-t- i-
t=G I--~L ~ '-'--
In'trCO'nned H
L..........- - - L\
[ Column Hi«lor --
Ifor
Fig. 2. Chip structure.
Finally, at the lower left end of the chip is an addi i + k and column j + 1 of the array, where the in
tional cell configured for serial processing of images. dexes k and 1 denote the relative row and column po
The boundary conditions for this cell are set by eight sition of the neighboring cells to respect c(i, j). In
boundary cells, three of them are at the lower left cor this chip each cell provides outputs to its eight closest
ner of the chip, two at the lower right corner of the neighboring cells located at the relative positions given
chip, and the other three are shared with three of the by (k, I) = (-1, -1), (-1,0), (-1, +1), (0, -1),
bottom boundary cells of the main array. This serial (0, + 1), (+ 1, -1), (+ 1,0), (+ 1, +1). Likewise, each
processing cell has its own input and output ports. cell c(i, j) receives eight input currents, one for each
of its eight neighboring cells.
The cell dynamics are determined by a set of 19
3. Description of Processing Cells weights, called template elements, which control the
gain of 18 VCCSs and the value of an independent cur
The core element of the chip is the CNN cell. Figure 3 rent source. These template elements are divided into
shows a schematic of the implemented cell which con nine feedback template elements, nine feedforward el
tains capacitors, resistors, and variable gain VCCSs ements and an independent term. The nine feedback
(multipliers). Observe that the cell provides eight out elements, denoted by a(k, I) where both k and 1 vary
put currents to the eight neighboring cells and receives from -1 to + 1, determine the gain of nine VCCSs con
eight input currents from the eight neighboring cells. trolled by the cell state voltage xU, j) which provides a
Each cell c(i, j) has three analog variables: the in current into each one of the nine cells in the local 3 x 3
put variable, the state variable and the output variable. neighborhood of iteration (the cell itself plus the eight
The indexes i and j denote the absolute row and col neighboring cells). The nine feedforward elements,
umn position of the cell in the array. Each cell cU, j) denoted by b(k, I) where both k and 1 vary from -1 to
at row i and column j provide output currents to a + 1, determine the gain of nine VCCSs controlled by
set of neighboring cells c(i + k, j + I) located at row
5
230 J. M. Cruz and L. O. Chua
Cell c(ij)
... ..,..tI~1-1J-oI)
T
Fig. 3. Ideal circuit diagram of one cell.
the cell input voltage u(i, j) which provide a current four-quadrant multipliers, that perform the multipli
into each one of the nine cells in the local 3 x 3 neigh cation with the nine feedback template coefficients
borhood of iteration. The independent term, denoted a(k, I). Likewise the input variable u(i, j) drives an
by io, represents the value of an independent source other amplifier, which drive another set of nine mul
providing an offset current into the state node of the tipliers that perform the multiplication with the nine
cell. feedforward template coefficients b(k, I). The outputs
Figure 4 shows a more detailed diagram of the cell. of the two set of multipliers are added by pairs, and
The state variable x(i, j) drives a differential output the results are provided in current mode into the state
nonlinear amplifier with a saturation-type transfer char nodes ofthe nine cells in the local 3 x 3 neighborhood.
acteristic f (.) [1]. The amplifier output drives nine Figure 5 shows a schematic of two differential pairs
6
A 16 x 16 Cellular Neural Network Universal Chip 231
in-NE
D-SE
1- ~
in-E
~'_M"I' .5
"c t,<,:
ct, TT
in2 Inl OUI
(pixel data)
in-SW In-S Ift-SE
Fig. 4. Cell structure.
that are used as amplifiers connected to a set of two mul Vb(k.l)' The final output current from each pair of mul
tipliers. The bias current and the transistor sizes of the tipliers is converted to single-ended and fed into the
differential pairs are set so that: (1) the output current state resistors of one of the neighboor cells. Figure 6
saturates to a constant value when the applied voltage shows the transistor implementation the state resistor
is out of the voltage range used to encode the external of a cell.
gray-scale images [1] and (2) the small signal transcon
ductance gain of the differential pair, G m, is greater
than II Rx, where Rx is the state resistor shown in Fig
ure 3, as this is necessary to obtain weight gains greater
than unity as required for most CNN templates [2]. In
4. Chip Layout
our case the voltage range used to encode gray-scale
images is (-250mV, +25OmV), and the Gml Rx ratio is
In a 0.8 /Lm technology, and using three levels of met
3.5, which will allow a gain range of (-3.5, +3.5) for
als, each cell occupies an area of232 /Lm x 263.5 /Lm.
all the template weights, which is sufficient for most
The cells are designed so they can be tiled together,
CNN operations reported in the literature.
as they include all the wiring for intercell connections,
Each one of the four-transistor stages on the right of
data loading, data output, control and bias. Figure 7
Figure 5 can multiply their incoming differential cur
shows a micrograph of one cell with portions of the
rent by multiplication factor between -1 and + 1. The
surrounding eight neighboor cells. Edge cells and the
transistors are sized so that all multiplication factors in
input/output circuit blocks are tiled at the borders of
the entire range (-1, + 1) can be obtained by appro
the processing array. Figure 8 shows a micrograph of
priate values of the external control voltages Va(k.1) and the entire 5.5 mm x 4.7 mm silicon die.
7