Table Of ContentNEURAL
NETWORK
DATA
ANALYSIS
USING
SIMULNET™
Springer Science+Business Media, LLC
J.
Edward Rzempoluck
NEURAL
NETWORK
DATA
ANALYSIS
USING
SIMULNETTM
EXIRA
MATERIALS
extras-springer.cam
With 45 Figures
i
Springer
Edward J. Rzempoluck
Brain-Behaviour Laboratory
School of Applied Sciences
Simon Fraser University
Burnaby, BC V5A lS6
Canada
Library of Congress Cataloging-in-Publication Data
Rzempoluck, Edward J.
Neural network data analysis using Simulnet / Edward J. Rzempoluck.
p. cm.
Includes index.
Additional material to this book can be downloaded from http://extras.springer.com.
ISBN 978-1-4612-7262-5 ISBN 978-1-4612-1746-6 (eBook)
DOI 10.1007/978-1-4612-1746-6
1. Neural networks (Computer science). 2. Numerical analysis-Data
processing. 3. Simulnet. I. Title.
QA76.87.R94 1997
519.5'OI'13632--dc21 97-16666
Printed on acid-free paper.
© 1998 Springer Science+Business Media New York
Originally published by Springer-Verlag New York, Inc. in 1998
Softcover reprint of the hardcover 1st edition 1998
All rights reserved. This work may not be translated or copied in whole or in part without the written permission
of the publisher Springer Science+Business Media, LLC, except for
brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now
known or hereafter developed is forbidden.
The use of general descriptive names, trade names, trademarks, etc., in this publication, even if the former are not
especially identified, is not to be taken as a sign that such names, as understood by the Trade Marks and
Merchandise Marks Act, may accordingly be used freely by anyone.
Although care has been taken to ensure that the material in this book and the accompanying software are free
from mistakes, the author makes no warranty, either express or implied, about the suitability of this material for
any particular purpose. In no event will the author be liable for any direct or indirect damages arising from the
use of the information presented in this book or the accompanying software.
Production managed by Terry Kornak; manufacturing supervised by Johanna Tschebull.
Photocomposed copy prepared by Springer, using the author's Microsoft Word files.
9 8 765 432 1
ISBN 978-1-4612-7262-5
Acknow ledgments
My sincere thanks to the members of the Brain Behaviour Laboratory at Simon Fraser
University for their many helpful comments and criticisms during the development of
Simulnet. I want to thank in particular Dr. Harold Weinberg, director of the Brain Be
haviour Laboratory, for his guidance and for his emphasis on the importance of defining
the substantive question at the beginning of a research project.
Parts of this book had their origins in a set of laboratory exercises for a biological
psychology course at Simon Fraser University. To the students who labored through
those exercises, my thanks for their patience and the useful feedback they provided.
I want to thank Dr. Martin Gilchrist at Springer for taking a chance on this book, and
Springer's production department for its patience with my many revisions to the text and
to the software.
Finally, I want to express my gratitude to my darling wife for her insight, help and
patience with the editing of the book.
Contents
Introd.uction......................................................................................... 1
Scope of this Text .................................................................................................. 1
What Is Expected from the Reader ........................................................................ 3
An Outline .............................................................................................................. 3
Computer Requirements......................................................................................... 4
1 The Simulnet Desktop .............................................................. . 5
Introduction............................................................................................................ 5
Desktop Components ............................................................................................. 5
2 Data Analysis .............................................................................. 13
Introduction........... ........... ...... ...... ... .............. ... .... ............. ...... .......... ...... ..... ... ... .... 13
The Substantive Question .. ... ......... ....... ..... ... ..... ......... ...... ........ ..... ...... ......... ......... 15
Neural Network Analysis....................................................................................... 15
Genetic Algorithms and Neural Networks ............................................................. 75
The Probabilistic Network ..................................................................................... 91
The Vector Quantizer Network.............................................................................. 102
Assessing the Significance of Network Results ..................................................... 113
Network Application Examples ............................................................................. 116
Fractal Dimension Analysis ................................................................................... 128
Fourier Analysis ..................................................................................................... 144
Eigenvalue Analysis............................................................................................... 146
Coherence and Phase Analysis............................................................................... 153
Mutual Information Analysis ................................................................................. 160
Correlation and Covariance Analysis..................................................................... 163
3 Acquiring and Conditioning Network Data ............................ 171
Introduction.............. ..... .......... ....... ....... ... ..... ....... ....... ...... ........ ... .......... ........ ........ 171
Data Specification .................................................................................................. 171
Data Collection....................................................................................................... 173
Data Inspection . ..... ... ...... ........ ...... ... ..... ....... ... .... ........ ....... .............. .... .......... ........ 177
Data Conditioning .................................................................................................. 180
Detrend-Order 0 .................................................................................................. 184
Standardize Columns ............................................................................................. 185
Frequency Filtering ................................................................................................ 187
Principal Component Analysis............................................................................... 189
Principal Component Data Reduction.................................................................... 199
viii Contents
4 A Data Analysis Protocol........................................................... 203
Introduction ........................................................................................................... . 203
A Preprocessing Checklist ................................................................................... .. 203
Analyzing Experimental Data .............................................................................. .. 204
Glossary .............................................................................................. . 213
Index .................................................................................................... . 219
Introduction
Scope of this Text
This text is intended to provide the reader with an introduction to the analysis of numeri
cal data using neural networks. Neural networks as data analytic tools allow data to be
analyzed in order to discover and model the functional relationships among the recorded
variables. Such data may be empirical. It may originate in an experiment in which the
values of one or more dependent variables are recorded as one or more independent vari
ables are manipulated. Alternatively, the data may be observational rather than empirical
in nature, representing historical records of the behavior of some set of variables. An ex
ample would be the values of a number of financial commodities, such as stocks or
bonds. Finally, the data may originate in a computational model of some physical proc
ess. Instead of recording variables of the physical process, the computer model could be
run to generate an artificial analog of the physical data.
Since data in virtually any native form can be expressed in numerical format, the
scope of the analytical techniques and procedures that will be presented in this text is es
sentially unlimited. Sources of data include research work in a range of disciplines as di
verse as neuroscience, biomedicine, geophysics, psychology, sociology, archeology, eco
nomics, and astrophysics.
An often fruitful approach to data analysis involves the use of neural network func
tions. Neural network functions discussed in this text include multilayer feedforward
networks trained using the error backpropagation algorithm; neural network-genetic al
gorithm hybrids; generalized regression neural networks, learning vector quantizer net
works; and self-organizing feature maps.
The obvious question is then, how can such neural networks be of use in the analysis
of data? Most significantly, neural networks can behave as universal function approxi
mators. As such, neural networks are capable of modeling the functional associations
among the independent and dependent variables in the data. As an example of this capa
bility, neural networks can be used as pattern classifiers. They can be trained to assign a
pattern of values of a set of independent variables, to one of a number of categories.
The application of a number of other data analytical techniques will also be dis
cussed. These techniques have been selected for inclusion on the basis of the following
criteria. First, the techniques can be usefully applied as adjuncts to neural network analy
sis, performing anyone of a variety of preprocessing operations on the data. Second, the
E. J. Rzempoluck, Neural Network Data Analysis Using Simulnet™
© Springer-Verlag New York, Inc. 1998
2 Neural Network Data Analysis Using Simulnet
techniques are some that are not generally discussed at a level that is accessible to a wide
range of nonspecialist users.
Techniques in the first category, those intended to perform a data preprocessing
function for subsequent network analysis, include correlation and covariance analysis,
eigenvalue analysis, and Fourier analysis. Correlation and covariance are both estimates
of the strength of linear association between a pair of variables. Eigenvalue analysis pro
vides an estimate of the complexity inherent in the data by finding a set of features, mu
tually independent combinations of the measured variables, that can account for a
majority of the variance in the data. Fourier analysis, like eigenvalue analysis, also re
packages the variance in the data. In the case of Fourier analysis however, the repackag
ing is done in terms of predefined features; sine and cosine functions. Essentially, Fourier
analysis acts as a mathematical prism. As a prism separates a beam of white light into its
component colors, so Fourier analysis shows the magnitudes of the different frequency
components within the data.
Techniques in the second category include fractal dimension analysis, mutual infor
mation analysis, and coherence analysis. Fractal dimension analysis can, under suitable
conditions, provide an estimate of the complexity of the multivariate dynamical system
from which the data has been sampled. Coherence and mutual information are both
measures of association. Mutual information is an index of the degree of general, rather
than simply linear, association between two data sets. If the data represent a time series
the data values are recorded at some interval over a span of time-then mutual informa
tion can be viewed as an index of general association in the time domain. Coherence is an
analogous measure in the frequency domain. Coherence is an index of the degree of as
sociation between the respective frequency components in two data sets.
All of these analytic techniques can be used in order to directly address questions
about the data. In addition many of these procedures can be used to preprocess data that
will then be analyzed using a neural network. Data preprocessing includes operations car
ried out on data that are intended to reduce the computational or analytic load on subse
quent analyses, in particular, analyses using neural networks. Preprocessing can
simultaneously achieve a number of different goals.
First, preprocessing the data can reduce the computational load on a network. Such a
reduction may not only be convenient, but it may be critically important if the analysis
has to be done within a limited time span. Neural networks are inherently parallel proc
esses. As such they can easily become effectively unusable when implemented on a serial
machine and, if at the same time, they are asked to deal with a large volume of data.
Second, preprocessing can reduce the analytic load on a network. When presented
with suitably preprocessed data, the network only needs to deal with a subset of the fea
tures within the data. These features would be ones that are hypothesized to be relevant
to the analytic task at hand. For example, if the task is to classify electroencephalo
graphic (EEG) recordings, the experimenter might entertain the hypothesis that the ex
perimental manipulation has had an effect on the EEG in terms of between-channel cor
relations. It might be useful to preprocess the data by computing between-channel
correlations. The volume of data represented by these correlations would be small in
comparison with the volume of the raw EEG data. The neural network would then be
trained on only the correlations, rather than on the raw data.
Introduction 3
In sum, data preprocessing can help the neural network by selecting from the entire
mass of data those components that the researcher has independent reason to believe may
be most relevant to the analytic question. To the extent that the selected features are per
tinent to this question, the neural network will often have a much easier job of learning to
model the data.
Supporting the discussions of the various data analytic techniques, a number of ex
amples and case studies will be presented and discussed. The procedures in the examples
can be carried out using the software supplied with this text, Simulnet. Simulnet is a pro
gram that is designed to carry out a wide range of analytical, transformational, and
graphical operations on numerical data. Simulnet shares some features with traditional
statistical software, but extends the capabilities of such software by offering access to
analytically powerful neural network-based functions. The significance of such neural
network-based functions is that they allow data to be explored for relationships or pat
terns that might not be discoverable using more traditional statistical procedures.
What Is Expected from the Reader
The reader is expected to bring to this text two skills. The first skill involves computer
literacy. The reader is expected to be familiar with the operation of Windows-based
software. Readers without this background can prepare themselves by working through
one of the many texts that are available to introduce novice users to the Windows oper
ating system. The second skill involves mathematical background. This text generally as
sumes a mathematical background on the part of the reader that is equivalent to a first
year course in calculus and related topics.
An Outline
• Section 1 presents a description and explanation of the components of the Simulnet
desktop.
• Section 2 discusses approaches to data analysis, with respect to the types of ques
tions that are correspondingly addressed. This section will describe a range of ana
lytical techniques, and will focus on the application of neural networks to the analy
sis of data. Each of these analytical approaches is illustrated using a number of
preprepared examples that demonstrate in detail how these functions are used.
• Section 3 addresses the issue of how readers can use the analytical functions in
Simulnet to explore their own data. This discussion will involve the data visualiza
tion and transformation functions that Simulnet provides in order to allow data ana
lysts to preprocess and condition their data prior to analysis.
• Section 4 presents a data analysis protocol; a generalized procedure that can be fol
lowed in a first attempt at examining a set of data.
• Section 5 is a glossary of some of the more complex terms used in this text.
