Table Of ContentFOUNDATIONS of
WAVELET NETWORKS
and APPLICATIONS
FOUNDATIONS of
WAVELET NETWORKS
and APPLICATIONS
S. Sitharama Iyengar
E. C. Cho
Vir V. Phoha
CHAPMAN & HALL/CRC
A CRC Press Company
Boca Raton London New York Washington, D.C.
Library of Congress Cataloging-in-Publication Data
Iyengar, S. S. (Sundararaja S.)
Foundations of wavelet network and applications / S.S. Iyengar, E.C. Cho, V.V. Phoha.
p. cm.
Includes bibliographical references and index.
ISBN 1-58488-274-3
1. Neural networks (Computer science) 2. Wavelets (Mathematics) I. Cho, E. C. II.
Phoha, Vir V. III. Title.
QA76.87 .I94 2002
006.3'2 —dc21 200205919
CIP
Catalog record is available from the Library of Congress
This book contains information obtained from authentic and highly regarded sources. Reprinted material
is quoted with permission, and sources are indicated. A wide variety of references are listed. Reasonable
efforts have been made to publish reliable data and information, but the authors and the publisher cannot
assume responsibility for the validity of all materials or for the consequences of their use.
Neither this book nor any part may be reproduced or transmitted in any form or by any means, electronic
or mechanical, including photocopying, microfilming, and recording, or by any information storage or
retrieval system, without prior permission in writing from the publisher.
All rights reserved. Authorization to photocopy items for internal or personal use, or the personal or
internal use of specific clients, may be granted by CRC Press LLC, provided that $1.50 per page
photocopied is paid directly to Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923
USA. The fee code for users of the Transactional Reporting Service is ISBN 1-58488-274-
3/02/$0.00+$1.50. The fee is subject to change without notice. For organizations that have been granted
a photocopy license by the CCC, a separate system of payment has been arranged.
The consent of CRC Press LLC does not extend to copying for general distribution, for promotion, for
creating new works, or for resale. Specific permission must be obtained in writing from CRC Press LLC
for such copying.
Direct all inquiries to CRC Press LLC, 2000 N.W. Corporate Blvd., Boca Raton, Florida 33431.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are
used only for identification and explanation, without intent to infringe.
Visit the CRC Press Web site at www.crcpress.com
© 2002 by Chapman & Hall/CRC CRC Press LLC
No claim to original U.S. Government works
International Standard Book Number 1-58488-274-3
Library of Congress Card Number 2002025919
Printed in the United States of America 1 2 3 4 5 6 7 8 9 0
Printed on acid-free paper
Dedication
S.S. Iyengar would like to dedicate this book to Professor Hart-
manis, a Turing Award winner, of Cornell for his pioneering
contribution to complexity theory and to Puneeth, Veneeth, and
Vijeth for their patience for my lectures,
E.C. Cho would like to dedicate this book to his parents and his
family,
V. V. Phoha would like to dedicate this work to Professor William
J.B, Oldham of Texas Tech University for his teachings and for
his contributions to neural networks.
Acknowledgments
This book was made possible and thanks to generous support
from the office of Naval Research Division of Electronics, DARPA-
Sense IT, DOE-ORNL programs, Board of Regents-LEQSF etc.
Special acknowledgments to the following individuals for their
contributions to this book. Chapter 5 was contributed by Ri
cardo Riaza, Raquel Gomez Sanchez, and Pedro J, Zuhria, Chap
ters 6 and 7 were contributed by Javier Eehauz and George
Vaehtsevanos, Chapter 8, contributed by Liangyue Cao, Yiguang
Hong, Harping Fang, and Guowei He, is reproduced with their
permission. Chapter 9 was contributed by Rao and Iyengar,
Thanks for Professor Rao’s technical expertise in fusion learn
ing.
We are grateful to the following individuals in the LSU
Robotics Research and LSU Networking Laboratories who have
helped in the publication of this book: Sridhar Karra, Kanthi
Adapa, and Sumanth Yenduri, Professor Seiden’s help in using
LaTex was very useful.
This book is an extension to our earlier (Prof Iyengar’s) book,
Wavelet Analysis with Applications to Image Processing, CRC
Press (1997), Boca Raton, Florida, which continues to be used
in many universities across the world.
Preface
This text grew out of our lecture notes at Louisiana State Uni
versity, Louisiana Tech University, and Kentucky State Univer
sity and from many questions from our students that indicated
the need for a comprehensive treatment of wavelet networks.
Our two main purposes in writing this text are (1) to present
a systematic treatment of wavelet networks based whenever
possible on fundamental principles, and (2) to present a self
contained treatment of wavelet networks in a clear and concise
manner.
Readers who sample the literature will note a recent vigor
ous growth in this area. Wavelet networks are relatively new;
however, their applications and growth have come from many
areas including wavelet theory, neural networks, statistics, com
puter science, pattern recognition, and communication, to name
a few. In this text, neural networks and wavelet theory, tradi
tionally taught in two different disciplines, are integrated under
the common theme of wavelet networks.
In our teaching and in research, we found that the material
required to teach wavelet networks in an organized and compre
hensive manner was scattered in many research monographs;
there is no book that presents a comprehensive treatment of
material in one place. This book is an attempt to fill that void.
Approach
We have followed a pedagogical approach in this text. The
book presents a rigorous treatment of both wavelets and neu
ral networks, because the foundations of wavelet networks can
be found in neural networks, and wavelets-wavelet networks
combine the decomposition powers of wavelets with universal
approximation properties of neural networks, so a good under
standing of wavelets and networks is essential to appreciate the
universal approximation properties of wavelet networks.
Theory and techniques of wavelet networks are mostly math
ematical in nature, but we have focused on providing insight and
understanding rather than establishing rigorous mathematical
foundations, so we have avoided detailed proofs of theorems. A
deliberate attempt is made to provide a balance between theory
and applications of wavelet networks. We present the mate
rial at a level that can be understood by senior undergraduate
students in engineering and science.
To follow this text no specific mathematical background is
expected, with the exception of basic calculus. All the other
requisite mathematical preliminaries are given in the hrst part.
Topics
The book has two parts. Part I deals with the foundational
aspects of wavelet networks, and Part II presents applications
of wavelet networks, A brief description of each part of the book
follows.
Part I is divided into four chapters. Chapter 1 deals with
mathematical preliminaries. Wavelet transforms have come to
be an important tool for approximation analysis of non-linear
equations, with a wide and ever increasing range of applica
tions, in recent years. The versatility of wavelet analytic tech
niques has forged new interdisciplinary bonds by offering com
mon solutions to diverse problems and providing a platform to
wavelet networks. Chapter 2 describes in detail wavelet anal
ysis and construction of wavelets. Chapter 3 addresses neural
networks, where the stress is on pereeptron type networks and
the back propagation learning algorithm with its various adap
tations, Starting with a single MeCulloeh-Pitts neuron model
to build intuition, we present pereeptrons and gradient-deseent
learning, Hebb type learning is the basis of most artificial neural
network models, and competition is a major learning principle,
so we develop these concepts at some length. We also present
other neural architectures, because it is our belief that the future
lies in adapting and integrating wavelets to other neural archi
tectures, Our presentation of other neural architecture gives
the reader a firm foundation in the concepts, which we hope
will generate new ideas. Chapter 4 is the capstone finale that
deals with wavelet networks. In these final segments, we outline
the theory underlying wavelet networks and present some basic
applications.
Part II, contributed by many distinguished researchers in
this area, is divided into five chapters. There is a great amount
of work available that uses wavelet networks, much of which de
serves inclusion here. Although we provide a review of current
work, we have selected that material which, in our view best
represents important applications of wavelet networks. Chap
ter 5 (contributed by Ricardo Riaza, Raquel Gomez Sanchez,
and Pedro J, Zufiria) presents recurrent learning in dynamical
wavelet networks. This chapter is a bridge between Part I and
Part II of the book. It could easily have been included in Part I,
but we chose to include it in Part II because of the advanced na
ture of the concepts presented therein. Chapter 6 (contributed
by Javier Eehauz and George Vaehtsevanos) presents the use
of wavelet networks as trading advisors in the stock market;
Chapter 7 (contributed by Javier Eehauz, and George Vaeht
sevanos) presents radial wavelet networks as classifiers in elee-
troeneephalographie drug detection; Chapter 8 (contributed by
Liangvue Cao, Yiguang Hong, Haiping Fang, and Guowei He)
presents the use and application of wavelet networks for pre
diction of chaotic time series including the short term and long
term predictions, where a number of times series generated from
chaotic systems such as the Maekev-Glass equation are tested.
Chapter 9 (contributed by Nageswara S.V, Rao and S.S.
Iyengar) presents concept learning and deals with approxima
tion by wavelet networks.
The book includes a detailed bibliography arranged under
chapter headings to help the reader easily locate references of
interest. The bibliography contains additional references not
explicitly cited in text, and will provide the reader with a rich
resource for further exploration.
