Computational intelligence paradigms : theory & applications using MATLAB

834 Pages·2010·8.75 MB·English

by  S. Sumathi and Surekha Paneerselvam

Computational
Intelligence
Paradigms
Theory and
Applications
using MATLAB®
© 2010 by Taylor and Francis Group, LLC
Computational
Intelligence
Paradigms
Theory and
Applications
using MATLAB®
S. sumathi
surekha p.
Boca Raton London New York
CRC Press is an imprint of the
Taylor & Francis Group, an informa business
A CHAPMAN & HALL BOOK
© 2010 by Taylor and Francis Group, LLC
MATLAB ® and Simulink ® are trademarks of the Math Works, Inc. and are used with permission. The Mathworks does
not warrant the accuracy of the text or exercises in this book. This book’s use or discussion of MATLAB ® and Simulink
® software or related products does not constitute endorsement or sponsorship by the Math Works of a particular peda-
gogical approach or particular use of the MATLAB ® and Simulink ® software.
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2010 by Taylor and Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Printed in the United States of America on acid-free paper
10 9 8 7 6 5 4 3 2 1
International Standard Book Number: 978-1-4398-0902-0 (Hardback)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been
made to publish reliable data and information, but the author and publisher cannot assume responsibility for the valid-
ity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright
holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this
form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may
rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or uti-
lized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopy-
ing, microfilming, and recording, or in any information storage or retrieval system, without written permission from the
publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://
www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923,
978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For
organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for
identification and explanation without intent to infringe.
Library of Congress Cataloging‑in‑Publication Data
Sumathi, S., 1968-
Computational intelligence paradigms : theory & applications using MATLAB / S. Sumathi and
Surekha Paneerselvam.
p. cm.
Includes bibliographical references and index.
ISBN 978-1-4398-0902-0 (hard back : alk. paper)
1. Computational intelligence. 2. MATLAB. I. Paneerselvam, Surekha, 1980- II. Title.
Q342.S94 2010
006.3--dc22 2009022113
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
© 2010 by Taylor and Francis Group, LLC
Contents
Preface xvii
1 Computational Intelligence 1
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Primary Classes of Problems for CI Techniques . . . . 5
1.2.1 Optimization . . . . . . . . . . . . . . . . . . . 5
1.2.2 NP-Complete Problems . . . . . . . . . . . . . 6
1.3 Neural Networks . . . . . . . . . . . . . . . . . . . . . 7
1.3.1 Feed Forward Neural Networks . . . . . . . . . 8
1.4 Fuzzy Systems . . . . . . . . . . . . . . . . . . . . . . 9
1.4.1 Fuzzy Sets . . . . . . . . . . . . . . . . . . . . 9
1.4.2 Fuzzy Controllers . . . . . . . . . . . . . . . . 11
1.5 Evolutionary Computing . . . . . . . . . . . . . . . . . 12
1.5.1 Genetic Algorithms . . . . . . . . . . . . . . . 13
1.5.2 Genetic Programming . . . . . . . . . . . . . . 14
1.5.3 Evolutionary Programming . . . . . . . . . . . 15
1.5.4 Evolutionary Strategies . . . . . . . . . . . . . 15
1.6 Swarm Intelligence . . . . . . . . . . . . . . . . . . . . 16
1.7 Other Paradigms . . . . . . . . . . . . . . . . . . . . . 17
1.7.1 Granular Computing . . . . . . . . . . . . . . 18
1.7.2 Chaos Theory . . . . . . . . . . . . . . . . . . 20
1.7.3 Artificial Immune Systems . . . . . . . . . . . 21
1.8 Hybrid Approaches . . . . . . . . . . . . . . . . . . . . 22
1.9 Relationship with Other Paradigms . . . . . . . . . . . 23
1.10 Challenges To Computational Intelligence . . . . . . . 25
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Review Questions . . . . . . . . . . . . . . . . . . . . . . . . 27
2 Artificial Neural Networks with MATLAB 29
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . 29
2.2 A Brief History of Neural Networks . . . . . . . . . . . 29
2.3 Artificial Neural Networks . . . . . . . . . . . . . . . . 32
2.3.1 Comparison of Neural Network to the Brain . 33
2.3.2 Artificial Neurons . . . . . . . . . . . . . . . . 33
© 2010 by Taylor and Francis Group, LLC
v
vi
2.3.3 Implementation of Artificial Neuron Electroni-
cally . . . . . . . . . . . . . . . . . . . . . . . . 35
2.3.4 Operations of Artificial Neural Network . . . . 37
2.3.5 Training an Artificial Neural Network . . . . . 40
2.3.6 Comparison between Neural Networks, Tradi-
tional Computing, and Expert Systems . . . . 44
2.4 Neural Network Components . . . . . . . . . . . . . . 46
2.4.1 Teaching an Artificial Neural Network . . . . . 52
2.4.2 Learning Rates . . . . . . . . . . . . . . . . . . 54
2.4.3 Learning Laws . . . . . . . . . . . . . . . . . . 55
2.4.4 MATLAB Implementation of Learning Rules . 61
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
Review Questions . . . . . . . . . . . . . . . . . . . . . . . . 67
3 Artificial Neural Networks - Architectures and Algo-
rithms 69
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . 69
3.2 LayeredArchitectures . . . . . . . . . . . . . . . . . . 71
3.2.1 Single-Layer Networks . . . . . . . . . . . . . . 71
3.2.2 Multilayer Networks . . . . . . . . . . . . . . . 73
3.3 Prediction Networks . . . . . . . . . . . . . . . . . . . 75
3.3.1 The Perceptron . . . . . . . . . . . . . . . . . 75
3.3.2 MATLABImplementationofaPerceptronNet-
work . . . . . . . . . . . . . . . . . . . . . . . 79
3.3.3 Feedforward Back-PropagationNetwork . . . . 83
3.3.4 Implementation of BPN Using MATLAB . . . 91
3.3.5 Delta Bar Delta Network . . . . . . . . . . . . 95
3.3.6 Extended Delta Bar Delta . . . . . . . . . . . 97
3.3.7 Directed Random Search Network . . . . . . . 100
3.3.8 Functional Link Artificial Neural Network
(FLANN) or Higher-Order Neural Network . . 102
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
Review Questions . . . . . . . . . . . . . . . . . . . . . . . . 106
4 Classification and Association Neural Networks 109
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . 109
4.2 Neural Networks Based on Classification . . . . . . . . 109
4.2.1 Learning Vector Quantization . . . . . . . . . 110
4.2.2 Implementation of LVQ in MATLAB . . . . . 115
4.2.3 Counter-PropagationNetwork . . . . . . . . . 116
4.2.4 Probabilistic Neural Network . . . . . . . . . . 123
4.2.5 Implementation of the Probabilistic Neural Net
Using MATLAB . . . . . . . . . . . . . . . . . 128
© 2010 by Taylor and Francis Group, LLC
vii
4.3 Data Association Networks . . . . . . . . . . . . . . . 129
4.3.1 Hopfield Network . . . . . . . . . . . . . . . . 130
4.3.2 Implementation of Hopfield Network in MAT-
LAB. . . . . . . . . . . . . . . . . . . . . . . . 133
4.3.3 Boltzmann Machine . . . . . . . . . . . . . . . 134
4.3.4 Hamming Network . . . . . . . . . . . . . . . . 137
4.3.5 Bi-Directional Associative Memory. . . . . . . 141
4.4 Data Conceptualization Networks . . . . . . . . . . . . 146
4.4.1 Adaptive Resonance Network . . . . . . . . . . 146
4.4.2 Implementation of ART Algorithm in MATLAB 151
4.4.3 Self-Organizing Map . . . . . . . . . . . . . . . 153
4.5 Applications Areas of ANN . . . . . . . . . . . . . . . 158
4.5.1 Language Processing . . . . . . . . . . . . . . 159
4.5.2 Character Recognition. . . . . . . . . . . . . . 160
4.5.3 Data Compression . . . . . . . . . . . . . . . . 160
4.5.4 Pattern Recognition . . . . . . . . . . . . . . . 160
4.5.5 Signal Processing . . . . . . . . . . . . . . . . 161
4.5.6 Financial . . . . . . . . . . . . . . . . . . . . . 161
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
Review Questions . . . . . . . . . . . . . . . . . . . . . . . . 162
5 MATLAB Programs to Implement Neural Networks 165
5.1 Illustration 1: Coin Detection Using Euclidean Distance
(Hamming Net) . . . . . . . . . . . . . . . . . . . . . . 165
5.2 Illustration2:LearningVectorQuantization-Clustering
Data Drawn from Different Regions . . . . . . . . . . 171
5.3 Illustration 3: Character Recognition Using Kohonen
Som Network . . . . . . . . . . . . . . . . . . . . . . . 174
5.4 Illustration 4: The Hopfield Network as an Associative
Memory . . . . . . . . . . . . . . . . . . . . . . . . . . 182
5.5 Illustration 5: Generalized Delta Learning Rule and
Back-Propagationof Errors for a Multilayer Network . 187
5.6 Illustration 6: Classification of Heart Disease Using
Learning Vector Quantization . . . . . . . . . . . . . . 189
5.7 Illustration7: Neural Network Using MATLAB Simulink 198
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
Review Questions . . . . . . . . . . . . . . . . . . . . . . . . 200
6 MATLAB-Based Fuzzy Systems 203
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . 203
6.2 Imprecision and Uncertainty . . . . . . . . . . . . . . 205
6.3 Crisp and Fuzzy Logic . . . . . . . . . . . . . . . . . . 205
6.4 Fuzzy Sets . . . . . . . . . . . . . . . . . . . . . . . . . 207
© 2010 by Taylor and Francis Group, LLC
viii
6.5 Universe . . . . . . . . . . . . . . . . . . . . . . . . . . 209
6.6 Membership Functions . . . . . . . . . . . . . . . . . . 210
6.6.1 Types of Membership Functions . . . . . . . . 212
6.6.2 Membership Functions in the MATLAB Fuzzy
Logic Toolbox . . . . . . . . . . . . . . . . . . 214
6.6.3 MATLAB Code to Simulate MembershipFunc-
tions . . . . . . . . . . . . . . . . . . . . . . . 217
6.6.4 TranslationofParametersbetweenMembership
Functions Using MATLAB . . . . . . . . . . . 223
6.7 Singletons . . . . . . . . . . . . . . . . . . . . . . . . . 224
6.8 Linguistic Variables . . . . . . . . . . . . . . . . . . . 225
6.9 Operations on Fuzzy Sets . . . . . . . . . . . . . . . . 225
6.9.1 Fuzzy Complements . . . . . . . . . . . . . . . 227
6.9.2 Fuzzy Intersections: t-norms . . . . . . . . . . 230
6.9.3 Fuzzy Unions: t-conorms . . . . . . . . . . . . 233
6.9.4 Combinations of Operations . . . . . . . . . . 235
6.9.5 MATLAB Codes for Implementation of Fuzzy
Operations . . . . . . . . . . . . . . . . . . . . 236
6.9.6 AggregationOperations . . . . . . . . . . . . . 239
6.10 Fuzzy Arithmetic . . . . . . . . . . . . . . . . . . . . . 242
6.10.1 Arithmetic Operations on Intervals . . . . . . 243
6.10.2 Arithmetic Operations on Fuzzy Numbers . . 244
6.10.3 Fuzzy Arithmetic Using MATLAB Fuzzy Logic
Toolbox . . . . . . . . . . . . . . . . . . . . . . 246
6.11 Fuzzy Relations . . . . . . . . . . . . . . . . . . . . . . 247
6.12 Fuzzy Composition . . . . . . . . . . . . . . . . . . . . 251
6.12.1 MATLAB Code to Implement Fuzzy Composi-
tion . . . . . . . . . . . . . . . . . . . . . . . . 254
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
Review Questions . . . . . . . . . . . . . . . . . . . . . . . . 260
7 Fuzzy Inference and Expert Systems 261
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . 261
7.2 Fuzzy Rules . . . . . . . . . . . . . . . . . . . . . . . . 261
7.2.1 Generation of Fuzzy Rules . . . . . . . . . . . 262
7.2.2 Disintegration of Rules . . . . . . . . . . . . . 262
7.2.3 Aggregationof Rules . . . . . . . . . . . . . . 263
7.3 Fuzzy Expert System Model . . . . . . . . . . . . . . . 264
7.3.1 Fuzzification . . . . . . . . . . . . . . . . . . . 264
7.3.2 Fuzzy Rule Base and Fuzzy IF-THEN Rules . 265
7.3.3 Fuzzy Inference Machine . . . . . . . . . . . . 266
7.3.4 Defuzzification . . . . . . . . . . . . . . . . . . 267
© 2010 by Taylor and Francis Group, LLC
ix
7.3.5 Implementation of Defuzzification using MAT-
LAB Fuzzy Logic Toolbox . . . . . . . . . . . 273
7.4 Fuzzy Inference Methods . . . . . . . . . . . . . . . . 276
7.4.1 Mamdani’s Fuzzy Inference Method . . . . . . 278
7.4.2 Takagi–Sugeno Fuzzy Inference Method . . . . 281
7.4.3 Tsukamoto Fuzzy Inference Method . . . . . . 282
7.5 Fuzzy Inference Systems in MATLAB . . . . . . . . . 283
7.5.1 Mamdani-Type Fuzzy Inference . . . . . . . . 286
7.5.2 Sugeno-Type Fuzzy Inference . . . . . . . . . 292
7.5.3 Conversionof Mamdani to Sugeno System . . 295
7.6 Fuzzy Automata and Languages . . . . . . . . . . . . 297
7.7 Fuzzy Control . . . . . . . . . . . . . . . . . . . . . . . 298
7.7.1 Fuzzy Controllers . . . . . . . . . . . . . . . . 300
7.7.2 A Fuzzy Controller in MATLAB . . . . . . . . 302
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
Review Questions . . . . . . . . . . . . . . . . . . . . . . . . 305
8 MATLAB Illustrations on Fuzzy Systems 307
8.1 Illustration 1: Application of Fuzzy Controller Using
MATLAB — Fuzzy Washing Machine . . . . . . . . . 307
8.2 Illustration 2 - Fuzzy Control System for a Tanker Ship 317
8.3 Illustration 3 - Approximation of Any Function Using
Fuzzy Logic . . . . . . . . . . . . . . . . . . . . . . . . 336
8.4 Illustration 4 - Building Fuzzy Simulink Models . . . . 343
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348
Review Questions . . . . . . . . . . . . . . . . . . . . . . . . 348
9 Neuro-Fuzzy Modeling Using MATLAB 351
9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . 351
9.2 Cooperative and Concurrent Neuro-Fuzzy Systems . . 352
9.3 Fused Neuro-Fuzzy Systems . . . . . . . . . . . . . . . 352
9.3.1 Fuzzy Adaptive Learning Control Network
(FALCON) . . . . . . . . . . . . . . . . . . . . 353
9.3.2 Adaptive Neuro-Fuzzy Inference System (AN-
FIS) . . . . . . . . . . . . . . . . . . . . . . . . 353
9.3.3 Generalized Approximate Reasoning-Based In-
telligent Control (GARIC) . . . . . . . . . . . 355
9.3.4 Neuro-Fuzzy Control (NEFCON) . . . . . . . 356
9.3.5 Fuzzy Inference and Neural Network in Fuzzy
Inference Software (FINEST) . . . . . . . . . . 360
9.3.6 Fuzzy Net (FUN) . . . . . . . . . . . . . . . . 362
9.3.7 Evolving Fuzzy Neural Network (EFuNN) . . . 363
© 2010 by Taylor and Francis Group, LLC
x
9.3.8 Self–Constructing Neural Fuzzy Inference Net-
work (SONFIN) . . . . . . . . . . . . . . . . . 364
9.3.9 Evolutionary Design of Neuro-Fuzzy Systems . 364
9.4 Hybrid Neuro-Fuzzy Model — ANFIS . . . . . . . . . 367
9.4.1 ArchitectureofAdaptiveNeuro-FuzzyInference
System . . . . . . . . . . . . . . . . . . . . . . 367
9.4.2 Hybrid Learning Algorithm . . . . . . . . . . 370
9.5 Classification and Regression Trees . . . . . . . . . . . 372
9.5.1 CART — Introduction . . . . . . . . . . . . . 372
9.5.2 Node Splitting Criteria . . . . . . . . . . . . . 373
9.5.3 Classification Trees . . . . . . . . . . . . . . . 374
9.5.4 Regression Trees . . . . . . . . . . . . . . . . . 375
9.5.5 Computational Issues of CART. . . . . . . . . 375
9.5.6 Computational Steps . . . . . . . . . . . . . . 376
9.5.7 Accuracy Estimation in CART . . . . . . . . . 379
9.5.8 Advantages of Classification and Regression
Trees . . . . . . . . . . . . . . . . . . . . . . . 380
9.6 Data Clustering Algorithms . . . . . . . . . . . . . . . 382
9.6.1 System Identification Using Fuzzy Clustering . 382
9.6.2 Hard C-Means Clustering . . . . . . . . . . . . 383
9.6.3 Fuzzy C-Means (FCM) Clustering . . . . . . . 386
9.6.4 Subtractive Clustering. . . . . . . . . . . . . . 388
9.6.5 Experiments . . . . . . . . . . . . . . . . . . . 389
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393
Review Questions . . . . . . . . . . . . . . . . . . . . . . . . 393
10 Neuro-Fuzzy Modeling Using MATLAB 395
10.1 Illustration 1 - Fuzzy Art Map . . . . . . . . . . . . . 395
10.2 Illustration 2: Fuzzy C-Means Clustering — Compara-
tive Case Study . . . . . . . . . . . . . . . . . . . . . . 402
10.3 Illustration 3 - Kmeans Clustering . . . . . . . . . . . 403
10.4 Illustration 4 - Neuro-Fuzzy System Using Simulink . 411
10.5 Illustration 5 - Neuro-Fuzzy System Using Takagi–
Sugeno and ANFIS GUI of MATLAB . . . . . . . . . 414
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
Review Questions . . . . . . . . . . . . . . . . . . . . . . . . 418
11 Evolutionary Computation Paradigms 419
11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . 419
11.2 Evolutionary Computation . . . . . . . . . . . . . . . 420
11.3 Brief History of Evolutionary Computation . . . . . . 422
11.4 Biologicaland Artificial Evolution . . . . . . . . . . . 423
11.4.1 Expressions Used in Evolutionary Computation 423
© 2010 by Taylor and Francis Group, LLC
xi
11.4.2 Biological Evolution Inspired by Nature . . . . 423
11.4.3 Evolutionary Biology . . . . . . . . . . . . . . 425
11.5 Flow Diagram of a Typical Evolutionary Algorithm . 428
11.6 Models of Evolutionary Computation . . . . . . . . . . 430
11.6.1 Genetic Algorithms (GA) . . . . . . . . . . . . 430
11.6.2 Genetic Programming (GP). . . . . . . . . . . 431
11.6.3 Evolutionary Programming (EP) . . . . . . . . 435
11.6.4 Evolutionary Strategies (ESs) . . . . . . . . . 436
11.7 Evolutionary Algorithms . . . . . . . . . . . . . . . . . 439
11.7.1 Evolutionary Algorithms Parameters . . . . . 442
11.7.2 Solution Representation . . . . . . . . . . . . . 442
11.7.3 Fitness Function . . . . . . . . . . . . . . . . . 443
11.7.4 Initialization of Population Size . . . . . . . . 444
11.7.5 Selection Mechanisms . . . . . . . . . . . . . . 444
11.7.6 CrossoverTechnique . . . . . . . . . . . . . . . 451
11.7.7 Mutation Operator . . . . . . . . . . . . . . . 455
11.7.8 Reproduction Operator . . . . . . . . . . . . . 458
11.8 Evolutionary Programming . . . . . . . . . . . . . . . 459
11.8.1 History . . . . . . . . . . . . . . . . . . . . . . 459
11.8.2 Procedure of Evolutionary Programming . . . 460
11.8.3 EPs and GAs. . . . . . . . . . . . . . . . . . . 461
11.8.4 Algorithm of EP . . . . . . . . . . . . . . . . . 461
11.8.5 Flowchart . . . . . . . . . . . . . . . . . . . . . 461
11.9 Evolutionary Strategies . . . . . . . . . . . . . . . . . 478
11.9.1 Solution Representation . . . . . . . . . . . . . 480
11.9.2 Mutation . . . . . . . . . . . . . . . . . . . . . 480
11.9.3 Recombination . . . . . . . . . . . . . . . . . . 481
11.9.4 Population Assessment . . . . . . . . . . . . . 482
11.9.5 Convergence Criteria . . . . . . . . . . . . . . 483
11.9.6 Computational Considerations . . . . . . . . . 483
11.9.7 Algorithm Performance . . . . . . . . . . . . . 484
11.10 Advantages and Disadvantagesof EvolutionaryCompu-
tation . . . . . . . . . . . . . . . . . . . . . . . . . . . 485
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487
Review Questions . . . . . . . . . . . . . . . . . . . . . . . . 488
12 EvolutionaryAlgorithmsImplementedUsingMATLAB 491
12.1 Illustration 1: Differential Evolution Optimizer . . . . 491
12.2 Illustration 2: Design of a Proportional-Derivative Con-
troller Using Evolutionary Algorithm for Tanker Ship
Heading Regulation . . . . . . . . . . . . . . . . . . . 502
© 2010 by Taylor and Francis Group, LLC