Pattern Recognition using Neural and Functional Networks

197 Pages·2009·11.353 MB·English

by  Vasantha Kalyani David, Sundaramoorthy Rajasekaran (auth.)

VasanthaKalyaniDavidandSundaramoorthyRajasekaran
PatternRecognitionUsingNeuralandFunctionalNetworks
StudiesinComputationalIntelligence,Volume160
Editor-in-Chief
Prof.JanuszKacprzyk
SystemsResearchInstitute
PolishAcademyofSciences
ul.Newelska6
01-447Warsaw
Poland
E-mail:[email protected]
Furthervolumesofthisseriescanbefoundonourhomepage: Vol.149.RogerLee(Ed.)
springer.com SoftwareEngineering,ArtificialIntelligence,Networkingand
Parallel/DistributedComputing,2008
Vol.138.BrunoApolloni,WitoldPedrycz,SimoneBassis ISBN978-3-540-70559-8
andDarioMalchiodi Vol.150.RogerLee(Ed.)
ThePuzzleofGranularComputing,2008 SoftwareEngineeringResearch,Managementand
ISBN978-3-540-79863-7 Applications,2008
ISBN978-3-540-70774-5
Vol.139.JanDrugowitsch
DesignandAnalysisofLearningClassifierSystems,2008 Vol.151.TomaszG.Smolinski,MariofannaG.Milanova
ISBN978-3-540-79865-1 andAboul-EllaHassanien(Eds.)
ComputationalIntelligenceinBiomedicineandBioinformatics,
Vol.140.NadiaMagnenat-Thalmann,LakhmiC.Jain 2008
andN.Ichalkaranje(Eds.) ISBN978-3-540-70776-9
NewAdvancesinVirtualHumans,2008 Vol.152.Jarosl(cid:2)awStepaniuk
ISBN978-3-540-79867-5 Rough–GranularComputinginKnowledgeDiscoveryandData
Mining,2008
Vol.141.ChristaSommerer,LakhmiC.Jain
ISBN978-3-540-70800-1
andLaurentMignonneau(Eds.)
TheArtandScienceofInterfaceandInteractionDesign(Vol.1), Vol.153.CarlosCottaandJanovanHemert(Eds.)
2008 RecentAdvancesinEvolutionaryComputationfor
ISBN978-3-540-79869-9 CombinatorialOptimization,2008
ISBN978-3-540-70806-3
Vol.142.GeorgeA.Tsihrintzis,MariaVirvou,RobertJ.Howlett
Vol.154.OscarCastillo,PatriciaMelin,JanuszKacprzykand
andLakhmiC.Jain(Eds.)
WitoldPedrycz(Eds.)
NewDirectionsinIntelligentInteractiveMultimedia,2008
SoftComputingforHybridIntelligentSystems,2008
ISBN978-3-540-68126-7
ISBN978-3-540-70811-7
Vol.143.UdayK.Chakraborty(Ed.) Vol.155.HamidR.TizhooshandM.Ventresca(Eds.)
AdvancesinDifferentialEvolution,2008 OppositionalConceptsinComputationalIntelligence,2008
ISBN978-3-540-68827-3 ISBN978-3-540-70826-1
Vol.144.AndreasFinkandFranzRothlauf(Eds.) Vol.156.DawnE.HolmesandLakhmiC.Jain(Eds.)
AdvancesinComputationalIntelligenceinTransport,Logistics, InnovationsinBayesianNetworks,2008
andSupplyChainManagement,2008 ISBN978-3-540-85065-6
ISBN978-3-540-69024-5
Vol.157.Ying-pingChenandMeng-HiotLim(Eds.)
Vol.145.MikhailJu.Moshkov,MarcinPiliszczuk LinkageinEvolutionaryComputation,2008
andBeataZielosko ISBN978-3-540-85067-0
PartialCovers,ReductsandDecisionRulesinRoughSets,2008 Vol.158.MarinaGavrilova(Ed.)
ISBN978-3-540-69027-6 GeneralizedVoronoiDiagram:AGeometry-BasedApproachto
ComputationalIntelligence,2009
Vol.146.FatosXhafaandAjithAbraham(Eds.)
ISBN978-3-540-85125-7
MetaheuristicsforSchedulinginDistributedComputing
Environments,2008 Vol.159.DimitriPlemenosandGeorgiosMiaoulis(Eds.)
ISBN978-3-540-69260-7 GeneralizedVoronoiDiagram:AGeometry-BasedApproachto
ComputationalIntelligence,2009
Vol.147.OliverKramer ISBN978-3-540-85125-7
Self-AdaptiveHeuristicsforEvolutionaryComputation,2008
ISBN978-3-540-69280-5 Vol.160.VasanthaKalyaniDavidand
SundaramoorthyRajasekaran
Vol.148.PhilippLimbourg PatternRecognitionUsingNeuralandFunctionalNetworks,
DependabilityModellingunderUncertainty,2008 2009
ISBN978-3-540-69286-7 ISBN978-3-540-85129-5
Vasantha Kalyani David
Sundaramoorthy Rajasekaran
Retracted Book: Pattern
Recognition Using Neural and
Functional Networks
123
ProfessorDr.VasanthaKalyaniDavid
DepartmentofComputerScience
AvinashilingamUniversityforWomen
Coimbatore-641043,TamilNadu
India
ProfessorDr.SundaramoorthyRajasekaran
InfrastructureEngineering
PSGCollegeofTechnology
Coimbatore-641004,TamilNadu
India
Email:[email protected]
ISBN978-3-540-85129-5 e-ISBN978-3-540-85130-1
DOI10.1007/978-3-540-85130-1
StudiesinComputationalIntelligence ISSN1860949X
LibraryofCongressControlNumber:2008931714
(cid:2)c 2009Springer-VerlagBerlinHeidelberg
This work is subject to copyright.All rights are reserved,whether the whole or part of the
materialisconcerned,specifically the rightsof translation,reprinting,reuseof illustrations,
recitation,broadcasting,reproductiononmicrofilmorinanyother way,andstorageindata
banks.Duplicationofthispublicationorpartsthereofispermittedonlyundertheprovisionsof
theGermanCopyrightLawofSeptember9,1965,initscurrentversion,andpermissionforuse
mustalwaysbeobtainedfromSpringer.ViolationsareliabletoprosecutionundertheGerman
CopyrightLaw.
The use of general descriptive names,registered names,trademarks,etc.in thispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom
therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
Typeset&CoverDesign:ScientificPublishingServicesPvt.Ltd.,Chennai,India.
Printedinacid-freepaper
987654321
springer.com
Authors
d
Vasantha Kalyani David and S. Rajasekaran
e
About the Authors
t
Vasantha Kalyani David, PhD., is a Selection Grade Lecturer in Computer
c
Science in AvinashilingamUniversity for Women, Coimbatore.Earliera mathe-
matician with a Master of Philosophy in Mathematics and later did research in
a
ComputerScience.Dr.VasanthaKalyaniDavidhaspublishedmanypapersinar-
easofSoftComputing,HerresearcrhinterestsincludeNeuralNetworks,Artificial
Intelligence, Fuzzy Logic, Genetic Algorithms, Cellular Automata, Theoretical
t
Computer Science and Automata Theory.
e
S. Rajasekaran, DSc, (Civil Engineering) is Professor of Infrastructure Engi-
neering.,PSGCollegeRofTechnology,Coimbatore.Hehasover41yearsofteach-
ingandresearchexperienceandmorethan300researchpaperstohiscredit.His
areasofspecialinterestincludeStructuralEngineering,FiniteElementAnalysis,
and application of Soft Computing techniques in Structural Engineering.
Preface
d
Biologicallyinspiredcomputing is differentfromconventionalcomputing.Ithas
a different feel; often the terminology does not sound like it’s talking about
e
machines.The activitiesofthis computing soundmorehumanthanmechanistic
as people speak of machines that behave, react, self-organize, learn, generalize,
t
remember and even to forget. Much of this technology tries to mimic nature’s
c
approachin order to mimic some of nature’s capabilities. They have a rigorous,
mathematical basis and neural networks for example have a statistically valid
a
set on which the network is trained.
Twooutlinesaresuggestedasthrepossibletracksforpatternrecognition.They
are neural networks and functional networks. Neural Networks (many intercon-
t
nected elements operating in parallel) carry out tasks that are not only beyond
the scope of conventionalperocessing but also cannotbe understood in the same
terms.Imagingapplicationsforneuralnetworksseemto be anaturalfit.Neural
R
networkslove to do pattern recognition.A new approachto pattern recognition
using microARTMAP together with wavelet transforms in the context of hand
written characters,gestures and signatures have been dealt. The Kohonen Net-
work, Back Propagation Networks and Competitive Hopfield Neural Network
have been considered for various applications.
Functionalnetworks,beingageneralizedformofNeuralNetworkswherefunc-
tionsarelearnedratherthanweightsiscomparedwithMultipleRegressionAnal-
ysis for some applications and the results are seen to be coincident.
New kinds of intelligence can be added to machines, and we will have the
possibility of learning more about learning. Thus our imaginations and options
are being stretched. These new machines will be fault-tolerant, intelligent and
self-programmingthustryingtomakethemachinessmarter.Soastomakethose
who use the techniques even smarter.
Chapter 1 is a brief introduction to Neural and Functional networks in the
contextof Patternrecognitionusing these disciplines Chapter 2 givesa review
of the architectures relevant to the investigation and the development of these
technologies in the past few decades.
VIII Preface
Chapter 3 begins with the look at the recognitionof handwritten alphabets
using the algorithm for ordered list of boundary pixels as well as the Koho-
nen Self-Organizing Map (SOM). Chapter 4 describes the architecture of the
MicroARTMAP and its capability.
Chapter 5 the MicroARTMAP is augmented with a moment based feature
extractorandappliedto characterrecognitionin this chapter.Chapter 6 illus-
tratestheuse ofwavelettransformstogetherwithMicroARTMAPforcharacter
recognition.The microARTMAP gavesolutions to problems in civilengineering
like classification of soil problem, finding the load from yield pattern of a plate
and finding earthquake parameters from a given response spectrum.
Chapter 7 MicroARTMAP and Back PropagationNetwork have been com-
pared and presented for gesture recognition and signature verification. Solving
schedulingproblembymeansofaCompetitiveHopfieldNeuralNetworkaredis-
cussedinChapter 8 Multiple Regressionmethods consideredasa recognizeris
d
compared with functional networks in solving certain problems as shown in
Chapter9.ConclusionandfurtherapplicationsaresuggestedinChapter10.
e
t
c
a
r
t
e
R
Contents
d
1 Retracted Chapter: Introduction .......................... 1
1.1 Introduction ............................................. 1
e
1.2 Recognition through Algorithm and Kohonen’s Self
Organizing Map...................t....................... 2
1.3 MicroARTMAP .......................................... 2
c
1.4 Wavelet Transforms....................................... 3
1.5 Gesture Recognition ...................................... 4
a
1.6 Competitive Hopfield Neural Network....................... 5
1.7 Neural and Functional Nretworks............................ 5
1.8 Objectives and Scope of the Investigation.................... 6
t
1.9 Organizationof the Book.................................. 6
1.10 Summary......e.......................................... 7
2 Retracted ChaRpter: Review of Architectures Relevant
to the Investigation ...... ............ ............ .......... 9
2.1 Introduction ............................................. 9
2.2 Recognition through Self Organizing Map ................... 10
2.3 The μARTMAP.......................................... 10
2.4 Wavelet Transforms and MicroARTMAP .................... 11
2.5 MicroARTMAP and Gesture Recognition.................... 11
2.6 Competitive Hopfield Neural Network....................... 11
2.7 Functional Networks and Multiple Regression Analysis ........ 12
2.8 Summary................................................ 12
3 Retracted Chapter: Recognition of English and Tamil
Alphabets Using Kohonen’s Self-organizing Map ........... 15
3.1 Introduction ............................................. 15
3.2 Recognition of Handwritten Characters Using Ordered List
of Image Pixels on Its Boundary............................ 15
3.3 The Kohonen Feature Map ................................ 18
X Contents
3.4 Normalization of a Vector ................................. 19
3.5 Training Law ............................................ 19
3.6 Neighbourhood Size ...................................... 19
3.7 The Kohonen Network .................................... 19
3.8 Representation of Characters .............................. 19
3.9 Weight Vector ........................................... 20
3.10 Summary................................................ 26
4 Retracted Chapter:Adaptive ResonanceTheory Networks... 27
4.1 Introduction ............................................. 27
4.2 ART Network............................................ 28
4.2.1 Resonant State..................................... 28
4.2.2 The STM and LTM Traces .......................... 28
d
4.2.3 The Structure of the ART Model..................... 30
4.2.4 Pattern-Matching Cycle in an ART Network........... 31
e
4.2.5 The 2/3 Rule ...................................... 31
4.2.6 Gain Control Mechanism .....t....................... 32
4.3 Fuzzy ART ....................c.......................... 33
4.3.1 Analogy between ART1 and Fuzzy ART .............. 33
4.3.2 Fast-Learn, Slow-Recoade and Complement Coding...... 34
4.3.3 Complement Coding ................................ 34
r
4.3.4 Weight Vectors..................................... 36
4.3.5 Parameters ..t...................................... 36
4.3.6 Category Cehoice ................................... 36
4.3.7 Resonance or Reset................................. 37
4.3.8 LearnRing Law...................................... 37
4.3.9 Normalization of Fuzzy ART Inputs .................. 38
4.3.10 Geometric Interpretation of Fuzzy ART ............... 39
4.3.11 Fuzzy ART Category Boxes in Fuzzy Cubes ........... 39
4.3.12 Fuzzy ART Stable Category Learning................. 42
4.4 Fuzzy ARTMAP ......................................... 43
4.4.1 Fuzzy ARTMAP and MicroARTMAP................. 43
4.4.2 MicroARTMAP Algorithm (Supervised Neural
Network Architecture) .............................. 44
4.4.3 Map Field Activation ............................... 45
4.4.4 Match Tracking .................................... 45
4.4.5 Map Field Learning................................. 46
4.4.6 Defining H ........................................ 46
4.4.7 Training of μARTMAP ............................. 47
4.4.8 Inter ART Reset ................................... 47
4.4.9 Offline Evaluation .................................. 48
4.4.10 μARTMAP Prediction .............................. 48
Contents XI
4.4.11 Fast Learning in μARTMAP......................... 48
4.4.12 Refining a Hyper Box............................... 49
4.4.13 μARTMAP Rules .................................. 49
4.5 Summary................................................ 49
5 Retracted Chapter: Applications of MicroARTMAP ...... 51
5.1 Introduction ............................................. 51
5.2 Recognition of Handwritten Alphabets by μARTMAP ........ 51
5.3 Recognition of Handwritten Words by μARTMAP............ 53
5.4 Recognition of Handwritten Alphabets by μARTMAP
Augmented with Moment-Based Feature Extractor ........... 53
5.4.1 Introduction ....................................... 53
5.4.2 Steps Involved in Obtaining Moment Invariants ........ 55
d
5.5 Recognition of Handwritten Numbers by μARTMAP Using
Hamming Distance ....................................... 59
5.6 Recognition of Alphabets and Numbers Useing μARTMAP
with Only One Exemplar for Training....................... 60
t
5.7 Recognition of Alphabets by μARTMAP with Increased
Sample Size....................c.......................... 61
5.8 BIS Classification of Soil .................................. 61
a
5.9 Plastification of Clamped Isotropic Plate .................... 64
5.10 Application to Earthquake Engineering...................... 65
r
5.11 Summary................................................ 71
t
6 Retracted Chaptere: Wavelet Transforms and
MicroARTMAP ............................................ 73
R
6.1 Introduction ............................................. 73
6.2 The Need for Transforms .................................. 73
6.3 Fourier Transform ........................................ 73
6.4 Transforms Available ..................................... 74
6.5 Wavelet Transforms....................................... 74
6.6 Continuous Wavelet Transforms (CWT)..................... 75
6.7 Discrete Wavelet Transforms (DWT)........................ 75
6.8 Wavelet Functions ........................................ 77
6.9 Wavelet Analysis ......................................... 77
6.10 Schematic Representation of the Working of a Wavelet ........ 78
6.11 Handwritten Characters Recognition Using Wavelet
Transforms and MicroARTMAP............................ 82
6.12 Wavelet Transforms in Two Dimensions ..................... 82
6.13 The Two-Dimensional DWT ............................... 84
6.14 RecognitionofHandwrittenAlphabets UsingWaveletPackets
and MicroARTMAP ...................................... 85