Multi-Objective Evolutionary Algorithms for Knowledge Discovery from Databases

168 Pages·2008·5.739 MB·English

by  Satchidananda Dehuri, Susmita Ghosh (auth.), Ashish Ghosh, Satchidananda Dehuri, Susmita Ghosh (eds.)

AshishGhosh,SatchidanandaDehuriandSusmitaGhosh(Eds.)
Multi-ObjectiveEvolutionaryAlgorithmsforKnowledgeDiscoveryfromDatabases
StudiesinComputationalIntelligence,Volume98
Editor-in-chief
Prof.JanuszKacprzyk
SystemsResearchInstitute
PolishAcademyofSciences
ul.Newelska6
01-447Warsaw
Poland
E-mail:[email protected]
Furthervolumesofthisseriescanbefoundonour Vol.85.ArpadKelemen,AjithAbrahamandYulanLiang
homepage:springer.com (Eds.)
ComputationalIntelligenceinMedicalInformatics,2008
ISBN978-3-540-75766-5
Vol.72.RaymondS.T.LeeandVincenzoLoia(Eds.)
ComputationIntelligenceforAgent-basedSystems,2007 Vol.86.ZbigniewLesandMogdalenaLes
ISBN978-3-540-73175-7 ShapeUnderstandingSystems,2008
ISBN978-3-540-75768-9
Vol.73.PetraPerner(Ed.)
Case-BasedReasoningonImagesandSignals,2008 Vol.87.YuriAvramenkoandAndrzejKraslawski
ISBN978-3-540-73178-8 CaseBasedDesign,2008
ISBN978-3-540-75705-4
Vol.74.RobertSchaefer
Vol.88.TinaYu,DavidDavis,CemBaydarandRajkumar
FoundationofGlobalGeneticOptimization,2007
Roy(Eds.)
ISBN978-3-540-73191-7
EvolutionaryComputationinPractice,2008
Vol.75.CrinaGrosan,AjithAbrahamandHisaoIshibuchi ISBN978-3-540-75770-2
(Eds.) Vol.89.ItoTakayuki,HattoriHiromitsu,ZhangMinjie
HybridEvolutionaryAlgorithms,2007 andMatsuoTokuro(Eds.)
ISBN978-3-540-73296-9 Rational,Robust,Secure,2008
Vol.76.SubhasChandraMukhopadhyayandGourabSen ISBN978-3-540-76281-2
Gupta(Eds.) Vol.90.SimoneMarinaiandHiromichiFujisawa(Eds.)
AutonomousRobotsandAgents,2007 MachineLearninginDocumentAnalysis
ISBN978-3-540-73423-9 andRecognition,2008
ISBN978-3-540-76279-9
Vol.77.BarbaraHammerandPascalHitzler(Eds.)
PerspectivesofNeural-SymbolicIntegration,2007 Vol.91.HorstBunke,KandelAbrahamandLastMark(Eds.)
ISBN978-3-540-73953-1 AppliedPatternRecognition,2008
ISBN978-3-540-76830-2
Vol.78.CostinBadicaandMarcinPaprzycki(Eds.)
Vol.92.AngYang,YinShanandLamThuBui(Eds.)
IntelligentandDistributedComputing,2008
SuccessinEvolutionaryComputation,2008
ISBN978-3-540-74929-5
ISBN978-3-540-76285-0
Vol.79.XingCaiandT.-C.JimYeh(Eds.)
Vol.93.ManolisWallace,MariosAngelidesandPhivos
QuantitativeInformationFusionforHydrological
Mylonas(Eds.)
Sciences,2008
AdvancesinSemanticMediaAdaptationand
ISBN978-3-540-75383-4
Personalization,2008
Vol.80.JoachimDiederich ISBN978-3-540-76359-8
RuleExtractionfromSupportVectorMachines,2008 Vol.94.ArpadKelemen,AjithAbrahamandYuehuiChen
ISBN978-3-540-75389-6 (Eds.)
Vol.81.K.Sridharan ComputationalIntelligenceinBioinformatics,2008
RoboticExplorationandLandmarkDetermination,2008 ISBN978-3-540-76802-9
ISBN978-3-540-75393-3 Vol.95.RaduDogaru
SystematicDesignforEmergenceinCellularNonlinear
Vol.82.AjithAbraham,CrinaGrosanandWitold
Networks,2008
Pedrycz(Eds.)
ISBN978-3-540-76800-5
EngineeringEvolutionaryIntelligentSystems,2008
ISBN978-3-540-75395-7 Vol.96.Aboul-EllaHassanien,AjithAbrahamandJanusz
Kacprzyk(Eds.)
Vol.83.BhanuPrasadandS.R.M.Prasanna(Eds.) ComputationalIntelligenceinMultimediaProcessing:
Speech,Audio,ImageandBiomedicalSignalProcessing RecentAdvances,2008
usingNeuralNetworks,2008 ISBN978-3-540-76826-5
ISBN978-3-540-75397-1
Vol.98.AshishGhosh,SatchidanandaDehuriandSusmita
Vol.84.MarekR.OgielaandRyszardTadeusiewicz Ghosh(Eds.)
ModernComputationalIntelligenceMethods Multi-ObjectiveEvolutionaryAlgorithmsforKnowledge
fortheInterpretationofMedicalImages,2008 DiscoveryfromDatabases,2008
ISBN978-3-540-75399-5 ISBN978-3-540-77466-2
Ashish Ghosh
Satchidananda Dehuri
Susmita Ghosh
(Eds.)
Multi-Objective Evolutionary
Algorithms for Knowledge
Discovery from Databases
With67Figuresand17Tables
123
AshishGhosh SatchidanandaDehuri
MachineIntelligenceUnit DepartmentofInformation
andCenterforSoftComputingResearch andCommunicationTechnology
IndianStatisticalInstitute F.M.University
203B.T.Road Balasore756019
Kolkata700108 India
India [email protected]
[email protected]
SusmitaGhosh
DepartmentofComputerScience
andEngineering
JadavpurUniversity
Kolkata700032
India
[email protected]
ISBN978-3-540-77466-2 e-ISBN978-3-540-77467-9
StudiesinComputationalIntelligenceISSN1860-949X
LibraryofCongressControlNumber:2008921361
(cid:176)c 2008Springer-VerlagBerlinHeidelberg
Thisworkissubjecttocopyright.Allrightsarereserved,whetherthewholeorpartofthematerial
isconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,broad-
casting,reproductiononmicrofilmorinanyotherway,andstorageindatabanks.Duplicationof
thispublicationorpartsthereofispermittedonlyundertheprovisionsoftheGermanCopyrightLaw
ofSeptember9,1965,initscurrentversion,andpermissionforusemustalwaysbeobtainedfrom
Springer-Verlag.ViolationsareliabletoprosecutionundertheGermanCopyrightLaw.
Theuseofgeneraldescriptivenames,registerednames,trademarks,etc.inthispublicationdoesnot
imply, even in the absence of a specific statement, that such names are exempt from the relevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Coverdesign:Deblik,Berlin,Germany
Printedonacid-freepaper
9 8 7 6 5 4 3 2 1
springer.com
ToOurParents
Preface
Withthegrowthofinformationtechnologyatunprecedentedrate,wehavewitnessed
aproliferationofdifferentkindsofdatabaseslikebiological,scientific,andcommer-
cial. Nowadays, it is fairly easy to create and customize a database tailored to our
need.Nevertheless,asthenumberofrecordsgrows,itisnotthateasytoanalyzeand
retrievehighlevelknowledgefromthesamedatabase.Therearenotasmanyof-the-
shelfsolutionsfordataanalysisastherearefordatabasecreationandmanagement;
furthermore,theyareprettyhardtosuittoourneed.
Data Mining (DM) is the most commonly used name to describe such compu-
tationalanalysisofdataandtheresultsobtainedmustconformtoseveralobjectives
suchasaccuracy,comprehensibility,interestfortheuseretc.Thoughtherearemany
sophisticated techniques developed by various interdisciplinary fields only a few
of them are well equipped to handle these multi-criteria issues of DM. Therefore,
the DM issues have attracted considerable attention of the well established multi-
objective genetic algorithm community to optimize the objectives in the tasks of
DM.
The present volume provides a collection of seven articles containing new and
highqualityresearchresultsdemonstratingthesignificanceofMulti-objectiveEvo-
lutionaryAlgorithms(MOEA)fordataminingtasksinKnowledgeDiscoveryfrom
Databases(KDD).Thesearticlesarewrittenbyleadingexpertsaroundtheworld.It
isshownhowthedifferentMOEAscanbeutilized,bothinindividualandintegrated
manner,invariouswaystoefficientlyminedatafromlargedatabases.
Chapter 1, by Dehuri et al., combines activities from three different areas of
activeresearch:knowledgediscoveryfromdatabases,geneticalgorithmsandmulti-
objective optimization. The goal of this chapter is to identify the objectives those
are implicitly/explicitly associated with the tasks of KDD like pre-processing, data
mining, and post processing and then discuss how MOEA can be used to optimize
them.
Chapter 2 contributed by Landa-Becerra et al. presents a survey of techniques
used to incorporate knowledge into evolutionary algorithms, with special empha-
sis on multi-objective optimization. They focus on two main groups of techniques:
techniques which incorporate knowledge into fitness evaluation, and those which
VIII Preface
incorporateknowledgeintheinitializationprocessandtheoperatorsofanevolution-
ary algorithm. Several methods, representative of each of these groups, are briefly
discussedtogetherwithsomeexamplesfoundinthespecializedliterature.Inthelast
partofthechapter,theauthorsprovidedsomeresearchideasthatareworthexploring
infuturebyresearchersinterestedinthistopic.
Classification rule mining is one of the fundamental tasks of data mining.
Ishibuchietal.hassolvedthisproblemusingevolutionarymulti-objectiveoptimiza-
tionalgorithmsandtheirworkisincludedinChapter3.Inthefieldofclassification
rulemining,classifiersaredesignedthroughthefollowingtwophases:rulediscov-
eryandruleselection.Inrulediscoveryphase,alargenumberofclassificationrules
areextractedfromtrainingdata.Thisphaseisbasedontworuleevaluationcriteria:
supportandconfidence.AnassociationruleminingtechniquesuchasApriorialgo-
rithmisusuallyusedtoextractclassificationrulessatisfyingpre-specifiedthreshold
valuesoftheminimumsupportandconfidence.Inthesecondphaseasmallnumber
ofrulesareselectedfromtheextractedrulestodesignanaccurateandcompactclas-
sifier.Inthischapter,firsttheauthorsexplainedtheabove-mentionedtwophasesin
classificationrulemining.NexttheydescribedhowtofindthePareto-optimalrules
andPareto-optimalrulesets.Thentheydiscussedevolutionarymulti-objectiverule
selectionasapostprocessingprocedure.
Chapter 4, written by Jin et al., showed that rule extraction from neural net-
works is a powerful tool for knowledge discovery from data. In order to facilitate
ruleextraction, trained neural networks areoften pruned sothat the extracted rules
are understandable to human users. This chapter presents a method for extracting
interpretable rules from neural networks that are generated using an evolutionary
multi-objectivealgorithm.Inthealgorithm,theaccuracyonthetrainingdataandthe
complexity of the neural networks are minimized simultaneously. Since there is a
tradeoff between accuracy and complexity, a number of Pareto-optimal neural net-
works, instead of a single optimal neural network, are obtained. They showed that
the Pareto-optimal networks, with a minimal degree of complexity, are often inter-
pretable as they can extract understandable logic rules. Finally they have verified
theirapproachusingtwobenchmarkproblems.
Alcala et al. contributed Chapter 5 which deals with the usefulness of MOEAs
forgettingcompactfuzzyrulebasedsystems(FRBSs)underparametertuningand
ruleselection.Thiscontributionbrieflyreviewsthestate-of-the-artofthistopicand
presents an approach to prove the ability of multi-objective genetic algorithms for
obtainingcompactfuzzyrulebasedsystemsunderruleselectionandparametertun-
ing,i.e.,toobtainlinguisticmodelswithimprovedaccuracyandminimumnumber
ofrules.
Chapter 6, contributed by Setzkorn, deals with the details of three MOEAs to
solve different data mining problems. The first approach is used to induce fuzzy
classificationrulesystemsandtheothertwoareusedforsurvivalanalysisproblems.
Till now, many evolutionary approaches use accuracy to measure the fitness of the
modeltothedata.Thisisinappropriatewhenthemisclassificationscostsandclass
priorprobabilitiesareunknown, whichisoftenthecaseinpractice.Hence,theau-
thorusesameasurecalledtheareaunderthereceiver-operatingcharacteristiccurve
Preface IX
(AUC)thatdoesnothavetheseproblems.Theauthoralsodeploysaself-adaptation
mechanism to reduce the number of free parameters and uses the state-of-the-art
multi-objectiveevolutionaryalgorithmcomponents.
Chapter7,writtenbyMurtyetal.,discussestheroleofevolutionaryalgorithms
(EAs)inclustering.Inthiscontext,theyhavepointedoutthatmostoftheGA-based
clusteringalgorithmsareappliedonthedatasetswithsmallnumberofpatternsand
or features. Hence, to cope up with large size and high dimensional data sets, they
proposedaGAbasedalgorithm,OCFTBA,employingtheclusterfeaturetree(CF-
tree)datastructure.Itscalesupwellwiththenumberofpatternsandfeatures.They
have also suggested that clustering for the large scale data can be formulated as a
multi-objectiveproblemandsolvingthemusingGAswillbeveryinterestingwitha
flavorofgoodsetofParetooptimalsolutions.
Kolkata,Balasore AshishGhosh
October2007 SatchidanandaDehuri
SusmitaGhosh
Contents
1 Genetic Algorithm for Optimization of Multiple Objectives in
KnowledgeDiscoveryfromLargeDatabases
SatchidanandaDehuri,SusmitaGhosh,AshishGhosh .................... 1
2 KnowledgeIncorporationinMulti-objectiveEvolutionaryAlgorithms
RicardoLanda-Becerra,LuisV.Santana-Quintero,CarlosA.CoelloCoello... 23
3 EvolutionaryMulti-objectiveRuleSelectionforClassificationRule
Mining
HisaoIshibuchi,IsaoKuwajima,YusukeNojima ......................... 47
4 RuleExtractionfromCompactPareto-optimalNeuralNetworks
YaochuJin,BernhardSendhoff,EdgarKo¨rner .......................... 71
5 OntheUsefulnessofMOEAsforGettingCompactFRBSsUnder
ParameterTuningandRuleSelection
R.Alcala´,J.Alcala´-Fdez,M.J.Gacto,F.Herrera ........................ 91
6 Classification and Survival Analysis Using Multi-objective
EvolutionaryAlgorithms
ChristianSetzkorn.................................................109
7 ClusteringBasedonGeneticAlgorithms
M.N.Murty,BabariaRashmin,ChiranjibBhattacharyya ..................137
List of Contributors
R.Alcala´ SatchidanandaDehuri
DepartmentofComputerScienceand DepartmentofInformationand
ArtificialIntelligence CommunicationTechnology
UniversityofGranada FakirMohanUniversity
E-18071-Granada,Spain VyasaVihar
[email protected] Balasore756019,India
[email protected]
J.Alcala´-Fdez M.J.Gacto
DepartmentofComputerScienceand DepartmentofComputerScienceand
ArtificialIntelligence ArtificialIntelligence
UniversityofGranada UniversityofGranada
E-18071-Granada,Spain E-18071-Granada,Spain
[email protected] [email protected]
AshishGhosh
MachineIntelligenceUnit
ChiranjibBhattacharyya
IndianStatisticalInstitute
DepartmentofComputerScience
203B.T.Road,Kolkata700108,India
andAutomation
[email protected]
IndianInstituteofScience
Bangalore560012,India SusmitaGhosh
[email protected] DepartmentofComputerScienceand
Engineering,JadavpurUniversity
Kolkata700032,India
CarlosA.CoelloCoello
[email protected]
CINVESTAV-IPN(Evolutionary
ComputationGroup) F.Herrera
DepartamentodeComputacio´n DepartmentofComputerScienceand
Av.IPNNo.2508 ArtificialIntelligence
Col.SanPedroZacatenco,Me´xico UniversityofGranada
D.F.07360,Mexico E-18071-Granada,Spain
[email protected] [email protected]