Information Processing with Evolutionary Algorithms: From Industrial Applications to Academic Speculations

340 Pages·2005·3.88 MB·English

by  T. Bäck (auth.)

Advanced Information and Knowledge Processing
Also in this series
Gregoris Mentzas, Dimitris Apostolou, Andreas Abecker and Ron Young
Knowledge Asset Management
1-85233-583-1
Michalis Vazirgiannis, Maria Halkidi and Dimitrios Gunopulos
Uncertainty Handling and Quality Assessment in Data Mining
1-85233-655-2
Asuncio´n Go´mez-Pe´rez, Mariano Ferna´ndez-Lo´pez, Oscar Corcho
Ontological Engineering
1-85233-551-3
Arno Scharl
Environmental Online Communication
1-85233-783-4
Shichao Zhang, Chengqi Zhang and Xindong Wu
Knowledge Discovery in Multiple Databases
1-85233-703-6
Jason T.L. Wang, Mohammed J. Zaki, Hannu T.T. Toivonen and Dennis
Shasha (Eds)
Data Mining in Bioinformatics
1-85233-671-4
C.C. Ko, Ben M. Chen and Jianping Chen
Creating Web-based Laboratories
1-85233-837-7
Manuel Gran˜a, Richard Duro, Alicia d’Anjou
and Paul P. Wang (Eds)
Information Processing
with Evolutionary
Algorithms
From Industrial Applications to Academic Speculations
With 137 Figures
Manuel Gran˜a, BSc, MSc, PhD
Universidad del Pais Vasco, San Sebastian, Spain
Richard J. Duro, BSc, MSc, PhD
Universidad da Corun˜a, La Corun˜a, Spain
Alicia d’Anjou, BSc, MSc, PhD
Universidad del Pais Vasco, San Sebastian, Spain
Paul P. Wang, PhD
Duke University, Durham, North Carolina, USA
SeriesEditors
XindongWu
LakhmiJain
BritishLibraryCataloguinginPublicationData
Informationprocessingwithevolutionaryalgorithms:from
industrialapplicationstoacademicspeculations.—
(Advancedinformationandknowledgeprocessing)
1. Evolutionarycomputation 2. Computeralgorithms
I. Gran˜a,Manuel
005.1
ISBN1852338660
LibraryofCongressCataloging-in-PublicationData
Informationprocessingwithevolutionaryalgorithms:fromindustrialapplicationsto
academicspeculations/ManuelGran˜a...[etal.].
p.cm.—(Advancedinformationandknowledgeprocessing)
ISBN1-85233-866-0(alk.paper)
1. Evolutionaryprogramming(Computerscience) 2. Geneticalgorithms. 3. Electronic
dataprocessing. I. Gran˜a,Manuel,1958– II. Series.
QA76.618.I56 2004
006.3′36—dc22 2004059333
Apartfromanyfairdealingforthepurposesofresearchorprivatestudy,orcriticismorreview,aspermittedunder
theCopyright,DesignsandPatentsAct1988,thispublicationmayonlybereproduced,storedortransmitted,in
anyformorbyanymeans,withthepriorpermissioninwritingofthepublishers,orinthecaseofreprographic
reproductioninaccordancewiththetermsoflicencesissuedbytheCopyrightLicensingAgency.Enquiriescon-
cerningreproductionoutsidethosetermsshouldbesenttothepublishers.
AI&KPISSN1610-3947
ISBN1-85233-866-0SpringerLondonBerlinHeidelberg
SpringerScience+BusinessMedia
springeronline.com
©Springer-VerlagLondonLimited2005
Theuseofregisterednames,trademarks,etc.inthispublicationdoesnotimply,evenintheabsenceofaspecific
statement,thatsuchnamesareexemptfromtherelevantlawsandregulationsandthereforefreeforgeneraluse.
Thepublishermakesnorepresentation,expressorimplied,withregardtotheaccuracyoftheinformationcon-
tainedinthisbookandcannotacceptanylegalresponsibilityorliabilityforanyerrorsoromissionsthatmaybe
made.
Typesetting:Electronictextfilespreparedbyauthors
PrintedandboundintheUnitedStatesofAmerica
34/3830-543210 Printedonacid-freepaper SPIN10984611
Preface
Thelastdecadeofthe20thcenturyhaswitnessedasurgeofinterestinnumer-
ical, computation-intensive approaches to information processing. The lines
thatdrawtheboundariesamongstatistics,optimization,arti(cid:1)cialintelligence
and information processing are disappearing, and it is not uncommon to (cid:1)nd
well-founded and sophisticated mathematical approaches in application do-
mains traditionally associated with ad-hoc programming. Heuristics has be-
comea branch of optimization and statistics. Clustering is applied toanalyze
soft data and to provide fast indexing in the World Wide Web. Non-trivial
matrix algebra is at the heart of the last advances in computer vision.
The breakthrough impulse was, apparently, due to the rise of the interest
in arti(cid:1)cial neural networks, after its rediscovery in the late 1980s. Disguised
as ANN, numerical and statistical methods made an appearance in the in-
formation processing scene, and others followed. A key component in many
intelligentcomputationalprocessingisthesearchforanoptimalvalueofsome
function.Sometimes,thisfunctionisnotevidentanditmustbemadeexplicit
inordertoformulatetheproblem asan optimizationproblem. The search of-
tentakesplaceinhigh-dimensional spacesthat can be eitherdiscrete, orcon-
tinuousormixed.Theshapeofthehigh-dimensionalsurfacethatcorresponds
totheoptimizedfunctionisusuallyverycomplex.Evolutionaryalgorithmsare
increasinglybeing applied toinformationprocessing applicationsthatrequire
anykind of optimization. They provide a systematic and intuitiveframework
to state the optimization problems, and an already well-established body of
theory that endorses their good mathematical properties. Evolutionary algo-
rithms have reached the status of problem-solving tools in the backpack of
theengineer. However, there are still exciting new developments taking place
intheacademiccommunity.The driving ideaintheorganizationof this com-
pilation istheemphasis inthe contrastbetween already acceptedengineering
practice and ongoing explorations in the academic community.
After the seminal works of Holland, Goldberg and Schwefel, the (cid:1)eld of
evolutionaryalgorithmshas experiencedan explosion ofboth researchers and
publicationsinboththeapplication-orientedandthefundamentalissues.Itis
vi Preface
obviouslydifficultto present ina singlebook a complete and detailedpicture
of the (cid:1)eld. Therefore, the point of view of this compilation is more mod-
est. Its aim has been to provide a glimpse of the large variety of problems
tackledwithevolutionaryapproximationsandof thediversityofevolutionary
algorithmsthemselvesbasedonsomeof thepaperspresented attheFrontiers
onEvolutionaryAlgorithms Conference within JCIS 2002and complemented
with some papers by well-known authors in the (cid:1)eld on topics that were not
fully covered in the sessions. Following the general trend in the (cid:1)eld, most
of the papers are application-oriented. However, we have made an effort to
include some that refer to fundamental issues as well as some that provide a
review of the state of the art in some sub(cid:1)eld.
As the subtitle (cid:4)From industrial applications to academic speculation-
s(cid:5) suggests, the organization of the compilation follows an axis of nearness
to practical applications. We travel from industrial day-to-day problems and
practice to the more speculative works. The starting collection of papers is
devoted to immediate applications of clear economical value at present.
(cid:1) The chapter by T. Bäck is an example of successful consulting with a
toolbox of computational methods that include evolutionary algorithms
addressing nontrivial industrial problems. Although the emphasis of the
chapter is on Evolutionary Strategies, Bäck(cid:6)s work is a bright example of
a host of evolutionary solutions to everyday problems being developed at
both universities and the industry RD labs.
(cid:1) The general approach of Deschaine and Francone is to reverse engineer a
systemwith Linear GeneticProgrammingat themachine code level.This
approach provides very fast and accurate models of the process that will
be subject to optimization. The optimization process itself is performed
using an Evolutionary Strategy with completely deterministic parameter
self-adaptation.The authors havetestedthis approachinavarietyof aca-
demic problems. They target industrial problems, characterized by low
formalization and high complexity. As a (cid:1)nal illustration they deal with
the design of an incinerator and the problem of subsurface unexploded
ordnance detection.
(cid:1) Nowadaysthereisabigindustryof3Dcomputermodelingbasedonseveral
3D scanning methods. The rendering of these 3D structures from a cloud
of scanned points requires a triangulation de(cid:1)ned on them which may be
very costly, depending on the number of scanned points, and subject to
noise. Smooth and efficient approximations are therefore desired. In the
chapter by Weinert et al. we (cid:1)nd the application of Evolution Strategies
to the problem of (cid:1)nding optimal triangulation coverings of a 3D object
described by a cloud of points. The authors introduce a special encoding
of thetriangulation on areal-valuedvector,a prerequisite fortheapplica-
tionofEvolutionStrategies. Thisencoding consistsof themodelingof the
triangulation as grid of springs and masses of varying coefficients. These
coefficients and the Z coordinate of the mass points result in a problem
Preface vii
encoding that is closedunder conventional Evolution Strategy genetic op-
erators.
(cid:1) Another practical application domain of current interest is the exploita-
tionof hyperspectral images,especially thosethatarise inremotesensing.
Therecent advancesinhyperspectral sensorsandthespaceprogramsthat
include them in modern and future satellites imply that a large amount
of data will be available in the near future. Fast, unsupervised analysis
methods will be needed to provide adequate preprocessing of these data.
Graña, Hernandez and d(cid:6)Anjou propose an evolutionary algorithm to ob-
tain an optimal unsupervised analysis of hyperspectral images given by a
set of endmembers identi(cid:1)ed in the image. The identi(cid:1)cation is based on
thenotion of morphologicalindependence, and Morphological Associative
Memories serve as detectors of this condition.
(cid:1) The processing of digital images is already an exploding application do-
main of computational methods. One of the issues of current interest,
especially in the medical image domainand Magnetic Resonance Imaging
(MRI), is the correction of illuminationinhomogeneity(bias). Algorithms
forilluminationcorrectionmaybeparametricornonparametric.Thelatter
aremorecomputationallydemanding.The formersrequireanappropriate
modeling framework. Fernandez et al. present a gradient-driven evolution
strategy for the estimation of the parameters of an illumination model
given by a linear combination of Legendre polynomials. The gradient in-
formation is used in the mutation operator and seems to improve the
convergence of the search, when compared with similar approaches.
(cid:1) Jobshopschedulingisaclassicaloperationsresearch problemandarecur-
rent problem in many industrial settings, ranging from the planning of a
smallworkshoptotheallocationofcomputingresources. Varelaetal.pro-
pose an encoding that allows the modular decomposition of the problem.
This modulardecomposition is of use forthe de(cid:1)nition of new genetic op-
eratorsthatalwaysproducefeasiblesolutions.Inaddition,thenewgenetic
operatorsbene(cid:1)tfromthelocal/globalstructuraltradeoffsoftheproblem,
producing an implicit search of local solutions, akin to the local search in
memetic approaches, but carried out in a parallel fashion.
The next batch of chapters includes works that present interesting and
innovative applications of evolutionary approaches. The emphasis is on the
departure of the application from the conventional optimization problems.
Topics range from archeology to mobile robotics control design.
(cid:1) The starting work is a fascinating application of evolution to the creation
of acomputationalmodelthatexplainstheemergenceof anarchaic state,
the Zapotec state. The model is composed of the ontogenies evolved by
speci(cid:1)c agentsdealing with thedata about thesites inthe Oaxaca valley,
embodied in a GIS developed in the project. One of the basic results is
the search for sites that may have been subject to warfare. A GA-driven
viii Preface
Rough Set data mining procedure was realized and its results compared
with a decision tree approach.
(cid:1) Phylogeneticisthesearch ofevolutionarypathwaysbetweenspecies based
onbiologicaldata.Thephylogenicrelationstaketheformoftrees.Species
are characterized by several attributes, and the construction of phyloge-
netictreesissomehowreminiscentofdecisiontreeconstruction.Attributes
usuallyconsistofphenotypicdata,althoughrecent approachesalsousege-
netic data. Measures of the qualityof phylogenetic trees are based on the
parsimonyevolutiverelation representation. Theseparsimoniousobjective
functions have been used to guide heuristic search algorithms applied to
phylogenetic tree construction. C.B. Congdon proposes an evolutionary
approach to their construction. A GA is applied because only binary val-
ued attributes are considered. A canonical tree is introduced to compare
phylogenies, andthegeneticmutation andcrossover operatorsare de(cid:1)ned
accordingly. Besides the comparison of the evolutionary approach with
standard algorithms, the effect of the genetic operators is studied.
(cid:1) An active area in evolutionary robotics is the (cid:1)eld of evolutionary devel-
opment of robotic controllers. The need to test these controllers on the
real robot to evaluate the (cid:1)tness function imposes stringent constraints
onthenumberof(cid:1)tness evaluationsallowable.Therefore,theconvergence
problems of conventional evolutionary approaches are worsened because
of the poor sampling of the (cid:1)tness landscape. Becerra et al. introduce
Macroevolutionary Algorithms for the design of robot controllers in the
domain of mobile robotics. Robot controllers take the form of arti(cid:1)cial
neural networks and the intended task is robust wall following in food or
poison rewarding environment. The Macroevolutionary Algorithms parti-
tion the population into races that may evolve independently and, some-
times, become extinct. The chapter studies the setting of the colonization
parameters that produce different exploitation/exploration balances.
(cid:1) Parsingbased on grammars is acommon tool fornatural language under-
standing. The caseof sublanguages associated with aspeci(cid:1)c activity,like
patent claiming, is that many features of the general language do not ap-
pear,sothat simpli(cid:1)edgrammarscouldbe designedforthem.Learningof
grammarsfromacorpusofthesublanguageispossible.Statisticallearning
techniques tendtoproduce rathercomplexgrammars.Cyreappliesevolu-
tionalgorithmstothetaskof(cid:1)ndingoptimalnaturallanguagecontext-free
statistical grammars. Because of the obvious difficulties in coding entire
grammars as individuals, Cyre(cid:6)s approach is a GA whose individuals are
grammar rules, endowed with bucket-brigade rewarding mechanisms as in
the classical Holland classi(cid:1)er systems. The evolutionary algorithm uses
only mutation in the form of random insertion of wildcards in selected
rules. The discovery of new rules is performed by instantiating the wild-
cardand evaluatingtheresultingrules. The(cid:1)tness of aruleisthenumber
of parsed sentences it has contributed to parse. Rules with small (cid:1)tness
Preface ix
are deleted in a culling step. Positive results are reported in this chapter
with some large corpora.
(cid:1) Discovering the spatial structure of proteins from their spectral images is
a result that may be in(cid:7)uential to pharmacological and biological stud-
ies. Gamalielsson and Olsson present the evaluation of protein structure
models with off-lattice evolutionary algorithms. The type of evolutionary
algorithmsappliedareevolutionstrategieswithandwithout(cid:1)tnesssharing
for premature convergence avoidance. The encoding of the protein struc-
tureis carried out bymeans of the angles between theresidues. The main
experimental argument of the paper is to study the effect of the (cid:1)tness
function de(cid:1)nition. The best results are obtained with a (cid:1)tness function
that assumes knowledge of the actual spatial structure. This is equivalent
toasupervisedtrainingproblem.Unsupervisedstructurediscoveryisreal-
izedby(cid:1)tnessfunctionsde(cid:1)nedoncharacteristicsofthecomposingamino
acids.
(cid:1) Classi(cid:1)cation is the most basic intelligent process. Among the diversity
of approaches, the decision trees and related rule-based systems have en-
joyed a great deal of attention, with some big success. Riquelme presents
the generation of hierarchical decision rules by evolutionary approaches,
comparingitto classicalC4.5decision treesovera well-knownbenchmark
collection of problems. The hierarchical decision rules possess some nice
intrinsic features, such as the parsimonious number of tests performed
to classify a data pattern. They are in fact a decision list, which is con-
structedincrementallywiththeevolutionaryalgorithm servingastherule
selector for each addition to the list. Individuals correspond to candidate
rules. Continuous-valued attributes are dealt with by the de(cid:1)nition of in-
tervals that quantize the attribute value range. Crossover and mutation
are accordingly de(cid:1)ned to deal with interval speci(cid:1)cations. The (cid:1)tness
function computation involves the correctly classi(cid:1)ed examples, the erro-
neously classi(cid:1)ed examples and the coverage of the problem space by the
rule.
(cid:1) Evolvable hardware is an active (cid:1)eld of research that aims at the unsu-
pervised generation of hardware ful(cid:1)lling some speci(cid:1)cations. A fruitful
area in this (cid:1)eld is that of evolving designs of gate circuits implement-
ing Boolean functions speci(cid:1)ed by truth tables, with great potential for
application to circuit design. Hernandez Aguirre reviews the evolutionary
approaches developed to handle this problem, which include classical bi-
nary GA and modi(cid:1)cations, Ant Colony Systems and variations of the
GP. Future lines of research include the design of appropriate platforms,
because most present work is performed in an extrinsic mode, while the
desiredgoalwouldbe toperformtheevolutionarysearch embeddedinthe
hardware being optimized, that is, in an intrinsic way.
(cid:1) System identi(cid:1)cation is the estimation of the parameters and structure
of a system processing an input signal, on the basis of the observed in-
put/outputpairs.Itisusedinthecontextofdesigningcontrolforprocesses