Table Of ContentAdaptation, Learning, and Optimization 14
SeriesEditors-in-Chief
Meng-HiotLim Yew-SoonOng
DivisionofCircuitsandSystems SchoolofComputerEngineering
SchoolofElectrical&Electronic NanyangTechnologicalUniversity
Engineering BlockN4,2b-39
NanyangTechnologicalUniversity NanyangAvenue
Singapore639798 Singapore639798
E-mail:[email protected] E-mail:[email protected]
Forfurthervolumes:
http://www.springer.com/series/8335
Siddhartha Shakya and Roberto Santana (Eds.)
Markov Networks in
Evolutionary Computation
ABC
Editors
SiddharthaShakya RobertoSantana
BusinessModellingandOperational IntelligentSystemsGroup
TransformationPractice FacultyofInformatics
BTInnovate&Design UniversityoftheBasqueCountry
Ipswich (UPV/EHU)
UK SanSebastian
Spain
ISSN1867-4534 e-ISSN1867-4542
ISBN978-3-642-28899-9 e-ISBN978-3-642-28900-2
DOI10.1007/978-3-642-28900-2
SpringerHeidelbergNewYorkDordrechtLondon
LibraryofCongressControlNumber:2012933771
(cid:2)c Springer-VerlagBerlinHeidelberg2012
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.Exemptedfromthislegalreservationarebriefexcerptsinconnection
with reviews or scholarly analysis or material supplied specifically for the purpose of being entered
and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of
this publication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’slocation,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.
PermissionsforusemaybeobtainedthroughRightsLinkattheCopyrightClearanceCenter.Violations
areliabletoprosecutionundertherespectiveCopyrightLaw.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Whiletheadviceandinformationinthisbookarebelievedtobetrueandaccurateatthedateofpub-
lication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityforany
errorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,withrespect
tothematerialcontainedherein.
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
Preface
SiddharthaShakyaandRobertoSantana
1 Why MarkovNetworks inEvolutionary Computation?
Inrecentyears,thefieldofheuristicoptimisationhasexperiencedanupsurgeinthe
numberofproposalsthatincorporatemachinelearningtechniquestothesearchof
optimalsolutions.Thetrueimpactthattheintroductionofintelligentsearchstrate-
gies has in terms of search efficiency and problemknowledgediscoveryhas been
confirmedinseveralapplicationstoresearchandindustrialdomains.
Estimation of distribution algorithms (EDAs) [3, 5, 6, 8] are a class of evolu-
tionary algorithms (EAs) characterised by the use of probabilistic modelling and
machine learning methods. Given their suitability to deal with complex optimisa-
tionproblemsandtheirscalabilityinmanyoptimisationdomains,EDAshavebeen
proposedasacompetentversionoftraditionalgeneticalgorithms.
EDAs work by replacing the genetic operators used by other EAs by a proba-
bilisticgraphicalmodel(PGM)[1,2,4,7,9]thatservesasastatisticaldescription
ofthebestsolutionsvisitedduringthesearch.Learningandsamplingmethodsare
respectivelyused to constructand sample the probabilistic model.By applyingin
each generation the steps of selection, learning, and sampling, EDAs are able to
progressivelymovethefocusofthesearchtopromisingareasofthesearchspace.
AcharacteristicfeatureofEDAsisthetypeofprobabilisticmodeltheyuse.Since
probabilistic modelling is very related to the representation of the solutions and
to the characteristics of the optimisation problem, model selection is not always
straightforward.A cleverchoiceofthePGM maybecriticalinthe solutionofthe
SiddharthaShakya
BusinessModellingandOperationalTransformationPractice,BTInnovate&Design,
Ipswich,UK
e-mail:[email protected]
RobertoSantana
IntelligentSystemsGroup,FacultyofInformatics,UniversityoftheBasqueCountry
(UPV/EHU),SanSebastian,Spain
e-mail:[email protected]
VI S.ShakyaandR.Santana
problem.ResearchonEDAshasmainlyfocusedontheuseofdirectedprobabilistic
modelssuchasBayesiannetworks.However,otherclassesofprobabilisticmodels
areabetterorcompetitivealternativeinmanysituations.
ThisbookfocusesontheanalysisandapplicationofMarkovnetworks,andundi-
rectedmodelsingeneral,asprobabilisticmodelsinEDAs.Markovnetworksarenot
merelyadifferentalternativetodirectedPGMs.Theysupportanumberofattributes
thatmakethemparticularlysuitableinthecontextofevolutionaryoptimisation.In
contrasttoBayesiannetworksandotherdirectedmodels,noinformationaboutthe
directionoftheinteractionbetweenthevariablesoftheproblemsisrequiredbythe
model.Thisfeatureisconsistentwiththecharacteristicsituationofmanyoptimisa-
tionproblemsforwhichthedirectionbetweentworelatedvariablesismeaningless.
Thebookcomprisesasetofrelevantcontributionsthatexplaintherationalebe-
hindtheuseofMarkovnetworksinEDAs,analyseimportantissuesfortheirdesign,
andpresentanumberofexemplaryapplicationofEDAsbasedonMarkovnetworks.
Readersinterestedinknowingaboutgeneralandefficientoptimisationapproaches,
basedontheuseofmachinelearningtechniques,willfindexamplesofhowtocon-
ceive, analyse and enhance algorithms fitted to their problems. More experienced
researchers from the evolutionarycomputation community,interested in getting a
detailed accountof the characteristicsof these algorithms,will find in this booka
panoplyofhotresearchtopicsandcurrentchallengesinthearea.
Markov networks have been profusely investigated in the field of statistical
physics and there is a strong link between them and other application fields of
machine learning techniques such as computer vision and sequence analysis. Re-
searchersin these areas, which are not familiar with evolutionaryalgorithms,will
findinthisbookanopportunitytodiscoveranexcitingapplicationareaforMarkov
network learning and sampling algorithms. Some of the chapters included in the
book address the question of how to adapt currently available methods based on
undirectedmodelstodealwithEDAslimitations.Theworkcomprisedinthisbook
showsthatresearchonEDAscouldcertainlyinspireotherdevelopmentsinthestudy
ofMarkovnetworks.Furthermore,thevarietyofavailabletheoreticalresultsinthe
field of Markov networks could provide a solid platform for better understanding
of advanced evolutionary algorithms and the conception of more powerful EDAs
basedonundirectedgraphicalmodels.
2 Book OrganisationandBrief Overviewofthe Chapters
Content
Thisbookisorganisedinthreemainparts:
1. Introductorypart.
2. TheoreticalcontributionorcurrentresearchonMarkovnetworkEDAs.
3. ApplicationsofEDAsbasedonMarkovnetworkstorealworldproblems.
Theintroductorypartcomprisesfivechaptersauthoredbytheeditorsofthisbook.
Chapter1givesanintroductiontoPGMs,withemphasisonMarkovnetworksand
Preface VII
thealgorithmsusedtolearnandsamplethesemodels.Chapter2explainsEDAsand
reviewsomeofthemostrecentadvancesinresearchonthesealgorithms,devoting
particularattentiontoEDAsbasedonMarkovnetworks.Thenextthreechaptersin
theintroductorypartdescribeindetailthreeparadigmaticEDAsbasedonMarkov
networks.Chapter3explainstheMarkovoptimisationalgorithm(MOA),arecently
introducedEDA. MOA has a simple structure and incorporatesadvancedfeatures
from previousEDAs based on Markov networks. Chapter 4 describes the DEUM
classofalgorithms.Acharacteristicfeatureofthesealgorithmsisthatthefitnessof
thesolutionsismodelledasaGibbsdistributiondefinedonthemodelstructure.In
contrast,theMN-FDAandMN-EDAapproachesdiscussedinchapter5whereorig-
inallyconceivedtolearnastatisticalmodelofthesolutionsstraightfromthedata.
The five chaptersincludedin this introductorypart should providethe reader less
familiar with EDAs with a basic understandingof the differentways for applying
MarkovnetworksinEDAs.
The second part of the book comprises a set of highly relevant and recent de-
velopments in the conception of EDAs based on Markov networks. This part is
openedby chapter6 whereMühlenbeinanalysessomeof the mostsensitivetheo-
reticalquestionsontheapplicationofEDAsbasedonMarkovnetworkstoadditively
decomposablefunctions.Twoofthequestionsconsideredarethederivationofcon-
vergenceresultsforfinitesamples,andtherelevanceandcomplexityrelatedtothe
useoftheGibbssamplingdistribution.Inchapter7,Ponce-de-LeónandDíaz-Díaz
introduceanalgorithmthatextendstherepresentationcapabilitiesoftreesbymeans
of adding edges based on a measure of edge relevance. The relationship between
modellearningandselection methodsis also investigated.Brownleeet al. present
inchapter8adeepanalysisoftheuseofMarkovnetworksasfitnessfunctionmod-
els. The chapter provides a detailed review of related approaches in evolutionary
computation and illustrates with examples the potential of fitness modelling with
Markov networks for the design of genetic operators and the extraction of a pri-
oriunknownproblemstructuralinformation.In chapter9,Mendiburuetal. inves-
tigate another hot topic in EDAs, the conception of alternative sampling methods
thatincorporateadvancedinferencetechniquesfromthemachinelearningdomain.
Thechapterillustrateshowtheimplementationofloopymax-propagationonfactor
graphsconstructedfrom the graphicalmodelslearntby the EDA can improvethe
efficiencyofthesealgorithms.Inchapter10,Karshenasetal.presenttheextension
ofEDAsbasedonMarkovnetworkstocontinuousproblems.Thechaptershowsthe
relationship between Gaussian distributions and Gaussian Markov random fields
andintroducesanalgorithmforlearningcontinuousmarginalproductmodelsinthe
context of EDAs. This second part of the book closes with chapter 11 by Höens
thatillustratesthebenefitsfromthecreativeapplicationofconceptsfromstatistical
physics to evolutionary computation. The chapter focuses on improving the EDA
probability estimation and sampling steps by means of subfunction merge strate-
gies that cover a higher number of interactions, and the use of generalised belief
propagationalgorithmsforefficientsampling.
The third and last part of the book comprises three chapters. In chapter 12,
McCall et al review different applications of the DEUM approach. This chapter
VIII S.ShakyaandR.Santana
illustrates on the applicability of EDAs based on Markov networks to real world
optimisation problems,coveringdomainssuch as chemotherapyoptimisation,dy-
namicpricing,agriculturalbiocontrol,andcase-basedfeatureselection.Chapter13,
by Soto et al., addresses problems with a continuous representation. A molecular
dockingproblemissolvedbymeansofvariantsofcopula-basedEDAs.Themod-
elling approach, a sophisticated PGM based on copulas is shown to outperform
UMDA and other evolutionary algorithms for this challenging optimisation from
bioinformatics.Chapter 14 by Handa investigatesthe use of EDAs based on con-
ditional random fields for solving reinforcement learning problems. This chapter
illustrateshowtheEDAapplicationtothisdomaincanresolvetasksthataregener-
allydifficultfortraditionalreinforcementlearningalgorithms.
References
1. Castillo,E.,Gutierrez,J.M.,Hadi,A.S.:ExpertSystemsandProbabilisticNetworkMod-
els.Springer(1997)
2. Koller,D.,Friedman,N.:ProbabilisticGraphicalModels:PrinciplesandTechniques.The
MITPress(2009)
3. Larrañaga,P.,Lozano,J.A.(eds.):EstimationofDistributionAlgorithms.ANewToolfor
EvolutionaryComputation.KluwerAcademicPublishers,Boston(2002)
4. Lauritzen,S.L.:GraphicalModels.OxfordClarendonPress(1996)
5. Lozano,J.A.,Sagarna,R.,Larrañaga,P.:ParallelEstimationofDistributionAlgorithms.
In:Larrañaga,P.,Lozano,J.A.(eds.)EstimationofDistributionAlgorithms.ANewTool
forEvolutionaryComputation,pp.129–145.KluwerAcademicPublishers(2001)
6. Mühlenbein,H.,Paaß,G.:Fromrecombinationofgenestotheestimationofdistributions
I.Binaryparameters.In:Ebeling,W.,Rechenberg,I.,Voigt,H.-M.,Schwefel,H.-P.(eds.)
PPSN1996.LNCS,vol.1141,pp.178–187.Springer,Heidelberg(1996)
7. Pearl, J.: Probabilistic Reasoning in Intelligent Systems. Morgan Kaufman Publishers,
PaloAlto(1988)
8. Pelikan, M., Goldberg, D.E.,Lobo, F.:A survey of optimization by building and using
probabilisticmodels.ComputationalOptimizationandApplications21(1),5–20(2002)
9. Whittaker,J.:GraphicalModelsinAppliedMultivariateStatistics.WileySeriesinProba-
bilityandMathematicalStatistics,NewYork(1991)
Contents
Preface ....................................................... V
SiddharthaShakyaandRobertoSantana
1 WhyMarkovNetworksinEvolutionaryComputation?......... V
2 BookOrganisationandBriefOverview
oftheChaptersContent ................................... VI
References.................................................... VIII
PartI Introduction
1 ProbabilisticGraphicalModelsandMarkovNetworks ........... 3
RobertoSantanaandSiddharthaShakya
1.1 Introduction............................................. 3
1.2 GraphsandIndependenceGraphs........................... 5
1.2.1 RelevantConceptsfromGraphTheory............... 6
1.3 BayesianNetworks....................................... 7
1.4 MarkovNetworks........................................ 8
1.4.1 MarkovNetworksofBoundedComplexity ........... 9
1.4.2 GaussianMarkovRandomFields ................... 10
1.5 LearningMarkovNetworks................................ 11
1.5.1 LearningofDiscreteMarkovNetworks .............. 11
1.5.2 LearningofGaussianMarkovRandomFields......... 13
1.6 SamplingMarkovNetworks ............................... 14
1.7 OtherProbabilisticGraphicalModels ....................... 15
1.7.1 MixtureModels.................................. 15
1.7.2 FactorGraphs ................................... 16
1.7.3 RegionGraphs................................... 16
References.................................................... 17
X Contents
2 AReviewofEstimationofDistributionAlgorithmsandMarkov
Networks ................................................. 21
SiddharthaShakyaandRobertoSantana
2.1 Introduction............................................. 21
2.2 EstimationofDistributionAlgorithm........................ 22
2.2.1 ASimpleEDAWorkflow.......................... 22
2.2.2 ProbabilisticGraphicalModelsandEDAs............ 23
2.3 EDACategorisation ...................................... 24
2.3.1 UnivariateEDA.................................. 25
2.3.2 MultivariateEDA ................................ 27
2.4 EstimationofDistributionAlgorithmswithMarkovNetworks... 29
2.4.1 Motivation ...................................... 29
2.4.2 History ......................................... 30
2.4.3 ReviewofPopularEDAswithMarkovNetworks...... 30
2.4.4 CurrentResearchonEDA
BasedonMarkovNetworks........................ 32
References.................................................... 33
3 MOA-MarkovianOptimisationAlgorithm .................... 39
SiddharthaShakyaandRobertoSantana
3.1 Introduction............................................. 39
3.2 LocalandGlobalPropertiesinGraphicalModels ............. 40
3.3 MarkovNetworkBasedEDAs ............................. 42
3.4 MarkovianitybasedOptimisationAlgorithm(MOA)........... 42
3.5 ResultsandAnalysis ..................................... 45
3.5.1 ComparisonwithOtherEAs ....................... 46
3.5.2 IntroducingPermutation........................... 47
3.5.3 IntroducingOverlaps ............................. 47
3.5.4 IncorporatingPriorInformation..................... 48
3.5.5 GibbsSamplingvs.TemperatureLessSampling....... 49
3.6 Conclusion.............................................. 51
References.................................................... 51
4 DEUM-DistributionEstimationUsingMarkovNetworks ........ 55
SiddharthaShakya,JohnMcCall,AlexanderBrownlee,
andGilbertOwusu
4.1 Introduction............................................. 55
4.2 MarkovNetworks........................................ 56
4.3 FitnessModellingandDEUMAlgorithms ................... 58
4.3.1 FitnessModelling ................................ 59
4.3.2 UnivariateMFM ................................. 60
4.3.3 MultivariateMFM................................ 60
4.3.4 EstimatingMRFParameters ....................... 61
4.3.5 SamplingMarkovNetworks ....................... 61
4.4 DEUM - A UnivariateDEUM Algorithm with Direct
d
SamplingofGibbsDistribution............................. 63
Contents XI
4.5 Is-DEUM -AMultivariateDEUMwithIsingStructureand
g
GibbsSampler........................................... 64
4.6 FullDEUM-AFullyMultivariateGeneralDEUMAlgorithm... 65
4.6.1 EstimationofUndirectedStructure.................. 66
4.6.2 FindingCliquesandAssigningPotentials ............ 66
4.6.3 SamplingNewSolution ........................... 67
4.7 Summary ............................................... 67
References.................................................... 68
5 MN-EDAandtheUseofClique-BasedFactorisationsinEDAs .... 73
RobertoSantana
5.1 Introduction............................................. 73
5.2 FactorisationsandFactorisedDistributionAlgorithms.......... 75
5.3 Region-BasedApproximationsofaProbabilityDistribution..... 77
5.3.1 KikuchiApproximation ........................... 78
5.3.2 AlgorithmforConstructingValidRegionBased
Decompositions.................................. 79
5.4 GibbsSamplingoftheKikuchiApproximation ............... 80
5.5 LearningtheKikuchiApproximationfromtheData ........... 81
5.6 TheMarkovNetworkEDA ................................ 82
5.7 RelatedWorkandExtensionstoMN-EDA ................... 83
5.7.1 MixtureofKikuchiApproximations................. 83
5.7.2 CombiningKikuchiApproximationswithInference
Methods ........................................ 84
5.7.3 GeneralisedEDAsBasedonUndirectedModels....... 84
5.8 Conclusions............................................. 86
References.................................................... 86
PartII Theory
6 ConvergenceTheoremsofEstimation
ofDistributionAlgorithms................................... 91
HeinzMühlenbein
6.1 Introduction............................................. 91
6.2 ConvergenceTheoryforInfiniteSamples .................... 92
6.2.1 AConvergenceTheoremforMN-GIBBS ............ 94
6.2.2 SomeImplementationsofEDAsUsing Markov
Networks ....................................... 97
6.3 ConvergenceofMN-GIBBSwithFiniteSamples.............. 98
6.3.1 LearningofMarkovNetworksandFactorGraphs ..... 99
6.3.2 SampleComplexityforLearningtheADFStructure
byProbing ...................................... 99
6.4 AComparisonofMarkovNetworksandBayesianNetworks ... 100
6.4.1 ASamplingMethodFindingtheMost
ProbableStates .................................. 102
6.4.2 ComputationalComplexityofGibbsSampling........ 102
