Multi Objective Optimization of Computationally Expensive Multi-Modal Functions with RBF ...

23 Pages·2014·0.28 MB·English

Online Supplement for
Multi Objective Optimization of Computationally Expensive
Multi-Modal Functions with RBF Surrogates and Multi-Rule
Selection
Journal of Global Optimization
TaimoorAkhtar1 andChristineA.Shoemaker2
1SchoolofCivilandEnvironmentalEngineering,CornellUniversity,Ithaca,NY14853,USA.
e-mail: [email protected]
2SchoolofCivilandEnvironmentalEngineering,CornellUniversity,Ithaca,NY14853,USA.
e-mail: [email protected]
A. Introduction
Thisdocumentmaybeconsideredasanonlinesupplementtothepapertitled“MultiObjectiveOptimiza-
tionofComputationallyExpensiveMulti-ModalFunctionswithRBFSurrogatesandMulti-RuleSelection”
(the paper will be referred as the main paper in subsequent discussions). The primary purpose of the
supplementarymaterialwithinthisdocumentistoprovideanelaboratediscussionontheuseof1)Alter-
nativealgorithmsforembeddedevolutionaryoptimizationinSteps2.2and2.3oftheGOMORSAlgorithm
FrameworkdescribedinSection3.1ofthemainpaper,and2)MultipleSelectionrulesemployedwithinthe
GOMORSalgorithmforselectionofpointsforexpensiveevaluations. Furthermore, thissupplementalso
providesadescriptionoftheTestProblemsusedforcomparisonofGOMORSwithParEGOandNSGA-II,
andadescriptionofthealgorithmparametersemployedforGOMORS,ParEGOandNSGA-II.
Section C of this document provides a summary of computer experiments performed with different
evolutionary algorithms for solving the embedded global optimization problems in Steps 2.2 and 2.3 of
theAlgorithm(definedinSection3.1ofthemainpaper). SectionDofthisdocumentprovidesasummary
of the results obtained from computer experiments performed for analyzing the comparative advantage
of using multiple selection rules for selection of points for expensive evaluation (especially, if they are
used to select multiple points in a single algorithm iteration for evaluation in parallel). Section B of this
document provides a description of the Synthetic Test Problems used for comparing algorithms, as well
asanelaboratedescriptionoftheHypotheticalGroundwaterRemediationProblemusedinthecomputer
experiments. Section E provides the parameters used for all algorithms compared in this study, which
includeNSGA-II,ParEGOandGOMORS.SectionFprovidesadditionalfiguresforcomparativeanalysisof
GOMORSandParEGO.
1
B. Test Problems
B.1 SyntheticTestSuite
TheperformanceofGOMORSwastestedviacomputationalexperimentsperformedonelevencontinuous
testfunctions(collectivelyreferredastheSyntheticTestSuite). Fiveofthetestfunctionsemployedinthe
experimentsarepartoftheZDTtestsuite[16],whiletheremainingsixarederivedfromtheworkofLiand
Zhang[7]. Theproblemsincorporatedinthesynthetictestsuitearenotexpensivetoevaluate. However,
theyincorporatevariousoptimizationchallenges(forinstancemultimodality,non-convexityofthePareto
Front,complicatedParetoSetsetc)inherentinrealworldmultiobjectiveoptimizationproblems.
TableB-1:DescriptionoftheZDTTestFunctions
Name MathematicalDescription ProblemChallenge
f (x)=x
1 1
(cid:112)
f (x)=g(x)[1− f (x)/g(x)]
2 1
ZDT1 whereg(x)=1+9((cid:80)ni=2xi)/(n−1)
x∈[0,1]n
nmayvarybetween2and30
f (x)=x
1 1
f (x)=g(x)[1−(f (x)/g(x))2]
2 1
ZDT2 whereg(x)=1+9((cid:80)ni=2xi)/(n−1) TheParetoFrontisnon-convex.
x∈[0,1]n
nmayvarybetween2and30
f (x)=x
1 1
(cid:112)
f (x)=g(x)[1− f (x)/g(x)− f1(x)sin(10πx )] TheParetofrontisdisconnected
2 1 g(x) 1
ZDT3 whereg(x)=1+9((cid:80)ni=2xi)/(n−1) andhasseveralnoncontiguous
x∈[0,1]n convexparts.
nmayvarybetween2and30
f (x)=x
1 1
(cid:112)
f (x)=g(x)[1− f (x)/g(x)]
2 1
ZDT4 whereg(x)=1+10(n−1)+(cid:80)ni=2[xi2−10cos(4πxi)] 219locallyoptimalfrontsexist.
x ∈[0,1],x ∈[−5,5],∀i∈{2,...,n}
1 i
nmayvarybetween2and30
f (x)=1−exp(−4x )sin6(6πx )
1 1 1
f (x)=g(x)[1−(f (x)/g(x))2] Thesearchspaceisnon-uniform,
2 1
ZDT6 whereg(x)=1+9[((cid:80)ni=2xi)/(n−1)]0.25 anddistributionofsolutionson
x∈[0,1]n theParetofrontisnon-uniform.
nmayvarybetween2and30
2
TableB-2:DescriptionoftheLZFTestFunctions
Name MathematicalDescription ProblemChallenge
f (x)= x + 2 (cid:80) [x −sin(6πx + jπ)]2
f1(x)=11− √|Jx1| +j∈J21 (cid:80)j [x −si1n(6πnx + jπ)]2,where TheParetoSetisanon-linear
2 1 |J2| j∈J2 j 1 n
LZF1 J ={j| jisoddand2≤ j≤n}, curve. Problemisalsoreferredas
1
J ={j| jisevenand2≤ j≤n},and F2in[7].
2
x∈[0,1]x[−1,1]n−1
f (x)= x + 2 (cid:80) y2
f1(x)=11− √|Jx1| +j∈J21 (cid:80)j z2,where
2 1 |J2| j∈J2 j
J ={j| jisoddand2≤ j≤n}, TheParetoSetisanon-linear
1
LZF2 J ={j| jisevenand2≤ j≤n}, curve. Problemisalsoreferredas
2
y = x −[0.3x2cos(24πx + 4jπ)+0.6x ]cos(6πx + jπ), F5in[7].
j j 1 1 n 1 1 n
z = x −[0.3x2cos(24πx + 4jπ)+0.6x ]sin6πx + jπ),
j j 1 1 n 1 1 n
andx∈[0,1]x[−1,1]n−1
ff12((xx))==1x1−+√|J2x1|1[4+(cid:80)|J21j∈|[J41y(cid:80)2jj−∈J21y(cid:81)2j −j∈J21(cid:81)cosj∈(J210√cyjjoπs)(+20√y2jj]π)+2] Manylocallyoptimalfronts
LZF3 whereJ andJ arethesameasinLZF1, exist. Problemisalsoreferredas
1 2
y = x −x0.5(1+3(nj−−22)),and F8in[7].
j j 1
x∈[0,1]n
f (x)= x + 2 (cid:80) h(y )
1 1 |J1| j∈J1 j
f (x)=1−x2+ 2 (cid:80) h(y ),where
2 1 |J2| j∈J2 j
J ={j| jisoddand2≤ j≤n}, ParetoFrontisnon-convexand
1
LZF4 J ={j| jisevenand2≤ j≤n}, ParetoSetisnon-linear. The
2
y =sin(6πx + jπ), ProblemisreferredasUF4in.
j 1 n
h(t)=|t|/(1+2e2|t|),and
. x∈[0,1]x[−2,2]n−1
f (x)= x + 2 (cid:80) [x −0.8x cos(6πx + jπ)]2
f1(x)=11− √|Jx1| +j∈J21 (cid:80)j [x −1 0.8x sin1(6πnx + jπ)]2 TheParetoSetisanon-linear
2 1 |J2| j∈J2 j 1 1 n
LZF5 J ={j| jisoddand2≤ j≤n} curve. Problemisalsoreferredas
1
J ={j| jisevenand2≤ j≤n} F3in[7].
2
x∈[0,1]x[−1,1]n−1
f (x)= x + 2 (cid:80) [x −0.8x cos(6πx1+jnπ)]2
f1(x)=11− √|Jx1| +j∈J21 (cid:80)j [x −1 0.8x si3n(6πx + jπ)]2 TheParetoSetisanon-linear
2 1 |J2| j∈J2 j 1 1 n
LZF6 J ={j| jisoddand2≤ j≤n} curve. Problemisalsoreferredas
1
J ={j| jisevenand2≤ j≤n} F4in[7].
2
x∈[0,1]x[−1,1]n−1
3
The ZDT test problems proposed by Zitzler et. al [16] include five continuous multi-objective opti-
mizationtestproblemsallofwhichproposedifferentchallengesinconvergingtotheParetooptimalfront.
Thesechallenges,alongwithmathematicaldescriptionsoftheZDToptimizationproblems,areprovidedin
TableB-1.TheParetofrontsforthefivetestproblemsareknown. Meaningfulcomparisonsofrelativeper-
formance of different algorithms on these test problems can be based on how the quality of optimization
solutionsincreasesasthenumberoffunctionevaluationsincrease,ratherthanontheactualrunningtime
ofthealgorithms(sincepurposeofcomparisonisevaluationofalgorithmperformanceforcomputationally
expensiveproblems).
PriorliteratureonthedesignofsuitabletestproblemsindicatethatcomplicatedshapesofParetoFronts
add to the challenges of solving multi objective optimization problems via heuristics [7]. The ZDT test
functions incorporate some of these challenges, including non-convex Pareto Fronts, disconnected Pareto
fronts etc. However, the shapes of Pareto Sets for the ZDT functions are very simple. The Pareto Set of a
problemreferstothesetofParetooptimalsolutionsoftheprobleminthedecisionspacethatcorresponds
totheParetoFront. ParetoSetsfortheZDTproblemsarelinesegments. Itishighlyunlikelyforrealworld
problemstohavesuchsimpleParetoSets.
Li and Zhang [7] introduced a general class of continuous multi objective optimization test problems
whereParetoSetscouldbedefinedarbitrarily. Furthermore, LiandZhang[7]proposedninespecifictest
problemsfromthegeneralclassofproblemsintroduced.Wehaveincorporatedfiveofthesetestinoursyn-
thetictestsuite. WewillreferthesetestproblemsastheLZFtestfunctionsforfuturediscussionreference.
The LZF test suite also includes a test problem which was proposed in the CEC 2009 MOEA competi-
tion [13]. This test function (LZF4) is also from the general class of optimization problems introduced by
LiandZhang. TheLZFtestproblemsproposedifferentoptimizationchallenges. Thesechallenges, along
withmathematicaldescriptionsoftheLZFoptimizationproblems,areprovidedinTableB-2.
TheZDTandLZFfunctionswillcollectivelybereferredastheSyntheticTestSuite(eleventestproblems)
infuturediscussions. WhileSyntheticTestSuiteidentifiedforperformanceanalysisaddressesmostofthe
optimizationchallengesdiscussed,someadditionaloptimizationchallengeshavebeenleftunexplored,for
instance,constrainthandling,andtestingonproblemswithmorethantwoobjectives.
B.2 GroundwaterRemediationDesignProblem
The test problems employed in our computer experiments also include a complex model describing the
detoxification of contaminated groundwater using aerobic bioremediation [8]. Aerobic bioremediation is
adecontaminationprocessthatinvolvestheinjectionofelectronacceptors(e.g. oxygen)ornutrients(e.g.
nitrogenandphosphorus)intothegroundwatertopromotethegrowthofmicroorganismsthatcantrans-
form the contaminant into a harmless substance. Cleaning up contaminated groundwater is financially
expensive and the cost could run into tens of millions of dollars. Hence, the use of a complex model de-
scribingthedecontaminationprocess,coupledwithanefficientglobaloptimizationalgorithmcouldassist
4
indevisingcleanupstrategiesthatmayyieldhugesavingsforenvironmentalcleanup.
The groundwater remediation model used in our experiments considers a hypothetical contaminated
aquiferwithcharacteristicsthataresymmetricaboutahorizontalaxis. Weassumethatthereare6injection
wells in the contaminated aquifer, the locations of which have been fixed. 84 monitoring wells are also
assumedwhichmeasuretheconcentrationofthecontaminantatspecifiedtimeperiods. Theaquiferisdis-
cretizedusingatwo-dimensionalfiniteelementmesh.Itisassumedthattheinjectionwellsandmonitoring
wellsarelocatedatthenodesofthemeshandthesewellsarealsosymmetricaboutthehorizontalaxis. A
two-dimensionalfiniteelementsimulationmodelcalledBio2D[10],isusedtodescribegroundwaterflow
andthechangesintheconcentrationsofthecontaminant,oxygen,andbiomass. Themodelequationsare
nonlinearbecausethegrowthequationsforthemicroorganismsrepresentMonodkinetics.
Theoptimizationprocessconsiderstheproblemofdecidingthepumpingratesforinjectionwells,which
areusedtosupplyoxygenintothegroundwater. Duetothesymmetryofthecontaminatedarea,weonly
needtomakepumpingdecisionsforthe3injectionwellsononesideoftheaxisofsymmetry. Theplanning
timeperiodisdividedintomultiplemanagementperiods,andpumpingratesremainconstantineachman-
agementperiod. Weuse2,4and8managementperiodsinourcomputerexperimentswhichcorrespondto
6, 12and24decisionvariables, respectively. Becausethepumpingofoxygenatedwaterisexpensive, our
goalistodeterminethepumpingratesforeachinjectionwellatthebeginningofeachmanagementperiod
that minimize the total pumping cost and the contaminant concentration at the monitoring wells. Yoon
and Shoemaker [12] incorporated theminimization ofcontaminant concentration, into thetotal pumping
costobjectivebymeansofapenaltyterm. Thisresultedintheformulationofthegroundwaterremediation
designproblemasasingleobjectiveglobaloptimizationproblem.
We visualize the groundwater remediation design problem as a multi objective optimization problem
wheretheconflictingobjectivesoftheproblemare1)minimizationofremediationcostand2)minimization
ofcontaminantconcentration.Thedecisionvariablesoftheproblemaretheinjectionratesat3differentwell
locationsduringeachofmmanagementperiods. Thenumberofdecisionvariablesvarybetween6and24
(asdiscussedabove),dependinguponthenumberofmanagementperiodsincorporatedinthenumerical
computation model. The groundwater problem can be formulated as two separate multi-objective opti-
mization problems. In the first formulation, the first objective is minimization of the cost of remediation,
and the second objective is minimization of the maximum contaminant concentration recorded at the 84
monitoring wells at the end of the planning time. This formulation is referred to as the GWDM problem
in the main paper and in this document. In the second formulation, the first objective is minimization of
thecostofremediationandthesecondobjectiveisminimizationoftheaveragecontaminantconcentration
recordedatthe84monitoringwellsattheendoftheplanningtime. Thisformulationisreferredtoasthe
GWDAprobleminthemainpaperandinthisdocument. Mathematicalformulationsofthetwomultiob-
jectivevariantsofthegroundwaterremediationdesignproblemareprovidedinTable2ofthemainpaper.
Thesimulationtimesforgroundwaterremediationproblemscanvarybetweenminutesandhoursde-
5
pendingonthecomplexityofthemodelandthesizeofthemodeledregion. Thisexampleusesarelatively
coarse grid and requires only about 0.5 sec. to run each simulation on a 1.3 GHz Intel Core i5 machine.
However, it is representative of the type of multi objective optimization formulations used in more com-
plexandcomputationallyexpensivegroundwaterbioremediationproblems.
C. Alternative Methods for Embedded Evolutionary Optimization
Section 3.1 of the main paper defines the general framework of the GOMORS algorithm where surrogate
optimization problems defined in Steps 2.2 and 2.3b of the algorithm framework are solved within each
algorithm iteration. The primary purpose of the surrogate optimizations is to find near optimal fronts
given that the expensive functions, F(x), are replaced by the corresponding response surface model ap-
proximations. Evaluation points are subsequently chosen from the solutions obtained via the surrogate
optimizations(alsoreferredascandidatesolutionsorcandidatepopulations)duringeachalgorithmitera-
tion.
Step2.2performsaglobalsurrogateassistedsearchandhence,solvesasurrogatemultiobjectiveopti-
mizationproblemoverthewholedecisionspace,whileStep2.3bperformsalocalsurrogateassistedsearch
andhence,solvesasurrogateoptimizationproblemwithinalocalizedneighborhoodoftheleastcrowded
solutionontheexpensivelyevaluatednon-dominatedfront(seeStep2.3aofthealgorithmframeworkde-
fined in Step 3.1 of the main paper). While solving them may be relatively inexpensive, the surrogate
optimizationproblemsarenottrivial(sincesurrogatescanbemultimodal).
TheoptimizationliteratureindicatesthatMultiObjectiveEvolutionaryAlgorithms(MOEAs)arearevo-
lutionaryheuristicapproachforsolvingnontrivialandmultimodalmultiobjectiveoptimizationproblems.
Debatal.[5]andCoelloetal.[3]suggestthatuseofMOEAscanbehighlybeneficial,sincethepopulation
based structure of an evolutionary algorithm can be exploited to simultaneously converge towards the
Pareto front, and maintain a diverse set of trade-offs. Since the surrogate optimization problems defined
in Steps 2.2 and 2.3 of the algorithm framework could potentially have non-linear and multi-modal ob-
jectives, and our aim is to find good approximate solutions of the surrogate problems (both in terms of
convergenceanddiversity),wemakeuseofevolutionaryoptimizationforsolvingthesurrogateproblems
withineachiterationofGOMORS.Inthisregard,weperformedcomputerexperimentswitheitherNSGA-
II[4],MOEA/D[14],orAMALGAM[11]asevolutionaryalgorithmsforsolvingthesurrogateoptimization
problemswithineachiterationoftheGOMORSalgorithmframework. Thepurposeoftheseexperiments
wastoidentifyoneofthesealgorithmasmostsuitedforsurrogateoptimizationwithinGOMORS.Please
notethatsurrogateoptimizationwithreferencetoGOMORSisalsoreferredasembeddedevolutionaryop-
timizationinsubsequentdiscussions. Theevolutionaryalgorithmsmentionedabovearedescribedbriefly
in sub sections C.1 - C.3. The experimental setup employed for comparison of performance of these al-
gorithms within the GOMORS framework, and pertaining results are discussed in sub sections C.4 and
6
C.5.
C.1 NonDominatedSortingGeneticAlgorithm-II(NSGA-II)
NSGA-II,proposedbyDebetal.[4]isarevolutionaryandextremelypopularmulti-objective(MOEA),and
has been applied to various applied engineering problems across numerous disciplines [3] [2]. NSGA-II
handles the evolutionary search optimization process by ranking and archiving parent and child popula-
tionsaccordingtoanon-dominationsorting(ameasureofconvergence),andcrowdingdistance,whichis
ameasureofdiversityofasolution,onaparticularfront. TheNSGA-IIaimsatusingPareto-dominanceto
movethethenon-dominatedfronttowardsconvergence,duringthesearchprocess,andcanhandlehighly
non-linearandmultimodalobjectives.
C.2 MultiObjectiveEvolutionaryAlgorithm-Decomposition(MOEA/D)
Aggregate functions can also be used in the multi-objective optimization process in order to convert a
multi-objective problem into many single- objective problems. The aggregate methodology falls under
a separate class of MOEAs within Coello et al.’s [3] classification, where the aim is to solve many sin-
gleobjectiveoptimizationproblems, withdifferentaggregationpriorities, toconvergetotheParetofront,
and maintain divergence. The MOEA/D [14] uses aggregate functions, and simultaneously solves many
single-objectiveTchebycheffdecompositions(Tchebycheffdecompositionisapreferencebasedaggregation
method for converting a vector of objectives into a single objective) of multi-objective problems in a sin-
gle run.The MOEA/D is an established benchmark algorithm and has won the 2009 IEEE Congress on
EvolutionaryComputation(CEC2009)competition.
C.3 AMALGAM - A Multi Algorithm Genetically Adaptive Multi-Objective Opti-
mizationMethod
AMALGAM[11]isamulti-methodevolutionaryalgorithm,whichincorporatessearchmechanicsofvari-
ousalgorithms. ThekeyfeatureofAMALGAMissimultaneousmulti-methodsearchandthesearchmech-
anismselectionisself-adaptive. AMALGAMincorporatesfourcandidateMOEAs,NSGA-II,PSO,DE,and
AMS. Recent literature has seen various applications of AMALGAM including Zhang et al.’s work [15],
whichincludesapplicationofAMALGAMformulti-sitecalibration,withcomparisonagainstSPEA2and
NSGA-II.
C.4 ExperimentalSetup
The relative performance of NSGA-II, MOEAD and AMALGAM as embedded MOEAs within the GO-
MORSalgorithmframeworkwasassessedonthesynthetictestsuite. Numberofdecisionvariablesforall
7
LZF1 LZF2 LZF3
0.1 0.08 0.14
me0.09 me0.07 me0.12
volu0.08 volu0.06 volu 0.1
er0.07 er er
Hyp0.06 Hyp0.05 Hyp0.08
ed 0.05 ed 0.04 ed 0.06
over0.04 over0.03 over0.04
nc0.03 nc0.02 nc
U U U0.02
0.02 0.01
0
NSGA−II MOEAD AMALGAM NSGA−II MOEAD AMALGAM NSGA−II MOEAD AMALGAM
Algorithm Algorithm Algorithm
LZF4 LZF5 LZF6
0.18
me0.17 me0.07 me0.08
olu olu0.06 olu
v0.16 v v
per per0.05 per0.06
y0.15 y y
H H H
ed 0.14 ed 0.04 ed 0.04
er er0.03 er
ov0.13 ov ov
c c c0.02
n n0.02 n
U0.12 U U
0.11 0.01 0
NSGA−II MOEAD AMALGAM NSGA−II MOEAD AMALGAM NSGA−II MOEAD AMALGAM
Algorithm Algorithm Algorithm
FigureC-1: ComparativeanalysisofevolutionaryalgorithmsembeddedintoGOMORS-Boxplotsofuncoveredhy-
pervolumemetricvaluesobtainedafter200functionevaluationswithapplicationofGOMORSusingeitherofNSGA-II,
MOEADorAMALGAMasembeddedevolutionaryoptimizationalgorithms.ResultsforthesixLZFfunctionswith8
decisionvariablesareshownineachsubplot.Inallcaseslowervaluesofuncoveredhypervolumearebest.
testproblemswerefixedateight. AllparametersettingsofGOMORSwerekeptconstant. Theparameter
settings of GOMORS used in the comparative analysis of the embedded MOEAs are defined in Table E-
6andSectionE.3. TheonlychangeinGOMORSwastheuseofeitherofthethreeembeddedMOEAsfor
solvingtheembeddedoptimizationproblemsdefinedinSteps2.2and2.3bofthealgorithmframework(see
Section3.1ofthemainpaper).Tenoptimizationexperimentswereperformedforeachofthethreemodified
versionsofGOMORS,oneachtestproblemandfunctionevaluationswerelimitedto200. ThesameLatin
hypercubesampleswereusedforcorrespondingtrialsofeachversionofGOMORS,oneachtestproblem.
The uncovered hypervolume metric [1] was used to compare the performance of the three versions
of GOMORS with different MOEAs. Uncovered hypervolume is the difference between the total feasible
objectivespace(illustratedinFigure1(a)ofthemainpaper)andtheobjectivespacedominatedbyestimate
of the Pareto front obtained by an algorithm. A lower value of the index indicates a better solution and
theidealvalueiszero. Thenextsectiondiscussestheresultsobtainedfromtheexperimentswithdifferent
MOEAs.
8
C.5 ResultsandAnalysis
TheuncoveredhypervolumemetricvaluesobtainedafterperformingoptimizationexperimentsontheLZF
problemswithapplicationofGOMORSwiththethreealternativeembeddedMOEAs,isdepictedviabox
plotsinFigureC-1. Eachofthesixsubplotsofthefiguredisplaytheresultspertainingtoeachofthesix
LZFproblems. Lowervaluesofthemetricsignifysuperiorityofperformanceandalowerspreadwithina
boxplotdepictsdefinitenessofperformance.
Alternate Embedded Evolutionary Algorithms
1.3
1.2
e
m
u
ol1.1
v
er
p
Hy 1
d
e
er
v0.9
o
c
n
U
0.8
0.7
NSGA−II MOEAD AMALGAM
Algorithm
FigureC-2: ComparativeanalysisofevolutionaryalgorithmsembeddedintoGOMORS-Boxplotsofuncoveredhy-
pervolumemetricvaluessummedovertheelevensynthetictestfunctions,andobtainedafter200functionevaluations
withapplicationofGOMORSusingeitherofNSGA-II,MOEADorAMALGAMasembeddedevolutionaryoptimiza-
tionalgorithms.Inallcaseslowervaluesofuncoveredhypervolumearebest.
ThefirstconclusionthatcanbedrawnfromtheresultsdepictedinFigureC-1isthatitishardtoidentify
the difference between performance of all three embedded MOEAs. This is true especially in the case of
LZF2, LZF5 and LZF6 (see Table B-2 and Figure C-1). However, in the case of LZF1, LZF3, and LZF4, it
canbeseenthattheoverallperformanceofAMALGAM(asembeddedMOEAofGOMORS)isbetterthan
NSGA-IIandMOEAD,bothintermsofattainmentoflowermedianmetricvaluesandlowerspreadinthe
box plots. After an overall analysis of the box plots of Figure C-1, it can concluded that performance of
AMALGAMisslightlybetterthanNSGA-IIandMOEADasanembeddedMOEAforGOMORS.
Resultsofallsynthetictestproblemswerealsocompiledforanalysisbysummingthemetricvaluesof
each of the ten optimization experiments performed on each test problem. The uncovered hypervolume
metricvalueobtainedfromeverytrialofeachindividualtestproblemandalgorithmcombinationcouldbe
consideredasarandomsamplefromanunknowndistribution. Assumingthattheunknowndistributions
9
of each individual test problem and algorithm combination are independent across a single algorithm, a
sum of metric values of each trial across a single algorithm is another random sample from an unknown
distribution which is a convolution of the independent unknown distributions. This convolution based
metric summarizes overall performance of an algorithm on all synthetic test problems and is depicted in
FigureC-2forcomparisonofperformanceofembeddedMOEAsacrossallsynthetictestfunctions.
Figure C-2 also highlights that performance of all embedded MOEAs is comparable with reference to
their application to the eleven synthetic test functions (and considering that results are shown as a con-
volutionacrossalltestproblems). However,FigureC-2alsoindicatesthatperformanceofAMALGAMis
slightlybetterthanNSGA-IIandMOEAD.TheworstattainedconvolutedmetricvaluefromAMALGAM
issignificantlylowerthantheworstattainedconvolutedmetricvaluesofNSGA-IIandAMALGAM.Based
ontheresultsdepictedinFiguresC-1andC-2,weusedAMALGAMastheembeddedMOEAinGOMORS.
However,itisworthmentioningthatperformancesofthetestedembeddedMOEAswereclose.
D. Experimental Assessment of Selection Rules
Section3.4ofthemainpaperprovidesadiscussiononthevariousselectionrulesemployedwithintheGO-
MORSalgorithmforselectionofpointsforexpensiveevaluationswithineachalgorithmiteration. Points
forexpensiveevaluationsareselectedfromthecandidatepointsobtainedafterembeddedoptimizationsof
Steps2.2and2.3ofthealgorithmframework(definedinSection3.1ofthemainpaper). Theselectedpoints
aresubsequentlyevaluatedbytheexpensiveobjectivefunctions,F(x),withinanalgorithmiteration.
Selection of points for expensive evaluations is a critical step in the algorithm because evaluation of
selected evaluations points is usually, by far, the most computationally expensive step of the algorithm,
when applied to a computationally expensive problem. Furthermore, (as is discussed in Section 3.4 of
themainpaper)thestrategyforselectionofevaluationpointsisimportantforretrainingofthesurrogate
functions in subsequent algorithm iterations, and can have a significant impact on the search dynamics
ofthealgorithm. Majordecisionspertainingtodeductionofanappropriateselectionstrategyincludethe
rule(s)forselectionofevaluationpoint(s)andthenumberofpointstobeselectedforexpensiveevaluation
ineachalgorithmiteration.
Section 3.4 of the main paper proposes five selection rules which are employed to choose points for
expensiveevaluationsineachiterationofGOMORS.InSection3.4ofthemainpaperitisstatedthatGO-
MORS chooses one point for expensive evaluation in each algorithm iteration from four of the five rules
stated(Rules1to4),whileonepointforexpensiveevaluationischosenfromRule0withaprobabilityof0.1.
Henceeither4or5pointsareselectedforexpensiveevaluationsineachiterationofGOMORS.Inorderto
deduce the above-mentioned strategy for selection of evaluation points, we performed various computer
experimentswithalternateevaluationpointselectionstrategies. Atotalofsixalternateselectionstrategies
werecomparedinouranalysis. ThesixselectionstrategiesarediscussedinSectionsD.1-D.6. Theexperi-
10