Guiding Cost-Based Anytime Planning with Landmarks

52 Pages·2011·0.79 MB·English

JournalofArtificialIntelligenceResearch39(2010)127–177 Submitted11/09;published09/10
TheLAMAPlanner:
GuidingCost-BasedAnytimePlanningwithLandmarks
SilviaRichter [email protected]
IIIS,GriffithUniversity,Australia
andNICTAQRL,Australia
MatthiasWestphal [email protected]
Albert-Ludwigs-Universita¨tFreiburg
Institutfu¨rInformatik
Freiburg,Germany
Abstract
LAMAisaclassicalplanningsystembasedonheuristicforwardsearch. Itscorefeatureis
theuseofapseudo-heuristicderivedfromlandmarks, propositionalformulasthatmustbetrue
ineverysolutionofaplanningtask. LAMAbuildsontheFastDownwardplanningsystem,using
finite-domainratherthanbinarystatevariablesandmulti-heuristicsearch.Thelatterisemployedto
combinethelandmarkheuristicwithavariantofthewell-knownFFheuristic. Bothheuristicsare
cost-sensitive,focusingonhigh-qualitysolutionsinthecasewhereactionshavenon-uniformcost.
AweightedA∗searchisusedwithiterativelydecreasingweights,sothattheplannercontinuesto
searchforplansofbetterqualityuntilthesearchisterminated.
LAMAshowedbestperformanceamongallplannersinthesequentialsatisficingtrackofthe
InternationalPlanningCompetition2008. Inthispaperwepresentthesystemindetailandinves-
tigatewhichfeaturesofLAMAarecrucialforitsperformance. Wepresentindividualresultsfor
someofthedomainsusedatthecompetition,demonstratinggoodandbadcasesforthetechniques
implementedinLAMA.Overall,wefindthatusinglandmarksimprovesperformance,whereasthe
incorporationofactioncostsintotheheuristicestimatorsprovesnottobebeneficial.Weshowthat
insomedomainsasearchthatignorescostsolvesfarmoreproblems,raisingthequestionofhow
todealwithactioncostsmoreeffectivelyinthefuture. TheiteratedweightedA∗ searchgreatly
improvesresults,andshowssynergyeffectswiththeuseoflandmarks.
1. Introduction
Inthelastdecade,heuristicsearchhasbecomethedominantapproachtodomain-independentsat-
isficingplanning. StartingwiththeadditiveheuristicbyBonetandGeffner(2001), implemented
intheHSPplanningsystem, muchresearchhasbeenconductedinsearchofheuristicestimators
thatareefficienttocalculateyetpowerfulinguidingthesearchtowardsagoalstate. TheFFplan-
ningsystembyHoffmannandNebel(2001),usingaheuristicestimatorbasedonrelaxedplanning
graphs, brokegroundbyshowingbestperformanceamongallfullyautomatedsystemsattheIn-
ternationalPlanningCompetitionin2000, andcontinuestobestateofthearttoday. Eversince,
heuristic-searchapproacheshaveplayedaprominentroleintheclassicalorsequentialsatisficing
tracksofthebiennialcompetition,withFastDownward(Helmert,2006)winningin2004andSG-
Plan(Chen,Wah,&Hsu,2006)placingfirstin2006.
TheLAMAplanningsystemistheyoungestmemberinthisline,winningthesequentialsatis-
ficingtrackattheInternationalPlanningCompetition(IPC)in2008.LAMAisaclassicalplanning
(cid:3)c2010AIAccessFoundation.Allrightsreserved.
127
Richter&Westphal
systembasedonheuristicsearch. ItfollowsinthefootstepsofHSP,FF,andFastDownwardand
usestheirearlierworkinmanyrespects. Inparticular,itbuildsonFastDownwardbyextendingit
inthreemajorways:
1. Landmarks.InLAMA,FastDownward’scausalgraphheuristicisreplacedwithavariantof
theFFheuristic(Hoffmann&Nebel,2001)andheuristicestimatesderivedfromlandmarks.
Landmarksarepropositionalformulasthathavetobecometrueatsomepointineveryplan
for the task at hand (Porteous, Sebastia, & Hoffmann, 2001). LAMA uses landmarks to
directsearchtowardsthosestateswheremanylandmarkshavealreadybeenachieved. Via
preferredoperators,landmarksarealsousedasanadditionalsourceofsearchcontrolwhich
complementstheheuristicestimates. Inrecentwork,wehaveshownthisuseoflandmarks
inadditiontotheFFheuristictoimproveperformance,byleadingtomoreproblemsbeing
solvedandshortersolutionpaths(Richter,Helmert,&Westphal,2008).
2. Actioncosts. Boththelandmarkheuristicweproposedearlier(Richteretal.,2008)andthe
FFheuristichavebeenadaptedtouseactioncosts.However,LAMAdoesnotfocuspurelyon
thecost-to-go,i.e.,theestimatedcostofreachingthegoalfromagivensearchnode. There
isadangerthatacost-sensitiveplannermayconcentratetoomuchonfindingacheapplan,
attheexpenseoffindingaplanatallwithinagiventimelimit. LAMAweighstheestimated
cost-to-go(asameasureofplanquality)againsttheestimatedgoaldistance(asameasureof
remainingsearcheffort)bycombiningthevaluesforthetwoestimates.
3. Anytimesearch. LAMAcontinuestosearchforbettersolutionsuntilithasexhaustedthe
searchspaceorisinterrupted.Afterfindinganinitialsolutionwithagreedybest-firstsearch,
itconductsaseriesofweightedA∗ searcheswithdecreasingweights, restartingthesearch
eachtimefromtheinitialstatewhenanimprovedsolutionisfound.Inrecentwork,wehave
shownthisapproachtobeveryefficientonplanningbenchmarkscomparedtootheranytime
methods(Richter,Thayer,&Ruml,2010).
At the International Planning Competition 2008, LAMA outperformed its competitors by a
substantialmargin. Thisresultwasnotexpectedbyitsauthors,astheirpreviousworkconcerning
LAMA’s putative core feature, the landmark heuristic (Richter et al., 2008), showed some, but
nottremendousimprovementoverthebaseconfigurationwithoutlandmarks. Thispaperaimsto
provideareferencedescriptionofLAMAaswellasanextensiveevaluationofitsperformancein
thecompetition.
• Detailed description of LAMA. We present all distinguishing components of the planner
indetail,describinghowlandmarksaregeneratedandusedinLAMA,howactioncostsare
incorporated into the heuristic estimators and how the anytime search proceeds. Some as-
pects of LAMA have been presented in previous publications (Richter et al., 2008, 2010;
Helmert, 2006). However, aspectsthathavenotbeenadequatelycoveredinthosepublica-
tions, inparticulartheprocedureforfindinglandmarks, aredescribedhereindetail. Other
relevantaspectsdescribedinpreviouswork,likethelandmarkheuristic,aresummarisedfor
theconvenienceofthereader.Ouraimisthatthispaper,togetherwithpreviousones,forma
comprehensivepictureoftheLAMAsystem.
• ExperimentalevaluationofLAMA.Buildingonthis,weconductanexperimentalevalua-
tionfocusingontheaspectsthatdifferentiateLAMAfrompredecessorsystemslikeFFand
128
TheLAMAPlanner:GuidingCost-BasedAnytimePlanningwithLandmarks
FastDownward.Wedonotrepeatcomparisonspublishedinearlierwork,likethecomparison
betweenLAMA’sanytimemethodandotheranytimealgorithms(Richteretal.,2010),orthe
comparisonofLAMA’smethodsforhandlinglandmarkstoalternativelandmarkapproaches
(Richter et al., 2008). Instead, we aim to elicit how much the performance of the LAMA
systemasawholeisenhancedbyeachofthethreedistinguishingfeaturesdescribedabove
(landmarks, actioncostsandanytimesearch). Toanswerthisquestion, wecontrastseveral
variationsofourplannerusingvarioussubsetsofthesefeatures.
Wefindthatusingcost-sensitiveheuristicsdidnotpayoffontheIPC2008benchmarktasks.
Our results show that the cost-sensitive variant of the FF heuristic used in LAMA performs sig-
nificantly worse than the traditional unit-cost version of the same heuristic. Similarly, all other
cost-sensitiveplannersinthecompetitionfaredworsethanthebaselineplannerFFthatignoredac-
tioncosts,demonstratingthatcost-basedplanningpresentsaconsiderablechallenge. Whilewedo
notconductafullanalysisofthereasonsforthis,weshowcasetheproblemsofthecost-sensitiveFF
heuristicinsomeexampledomainsandprovideinformedhypothesesfortheencounteredeffects.
Landmarksprovetobeparticularlyhelpfulinthiscontext. Whileintheunit-costcaselandmarks
only lead to a moderate increase in performance, in the case of planning with action costs they
substantially improve coverage (the number of problems solved), thus effectively mitigating the
problemsofthecost-sensitiveFFheuristicinLAMA. Theanytimesearchsignificantlyimproves
thequalityofsolutionsthroughoutandevenactsinsynergywithlandmarksinonedomain.
2. Preliminaries
Weuseaplanningformalismwithstatevariablesoffinite(ratherthanbinary)range,similartothe
oneemployedbyHelmert(2009). ItisbasedontheSAS+ planningmodel(Ba¨ckstro¨m&Nebel,
1995),butextendsitwithconditionaleffects. WhileLAMAalsohandlesaxiomsinthesameway
asFastDownward(Helmert,2006),wedonotformaliseaxiomshere,sincetheyarenotimportant
forourpurposes.
Definition1. Planningtasksinfinite-domainrepresentation(FDRtasks)
Aplanningtaskinfinite-domainrepresentation(FDRtask)isgivenbya5-tuple(cid:4)V,s0,s(cid:2),O,C(cid:5)
withthefollowingcomponents:
• Visafinitesetofstatevariables,eachwithanassociatedfinitedomainD .
v
Afactisapair(cid:4)v,d(cid:5)(alsowrittenv (cid:6)→ d), wherev ∈ V andd ∈ D . Apartialvariable
v
assignment sisasetoffacts, eachwithadifferentvariable. (Weusesetnotationsuchas
(cid:4)v,d(cid:5) ∈ s and function notation such as s(v) = d interchangeably.) A state is a variable
assignmentdefinedonallvariablesV.
• s isastatecalledtheinitialstate.
0
• s(cid:2)isapartialvariableassignmentcalledthegoal.
• Oisafinitesetofoperators. Anoperator(cid:4)pre,eff(cid:5)consistsofapartialvariableassignment
pre called its precondition, and a finite set of effects eff. Effects are triplets (cid:4)cond,v,d(cid:5),
wherecondisa(possiblyempty)partialvariableassignmentcalledtheeffectcondition,vis
theaffectedvariableandd∈D iscalledthenewvalueforv.
v
129
Richter&Westphal
• C:O→N+isaninteger-valuednon-negativeactioncostfunction.
0
Anoperatoro = (cid:4)pre,eff(cid:5) ∈ Oisapplicableinastate sifpre ⊆ s,anditseffectsareconsistent,
i.e.,thereisastates(cid:10)suchthats(cid:10)(v)=dforall(cid:4)cond,v,d(cid:5)∈effwherecond⊆ s,ands(cid:10)(v)= s(v)
otherwise. Inthiscase,wesaythattheoperatorocanbeappliedtosresultinginthestates(cid:10)and
writes[o]fors(cid:10).
Foroperatorsequencesπ=(cid:4)o ,...,o (cid:5),wewrites[π]fors[o ]...[o ](onlydefinedifeachoper-
1 n 1 n
atorisapplicableintherespectivestate). Theoperatorseq(cid:2)uenceπiscalledaplanif s(cid:2) ⊆ s0[π].
Thecostofπisthesumoftheactioncostsofitsoperators, n C(o).
i=1 i
Eachstatevariablevofaplanningtaskinfinite-domainrepresentationhasanassociateddirected
graph called the domain transition graph, which captures the ways in which the value of v may
change (Jonsson & Ba¨ckstro¨m, 1998; Helmert, 2006). The vertex set of this graph is D , and it
v
containsanarcbetweentwonodesdandd(cid:10)ifthereexistsanoperatorthatcanchangethevalueof
vfromdtod(cid:10).Formally:
Definition2. Domaintransitiongraph
Thedomaintransitiongraph(DTG)ofastatevariablev ∈ VofanFDRtask(cid:4)V,s0,s(cid:2),O,C(cid:5)is
thedigraph(cid:4)D ,A(cid:5)whichincludesanarc(cid:4)d,d(cid:10)(cid:5)iffd(cid:2)d(cid:10),thereisanoperator(cid:4)pre,eff(cid:5)∈Owith
v
(cid:4)cond,v,d(cid:10)(cid:5)∈eff,andfortheunionofconditionspre∪conditholdsthateitheritcontainsv=dor
itdoesnotcontainv=d˜foranyd˜∈D .
v
3. SystemArchitecture
LAMAbuildsontheFastDownwardsystem(Helmert,2006),inheritingtheoverallstructureand
large parts of the functionality from that planner. Like Fast Downward, LAMA accepts input in
thePDDL2.2Level1format(Fox&Long,2003;Edelkamp&Hoffmann,2004),includingADL
conditions and effects and derived predicates (axioms). Furthermore, LAMA has been extended
tohandletheactioncostsintroducedforIPC2008(Helmert, Do,&Refanidis, 2008). LikeFast
Downward,LAMAconsistsofthreeseparatecomponents:
• Thetranslationmodule
• Theknowledgecompilationmodule
• Thesearchmodule
These components are implemented as separate programs that are invoked in sequence. In the
following, weprovideabriefdescriptionofthetranslationandknowledgecompilationmodules.
ThemainchangesinLAMA,comparedtoFastDownward,areimplementedinthesearchmodule,
whichwediscussindetail.
3.1 Translation
Thetranslationmodule,shorttranslator,transformsthePDDLinputintoaplanningtaskinfinite-
domainrepresentationasspecifiedinDefinition1. Themaincomponentsofthetranslatorarean
efficient grounding algorithm for instantiating schematic operators and axioms, and an invariant
130
TheLAMAPlanner:GuidingCost-BasedAnytimePlanningwithLandmarks
synthesisalgorithmfordetermininggroupsofmutuallyexclusivefacts.Suchfactgroupsareconse-
quentlyreplacedbyasinglestatevariable,encodingwhichfact(ifany)fromthegroupissatisfied
inagivenworldstate.DetailsonthiscomponentcanbefoundinarecentarticlebyHelmert(2009).
The groups of mutually exclusive facts (mutexes) found during translation are later used to
determine orderings between landmarks. For this reason, LAMA does not use the finite-domain
representationsofferedatIPC2008(objectfluents),butinsteadperformsitsowntranslationfrom
binarytofinite-domainvariables. Whilenotallmutexescomputedbythetranslationmoduleare
neededforthenewencodingoftheplanningtask,themodulehasbeenextendedinLAMAtoretain
allfoundmutexesfortheirlaterusewithlandmarks.
Further changes we made, compared to the translation module described by Helmert, were
to add the capability of handling action costs, implement an extension concerning the parsing of
complex operator effect formulas, and limit the runtime of the invariant synthesis algorithm. As
invariantsynthesismaybetimecritical,inparticularonlarge(grounded)PDDLinput,welimitthe
maximumnumberofconsideredmutexcandidatesinthealgorithm,andabortit,ifnecessary,after
fiveminutes. Notethatfindingfewornomutexesdoesnotchangethewaythetranslationmodule
works; if no mutexes are found, the resulting encoding of the planning task contains simply the
same(binary-domain)statevariablesasthePDDLinput. Whenanalysingthecompetitionresults,
wefoundthatthesynthesisalgorithmhadabortedonlyinsomeofthetasksofonedomain(Cyber
Security).
3.2 KnowledgeCompilation
Usingthefinite-domainrepresentationgeneratedbythetranslator,theknowledgecompilationmod-
uleisresponsibleforbuildinganumberofdatastructuresthatplayacentralroleinthesubsequent
landmarkgenerationandsearch. Firstly,domaintransitiongraphs(seeDefinition2)areproduced
whichencodethewaysinwhicheachstatevariablemaychangeitsvaluethroughoperatorapplica-
tionsandaxioms.Furthermore,datastructuresareconstructedforefficientlydeterminingthesetof
applicableoperatorsinastateandforevaluatingthevaluesofderivedstatevariables. Wereferto
Helmert(2006)formoredetailontheknowledgecompilationcomponent,whichLAMAinherits
unchangedfromFastDownward.
3.3 Search
Thesearchmoduleisresponsiblefortheactualplanning. Twoalgorithmsforheuristicsearchare
implemented in LAMA: (a) a greedy best-first search, aimed at finding a solution as quickly as
possible,and(b)aweightedA∗ searchthatallowsbalancingspeedagainstsolutionquality. Both
algorithmsarevariationsofthestandardtextbookmethods,usingopenandclosedlists.Thegreedy
best-firstsearchalwaysexpandsastatewithminimalheuristicvaluehamongallopenstatesand
neverexpandsastatemorethanonce. Inordertoencouragecost-efficientplanswithoutincurring
muchoverhead,itbreakstiesbetweenequallypromisingstatesbypreferringthosestatesthatare
reachedbycheaperoperators,i.e.,takingintoaccountthelastoperatoronthepathtotheconsidered
stateinthesearchspace.(Thecostoftheentirepathcouldonlybeusedattheexpenseofincreased
time or space requirements, so that we do not consider this.) Weighted A∗ search (Pohl, 1970)
associatescostswithstatesandexpandsastatewithminimal f(cid:10)-value,where f(cid:10) = w·h+g,the
weightwisaninteger≥ 1,andgisthebestknowncostofreachingtheconsideredstatefromthe
131
Richter&Westphal
initialstate.Incontrasttothegreedysearch,weightedA∗searchre-expandsstateswheneveritfinds
cheaperpathstothem.
Inaddition,bothsearchalgorithmsusethreetypesofsearchenhancementsinheritedfromFast
Downward (Helmert, 2006; Richter & Helmert, 2009). Firstly, multiple heuristics are employed
withinamulti-queueapproachtoguidethesearch. Secondly,preferredoperators—similartothe
helpfulactionsinFF—allowgivingprecedencetooperatorsthataredeemedmorehelpfulthan
others in a state. Thirdly, deferred heuristic evaluation mitigates the impact of large branching
factors assuming that heuristic estimates are fairly accurate. In the following, we discuss these
techniquesandtheresultingalgorithmsinmoredetailandgivepseudocodeforthegreedybest-first
search. TheweightedA∗ searchisverysimilar, sowepointoutthedifferencesbetweenthetwo
algorithmsalongtheway.
Multi-queueheuristicsearch. LAMAusestwoheuristicfunctionstoguideitssearch:thename-
givinglandmarkheuristic(seeSection5),andavariantofthewell-knownFFheuristic(seeSec-
tion6). Thetwoheuristicsareusedwithseparatequeues,thusexploitingstrengthsoftheutilised
heuristics in an orthogonal way (Helmert, 2006; Ro¨ger & Helmert, 2010). To this end, separate
openlistsaremaintainedforeachofthetwoheuristics. Statesarealwaysevaluatedwithrespectto
bothheuristics,andtheirsuccessorsareaddedtoallopenlists(ineachcasewiththevaluecorre-
spondingtotheheuristicofthatopenlist).Whenchoosingwhichstatetoevaluateandexpandnext,
thesearchalgorithmalternatesbetweenthedifferentqueuesbasedonnumericalprioritiesassigned
toeachqueue.Theseprioritiesarediscussedlater.
Deferred heuristic evaluation. The use of deferred heuristic evaluation means that states are
notheuristicallyevaluatedupongeneration,butuponexpansion,i.e.,whenstatesaregeneratedin
greedybest-firstsearch,theyareputintotheopenlistnotwiththeirownheuristicvalue,butwith
thatoftheirparent. Onlyafterbeingremovedfromtheopenlistaretheyevaluatedheuristically,
andtheirheuristicestimateisinturnusedfortheirsuccessors. Theuseofdeferredevaluationin
weighted A∗ search is analogous, using f(cid:10) instead of h as the sorting criterion of the open lists.
Ifmanymorestatesaregeneratedthanexpanded,deferredevaluationleadstoasubstantialreduc-
tioninthenumberofheuristicestimatescomputed. However,deferredevaluationincursalossof
heuristicaccuracy,asthesearchcannolongeruseh-valuesor f(cid:10)-valuestodifferentiatebetweenthe
successorsofastate(allsuccessorsareassociatedwiththeparent’svalueintheopenlist).Preferred
operatorsareveryhelpfulinthiscontextastheyprovideanalternativewaytodeterminepromising
successors.
Preferredoperators. Operatorsthataredeemedparticularlyusefulinagivenstatearemarked
as preferred. They are computed by the heuristic estimators along with the heuristic value of a
state(seeSections6and5). Tousepreferredoperators,inthegreedybest-firstsearchaswellas
intheweightedA∗ search,theplannermaintainsanadditionalpreferred-operatorqueueforeach
heuristic. When a state is evaluated and expanded, those successor states that are reached via a
preferredoperator(thepreferredstates)areputintothepreferred-operatorqueues, inadditionto
beingputintotheregularqueueslikethenon-preferredstates. (Analogouslytoregularstates,any
statepreferredbyatleastoneheuristicisaddedtoallpreferred-operatorqueues. Thisallowsfor
cross-fertilisation through information exchange between the different heuristics.) States in the
preferred-operatorqueuesareevaluatedearlieronaverage, astheyformpartofmorequeuesand
haveahigherchanceofbeingselectedatanypointintimethanthenon-preferredstates.Inaddition,
132
TheLAMAPlanner:GuidingCost-BasedAnytimePlanningwithLandmarks
LAMA(liketheIPC2004versionofFastDownward)givesevenhigherprecedencetopreferred
successors via the following mechanism. The planner keeps a priority counter for each queue,
initialised to 0. At each iteration, the next state is removed from the queue that has the highest
priority. Wheneverastateisremovedfromaqueue,thepriorityofthatqueueisdecreasedby1. If
theprioritiesarenotchangedoutsideofthisroutine,thismethodwillalternatebetweenallqueues,
thusexpandingstatesfrompreferredqueuesandregularqueuesequallyoften. Toincreasetheuse
ofpreferredoperators,LAMAincreasestheprioritiesofthepreferred-operatorqueuesbyalarge
numberboostofvalue1000wheneverprogressismade,i.e.,wheneverastateisdiscoveredthathas
abetterheuristicestimatethanpreviouslyexpandedstates. Subsequently,thenext1000stateswill
beremovedfrompreferred-operatorqueues. Ifanotherimprovingstateisfoundwithinthe1000
states,theboostsaccumulateand,accordingly,ittakeslongeruntilstatesfromtheregularqueues
areexpandedagain.
Alternative methods for using preferred operators include the one employed in the YAHSP
system(Vidal,2004),wherepreferredoperatorsarealwaysusedovernon-preferredones. Bycon-
trast,ourschemedoesnotnecessarilyemptythepreferredqueuesbeforeswitchingbacktoregular
queues. IntheFFplanner(Hoffmann&Nebel,2001),theemphasisonpreferredoperatorsiseven
strongerthaninYAHPS:thesearchinFFisrestrictedtopreferredoperatorsuntileitheragoalis
foundortherestrictedsearchspacehasbeenexhausted(inwhichcaseanewsearchisstartedwith-
outpreferredoperators). Comparedtotheseapproaches,themethodforusingpreferredoperators
inLAMA,inconjunctionwithdeferredheuristicevaluation,hasbeenshowntoresultinsubstantial
performanceimprovementanddeliverbestresultsintheclassicalsettingofoperatorswithunitcosts
(Richter&Helmert,2009). Thechoiceof1000astheboostvalueisnotcriticalhere,aswefound
variousvaluesbetween100and50000togivesimilarlygoodresults. Onlyoutsidethisrangedoes
performancedropnoticeably.
Note that when using action costs, the use of preferred operators may be even more helpful
thanintheclassicalsetting. Forexample,ifalloperatorshaveacostof0,aheuristicusingpure
costestimatesmightassignthesameheuristicvalueof0toallstatesinthestatespace,givingno
guidancetosearchatall. Preferredoperators, however, stillprovidethesameheuristicguidance
inthiscaseasinthecasewithunitactioncosts. Whilethisisanextremeexample,similarcases
appearinpractice,e.g.intheIPC2008domainOpenstacks,wherealloperatorsexceptfortheone
openinganewstackhaveanassociatedcostof0.
Pseudo code. Algorithm 1 shows pseudo code for the greedy best-first search. The main loop
(lines 25–36)runsuntileitheragoalhasbeenfound(lines 27–29)orthesearchspacehasbeen
exhausted(lines32–33). Theclosedlistcontainsallseenstatesandalsokeepstrackofthelinks
between states and their parents, so that a plan can be efficiently extracted once a goal state has
beenfound(line28). Ineachiterationoftheloop, thesearchaddsthecurrentstate(initiallythe
start state) to the closed list and processes it (lines 30–31), unless the state has been processed
before,inwhichcaseitisignored(line26).Bycontrast,weightedA∗searchprocessesstatesagain
whenever they are reached via a path with lower cost than before, and updates their parent links
intheclosedlistaccordingly. Thenthesearchselectsthenextopenlisttobeused(theonewith
highest priority, line 34), decreases its priority and extracts the next state to be processed (lines
35–36). Theprocessingofastateincludescalculatingitsheuristicvaluesandpreferredoperators
withbothheuristics(lines3–4),expandingit,andinsertingthesuccessorsintotheappropriateopen
133
Richter&Westphal
Globalvariables:
Π=(cid:4)V,s0,s(cid:2),O,C(cid:5) (cid:4)Planningtasktosolve
reg , pref , reg , pref (cid:4)Regularandpreferredopenlistsforeachheuristic
FF FF LM LM
best seen value (cid:4)Bestheuristicvalueseensofarforeachheuristic
priority (cid:4)Numericalpriorityforeachqueue
1: functionexpand state(s)
2: progress←False
3: forh∈{FF,LM}do
4: h(s),preferred ops(h,s)←heuristicvalueofsandpreferredoperatorsgivenh
5: ifh(s)<best seen value[h]then
6: progress←True
7: best seen value[h]←h(s)
8: if progressthen (cid:4)Boostpreferred-operatorqueues
9: priority[prefFF]←priority[prefFF]+1000
10: priority[prefLM]←priority[prefLM]+1000
11: succesor states←{s[o]|o∈Oandoapplicableins}
12: fors(cid:10)∈succesor statesdo
13: forh∈{FF,LM}do
14: adds(cid:10)toqueuereghwithvalueh(s) (cid:4)Deferredevaluation
15: if s(cid:10)reachedbyoperatoro∈preferred ops(h,s)then
16: adds(cid:10)toqueueprefFFwithvalueFF(s),andtoqueueprefLMwithvalueLM(s)
17: functiongreedy bfs lama
18: closed list←∅
19: forh∈{FF,LM}do (cid:4)InitializeFFandlandmarkheuristics
20: best seen value[h]←∞
21: forl∈{reg,pref}do (cid:4)Regularandpreferredopenlistsforeachheuristic
22: lh←∅
23: priority[lh]←0
24: current state←s0
25: loop
26: ifcurrent state(cid:3)closed listthen
27: if s=s(cid:2)then
28: extractplanπbytracingcurrentstatebacktoinitialstateinclosedlist
29: returnπ
30: closed list←closed list∪{current state}
31: expand state(current state)
32: ifallqueuesareemptythen
33: returnfailure (cid:4)Noplanexists
34: q←non-emptyqueuewithhighestpriority
35: priority[q]←priority[q]−1
36: current state←pop state(q) (cid:4)Getlowest-valuedstatefromqueueq
Algorithm1:Thegreedybest-first-searchwithsearchenhancementsusedinLAMA.
134
TheLAMAPlanner:GuidingCost-BasedAnytimePlanningwithLandmarks
lists(lines11–16).Ifitisdeterminedthatanewbeststatehasbeenfound(lines5-7),thepreferred-
operatorqueuesareboostedby1000(lines8-10).
3.3.1 RestartingAnytimeSearch
LAMAwasdevelopedfortheInternationalPlanningCompetition2008andistailoredtothecon-
ditionsofthiscompetitioninseveralways. Indetail,thoseconditionswereasfollows. Whilein
previouscompetitionscoverage,planqualityandruntimewereallusedtovaryingdegreesinorder
todeterminetheeffectivenessofaclassicalplanningsystem,IPC2008introducedanewintegrated
performance criterion. Each operator in the PDDL input had an associated non-negative integer
actioncost, andtheaimwastofindaplanoflowest-possibletotalcostwithinagiventimelimit
of 30 minutes per task. Given that a planner solves a task at all within the time limit, this new
performancemeasuredependsonlyonplanquality,notonruntime,andthussuggestsguidingthe
searchtowardsacheapestgoalratherthanaclosestgoalaswellasusingalloftheavailabletimeto
findthebestplanpossible.
Guidingthesearchtowardscheapgoalsmaybeachievedintwoways,bothofwhichLAMA
implements: firstly,theheuristicsshouldestimatethecost-to-go,i.e.,thecostofreachingthegoal
fromagivenstate,ratherthanthedistance-to-go,i.e.,thenumberofoperatorsrequiredtoreachthe
goal.BoththelandmarkheuristicandtheFFheuristicemployedinLAMAarethereforecapableof
usingactioncosts.Secondly,thesearchalgorithmshouldnotonlytakethecost-to-gofromagiven
stateintoaccount,butalsothecostnecessaryforreachingthatstate. Thisisthecaseforweighted
A∗searchasusedinLAMA. Tomakethemostoftheavailabletime,LAMAemploysananytime
approach: itfirstrunsagreedybest-firstsearch,aimedatfindingasolutionasquicklyaspossible.
Onceaplanisfound,itsearchesforprogressivelybettersolutionsbyrunningaseriesofweighted
A∗ searcheswithdecreasingweight. Thecostofthebestknownsolutionisusedforpruningthe
search,whiledecreasingtheweightovertimemakesthesearchprogressivelylessgreedy,trading
speedforsolutionquality.
SeveralanytimealgorithmsbasedonweightedA∗havebeenproposed(Hansen&Zhou,2007;
Likhachev, Ferguson, Gordon, Stentz, & Thrun, 2008). Their underlying idea is to continue the
weightedA∗ searchpastthefirstsolution,possiblyadjustingsearchparametersliketheweightor
pruningbound,andthusprogressivelyfindbettersolutions. TheanytimeapproachusedinLAMA
differs from these existing algorithms in that we do not continue the weighted A∗ search once it
finds a solution. Instead, we start a new weighted A∗ search, i.e., we discard the open lists of
theprevioussearchandre-startfromtheinitialstate.Whileresultinginsomeduplicateeffort,these
restartscanhelpovercomebaddecisionsmadebytheearly(comparativelygreedy)searchiterations
withhighweight(Richteretal.,2010).Thiscanbeexplainedasfollows:Afterfindingagoalstate
s ,theopenlistswillusuallycontainmanystatesthatareclosetos inthesearchspace,becausethe
g g
ancestorsofs havebeenexpanded;furthermore,thosestatesarelikelytohavelowheuristicvalues
g
becauseoftheirproximityto s . Hence, ifthesearchiscontinued(evenafterupdatingtheopen
g
listswithlowerweights),itislikelytoexpandmostofthestatesarounds beforeconsideringstates
g
thatareclosetotheinitialstate. Thiscanbecritical, asitmeansthatthesearchisconcentrating
onimprovingtheendofthecurrentplan,asopposedtoitsbeginning. Abadbeginningofaplan,
however,mayhaveseverenegativeinfluenceonitsquality,asitmaybeimpossibletoimprovethe
qualityoftheplansubstantiallywithoutchangingitsearlyoperators.
135
Richter&Westphal
3.8 3.8 3.8 4.0
s
10.6 9.6 8.6 9.0
3.4 3.4
9.8 8.8
2.6 2.6 2.6
8.2 8.2 8.2
1.8 1.8 1.8
7.6 7.6 7.6
1.0 1.0 1.0
g2
7.0 7.0 7.0
1.0 1.0
g1
8.0 8.0
(a)initialsearch,w=2
3.8 3.8 X s 4.0
8.7 7.7 7.0
3.4
X
8.1
2.6 2.6 2.6 2.6 1.9 2.0 2.0
X
8.9 7.9 6.9 6.9 6.85 8.0 9.0
2.6 1.8 1.8 1.8 1.9 1.0 1.0
X
8.9 6.7 6.7 6.7 6.85 6.5 7.5
2.6 1.8 1.0 1.0 1.9 1.0
X g2
8.9 7.7 6.5 6.5 7.85 6.5
1.8 1.0 1.0 1.9
g1
8.7 7.5 7.5 8.85
(b)continuedsearch,w=1.5
3.8 3.8 4.0 4.0
s
7.7 6.7 7.0 8.0
3.4 3.0 3.0
7.1 6.5 7.5
2.0 2.0 2.0
6.0 6.0 6.0
1.0 1.0 1.0
5.5 5.5 5.5
1.0 1.0
g2
6.5 6.5
g1
(c)restartedsearch,w=1.5
Figure1: Theeffectoflow-hbias. Forallgridstatesgeneratedbythesearch,h-valuesareshown
above f(cid:10)-values. (a) Initial weighted A∗ search finds a solution of cost 6. (b) Continued search
expandsmanystatesaroundthepreviousOpenlist(greycells),findinganothersub-optimalsolution
ofcost6.(c)Restartedsearchquicklyfindstheoptimalsolutionofcost5.
136