Table Of ContentAstron.Nachr./AN000,No.00,1–5(0000)/DOIpleasesetDOI!
An autonomous adaptive scheduling agent for period searching
EricS.Saunders1,2,⋆,TimNaylor1,andAlasdairAllan1
1 SchoolofPhysics,UniversityofExeter,StockerRoad,EX44QL
2 LasCumbresObservatory,6740CortonaDr.,Suite102,SantaBarbara,CA93117
Received;accepted;publishedonline
8 Wedescribethedesignandimplementationofanautonomousadaptivesoftwareagentthataddressesthepracticalproblem
0 ofobservingundersampled,periodic,time-varyingphenomenausinganetworkofHTN-compliantrobotictelescopes.The
0 algorithmgoverningthebehaviouroftheagentusesanoptimalgeometricsamplingtechniquetocovertheperiodrange
2 ofinterest,butadditionallyimplementsproactivebehaviourthatmaximisestheoptimalityofthedatasetinthefaceofan
n uncertainandchangingoperatingenvironment.
a
J (cid:13)c 0000WILEY-VCHVerlagGmbH&Co.KGaA,Weinheim
4
2
1 Introduction range of periods among cluster members is large, ranging
] fromafewhourstomanydays.Additionally,aliasing,par-
h
The eSTAR project (Allanetal., 2007, 2004) is an agent- ticularly as a consequence of diurnal sampling, is usually
p
based software system that aims to establish an intel- a problem for data obtained in the “classical paradigm”,
-
o ligent robotic telescope network for efficient and auto- where an on-site observer performs observations using a
r
mated observing. The system comprises user agents that singletelescope.Aroboticnetworkspreadacrosslongitude
t
s
run an observing programme on behalf of an astronomer, canbreakthisaliasingpattern,butdatasetsaretypicallyun-
a
[ and embedded agents that provide the interface to ob- dersampled with respect to shorter periods in the range of
servational resources such as telescopes. One role of eS- interest. Given a limited number of observations,the opti-
1
TAR is toprovidethe softwareinfrastructureforRobonet- malgeometricsamplingtechniquedeterminesthebesttime
v
6 1.0 (e.g. Bodeetal., 2004; Mottram, 2006), a network of to make individual observations in order to achieve simi-
4 three autonomous 2-m robotic telescopes strategically lo- larsensitivitytoperiodsacrossthefullrangeofinterest.Of
8 cated across the globe. eSTAR implementsthe HTN1 pro- particularimportanceisthepropertythataslongastheset
3
tocol (Allanetal., 2006) using the XML dialect RTML ofgapsbetweenobservationsremainsunchanged,observa-
.
1 (Hessman, 2006; Pennypackeretal., 2002). The Robonet- tionsmaybearbitrarilyreordered(Saundersetal.,2006b).
0 1.0 network thus providesa full reference implementation Thispropertyisfullyexploitedbytheadaptivealgorithm.
8
ofanHTN-compliantenvironment. Unfortunately, observations are not guaranteed. In the
0
: Thispaperdescribesthe designand implementationof dispatch model of telescope scheduling, users request ob-
v auseragentthatappliesadaptiveschedulingtooptimisean servations but it is up to the telescope scheduler to decide
i
X observingrun.Adaptiveschedulingwasdescribedinanear- whethersuchrequestsarehonoured(Fraser,2006).Obser-
r lierpaper(Saundersetal.,2006a)as“techniques...toallow vations may still fail even when submitted to an entirely
a anautonomoussoftwareentitytocalculateanidealisedob- cooperative scheduler, due to the possibility of telescope
serving strategy, while remaining able to respond flexibly downtime or inclement weather. Thus any system seeking
toschedulingconstraintsoutsideofitsimmediatecontrol.” toreliablyimplementanoptimalgeometricsamplingtech-
In particular, the adaptive scheduling agent (ASA) imple- nique must in practice deal with such observation uncer-
ments the optimal geometric sampling strategy for period tainty.
searching described in Saundersetal. (2006b) to address
thepracticalproblemofobservingundersampled,periodic, 2 Designprinciples
time-varyingphenomena.
Anexampleofthetypeoftime-domainastronomythat Theenvironmentinwhichtheadaptiveschedulingagentop-
thisagentwasdesignedtoperformisthesurveyingofvari- eratesisinherentlyuncertain.Thisfactwasthefundamental
ability in star-forming regions. Especially in the youngest designconstraintfromwhichthedesirableattributesofthe
clusters,alargefractionoftheclustermembersareT-Tauri agentwerederived.Fourprincipalideasdrovethedesign.
starsthatcanexhibitsignificantvariabilityduetorotational – Robustness.Buildingsoftwarethatwilloperatereliably
modulation of features at the stellar surface. However the overanetworkiscomplicated.Thereforeaguidingprin-
⋆ Correspondingauthor:e-mail:[email protected] ciple was to keep the algorithm as simple as possible,
tominimisethepossibilityofunforseeninteractionsbe-
1 HeterogeneousTelescopeNetworks tweensystemcomponents.Withadistributedsystemit
(cid:13)c 0000WILEY-VCHVerlagGmbH&Co.KGaA,Weinheim
EricS.Saundersetal.:Anautonomousadaptiveschedulingagentforperiodsearching
Astron.Nachr./AN(0000)
mustbeassumedthatanyremotepartofthesystemcan Adaptive Scheduling (User) Agent
failatanytime,eithertemporarilyorpermanently.The Time Server
Observational Resource
algorithmneedstobeabletomakeareasonableattempt
tocontinueundersuchcircumstances. ASA Core User
– Cynicism.AlthoughalltheagentsintheeSTARsystem Shared Agent Node Agent TCS
Memory
areinprincipletrustworthy,inpracticeitismuchsafer
(Eval)
to assume nothing binding about events or entities ex-
Evaluator
ternal to the main code. The benevolence assumption,
namely thatagentsmay actfor the greater goodof the BPololyckmorphic STOCAP/PI Pre sqouceksett o rre qseurevsetr or server
system, at their own expense,is not valid here. This is Internal message
Network message
becausethe telescope schedulers,and byproxythe ac- Run−time evaluation
tionsoftheembeddedagents,arenotunderthecontrol Fig.1 Architectureoftheadaptiveschedulingagent.The
of eSTAR. The goal of the embedded agents is to op- multithreaded ASA core implements the adaptive evalua-
timisethescheduleinsomeinternallysatisfactoryway, tion engine, a SOAP submission mechanism for making
which may or may not coincide with the goals of the observation requests, and a TCP/IP socket listener for re-
user agent. This is most important with respect to the ceivingobservationfeedbackinformation.Date/timeinfor-
scoring information returned by the observing nodes: mation is abstracted by a time server, which can be accel-
sincethereisnopenaltytotheembeddedagentforpro- erated to allow fast simulations to be performed. Actual
viding an inaccurate score, there is no compelling rea- RTML document construction and HTN protocol negotia-
sonfortheuseragenttotrustthatvalue.Evenifthereis tionsare handledbytheuser agent,a webservicethatim-
reasontobelievethatthenodeisactingingoodfaith,we plementsaSOAPserverforsubmittingobservationsandre-
still have no a priori idea of the accuracyof the infor- ceivingasynchronousmessagesfromthe embeddedagent.
mation being supplied. Therefore external information Asynchronousmessagestriggerarun-timecodeevaluation
should be considered,butnotrelied on,and in general based on the observationtype, allowing arbitraryresponse
theagentneedstoadequatelyhandletheworstcasesce- behaviourtobedefinedfordifferentobservingprogrammes.
nario(beingsuppliedwithfalseinformation). Inthiscase,thepolymorphicblockstripspertinentinforma-
– Stability.Apooralgorithmwouldrequirechainsofspe- tion(primarilytheobservationtimestamp)fromtheincom-
cific events in order to be successful. If the future is ingmessageandpipesittothededicatedsocketlistenerin
highlyuncertain,thenanalgorithmthatisreliantonthat the ASA core. In this way the agent receives observation
futuretosucceedisriskyandfragile.Ideally,theperfor- feedbackastherunprogresses,allowingdirectoptimisation
manceofthealgorithmshoulddegradegracefullyasthe ofthefutureobservingstrategy.
environmentbecomesmorehostile,sothatitcontinues
tomakeabestefforttosucceed.Inpracticalterms,this
behaviourisofcriticalimportancetothesuccessofthis
meansthatpartialrunsneedtobeoptimal,i.e.thatwhat-
approach.
evertheagenthasachievedtodateneedstobethebest
setofobservationsthatitcouldmakeunderthecircum-
stances, since we cannotguaranteefuture observations
3 Implementation
atthetemporalpositionswespecify.
– Adaptability. Finally, the agent needs to constantly
Figure 1 shows the detailed architecture of the adaptive
evaluatethecurrentstateoftherunandthecurrentob-
scheduling agent. The core decision-making algorithm is
servingconditions,andalteritsbehaviourasnecessary.
a multi-threadedPerl process that determines when to ob-
Somesimpleexamplesillustratethispoint.Theagentis
serve, translates between abstract theoretical timestamps
aiming to acquire a set of observationsthat are spaced
and real dates and times, and monitorsthe progressof the
by both small and large gaps of particular sizes. If the
observingrun.
agenthasacquiredmanysmallgaps,howshoulditalter
Observationrequestsaremadebyremotemethodinvo-
itsbehaviourtotrytoachievelargergaps?Whatshould
cation of the user agent web service running on the same
it do in the opposite case? Indeed, how should it deal
machine.Followingthe HTN protocol(Allanetal., 2006),
with the general problem of a set of gaps that are not
theuseragentmakesscoringrequestsofeachtelescope(by
quite correct? Another plausible scenario is that of a
wayoftheembeddedagentateachsite),andthenpicksthe
telescopethatconsistentlyprovideslowscores,butstill
highestnon-zeroscore.A “request”documentisthensub-
manages to complete observations successfully. How
mitted. Under normal conditions, a “confirmation” docu-
should that be handled? The agent programming ap-
mentisreturned.Thisindicatesthattheobservationrequest
proach explicitly assumes uncertaintyand change, and
hasbeenacceptedbythetelescopeandisnowqueued.
defines a feedback cycle to interact with the environ-
ment. How to successfully implement useful adaptive Eventually, the observation request is resolved. Al-
though multiple exposure observation requests are possi-
ble, the ASA always requests single observations, as this
2
EricS.Saundersetal.:Anautonomousadaptiveschedulingagentforperiodsearching
Astron.Nachr./AN(0000)
0 1 Theoptimalitycriterion,w,isdefinedbythefollowing
Optimal spacing
simpleseriesofoperations.
Actual spacing
1. Orderthesetofoptimalgaps,fromsmallesttolargest.
Actual spacing, reordered 2. Orderthesetofobtainedgaps,fromsmallesttolargest.
Calculate offsets δ1= 0 δ2 δ3 3. For each obtained gap,find the offsetof that gap from
Sum offsets ω = Σδ = δ1+δ2+δ3
theoptimalgapatthatpositionintheset.
Fig.2 Calculatingthe optimality,w, ofa setofobserva- 4. Theoveralloptimality,wisthesumoftheseoffsets.
tions.Theoptimalandactualsetsofgapsaresorted,small-
esttolargest,andeachgapcompared.Thesumofoffsetsis Figure2illustratesthis process.Theoptimalgapset is
w. expressed in theoretical time, i.e. so that the run begins at
timestamp0andthefinalobservationoccursattimestamp1.
Theobtainedgapsarescaledtothesameunits.Ifthesetof
maximisestheamountofschedulingcontrolretainedbythe
actualtimestampsisinperfectagreementwith theoptimal
agent. Either the observation was successful, and an “ob-
set,thenthevalueoftheoptimalitymetricis0.Thereisno
servation”documentisreturned,ortheobservationdidnot
upper limit on the value since the length of the actual run
succeed,andamessageoftype“fail”isreturned.Thereturn
canexceedthatoftheoptimalsequence.
document is received by the user agent web service, and
Notethatthesimplicityofthisapproachisonlypossible
scannedforobservationtype.Theagentcomparesthetype
becauseofthereorderingpropertyoftheoptimalsequence.
to the set of types it knows how to deal with, and if there
Thisallowsthegapstobecomparedinorderofsize,regard-
isamatch,thecodeforthattypeisdynamicallyloadedand
lessoftheactualobservedordering.
executed. This plug-in architecture allows custom actions
to occur for differenttypes of observation,allowing a sin-
gleinstanceoftheuseragentwebservicetohandleRTML 4.2 Choosingthenextobservation
marshallingformanydistinctobservingprogrammes.
The optimality function allows future observation times-
ThealgorithmicblockfortheASA extractsthecritical
tamps to be compared. The agent seeks to minimise the
piecesofinformationfromtheincomingRTMLdocument.
value of the optimality,which will increase monotonically
These include the requested start time, and if the observa-
asmoreobservationsaccrue.Thequestionis,giventheex-
tionwassuccessful,theactualstarttimeoftheobservation,
istingsetofobservations,whatnewtimestampwillachieve
gleanedfromtheFITSheaderincludedinthe“observation”
this?Sincetime iscontinuous,inprinciplethereareanin-
document.Anyerrormessagesarealsopickeduphere.This
finite numberof possible futuresto choose between.Once
subset of key information is then transmitted by a simple
again however it is the ability to reorder optimal observa-
socket connection to the listener thread of the ASA core
tions that allows some elegant simplifications to be made.
code.Inthiswaytheloopisclosed.
Firstly,if allreorderingsofa givenoptimalseriesare con-
sideredequallyoptimal,thenthesetis degeneratewithre-
4 Algorithm spect to reordering:for any set of gapswe need only con-
sider one possible reordering. The most straightforward is
4.1 Optimality to place the gaps in order of size. Secondly, it is apparent
thatanynewobservationshouldbeplacedinsuchawayas
Ifanagentcouldplaceobservationsatanytimeandbeguar- to exactly achieve one of the optimal gaps, since anything
anteed ofsuccess, then the choiceofobservationsis clear: elsewouldimmediatelyincreasethevalueofwfornogain.
they should be placed with the gap spacings indicated by Thisinsightdrasticallyreducesthesearchspaceofpossible
theoptimalsampling.Howeverinthenormaloperatingen- timestamps.
vironmentmanyobservationscanfail.Whenanobservation Animportantside-effectofthedefinitionoftheoptimal-
eventually doessucceed, the gap between that observation ity is thatthe agentusually prefersto acquire shorter gaps
and the last successful observation is unlikely to be of an first.Thisbehaviouremergesbecausetheoptimalityiscal-
ideallength—butitcouldbeclose. culatedbycomparingoptimalgapstoactualacquiredgaps.
What the agent requires is some unambiguous way to Themagnitudeoftheincreasein w accruedfromanynew
determinehowwellitscompletedspacingscomparetothe observation is directly proportional to the distance of that
optimalsetofgaps.Itisnotpossibletosimplycomparethe gap from its comparisongap. Because we choose to order
twosetsofgapsand“tickoff”perfectgapsastheyareob- the comparison gaps from shortest to longest, this means
tained,becauseevena“correct”observationisnotprecisely thatlargegapsearlyonarepenalisedbyrelativelylargein-
located. Telescope overheads mean that in practice an ac- creases in w. This is an important feature. It makes good
ceptablewindowfortheobservationmustbeprovided,and practicalsensetogofortheshortestusefulgapatanygiven
the observation can take place anywhere within that win- time, becausethe agentmustwait untilthattimestampbe-
dow. Some sort of fuzzy criterion could be used, but this fore it can discover whether the observation actually hap-
mustbeexplicitlydefinedandissomewhatarbitrary. pened(andhencetherequiredgapcollected).
3
EricS.Saundersetal.:Anautonomousadaptiveschedulingagentforperiodsearching
Astron.Nachr./AN(0000)
Itshouldbenoted,however,thatitisnotsufficienttoal- replies have random values between 0 and 1, and have no
waysseektheshortest“uncollected”gap,observationafter correlation with the likelihood that the observation will in
observation.Thiswouldindeedbethebeststrategyifobser- factbesuccessful.
vationswere alwaysguaranteedtosucceed,butin practice The passage of time in the simulation is regulated by
two types of forced deviations from optimality can be ex- the time server. This is a simple standalone process that
pectedtooccur.Thefirstisthatsometimestampsareknown providesthecurrentsimulationtimeviaasocketinterface.
priortoobservationtobeinaccessible,andhenceareguar- Internally,thetimeservercalculatesthecurrentsimulation
anteedtofail(e.g.becauseitisday,thetargetisnotvisible, timebyscalingtheamountofrealtimeelapsedsinceinstan-
thetargetistooclosetothemoonetc.).Thisinformationis tiation byan accelerationfactor,providedatstartup.Since
passedbacktotheagentintheformofascoreof0.Thesec- all agents use the time server as the canonical source for
ondisthatsomefractionofobservationsthatwereexpected timestampsandtimingcalculations,the timeserverallows
tosucceedwillinfactfail“onthenight,”duetouncertain- the simulation to be run many times faster than the real
tiesintheobservingprocessitself(weather,telescopeload, world, enabling full simulations of a likely observing run
etc.). tocompleteinareasonabletimeframe.Settingtheaccelera-
Thesefailuresforcethecreationofsub-optimalgapsin tionfactorto1allowsthetimeservertorunatnormalspeed,
the observing pattern. The specific shape of this observed andthereforeprovidestransparentaccessto thereal-world
pattern drives the choice of the next gap. This is because time.
sub-optimalityisdefinedasacontinuousspectrum—agap The probability of observation success is specified at
of a similar size to an optimal gap is “more optimal” than startup for each virtual telescope, and a random number
a gapofa verydifferentsize.Theeffectoffailedobserva- generator used to determine the success or failure of each
tionsistoincreasetheaveragesizeofgapsintheobserved observation at run-time. If successful, a fake FITS header
set. Althoughtheselargergapswere notplanned,theycan isgenerated,withthecorrectobservationtimestampplaced
nevertheless be useful, since they will be nearer in size to within the header. This header is encapsulated inside the
the larger gaps in the optimal set. It may therefore make RTMLmessageoftype“observation”thatisreturned.Oth-
sense for the agent to aim for a smaller gap, because this erwise, a “fail” message with no data product is returned.
will shuntthelargergapsalongin theoptimalitycompari- Using the existing user agent and node agent codebase
son,improvingtheoveralloptimalityoftheseries.Inother meantthatattheprotocolleveltheinteractionbetweeneS-
situationshowever,the agentmay choosea slightlylonger TARcomponentsinthesimulatorwasidenticaltothatinthe
gap,evenifithasnotachievedalltheshortergapspreceding realworld,allowingmostaspectsofdocumentexchangeto
it, becausetoomanyshortgapswillworsentheoptimality betestedbeforelivedeployment.
ofthelongerendoftherun.Thisisadirectconsequenceof
One limitation of the simulation environmentwas that
findingabettervalueofwforalongergap.
fail rates at the virtual telescopes were not correlated, that
Itisworthnotingthatatnopointarethedatathemselves
is,periodsoffailurewerenotconsecutiveintime.Thiswas
examined. This is by design; observing the Fourier trans-
notimplementedbecauseofthecomplexityofchoosingre-
form of the data during the experiment, for example, and
alistic correlationbehaviour.Evenwithoutsuchbehaviour,
changingthesamplingbehaviourbasedontheresultsleads
the most critical aspects of the agent were able to be ade-
toacomplexquagmireofinteractionsbetweentheobserver
quatelytested.Nevertheless,thisisanobviouswayinwhich
and the observations. It is possible, for example, to home
thevirtualtelescopeenvironmentcouldbeimproved.
in on a spurious signal in the periodogram,and to choose
The purpose of the tests performed in simulation was
asamplingthatselectivelyreinforcesthatsignal.Theguid-
to identify bugs at several levels, and to evaluate the per-
ing principle of simplicity suggests that such complexities
formance of the algorithm and iterate improvements in a
shouldbeavoided.
tightfeedbackcycle.Thisincludedanalysisatthemessage-
passing levelfor conformitywith the HTN protocolspeci-
5 Testing andresults fication. The behaviourof the implementedalgorithm was
carefullycomparedwiththedesignstatement.Anumberof
The agentwas extensivelytested in a virtualtelescope en- discrepancieswerefoundandcorrected.Mostimportantly,
vironment before being deployed on the eSTAR network, the performance of the algorithm under pseudo-realistic
whereitacquiredobservationsofavariablestarwithawell- conditionsallowedanumberofunforeseencornercasesto
knownperiodasaproof-of-concept.Thisenvironmentwas beidentifiedandcorrectlyhandled.
designedtoprovideavirtualtelescopenetworkthatwould Theschedulingalgorithmwastestedinanumberofsim-
replicate as closely as possible the actual operatingcondi- ulatedobservingruns,butthediscussionofasingleexperi-
tions under which the agent was to run. Each virtual tele- mentissufficienttoillustratethegeneralperformancechar-
scopeinstancecanbe“sited”atadifferentvirtuallocation, acteristics. In this simulation a 10 day, 60 observation run
allowingthecalculationofsunriseandsunsettimes,andthe wasattempted.Anaccelerationfactorof50,appliedtothe
riseandsettimesofarbitrarypointsonthecelestialsphere. time server,alloweda single 30minuteobservingwindow
Tomaketheenvironmentmorechallenging,non-zeroscore to complete in 36s. The start of the run was set to a date
4
EricS.Saundersetal.:Anautonomousadaptiveschedulingagentforperiodsearching
Astron.Nachr./AN(0000)
of 27/04/07, at which time the target was observable ap- Acknowledgements. ESSwasfundedthroughaPPARCe-Science
proximately 2/3 of the time from at least one of the three studentship. Thiswork ispart of the eSTARproject, which sup-
Robonet-1.0virtualtelescopes. In this simulation, in addi- ports AA and which is jointly funded by the DTI, PPARC and
tiontothenon-observableblocks(whichreturnedascoreof EPSRC.
0totheagent),thevirtualtelescopesweresettorandomly
failobservationsatrun-timewith50%probability(anarbi- References
traryfractionthatseemedreasonable).
Allan,A.,Hessman,F.,Bischoff,K.,etal.:2006,AN,327,744
The experiment was halted after 10 days had elapsed
Allan,A.,Naylor,T.,&Saunders,E.S.:2007,AN,thisvolume
in simulation time. It was found that of all the observa-
Allan,A.,Naylor,T.,Steele,I.A.,etal.:2004,inAdvancedSoft-
tionssubmittedtothenetwork,40hadsucceeded(i.e.,been
ware, Control, and Communication Systems for Astronomy.
accepted to the queue, and then been declared successful
EditedbyLewis,Hilton;Raffi,Gianni.ProceedingsoftheSPIE,
at some point in the specified observing window), while
5496,313-322
50 of the queued observations had failed (queued but not Bode, M. F., Cardiff U., U. Hertfordshire, et al.: 2004, Bulletin
observed). All the successful observations were found to oftheAmericanAstronomicalSociety,Vol.36,Bulletinofthe
haveclose-to-optimaltimestamps,subjecttotheobservabil- AmericanAstronomicalSociety
ityandsuccessconstraints. Fraser,S.N.:2006,AN,327,779
Hessman,F.V.:2006,AN,327,751
Theresultsindicatedcrudelythatiftherealnetworkex-
Mottram,C.:2006,AN,327,806
hibitedasimilarfailrate,thenaballparkfigureofapproxi-
Pennypacker,C.,Boer,M.,Denny,R.,etal.:2002A&A,395,727
mately2/3oftheobservationstobecompletedsuccessfully
Saunders,E.S.:2007,Thesis(UniversityofExeter)
wasareasonableexpectationbythe10thcontinuousdayof
Saunders,E.S.,Naylor,T.,&Allan,A.:2006a,AN,327,783
observing,andadditionally,thatthespacingsbetweenindi-
Saunders,E.S.,Naylor,T.,&Allan,A.:2006b,A&A,455,757
vidualobservationswerelikelytobenear-optimal. Saunders,E.S.,Naylor,T.,&Allan,A.:2007,inprep.
Theanalysisoftheresultsofthesubsequentobserving
runareoutsidethescopeofthiswork,butwillbepresented
indetailinaforthcomingpaper(Saundersetal.,2007).For
adetaileddiscussionofoptimalgeometricsamplingandthe
adaptive scheduling agent, including results, see Saunders
(2007).
6 Conclusions
Wehavebuiltanadaptiveschedulingagentthatimplements
theoptimalgeometricsamplingstrategyforperiodsearch-
ingdescribedbySaundersetal.(2006b)inthecontextofan
HTN-compliantnetworkoftelescopes.Thealgorithmauto-
matically penalises the agent for choosing too many short
gaps, but is similarly hostile to the chasing of large gaps
when there are shorter, more valuable alternatives. In this
way a balancepointis determinedthatunambiguouslyde-
fines the best option for the agent at any point in the run.
The deceptively simple rule sequence for finding the opti-
malityimplicitlyutilisesthereorderingpropertyofoptimal
sequences, maximises the agent’s cautiousness in the face
of uncertainty, and provides a computationally cheap and
scalablewaytounambiguouslycalculatethebestchoiceof
actionatanypoint.Itisalsostable—asnapshotoftherun
atanypointintimeisoptimalforthatsubsetofpoints,and
minimisestheeffectsofobservationfailureonthesequence.
Inthecaseofperfectobservingbehaviour(nofailures),the
optimal sequence is naturally recovered. Importantly, the
degreeoffailureexhibitedbythenetworkcanalsochange
dynamicallywithoutadverselyaffectingthealgorithm,be-
causeitmakesnoassumptionsaboutthestabilityofthenet-
work,andmakesnoattempttomodelthebehaviourofthe
telescopesonwhichitrelies.
5
