Construction of Component-Based Applications by Planning

231 Pages·2005·1.1 MB·English

Construction of Component-Based
Applications by Planning
by
Tatiana Kichkaylo
Adissertationsubmittedinpartialfulfillment
oftherequirementsforthedegreeof
DoctorofPhilosophy
DepartmentofComputerScience
NewYorkUniversity
January2005
ResearchAdvisors:
VijayKaramcheti
ErnestDavis
(cid:13)c TatianaKichkaylo
AllRightsReserved2005
You are never given a wish without also being
given the power to make it true.
You may have to work for it, however.
Richard Bach
Acknowledgments
Firstofall,Iwouldliketothankmyadvisors,VijayandErnie,forlettingmeexplore
andinventwhilestillkeepingmeontrack. Strikingthisbalancewasnotaneasytask,
but you did it. I would also like to thank all people who asked tough questions. John,
Dan,andRobert,thankyouforyourtimeandhelp.
IameternallygratefultomyparentsformakingmewhoIam. Youtaughtmethat,
nomatterwhathappens,itisimportanttokeepworking. Remindingmyselfabout“at
leasttwowaysout”helpedmemanytimes.
I thank Anca, my sister for all practical purposes, for taking good care of me
and making me feel home. I thank my friends, Sveta, Dana, Igor, Oana, Gela, for
accepting my not exactly angelic personality. And I thank all my teachers for telling
methatnomatterhowfarweget,thereisstillaroadahead.
iv
Abstract
Many modern wide-area distributed systems are component-based. This approach
provides great flexibility in adapting applications to the changing state of the envi-
ronmentanduserrequirements,butincreasesthecomplexityofconfiguringtheappli-
cations. Because of the scale and heterogeneity of modern wide-area environments,
manual configuration is hard, inefficient, suboptimal, and error-prone. Automated
applicationconfigurationisdesired.
Constructing distributed applications requires choosing a set of components that
willconstitutetheapplicationinstanceandassigningnetworkresourcestocomponent
executions and data transfers. Stated this way, the application configuration problem
(ACP) is similar to the planning (action selection) and scheduling (resource alloca-
tion)problemsstudiedbytheArtificialIntelligence(AI)community.
This thesis investigates the problem of solving the ACP using AI planning tech-
niques. However, the ACP poses several challenges not usually encountered and ad-
dressed by the traditional AI solutions. The problem specification for the ACP can
be much larger than the solution, with the relevant portions only identified during the
search. Additionally, the interactions between planning operators are numeric rather
thanlogical. Finally,itisdesirabletobeabletotradeoffqualityofthesolutionversus
v
searchtime.
We show that the ACP is undecidable in general. Therefore, instead of a single
algorithm, we propose a set of techniques that can be used to compose an algorithm
for a particular variety of the ACP that can exploit natural restrictions exhibited by
thatvariety. Thesetechniquesaddressthechallengesabovebydynamicallyobtaining
portions of the problem specification as necessary during the search, using envelope
hierarchies based on numeric information for pruning and search guidance, and dis-
cretizingcontinuous variablestoapproximate numericparameterswithout restricting
theformofsupportednumericfunctions.
We illustrate these techniques by describing their use in algorithms tailored for
twospecificvarietiesoftheACP—snapshotconfigurationsfordynamiccomponent-
based frameworks, and scheduling of grid workflows with replica selection and ex-
plicitresourcereservations. Experimentalevaluationoftheperformanceofthesetwo
algorithmsshowsthatthetechniquessuccessfullyachievetheirgoals,withacceptable
run-timeoverhead.
Contents
Acknowledgments iv
Abstract v
ListofFigures xiii
ListofTables xvii
ListofAppendices xviii
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Approachandchallenges . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.4 Thesisorganization . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.5 Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2 Background 11
2.1 Component-basedframeworks . . . . . . . . . . . . . . . . . . . . . 11
vii
2.1.1 Dynamiccomponent-basedframeworks . . . . . . . . . . . . 12
2.1.2 Computationalgrids . . . . . . . . . . . . . . . . . . . . . . 13
2.1.3 PlannersfortheACP . . . . . . . . . . . . . . . . . . . . . . 15
2.2 AIplanning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.2.1 Theclassicalplanningproblem . . . . . . . . . . . . . . . . 18
2.2.2 Planningwithnumericstatevariables . . . . . . . . . . . . . 20
2.2.3 Searchinplanning . . . . . . . . . . . . . . . . . . . . . . . 22
2.2.4 Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.2.5 Planningwithresources . . . . . . . . . . . . . . . . . . . . 26
2.3 Exampleproblems . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.3.1 Mailapplication . . . . . . . . . . . . . . . . . . . . . . . . 29
2.3.2 Webcastapplication . . . . . . . . . . . . . . . . . . . . . . 32
2.3.3 Gridplanning . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3 TheApplicationConfigurationProblem 39
3.1 GeneralmodelofACP . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.1.1 Environment . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.1.2 Componentsanddata . . . . . . . . . . . . . . . . . . . . . . 40
3.1.3 Numericproperties . . . . . . . . . . . . . . . . . . . . . . . 42
3.1.4 Usergoal . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.1.5 Applicationconfiguration . . . . . . . . . . . . . . . . . . . 43
3.1.6 Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.1.7 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
viii
3.1.8 Correctnessofinformation . . . . . . . . . . . . . . . . . . . 46
3.2 ComponentPlacementProblem . . . . . . . . . . . . . . . . . . . . 46
3.2.1 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.2.2 Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.2.3 Compilationintoaplanningproblem . . . . . . . . . . . . . 51
3.3 CPPandAIproblems . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.3.1 CPPandSTRIPSplanning . . . . . . . . . . . . . . . . . . . 56
3.3.2 Networkspaceandphysicaltime . . . . . . . . . . . . . . . 58
3.3.3 CPPandmetricplanning . . . . . . . . . . . . . . . . . . . . 60
3.3.4 CPPandscheduling . . . . . . . . . . . . . . . . . . . . . . 61
3.4 Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4 Dealingwithlarge-scaleopenworlds 65
4.1 Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.1.1 On-demandcompilation . . . . . . . . . . . . . . . . . . . . 66
4.1.2 Combiningregressionandprogression . . . . . . . . . . . . . 69
4.2 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.2.1 Thecorealgorithm . . . . . . . . . . . . . . . . . . . . . . . 71
4.2.2 Reasoningaboutresources . . . . . . . . . . . . . . . . . . . 78
4.2.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
4.3 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
4.3.1 Planningundervariousconditions . . . . . . . . . . . . . . . 84
4.3.2 Scalabilitywithrespecttonetworksize . . . . . . . . . . . . 85
ix
4.3.3 Scalabilitywithrespecttoirrelevantcomponents . . . . . . . 87
4.3.4 Reusabilityofexistingdeployments. . . . . . . . . . . . . . . 88
4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
5 Dealingwithcomplexresourcefunctions 91
5.1 Ramificationsofnumericinteractionsbetweenoperators . . . . . . . 91
5.2 Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5.2.1 Addinganewenvelope . . . . . . . . . . . . . . . . . . . . . 93
5.2.2 Boundedenvelopegraphs . . . . . . . . . . . . . . . . . . . 96
5.2.3 Hierarchyofenvelopes . . . . . . . . . . . . . . . . . . . . . 96
5.2.4 Othermodificationsofthealgorithm . . . . . . . . . . . . . . 97
5.3 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
5.3.1 Firstlevelgraph . . . . . . . . . . . . . . . . . . . . . . . . 98
5.3.2 Secondlevelgraph . . . . . . . . . . . . . . . . . . . . . . . 99
5.3.3 Thirdlevelgraph . . . . . . . . . . . . . . . . . . . . . . . . 100
5.3.4 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
5.4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
5.4.1 Experimentalsetup . . . . . . . . . . . . . . . . . . . . . . . 104
5.4.2 Planningtime . . . . . . . . . . . . . . . . . . . . . . . . . . 106
5.4.3 Memoryconsumption . . . . . . . . . . . . . . . . . . . . . 108
5.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
6 Optimizingresourceconsumption 110
6.1 Limitationsofthegreedyapproach . . . . . . . . . . . . . . . . . . . 110
6.2 Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
x