Table Of ContentOnline Stochastic Combinatorial Optimization
Pascal Van Hentenryck and Russell Bent
The MIT Press
Cambridge, Massachusetts
London, England
(cid:2)c 2006Massachusetts Institute ofTechnology
LibraryofCongressCataloging-in-PublicationData
VanHentenryck, Pascal.
Onlinestochasticcombinatorialoptimization / PascalVanHentenryckandRussellBent.
p. cm.
ISBN-13;978-0-262-22080-4(alk.paper)
ISBN-10;0-262-22080-6(alk.paper)
1. Stochastic processes. 2. Combinatorialoptimization. 3. Onlinealgorithms.
4. Operationsresearch. I.Bent,Russell. II.Title.
T57.32.V36 2006
003—dc22
2006048141
Contents
1 Introduction 1
1.1 From A Priori to Online Stochastic Optimization 1
1.2 Online Stochastic CombinatorialOptimization 2
1.3 Online Anticipatory Algorithms 4
1.4 Online Stochastic CombinatorialOptimization in Context 5
1.5 Organizationand Themes 13
I ONLINE STOCHASTIC SCHEDULING
2 Online Stochastic Scheduling 19
2.1 The Generic Offline Problem 19
2.2 The Online Problem 20
2.3 The Generic Online Algorithm 20
2.4 Properties of Online Stochastic Scheduling 22
2.5 Oblivious Algorithms 23
2.6 The Expectation Algorithm 24
2.7 The Consensus Algorithm 26
2.8 The Regret Algorithm 27
2.9 Immediate Decision Making 30
2.10 The Suboptimality Approximation Problem 31
2.11 Notes and Further Reading 33
3 Theoretical Analysis 35
3.1 Expected Loss 35
3.2 Local Errors 36
3.3 Bounding Local Errors 38
3.4 The Theoretical Results 40
3.5 Discussion on the Theoretical Assumptions 41
3.6 Notes and Further Reading 43
4 Packet Scheduling 45
4.1 The PacketScheduling Problem 45
4.2 The Optimization Algorithm 46
4.3 The Greedy Algorithm is Competitive 50
4.4 The Suboptimality Approximation 51
4.5 Experimental Setting 53
4.6 Experimental Results 54
4.7 The Anticipativity Assumption 60
4.8 Notes and Further Reading 66
II ONLINE STOCHASTIC RESERVATIONS
5 Online Stochastic Reservations 71
5.1 The Offline Reservation Problem 71
5.2 The Online Problem 72
5.3 The Generic Online Algorithm 73
5.4 The Expectation Algorithm 74
5.5 The Consensus Algorithm 74
5.6 The Regret Algorithm 75
5.7 Cancellations 77
6 Online Multiknapsack Problems 79
6.1 Online Multiknapsack with Deadlines 79
6.2 The Suboptimality Approximation 81
6.3 Experimental Results 89
6.4 Notes and Further Reading 98
III ONLINE STOCHASTIC ROUTING
7 Vehicle Routing with Time Windows 103
7.1 Vehicle Dispatching and Routing 103
7.2 Large Neighborhood Search 107
7.3 Notes and Further Reading 112
8 Online Stochastic Routing 113
8.1 Online Stochastic Vehicle Routing 113
8.2 Online Single Vehicle Routing 116
8.3 Service Guarantees 118
8.4 A Waiting Strategy 120
8.5 A Relocation Strategy 121
8.6 Multiple Pointwise Decisions 122
8.7 Notes and Further Reading 125
9 Online Vehicle Dispatching 127
9.1 The Online Vehicle Dispatching Problems 127
9.2 Setting of the Algorithms 128
9.3 Experimental Results 129
9.4 Visualizations of the Algorithms 138
9.5 Notes and Further Reading 149
10 Online Vehicle Routing with Time Windows 153
10.1 The Online Instances 153
10.2 Experimental Setting 155
10.3 Experimental Results 156
10.4 Notes and Further Reading 165
IV LEARNING AND HISTORICAL SAMPLING
11 Learning Distributions 171
11.1 The Learning Framework 171
11.2 Hidden Markov Models 172
11.3 Learning Hidden Markov Models 174
11.4 Notes and Further Reading 182
12 Historical Sampling 185
12.1 Historical Averaging 185
12.2 Historical Sampling 186
V SEQUENTIAL DECISION MAKING
13 Markov Chance-Decision Processes 193
13.1 Motivation 193
13.2 Decision-Chance versus Chance-Decision 194
13.3 Equivalence of MDCPs and MCDPs 196
13.4 Online Anticipatory Algorithms 199
13.5 The Approximation Theorem for Anticipative MCDPs 202
13.6 The Anticipativity Assumption 208
13.7 Beyond Anticipativity 210
13.8 The General Approximation Theorem for MCDPs 214
13.9 Notes and Further Reading 218
References 219
Index 229
Preface
FromAPrioritoOnlineOptimization Optimizationsystemstraditionallyhavefocusedonapriori
planningandarerarelyrobusttodisruptions. Theairlineindustry,asophisticateduserofoptimiza-
tion technology, solves complex fleet assignment, crew scheduling, and gate allocation problems as
partofitsoperationsusingsomeofthemostadvancedoptimizationalgorithmsavailable. Yetunex-
pected events such as weather conditions or strikes produce major disruptions in its operations. In
August 2005,British Airwaystook three to four days to resume its normaloperationsafter a strike
ofoneofitscateringsubcontractors;manyofitsplanesandcrewswereatthewrongplacesbecause
of the airline’s inability to anticipate and quickly recoverfrom the event. The steel industry, also a
significantcustomerofoptimizationtechnology,typicallyseparatesstrategicplanning(whichorders
to accept) and tactical planning (which priorities to assign them) from daily scheduling decisions.
The strategicdecisionsarebasedoncoarseapproximationsofthe factorycapabilitiesandforecasts,
sometimesleadingtoinfeasibleschedules,misseddeadlines,andpoordecisionswhennovelincoming
orders arrive online.
Fortunately, the last decades have witnessedsignificant progressin optimization andinformation
technology. The progress in speed and functionalities of optimization software has been simply
amazing. Advancesintelecommunications,suchastheglobalpositioningsystem(GPS),sensorand
mobile networks, and radio frequency identification (RFID) tags, enable organizations to collect a
wealth of data on their operations in real time.
It also is becoming increasingly clear that there are significant opportunities for optimization
algorithms that make optimization decisions online. Companies such as UPS have their own me-
teorologists and political analysts to adapt their operations and schedules online. Pharmaceutical
companies must schedule drug design projects with uncertainty on success, duration, and new de-
velopments. Companies such as Wal-Mart now try to integrate their supply chains with those of
theirs suppliers, merging their logistic systems and replenishing inventories dynamically.
Asaconsequence,wemayenvisionanewerainwhichoptimizationsystemswillnotonlyallocate
resourcesoptimally: they will reactand adaptto externaleventseffectively under time constraints,
anticipating the future and learning from the past to produce more robust and effective solutions.
These systems may deal simultaneously with planning, scheduling, and control, complementing a
priori optimization with integrated online decision making.
Online Stochastic Combinatorial Optimization This book explores some of this vision, trying
to understand its benefits and challenges and to develop new models, algorithms,and applications.
It studies online stochastic combinatorial optimization (OSCO), a class of optimization applica-
tions where the uncertainty does not depend on the decision-making process. OSCO problems are
ubiquitous in our society and arise in networking, manufacturing, transportation, distribution, and
reservation systems. For instance, in courier service or air-taxi applications, customers make re-
quests at various times and the decision-making process must determine which requests to serve
and how under severe time constraints and limited resources.
Different communities approach new classes of problems in various ways. A problem-driven
community studies individual applicationsanddesigns dedicatedsolutions for eachofthem. A the-
oreticallyorientedcommunityoftensimplifiestheapplicationstoidentifycorealgorithmicproblems
that hopefully are amenable to mathematical analysis and efficient solutions. These approaches
are orthogonal and often produce invaluable insights into the nature of the problems. However,
many professionalsin optimizationlike to saythat“therearetoo manyapplicationswith toomany
idiosyncratic constraints” and that “an approximated solution to a real problem is often prefer-
able to an optimal solution to an approximated problem.” As a result, this book takes a third,
engineering-oriented, approach. It presents the design of abstract models and generic algorithms
that are applicable to many applications, captures the intricacies of practical applications, and
leverages existing results in deterministic optimization.
OnlineAnticipatoryAlgorithms Moreprecisely,totackleOSCOapplications,thisbookproposes
the class of online anticipatory algorithms that combine online algorithms (from computer science)
andstochasticprogramming(fromoperationsresearch). Onlineanticipatoryalgorithmsassumethe
availabilityofadistributionoffutureeventsoranapproximationthereof. Theytakedecisionsduring
operationsbysolvingdeterministicoptimizationproblemsthatrepresentpossiblerealizationsofthe
future. By exploiting insights into the problem structure, online anticipatory algorithms address
the time-critical nature of decisions, which allows for only a few optimizations at decision time or
between decisions.
The main purpose of this book is thus to present online anticipatory algorithms and to demon-
strate their benefits on a variety of applications including online packet scheduling, reservation
systems,vehicledispatching,andvehiclerouting. Oneachoftheseapplications,onlineanticipatory
algorithmsareshowntoimprovecustomerserviceorreducecostssignificantlycomparedtooblivious
algorithms that ignore the future. The applications are diverse. For some of them, the underlying
optimization problem is solvable in polynomial time. For others, even finding optimal solutions to
the deterministic optimization where all the uncertainty is revealed is beyond the scope of current
optimizationsoftware. Moreover,theseapplicationscapturedifferenttypesofdecisions. Onsomeof
them,theissueistochoosewhichrequesttoserve,while,onothers,thequestionishowtoservethe
request. Onsomeoftheseapplications,itisnotclearevenwhatthedecisionsshouldbeinanonline
setting, highlighting some interesting modeling issues raised by online applications. In particular,
the bookpresentssomeresultsonvehicle-routingstrategiesthatwereamazinglycounterintuitiveat
first and seem natural retrospectively.
Of course,in practice, not all applications come with a predictive model of the future. The book
alsostudiesapplicationsinwhichonlythestructureofmodelorhistoricaldataisavailable. Itshows
how to integrate machine learning and historical sampling into online anticipatory algorithms to
address this difficulty.
FromPracticetoTheoryandBack Demonstratingthebenefitsofonlineanticipatoryalgorithms
on a number of applications, however diverse and significant, is hardly satisfying. It would be
desirable to identify the class of applications that are amenable to effective solutions by online an-
ticipatoryalgorithms. Suchcharacterizationsareinherentlydifficult,however,evenindeterministic
optimization: indeed optimization experts sometimes disagree about the best way to approach a
novel application. OSCO applications further exacerbate the issue by encompassing online and
stochastic elements.
Thisbookattemptstoprovidesomeinsightsaboutthebehaviorofonlineanticipatoryalgorithms
byidentifyingassumptionsunderwhichtheydelivernear-optimalsolutionswithapolynomialnum-
ber of optimizations. At the core of online anticipatory algorithms lies an anticipatory relaxation
that removes the interleaving of decisions and observations. When the anticipatory relaxation is
tight,apropertycalled(cid:2)-anticipativity, onlineanticipatoryalgorithmscloselyapproximateoptimal,
a posteriori solutions. Although this is a strong requirement, the applications studied in this book
are shown to be (cid:2)-anticipative experimentally. The inherent temporal structure of these applica-
tions, together with well-behaved distributions, seems to account for this desirable behavior. The
analysispresentedhereisonlyasmallfirststepandmuchmoreresearchisnecessarytocomprehend
the nature of OSCO applications and to design more advanced online anticipatory algorithms.
A Model for Sequential Decision Making The theoretical analysis has an interesting side ef-
fect: it highlights the common abstract structure that was buried under the idiosyncracies of the
applications, their models, and their algorithms. It also establishes clear links with Markov Deci-
sion Processes(MDP),afundamentalapproachtosequentialdecisionmakingextensivelystudiedin
artificialintelligenceandoperationsresearch. LikeMDPs,onlinestochasticcombinatorialoptimiza-
tion alternates between decisions and observations, but with a subtle difference: the uncertainty
in MDPs is endogenous and depends on the decider’s actions. It is thus possible to define a vari-
ant of MDPs, called Markov Chance-Decision Processes (MCDPs), that captures online stochastic
combinatorial optimization and whose uncertainty is exogenous. In MCDPs, the decision process
alternates between observing uncertain inputs and deterministic actions. In contrast, the decision
process in traditional MDPs, called Markov Decision-Chance Processes (MDCPs) here, alternates
between actions whose effects are uncertain and observations of the action outcomes. As a conse-
quence, MCDPs crystallize the essence of online stochastic combinatorial optimization that can be
summarized as follows:
Anticipatory Relaxation: Contrary to MDCPs, MCDPs naturally encompass an anticipatory
relaxationfor estimating the future. The anticipatory relaxationcan be approximatedby the
solutions of deterministic optimization problems representing scenarios of the future.
Online Anticipatory Algorithms: MCDPs naturally lead to a class of online anticipatoryalgo-
rithmstakingdecisionsonlineateachtime stepusingtheobservationsandapproximationsto
the anticipatory relaxations.
Anticipativity: When the anticipatoryrelaxationis (cid:2)-anticipative,online anticipatoryalgorithms
produce near-optimal solutions with a polynomial number of optimizations.
Learning: The distribution of the inputs can be sampled and learned independently from the un-
derlyingdecisionprocess,providingacleanseparationofconcernsandcomputationalbenefits.
So, perhaps now that this book is written, twenty minutes are sufficient to describe its contents,
makingitswritingyetanotherhumblingexperience. However,theopenissuesitraisesaredaunting
both in theory and practice.
