Stochastic Programming 84 Part II

188 Pages·1986·2.333 MB·English

by  J. L. Nazareth, R. J.-B. Wets (auth.), Andras Prékopa, Roger J.- B. Wets (eds.)

MATHEMATICAL PROGRAMMING STUDIES
PREFACE
One may think of stochastic programming as simply a subfield of nonlinear
programming. The fact that the objective function or some of the constraints are
expressed in terms of multidimensional integrals does not change the essence of
the problem, at least in theory. However, it is precisely because the problem at hand
demands the evaluation of multidimensional integrals that its nature is altered in a
fundamental way from a numerical viewpoint.
Let us consider the following type of problem
find x e S c R that minimizes F(x)
where F(x)-E{f(x,w)}=~f(x, to)dP(w), and S is a closed set determined by
some constraints that could be of probabilistic type. For simplicity, let us consider
the case in which only the objective is given by a multidimensional integral with
respect to the probability measure P. Because the operator E has a 'smoothing'
effect, this optimization problem usually has many desirable properties. For example,
if the integrand f is convex in x, then so is F. If f is differentiable with respect to
x or the measure P is absolutely continuous, then it usually turns out that F is also
differentiable. Thus, in principle the problem could be solved by relying on some
of the existing subroutines for nonlinear programming problems; all that is needed
is to appeal to a multidimensional integration subroutine to evaluate the function
F, its gradients, or subgradients, as the case may be. However, general purpose
integration subroutines are available only for the 1-dimensional case. In 2-
dimensions some serious difficulties already must be dealt with, and in 3-dimensions
subroutines are available only for very special cases. Typically, a stochastic program-
ming problem involves anywhere from 5 to 100 random variables, making it totally
impossible to rely on existing numerical integration subroutines. Naturally, some
efforts have been made to design multidimensional integration subroutines--and
some of the papers in this collection report the progress made in that direction--but
essentially they rely on,: sampling techniques (involving the generation of pseudo-
or quasi-random numbers). This presupposes that the integrand is sufficiently easy
to evaluate, and for stochastic programming models that is the exception, not the
rule. The integrand f(x, )ot is often defined implicitly, for example as the optimal
value of an optimization problem. Thus efficient procedures must avoid numerous
evaluations of the integrand.
Although the search for reliable multidimensional integration subroutines has not
been abandoned, the design of solution procedures for stochastic optimization
problems has been chiefly oriented toward methods that in one way or another
avoid coming to grips with this potential stumbling block. Excluding some very
V
ecaferP
specific classes of stochastic programming problems, the suggested solution strategies
can be divided into two major categories:
(cid:12)9 'descent' methods that rely on directions determined by statistical estimates of
the subgradients of F, and
(cid:12)9 approximation methods that replace either the original distribution P by one
that would be more manageable or the integrand f by a 'simpler' one that would
make it possible to carry out the multidimensional integration.
All these possibilities are illustrated in the articles of this collection.
In Volume 1, the first three articles deal with evaluating multidimensional integrals
as they arise in stochastic programming (Szantai, Niederreiter) or obtaining bounds
for them (Gassman/Ziemba). The next group of three articles deal with approxima-
tion schemes. We start with a review of the existing results as well as some suggestions
for implementation (Birge/Wets). Approximating by problem redefinition is illus-
trated in the article by Beale, Dantzig and Watson, whereas Andreatta and Runggal-
dier work by approximating the probability measure. Intimately related to approxi-
mation is the question of the stability of the problem under various perturbations,
in particular perturbations of the probability distribution function. This is the subject
of the contributions of Dupa~ov~i and Wang.
The remaining articles deal with specific procedures for solving particular or
general stochastic programming problems. In Volume 1, the articles by Klein
Haneveld and Qi deal with stochastic network problems. The structure of the
problems plays a very important role in the procedures they suggest. In Volume 2,
the first four articles deal with stochastic programs with recourse models
(Nazareth/Wets, Wallace, Louveaux, Rockafellar/Wets). Next, Komaromi suggests
a new dual-based procedure for solving problems with probabilistic constraints.
The last three articles introduce modifications of the stochastic-gradient method to
make the calculations of the step size more directly adaptive (Ruszczynski), to
include nonstochastic descent information (Marti/Fuchs), and to allow for its
application in the case where the decision variables themselves are probability
measures (Gaivoronski).
The decision to submit these contributions in the form of a Study was made at
the first meeting of COSP (Committee on Stochastic Programming) on December
1, 1983, at IIASA (International Institute for Applied Systems Analysis), Laxenburg,
Austria. These two vo4nmes could very well serve as commemorative issues to mark
that occasion.
Andras Pr6kopa
Roger J.-B. Wets
Founder and first Editor-in-Chief
M.L. BALANSK1
Editor-in-Chief
R.W. COTTLE, Department of Operations Research, Stanford University, Stanford, CA 94305, U.S.A.
Co-Editors
L.C.W. DIXON, Numerical Optimisation Centre, The Hatfield Polytechnic, College Lane, Hatfield,
Hertfordshire ALl0 9AB, England
B. KORTE, lnstitut ffir Okonometrie und Operations Research, Universitfit Bonn, Nassestrasse 2,
D-5300 Bonn ,1 FR Germany
M.J. TODD, School of Operations Research and Industrial Engineering, Upson Hall, Cornell Universi-
ty, Ithaca, NY 14853, U.S.A.
Associate Editors
E.L. ALLGOWER, Colorado State University, Fort Collins, CO, U.S.A.
W.H. CUNN1NGHAM, Carteton University, Ottawa, Ontario, Canada
J.E. DENNIS, Jr., Rice University, Houston, TX, U.S.A.
B.C. EAVES, Stanford University, CA, U.S.A.
R. FLETCHER, University of Dundee, Dundee, Scotland
D. GOLDFARB, Columbia University, New York, USA
J.-B. HIRIART-URRUTY, Universit6 Paul Sabatier, Toulouse, France
M. IRI, University of Tokyo, Tokyo, Japan
R.G. JEROSLOW, Georgia Institute of Technology, Atlanta, GA, U.S.A.
D.S. JOHNSON, Bell Telephone Laboratories, Murray Hill, N J, U.S.A.
C. LEMARECHAL, INR1A-Laboria, Le Chesnay, France
L. LOVASZ, University of Szeged, Szeged, Hungary
L. MCLINDEN, University of Illinois, Urbana, 1L, U.S.A.
M.J.D. POWELL, University of Cambridge, Cambridge, England
W.R. PULLEYBLANK, University of Calgary, Calgary, Alberta, Canada
A.H.G. RINNOOY KAN, Erasmus University, Rotterdam, The Netherlands
K. RITTER, University of Stuttgart, Stuttgart, FR Germany
R.W.H. SARGENT, Imperial College, London, England
D.F. SHANNO, University of California, Davis, CA, U.S.A.
L.E. TROTTER, Jr., Cornell University, Ithaca, NY, U.S.A.
H. TUY, Institute of Mathematics, Hanoi, Socialist Republic of Vietnam
R.J.B. WETS, University of Kentucky, Lexington, KY, U.S.A.
Senior Editors
E.M.L. BEALE, Scicon Computer Services Ltd., Milton Keynes, England
G.B. DANTZIG, Stanford University, Stanford, CA, U.S.A.
L.V. KANTOROVICH, Academy of Sciences, Moscow, U.S.S.R.
T.C. KOOPMANS, Yale University, New Haven, CT, U.S.A.
A.W. TUCKER, Princeton'University, Princeton, N J, U.S.A.
P. WOLFE, IBM Research Center, Yorktown Heights, NY, U.S.A.
MATHEMATICAL
P R O G R A M M I N G
STUDY28
A PUBLICATION OF
THE MATHEMATICAL PROGRAMMING SOCIETY
Stochastic Programming 84
Part II
Edited by A. PRI~KOPA and R.J.-B. WETS
C H ~ p (
N
April 6891
NORTH-HOLLAND - AMSTERDAM
© The Mathematical Programming Society, Inc. - 1986
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or trans-
mitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without
the prior permission of the copyright owner.
Submission to this journal of a paper entails the author's irrevocable and exclusive authorization of the
publisher to collect any sums or considerations for copying or reproduction payable by third parties (as
mentioned in article 17 paragraph 2 of the Dutch Copyright Act of 1912 and in the Royal Decree of June
20, 1974 (S. 351) pursuant to article 16b of the Dutch Copyright Act of 1912) and/or to act in or out of Court
in connection therewith.
This STUDY is also available to nonsubscribers in a book edition.
Printed in The Netherlands
CONTENTS
PART I
(Mathematical Programming Study )72
Preface
.T Szfintai, Evaluation of a special multivariate gamma distribution function
.H Niederreiter, Multidimensional numerical integration using pseudo ran-
dom numbers
.H Gassmann and W. Ziemba, A tight upper bound for the expectation of a
convex function of a multivariate random variable
.J Birge and R.J.-B. Wets, Designing approximation schemes for stochastic
optimization problems, in particular for stochastic programs with recourse
E.M.L. Beale, G.B. Dantzig and R.D. Watson, A first order approach to a
class of multi-time period stochastic programming problems
G. Andreatta and W. Runggaldier, An approximation scheme for stochastic
dynamic optimization problems
.J Duparovfi, Stability in stochastic programming with recourse. Contami-
nated distributions
.J Wang, Lipschitz continuity of objective functions in stochastic programs
with fixed recourse and its applications
.W Klein Haneveld, Robustness against dependence in PERT: An application
of duality and distributions with known marginals
.L Qi, An alternate method for stochastic linear programming with simple
recourse
PART II
(this volume)
Preface v
.L Nazareth and R.J.-B. Wets, Algorithms for stochastic programs: The case
of nonstochastic tenders 1
.S Wallace, Decomposing the requirement space of a transportation problem
into polyhedral cones 92
.F Louveaux, Multistage stochastic programs with block-separable recourse 84
R.T. Rockafellar and R.J.B. Wets, A Lagrangian finite generation technique
for solving linear-quadratic problems in stochastic programming 36
.~I Komgtromi, A dual method for probabilistic constrained problems 94
.A Ruszczyfiski and W. Syski, A method of aggregate stochastic subgradients
with on-line stepsize rules for convex stochastic programming problems 311
.K Marti and E. Fuchs, Computation of descent directions and efficient points
in stochastic optimization problems without using derivatives 231
.A Gaivoronski, Linearization methods for optimization of functionals which
depend on probability measures 751
Mathematical Programming Study 28 (1986) 1-28
North-Holland
ALGORITHMS FOR STOCHASTIC PROGRAMS:
THE CASE OF NONSTOCHASTIC TENDERS
J.L. NAZARETH
International Institute for Applied Systems Analysis, A-2361 Laxenburg, Austria*
R.J.-B. WETS
Department of Mathematics. University of California at Davis, CA 95616, USA
Received 51 January 1984
Revised manuscript received 72 February 5891
We consider solution strategies for stochastic programs whose deterministic equivalent programs
take on the form: Find xeR', XeR" such that ,O~>x Ax=b, Tx=x and z=cx+~(X) is
minimized. We suggest algorithms based upon (i) extensions of the revised simplex method, (ii)
inner approximations (generalized programming techniques), (iii) outer approximations (min-
max) strategies.
Key words: Stochastic Programs with Recourse, Generalized Programming, Nonstochastic
Tenders, Inner Linearization.
1. Introduction
The class of stochastic programs (with recourse) that we consider in this paper,
and to which we refer as having nonstochastic tenders, arise as models for the
following decision process. An (optimal) decision vector x must be selected when
some of the parameters of the problem are only known in probability, i.e. only in
a statistical sense, the actual cost depending in part on how well a transformation
of x, X = Tx matches a random demand or recourse vector p.
We think of X as a tender, nonstochastic if the transformation T does not depend
on the (unknown) values of the random parameters. For example, stochastic pro-
grams with simple recourse and fixed technology matrix are of this type. As we
shall see in Section 2, for stochastic (linear) programs, the equivalent deterministic
program can then be e~pressed as:
find xER", X~R" such that
Ax=b, Tx=x, x>~O, (1.1)
and z = cx + g'(X) is minimized.
The algorithms that we analyze could be viewed as procedures for convex programs
of the type (1.1) that seek to take advantage of the special structure, and to some
extent that view is certainly correct. In fact we expect that the suggested techniques
* Present address: CDSS, P.O. Box 4908, Berkeley, CA 94704, USA
2 .I.L. Nazareth, R.J.-B. Wets / Stochastic programs: Nonstochastic tenders
will also be efficient whenever nonlinear optimization problems can be cast in the
form (1.1). However, because stochastic programming problems present computa-
tional challenges of their own, it is their specific properties that are always in the
background of our solution strategies. For example, our title is intended to suggest
that eht major task of the solution procedure si the finding of optimal ro nearly optimal
.srednet
In Section 2, we review briefly the properties of stochastic programs that will be
used in the design of algorithmic procedures. In Section 3, we examine the issue
of what information can be made available and its cost, and we also exhibit some
important special cases when the objective and the underlying distribution functions
are such, that the equivalent deterministic programs can be conveniently and
inexpensively specified. We then turn to the three main solution strategies that we
consider here. They are based upon
(i) extensions of the revised simplex method,
(ii) inner approximations (generalized programming techniques),
(iii) outer approximations (min-max strategies).
In order to give the essence of each solution strategy, we consider first, in Section
4, a very simple case, viz., equivalent linear programming formulations for finding
the minimum of a convex piecewise linear function of one variable. In Sections 5,
6 and 7 we then go into each one as it applies to our class of stochastic programming
problems. The detailed design and implementation of two specific algorithms can
be found in a companion paper, Nazareth and Wets (1985).
2. Stochastic programs with recourse: Nonstochastic tenders
We consider stochastic (linear) programs of the type
find x e R ,n such that
Ax=b, x>~O, (2.1)
and z = E{c(w)x+ Q(x, w)} is minimized,
where Q is calculated by finding for given decision x and event ,w an optimal
recourse y c R"~, viz.
Q(x, w)= ~nf q(y, w)l Wy =p(w)- Tx. (2.2)
Here A(ml x nl), T(m2xnt), W(m2xn2) and b(m~) are given (fixed) matrices,
c(. )(n~) and p (.) (m2) are random vectors, y ~ q (y, (cid:12)9 ) : 2"R ~ R is a random finite-
valued convex function and C is a convex polyhedral subset of R ~, usually C ~n
-~" R +-.
Because W is nonstochastic one refers to (2.1) as having fixed .esruocer Tenders are
nonstochastic because T is fixed. With
c= E{c(w)} and 0(x)= E{Q(x, w)}
J.L. Nazareth, R.J.-B. Wets/Stochastic programs: Nonstochastic tenders
we obtain the equivalent deterministic form of (2.1):
find xeR ,n such that
Ax = b, x ~> 0, (2.3)
and z = cx+ Q(x) is minimized.
We assume that the random elements of the problem are such that all quantities
introduced are well-defined, with 0(x) finite, unless
P{wl(p(w)- Tx)~ W(C)}>O
where W(C) = {t = Wyly ~ C}, i.e. there is no feasible recourse with positive proba-
bility, in which case (~(x)= +cv. Detailed conditions have been made explicit in
Wets, 1974; extensions to the multistage and nonconvex cases hav~ been provided
by P. Olsen, 1976 and J.B. Hiriart-Urruty, 1978 respectively.
As background to the algorithmic development, we review the basic properties
of (2.3), proofs and further details can be found in Wets, 1974; see also Kall, 1982
for a compact treatment of stochastic prqgrams with complete recourse, i.e. when
W(C) = R "2 and consequently Q is everywhere finite.
2.4. Properties. The function Q is lower semicontinuous and convex. It is Lipschitz if
for (almost) all w, y~-~q(y, w) is Lipschitz. Also the set
K2 = {xl (~(x) < +~}
si a convex polyhedron that can be expressed as
K2 = {x I Dx>~ d}
for some matrix D and vector d. Moreover if the distribution of the random elements
of the problem is absolutely continuous then 0 is differentiable relative to K2.
Because q(., w) is Lipschitz rather than linear, the assertion about t~ being
Lipschitz does not follow directly from Theorem 7.7 of Wets, 1974 but can be
gathered from its proof, orsee Wets, 1972, for example.
In the case of nonstochastic tenders it is useful to consider another representation
of the deterministic equivalent program. Let us define X and other associated
quantities by
X : Tx, ~b(X, w): inf q(y, w)l Wy =p(w)-x
yeC
and
q'(x) = E{q,(x, w)}.