Table Of ContentComputational Mathematical Programming
NATO ASI Series
Advanced Science Institutes Series
A series presenting the results of activities sponsored by the NA TO Science Committee,
which aims at the dissemination of advanced scientific and technological knowledge,
with a view to strengthening links between scientific communities.
The Series is published by an international board of publishers in conjunction with the
NATO Scientific Affairs Division
A Life Sciences Plenum Publishing Corporation
B Physics London and New York
C Mathematical and D. Reidel Publishing Company
Physical Sciences Dordrecht, Boston and Lancaster
D Behavioural and Martinus Nijhoff Publishers
Social Sciences Boston, The Hague, Dordrecht and Lancaster
E Applied Sciences
F Computer and Springer-Verlag
Systems Sciences Berlin Heidelberg New York Tokyo
G Ecological Sciences
Series F: Computer and Systems Sciences Vol. 15
Computational Mathematical
Programming
Edited by
Klaus Schittkowski
Institut fur Informatik, Universitat Stuttgart, AzenbergstraBe 12
7000 Stuttgart 1, Federal Republic of Germany
Springer-Verlag Berlin Heidelberg New York Tokyo
Published in cooperation with NATO Scientific Affairs Division
Proceedings of the NATO Advanced Study Institute on Computational Mathematical
Programming held at Bad Windsheim, Federal Republic of Germany, July 23-August 2,
1984
ISBN-13: 978-3-642-82452-4 e-ISBN-13: 978-3-642-82450-0
001: 10.1007/978-3-642-82450-0
Library of Congress Cataloging in Publication Data.
NATO Advanced Study Institute on Computational Mathematical Programming (1984 : Bad Windsheim, Germany)
Computational mathematical programming. (NATO ASI series. Series F, Computer and systems sciences; vol. 15)
"Proceedings of the NATO Advanced Study Institute on Computational Mathematical Programming held at Bad
Windsheim. Federal Republic of Germany. July 23-August 2, 1984"-T. p. verso. "Published in cooperation with NATO
Scientific Affairs Division." 1. Programming (Mathematics)-Data processing-Congresses.1. Schittkowski, Klaus.
1946-.11. North Atlantic Treaty Organization. Scientific Affairs Division. III. Series: NATOASI series. Series F, Computer
and system sciences; no. 15. QA402.5.N3651984 519.7 85-8055
This work is subject to copyright. All rights are reserved, whether the Whole or part of the material is concerned,
specifically those of translating. reprinting, re-use of illustrations. broadcastings, reproduction by photocopying
machine or similar means, and storage in data banks. Under § 54 of the Gerrnan Copyright Law where copies are
made for other than private use. a fee is payable to "Verwertungsgesellschaft Wort", Munich.
© Springer-Verlag Berlin Heidelberg 1985
Softcover reprint of the hardcover 1s t edition 1985
Printing: Beltz Offsetdruck, Hemsbach; Bookbinding: J. Schaffer OHG, Grunstadt
2145/3140-54321 0
Preface
This book contains the written versions of main lectures presented at the Advanced Study Institute (ASI)
on Computational Mathematical Programming, which was held in Bad Windsheim, Germany F. R., from
July 23 to August 2, 1984, under the sponsorship of NATO. The ASI was organized by the Committee on
Algorithms (COAL) of the Mathematical Programming Society. Co-directors were Karla Hoffmann
(National Bureau of Standards, Washington, U.S.A.) and Jan Teigen (Rabobank Nederland, Zeist, The
Netherlands). Ninety participants coming from about 20 different countries attended the ASI and
contributed their efforts to achieve a highly interesting and stimulating meeting.
Since 1947 when the first linear programming technique was developed, the importance of
optimization models and their mathematical solution methods has steadily increased, and now plays a
leading role in applied research areas. The basic idea of optimization theory is to minimize (or maximize)
a function of several variables subject to certain restrictions. This general mathematical concept covers
a broad class of possible practical applications arising in mechanical, electrical, or chemical
engineering, physics, economics, medicine, biology, etc. There are both industrial applications (e.g.
design of mechanical structures, production plans) and applications in the natural, engineering, and
social sciences (e.g. chemical equilibrium problems, christollography problems).
In most cases, a theoretical foundation which predicts the numerical performance of a mathematical
programming algorithm, does not exist Nor can one at the present time prove theoretically that one
algorithm is best for a given application. As a consequence, the performance of optimization methods
must be evaluated by experiments. By defining a sample, i. e. by searching for suitable test problems,
and by designing a computational experiment, the efficiency, robustness, and reliability of an algorithm
can be investigated. Such experimental optimization will become even more important in future years as
modeling efforts attempt to solve problems related to evermore complex systems and consequently, the
mathematical algorithms used to solve these problems become more and more sophisticated. The
need to disseminate information about computational mathematical programming procedures was the
main impulse for organizing the Advanced Study Institute.
Consequently, the general purpose of the Advanced Study Institute was to link new algorithmic
developments for optimization models with practical applications. From the organizational point of view,
the ASI consisted of tutorilas, research seminars, and a software fair. The tutorials covered all major types
of optimization models currently used in practice. Leading experts gave an introduction to the subject,
provided any mathematical background, presented suitable algorithms stressing information about
their computer implementations, usage and numerical performance. The topics of the tutorials ranged
from linear and integer programming, networks, and various aspects of nonlinear programming to
optimal control, stochastic and nonsmooth optimization. They were accomplished by about 60 research
seminars of other participants leading to a more detailed insight in special individual research projects.
Moreover, information material about existing optimization software was displayed at the ASI e. g. in form
of user's guides or code descriptions.
The purpose of this book is to compose the tutorials and to offer them to the mathematical
programming community. It is expected that only minor knowledge of mathematical theory and
terminology is required to understand the most important opinions of the individual authors. In addition,
the book contains the main results of the software fair, i. e. a collection of about 50 code descriptions, with
the intention to disseminate information about existing optimization programs.
The organizer of the ASI is indepted to the NATO Science Committee for the generous financial
support. Other sponsors were the National Bureau of Standards of the U. S. Department of Commerce,
and the Deutscher Akademischer Austauschdienst. The success of the meeting is based on the quality
of the tutorials held at the ASI. Without any exception, the organizer acknowledges the extensive amount
of time the invited lecturers had to spend for preparing the tutorials and, in particular, for preparing the
manuscripts submitted for publication. Other individuals who deserve special thanks, are the co
directors Karla Hoffmann and Jan Teigen, furthermore Marlis and Jochem Zowe for their extensive
assistance concerning all local organizational arrangements.
K. Schittkowski
Contents
Integer Programming
EML.Beale . ............................................. .
Model Building in Linear and Integer Programming
H. P. Williams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
LP-Based Combinatorial Problem Solving
K Hoffman, M Padberg. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Network Optimization
R.R.Meyer . .............................................. . 125
Reflections on Geometric Programming
M J. Rijckaert E J. C. Walraven. . .. ................................ 141
Principles of Sequential Quadratic Programming Methods for Solving Nonlinear Programs
J. Stoer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
Model Building and Practical Aspects of Nonlinear Programming
P. E Gill, W Murray, M A. Saunders, M H. Wright . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
Comparative Performance Evaluation, Experimental Design, and Generation of Test Problems
in Nonlinear Optimization
FA. Lootsma. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
On Converting Optimal Control Problems into Nonlinear Programming Problems
0. Kraft . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
A Stochastic Approach to Global Optimization
A. H. G. Rinnooy Kan, C. G. E Boender, G. Th. Timmer . . . . . . . . . . . . . . . . . . . . . . . . 281
Algorithmic Procedures for Stochastic Optimization
R. J. B. Wets .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
Nondifferentiable Optimization
J.Zowe . 323
Parallel Computing in Optimization
R. B Schnabel . 357
Software for Mathematical Programming
K. Schittkowski . 383
INTEGER PROGRAMMING
E.M.L. Beale
Scicon Limited
Brick Close, Kiln Farm,
Milton Keynes, Great Britain.
SUMMARY
Following an introduction, which discusses the motivation for studying
integer programming, the relevance of computational complexity and the
relative merits of integer and dynamic programming, the branch and bound
method is introduced in general terms. Various types of global entity
to which it can be applied are introduced. These are integer variables,
semicontinuous variables, special ordered sets and chains of linked
ordered sets. A discussion of the algorithmic details follows. Finally,
various approaches to automatic model reformulation are discussed: this
seems to be the most important current area of integer programming research
1. INTRODUCTION
The requirement that the variables must take integer values is a
natural mathematical extension of linear programming, which is
known as Pure Integer Programming. Alternatively, one may think
of planning problems in which some questions require qualitative
answers - Yes or No, while others require quantitative answers
- How much. The Yes-or-No decisions can be coded as zero-one (or
binary) variables. And many of these problems can be formulated as
linear programming problems with the additional restriction that
some variables must be either zero or one. This is the problem of
Mixed Integer Programming if there are other variables that can take
continuous values. And it is sometimes useful to give some integer
variables upper bounds greater than one.
In any integer programming problem, a local optimum may not be a
global optimum. This property is shared by nonlinear problems in
which either the objective function to be maximized is not concave
or the feasible region is not convex or both. Markowitz and Manne
(1957) showed that some of these problems can be formulated as
mixed integer programming problems. But it has been found better
to introduce other types of global entity as described in Section 3
below, rather than to reduce these problems explicitly to mixed
integer programming. So the topiC of integer programming can be
considered as including global optimization of other models using
the methodology developed for integer programming.
NATOASISeri~ Vol.F15
Computational Mathematical Programming
Edited by K Schittkowski
© Springer-Verlag Berlin Heidelberg 1985
2
Integer programming is one approach to problems of combinatorial
optimization. Elementary considerations can teach some important
lessons, notably that it is unrealistic to expect to solve all
problems by waiting for computing to become cheaper or quicker. Any
method that requires the enumeration of all 2n possible combinations
of Yes and No decisions on n items may well work for,up to 10 or
even 20 items. But an increase of computer speed by a factor of 1000
only allows n to increase by 10 with this approach. The situation is
even worse if one has to enumerate all n! permutations of n items.
A value of n around 10 may then be acceptable, since 10! is only
about 3.6 x 106, but each further increase in n by a single unit
multiplies the amount of work by a factor of more than 10. Similar
remarks apply to methods that improve on complete enumeration but
save "only" 99.9% of the work.
These facts also illustrate two facts about model building. If the
essence of a problem is to make a selection from n items, or to
find an optimum ordering of them, then it may be important to
simplify the representation of each individual item, even if this
requires a fair amount of computing. And it may also be important
to suppress minor details from a combinatorial model designed to
suggest an overall strategy.
Most integer programming problems are such that the amount of work
needed to produce a guaranteed optimum solution may increase
exponentially with the problem size. The significance of this
proposition is limited by two facts: we usually wish to solve a
particular class of problems of a given size, and we can also often
accept a "good" solution that is "probably nearly optimum" if a
guaranteed optimum is unavailable. We should remember that many
practical problems with thousands of integer variables can be
completely solved in a few minutes. But the result about exponential
growth warns us that some problems can become much harder when the
problem size increases by an apparently modest amount. We have
already noted that complete enumeration of all 2n combinations
requires 1000 times as much work when n increases from 20 to 30.
Before studying integer programming in detail, it is worth
considering dynamic programming as an alternative approach to some
problems. For most practical purposes, dynamic programming can be
considered as the process of formulating and solving problems as
shortest-route problems. Its scope is much more restricted than
