Table Of ContentReiner Horst· Hoang Tuy
Global
Optimization
Deterministic Approaches
With 55 Figures
Springer-Verlag
Berlin Heidelberg GmbH
Professor Or. Reiner Horst
University of Trier
Oepartment of Mathematics
P.O.Box 3825
0-5500 Trier, FRG
Professor Or. Hoang Tuy
Vien Toan Hoc
Institute of Mathematics
P.O.Box 631, BO-HO
10000 Hanoi, Vietnam
ISBN 978-3-662-02600-7 ISBN 978-3-662-02598-7 (eBook)
DOI 10.1007/978-3-662-02598-7
This work is subject to copyright. Ali rights are reserved, whether the whole or part ofthe
material is concemed, specifically the rights oftranslation, reprinting, reuse ofillustrations,
recitation, broadcasting, reproduction on microfilms or in other ways, and storage in data
banks. Duplication ofthis publication or parts thereof is ooly permitted under the provisi
ons ofthe German Copyright Law ofSeptember 9,1965, in its version ofJune 24, 1985, and a
copyright fee must always be paid. Violations fali under the prosecution act ofthe German
Copyright Law.
© Springer-Verlag Berlin Heidelberg 1990
OriginaIly published by Springer-Verlag Berlin Heidelberg New York Tokyo in 1990
Softcover reprint of the hardcover 1s t edition 1990
The use ofregistered names, trademarks, etc. in this publication does not imply, even in the
absence of a specific statement, that such names are exempt from the relevant protective
laws and regulations and therefore free for general use.
2142/7130-543210
PREFACE
The enormous practical need for solving global optimization problems coupled
with a rapidly advancing computer technology has allowed one to consider problems
which a few years ago would have been considered computationally intractable. As a
consequence, we are seeing the creation of a large and increasing number of diverse
algorithms for solving a wide variety of multiextremal global optimization problems.
The goal of this book is to systematically clarify and unify these diverse
approaches in order to provide insight into the underlying concepts and their pro
perties. Aside from a coherent view of the field much new material is presented.
By definition, a multiextremal global optimization problem seeks at least one
global minimizer of a real-valued objective function that possesses different local
minimizers. The feasible set of points in IRn is usually determined by a system of
inequalities. It is well known that in practically all disciplines where mathematical
models are used there are many real-world problems which can be formulated as
multi extremal global optimization problems.
Standard nonlinear programming techniques have not been successful for solving
these problems. Their deficiency is due to the intrinsic multiextremality of the
formulation and not to the lack of smoothness or continuity, for often the latter
properties are present. One can observe that local· tools such as gradients, subgra
dients, and second order constructions such as Hessians, cannot be expected to yield
more than local solutions. One finds, for example, that a stationary point is often
detected for which there is even no guarantee of local minimality. Moreover, deter
mining the local minimality of such a point is known to be NP-hard in the sense of
computational complexity even in relatively simple cases. Apart from this deficiency
in the local situation, classical methods do not recognize conditions for global
optimality.
VI
For these reasons global solution methods must be significantly different from
standard nonlinear programming techniques, and they can be expected to be - and
are - much more expensive computationally. Throughout this book our focus will
be on typical procedures that respond to the inherent difficulty of multiextremality
and which take advantage of helpful specific features of the problem structure. In
certain sections, methods are presented for solving very general and difficult global
problems, but the reader should be aware that difficult large scale global optimiza
tion problems cannot be solved with sufficient accuracy on currently available
computers. For these very general cases our exposition is intended to provide useful
tools for transcending local optimality restrictions, in the sense of providing valuable
information about the global quality of a given feasible point. Typically, such in
formation will give upper and lower bounds for the optimal objective function value
and indicate parts of the feasible set where further investigations of global optima
lity will not be worthwhile.
On the other hand, in many practical global optimizations, the multiextremal
feature involves only a small number of variables. Moreover, many problems have
additional structure that is amenable to large scale solutions.
Many global optimization problems encountered in the decision sciences, engi
neering and operations research have at least the following closely related key
properties:
(i) convexity is present in a limited and often unusual sense;
(ii) a global optimum occurs within a subset of the boundary of the feasible set.
With the current state of the art, these properties are best exploited by determini
stic methods that combine analytical and combinatorial tools in an effective way.
We find that typical approaches use techniques such as branch and bound, relaxa
tion, outer approximation, and valid cutting planes, whose basic principles have long
appeared in the related fields of integer and combinatorial optimization as well as
convex minimization. We have found, however, that application of these fruitful
ideas to global optimization is raising many new interesting theoretical and compu
tational questions whose answers cannot be inferred from previous successes. For
example, branch and bound methods applied to global optimization problems
generate infinite processes, and hence their own convergence theory must be deve
loped. In contrast, in integer programming these are finite procedures, and so their
convergence properties do not directly apply. Other examples involve important
VII
results in convex minimization that reflect the coincidence of local and global solu
tions. Here also one cannot expect a direct application to multi extremal global
minimization.
In an abundant class of global optimizations, convexity is present in a reverse
sense. In this direction we focus our exposition on the following main topics:
(a) minimization of concave functions subject to linear and convex constraints (i.e.,
"concave minimization");
(b) convex minimization over the intersection of convex sets and complements of
convex sets (i.e., "reverse convex programming"); and
(c) global optimization of functions that can be expressed as a difference of two convex
functions (i.e., "d.c.-programming").
Another large class of global optimization that we shall discuss in some detail has
been termed "Lipschitz Programming", where now the functions in the formulation
are assumed to be Lipschitz continuous on certain subsets of their domains. Al
though neither of the aforementioned properties (i)-(ii) is necessarily satisfied in
Lipschitz problems, much can be done here by applying the basic ideas and tech
niques which we shall develop for the problem classes (a), (b), and (c) mentioned
above.
Finally, we also demonstrate how global optimization problems are related to solving
systems of equations and/or inequalities. As a by-product, then, we shall present
some new solution methods for solving such systems.
The underlying purpose of this book is to present general methods in such a way
as to enhance the derivation of special techniques that exploit frequently encoun
tered additional problem structure. The multifaceted approach is manifested occa
sionally in some computational results for these special but abundant problems.
However, at the present stage, these computational results should be considered as
preliminary.
The book is divided into three main parts.
Part A introduces the main global optimization problem classes we study, and
develops some of their basic properties and applications. It then discusses the funda
mental concepts that unify the various general methods of solution, such as outer
approximation, concavity cuts, and branch and bound.
VIII
Part B treats concave minimization and reverse convex programming subject to
linear and reverse convex constraints. In this part we present additional detail on
specially structured problems. Examples include decomposition, projection, separa
bility, and parametric approaches.
In Part C we consider rather general global optimization problems. We study
d.c.-programming and Lipschitz optimization, and present our most recent attempts
at solving more general global optimization problems. In this part, the speciali
zations most naturally include biconvex programming, indefinite "all-quadratic"
optimization, and design centering as encountered in engineering design.
Each chapter begins with a summary of its contents.
The technical prerequisites for this book are rather modest, and are within reach
of most advanced undergraduate university programs. They include a sound know
ledge of elementary real analysis, linear algebra, and convexity theory. No familia
rity with any other branch of mathematics is required.
In preparing this book, we have received encouragement, advice, and suggestions
from a large group of individuals. For this we are grateful to Faiz AI-Khayyal,
Harold P. Benson, Neil Koblitz, Ken Kortanek, Christian Larsen, Janos Pinter,
Phan Thien Thach, Nguyen van Thoai, Jakob de Vries, Graham Wood, and to
several other friends, colleagues and students. We are indebted to Michael Knuth for
drawing the figures.
We acknowledge hospitality and / or financial support given by the following
organizations and institutions: the German Research Association (DFG), the
Alexander von Humboldt Foundation, the Minister of Science and Art of the State
Lower Saxony, the National Scientific Research Center in Hanoi, the Universities of
Oldenburg and Trier.
Our special thanks are due to Frau Rita Feiden for the efficient typing and
patient retyping of the many drafts of the manuscript.
Finally, we thank our families for their patience and understanding.
December 1989 Reiner Horst Hoang Tuy
CONTENTS
PART A: INTRODUCTION AND BASIC TECHNIQUES 1
CHAPTER I. SOME IMPORTANT CLASSES OF GLOBAL
OPTIMIZATION PROBLEMS 3
1. Global Optimization 3
2. Concave Minimization 9
2.1. Definition and Basic Properties 9
2.2. Brief Survey of Direct Applications 12
2.3. Integer Programming and Concave Minimization 14
2.4. Bilinear Programming and Concave Minimization 19
2.5. Complementarity Problems and Concave Minimization 23
2.6. Max-Min Problems and Conc1J,ve Minimization 25
3. D.C. Programming and Reverse Convex Constraints 26
3.1. D.C. Programming: Basic Properties 26
3.2. D.C. Programming: Applications 32
3.3. Reverse Convex Constraints 36
3.4. Canonical D.C. Programming Problems 39
4. Lipschitzian Optimization and Systems of Equations
and Inequalities 42
4.1. Lipschitzian Optimization 42
4.2. Systems of Equations and Inequalities 46
CHAPTER II. OUTER APPROXIMATION 51
1. Basic Outer Approximation Method 51
2. Outer Approximation by Convex Polyhedral Sets 56
3. Constraint Dropping Strategies 65
4. On Solving the Subproblems (Qk) 68
4.1. Finding an Initial Polytope Dl and its Vertex Set VI 69
4.2. Computing New Vertices and New Extreme Directions 71
4.3. Identifying Redundant Constraints 82
x
CHAPTER m. CONCAVITY CUTS 85
1. Concept of a Valid Cut 85
2. Valid Cuts in the Degenerate Case 91
3. Convergence of Cutting Procedures 95
4. Concavity Cuts for Handling Reverse Convex Constraints 100
5. A Class of Generalized Concavity Cuts 104
6. Cuts Using Negative Edge Extensions 108
CHAPTER IV. BRANCH AND BOUND 111
1. A Prototype Branch and Bound Method 111
2. Finiteness and Convergence Conditions 121
3. Typical Partition Sets and their Refinement 132
3.1. Simplices 132
3.2. Rectangles and Polyhedral Cones 137
4. Lower Bounds 139
4.1. Lipschitzian Optimization 140
4.2. Vertex Minima 141
4.3. Convex Subfunctionals 142
4.4. Duality 153
4.5. Consistency 158
5. Deletion by Infeas:ibility 163
6. Restart; Branch and Bound Algorithm 169
PART B: CONCAVE MINIMIZATION 173
CHAPTER V. CUTTING METHODS 175
1. A Pure Cutting Algorithm 175
1.1. Valid Cuts and a Sufficient Condition
for Global Optimality 176
1.2. Outline of the Method 181
2. Facial Cut Algorithm 184
2.1. The Basic Idea 184
2.2. Finding an Extreme Face of D Relative to M 186
2.3. Facial Valid Cuts 190
2.4. A Finite Cutting Algorithm 192
