Table Of ContentJochen Werner
Optimization
Theory and Applications
Advanced Lectures
in Mathematics
Edited by Gerd Fischer
Jochen Werner
Optimization. Theory and Applications
Manfred Denker
Nonparametric Statistics
Jochen Werner
Optimization
Theory and Applications
M
Friedr. Vieweg & Sohn Braunschweig/Wiesbaden
CIP-Kurztlteleufnehme der Deutschen Bibliothek
Werner, Jochen:
Optimizetion - theory end epplications I Jochen
Werner. -,Breunschweig; Wiesbeden: Vieweg,
1984.
(Advences lectures in metherneticsl
ISBN-13: 978-3-528'()8594'() e-ISBN-13:978-3-322-84035-6
DOl: 10.1007/978-3-322-84035-6
1984
All rights reserved
© Friedr. Vieweg & Sohn Verlegsgesellscheft mbH. Breunschweig 1984
No pert of this publicetion mey be reproduced, stored in e retrievel system or trensmitted in
eny form or by eny meens, electronic, mechenicel, photocopying, recording or otherwise,
without prior permission of the copyright holder.
Produced,by IVD, Welluf b. Wiesbeden
ISBN-13:97S-3-52S-GS594-0
v
PREFACE
This book is a slightly augmented version of a set of lec
tures on optimization which I held at the University of Got
tingen in the winter semester 1983/84. The lectures were in
tended to give an introduction to the foundations and an im
pression of the applications of optimization theory. Since in
finite dimensional problems were also to be treated and one
could only assume a minimal knowledge of functional analysis,
the necessary tools from functional analysis were almost com
pletely developed during the course of the semester. The most
important aspects of the course are the duality theory for
convex programming and necessary optimality conditions for
nonlinear optimization problems; here we strive to make the
geometric background particularly clear. For lack of time and
space we were not able to go into several important problems
in optimization - e.g. vector optimization, geometric program
ming and stability theory.
I am very grateful to various people for their help in pro
ducing this text. R. Schaback encouraged me to publish my lec
tures and put me in touch with the Vieweg-Verlag. W. BrUbach
and O. Herbst proofread the manuscript; the latter also pro
duced the drawings and assembled the index. I am indebted to
W. LUck for valuable suggestions for improvement. I am also
particularly grateful to R. Switzer, who translated the German
text into English. Finally I wish to thank Frau P. Trapp for
her Gare and patience in typing the final version.
Gottingen, June 1984 Jochen Werner
VI
CONTENTS
§ 1 INTRODUCTION, EXAMPLES, SURVEY
1.1 Optimization problems in elementary g-eometry
1.2 Calculus of variations 10
1.3 Approximation problems 14
1.4 Linear programming 19
1.5 Optimal Control 24
1.6 Survey 27
1.7 Literature 28
§ 2 LINEAR PROGRAMMING
2.1 Definition and interpretation of the dual
program ,30
2.2 The FARKAS-Lemma and the Theorem of
CARATHEODORY 37
2.3 The strong duality theorem of linear
programming 44
2.4 An application: relation between inradius
and width of a polyhedron 50
2.5 Literature 55
§ 3 CONVEXITY 'IN LINEAR AND NORMED LINEAR SPACES
3.1 Separating convex sets in linear spaces 56
3.2 Separation of convex sets in norrned
linear spaces 67
3.3 Convex functions 73
3.4 Literature 84
§ 4 CONVEX OPTIMIZATION PROBLEMS
4.1 Examples of convex optimization problems 85
4.2 Definition and motivation of the dual pro
gram. The weak duality theorem 90
4.3 Strong duality, KUHN-TUCKER saddle point
theorem 100
4.4 Quadratic programming 126
4.5 Literature 140
VII
§ 5 NECESSARY OPTIMALITY CONDITIONS
5.1 GATEAUX and FRECHET Differential 142
5.2 The Theorem of LYUSTERNIK 150
5.3 LAGRANGE multipliers. Theorems of
KUHN-TUCKER and F. JOHN type 164
5.4 Necessary optimality conditions of first
order in the calculus of variations and
in optimal control theory 177
5.5 Necessary and sufficient optimality
conditions of second order 196
5.6 Literature 206
§ 6 EXISTENCE THEOREMS FOR SOLUTIONS OF
OPTIMIZATION PROBLEMS
6.1 Functional analytic existence theorems 208
6.2 Existence of optimal controls 217
6.3 Literature· 223
BIBLIOGRAPHY 225
SYMBOL INDEX 229
SUBJECT INDEX 230
Bei dem studio der Mathema
tik kann wohl nichts star
keren Trost bei Unverstand
lichkeiten gewahren, als
daB es sehr viel schwerer
ist eines anderen Meditata
zu verstehen, als selbst zu
medi tieren.
G. Chr. Lichtenberg
§ INTRODUCTION, EXAMPLES, SURVEY
An optimization problem consists in minimizing a function
f M ~ lR on a given set M. x E M is a solution of
(P) minimize f{x) on M,
if f{x) S f{x) for all x E M. f is often called cost or objec
tive function, M the set of feasible solutions. The value of
the optimization problem (P) is
*
inf{f{x) x E M} if M {II
inf (P) := {
+ co if M {II.
If (P) has a solution, then we shall always write min (P) in
stead of inf {Pl. The maximization of a function g : M ~ lR on
M can of course be reduced to the problem (P) by introducing
f := - o.
In this introduction we shall begin by giving several ex
amples of optimization problems. We shall see that widely dif
fering types of problems can be formulated in our general set
ting.At the same time we hope to whet the reader's appetite
for concrete problems.
1.1 QEtimization problems in elementary geometry
According to CANTOR [12, p. 228] the first optimization p~
lem in the history of mathematics occurs in EUCLID's Elements,
Book VI, Theorem 27. The problem is essentially the following:
2
1} Find a point E on the side BC of the triangle ABC such that
the parallelogram ADEF with vertices D resp. F lying on the
sides AB resp. AC has maximal area.
c The solution is obviously
given by choosing E to be the
midpoint of BC. In fact for
arbitrary E on BC with
X : = BE/BC the area of the oor
responding parallelogram is
=
area (ADEF) 2X(1-X}area(ABC}
and this function is maximal
A 8 for X = 1/2.
HERON (ca. 100 BC) gave a solution to the following problem:
2} On a given line find a pOint C such that the sum of the
distances to the pOints A and B is minimal.
If A and B lie on opposite
sides of the given line, then
obviously the intersection of
the line with the segment AB
is the desired point~ other-
, wise one reflects A in the
/,' C
: line getting A' and determines
, , ,,/
... C as the intersection of the
A' line with the segment A'B.
Hence: a light ray reflected in the line with angle of inci-
dence and angle of reflection equal takes the shortest pos
sible path from A to B via C. Or: if one wants to go from A
to B on the shortest possible route and on the way fetch a
pail of water from a (straight) river, then one must solve the
same problem (COURANT-ROBBINS [16]).
The origins of the classical isoperimetric problem
3} among all plane closed curves of a given constant length
(resp. "figures" of a given perimeter, isoperimetric fig
ures) find the one which encloses the greatest possible
area
go back at least to ZENODORES (between 200 BC and 90 AD).
From him we have the following assertions (try to prove these
3
with elementary or analytical geometry):
i) Given two regular polygons of the same perimeter the
one with the larger number of sides has the greater
enclosed area.
ii) If a circle has the same perimeter as a regular poly
gon, then the circle encloses the greater area.
iii) Among all isoperimetric triangles with the same base
the isosceles triangle has the greatest area.
iv) Among all isoperimetric polygons with a given number
of sides the regular one has the greatest area.
Thus the circle has a greater area than any isoperimetric pol
ygon. STEINER (1836) showed: if the isoperimetric problem 3)
has a solution, then it is necessarily a circle. The existence
of a solution was only proved much later. While the existence
is evident for the problems 1 land 2), it is by no means evi
dent for the isoperimetric problem 3) (not to mention the spa
tial analog).
After appropriate generalization the following geometric
problem is also of considerable interest to economists:
4) Suppose given three points in the plane. Find a point such
that the sum of the distances from this point to -the three
given pOints is a minimum.
This problem appears to have been formulated for the first
time by FERMAT in 1629. The names TORRICELLI- and STEINER are
also associated with the problem. The generalization to m
points in lRn is often called the FERMAT-WEBER problem:
Suppose given m pairwise d-istinct. 'points aJ, ••• ,arnE lRn
and positive weights w1"" ,wm" Find a point x E lRn which
minimizes the function f : m.Il ... lR defined by
m .
f(x) := L w. I x-a1. I .
i=1 1.
Here I I denotes the euclidean norm, i.e. Iyl := (\ nL Y i2l\ 1/2 .
i=1
