Table Of Contentl..I
Smooth and Nonsmooth Case
0. GIORGI, A. GUERRAGGIO and J. THIERFELDER
Mathematics of Optimization:
Smooth and Nonsmooth Case
0. Giorgi
Thcuity of Economics
University of Pa via
Pa
via, Eta);'
A, Guerraggio
Facultyof Economics
Insubria Llriivcrsity
I/arose, italy
J. Thierfelder
Ins-Lila ft thcni a tics
Technical Universityilrnenau
.llmena u, Germany
2004
ELSFVIER
Amsterdam— Boston — Heidelberg — London — New York — Oxtord
Paris — San Diego — San Francisco — Singapore — Sydney — Tokyo
ELSEVIERWV. ELSKVyR Inc LJ.SVoiEO Lid ELSEVIOKLI.J
[ian Ikangrrlaartatraat 25 325 B Street The 84 llrrobaids Road
PA). Ran 2fl, liliHartU. Susie San Diego Laaagthrdln.Kid[iaagaoc, Lotad,onWC:X4Rft
Amsierdam,The Netheu-Iaads CA 92101-449.5, IsA C5rkwrl OX) 1(113, IlK OK
o2004Finest WV. All rzØirs rearmed
thiswork in protected oaadorccrpyrighr lay Elsevior wV ant rIm Irl: :rannl: sod c apply in its rae:
Piioaocogying
phLt copirnolnangin cnagtrrs may be made for pottorral are at aBetted by oaSenai laws Pemrissira-i
rslulae Publisher ama! payarsriar oft lee is reqaired for all other phororopyino. 3aclu:[rng on ilIai:r Sr syrit-aralic copying.
copying far adseriirrir.girr 700mcriLIrarII pcronaa. resale, all lbrrnaofdecarer.rrai deiLveat Special rates are assilable
for edaeaiaaiaa] rnstrraaioiaa [Stat wish in rsake p1 aoco[5aart fort ion-poole ardiaeaiional slatarsenri uSC
l'rtrrni.roiorai may Ire sortyjia dwec[]y Il-am Riglcs Depastmrre ii: PafrerL UK phroae('t44) 1865 84311415, faa
[+44) 1865 053333, e.rinall: pam tsrs:,elnesarur,com. Resjuieora rosy a'sea be corarglened eta-lose via the Flaeoin
psp(htip./umw.cloverercorm'looate)tcriainninun).
lii -jan may clear peaaniasioiisand rraalcepasinenmat2rrnngh rhoCopysiglt[ Ciraranco (sorer Iris
Rosrwrace! Jrtvc, Daa:smn, lolA01923, USA.phone:(-][ (9Th)7.508401), 1—il[9Th) 7504741, and in theUK
rho Copytgla[ Liconatrao Agency Rapid Cearaisee Stealer[CLARCS), 90 Tortcaliana Cairn Road,liStidrun Wi Ii
[5(2. LI; phone lr44)207451 1515; Isa- 44) 7611 55110 tit[sercos,-rniesmayhavea localrepmpaphic ngtrir
agancy 0r paynimia.
Dcr,salive Works
Tables of coatrenra may lana rea,sroilocnd fee mrmnai circailatior, SrI perniosiossolrhe Publisher a reqraand for ercernal
renarne or dir-Inbiasiomef ascii marerial. Pormiaatoaa attlat Parb[5aher is required for all omlaer wsrkt! inc liatiny
andisaaalalirann.
Eiaoarenic Storagc-ina Usaga
Pemraission otilw Publiahea is reqatred to oiorrornoechecironically arty ataleelaS eoaitaiazd itt ibis worE aieladiatgarey
elsaçter tan pail of a chaptna
Eaceçae as collared abort, icr pail eat iiai mark amy be rnprodniced, storer! on a atbirinal oyntaan or tranrrmerd Ira sri forTs
irebsarty arenas, e3ectronle, rarcharacal, ph docc'psaag. recarduag or othenisne, wiflttios print ftatnitsiassi &Lbe
Publiaber
Addresspernosoonorequests to:EIsc',ot'(s RightsDepaetaitau. atrIteIt and e-mailadalmesara solidahm'a.
Notice
NoreapoaSatrya aaataaed by that Pulaldier fee tiny injuryactOrsdarnagu to perarcia or pmperty ana citric of çuonloeia
Iialasbry, negligence oronherwiae, arfrrura asay tat oroirmalion tat any rnmlroda5 prodacins, inalraethoaa or dot ecarainad
tar rIse snatenal Iteecia Encase at rnjnid advances an ide medical scimacco, :r particular, oadeeeitderar vedSoatian of
d'agooses and drug dosages sheatH be made.
tsarrdiciaa: 2004
Libraiy afCorr Catalaopno ci l'aobl:rists'rn I)ato
A catalog record :s availoble larson [hr 1.ibraey olCoogtcss.
Eleitith Umary (:nlogarag or Publacaiioa [lola
A catalogue accord in assaitabk front Ilteinaitith Library.
lIEN: 0-444-505304
line paper used iii this publieaiioaa meets the reqeipenararo of ANSTIWSO 139.48-1991 iPernianercr oF Paper)
Prbaled ha TIre Netherlands
PREFACE
The present book is primarily dedicated to the study of the optimal-
ity conditions for a nonlinear programming problem, also known as a
mathematical programming problem. Indeed one of the main subjects of
mathematical optimization (a relatively modern branch of applied math-
ematics which has grown at an exponential rate, both from a theoretical
point of view and on the side of applications) is the study of a class of ex-
tremum problems where the objective function f is to be optimized under
some restrictions, usually in the form of equalities and/or inequalities.
As an example for a nonlinear programming problem in Rit, we mean
an extremum problem of the following type:
f(x) (P)
Mm
x€S
where S = {x X g2(x) 0, i 1, ...,m; hk(X) = 0, k 1, ...,p},
X C f, gj, hk : X —p JR.
If all the functions involved in (P) are linear, we speak of linear pro-
gramming problems. A more general version of (P) is one where the
objective function I is a vector-valued function. This latter case (espe-
cially important from an economic point of view) will be treated in the
last chapter of the book.
Problems similar to (F) arise in several contexts: the building and inter-
preting of economic models; the study of various technological processes;
the development of optimal choices in finance; operations research; man-
agement science; production processes; transportation models; statistical
decisions, etc. Therefore it is of the utmost importance to study the exis-
tence of solutions for (P) and to study the effective methods (i.e. numerical
algorithms) for finding solutions of the problem. This second aspect of the
study of (F) will not be treated here; we shall be concerned only with
the study of the optimality conditions for (P), i.e. of the necessary and
VII'
sufficient conditions such that a point e S is a solution (local or global)
for (P).
We have decided to conduct our analysis in the Euclidean space lAY1'
(there is only one exception in Section 13 of Chapter III, where problems
defined in infinite dimensional spaces are briefly mentioned). We have
renounced to a more general mathematical setting, because of two main
reasons:
a) Mathematical programming in Jfltt is a relatively self-contained field
within the larger class of optimization problems (which includes sta-
tic and dynamic optimization problems, calculus of variations, optimal
control problems, etc.). The reader can quickly become familiar with
the main topics; only some previous knowledge in linear algebra, mathe-
matical analysis and convex analysis is necessary in order to understand
the subject.
b) Many optimization problems in practice are defined on or can be
approximated by a problem defined on AT1'; moreover, the numerical
solution of optimization problems in general spaces by means of com-
puters usually requires a former embedding of the problem in a finite
dimensional space.
Besides the classical optimality conditions for a problem (P), where all
the functions involved are assumed differentiable or even continuously dif-
ferentiable (i.e. "smooth"), special attention has been given to the study
of optimality conditions when the functions of (P) are "nonsmooth". The
title itself of this book obviously points out this aspect. Moreover, we have
dedicated an entire chapter (the last one) to the study of so-called vec-
tor optimization problems, which are rarely considered in books treating
mathematical programming problems. Indeed, in several practical problems
we have to optimize simultaneously, on the basis of certain criteria, sev-
eral objective functions (e.g. costs of production, production times, degrees
of mechanization and automatization, etc.). Such problems are obviously
more complex than problem (P) and, accordingly, their mathematical han-
dling is more complex.
ix
In preparing this book a special effort has been made to obtain a self-
contained treatment of the subjects; so we hope that this may be a suitable
book as a textbook and a reference (every chapter ends with an extensive
bibliography that should be useful to the reader for further investigations).
We do not claim to have produced an "advanced" book on mathemat-
ical programming; this book is addressed to those researchers, graduate
students and post-graduates wishing to begin the study of nonlinear pro-
gramming problems, in view of further deepenings. We think and hope
that this book may also be useful to theoretical economists, engineers and
applied researchers involved in such problems.
The book is divided into six chapters:
Chapter 1 briefly deals with the notion of nonlinear programming prob-
lems and with basic notations, conventions, definitions and results pertain-
ing to set theory, mathematical analysis and linear algebra.
Chapter Il deals with convex sets, convex functions and generalized
convex functions; indeed in the study of optimality conditions for (P)
and in the study of the so-called "dual problems" of (P), a fundamental
role is played by the convexity or also by the generalized convexity of the
functions involved. This explains why we have dedicated an entire chapter
to the following topics: topological properties of convex sets, separation
of convex sets, theorems of the alternative, properties of convex functions
and generalized convex functions.
Chapter III covers the classical optimality conditions for a nonlinear
programming problem in the smooth case, i.e. when all the functions in-
volved are differentiable or continuously differentiable. The Fritz John and
Karush-Kuhn-Tucker optimality conditions are developed, both for equal-
ity and/or inequality constrained problems. Both first-order and second-
order optimality conditions are developed; some fundamental material on
(first-order) constraint qualifications is presented. Sufficient optimality
conditions are obtained by use of various generalizations of convex func-
tions. The last section of the chapter deals with so-called "saddle point
problems".
Chapter IV deals with the study of optimality conditions for (P) in the
nonsmooth case, i.e. when the functions involved in (F) are not differ-
x
entiable in the classical sense. Thus, different kinds of generalized dif-
ferentiability are introduced and compared. After some general remarks
about classical differentiability notions, we present some generalized di-
rectional derivatives and generalized subdifferential mappings and discuss
these notions in connection with their application in mathematical opti-
mization. In this context, the notion of "local cone approximation" of
sets, introduced in Chapter II, is of great importance. Using this notion,
we develop the so-called K-directional derivatives and K-subdifferentials
and give an abstract approach for the development of general necessary
optimality conditions for the nonsmooth case.
In Chapter V we give a survey about the most important duality con-
cepts in mathematical optimization. For different kinds of optimization
problems we discuss properties of their dual problems and develop different
duality assertions. Starting with the well-known duality in linear optimiza-
tion, we describe the Wolfe duality for convex optimization problems, the
Lagrange duality in connection with saddle-point assertions and the per-
turbation concept, using the notion of conjugate functions. In this manner
we shall see that the concepts will be generalized step by step.
Chapter VI is concerned with vector nonlinear programming problems.
The presentation of the main optimality notions (weak efficient points,
efficient points, proper efficient points) is followed by providing theorems
giving conditions about the existence of said points. A classical prob-
lem of vector optimization is then discussed by means of the Fritz John
and Kuhn-Tucker conditions, both for the smooth and nonsmooth (Lip-
schitzian) case. Links with a scalar optimization problem are emphasized
(especially by means of some generalized convex functions, described in
Chapter II) through various scalarization techniques and in a vector version
of the Wolfe dual problem presented in Chapter IV.
The idea to write the present book emerged several years ago, during
a trip of the two Italian authors to the Department of Mathematics of
the Technical University of llmenau (Germany), in those days a relatively
small university department among the woods of Thuringia ("Das grüne
Thüringen), where Prof. Karl-Heinz Elster had founded the now renowned
xi
review "Optimization" and created an important center for studies in
mathematical programming.
We recognize our intellectual debt toward many people from whom
we have borrowed many ideas and results; indeed we owe much to the
authors cited in the bibliographical list at the end of each chapter. We
want, however, to pay a particular respect to the memory of Prof. Karl-
Heinz Elster.
We want to express our gratitude to the Italian Ministry for University,
Scientific and Technological Research for the financial support provided.
Finally we want to thank Drs. A. Sevenster of Elsevier BV (Amster-
dam) for his patience and precious collaboration and express our most deep
and sincere thanks to Mrs. Anita Klooster for her diligent and hard work
in typing our almost unreadable manuscript.
The book has been planned and discussed throughout its various phases
of developments by all three authors. However, Chapters I, II and Ill have
been written by G. Giorgi; Chapters IV and V by J. Thierfelder; Chapter VI
by A. Guerraggio.
The authors.
CHAPTER I. INTRODUCTION
1.1. Optimization Problems
In the analysis of numerous mathematical problems (arising in Eco-
nomics, Statistics, Operations Research, Engineering, Physics, etc.) situa-
tions often occur where a decison maker must make a decision in order to
rule a system in an optimal way (on the basis of specified criteria), a model
containing several alternative possibilities and which is, at least partially,
under the control of the decision maker.
Such a problem can be considered as an optimization problem; "op-
timization" is a catch-all term for maximization or minimization and lies
at the heart of several applied sciences, but perhaps the more natural
application of optimization theory is in the field of economic sciences. In-
deed optimization (or better, optimization subject to constraints) has been
considered by many authors as defining the essential nature of Economics.
We quote only the famous classical statement of Robbins (1932, p. 16):
"Economics is the science which studies human behaviour as a relationship
between ends and scarce means which have alternative uses".
The first mathematicians dealing with optimization problems were Fer-
mat, Euler, the Bernoullis, Lagrange and others in connection with the de-
velopment of Calculus in the 17th and 18th centuries. However, the first
results in the modern theory of mathematical optimization were presented
by the Russian Nobel prize winner L.V. Kantorovich and the American
mathematician G.B. Dantzig, in 1939 and 1947 respectively. In 1951 the
American mathematicians H.W. Kuhn and A.W. Tucker published impor-
tant theoretical results in the theory of mathematical optimization, as
extensions of the classical methods of Euler-Lagrange for the solution of
optimization problems with equality constraints and in presence of differ-
entiability assumptions.
Prior to the work of Kuhn and Tucker and in the same direction,
