Fundamentals of Convex Analysis

267 Pages·2004·20.425 MB·English

by  Jean-Baptiste Hiriart-Urruty, Claude Lemaréchal

Grundlehren Text Editions
Editors
A.Chenciner S.S.Chern B.Eckmann
P.delaHarpe F.Hirzebruch N.Hitchin
1. Horrnander M.-A.Knus A.Kupiainen
G.Lebeau M.Ratner D.Serre Ya.G.Sinai
N.J. A.Sloane J.Tits B.Totaro A.Vershik
M.Waldschmidt
ManagingEditors
M.Berger J.Coates S.R.S.Varadhan
Springer-Verlag Berlin Heidelberg GmbH
J
ean -Baptiste Hiriart -Urruty
Claude Lemarechal
Fundatnentals
of Convex Analysis
With 66 Figures
i
Springer
Jean-Baptiste Hiriart-Urruty
Departement de Mathematiques
Universite Paul Sabatier
118, route de Narbonne
31062 Toulouse
France
e-mail: [email protected]
Claude Lemarechal
INRIA, Rhone Alpes
ZIRST
655, avenue de l'Europe
38330 Montbonnot
France
e-mail: [email protected]
Library ofCongress Cataloging-in-Publication Data
Hiriart-Unuty, Jean-Baptiste, 1949-
Fundamentals of convex ana1ysis / Jean-Baptiste Hiriart-Unuty, Claude Lemarecbal.
p. em. -- (Gnmdlehren text editions)
Includes bibliographical references and index.
ISBN 978-3-540-42205-1 ISBN 978-3-642-56468-0 (eBook)
DOI 10.1007/978-3-642-56468-0
1. Convex timctions. 2. Convex sets. 3. Mathematical ana1ysis. 1. Lemarecbal, Claude,
1944-II. Title. III. Series.
QA331.5 .H58 2001
51S.8--de21
2001053271
Corrected Second Printing 2004
Mathematics Subject Classification (2000):
21-01, 26B05, 52A41, 26Axx, 49Kxx, 49Mxx, 49-01, 93B30, 90CxX
ISSN 1618-2685
ISBN 978-3-540-42205-1
This work is subject to copyright. All rights are reserved, whether the whole or part
of the material is concerned, specifically the rights of translation, reprinting, reuse
of illustrations, re citation, broadcasting, reproduction on microfilm or in any other
way, and storage in data banks. Duplication of this publication or parts thereof is
permitted only under the provisions of the German Copyright Law of September 9,
1965, in its current version, and permission for use must always be obtained from
Springer-Verlag. Violations are liable for prosecution under the German Copyright
Law.
springeronline.com
© Springer-Verlag Berlin Heidelberg 2001
Originally published by Springer-Verlag Berlin Heidelberg New York in 2001
The use of general descriptive names, registered names, trademarks, etc. in this
publication does not imply, even in the absence of a specific statement, that such
names are exempt from the relevant protective laws and regulations and therefore
free for general use.
Cover design: Erich Kirchner, Heidelberg
Typeset by the authors using a Springer TJlX macro package
Printed on acid-free paper 41/3142/LK -5 43 210
Preface
This book is an abridged version of our two-volume opus Convex Analysis and
Minimization Algorithms[18],aboutwhichwehavereceivedverypositivefeedback
from users, readers, lecturers ever since it was published - by Springer-Verlag in
1993. Its pedagogical qualities were particularly appreciated, in the combination
witharather advanced technical material.
Now [18]hasadual butclearly definednature:
- anintroductiontothebasicconceptsinconvexanalysis,
- a study of convex minimization problems (with an emphasis on numerical algo-
rithms),
and insists on their mutual interpenetration. Itis our feeling that the above basic
introduction ismuch needed in thescientificcommunity. This isthemotivation for
thepresent edition, our intention being tocreate atool usefulto teachconvexanal
ysis.Wehave thus extracted from [18] its "backbone"devoted toconvex analysis,
namely ChapsIII-VIandX.Apart from some local improvements,thepresent text
ismostly acopyofthecorrespondingchapters. Themaindifference isthatwehave
deleted materialdeemed tooadvancedforanintroduction, ortoocloselyattached to
numerical algorithms.
Further, we haveincluded exercises, whose degree ofdifficulty issuggested by
0, I or2stars *.Finally,theindexhasbeenconsiderablyenriched.
Just asin [18], each chapteris presented as a "lesson", in the sense of our old
masters,treating of agiven subject in its entirety. After an introduction presenting
orrecalling elementarymaterial, thereare fivesuchlessons:
- A Convexsets(correspondingtoChap.III in[18]),
- B Convexfunctions (Chap.IVin[18]),
- C Sublinearityandsupport functions (Chap.V),
- D Subdifferentialsinthefinite-valuedcase (VI),
- E Conjugacy (X).
Thus,wedonotgobeyondconjugacy.Inparticular,subdifferentiabilityofextended
valued functions is intentionally left aside. This allows a lighter book, easier to
master andtogothrough. The samereason ledustoskipduality which, besides, is
morerelated tooptimization.Readers interestedbythesetopicscanalwaysreadthe
relevantchapters in[18](namely Chaps XIandXII).
VI Preface
During the French Revolution, the writer of a bill on public instruction com
plained: "Le defaut ou la disette de bons ouvrages elementaires a ete, jusqu'a
present, un des plus grands obstacles qui s'opposaient au perfectionnement de
I'instruction. La raison de cette disette, c'est quejusqu'apresent les savants d'un
merite eminent ont,presque toujours, prefere fagloire d'eleverl'edifice de fasci
a
ence fapeine d'en eclairer l'entree:" Our main motivation here is precisely to
"light theentrance" of the monumentConvex Analysis.This istherefore not aref
erence book, to bekept on the shelfbyexperts who already knowthe building and
can findtheir way through it; it is far more a book for the purpose of learning and
teaching. Wecall above all onthe intuition of thereader, and our approach isvery
gradual. Nevertheless, we keep constantly in mind the suggestion of A. Einstein:
"Everything should be made as simple as possible, but not simpler". Indeed, the
content is by no means elementary,and will be hard for areadernot possessing a
firmmastery ofbasic mathematical skill.
Wecould not completely avoid cross-references between the various chapters;
butformanyofthem,themotivationistosuggestanintellectuallinkbetween appar
ently independentconcepts,rather than atechnical need forprevious results. More
thanatree,ourapproach evokes aspiral, made upofloosely interrelated elements.
Many sections are set in smaller characters. They are by no means reserved to
advancedmaterial; rather,theyaretheretohelpthereaderwithillustrativeexamples
andsideremarks,thathelp tounderstand adelicate point,orprepare some material
tocome ina subsequentchapter. Roughly speaking, sections in smallercharacters
can be compared to footnotes, used to avoid interrupting the flowof the develop
ment;itcan behelpful toskipthem during adeeper reading, withpencil andpaper.
They can often be considered as additional informal exercises, useful to keep the
readeralert.
Thenumberingofsections restartsat Iineachchapter,andchapternumbers are
dropped inareferencetoanequation orresult from within thesamechapter.
Toulouseand Grenoble,
March 2001 J.-B.Hiriart-Urruty,C.Lemarechal,
1"Thelackorscarcityofgood,elementarybookshasbeen,untilnow,oneofthegreatest
obstacles in the wayofbetterinstruction. The reason for thisscarcity is that,until now,
scholars ofgreat merit havealmost alwayspreferred theglory ofconstructing themon
ument of science over the effort of lighting its entrance." D.Guedj:La Revolution des
Savants,Decouvertes,GallimardSciences(1988)130- 131.
Contents
Preface ... ... ....... ... ......... ........ ...... ....... .. .... .... V
o.
Introduction: Notation,ElementaryResults 1
1 Some FactsAbout LowerandUpperBounds 1
2 TheSetofExtended RealNumbers 5
3 Linear andBilinear Algebra.................................. 6
4 Differentiation inaEuclidean Space........................... 9
5 Set-ValuedAnalysis 12
6 Recalls onConvexFunctions oftheRealVariable 14
Exercises ...................................................... 16
A. ConvexSets ... ............................................. 19
1 Generalities 19
1.1 DefinitionandFirstExamples.......................... 19
1.2 Convexity-Preserving Operations onSets ................ 22
1.3 ConvexCombinations andConvexHulls. ................ 26
1.4 Closed ConvexSetsandHulls 31
2 ConvexSets Attached toaConvexSet ......................... 33
2.1 TheRelativeInterior ............... 33
2.2 TheAsymptotic Cone 39
2.3 Extreme Points ...................................... 41
2.4 Exposed Faces 43
3 Projection ontoClosedConvexSets ........................... 46
3.1 TheProjection Operator 46
3.2 Projection ontoaClosed ConvexCone .................. 49
4 Separation andApplications.................................. 51
4.1 Separation Between ConvexSets ....................... 51
4.2 FirstConsequences oftheSeparation Properties 54
- Existence ofSupporting Hyperplanes.................. 54
- OuterDescription ofClosedConvexSets 55
- ProofofMinkowski'sTheorem....................... 57
- BipolarofaConvexCone 57
4.3 TheLemma ofMinkowski-Farkas ...................... 58
5 Conical ApproximationsofConvexSets 62
VIII Contents
5.1 ConvenientDefinitionsofTangentCones ................ 62
5.2 TheTangentandNormalConestoaConvexSet .......... 65
5.3 SomePropertiesofTangentandNormal Cones ........... 68
Exercises ...................................................... 70
B. ConvexFunctions .. ......................................... 73
1 BasicDefinitionsandExamples 73
1.1 The DefinitionsofaConvexFunction ................... 73
1.2 SpecialConvexFunctions:AffinityandC1osedness 76
- Linear andAffineFunctions.......................... 77
- ClosedConvexFunctions............................ 78
- OuterConstruction ofClosed ConvexFunctions......... 80
1.3 FirstExamples 82
2 Functional OperationsPreservingConvexity.................... 87
2.1 OperationsPreservingClosedness ...................... 87
2.2 Dilations andPerspectivesofaFunction ................. 89
2.3 InfimalConvolution. ................................. 92
2.4 Image ofaFunction Under aLinearMapping 96
2.5 ConvexHullandClosedConvexHullofaFunction .. 98
3 Local andGlobalBehaviourofaConvexFunction 102
3.1 Continuity Properties 102
3.2 Behaviour atInfinity 106
4 First- andSecond-OrderDifferentiation 110
4.1 DifferentiableConvexFunctions 110
4.2 NondifferentiableConvexFunctions 114
4.3 Second-OrderDifferentiation II5
Exercises 117
C. SublinearityandSupportFunctions. ........................... 121
1 Sublinear Functions 123
1.1 DefinitionsandFirstProperties 123
1.2 SomeExamples 127
1.3 TheConvexCone ofAllClosedSublinearFunctions 131
2 TheSupport Function ofaNonempty Set 134
2.1 Definitions,Interpretations 134
2.2 BasicProperties 136
2.3 Examples 140
3 CorrespondenceBetweenConvexSetsandSublinear Functions 143
3.1 TheFundamentalCorrespondence 143
3.2 Example:Norms andTheirDuals, Polarity 146
3.3 CalculuswithSupportFunctions 151
3.4 Example:SupportFunctions ofClosedConvexPolyhedra .. 158
Exercises ................................ 161
Contents IX
D. SubdifferentialsofFiniteConvex Functions 163
1 The Subdifferential:DefinitionsandInterpretations 164
1.1 FirstDefinition:Directional Derivatives 164
1.2 Second Definition:MinorizationbyAffineFunctions 167
1.3 Geometric ConstructionsandInterpretations 169
2 LocalProperties oftheSubdifferential 173
2.1 First-OrderDevelopments 173
2.2 Minimality Conditions 177
2.3 Mean-ValueTheorems 178
3 FirstExamples 180
4 Calculus Rules withSubdifferentials 183
4.1 PositiveCombinationsofFunctions 183
4.2 Pre-CompositionwithanAffineMapping 184
4.3 Post-CompositionwithanIncreasing ConvexFunction
ofSeveralVariables 185
4.4 Supremum ofConvexFunctions 188
4.5 Image ofaFunction Under aLinearMapping 191
5 Further Examples 194
5.1 Largest EigenvalueofaSymmetric Matrix 194
5.2 Nested Optimization 196
5.3 Best Approximation of aContinuous Function on aCom-
pactInterval 198
6 The Subdifferential asaMultifunction 199
6.1 MonotonicityProperties oftheSubdifferential 199
6.2 Continuity Properties oftheSubdifferential 201
6.3 SubdifferentialsandLimits ofSubgradients 204
Exercises 205
E. Conjugacyin Convex Analysis................................. 209
I TheConvexConjugate ofaFunction 211
1.1 Definition andFirstExamples 211
1.2 Interpretations 214
1.3 FirstProperties 216
- Elementary CalculusRules 216
- The BiconjugateofaFunction 218
- Conjugacy andCoercivity 219
1.4 SubdifferentialsofExtended-ValuedFunctions 220
2 Calculus RulesontheConjugacy Operation 222
2.1 Image ofaFunction Under aLinearMapping 222
2.2 Pre-CompositionwithanAffineMapping 224
2.3 SumofTwoFunctions 227
2.4 Infimaand Suprema 229
2.5 Post-CompositionwithanIncreasing ConvexFunction 231
3 VariousExamples 233
3.1 TheCramer Transformation 234
X Contents
3.2 TheConjugateofConvexPartiallyQuadratic Functions 234
3.3 PolyhedralFunctions 235
4 DifferentiabilityofaConjugateFunction 237
4.1 First-OrderDifferentiability 238
4.2 LipschitzContinuityoftheGradientMapping 240
Exercises 241
BibliographicalComments. ....................................... 245
The Founding FathersoftheDiscipline 249
References ..................................................... 251
Index 253