Table Of ContentDe Gruyter Series in Nonlinear Analysis
and Applications 13
Editor in Chief
Jürgen Appell, Würzburg, Germany
Editors
Catherine Bandle, Basel, Switzerland
Alain Bensoussan, Richardson, Texas, USA
Avner Friedman, Columbus, Ohio, USA
Karl-Heinz Hoffmann, Munich, Germany
Mikio Kato, Kitakyushu, Japan
Umberto Mosco, Worchester, Massachusetts, USA
Louis Nirenberg, New York, USA
Katrin Wendland, Augsburg, Germany
Alfonso Vignoli, Rome, Italy
Peter Kosmol
Dieter Müller-Wichards
Optimization in
Function Spaces
With Stability Considerations in Orlicz Spaces
De Gruyter
Mathematics Subject Classification 2010: Primary: 46B20, 46E30, 52A41; Secondary: 41-02,
41A17,41A25,46B10,46T99,47H05,49-02,49K05,49K15,49K40,49M15,90C34.
ISBN 978-3-11-025020-6
e-ISBN 978-3-11-025021-3
ISSN 0941-813X
LibraryofCongressCataloging-in-PublicationData
Kosmol,Peter.
Optimization in function spaces with stability considerations on
Orliczspaces/byPeterKosmol,DieterMüller-Wichards.
p. cm. (cid:2) (De Gruyter series in nonliniear analysis and appli-
cations;13)
Includesbibliographicalreferencesandindex.
ISBN978-3-11-025020-6(alk.paper)
1. Stability (cid:2) Mathematical models. 2. Mathematical opti-
mization. 3. Orlicz spaces. I. Müller-Wichards, D. (Dieter),
1949(cid:2) II.Title.
QA871.K8235 2011
5151.392(cid:2)dc22
2010036123
BibliographicinformationpublishedbytheDeutscheNationalbibliothek
TheDeutscheNationalbibliothekliststhispublicationintheDeutscheNationalbibliografie;
detailedbibliographicdataareavailableintheInternetathttp://dnb.d-nb.de.
”2011WalterdeGruyterGmbH&Co.KG,Berlin/NewYork
Typesetting:Da-TeXGerdBlumenstein,Leipzig,www.da-tex.de
Printingandbinding:Hubert&Co.GmbH&Co.KG,Göttingen
(cid:3)Printedonacid-freepaper
PrintedinGermany
www.degruyter.com
Preface
In this text we present a treatise on optimization in function spaces. A concise but
thorough account of convex analysis serves as a basis for the treatment of Orlicz
spaces and Variational Calculus as a contribution to non-linear and linear functional
analysis.
As examples may serve our discussion of stability questions for families of opti-
mization problems or the equivalence of strong solvability, local uniform convexity,
andFréchetdifferentiabilityoftheconvexconjugate, whichdoprovidenewinsights
andopenupnewwaysofdealingwithapplications.
A further contribution is a novel approach to the fundamental theorems of Varia-
tional Calculus, where pointwise (finite-dimensional) minimization of suitably con-
vexified Lagrangians w.r.t. the state variables x and x˙ is performed simultaneously.
Convexification in this context is achieved via quadratic supplements of the La-
grangian. Thesesupplementsareconstantontherestrictionsetandthusleadtoequiv-
alentproblems.
Itisouraimtopresentanessentially self-contained bookonthetheoryofconvex
functionsandconvexoptimizationinBanachspaces. Weassumethatthereaderisfa-
miliarwiththeconceptsofmathematicalanalysisandlinearalgebra. Someawareness
oftheprinciplesofmeasuretheorywillturnouttobehelpful.
ThemethodsmentionedaboveareappliedtoOrliczspacesinatwofoldsense: not
onlydoweconsider optimization andinparticular normapproximation problemsin
Orlicz spaces but we also use these methods to develop a complete theory of Orlicz
spacesforσ-finitemeasures. Wealsoemploythestabilityprinciplesdevelopedinthis
textinordertoestablishoptimalityforsolutionsoftheEuler–Lagrangeequationsof
certainnon-convexvariationalproblems.
Overview
In Chapter 1 we present the classical approximation theory for L1 and C(T) em-
bedded into the category of Orlicz spaces. In particular we present the theorems of
Jackson (zeros of error function) and Bernstein (error estimates) and their extension
toOrliczspaces. Fordifferentiable (1-D)Youngfunctionsthenon-linear systemsof
equations that arise from linear modular and norm approximation problems are dis-
cussed. FortheirsolutiontherapidlyconvergentnumericalmethodsofChapter4can
beused.
Chapter2isdevotedtoPolya-typealgorithmsinOrliczspacestodetermineabest
Chebyshev(orL1)approximation. Theideaistoreplaceanumericallyillconditioned
problem, affected by non-uniqueness and non-differentiability, with a sequence of
vi Preface
well-behavedproblems. Theclosednessofthealgorithmisalreadyguaranteedbythe
pointwiseconvergenceofthesequenceofYoungfunctions. Thisisduetothestability
principles presented in Chapter 5. Convergence estimates for the corresponding se-
quenceofLuxemburgnormscanbegleanedfromtheYoungfunctionsinthediscrete
andcontinuouscase.
TheseestimatesinturncanbeutilizedtoregularizethePolyaalgorithminorderto
achieveactualconvergencetoatwo-stagesolution,asubjectthatisdiscussedwithin
a general framework in more detail in Chapter 5. For sequences of Young functions
withseparationpropertytheconvergenceofdiscreteapproximationstothestrictap-
proximationofRiceisshown.
The last application in this chapter is of a somewhat different nature: it is well
knownthatmanyapplicationscanberestatedaslinearprogrammingproblems. Semi-
infinite optimization problems are a generalization with potentially infinitely many
restrictions(forwhichafairlylargeliteratureexists). Wesolvetheproblembymaking
useofthestabilityresultsdevelopedinthisbook: weapproximatetherestrictionset
byasequenceofsmoothrestrictionsetsmakinguseoftheLagrangemechanismand
the existence of Lagrange multipliers, which in turn is guaranteed by corresponding
regularityconditions.
InChapter3wedevelopthetheoryofconvexfunctionsandconvexsetsinnormed
spaces. We stress the central role of the (one-sided) directional derivative and – by
usingitsproperties–derivenecessaryandsufficientconditionsforminimalsolutions
ofconvexoptimizationproblems.
For later use we consider in particular convex functions on finite dimensional
spaces. It turns out that real-valued convex functions are already continuous, and
differentiableconvexfunctionsarealreadycontinuouslydifferentiable. Thelatterfact
is used later to show that the Gâteaux derivative of a continuous convex function is
already demi-continuous (a result that is used in Chapter 8 to prove that a reflexive
anddifferentiableOrliczspaceisalreadyFréchetdifferentiable).
WediscusstherelationshipbetweentheGâteauxandFréchetderivativesofacon-
vexfunctioninnormedspacesandgivetheproofofanotverywell-knowntheoremof
PhelpsontheequivalenceofthecontinuityoftheGâteauxderivativeandtheFréchet
differentiabilityofaconvexfunction.
BasedontheHahn–Banachextensiontheoremweproveseveralvariantsofsepara-
tiontheorems(Mazur,Eidelheit,strict). Usingtheseseparationtheoremsweshowthe
existenceofthesubdifferentialofacontinuousconvexfunction,andderiveanumber
of properties of a convex function together with its convex conjugate (in particular
Young’sequalityandthetheoremofFenchel–Moreau).
The separation theorems are then used to prove the Fechel-duality theorem and –
in an unconventional way – by use of the latter theorem the existence of minimal
solutions of a convex function on a bounded convex subset of a reflexive Banach
space(theoremofMazur–Schauder).
Preface vii
ThelastsectionofthischapterisdevotedtoLagrangemultiplierswhereweshow
theirexistence–againusingtheseparationtheorem–andtheLagrangedualitytheo-
remwhichisusedinChapter7toprovethe(general)AmemiyaformulafortheOrlicz
norm.
In Chapter 5 we turn our attention to stability principles for sequences (and fam-
ilies) of functions. We show the closedness of algorithms that are lower semi-con-
tinuously convergent. Using this result we derive stability theorems for monotone
convergence. Among the numerous applications of this result is the computation of
the right-handed derivative of the maximum norm on C(T) which in turn is used to
provide a simple proof for the famous Kolmogoroff criterion for a best Chebyshev
approximation(seeChapter1).
Ourmainobjectiveinthischapteristoconsiderpointwiseconvergentsequencesof
convexfunctions. BasedonanextensionoftheuniformboundednessprincipleofBa-
nach, formulated originally for linear functionals, to families of convex functions, it
turnsoutthatpointwiseconvergentsequencesofconvexfunctionsarealreadycontin-
uouslyconvergent. Wearethusenabledtoconsidersequencesofconvexoptimization
problems(f ,M )wherethefunctionsf convergepointwisetof andthesetsM in
n n n n
thesenseofKuratowskitoM,whileclosednessofthealgorithmispreserved. Inthe
finitedimensionalcasewecanevenguaranteetheexistenceofpointsofaccumulation,
providedthatthesetofminimalsolutionsof(f,M)isnon-emptyandbounded.
In the next section of this chapter we consider two-stage solutions where – un-
der certain conditions – we can guarantee the actual convergence of the sequence
of minimal solutions and characterize it. Among the methods being considered is
differentiation w.r.t. the family parameter: for example, it turns out that the best Lp-
approximationsconvergeforp→1tothebestL1-approximationofmaximalentropy.
‘Outerregularizations’inthespiritofTikhonovarealsopossible,providedthatcon-
vergenceestimatesareavailable.
In the final section of this chapter we show that a number of stability results for
sequences of convex functions carry over to sequences of monotone operators. If P
isanon-expansiveFéjercontractionthenI−P ismonotone. Suchoperatorsoccurin
the context of smoothing of linear programming problems where the solution of the
latter(non-smooth)problemappearsasasecondstagesolutionofthesolutionsofthe
smoothoperatorsequence.
InChapter6weintroducetheOrliczspaceLΦ forgeneralmeasuresandarbitrary
(not necessarily finite) Young functions Φ. We discuss the properties and relations
ofYoungfunctionsandtheirconjugatesandinvestigatethestructureofOrliczspaces
equipped with the Luxemburg norm. An important subspace is provided by the clo-
sureofthespaceofstepfunctionsMΦ. ItturnsoutthatMΦ = LΦ ifandonlyifΦ
satisfiesanappropriateΔ -condition. Moreover,ifΦdoesnotsatisfyaΔ -condition
2 2
thenLΦcontainsaclosedsubspaceisomorphicto(cid:3)∞,inthecaseofafinitenotpurely
atomicmeasurethisisomorphyisevenisometric. Thesestatementsareduetothethe-
viii Preface
oremofLindenstrauss–TsafririforOrliczsequencespacesandatheoremofTurettfor
non-atomicmeasures. Theseresultsbecomeimportantinourdiscussionofreflexivity
inChapters7and8.
InChapter7weintroducetheOrlicznormwhichturnsouttobeequivalenttothe
Luxemburgnorm. UsingJensen’sintegralinequalityweshowthatnormconvergence
already implies convergence in measure. We show that the convex conjugate of the
modular fΦ is fΨ, provided that Ψ is the conjugate of the Young function Φ. An
important consequence is that the modular is always lower semi-continuous. These
facts turn out to be another ingredient in proving the general Amemiya formula for
theOrlicznorm.
ThemainconcernofthischapteristocharacterizethedualspaceofanOrliczspace:
itturnsoutthatforfiniteΦandσ-finitemeasuresweobtain(MΦ)∗ = LΨ,provided
that Ψ is the conjugate of Φ. Reflexivity of an Orlicz space is then characterized by
anappropriate(dependingonthemeasure)Δ -conditionforΦandΨ.
2
Based on the theorem of Lusin and a theorem of Krasnosielski, establishing that
the continuous functions are dense in MΦ, we show that, if Φ and Ψ are finite, and
T is a compact subset of Rm, and μ the Lebesgue measure, then MΦ is separable,
where separability becomes important in the context of greedy algorithms and the
Ritzmethod(Chapter8).
WeconcludethechapterbystatingandprovingthegeneralAmemiyaformulafor
theOrlicznorm.
Based on the results of Chapter 6 and 7 we now (in Chapter 8) turn our attention
to the geometry of Orlicz spaces. Based on more general considerations in normed
spacesweobtainfornotpurelyatomicmeasuresthatMΦ issmoothifandonlyifΦ
is differentiable. For purely atomic measures the characterization is somewhat more
complicated.
Our main objective is to characterize strong solvability of optimization problems
where convergence of the values to the optimum already implies norm convergence
oftheapproximationstotheminimalsolution. Itturnsoutthatstrongsolvabilitycan
be geometrically characterized by the local uniform convexity of the corresponding
convex functional (provided that the term local uniform convexity is appropriately
defined,whichwedo). Moreover,weestablishthatinreflexiveBanachspacesstrong
solvability is characterized by the Fréchet differentiability of the convex conjugate,
providedthatbothareboundedfunctionals. Theseresultsarebasedinpartonapaper
of Asplund and Rockafellar on the duality of A-differentiability and B-convexity of
conjugatepairsofconvexfunctions,whereB isthepolarofA.
BeforeweapplytheseresultstoOrliczspaces,weturnourattentiontoE-spacesin-
troducedbyFanandGlicksberg,whereeveryweaklyclosedsubsetisapproximatively
compact. ABanach space is an E-space if and only if it is reflexive, strictly convex,
and satisfies the Kadec–Klee property. In order to establish reflexivity the theorem
ofJamesisapplied. AnothercharacterizationisgivenbyAnderson: X isanE-space
Preface ix
∗
ifandonlyifX isFréchetdifferentiable. ThelinkbetweenFréchetdifferentiability
andKadec–KleepropertyisthenprovidedthroughthelemmaofShmulian[42].
With these tools at hand we can show that for finite not purely atomic measures
Fréchet differentiability of an Orlicz space already implies its reflexivity. The main
theoremgivesin17equivalentstatementsacharacterizationofstrongsolvability,lo-
cal uniform convexity, and Fréchet differentiability of the dual, provided that LΦ is
reflexive. Itisremarkablethatallthesepropertiescanalsobeequivalentlyexpressed
by the differentiability of Φ or the strict convexity of Ψ. In particular it turns out
thatinOrliczspacesthenecessaryconditionsofstrictconvexityandreflexivityforan
E-spacearealreadysufficient.
Weconcludethegeometricalpartofthischapterbyadiscussiononthedualityof
uniform convexity and uniform differentiability of Orlicz spaces based on a corre-
sponding theorem by Lindenstrauss. We restate a characterization by Milne of uni-
formly convex Orlicz spaces equipped with the Orlicz norm and present an example
of a reflexive Orlicz space, also due to Milne, that is not uniformly convex but (ac-
cordingtoourresults)thesquareofitsnormislocallyuniformlyconvex. Following
A. Kaminska we show that uniform convexity of the Luxemburg norm is equivalent
toδ-convexityofthedefiningYoungfunction.
Inthelastsectionwediscussanumberofunderlyingprinciplesforcertainclasses
ofapplications:
• Tikhonov regularization: this method was introduced for the treatment of ill-
posedproblems(ofwhichthereisawholelot). Theconvergenceofthemethod
wasprovedbyLevitinandPolyakforuniformlyconvexregularizing function-
als. We show here that locally uniformly convex regularizations are sufficient
for that purpose. As we have given a complete description of local uniform
convexityinOrliczspaceswecanstatesuchregularizingfunctionalsexplicitly.
• Ritzmethod: theRitzmethodplaysanimportantroleinmanyapplications(e.g.
FEM-methods). It is well known that the Ritz procedure generates a minimiz-
ing sequence. Actual convergence of the solutions on each subspace is only
achievediftheoriginalproblemisstronglysolvable.
• Greedy algorithms have indeed drawn a growing attention and experienced a
rapid development in recent years (see e.g. Temlyakov). The aim is to arrive
at a ‘compressed’ representation of a function in terms of its dominating ‘fre-
quencies’. TheconvergenceproofmakesuseoftheKadec–Kleepropertyofan
E-space.
Viewing these 3 ‘applications’ at one glance it turns out that there is an inherent
relationship: localuniformconvexity,strongsolvability,andKadec–Kleepropertyare
3facetsofthesamepropertywhichwehavecompletelydescribedinOrliczspaces.
Inthelastchapterwedescribeanapproachtovariationalproblems,wherethesolu-
tionsappearaspointwise(finitedimensional)minimaforfixedtofthesupplemented
