Table Of ContentCAMBRIDGEMONOGRAPHSON
APPLIEDANDCOMPUTATIONAL
MATHEMATICS
SeriesEditors
M.ABLOWITZ,S.DAVIS,J.HINCH,
A.ISERLES,J.OCKENDON,P.OLVER
20 Greedy Approximation
The Cambridge Monographs on Applied and Computational Mathematics series
reflectsthecrucialroleofmathematicalandcomputationaltechniquesincontemporary
science. The series publishes expositions on all aspects of applicable and numeri-
cal mathematics, with an emphasis on new developments in this fast-moving area of
research.
State-of-the-art methods and algorithms as well as modern mathematical descrip-
tions of physical and mechanical ideas are presented in a manner suited to graduate
researchstudentsandprofessionalsalike.Soundpedagogicalpresentationisaprereq-
uisite.Itisintendedthatbooksintheserieswillservetoinformanewgenerationof
researchers.
Acompletelistofbooksintheseriescanbefoundat
www.cambridge.org/mathematics
Recenttitlesincludethefollowing:
8. Schwarz–Christoffelmapping,TobinA.Driscoll&LloydN.Trefethen
9. High-ordermethodsforincompressiblefluidflow,M.O.Deville,P.F.Fischer
&E.H.Mund
10. Practicalextrapolationmethods,AvramSidi
11. Generalized Riemann problems in computational fluid dynamics, Matania
Ben-Artzi&JosephFalcovitz
12. Radialbasisfunctions,MartinD.Buhmann
13. IterativeKrylovmethodsforlargelinearsystems,HenkvanderVorst
14. SimulatingHamiltoniandynamics,BenedictLeimkuhler&SebastianReich
15. Collocation methods for Volterra integral and related functional differential
equations,HermannBrunner
16. Topologyforcomputing,AfraJ.Zomorodian
17. Scattereddataapproximation,HolgerWendland
18. Moderncomputerarithmetic,RichardBrent&PaulZimmermann
19. Matrixpreconditioningtechniquesandapplications,KeChen
20. Greedyapproximation,VladimirTemlyakov
21. Spectralmethodsfortime-dependentproblems,JanHesthaven,SigalGottlieb
&DavidGottlieb
22. The mathematical foundations of mixing, Rob Sturman, Julio M. Ottino &
StephenWiggins
23. Curveandsurfacereconstruction,TamalK.Dey
24. Learningtheory,FelipeCucker&DingXuanZhou
25. Algebraicgeometryandstatisticallearningtheory,SumioWatanabe
26. Apracticalguidetotheinvariantcalculus,ElizabethLouiseMansfield
Greedy Approximation
VLADIMIR TEMLYAKOV
UniversityofSouthCarolina
CAMBRIDGE UNIVERSITY PRESS
Cambridge,NewYork,Melbourne,Madrid,CapeTown,
Singapore,SãoPaulo,Delhi,Tokyo,MexicoCity
CambridgeUniversityPress
TheEdinburghBuilding,CambridgeCB28RU,UK
PublishedintheUnitedStatesofAmericabyCambridgeUniversityPress,NewYork
www.cambridge.org
Informationonthistitle:www.cambridge.org/9781107003378
(cid:2)c CambridgeUniversityPress2011
Thispublicationisincopyright.Subjecttostatutoryexception
andtotheprovisionsofrelevantcollectivelicensingagreements,
noreproductionofanypartmaytakeplacewithoutthewritten
permissionofCambridgeUniversityPress.
Firstpublished2011
PrintedintheUnitedKingdomattheUniversityPress,Cambridge
AcatalogrecordforthispublicationisavailablefromtheBritishLibrary
LibraryofCongressCataloginginPublicationdata
Temlyakov,Vladimir,1953–
Greedyapproximation/VladimirTemlyakov.
p. cm.–(Cambridgemonographsonappliedandcomputationalmathematics;20)
ISBN978-1-107-00337-8(hardback)
1. Approximationtheory. I. Title. II. Series.
QA221.T455 2011
(cid:3)
518.5–dc23
2011025053
ISBN978-1-107-00337-8Hardback
CambridgeUniversityPresshasnoresponsibilityforthepersistenceor
accuracyofURLsforexternalorthird-partyinternetwebsitesreferredtoin
thispublication,anddoesnotguaranteethatanycontentonsuchwebsitesis,
orwillremain,accurateorappropriate.
Contents
Preface pageix
1 Greedyapproximationwithregardtobases 1
1.1 Introduction 1
1.2 SchauderbasesinBanachspaces 6
1.3 Greedybases 15
1.4 Quasi-greedyandalmostgreedybases 33
1.5 WeakGreedyAlgorithmswithrespecttobases 39
1.6 Thresholdingandminimalsystems 43
1.7 Greedy approximation with respect to the trigonometric
system 47
1.8 Greedy-typebases;directandinversetheorems 58
1.9 Somefurtherresults 63
1.10 Systems L -equivalenttotheHaarbasis 68
p
1.11 Openproblems 76
2 Greedyapproximationwithrespecttodictionaries:
Hilbertspaces 77
2.1 Introduction 77
2.2 Convergence 84
2.3 Rateofconvergence 89
2.4 Greedyalgorithmsforsystemsthatarenotdictionaries 97
2.5 Greedy approximation with respect to λ-quasi-orthogonal
dictionaries 101
2.6 Lebesgue-typeinequalitiesforgreedyapproximation 111
2.7 Saturationpropertyofgreedy-typealgorithms 122
2.8 Somefurtherremarks 135
2.9 Openproblems 141
3 Entropy 143
3.1 Introduction:definitionsandsomesimpleproperties 143
v
vi Contents
3.2 Finitedimensionalspaces 144
3.3 Trigonometricpolynomialsandvolumeestimates 151
3.4 Thefunctionclasses 165
3.5 Generalinequalities 168
3.6 Somefurtherremarks 175
3.7 Openproblems 182
4 Approximationinlearningtheory 183
4.1 Introduction 183
4.2 Somebasicconceptsofprobabilitytheory 189
4.3 Improperfunctionlearning;upperestimates 206
4.4 Properfunctionlearning;upperestimates 235
4.5 Thelowerestimates 253
4.6 Applicationofgreedyalgorithmsinlearningtheory 270
5 Approximationincompressedsensing 277
5.1 Introduction 277
5.2 Equivalenceofthreeapproximationpropertiesofthe
compressedsensingmatrix 283
5.3 Constructionofagoodmatrix 287
5.4 Dealingwithnoisydata 294
5.5 First results on exact recovery of sparse signals;
theOrthogonalGreedyAlgorithm 298
5.6 Exact recovery of sparse signals; the Subspace Pursuit
Algorithm 305
5.7 Onthesizeofincoherentsystems 314
5.8 RestrictedIsometryPropertyforrandommatrices 327
5.9 Somefurtherremarks 330
5.10 Openproblems 332
6 Greedyapproximationwithrespecttodictionaries:
Banachspaces 334
6.1 Introduction 334
6.2 TheWeakChebyshevGreedyAlgorithm 340
6.3 Relaxation;co-convexapproximation 347
6.4 Freerelaxation 350
6.5 Fixedrelaxation 354
6.6 Thresholdingalgorithms 359
6.7 Greedyexpansions 363
6.8 Relaxation; X-greedyalgorithms 378
6.9 Incoherentdictionariesandexactrecovery 381
6.10 Greedy algorithms with approximate evaluations
andrestrictedsearch 385
Contents vii
6.11 An application of greedy algorithms for the discrepancy
estimates 390
6.12 Openproblems 404
References 405
Index 415
