Table Of ContentPrediction, Learning, and Games
This important new text and reference for researchers and students in
machinelearning,gametheory,statistics,andinformationtheoryoffers
thefirstcomprehensivetreatmentoftheproblemofpredictingindividual
sequences.Unlikestandardstatisticalapproachestoforecasting,predic-
tionofindividualsequencesdoesnotimposeanyprobabilisticassump-
tion on the data-generating mechanism. Yet, prediction algorithms can
be constructed that work well for all possible sequences, in the sense
that their performance is always nearly as good as the best forecasting
strategyinagivenreferenceclass.
The central theme is the model of prediction using expert advice, a
generalframeworkwithinwhichmanyrelatedproblemscanbecastand
discussed.Repeatedgameplaying,adaptivedatacompression,sequential
investment in the stock market, sequential pattern analysis, and several
other problems are viewed as instances of the experts’ framework and
analyzed from a common nonstochastic standpoint that often reveals
new and intriguing connections. Old and new forecasting methods are
describedinamathematicallyprecisewayinordertocharacterizetheir
theoreticallimitationsandpossibilities.
Nicolo` Cesa-BianchiisProfessorofComputerScienceattheUniversity
of Milan, Italy. His research interests include learning theory, pattern
analysis,andworst-caseanalysisofalgorithms.Heisanactioneditorof
thejournalMachineLearning.
Ga´borLugosihasbeenworkingonvariousproblemsinpatternclassifi-
cation,nonparametricstatistics,statisticallearningtheory,gametheory,
probability, and information theory. He is coauthor of the monographs
AProbabilisticTheoryofPatternRecognitionandCombinatorialMeth-
ods of Density Estimation. He has been an associate editor of various
journals, including the IEEE Transactions of Information Theory, Test,
ESAIM:ProbabilityandStatistics,andStatisticsandDecisions.
Prediction, Learning, and Games
NICOLO` CESA-BIANCHI
Universita` degliStudidiMilano
GA´ BOR LUGOSI
UniversitatPompeuFabra,Barcelona
cambridge university press
Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, São Paulo
Cambridge University Press
The Edinburgh Building, Cambridge cb2 2ru,UK
Published in the United States of America by Cambridge University Press, New York
www.cambridge.org
Informationo nthi stitle :www.cambri dge.org/9780521841085
© Nicolo Cesa-Bianchi and Gabor Lugosi
This publication is in copyright. Subject to statutory exception and to the provision of
relevant collective licensing agreements, no reproduction of any part may take place
without the written permission of Cambridge University Press.
First published in print format 2006
isbn-13 978-0-511-19178-7 eBook (NetLibrary)
isbn-10 0-511-19178-2
eBook (NetLibrary)
isbn-13 978-0-521-84108-5 hardback
isbn-10 0-521-84108-9 hardback
Cambridge University Press has no responsibility for the persistence or accuracy of urls
for external or third-party internet websites referred to in this publication, and does not
guaranteethatanycontentonsuchwebsitesis,orwillremain,accurateorappropriate.
to Betta,Silvia, and Luca–NCB
to Arrate,Dani, and Frida
to thememoryofmyparents–GL
Contents
Preface pagexi
1 Introduction 1
1.1 Prediction 1
1.2 Learning 3
1.3 Games 3
1.4 AGentleStart 4
1.5 ANotetotheReader 6
2 PredictionwithExpertAdvice 7
2.1 WeightedAveragePrediction 9
2.2 AnOptimalBound 15
2.3 BoundsThatHoldUniformlyoverTime 17
2.4 AnImprovementforSmallLosses 20
2.5 ForecastersUsingtheGradientoftheLoss 22
2.6 ScaledLossesandSignedGames 24
2.7 TheMultilinearForecaster 25
2.8 TheExponentialForecasterforSigned
Games 27
2.9 SimulatableExperts 29
2.10 MinimaxRegret 30
2.11 DiscountedRegret 32
2.12 BibliographicRemarks 34
2.13 Exercises 37
3 TightBoundsforSpecificLosses 40
3.1 Introduction 40
3.2 FollowtheBestExpert 41
3.3 Exp-concaveLossFunctions 45
3.4 TheGreedyForecaster 49
3.5 TheAggregatingForecaster 52
3.6 MixabilityforCertainLosses 56
3.7 GeneralLowerBounds 59
3.8 BibliographicRemarks 63
3.9 Exercises 64
vii
viii Contents
4 RandomizedPrediction 67
4.1 Introduction 67
4.2 WeightedAverageForecasters 71
4.3 FollowthePerturbedLeader 74
4.4 InternalRegret 79
4.5 Calibration 85
4.6 GeneralizedRegret 90
4.7 CalibrationwithCheckingRules 93
4.8 BibliographicRemarks 94
4.9 Exercises 95
5 EfficientForecastersforLargeClassesofExperts 99
5.1 Introduction 99
5.2 TrackingtheBestExpert 100
5.3 TreeExperts 109
5.4 TheShortestPathProblem 116
5.5 TrackingtheBestofManyActions 121
5.6 BibliographicRemarks 124
5.7 Exercises 125
6 PredictionwithLimitedFeedback 128
6.1 Introduction 128
6.2 LabelEfficientPrediction 129
6.3 LowerBounds 136
6.4 PartialMonitoring 143
6.5 AGeneralForecasterforPartialMonitoring 146
6.6 HannanConsistencyandPartialMonitoring 153
6.7 Multi-armedBanditProblems 156
6.8 AnImprovedBanditStrategy 160
6.9 LowerBoundsfortheBanditProblem 164
6.10 HowtoSelecttheBestAction 169
6.11 BibliographicRemarks 173
6.12 Exercises 175
7 PredictionandPlayingGames 180
7.1 GamesandEquilibria 180
7.2 MinimaxTheorems 185
7.3 RepeatedTwo-PlayerZero-SumGames 187
7.4 CorrelatedEquilibriumandInternalRegret 190
7.5 UnknownGames:Game-TheoreticBandits 194
7.6 CalibrationandCorrelatedEquilibrium 194
7.7 Blackwell’sApproachabilityTheorem 197
7.8 Potential-basedApproachability 202
7.9 ConvergencetoNashEquilibria 205
7.10 ConvergenceinUnknownGames 210
7.11 PlayingAgainstOpponentsThatReact 219
7.12 BibliographicRemarks 225
7.13 Exercises 227
Contents ix
8 AbsoluteLoss 233
8.1 SimulatableExperts 233
8.2 OptimalAlgorithmforSimulatableExperts 234
8.3 StaticExperts 236
8.4 ASimpleExample 238
8.5 BoundsforClassesofStaticExperts 239
8.6 BoundsforGeneralClasses 241
8.7 BibliographicRemarks 244
8.8 Exercises 245
9 LogarithmicLoss 247
9.1 SequentialProbabilityAssignment 247
9.2 MixtureForecasters 249
9.3 GamblingandDataCompression 250
9.4 TheMinimaxOptimalForecaster 252
9.5 Examples 253
9.6 TheLaplaceMixture 256
9.7 ARefinedMixtureForecaster 258
9.8 LowerBoundsforMostSequences 261
9.9 PredictionwithSideInformation 263
9.10 AGeneralUpperBound 265
9.11 FurtherExamples 269
9.12 BibliographicRemarks 271
9.13 Exercises 272
10 SequentialInvestment 276
10.1 PortfolioSelection 276
10.2 TheMinimaxWealthRatio 278
10.3 PredictionandInvestment 278
10.4 UniversalPortfolios 282
10.5 TheEGInvestmentStrategy 284
10.6 InvestmentwithSideInformation 287
10.7 BibliographicRemarks 289
10.8 Exercises 290
11 LinearPatternRecognition 293
11.1 PredictionwithSideInformation 293
11.2 BregmanDivergences 294
11.3 Potential-BasedGradientDescent 298
11.4 TheTransferFunction 301
11.5 ForecastersUsingBregmanProjections 307
11.6 Time-VaryingPotentials 314
11.7 TheEllipticPotential 316
11.8 ANonlinearForecaster 320
11.9 LowerBounds 322
11.10 MixtureForecasters 325
11.11 BibliographicRemarks 328
11.12 Exercises 330
