A Course in Stochastic Game Theory

279 Pages·2022·2.085 MB·English

by  Eilon Solan

LONDONMATHEMATICALSOCIETYSTUDENTTEXTS
ManagingEditor:IanJ.Leary,
MathematicalSciences,UniversityofSouthampton,UK
63 Singularpointsofplanecurves,C.T.C.WALL
64 AshortcourseonBanachspacetheory,N.L.CAROTHERS
65 ElementsoftherepresentationtheoryofassociativealgebrasI,IBRAHIMASSEM,DANIEL
SIMSON&ANDRZEJSKOWRON´SKI
66 Anintroductiontosievemethodsandtheirapplications,ALINACARMENCOJOCARU
&M.RAMMURTY
67 Ellipticfunctions,J.V.ARMITAGE&W.F.EBERLEIN
68 Hyperbolicgeometryfromalocalviewpoint,LINDAKEEN&NIKOLALAKIC
69 LecturesonKa¨hlergeometry,ANDREIMOROIANU
70 Dependencelogic,JOUKUVA¨A¨NA¨NEN
71 ElementsoftherepresentationtheoryofassociativealgebrasII,DANIELSIMSON&ANDRZEJ
SKOWRON´SKI
72 ElementsoftherepresentationtheoryofassociativealgebrasIII,DANIELSIMSON&ANDRZEJ
SKOWRON´SKI
73 Groups,graphsandtrees,JOHNMEIER
74 RepresentationtheoremsinHardyspaces,JAVADMASHREGHI
75 Anintroductiontothetheoryofgraphspectra,DRAGOSˇCVETKOVIC´,PETERROWLINSON
&SLOBODANSIMIC´
76 NumbertheoryinthespiritofLiouville,KENNETHS.WILLIAMS
77 Lecturesonprofinitetopicsingrouptheory,BENJAMINKLOPSCH,NIKOLAYNIKOLOV
&CHRISTOPHERVOLL
78 Cliffordalgebras:Anintroduction,D.J.H.GARLING
79 IntroductiontocompactRiemannsurfacesanddessinsd’enfants,ERNESTOGIRONDO&
GABINOGONZA´LEZ–DIEZ
80 TheRiemannhypothesisforfunctionfields,MACHIELVANFRANKENHUIJSEN
81 Numbertheory,Fourieranalysisandgeometricdiscrepancy,GIANCARLOTRAVAGLINI
82 Finitegeometryandcombinatorialapplications,SIMEONBALL
83 Thegeometryofcelestialmechanics,HANSJO¨RGGEIGES
84 Randomgraphs,geometryandasymptoticstructure,MICHAELKRIVELEVICHetal.
85 Fourieranalysis:PartI–Theory,ADRIANCONSTANTIN
86 Dispersivepartialdifferentialequations,M.BURAKERDOG˘AN&NIKOLAOSTZIRAKIS
87 Riemannsurfacesandalgebraiccurves,R.CAVALIERI&E.MILES
88 Groups,languagesandautomata,DEREKF.HOLT,SARAHREES&CLAASE.RO¨VER
89 AnalysisonPolishspacesandanintroductiontooptimaltransportation,D.J.H.GARLING
90 Thehomotopytheoryof(∞,1)-categories,JULIAE.BERGNER
91 TheblocktheoryoffinitegroupalgebrasI,MARKUSLINCKELMANN
92 TheblocktheoryoffinitegroupalgebrasII,MARKUSLINCKELMANN
93 Semigroupsoflinearoperators,DAVIDAPPLEBAUM
94 Introductiontoapproximategroups,MATTHEWC.H.TOINTON
95 RepresentationsoffinitegroupsofLietype(2ndEdition),FRANC¸OISDIGNE&JEANMICHEL
96 TensorproductsofC*-algebrasandoperatorspaces,GILLESPISIER
97 Topicsincyclictheory,DANIELG.QUILLEN&GORDONBLOWER
98 Fasttracktoforcing,MIRNADZˇAMONJA
99 Agentleintroductiontohomologicalmirrorsymmetry,RAFBOCKLANDT
100 Thecalculusofbraids,PATRICKDEHORNOY
101 Classicalanddiscretefunctionalanalysiswithmeasuretheory,MARTINBUNTINAS
102 NotesonHamiltoniandynamicalsystems,ANTONIOGIORGILLI

LondonMathematicalSocietyStudentTexts103
A Course in Stochastic Game Theory
EILON SOLAN
Tel-AvivUniversity
UniversityPrintingHouse,CambridgeCB28BS,UnitedKingdom
OneLibertyPlaza,20thFloor,NewYork,NY10006,USA
477WilliamstownRoad,PortMelbourne,VIC3207,Australia
314–321,3rdFloor,Plot3,SplendorForum,JasolaDistrictCentre,
NewDelhi–110025,India
103PenangRoad,#05–06/07,VisioncrestCommercial,Singapore238467
CambridgeUniversityPressispartoftheUniversityofCambridge.
ItfurtherstheUniversity’smissionbydisseminatingknowledgeinthepursuitof
education,learning,andresearchatthehighestinternationallevelsofexcellence.
www.cambridge.org
Informationonthistitle:www.cambridge.org/9781316516331
DOI:10.1017/9781009029704
©EilonSolan2022
Thispublicationisincopyright.Subjecttostatutoryexception
andtotheprovisionsofrelevantcollectivelicensingagreements,
noreproductionofanypartmaytakeplacewithoutthewritten
permissionofCambridgeUniversityPress.
Firstpublished2022
AcataloguerecordforthispublicationisavailablefromtheBritishLibrary.
LibraryofCongressCataloging-in-PublicationData
Names:Solan,Eilon,author.
Title:Acourseinstochasticgametheory/EilonSolan,
SchoolofMathematicalSciences.
Description:Firstedition.|NewYork:CambridgeUniversityPress,2022.|
Series:LMSTlondonmathematicalsocietystudenttexts|
Includesbibliographicalreferencesandindex.
Identifiers:LCCN2021055382(print)|LCCN2021055383(ebook)|
ISBN9781316516331(hardback)|ISBN9781009014793(paperback)|
ISBN9781009029704(epub)
Subjects:LCSH:Gametheory.|BISAC:MATHEMATICS/General
Classification:LCCQA269.S652022(print)|LCCQA269(ebook)|
DDC519.3–dc23/eng20220207
LCrecordavailableathttps://lccn.loc.gov/2021055382
LCebookrecordavailableathttps://lccn.loc.gov/2021055383
ISBN978-1-316-51633-1Hardback
ISBN978-1-009-01479-3Paperback
CambridgeUniversityPresshasnoresponsibilityforthepersistenceoraccuracyof
URLsforexternalorthird-partyinternetwebsitesreferredtointhispublication
anddoesnotguaranteethatanycontentonsuchwebsitesis,orwillremain,
accurateorappropriate.
Thebookisdedicatedtothepeoplewhosharedmylife,mywork,andmylove
ofgamesinthelastthirtyyears.ToAbrahamNeyman,whointroducedmeto
gametheoryandtostochasticgames;toNicolasVieilleandDinahRosenberg,
withwhomIspentcountlessfunweeksofstudyingstochasticgames;toEhud
Lehrer,whohasbeenmycolleagueandpartneratTelAvivUniversityforthe
past 20 years; to my parents, Chaim and Zafrira; and to my two sons, Omri
andRon,wholistenedtogame-theoretic problems sincebirthand eventually
becamecoauthors.

Contents
Introduction page1
Notation 3
1 MarkovDecisionProblems 5
1.1 OnHistories 8
1.2 OnStrategies 8
1.3 TheT-StagePayoff 11
1.4 TheDiscountedPayoff 17
1.5 ContractingMappings 21
1.6 ExistenceofanOptimalStationaryStrategy 22
1.7 UniformOptimality 27
1.8 CommentsandExtensions 30
1.9 Exercises 31
2 A Tauberian Theorem and Uniform (cid:2)-Optimality in Hidden
MarkovDecisionProblems 35
2.1 ATauberianTheorem 36
2.2 HiddenMarkovDecisionProblems 38
2.3 CommentsandExtensions 45
2.4 Exercises 46
3 Strategic-FormGames:AReview 49
3.1 Exercises 53
4 StochasticGames:TheModel 55
4.1 OnHistoriesandStrategies 57
4.2 AbsorbingStatesandAbsorbingGames 58
4.3 CommentsandExtensions 59
4.4 Exercises 60
vii
viii Contents
5 Two-PlayerZero-SumDiscountedGames 62
5.1 TheDiscountedPayoff 62
5.2 TheDiscountedValue 66
5.3 ExistenceoftheDiscountedValue 68
5.4 ContinuityoftheValueFunction 73
5.5 CommentsandExtensions 75
5.6 Exercises 75
6 Semi-AlgebraicSetsandtheLimitoftheDiscountedValue 82
6.1 Semi-AlgebraicSets 82
6.2 Semi-AlgebraicSetsandZero-SumStochasticGames 87
6.3 CommentsandExtensions 89
6.4 Exercises 90
7 B-GraphsandtheContinuityoftheLimitlimλ→0vλ(s;q,r) 94
7.1 B-Graphs 94
7.2 TheMeanDiscountedTime 99
7.3 TheMeanDiscountedTimeandStochasticGames 101
7.4 ADistancebetweenTransitionRules 102
7.5 ContinuityoftheValue 104
7.6 CommentsandExtensions 108
7.7 Exercises 108
8 Kakutani’sFixed-PointTheoremandMultiplayerDiscounted
StochasticGames 111
8.1 Kakutani’sFixed-PointTheorem 111
8.2 DiscountedEquilibrium 114
8.3 ExistenceofStationaryDiscountedEquilibria 116
8.4 Semi-AlgebraicSetsandDiscountedStochasticGames 119
8.5 CommentsandExtensions 120
8.6 Exercises 121
9 UniformEquilibrium 128
9.1 DefinitionofUniformEquilibrium 129
9.2 The“BigMatch” 134
9.3 ExistenceoftheUniformValueinTwo-PlayerZero-Sum
StochasticGames 139
9.4 TheAverageCostOptimalityEquation 148
9.5 Extensive-FormCorrelatedEquilibrium 155
9.6 CommentsandExtensions 161
9.7 Exercises 163
Contents ix
10 TheVanishingDiscountFactorApproachandUniform
EquilibriuminAbsorbingGames 173
10.1 Preliminaries 173
10.2 Uniform Equilibrium in Absorbing Games with Positive
AbsorbingProbability 177
10.3 UniformEquilibriuminTwo-PlayerAbsorbingGames 178
10.4 CommentsandExtensions 189
10.5 Exercises 190
11 Ramsey’sTheoremandTwo-PlayerDeterministic
StoppingGames 195
11.1 Ramsey’sTheorem 195
11.2 UndiscountedEquilibrium 197
11.3 Two-PlayerRecursiveAbsorbingGameswithaSingle
NonabsorbingEntry 198
11.4 Two-PlayerDeterministicStoppingGames 201
11.5 PeriodicDeterministicStoppingGames 202
11.6 ProofofTheorem11.12 204
11.7 CommentsandExtensions 207
11.8 Exercises 209
12 InfiniteOrbitsandQuittingGames 211
12.1 ApproximatingInfiniteOrbits 211
12.2 AnExampleofaThree-PlayerQuittingGame 216
12.3 Multiplayer Quitting Games in Which Players Do Not Want
OtherstoJoinTheminQuitting 223
12.4 CommentsandExtensions 234
12.5 Exercises 234
13 LinearComplementarityProblemsandQuittingGames 236
13.1 LinearComplementarityProblems 236
13.2 StationaryEquilibriainQuittingGames 239
13.3 CyclicEquilibriuminThree-PlayerPositiveQuittingGames 245
13.4 CommentsandExtensions 250
13.5 Exercises 252
References 255
Index 265