Noncooperative Game Theory: An Introduction for Engineers and Computer Scientists

230 Pages·2017·1.507 MB·English

by  João P. Hespanha

N
ONCOOPERATIVE
G
AME
T
HEORY
AN INTRODUCTION FOR ENGINEERS
AND COMPUTER SCIENTISTS
J ˜ P. H
OAO ESPANHA
PRINCETONUNIVERSITYPRESS ~ PRINCETONANDOXFORD
Copyright©2017byPrincetonUniversityPress
PublishedbyPrincetonUniversityPress,41WilliamStreet,
Princeton,NewJersey08540
IntheUnitedKingdom:PrincetonUniversityPress,6OxfordStreet,
Woodstock,OxfordshireOX201TR
press.princeton.edu
LibraryofCongressCataloging-in-PublicationData
Names:Hespanha,Joa˜oP.
Title:Noncooperativegametheory:anintroductionforengineersand
computerscientists/Joa˜oP.Hespanha.
Description:Princeton:PrincetonUniversityPress,[2017]|Includes
bibliographicalreferencesandindex.
Identifiers:LCCN2017001998|ISBN9780691175218(hardcover:alk.paper)
Subjects:LCSH:Noncooperativegames(Mathematics)—Textbooks.|Game
theory—Textbooks.|Cooperativegames(Mathematics)—Textbooks.
Classification:LCCQA272.5.H472017|DDC519.3—dc23
LCrecordavailableathttps://lccn.loc.gov/2017001998
BritishLibraryCataloging-in-PublicationDataisavailable
ThisbookhasbeencomposedinMinionandBankGothicusingZzTEX
byWindfallSoftware,Carlisle,Massachusetts
PrintedintheUnitedStatesofAmerica
CONTENTS
Preamble xi
I INTRODUCTION
1 NoncooperativeGames
1.1 ElementsofaGame 3
1.2 Cooperativevs.NoncooperativeGames:Rope-Pulling 4
1.3 RobustDesigns:ResistiveCircuit 8
1.4 MixedPolicies:NetworkRouting 9
1.5 NashEquilibrium 11
1.6 PracticeExercise 11
2 Policies
2.1 Actionsvs.Policies:AdvertisingCampaign 13
2.2 Multi-StageGames:WarofAttrition 16
2.3 Openvs.Closed-Loop:ZebraintheLake 18
2.4 PracticeExercises 19
II ZERO-SUM GAMES
3 Zero-SumMatrixGames
3.1 Zero-SumMatrixGames 25
3.2 SecurityLevelsandPolicies 26
3.3 ComputingSecurityLevelsandPolicieswithMATLAB® 27
3.4 Securityvs.Regret:AlternatePlay 28
3.5 Securityvs.Regret:SimultaneousPlays 28
3.6 Saddle-PointEquilibrium 29
3.7 Saddle-PointEquilibriumvs.SecurityLevels 30
3.8 OrderInterchangeability 32
3.9 ComputationalComplexity 32
3.10 PracticeExercise 34
3.11 AdditionalExercise 34
4 MixedPolicies
4.1 MixedPolicies:Rock-Paper-Scissor 35
4.2 MixedActionSpaces 37
4.3 MixedSecurityPoliciesandSaddle-PointEquilibrium 38
4.4 MixedSaddle-PointEquilibriumvs.AverageSecurityLevels 41
4.5 GeneralZero-SumGames 43
viii CONTENTS
4.6 PracticeExercises 47
4.7 AdditionalExercise 50
5 MinimaxTheorem
5.1 TheoremStatement 52
5.2 ConvexHull 53
5.3 SeparatingHyperplaneTheorem 54
5.4 OntheWaytoProvetheMinimaxTheorem 55
5.5 ProofoftheMinimaxTheorem 57
5.6 ConsequencesoftheMinimaxTheorem 58
5.7 PracticeExercise 58
6 ComputationofMixedSaddle-PointEquilibriumPolicies
6.1 GraphicalMethod 60
6.2 LinearProgramSolution 61
6.3 LinearProgramswithMATLAB® 63
6.4 StrictlyDominatingPolicies 64
6.5 “Weakly”DominatingPolicies 66
6.6 PracticeExercises 67
6.7 AdditionalExercise 70
7 GamesinExtensiveForm
7.1 Motivation 71
7.2 ExtensiveFormRepresentation 72
7.3 Multi-StageGames 72
7.4 PurePoliciesandSaddle-PointEquilibria 74
7.5 MatrixFormforGamesinExtensiveForm 75
7.6 RecursiveComputationofEquilibriaforSingle-StageGames 77
7.7 FeedbackGames 79
7.8 FeedbackSaddle-PointforMulti-StageGames 79
7.9 RecursiveComputationofEquilibriaforMulti-StageGames 83
7.10 PracticeExercise 85
7.11 AdditionalExercises 86
8 StochasticPoliciesforGamesinExtensiveForm
8.1 MixedPoliciesandSaddle-PointEquilibria 87
8.2 BehavioralPoliciesforGamesinExtensiveForm 90
8.3 BehavioralSaddle-PointEquilibria 91
8.4 Behavioralvs.MixedPolicies 92
8.5 RecursiveComputationofEquilibriaforFeedbackGames 93
8.6 Mixedvs.BehavioralOrderInterchangeability 95
8.7 Non-FeedbackGames 95
8.8 PracticeExercises 96
8.9 AdditionalExercises 102
CONTENTS ix
III NON-ZERO-SUM GAMES
9 Two-PlayerNon-Zero-SumGames
9.1 SecurityPoliciesandNashEquilibria 105
9.2 BimatrixGames 107
9.3 AdmissibleNashEquilibria 108
9.4 MixedPolicies 110
9.5 Best-ResponseEquivalentGamesandOrderInterchangeability 111
9.6 PracticeExercises 114
9.7 AdditionalExercises 116
10 ComputationofNashEquilibriaforBimatrixGames
10.1 CompletelyMixedNashEquilibria 118
10.2 ComputationofCompletelyMixedNashEquilibria 120
10.3 NumericalComputationofMixedNashEquilibria 121
10.4 PracticeExercise 124
10.5 AdditionalExercise 126
11 N-PlayerGames
11.1 N-PlayerGames 127
11.2 PureN-PlayerGamesinNormalForm 129
11.3 MixedPoliciesforN-PlayerGamesinNormalForm 130
11.4 CompletelyMixedPolicies 131
12 PotentialGames
12.1 IdenticalInterestsGames 133
12.2 PotentialGames 135
12.3 CharacterizationofPotentialGames 138
12.4 PotentialGameswithIntervalActionSpaces 139
12.5 PracticeExercises 142
12.6 AdditionalExercise 144
13 ClassesofPotentialGames
13.1 IdenticalInterestsPlusDummyGames 145
13.2 DecoupledPlusDummyGames 146
13.3 BilateralSymmetricGames 147
13.4 CongestionGames 148
13.5 OtherPotentialGames 149
13.6 DistributedResourceAllocation 150
13.7 ComputationofNashEquilibriaforPotentialGames 153
13.8 FictitiousPlay 156
13.9 PracticeExercises 159
13.10 AdditionalExercises 167
x CONTENTS
IV DYNAMIC GAMES
14 DynamicGames
14.1 GameDynamics 171
14.2 InformationStructures 173
14.3 Continuous-TimeDifferentialGames 175
14.4 DifferentialGameswithVariableTerminationTime 177
15 One-PlayerDynamicGames
15.1 One-PlayerDiscrete-TimeGames 178
15.2 Discrete-TimeCost-To-Go 179
15.3 Discrete-TimeDynamicProgramming 179
15.4 ComputationalComplexity 184
15.5 SolvingFiniteOne-PlayerGameswithMATLAB® 186
15.6 LinearQuadraticDynamicGames 187
15.7 PracticeExercise 187
15.8 AdditionalExercise 189
16 One-PlayerDifferentialGames
16.1 One-PlayerContinuous-TimeDifferentialGames 190
16.2 Continuous-TimeCost-To-Go 191
16.3 Continuous-TimeDynamicProgramming 191
16.4 LinearQuadraticDynamicGames 195
16.5 DifferentialGameswithVariableTerminationTime 196
16.6 PracticeExercise 198
17 State-FeedbackZero-SumDynamicGames
17.1 Zero-SumDynamicGamesinDiscreteTime 201
17.2 Discrete-TimeDynamicProgramming 203
17.3 SolvingFiniteZero-SumGameswithMATLAB® 205
17.4 LinearQuadraticDynamicGames 206
17.5 PracticeExercise 209
18 State-FeedbackZero-SumDifferentialGames
18.1 Zero-SumDynamicGamesinContinuousTime 214
18.2 LinearQuadraticDynamicGames 216
18.3 DifferentialGameswithVariableTerminationTime 219
18.4 Pursuit-Evasion 220
18.5 PracticeExercise 222
References 223
Index 225
PREAMBLE
Totheuninitiated,GameTheoryconjuresimagesofdevelopingcomputerprogramsto
solveboardgameslikechessorcardgameslikepokerand,infact,thetoolsbehindthis
disciplinecanindeedbeusedforsuchpurposes.However,gametheorygoesmuch
beyond such functions and provides a framework for reasoning about problems in
whichmultiple“players”mustmakedecisions,withtheunderstandingthattheresults
ofthesedecisionsaffectandareaffectedbythedecisionsoftheotherplayers.Board
andcardgamesareobviousexamplesofsuchproblems,butgametheoryisapplicable
tomuchmore“serious”domains.
The first question one typically asks when faced with a multiplayer problem is
probably“HowshouldIplay?”Immediatelyfollowedbythequestion“Howwillmy
opponent play?” The way out of this fundamental chicken and egg problem faced
by game theorists is that one needs to find “equilibrium” strategies that somehow
simultaneously satisfy all the players. Game theory thus provides a framework to
predictthebehaviorofrationalplayers,eitherinboardgamesorineconomics.
Oncethenotionof“equilibrium”isunderstood,onestartstowonderwhetheritis
possibletofindsuchequilibriaforallgames.Itisnotnecessarytogofartofindtrouble:
equilibriadonotalwaysexistandsometimesthereismorethanoneequilibrium.How
isonethensupposedtopredictthebehaviorofrationalplayers?Andwhatifasingle
equilibrium exists, but we do not like the predicted behavior of the players? These
questions lead to some of the most interesting problems in game theory: “How to
designgamesthatpredictablyleadtodesirableactionsbytheplayers?”Thesequestions
areofkeyinteresttoeconomists,socialscientists,andengineers.
Modern game theory was born in the 1930s, mostly propelled by the work of
JohnvonNeumann,andfurtherrefinedbyMorgenstern,Kuhn,Nash,Shapleyand
others.Throughoutmostofthe1940sand1950s,economicswasitsmainapplication,
eventuallyleadingtothe1994NobelprizeinEconomicScienceawardedtoJohnNash,
JohnC.Harsanyi,andReinhardSeltenfortheircontributionstogametheory.Itwas
notuntilthe1970sthatitstartedtohaveasignificantimpactonengineering;andin
thelate1980sitledtosignificantbreakthroughsincontroltheoryandrobustfiltering.
Currently,gametheorypervadesallareasofengineering.
Problemsrelatedtothedevelopmentofpricingstrategiesfortechnologicalproducts
orserviceshavealwaysinterestedengineers,buttheuseofgametheoryintechnology
design is a more recent development that arose from the intrinsic limitations of
classicaloptimization-baseddesigns.Inoptimization,oneattemptstofindvaluesfor
parameters that minimize suitably defined criteria (such as monetary cost, energy
consumption,heatgenerated,etc.)However,inmostengineeringapplicationsthere
is always some uncertainty as to how the selected parameters will affect the final
objective.Onecanthenposetheproblemofhowtomakesurethattheselectionwill
leadtoacceptableperformance,eveninthepresenceofsomedegreeofuncertainty—
theunforgivingplayerthat,behindthescenes,conspirestowreckengineeringdesigns.
Thisquestionisattheheartofmanygamesthatappearinengineeringapplications.
xii PREAMBLE
Infact,gametheoryprovidesabasicmathematicalframeworkforrobustdesignin
engineering.
A feature common to many engineering applications of game theory is that the
problem does not start as a “game.” In fact, the most interesting design challenge
isoftentoconstructagamethatcapturestheessenceoftheproblem:Whoarethe
players?Whataretheirgoals?Willthe“solution”tothegamesolvetheoriginaldesign
problem?Thesearequestionsthatwewillencounterinthisbook.
Content
Lectures1–2introducethebasicelementsofamathematicalgamethroughasetof
simpleexamples.Notionsofplayer,gamerulesandobjectives,informationstructure,
playerrationality,cooperativeversusnoncooperativesolutions,andNashequilibrium
are introduced to provide the reader with an overview of the main issues to come.
Subsequentlecturessystematicallyreturntoallofthesetopics.
Lectures 3–8 are focused on zero-sum games. Starting with matrix games in lec-
tures 3–6, we introduce the fundamental concept of saddle-point equilibrium and
exploreitskeyproperties,bothforpureandmixedpolicies.TheMinimaxTheorem
andcomputationalissuesarealsocovered.Theinformationstructureofagameisfirst
treatedinlectures7–8withtheintroductionof(zero-sum)gamesinextensiveform.
Complexinformationstructuresleadtothedistinctionbetweentwotypesofstochas-
ticpolicies:mixedandbehavioralpolicies.Intheselectureswealsointroduceageneral
recursivemethodthatwillevolveinlaterlecturesintoDynamicProgramming.
Non-zero sum games are treated in lectures 9–13. We introduce the concept of
Nashequilibriuminageneralsettinganddiscussitsnumericalcomputationfortwo-
player bimatrix games. Lectures 12–13 are focused exclusively on the rich class of
potential games. In these lectures we discuss several classical potential games, with
some emphasis on the design of potential games to solve distributed optimization
problems.
Thelastsetoflectures14–18isdevotedtothesolutionofdynamicgames.Westart
byreviewingDynamicProgrammingfor(single-player)optimizationinlectures15–
16 and use it as the starting point to construct saddle-point policies for zero-sum
gamesinlectures17–18.Wetreatbothdiscrete-andcontinuous-timegames,witha
fixedoravariableterminationtime.
Learning and Teaching Using This Textbook
Thisbookwaspurposelydesignedasatextbook,andconsequently,themainemphasis
isonpresentingmaterialinafashionthatmakesitinterestingandeasyforstudents
tounderstand.
Attention!Whenamarginal Inwritingthismanuscript,therewasaconsciousefforttoreduceverbosity.This
notefinisheswith (cid:2)p.xxx is not to say that there was no attempt to motivate the concepts or discuss their
moreinformationabout significance(tothecontrary),buttheamountoftextwaskepttoaminimum.Typically,
thattopiccanbefoundon
discussion,remarks,andsidecommentsarerelegatedtomarginalnotessothatthe
pagexxx.
PREAMBLE xiii
readercaneasilyfollowthematerialpresentedwithoutdistractionandyetenjoythe
benefitofcommentsonthenotationandterminology,orbemadeawarethatathere
isarelatedMATLAB®command.
Note.Joa˜oHespanhaisa At the University of California at Santa Barbara, I teach the material in these
ProfessorattheElectrical lecturesinonequarterwithabout36hoursofclasstime.TheclassIteachisprimarily
andComputerEngineering aimedatfirst-yeargraduatestudentsintheCollegeofEngineering, butthesenotes
DepartmentattheUniversity
were written so that they can also serve as the primary textbook for a senior-level
ofCalifornia,SantaBarbara.
undergraduateclass,asmostofthelecturesonlyrequirefamiliaritywithlinearalgebra
and probabilities at an undergraduate level. Two lectures (16 and 18) also require
somefamiliaritywithdifferentialequations,buttheycouldbeskippedorpresented
asoptionaladvancedmaterialifstudentsdonothavetheappropriateprerequisites.
Ihavetailoredtheorganizationofthetextbooktosimplifytheteachingandlearning
ofthematerial.Inparticular,thesequenceofthechaptersemphasizescontinuity,with
eachchaptermotivatedbyandinlogicalsequencewiththeprecedingones.Ialways
avoid introducing a concept in one chapter and using it again only many chapters
later.Ithasbeenmyexperiencethatevenifthismaybeeconomicalintermsofspace,
itispedagogicallycounterproductive.Thechaptersarebalancedinlengthsothaton
averageeachcanbecoveredinroughly2hoursoflecturetime.Notonlydoesthis
greatlyaidtheinstructor’splanning,butitmakesiteasierforthestudentstoreview
thematerialstaughtinclass.
Thebookincludesexercisesthatshouldbesolvedasthereaderprogressesthrough
thematerial.Someoftheseexercisesclarifyissuesraisedinthebodyofthetextand
thereaderisgenerallypointedtosuchexercisesinmarginalnotes;forexample,Exer-
cise6.3,whichisreferencedinamarginalnoteonpage62.Otherexercisesareaimed
atconsolidatingtheknowledgeacquired,byaskingthereadertoapplyalgorithmsor
approachespreviouslydiscussed;forexample,Exercise13.9onpage167.Thebookin-
cludesdetailedsolutionsforalltheexercisesthatappearinthesectionstitled“Practice
Exercises,”butitdoesnotincludesolutionstothoseinthesectionstitled“Additional
Exercises.”
MATLAB®
ComputationaltoolssuchastheMATLAB®softwareenvironmentofferasignificant
step forward in teaching this class because they allow students to solve numerical
problemswithoutbeingsubjecttoadetailedtreatmentofnumericalmethods.Bysys-
tematicallyannotatingthetheoreticaldevelopmentswithmarginalnotesthatdiscuss
therelevantcommandsavailableinMATLAB®,thistextbookhelpsstudentslearnto
usethesetools.Anexampleofthiscanbefound,e.g.,inMATLAB®Hint1onpage26,
whichisfurtherexpandedonpage27.WealsoprovideMATLAB®functionsthatim-
plementsomeofthekeyalgorithmsdiscussed.Anexampleofthiscanbefound,e.g.,
inMATLAB®Hint4onpage155,whichislaterused,e.g.,inExercise13.5onpage163.
Thecommandsdiscussedinthe“MATLAB®Hints”assumethatthereaderhasver-
sionR2015bofMATLAB®andtheOptimizationToolbox.However,essentiallyallthe
commandsusedhavebeenfairlystableforseveralversions,sotheyarelikelytowork
withpreviousandsubsequentversionsforseveralyearstocome.Lecture6assumes
xiv PREAMBLE
thatthereaderhasinstalledCVX,whichisaMATLAB®packageforDisciplinedCon-
vexProgramming,distributedundertheGNUGeneralPublicLicense2.0[4].
MATLAB® is a registered trademark of The MathWorks, Inc. and is used with
permission.TheMathWorksdoesnotwarranttheaccuracyofthetextorexercises
in this book. This book’s use or discussion of MATLAB® or related products does
not constitute an endorsement or sponsorship by The MathWorks of a particular
pedagogicalapproachorparticularuseoftheMATLAB®software.
Web
The reader is referred to the author’s website at www.ece.ucsb.edu/~hespanha for
corrections,updatesonMATLAB®andCVX,andothersupplementalmaterial.
Acknowledgments
Several friends and colleagues have helped me improve this manuscript through
theirthoughtfulconstructivecommentsandsuggestions.Amongthese,Iowespecial
thankstoJasonMarden,Se´rgioPequito,FarshadPourSafaei,aswellasallthestudents
attheUniversityofCaliforniaatSantaBarbarawhousedearlydraftsofthesenotes
and provided me with numerous comments and suggestions. I would also like to
acknowledge the support of several organizations, including the National Science
Foundation(NSF),theArmyResearchOffice(ARO),theAirForceOfficeofScientific
Research(AFOSR),andtheUniversityofCaliforniaatSantaBarbara.