Game Theory and Learning for Wireless Networks. Fundamentals and Applications

316 Pages·2011·4.037 MB·English

by  Samson Lasaulce and Hamidou Tembine (Auth.)

Lasaulce — 01-fm-i-iv-9780123846983 — 2011/8/5 — 14:52 — Page 1 — #1
Game Theory and
Learning for Wireless
Networks
Fundamentals and Applications
Lasaulce — 01-fm-i-iv-9780123846983 — 2011/8/5 — 14:52 — Page 3 — #3
Game Theory and
Learning for Wireless
Networks
Fundamentals and Applications
Samson Lasaulce
Hamidou Tembine
AMSTERDAM•BOSTON•HEIDELBERG•LONDON
NEWYORK•OXFORD•PARIS•SANDIEGO
SANFRANCISCO•SINGAPORE•SYDNEY•TOKYO
AcademicPressisanimprintofElsevier
Lasaulce — 01-fm-i-iv-9780123846983 — 2011/8/5 — 14:48 — Page 4 — #4
AcademicPressisanimprintofElsevier
TheBoulevard,LangfordLane,Kidlington,Oxford,OX51GB
225WymanStreet,Waltham,MA02451,USA
Firstpublished2011
Copyright(cid:13)c 2011ElsevierLtd.Allrightsreserved.
Nopartofthispublicationmaybereproducedortransmittedinanyformorbyanymeans,electronicor
mechanical,includingphotocopying,recording,oranyinformationstorageandretrievalsystem,without
permissioninwritingfromthepublisher.Detailsonhowtoseekpermission,furtherinformationaboutthe
Publisher’spermissionspoliciesandourarrangementwithorganizationssuchastheCopyrightClearance
CenterandtheCopyrightLicensingAgency,canbefoundatourwebsite:www.elsevier.com/permissions.
ThisbookandtheindividualcontributionscontainedinitareprotectedundercopyrightbythePublisher
(otherthanasmaybenotedherein).
Notices
Knowledgeandbestpracticeinthisfieldareconstantlychanging.Asnewresearchandexperience
broadenourunderstanding,changesinresearchmethods,professionalpractices,ormedicaltreatment
maybecomenecessary.
Practitionersandresearchersmustalwaysrelyontheirownexperienceandknowledgeinevaluatingand
usinganyinformation,methods,compounds,orexperimentsdescribedherein.Inusingsuchinformation
ormethodstheyshouldbemindfuloftheirownsafetyandthesafetyofothers,includingpartiesfor
whomtheyhaveaprofessionalresponsibility.
Tothefullestextentofthelaw,neitherthePublishernortheauthors,contributors,oreditors,assumeany
liabilityforanyinjuryand/ordamagetopersonsorpropertyasamatterofproductsliability,negligence
orotherwise,orfromanyuseoroperationofanymethods,products,instructions,orideascontainedin
thematerialherein.
BritishLibraryCataloguinginPublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary
LibraryofCongressNumber:2011930747
ISBN:978-0-12-384698-3
ForinformationonallAcademicPresspublications
visitourwebsiteatwww.elsevierdirect.com
Typesetby:diacriTech,India
PrintedandboundintheUnitedKingdom
11 12 13 14 10 9 8 7 6 5 4 3 2 1
Lasaulce — 03-pref-xi-xvi-9780123846983 — 2011/8/4 — 12:39 — Page xi — #1
Preface
GAME THEORY AND WIRELESS
In everyday life, it is common to be in situations where the outcome of a situation
dependsnotonlyonwhatwedo,butalsoonwhatotherpeopledo.Thisisclearlythe
case when participating in an auction on the Internet, voting at a presidential elec-
tion, negotiating a price with a seller, trying to find a seat in a bus or in the metro,
etc. All these situations are referred to as a “game” in game theory. Indeed, they
do have something in common with what is usually known as a game (e.g., chess,
football, monopoly, video games): several decision makers are involved, and there
isinteractionbetweentheminthesensethattheirdecisionsareinterdependent.This
broad definition of a game explains why game theory, which is the formal study of
interactions between several decision makers who can have conflicting or common
interests,hasbecomemoreandmoreimportantinmanyareas.Amongthesewefind
for example, economics, politics, biology, and computer science. Game theory has
recentlybeenappliedtotelecommunications.Indeed,itisnowwellknownthatgame
theorycanbeusedtoanalyzeinteractionsbetweenentitiessuchastelecomsregula-
tors,operators,manufacturers,andcustomers;forinstance,gametheoreticianshave
beeninvolvedindesigningradiospectrumauctionsintheUSandinEurope.More
specifically,thespectrumforthethirdgenerationmobilesystem(3G)inEuropehas
beenauctionedintheUKandGermany.Carriersacrossthecontinenthavepaid110
billioneurofortherightstotheradiospectrumreservedfor3G,showingthestrategic
importance of the radio spectrum in the telecommunications sector. In such a case,
theplayersofthegamearetypicallythegovernmentsandthecarriers.Althoughthis
book is about applying game theory to wireless communications, the players in the
gamesunderstudyarenoteconomicactorssuchasregulatorsorcarriers,atleastnot
explictly.Rather,theplayersofthegamesunderconsiderationare“machines”such
asbase stations,laptops, mobilephones,routers, servers,and soon. Obviously,the
engineers who design these machines are directed by the economic policy of their
company or institution, which indicates that identifying the players is not always
obviousinwirelesscommunications.Nonetheless,thereisadifferencebetweencon-
sidering a telecoms operator or a mobile phone to be a player – the nature of the
action or decision to be taken is different. For an operator involved in a spectrum
auction, the action to be taken may be a price, whereas the possible actions avail-
abletoamobilephonemaybetodecidewhichaccesspointsorbasestationstobe
connected to. Indeed, interaction between wireless terminals like mobile phones is
naturally present in wireless networks, since interference often exists or/and com-
mon resources must be shared. This book therefore focuses on technical aspects of
communicationproblemsandnottheireconomicaspects.
Gametheoryandcommunicationproblemsseemtohavebeenconnectedforthe
firsttimebyBenoˆıtMandelbrotinhisthesis(in1952),butitisonlyduringthepast
decade that game theory has experienced a strong surge of interest in the area of
xi
Lasaulce — 03-pref-xi-xvi-9780123846983 — 2011/8/4 — 12:39 — Page xii — #2
xii Preface
wireless communications. Wireless networks have evolved enormously during this
timeandnewcommunicationparadigmshaveappeared,makinggametheoryespe-
ciallyrelevantintheiranalysisanddesign.Toemphasizethis,letusconsiderafew
examples.
• Ad hoc networks. In fixed communication networks, the paradigm of peer-to-
peercommunicationshasundergoneapowerfulsurgeofinterestduringthetwo
pastdecades,inapplicationssuchastheInternet.Remarkably,thisparadigmhas
also been found to be very useful for wireless networks, especially ad hoc net-
workswhichdonotrelyonapre-existinginfrastructure,suchasroutersinwired
networksoraccesspointsinmanagedwirelessnetworks.Typically,insuchnet-
works,awirelessnodeorterminalmayhavetodecidewhetherornottoforward
dataorpackets;thatis,itmustchoosearoutingpolicy.
• Cognitiveradio.Spectrumcongestionhasbecomeamoreandmorecriticalissue,
sotheFederalCommunicationsCommissionreleasedanimportantreportin2002
(FCC,2002),providingalegalframeworkfordeployingnetworkswithcognitive
terminals. Roughly, the fundamental idea of cognitive radio is that a terminal is
abletosenseitsenvironment(mainlyradioresources)andreacttoitaccordingly.
In the context of cognitive radio, the decision of a terminal (a transmitter or a
receiver)isessentiallyhowtousethespectrum(e.g.,transmitornottransmiton
agivenband,listenornotlistentousers’activityinagivenband).
• Sensornetworks.Wirelesssensornetworkscompriseseveral(possiblymany)spa-
tiallydistributedautonomoussensors.Atypicalobjectiveofthesenetworksisto
monitor physical conditions, e.g., estimating a field of temperature or pressure
inadistributedmanner.Forthispurpose,communicationsbetweensensorsmay
beusedtoreachaconsensusontheestimatedquantitiesoranefficientsolution.
Sensors have therefore to take decisions, in terms of estimates and what data to
sendtoneighboringnodes.
• Unlicensed bands. In the scenario of unlicensed bands, the spectrum available
to wireless terminals is not owned by an operator and is free to use, provided
someemissionlevelsarerespected.Typically,therecanbeseveralpoint-to-point
communications operating on the same band, at the same time, and in the same
geographical area. Even if the terminals are not cognitive, the engineers who
design terminals operating in unlicensed bands must deal with interfering sig-
nalsespeciallyifgoodperformanceissought.InWiFisystems,channelselection
caninfluencethequalityperceivedbytheuser.
To these examples, one might add more classical examples, such as medium
access control and power control in multiuser channels. In all these examples, ter-
minals are (quasi) autonomous in their decision making, and may share common
resources like energy, power, routes, space, spectrum, and time, which implies
potential interaction between them. Modeling the network as a set of independent
point-to-pointcommunicationsmaybestronglysuboptimalandeveninappropriate.
Game theory becomes especially relevant when one wants to assume that the envi-
ronmentofthewirelessterminalsisstructured,otherwiseclassicaloptimizationtools
Lasaulce — 03-pref-xi-xvi-9780123846983 — 2011/8/4 — 12:39 — Page xiii — #3
Preface xiii
may suffice. The problem in interactive situations, which include several decision
makers having a certain knowledge about the situation, is that the optimal deci-
sion or strategy is generally unclear, since no one player completely controls the
final outcome. This means that the problem needs to be defined before it can be
solved. Game theory is a means of proposing, designing interaction models, study-
ingtheconditionsunderwhichsomeoutcomescanbereached,anddesigninggood
strategies.Game-theoreticapproachesmayleadtomoreefficientnetworkstatesthan
the worst-case approach, where structural elements are ignored or not accounted
for. Remarkably, considering a terminal as an intelligent entity which is capable of
observingastructuredenvironmentandreactingsufficientlyrapidly,hasbecomean
increasingly realistic assumption with the significant progress in signal processing
(e.g.,spectrumsensingalgorithmsanddramaticincreaseofadmissiblecomputational
complexity).Foralongtime,gametheorywasusedquitemarginallyandmorelike
ananalysistoolincommunicationproblems.Withrecenttechnologicalprogressand
the arrival of new wireless paradigms, the era of using game theory for design has
come.Multiuserinformationtheoryhasinspiredresearchersandengineers,andled
themtoinventimportantconceptssuchassuccessiveinterferencecancellation,coop-
erative diversity, and multiuser diversity. Although applied almost ten years ago to
design algorithms such as the iterative water-filling algorithm, much work remains
todesignalgorithmsimplementableinarealnetworkandcompatiblewithrequired
information assumptions, behavior assumptions, convergence rate, quality of ser-
vice,oradmissiblecomplexity.Therightdynamicgamemodelstostudydistributed
wirelessnetworksneedtodesigned.
ABOUT THE CONTENTS OF THE BOOK
This book is dedicated to students, engineers, teachers, and researchers who know
aboutwirelesscommunications.Inthisrespect,almostallthechosenexamplesand
treatedcasestudiesconcernwirelesscommunicationproblems.Thebookcomprises
of 10 chapters and is divided in three parts: Part I, Part II, and Part III. Part I tries
to provide, in a concise manner, a certain number of fundamental notions of game
theory, such as mathematical models for a game and game-theoretic solution con-
cepts. Part II is dedicated to games with incomplete information; that is, those in
whichplayersdonotknowalltherelevantdataregardingthegametheyareplaying.
ThispartisstronglybasedonthenotionsdiscussedinPartI.Inparticular,iterative
algorithmsareconsideredforlearninggoodstrategies,andtheconvergenceanalysis
ofthesealgorithmsreliesongame-theoreticnotions(e.g.,potentialgamesandNash
equilibria).PartIIIcomprisesthreedetailedcasestudies,andshowshowgametheory
hasbeenappliedinthewirelessliterature.Itisbasedstronglyontheideasdiscussed
inPartsIandII.
Movingtoahigherresolutionlevelofthedescriptionofthebookcontents,PartI
comprises three chapters. In particular, the first chapter contains some motivating
examples for using game theory, the main mathematical models for a game (called
Lasaulce — 03-pref-xi-xvi-9780123846983 — 2011/8/4 — 12:39 — Page xiv — #4
xiv Preface
representations),andcommentsonvarioussolutionconceptsforagame.TheNash
equilibrium is one of the considered solution concepts. As the concept of Nash
equilibrium is a building block of game theory, a complete chapter (Chapter 2) is
dedicatedtoequilibriumanalysis(existence,uniqueness,etc).Indeed,throughoutthe
book,itshouldbecomecleartothereaderthatNashequilibriumisessentialtogame
theory since it both inspires new solution concepts and corresponds to the solution
ofmanyinteractivesituations;inparticular,Nashequilibriamayresultfromevolu-
tionorlearning.InChapter3,someimportantmodelsofdynamic,non-cooperative
gamesarepresented.By“non-cooperative”itismeantthattheindividualobjectives
oftheplayersareclearlyidentifiableandareassumedtobeselfish.Roughly,dynamic
games correspond to general game models, including one-shot games, where the
playerscanobservepastmovesordataandreacttothem.
ThepurposeofPartIIistoshowsomewaysofrelaxinginformationandbehavior
assumptions for the players, which is important in wireless communications (typi-
callyonlylocalobservationsareavailable,quantitiesareestimated,etc.).Chapter4
isaveryshortchapterpresentingBayesiangames,whichconsidershowtotakeinto
accountthefactthatonlypartialknowledgeisavailabletotheplayersofagame,and
showshowtheclassicalmachineryofequilibriumanalysisingameswithcomplete
information can be adapted. Chapters 5 and 6 are devoted to learning; that is, how
playerslearntheirstrategies.Insuchaframework,thegameisassumedtobeplayed
several/many times, so that repeated interaction between players who are partially
informed may converge to some steady points: players learn their strategies over
time.Chapter5focusesonsomelearningtechniqueswhicharepartiallydistributed;
that is, a player has some knowledge about the others. Chapter 6 focuses on fully
distributed learning techniques including reinforcement learning techniques. In this
setting,playersaremodeledasautomatawhoarethereforenotnecessarilyrational,
do not assume that rationality is common knowledge, and only observe a feedback
ontheirownperformancemetric.
Finally, Part III comprises three case studies. Chapter 7 describes typical power
allocationgames,wheretransmissionrateshavetobeoptimized.Chapter8focuses
onenergy-efficientpowercontrol,andChapter9isdevotedtomediumaccesscon-
trol. The purpose of these chapters is show how the important notions and models,
presented in Parts I and II (one-shot games, dynamic games, learning algorithms),
canbeappliedtotypicalwirelessproblems.Finally,PartIVprovidestheappendix,
references,andindex.Thepurposeoftheappendixistoreviewsomenotions(mainly
inmathematics,stochasticapproximation,andinformationtheory)whichcanmake
thereadingeasier.
ACKNOWLEDGMENTS
The authors would like to thank all the MSc and PhD students we have taught or
supervised in this area. Their feedback has contributed to making some parts of
Lasaulce — 03-pref-xi-xvi-9780123846983 — 2011/8/4 — 12:39 — Page xv — #5
Preface xv
this book more understandable. Also, we would like to thank the researchers who
probablyinfluencedthisbooksincetheauthorsattendedtheirclassesorseminarsand
interactedwiththem.Wewilljustmentionsomeofthem:ProfessorJo¨rgenWeibull
(Stockholm School of Economics, Sweden), Professor Sylvain Sorin (Universite´
PierreetMarieCurie–Paris6),ProfessorShmuelZamir(TheHebrewUniversityof
Jerusalem, Israel), Professor Rida Laraki (CNRS – E´cole Polytechnique, Palaiseau,
France),andProfessorEitanAltman(INRIA,SophiaAntipolis,France).
Lasaulce — 04-ch01-001-040-9780123846983 — 2011/8/4 — 13:55 — Page 3 — #1
CHAPTER
1
A Very Short Tour of Game
Theory
1.1 INTRODUCTION
First of all, what is game theory? It is always difficult to define a theory in a few
words without limiting its actual scope and hiding some of the fundamental ideas
which originate from it. This is especially true for theories which evolve consid-
erably over time. The essence of game theory is the formal study of interactions
betweenseveraldecision-makerswhocanhaveconflictingorcommoninterests.By
interactions,itismeantthatwhattheothersdohasanimpactoneachdecision-maker
(called a player), and what he/she gets from an interactive situation does not only
depend on his/her decisions. Although the focus of this book is clearly on wireless
games, it will be seen that the meaning of a player can be very broad; it can be a
human being, a machine, an automaton, an animal, a plant, etc. Very often, a ques-
tion arises concerning how game theory differs from optimization. Although some
people consider game theory as a part of optimization, it is important to note that
gameshavesomefeatureswhicharenotcommoninclassicaloptimizationproblems.
Incontrasttoaclassicaloptimizationproblemwhereacertainfunctionmustbeopti-
mizedundersomeconstraints,theoptimaldecisionorstrategyisgenerallyunclearin
interactivesituationsinvolvingseveraldecision-makers,sincenosingleplayercom-
pletelycontrolsthefinaloutcome.Thismeansthattheproblemneedstobedefined
beforeitcanbesolved.Afterreadingthisbook,thereadershouldbeconvincedthat
gametheorypossessesitsowntoolsandnotions,andisenrichedwithnotionscoming
from different areas, such as economics, biology, and computer science. This leads
to concepts we do not encounter in studying convex optimization1 – for instance
cooperativeplans,punishments,rationality,riskaversion,andunbeatablestrategies.
As said above, the meaning of a player can be very broad, and because of this,
games have existed in nature and human societies for a very long time. What is
more recent is the appearance of a theory of games. Game theory appeared gradu-
allythroughtheworksofBorel(1921),Cournot(1838),Darwin(1871),Edgeworth
(1881), Waldegrave (1713), Zermelo (1913) and Ville (1938), but the major works
byNash(1950)aregenerallyconsideredtobethosefoundwhichmarkedtheofficial
1Toelaboratefurtheronthispoint,notethatgametheoryanddistributedoptimizationobviouslyhave
manyintersections,butoneofthemaindifferencesbetweenthetwoapproaches,intheircurrentstate,
isthatthelatterassumesthattheagentsfollowsomegivenrules,andaregenerallycoordinated,while
intheformerthedecisionisgenerallylefttotheplayer.
GameTheoryandLearningforWirelessNetworks.DOI:10.1016/B978-0-12-384698-3.00001-3 3
Copyright(cid:13)c 2011ElsevierLtd.Allrightsreserved.
Lasaulce — 04-ch01-001-040-9780123846983 — 2011/8/4 — 13:55 — Page 4 — #2
4 CHAPTER 1 A Very Short Tour of Game Theory
birthofthetheoryofgames.Sincethen,gametheoryhasresoundingsuccessinmany
areas, such as biology, economics, and information sciences. It has been enriched
by numerous brilliant game-theorists, such as Allais, Aumann, Debreu, Harsanyi,
Maskin, Maynard-Smith, Myerson, Selten, to name just a few. The purpose of the
theoryofgamesistomodelinteractionsbetweenplayers,todefinedifferenttypesof
possibleoutcomefortheinteractivesituationunderstudy(solutionconcepts),topre-
dictthesolution(s)ofagameundergiveninformationandbehaviorassumptions,to
characterizeallpossibleoutcomesofthegame,andtodesigngoodstrategiestoreach
someoftheseoutcomes.
Beforedefiningthemainrepresentationsofagame(Sec.1.3),whichcorresponds
todefiningagamemathematically,wenowprovidefourexamples,wheretheideas
ofgametheorycanbeused.
1.2 A BETTER UNDERSTANDING OF THE NEED FOR GAME
THEORY FROM FOUR SIMPLE EXAMPLES
Beforedefiningthefundamentalconceptspresentedinthischapter,itisusefultocon-
sidercommonsituationswheregametheoryisanaturalparadigm.Wehavepicked
four examples. The first is not related to communication problems, but the other
three correspond to models used in the area of wireless networks. All these situa-
tions need a tool to predict or understand the outcome of the interactive situation
underconsideration.
Example 1: A Bankruptcy Problem
The problem is as follows. A man dies, leaving debts totaling more than the value of his
estate.Assumethattherearethreecreditorsandthedebtstothemare100,200,and300.
Howshouldtheestatebedividedamongthecreditors?Abouttwothousandyearsago,the
BabylonianTalmudgavetheanswer(reportedintheTable1.1).Whiletherecommendation
inthecasewheretheestateis100(theestateisdividedequally)seemsreasonable,those
Table1.1 Abankruptcyproblemand
thesolutionrecommendedbythe
Talmud:Theestateisdivided
betweenthreecreditors,depending
onhowmuchtheestateisworthand
theclaimsofthecreditors
Claim
100 200 300
100 100 100 100
3 3 3
e
at 200 50 75 75
st
E
300 50 100 150