Equilibria in Quitting Games - Basics

0.24 MB·

by  Katharina Fischer

ε
-Equilibria in Quitting Games – Basics
Katharina Fischer∗
January 13, 2011
1
1
0
Abstract
2
Quitting games are one of the simplest stochastic games in which at any stage each
n
a playerhasonlytwopossibleactions,continue andquit. Thegameendsassoonasatleast
J one playerchoosesto quit. The playersthenreceivea payoff,whichdepends onthe setof

2 playersthat didchooseto quit. Ifthe gameneverends,the payofftoeachplayeris zero.
1
Examples of quitting games were studied first by Flesh, Thuijsman and Vrieze in

] 1997([1]). Solan1999([6]) provedthat all three-playerquitting games haveapproximate
R
equilibria. In the paper Quitting Games SolanandVieille 2001([7]) provedthe existence
P
ofsubgame-perfectapproximateequilibriaundersomerestrictionsonthepayofffunction.
.
h FurthermoreSolanandVieillestudiedin[8]afour-playerquittinggameexampleinwhich
at the simplest equilibrium strategy is periodic with period two. In The structure of non-
m zero-sum stochastic games ([5]) Simon showed under which properties quitting games
have approximate equilibria among other things by generalization of the solution-idea
[
from Solan an Vieille.

1
v This paper gives a short introduction into the topic quitting games and tries to illus-
7 trate severalproperties with examples. First the mathematical model of a quitting game
3
is presented. After the definition of the strategy and strategy profile the corresponding
3
probability space and the underlying stochastic process are stated. This leads to the
2
. expected payoff and the definition of some equilibria.
1
For a better analysis of quitting games the so called one-step game is introduced in
0
the secondpartofthis paper. Importantpropertiesofstrategyprofilesinone-stepgames
1
1 are posted and proved.
: In the third section an important theorem from Solan and Vieille (cf. [7]) is cited,
v
in which the existence of approximate equilibria, under some assumptions to the payoff
i
X
function, is postulated. It’s proof is divided into three parts, however this paper concen-
r trates only on the first one. In the referred literature only a few steps of the proof are
a
denoted. It is the aimto showthe proofatlengthunder usage ofthe then knownresults.
1 The model
A quitting game is a sequential N-player (N ∈ N) game and played as follows. In every game
turneach player hasonly two possibleactions continue andquit. Thegame endsas soonas at
least one of the N-players chooses to quit. We denote S (quitting coalition) as the subset of
∗Thisarticleappearedundertheformerfamily nameHeimannintheconferenceproceedingsofthe”Inter-
national Symposium on DynamicGames and Applications”, Wro law 2008.
1
the players who choose to quit. If S = ∅ the players receive no payoff and the game continues
to the next stage. If S 6= ∅ each player n ∈ {1,...,N} receives the payoff rn ∈ R and the
S
game terminates.
Definition 1.1 (Quitting Game) A quitting game is a tuple
G = (N,(r ) ) (1)
S ∅⊆S⊆N
where
– N ={1,...,N} ⊂ N is a finite set of players, N ∈ N,
– S ∈P(N) denotes the quitting coalition and
– (r ) ∈ RN is a sequence of payoff-vectors to the players under the quitting coali-
S S∈P(N)
tion S with r = 0 (0 := (0,...,0)T ∈ RN) and r = (r1,...,rN)T.
∅ S S S
Remark 1.2 A quitting game is a special case of a (stochastic) game, where transition prob-
abilities are even deterministic. For comparison (cf. e.g. [4]):
– The state space is given by Z := {S | ∅ ⊆ S ⊆ N} = P(N).
– The action space is given by A:= {0,1}N, where 0 stands for continue and 1 for quit.
We denote a = (a1,...,aN)T as element of A with
S S S
0 for n ∈ N \S
an := ∀n∈ N, ∅ ⊆ S ⊆ N. (2)
S
(1 for n ∈ S
– The transition law t :Z ×A×Z → [0,1] is given by
1 for z = S
t(z|∅,a ) :=
S
(0 otherwise
1 for z = S˜
t(z|S˜,a ) := (3)
S
(0 otherwise
where z,S,S˜ ∈ Z, S˜ 6= ∅ and a ∈ A.
S
– The payoff function is given by r˜:A → RN, a 7→ r˜(a ):= r .
S S S
– There is no discounting in this model.
Example 1: A typical way to describe two- or three-player quitting games is in a matrix.
For example let two players be given. Player one is the so called row player and player two
the column player.
Player 2
continue quit
continue (cid:9) ( 1 , 1 )
Player 1
quit ( 1 ,−1 ) (−2 ,−2 )
2
Where (cid:9) means, that the players does not receive any payoff and the game continues to the
next round.
In this case the quitting game is given by
0 1 1 −2
G = {1,2}, r = ,r = ,r = ,r = .
∅ 0 {1} −1 {2} 1 N −2
(cid:18) (cid:18) (cid:18) (cid:19) (cid:18) (cid:19) (cid:18) (cid:19) (cid:18) (cid:19)(cid:19)(cid:19)
For further analysis the term “strategy profile” is needed.
Definition 1.3 (strategy profile, strategy) Let G = (N,(r ) ) be a given quitting
S ∅⊆S⊆N
game. A sequence of probability vectors π := (pi)i∈N with pi = (p1i,...,pNi )T ∈ [0,1]N is called
strategy profile in the quitting game G for the players 1,...,N. pn stands for the probability
i
that player n will play the action quit at stage i. The sequence πn := (pni)i∈N is called strategy
for player n, n ∈ N.
Let Π be the set of all strategy profiles for the given quitting game.
Definition 1.4 (subgame profile) Let G = (N,(r ) ) be a given quitting game and
S ∅⊆S⊆N
π = (pi)i∈N a strategy profile in G. For each j ∈ N, πj := (pi)j≤i∈N denotes the subgame
profile induced by π in the quitting game starting at time j.
Definition 1.5 (pure, cyclic, stationary) Let π = (pi)i∈N be a strategy profile in a quit-
ting game G. A strategy πn = (pni)i∈N for player n is called
– pure, if pn ∈ {0,1} for all i∈ N.
i
– cyclic, if a k ∈ N exists such that pn = pn for every k ∈N.
0 k k+k0
– stationary, if pn = pn for all k ∈ N.
k 1
A strategy profile π is called pure, if all strategies πn, n ∈ N, are pure. It is cyclic, if all
strategies are cyclic, and stationary, if all strategies are stationary.
Notation 1.6 Let π = (pi)i∈N be a strategy profile and π˜n = (p˜ni)i∈N an alternative strategy
for player n, n ∈N. We denote by π−n the strategy profile for the players j ∈N \{n} and by
(π−n,π˜n) an alternative strategy profile for player n in which the players j ∈ N \{n} carry
on playing π−n, that means
p1 p1 ...
p1 p1 ... 1 2
1 2 . .
.. ..  .. .. 
 . .  pn−1 pn−1 ...
pn−1 pn−1 ...  1 2 
π−n :=  1 2  and (π−n,π˜n):=  p˜n p˜n ... .
 pn+1 pn+1 ...   1 2 
 1 2   pn+1 pn+1 ... 
 ... ...   ..1 ..2 
   . . 
 pN pN ...   
 1 2   pN pN ... 
   1 2 
 
3
1.1 The underlying stochastic process
LetG= (N,(r ) )bethegiven quittinggame, Z ={S |∅ ⊆ S ⊆ N}thecorresponding
S ∅⊆S⊆N
state space and A = {0,1}N the corresponding action space (cf. remark 1.2). Set
Ω := (Z ×A)∞
and
A := P(Z)⊗P(A)⊗P(Z)⊗P(A)⊗....
Furthermore and without loss of generality let z = ∅ be the initial state. If a strategy profile
π = (pi)i∈N ∈ Π is given, a unique probability measure Pπ on (Ω,A) and a stochastic process
(Xk,Yk)k∈N with values in (Z ×A) exist, where
– X (ω) = X (z ,a ,z ,a ,...) := z
k k 1 1 2 2 k
(Xk denotes(cid:0)the random state(cid:1)of the system at time k, k ∈N, ω ∈ Ω),
– Y (ω) =Y (z ,a ,z ,a ,...) := a
k k 1 1 2 2 k
(Yk denotes(cid:0)the random actio(cid:1)n taken at time k, k ∈ N, ω ∈ Ω),
– H := (X ,Y ,...,X ),
k 1 1 k
that means H (ω) = H (z ,a ,z ,a ,...) = (z ,a ,z ,a ,...,z )
k k 1 1 2 2 1 1 2 2 k
(Hk describes the random(cid:0) history at time (cid:1)k, k ∈N, ω ∈ Ω)
hold and P is defined by
π
– P (X = ∅) := 1,
π 1
– P (X = z|H = (z ,a ,...,z ),Y = a) := t(z|z ,a)
π k+1 k 1 1 k k k
if P (H = (z ,a ,...,z ),Y = a) > 0 and
π k 1 1 k k
– P (Y = a|H = (z ,a ,...,z )) := pn (1−pm)
π k k 1 1 k k k
{n∈N:an=1} {m∈N:am=0}
Q Q
if P (H = (z ,a ,...,z )) > 0
π k 1 1 k
with z,z ∈ Z for all i= 1...,k and a,a ∈A for all i = 1,...,k−1.
i i
Equivalently, P can also be described as the unique probability measure on (Ω,A) for which
π
– P (X = ∅) := 1 and
π 1
– P (H = (z ,a ,z ,a ,...,z ))
π k 1 1 2 2 k
k−1
:= P (X = z ) t(z |z ,a )· pn (1−pm) ,
π 1 1 i+1 i i i i
i=1 {n∈N:ani=1} {m∈N:ami =0}
Q (cid:0) Q Q (cid:1)
where z ∈ Z for all i = 1,...,k and a ∈ A for all i = 1,...,k−1.
i i
4
1.2 Expected payoffs and equilibria
In this section the expected payoff for a given quitting game will be defined and additionally
the terms Nash-equilibria, ε-equilibria and approximate equilibria.
In [7] the expected payoff for a quitting game G = (N,(r ) ) and the strategy profile
S ∅⊆S⊆N
π ∈ Π is defined with a stopping time τ :Ω → N∪{+∞} where
τ(ω) := inf k ∈N : Y (ω) ∈ A\{(0,...,0)T}
k
concerning the fil(cid:8)tration (A ) with A := σ{Y :(cid:9)1 ≤ i ≤ k}.
k k∈N k i
Definition 1.7 (Expected payoff) Let G = (N,(r ) ) be a quitting game and π ∈ Π
S ∅⊆S⊆N
the chosen strategy profile. The expected payoff of the game is given by
γ(π) := E (r˜(Y )1 )
π τ {τ<∞}
with r˜from remark 1.2, γ(π) = (γ1(π),...,γN(π))T and E as expected value with respect to
π
the probability measure P .
π
With use of the definition of P and r˜(0) = r = 0 one obtains
π ∅
γ(π) = E (r˜(Y )1 )
π τ τ<∞
= P (τ = k,Y = a )·r˜(a )
π k k k
kX∈NaXk∈A
= P H = (∅,0,∅,0,...,∅),Y = a ·r˜(a )
π k−1 k k k
kX∈N(cid:16)aXk∈A (cid:0) (cid:1) (cid:17)
k−1
= (1−pn)· r˜(a )· pn (1−pm)
i k k k
kX∈N(cid:16)Yi=1nY∈N aXk∈A {n∈NY:ank=1} {m∈NY:amk =0} (cid:17)
and with remark 1.2 follows
k−1
γ(π) = (1−pn)· r · pn (1−pm) .
i S k k
kX∈N(cid:16)Yi=1nY∈N S∈XP(N) nY∈S m∈YN\S (cid:17)
Definition 1.8 (ε-equilibrium, Nash-equilibrium, approximate equilibria) Let G =
(N,(rS)∅⊆S⊆N) be a quitting game. A strategy profile π = (pi)i∈N is called ε-equilibrium
(ε ≥ 0) if for every player n ∈ N and every strategy π˜n of player n
γn(π) ≥ γn((π−n,π˜n))−ε (4)
holds.
The strategy profile π = (pi)i∈N is called Nash-equilibrium or (0−)equilibrium if π is an
ε-equilibrium for ε= 0.
A game has got approximate equilibria, if for all ε >0 an ε-equilibrium exists.
Definition 1.9 (subgame ε-equilibrium) Let G = (N,(r ) ) be a quitting game. A
S ∅⊆S⊆N
strategy profile π = (pi)i∈N iscalled subgame ε-equilibrium(ε ≥ 0)if for all j ∈ N the subgame
profile π is also an ε-equilibrium in G.
j
5
2 One-step game
The consideration of so called one-step games is an instrument for analyzing quitting games.
These games are also known as one-stage games ([5], p. 15) or as one-shot game ([7], p. 269).
Definition 2.1 (One-step game) Let G = (N,(r ) ) be a given quitting game. For
S ∅⊆S⊆N
every v ∈ RN the tuple
Γ := (G,v) = (N,(r ) ,v)
v S ∅⊆S⊆N
denotes the one-step(cid:0)game correspon(cid:1)ding to the quitting game G, where the players receive
the payoff v if S = ∅ and r otherwise (∅ =6 S ⊆N).
S
A one-step game has only one stage. The transition law, the state and action space are the
same as the transition law, the state and action space of the quitting game (cf. remark 1.2).
For the payoff function r˜ :A → RN
v
v if a = 0
a 7→ r˜ (a) :=
v
(r{n∈N|an=1} otherwise.
Definition 2.2 (Strategy profile, strategy in the one-step game) Let Γ = (G,v) be
v
a given one-step game. A vector p = (p1,...,pN)T ∈ [0,1]N is called strategy profile for the
one-step game Γ , where pn stands for the probability that player n will play the action quit.
v
pn denotes the strategy for player n, n ∈ N, in the one-step game Γ .
v
Notation 2.3 Letp ∈[0,1]N be astrategy profile fora one-stepgame Γ and p˜n astrategy for
v
player n. Similar to notation 1.6, p−n denotes the strategy profile for the players j ∈ N \{n}
and (p−n,p˜n) an alternative strategy profile for player n in which the players j ∈ N \ {n}
carry on playing p−n, that means
p1
p1
.
..  .. 
 .  pn−1
pn−1  
p−n :=   and (p−n,p˜n):=  p˜n .
 pn+1   
   pn+1 
 ...   .. 
   . 
 pN   
   pN 
   
 
Let Γ = (G,v) and p ∈ [0,1]N be given. Without loss of generality the game starts in
v
the state z = ∅. The corresponding probability space (Ω¯,A¯,P ) and the stochastic process
p
(X¯ ,Y¯ ,X¯ ) are defined by
1 1 2
– Ω¯ := Z ×A×Z,
– A¯:= P(Z)⊗P(A)⊗P(Z),
– X¯ (ω) = X¯ ((z ,a,z )) := z , i = 1,2,
i i 1 2 i
6
– Y¯ (ω) = Y¯ ((z ,a,z )) := a,
1 1 1 2
– P (X¯ = ∅) := 1,
p 1
P ({ω}) = P ((z ,a,z ))
p p 1 2
:= P (X¯ = z )·t(z |z ,a) pn (1−pm),
p 1 1 2 1
{n∈N:an=1} {m∈N:am=0}
Q Q
with ω ∈ Ω¯, z ,z ∈ Z, a ∈A.
1 2
Theexpected payoff γ for theone-step game Γ underthe strategy profilep ∈ [0,1]N is given
v v
by
γ (p) := E r˜ (Y )
v p v 1
= (cid:0) Pp(Y(cid:1)1 = aS)·r˜v(aS)
aXS∈A
= P (Y = a )·v+ P (Y = a )·r
p 1 ∅ p 1 S S
S∈P(N)
X
whereγ (p) = (γ1(p),...,γN(p))T andE istheexpectedvaluewithrespecttotheprobability
v v v p
measure P . γn(p) is the expected payoff for player n (n ∈ N) in the one-step game Γ under
p v v
the strategy profile p.
Notation 2.4 The function ̺: [0,1]N ×P(N) → [0,1],
(p,S) 7→ ̺(p,S) := pn (1−pm),
n∈S m∈N\S
Y Y
with p = (p1,...,pN)T, denotes the probability that a quitting coalition S or – equivalent to
that – an action a ∈ A is chosen under the vector p.
S
With this notation for the expected payoff γ under the strategy profile p ∈ [0,1]N follows
v
γ (p) = ̺(p,∅)·v+ ̺(p,S)·r .
v S
S∈P(N)
X
Proposition 2.5 Let Γ be a given one-step game and p ∈ [0,1]N a strategy profile in Γ .
v v
Then for the expected payoff γ (p)
v
γn(p) ∈ [−δ ,δ ]
v v v
holds for all n∈ N, where
δ := max max|vn|, max |rn| S ∈ P(N) . (5)
v S
n∈N
(cid:8) (cid:8) (cid:12) (cid:9)(cid:9)
(cid:12)
Proof: For all p ∈ [0,1]N and all n ∈ N
γn(p) = ̺(p,∅)·vn+ ̺(p,S)·rn
v S
S∈P(N)\{∅}
X
≤ ̺(p,∅)·|vn|+ ̺(p,S)·|rn|
S
S∈P(N)\{∅}
X
7
holds. With δ like in (5)
v
γn(p) ≤ ̺(p,∅)·δ + ̺(p,S)·δ
v v v
S∈P(N)\{∅}
X
= δ · ̺(p,∅)+ ̺(p,S)
v
S∈P(N)\{∅}
(cid:0) X (cid:1)
follows on the one hand and on the other
γn(p) ≥ −̺(p,∅)·|vn|− ̺(p,S)·|rn|
v S
S∈P(N)\{∅}
X
≥ −δ · ̺(p,∅)+ ̺(p,S) .
v
S∈P(N)\{∅}
(cid:0) X (cid:1)
With
̺(p,∅)+ ̺(p,S) = 1,
S∈P(N)\{∅}
X
γn(p) ∈ [−δ ,δ ] holds for all p ∈ [0,1]N and n ∈ N.
v v v
(cid:4)
In order to show that the expected payoff γ (p) is linear in the strategy pn of player n for all
v
n ∈ N the following proposition is needed.
Proposition 2.6 For all p ∈ [0,1]N, all S ∈ P(N) and all i ∈ N
̺(p,S) = pi·̺ (p−i,1),S +(1−pi)·̺ (p−i,0),S
(cid:0) (cid:1) (cid:0) (cid:1)
holds.
Proof:
Case 1: i∈ S
Because
̺(p,S) = pn (1−pn)= pi · pn (1−pn),
n∈S n∈N\S n∈S\{i} n∈N\S
Y Y Y Y
with (p−i,1)i = 1 and pn = (p−i,1)n for all n ∈ N \{i}, where (p−i,1)n denotes the n-th
component of the alternative strategy profile for player i,
̺(p,S) = pi·(p−i,1)i · (p−i,1)n 1−(p−i,1)n
n∈S\{i} n∈N\S
Y Y (cid:0) (cid:1)
= pi· (p−i,1)n 1−(p−i,1)n
n∈S n∈N\S
Y Y (cid:0) (cid:1)
= pi·̺ (p−i,1),S
follows. (cid:0) (cid:1)
Case 2: i∈ N \S
8
Because
̺(p,S) = (1−pi)· pn (1−pn),
n∈S n∈N\(S∪{i})
Y Y
with (p−i,0)i = 0 and pn = (p−i,0)n for all n ∈ N \{i}
̺(p,S) = (1−pi)· 1−(p−i,0)i · (p−i,0)n 1−(p−i,0)n
n∈S n∈N\(S∪{i})
(cid:0) (cid:1) Y Y (cid:0) (cid:1)
= (1−pi)· (p−i,0)n 1−(p−i,0)n
n∈S n∈N\S
Y Y (cid:0) (cid:1)
= (1−pi)·̺ (p−i,0),S
follows. (cid:0) (cid:1)
Because of ̺ (p−i,1),S = 0 for i ∈ N \S and ̺ (p−i,0),S = 0 for i ∈S, case 1 and case 2
imply the proposition.
(cid:0) (cid:1) (cid:0) (cid:1)
(cid:4)
Proposition 2.7 Let Γ be a given one-step game. Then for all p ∈ [0,1]N and all n ∈ N
v
γ (p) = γ ((p−n,0))+pn· γ ((p−n,1))−γ ((p−n,0))
v v v v
holds, that means the expected p(cid:0)ayoff γ (p) is linear in th(cid:1)e strategy pn of player n for all
v
p ∈ [0,1]N and all n ∈ N.
Proof:
γ (p) = ̺(p,∅)·v+ ̺(p,S)·r
v S
S∈P(N)
X
With proposition 2.6 and ̺ (p−n,1),∅ = 0 one obtains for all n ∈ N
(cid:0) (cid:1)
γ (p) = pn·̺ (p−n,1),∅ +(1−pn)·̺ (p−n,0),∅ ·v
v
(cid:16) (cid:17)
+ (cid:0) pn·̺ ((cid:1)p−n,1),S +(1(cid:0)−pn)·̺ ((cid:1)p−n,0),S ·r
S
S∈XP(N)(cid:16) (cid:0) (cid:1) (cid:0) (cid:1)(cid:17)
= (1−pn)·̺ (p−n,0),∅ ·v+(1−pn) ̺ (p−n,0),S ·r
S
S∈P(N)
(cid:0) (cid:1) X (cid:0) (cid:1)
+ pn ̺ (p−n,1),S ·r
S
S∈P(N)
X (cid:0) (cid:1)
Furthermore
γ ((p−n,0)) = ̺ (p−n,0),∅ ·v+ ̺((p−n,0),S)·r
v S
S∈P(N)
(cid:0) (cid:1) X
9
and
γ ((p−n,1)) = ̺((p−n,1),S)·r .
v S
S∈P(N)
X
That implies
γ (p) = (1−pn)·γ ((p−n,0))+pn·γ ((p−n,1)).
v v v
(cid:4)
Conclusion 2.8 Let p ∈ [0,1]N be a strategy profile in the one-step game Γ . The following
v
equations hold:
1. γ ((p−n,1)) = piγ (p−n,1)−i,1 +(1−pi)γ (p−n,1)−i,0
v v v
2. γ ((p−n,0)) = piγ (cid:0)(cid:0)(p−n,0)−i,1(cid:1)(cid:1)+(1−pi)γ (cid:0)(cid:0)(p−n,0)−i,0(cid:1)(cid:1)
v v v
for all i,n ∈ N, i 6= n. (cid:0)(cid:0) (cid:1)(cid:1) (cid:0)(cid:0) (cid:1)(cid:1)
Definition 2.9 (ε-equilibrium, Nash-equilibrium, approximate equilibria of the
one-step game) Let Γ be a one-step game corresponding to a quitting game G. The strategy
v
profile p ∈ [0,1]N is called an ε-equilibrium for ε ≥0 if
∀n ∈N ∀p˜n ∈ [0,1] : γn(p) ≥ γn (p−n,p˜n) −ε.
v v
(cid:0) (cid:1)
If p is an ε-equilibrium with ε = 0, p is also called (Nash-)equilibrium.
A one-step game Γ has got approximate equilibria, if for all ε > 0 an ε-equilibrium in Γ
v v
exists.
Becauseofthelinearity oftheexpectedpayoffγ (p)inthestrategies pn (n ∈ N)itissufficient
v
to consider the expected payoff only for pure strategies in order to find out whether a given
strategy profile in a one-step game is an equilibrium or not, since the extreme values of γ (p)
v
is for each single player attained in a border point. The following example illustrates this
fact.
Example 2: Consider example 1 again. The corresponding one-step game Γ for a vector
v
v = (v1,v2)T ∈ R2 is given by
Player 2
continue quit
continue ( v1 , v2) ( 1 , 1 )
Player 1
quit ( 1 ,−1 ) ( −2 ,−2 )
Consider four different given v’s:
10