The M Algorithm: Incorporating Opponent Models into Adversary Search

29 Pages·2004·0.65 MB·English

Technion (cid:13) Israel Institute of Technology
Computer Science Department
4
9
9
1

-

2
0
4
9
S
I
C

(cid:0)
rt The M Algorithm(cid:3) Incorporating Opponent Models
o
into Adversary Search
p
e
R

l
a
c by
i
n
h
c
e
T David Carmel and Shaul Markovitch

-

t
n
e CIS Report (cid:31)(cid:8)(cid:17)(cid:12)(cid:11)
m
March (cid:6)(cid:8)(cid:8)(cid:17)
t
r
a
p
e
D

e
c
n
e
i
c
S

r
e
t
u
p
m
o
C

-

n
o
i
n
h
c
e
T
(cid:0)
4 M
The Algorithm(cid:0) Incorporating Opponent Models into Adversary
9
9
1 Search

-

2
0 (cid:0)
4 David Carmel and Shaul Markovitch
9
S
I Computer Science Department
C
Technion(cid:0) Haifa (cid:1)(cid:2)(cid:3)(cid:3)(cid:3)

t Israel
r
o
carmel(cid:4)cs(cid:5)technion(cid:5)ac(cid:5)il shaulm(cid:4)cs(cid:5)technion(cid:5)ac(cid:5)il
p
e
R

l
a
c
i Abstract
n
h
c While human players adjust their playing strategy according to their opponent(cid:4) computer
e
T programs(cid:4)which are based on the minimaxalgorithm(cid:4)use tha same playingstrategy against a
novice as against an expert(cid:0) This is due to the assumption of minimaxthat the opponent uses
-
the samestrategy as the player(cid:0) This work studies the problemof opponent modellingin game
t
n
playing(cid:0) We recursively de(cid:13)ne a player as a pair of astrategy and an opponent model(cid:4)which is
e
m also a player(cid:0) A strategy can be determined by the static evaluation function and the depth of
t search(cid:0) M(cid:0)(cid:4) an algorithmfor searching game(cid:14)trees using an n(cid:14)level modelling player that uses
r
a such a strategy(cid:4) is described and analyzed(cid:0) Wedemonstrate experimentallythe bene(cid:13)t of using
p
anopponent modelandshow the potentialharmcaused bythe use ofan inaccurate model(cid:0) We
e
D then describe an algorithm(cid:4)M(cid:3)(cid:0)(cid:4) for using uncertain models when a bound on the model error
is known(cid:0) Pruning in M(cid:0) is impossiblein the general case(cid:0) We prove a su(cid:16)cient condition for
e
c pruning and present a pruning algorithm(cid:4)(cid:1)(cid:2)(cid:0)(cid:4) that returns the M(cid:0) value of a tree(cid:4) searching
n
only necessary subtrees(cid:0) Finally(cid:4) we present a method for acquiring a model for an unknown
e
i player(cid:0) First(cid:4) we describe a learning algorithm that acquires a model of the opponent(cid:1)s depth
c
S ofsearch by using its past movesas examples(cid:0) Then(cid:4) an algorithmfor acquiringa modelof the

r player(cid:1)s strategy(cid:4) both depth and function(cid:4) is described and evaluated(cid:0) Experiments with this
e
t algorithmshowthat when a superset ofthe set offeatures used by a(cid:13)xed opponent is available
u
tothe learner(cid:4)fewexamplesare su(cid:16)cientforlearningamodelthatagrees almostperfectly with
p
m the opponent(cid:0)
o
C

- (cid:11) Introduction
n
o
i
n (cid:26)(cid:3)(cid:3)(cid:3)At the press conference(cid:5) it quickly became clear that Kasparov had done his home(cid:13)
h
work(cid:3) He admitted that he had reviewed about (cid:19)fty of DEEP THOUGHT(cid:18)s games and
c
e felt con(cid:19)dent he understood the machine(cid:3)(cid:3)(cid:3)(cid:27) (cid:0)(cid:6)(cid:6)(cid:2)
T
One ofthe mostnotable challenges that the Arti(cid:19)cial Intelligence research community has been
trying to face during the last (cid:19)ve decades is the creation of a computer program that can beat
the world chess champion(cid:3) Most activity in the area of game(cid:13)playing programshas been concerned
with e(cid:20)cient ways of searching large game trees(cid:3) However(cid:5) good playing performance involves
(cid:0)
This research was partially supported by the Fund forthe Promotion of Research at the Technion
(cid:6)
additional types of intelligent processes(cid:3) The quote above highlights one type of such a process
that is performed by expert human players(cid:16) acquiring a model of their opponent(cid:18)s strategy(cid:3)
While human players adjust their playing strategy according to their opponent(cid:5) computer pro(cid:13)
4
9 gramsplay thesameagainstanovice asagainstanexpert(cid:3) Mostplaying programsuse theminimax
9
algorithm for search (cid:0)(cid:6)(cid:8)(cid:2)(cid:3) The main assumption of this algorithm is that the opponent uses the
1
same strategy as the player(cid:3)
-

2
Thereareseveralsituationswherethemodelling approachhasadvantageoverthenon(cid:13)modelling
0
4 approach of the standard minimax procedure(cid:3) Jansen (cid:0)(cid:1)(cid:2)(cid:5) describes two situations in which it is
9
S important to consider the opponent(cid:18)s playing ability(cid:3) One is a swindle position(cid:5) where the player
I has reason to believe that the opponent will underestimate a good move(cid:5) and will therefore play a
C
poorer move instead(cid:3) Another situation is a trap position(cid:5) where the player expects the opponent

t
r tooverestimateand therefore play a bad move(cid:3) Choosing traporswindle positions is good strategy
o
p when the player has reason to believe that its opponent searches to shallower depth than itself(cid:3)
e
Anothersituation(cid:5)where anopponent modelcan bebene(cid:19)cial(cid:5) isalosing position(cid:0)(cid:11)(cid:2)(cid:3) Ifall possible
R
moves lead to a loss(cid:5) minimax chooses one of them arbitrarily(cid:5) in contrast to human players that
l
a
can utilize their opponent model in order to select a swindle move(cid:3)
c
i
n Several researchers have pointed out the importance of opponent modelling (cid:0)(cid:6)(cid:1)(cid:5) (cid:11)(cid:5) (cid:10)(cid:5) (cid:8)(cid:5) (cid:6)(cid:5) (cid:6)(cid:10)(cid:2)(cid:5)
h
c but the acquisition and utilization of an opponent model have not received much attention in the
e
T computational game research community(cid:3) Korf (cid:0)(cid:8)(cid:2) outlined a method of generalizing the minimax
algorithm for utilizing multiple(cid:13)level models of evaluation functions(cid:3) The work described in this
-

t paper builds on Korf(cid:18)s research and expands it in several ways(cid:3)
n
e
m Thegoal ofthis researchis tostudy the utilization and acquisition of opponentmodels in game(cid:13)
t playing(cid:3) In order to do so(cid:5) we will make an attempt to (cid:19)nd answers to the following questions(cid:16)
r
a
p
e (cid:6)(cid:3) What is a model of opponent(cid:18)s strategy(cid:29)
D
e (cid:11)(cid:3) Assuming that we possess such a model(cid:5) how can we utilize it(cid:29)
c
n
(cid:15)(cid:3) What are the potential bene(cid:19)ts of using opponent models(cid:29)
e
i
c
S (cid:17)(cid:3) How does the accuracy of the model e(cid:4)ect its bene(cid:19)t(cid:29)

r
e (cid:14)(cid:3) How can we use an uncertain model(cid:29)
t
u
p
(cid:7)(cid:3) How can a program acquire a model of its opponent(cid:29)
m
o
C Section (cid:11) of this paper deals with the (cid:19)rst two questions(cid:16) De(cid:19)ning an opponent model and de(cid:13)
- veloping algorithms forusing suchamodel(cid:3) Section (cid:15)deals withthe third andthe fourthquestions(cid:16)

n Measuring the potential bene(cid:19)ts of modelling and testing the e(cid:4)ects of modelling accuracy on its
o
i bene(cid:19)ts(cid:3) In section (cid:17)(cid:5) we describe an algorithm for using uncertain models(cid:3) Section (cid:14) describes a
n
h method for incorporating pruning into the algorithms(cid:3) Section (cid:7) describes algorithms for acquiring
c
opponent models(cid:3) Finally(cid:5) section (cid:1) concludes(cid:3)
e
T
(cid:10) Using opponent models
Every human player has either an explicit or an implicit model of the way that its opponent plays(cid:3)
In this section wegive aprecise de(cid:19)nition ofan opponent model and develop analgorithm forusing
such a model(cid:3)
(cid:11)
De(cid:8)nition (cid:3) A player is de(cid:1)ned by the strategy that it uses and by the model of its opponent(cid:3) The
opponent model is also a player(cid:3)
4 (cid:12)(cid:3) Given a strategy S(cid:0) P (cid:25) (cid:23)S(cid:3)NIL(cid:24) is a player(cid:3)
9
9
1 (cid:5)(cid:3) Given a strategy S and a player O(cid:0) P (cid:25) (cid:23)S(cid:3)O(cid:24) is a player(cid:3)

-

2
0 The (cid:1)rst element of a player is called the player(cid:18)s strategy and the second element is called the
4 opponent model(cid:3)
9
S
I
C De(cid:8)nition (cid:7) Given a player P (cid:25) (cid:23)S(cid:3)O(cid:24)(cid:3) The modelling level of a player is de(cid:1)ned by the recur(cid:8)

rence(cid:16)
t
r (cid:12) if O (cid:25) NIL
o (cid:23)(cid:6)(cid:24) ML(cid:23)P(cid:24) (cid:25)
p (cid:1) ML(cid:23)O(cid:24)(cid:28)(cid:6) otherwise
e
R

al Thus(cid:5) a zero(cid:13)level modelling player(cid:5) (cid:23)S(cid:2)(cid:3)NIL(cid:24)(cid:5)is one that does not model its opponent(cid:3) A one(cid:13)
c
i level modelling player(cid:5) (cid:23)S(cid:0)(cid:3)(cid:23)S(cid:2)(cid:3)NIL(cid:24)(cid:24)(cid:5) is one that has a model of its opponent(cid:5) but assumes that
n
h its opponent is a zero(cid:13)level modelling player(cid:3) A two(cid:13)level modelling player(cid:5) (cid:23)S(cid:4)(cid:3)(cid:23)S(cid:0)(cid:3)(cid:23)S(cid:2)(cid:3)NIL(cid:24)(cid:24)(cid:24)(cid:5)
c is one that uses a strategy S(cid:4)(cid:5) and has a model of its opponent(cid:5) (cid:23)S(cid:0)(cid:3)(cid:23)S(cid:2)(cid:3)NIL(cid:24)(cid:24)(cid:3) The opponent(cid:18)s
e
T model uses a strategy S(cid:0) and has a model(cid:5) (cid:23)S(cid:2)(cid:3)NIL(cid:24)(cid:5) of the player(cid:3) The recursive de(cid:19)nition of a

- player is in the spirit of the Recursive Modelling Method (cid:23)RMM(cid:24) by Gmytrasiewicz(cid:5) Durfee and

nt Wehe (cid:0)(cid:6)(cid:14)(cid:2)(cid:3)
e
m
(cid:0)
t
r (cid:4)(cid:1)(cid:6) The M algorithm
a
p
e
Most of the game(cid:13)playing programs use a minimax search procedure in which the player evaluates
D
boards by a function f(cid:5) and believes that the opponent evaluates boards by the function f(cid:3)
e (cid:1)
c Assume that the player uses a function f(cid:0)(cid:5) but believes that the opponent uses a di(cid:4)erent function
n
e f(cid:2)(cid:3) What is a good search strategy that incorporates this belief(cid:29) What is a good search strategy
ci when the player uses a function f(cid:4)(cid:5) believes that the opponent uses function f(cid:0)(cid:5) and also believes
S
that the opponent believes that the player evaluates boards using a third function f(cid:2)(cid:29)

r
e
t We have developed an algorithm(cid:5) M(cid:0)(cid:5) shown in (cid:19)gure (cid:6)(cid:5) that can handle such modelling to
u (cid:0)
p any level (cid:3) Let us assume that a playing strategy is determined by a static evaluation function
m f(cid:3) A player is then(cid:5) according to de(cid:19)nition (cid:6)(cid:5) a pair (cid:23)f(cid:3)MODEL(cid:24) where MODEL is also a player(cid:3)
o The input for the algorithm is a position(cid:5) a depth limit(cid:5) and a player(cid:5) and the output is the move
C
selected by the player and its value(cid:3)

-

n The algorithm generates the successor boards and simulates the opponent(cid:18)s search from each of
o
i themin ordertoanticipate itschoice(cid:3) Thissimulation is donebyapplying thealgorithmrecursively
n
h with the opponent model as the player(cid:3) The player then evaluates each of its optional moves by
c
evaluating the outcome of its opponent(cid:18)s reaction by applying the algorithm recursively using its
e
T own strategy(cid:3)
Figure (cid:11) shows an example for a search tree spanned by M(cid:0)(cid:23)a(cid:3)(cid:15)(cid:3)f(cid:4)(cid:23)f(cid:0)(cid:3)f(cid:2)(cid:24)(cid:24)(cid:3) The numbers at
the bottom are the static values of the leaves(cid:3) The recursive calls applied to each node are listed
next to the node(cid:3) The dashed lines indicate what move is selected by each recursive call(cid:3)
(cid:6) (cid:0)
We have presented an earlier version of the M algorithm in a previous publication (cid:7)(cid:8)(cid:9)(cid:2) However(cid:1) the earlier
version was limited to one(cid:3)level modelling players(cid:2)
(cid:15)
Procedure M(cid:0)(cid:23)pos(cid:3)depth(cid:3)(cid:23)fpl(cid:3)OPP MODEL(cid:24)(cid:24)
if depth (cid:25) (cid:12)
4 return NIL(cid:3)fpl(cid:23)pos(cid:24)
9 h i
9 else
1 SUCC MoveGen(cid:23)pos(cid:24)
- (cid:0)
for each succ SUCC
2 (cid:7)
0 if depth (cid:25) (cid:6)
4
player value fpl(cid:23)succ(cid:24)
9
(cid:0)
S else
I
C opp board(cid:3)opp value M(cid:0)(cid:23)succ(cid:3)depth (cid:6)(cid:3)OPP MODEL(cid:24)
h i(cid:0) (cid:1)
player board(cid:3)player value M(cid:0)(cid:23)opp board(cid:3)depth (cid:11)(cid:3)(cid:23)fpl(cid:3)OPP MODEL(cid:24)(cid:24)
t h i(cid:0) (cid:1)
r if player value(cid:6) max value
o
p max value player value
e (cid:0)
R max board succ
(cid:0)
return max board(cid:3)max value
l
a h i
c
i
n
h
c
e Figure (cid:6)(cid:16) The M(cid:0) algorithm
T

-

t The player simulates its opponent(cid:18)s search from nodes b and c(cid:3) The opponent simulates the
n
e player by using its own model of the player from nodes d and e(cid:3) At node d the model of the player
m
used by the opponent (cid:23)f(cid:2)(cid:24) selects node h(cid:3) The opponent then applies its f(cid:0) function to node h and
t
r concludes thatnode h(cid:5) and therefore node d(cid:5)are worth (cid:7)(cid:3) The opponent then applies the player(cid:18)s
a
(cid:1)
p model (cid:23)f(cid:2)(cid:24) to node e(cid:5) concludes that the player will select node j(cid:5) applies its own function (cid:23)f(cid:0)(cid:24)
e
D to node j(cid:5) and decides that node j(cid:5) and therefore node e(cid:5) are worth (cid:10)(cid:3) Therefore(cid:5) the opponent
(cid:1)
model(cid:5) when applied to node b(cid:5) selects the move that leads to node d(cid:3) The player then tests how
e
c much node d is worth according to its criterion (cid:23)f(cid:4)(cid:24)(cid:3) It applies M(cid:0) to node d and concludes that
n
e node d(cid:5) and therefore node b(cid:5) are worth (cid:10)(cid:3) Using a similar search from node c yields (cid:6)(cid:12)(cid:3) Therefore(cid:5)
i
c the player selects the move that leads to c with a value of (cid:6)(cid:12)(cid:3) Note that using a regular minimax
S
search with f(cid:4) would have resulted in selecting the move leads to node b with a value of (cid:1)(cid:3)
r
e
t How does minimax (cid:19)t into our new algorithm(cid:29) It turns out that a minimax search to depth
u
p d with evaluation function f is in fact equivalent to the M(cid:0) algorithm called with the d(cid:13)level
m d
o
modelling player (cid:23)f(cid:3)(cid:23) f(cid:3)(cid:23)f(cid:3)(cid:5)(cid:5)(cid:5)(cid:24)(cid:24)(cid:24)(cid:3)
C
(cid:1)
z (cid:10)(cid:11) (cid:12)
- There is one potential complication that deserves special handling(cid:3) Each simulation of the

n
opponent(cid:18)ssearchinvolvesacalltoM(cid:0) withalowerlevel modelling player(cid:3) Whatwill theprocedure
o
i do if the modelling level of the original player n is smaller than the depth of the search tree d(cid:29)
n
h In such a case(cid:5) we assume that the (cid:12)(cid:13)level modelling player f(cid:2) is playing minimax(cid:5) and therefore
c d n
e (cid:1)
T
replace it by (cid:23)f(cid:2)(cid:3)(cid:23) f(cid:2)(cid:3)(cid:23)f(cid:2)(cid:3)(cid:5)(cid:5)(cid:5)(cid:24)(cid:24)(cid:24)(cid:3)
(cid:1)
z (cid:10)(cid:11) (cid:12)
It is obvious that the M(cid:0) algorithm does multiple expansions of parts of the search tree(cid:3) For
analyzing the complexity of M(cid:0)(cid:5) let us assume that the search tree has a uniform branching factor
b(cid:3) The number of leaves spanned by M(cid:0) for such a tree with depth d can be measured by the
(cid:17)
a M*(f2,(f1,f0))=10
M*(f1,f0)=-6 b c M*(f1,f0)=0
4
9
9
1 M*(f0)=4 d M*(f0)=10 e f M*(f0)=4 g M*(f0)=6

- M*(f2,(f1,f0))=8
M*(f2,(f1,f0))=10
2
0
4 h i j k l m n o
9
M*(f1,f0)=-6 M*(f1,f0)=-8 M*(f1,f0)=-2 M*(f1,f0)=0
S
I f2= 8 f2=-4 f2= 4 f2= 7 f2=-6 f2= 1 f2=10 f2= 2
C f1=-6 f1= 6 f1=-8 f1=-7 f1= 7 f1=-2 f1=-4 f1= 0
f0= 4 f0=-8 f0=10 f0= 3 f0=-4 f0= 4 f0= 4 f0= 6

t
r
o
p Figure (cid:11)(cid:16) The set of recursive calls generated by calling M(cid:0)(cid:10)a(cid:4)(cid:6)(cid:4)(cid:10)f(cid:4)(cid:10)f(cid:0)(cid:4)f(cid:2)(cid:11)(cid:11)(cid:11)(cid:0) For sake of clarity we have
e
R included only the player in the parameter list of the calls(cid:0) Each call is written next to the node it is called
from(cid:0) The dashed lines show what moveis selected by each call(cid:0)
l
a
c
i
n following recurrence (cid:16)
h
c
e (cid:6) if d(cid:25) (cid:12)
T
(cid:23)(cid:11)(cid:24) Tb(cid:23)d(cid:24)(cid:25) (cid:2) b if d(cid:25) (cid:6)
-
t (cid:3)(cid:4) b(cid:0)Tb(cid:23)d (cid:6)(cid:24)(cid:28)Tb(cid:23)d (cid:11)(cid:24)(cid:2) otherwise
n (cid:1) (cid:1)
e (cid:3)
m Lemma (cid:3) (cid:12)(cid:3) The number of l(cid:5)eaves spanned by M(cid:0) for a tree with uniform branching factor b
rt and depth d is
a d(cid:9)(cid:0) d(cid:9)(cid:0)
p (cid:4)b (cid:4)b
e (cid:23)(cid:15)(cid:24) Tb(cid:23)d(cid:24)(cid:25) (cid:4)(cid:1)
D pb (cid:28)(cid:17)b
ce where (cid:4)b (cid:25) b(cid:9)pb(cid:4)(cid:1)(cid:9)(cid:8)b(cid:0) and (cid:4)b (cid:25) b(cid:1)pb(cid:4)(cid:1)(cid:9)(cid:8)b (cid:4)
n
d d
e (cid:5)(cid:3) Tb(cid:23)d(cid:24) (cid:4)b (cid:23)b(cid:28)(cid:6)(cid:24)
i (cid:5) (cid:5)
c
S
Proof(cid:1)
r
e By induction on d(cid:16) It is easy to verify equation (cid:15) for d (cid:25) (cid:12) and d(cid:25) (cid:6)(cid:3) Assume it(cid:18)s correctness
t
u
for depths less than d(cid:3)
p
m
o (cid:4)db (cid:4)bd (cid:4)db(cid:1)(cid:0) (cid:4)bd(cid:1)(cid:0)
C (cid:23)(cid:17)(cid:24) Tb(cid:23)d(cid:24)(cid:25) b(cid:0)Tb(cid:23)d (cid:6)(cid:24)(cid:28)Tb(cid:23)d (cid:11)(cid:24)(cid:2)(cid:25) b (cid:1) (cid:28) (cid:1) (cid:25)
(cid:1) (cid:1) (cid:6)pb(cid:4)(cid:28)(cid:17)b pb(cid:4)(cid:28)(cid:17)b (cid:8)
-
n (cid:7) (cid:9)
nio (cid:25) pb(cid:1)b(cid:9)(cid:8)b (cid:23)(cid:4)b(cid:28)(cid:6)(cid:24)(cid:4)db(cid:1)(cid:0) (cid:23)(cid:4)b(cid:28)(cid:6)(cid:24)(cid:4)bd(cid:1)(cid:0)
(cid:1)
h h i
c
(cid:4)
e (cid:4)
T It is also easy to verify that b(cid:23)(cid:4)b (cid:28) (cid:6)(cid:24) (cid:25) (cid:4)b and b(cid:23)(cid:4)b (cid:28) (cid:6)(cid:24) (cid:25) (cid:4)b (cid:3) Placing the two equalities in
equation (cid:17) completes the proofof the (cid:19)rst statement(cid:3) Forthe second statement(cid:5)the (cid:19)rstinequality
will be proved by induction on d(cid:3) It is easy to verify the inequality for d (cid:25) (cid:12) and d (cid:25) (cid:6)(cid:3) Assume
its correctness for depth (cid:7) d(cid:3)
d (cid:0) d (cid:4) d (cid:4) d
(cid:23)(cid:14)(cid:24) Tb(cid:23)d(cid:24)(cid:25) b(cid:0)Tb(cid:23)d (cid:6)(cid:24)(cid:28)Tb(cid:23)d (cid:11)(cid:24)(cid:2) b (cid:4)b(cid:1) (cid:28)(cid:4)b(cid:1) (cid:25) b (cid:23)(cid:4)b(cid:28)(cid:6)(cid:24)(cid:4)b(cid:1) (cid:25) (cid:4)b
(cid:1) (cid:1) (cid:5)
(cid:5) h i h i
For b(cid:5)(cid:6) weget the Binet formula for Fibonacci sequence(cid:2)
(cid:14)
The second inequality follows from the inequality b(cid:0)(cid:4)b b(cid:28)(cid:6)(cid:2)(cid:3) (cid:0)
(cid:12) (cid:5)
d d
Therefore(cid:5) the number of leaves spanned by M(cid:0) is bounded by (cid:23)b(cid:28)(cid:6)(cid:24) (cid:5) while minimax spans b
leaves(cid:3) For games with large branching factors(cid:5) the overhead of using M(cid:0) (cid:23)compared to minimax(cid:24)
4
9 is therefore neglectable(cid:5) while for games with small branching factors it is quite signi(cid:19)cant(cid:3)
9
1
(cid:0)
-
(cid:4)(cid:1)(cid:4) A one(cid:11)pass version of M
2
0
4
We have developed another version of the M(cid:0) algorithm(cid:5) called M(cid:0)(cid:0) pass(cid:5) that expands the tree one
9 (cid:1)
S time only(cid:5) just as minimax does(cid:3) The algorithm expands the search tree in the same manner as
I
C minimax(cid:3) However(cid:5) node values are propagated di(cid:4)erently(cid:3) Whereas minimax propagatesonly one
value(cid:5) M(cid:0) propagates n(cid:28)(cid:6) values(cid:5) (cid:23)Vn(cid:3)(cid:5)(cid:5)(cid:5)(cid:3)V(cid:2)(cid:24)(cid:3) The value Vi represents the merit of the current
t
r node according to the i(cid:13)level model(cid:5) fi(cid:3) M(cid:0) passes values to V di(cid:4)erently for values associated
o
p with the player and values associated with the opponent(cid:3) In a player(cid:18)s node (cid:23)a node where it is the
e (cid:10)
R player(cid:18)s turn to play(cid:24) (cid:5) for values associated with the player (cid:23)Vn(cid:3)Vn (cid:4)(cid:3)(cid:5)(cid:5)(cid:5)(cid:24)(cid:5) Vi gets the maximal
(cid:1)
Vi value among its children(cid:3) For values associated with the opponent (cid:23)Vn (cid:0)(cid:3)Vn (cid:10)(cid:3)(cid:5)(cid:5)(cid:5)(cid:24) (cid:5) Vi gets
l (cid:1) (cid:1)
a
c the Vi value of the child that gave the maximal value to Vi(cid:1)(cid:0)(cid:3) For example(cid:5) the opponent believes
ni (cid:23)according tothe model(cid:24) thatthe player evaluates nodes by Vn (cid:4)(cid:3) Ata player(cid:18)snode(cid:5) the opponent
(cid:1)
h assumes that the player will select the child with maximal Vn (cid:4) value(cid:3) Therefore(cid:5) the value of the
c (cid:1)
e current node for the opponent is the Vn (cid:0) value of the selected child with the maximal Vn (cid:4) value(cid:3)
T (cid:1) (cid:1)
At an opponent(cid:18)s node(cid:5) we do the same but the roles of the opponent and the player are switched(cid:3)

-
nt Figure (cid:15) lists the M(cid:0)(cid:0)(cid:1)pass algorithm(cid:3)
e
m
rt Procedure M(cid:0)(cid:0) pass(cid:23)pos(cid:3)depth(cid:3)(cid:23)fn(cid:3)(cid:23)fn (cid:0)(cid:3)(cid:23)(cid:5)(cid:5)(cid:5)(cid:3)f(cid:2)(cid:24)(cid:5)(cid:5)(cid:5)(cid:24)(cid:24)(cid:24)
a (cid:1) (cid:1)
p if depth (cid:25) (cid:12)
e return fn(cid:23)pos(cid:24)(cid:3)(cid:5)(cid:5)(cid:5)(cid:3)f(cid:2)(cid:23)pos(cid:24)
D
h i
else SUCC MoveGen(cid:23)pos(cid:24)
e (cid:0)
c for each succ SUCC
n (cid:7)
e succ V (cid:0) M(cid:0)(cid:0)(cid:1)pass(cid:23)succ(cid:3)depth(cid:1)(cid:6)(cid:3)(cid:23)fn(cid:3)(cid:23)fn(cid:1)(cid:0)(cid:3)(cid:5)(cid:5)(cid:5)(cid:3)f(cid:2)(cid:24)(cid:5)(cid:5)(cid:5)(cid:24)(cid:24)
ci for each i associated with current player
S
if succ V(cid:0)i(cid:2)(cid:6) V(cid:0)i(cid:2)

er V(cid:0)i(cid:2) succ V(cid:0)i(cid:2)
t (cid:0)
u if i(cid:7) n
p
V(cid:0)i(cid:28)(cid:6)(cid:2) succ V(cid:0)i(cid:28)(cid:6)(cid:2)
m
(cid:0)
return V(cid:0)n(cid:2)(cid:3)(cid:5)(cid:5)(cid:5)(cid:3)V(cid:0)depth(cid:2)
o
h i
C

-

n
o
ni Figure (cid:15)(cid:16) M(cid:0)(cid:0)(cid:1)pass(cid:15) A version of the M(cid:0) algorithmthat performs only one pass over the search tree
h
c The maxn algorithm (cid:0)(cid:6)(cid:15)(cid:2) also propagates vectors of values up the game tree(cid:3) However(cid:5) M(cid:0)(cid:0) pass
e (cid:1)
T and maxn are targeted at di(cid:4)erent goals(cid:3) Maxn is an extension of minimax that can handle n
players while M(cid:0)(cid:0) pass is an extension of minimax that can handle n(cid:13)level modelling(cid:3) While all the
(cid:1)
values of a vector in maxn come from the same leaf(cid:5) the values of a vector in M(cid:0)(cid:0) pass come from
(cid:1)
di(cid:4)erent leaves(cid:3) It should be relatively easy to combine the two algorithms(cid:3) Figure (cid:17) shows an
example for a tree spanned by M(cid:0)(cid:3) Note that the values in the vectors correspond to the results
of the recursive calls in (cid:19)gure (cid:11)(cid:3)
(cid:4)
Traditionally such a node is called a MAX node(cid:2) However(cid:1)we assume that both players are maximizers
(cid:7)
a V[2]=10
V[2]= 8 V[2]=10
V[1]=-6 b c V[1]= 0
4
9
9
1 V[2]= 8 V[2]= 7 V[2]= 1 V[2]=10
V[1]=-6 d V[1]=-8 e V[1]=-2 f V[1]= 0 g
- V[0]= 4 V[0]=10 V[0]= 4 V[0]= 6
2
0
4 h i j k l m n o
9
f2= 8 f2=-4 f2= 4 f2= 7 f2=-6 f2= 1 f2=10 f2= 2
S
f1=-6 f1= 6 f1=-8 f1=-7 f1= 7 f1=-2 f1=-4 f1= 0
I f0= 4 f0=-8 f0=10 f0= 3 f0=-4 f0= 4 f0= 4 f0= 6
C

t
r
o Figure (cid:17)(cid:16) The value vectors propagated by M(cid:0)(cid:0) pass(cid:0)
p (cid:1)
e
R
Lemma (cid:7) Assumethat PLAYERisan(cid:8)levelmodellingplayer(cid:3) Let v(cid:3)b (cid:25) M(cid:0)(cid:23)pos(cid:3)depth(cid:3)PLAYER(cid:24)(cid:0)
l
a h i
c and let V(cid:0)n(cid:2) (cid:25) M(cid:0)(cid:0) pass(cid:23)pos(cid:3)depth(cid:3)PLAYER(cid:24)(cid:3) Then v (cid:25) V(cid:0)n(cid:2)(cid:3)
h i (cid:1)
i
n
h
c Proof(cid:1)
e
T For d(cid:25) (cid:12) and d(cid:25) (cid:6) the proof is immediate(cid:3)

- Assume that M(cid:0) and M(cid:0)(cid:0) pass return the same value for a tree of depth d depth (cid:6)(cid:3) We will
t (cid:1) (cid:5) (cid:1)
n prove that they return the same value for d(cid:25) depth(cid:3)
e
m
rt (cid:6)(cid:3) Foreachsuccessoratlevel (cid:6)(cid:5)M(cid:0) (cid:19)rstdeterminestheboardatlevel (cid:11)selected byitsopponent(cid:3)
a
p By the induction hypothesis(cid:5)
e
D
M(cid:0)(cid:23)succ(cid:3)d (cid:6)(cid:3)OPP MODEL(cid:24) (cid:25) M(cid:0)(cid:0) pass(cid:23)succ(cid:3)d (cid:6)(cid:3)OPP MODEL(cid:24)(cid:25) V(cid:0)n (cid:6)(cid:2)
e (cid:1) (cid:1) (cid:1) (cid:1)
c
n and therefore these will be the boards with the maximal V(cid:0)n (cid:6)(cid:2)(cid:3)
e (cid:1)
i
c (cid:11)(cid:3) M(cid:0) determines the value of each successor by calling itself on the board selected by the
S
opponent(cid:3) By the induction hypothesis
r
e
t
u
p M(cid:0)(cid:23)b(cid:3)d (cid:11)(cid:3)PLAYER(cid:24)(cid:25) M(cid:0)(cid:0) pass(cid:23)b(cid:3)d (cid:11)(cid:3)PLAYER(cid:24)(cid:25) V(cid:0)n(cid:2)
m (cid:1) (cid:1) (cid:1)
o
C and therefore these M(cid:0) values will be equal to the V(cid:0)n(cid:2) values of these nodes(cid:3)

-
(cid:15)(cid:3) M(cid:0) associates with each successor the player(cid:18)s value of the board selected by the opponent(cid:3)
n
o M(cid:0)(cid:0) pass assigns to each successor the V(cid:0)n(cid:2) value associated with the board with maximal
i (cid:1)
n V(cid:0)n (cid:6)(cid:2) value(cid:3) Thus(cid:5) the value assigned to each successor by M(cid:0) is equal to its V(cid:0)n(cid:2) value(cid:3)
h (cid:1)
c
e (cid:17)(cid:3) M(cid:0) returns the maximal value for its successors(cid:5) while M(cid:0)(cid:0) pass returns the maximal V(cid:0)n(cid:2) of
T (cid:1)
its successors(cid:3) These are the same values(cid:3)
(cid:0)
(cid:1)
(cid:0)
(cid:4)(cid:1)(cid:2) The relationship between M and minimax
An interesting property of M(cid:0) is that it always selects a move with a value greater or equal to the
4 value of the move selected by minimax that searches the same tree with the same strategy(cid:3)
9
9
1
Lemma (cid:6) Assume that M(cid:0) and Minimax use the same fplayer(cid:3) Then

-

2
0 (cid:23)(cid:7)(cid:24) Minimax(cid:23)pos(cid:3)depth(cid:3)fplayer(cid:24) M(cid:0)(cid:23)pos(cid:3)depth(cid:3)(cid:23)fplayer(cid:3)OPP MODEL(cid:24)(cid:24)
(cid:5)
4
9
for any OPP MODEL(cid:3)
S
I
C
Proof(cid:1)

t
r Wewill provethatthis property exists foreverynode in the tree spanned by the twoalgorithms
o
p by induction on the depth of search(cid:3) For convenience we will prove it for the version of M(cid:0)(cid:0) pass(cid:3)
(cid:1)
e
R For depth (cid:25) (cid:12)(cid:5) for any OPP MODEL

l
a
c (cid:23)(cid:1)(cid:24) Minimax(cid:23)pos(cid:3)depth(cid:24)(cid:25) fplayer(cid:23)pos(cid:24)(cid:25) M(cid:0)(cid:0) pass(cid:23)pos(cid:3)depth(cid:3)(cid:23)fplayer(cid:3)OPP MODEL(cid:24)
i (cid:1)
n
h Let us assume correctness for d depth and prove it for d (cid:25)depth(cid:28)(cid:6)
c (cid:5)
e If pos is a player(cid:18)s node(cid:5)
T

- (cid:23)(cid:10)(cid:24) Minimax(cid:23)pos(cid:3)depth(cid:28)(cid:6)(cid:24)(cid:25) max Minimax(cid:23)succ(cid:3)depth(cid:24) succ SUCC(cid:23)pos(cid:24)

t f j (cid:7) g
n
e
According to the inductive assumption(cid:5)
m
t
r
a max M(cid:0)(cid:0) pass(cid:23)succ(cid:3)depth(cid:3)(cid:23)fplayer(cid:3)OPP MODEL(cid:24)(cid:24) succ SUCC(cid:23)pos(cid:24)
p (cid:5) (cid:1) j (cid:7)
e (cid:25) M(cid:0)(cid:0) pnass(cid:23)pos(cid:3)depth(cid:28)(cid:6)(cid:3)(cid:23)fplayer(cid:3)OPP MODEL(cid:24)(cid:24) o
D (cid:1)

e
c If pos is an opponent(cid:18)s node(cid:5)
n
let s be the successor with the maximal value according to the opponent(cid:18)s model(cid:3)
e
i
c
S (cid:23)(cid:8)(cid:24) Minimax(cid:23)pos(cid:3)depth(cid:28)(cid:6)(cid:24)(cid:25) min Minimax(cid:23)succ(cid:3)depth(cid:24) succ SUCC(cid:23)pos(cid:24)
f j (cid:7) g
r
e
t Minimax(cid:23)s(cid:3)depth(cid:24)
u
(cid:5)
p
m
According to the inductive assumption(cid:5)
o
C

- M(cid:0)(cid:0) pass(cid:23)s(cid:3)depth(cid:3)(cid:23)fplayer(cid:3)OPP MODEL(cid:24)(cid:24)
n (cid:5)(cid:25) M(cid:0)(cid:0)(cid:1)pass(cid:23)pos(cid:3)depth(cid:28)(cid:6)(cid:3)(cid:23)fplayer(cid:3)OPP MODEL(cid:24)(cid:24) (cid:0)
o (cid:1)
i
n
h The intuition behind the above lemma is that minimax assumes an adversary model of the
c
e opponent(cid:3) It assumes that the opponent knows the player(cid:18)s strategy(cid:5) and its only interest is to
T
select moves that are worst according to the player(cid:18)s strategy(cid:3) The lemma says(cid:5) that if you have a
good reason to believe that your opponent(cid:18)s model is di(cid:4)erent than that(cid:5) you could only bene(cid:19)t by
using M(cid:0) instead of minimax(cid:3) The reader should note that the above lemma does not mean that
M(cid:0) selects a move that is better according to some objective criterion(cid:5) but rather a subjectively
better move(cid:23)from the player(cid:18)s point of view(cid:5) according to its strategy(cid:24)(cid:3) If the player does not have
a reliable model of its opponent(cid:5) then playing minimax is a good cautious strategy(cid:3)
(cid:10)
(cid:4)(cid:1)(cid:8) Incorporating depth of search into the model
The M(cid:0) algorithm as well as minimax (cid:23)which is a special case of M(cid:0)(cid:24)(cid:5) incorporates an implicit
4 assumption about the depth to which the opponent searches(cid:3) M(cid:0) that searches a tree of depth
9
9 d(cid:5) assumes that its opponent searches to level d (cid:6)(cid:5) and then assumes that its opponent assumes
(cid:1)
1 that the player searches to level d (cid:11)(cid:5) etc(cid:3) This is a potentially wrong assumption(cid:5) but it is a
- (cid:1)
good defensive mechanism(cid:3) The player assumes that its opponent searches as deep as the player
2
0 can simulate the opponent(cid:18)s search(cid:3) It is meaningless for the player to assume that its opponent
4
searchestoadeeperlevelsince itcannotsimulateasearchtosuchdepth(cid:3) However(cid:5)aswasdiscussed
9
S in the introduction(cid:5) if the player has reason to believe that its opponent searches to a lesser depth(cid:5)
I
C then it can utilize this belief against the opponent (cid:23)to set traps(cid:5) for example(cid:24)(cid:3)

t But whydoes the player havetosearch tolevel d(cid:0) if it knows thatits opponent searches to level
r
o d(cid:2) d(cid:0)(cid:29) In order to predict the opponent(cid:18)s selection(cid:5) it is indeed enough to simulate its search
p (cid:11)
e to level d(cid:2)(cid:3) However(cid:5) in order to evaluate the merit of the opponent(cid:18)s selection for the player(cid:5) it
R
searches as deep as it can(cid:3)

l
a We have extended M(cid:0) to handle models of depth of search(cid:3) We de(cid:19)ne a strategy to be a pair
c
ni (cid:23)fplayer(cid:3)dplayer(cid:24)(cid:3) A player is then a pair (cid:23)(cid:23)fplayer(cid:3)dplayer(cid:24)(cid:3)MODEL(cid:24) where MODEL is also a player(cid:3)
h Figure (cid:14) lists the extended M(cid:0) algorithm(cid:3)
c
e
T

- Procedure M(cid:0)(cid:23)pos(cid:3)depth(cid:3)(cid:23)(cid:23)fpl(cid:3)dpl(cid:24)(cid:3)OPP MODEL(cid:24)(cid:24)
t
n if depth (cid:25) (cid:12)
e
m return NIL(cid:3)fpl(cid:23)pos(cid:24)
h i
t else
r
a SUCC MoveGen(cid:23)pos(cid:24)
p (cid:0)
e for each succ SUCC
D (cid:7)
if depth (cid:25) (cid:6)

e player value fpl(cid:23)succ(cid:24)
c
(cid:0)
n else
e
i opp board(cid:3)opp value M(cid:0)(cid:23)succ(cid:3)min(cid:23)depth (cid:6)(cid:3)dopp(cid:24)(cid:3)OPP MODEL(cid:24)
c h i(cid:0) (cid:1)
S player board(cid:3)player value M(cid:0)(cid:23)opp board(cid:3)depth (cid:11)(cid:3)(cid:23)(cid:23)fpl(cid:3)dpl(cid:24)(cid:3)OPP MODEL(cid:24)(cid:24)
h i(cid:0) (cid:1)
r if player value(cid:6) max value
e
t max value player value
u
(cid:0)
p max board succ
m (cid:0)
return max board(cid:3)max value
o h i
C

-

n
o Figure (cid:14)(cid:16) An extended version of the M(cid:0) algorithmthat can handle a modelof depth
i
n
h
The extended algorithm is di(cid:4)erent from its simpler version on one point(cid:3) Whereas the simple
c
e algorithm allocates as many resources as possible to the call that simulates the opponent(cid:18)s search
T
(cid:23)which is the current depth allocation minus one(cid:24)(cid:5) the extended algorithm allocates the simulation
as maby resources as it believes that the opponent would have used(cid:3) Of course(cid:5) regardless of
the model of its opponent(cid:18)s depth of search(cid:5) the player should never exceed the total depth limit
allocated for the procedure (cid:23)hence the min in the recursive call(cid:24)(cid:3) Figure (cid:7) shows an example of
applying M(cid:0) with a player that searches to depth (cid:15) and an opponent model that searches to depth
(cid:6)(cid:3)
(cid:8)