GamePlaying

29/2/2012 1
Outline
Whataregames?
Optimaldecisionsingames
Whichstrategyleadstosuccess?


pruning
Gamesofimperfectinformation
Gamesthatincludeanelementofchance
29/2/2012 2
Whatareandwhystudygames?
Gamesareaformofmultiagentenvironment
Whatdootheragentsdoandhowdotheyaffectoursuccess?
Cooperativevs.competitivemultiagentenvironments.
Competitivemultiagentenvironmentsgiverisetoadversarial

problemsa.k.a.games
GameplayingisagoodproblemforAIresearch
Gameplayingisnontrivial
Playersneed"humanlike"intelligence
Gamescanbeverycomplex(e.g.chess,go)
Requiresdecisionmakingwithinlimitedtime
Gamesusuallyare:
Welldefinedandrepeatable
Limitedandaccessible
29/2/2012 3
TypesofGames
Deterministic Chance
Perfect

Information
Chess,Checkers,Go,

Othello
Backgammom,

Monopoly
Inperfect

Information
Brigde,Poker,

Scrabble,Nuclear

War
29/2/2012 4
RelationofGamestoSearch
Search noadversary
Solutionis(heuristic)

methodforfindinggoal
HeuristicsandCSP

(ConstraintSatisfaction

Problems)techniquescan

findoptimal

solution
Evaluationfunction:

estimateofcostfromstart

togoalthroughgivennode
Examples:pathplanning,

schedulingactivities
Games adversary
Solutionisstrategy

(strategyspecifiesmovefor

everypossibleopponent

reply).
Timelimitsforcean

approximate

solution
Evaluationfunction:

evaluate"goodness"of

gameposition
Examples:chess,checkers,

Othello,backgammon
29/2/2012 5
Gamesetup
Twoplayers:MAXandMIN
MAXmovesfirstandtheytaketurnsuntilthegame

isover.Winnergetsaward,loosergetspenalty.
Gamesassearch:
Initialstate:e.g.boardconfigurationofchess
Successorfunction:listof(move,state)pairsspecifying

legalmoves.
Terminaltest:Isthegamefinished?
Utilityfunction(hmtinch):Givesnumericalvalueof

terminalstates.E.g.win(+1),loose(1)anddraw(0)intic

tactoe(next)
MAXusessearchtreetodeterminenextmove.
29/2/2012 6
PartialGameTreeforTicTacToe
29/2/2012 7
ComplexityofGamePlaying
Howcomplexwouldsearchbeinthiscase?
Worstcase:O(b
d
)
TicTacToe:~5legalmoves,maxof9moves
5
9

=1,953,125states
Chess:~35legalmoves,~100movespergame
35
100

~10154states(butonly

~1040legalstates)
Commongamesproduceenormoussearchtrees!!
29/2/2012 8
GreedySearchforGames
Expandthesearchtreetotheterminalstates
Evaluateutilityofeachterminalboardstate
Maketheinitialmovethatresultsintheboard

configurationwiththemaximumvalue
But this still ignores what the opponent is likely to do
Computer chooses C because its utility is 9
Opponent chooses J and wins!
29/2/2012 9
Minimaxprinciple

Optimalstrategies
Choosesthebestmoveconsideringbothitsmove

andtheopponentsbestmove
Assumption:Bothplayersplayoptimally!!
MAXmaximizingtheutilityundertheassumptionafterit

movesMINwillchoosetheminimizingmove.
Givenagametree,theoptimalstrategycanbe

determinedbyusingtheminimaxvalueofeach

node:
MINIMAXVALUE(n)=
UTILITY(n)

Ifnisaterminal
max
s

successors(n)

MINIMAXVALUE(s)

Ifnisamaxnode
min
s

successors(n)

MINIMAXVALUE(s)

Ifnisaminnode
29/2/2012 10
TwoPlyGameTree
29/2/2012 11
TwoPlayerGameTree
29/2/2012 12
TwoPlayerGameTree
29/2/2012 13
TwoPlayerGameTree
T

he m

i

nim

a

x decision
Minimax maximizes the worst-case outcome for max.
29/2/2012 14
WhatifMINdoesnotplayoptimally?
DefinitionofoptimalplayforMAXassumesMIN

playsoptimally:maximizesworstcaseoutcomefor

MAX.
ButifMINdoesnotplayoptimally,MAXwilldo

evenbetter.[Canbeproved.]
29/2/2012 15
function MINIMAX-DECISION(state)

returns an action
inputs: state, current state in game
v

MAX-VALUE(state)
return the action in SUCCESSORS(state) with value v
function MIN-VALUE(state)

returns a utility value
if TERMINAL-TEST(state) then return

UTILITY(state)
v

for a, s in SUCCESSORS(state) do
v

MIN(v, MAX-VALUE(s))
return v
function MAX-VALUE(state)

returns a utility value
if TERMINAL-TEST(state) then return

UTILITY(state)
v

for a, s in SUCCESSORS(state) do
v

MAX(v, MIN-VALUE(s))
return v
MinimaxAlgorithm
29/2/2012 16
PropertiesofMinimax
Criterion Minimax
Complete? Yes (against an optimal opponent)
Time complexity given branching factor b, O(b
m
)
Space complexity O(b
m
) (depth-first exploration)
Optimal? Yes (if tree is finite)
Timecomplexityisamajorproblemsinceplayer

typicallyonlyhasafiniteamountoftimetomakea

move!!
Forchess,b

35,m100for"reasonable"games

exactsolutioncompletelyinfeasible
29/2/2012 17
Multiplayergames
Gamesallowmorethantwoplayers
Singleminimaxvaluesbecomevectors
29/2/2012 18
Problemofminimaxsearch
Numberofgamesstatesisexponentialtothe

numberofmoves.
Someofthebranchesofthegametreewontbe

takenifplayingagainstanintelligentopponent
Solution:canprune

thosebranchesfromthetree

==>Alphabetapruning
WhiledoingDFSofgametree,keeptrackof:
Alpha

=Highestvaluefoundsofaratanychoicepoint

alongtheMAXpath
Lowerbound(cndi)onnodesutility
Beta

=Lowestvaluefoundsofaratanychoicepointalong

theMINpath
Higherbound(cntrn)onnodesutility
29/2/2012 19
AlphaBetapruning
Betacutoff

pruningoccurswhenmaximizing

(MAXsturn):
Ifalphaparentsbeta,stopexpanding
Whystopexpandingchildren?
OpponentshouldntallowtheMAXtomakethismove
Alphacutoff

pruningoccurswhenminimizing

(MINsturn):
Ifbetaparentsalpha,stopexpanding
Whystopexpandingchildren?
MAXshouldnttakethisroute
29/2/2012 20
AlphaBetapruning
29/2/2012 21
AA
A
=-
O
W
-3
B
N
4
F
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
L
2
AlphaBetaSearchExample
minimax(A,0,4)
max
Call
Stack
A
alpha initialized to -infinity
Expand A? Yes since there are successors, no cutoff test for root
BB
B
=+
O
W
-3
N
4
F
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
L
2
A
=-
AlphaBetaSearchExample
max
Call
Stack
A
B
min
Expand B? Yes since As alpha >= Bs beta is false, no alpha cutoff
minimax(B,1,4) beta initialized to +infinity
FF
F
=-
O
W
-3
B
=+
N
4
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
L
2
A
=-
AlphaBetaSearchExample
max
Call
Stack
A
B
min
max
F
Expand F? Yes since Fs alpha >= Bs beta is false, no beta cutoff
minimax(F,2,4) alpha initialized to -infinity
O
W
-3
B
=+
N
4
F
=-
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
L
2
A
=-
AlphaBetaSearchExample
max
Call
Stack
A
N
4
B
min
max
F
green: terminal state
N
minimax(N,3,4) evaluate and return SBE value
F
=-
F
=4
O
W
-3
B
=+
N
4
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
L
2
A
=-
AlphaBetaSearchExample
max
Call
Stack
A
B
min
max
F
Keep expanding F? Yes since Fs alpha >= Bs beta is false, no beta cutoff
alpha = 4, since 4 >= -infinity (maximizing) back to
minimax(F,2,4)
OO
O
=+
W
-3
B
=+
N
4
F
=4
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
L
2
A
=-
AlphaBetaSearchExample
max
Call
Stack
A
B
min
max
F
O
min
Expand O? Yes since Fs alpha >= Os beta is false, no alpha cutoff
minimax(O,3,4) beta initialized to +infinity
blue: non-terminal state (depth limit)
O
=+
W
-3
B
=+
N
4
F
=4
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
L
2
A
=-
AlphaBetaSearchExample
max
Call
Stack
A
B
min
max
F
O
W
-3
min
W
minimax(W,4,4) evaluate and return SBE value
O
=+
O
=-3
W
-3
B
=+
N
4
F
=4
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
L
2
A
=-
AlphaBetaSearchExample
max
Call
Stack
A
B
min
max
F
O
min
Keep expanding O?
beta = -3, since -3 <= +infinity (minimizing) back to
minimax(O,3,4)
No since Fs alpha >= Os beta is true: alpha cutoff
red: pruned state
O
=-3
W
-3
B
=+
N
4
F
=4
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
L
2
A
=-
AlphaBetaSearchExample
Why?
SmartopponentwillchooseWorworse,thusO'supperboundis3.

ComputeralreadyhasbettermoveatN.
max
Call
Stack
A
B
min
max
F
O
min
X
-5
O
=-3
W
-3
B
=+
N
4
F
=4
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
L
2
A
=-
AlphaBetaSearchExample
max
Call
Stack
A
B
min
max
F
min
X
-5
back to
minimax(F,2,4)
alpha doesnt change, since -3 < 4 (maximizing)
Keep expanding F? No since no more successors for F
B
=+
B
=4
back to
minimax(B,1,4)
O
=-3
W
-3
N
4
F
=4
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
L
2
A
=-
AlphaBetaSearchExample
max
Call
Stack
A
B
min
max
min
X
-5
beta = 4, since 4 <= +infinity (minimizing)
Keep expanding B? Yes since As alpha >= Bs beta is false, no alpha cutoff
O
=-3
W
-3
B
=4
N
4
F
=4
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
L
2
A
=-
AlphaBetaSearchExample
max
Call
Stack
A
B
min
max
min
X
-5
G
G
-5
minimax(G,2,4)
green: terminal state
evaluate and return SBE value
B
=4
B
=-5
O
=-3
W
-3
N
4
F
=4
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
L
2
A
=-
AlphaBetaSearchExample
max
Call
Stack
A
B
min
max
min
X
-5
back to
minimax(B,1,4)
beta = -5, since -5 <= 4 (minimizing)
Keep expanding B? No since no more successors for B
A
=
A
=-5
O
=-3
W
-3
B
=-5
N
4
F
=4
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
L
2
AlphaBetaSearchExample
max
Call
Stack
A
min
max
min
X
-5
back to
minimax(A,0,4)
alpha = -5, since -5 >= -infinity (maximizing)
Keep expanding A? Yes since there are more successors, no cutoff test
CC
C
=+
O
=-3
W
-3
B
=-5
N
4
F
=4
G
-5
X
-5
E
D
0
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
L
2
A
=
AlphaBetaSearchExample
max
Call
Stack
A
min
max
min
X
-5
A
=-5
C
minimax(C,1,4) beta initialized to +infinity
Expand C? Yes since As alpha >= Cs beta is false, no alpha cutoff
green: terminal state
O
=-3
W
-3
B
=-5
N
4
F
=4
G
-5
X
-5
E
D
0
C
=+
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
L
2
A
=
AlphaBetaSearchExample
minimax(H,2,4)
max
Call
Stack
A
min
max
min
X
-5
A
=-5
C
H
3
H
evaluate and return SBE value
C
=+
C
=3
O
=-3
W
-3
B
=-5
N
4
F
=4
G
-5
X
-5
E
D
0
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
L
2
A
=
AlphaBetaSearchExample
max
Call
Stack
A
min
max
min
X
-5
A
=-5
C
back to
minimax(C,1,4)
beta = 3, since 3 <= +infinity (minimizing)
Keep expanding C? Yes since As alpha >= Cs beta is false, no alpha cutoff
green: terminal state
O
=-3
W
-3
B
=-5
N
4
F
=4
G
-5
X
-5
E
D
0
C
=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
L
2
A
=
AlphaBetaSearchExample
minimax(I,2,4)
max
Call
Stack
A
min
max
min
X
-5
A
=-5
C
I
8
I
evaluate and return SBE value
O
=-3
W
-3
B
=-5
N
4
F
=4
G
-5
X
-5
E
D
0
C
=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
L
2
A
=
AlphaBetaSearchExample
max
Call
Stack
A
min
max
min
X
-5
A
=-5
C
back to
minimax(C,1,4)
beta doesnt change, since 8 > 3 (minimizing)
Keep expanding C? Yes since As alpha >= Cs beta is false, no alpha cutoff
JJ
J
=-
O
=-3
W
-3
B
=-5
N
4
F
=4
G
-5
X
-5
E
D
0
C
=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
L
2
A
=
AlphaBetaSearchExample
minimax(J,2,4)
max
Call
Stack
A
min
max
min
X
-5
A
=-5
C
J
alpha initialized to -infinity
Expand J? Yes since J s alpha >= Cs beta is false, no beta cutoff
green: terminal state
O
=-3
W
-3
B
=-5
N
4
F
=4
G
-5
X
-5
E
D
0
C
=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
=-
L
2
A
=
AlphaBetaSearchExample
minimax(P,3,4)
max
Call
Stack
A
min
max
min
X
-5
A
=-5
C
J
P
P
9
evaluate and return SBE value
J
=-
J
=9
O
=-3
W
-3
B
=-5
N
4
F
=4
G
-5
X
-5
E
D
0
C
=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
L
2
A
=
AlphaBetaSearchExample
max
Call
Stack
A
min
max
min
X
-5
A
=-5
C
J
back to
minimax(J,2,4)
alpha = 9, since 9 >= -infinity (maximizing)
Keep expanding J?
Q
-6
R
0
red: pruned states
No since J s alpha >= Cs beta is true: beta cutoff
O
=-3
W
-3
B
=-5
N
4
F
=4
G
-5
X
-5
E
D
0
C
=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
=9
L
2
A
=
AlphaBetaSearchExample
max
Call
Stack
A
min
max
min
X
-5
A
=-5
C
J
Why?
Computer will choose P or better, thus J's lower bound is 9.
Smart opponent wont let computer take move to J
(since opponent already has better move at H).
red: pruned states
O
=-3
W
-3
B
=-5
N
4
F
=4
G
-5
X
-5
E
D
0
C
=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
=9
L
2
A
=
AlphaBetaSearchExample
max
Call
Stack
A
min
max
min
X
-5
A
=-5
C
back to
minimax(C,1,4)
beta doesnt change, since 9 > 3 (minimizing)
Keep expanding C? No since no more successors for C
A
=-5
A
=3
O
=-3
W
-3
B
=-5
N
4
F
=4
G
-5
X
-5
E
D
0
C
=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
=9
L
2
AlphaBetaSearchExample
max
Call
Stack
A
min
max
min
X
-5
back to
minimax(A,0,4)
alpha = 3, since 3 >= -5 (maximizing)
Keep expanding A? Yes since there are more successors, no cutoff test
green: terminal state
O
=-3
W
-3
B
=-5
N
4
F
=4
G
-5
X
-5
E
D
0
C
=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
=9
L
2
A
=
AlphaBetaSearchExample
minimax(D,1,4)
max
Call
Stack
A
min
max
min
X
-5
A
=3
D
D
0
evaluate and return SBE value
O
=-3
W
-3
B
=-5
N
4
F
=4
G
-5
X
-5
E
D
0
C
=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
=9
L
2
A
=
AlphaBetaSearchExample
max
Call
Stack
A
min
max
min
X
-5
A
=3
back to
minimax(A,0,4)
alpha doesnt change, since 0 < 3 (maximizing)
Keep expanding A? Yes since there are more successors, no cutoff test
O
=-3
W
-3
B
=-5
N
4
F
=4
G
-5
X
-5
E
D
0
C
=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
=9
L
2
A
=
AlphaBetaSearchExample
Howdoesthealgorithmfinishsearchingthetree?
max
Call
Stack
A
min
max
min
X
-5
A
=3
O
=-3
W
-3
B
=-5
N
4
F
=4
G
-5
X
-5
E
=2
D
0
C
=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K
=5
M
H
3
I
8
J
=9
L
2
A
=
AlphaBetaSearchExample
StopExpandingEsinceA'salpha>=E'sbetaistrue:

alphacutoff
max
Call
Stack
A
min
max
min
X
-5
A
=3
Why?
Smart opponent will choose L or worse, thus E's upper bound is 2.
Computer already has better move at C.
green: terminal states, red: pruned states
blue: non-terminal state (depth limit)
O
=-3
W
-3
B
=-5
N
4
F
=4
G
-5
X
-5
E
=2
D
0
C
=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K
=5
M
H
3
I
8
J
=9
L
2
A
=
AlphaBetaSearchExample
Result:ComputerchoosesmovetoC.
max
Call
Stack
A
min
max
min
X
-5
A
=3
AlphaBetaAlgorithm
function ALPHA-BETA-SEARCH(state)

returns an action
inputs: state, current state in game
v

MAX-VALUE(state, - , +)
return the action in SUCCESSORS(state) with value v
function MAX-VALUE(state, , )

returns a utility value
if TERMINAL-TEST(state) then return

UTILITY(state)
v

-

for each

s in

SUCCESSORS(state) do
v

MAX(v, MIN-VALUE(s, , ))
if

v

then return

v


MAX(, v)
return v
29/2/2012 52
AlphaBetaAlgorithm
function MIN-VALUE(state, , )

returns a utility value
if TERMINAL-TEST(state) then return

UTILITY(state)
v

+
for each s in

SUCCESSORS(state) do
v

MIN(v, MAX-VALUE(s, , ))
if

v

then return

v


MIN( ,v)
return v
29/2/2012 53
AlphaBetaExample
[-, +]
[-,+]
Range of possible values
Do DF-search until first leaf
29/2/2012 54
AlphaBetaExample(continued)
[-, 3]
[-,+]
29/2/2012 55
AlphaBetaExample(continued)
[-, 3]
[-,+]
29/2/2012 56
AlphaBetaExample(continued)
[3, +]
[3, 3]
29/2/2012 57
AlphaBetaExample(continued)
[-, 2]
[3, +]
[3, 3]
This node is worse
for MAX
29/2/2012 58
AlphaBetaExample(continued)
[-, 2]
[3, 14]
[3, 3] [-, 14]
,
29/2/2012 59
AlphaBetaExample(continued)
[, 2]
[3, 5]
[3, 3] [-, 5]
,
29/2/2012 60
AlphaBetaExample(continued)
[2, 2]
[, 2]
[3, 3]
[3, 3]
29/2/2012 61
AlphaBetaExample(continued)
[2, 2]
[-, 2]
[3, 3]
[3, 3]
29/2/2012 62
Consideranoden

somewhereinthetree
Ifplayerhasabetter

choiceat
Parentnodeofn
Oranychoicepoint

furtherup
n

willnever

bereached

inactualplay.
Hencewhenenoughis

knownaboutn,itcanbe

pruned.
Generalalphabetapruning
29/2/2012 63
FinalComments:AlphaBetaPruning
Pruningdoesnotaffectfinalresults
Entiresubtreescanbepruned.
Goodmoveordering

improveseffectivenessof

pruning
Withperfectordering,

timecomplexityisO(b
m/2
)
Branchingfactorofsqrt(b)!!
Alphabetapruningcanlooktwiceasfarasminimaxinthe

sameamountoftime
Repeatedstatesareagainpossible.
Storetheminmemory=transpositiontable
29/2/2012 64
Gamesofimperfectinformation
Minimaxandalphabetapruningrequiretoomuch

leafnodeevaluations.
Maybeimpracticalwithinareasonableamountof

time.
SHANNON(1950):
Cutoffsearchearlier(replaceTERMINALTESTbyCUTOFF

TEST)
ApplyheuristicevaluationfunctionEVAL(replacingutility

functionofalphabeta)
29/2/2012 65
Cuttingoffsearch
Change:
if

TERMINALTEST(state)thenreturnUTILITY(state)
into
if

CUTOFFTEST(state,depth)thenreturnEVAL(state)
Introducesafixeddepthlimitdepth
Isselectedsothattheamountoftimewillnotexceed

whattherulesofthegameallow.
Whencuttoffoccurs,theevaluationisperformed.
29/2/2012 66
HeuristicEVAL
EVALfunctionretunsanestimateoftheexpected

utilityofthegamefromagivenposition.
Performanceofgameplayingdependsonqualityof

EVAL.
Requirements:
EVALmustagreewithterminalnodesinthesamewayas

UTILITY.
Computationmaynottaketoolong.
FornonterminalstatestheEVALshouldbestrongly

correlatedwiththeactualchanceofwinning.
Onlyusefulforquiescent(nowildswingsinvaluein

nearfuture)states
29/2/2012 67
HeuristicEVALexample
Eval(s) = w
1
f
1

(s) + w
2
f
2

(s) +

+ w
n

f
n

(s)
29/2/2012 68
HeuristicEVALexample
Eval(s) = w
1
f
1

(s) + w
2
f
2

(s) +

+ w
n

f
n

(s)
Addition assumes
independence
29/2/2012 69
Heuristicdifficulties
Heuristic counts pieces won
29/2/2012 70
Horizoneffect
Fixed depth search
thinks it can avoid
the queening move
29/2/2012 71
Gamesthatincludechance
Possiblemoves(510,511),(511,1924),(510,10

16)and(511,1116)
29/2/2012 72
Gamesthatincludechance
Possiblemoves(510,511),(511,1924),(510,1016)

and(511,1116)
[1,1],[6,6]chance1/36,allotherchance1/18
29/2/2012 73
chance nodes
Gamesthatincludechance
[1,1],[6,6]chance1/36,allotherchance1/18
Cannotcalculatedefiniteminimaxvalue,onlyexpected

value
29/2/2012 74
Expectedminimaxvalue
EXPECTEDMINIMAXVALUE(n)=
UTILITY(n)

Ifnisaterminal
max
s

successors(n)

MINIMAXVALUE(s)

Ifnisamaxnode
min
s

successors(n)

MINIMAXVALUE(s)

Ifnisamaxnode

s

successors(n)

P(s).EXPECTEDMINIMAX(s)

Ifnisachancenode
Theseequationscanbebackeduprecursivelyallthe

waytotherootofthegametree.
29/2/2012 75
Positionevaluationwithchancenodes
Left,A1wins
RightA2wins
Outcomeofevaluationfunctionmaynotchangewhen

valuesarescaleddifferently.
Behaviorispreservedonlybyapositivelinear

transformationofEVAL.
29/2/2012 76
Discussion
Examinesectiononstateoftheartgamesyourself
Minimaxassumesrighttreeisbetterthanleft,yet
Returnprobabilitydistributionoverpossiblevalues
Yetexpensivecalculation
29/2/2012 77
Discussion
Utilityofnodeexpansion
Onlyexpandthosenodeswhichleadtosignificanltybetter

moves
Bothsuggestionsrequiremetareasoning
29/2/2012 78
Summary
Gamesarefun(anddangerous)
TheyillustrateseveralimportantpointsaboutAI
Perfectionisunattainable>approximation
Goodideawhattothinkabout
Uncertaintyconstrainstheassignmentofvaluestostates
GamesaretoAIasgrandprixracingisto

automobiledesign.
29/2/2012 79