CS 863 / CSE 860

Artificial Intelligence
Lecture # 11: Evaluation Functions &
Expectiminimax

Instructor: Dr. Zuhair Zafar

 Heuristic Alpha-Beta Tree Search

• Computation time of minimax searching is 𝑏 𝑚

• Alpha-beta pruning (best case) = 𝑏 𝑚/2 which means

• For chess, branching factor = 35 ≈ 6

• Even with alpha-beta pruning and clever move ordering,

minimax won’t work for games such as chess, go etc.

• Too many states to explore in the available time.

 Heuristic Alpha-Beta Tree Search

• Solution: cut off search early and apply heuristic evaluation

function to states.

• Treating nonterminal states as if they were terminal.

• In other words: Utility function is replaced with EVAL.

• EVAL function estimates the states utility.

• Terminal test will be replaced by a cut off test.

 Evaluation Function

• EVAL (s, p) returns an estimate of the expected utility of state

s to player p.
• For terminal nodes, Eval(s, p) = Utility (s, p)
• For nonterminal nodes:
𝑈𝑡𝑖𝑙𝑖𝑡𝑦 𝑙𝑜𝑠𝑠, 𝑝 ≤ 𝐸𝑣𝑎𝑙 𝑠, 𝑝 ≤ 𝑈𝑡𝑖𝑙𝑖𝑡𝑦(𝑤𝑖𝑛, 𝑝)
 Evaluation Function
• Good evaluation function
– Computations must not take too long
– Evaluation function should be strongly correlated with actual chances of
winning
• If the search is cut off at nonterminal states, then the algorithm
will necessary be uncertain about the final outcomes of those
states.
• For example: In chess, there are states which contain all two-
pawn versus one-pawn endgames.
• Some states can lead to wins, some to draws and some to
losses.
 Evaluation Function
• Suppose our experience suggests that 82% of the states
encountered in the two-pawns versus one-pawn category lead
to a win (Utility +1); 2% to a loss (0) and 16% to a draw(1/2).
• Evaluation function for states with such a situation (2-pawns
versus 1-pawn) is the expected value:
1
0.82 × +1 + 0.02 × 0 + 0.16 × = 0.90
2

• Expected value can be estimated for each type of situations,

resulting in an evaluation function for each state.
• Practically, there exists too many situations in a game and
requires too much experience to estimate all the probabilities.
 Evaluation Function
• Most evaluation functions computer compute separate
numerical contributions from each feature and then combine
them to get a total.
• For example: In chess, worth of Pawn is 1, a knight or a bishop
is 3, a rook 5 and queen 9.
• Other features can be “good pawn structure” and “king safety”
with half a worth.
𝑛

𝐸𝑉𝐴𝐿 𝑠 = 𝑤1 𝑓1 𝑠 + 𝑤2 𝑓2 𝑠 + ⋯ + 𝑤𝑛 𝑓𝑛 𝑠 = ෍ 𝑤𝑖 𝑓𝑖 (𝑠)
𝑖=1
• Fi is a feature of the position (such as “number of white
bishops”) and wi is a weight.
Evaluation Function
 Cut off Search
• When it is appropriate to cut off the search.
• Simplest approach: set a fixed depth limit, d.
• Depth d is chosen such that a move is selected within the
allocated time.
• A more robust approach is iterative deepening.
• When the time runs out the algorithm return the move
selected by the deepest completed search.
• These approaches can lead to errors due to the approximate
nature of the evaluation function. (See the chess example)
 Cut off Search
• The evaluation function should be applied only to positions
that are quiescent, i.e., positions in which there is no pending
move (such a capturing the queen).
• For nonquiescent positions, cut off should return false and the
search continues until quiescent position is reached.
• What could be the quiescence positions?
– Capture moves that could swing the evaluation wildly.
 Horizon Effect
• It arises when the program is facing an opponent’s move that
causes serious damage and is unavoidable. It can be
unavoidable temporarily by the use of delaying tactics.
 Horizon Effect
• Strategy to mitigate the horizon effect is to allow singular
extensions, i.e., moves that are clearly better than all the other
moves.
 Forward Pruning
• Alpha-beta pruning prunes branches of the tree that can have
no effect on the final evaluation
• Forward pruning prunes moves that appear to be poor moves
but might possibly be good ones.
• This strategy saves computation time at the risk of making an
error.
• One approach to forward pruning is beam search.
– On each ply, consider only a beam of n best moves (according to the
evaluation function) rather than considering all moves.
– Dangerous approach, because there is no guarantee that the best move
will not be pruned away.
 Forward Pruning: ProbCut
• Probabilistic Cut algorithm.
• It uses statistics gained from prior experience to lessen the
chance of pruning the best move.
• Like Alpha-beta pruning, ProbCut also prunes away all those
moves which are outside of alpha-beta range.
• Computes probability using past experience to estimate how
likely it is that a score of state s at depth d in the tree would be
outside of alpha-beta range.
 Search vs. Lookup
• Overkill for a chess program to start a game by considering a
tree of a billion game states only to conclude that it will play
pawn to e4. (a standard move)
• First few moves, there are few possibilities.
• After 10-15 moves, we end up in rarely seen position, and the
program must now switch from table lookup to search.
• Near the ending of the game, there are again fewer possible
positions and it may be easier to do lookup.
• However, search analysis by computer goes far beyond human
abilities. Endings such as King, bishop and knight versus king
are difficult to master.
 Stochastic Games
• Environmental uncertainty
• Backgammon is a typical stochastic game that combines luck
and skill.
• Rolling of dice is luck or uncertainty in the game environment.
 Stochastic Games
• Black knows what moves it can make
• It does not know what white legal moves will be.
• Black can not construct a standard game tree like we saw in
chess or tic-tac-toe.
 Game Trees with chance nodes
• A game tree in backgammon includes chance nodes.
 Game Trees with chance nodes
• 36 ways to roll two dice, each equally likely.
• 21 distinct rolls (5-6 or 6-5 are considered same)
• The six doubles (1-1,2-2,…,6-6) each have probability 1/36
• The other 15 distinct rolls each have a 1/18 probability.
 Expectiminimax Value
• Terminal nodes and max and min nodes work exactly the same
way as before. However legal moves for max and min will
depend on the outcome of the dice roll.
• For chance nodes we compute the expected value.
 Expectiminimax Value
• Terminal nodes and max and min nodes work exactly the same
way as before. However legal moves for max and min will
depend on the outcome of the dice roll.
• For chance nodes we compute the expected value.
 Expectiminimax Value
• Time complexity 𝑂 𝑏 𝑚 𝑛𝑚 where n is the number of distinct
dice rolls.
• For chance nodes we compute the expected value.