![Alpha-Beta Pruning
• Typically can only look 3-4 ply in allowable chess time
• Alpha-beta pruning simplifies search space without eliminating
optimality
– By applying common sense
– If one route allows queen to be captured and a better move is available
– Then don’t search further down bad path
– If one route would be bad for opponent, ignore that route also
Maintain [alpha, beta] window at each node during depth-first search
alpha = lower bound, change at max levels
beta = upper bound, change at min levels
2 7 1 No need to
look here!
Max](https://image.slidesharecdn.com/l3-231115172747-53526273/85/l3-pptx-15-320.jpg)
l3.pptx
- 2. Game Playing
- 3. Why Study Game Playing? • Games allow us to experiment with easier versions of real-world situations • Hostile agents act against our goals • Games have a finite set of moves • Games are fairly easy to represent • Good idea to decide about what to think • Perfection is unrealistic, must settle for good • One of the earliest areas of AI – Claude Shannon and Alan Turing wrote chess programs in 1950s • The opponent introduces uncertainty • The environment may contain uncertainty (backgammon) • Search space too hard to consider exhaustively – Chess has about 1040 legal positions – Efficient and effective search strategies even more critical • Games are fun to target!
- 4. Assumptions • Static or dynamic? • Fully or partially observable? • Discrete or continuous? • Deterministic or stochastic? • Episodic or sequential? • Single agent or multiple agent?
- 5. Zero-Sum Games • Focus primarily on “adversarial games” • Two-player, zero-sum games As Player 1 gains strength Player 2 loses strength and vice versa The sum of the two strengths is always 0.
- 6. Search Applied to Adversarial Games • Initial state – Current board position (description of current game state) • Operators – Legal moves a player can make • Terminal nodes – Leaf nodes in the tree – Indicate the game is over • Utility function – Payoff function – Value of the outcome of a game – Example: tic tac toe, utility is -1, 0, or 1
- 7. Using Search • Search could be used to find a perfect sequence of moves except the following problems arise: – There exists an adversary who is trying to minimize your chance of winning every other move • You cannot control his/her move – Search trees can be very large, but you have finite time to move • Chess has 1040 nodes in search space • With single-agent search, can afford to wait • Some two-player games have time limits • Solution? – Search to n levels in the tree (n ply) – Evaluate the nodes at the nth level – Head for the best looking node
- 8. Game Trees • Tic tac toe • Two players, MAX and MIN • Moves (and levels) alternate between two players
- 9. Minimax Algorithm • Search the tree to the end • Assign utility values to terminal nodes • Find the best move for MAX (on MAX’s turn), assuming: – MAX will make the move that maximizes MAX’s utility – MIN will make the move that minimizes MAX’s utility • Here, MAX should make the leftmost move • Minimax applet
- 10. Minimax Properties • Complete if tree is finite • Optimal if play against opponent with same strategy (utility function) • Time complexity is O(bm) • Space complexity is O(bm) (depth-first exploration) • If we have 100 seconds to make a move – Can explore 104 nodes/second – Can consider 106 nodes / move • Standard approach is – Apply a cutoff test (depth limit, quiescence) – Evaluate nodes at cutoff (evaluation function estimates desirability of position)
- 11. Static Board Evaluator • We cannot look all the way to the end of the game – Look ahead ply moves – Evaluate nodes there using SBE • Tic Tac Toe example – #unblocked lines with Xs - #unblocked lines with Os • Tradeoff – Stupid, fast SBE: Massive search • These are “Type A” systems – Smart, slow SBE: Very little search • These are “Type B” systems – Humans are Type B systems – Computer chess systems have been more successful using Type A – They get better by searching more ply
- 12. Comparison 0 2 4 6 8 10 12 14 16 18 20 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 ply U.S. Chess Federation Rating x 103 40 … Belle Belle Belle Belle Bobby Fischer Deep Thought Deep Blue Anatoly Karpov Gary Kasparov Hitech
- 13. Example • Chess, SBE is typically linear weighted sum of features – SBE(s) = w1f1(s) + w2f2(s) + … + wnfn(s) – E.g., w1 = 9 • F1(s) = #white queens - #black queens • For chess: – 4 ply is human novice – 8 ply is typical PC or human master – 12 ply is grand master
- 14. Example • Othello • SBE1: #white pieces - #black pieces • SBE2: weighted squares
- 15. Alpha-Beta Pruning • Typically can only look 3-4 ply in allowable chess time • Alpha-beta pruning simplifies search space without eliminating optimality – By applying common sense – If one route allows queen to be captured and a better move is available – Then don’t search further down bad path – If one route would be bad for opponent, ignore that route also Maintain [alpha, beta] window at each node during depth-first search alpha = lower bound, change at max levels beta = upper bound, change at min levels 2 7 1 No need to look here! Max
- 16. Example
- 17. Example
- 18. Example
- 19. Example
- 20. Example
- 21. Example
- 22. Example
- 23. Example
- 24. Example
- 25. Example
- 26. Example
- 27. Example
- 28. Example
- 29. Example
- 30. Example
- 31. Example
- 32. Example
- 33. Example
- 34. Example
- 35. Example
- 36. Example
- 37. Example
- 38. Example
- 39. Example
- 40. Example
- 41. Example
- 42. Example
- 43. Example
- 44. Example
- 45. Example
- 46. Example
- 47. Example
- 48. Example
- 49. Example
- 50. Example
- 51. Example
- 52. Example
- 53. Example
- 54. Example
- 55. Bad and Good Cases for Alpha-Beta Pruning • Bad: Worst moves encountered first • Good: Good moves ordered first • If we can order moves, we can get more benefit from alpha-beta pruning 4 MAX +----------------+----------------+ 2 3 4 MIN +----+----+ +----+----+ +----+----+ 6 4 2 7 5 3 8 6 4 MAX +--+ +--+ +--+ +-+-+ +--+ +--+ +--+ +--+ +--+--+ 6 5 4 3 2 1 1 3 7 4 5 2 3 8 2 1 6 1 2 4 4 MAX +----------------+----------------+ 4 3 2 MIN +----+----+ +----+----+ +----+----+ 4 6 8 3 x x 2 x x MAX +--+ +--+ +--+ +--+ +-+-+ 4 2 6 x 8 x 3 2 1 2 1
- 56. Alpha Beta Properties • Pruning does not affect final result • Good move ordering improves effectiveness of pruning • With perfect ordering, time complexity is O(bm/2)
- 57. Problems with a fixed ply: The Horizon Effect • Inevitable losses are postponed • Unachievable goals appear achievable • Short-term gains mask unavoidable consequences (traps) Lose queen Lose pawn Lose queen!!! The “look ahead horizon”
- 58. Solutions • How to counter the horizon effect – Feedover • Do not cut off search at non-quiescent board positions (dynamic positions) • Example, king in danger • Keep searching down that path until reach quiescent (stable) nodes – Secondary Search • Search further down selected path to ensure this is the best move – Progressive Deepening • Search one ply, then two ply, etc., until run out of time • Similar to IDS
- 59. Variations on 2-Player Games Multiplayer Games • Each player maximizes utility • Each node stores a vector of utilities • Entire vector is backed up the tree • 3-player example: If in leftmost state, should player 3 choose first move because higher utility values? • Result will be terminal state with utility values (v1=1, v2=2, v3=3) • This vector is backed up to the parent node • Need to consider cooperation among players to move 1 (1 2 3) +------------------+ +---------------------+ 2 (1 2 3) (-1 5 2) +--------+ +-----+ +--------+ +-------+ 3 (1 2 3) (6 1 2) (-1 5 2) (5 4 5) / / / / 1 (1 2 3) (4 2 1) (6 1 2) (7 4 -1) (5 -1 -1) (-1 5 2) (7 7 -1) (5 4 5)
- 60. Nondeterministic Games • In backgammon, the dice rolls determine legal moves
- 62. Nondeterministic Game Algorithm • Just like Minimax except also handle chance nodes • Compute ExpectMinimaxValue of successors – If n is terminal node, then ExpectMinimaxValue(n) = Utility(n) – If n is a Max node, then ExpectMinimaxValue(n) = maxsSuccessors(n) ExpectMinimaxValue(s) – If n is a Min node, then ExpectMinimaxValue(n) = minsSuccessors(n) ExpectMinimaxValue(s) – If n is a chance node, then ExpectMinimaxValue(n) = sSuccessors(n) P(s) * ExpectMinimaxValue(s)
- 63. Status of AI Game Players • Tic Tac Toe – Tied for best player in world • Othello – Computer better than any human – Human champions now refuse to play computer • Scrabble – Maven beat world champions Joel Sherman and Matt Graham • Backgammon – 1992, Tesauro combines 3-ply search & neural networks (with 160 hidden units) yielding top-3 player • Bridge – Gib ranked among top players in the world • Poker – Pokie plays at strong intermediate level • Checkers – 1994, Chinook ended 40-year reign of human champion Marion Tinsley • Chess – 1997, Deep Blue beat human champion Gary Kasparov in six-game match – Deep Blue searches 200M positions/second, up to 40 ply – Now looking at other applications (molecular dynamics, drug synthesis) • Go – 2008, MoGo running on 25 nodes (800 cores) beat Myungwan Kim – $2M prize available for first computer program to defeat a top player