2

Unit

PROBLEM SOLVING METHODS

Problem solving Methods – Search Strategies- Uninformed – Informed – Heuristics – Local

Search Algorithms and Optimization Problems – Searching with Partial Observations –
Constraint Satisfaction Problems – Constraint Propagation – Backtracking Search – Game
Playing – Optimal Decisions in Games – Alpha – Beta Pruning – Stochastic Games

2.1 PROBLEM DEFINITION

1) Define a state space that contains all the possible configurations of the relevant objects.
2) Specify one or more states within that space that describe possible situations from which
the
3) problem solving process may start ( initial state).
4) Specify one or more states that would be acceptable as solutions to the problem. ( goal
states)
5) Specify a set of rules that describe the actions (operations) available.

Well-Defined Problems And Solutions

A problem can be defined formally by four components:

2.2 Artificial Intelligence
1) Initial state
2) Successor function
3) Goal test
4) Path cost
(1) Initial State
The starting state which agent knows itself.

(2) Successor Function

A description of the possible actions available to the agent.

State x, successor – FN (x) returns a set of < action, successor> ordered pairs, where each
action is a legal action in a state x and each successor is a state that can be reached from x by
applying that action.

State Space
The set of all possible states reachable from the initial state by any sequence of actions. The
initial state and successor function defines the state space. The state space forms a graph in
which nodes are state and axis between the nodes are action.

Path
A path in the state space is a sequence of state connected by a sequence of actions.

(3) Goal Test

Test to determine whether the given state is the goal state. If there is an explicit set of
possible goal states, then we can whether any one of the goal state is reached or not.

Example: In chess, the goal is to reach a state called ―checkmate‖ where the opponent‘s
king is under attack and can‘t escape.

(4). Path cost

A function that assigns a numeric cost to each path. The cost of a path can be described as
the sum of the costs of the individual actions along that path.

Step cost of taking an action ‗a‘to go from one state ‗x‘to state ‗y‘is denoted by C(x,a,y)
C-Cost , x,y- states , Action , Step costs are non-negative

These 4 elements are gathered into a data structure that is given as input to problem solving
algorithm. A solution quality is measured by path cost function. An optimal solution has lowest path
cost among all solutions.

Total cost = Path cost + Search cost

Example: Route finding problem

 Problem Solving Methods 2.3

• Holiday planning on holiday in Romania;

• Currently in Arad. Flight leaves tomorrow from Bucharest.
• Formulate Goal: Be in Bucharest
• Formulate Problem:
◦ States: various cities
◦ Actions: drive between cities
• Find solution: Sequence of cities: Arad, Sibiu, Fagaras, Bucharest

Initial State: In (Arad)

Successor Function: {< Go (Sibia), In (Sibia)>
< Go (Faragas), In (Faragas)>
< Go (Bucharest), In (Bucharest)>}
Goal Test: In (Chennai)
Path Cost: {(In (Arad),}
{Go (Sibia),} = 140 [kilometers]
{In (Erode)}
Path cost = 140 + 99 + 211 = 450
TOY PROBLEM
Example-1 : Vacuum World
Problem Formulation
• • States
◦ 2 x 22 = 8 states
Formula n2n states
2.4 Artificial Intelligence
• Initial State
◦ Any one of 8 states
• Successor Function
◦ Legal states that result from three actions (Left, Right, Absorb)
• Goal Test
◦ All squares are clean
• Path Cost
◦ Number of steps (each step costs a value of 1)

Figure: Vaccuum World

Figure: State Space for the Vacuum World

Labels on Arcs denote L: Left, R: Right, S: Suck

2.2 TYPES OF SEARCH STRATEGIES

 Problem Solving Methods 2.5
Search – the process of finding a path to desired goal state from all possible future states.
The major work in the field of search is to find the right Search Strategy for a particular problem.

There are 2 kinds of search, based on whether they use information about the goal.

Uninformed or blind search strategies uses only the information available in the problem
definition

Informed or heuristic search strategies use additional information. Heuristic tells us

approximately how far the state is from the goal state. Heuristics might underestimate or
overestimate the merit of a state.

Criteria for Evaluation of Search Strategies:

• Completeness: Is the strategy sure to find a solution when there is one?
• Time Complexity: How much time is required to find a solution?
• Space Complexity: How much memory is required to perform the search?
• Optimality: Will it produce high end solution when several different solutions are
available?

The time and space complexity are measured in terms of:

• b: maximum branching factor of the search tree (actions per state).
• d: depth of the solution
• m: maximum depth of the state space (may be ∞) (also noted sometimes D).

2.3 TYPES OF SEARCH

2.6 Artificial Intelligence
2.3.1 Uninformed Search/Blind Search
This type of search does not use any domain knowledge.

These search algorithms do not have additional info about states beyond problem definition.
No information is used to determine preference of one child over other, This means that it does not
use any information that helps it to reach the goal, like closeness or location of the goal. This form
of search, ignores where they are going until they find a goal and report success.

The basic uninformed search strategies are:

• BFS(Beadth First Search): It expands the shallowest node(node having lowest depth)
first.
• DFS(Depth First Search): It expands deepest node first.
• DLS(Depth Limited Search): It is DFS with a limit on depth.
• IDS(Iterative Deepening Search): It is DFS with increasing limit
• UCS(Uniform Cost Search): It expands the node with least cost(Cost for expanding the
node).
2.3.2 Breadth First Search (BFS)
• It expands nodes from the root of the tree and then generates one level of the tree at a
time until a solution is found.
• It is very easily implemented by maintaining a queue of nodes.
• Initially the queue contains just the root.
• In each iteration, node at the head of the queue is removed and then expanded.
• The generated child nodes are then added to the tail of the queue.

Algorithm: Breadth-First Search

1) Create a variable called NODE-LIST and set it to the initial state.
2) Loop until the goal state is found or NODE-LIST is empty.
• Remove the first element, say E, from the NODE-LIST. If NODE-LIST was empty then
quit.
• For each way that each rule can match the state described in E do:
(i) Apply the rule to generate a new state.
(ii) If the new state is the goal state, quit and return this state.
(iii) Otherwise add this state to the end of NODE-LIST
Since each node can be generated in constant time, the amount of time used by Breadth first
search is proportional to the number of nodes generated, which is a function of the branching factor
b and the solution d. Since the number of nodes at level d is bd, the total number of nodes generated
in the worst case is b + b2 + b3 +… + bd i.e. O(bd) , the asymptotic time complexity of breadth first
search.
 Problem Solving Methods 2.7

Look at the above tree with nodes starting from root node, R at the first level, A and B at
the second level and C, D, E and F at the third level. If we want to search for node E then BFS will
search level by level. First it will check if E exists at the root. Then it will check nodes at the second
level. Finally it will find E a the third level.

Advantages of Breadth-First Search

• Breadth first search will never get trapped exploring the useless path forever.
• If there is a solution, BFS will definitely find it out.
• If there is more than one solution then BFS can find the minimal one that requires less
number of steps.
Disadvantages of Breadth-First Search
• The main drawback of Breadth first search is its memory requirement. Since each level
of the tree must be saved in order to generate the next level, and the amount
of memory is proportional to the number of nodes stored.
• The space complexity of BFS is O(bd). As a result, BFS is severely space-bound in
practice so will exhaust the memory available on typical computers in a matter of
minutes.
• If the solution is farther away from the root, breath first search will consume lot of time.
Performance Evaluation:
• Time complexity – O(bd+1)
• Space complexity - O(bd+1)

2.3.3 DFS (Depth First Search)

• This type of algorithm always chooses to go deeper into the graph.
• After DFS visited all the reachable vertices from a particular sources vertices it chooses
one of the remaining undiscovered vertices and continues the search.
• DFS reminds the space limitation of breath first search by always generating next a child
of the deepest unexpanded nodded.
• The data structure stack or last in first out (LIFO) is used for DFS.
Algorithm:
2.8 Artificial Intelligence
Step 1: PUSH the starting node into the stack.
Step 2: If the stack is empty then stop and return failure.
Step 3: If the top node of the stack is the goal node, then stop and return success.
Step 4: Else POP the top node from the stack and process it. Find all its neighbors that are
in ready state and PUSH them into the stack in any order.
Step 5: Go to step 3.
Step 6: Exit.

Advantages:
 Problem Solving Methods 2.9
• DFS consumes very less memory space.
• It will reach at the goal node in a less time period than BFS if it traverses in a right path.
• It may find a solution without examining much of search because we may get the desired
solution in the very first go.
Disadvantages:
• It is possible that many states keep reoccurring. There is no guarantee of finding the
goal node.
• Sometimes the states may also enter into infinite loops.
• In the worst case that goal will be in the shallowest level in the search tree resulting in
generating all tree nodes which are O(bm).
• With branching factor b and maximum depth m, DFS requires storage of only bm + 1
nodes which are O(bm) compared to the O(bd+1) of the BFS.
◦ Time Complexity - O(bd)
◦ Space Complexity - O(bd+1)

2.3.4 Uniform-Cost Search (UCS)

• Uniform-cost is guided by path cost rather than path length like in BFS, the algorithms
starts by expanding the root, then expanding the node with the lowest cost from the root,
the search continues in this manner for all nodes.
• Dijkstra’s algorithm, which is perhaps better-known for shortest path, can be regarded
as a variant of uniform-cost search, where there is no goal state and processing continues
until the shortest path to all nodes has been determined.

Uniform Cost Search (Graph, start, goal)

node ← start
cost ← 0
frontier ← priority queue containing node only
explored ← empty set
do
if frontier is empty
return failure
node ← frontier.pop()
if node is goal
return solution
explored.add(node)
for each of node's neighbors n
if n is not in explored
frontier.add(n)
Time Complexity:
2.10 Artificial Intelligence
UCS is guided by path cost rather than path length so it is hard to determine its complexity
in terms of b and d, so if we consider C to be the cost of the optimal solution, and every action costs
at least e, then the algorithm worst case is O(bC/e).

Space Complexity:
The space complexity is O(bC/e) as the time complexity of UCS.

2.3.5 Depth Limited Search

The unbounded tree problem appeared in DFS can be fixed by imposing a limit on the depth
that DFS can reach, this limit we will call depth limit l, this solves the infinite path problem.

The basic idea is not allowing expansion after certain depth. This problem is most useful if
one is aware of the maximum depth of the solution.

Algorithm:
1) Set the depth limit to the maxdepth of search
2) Initial node = current node
If initial node = goal, return
3) If depth(initial node) > depth limit
return
else
expand (initial node)
save (successors) using stack
go to step2

The search method is not optimal, there is no guarantee that the search will give a solution
that is optimal, since if finds the solution that is within the limits.

One can view DFS as a special case of the depth DLS, that DFS is DLS with l = infinity.
DLS is not optimal even if l > d. (d – depth)

Time Complexity: O(bl) & Space Complexity: O(bl)

Iterative deepening depth-first search (IDS)

IDS is a search strategy resulting when BFS and DFS are combined to make use of the
advantages of each strategy, taking the completeness and optimality of BFS and the modest memory
requirements of DFS.

IDS works by looking for the best search depth d, thus starting with depth limit 0 and make
a BFS and if the search failed it increase the depth limit by 1 and try a BFS again with depth 1 and
so on – first d = 0, then 1 then 2 and so on – until a depth d is reached where a goal is found.

Algorithm: IDS
 Problem Solving Methods 2.11
1) Set depth-limit < -- 0
2) Do
a) Solution =DLS(depth-limit, initial node)
b) If (solution = goal state) then return
else
depth-limit=depth-limit + 1
Continue

Major issue with this algorithm is regeneration of tree for every depth searched. In cases
where depth factor is unknown IDS is preferred. It is complete and optimal, its time complexity is
O(bd) and space complexity is O(bd).

2.3.6 Bidirectional search

Bidirectional search suggests to run 2 simultaneous searches, one from the initial state and
the other from the goal state, those 2 searches stop when they meet each other at some point in the
middle of the graph. The below figure shows the semantic view of a bidirectional search. Best
optimal solution is achieved when BFS is used. The time and space complexity of bidirectional
search is O(bd/2).

2.4 INFORMED SEARCH / HEURISTIC SEARCH

• This type of search uses domain knowledge.
• Some information related to problem space (heuristic) is used to compute preference
among the children for exploration and expansion.
• It generally uses a heuristic function that estimates how close a state is to the goal. This
heuristic need not be perfect.
• This function is used to estimate the cost from a state to the closest goal.
Heuristic Function: It maps each state to a numerical value which depicts goodness of a
node.
H(n) = value where, H() is a heuristic function and ‘n’ is the current state.

The basic informed search strategies are:

2.12 Artificial Intelligence
• Generate and test
• Greedy search (best first search) : It expands the node that appears to be closest to goal
• A* search : Minimize the total estimated solution cost, that includes cost of reaching a
state and cost of reaching goal from that state.

2.4.1 Generate and Test

The most easiest and obvious way to solve any problem is to generate solution and then
check whether the solution is real or not.

Algorithm:
1) Generate a possible solution, which can be a node in the problem space or it can be a
path from start node.
2) Test if the possible solution is real, ie, compare with the goal state
3) Check solution, if true, return the solution, else go to step 1.
Evaluation of which path to be considered is done by heuristic function.

Generator

Possible Solution

Correct Solution
Tester Stop

Incorrect Solution

Best First Search

Best-first search, which is a way of combining the advantages of both Depth and Breadth
First Search
• DFS: follows a single path, don’t need to generate all competing paths.
• BFS: doesn’t get caught in loops or dead-end-paths.
• Best First Search: explore the most promising path seen so far. Nodes are ordered and
expanded using evaluation function. The best evaluation function is expanded first.
Algorithm:
Step 1: Place the starting node or root node into the queue.
Step 2: If the queue is empty, then stop and return failure.
Step 3: If the first element of the queue is our goal node, then stop and return success.
Step 4: Else, remove the first element from the queue. Expand it and compute the estimated
goal distance for each child. Place the children in the queue in ascending order to the
goal distance.
 Problem Solving Methods 2.13
Step 5: Go to step-3
Step 6: Exit.

Example:

Step 1:
Consider the node A as our root node. So the first element of the queue is A whish is not our
goal node, so remove it from the queue and find its neighbour that are to inserted in ascending
order.
2.14 Artificial Intelligence
 Problem Solving Methods 2.15

Advantage:
• It is more efficient than that of BFS and DFS.
• Time complexity of Best first search is much less than Breadth first search.
• The Best first search allows us to switch between paths by gaining the benefits of both
breadth first and depth first search. Because, depth first is good because a solution can
be found without computing all nodes and Breadth first search is good because it does
not get trapped in dead ends.

Disadvantages:
• Sometimes, it covers more distance than our consideration.

Branch and Bound Search

Branch and Bound is an algorithmic technique which finds the optimal solution by keeping
the best solution found so far. If partial solution can’t improve on the best it is abandoned, by this
method the number of nodes which are explored can also be reduced.

Branch and Bound is an algorithmic technique which finds the optimal solution by keeping
the best solution found so far. If partial solution can’t improve on the best it is abandoned, by this
method the number of nodes which are explored can also be reduced. It also deals with the
optimization problems over a search that can be presented as the leaves of the search tree. The usual
technique for eliminating the sub trees from the search tree is called pruning. For Branch and Bound
algorithm we will use stack data structure.

Concept:
Step 1: Traverse the root node.
Step 2: Traverse any neighbour of the root node that is maintaining least distance from the
root node.
Step 3: Traverse any neighbour of the neighbour of the root node that is maintaining
least distance from the root node.
Step 4: This process will continue until we are getting the goal node.
2.16 Artificial Intelligence
Algorithm:
Step 1: PUSH the root node into the stack.
Step 2: If stack is empty, then stop and return failure.
Step 3: If the top node of the stack is a goal node, then stop and return success.
Step 4: Else POP the node from the stack. Process it and find all its successors. Find out
the path containing all its successors as well as predecessors and then PUSH
the successors which are belonging to the minimum or shortest path.
Step 5: Go to step 5.
Step 6: Exit.

Let us take the following example for implementing the Branch and Bound algorithm.
 Problem Solving Methods 2.17
Step 2:

Step 3:

Step 4:

Hence the searching path will be A-B -D-F

2.18 Artificial Intelligence
Advantages:
• As it finds the minimum path instead of finding the minimum successor so there should
not be any repetition.
• The time complexity is less compared to other algorithms.

Disadvantages:
• The load balancing aspects for Branch and Bound algorithm make it parallelization
difficult.
• The Branch and Bound algorithm is limited to small size network. In the problem of
large networks, where the solution search space grows exponentially with the scale of
the network, the approach becomes relatively prohibitive.
OR Graph
Here a graph is used that is named as an OR - graph, since each of its branches represents
alternative problem solving path. The Best First Search, selects the most promising of the nodes
that has been generated so far. This can be achieved by applying appropriate Heuristic function to
each of them.

Heuristic function:
f(n) = h(n) where, h(n) - estimated straight line distance from node n to goal
To implement the graph search procedure , two list of nodes are used.
OPEN- nodes that have been generated but have not been visited yet
Closed - nodes that have been already visited

Algorithm:
1) The 1st step is to define the OPEN list with a single node, the starting node.
2) The 2nd step is to check whether or not OPEN is empty. If it is empty, then the algorithm
returns failure and exits.
3) The 3rd step is to remove the node with the best score, n, from OPEN and place it in
CLOSED.
4) The 4th step “expands” the node n, where expansion is the identification of successor
nodes of n.
5) The 5th step then checks each of the successor nodes to see whether or not one of them
is the goal node. If any successor is the goal node, the algorithm returns success and the
solution, which consists of a path traced backwards from the goal to the start node.
Otherwise, proceeds to the sixth step.
6) In 6th step, for every successor node, the algorithm applies the evaluation function, f, to
it, then checks to see if the node has been in either OPEN or CLOSED. If the node has
not been in either, it gets added to OPEN.
 Problem Solving Methods 2.19
7) Finally, the 7th step establishes a looping structure by sending the algorithm back to the
2nd step. This loop will only be broken if the algorithm returns success in step 5 or
failure in step 2.
Consider the following graph as an example,

S: Initial state, G: goal.

Table shows the heuristic estimates:

node h(n) node h(n) node h(n)

A 11 E 4 I,J 3
B 5 F 2 S 15
C,D 9 H 7 G 0
Initialization
S (15) Expand the nodes of S and add S into CLOSED
Iteration 1:
1) Add the successors of node s into OPEN and compute f(n)
2) Choose the most promising node (min h(n))
3) Remove the node from OPEN and add it into the CLOSED
Iteration 2:
1) Add the successors of node B into OPEN and compute f(n)
2) Choose the most promising node (min h(n))
3) Remove the node from OPEN and add it into the CLOSED
Iteration 3:
1) Add the successors of node F into OPEN and compute f(n)
2.20 Artificial Intelligence
2) Choose the most promising node (min h(n))
3) Remove the node from OPEN and add it into the CLOSED
The shortest path form S -> G is S -> B -> F -> G
Total cost: 1+3+1=4

A* Search
• A* is an informed search algorithm. The best first search is a simplified version of A*
algorithm. This is the widely used approach for pathfinding.
• Evaluation Function f(n) represents the total cost.
• f(n) = g(n) + h(n), where g(n) = cost incurred so far to reach n, h(n) = estimated cost
from n to goal state.
Algorithm
1) Begin with open list containing only the initial state.
◦ Let g(n) =0 and h(n) = value whatever is
◦ So, f(n) = h(n) as g(n)=0
2) Do
If no nodes exists in open list, return failure to goal achieved
Else
a) Select and remove node from open list with low f value
b) Let this be current node ’c’
c) Put c on the closed list
d) Check if c= goal state, return solution.
e) If not generate successor for c and for every successor,
(i) Save path from successor to C
(ii) Calculate g(successor)g(c) + cost of reaching to successor from c
3) End

This algorithm will be successful when the following three cases are verified.
• Check if the successor is on open list
• Check if the successor is on only closed list.
• Check if the successor is neither on the open list nor on the closed list.

Admissible A*
The heuristic function h (n) is called admissible if h(n) is never larger than h*(n), namely
h(n) is always less or equal to true cheapest cost from n to the goal.
 Problem Solving Methods 2.21
A* is admissible if it uses an admissible heuristic, and h (goal) = 0.

If the heuristic function, h always underestimates the true cost (h (n) is smaller than h*(n)),
then A* is guaranteed to find an optimal solution.

Example: 8 puzzle game.

Advantages:
• It is complete and optimal.
• It is the best one from other techniques. It is used to solve very complex problems.
• It is optimally efficient, i.e. there is no other optimal algorithm guaranteed to expand
fewer nodes than A*.
Disadvantages:
• This algorithm is complete if the branching factor is finite and every action has fixed
cost.
• The speed execution of A* search is highly dependant on the accuracy of the heuristic
algorithm that is used to compute h (n).
• It has complexity problems.

2.5 HEURISTIC FUNCTIONS

A heuristic function is a function that maps from problem state descriptions to measures of
desirability, usually represented as numbers.

Aspects of problem states are evaluated, and the weights given to individual aspects are
chosen in a way that the value of the heuristic function at a given node in the search process gives
as good an estimate as possible of whether that node is on the desired path to a solution.

Heuristic functions can play an important part in efficiently guiding a search process.
Heuristic functions provide a good estimate of a path. The below fig shows the heuristic functions
for a few problems.

Chess The material advantage of our side over the opponent

Travelling Salesman The sum of the distances so far

Tic-Tac-Toe
I for each row in which we could win and in which we already
have one piece plus 2 for each such row in which we have two
pieces
Sometimes a high value of the heuristic function indicates a good position, low value
indicates an advantageous situation. No matter the way function is stated. The program uses the
values of the function attempt to minimize it or to maximize it as appropriate.
2.22 Artificial Intelligence
• The Heuristic function (HF) guide the search process in the most profitable direction by
suggesting which path to follow first when more than one is available.
• H.F estimates the true merits of each node in the search tree, the more direct the solution
process.
• H.F is so good that definitely no search would be required.
• For many problems, the cost of computing the value of such a function would outweigh
the effort in the search process.
• Compute a perfect Heuristic function by doing a complete search from the node in
question and determining whether it leads to a good solution.
• Trade – off (Balance) between the cost of evaluating a heuristic function and the savings
in search time that the function provides.
• Some heuristics used to define the control structure that guides the application of rules
in the search process.

2.6 LOCAL SEARCH ALGORITHM AND OPTIMIZATION PROBLEMS

2.6.1 Hill Climbing
It is a variant of generate and test in which feedback from the test procedure is used to help
the generator decide which direction to move in the search space.

The test function responds yes or no.

Test function provides an estimate of how close a given state is to a goal state.

The generate procedure that often the computation of the heuristic function is done at
almost no cost at the same time that the test for a solution is being performed.

Example:
Suppose we are in an unfamiliar city without a map and want to get downtown. Our Aim is
for the tall buildings.

Heuristic Function is distance between the current location and the location of the tall
buildings and desirable states are those in which this distance is minimized. For this problem,
hill climbing can terminate whenever a goal state is reached.
• Absolution solution exist whenever it is possible to recognize a goal state just by
examining it.
• Relative solution exist, for maximization problems, such as T S P , there is no a prior
goal state.
2.6.2 Simple Hill climbing
Algorithm: Simple Hill climbing
1) Evaluate the initial state. If it is also a goal state, then return it and quit. Otherwise,
continue with the initial state as the current state.
 Problem Solving Methods 2.23
2) Loop until a solution is found or until there are no new operators left to be applied in
the current state:
a) Select an operator that has not yet been applied to the current state and apply it to
produce a new state.
b) Evaluate the new state.
(i) If it is a goal state, then return it and quit.
(ii) If it is not a goal state but it is better than the current state, then make it the
current state.
(iii) If it is not better than the current state, then continue in the loop.

Algorithm: Generate-and- test:

1) Generate a possible solution. For some problems, this means generating a particular
point in the problem space. For others, it means generating a path from a start state.
2) Test to see if this is actually a solution by comparing the chosen point or the endpoint
of the chosen path to the set of acceptable goal state.
3) If a solution has been found, quit. Otherwise, return to step 1.
Generate – and – test Simple Hill climbing
(i) Evaluation function is used as a way to (i) Execution of this alg,,.. let’s take four
inject task – specific knowledge into the colored blocks puzzle, to solve this, need
control process. to define a heuristic function that
(ii) Knowledge give heuristic search describes how close a particular
methods their power to solve intractable configuration is to being a solution.
problems. (ii) Function is the sum of the number of
(iii) Unclear question is asked, “IS one state different colors on each of the four sides.
better than another?” For the algorithm (iii) Solution for this puzzle, is value of 16.
to execute, definition of better is (iv) A set of rules that describe ways of
provided. transforming one configuration into
(iv) In some cases, higher value of the another.
heuristic function. Ex: pick a block and rotate it 90
(v) In some, lower value of the H.S. degrees in any direction.
(v) Next step is to generate a starting
configuration.
(vi) Hill climbing begin, generate a new
state by selecting a block and rotating it.
(vii) Result state is not good, return to
previous state and try a different
perturbation.

2.6.2.1 Steepest –Ascent Hill climbing

2.24 Artificial Intelligence
Simple hill climbing considers all the moves from the current state and selects the best one
as the next state. This method is called Steepest – ascent hill climbing or gradient search.

Algorithm: Steepest – Ascent Hill climbing

1) Evaluate the initial state. If it is also a goal state, then return it and quit. Otherwise,
continue with the initial state as the current state.
2) Loop until a solution is found or until a complete iteration produces no change to current
state:
a) Let SUCC be a state such that any possible successor of the current state will be
better than SUCC.
b) For each operator that applies to current state do:
(i) Apply the operator and generate a new state.
(ii) Evaluate the new state. If it is a goal state, then return it and quit. If not,
compare it to SUCC. If it is better, then set SUCC to this state. If it is not better,
leaves SUCC alone.
c) If the SUCC is better than current state, then set current state to SUCC.
Applying steepest – ascent hill climbing to the colored blocks problems, consider all
perturbations of the initial state and choose the best. There are many moves in a problem. There is
a balance between the time required to select a move and the number of moves required to get to a
solution that is considered when deciding which method will work better.

Algorithm terminate by not finding a goal state but by getting to a state from which no better
states can be generated. (i.e.) if the program has reached either a local maximum, a plateau or a
ridge.

A local maximum is better than all its neighbors but not better than far away states. All
moves appear to make things worse. Local maxima occur within sight of a solution. They are called
foothills at this point.

A Plateau is a flat area in which a whole set of neighboring states have the same value. It
is not possible to determine the best direction in which to move by making local comparisons,.

A ridge is a kind of local maximum. It is higher than surrounding areas and that itself has a
slope. But the orientation of the high region, Compared to the set of available moves and directions
they move, makes it impossible to traverse a ridge by single moves.

Problem dealing ways:

(i) Backtrack to earlier node and go in a different direction. The node has another direction
which is promising or promising as the one that was chosen earlier. To implement this,
list of paths taken and go back to one of them if the path that was taken leads to a dead
end.
(ii) Make a big jump in some direction to get to a new section. Rules describe single small
steps, apply them several times in the same direction.
 Problem Solving Methods 2.25
(iii) Two or – more rules are applied before doing the test. This corresponds to moving in
several directions at once. It is a good strategy for dealing with ridges.
◦ Hill climbing is not always very effective.
◦ Value of the heuristic function drops off suddenly as moving away from a
solution. This happens whenever there is sort of threshold effect is present.
◦ Hill climbers is a local method, decides what to do next by looking only at the
“immediate” consequences of its choice rather than by exploring all the
consequences.
◦ Hill climbing shares with other local methods like nearest neighbor heuristic.
◦ It shares with other local methods a lack of a guarantee that it will be effective.
◦ Hill - climbing procedure looks only one move ahead and that examination may
exploit an arbitrary amount of global information if that information is encoded
in the heuristic function.
Example:
Consider the blocks world problem shown in below fig.

A H
H G
G F
F E
E D
D C
C B
B A
Initial state Final state

Fig shows (a) A Hill climbing problem

We use the following heuristic function:

Local:
Add one point for every block that is resting on the thing it is supposed to be resting on.
Subtract one point for every block that is sitting on the wrong thing.
• Using this function, the goal state has a score of 8.
• The initial state has a score of 4.
• There is only one move from the initial state, namely to move block A to the table that
produces a state with a score of 6.
• The hill – climbing procedure accept the move.
2.26 Artificial Intelligence
• From the new state, there are 3 possible moves, leading to the three states show in below
fig

A
H
G G G
F F F
E E E
D D D
C H C C
B A B A H B
(a) (b) (c)
Fig – shows the Three possible Moves
• These above states have the scores: ( c ) 4, (d) 4, (e) 4.
• Hill climbing will halt because all these states have lower scores than the current state.
• The process has reached a local maximum that is not the global maximum.
• The problem is that by purely local examination of support structures, the current state
appears to be better than any of its successors because more blocks rest on the correct
objects.
• To solve this problem, necessary to disassemble a good local structure because it is in
the wrong global context.
Hill climbing is blamed for this failure to look far enough ahead to find a solution. But we
could also blame the heuristic function and modify it. The following heuristic function in place of
the first one:

Global:
For each block that has the correct support structure, add one point for every block in the
support structure. For each block that has an incorrect support structure, subtract one point for
every block in the existing support structure.
• Using this function the goal state has the score 28.
• The initial state has the score-2.8 moving A to the table yields a state with a score of 21
since A no longer has seven wrong blocks under it
• The three states that can be produced next have the following scores: (c) – 28 (d) – 16,
and (e) – 15.
• This time, steepest – ascent hill climbing will choose move (e), which is the correct one.
 Problem Solving Methods 2.27
• This new heuristic function captures the two key aspects of this problem: incorrect
structures are bad and should be taken apart – and correct structures are good and should
be built up.
• Hill climbing procedure fails with the earlier heuristic function works perfectly now.
Example:
Problem of driving down own. The perfect heuristic function need to have knowledge about
one way and deal – end streets, which in case of a strange city is not always available. Sometimes
even perfect knowledge available but not usable.

Example:
Imagine a heuristic function that computes a value for a state by invoking its own problem-
solving procedure to look ahead from the state it is given to find a solution. It knows the exact cost
of finding solution and return cost’s as it’s value. A heuristic function that converts the local hill –
climbing procedure in to a global method by embedding a global method. So far hill climbing is
very inefficient in a large problem space.

2.6.2.2 Stimulated Annealing

It is a variation of hill climbing in which, downhill moves made at the beginning of the
process. The final solution is insensitive to the starting state. This lower the changes of getting
caught at a local maximum a plateau, or a ridge

Two changes:
(i) The objects function is used in place of the term heuristic function.

(ii) Minimize rather than maximize the value of the objective function. We describe a
process of valley descending rather than hill climbing.

Stimulated annealing, a process is patterned after the physical process of annealing, physical
substances such as metals are melted and then cooled until some solid state is reached. The goal of
this process is to produce a minimal energy final state. This process is one of valley descending in
which the objects function is the energy level. Physical substances move from high to low, so the
valley descending occurs. There is some probability that a translation to a higher energy state will
occur. This probability is given by the function:

P = e-ΔE/kT

Where, At is the positive change, in the energy level T is the temperature and K is
Boltzmann’s constant.

Valley descending that occurs during annealing the probability of a large uphill move is
lower than the probability of a small one.

Probability that an uphill move decrease as the temperature decreases. These moves possible
during the beginning of the process when the temperature is high, and they become less at the end
as the temperature becomes lower. Downhill moves allowed anytime. Large upward moves occur
2.28 Artificial Intelligence
but as the process progresses, only small upward moves are allowed finally the process converges
to a local minimum configuration.
• The rate at which the system is cooled is called the annealing schedule.
• Physical Annealing process are sensitive to the annealing schedule.
• If cooling occurs too rapidly, stable regions of high energy will form or else a local but
not global minimum is reached.
• At high temperatures, random motion is allowed.
• At lower temperature, a lot of time wasted after the final structure has been formed.

Properties of physical annealing:

• It can be used to define an analogous process of simulated annealing which can be used
whenever simple hill climbing can be used.
• E is generalized so it represents not just the change in energy but more particularly the
change in the value of the objective function.
• Analogy for KT is less straight forward.
• The variable K describes the correspondence between the units of temperature and the
units of energy.
• In the analogous process, the units for both E and T are artificial, it sense to incorporate
K in T, selecting values for T that produces desirable behavior on the part of the
algorithm.
• Revised probability formula
P = e-ΔE/T

The different between stimulated annealing and the simple hill – climbing procedure are
given below:
a) The annealing schedule must be maintained.
b) Moves to worse states may to accepted.
c) It is a good idea to maintain the current state, best state found so far. If the final state is
worse than that earlier state the earlier state is still available.

Algorithm: Simulated Annealing

1) Evaluate the initial state. If it is also a goal state, then return it and quit. Otherwise,
continue with the initial state as the current state.
2) Initialize BEST-SO-FAR to the current sate.
3) Initialize T according to the annealing schedule.
4) Loop until a solution is found or until there are no new operators left to be applied in
the current state.
 Problem Solving Methods 2.29
a) Select an operator that has not yet been applied to the current state and apply it to
produce a new state.
b) Evaluate the new state. Compute
E = (Value of current) – (Value of new state)
◦ If the new state is a goal state, then return it and quit.
◦ If it is not a goal sate but is better than the current state, then make it the current
state. Also set BEST-SO-FAR to this new state.
◦ If it is not better than the current state, then make it the current state with
probability p’ as defined above. This step is usually implemented by invoking a
random number generator to produce a number in the range [0, 1]. If that number
is less than p’, then the move is accepted otherwise, do nothing.
c) Revise T as necessary according to the annealing schedule.
5) Return BEST-SO-FAR, as the answer.

Implementation of Algorithm”
• Select Annealing schedule, which has three components.
a) The first is the initial value user for temperature.
b) The second is the criteria will be used to decide when the temperature of the system
should be reduced.
1) The third is the amount by which the temperature will be reduced each time it
is changed.
2) Fourth component of the schedule, when to quit.
• Simulated Annealing solve the problems where number of moves from a given state is
very large.
• For such problems, it makes no sense to try all possible moves.
• The Best annealing schedule should be observed in a way that the effect on both the
quality of the solution that is found and the rate at which the process converges.
(i) T approaches zero, the probability of accepting a move to a worse state goes to zero
and simulated annealing becomes identical to simple hill climbing.
(ii) Computing the probability of accepting a move is the ratio E/ T. Thus it’s
important that values of T be scaled so this ratio is meaningful.

2.7 CONSTRAINT SATISFACTION

• a CSP is a triple < V , D, C > where :
• V={V1,...,Vn} is a (finite) set of variables
• D ={D1,...,Dn} a set of domains Di for each variable Vi (finite sets of possible
values) Constraint Satisfaction Problem (CSP)
2.30 Artificial Intelligence
• C={C1,...,Cp} is a set of constraints on variables of V
• a constraint Ci(Vi1,...,Vik) on variables {Vi1,...,Vik} is defined as a subset of the cross-
product Di1x ... x Dik
The goal is to discover some problem state that satisfies a given set of constraints.

Example:
cyptarithmatic puzzles.

SEND DONALD CROSS

+MORE +GERALD +ROADS
……… ……… ………
MONEY ROBERT DANGER

Fig – shows Some cyptarithmatic problem

Assigning a decimal digit to each of the letters in a way that the answer to the problem is
correct. If the same letter occurs more than once, it is assigned the same digit each time.

No 2 different letters may be assigned the same digit.

• Design tasks are constraint – satisfaction problems in which design must be created
within fixed limits on time, cost and materials.
• The amount of search is reduced which requires comparison with a method that attempts
to form partial solutions directly by choosing specific values for components of the
eventual solution.
Example:
A straight forward search procedure to solve a cryptarimetic problem operate in a state space
of partial solutions in which letters are assigned particular number as their values.

Constraint satisfaction approach to solving this problem avoids making guesses on

particular assignments of numbers to letters until it has otherwise, the initial set of
constraints, each number correspond to only one letters and that the sums of the digits must be as
they are given in the problem is first augmented to include restrictions that can be inferred (or)
concluded from the rules of arithmetic.
• Constraint satisfaction is a search procedure that operates in a space of constraint sets.
• The initial state contains the constraints that are given in the problem description.
• A goal state is any state that has been constrained “enough”, where “enough” must be
defined for each problem.
• For cryptarithmetic, each letter assigned a unique numeric value.
Constraint – satisfaction – Two – step process:
 Problem Solving Methods 2.31
Constraints are discovered and propagated throughout the system. If no solution, search
begins and added as a new constraint. Propagation occur with new constraint.

(i) First step:

Propagation arises from the fact that there are usually dependencies among the constraints.
These dependencies occur because many constraints involve more than one object and many objects
particular is more than one constraint.

Example:
Assume we start with one constraint, N = E + 1. If we added the constraint N= 3, we
propagate that to get a stronger constraint on E namely.

that E= 2. Constraint propagation arises from the presence of inference rules that allows
additional constraints to be inferred from the ones.

Constraint propagation terminates for two reasons:

(i) A contradiction is detected, it so, then there is no solution consistent with all the known
constraints. If the contradiction involves only those constraints that were given as part
of the problem specification then no solution exists.
(ii) Propagation has run out of steam and no further changes that can be made on the basis
of current knowledge. Solution has not been adequately specified, then search is
necessary to get the process again.
(ii) Second step:
Hypothesis is a way to strengthen the constraints must be made.

Example:
Cryptarithmetic problem, guessing a particular value for some letter.

Constraint propagation can begin from this new state. Solution is reported. If more guesses
are necessary, then it can proceed.
• If a contradiction is detected, then backtracking used to try a different guess and
proceed.
Algorithm: constraint satisfaction:
1) Propagate available constraints. To do this, first set OPEN to the set of all objects that
must have values assigned to them in a complete solution. Then do until an
inconsistency is detected or until OPEN is empty:
a) Select an object OB from OPEN. Strengthen as much as possible the set of
constraints that apply to OB.
b) If this set is different from the set that was assigned the last time OB was examined
or if this is the first time OB has been examined, then add to OPEN all objects that
share any constraints with OB.
c) Remove OB from OPEN.
2.32 Artificial Intelligence
2) If the union of the constraints discovered above defines a solution, then quit and report
the solution.
3) If the union of the constraints discovered above defines a contradiction, then return
failure.
4) If neither of the above occurs, then it is necessary to make a guess at something in order
to proceed. To do this, loop until a solution is found or all possible solutions have been
eliminated:
a) Select an object whose value is not yet determined and select a way of strengthening
the constraints on that object.
b) Recursively invoke constraint satisfaction with the current set of constraints
augmented by the strengthening constraint just selected.
Algorithm application – Two rules:
(i) Rules that define the way constraints validly be propagated.
(ii) Rules that suggest guesses when guesses are necessary. In some problems, guesses are
not required.
Algorithm – Working:
Consider the crypt arithmetic problem shown in the below fig

Problem:
SEND
+MORE
………
MONEY
Initial state:
No two letters have the same value. The sums
of the digits must be as shown in the problem.

Fig –shows A crypt arithmetic problem

The goal state is a problem state in which all letters have been assigned a digit in such a way
that all the initial constraints are satisfied.
• The solution process proceeds in cycles.
• At each cycle, 2 important things are done
(i) Constraints are propagated by using rules that correspond to the properties of
arithmetic.
(ii) A value is guessed for some letter whose value is not determined.

First step:
It does not matter what order the propagation is done since all available propagations will
be performed before the step ends.
 Problem Solving Methods 2.33
Second steps:
Though in which order the guesses are tried have a substantial impact on the degree of
search that is necessary.
a) A heuristics help to select the best guess to try first.
b) Example:
c) If there is a letter that has only two possible values and another with six values, there is
better chance of guessing right on the first than on the second.
d) Heuristic explains if there is a letter that participates in many constraints then it is a
good idea to prefer it to a letter that participates in a few.
e) A guess on a highly constrained letter lead to a contradiction or to the generation of
additional constraints.
f) A guess on a less constrained letter, provides less information.
g) The result of the few cycles of processing this example is shown in below fig
Initial state: SEND
+MORE
M=1 ………
S=8 or 9 MONEY
O=0 or 1  O=0
N=E or E+1  N=E+1
C2=1
N+R>8
E< >9

E=2
N=3
R=8 or 9
2+D=Y or 2+D = 10+Y

C1=0 C1=1

2+D=Y 2+D=10+Y
N+R=10+E D=8+Y
R=9 D=8 or 9
S=8 D=9
D=8

Y=0 Y=1

Conflict Conflict

Fig – showsSolving a cryptarithmetic problem

Constraints never disappear at lower levels only the ones being added are shown for each
level. This approach is efficient in terms of storage space and backtracking. This approach store all
2.34 Artificial Intelligence
the constraints in d/b and to record at each node the changes that must be undone during
backtracking.

C1,C2, C3 and C4 indicate the carry bits out of the columns, numbering from the right.

Rules for propagating constraints generate the following additional constraints:

• M=1, since two single digit numbers plus a carry cannot total more than 19.
• S=8 or 9, since S+M+C3>9 (to generate the carry) and M=1, S+1+C3>9, so S+C3>8
and C3 is at most 1.
• O=0, since S+M(1) + C3(<=1) must be at least 10 to generate a carry and it can be at
most 11. But M is already 1, so O must be 0.
• N = E or E+1, depending on the value of C2. But N cannot have the same value as E.
So N=E+1 and C2 is 1.
• In order for C2 to be 1, the sum of N+R+C1 must be greater than 9, so N+R must be
greater than 8.
• N+R cannot be greater than 18, even with a carry in, so E cannot be 9.
Assume that no more constraints can be generated. To progress at this point, guessing
happens. If E is assigned the value 2. Now the next cycle begins.

Constraint propagator shows that:

• N = 3, since N = E +1
• R = 8 or 9, since R + N(3) + C1(1 or 0) = 2 or 12. But since N is already 3, the sum of
these nonnegative numbers cannot be less than 3. Thus R + 3 + (0 or 1) = 12 and R = 8
or 9
• 2+D = Y or 2+D = 10+Y, from the sum in the rightmost column.
Assuming no more constraint generation then guess is required.

Suppose C1 is chosen to guess a value for 1, then we reach dead end as shown in the fig –
1.11 ( b), when this happens, the process will backtrack and C1 = 0

First thing to notice:

Constraint propagation rules is that they do not conclude fake constraints. They do not have
to conclude all legal ones.

Example:
Reasons through to the result that C1 equals 0. We can process by observing that for C1 to
be 1, it should held:

2+D = 10+Y. For this to be the case, D would have to be 8 or 9. Both S and R must be either
8 or 9 and three letters can’t share two values. So C1 can’t be 1. Some search could be avoided.
Search route takes more or less actual time than does the constraint propagation route depends on
how long it takes to perform the reasoning required for constraint propagation.
 Problem Solving Methods 2.35
Second thing to notice:
There are two kinds of constraints:
(i) First kind is simple, list values for a single objects.
(ii) Second kind is complex, describe relationship between or among objects.
Both kinds of constraints play the same role in the constraint satisfaction process and in the
cryptarithmetic example they were treated identically.

Other two kinds of constraints:

(i) Simple, value – listing constraints are always dynamic and must always be represented
explicitly in each problem state.
(ii) Complicated, relationship – expressing constraints are dynamic in the cryptarithmatic
domain since they are different for each cryptarithmetic problem. But in many other
domains they are static.
Algorithm for constraint satisfaction in which chronological backtracking is used when
guessing leads to an inconsistent set of constraints.
Constraints are generated are left alone if they are independent of the problem and its cause.
This approach is called dependency directed Backtracking (DDB).

2.8 SOLVING CSPS USING BACKTRACKING SEARCH

• CSPs can be solved by a specialized version of depth first search.
• Key intuitions:
◦ a solution is built by searching through the space of partial assignments.
◦ Order in which variables are assigned does not matter---eventually they all have
to be assigned.
◦ If during the process of building up a solution a constraint failsreject all possible
ways of extending the current partial assignment.
2.8.1 Backtracking Search Algorithm
2.36 Artificial Intelligence
Measure of Performance and Analysis f Search Algorithm:
Search strategies that can reason either forward or backward but for a given problem, one
direction or the other must be chosen. A mixture of the two directions is appropriate mixed strategy
solve the major parts of the problem first and then go back and solve the small problems. This
technique is known as Means – ends Analysis.

• Difference between the current state and the goal state.

• Operator reduce the difference must be found.

• Operator can’t be applied to the current state.

• Setting up a sub problem of getting to a state in which it can be applied.

• Backward chaining in which operators are selected and then sub goals are setup to
establish the pre-conditions of the operators is called operator sub goaling.
• If the operator doesn’t produce the exact goal state then the second sub problem of
getting from the state it does produce a goal.

• If the difference chosen correctly and if operator is effective at reducing the difference,
then the two sub problems should be easier to solve than the original problem.

• To solve big problems first, the differences assigned priority levels.

• Higher priority are considered before lower priority.

• General problem solver (GPS) , people use this GPS to solve problem.

• GPS provide fuzziness of the boundary between building programs that simulate what
people do and building programs that solve a problem.

• Mean – ends Analysis relies on a set of rules that transform one problem state into
another.

• Rules are represented as a left side that describes the conditions that must be met for the
rule to be applicable and a right side describes aspects of the problem state that will be
changed by the application of the rule.

• A separate data structure called a difference table indexes the rules by the differences
that they can be used to reduce.

Example: House hold robot domain

The available operators shown in below fig. along with preconditions and results.
 Problem Solving Methods 2.37

Operator Preconditions Results

PUSH(obj, loc) at (robot, obj )^ at (obj, loc)^
large(obj)^ at (robot, loc )
clear(obj)^
armempty
CARRY(obj, loc) at (robot, obj )^ at (obj, loc)^
small (obj) at (robot, loc )
none at (robot, loc )
at (robot, obj ) holding(obj)
WALK(loc)
PICKUP(obj) holding (obj) holding(obj)
PUTDOWN(obj) at (robot, obj 2)^ on(obj1, obj2)
PLACE(obj1, obj2) holding (obj1)

Fig Shows The Robot’s operation

The below fig shows the different table that describes when each of the operator is
appropriate.

More operator reduce a different and a given operator able to reduce more than one
difference.

Working of Robot:
• Robot job is to moving a desk with two things on it from one room to another.
• The objects on top must be moved.
• Difference between the start state and the goal state is location of the desk.
• To reduce the difference, either PUSH or CARRY could be chosen.
• If CARRY chosen, its preconditions met.
• It Result in two more differences that must be reduced, the location of robot and size of
desk.
• Location of robot by applying WALK. But no operator change the size of on object.
• This path leads to dead – end.
• We attempt to apply PUSH.
Below fig shows the problem solver’s progress.

A B C D

Push

Start Goal

Fig –ShowsThe progress of the Means – ends Analysis method

2.38 Artificial Intelligence
• The thing does not get to goal state.
• So, the difference between A & B and between C & D must be reduced.
• PUSH has four preconditions, two of which produce differences between the start and
the goal states:
a) The robot must be at the desk

b) The desk must be clear.

• Since the desk is already large, and the robot’s arm is empty, those two preconditions
can be ignored.
• The robot can be brought to the correct location by using WALK.
• Surface of the desk can be cleared by two uses of PICKUP.
• After one PICKUP, an attempt to do the second results in another difference the arm
must be empty.
• PUTDOWN used to reduce that difference.
• ONE PUSH is performed, the problem state is close to the goal state.
• The objects placed back on the desk.
• PLACE will put them there.
• Difference is eliminated.
• Robot holding the object.
• The progress of the problem solver shown below fig.

A B C E
D

Push Pickup Put down Pick up Put down Place

Pick
Start Goal

Fig –ShowsMore progress of the Means – Ends method

• The final difference between C and E reduced by using WALK to get the robot back to
the objects, followed by PICKUP and CARRY.

Algorithm: Means – Ends Analysis (CURRENT, GOAL )

1) Compare CURRENT to GOAL. If there are no difference between them then return.
2) Otherwise, select the most important difference and reduce it by doing the following
until success or failure is signaled:
a) Select an as yet untried operator O that is applicable to the current difference. If
there are no such operator, then signal failure.
 Problem Solving Methods 2.39
b) Attempt to apply O to CURRENT. Generate description of two states: O-START,
a state in which O’s preconditions are satisfied and O-RESULT, the state that would
result if O were applied in O-START.

c) If

(FIRST-PART  MEA(CURRENT, O-START)

And
(LAST-PART  MEMO-RESULT, GOAL)
Are successful, then signal success and return the result of concatenating
FIRST-PART, O, and LAST-PAR

2.9 GAME PLAYING

Game playing is considered one of the major topic of AI. Game playing is a science which
is used to analyze various scenarios in different domains. Game playing is an important aspect of
the system, where two or more entities compete to achieve a state of maximum gain at the cost of
other.

To put it simple: Game playing shows several sides of intelligence, particularly the ability
to plan and ability to learn.

Game – Definition:
• A game is an activity or sport usually involving skill, knowledge, or chance, in
which you follow fixed rules and try to win against an opponent or to solve a puzzle.
• A game is an activity among two or more self-regulating decision-makers looking
for to achieve their goals in some limiting environment.
• A game is a form of play with goals and structure.
• When you strip away the category differences and the technical difficulties, all
games share four defining characters: a goal, rules, a feedback system, and voluntary
participation.

Why are games relevant to AI?

• Games are fun!
• They are limited, well-defined rules
• They provide advanced, existing test-beds for developing several ideas and techniques
that are useful elsewhere.
• They are one of the few domains that allows us to build agents.
• Studying games teaches us how to deal with other agents trying to foil our plans
• Huge state spaces – Games are highly complex, e.g. Chess.
• Game Playing is considered as intelligent human activity.
2.40 Artificial Intelligence
Procedure Required:
Heuristic approach is used in Game playing.

Basically heuristic procedure is generate and test procedure, where testing is performed after
the generator has done its part.

The procedure is required to generate only good moves.

The procedure uses the test strategies that will identify and explore the best moves in first
place.

Example: playing chess

Legal-move generator, the test procedure will have a look at each of them in a faster way.
So there is no accuracy in a job.

Possible-move generator, only small number of moves are generated. Heuristic is applied
to only some kind of promise. In this generator, the test procedure spend more time evaluating each
of the moves which produce a trustworthy result.

Conclusion - Include heuristic (exploratory) knowledge into both the generator and the
tester, the performance of the overall system will be improved.

Search Procedure –Use:

• Search procedure is used to generate moves through the problem space until a goal state
is reached.
• In game-playing programs, a goal state is the one which is considered as win.
• In chess, it is not possible to search until a goal state is found.
• The depth of the resulting tree and its branching factor are great.
• To choose the best Move, Static Evaluating function uses information it has to evaluate
individual board positions by estimating which leads to a win.
• This function is similar to that of the heuristic function h’.
2.9.1 Game Theory
Game theory does not prescribe a way or say how to play a game.

Game theory is a set of ideas and techniques for analyzing conflict situations between two
or more parties. The outcomes are determined by their decisions.

Frequently used terms:

Game: A sort of conflict in which ‘n’ individuals or groups participate.
Rules: Conditions which are responsible for the game to begin
Strategy: List of best choices for every player at every point of a given game.
Move: Way in which the game advances from stage one to final stage.
 Problem Solving Methods 2.41
Important concepts used in Game theory:
Set of Rules, Number of Players, Possible Strategies for players, State of the game

Equilibrium, Information about moves of opponent, Turn to make move, Probabilistic

Approach, Finiteness of turns etc., the above concepts are considered while describing a game.

2.9.1.1 Game Types

Different approaches have given rise to different classes. They are as follows:
• Symmetric Game: This game strategy depends on other techniques that are in practice.
• Zero-sum Game: In this game the loss or gain of winner is balanced by the gain or loss
of other participants.
• Non-Zero sum Game: This game provides situations in which participants may lose or
gain simultaneously.
• Perfect Information Game: A game where, if all moves taken till any point in the game
by all players are known to all players. Example – chess
• Simultaneous Game: Here the actions/moves taken by a player is independent of the
moves/actions of other players.
• Sequential Game: One player takes the first move thus making other players get some
information of the first player’s choice. Example – Combinatorial games.
• Repeated Game: This is also called stage game. Its two-player game. The effects of the
moves of any players affects the future moves.
• Signaling Game: Two players namely transmitter and receiver. In these type of games
incomplete information exists, often one player is more informed than the other.
• Large Poisson Game: Random number of players play such games, where every player
has his own strategy to win. Example – Voting system
• Non-transitive Game: The games where multiple strategies generate loops fall in this
category.
• Global Game: If a game is with incomplete information then it can be categorized as
Global, which is associated with some probabilities and signaling about the
environment.
2.42 Artificial Intelligence
2.9.1.2 Game Strategies
Strategy is a complete approach for playing a game, while move is an action taken by a
player at some point during the course of the game.
• Dominant Strategies: One strategy is better than another for the same player,
irrespective of other player’s game.
• Pure Strategies: It’s the complete approach of a player’s game plan.
• Mixed Strategy: Randomly the player can choose a strategy based on probability
assigned for any strategy.
• Tit for Tat: Starts with co-operative players, but as soon as a player betrays, opponent
in the next cycle betrays.
• Collusion: Illegal and back door agreement between multiple players to impose
restriction with regard to the use of its rights.
• Backward Induction: Optimal actions are executed by backward reasoning.

2.9.1.3 Game Equilibrium

Equilibrium is the process of selection of a stable state. Game theory tries to find state of
equilibrium. It depends on field of application, which might coincide. Some important equilibrium
are Nash Equilibrium, Sub-Game Perfection, Bayesian Nash, Perfect Bayesian, Proper
Equilibrium, Correlated Equilibrium, Sequential Equilibrium, Parto Efficiency, Self-Confirming
Equilibrium, Trembling hand. Nash Equilibrium is one of the most popular concept.

Nash Equilibrium
In Nash Equilibrium multiple players are involved along with the following rules.
1) The participants in the game are aware of the equilibrium of the other players.
2) None of the participants can gain by selecting or changing the strategy in such a way
that it affects only one player.
3) If a player changes or selects a strategy then none of the other players can get any gain
whether they change or don’t change their strategy.
The above rules along with set of strategy choices and their gains(payoffs) forms the Nash
Equilibrium.

2.9.2 Knowledge structure

Structural Knowledge is base for problem solving that is a requirement for creating plans
and strategies to set conditions, and also to determine what to do when a failure occurs or when
certain information is missing.

Knowledge Structure is capable of capturing different aspects of the game. Following

briefly describes some of the important structures depending upon the form of games.
• Extensive Form Game: This is used to acquire games in an order and represent them
in the form of a game tree. Nodes represent decision the player has taken and branches
 Problem Solving Methods 2.43
represents the move the player has used to reach the node. Leaf nodes represent the
payoffs or gain.
• Strategic Form Game: If the Nash equilibrium discussed above is represented with
some function that is able to acquire the gains with the help of payoff matrix is Strategic
Form Game.
• Co-operative/Coalition Form Game: Coalitions occur in co-operative games, where
multiple players co-operatively fight to achieve target.
2.9.3 Game as a Search Problem
Any intelligent games need an optimally correct choice so it may result in superior gain.
This may be achieved by generating search tree and comparing profit and loss function. Various
components that are used to formalize the game theory are initial state, successor function, terminal
test and payoff function.

2.9.3.1 The Minimax Search Procedure

• The minimax search procedure is a depth-first, depth-limited search procedure.
• The idea is to start at the current position and use the possible-move generator to
generate the set of possible successor positions.
• Applying the fixed evaluation function to those positions and choose the best one.
• Then assign the value to the starting position by the best move that could be taken next.

Procedure
Minimax is a procedure used for minimizing the possible loss while maximizing the
potential gain. Originally formulated for two player game. In simple way minimax is Considered as
two player game, the players are referred as MAX(player) and MIN(the opponent). MAX is trying
to maximize its score and MIN is trying to minimize MAX’s score.

Assumption:
• Let us consider two players MAX and MIN
• The game proceeds with the assuming ahead – opponent’s best move.
• Task is to find “best” move for MAX
• The game starts with MAX making the first move and then MIN.
• Type of payoff’s could be Win, Lose or Draw.

Principle:
• Each move has a value (Evaluation number) associated with it.
• Higher the number greater the opportunity for player, lower the number opportunity is
high for opponent.
• Basic principle – the player assume the opponent will choose the minimizing move next
after player made his move first.
2.44 Artificial Intelligence
• Best moves are made under assumption.
• The fixed evaluation function returns large values to indicate good situation for us.

Static (Heuristic) Evaluation Functions:

• Estimates how good the current board configuration is for a player.
• Typically one has to find out how good it is for the player, and how good it is for the
opponent, Subtract the opponents score from the players
E(n) = M(n) – O(n)

Goal:
To maximize the value of the static evaluation function of the next board position.

Minimax Algorithm:
1) DEEP-ENOUGH(Position, Depth), then return the structure
VALUE=STATIC (Position, Player);
PATH=nil
This indicates that there is no path from this node and that its value is that determined
by the static evaluation function.
2) Otherwise, generate one more play of the tree by calling the function MOVE-GEN
(Position, Player) and setting SUCCESSORS to the list it returns.
3) If SUCCESSORS is empty, then there are no moves to be made, so return the same
structure that would have been returned if DEEP-ENOUGH had returned true.
 Problem Solving Methods 2.45
4) If SUCCESSORS is not empty, then examine each element in turn and keep track of the
best one. This is done as follows;
Initialize BEST-SCORE to the minimum value that STATIC can return. It will be
updated to reflect the best score that can be achieved by an element of SUCCESSORS.
For each element SUCC of SUCCESSORS, do the following:
a) Set RESULT-SUCC to
MINIMAX(SUCC, Depth+1, OPPOSITE(Player))ᶟ
This recursive call to MINIMAX will actually carry out the exploration of SUCC.
b) Set NEW-VALUE to –VALUE (RESULT-SUCC). This will cause it to reflect the
merits of the position from the opposite perspective from that of the next lower level.
c) If NEW-VALUE>BEST-SCORE, then we have found a successor that is better than
any that have been examined so far. Record this by doing the following:
i) Set BEST-SCORE to NEW-VALUE.
ii) The best known path is now from CURRENT to SUCC and then on to the
appropriate path down from SUCC as determined by the recursive call to
MINIMAX. So set BEST-PATH to the result of attaching SUCC to the front of
PATH(RESULT-SUCC).
5) Now that all the successors have been examined, we know the value of Position as well
as which path to take from it. So return the structure
VALUE=BEST-SCORE
PATH=BEST-PATH
When the initial call to MINIMAX returns, the best move from CURRENT is the first
element on PATH.
Example:
Assume a static evaluation function that returns values ranging from -10 to 10, with 10
indicating a win for us, -10 a win for the opponent and 0 an even match. Since our goal is to
maximize the value of the heuristic function, we choose to move to B. Backing B’s value up to A,
we can conclude that A’.
2.46 Artificial Intelligence

Step 1:

Step 2:
 Problem Solving Methods 2.47
Step 3:

Step 4:

Step 5: (The Best Path )

2.48 Artificial Intelligence
2.10 ALPHA – BETA PRUNING
α - β pruning
• It is a decision rule algorithm which is represented as game tree.
• It is used to find best way from number of possibility by pruning the nodes.
• Min-Max with alpha-beta pruning yields the same result as a complete search, but with
much greater efficiency (i.e. reduced its effective branching).
• Here the best value for Max play found so far is stored in ALPHA and the best valuefor
Min play found so for in BETA.
• If, at any given point BETA becomes smaller than ALPHA (α≥β)we can prune the game
tree at this point.
• It is required to evaluate atleast on set of leaf nodes branching of one of the deepest Min
nodes and then use the acquired α value to prune the rest of the search.
Alpha-beta pruning is a way of finding the optimal minimax solution while avoiding
searching subtrees of moves which won't be selected. In the search tree for a two-player game, there
are two kinds of nodes, nodes representing your moves and nodes representing your
opponent's moves. Nodes representing your moves are generally drawn as squares (or possibly
upward pointing triangles):

These are also called MAX nodes. The goal at a MAX node is to maximize the
value of the subtree rooted at that node. To do this, a MAX node chooses the
child with the greatest value, and that becomes the value of the MAX node.

Nodes representing the opponent's moves are generally drawn as circles (or possibly as
downward pointing triangles):
These are also called MIN nodes. The goal at a MIN node is to minimize the
value of the subtree rooted at that node. To do this, a MIN node chooses the
child with the least (smallest) value, and that becomes the value of the MIN
node.

Alpha-beta pruning gets its name from two bounds that are passed along during the
calculation, which restrict the set of possible solutions based on the portion of the search tree that
has already been seen. Specifically, β - Beta is the minimum upper bound of possible solutions, α -
Alpha is the maximum lower bound of possible solutions

Thus, when any new node is being considered as a possible path to the solution, it can only
work if: α≤N≤β where N is the current estimate of the value of the node.
 Problem Solving Methods 2.49

Example:
2.50 Artificial Intelligence
 Problem Solving Methods 2.51

2.11 STOCHASTIC GAMES

A stochastic game is a collection of normal-form games that the agents play repeatedly

The particular game played at any time depends probabilistically on

• the previous game played
• the actions of the agents in that game

Like a probabilistic FSA (Finite state Automata) in which

• the states are the games
• the transition labels are joint action-payoff pairs
Markov Games
A stochastic (or Markov) game includes the following:
• a finite set Q of states (games),
• a set N = {1, …, n} of agents,
2.52 Artificial Intelligence
• For each agent i, a finite set Ai of possible actions
• A transition probability function P : Q × A1 ×· · ·× An × Q → [0, 1]
P(q, a1, …, an , q") = probability of transitioning to state q" if the action profile
(a1, …, an) is used in state q
• For each agent i, a real-valued payoff function
ri : Q × A1 ×· · ·× An → ℜ
This definition makes the inessential but simplifying assumption that each agent’s
strategy space is the same in all games
• So the games differ only in their payoff functions
Eg: Two-player Zero-sum Stochastic Games (Backgammon)