Backtracking- module-5

• to solve some large instances of difficult combinatorial problems.

• The principal idea is to construct solutions one component at a time and evaluate such
partially constructed candidates as follows.

• If a partially constructed solution can be developed further without violating the problem's
constraints, it is done by taking the first remaining legitimate option for the next
component.

• If there is no legitimate option for the next component, no alternatives for any remaining
component need to be considered.

• In this case, the algorithm backtracks to replace the last component of the partially
constructed solution with its next option.
 State Space Tree
• It is convenient to implement this kind of processing by constructing a tree of
choices being made, called the state-space tree.
• Its root represents an initial state before the search for a solution begins. The
nodes of the first level in the tree represent the choices made for the first
component of a solution,
• the nodes of the second level represent the choices for the second component,
and so on
• A node in a state-space tree is said to be promising if it corresponds to a
partially constructed solution that may still lead to a complete solution;
• otherwise, it is called nonpromising. Leaves represent either nonpromising
dead ends or complete solutions found by the algorithm.
• In the majority of cases, a state space tree for a backtracking algorithm is
constructed in the manner of depth first search.
 N-Queen’s Problem
• The problem is to place n queens on an n-by-n chessboard so that no two
queens attack each other by being in the same row or in the same column or
on the same diagonal.

• consider the four-queens problem and solve it by the backtracking technique.

Since each of the four queens has to be placed in its own row, all we need to
do is to assign a column for each queen on the board.
 Sum of Subset Problem
• consider the subset-sum problem: find a subset of a given set S= {s sn) of n
1, ... ,

positive integers whose sum is equal to a given positive integer d.

• For example, for S = {1, 2, 5, 6, 8] and d = 9, there are two solutions:
{1,2, 6] and {1, 8].

• The state-space tree can be constructed as a binary tree, The root of the tree
represents the starting point, with no decisions about the given elements
made as yet.
• Its left and right children represent, respectively, inclusion and exclusion of s1
in a set being sought.
• going to the left from a node of the first level corresponds to inclusion
of s while going to the right corresponds to its exclusion, and so on.
2,
 Branch and Bound
• In Backtracking, to cut off a branch of the problem's state-space tree as soon
as we can deduce that it cannot lead to a solution.
• But in Branch and Bound need to find an optimal solution out of many
feasible solution.
• Compared to backtracking, branch-and-bound requires two additional items:

• A way to provide, for every node of a state-space tree, a bound on the best
value of the objective function1

• the value of the best solution seen so far.

Knapsack Problem – Branch and Bound

• Given n items of known weights w and values v i = 1, 2, ... ,n and a

1 i, ,

knapsack of capacity W, find the most valuable subset of the items that
fit in the knapsack.
• It is convenient to order the items of a given instance in descending
order by their value-to-weight ratios.
• Then the first item gives the best payoff per weight unit and the last
one gives the worst payoff per weight unit, with ties resolved
arbitrarily:
 State Space Tree
• structure the state-space tree for this problem as a binary tree constructed
in such a way that Each node on the i th level of this tree, 0 <= i <=n
,

represents all the subsets of n items that include a particular selection

made from the first i ordered items.

• a branch going to the left indicates the inclusion of the next item, while a
branch going to the right indicates its exclusion.
• We record the total weight w and the total value v of this selection in the
node, along with some upper bound ub.
 Ub=0+(10-0)*10=100

Ub=0+(10-0)*6=60
Ub=40+(10-4)*6=76

Ub=40+(10-4)*5=70

Ub=65+(10-9)*4=69 Ub=40+(10-4)*4=64

Knapsack capacity=10
 Decision Trees
Many important algorithms, especially those for sorting and searching, work
by comparing items of their inputs. We can study the performance of such
algorithms with a device called the decision tree.
• Internal node of a binary decision tree represents a key comparison,
• The node's left subtree contains the information about subsequent
comparisons made if k < k’.
• while its right subtree does the same for the case of k > k’.
• Each leaf represents a possible outcome of the algorithm's run on some
input of size n.
The algorithm's work on a particular input of size n can be traced by a path
from the root to a leaf in its decision tree, and the number of comparisons
made by the algorithm on such a run is equal to the number of edges in this
path. Hence, the number of comparisons in the worst case is equal to the
Decision Trees for Sorting Algorithms
Decision Trees for Searching a Sorted Array
 Basic Concepts
- Classifying Problems
- Tractable and non-tractable
- Optimization and Decision Problems

- Problem Classes
-P
- NP
- NP Complete

18
 Classifying Problem Complexity
Is the problem tractable, i.e., is there a polynomial-time (O(p(n))
algorithm that solves it?

Possible answers:
• yes (give examples)

• no
• because it’s been proved that no algorithm exists at all
(e.g., Turing’s halting problem)
• because it’s been be proved that any algorithm takes exponential time

• unknown
19
Problem Types: Optimization and Decision

• Optimization problem: find a solution that maximizes or minimizes some objective

function

• Decision problem: answer yes/no to a question

Many problems have decision and optimization versions.

E.g.: traveling salesman problem

• optimization: find Hamiltonian cycle of minimum length
• decision: find Hamiltonian cycle of length  m

Decision problems are more convenient for formal investigation of their complexity.
20
Class P
P: the class of decision problems that are solvable in O(p(n)) time, where p(n)
is a polynomial of problem’s input size n

Examples:
• searching

• element uniqueness

• graph connectivity

• graph acyclicity

• primality testing (finally proved in 2002) 21

 Class NP
NP (nondeterministic polynomial): class of decision problems whose proposed
solutions can be verified in polynomial time = solvable by a nondeterministic
polynomial algorithm

A nondeterministic polynomial algorithm is an abstract two-stage procedure

that:

• generates a random string that may solve the problem

• checks whether this solution is correct, in polynomial time

By definition, it solves the problem if it will generate and verify a solution on one
of its tries 22
 What problems are in NP?
• Hamiltonian circuit existence
• Partition problem: Is it possible to partition a set of n integers into two
disjoint subsets with the same sum?
• Decision versions of TSP, knapsack problem, graph coloring, and many
other combinatorial optimization problems.
• (Few exceptions include: MST, shortest paths)

• All the problems in P can also be solved in this manner (no guessing is
necessary), so we have:
P  NP

• Big question: Is P = NP ? Or is P ⊂ NP ?
• Answer unknown. Current approach: focus on hardest NP problems 23
 NP-Complete : The Hardest NP Problems
A decision problem D in NP is NP-complete if it’s as hard as (ie no easier than)
any and every problem in NP. That is:
• D is in NP
• every problem in NP is polynomial-time reducible to D
NP problems

NP-complete
problem

Think: Which problems can be easier? 24

 Proving Other Problems are NP-Complete
Prove other problems are NP-complete by polynomial-
time reductions from a known NP-complete problem
NP problems

known
NP-complete
problem
candidate
for NP -
completeness

Pay attention to the direction of the reductions

Examples: TSP, knapsack, partition, graph-coloring and
hundreds of other problems of combinatorial nature
25
 P = NP ? Dilemma Revisited
• If P = NP then every problem in NP (including all NP-complete problems) could
be solved in polynomial time
• If a polynomial-time algorithm for just one NP-complete problem is discovered,
then every problem in NP can be solved in polynomial time, i.e., P = NP
NP problems

NP-complete
problem

• Most but not all researchers believe that P  NP , i.e. P is a proper subset of NP

26