Undecidable Problems and Approximation Algorithms

Subject: Design and Analysis of Algorithms
Unit 5: Coping with an Algorithm Limitations
( Undecidable Problems and Approximation Algorithms)
Prepared by: B. Muthukrishna Vinayagam, AP/CSE

Limitation of an Algorithm Power
• Algorithm ?
• Three Things about an Algorithm?
• Some Problems have no algorithm – No Solution
• Some Problems can be solved in Non Polynomial Time
• Some Problems can solved in Polynomial Time
• Three bounds of an Algorithms?
• Lower bound: Minimum amount of time taken by an algorithm to solve
problem
• Upper bound: Maximum amount of time required by an algorithm to solve
problem
• Tight bound: Time taken by an algorithm same as the efficiency class.

Lower Bounds:
• Lower bound can be
• an exact count
• an efficiency class ()
• Tight lower bound: there exists an algorithm with the
same efficiency as the lower bound
Problem Lower bound Tightness
sorting (comparison-based) (nlog n) yes
searching in a sorted array (log n) yes
element uniqueness (nlog n) yes
n-digit integer multiplication (n) unknown
multiplication of n-by-n matrices (n2) unknown

Four Type of Obtaining Algorithm Lower Bound
1. Trivial Lower Bound: based on counting no of inputs are given and
counting number of outputs are generated.
Ex: Matrix Multiplication: inputs: 2*n2 and output:n2
2. Information Theoretical Arguments: based on comparison model.
Ex: Guessing a number with in the range
3. Adversary Arguments: for the legitimate input an algorithm takes long run
to give results. Ex: Eating: Last person completion time is the result.
4. Reduction: P is Hard Problem, Q is Known Problem has an algorithm. So
All the instance of Q is transferred to P. (reduction). If the Problem P is
deduced to Q so that Q has the result then P has also result.

Decision tree —model algorithms that use comparisons:
• internal nodes represent comparisons
• leaves represent outcomes

Decision Trees and Sorting Algorithms
• Any comparison-based sorting algorithm can be represented by a
decision tree (for each fixed n)
• Number of leaves (outcomes)  n!
• Height of binary tree with n! leaves  log2n!
• Minimum number of comparisons in the worst case  log2n! for any
comparison-based sorting algorithm, since the longest path represents
the worst case and its length is the height
• log2n!  n log2n (by Sterling approximation)
• This lower bound is tight (mergesort or heapsort)

Insertion sort
a < b
b < c a < c
y es
y es no
noy esno
a < c b < c
a < b < c
c < a < b
b < a < c
b < c < a
no y es
abc
abc bac
bcaacb
y es
a < c < b c < b < a
no

Undecidable Problems and Approximation Algorithms

 P Problems
A problem is assigned to the P (polynomial time) class if there exists at least one algorithm to solve that
problem, such that the number of steps of the algorithm is bounded by a polynomial in n, where n is the length
of the input.
Time complexity of Polynomial Problems are O(log n), O(n) and O(n2). The running time of Polynomial
Problems is efficient and tractable Problems.
 Example: Kruskal’s Algorithm
Class P & NP Problems

• # NP Problems
• NP is the set of decision problems where the "yes"-instances can be decided in polynomial time
by a non-deterministic Turing machine.
• Example : Travelling salesman problem is in NP.
• Time complexity of Polynomial Problems are O(10n) and O(n!). The running time of Non
Polynomial Problems is inefficient

• #NP Complete Problems
Reduction: It allows proof by giving an algorithm to perform the
transformation. That is Problem A is deduced to problem B, if B is
decidable then A is decidable
Graph coloring problem:
•Adjacency or neighbour nodes do not have same color.
•If n vertex use m colors ( m<=n )

Boolean satisfiability problem (Sat.)
• A Boolean expression E is said to be satisfiable if there exists at-least
one truth assignment T that satisfies E.
Truth Assignment:
A Truth Assignment for a given Boolean expression E assign
either true or false to each of the variables mentioned in E.
• Boolean Expression operators: logical AND(), OR(), Negation().
• Ex: Boolean Expression: x(yz)
• Truth Assignment: T(x)=true or 1 , T(y) and T(z) =false or 0 then result of
expression is E(T)=1.

Backtracking
• It is the variation of exhaustive search, Where search is refined by
eliminating other possibilities.
• Backtracking is to solve combinatorial Problem, decision Problem, and
Optimization Problem.
• It uses depth First Search.
• To construct a solution one component at a time.
• If partially constructed solution can be developed further with out violating the
problem constraints (It is done by taking first legitimate option).
• If there is no legitimate option for the next component (Do Backtracking). i.e. replace
the last partially constructed component solution with its next option. Repeat the
same step(backtrack & check the parent) until legitimate option.
• State Space Tree: It is an arrangement of all possible solution in a tree like
fashion.
• Answer Node: It is solution from root to leave.
• Live Node: Currently generated node; Yet to generate children node.
• E – Node: A node under consideration or Promising
• Dead Node: A Node cannot be considered further.

Hamiltonian
Circuit
Problem:
It is a path start
& end with same
vertex pass
through all the
vertices exactly
once.
Hamiltonian tour
with minimum
cost is called as
Travelling
Salesman
Problems

Sum of Subset:
A Set consists of n positive
integers
{ a1,a2, …, an}
a1<a2<a3<…<an
d=Sum of Subset of n +ve
interger.
Termination:
s +ai+1 >d (too large)
s + 𝑗=𝑖+1
𝑛
𝑠𝑗 < d (too small)

Branch and Bound Problem
• Only for optimization problem, use DFS, BFS, Least Cost Search.
• Promising Node: Leads to complete the solution otherwise Non
Promising Node.
• Feasible Solution: Satisfies all the problem constraints.
• Optimal Solution: Feasible Solution with Object Function.
• Set a Initial Bound at Root.
• To Generate branches when search solution(Adding Further Component to
the Partial Solution). Which node has best bound, chosen as Expanding Node
or Branch Node.
• Each generated Node calculate bound; so that branch can be terminated.
• The Node bound is best & No Component has left
• Other Nodes are no longer feasible solution.
• No Nodes are promising.

In Maximization problems:
0<S≤S* , Performance ratio R(A) = S*/S
In Minimization Problems:
0<S*≤S , Performance ratio R(A) = S/S*
Approximation Algorithm:
An algorithm that returns an answer S
which is “close” to the optimal solution
S* is called an approximation algorithm

Some Characteristics of
Approximation Algorithms
• Time-efficient (sometimes not as efficient as heuristics)
• Don’t guarantee optimal solution
• Guarantee good solution within some factor of the optimum
• Rigorous mathematical analysis to prove the approximation
guarantee

Traveling Salesman Problem
• Allow the tour to visit a vertex more than once. Although this violates
the problem requirements, this restriction can be relaxed initially and
imposed later.
• Restrict the TSP to a Metric TSP by introduction metric conditions as
approximation algorithm exists for Euclidean TSP.
• Triangle inequality: d(i,j)<=d(i,k)+d(k,j) for all vertices i, j & k
• Symmetry: d(i,j)=d(j,i)

Nearest-neighbor algorithm
The following well-known greedy algorithm is based on the nearest-
neighbor heuristic: always go next to the nearest unvisited city.
• Step 1 Choose an arbitrary city as the start.
• Step 2 Repeat the following operation until all the cities have been
visited: go to the unvisited city nearest the one visited last (ties can be
broken arbitrarily).
• Step 3 Return to the starting city.
Sa : a – b –c –d – a is 10
S*= a – b – d – c – a is 8

Multifragment-heuristic algorithm
• Step 1 Sort the edges in increasing order of their weights. (Ties can be
broken arbitrarily.) Initialize the set of tour edges to be constructed to
the empty set.
• Step 2 Repeat this step n times, where n is the number of cities in the
instance being solved: add the next edge on the sorted edge list to
the set of tour edges, provided this addition does not create a vertex
of degree 3 or acycle of length less than n; otherwise, skip the edge.
• Step 3 Return the set of tour edges.
Sa : (B,C) (A,C), (B,D) (A,D) is 24
S*: A – D – B – C – A is 24
A – C – B – D – A is 24

Using MST
• Twice-around-the-tree algorithm
Step 1 Construct a minimum spanning tree of the graph corresponding
to a given instance of the traveling salesman problem.
Step 2 Starting at an arbitrary vertex, perform a walk around the
minimum spanning tree recording all the vertices passed by. (This can
be done by a DFS traversal.)
Step 3 Scan the vertex list obtained in Step 2 and eliminate from it all
repeated occurrences of the same vertex except the starting one at the
end of the list. (This step is equivalent to making shortcuts in the walk.)
The vertices remaining on the list will form a Hamiltonian circuit, which
is the output of the algorithm.

Spring 2003 Approximation Algorithmes 35
Traveling salesman problem with triangle
inequality.
root
MST
Pre-Order walk Hamiltonian
Cycle

Thanks to: Swati Singhal
(28,MCS '09)
Edges shown dotted do not carry weight and are assumed to be
shortest path between the pair of vertices( due to triangular inequality).
Example
Given a Complete Graph
V5
V1
V4
V2
V3
V6 V9
V7
V8
V10
3 3
7
1
3
2
2
1
4
6
2
2
5
3
5

(28,MCS '09)
7
1
3
2
2
1
4
6
2
2
5
3
5
3 3V1 V2 V8
V4 V3 V7
V5
V6 V9
V10
Step1: Compute Minimum Spanning Tree

(28,MCS '09)
Step 2: Double each edge of MST
1
3
2
2
1
2
2 3
3
V5
V10
V7
V8V2V1
V6
V3V4
V9
3
13
2
2
3
1
2
2

(28,MCS '09)
Step 3: Computing Eulerian Cycle
A cycle is one in which each edge visited exactly once
v1  v2  v3  v4 v5  v6  v5  v4  v3 v7 v9 v10 v9
v7v8v7v3v2v1
1
3
2
2
1
2
2
3
3
V5
V6
V4 V3
V1 V2 V8
V7
V10
V9
3
1
3
2
2
3
2
12

(28,MCS '09)
Step 4: Computing solution for TSP
1
3
2
2
1
2
2
3
3
V4
V9
V10
V6
V8
V5
V7V3
V2V1
3
3
3
2
2
1
2
2
1
v1  v2  v3  v4 v5  v6  v5  v4  v3 v7 v9 v10 v9
v7v8v7v3v2v1

(28,MCS '09)
1
2
1
2
2
3
3
v1  v2  v3  v4 v5  v6 v7 v9 v10 v8v1
Approximate solution for TSP
V1
V9
V8
V3
V2
V4
V5
V6
V7
V10

43
Time Complexity
APPROX-TSP-TOUR2(G, c)
1 Select a vertex r Є V[G] to be root.
2 Compute a MST T for G from root r using Prim
Alg.
3 Find a minimal-weight matching M for vertices of
odd degree in T.
4 Find an Euler cycle A in G’ = (V, T U M).
5 L=list of vertices in preorder walk of A.
6 return the Hamiltonian cycle H in order L.
O(1)
O(n lg n)
O(n3)
O(n2)
O(n)

Cristofides Improvement
A more sophisticated algorithm (which again uses the MST algorithm as a subroutine)
guarantees a solution within factor of 1.5 of the optimum (Christofides).
• Start with MST T as before
• Identify nodes with odd degree
• Find a minimum weight matching M on these nodes
• Now compute an Euler tour of the graph of T union M (with shortcuts to prevent visiting
an edge twice)
• This solution is guaranteed to have length at most 3/2.
Step 1: Find an MST, T, of G.
Step 2: Compute a minimum cost perfect matching, M, on the odd degree vertices of T.
Add M to T and obtain an Eulerian graph.
Step 3: Find an Euler tour, T’, of this graph.
Step 4: Output the tour that visits vertices of G in order of their first appearance in T’.
Call this tour C.

9-46
• Step1: Find a minimal spanning tree.
An example for ETSP algorithm

9-47
• Step2: Perform weighted matching. The number of points with odd
degrees must be even because is even.

n
i
i Ed
1
2

9-48
• Step3: Construct the tour with an Eulerian cycle
and a Hamiltonian cycle.

9-49
• Time complexity: O(n3)
Step 1: O(nlogn)
Step 2: O(n3)
Step 3: O(n)
• How close the approximate solution to an optimal
solution?
• The approximate tour is within 3/2 of the optimal one.
(The approximate rate is 3/2.)
(See the proof on the next page.)

50
h
c
b
a d
e
gf
MST
h
c
b
d
e
gf
Input
(assume Euclidean distances)
a
Example

51
h
c
b
a d
e
gf
Walk W
a b c b h b a d e f e g e d a
h
c
b
a d
e
gf
Hamiltonian Cycle H
a b c h d e f g a
2-approx.

h
c
b
a d
e
gf
MST T
h
c
b
e
gf
Matching M
1.5 approx.

h
c
b
a d
e
gf
G' = MST + Matching
h
c
b
e
gf
Matching M
Matching
h
c
b
a d
e
gf
Euler Tour: a,b,c,b,h,f,e,g,e,d,a
Hamilton Tour: a,b,c,h,f,e,g,d,a

Greedy algorithm for the continuous knapsack problem
Step 1 Compute the value-to-weight ratios vi/wi, i = 1, . . . , n, for the items
given.
Step 2 Sort the items in non increasing order of the ratios computed in
Step 1. (Ties can be broken arbitrarily.)
Step 3 Repeat the following operation until the knapsack is filled to its full
capacity or no item is left in the sorted list: if the current item on the list fits
into the knapsack in its entirety, take it and proceed to the next item;
otherwise, take its largest fraction to fill the knapsack to its full capacity
and stop.
Item 1, Item 2 (6/7) ratio because of Continuous
40 + 42*6/7=76

Vertex Cover Problem: Graph(G,E) is to find a vertex cover such that every edge of G is
incident on at least one vertex in VC. A Vertex is said to be cover if the edges of the graph G
are incident on it. A Cover is thus a minimum set vertices which ensures all the edges of the
given graph are incident on at least one vertex of the set VC.
Step 1: Initialize the vertex covet VC as NULL
Step 2: while the edge list E in not empty do the following
a. Pick an edge e={u,v} arbitrarily.
b. VC=VC U { u,v}
c. Delete e and all edges that has u and v end points
Step 3: Return vertex Cover VC

VERTEX COVER ALGORITHM
• Here is a trivial 2-approximation algorithm
• Input is some graph G = (V,E).
APPROX-VERTEX-COVER
1: C ← Ø ;
2: E′ ← E
3: while E′ ≠ Ø; do
4: let (u, v) be an arbitrary edge of E′
5: C ← C {(u, v)}
6: remove from E′ all edges incident on either u or v
7: end while


VERTEX COVER EXAMPLE
Input Graph
a
b c
e
d
f g

VECTEX COVER ALGORITHM
EXAMPLE
a
b c
e
d
f g
Step 1: choose
edge (c,e)
a
b c
e
d
f g
Step 2: choose
edge (d,g)
a
b c
e
d
f g
Step 3: choose
edge (a,b)
a
b c
e
d
f g
Result: Size 6

Bin Packing Problem: Backing of an items W=10 , { 5,6,3,7,5,4}
Next Fit: While placing element in bin checking start from current bin.
First Fit: While placing element in bin checking start from first bin.
Best Fit: Least Space ( room left) should be checked choosing optimal bin place.
Worst Fit: Most space or room left should be checked.
Number of bins are required:4
3 4
5 6 7 5
Bin 1 Bin 2 Bin 3 Bin 4
3 4
5 6 7 5
5 3
5 6 7 4
3 4
5 6 7 5
W=10 { 7,6,5,5,4,3}
First Fit Decreasing: First arranging of an
items in decreasing order and follow the
first fit approach
Best Fit Decreasing: First arranging of an
items in decreasing order and follow the
best fit approach
3 4 5
7 6 5
Bin 1 Bin 2 Bin 3
3 4 5
7 6 5
Bin 1 Bin 2 Bin 3
Number of bins are required: 3

References
1. AnanyLevtin “Introduction to The Design & Analysis of Algorithms”

Undecidable Problems and Approximation Algorithms

Recommended

More Related Content

What's hot (19)

Similar to Undecidable Problems and Approximation Algorithms (20)

Recently uploaded (20)

Undecidable Problems and Approximation Algorithms