Decrease & Conquer
Uploaded by
muhammadusamahoyrrDecrease & Conquer
Uploaded by
muhammadusamahoyrrDesign and Analysis of Algorithm
Tanveer Ahmed Siddiqui
Department of Computer Science
COMSATS University, Islamabad
Recap and Today Covered
Algorithm Design and Analysis Process
Understand the problem
Decide on : algorithm
design techniques etc.
Design an algorithm
Prove correctness
Analyze efficiency etc
Code the algorithm
Department of Computer Science
Reading Material
Read Chapter 4
Decrease and Conquer
Department of Computer Science
Objectives
How to Design Algorithm using decrease and
conquer Approach
To sesign a variety of solutions using decrease and
conquer:
Problems: Decrease (by a constant) and conquer
Problems: Decrease (by constant factor) and conquer
Problems: Decrease (by variable size) and conquer.
Department of Computer Science
Solving Complex problem
How to design algorithm of a complex
problem? i.e. How to solve a complex
problem?
Break down the problem into segments
(smaller chunks/parts) and then target to solve
the segments, ultimately the problem will be
solved.
How we reduce the size of an instance of a
problem?
Decreasing
Decrease and conquer
Dividing
Divide and conquer
Department of Computer Science
Decrease and Conquer
Decrease or reduce problem instance to
smaller instance of the same problem and
extend solution.
Conquer the problem by solving smaller
instance of the problem.
Extend solution of smaller instance to obtain
solution to original problem.
Other Names:
This approach is also known as incremental or
inductive approach.
Department of Computer Science
Decrease and Conquer
Exploit the relationship between a solution to a
given instance of a problem and a solution to
its smaller instance
relationship between solution of P(n) {a given
instance of a problem} and a solution to P(n-1) or
P(n/2) {a smaller instance} of the same problem.
3 major variants:
Decrease by a constant
Decrease by a constant factor
Variable size decrease
Bottom-up: iterative
Top-down: recursive
Department of Computer Science
Top Down and Iterative
Top Down Iterative
It is a top down approach It is bottom up approach
Department of Computer Science
Decrease by
Constant
Department of Computer Science
Decrease by a constant factor
Decrease by a constant: Reduce the size of
the problem by the same constant on each
iteration/recursion of the algorithm.
Department of Computer Science 10
A Typical Decrease by one Technique
a problem of size n
subproblem
Recursive
of size n-1
a solution to the
subproblem
e.g., n!
a solution to
the original problem 11
Iterative
Department of Computer Science
Example-1: Computing
Brute Force Example a n
Design an algorithm that computes an using
decrease by constant
Solution: BF an = a × a × … … × a
What is input
Two numbers say a and n.
What should be the output
Interchange the value of an
Which designing technique?
Decrease and conquer
Top down: recursive
Logic/Idea a×f(n-1) if n > 0
f(n) =
an = a×an-1 1 if n = 0
Department of Computer Science 12
Example-1: Computing
Brute Force Example a n
Design an algorithm that computes an using
decrease by constant
BF an = a × a × … … × a
Can you improve
Top down: recursive
an computation ?
a×f(n-1) if n > 0
f(n) =
1 if n = 0
Department of Computer Science 13
Example-2:
Brute Maximum Number
Force Example
Design an algorithm that find maximum in an
array A[0..n-1] using decrease by constant
Solution: BF
What is input
An array A of n numbers.
What should be the output
The largest value from given array A
Which designing technique?
Decrease and conquer
Top down: recursive
Logic/Idea
Department of Computer Science 14
Example-2:
Brute Maximum Number
Force Example
Design an algorithm that find minimum in an
array A[0..n-1] using decrease by constant
Solution:
Decrease and Conquer
BF
Department of Computer Science 15
Example-3:
Brute Force Sorting
Example
Design an algorithm that arrange elements of an
array A[0..n-1] in ascending order using
decrease by constant BF
Solution:
What is input
An array A of n numbers.
What should be the output
The array A with increasing order
Which designing technique?
Decrease and conquer
Logic/Idea
Department of Computer Science 16
Sorting : Decrease By one
We assume that the smaller instance A[0..n-2]
has already been sorted.
How can we solve for A[0..n-1] now?
Department of Computer Science 17
Sort: Decrease by one(Iterative)
Obviously, all we need is to find an appropriate
position for A[n-1] among the sorted elements and
insert it there.
There are three alternatives for doing this:
1st Method: Scan the sorted subarray from left to
right until the first grater than or equal to A[n-1] is
encountered and then insert A[n-1] right before that
element.
29 45 68 89 90 | 34 17 45 68 89 90 | 29 34 17
29 34 45 68 89 90 | 17 29 45 68 89 90 | 34 17
45 68 89 | 90 29 34 17
Department of Computer Science
45 68 89 90 | 29 34 17 18
Sort: Decrease by one(Iterative)
Obviously, all we need is to find an appropriate
position for A[n-1] among the sorted elements and
insert it there.
There are three alternatives for doing this:
2nd Method: Scan the sorted subarray from right to
left until the first smaller than or equal to A[n-1] is
encountered and then insert A[n-1] right after that
element.
29 45 68 89 90 | 34 17 45 68 89 90 | 29 34 17
29 34 45 68 89 90 | 17 29 45 68 89 90 | 34 17
45 68 89 | 90 29 34 17
Department of Computer Science
45 68 89 90 | 29 34 17 19
Comparison of two strategy
1st Method: Scan the sorted subarray from left to
right until the first grater than or equal to A[n-1] is
encountered and then insert A[n-1] right before that
element.
2nd Method: Scan the sorted subarray from right to
left until the first smaller than or equal to A[n-1] is
encountered and then insert A[n-1] right after that
element.
Which one is better and why?
Department of Computer Science 20
Sort: Decrease by one(Iterative)
Logic/Idea
Obviously, all we need is to find an
appropriate position for A[n-1] 89 | 45 68 90 29 34 17
among the sorted elements and
insert it there. 45 89 | 68 90 29 34 17
45 68 89 | 90 29 34 17
45 68 89 90 | 29 34 17
29 45 68 89 90 | 34 17
29 34 45 68 89 90 | 17
17 29 34 45 68 89 90
Department of Computer Science
Insertion Sort: Analysis of Iterative Solution
ALGORITHM InsertionSortIter(A[0..n-1])
//An iterative implementation of insertion sort
//Input: An array A[0..n-1] of n orderable elements
//Output: Array A[0..n-1] sorted in nondecreasing order
for i 1 to n – 1 do // i: the index of the first element of the unsorted part.
key = A[i]
for j i – 1 to 0 do // j: the index of the sorted part of the array
if (key< A[j]) //compare the key with each element in the sorted part
A[j+1] A[j]
else
break
A[j +1] key //insert the key to the sorted part of the array
Department of Computer Science 22
Insertion Sort: Analysis
ALGORITHM InsertionSort(A[0..n-1])
for i <- 1 to n-1 do What is the worst case scenario ?
v <- A[i]
j <- i-1
A[j] > v executes highest # of times
while j ≥ 0 and A[j] > v do When does that happen ?
A[j+1] <- A[j] A[j] > A[i] for j = i-1, i-2, …, 0
Can you improve it j?<- j-1
A[j+1] <- v Worst case input:
An array of strictly decreasing values
For almost sorted files,
What is the best case ?
insertion sort’s performance
is excellent! A[i-1] ≤ A[i] for i = 1, 2, …, n-1
In place? Stable ?
Department of Computer Science
Insertion Sort: Decrease by one(Recursive Solution)
ALGORITHM InsertionSortRecur(A[0..n-1], n-1)
//A recursive implementation of insertion sort
//Input: An array A[0..n-1] of n orderable elements Index of the last element
//Output: Array A[0..n-1] sorted in nondecreasing order
if n > 1
InsertionSortRecur(A[0..n-1], n-2)
Insert(A[0..n-1], n-1) //insert A[n-1] to A[0..n-2]
Index of the element/key to be inserted.
ALGORITHM Insert(A[0..m], m)
//Insert A[m] to the sorted subarray A[0..m-1] of A[0..n-1]
//Input: A subarray A[0..m] of A[0..n-1]
//Output: Array A[0..m] sorted in nondecreasing order
key = A[m]
for j m – 1 to 0 do
if (key< A[j])
A[j+1] A[j]
else
break
A[j +1] Department
key of Computer Science 24
Example -2
Recurrence relation:
T(n) = T(n-1) + n
T(1) = 1
Telescoping:
T(n) = T(n-1) + n
T(n-1) = T(n-2) + n-1
T(n-2) = T(n-3) + n-2
…
T(2) = T(1 ) + 1
Add the equations and cross equal terms on opposite sides:
T(n) = n+n-1+ n-2…1 =n(n+1)/2=O(n2)
Department of Computer Science 25
Example
T (n) = T (n − 1) + n
Department of Computer Science
Example
T(n) = T(n-1) + n
Guess: T(n) = O(n2)
Induction goal: T(n) ≤ c n2, for some c and n ≥ n0
Induction hypothesis: T(n-1) ≤ c(n-1)2 for all k < n
Proof of induction goal:
T(n) = T(n-1) + n ≤ c (n-1)2 + n
= cn2 – (2cn – c - n) ≤ cn2
if: 2cn – c – n ≥ 0 c ≥ n/(2n-1) c ≥ 1/(2 – 1/n)
For n ≥ 1 2 – 1/n ≥ 1 any c ≥ 1 will work
Department of Computer Science 27
Insertion Sort: Food for thought
Compare the above version of insertion sort with
the following version.
What is the time efficiency of this algorithm?
How is it compared to that of the version
discussed in previous slide.
Department of Computer Science
Insertion Sort: Food for thought
Observe that the while loop of the Insertion-
sort procedure uses a linear search to scan
(backward) through the sorted subarray A[0..i-
1]. Can we use a binary search instead to
improve the overall worst-case running time of
insertion sort to Θ (n㏒n)?
Department of Computer Science
Exercise
Design a recursive decrease-by-one algorithm
for finding the position of the largest element in
an array of n real numbers.
Determine the time efficiency of this algorithm
and compare it with that of the brute-force
algorithm for the same problem.
Department of Computer Science 30
Sorting
BF
Decrease and Conquer
Department of Computer Science
Graph Representation
Adjacency matrix
n x n boolean matrix if |V| is n.
The element on the ith row and jth column is 1 if there’s an
edge from ith vertex to the jth vertex; otherwise 0.
The adjacency matrix of an undirected graph is symmetric.
It takes (comparisons) to figure out all the adjacent
nodes of a certain node i.
Adjacency linked lists
A collection of linked lists, one for each vertex, that contain all
the vertices adjacent to the list’s vertex.
An example
Department of Computer Science 32
Depth First Search
The idea
traverse “deeper” whenever possible.
On each iteration, the algorithm proceeds to an unvisited vertex that is
adjacent to the one it is currently in. If there are more than more
neighbors, break the tie by the alphabetic order of the vertices.
When reaching a dead end ( a vertex with no adjacent unvisited
vertices), the algorithm backs up one edge to the parent and tries to
continue visiting unvisited vertices from there.
The algorithm halts after backing up to the starting vertex, with the
latter being a dead end.
Similar to preorder tree traversals
It’s convenient to use a stack to trace the operation of depth-first
search.
Push a vertex onto the stack when the vertex is reached for the first
time.
Pop a vertex off the stack when it becomes a dead end.
An example
Department of Computer Science 33
Example: Undirected Graph
a b c d
e f g h
Depth-first traversal: Give the order in which the
vertices were reached.
abfegcdh
Department of Computer Science 34
Depth First Search
Pseudocode for Depth-first-search of graph G=(V,E)
DFS(G) // Use depth-first to visit a graph, which might
// contain multiple connected components.
count 0 //visiting sequence number
mark each vertex with 0 // (unvisited)
for each vertex v∈ V do
if v is marked with 0 //w has not been visited yet.
dfs(v)
dfs(v)
//Use depth-first to visit a connected component starting
//from vertex v.
count count + 1
mark v with count //visit vertex v
for each vertex w adjacent to v do
if w is marked with 0 //w has not been visited yet
dfs(w)
Department of Computer Science 35
Example: Directed Graph
a b c d
e f g h
Depth-first traversal: Give the order in which the
vertices were reached.
abfghecd
Department of Computer Science 37
Depth First Search: Notes
DFS can be implemented with graphs
represented as:
Adjacency matrices: Θ(|V |2)
Adjacency linked lists: Θ(|V |+|E| )
Applications:
Checking connectivity
Finding connected components
Department of Computer Science 38
Breadth First Search
The idea
Traverse “wider” whenever possible.
Discover all vertices at distance k from s (on level k)
before discovering any vertices at distance k +1 (at
level k+1)
Similar to level-by-level tree traversals
Instead of a stack, breadth-first uses a queue.
Department of Computer Science 39
Example: Undirected Graph
a b c d
e f g h
Breadth-first traversal:
abefgchd
Department of Computer Science 40
Breadth First Search Algorithm
BFS(G)
count 0
bfs(v)
mark each vertex with 0
for each vertex v∈ V do count count + 1
if v is marked with 0 mark v with count //visit v
bfs(v) initialize queue with v //enqueue
while queue is not empty do
a front of queue //dequeue
for each vertex w adjacent to a do
if w is marked with 0 //w hasn’t been visited.
count count + 1
mark w with count //visit w
add w to the end of the queue //enqueue
remove a from the front of the queue
Department of Computer Science 41
Example: Directed Graph
a b c d
e f g h
Breadth-first traversal:
abefghcd
Department of Computer Science 42
Example-4:
Brute Topological Sorting
Force Example
Design an algorithm that orders a Directed
Acyclic Graph in such a way each node appears
before all the nodes it points to in the returned
order, i.e. if we have a --> b, a must appear
before b in the topological order.
Solution:
What is input
A 2D Array representing the given graph.
What should be the output
Linear ordering of vertices of DAG
Which designing technique?
Decrease and conquer
Logic/Idea
Department of Computer Science 43
Directed Graph or Digraph
Directions for all edges a b c d e
a 0 1 1 0 0
a b
b 1 0 1 0 0
c c 0 0 0 0 0
0 0 1 0 1
d d
0 0 0 0 0
e
a b c
e
b a c Not symmetric
c
d c e
e One node for one edge
Department of Computer Science
Directed Graph or Digraph
a b a b c
b a c
c c
d c e
e
d
a d
Tree edge
e
Back edge
b e
Forward edge
Directed cycle: a, b, a
Cross edge
c
DFS forest
If no directed cycles, the digraph
is called “directed acyclic graph” or “DAG”
Department of Computer Science
UPSHOT
A directed graph with no (directed) cycles is
called directed acyclic graph(DAG)
a b a b
not a DAG
c d c d
DAG
Department of Computer Science 46
DFS based Algorithm
DFS-based algorithm for topological sorting
Perform DFS traversal, noting the order vertices are
popped off the traversal stack
Reverse order solves topological sorting problem
Example:
Set of 5 courses: { C1, C2, C3, C4, C5 }
C1 and C2 have no prerequisites
C3 requires C1 and C2 C1 C4
C4 requires C3 C3
C5 requires C3 and C4
C2 C5
Department of Computer Science
DFS based Algorithm: Example
Set of 5 courses: { C1, C2, C3, C4, C5 }
C1 and C2 have no prerequisites
C3 requires C1 and C2
C4 requires C3 C 1 , C 2 , C 3 , C 4 , C5 !
C5 requires C3 and C4
Department of Computer Science
DFS based Algorithm: Your Turn
DFS-based algorithm for topological sorting
Perform DFS traversal, noting the order vertices are
popped off the traversal stack
Reverse order solves topological sorting problem
Example:
a b c d
e f g h
Department of Computer Science
Source Removal Algorithm
Source removal algorithm
Repeatedly identify and remove a source (a vertex with
no incoming edges) and all the edges incident to it until
either no vertex is left (problem is solved) or there is no
source among remaining vertices (not a dag)
Example:
Department of Computer Science
Source Removal Algorithm: Your Turn
Source removal algorithm
Repeatedly identify and remove a source (a vertex with
no incoming edges) and all the edges incident to it until
either no vertex is left (problem is solved) or there is no
source among remaining vertices (not a dag)
Example:
a b c d
e f g h
Department of Computer Science
Application of Topological Sorting
A large project – e.g., in construction, research,
or software development – that involves a
multitude of interrelated tasks with known
prerequisites
Schedule to minimize the total completion time
Instruction scheduling in program compilation,
resolving symbol dependencies in linkers, etc.
Department of Computer Science
Application of Topological Sorting
Other Examples:
Finding cycle in a graph.
Operation System deadlock detection.
Dependency resolution.
Sentence Ordering.
Critical Path Analysis.
Course Schedule problem.
Department of Computer Science
Decrease by a
constant factor
Department of Computer Science
Variants of Decrease and Conquer
Decrease by a constant factor: the size of the problem is
reduced by the same constant factor on each
iteration/recursion of the algorithm.
Department of Computer Science 55
Example-1: Computing
Brute Force Example a n
Design an algorithm that computes an using
decrease by constant factor
Decrease by a constant factor
Solution:
What is input
Two numbers say a and n.
What should be the output
Return the value of an
Which designing technique?
Decrease(by a constant factor and conquer
Logic/Idea Decrease by a constant factor
Divide the power in half
at each step
Department of Computer Science 56
Example-1: Computing
Brute Force Example a n
Design an algorithm that computes an using
decrease by constant factor
Decrease by a constant factor
Solution:
Effectively, when
power is not
Suppose we want to compute 95
divisible by 2, we
make power even
by taking out the
extra 9. Then we
already know the
solution when
power is divisible by
2. Divide the power
Department of Computer Science 57
by 2 and multiply
Example-1: Computing
Brute Force Example a n
Design an algorithm that computes an using
decrease by constant factor
Decrease by a constant factor
Solution:
Now our problem is to find (812) * 9
Department of Computer Science 58
Example-1: Computing
Brute Force Example a n
Design an algorithm that computes an using
decrease by constant factor
Decrease by a constant factor
Solution:
Department of Computer Science 59
Example-1: Computing
Brute Force Example a n
Design an algorithm that computes an using
decrease by constant factor
Decrease by a constant factor
Solution:
Department of Computer Science 60
Example 2: Searching a Key
a problem of size n
subproblem
of size n/2
a solution to the
subproblem
a solution to
the original problem 61
Department of Computer Science
Example-2:
Brute Searching a Key
Force Example
Design an algorithm that find a value in a given
sorted array using decrease by constant
factor
Solution:
What is input
An array A of n numbers and a key.
What should be the output
Return location of key if found otherwise return false
Which designing technique?
Decrease(by a factor) and conquer
Logic/Idea
repeatedly dividing the search interval in half.
Department of Computer Science 62
Example-2:
Brute Searching a Key
Force Example
nBegin with the mid element of the whole array as a search key.
nIf the value of the search key is equal to the item then return an index
of the search key.
nOr if the value of the search key is less than the item in the middle of
the interval, narrow the interval to the lower half.
nOtherwise, narrow it to the upper half.
nRepeatedly check from the second point until the value is found or the
interval is empty.
If ( value == middle element )
value is found
else if ( value < middle element )
search left-half of list with the same method
else
search right-half of list with the same method
Department of Computer Science 63
Searching an Array: Decrease by constant factor(half) Technique
Binary search is like looking up a phone number
or a word in the dictionary
Start in middle of book
If name you're looking for comes before names on
page, look in first half
Otherwise, look in second half
Department of Computer Science
Binary search: Iterative Algorithm
Algorithm isPresent(A[0…N], val)
{
low = 0;
high = N;
mid;
while ( low <= high ){
mid = ( low + high )/2;
if (A[mid]== val)
return 1; // found!
else if (A[mid] < val)
low = mid + 1;
else
high = mid - 1;
}
return 0; // not found
}
Department of Computer Science 65
Binary search
Case 1: val == a[mid]
val = 10
low = 0, high = 8
mid = (0 + 8) / 2 = 4
a: 1 5 7 9 10 13 17 19 27
0 1 2 3 4 5 6 7 8
low mid high
Department of Computer Science 66
Binary Search
Case 2: val > a[mid]
val = 19
low = 0, high = 8
mid = (0 + 8) / 2 = 4
new low = mid+1 = 5
a: 1 5 7 9 10 13 17 19 27
0 1 2 3 4 5 6 7 8
low new high
mid
low
Department of Computer Science 67
Example 2: Binary Search
Case 3: val < a[mid]
val = 7
low = 0, high = 8
mid = (0 + 8) / 2 = 4
new high = mid-1 = 3
a: 1 5 7 9 10 13 17 19 27
0 1 2 3 4 5 6 7 8
low new mid high
high
Department of Computer Science 68
Binary search: Example 3(contd.)
val = 7
a: 1 5 7 9 10 13 17 19 27
0 1 2 3 4 5 6 7 8
a: 1 5 7 9 10 13 17 19 27
0 1 2 3 4 5 6 7 8
a: 1 5 7 9 10 13 17 19 27
0 1 2 3 4 5 6 7 8
Department of Computer Science 69
Binary Search: Recursive Algorithm
For an ordered array A, finds if x is in the array A[lo…hi]
Algorithm BINARY-SEARCH (A, lo, hi, x)
if (lo > hi)
1 2 3 4 5 6 7 8
return FALSE
mid (lo+hi)/2 2 3 5 7 9 10 11 12
if x = A[mid]
return TRUE mid
lo hi
if ( x < A[mid] )
BINARY-SEARCH (A, lo, mid-1, x)
if ( x > A[mid] )
BINARY-SEARCH (A, mid+1, hi, x)
Department of Computer Science 70
Decrease by
variable size
Department of Computer Science
Decrease by variable size
Variable-size decrease: Problem size decreases at each
iteration in variable amount
Euclid’s algorithm(used in cryptography, e.g. RSA Algorithm)
gcd(m, n) = gcd(n, m mod n)
gcd( 31415, 14142 )= gcd( 14142, 3131 ) Decr. By 11011
= gcd( 3131, 1618 ) Decr. By 1513
= gcd( 1618, 1513 ) Decr. By 105
= gcd( 1513, 105 ) Decr. By 1408
= gcd( 105, 43 ) Decr. By 62
= gcd( 43, 19 ) Decr. By 24
= gcd( 19, 5 ) Decr. By 14
= gcd( 5, 4 ) Decr. By 1
= gcd( 4, 1 ) Decr. By 3
= gcd( 1, 0 ) Decr. By 1
Department of Computer Science
Pseudo code for ()Euclid’s)Algorithm, gcd(m,n)
gcd(m, n) = gcd(n, m mod n)
gcd(24, 60) = gcd(60, 24 mod 60)
What happens if m < n ?
Department of Computer Science 11/2/2024 73
How we do it?
CONCLUSION
Department of Computer Science
Decrease and Conquer
Exploit the relationship between a solution to a
given instance of a problem and a solution to
its smaller instance
Top-down: recursive
Bottom-up: iterative
3 major types:
Decrease by a constant
Decrease by a constant factor
Variable size decrease
Department of Computer Science
Three Types of Decrease and Conquer
Decrease by a constant (usually by 1):
insertion sort
graph traversal algorithms (DFS and BFS)
Topological Sorting
Algorithms for generating permutations, subsets
Decrease by a constant factor (usually by half)
Binary Search and Bisection Method
Exponentiation by Squaring
Multiplication à la russe
Variable-size decrease
Euclid’s algorithm
Selection by partition
Nim-like games
Department of Computer Science
Design and Analysis of Algorithm
Tanveer Ahmed Siddiqui
Department of Computer Science
COMSATS University, Islamabad
Recap and Today Covered
Algorithm Design and Analysis Process
Understand the problem
Decide on : algorithm
design techniques etc.
Design an algorithm
Prove correctness
Analyze efficiency etc
Code the algorithm
Department of Computer Science
Reading Material
Read Chapter 4
Decrease and Conquer
Department of Computer Science
Objectives
How to Design Algorithm using decrease and
conquer Approach
To sesign a variety of solutions using decrease and
conquer:
Problems: Decrease (by a constant) and conquer
Problems: Decrease (by constant factor) and conquer
Problems: Decrease (by variable size) and conquer.
Department of Computer Science
Solving Complex problem
How to design algorithm of a complex
problem? i.e. How to solve a complex
problem?
Break down the problem into segments
(smaller chunks/parts) and then target to solve
the segments, ultimately the problem will be
solved.
How we reduce the size of an instance of a
problem?
Decreasing
Decrease and conquer
Dividing
Divide and conquer
Department of Computer Science
Decrease and Conquer
Decrease or reduce problem instance to
smaller instance of the same problem and
extend solution.
Conquer the problem by solving smaller
instance of the problem.
Extend solution of smaller instance to obtain
solution to original problem.
Other Names:
This approach is also known as incremental or
inductive approach.
Department of Computer Science
Decrease and Conquer
Exploit the relationship between a solution to a
given instance of a problem and a solution to
its smaller instance
relationship between solution of P(n) {a given
instance of a problem} and a solution to P(n-1) or
P(n/2) {a smaller instance} of the same problem.
3 major variants:
Decrease by a constant
Decrease by a constant factor
Variable size decrease
Bottom-up: iterative
Top-down: recursive
Department of Computer Science
Top Down and Iterative
Top Down Iterative
It is a top down approach It is bottom up approach
Department of Computer Science
Decrease by
Constant
Department of Computer Science
Decrease by a constant factor
Decrease by a constant: Reduce the size of
the problem by the same constant on each
iteration/recursion of the algorithm.
Department of Computer Science 86
A Typical Decrease by one Technique
a problem of size n
subproblem
Recursive
of size n-1
a solution to the
subproblem
e.g., n!
a solution to
the original problem 87
Iterative
Department of Computer Science
Example-1: Computing
Brute Force Example a n
Design an algorithm that computes an using
decrease by constant
Solution: BF an = a × a × … … × a
What is input
Two numbers say a and n.
What should be the output
Interchange the value of an
Which designing technique?
Decrease and conquer
Top down: recursive
Logic/Idea a×f(n-1) if n > 0
f(n) =
an = a×an-1 1 if n = 0
Department of Computer Science 88
Example-1: Computing
Brute Force Example a n
Design an algorithm that computes an using
decrease by constant
BF an = a × a × … … × a
Can you improve
Top down: recursive
an computation ?
a×f(n-1) if n > 0
f(n) =
1 if n = 0
Department of Computer Science 89
Example-2:
Brute Maximum Number
Force Example
Design an algorithm that find maximum in an
array A[0..n-1] using decrease by constant
Solution: BF
What is input
An array A of n numbers.
What should be the output
The largest value from given array A
Which designing technique?
Decrease and conquer
Top down: recursive
Logic/Idea
Department of Computer Science 90
Example-2:
Brute Maximum Number
Force Example
Design an algorithm that find minimum in an
array A[0..n-1] using decrease by constant
Solution:
Decrease and Conquer
BF
Department of Computer Science 91
Example-3:
Brute Force Sorting
Example
Design an algorithm that arrange elements of an
array A[0..n-1] in ascending order using
decrease by constant BF
Solution:
What is input
An array A of n numbers.
What should be the output
The array A with increasing order
Which designing technique?
Decrease and conquer
Logic/Idea
Department of Computer Science 92
Sorting : Decrease By one
We assume that the smaller instance A[0..n-2]
has already been sorted.
How can we solve for A[0..n-1] now?
Department of Computer Science 93
Sort: Decrease by one(Iterative)
Obviously, all we need is to find an appropriate
position for A[n-1] among the sorted elements and
insert it there.
There are four alternatives for doing this:
1st Method: Take the element adjacent to the sorted
area. Insert this element into the right place in the
sorted area. This is accomplished by bubbling down
the element to the correct spot in the sorted area.
29 45 68 89 90 | 34 17
29 45 68 89 34 | 90 17
29 34 45 68 89 | 90 17
29 45 68 34 89 | 90 17
29 45 34 68 89 | 90 17
Department of Computer Science 94
1st Method
1st Method: Take the element adjacent to the sorted
area. Insert this element into the right place in the
sorted area. This is accomplished by bubbling down
the element to the correct spot in the sorted area.
29 45 68 89 90 | 34 17
29 45 68 89 34 | 90 17
29 45 68 34 89 | 90 17
29 45 34 68 89 | 90 17
29 34 45 68 89 | 90 17
Department of Computer Science 95
1st Method: Analysis
29 45 68 89 90 | 34 17
29 45 68 89 34 | 90 17
29 45 68 34 89 | 90 17
29 45 34 68 89 | 90 17
29 34 45 68 89 | 90 17
If N swaps are performed by the inner loop, the
given version of sort algorithm requires N *
3 assignment statements to perform those
swaps.
Can we improve it? That is can we minimize
assignment statements?
Department of Computer Science 96
2nd Method
29 45 68 89 90 | 34 17
29 45 68 89 34 | 90 17
29 45 68 34 89 | 90 17
29 45 34 68 89 | 90 17
29 34 45 68 89 | 90 17
Performing a full swap of the array elements in
each inner for loop iteration is not necessary.
Instead, we save the value that we want to insert into
the sorted subarray in temporary storage.
In place of performing a full swap, we simply copy
elements to the right. The saved value can then be
inserted into its proper position once that has been
located.
Department of Computer Science 97
2nd Method
There are two alternatives for doing this:
2nd Method: Scan the sorted subarray from left to
right until the first grater than or equal to A[n-1] is
encountered and then insert A[n-1] right before that
element.
29 45 68 89 90 | 34 17 45 68 89 90 | 29 34 17
29 34 45 68 89 90 | 17 29 45 68 89 90 | 34 17
45 68 89 | 90 29 34 17
45 68 89 90 | 29 34 17
Department of Computer Science 98
3rd Method
3rd Method: Scan the sorted subarray from right to
left until the first smaller than or equal to A[n-1] is
encountered and then insert A[n-1] right after that
element.
29 45 68 89 90 | 34 17 45 68 89 90 | 29 34 17
29 34 45 68 89 90 | 17 29 45 68 89 90 | 34 17
45 68 89 | 90 29 34 17
45 68 89 90 | 29 34 17
Department of Computer Science 99
Comparison of two strategy
2nd Method: Scan the sorted subarray from left to
right until the first grater than or equal to A[n-1] is
encountered and then insert A[n-1] right before that
element.
3rd Method: Scan the sorted subarray from right to
left until the first smaller than or equal to A[n-1] is
encountered and then insert A[n-1] right after that
element.
Which one is better and why?
Department of Computer Science 100
Sort: Decrease by one(Iterative)
Department of Computer Science
Sort: Decrease by one(Iterative)
89 | 45 68 90 29 34 17
45 89 | 68 90 29 34 17
45 68 89 | 90 29 34 17
45 68 89 90 | 29 34 17
The improved version only requires N
29 45 68 89 90 | 34 17
+ 2 assignment statements to
accomplish the same task.
29 34 45 68 89 90 | 17
17 29 34 45 68 89 90
Department of Computer Science
Insertion Sort: Analysis of Iterative Solution
ALGORITHM InsertionSortIter(A[0..n-1])
//An iterative implementation of insertion sort
//Input: An array A[0..n-1] of n orderable elements
//Output: Array A[0..n-1] sorted in nondecreasing order
for i 1 to n – 1 do // i: the index of the first element of the unsorted part.
key = A[i]
for j i – 1 to 0 do // j: the index of the sorted part of the array
if (key< A[j]) //compare the key with each element in the sorted part
A[j+1] A[j]
else
break
A[j +1] key //insert the key to the sorted part of the array
Department of Computer Science 103
Insertion Sort: Decrease by one(Iterative)
ALGORITHM InsertionSort(A[0..n-1])
for i <- 1 to n-1 do
v <- A[i] 89 | 45 68 90 29 34 17
j <- i-1 45 89 | 68 90 29 34 17
while j ≥ 0 and A[j] > v do
45 68 89 | 90 29 34 17
A[j+1] <- A[j]
j <- j-1 45 68 89 90 | 29 34 17
A[j+1] <- v
29 45 68 89 90 | 34 17
Input size: n 29 34 45 68 89 90 | 17
Basic op: A[j] > v
Why not j ≥ 0 ? 17 29 34 45 68 89 90
C(n) depends on input type ?
Department of Computer Science
Insertion Sort: Analysis
ALGORITHM InsertionSort(A[0..n-1])
for i <- 1 to n-1 do What is the worst case scenario ?
v <- A[i]
j <- i-1
A[j] > v executes highest # of times
while j ≥ 0 and A[j] > v do When does that happen ?
A[j+1] <- A[j] A[j] > A[i] for j = i-1, i-2, …, 0
Can you improve it j?<- j-1
A[j+1] <- v Worst case input:
An array of strictly decreasing values
For almost sorted files,
What is the best case ?
insertion sort’s performance
is excellent! A[i-1] ≤ A[i] for i = 1, 2, …, n-1
In place? Stable ?
Department of Computer Science
Insertion Sort: Decrease by one(Recursive Solution)
ALGORITHM InsertionSortRecur(A[0..n-1], n-1)
//A recursive implementation of insertion sort
//Input: An array A[0..n-1] of n orderable elements Index of the last element
//Output: Array A[0..n-1] sorted in nondecreasing order
if n > 1
InsertionSortRecur(A[0..n-1], n-2)
Insert(A[0..n-1], n-1) //insert A[n-1] to A[0..n-2]
Index of the element/key to be inserted.
ALGORITHM Insert(A[0..m], m)
//Insert A[m] to the sorted subarray A[0..m-1] of A[0..n-1]
//Input: A subarray A[0..m] of A[0..n-1]
//Output: Array A[0..m] sorted in nondecreasing order
key = A[m]
for j m – 1 to 0 do
if (key< A[j])
A[j+1] A[j]
else
break
A[j +1] Department
key of Computer Science 106
Example -2
Recurrence relation:
T(n) = T(n-1) + n
T(1) = 1
Telescoping:
T(n) = T(n-1) + n
T(n-1) = T(n-2) + n-1
T(n-2) = T(n-3) + n-2
…
T(2) = T(1 ) + 1
Add the equations and cross equal terms on opposite sides:
T(n) = n+n-1+ n-2…1 =n(n+1)/2=O(n2)
Department of Computer Science 107
Example
T (n) = T (n − 1) + n
Department of Computer Science
Example
T(n) = T(n-1) + n
Guess: T(n) = O(n2)
Induction goal: T(n) ≤ c n2, for some c and n ≥ n0
Induction hypothesis: T(n-1) ≤ c(n-1)2 for all k < n
Proof of induction goal:
T(n) = T(n-1) + n ≤ c (n-1)2 + n
= cn2 – (2cn – c - n) ≤ cn2
if: 2cn – c – n ≥ 0 c ≥ n/(2n-1) c ≥ 1/(2 – 1/n)
For n ≥ 1 2 – 1/n ≥ 1 any c ≥ 1 will work
Department of Computer Science 109
Insertion Sort: Food for thought
Compare the above version of insertion sort with
the following version.
What is the time efficiency of this algorithm?
How is it compared to that of the version
discussed in previous slide.
Department of Computer Science
Insertion Sort: Food for thought
Observe that the while loop of the Insertion-sort
procedure uses a linear search to scan
(backward) through the sorted subarray A[0..i-
1]. Can we use a binary search instead to
improve the overall worst-case running time of
insertion sort to Θ (n㏒n)?
Department of Computer Science
Exercise
Design a recursive decrease-by-one algorithm
for finding the position of the largest element in
an array of n real numbers.
Determine the time efficiency of this algorithm
and compare it with that of the brute-force
algorithm for the same problem.
Department of Computer Science 112
Sorting
BF
Decrease and Conquer
Department of Computer Science
Graph Representation
Adjacency matrix
n x n boolean matrix if |V| is n.
The element on the ith row and jth column is 1 if there’s an
edge from ith vertex to the jth vertex; otherwise 0.
The adjacency matrix of an undirected graph is symmetric.
It takes (comparisons) to figure out all the adjacent
nodes of a certain node i.
Adjacency linked lists
A collection of linked lists, one for each vertex, that contain all
the vertices adjacent to the list’s vertex.
An example
Department of Computer Science 114
Depth First Search
The idea
traverse “deeper” whenever possible.
On each iteration, the algorithm proceeds to an unvisited vertex that is
adjacent to the one it is currently in. If there are more than more
neighbors, break the tie by the alphabetic order of the vertices.
When reaching a dead end ( a vertex with no adjacent unvisited
vertices), the algorithm backs up one edge to the parent and tries to
continue visiting unvisited vertices from there.
The algorithm halts after backing up to the starting vertex, with the
latter being a dead end.
Similar to preorder tree traversals
It’s convenient to use a stack to trace the operation of depth-first
search.
Push a vertex onto the stack when the vertex is reached for the first
time.
Pop a vertex off the stack when it becomes a dead end.
An example
Department of Computer Science 115
Example: Undirected Graph
a b c d
e f g h
Depth-first traversal: Give the order in which the
vertices were reached.
abfegcdh
Department of Computer Science 116
Depth First Search
Pseudocode for Depth-first-search of graph G=(V,E)
DFS(G) // Use depth-first to visit a graph, which might
// contain multiple connected components.
count 0 //visiting sequence number
mark each vertex with 0 // (unvisited)
for each vertex v∈ V do
if v is marked with 0 //w has not been visited yet.
dfs(v)
dfs(v)
//Use depth-first to visit a connected component starting
//from vertex v.
count count + 1
mark v with count //visit vertex v
for each vertex w adjacent to v do
if w is marked with 0 //w has not been visited yet
dfs(w)
Department of Computer Science 117
Example: Directed Graph
a b c d
e f g h
Depth-first traversal: Give the order in which the
vertices were reached.
abfghecd
Department of Computer Science 119
Depth First Search: Notes
DFS can be implemented with graphs
represented as:
Adjacency matrices: Θ(|V |2)
Adjacency linked lists: Θ(|V |+|E| )
Applications:
Checking connectivity
Finding connected components
Department of Computer Science 120
Breadth First Search
The idea
Traverse “wider” whenever possible.
Discover all vertices at distance k from s (on level k)
before discovering any vertices at distance k +1 (at
level k+1)
Similar to level-by-level tree traversals
Instead of a stack, breadth-first uses a queue.
Department of Computer Science 121
Example: Undirected Graph
a b c d
e f g h
Breadth-first traversal:
abefgchd
Department of Computer Science 122
Breadth First Search Algorithm
BFS(G)
count 0
bfs(v)
mark each vertex with 0
for each vertex v∈ V do count count + 1
if v is marked with 0 mark v with count //visit v
bfs(v) initialize queue with v //enqueue
while queue is not empty do
a front of queue //dequeue
for each vertex w adjacent to a do
if w is marked with 0 //w hasn’t been visited.
count count + 1
mark w with count //visit w
add w to the end of the queue //enqueue
remove a from the front of the queue
Department of Computer Science 123
Example: Directed Graph
a b c d
e f g h
Breadth-first traversal:
abefghcd
Department of Computer Science 124
Directed Acyclic Graph
A directed graph with no (directed) cycles is
called directed acyclic graph(DAG)
a b a b
not a DAG
c d c d
DAG
Department of Computer Science 125
Example-4:
Brute Topological Sorting
Force Example
Topological sorting for Directed Acyclic Graph
(DAG) is a linear ordering of vertices such that
for every directed edge u-v, vertex u comes
before v in the ordering.
Note: Topological Sorting for a graph is not
possible if the graph is not a DAG.
Design an algorithm that find topological order
in a DAG.
Department of Computer Science 126
Example-4:
Brute Topological Sorting
Force Example
Design an algorithm that find topological order
in a DAG.
Solution:
What is input
A 2D Array representing the given graph.
What should be the output
Linear ordering of vertices of DAG
Which designing technique?
Decrease and conquer
Logic/Idea
Department of Computer Science 127
DFS based Algorithm
DFS-based algorithm for topological sorting
Perform DFS traversal, noting the order vertices are
popped off the traversal stack
Reverse order solves topological sorting problem
Example:
Set of 5 courses: { C1, C2, C3, C4, C5 }
C1 and C2 have no prerequisites
C3 requires C1 and C2 C1 C4
C4 requires C3 C3
C5 requires C3 and C4
C2 C5
Department of Computer Science
DFS based Algorithm: Example
Set of 5 courses: { C1, C2, C3, C4, C5 }
C1 and C2 have no prerequisites
C3 requires C1 and C2
C4 requires C3 C 1 , C 2 , C 3 , C 4 , C5 !
C5 requires C3 and C4
Department of Computer Science
DFS based Algorithm: Your Turn
DFS-based algorithm for topological sorting
Perform DFS traversal, noting the order vertices are
popped off the traversal stack
Reverse order solves topological sorting problem
Example:
a b c d
e f g h
Department of Computer Science
Source Removal Algorithm
Source removal algorithm
Repeatedly identify and remove a source (a vertex with
no incoming edges) and all the edges incident to it until
either no vertex is left (problem is solved) or there is no
source among remaining vertices (not a dag)
Example:
Department of Computer Science
Source Removal Algorithm: Your Turn
Source removal algorithm
Repeatedly identify and remove a source (a vertex with
no incoming edges) and all the edges incident to it until
either no vertex is left (problem is solved) or there is no
source among remaining vertices (not a dag)
Example:
a b c d
e f g h
Department of Computer Science
Application of Topological Sorting
A large project – e.g., in construction, research,
or software development – that involves a
multitude of interrelated tasks with known
prerequisites
Schedule to minimize the total completion time
Instruction scheduling in program compilation,
resolving symbol dependencies in linkers, etc.
Department of Computer Science
Application of Topological Sorting
Other Examples:
Finding cycle in a graph.
Operation System deadlock detection.
Dependency resolution.
Sentence Ordering.
Critical Path Analysis.
Course Schedule problem.
Department of Computer Science
Decrease by a
constant factor
Department of Computer Science
Variants of Decrease and Conquer
Decrease by a constant factor: the size of the problem is
reduced by the same constant factor on each
iteration/recursion of the algorithm.
Department of Computer Science 136
Example-1: Computing
Brute Force Example a n
Design an algorithm that computes an using
decrease by constant factor
Decrease by a constant factor
Solution:
What is input
Two numbers say a and n.
What should be the output
Return the value of an
Which designing technique?
Decrease(by a constant factor and conquer
Logic/Idea Decrease by a constant factor
Divide the power in half
at each step
Department of Computer Science 137
Example-1: Computing
Brute Force Example a n
Design an algorithm that computes an using
decrease by constant factor
Decrease by a constant factor
Solution:
Effectively, when
power is not
Suppose we want to compute 95
divisible by 2, we
make power even
by taking out the
extra 9. Then we
already know the
solution when
power is divisible by
2. Divide the power
Department of Computer Science 138
by 2 and multiply
Example-1: Computing
Brute Force Example a n
Design an algorithm that computes an using
decrease by constant factor
Decrease by a constant factor
Solution:
Now our problem is to find (812) * 9
Department of Computer Science 139
Example-1: Computing
Brute Force Example a n
Design an algorithm that computes an using
decrease by constant factor
Decrease by a constant factor
Solution:
Department of Computer Science 140
Example-1: Computing
Brute Force Example a n
Design an algorithm that computes an using
decrease by constant factor
Decrease by a constant factor
Solution:
Department of Computer Science 141
Example 2: Searching a Key
a problem of size n
subproblem
of size n/2
a solution to the
subproblem
a solution to
the original problem 142
Department of Computer Science
Example-2:
Brute Searching a Key
Force Example
Design an algorithm that find a value in a given
sorted array using decrease by constant
factor
Solution:
What is input
An array A of n numbers and a key.
What should be the output
Return location of key if found otherwise return false
Which designing technique?
Decrease(by a factor) and conquer
Logic/Idea
repeatedly dividing the search interval in half.
Department of Computer Science 143
Example-2:
Brute Searching a Key
Force Example
nBegin with the mid element of the whole array as a search key.
nIf the value of the search key is equal to the item then return an index
of the search key.
nOr if the value of the search key is less than the item in the middle of
the interval, narrow the interval to the lower half.
nOtherwise, narrow it to the upper half.
nRepeatedly check from the second point until the value is found or the
interval is empty.
If ( value == middle element )
value is found
else if ( value < middle element )
search left-half of list with the same method
else
search right-half of list with the same method
Department of Computer Science 144
Searching an Array: Decrease by constant factor(half) Technique
Binary search is like looking up a phone number
or a word in the dictionary
Start in middle of book
If name you're looking for comes before names on
page, look in first half
Otherwise, look in second half
Department of Computer Science
Binary search: Iterative Algorithm
Algorithm isPresent(A[0…N], val)
{
low = 0;
high = N;
mid;
while ( low <= high ){
mid = ( low + high )/2;
if (A[mid]== val)
return 1; // found!
else if (A[mid] < val)
low = mid + 1;
else
high = mid - 1;
}
return 0; // not found
}
Department of Computer Science 146
Binary search
Case 1: val == a[mid]
val = 10
low = 0, high = 8
mid = (0 + 8) / 2 = 4
a: 1 5 7 9 10 13 17 19 27
0 1 2 3 4 5 6 7 8
low mid high
Department of Computer Science 147
Binary Search
Case 2: val > a[mid]
val = 19
low = 0, high = 8
mid = (0 + 8) / 2 = 4
new low = mid+1 = 5
a: 1 5 7 9 10 13 17 19 27
0 1 2 3 4 5 6 7 8
low new high
mid
low
Department of Computer Science 148
Example 2: Binary Search
Case 3: val < a[mid]
val = 7
low = 0, high = 8
mid = (0 + 8) / 2 = 4
new high = mid-1 = 3
a: 1 5 7 9 10 13 17 19 27
0 1 2 3 4 5 6 7 8
low new mid high
high
Department of Computer Science 149
Binary search: Example 3(contd.)
val = 7
a: 1 5 7 9 10 13 17 19 27
0 1 2 3 4 5 6 7 8
a: 1 5 7 9 10 13 17 19 27
0 1 2 3 4 5 6 7 8
a: 1 5 7 9 10 13 17 19 27
0 1 2 3 4 5 6 7 8
Department of Computer Science 150
Binary Search: Recursive Algorithm
For an ordered array A, finds if x is in the array A[lo…hi]
Algorithm BINARY-SEARCH (A, lo, hi, x)
if (lo > hi)
1 2 3 4 5 6 7 8
return FALSE
mid (lo+hi)/2 2 3 5 7 9 10 11 12
if x = A[mid]
return TRUE mid
lo hi
if ( x < A[mid] )
BINARY-SEARCH (A, lo, mid-1, x)
if ( x > A[mid] )
BINARY-SEARCH (A, mid+1, hi, x)
Department of Computer Science 151
Decrease by
variable size
Department of Computer Science
Decrease by variable size
Variable-size decrease: Problem size decreases at each
iteration in variable amount
Euclid’s algorithm(used in cryptography, e.g. RSA Algorithm)
gcd(m, n) = gcd(n, m mod n)
gcd( 31415, 14142 )= gcd( 14142, 3131 ) Decr. By 11011
= gcd( 3131, 1618 ) Decr. By 1513
= gcd( 1618, 1513 ) Decr. By 105
= gcd( 1513, 105 ) Decr. By 1408
= gcd( 105, 43 ) Decr. By 62
= gcd( 43, 19 ) Decr. By 24
= gcd( 19, 5 ) Decr. By 14
= gcd( 5, 4 ) Decr. By 1
= gcd( 4, 1 ) Decr. By 3
= gcd( 1, 0 ) Decr. By 1
Department of Computer Science
Pseudo code for ()Euclid’s)Algorithm, gcd(m,n)
gcd(m, n) = gcd(n, m mod n)
gcd(24, 60) = gcd(60, 24 mod 60)
What happens if m < n ?
Department of Computer Science 11/2/2024 154
How we do it?
CONCLUSION
Department of Computer Science
Decrease and Conquer
Exploit the relationship between a solution to a
given instance of a problem and a solution to
its smaller instance
Top-down: recursive
Bottom-up: iterative
3 major types:
Decrease by a constant
Decrease by a constant factor
Variable size decrease
Department of Computer Science
Variants of Decrease and Conquer
Decrease by a constant: the size of the problem is reduced by the
same constant on each iteration/recursion of the algorithm.
Insertion sort
Graph Traversing Algorithms:
DFS
BFS
Topological Sorting
Algorithms for generating permutations, subsets
Decrease by a constant factor: the size of the problem is reduced by
the same constant factor on each iteration/recursion of the algorithm.
Binary search
Fake-coin problem
Bisection Method
Exponentiation by Squaring
Multiplication à la russe
Variable-size decrease: the size reduction pattern varies from one
iteration of an algorithm to another.
Euclid’s algorithm(used in cryptography, e.g. RSA Algorithm)
Selection by partition
Nim-like games
Department of Computer Science 157
