Introduction to Design and Analysis of Algorithms

DIVIDE AND CONQUER

Nguyễn Ngọc Thảo

[email protected]
ALLPPT.com _ Free PowerPoint Templates, Diagrams and Charts

ALLPPT.com _ Free PowerPoint Templates, Diagrams and Charts

This slide is adapted from the Lecture notes of the course
CSC14111 – Introduction to the Design and Analysis of
Algorithms taught by Dr. Nguyen Thanh Phuong (2023).

2
Introduction to
Divide and Conquer
Divide and Conquer
• Divide and Conquer (DAC) is probably the best known
general algorithm design technique.

4
Divide and Conquer

Step 1. Divide the problem into several subproblems of the

same type, ideally of about equal size.
Step 2. Solve the subproblems, typically recursively.
We may use a different algorithm, especially when the subproblems
become small enough.
Step 3. Combine the solutions to the subproblems to get a
solution to the original problem.

5
DAC: The general recurrence
• The most typical case is to divide a problem instance of size
𝑛 into 𝒂 𝑎 > 1 instances of size 𝒏/𝒃 𝑏 > 1 .
• For simplicity, assume that size 𝑛 is a power of 𝑏.
• The general divide-and-conquer recurrence for running time
is
𝒏
𝑻 𝒏 =𝒂𝑻 + 𝒇(𝒏)
𝒃
• 𝑓(𝑛) is a function for the time spent on dividing an instance of size 𝑛
into those of size 𝑛/𝑏 and combining their solutions.

6
Master theorem
• Given the divide-and-conquer recurrence
𝑻 𝒏 = 𝒂 𝑻 𝒏/𝒃 + 𝒇(𝒏)
• If 𝑓(𝑛) ∈ Θ 𝑛𝑑 , where 𝑑 ≥ 0, then

𝚯 𝒏𝒅 𝒂 < 𝒃𝒅
𝑻 𝒏 ∈ 𝚯 𝒏𝒅 𝐥𝐨𝐠 𝒏 𝒂 = 𝒃𝒅
𝚯 𝒏𝐥𝐨𝐠𝒃 𝒂 𝒂 > 𝒃𝒅

• Analogous results hold for the Ο and Ω notations, too.

7
 Example: Find the maximum value from an array

Finding the maximum value from an array of 𝑛 numbers (for simplicity, 𝑛 = 2𝑘 )

findMax(a, l, r) {
if (l == r) return a[l];
m = (l + r) / 2;
return max(findMax(a, l, m), findMax(a, m + 1, r));
}

𝒏 𝒏>𝟏
The divide-and-conquer recurrence is: 𝑻 𝒏 = ቐ𝟐𝑻 + 𝚯(𝟏)
𝟐
𝟎 𝒏=𝟏

Thus, 𝑻(𝒏) ∈ 𝜣 𝒏 since 𝑇(𝑛) ∈ Θ 𝑛log𝑏 𝑎 with 𝑑 = 0, 𝑎 = 2, 𝑏 = 2.

8
 Example: Find both the maximum and minimum values

Finding simultaneously the maximum and minimum values from an array

of 𝑛 numbers (for simplicity, 𝑛 = 2𝑘 )

MinMax(a[1..n]) { MinMax(a[1..n]) {
min = max = a[1]; min = max = a[1];
for (i = 2; i ≤ n; i++) { for (i = 2; i ≤ n; i++)
if (a[i] > max) if (a[i] > max)
max = a[i]; max = a[i];
if (a[i] < min) else
min = a[i]; if (a[i] < min)
} min = a[i];
return <min, max> return <min, max>
} }
9
MinMax(l, r, & min, & max) {
if (l ≥ r - 1)
if (a[l] < a[r]) {
min = a[l];
max = a[r];
}
else {
min = a[r];
max = a[l];
}
else {
…
m = (l + r) / 2;
MinMax(l, m, minL, maxL);
MinMax(m + 1, r, minR, maxR);
min = (minL < minR) ? minL : minR;
max = (maxL < maxR) ? maxR : maxL;
}
} 10
 Example: Find both the maximum and minimum values

Finding simultaneously the maximum and minimum values from an array

of 𝑛 numbers (for simplicity, 𝑛 = 2𝑘 )

The general divide-and-conquer recurrence is:

𝒏 𝒏 𝒏>𝟐
𝑻 𝒏 = ቐ𝑻 𝟐
+𝑻
𝟐
+𝟐
𝟎 𝒏≤𝟐

Thus, 𝑻(𝒏) ∈ 𝚯 𝒏 since 𝑇(𝑛) ∈ Θ 𝑛log𝑏 𝑎 with 𝑑 = 0, 𝑎 = 2, 𝑏 = 2.

11
 Example: Merge sort

This approach sorts a given array 𝑎1 , 𝑎2 , … , 𝑎𝑛 by dividing it into two halves.

𝑎1 , 𝑎2 , … , 𝑎 𝑛 and 𝑎 𝑛 +1 , 𝑎 𝑛 +2 , … , 𝑎𝑛
2 2 2

sorting each of them recursively, and then merging the two smaller sorted
arrays into a single sorted one.

12
 merge(a[1 .. n], low, mid, high) {
i = low; j = mid + 1;
k = low;
while (i  mid) && (j  high)
if (a[i]  a[j]) buf[k++] = a[i++];
else buf[k++] = a[j++];
if (i > mid) buf[k .. high] = a[j .. high];
else buf[k .. high] = a[i .. mid];
a[low .. high] = buf[low .. high];
}
mergeSort(a[1 .. n], low, high) {
if (low < high) {
mid =  (low + high) / 2;
mergeSort(a, low, mid);
mergeSort(a, mid + 1, high);
merge(a, low, mid, high);
}
}
mergeSort(a, 1, n); 13
 Example: Merge sort

Assuming that the key comparison is the basic operation.

𝒏 𝒏 𝒏 𝑛>1
• Best case: 𝑻 𝒏 = ቐ𝑻 𝟐
+𝑻
𝟐
+
𝟐 ∈ 𝜣 𝒏 𝐥𝐨𝐠 𝒏
𝟎 𝑛≤1

𝒏 𝒏
𝑻 +𝑻 +𝒏−𝟏 𝑛 > 1
• Worst case: 𝑻 𝒏 = ቐ 𝟐 𝟐 ∈ 𝜣 𝒏 𝐥𝐨𝐠 𝒏
𝟎 𝑛≤1
Assuming that the assignment statement is the basic operation.
𝒏 𝒏 𝑛>1
𝑻 𝒏 = ቐ𝑻 𝟐
+𝑻
𝟐
+𝒏 ∈ 𝜣 𝒏 𝐥𝐨𝐠 𝒏
𝟎 𝑛≤1
14
 Example: Merge sort

This approach sorts a given array 𝑎1 , 𝑎2 , … , 𝑎𝑛 by partitioning it into two

subregions following the pivot 𝑎𝑠 , such that
𝑎1 , 𝑎2 , … , 𝑎𝑠−1 ≤ 𝑎𝑠 ≤ 𝑎𝑠+1 , … , 𝑎𝑛

≤ 𝑎𝑠 ≤
𝑎1 , 𝑎2 , … , 𝑎𝑠−1 𝑎𝑠+1 , … , 𝑎𝑛

15
 Partition(a[left .. right]) {
p = a[left];
i = left; j = right + 1;
do {
do i++; while (a[i] < p);
do j--; while (a[j] > p); Does this design work?
swap(a[i], a[j]);
} while (i < j);
swap(a[i], a[j]);
swap(a[left], a[j]);
return j;
} Quicksort(a[left .. right]) {
if (left < right){
s = Partition(a[left .. right]);
Quicksort(a[left .. s – 1]);
Quicksort(a[s + 1 .. right]);
}
} 16
 Example: Quick sort

For simplicity, assume that 𝑎1 , 𝑎2 , … , 𝑎𝑛 contains no duplicate values and

the size 𝑛 = 2𝑘 .
Two comparisons in loops are the basic operations.

• Best case: 𝑩(𝒏) ∈ 𝚯(𝒏 𝐥𝐨𝐠 𝒏)

• Worst case: 𝑾(𝒏) ∈ 𝚯(𝒏𝟐 )

• Average case: 𝑨(𝒏) ≈ 𝚯(𝟏, 𝟑𝟗 𝒏 𝐥𝐨𝐠 𝒏)

17
18