Divide and Conquer

Divide and conquer is a design strategy which is well known to breaking down
efficiency barriers. When the method applies, it often leads to a large improvement
in time complexity. For example, from O (n^2 ) to O (n log n) to sort the elements.

Divide and conquer strategy is as follows: divide the problem instance into two or
more smaller instances of the same problem, solve the smaller instances
recursively, and assemble the solutions to form a solution of the original instance.
The recursion stops when an instance is reached which is too small to divide.
When dividing the instance, one can either use whatever division comes most
easily to hand or invest time in making the division carefully so that the assembly
is simplified.

Divide and conquer algorithm consists of two parts:

Divide : Divide the problem into a number of sub problems. The sub problems are
solved recursively.

Conquer : The solution to the original problem is then formed from the solutions
to the sub problems (patching together the answers). Traditionally, routines in
which the text contains at least two recursive calls are called divide and conquer
algorithms, while routines whose text contains only one recursive call are not.
Divide–and–conquer is a very powerful use of recursion

Maximum sub array problem

In the “Maximum Subarray Sum using Divide and Conquer” problem we have
given an array of both positive and negative integers. Write a program that will
find the largest sum of the contiguous subarray

The problem of maximum subarray sum is basically finding the part of an array
whose elements has the largest sum. If all the elements in an array are positive then
it is easy, find the sum of all the elements of the array and it has the largest sum
over any other subarrays you can make out from that array.

But the problem gets more interesting when some of the elements are negative then
the subarray whose sum of the elements is largest over the sum of the elements of
any other subarrays of that element can lie anywhere in the array.
The Brute Force technique to solve the problem is simple. Just iterate through every element of
the array and check the sum of all the subarrays that can be made starting from that element i.e.,
check all the subarrays and this can be done in nC2 ways i.e., choosing two different elements of
the array to make a subarray.

Thus, the brute force technique is of Θ(n2) time. However, we can solve this in Θ(nlog(n)) time
using divide and conquer.

As we know that the divide and conquer solves a problem by breaking into subproblems, so let's
first break an array into two parts. Now, the subarray with maximum sum can either lie entirely
in the left subarray or entirely in the right subarray or in a subarray consisting both i.e., crossing
the middle element.

The first two cases where the subarray is entirely on right or on the left are actually the smaller
instances of the original problem. So, we can solve them recursively by calling the function to
calculate the maximum sum subarray on both the parts

Following is the Divide and Conquer algorithm.

1. Divide the given array in two halves
2. Return the maximum of following three
• Maximum subarray sum in left half (Make a recursive call)
• Maximum subarray sum in right half (Make a recursive call)
• Maximum subarray sum such that the subarray crosses the midpoint
The idea is simple, find the maximum sum starting from mid point and ending at some point
on left of mid, then find the maximum sum starting from mid + 1 and ending with some point
on right of mid + 1. Finally, combine the two and return the maximum among left, right and
combination of both

Algorithm:

First is divide step,

maxSubarray(array)
 if start = end
return array[start]
else
middle = (start + end) / 2
return max(maxSubarray(array(From start to middle)), maxSubarray(array(From
middle + 1 to end)), maxCrossover(array))

In second part, separate the different part that are created in first part.

maxCrossover(array)
currentLeftSum = 0
leftSum = 0
currentRightSum = 0
rightSum = 0
for i in array
currentLeftSum += array[i]
if currentLeftSum > leftSum
leftSum = currentLeftSum
for i in array
currentRightSum += array[i]
if currentRightSum > rightSum
rightSum = currentRightSum
return leftSum + rightSum

Time Complexity: maxSubArraySum() is a recursive method and time complexity can be

expressed as following recurrence relation.
T(n) = 2T(n/2) + Θ(n)
The above recurrence is similar to Merge Sort and can be solved either using Recurrence Tree
method or Master method. It falls in case II of Master Method and solution of the recurrence
is Θ(nLogn)

Quick Sort-
Quick Sort is a famous sorting algorithm.
• It sorts the given data items in ascending order.
• It uses the idea of divide and conquers approach.
• It follows a recursive algorithm.
• Quick Sort follows a recursive algorithm.
• It divides the given array into two sections using a partitioning element called as pivot
• All the elements to the left side of pivot are smaller than pivot.
• All the elements to the right side of pivot are greater than pivot

Step-01:
Initially-

• Left and Loc (pivot) points to the first element of the array.
• Right points to the last element of the array
• So to begin with, we set loc = 0, left = 0 and right = 5

Step-02:
Since loc points at left, so algorithm starts from right and move towards left.
As a[loc] < a[right], so algorithm moves right one position towards left as
Step-03:
Since loc points at left, so algorithm starts from right and move towards left.
As a[loc] > a[right], so algorithm swaps a[loc] and a[right] and loc points at right

Step-04:
Since loc points at right, so algorithm starts from left and move towards right.
As a[loc] > a[left], so algorithm moves left one position towards right

Step-05:
Since loc points at right, so algorithm starts from left and move towards right.
As a[loc] > a[left], so algorithm moves left one position towards right
Step-06:
Since loc points at right, so algorithm starts from left and move towards right.
As a[loc] < a[left], so we algorithm swaps a[loc] and a[left] and loc points at left

Step-07:
Since loc points at left, so algorithm starts from right and move towards left.
As a[loc] < a[right], so algorithm moves right one position towards left

Step-08:
Since loc points at left, so algorithm starts from right and move towards left.
 As a[loc] > a[right], so algorithm swaps a[loc] and a[right] and loc points at right

Step-09:
Since loc points at right, so algorithm starts from left and move towards right.
As a[loc] > a[left], so algorithm moves left one position towards right

Now,
• loc, left and right points at the same element.
• This indicates the termination of procedure.
• The pivot element 25 is placed in its final position.
• All elements to the right side of element 25 are greater than it.
• All elements to the left side of element 25 are smaller than it.
 Now, quick sort algorithm is applied on the left and right sub arrays separately in the
similar manner
Worst Case-
• Quick Sort is sensitive to the order of input data.
• It gives the worst performance when elements are already in the ascending order.
• It then divides the array into sections of 1 and (n-1) elements in each call.
• Then, there are (n-1) divisions in all.
• Therefore, here total comparisons required are f(n) = n x (n-1) = O(n2).
Advantages of Quick Sort-
The advantages of quick sort algorithm are-
• Quick Sort is an in-place sort, so it requires no temporary memory.
• Quick Sort is typically faster than other algorithms.
(because its inner loop can be efficiently implemented on most architectures)
Disadvantages of Quick Sort-
The worst case complexity of quick sort is O(n2).
• This complexity is worse than O(nlogn) worst case complexity of algorithms like
merge sort, heap sort etc.
• It is not a stable sort i.e. the order of equal elements may not be preserved.
Time Complexity of Quick sort

• Best case scenario: The best case scenario occurs when the partitions are
as evenly balanced as possible, i.e their sizes on either side of the pivot
element are either are equal or are have size difference of 1 of each other.
o Case 1: The case when sizes of sublist on either side of pivot becomes
equal occurs when the subarray has an odd number of elements and
the pivot is right in the middle after partitioning. Each partition will
have (n-1)/2 elements.
o Case 2: The size difference of 1 between the two sublists on either side
of pivot happens if the subarray has an even number, n, of elements.
One partition will have n/2 elements with the other having (n/2)-1.
In either of these cases, each partition will have at most n/2 elements,
The best-case complexity of the quick sort algorithm is O(n logn)
 Worst case scenario: This happens when we encounter the most
unbalanced partitions possible, then the original call takes n time, the
recursive call on n-1 elements will take (n-1) time, the recursive call on (n-
2) elements will take (n-2) time, and so on. The worst case time complexity
of Quick Sort would be O(n^2)

Space Complexity of Quick sort

The space complexity is calculated based on the space used in the recursion stack.
The worst case space used will be O(n) . The average case space used will be of
the order O(log n). The worst case space complexity becomes O(n), when the
algorithm encounters its worst case where for getting a sorted list, we need to
make n recursive calls.

Algorithm

void quicksort(int number[25],int first,int last){

int i, j, pivot, temp;
if(first<last){
pivot=first;
i=first;
j=last;
while(i<j){
while(number[i]<=number[pivot]&&i<last)
i++;
while(number[j]>number[pivot])
j--;
if(i<j){
temp=number[i];
number[i]=number[j];
number[j]=temp;
}
}
temp=number[pivot];
number[pivot]=number[j];
number[j]=temp;
quicksort(number,first,j-1);
 quicksort(number,j+1,last);
}
}
int main(){
int i, count, number[25];
printf("How many elements are u going to enter?: ");
scanf("%d",&count);
printf("Enter %d elements: ", count);
for(i=0;i<count;i++)
scanf("%d",&number[i]);
quicksort(number,0,count-1);
printf("Order of Sorted elements: ");
for(i=0;i<count;i++)
printf(" %d",number[i]);
return 0; }
Strassen's Matrix multiplication and its recurrence relation
So the main idea is to use the divide and conquer technique in this algorithm – divide matrix A
& matrix B into 8 submatrices and then recursively compute the submatrices of C
Follow this link for details
https://www.codesdope.com/blog/article/strassens-matrix-multiplication/

Here we save one recursive call, but have several new additions of n/2 x n/2 matrices.
M1=(A11+A22)(B11+B22)M1=(A11+A22)(B11+B22)
M2=(A21+A22)B11M2=(A21+A22)B11
M3=A11(B12−B22)M3=A11(B12−B22)
M4=A22(B21−B−11)M4=A22(B21−B−11)
M5=(A11+A12)B22M5=(A11+A12)B22
M6=(A21−A11)(B11+B12)M6=(A21−A11)(B11+B12)
M7=(A12−A22)(B21+B22)M7=(A12−A22)(B21+B22)
C11=M1+M4−M5+M7C11=M1+M4−M5+M7
C12=M3+M5C12=M3+M5
C21=M2+M4C21=M2+M4
C22=M1−M2+M3+M6C22=M1−M2+M3+M6

Algorithm:
Algorithm Strassen(n, a, b, d)
begin
If n = threshold then compute
C = a * b is a conventional matrix.
Else
Partition a into four sub matrices a11, a12, a21, a22.
Partition b into four sub matrices b11, b12, b21, b22.
Strassen ( n/2, a11 + a22, b11 + b22, d1)
Strassen ( n/2, a21 + a22, b11, d2)
Strassen ( n/2, a11, b12 – b22, d3)
Strassen ( n/2, a22, b21 – b11, d4)
Strassen ( n/2, a11 + a12, b22, d5)
Strassen (n/2, a21 – a11, b11 + b22, d6)
Strassen (n/2, a12 – a22, b21 + b22, d7)

C = d1+d4-d5+d7 d3+d5
d2+d4 d1+d3-d2-d6

end if

return (C)
end.

Time Complexity Analysis of Merge Sort

First step was to divide the input into two halves which comprised us of a logarithmic time
complexity ie. log(N) where N is the number of elements.
our second step was to merge back the array into a single array, so if we observe it in all the
number of elements to be merged N, and to merge back we use a simple loop which runs over all
the N elements giving a time complexity of O(N).
finally, total time complexity will be - step -1 + step-2

For the best case, one can assume that the array is already sorted so in that case the number of
comparisons would be minimum.
So, the first element of the 2nd array is compared with each element of the first array and then,
the 2nd array is attached at the end of 1st array.
At every step, only N/2 elements are compared and there are O(logN) such steps
INPUT - [1,2,3,4,5]
STEP -1 divide it into equal parts
[1,2,3] and [4,5]
We can see that it can be further divided into smaller parts
[1,2] [3] [4,5]
Intially, let us assume that the count of swaps is 0, now if we check for the condition where first
< second and if not than we increase the count swap by 1.
[1,2] == first < second count=0;
[3] == only one element so no comparisons count=0;
[4,5] == first < second count=0;
// Now we have
[1,2] [3] [4,5]
Now lets merge back all the parts into one sorted array
[1,2,3,4,5]--> one can see that there is no swaps required in merging as well.
Therefore in Best Case,
• Input is already sorted
• Best Case Time Complexity: O(N logN)
• Number of Comparisons: 0.5 N logN
Average case
Number of comparisons decide the complexity to be best , average or worst.
INPUT - [1,3,4,11,7,9,5]
// we can clearly see that one part is going to have more elements so if we split it into two
possible way
[1,3,4,11] and [7,9,5]
we can see that it can be further divided into smaller parts
[1,3] [4,11] [7,9] [5]
intially lets assume that the count of swaps is 0, now if we check for the condition where first <
second and if not than we increase the count swap by 1.
[1,3] == first < second count=0;
[4,11] == first < second =0;
[7,9] == first < second count=0;
[5] == count=0;
// Now we have
[1,3] [11,4] [7,9] [5]
Now lets merge back all the parts into one sorted array
[1,3,4,11] [5,7,9] ----> here we will be comparing 5 with 1 3 4 11
-----> and then insert 5 after 4 and then same with 7 and 9.
[0,1,2,3,4,5,6,7]
Average case takes 0.26N less comparisons than worst case.
Therefore, in Average Case:
• Average Case Time Complexity: O(N logN)
• Number of Comparisons: 0.74 N logN

Worst case:
INPUT - [4,0,6,2,5,1,7,3]
STEP -1 divide it into equal parts
[4,0,6,2] and [5,1,7,3]
we can see that it can be further divided into smaller parts
[4,0] [6,2] [5,1] [7,3]
intially lets assume that the count of swaps is 0, now if we check for the condition where first <
second and if not than we increase the count swap by 1.
[4,0] != first < second count=1;
[6,2] != first < second count=2;
[5,1] != first < second count=3;
[7,3] != first < second coutn=4;
// Now we have
[0,4] [2,6] [1,5] [3,7]
Now lets merge back all the parts into one sorted array
[0,4,2,6] [1,3,5,7] ----> maximum comparison again every pair of set compared.
[0,1,2,3,4,5,6,7]
Therefore, in Worst Case:
• Input: Specify distribution
• Worst Case Time Complexity: O(N logN)
• Number of Comparisons: N logN

Largest Sub-array sum

Write an efficient program to find the sum of contiguous subarray within a one-dimensional
array of numbers that has the largest sum.

Consider the array which is given below:

B: {-5, 4, 6, -3, 4, 1}

There are multiple techniques to solve this problem. First, we look at brute force to solve the
problem. In the case of brute force, first, we have to find all the sub-arrays, and then we look at
the sub-array, which has the maximum sum
The below algorithm has the time complexity of O(n2). The time complexity is high, so we need
to optimize the problem. In order to optimize the problem, we use Kaden's algorithm.

Kaden's algorithm is an iterative dynamic programming algorithm that looks for the maximum
contiguous subarray. Using the Kaden's algorithm, we should consider only the positive elements
of the array and keep track of only the maximum contiguous sum subarray
1) Brute force 2) Kaden’s algo
 2)

Master theorem
Master theorem is and how it is used for solving recurrence relations.
The master method is a formula for solving recurrence relations of the form:
T(n) = aT(n/b) + f(n),

where,

n = size of input

a = number of subproblems in the recursion

n/b = size of each subproblem. All subproblems are assumed

to have the same size.

f(n) = cost of the work done outside the recursive call,

which includes the cost of dividing the problem and

cost of merging the solutions

If a ≥ 1 and b > 1 are constants and f(n) is an asymptotically positive function, then the
time complexity of a recursive relation is given by

T(n) = aT(n/b) + f(n)

where, T(n) has the following asymptotic bounds:

1. If f(n) = O(nlog b a-ϵ), then T(n) = Θ(nlog b a).

2. If f(n) = Θ(nlog b a), then T(n) = Θ(nlog b a * log n).

3. If f(n) = Ω(nlog b a+ϵ), then T(n) = Θ(f(n)).

ϵ > 0 is a constant.

Proof of Master Theorem

 The above form of master theorem expresses that the problem is in the form of tree and
the tree is formed as show below:

Also, we all know that if a problem can be represented in the form of tree as above, it
goes to at-most to level log(n)[base b]. At this level we left with 1, that is we have
divided to the problem to the shortest problem possible.
Work done at each level is represented as:

• In a Geometric series, as each next element in the series differ by a common multiple
integer, conventionally, we take it as “r”.
• Sum of the Geometric series is formula is a * ( (1-r^n) / (1–r) ), where a is the first term.
Proof of Case 1; d < log(b) [base a]:
• The above case can also be understood as the work done is increasing in each of the
subsequent levels of the tree.
• Also, it is the case when r < 1 in a Geometric series, so, in this case, the major term will
be the last term of the geometric series which will be having the maximum effect.
• So, in the total work done formula we take to take only the last term, i.e. substitute k =
log(n) [base b] in the time complexity expression

Proof of Case 2; d= log(b) [base a]:

It means that the work done in each of the levels is equal, so we can easily obtain the
final work done by multiplying the number of levels and work done on each level.

Proof of Case 3; d > log(a) [base b]:

This means that that the work required is constantly decreasing in the subsequent levels
=> Work done in the first level is the maximum => We need to consider only the first
term.
This clearly provides O(n^d).
Hence Proved.

Closest pair problem

We are given an array of n points in the plane, and the problem is to find out the closest
pair of points in the array. This problem arises in a number of applications. For example,
in air-traffic control, you may want to monitor planes that come too close together, since
this may indicate a possible collision. Recall the following formula for distance between
two points p and q.

The Brute force solution is O(n^2), compute the distance between each pair and return
the smallest. We can calculate the smallest distance in O(nLogn) time using Divide and
Conquer strategy.
Algorithm.
Divide: draw vertical line L so that roughly ½n points on each side.
Conquer: find closest pair in each side recursively.
Combine: find closest pair with one point in each side.
Return best of 3 solutions
Steps:
1) We sort all points according to x coordinates.
2) Divide all points in two halves.
3) Recursively find the smallest distances in both subarrays.
4) Take the minimum of two smallest distances. Let the minimum be d.
5) Create an array [] that stores all points which are at most d distance away from
the middle line dividing the two sets.
6) Find the smallest distance in [].
7) Return the minimum of d and the smallest distance calculated in above step 6.
The great thing about the above approach is, if the array [] is sorted according to y
coordinate, then we can find the smallest distance in strip[] in O(n) time,
Algo:

.
Convex Hull problem

Convex Hull Problem: Given: A set of points P in the plane. Goal: Find the
smallest, convex polygon containing all points in P
This problem is equivalent to
• Finding the largest convex polygon whose vertices are points in P.
• Finding the set of all convex combinations of points in P (i.e. all points on any
line segment between any pair of points).
• Finding the intersection of all convex regions containing P
Applications
 Applications. Among oldest and most well-studied problems in computational
geometry problems.
• Robot motion planning.
• Shortest perimeter fence enclosing P.
• Smallest area polygon enclosing P.
• Unique convex polygon whose vertices are points in P that encloses P.
• Smallest convex set containing all N points (i.e., intersection of all convex sets
containing the N points).

Graham's scan Algorithm

The algorithm first finds the bottom-most point P0. If there are multiple points with the
same Y coordinate, the one with the smaller X coordinate is considered. This step
takes O(N) time.

Next, all the other points are sorted by polar angle in counterclockwise order. If the polar
angle between two points is the same, the nearest point is chosen instead.

Then we iterate through each point one by one, and make sure that the current point and
the two before it make a counterclockwise turn, otherwise the previous point is discarded,
since it would make a non-convex shape. Checking for clockwise or anticlockwise nature
can be done by checking the orientation.

We use a stack to store the points, and once we reach the original point P0, the algorithm
is done and we return the stack containing all the points of the convex hull in clockwise
order.

Time Complexity: The merging of the left and the right convex hulls take O(n) time and
as we are dividing the points into two equal parts, so the time complexity of the above
algorithm is O(n * log n).

void convex_hull(vector<pt>& a, bool include_collinear = false) {

if (a.size() == 1)
return;

sort(a.begin(), a.end(), [](pt a, pt b) {

return make_pair(a.x, a.y) < make_pair(b.x, b.y);
});
pt p1 = a[0], p2 = a.back();
vector<pt> up, down;
up.push_back(p1);
down.push_back(p1);
for (int i = 1; i < (int)a.size(); i++) {
 if (i == a.size() - 1 || cw(p1, a[i], p2, include_collinear)) {
while (up.size() >= 2 && !cw(up[up.size()-2], up[up.size()-1],
a[i], include_collinear))
up.pop_back();
up.push_back(a[i]);
}
if (i == a.size() - 1 || ccw(p1, a[i], p2, include_collinear)) {
while (down.size() >= 2 && !ccw(down[down.size()-2],
down[down.size()-1], a[i], include_collinear))
down.pop_back();
down.push_back(a[i]);
}
}

if (include_collinear && up.size() == a.size()) {

reverse(a.begin(), a.end());
return;
}
a.clear();
for (int i = 0; i < (int)up.size(); i++)
a.push_back(up[i]);
for (int i = down.size() - 2; i > 0; i--)
a.push_back(down[i]);
}