CS/IS F211 Data Structures & Algorithms

2/8/2017
Sundar B.
REVIEW: TOP DOWN DESIGN

CSIS, BITS, Pilani

Space Complexity:
Function Call Overhead: Stack Space, Recursion
1
Recursion vs. Iteration:
Divide-and-Conquer Designs
 2/8/2017
Sundar B.
FUNCTION CALL OVERHEAD
- STACK SPACE AND RECURSION

CSIS, BITS, Pilani

2
SPACE COMPLEXITY OF SORTING ALGORITHMS
Insertion Sorting Merge Sorting

insertSort(A, n) { mergeSort(A, st, en) {

if (n>1) { if (en-st < 1) return;
insertSort(A,n-1); mid=floor((st+en)/2);
insertInOrder(A[n-1], A, n-1); mergeSort(A, st, mid);
} mergeSort(A, mid+1,en);
} mergeIn(A, st, mid, en);
// Time Complexity: O(N*N) }
// Space Complexity: O(1) // Time Complexity: O(N*N)
// Space Complexity: O(N)
3
Incorrect?
SPACE COMPLEXITY – COUNTING CALL FRAMES
Insertion Sorting Merge Sorting

insertSort(A, n) { mergeSort(A, st, en) {

if (n>1) { if (en-st < 1) return;
insertSort(A,n-1); mid=floor((st+en)/2);
insertInOrder(A[n-1], A, n-1); mergeSort(A, st, mid);
} mergeSort(A, mid+1,en);
} mergeIn(A, st, mid, en);
// Space Complexity: }
/* How many function calls (i.e. // Space Complexity:
stack frames) in sequence?
/* How many function calls (i.e.
*/
stack frames) in sequence? */
4
COUNTING CALL FRAMES – NON-RECURSIVE CALLS
Insertion Sorting Merge Sorting

insertSort(A, n) { mergeSort(A, st, en) {

if (n>1) { if (en-st < 1) return;
insertSort(A,n-1); mid=floor((st+en)/2);
insertInOrder(A[n-1], A, n-1); mergeSort(A, st, mid);
} mergeSort(A, mid+1,en);
} mergeIn(A, st, mid, en);
Call frames are allocated and
deallocated (before next call) }
i.e. maximum additional
space is O(1)
5
COUNTING CALL FRAMES – RECURSIVE CALLS
Insertion Sorting Merge Sorting

mergeSort(A, st, en) {

insertSort(A, n) { if (en-st < 1) return;
if (n>1) { mid=floor((st+en)/2);
mergeSort(A, st, mid);
insertSort(A,n-1);
mergeSort(A, mid+1,en);
insertInOrder(A[n-1], A, n-1);
mergeIn(A, st, mid, en);
}
}
} /* How many recursive calls in
/* How many recursive calls in sequence?
sequence? i.e. what is the depth of recursion (in
i.e. what is the depth of recursion divide-and-conquer) ?*/
(in divide-and-conquer) ?*/
COUNTING CALL FRAMES – RECURSIVE CALLS
Insertion Sorting Merge Sorting

insertSort(A, n) { mergeSort(A, st, en) {

if (n>1) { if (en-st < 1) return;
insertSort(A,n-1); mid=floor((st+en)/2);
insertInOrder(A[n-1], A, n-1); mergeSort(A, st, mid);
} mergeSort(A, mid+1,en);
} mergeIn(A, st, mid, en);
// depth of recursion: N }
// Space Complexity: O(N) // depth of recursion: log2N
// Space Complexity:
// N + logN which is O(N)
SPACE COMPLEXITY AND RECURSION
 Typically, stack space (i.e. number of call frames)
 is not dependent on input size except for recursive procedures.

2/8/2017
 e.g.
 for i = 1 to N { f(N); }

Sundar B.
 where N is input size, and f does not call other procedures

 uses O(1) space;

 but time overhead (due to function calls) is O(N) – Why?

 e.g.

CSIS, BITS, Pilani

 for i = 1 to N { f(N); }
 where N is input size, and neither f nor its descendants in the call

sequence are recursive.

 uses O(k) space
 where k is the length of the call sequence starting at f.

8
 2/8/2017
Sundar B.
DIVIDE-AND-CONQUER
- RECURSIVE VS. ITERATIVE ALGORITHMS

CSIS, BITS, Pilani

9
DIVIDE-AND-CONQUER: SPACE COMPLEXITY

2/8/2017
 Divide-and-Conquer designs translate naturally to
recursive algorithms:

Sundar B.
 but recursive algorithms incur stack space overhead.
 Solutions?
 Can we translate Divide-and-Conquer designs to iterative
algorithms?

CSIS, BITS, Pilani

 Can we rewrite recursive algorithms using iteration?

10
RECURSIVE VS. ITERATIVE ALGORITHMS – INSERTION SORTING
 Problem: Sort, in-place, a list of N elements.
 Assume list is stored as an array A[0], A[1], … A[n-1]

2/8/2017
 Design : Divide-And-Conquer
 Sub-problem: Sort a list of size N-1 (A[0], A[1],…A[n-2])

Sundar B.
 Combination: Insert A[n-1] in order (i.e. in the right position)
 Termination: when size is <= 1.

insertInOrd(A[n-2],A,n-2)

CSIS, BITS, Pilani

Sort(A, n)
Sort(A,n-1) Sort(A,n-2) … Sort (A,1)

insertInOrd(A[n-1],A,n-1) insertInOrd(A[1],A,1)

insertSort(A, n) {
11
if (n>1) { insertSort(A,n-1); insertInOrder(A[n-1], A, n-1); }
}
RECURSIVE VS. ITERATIVE ALGORITHMS – INSERTION SORTING
insertInOrd(A[n-2],A,n-2)

Sort(A, n)
…

2/8/2017
Sort(A,n-1) Sort(A,n-2) Sort (A,1)

insertInOrd(A[n-1],A,n-1)

Sundar B.
insertInOrd(A[1],A,1)

 Iterative Algorithm
Can be inlined.
insertionSort(A,n) {

CSIS, BITS, Pilani

for j := 1 to n-1 insertInOrder(A[j],A,j) ; Exercise:
} Find out option in gcc.
// Pre-condition: (length(A) > last) & forall j: 0<=j<last-1 --> A[j]<=A[j+1]
insertInOrder(v,A,last) {
j := last;
while (j>0 && v<A[j]) { A[j] := A[j-1]; j--; }
A[j] := v;
} 12
// Post-condition: forall j: 0<=j<last --> A[j]<=A[j+1]
DIVIDE-AND-CONQUER DESIGN – MERGE SORTING

Sort : A[0] to A[N-1]

M
Sort : A[0] to A[N/2] Sort : A[N/2+1] to A[N-1]

Sort : A[0] to A[N/4 ] Sort A[N/4  +1] to A[N/2]

…
 MERGE SORTING - DESIGN Merge 1 pair of lists
S [N] of size N/2 each

S[N/2] M S [N/2]

S[N/4] S[N/4] S[N/4] S[N/4]

M M
Merge 2 pairs of lists of size N/4 each

S[1] S[1] S[1] S[1] ... S[1] S[1] S[1] S[1]

M M M M
14
Merge N/2 pairs of lists of size 1 each
 MERGESORT – ITERATIVE ALGORITHM VER.0.5
S [N]

S[N/2] M slSz=N/2 S [N/2]

S[N/4] S[N/4] S[N/4] S[N/4]

M M
slSz=N/4

for (slSz=1; slSz <= maxSlSz; slSz*=2) {

merge N/(2*slSz) pairs of lists of size slSz each
}

S[1] S[1] S[1] S[1] ... S[1] S[1] S[1] S[1]

M M M M
slSz=1 15
MERGE SORT – ITERATIVE ALGORITHM VER.0.7
for (slSz=1; slSz<=maxSlSz; slSz*=2) {
//Merging N/(2*slSz) pairs of lists of size slSz each

2/8/2017
for (st1=0; st1<=last; st1+=2*slSz) {

Sundar B.
st2=st1+slSz; en2=st2+slSz-1;
//merging ls[st1..st2-1] with ls[st2..en2]
// and putting the result back in ls[st1..en2]

CSIS, BITS, Pilani

mergeIn(ls, st1, st2, en2);
}
} // Limitation:
// Assumes (sub)lists are of equal size when merging
// i.e. this works only on lists of size=2k for some k>0 16
MERGE SORT – ITERATIVE ALGORITHM VER.0.8
for (slSz=1; slSz<=maxSlSz; slSz*=2) {
for (st1=0; st1<=last; st1=st1+2*slSz) {

2/8/2017
st2=st1+slSz; en2=st2+slSz-1;
if (st2 > last) { // Handle uneven-sized sublists

Sundar B.
en2 = last;
if (st2-1 > last) { st2=last+1; }
}
mergeIn(ls, st1, st2, en2);

CSIS, BITS, Pilani

}
}
// maxSlSize has to be computed such that
// if 2^(k-1) < n <= 2^k, then maxSlSz = 2^(k-1)
17
MERGE SORT – ITERATIVE ALGORITHM VER.0.9
for(maxSlSz=1;n>maxSlSz;maxSlSz*=2) ;
/* Postcondition: maxSlSz/2 < n <= maxSlSz */
maxSlSz /= 2;

2/8/2017
/* Postcondition: maxSlSz < n <= 2*maxSlSz */
for (slSz=1; slSz<=maxSlSz; slSz*=2) {

Sundar B.
for (st1=0; st1<=last; st1=st1+2*slSz) {
st2=st1+slSz; en2=st2+slSz-1;
if (st2 > last) { // Handle uneven-sized sublists
en2 = last;

CSIS, BITS, Pilani

if (st2-1 > last) { st2=last+1; }
}
mergeIn(ls, st1, st2, en2);
}
}
/* mergeIn uses a temp. array to merge and copy back to ls */ 18
MERGE SORT – ITERATIVE ALGORITHM VER.1.0

2/8/2017
 Algorithm same as ver. 0.9 but, mergeIn uses two arrays, ls1
and ls2:

Sundar B.
 in odd iterations of outer loop:
 ls1 is an array of sorted sublists:

 merged into temporary array ls2

 in even iterations of outer loop:

CSIS, BITS, Pilani

 ls2 is an array or sorted sublists

 merged into temporary array ls1

 Exercise:
 Write mergeIn to meet this requirement.
 Modify mergeSort suitably. 19