SWEG 5012

ADVANCED ALGORITHMS
AND PROBLEM SOLVING

LECTURE 3 DIVIDE AND CONQUER

Beakal Gizachew

Based on slides of Shafi Goldwasser, David Luebke, George Kollios, Roger Crawfis, and Cevdet Aykanat
RECAP
• What have we seen?
• Worst-case vs. average-case analysis
• Stable Marriage Problem
• Fun problem applicable to many scenarios
• Algorithm design must take into account the goals
• Simple correctness proofs
• Insertion Sort
• A daily algorithm (sorting a deck of cards)
• Merge Sort
• Analyzing recurrence relations
• Divide-and-Conquer paradigm

2
DIVIDE AND CONQUER
• Divide the problem into a number of sub-problems
• Similar sub-problems of smaller size

• Conquer the sub-problems

• Solve the sub-problems recursively
• Sub-problem size small enough
•  solve the problems in straightforward manner

• Combine the solutions of the sub-problems

• Obtain the solution for the original problem
MULTIPLYING LARGE INTEGERS
• Given two n-bit integers a an-1 an-2 …… a0
and b, compute c=ab bn-1 bn-2 …… b0
• Naive (grade-school)
algorithm n bits
n bits
• Cost?
. . .
• Total work Θ(n2)
n bits

2n bit output

4
 MULTIPLYING LARGE INTEGERS:
DIVIDE AND CONQUER
for A0 , A1 , B0 , B1
Divide: Write a = A1 2n/2
+A0
n/2 bit integers
b = B1 2n/2 +B0
Assume n=2k w.l.o.g.

Conquer: ab = A1B1 2n + A1B0 2n/2 + B1A0 2n/2 + A0B0

= A1B1 2n + (A1B0 + B1A0) 2n/2 + A0B0

Reduces to 4 multiplications of n/2-bit integers (done recursively),

plus 3 additions and 2 shifts.
Additions and shifts cost Θ(n).

5
MULTIPLYING LARGE INTEGERS:
DIVIDE AND CONQUER

T(n) = 4 T(n/2)+cn

# subproblems Subproblem Work dividing

size and combining
recursive calls additions and shifts

❑ Case 1 of Master Theorem

❑ T(n)= Θ(n2)

No better than the grade school algorithm???

6
MULTIPLYING LARGE INTEGERS:
KARATSUBA’S ALGORITHM
Compute x= A1B1 , y= A0B0 , z= (A0+A1)(B0+B1)
Multiply (n,a,b) Let ab = x 2n +(z-y-x) 2n/2 +y

• a and b are n-bit integers

• assume n is a power of 2 for simplicity
1. If n<2 then use grade-school algorithm else
2. A1 a div 2n/2 ;B b div 2n/2
1

3. A1 a mod 2n/2 ;B b mod 2n/2

1

4. x MULTIPLY (n/2, A1, B1)

5. y MULTIPLY (n/2, A0, B0)

6.z MULTIPLY (n/2, A0+A1, B0+B1)
7. Output x 2n + (z-x-y) 2n/2 + y
7
MULTIPLYING LARGE INTEGERS:
KARATSUBA’S ALGORITHM

T(n) = 3 T(n/2) + cn

# subproblems Subproblem Work dividing

size and combining
recursive calls additions and shifts

❑ Case 1 of Master Theorem

❑ T(n)= Θ(𝑛log𝟐 𝟑=n1.58496)

Much better. But can we do even better??

8
WHY STOP HERE?
• We can obtain a sequence of asymptotically faster integer
multiplication algorithm by splitting the inputs into more pieces.
• If we split A and B into k equal parts than the corresponding
multiplication algorithm is obtained from an interpolation-based
polynomial multiplication algorithm of two degree k-1 polynomials.
• Since the product is of degree 2(k-1), we need to evaluate 2k-1
points. Thus there are 2k-1 multiplications each of size n/k and
time for splitting and adding is still O(n).
• T(n) = (2k-1)T(𝒏)+cn = Θ(𝒏𝐥𝐨𝐠𝒌 𝟐𝒌−𝟏) ≈ 𝒏𝜺 for any 𝜺 > 𝟏
𝒌

• Fastest: T(n) = Θ (n log n log log n)

• Based on the Fast Fourier Transform

9
EXPONENTIATION
• Problem: compute an, where n is in N.
• Naive algorithm: Compute a1, a2, a3, ..., an Complexity: Θ(n)
multiplications
• Divide-and-Conquer algorithm:

an/2 . an/2 if n is even

an =
an-1/2 . an-1/2 . a if n is odd

T(n) = T(n/2) + c => T(n) = Θ(log n) multiplications

10
 MATRIX MULTIPLICATION

Input: A = [aij ] ; B = [bij ]

i , j = 1, 2, ……. n
Output: C = [cij ] = AB

𝒄𝟏𝟏 𝒄𝟏𝟐 … … 𝒄𝟏𝒏 𝒂𝟏𝟏 𝒂𝟏𝟐 … … 𝒂𝟏𝒏 𝒃𝟏𝟏 𝒃𝟏𝟐 … … 𝒃𝟏𝒏

⋮ 𝒄𝟐𝟐 ⋱ ⋱ ⋮ = ⋮ 𝒂𝟐𝟐 ⋱ ⋱ ⋮ . ⋮ 𝒃𝟐𝟐 ⋱ ⋱ ⋮
𝒄𝒏𝟏 𝒄𝒏𝟐 … ⋯ 𝒄𝒏𝒏 𝒂𝒏𝟏 𝒂𝒏𝟐 … ⋯ 𝒂𝒏𝒏 𝒃𝒏𝟏 𝒃𝒏𝟐 … ⋯ 𝒃𝒏𝒏

𝐜𝐢𝐣 = ෍ aik . bkj

𝟏≤𝒌≤𝒏
11
STANDART ALGORITHM

for i ← 2 to length[A]
do for j ← 1 to n
do cij ← 0
for k ← 1 to n
do cij ← cij + aik . bkj

Running time = Θ(n3)

unit multiplications and additions

Non-square matrices?
12
DIVIDE AND CONQUER
ALGORITHM
• Idea: n x n matrix = 2 x 2 matrix of (n/2) x (n/2) submatrices

𝒄𝟏𝟏 𝒄𝟏𝟐 𝒂 𝒂𝟏𝟐

= 𝟏𝟏 . 𝒃𝟏𝟏 𝒃𝟏𝟐
𝒄𝟐𝟏 𝒄𝟐𝟐 𝒂𝟐𝟏 𝒂𝟐𝟐 𝒃𝟐𝟏 𝒃𝟐𝟐

C = A ∙ B

𝒄𝟏𝟏 = 𝒂𝟏𝟏 𝒃𝟏𝟏+ 𝒂𝟏𝟐 𝒃𝟐𝟏

𝒄𝟏𝟐 = 𝒂𝟏𝟏 𝒃𝟏𝟐 + 𝒂𝟏𝟐 𝒃𝟐𝟐 8 mults of (n/2) x (n/2) submatrices
𝒄𝟐𝟏 = 𝒂𝟐𝟏 𝒃𝟏𝟏 + 𝒂𝟐𝟐 𝒃𝟐𝟏 4 adds of (n/2) x (n/2) submatrices
𝒄𝟐𝟐 = 𝒂𝟐𝟏 𝒃𝟐𝟏 + 𝒂𝟐𝟐 𝒃𝟐𝟐

13
ANALYSIS OF D&C ALGORITHM

T(n) = 8 T(n/2) + cn2

# subproblems Subproblem Work dividing

size submatrices
recursive calls on
(n/2) x (n/2) matrices

❑ Case 1 of Master Theorem

❑ T(n)= Θ(𝑛log𝟐 8=n3)

No better than the standard algorithm??

14
STRASSEN’S IDEA
• Same Idea: n x n matrix = 2 x 2 matrix of (n/2) x (n/2) submatrices
• But multiply 2 x 2 matrices with only 7 recursive mults

• P1 = a11 x (b12 – b22) • C11 = P5 + P4 – P2 + P6

• P2 = (a11 + a12) x b22 • C12 = P1 + P2
• C21 = P4 + P3
• P3 = (a21 + a22) x b 11 • C22 = P5 + P1 – P3 – P7
• P4 = a22 x (b21 – b11)
• P5 = (a11 + a22) x (b11 +b22)
• P6 = (a11 - a22) x (b21 +b22)
• P7 = (a11 - a21) x (b11 +b12) 7 mults
18 adds/subs 15
ANALYSIS OF STRASSEN’S
ALGORITHM

T(n) = 7 T(n/2) + cn2

# subproblems Subproblem Work dividing

size submatrices
recursive calls on
(n/2) x (n/2) matrices

❑ Case 1 of Master Theorem

❑ T(n)= Θ(𝑛log𝟐 7=n2.81)

Beats the standard algorithm for n > 30.

Best known today: Θ(n2.376) [Coppersmith-Winograd] only of theoretical interest.
16
DRAWBACKS OF STRASSEN’S
ALGORITHM
▪ The constants used in Strassen’s method are
high and for a typical application Naive
method works better.

▪ For Sparse matrices, there are better methods

especially designed for them.

▪ The submatrices in recursion take extra space.

▪ Because of the limited precision of computer
arithmetic on noninteger values,
CONCLUSIONS
• Divide and conquer is just one of several powerful techniques for
algoritm design.
• Can lead to more efficient algorithms.

• Divide and conquer algorithms can be analyzed using recurrence

relations.
• So practice this math.

18
QUICKSORT
• Proposed by C.A.R. Hoare in 1962.
• Divide and conquer algorithm but the work is in divide rather than
in combine
• Different versions:
• Basic: Good in average case (for a random input)
• Randomized: good for all inputs in expectation (Randomized Las
Vegas algorithm)
• Sorts in place (like insertation sort, but unlike merge sort).
• Very practical (even though asymptotically not optimal).

19
IDEA: DIVIDE AND CONQUER
• Quicksort an n-element array A:
• Divide:
1. Pick a pivot element x in A
2. Partition the array into three sub-arrays
L(elements < x), E(elements = x), G(elements > x)

Conquer: Recursively sort sub-arrays L and G

Combine: Do nothing.

20
PSEDOCODE FOR BASIC QUICKSORT

QUICKSORT (A,p,r)
if p < r then
q ← PARTITION(A,p,r)
QUICKSORT(A,p,q-1)
QUICKSORT(A,q+1,r)

• p and r denote beginning and ending indices

• Initial call: QUICKSORT(A,1,n)

21
HOW TO CHOOSE PIVOT X
Basic Quick Sort:
Pivot is the first element: X = A[1]
Time: worst case O(n2) time
O(n log n) time for average input

Randomized Quick Sort:

X is chosen at random from the array A
(recursively each time a random choice)
Time: Expected O(n log n) for all inputs

Randomness gives us more control over runtime.

22
TWO PARTITIONING ALGORTIHMS

1. Hoare’s algorithm: Partitions around the first element of

sub- array (pivot x = A[p] ). Partitions grow in opposite
directions.

2. Lomuto’s algorithm: Partitions around the last element of

sub- array (pivot x = A[r] ). Partitions grow in the same
direction.

23
HOARE PARTITIONING
• Pick first element as pivot

40 20 10 80 60 50 7 30 100

• Partition array into sub-arrays:

• Less than the pivot
• Greater than the pivot
• At the end, put pivot into the middle

24
PARTITIONING EXAMPLE

pivot_index = 0 40 20 10 80 60 50 7 30 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index

25
PARTITIONING EXAMPLE
1. While data[too_big_index] <= data[pivot]
++too_big_index

pivot_index = 0 40 20 10 80 60 50 7 30 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index

26
PARTITIONING EXAMPLE
1. While data[too_big_index] <= data[pivot]
++too_big_index

pivot_index = 0 40 20 10 80 60 50 7 30 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index

27
PARTITIONING EXAMPLE
1. While data[too_big_index] <= data[pivot]
++too_big_index

pivot_index = 0 40 20 10 80 60 50 7 30 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index

28
PARTITIONING EXAMPLE
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index

pivot_index = 0 40 20 10 80 60 50 7 30 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index

29
PARTITIONING EXAMPLE
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index

pivot_index = 0 40 20 10 80 60 50 7 30 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index

30
PARTITIONING EXAMPLE
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]

pivot_index = 0 40 20 10 80 60 50 7 30 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index

31
PARTITIONING EXAMPLE
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]

pivot_index = 0 40 20 10 30 60 50 7 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index

32
PARTITIONING EXAMPLE
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 60 50 7 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index

33
PARTITIONING EXAMPLE
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 60 50 7 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index

34
PARTITIONING EXAMPLE
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 60 50 7 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index

35
PARTITIONING EXAMPLE
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 60 50 7 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index

36
PARTITIONING EXAMPLE
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 60 50 7 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index

37
PARTITIONING EXAMPLE
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 60 50 7 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index

38
PARTITIONING EXAMPLE
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index

39
PARTITIONING EXAMPLE
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index

40
PARTITIONING EXAMPLE
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index

41
PARTITIONING EXAMPLE
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index

42
PARTITIONING EXAMPLE
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index

43
PARTITIONING EXAMPLE
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index

44
PARTITIONING EXAMPLE
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index

45
PARTITIONING EXAMPLE
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index

46
PARTITIONING EXAMPLE
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index

47
PARTITIONING EXAMPLE
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
5. Swap data[too_small_index] and data[pivot_index]

pivot_index = 0 40 20 10 30 7 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index

48
PARTITIONING EXAMPLE
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
5. Swap data[too_small_index] and data[pivot_index]

pivot_index = 4 7 20 10 30 40 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index

49
PARTITION RESULT

7 20 10 30 40 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

<= data[pivot] > data[pivot]

RUNTIME: O(n)

50
RECURSION: QUICKSORT SUB- ARRAYS

7 20 10 30 40 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

<= data[pivot] > data[pivot]

51
ANOTHER EXAMPLE OF PARTITIONING

• choose pivot: 4 3 6 9 2 4 3 1 2 1 8 9 3 5 6
• search:4 3 6 9 2 4 3 1 2 1 8 9 3 5 6
• swap:4 3 3 9 2 4 3 1 2 1 8 9 6 5 6
• search:4 3 3 9 2 4 3 1 2 1 8 9 6 5 6
• swap:4 3 3 1 2 4 3 1 2 9 8 9 6 5 6
• search:4 3 3 1 2 4 3 1 2 9 8 9 6 5 6
• swap:4 3 3 1 2 2 3 1 4 9 8 9 6 5 6
• search:4 3 3 1 2 2 3 1 4 9 8 9 6 5 6
• swap with pivot:1 3 3 1 2 2 3 4 4 9 8 9 6 5 6

52
ANOTHER PARTITIONING SUBROUTINE

53
EXAMPLE OF PARTITIONING
6 10 13 5 8 3 2 11
i j

54
EXAMPLE OF PARTITIONING
6 10 13 5 8 3 2 11
i j

55
EXAMPLE OF PARTITIONING
6 10 13 5 8 3 2 11
i j

56
EXAMPLE OF PARTITIONING

6 10 13 5 8 3 2 11

6 5 13 10 8 3 2 11

i j

57
EXAMPLE OF PARTITIONING

6 10 13 5 8 3 2 11

6 5 13 10 8 3 2 11

i j

58
EXAMPLE OF PARTITIONING

6 10 13 5 8 3 2 11

6 5 13 10 8 3 2 11

i j

59
EXAMPLE OF PARTITIONING

6 10 13 5 8 3 2 11

6 5 13 10 8 3 2 11

6 5 3 10 8 13 2 11
i j

60
EXAMPLE OF PARTITIONING

6 10 13 5 8 3 2 11

6 5 13 10 8 3 2 11

6 5 3 10 8 13 2 11
i j

61
EXAMPLE OF PARTITIONING

6 10 13 5 8 3 2 11

6 5 13 10 8 3 2 11

6 5 3 10 8 13 2 11

6 5 3 2 8 13 10 11
i j

62
EXAMPLE OF PARTITIONING

6 10 13 5 8 3 2 11

6 5 13 10 8 3 2 11

6 5 3 10 8 13 2 11

6 5 3 2 8 13 10 11
i j

63
EXAMPLE OF PARTITIONING

6 10 13 5 8 3 2 11

6 5 13 10 8 3 2 11

6 5 3 10 8 13 2 11

6 5 3 2 8 13 10 11
i j

64
EXAMPLE OF PARTITIONING

6 10 13 5 8 3 2 11

6 5 13 10 8 3 2 11

6 5 3 10 8 13 2 11

6 5 3 2 8 13 10 11

2 5 3 6 8 13 10 11
i

65
LOMUTO PARTITIONING
• Select the last element A[r] in the subarray A[p..r] as
the pivot.
• The array is partitioned into four (possibly empty)
regions.
1. A[p..i ] — All entries in this region are < pivot.
2. A[i+1..j – 1] — All entries in this region are > pivot.
3. A[r] = pivot.
4. A[j..r – 1] — Not known how they compare to pivot.
• These hold before each iteration of the for loop, and
constitute a loop invariant.

66
CORRECTNESS OF PARTITION
• Use loop invariant. 1.
2.
A[p..i ] < pivot
A[i+1..j – 1] > pivot
• Initialization: 3. A[r] = pivot

• Before first iteration

• A[p..i] and A[i+1..j – 1] are empty
• r is the index of the pivot
• Maintenance: Partition(A, p, r)
x := A[r], i := p – 1;
• Use inductive hypothesis for j := p to r – 1 do
if A[j]  x then
i := i + 1;
A[i]  A[j];
A[i + 1]  A[r];
return i + 1;
67
CORRECTNESS OF PARTITION
Case 1: A[j] > x Increment j only.
p i j r
>x x

x >x
p i j r
x

x >x
1. A[p..i ] < pivot
2. A[i+1..j – 1] > pivot
3. A[r] = pivot

68
CORRECTNESS OF PARTITION
• Case 2: A[j]  x
• Increment i 1. A[p..i ] < pivot
• Swap A[i] and A[j] 2. A[i+1..j – 1] > pivot
• Increment j 3. A[r] = pivot

p i j r
x x

x >x
p i j r
x

x >x
69
CORRECTNESS OF PARTITION
• Termination:
• When the loop terminates, j = r, so all elements in A are partitioned
into one of the three cases:
• A[p..i]  pivot
• A[i+1..j – 1] > pivot
• A[r] = pivot
• At last, swap A[i+1] and A[r].
• Before swap A[i+1] > pivot
• After swap pivot moves from the end of the array to between the two
subarrays.
• Thus, procedure partition correctly performs the divide step.

70
ANALYZING QUICKSORT
• What will be the worst case for the algorithm?
• Partition is always unbalanced
• What will be the best case for the algorithm?
• Partition is perfectly balanced
• Which one is more likely?
• Balanced, except...
• Will any particular input cause the worst case?
• Yes: Already-sorted or reverse-sorted input

71
Worst Case Partitioning
Worst-case partitioning
One region has one element and the other has n – 1 elements
Maximally unbalanced

Recurrence: q=1 n n
1 n-1 n
T(n) = T(1) + T(n – 1) + n, n-1
1 n-2
T(1) = (1) n 1 n-3 n-2

T(n) = T(n – 1) + n 1
2 3
1 1 2
=  n 
n +   k  − 1 = (n) + (n 2 ) = (n 2 ) (n2)
 k =1 
72
WORST CASE
Depth Partition Time
0 n
1 n-1
… …

n-1 1

• Total: n + ( n - 1 ) + …… + 2 + 1
• Thus, the worst-case running time of quicksort is O(n2)
73
WORST-CASE RECURSION TREE
𝑻 𝒏 = 𝑻 𝟎 + 𝑻 𝒏 − 𝟏 + 𝒄𝒏

𝒄𝒏

𝑻𝟎 𝒄 𝒏 −𝟏

𝒉= 𝒏 𝑻𝟎 𝒄(𝒏 − 𝟐)

𝑻𝟎 ⋱
𝑻(𝟏)

74
WORST-CASE RECURSION TREE
𝑻 𝒏 = 𝑻 𝟏 + 𝑻 𝒏 − 𝟏 + 𝒄𝒏

𝒏
𝒄𝒏
𝜣 ∑𝒌 = 𝜣(𝒏𝟐)
𝒌=𝟏
Θ𝟏 𝒄 𝒏 −𝟏

𝒉= 𝒏 Θ𝟏 𝒄(𝒏 − 𝟐)

Θ𝟏 ⋱
Θ(𝟏)

𝑻 𝒏 = 𝜣 𝒏 + 𝜣(𝒏𝟐) = 𝜣(𝒏𝟐)
75
Best Case Partitioning
• Best-case partitioning
• Partitioning produces two regions of size n/2

• Recurrence: q=n/2
• T(n) = 2T(n/2) + (n)
• T(n) = (nlgn)
(Master theorem)

76
What if the split is ??
• Partition into two halves
• T(n) = 2 T(n/2) + cn
• T(n) = (n log n)

1 9
• What if the split is always ∶ ?
10 10
• T(n) = T( 1 n) + T( 9 n) + cn
10 10

77
 n cn
log𝟏𝟎 𝒏
1/10 n 9/10 n cn

1/100 n 9/100 n 9/100 n 81/100 n cn

81/1000 n 729/1000 n cn
1
≤cn
log𝟏𝟎 𝒏
𝟗 1 ≤cn

O(n logn)
78
How does partition affect
performance?

79
INTUTION FOR AVERAGE CASE
• Assumption: All permutations equally likely (realistic??)
• Unlikely: Splits always the same way at every level
• Expectation: Some splits will be reasonably balanced.
Some splits will be fairly unbalanced.
• Average case: A mix of good and bad splits.
Good and bad splits distributed randomly through the tree.
Good and bad splits occur in the alternate levels of the tree.
• Good split: Best-case split (n/2 and n/2)
• Bad split: Worst-case split (1 and n-1)

80
INTUTION FOR AVERAGE CASE
Average case
• All permutations of the input numbers are equally likely
• On a random input array, we will have a mix of well balanced and
unbalanced splits
• Good and bad splits are randomly distributed across throughout the
tree

combined partitioning cost: partitioning cost:

2n-1 = (n) n = (n)
n n
1 n-1
(n – 1)/2 + 1 (n – 1)/2
(n – 1)/2 (n – 1)/2
Nearly well balanced split
Alternate of a good and a bad split

• Running time of Quicksort when levels alternate

between good and bad splits is O(nlgn)
81
INTUTITION FOR AVERAGE CASE
• Two successive levels of Assume n is even
average-case produce a
half-and-half split. Average-case Best-case

• Same result as a single n n

level of the best case.
• Thus, after spending 1 n-1 n/2 n/2
Θ(n) + Θ(n-1) = Θ(n)
partitioning cost, we
reach the best case. n/2 -1 n/2
• Total tree height doubles
(2 log n).
• Runtime still Θ(n log n)
82
CONCLUSIONS
• Average-case analysis
• Hard in general
• Need to make assumptions on the input distribution or
workload
• May not represent the real scenario
• Somewhere between best- and worst- cases
• May be as bad as the worst case (e.g., insertion sort)
• Or as good as the best case (e.g., quicksort)

• Next: What if we can enforce input distribution?

83