Chapter 6

Transform-and-
Conquer

Copyright © 2007 Pearson Addison-Wesley. All rights reserved.

Transform and Conquer

This group of techniques solves a problem by a

transformation

● to a simpler/more convenient instance of the same

problem (instance simplification)

● to a different representation of the same instance

(representation change)

● to a different problem for which an algorithm is

already available (problem reduction)

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-1
Instance simplification -
Presorting
Solve a problem’s instance by transforming it into
another simpler/easier instance of the same problem

Presorting
Many problems involving lists are easier when list is
sorted.
● searching
● computing the median (selection problem)
● checking if all elements are distinct (element
uniqueness)

Also:
● Topological sorting helps solving some problems
for dags.
● Presorting is used inA. Levitin
many “Introduction togeometric algorithms.
the Design & Analysis of Algorithms,”
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
2 ed., Ch. 6 6-2
How fast can we sort ?

Efficiency of algorithms involving sorting depends on

efficiency of sorting.

Theorem (see Sec. 11.2): ⎡log2 n!⎤ ≈ n log2 n

comparisons are necessary in the worst case to
sort a list of size n by any comparison-based
algorithm.

Note: About nlog2 n comparisons are also sufficient to

sort array of size n (by mergesort).

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-3
Searching with presorting

Problem: Search for a given K in A[0..n-1]

Presorting-based algorithm:
Stage 1 Sort the array by an efficient sorting
algorithm
Stage 2 Apply binary search

Efficiency: Θ(nlog n) + O(log n) = Θ(nlog n)

Good or bad?
Why do we have our dictionaries, telephone
directories, etc. sorted?
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-4
Element Uniqueness with
presorting

● Presorting-based algorithm
Stage 1: sort by efficient sorting algorithm (e.g.
mergesort)
Stage 2: scan array to check pairs of adjacent
elements

Efficiency: Θ(nlog n) + O(n) = Θ(nlog n)

● Brute force algorithm

Compare all pairs of elements

Efficiency: O(n2)

6-5
Another algorithm? Hashing, which works well on
Copyright © 2007 Pearson Addison-Wesley. All rights reserved.
●
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6
Instance simplification – Gaussian
Elimination
Given: A system of n linear equations in n unknowns with an
arbitrary coefficient matrix.

Transform to: An equivalent system of n linear equations in

n unknowns with an upper triangular coefficient matrix.

Solve the latter by substitutions starting with the last

equation and moving up to the first one.

a11x1 + a12x2 + … + a1nxn = b1 a1,1x1+ a12x2 + … + a1nxn = b1

a21x1 + a22x2 + … + a2nxn = b2 a22x2 + … + a2nxn = b2

an1x1 + an2x2 + … + annxn = bn annxn = bn

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-6
Gaussian Elimination (cont.)

The transformation is accomplished by a sequence of

elementary operations on the system’s coefficient
matrix (which don’t change the system’s solution):

for i ←1 to n-1 do
replace each of the subsequent rows (i.e., rows
i+1, …, n) by
the difference between that row and an
appropriate multiple
of the i-th row to make the new coefficient in the i-
th column
of that row 0

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-7
Example of Gaussian Elimination
Solve 2x1 - 4x2 + x3 = 6
3x1 - x2 + x3 = 11
x1 + x2 - x3 = -3
Gaussian elimination
2 -4 1 6 2 -4 1 6
3 -1 1 11 row2 – (3/2)*row1 0 5 -1/2 2
1 1 -1 -3 row3 – (1/2)*row1 0 3 -3/2 -6 row3–(3/5)*row2

2 -4 1 6
0 5 -1/2 2
0 0 -6/5 -36/5
Backward substitution
x3 = (-36/5) / (-6/5) = 6
x2 = (2+(1/2)*6) / 5 = 1
x1 = (6 – 6 + 4*1)/2 = 2

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-8
Pseudocode and Efficiency of Gaussian
Elimination

Stage 1: Reduction to the upper-triangular matrix

for i ← 1 to n-1 do
for j ← i+1 to n do
for k ← i to n+1 do
A[j, k] ← A[j, k] - A[i, k] * A[j, i] / A[i, i] //improve!

Stage 2: Backward substitution

for j ← n downto 1 do
t←0
for k ← j +1 to n do
t ← t + A[j, k] * x[k]
x[j] ← (A[j, n+1] - t) / A[j, j]

Efficiency: Θ(n 3) + Θ(n 2) = Θ(n 3)

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-9
Searching Problem

Problem: Given a (multi)set S of keys and a search

key K, find an occurrence of K in S, if any

● Searching must be considered in the context of:

• file size (internal vs. external)
• dynamics of data (static vs. dynamic)

● Dictionary operations (dynamic data):

• find (search)
• insert
• delete

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-10
Taxonomy of Searching
Algorithms
● List searching (good for static data)
• sequential search
• binary search
• interpolation search

● Tree searching (good for dynamic data)

• binary search tree
• binary balanced trees: AVL trees, red-black
trees
• multiway balanced trees: 2-3 trees, 2-3-4 trees, B
trees

● Hashing (good on average case)

• open hashing (separate chaining)
• closed hashing (open addressing)
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 6 6-11
Binary Search Tree

Arrange keys in a binary tree with the

binary search tree property:
K

< > 5
K K 1
3
0
1
1 7
Example: 5, 3, 1, 10, 12, 7, 2
9 9
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-12
 Dictionary Operations on Binary Search
Trees

Searching – straightforward
Insertion – search for key, insert at leaf where search
terminated
Deletion – 3 cases:
deleting key at a leaf
deleting key at node with single child
deleting key at node with two children

Efficiency depends of the tree’s height: ⎣log2 n⎦ ≤ h ≤ n-1,

with height average (random files) be about 3log2 n

Thus all three operations have

• worst case efficiency: Θ(n)
• average case efficiency: Θ(log n) (CLRS, Ch. 12)

Bonus: inorder traversal produces sorted list

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-13
Balanced Search Trees
Attractiveness of binary search tree is marred by the
bad (linear) worst-case efficiency. Two ideas to
overcome it are:

● to rebalance binary search tree when a new

insertion
makes the tree “too unbalanced”
• AVL trees
• red-black trees

● to allow more than one key and two children

• 2-3 trees
• 2-3-4 trees
• B-trees A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 ed., Ch. 6
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd 6-14
Balanced trees: AVL trees

Definition An AVL tree is a binary search tree in

which, for every node, the difference between the
heights of its left and right subtrees, called the
balance factor, is at most 1 (with the height of an
empty tree defined as -1)

Tree (a) is an AVL tree; tree (b) is not an AVL

tree
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-15
Rotations

If a key insertion violates the balance requirement at

some node, the subtree rooted at that node is
transformed via one of the four rotations. (The
rotation is always performed for a subtree rooted
at an “unbalanced” node closest to the new leaf.)

Single R- Double LR-

rotation rotation

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-16
General case: Single R-rotation

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-17
General case: Double LR-
rotation

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-18
AVL tree construction - an
example
Construct an AVL tree for the list 5, 6, 8, 3, 2, 4, 7

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-19
AVL tree construction - an example
(cont.)

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-20
 Analysis of AVL trees
● h ≤ 1.4404 log2 (n + 2) - 1.3277 N(h-1) + N(h-2) ≤
average height: 1.01 log2n + 0.1 N(h)
for large n (found
empirically)

● Search and insertion are O(log n)

● Deletion is more complicated but is also O(log n)

● Disadvantages:
• frequent rotations
• complexity

● A similar idea: red-black trees (height of subtrees is

allowed to differ by up to a factor of 2)
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-21
 Multiway Search Trees
Definition A multiway search tree is a search tree that allows
more than one key in the same node of the tree.

Definition A node of a search tree is called an n-node if it

contains n-1 ordered keys (which divide the entire key range
into n intervals pointed to by the node’s n links to its
children):

k1 < k2 < … <

kn-1

[k1, ≥ kn-
< k1
k2 ) 1

Note: Every node in a classical binary search tree is a 2-node

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-22
2-3 Tree
Definition A 2-3 tree is a search tree that
● may have 2-nodes and 3-nodes
● height-balanced (all leaves are on the same level)

A 2-3 tree is constructed by successive insertions of keys

given, with a new key always inserted into a leaf of the
tree. If the leaf is a 3-node, it’s split into two with the
middle key promoted to the parent.
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-23
2-3 tree construction – an
example
Construct a 2-3 tree the list 9, 5, 8, 3, 2, 4, 7

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-24
Analysis of 2-3 trees

● log3 (n + 1) - 1 ≤ h ≤ log2 (n + 1) - 1

● Search, insertion, and deletion are in Θ(log n)

● The idea of 2-3 tree can be generalized by

allowing more keys per node
• 2-3-4 trees
• B-trees

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-25
Heaps and Heapsort

Definition A heap is a binary tree with keys at its

nodes (one key per node) such that:
● It is essentially complete, i.e., all its levels are full
except possibly the last level, where only some
rightmost keys may be missing

● The key at each node is ≥ keys at its children (this

is called a max-heap)

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-26
Illustration of the heap’s
definition

a heap not a not a

heap heap

Note: Heap’s elements are ordered top down (along

any path
down from its root), but they are not ordered
left to right
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-27
Some Important Properties of
a Heap

● Given n, there exists a unique binary tree with n

nodes that
is essentially complete, with h = ⎣log2 n⎦

● The root contains the largest key

● The subtree rooted at any node of a heap is also a

heap

● A heap can be represented as an array

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-28
Heap’s Array Representation
Store heap’s elements in an array (whose elements
indexed, for convenience, 1 to n) in top-down left-
to-right order
Example:
9 1 2 3 4 5
6
5 3 9 5 3 1 4
2
1 4 2

● Left child of node j is at 2j

● Right child of node j is at 2j+1
● Parent of node j is at ⎣j/2⎦
● Parental nodes are represented in the first ⎣n/2⎦ 6-29 nd
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 ed., Ch. 6
locations
Copyright © 2007 Pearson Addison-Wesley. All rights reserved.
Heap Construction (bottom-up)

Step 0: Initialize the structure with keys in the

order given

Step 1: Starting with the last (rightmost) parental

node, fix the heap rooted at it, if it doesn’t satisfy
the heap condition: keep exchanging it with its
larger child until the heap condition holds

Step 2: Repeat Step 1 for the preceding parental

node

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-30
Example of Heap Construction

Construct a heap for the list 2, 9, 7,

6, 5, 8

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-31
Pseudopodia of bottom-up heap
construction

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-32
Heapsort

Stage 1: Construct a heap for a given list of n keys

Stage 2: Repeat operation of root removal n-1 times:

– Exchange keys in the root and in the last
(rightmost) leaf
– Decrease heap size by 1
– If necessary, swap new root with larger
child until the heap condition holds

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-33
Example of Sorting by Heapsort
Sort the list 2, 9, 7, 6, 5, 8 by heapsort

Stage 1 (heap construction) Stage 2 (root/max removal)

1 9 7 6 5 8 9 6 8 2 5 7
2 9 8 6 5 7 7 6 8 2 5|9
2 9 8 6 5 7 8 6 7 2 5|9
9 2 8 6 5 7 5 6 7 2|8 9
9 6 8 2 5 7 7 6 5 2|8 9
2 6 5|7 8 9
6 2 5|7 8 9
5 2|6 7 8 9
5 2|6 7 8 9
2|5 6 7 8 9

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-34
Analysis of Heapsort
Stage 1: Build heap for a given list of n keys
worst-case h-1
C(n) = Σ 2(h-i) 2i = 2 ( n – log2(n + 1)) ∈ Θ(n)
i=0
# nodes
at level i
Stage 2: Repeat operation of root removal n-1 times
(fix heap)
worst-case n-1 2log2 i ∈
C(n) = Σ Θ(nlogn) i=1
Both worst-case and average-case efficiency:
Θ(nlogn)
In-place: yes
Stability: no (e.g., 1 A.1)
Levitin “Introduction to the Design & Analysis of Algorithms,” 2 ed., Ch. 6
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd 6-35
 Priority Queue

A priority queue is the ADT of a set of elements with

numerical priorities with the following operations:
• find element with highest priority
• delete element with highest priority
• insert (new) element with assigned priority (see
below)

● Heap is a very efficient way for implementing

priority queues

● A min-heap can also be used to handle priority

queue in which
highest priority = smallest number
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-36
Insertion of a New Element into a
Heap
● Insert the new element at last position in heap.
● Compare it with its parent and, if it violates heap
condition,
exchange them
● Continue comparing the new element with nodes
up the tree until the heap condition is satisfied

Example: Insert key 10

Efficiency: O(log n)
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-37
Horner’s Rule For Polynomial
Evaluation
Given a polynomial of degree n
p(x) = anxn + an-1xn-1 + … + a1x + a0
and a specific value of x, find the value of p at that
point.

Two brute-force algorithms:

p←0 p ← a0; power ← 1
for i ← n downto 0 do for i ← 1 to n do
power ← 1 power ← power * x
for j ← 1 to i do p ← p + ai * power
power ← power * x return p
p ← p + ai * power
return p
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-38
 Horner’s Rule
Example: p(x) = 2x4 - x3 + 3x2 + x - 5
= x(2x3 - x2 + 3x + 1) - 5
= x(x(2x2 - x + 3) + 1) - 5
= x(x(x(2x - 1) + 3) + 1) - 5
Substitution into the last formula leads to a faster
algorithm

Same sequence of computations are obtained by

simply
arranging the coefficient in a table and proceeding
as follows:
2 3*2+(-1)=5 3*5+3=18 3*18+1=55
coefficients 2 3*55+(-5)=160
-1 3 1 -5
x=3
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-39
Horner’s Rule pseudocode

Efficiency of Horner’s Rule: # multiplications = #

additions = n

Synthetic division of p(x) by (x-x0)

Example: Let p(x) = 2x4 - x3 + 3x2 + x - 5. Find p(x):
(x-3) 2x^3 + 5x^2 + 18x + 55
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 ed., Ch. 6
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd 6-40
 n
Computing a (revisited)
Left-to-right binary exponentiation
Initialize product accumulator by 1.
Scan n’s binary expansion from left to right and do
the following:
If the current binary digit is 0, square the
accumulator (S);
if the binary digit is 1, square the accumulator and
multiply it by a (SM).

Example: Compute a13. Here, n = 13 = 11012.

binary rep. of 13: 1 1 0 1
SM SM S SM
accumulator: 1 12*a=a a2*a = a3 (a3)2 = a6
(a6)2*a= a13 (computed left-to-right)

Efficiency: (b-1) ≤ M(n) ≤ 2(b-1) where b = ⎣log2 n⎦

A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 ed., Ch. 6
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd 6-41
+
1
 n
Computing a (cont.)
Right-to-left binary exponentiation
Scan n’s binary expansion from right to left and
compute an as the product of terms a2 i
corresponding to 1’s in this expansion.

Example Compute a13 by the right-to-left binary

exponentiation. Here, n = 13 = 11012.

1 1 0 1
a8 a2
a4 a : a2 i terms
a8 * a4 * a :
product (computed right-to-left)

Efficiency: same as that of left-to-right binary

exponentiation A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 ed., Ch. 6
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd 6-42
Problem Reduction

This variation of transform-and-conquer solves a

problem by a transforming it into different
problem for which an algorithm is already
available.

To be of practical value, the combined time of the

transformation and solving the other problem
should be smaller than solving the problem as
given by another method.

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-43
Examples of Solving Problems by
Reduction

● computing lcm(m, n) via computing gcd(m, n)

lcm(m,n)*gcd(m,n) = m*n
● counting number of paths of length n in a graph
by raising the graph’s adjacency matrix to the n-
th power

● transforming a maximization problem to a

minimization problem and vice versa (also, min-
heap construction)

● linear programming

● reduction to graph problems (e.g., solving puzzles

via state-space graphs)
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 ed., Ch. 6
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd 6-44
Examples of Solving Problems by
Reduction

● Counting number of paths of length n in a graph

by raising the graph’s adjacency matrix to the n-
th power

If (directed) graph G has adjacency matrix A,

then for any k, the (i,j) entry in A^k gives the
number of paths from vertex i and j of length k.
(Levitin, Ch. 6.6)

[ ] [ ]
1 0
0
101 100
2 4 1
0 01 0
2 01
A= A^2 = 0
0
1 0 0 000
3 0
0
000 000

Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2 nd ed., Ch. 6 6-45