Transform and Conquer

 Data structure: Heap

 Representation change: Heapsort
 Problem reduction

1
Expected Outcomes
 Students should be able to
 Explain the idea of heapsort
 Explain the ideas of problem reduction
 Analyze the time complexity of heapsort

2
 Heaps
A heap can be defined as a binary tree with keys assigned to its nodes
provided the following two conditions are met:

 The tree’s shape requirement: The binary tree is essentially

complete, that is, all its levels are full except possibly the last level,
where only some rightmost leaves may be missing.

 The parental dominance requirement: The key at each node is ≥

the keys at its children.

3
 Illustration of the heap’s definition
10 10 10

5 7 5 7 5 7

4 2 1 2 1 6 2 1

a heap not a heap not a heap

Second one is not heap because the tree’s shape requirement is violated and
third one because parental dominance requirement fails for the node with key 5.
Note that key values in a heap are ordered top down; that is a sequence of values
on any path from the root to a leaf is decreasing.
4
 Heaps and Heapsort
 Not only is the heap structure useful for heapsort,
but it also makes an efficient priority queue.
 Heapsort
 In place
 O(nlogn)
 A priority queue is the ADT for maintaining a set S
of elements, each with an associated value called a
key/priority. It supports the following operations:
 find element with highest priority
 delete element with highest priority
 insert element with assigned priority

5
 Properties of Heaps (1)
 Heap and its array representation. 9
 Conceptually, we can think of a heap as a
binary tree.
 But in practice, it is easier and more efficient 5 3
to implement a heap using an array.
 Store heap’s elements in an array (whose
elements indexed, for convenience, 1 to n) 1 4 2
in top-down left-to-right order
 Relationships between indexes of parents
and children. 1 2 3 4 5 6
PARENT(i) LEFT(i) RIGHT(i) 9 5 3 1 4 2
return i/2 return 2i return 2i+1

Parental nodes are represented in the first n/2 locations

6
 Properties of Heaps (2)
 Max-heap property and min-heap property
 Max-heap: for every node other than root, A[PARENT(i)] >= A(i)
 Min-heap: for every node other than root, A[PARENT(i)] <= A(i)
 The root has the largest key (for a max-heap)
 The subtree rooted at any node of a heap is also a heap
 Given a heap with n nodes, the height of the heap,
h = log n .
- Height of a node: the number of edges on the longest simple
downward path from the node to a leaf.
- Height of a tree: the height of its root.
- level of a node: A node’s level + its height = h, the tree’s height.

7
 Bottom-up Heap construction
 How can we construct a heap for a given list of keys? There are two
principal alternatives for doing that. The first is bottom-up heap
construction algorithm as given below:

 Build an essentially complete binary tree by inserting n keys in the

given order.

 Heapifies a series of trees

 Starting with the last (rightmost) parental node, heapify/fix the subtree rooted
at it: if the parental dominance condition does not hold for the key at this
node:
1. exchange its key with the key of its larger child
2. Heapify/fix the subtree rooted at it (now in the child’s position)
 Proceed to do the same for the node’s immediate predecessor.
 Stops after this is done for the tree’s root.
8
 Example of Heap Construction
Construct a heap for the list 2, 9, 7, 6, 5, 8
2 2 2

9 7 > 9 8 9 8

6 5 8 6 5 7 6 5 7

2 9 9

9 8 > 2 8 > 6 8

6 5 7 6 5 7 2 5 7

Try Example: 4 1 3 2 16 9 10 14 8 7 16 14 10 8 7 9 3 2 4 1
9
 Bottom-up heap construction algorithm(A
Recursive version)
ALGORITHM HeapBottomUp(H[1..n])
//Constructs a heap from the elements Given a heap of n nodes, what’s
//of a given array by the bottom-up algorithm the index of the last parent?
//Input: An array H[1..n] of orderable items
//Output: A heap H[1..n] n/2
for i  n/2 downto 1 do
MaxHeapify(H, i) ALGORITHM MaxHeapify(H, i)
l  LEFT(i)
r  RIGHT(i)
if l <= n and H[l] > H[i] // if left child exists and > H[i]
then largest  l
else largest  i
if r <= n and H[r] > H[largest] // if R child exists and > H[largest]
then largest  r
if largest  i
then exchange H[i] H[largest] // heapify the subtree
MaxHeapify(H, largest)
10
Bottom-up heap construction algorithm

// from the last parent down to 1, heapify the subtree rooted at i

// k: the root of the subtree to be heapified; v: the key of the root

// if not a heap yet and the left child exists

// find the larger child, j: its index.

// if the key of the root > that of the larger child, done.

// exchange the key with the key of the larger child

// again, k: the root of the subtree to be heapified; v: the key of the root

11
Worst-Case Efficiency
 Worst case:
a full tree; each key on a certain level will travel to the leaf.
 Fix a subtree rooted at height j: 2j comparisons

 Fix a subtree rooted at level i : 2(h-i) comparisons

 A node’s level + its height = h, the tree’s height.

 Total for heap construction phase:

h-1

Σ
i=0
2(h-i) 2i = 2 ( n – log (n + 1)) = Θ(n)

# nodes at level i

12
Bottom-up vs. Top-down Heap Construction

 Bottom-up: Put everything in the array

and then heapify/fix the trees in a
bottom-up way.
 Top-down: Heaps can be constructed
by successively inserting elements (see
the next slide) into an (initially) empty
heap.

13
 Insertion of a New Element
 The algorithm
 Insert element at the last position in heap.
 Compare with its parent, and exchange them if it violates the parental dominance
condition.
 Continue comparing the new element with nodes up the tree until the parental
dominance condition is satisfied.
 Example 1: add 10 to a heap: 9 6 8 2 5 7
 Efficiency:
h element
Inserting one new O(logn)
to a heap with n-1 nodes requires no more comparisons than
the heap’s height
 Example 2: Use the top-down method to build a heap for numbers 2 9 7 6 5 8
 Questions
 What is the efficiency for a top-down heap construction algorithm for a heap of size n?
 Which one is better, a bottom-up or a top-down heap construction ?

14
 Root Deletion
The root of a heap can be deleted and the heap
fixed up as follows:
1. Exchange the root with the last leaf
2. Decrease the heap’s size by 1
3. Heapify the smaller tree in exactly the same
way we did it in MaxHeapify().
It can’t make key comparison more
Efficiency: 2h  Θ(logn) than twice the heap’s height

Example: 9 8 6 2 5 1
15
Update keys
 How to do it?

16
 Heapsort Algorithm
 The algorithm
 (Heap construction) Build heap for a given
array (either bottom-up or top-down)
 (Maximum deletion ) Apply the root-
deletion operation n-1 times to the
remaining heap until heap contains just
one node.
 An example: 2 9 7 6 5 8
17
 Analysis of Heapsort
Recall algorithm:
Θ(n) 1. Bottom-up heap construction
Θ(log n) 2. Root deletion

Repeat 2 until heap contains just one node.

n – 1 times
Total: Θ(n) + Θ( n log n) = Θ(n log n)

• Note: this is the worst case. Average case also Θ(n log n).

18
 Problem Reduction
 Problem Reduction
 If you need to solve a problem, reduce it to another problem that you
know how to solve.
 Linear programming
 A problem of optimizing a linear function of several variables subject to
constraints in the form of linear equations and linear inequalities .
 Formally,
Maximize(or minimize) c1x1+ …cnxn
Subject to ai1x1+…+ ainxn ≤ (or ≥ or =) bi, for i=1…n
x1 ≥ 0, …, xn ≥ 0
 Reduction to graph problems

19
 Linear Programming—Example 1:
Investment Problem
 Scenario
 A university endowment needs to invest $100million
 Three types of investment:
 Stocks (expected interest: 10%)
 Bonds (expected interest: 7%)
 Cash (expected interest: 3%)
 Constraints
 The investment in stocks is no more than 1/3 of the money
invested in bonds
 At least 25% of the total amount invested in stocks and bonds
must be invested in cash
 Objective:
 An investment that maximizes the return

20
Example 1 (cont’)
 Maximize 0.10x + 0.07y + 0.03z
subject to x + y + z = 100
x (1/3)y
z  0.25(x + y)
x  0, y  0, z  0

21
 Linear Programming—Example 2 :
Election Problem
 Scenario:
 Objective:
 A politician that tries to win an election.  Figure out the minimum
 Three types of areas of the district: amount of money that you
 urban (100,000 voters),
need to spend in order to
win
 suburban (200,000 voters), and
 rural(50,000 voters).  50,000 urban votes
 Primary issues:  100,000 suburban votes
 Building more roads  25,000 rural votes
Gun control
constraints:


 Farm subsidies
 Gasoline tax Policy Urban Suburban rural
 Advertisement fee Build roads -2 5 3
 For every $1,000… Gun control 8 2 -5
Farm subsidies 0 0 10
Gasoline tax 10 0 -2 22
Example 2 (cont’)
x: the number of thousand of dollars spent on advertising on building
roads
y: the number of thousand of dollars spent on advertising on gun control
z: the number of thousand of dollars spent on advertising on farm
subsidies
w: the number of thousand of dollars spent on advertising on gasoline
taxes
 Minimize x + y + z + w
subject to –2x + 8y + 0z + 10w  50
5x + 2y + 0z + 0w  100
3x – 5y + 10z - 2w  25
x, y, z, w  0

23
 Linear Programming—Example 3: Knapsack
Problem (Continuous/Fraction Version)
 Scenario
Given n items:
 weights: w1 w2 … wn
 values: v1 v2 … vn
 a knapsack of capacity W

 Constraints
 Any fraction of any item can be put into the knapsack.
 All the items must fit into the knapsack.
 Objective:
 Find the most valuable subset of the items

24
Example 3 (cont’)
 Maximize n

v x
j 1
j j

subject to
n

w x
j 1
j j W

0  xj  1 for j = 1,…, n.

25
 Linear Programming—Example 3: Knapsack
Problem (Discrete Version)

 Scenario
Given n items:
 weights: w1 w 2 … w n
 values: v1 v2 … vn
 a knapsack of capacity W

 Constraints
 an item can either be put into the knapsack in its entirely or
not be put into the knapsack.
 All the items must fit into the knapsack.
 Objective:
 Find the most valuable subset of the items

26
Example 3 (cont’)
 Maximize n

v x
j 1
j j

subject to
n

w x
j 1
j j W

xj  {0,1} for j = 1,…, n.

27
 Algorithms for Linear Programming
 Simplex algorithm: exponential time.
 Ellipsoid algorithm: polynomial time.
 Interior-point methods: polynomial time.

 Integer linear programming problem

 no polynomial solution.
 requires the variables to be integers.

28
 Examples of Solving Problems by Reduction

 computing lcm(m, n) via computing gcd(m, n)

 counting number of paths of length at most n in a graph
by raising the graph’s adjacency matrix to the n-th power
 transforming a maximization problem to a minimization
problem and vice versa (also, min-heap construction)
 reduction to graph problems (e.g., solving River-crossing
puzzle via state-space graphs)

29