Greedy Algorithms

Design and
Analysis of
Algorithms
GREEDY METHOD
KRUSKAL’S ALGORITHM USING UNION FIND
MINIMUM SPANNING TREE
GREEDY ALGORITHMS AND MATROIDS

 Instructor
Prof. Amrinder Arora
amrinder@gwu.edu
Please copy TA on emails
Please feel free to call as well

 Available for study sessions
Science and Engineering Hall
GWU
Algorithms Greedy Algorithms 2
LOGISTICS

CS 6212
Analysis
Asymptotic
NP-
Completeness
Design
D&C
Greedy
DP
Graph
B&B
Applications
WHERE WE ARE
 Done
 Done
 Starting today..

 A technique to build a complete solution by making a
sequence of “best selection” steps
 Selection depends upon actual problem
 Focus is simply on “what is best step from this point”
GREEDY METHOD

 Applications of greedy method are very broad.
 Examples:
 Sorting
 Merging sorted lists
 Knapsack
 Minimum Spanning Tree (MST)
 Hoffman Encoding
APPLICATIONS

 Select the minimum element
 Move it to the beginning
 Continue doing it for the remaining array
Given array a[1..n] of unsorted numbers
 For i = 1 to n-1
 For j = i+1 to n
 If (a[i] > a[j]) swap (a[i], a[j])
SORTING USING GREEDY METHOD

 How long does it take to sort using greedy method?
 Is it optimal?
TIME COMPLEXITY ANALYSIS

 Input: n sorted arrays of lengths
L[1], L[2],...,L[n]
 Problem: To merge all the arrays into one array as
fast as possible. Which pair to merge every time?
 We observe that:
 The final list will be a list of length L[1] + L[2] + … + L[n]
 The final list will be same regardless of the sequence in which
we merge lists
 However, the time taken may not be the same.
MERGING SORTED LISTS

 Greedy method: Merge the two shortest remaining
arrays.
 To Implement, we can keep a data structure, that
allows us to:
 Remove the two smallest arrays
 Add a larger array
 Keep doing this until we have one array

 Implement using heap
 Build the original heap – O(n) time
 For i = 1 to n-1
 Remove two smallest elements: 2 log (n)
 Add a new element log(n) time
 Total time: O(n log n)

 Input: A weight capacity C, and n items of weights W[1:n] and
monetary value V[1:n].
 Problem: Determine which items to take and how much of
each item so that the total weight is ≤ C, and the total value
(profit) is maximized.
 Formulation of the problem: Let x[i] be the fraction taken
from item i. 0 ≤ x[i] ≤ 1.
The weight of the part taken from item i is x[i]*W[i]
The Corresponding profit is x[i]*V[i]
 The problem is then to find the values of the array x[1:n] so
that x[1]V[1] + x[2]V[2] + ... + x[n]V[n] is maximized subject to
the constraint that x[1]W[1] + x[2]W[2] + ... + x[n]W[n] ≤ C
KNAPSACK PROBLEM

3 options
Policy 1: Choose the lightest
remaining item, and take as
much of it as can fit.
Policy 2: Choose the most
profitable remaining item,
and take as much of it as
can fit.
Policy 3: Choose the item
with the highest price per
unit weight (V[i]/W[i]), and
take as much of it as can fit.
Exercise: Prove by a counter example that Policy 1 does
not guarantee an optimal solution. Same with Policy 2.
Policy 3 always gives an optimal solution

Capacity = 7
Solution:
1. All of items {1, 2} and a fraction of item 3
2. But, how to handle this problem instance if we
cannot take “fractional” portions of items.
EXAMPLE
Item # 1 2 3 4 5
V ($) 3 5 10 11 9
W (lb) 1 2 5 6 7
V/W 3 2.5 2 1.83 1.28

 No, in fact, it can be as bad as you want to make it
to be.
 Example?
 A simple fix can make this algorithm only as bad as
a ratio of 2.
 How?
IS GREEDY ALGORITHM FOR INTEGER
KNAPSACK PROBLEM OPTIMAL?

 Definitions
 A spanning tree of a graph is a tree that has all nodes in the
graph, and all edges come from the graph
 Weight of tree = Sum of weights of edges in the tree
 Statement of the MST problem
 Input : a weighted connected graph G=(V,E). The weights are
represented by the 2D array (matrix) W[1:n,1:n], where W[i,j] is
the weight of edge (i,j).
 Output: Find a minimum-weight spanning tree of G.
MINIMUM SPANNING TREE

 Selection Policy: Minimum weighted edge that
does NOT create a cycle.
 Procedure ComputeMST(in:G, W[1:n,1:n]; out:T)
Sort edges: e[1], e[2], .. e[m].
Initialize counter j = 1
Initialize tree T to empty
While (number of edges in Tree < n-1) {
Does adding an edge e[j] create a cycle?
If No, add edge e[j] to tree T
}
GREEDY ALGORITHM

Sort edges: e[1], e[2], .. e[m].
Initialize counter j = 1
Initialize tree T to empty
While (number of edges in Tree < n-1) {
Does adding an edge e[j] create a cycle?
If No, add edge e[j] to tree T
}
HOW TO MAKE THIS EFFICIENT?

Each set is marked by a leader
When calling “find” on a set’s member, it
returns the leader
Leader maintains a rank (or height)
When doing a union, make the tree with
smaller height (or rank) to be a child of the
tree with the larger height
Note that this is NOT a binary tree.
UNION FIND DATA STRUCTURE

 When doing a find, follow that up by compressing the
path to the root, by making every node (along the
way) point to the root.
 This is not easy to prove, but Union Find with Path
compression, when starting with n nodes and m
operations, takes O(m log*(n)) time instead of O(m
log n) time, where the log* function is the iterated
logarithm (also called the super logarithm) and is an
extremely slow growing function.
 log*(n) is defined as follows:
 0, if n <= 1
 1 + log*(log n) if n > 1
UNION FIND – PATH COMPRESSION

 Using 2 Find operations to check if adding an edge
will create a cycle or not.
 When adding an edge, use a Union Operation
TIME COMPLEXITY ANALYSIS OF KRUSKAL’S
ALGORITHM

 Proof by contradiction
WHY DOES KRUSKAL’S ALGORITHM WORK?

 Optimal Substructure Property: A problem has
optimal substructure if an optimal solution to the
problem contains within it optimal solutions to
its sub problems.
 Greedy Choice Property: If a local greedy choice is
made, then an optimal solution including this choice
is possible.
TWO BASIC PROPERTIES OF OPTIMAL
GREEDY ALGORITHMS

 A subset system is a set E together with a set of
subsets of E, called I, such that I is closed under
inclusion. This means that if X ⊆ Y and Y ∈ I, then X
∈ I. (I is sometimes referred to as set of
independent sets.)
 A subset system is a matroid if it satisfies the
exchange property: If i1 and i2 are sets in I and i1 has
fewer elements than i2, then there exists an element
e ∈ i2 i1 such that i1 ∪ {e} ∈ I.
 For any subset system (E,I), the greedy algorithm
solves the optimization problem for (E,I) if and only if
(E,I) is a matroid.
GREEDY ALGORITHMS AND MATROIDS

 Prone to overuse
 You shouldn’t use this algorithm unless you can prove that the
solution is optimal.
 That is, no points in MT/Final for using greedy algorithm to
produce a suboptimal solution, where another algorithmic
technique (such as D&C) would have resulted in an optimal
solution.
 Why?
 Optimality has a “business value”. Suppose you are trying to
maximize the flights that you can schedule using 3 aircrafts.
 Time complexity merely represents a “cost of computation” of
that schedule.
 If one algorithm runs in 1 minute, but schedules only 7 flights,
and another algorithm runs in 2 hours, but schedules 8 flights,
which one would you use?
WHEN NOT TO USE GREEDY ALGORITHM

 Chess
 Sorting
 Shortest path computation
 Knapsack
GREEDY: TO APPLY OR NOT TO APPLY

CS 6212
Analysis
Asymptotic
NP-
Completeness
Design
D&C
Greedy
DP
Graph
B&B
Applications
WHERE WE ARE
 Done
 Done
 Done

 Greedy
 Book – first problem on interval
scheduling classes
 http://en.wikipedia.org/wiki/Huffman_coding
 http://www.cs.kent.edu/~dragan/AdvAlg05/GreedyAlg-1x1.pdf
 Dynamic Programming
 Dynamic Programming: Book sections 6.1 – 6.4
 http://www.yaroslavvb.com/papers/wagner-dynamic.pdf
READING ASSIGNMENT
Application # 5

Greedy Algorithms

More Related Content

What's hot (20)

Similar to Greedy Algorithms (20)

More from Amrinder Arora (20)

Recently uploaded (20)

Greedy Algorithms