Greedy

Greedy Algorithms
Algorithms
• General principle of greedy algorithm
• Activity-selection problem
- Optimal substructure
- Recursive solution
- Greedy-choice property
- Recursive algorithm
• Minimum spanning trees
- Generic algorithm
- Definition: cuts, light edges
- Prim’s algorithm
Jan. 2018
 Overview
 Like dynamic programming (DP), used to solve
optimization problems.
 Problems exhibit optimal substructure (like DP).
 Problems also exhibit the greedy-choice property.
» When we have a choice to make, make the one that looks
best right now.
» Make a locally optimal choice in hope of getting a globally
optimal solution.

eedy - 2
 Greedy Strategy
 The choice that seems best at the moment is the one we
go with.
» Prove that when there is a choice to make, one of the optimal
choices is the greedy choice. Therefore, it’s always safe to
make the greedy choice.
» Show that all but one of the subproblems resulting from the
greedy choice are empty.

eedy - 3
 Activity-selection Problem
 Input: Set S of n activities, a1, a2, …, an.
» si = start time of activity i.
» fi = finish time of activity i.
 Output: Subset A of maximum number of compatible
activities.
» Two activities are compatible, if their intervals don’t overlap.

Example: Activities in each line

are compatible.

eedy - 4
 Example:
i 1 2 3 4 5 6 7 8 9 10 11
si 1 3 0 5 3 5 6 8 8 2 12
fi 4 5 6 7 8 9 10 11 12 13 14

{a3, a9, a11} consists of mutually compatible activities.

But it is not a maximal set.
{a1, a4, a8, a11} is a largest subset of mutually compatible activities.
Another largest subset is {a2, a4, a9, a11}.

eedy - 5
 Optimal Substructure
 Assume activities are sorted by finishing times.
» f 1  f2  …  fn .
 Suppose an optimal solution includes activity ak.
» This generates two subproblems.
» Selecting from a1, …, ak-1, activities compatible with one
another, and that finish before ak starts (compatible with ak).
» Selecting from ak+1, …, an, activities compatible with one
another, and that start after ak finishes.
» The solutions to the two subproblems must be optimal.

eedy - 6
 Recursive Solution
 Let Sij = subset of activities in S that start after ai
finishes and finish before aj starts.
 Subproblems: Selecting maximum number of mutually
compatible activities from Sij.
 Let c[i, j] = size of maximum-size subset of mutually
compatible activities in Sij.

 0 if Sij  
Recursive
c[i, j ]  max{c[i, k ]  c[k , j ]  1} if S  
Solution:  ik  j ij

eedy - 7
 Greedy-choice Property
 The problem also exhibits the greedy-choice property.
» There is an optimal solution to the subproblem Sij, that
includes the activity with the smallest finish time in set Sij.
» Can be proved easily.
 Hence, there is an optimal solution to S that includes
a1.
 Therefore, make this greedy choice without solving
subproblems first and evaluating them.
 Solve the subproblem that ensues as a result of making
this greedy choice.
 Combine the greedy choice and the solution to the
subproblem.

eedy - 8
 Recursive Algorithm
Recursive-Activity-Selector
Recursive-Activity-Selector(s, (s,f,f,i,i,j)j)
1.1. mm i+1
i+1
2.2. while
whilemm<<jjand andssmm<<ffi i
3.3. domm
do m+1
m+1
4.4. ifif mm<<jj
5.5. then
thenreturn
return{a{amm}}

Recursive-Activity-Selector(s,
Recursive-Activity-Selector(s,f,f,m,
m,j)j)
6.6. else return
elsereturn

Initial Call: Recursive-Activity-Selector (s, f, 0, n + 1)

Complexity: (n)
Straightforward to convert the algorithm to an iterative one.
See the text.
eedy - 9
 Typical Steps
 Cast the optimization problem as one in which we
make a choice and are left with one subproblem to
solve.
 Prove that there’s always an optimal solution that
makes the greedy choice, so that the greedy choice is
always safe.
 Show that greedy choice and optimal solution to
subproblem  optimal solution to the problem.
 Make the greedy choice and solve top-down.
 May have to preprocess input to put it into greedy
order.
» Example: Sorting activities by finish time.

eedy - 10
 Elements of Greedy Algorithms
 Greedy-choice Property.
» A globally optimal solution can be arrived at by making a
locally optimal (greedy) choice.
 Optimal Substructure.

eedy - 11
 Minimum Spanning Trees
• Given: Connected, undirected, weighted graph, G
• Find: Minimum - weight spanning tree, T
• Example:
b 7 c
5 Acyclic subset of edges(E) that connects
a 1
all vertices of G.
3 -3
11
d e f
0 2 b c
5

a 1
3 -3
weight of T:
w(T) =(u , w(u, v)
v )T
d
0
e f

eedy - 12
 Generic Algorithm
• A - subset of some Minimum Spanning tree (MST).
• “Grow” A by adding “safe” edges one by one.

• Edge is “safe” if it can be added to A without destroying this

invariant.

:=;
AA:= ;
while
whileAAnot
notcomplete
completetree
treedo
do
find
findaasafe
safeedge
edge(u,
(u,v);
v);
AA:=:=AA{(u,
{(u,v)}
v)}
od
od

eedy - 13
 Definitions
• Cut – A cut (S, V – S) of an undirected graph G = (V, E) is a
partition of V.
• A cut respects a set A of edges if no edge in A crosses the cut.
• An edge is a light edge crossing a cut if its weight is the minimum
of any edge crossing the cut.

cut that respects an edge set A = {(a, b), (b, c)}

a
5
b 7 c a light edge crossing cut
(could be more than one)
1 -3
11 3
 cut partitions vertices into
d
0
e
2
f disjoint sets, S and V – S.

eedy - 14
 Theorem 23.1
Theorem
Theorem23.1:
23.1:Let
Let(S,
(S,VV--S)
S)be
beany
anycut
cutthat
thatrespects
respectsA, A,and
andlet
let(u,
(u,v)
v)
be
beaalight
lightedge
edgecrossing
crossing(S,
(S,VV--S).
S).Then,
Then,(u,
(u,v)
v)isissafe
safefor
forA.
A.
Proof:
Let T be a MST that includes A.
Case 1: (u, v) in T. We’re done.
Case 2: (u, v) not in T. We have the following:
edge in A
(x, y) crosses cut.
x Let T´ = {T - {(x, y)}}  {(u, v)}.
y cut Because (u, v) is light for cut,
u
w(u, v)  w(x, y). Thus,
w(T´) = w(T) - w(x, y) + w(u, v)  w(T).
shows edges Hence, T´ is also a MST.
in T v So, (u, v) is safe for A.
eedy - 15
 Corollary

In general, A will consist of several connected components (CC).

Corollary:
Corollary:IfIf(u,
(u,v)
v)isisaalight
lightedge
edgeconnecting
connectingone
oneCC
CCin
inGGAA==(V,
(V,A)
A)
to
toanother
anotherCC
CCin inGGAA,,then
then(u,
(u,v)
v)isissafe
safefor
forA.
A.

eedy - 16
 Kruskal’s Algorithm
 Starts with each vertex in its own component.
 Repeatedly merges two components into one by choosing a light
edge that connects them (i.e., a light edge crossing the cut
between them).
 Scans the set of edges in monotonically increasing order by
weight.
 Uses a disjoint-set data structure to determine whether an edge
connects vertices in different components.

eedy - 17
 Prim’s Algorithm
 Builds one tree. So A is always a tree.
 Starts from an arbitrary “root” r.
 At each step, adds a light edge crossing cut (VA, V - VA) to A.
» VA = vertices that A is incident on.

VA

V - VA

cut

eedy - 18
 Prim’s Algorithm
 Uses a priority queue Q to find a light edge quickly. a p
n -h e
 Each object in Q is a vertex in V - VA. a s a mi
m en te d
imple
Min-heap as a binary
tree. 1 11

2 14 3 13

4 18 5 17 6 19 7 20

8 18 9 24 26 10
eedy - 19
 Prim’s Algorithm
 key(v) (key of v) is minimum weight of any edge (u, v),
where u  VA.
 Then the vertex returned by Extract-Min is v such that
there exists u  VA and (u, v) is light edge crossing (VA,
V - VA).
 key(v) is  if v is not adjacent to any vertex in VA.
VA u1 w1 V - VA
u2 w2 key(v) = min{w1, w2, w3}
u3 w3 v

v’
v’’ key(v’) = key(v’’) = 
cut
eedy - 20
 Prim’s Algorithm
QQ:=
:=V[G];
V[G];
for eachuuQQdo
foreach do Complexity:
key[u]
key[u]:= := Using binary heaps: O(E lgV).
od;
od; Initialization – O(V).
key[r]
key[r]:=
:=0;0; Building initial queue – O(V).
[r]
[r]:=
:=NIL;
NIL; V Extract-Min’s – O(V lgV).
whileQQ
while dodo E Decrease-Key’s – O(E lgV).
uu:=
:=Extract-Min(Q);
Extract-Min(Q);
for eachvvAdj[u]
foreach Adj[u]dodo Using min-heaps: O(E + VlgV).
ififvvQQw(u,
w(u,v)v)<<key[v]
key[v]then
then (see book)
[v]
[v]:=
:=u;u;
key[v]
key[v]:=
:=w(u,
w(u,v)v) decrease-key operation
fifi
od
od
od
od

Note: A = {([v], v) : v  V - {r} - Q}.

eedy - 21
 Example of Prim’s Algorithm

Not in tree
5 7
a/0 b/ c/

11 3 1 -3 Q=a b c d e f
0  
d/ e/ f/
0 2

eedy - 22
 Example of Prim’s Algorithm

5 7
a/0 b/5 c/

11 3 1 -3 Q=b d c e f
5 11  
d/11 e/ f/
0 2

eedy - 23
 Example of Prim’s Algorithm

5 7
a/0 b/5 c/7

11 3 1 -3 Q=e c d f
3 7 11 
d/11 e/3 f/
0 2

eedy - 24
 Example of Prim’s Algorithm

5 7
a/0 b/5 c/1

11 3 1 -3 Q=d c f
0 1 2
d/0 e/3 f/2
0 2

eedy - 25
 Example of Prim’s Algorithm

5 7
a/0 b/5 c/1

11 3 1 -3 Q=c f
1 2
d/11 e/3 f/2
0 2

eedy - 26
 Example of Prim’s Algorithm

5 7
a/0 b/5 c/1

11 3 1 -3 Q=f
-3
d/11 e/3 f/-3
0 2

eedy - 27
 Example of Prim’s Algorithm

5 7
a/0 b/5 c/1

11 3 1 -3 Q=

d/11 e/3 f/-3

0 2

eedy - 28
 Example of Prim’s Algorithm

5
a/0 b/5 c/1

3 1 -3

d/0 e/3 f/-3

0

eedy - 29