Lecture 17
Lecture 17
Part 1 Part 2
Today
• Graphs are directed and edges have “capacities” (weights)
• We have a special “source” vertex s and “sink” vertex t.
• s has only outgoing edges*
• t has only incoming edges*
3
4
4 2 6
6
2 4
s t
3
2 6
3 4
6
10
*at least for this class
An s-t cut
is a cut which separates s from t
3
4
4 2 6
6
2 4
s t
3
2 6
3 4
6
10
An s-t cut
is a cut which separates s from t
3
4
4 2 6
6
2 4
s t
3
2 6
3 4
6
10
An s-t cut
is a cut which separates s from t
• An edge crosses the cut if it goes from s’s side to t’s side.
3
4
4 2 6
6
2 4
s t
3
2 6
3 4
6
10
An s-t cut
is a cut which separates s from t
• An edge crosses the cut if it goes from s’s side to t’s side.
• The cost (or capacity) of a cut is the sum of the
capacities of the edges that cross the cut.
This cut has cost
3 4 + 2 + 10 = 16
4
4 2 6
6
2 4
s t
3
2 6
3 4
6
this edge does not
cross the cut; it’s going 10
in the wrong direction.
A minimum s-t cut
is a cut which separates s from t
with minimum capacity.
• Question: how do we find a minimum s-t cut?
This cut has cost
4 + 3 + 4 = 11
3
4
4 2 6
6
2 4
s t
3
2 6
3 4
6
10
Example where this comes up
• 1955 map of rail
networks from the Soviet
this says
Union to Eastern Europe.
“the bottleneck”
• Declassified in 1999.
• 44 edges, 105 vertices
3
2
2 4
4 6
2 1
6 1
4
2 4
s 1 2 t
3
2 6
3 4
6
The value of 10
this flow is 4.
A maximum flow
is a flow of maximum value.
• This example flow is pretty wasteful, I’m not utilizing
the capacities very well.
3
2
2 4
4 6
2 1
6 1
4
2 4
s 1 2 t
3
2 6
3 4
6
The value of 10
this flow is 4.
That’s the same as the
3
3
3 4
4 6
4
2 6 1
2 4
s 2 3 4 t
3
5 2 6
3 1
4
6 4
5
10
Example
• 1955 map of rail
networks from the Soviet
Union to Eastern Europe.
• Declassified in 1999.
• 44 edges, 105 vertices
These numbers
are flows.
Schriver 2002
Pre-lecture exercise
• Each edge is a (directed) rickety bridge.
• How many bridges need to fall down to disconnect
s from t?
• If only one person can be on a bridge at a time, and
you want to keep traffic moving (aka, no waiting at
vertices allowed), how many people can get to t at
a time?
s t
How about now?
• Each edge is a (directed) rickety bridge.
• How many bridges need to fall down to disconnect
s from t?
• If only one person can be on a bridge at a time, and
you want to keep traffic moving (aka, no waiting at
vertices allowed), how many people can get to t at
a time?
s t
How about now?
• Each edge is a (directed) rickety bridge.
• How many bridges need to fall down to disconnect
s from t?
• If only one person can be on a bridge at a time, and
you want to keep traffic moving (aka, no waiting at
vertices allowed), how many people can get to t at
a time?
s t
Pre-lecture exercise
• Can you come up with a graph where the two
numbers are different?
Theorem
Max-flow min-cut theorem
The value of a max flow from s to t Intuition: in a max flow,
is equal to the min cut better fill up,
and this is the bottleneck.
the cost of a min s-t cut.
3
3
3 4
4 6
4
2 6 1
2 4
s 2 3 4 t
3
5 2 6
3 1
4
6 4
5
10
Proof outline
• Lemma 1: max flow ≤ min cut.
• Proof-by-picture
• Lemma 2: max flow ≥ min cut.
• Proof-by-algorithm, using a “Residual graph” #$
• Sub-Lemma: t is not reachable from s in #$ ⇔ f is a max flow.
• ⇐ first we do this direction:
• Claim: If there is a path from s to t in #$ , then we can increase the
flow in G.
• Hence we couldn’t have started with a max flow.
• ⇒ for this direction, proof-by-picture again.
x stuff
comes
out of s
t
s
Proof of Min-Cut Max-Flow Thm
• Lemma 1:
• For ANY s-t flow and ANY s-t cut, the value of the flow is
at most the cost of the cut.
• Hence max flow ≤ min cut.
Outline of algorithm:
• Start with zero flow
• We will maintain a “residual graph” Gf
• A path from s to t in Gf will give us a Assume for today that we
way to improve our flow. don’t have edges like this,
although it’s not necessary.
• We will continue until there are no s-t
paths left.
Tool: Residual networks
Say we have a flow ! #, % − ) #, %
!" #, % = ' ) %, #
*) #, % ∈ ,
*) %, # ∈ ,
a 0 ./0.
2 • )(#, %) is the flow on edge #, % .
• c(#, %) is the capacity on edge #, %
2
3 6
4 1
t
s
8
1 3 2
a
b 3
Call the flow ) 2
Call the graph 4 0
1 5
1 t
Create a new residual s 7 1
network from this flow:
1 b 2
Call this graph 4"
Tool: Residual networks
Say we have a flow
a
Forward edges are the
2 amount that’s left.
6
2 Backwards edges are the
3
4 1
amount that’s been used.
t
s
8
1 3 2
a
b 3
Call the flow !
2
Call the graph "
1 5
1 t
Create a new residual s 7 1
network from this flow:
1 b 2
Call this graph "#
Why look at residual networks?
Lemma:
• t is not reachable from s in !" ⇔ f is a max flow.
Example: s is reachable from t in this
example, so not a max flow.
2
2 3
3 6 2
4
1 1 5
t
s 8 s 1 t
1 3 2 7 1
Call the flow $ 1 2
Call the graph ! Call this graph !"
Why look at residual networks?
Lemma:
• t is not reachable from s in !" ⇔ f is a max flow.
To see that this flow is not
Example: s is reachable from t in this
maximal, notice that we can example, so not a max flow.
improve it by sending one
more unit more stuff along Now update the residual graph…
this path:
2
2 3
43 6 2
4 21 1 5
t
s 8 s 1 t
1 3 32 7 1
Call the flow # 1 2
Call the graph ! Call this graph !"
Why look at residual networks?
Lemma:
• t is not reachable from s in !" ⇔ f is a max flow.
Example:
To see that this flow is not
maximal, notice that we can Now we get this residual graph:
improve it by sending one
more unit more stuff along
this path:
2
2 4
43 6 2
4 21 0 4
t
s 8 s 2 t
1 3 32 7 0
Call the flow # 1 3
Call the graph ! Call this graph !"
Why look at residual networks?
Lemma:
• t is not reachable from s in !" ⇔ f is a max flow.
Example:
Now we get this residual graph:
Now we can’t reach t from s.
So the lemma says that f is a
max flow.
2
2 4
43 6 2
4 21
t 4
s 8 s 2 t
1 3 32 7
Call the flow $ 1 3
Call the graph ! Call this graph !"
Let’s prove the Lemma
• t is not reachable from s in !" ⇔ f is a max flow.
We will prove the
Lemma: ⇐ first this direction ⇐ contrapositive
2
2 3
43 6 2
4 21 1 5
t
s 8 s 1 t
1 3 32 7 1
Call the flow $ 1 2
Call the graph ! Call this graph !"
claim:
if there is an augmenting path, we can
increase the flow along that path.
• In the situation we just saw, this is pretty obvious.
2
2 3
43 6 2
4 21 1 5
t
s 8 s 1 t
1 3 32 7 1
Call the flow # 1 2
Call the graph ! Call this graph !"
2
2 1
1 6 2
4 1 3 5
t
s 3 s 1 t
1 3 0 2 3
Call the flow # 1
Call the graph ! Call this graph !"
I changed some of
the weights and
edge directions.
claim:
if there is an augmenting path, we can
increase the flow along that path.
• But maybe there are backward edges in the path.
• Here’s a slightly different example of a flow:
2
2 1
1 6 2
4 1 3 5
t
s 3 s 1 t
1 3 0 2 3
Call the flow # 1
Call the graph ! Call this graph !"
Now we should NOT increase the flow at
all the edges along the path! I changed some of
• For example, that will mess up the the weights and
conservation of stuff at this vertex. edge directions.
claim:
if there is an augmenting path, we can
increase the flow along that path.
• In this case we do something a bit different:
We will add We will remove flow here,
flow here since our augmenting path
is going backwards along
2 this edge.
2 1
21 6 2
4
01 3 5
t
s 3 s 1 t
1 3 10 2 3
Call the flow # 1
Call the graph ! Call this graph !"
We will add
flow here
claim:
if there is an augmenting path, we can
increase the flow along that path.
• In this case we do something a bit different:
Then we’ll update the residual graph:
2
2 2
12 6 2
4 10 2 6
t
s 3 s
t
1 3 01 2 2
Call the flow # 1 1
Call the graph ! Call this graph !"
2 in, 2 out
Before: flow value is 2
2
2 1
1 6 2
4 1 3 5
t
s 3 s 1 t
1 3 0 2 3
Call the flow # 1
Call the graph ! 1 in, 1 out Call this graph !"
2 flow value is 3
2 2
12 6 2
4
10 2 6
t
s 3 s t
1 3 01 2 2
Call the flow # 1 1
Call the graph ! 1 in, 1 out Call this graph !"
4 5 3 1 5 2 2
5 2 1 3 0
s t
flow f in G
2
x=2
3 4 3
s t
path P in !"
That proves the claim
2
2 1
43 6 2
4 21
t 3 5
s 8 s 1 t
1 3 32 7 1
Call the flow $ 1 2
Call the graph ! Call this graph !"
Lemma: ⇒ now this direction ⇒
t is not reachable from s in !" ⇔ f is a max flow.
• Suppose there is not a path from s to t in !$.
• Consider the cut given by:
{things reachable from s} , {things not reachable from s}
2
2
4
4 6 2
4
2
t 5
s 8 s 1 t
1 3 3 7
Call the flow $ 1 3
Call the graph ! Call this graph !"
Lemma: ⇒ now this direction ⇒
t is not reachable from s in !" ⇔ f is a max flow.
• Suppose there is not a path from s to t in !$.
• Consider the cut given by:
t lives here
{things reachable from s} , {things not reachable from s}
• The flow from s to t is equal to the cost of this cut.
• Similar to proof-by-picture we saw before:
• All of the stuff has to cross the cut. 1
m a
• The edges in the cut are full because they don’t exist in !$ m
Le
• thus: this )low value = cost of this cut ≥ cost of min cut ≥ max )low
2
2
4
4 6 2
4
2 1 5
t
s 8 s t
1 3 3 7
Call the flow $ 1 3
Call the graph ! Call this graph !"
Lemma: ⇒ now this direction ⇒
t is not reachable from s in !" ⇔ f is a max flow.
• Suppose there is not a path from s to t in !$.
• Consider the cut given by:
t lives here
{things reachable from s} , {things not reachable from s}
• The flow from s to t is equal to the cost of this cut.
• Similar to proof-by-picture we saw before:
• All of the stuff has to cross the cut. 1
m a
• The edges in the cut are full because they don’t exist in !$ m
Le
• thus: this )low value = cost of this cut ≥ cost of min cut ≥ max )low
But this
2 flow value ≤ max flow value,
so
2 they must be equal!
4
4 6 2
4
2 1 5
t
s 8 s t
1 3 3 7
Call the flow $ 1 3
Call the graph ! Call this graph !"
We’ve proved:
• t is not reachable from s in !" ⇔ f is a max flow
3 1
4 5 3
1
s
2 t
4
5 2 3
1 4
1 5
5
Example of Ford-Fulkerson Now
have we
3 n ot h i
3 left to ng
4 d o!
4 1 3
4 6
s 2 2 3 4 4 t
3
5
3 1
6 4 4
5
10
3
1
4 5 3
1
s
2 t
3 4
5 2 4
1
1 5
5
Example of Ford-Fulkerson Now
have we
3 n ot h i
3 left to ng
4 d o!
4 1 3
4 6
s 2 2 3 4 4 t
3
5 Max flow and min
3 1 cut are both 11.
6 4 4
5
10
3
1
4 5 3
1
s
2 t
3 4
There’s no path 5 2 4
from s to t, and
1
1 5
here’s the cut to
prove it. 5
What have we learned?
• Max s-t flow is equal to min s-t cut!
• The USSR and the USA were trying to
solve the same problem…
• The Ford-Fulkerson algorithm can
find the min-cut/max-flow.
• Repeatedly improve your flow along
an augmenting path.
• How long does this take???
Why should we be concerned?
Suppose we just picked paths arbitrarily.
Choose a really
a
C big number C.
C
0
0
0 1
t
s
C
0 C 0
a
b
C
C
1 t
s
C
C
b
Why should we be concerned?
Suppose we just picked paths arbitrarily.
Choose a really
a
C big number C.
C
0
0
0 1
t
s
C
0 C 0
a
b
C
C
1 t
s
C
C
b
Why should we be concerned?
Suppose we just picked paths arbitrarily.
Choose a really
a
C big number C.
C
1
0
1 1
t
s
C
1 C 0
b
a 1
C
C-1
1
The edge (b,a) disappeared s 1 t
from the residual graph!
C
C-1 b
Why should we be concerned?
Suppose we just picked paths arbitrarily.
Choose a really
a
C big number C.
C
1
0
1 1
t
s
C
1 C 0
b
a 1
C
C-1
1
t
s 1
C
C-1 b
Why should we be concerned?
Suppose we just picked paths arbitrarily.
Choose a really
a
C big number C.
C
1
1
0 1
t
s
C
1 C 1
b
a 1
1
C-1 C-1
The edge (b,a) re-appeared 1 t
s 1 1
in the residual graph!
C-1
C-1 b
Why should we be concerned?
Suppose we just picked paths arbitrarily.
Choose a really
a
C big number C.
C
1
1
0 1
t
s
C
1 C 1
b
a 1
1
C-1 C-1
1 t
s 1 1
C-1
C-1 b
Why should we be concerned?
Suppose we just picked paths arbitrarily.
Choose a really
a
C big number C.
C
2
1
1 1
t
s
C
2 C 1
b
a 2
1
C-1 C-2
1
The edge (b,a) disappeared 2 t
s 1
from the residual graph!
C-1
C-2 b
Why should we be concerned?
Suppose we just picked paths arbitrarily.
Choose a really
a
C big number C.
C
2
1 This will go on for C steps,
1 1 adding flow along (b,a) and
t
s
C then subtracting it again.
2 C 1
b
a 2
1
C-1 C-2
1
The edge (b,a) disappeared 2 t
s 1
from the residual graph!
C-1
C-2 b
Doing Ford-Fulkerson with BFS is called
Theorem
the Edmonds-Karp algorithm.
• Basic idea:
• The number of times you remove an edge from the residual
graph is O(n).
• This is the hard part
• There are at most m edges.
• Each time we remove an edge we run BFS, which takes time
O(n+m).
• Actually, O(m), since we don’t need to explore the whole graph, just
the stuff reachable from s.
One more useful thing
• If all the capacities are integers, then the flows in
any max flow are also all integers.
• When we update flows in Ford-Fulkerson, we’re only
ever adding or subtracting integers.
• Since we started with 0 (an integer), everything stays
integral.
But wait, there’s more!
• Min-cut and max-flow are not just useful for the
USA and the USSR in 1955.
• An important algorithmic primitive!
• The Ford-Fulkerson algorithm is the basis for many
other graph algorithms.
• For the rest of today, we’ll see a few:
• Maximum bipartite matching
• Integer assignment problems
Some of the following material shamelessly stolen from Jeff Erickson’s excellent lecture notes:
http://jeffe.cs.illinois.edu/teaching/algorithms/2009/notes/17-maxflowapps.pdf
Maximum matching in bipartite graphs
s t
1
1
1
1
s 1 t
1
1
1
1
1
1
1 1 10
5
10 3
10
This person is vegan 2
and not that hungry;
they only want two
2 6
scoops of the sorbet.
Stanford Students Tubs of ice cream
Solution via max flow
1
3
6
4
3 3
3
1
s 1 10
t
1
10
5
3
2
10 6
2
1
1
3 4
4 3 6
4
3 3
3 3
3 3
1
s 1 10
1 1 3 t
1
7 10 1
5
1 3
2 6
2
2 10 6
2 6
W oop
1
e
sc
di s of
4
sh ic
4 3 3 6
ou e c
4
t 1 re
7 am
3 3
3 3
3 3
1
.
s 1 10
1 1 3 t
1
7 10 1
5 3
1
2 6
2
This student can 6
2 10
have flow at most No m
10 going in, and so 2 Che ore tha
rry G n
6
as s i 1 0
at most 10 going gne arcia c scoop
d to an b s of
out, so at most 10 this e
stud
ent.
scoops assigned.
As before, flows correspond to assignments, and
max flows correspond to max assignments.
What have we learned?
• Max flows and min cuts aren’t just for railway routing.
• Immediately, they apply to other sorts of routing too!
• But also they are useful for assigning items to Stanford
students!
Recap
• Today we talked about s-t cuts and s-t flows.
• The Min-Cut Max-Flow Theorem says that minimizing
the cost of cuts is the same as maximizing the value of
flows.
• The Ford-Fulkerson algorithm does this!
• Find an augmenting path
• Increase the flow along that path
• Repeat until you can’t find any more paths and then you’re
done!
• An important algorithmic primitive!
• eg, assignment problems.
Next time
• More recap
• A look back at the class