Filtering: a method for solving graph problems in map reduce

Filtering: A Method for Solving
Graph Problems in
MapReduce
Benjamin Moseley Silvio Lattanzi Siddharth Suri Sergei Vassilvitski
UIUC Google Yahoo! Yahoo!

Large Scale Distributed Computation, Jan 2012
Tuesday, January 17, 2012

Overview

• Introduction to MapReduce model

• Our settings

• Our results

• Open questions


Introduction to the MapReduce
model


XXL Data

• Huge amount of data

• Main problem is to analyze information quickly


XXL Data

• Huge amount of data

• Main problem is to analyze information quickly

• New tools

• Suitable efficient algorithms


MapReduce

• MapReduce is the platform of choice for processing
massive data
INPUT

MAP
PHASE

REDUCE
PHASE


MapReduce

massive data
INPUT

• Data are represented as tuple
< key, value >
MAP
PHASE

REDUCE
PHASE


MapReduce

massive data
INPUT

< key, value >
MAP
PHASE
• Mapper decides how data is
distributed
REDUCE
PHASE


MapReduce

massive data
INPUT

< key, value >
MAP
PHASE
• Mapper decides how data is
distributed
REDUCE

• Reducer performs non-trivial PHASE

computation locally


How can we model MapReduce?

PRAM

•No limit on the number of processors

•Memory is uniformly accessible from any
processor

•No limit on the memory available


How can we model MapReduce?

STREAMING

•Just one processor

•There is a limited amount of memory

•No parallelization


MapReduce model

[Karloff, Suri and Vassilvitskii]

• N is the input size and 0 is some fixed
constant


MapReduce model


constant
1−
•Less than N machines


MapReduce model


constant
1−
1−
•Less than N memory on each machine


MapReduce model


constant
1−
1−
•Less than N memory on each machine

i i
MRC : problem that can be solved in O(log N )
rounds

Combining map and reduce phase

•Mapper and Reducer work only on a subgraph




•Keep time in each round polynomial




•Keep time in each round polynomial

•Time is constrained by the number of rounds


Algorithmic challenges

•No machine can see the entire input




•No communication between machines during each
phase




•No communication between machines during each
phase

2−2
•Total memory is N


MapReduce vs MUD algorithms

•In MUD framework each reducer operates on a
stream of data.

•In MUD, each reducer is restricted to only using
polylogarithmic space.


Our settings


Our settings

•We study the Karloff, Suri and Vassilvitskii model


Our settings

•We study the Karloff, Suri and Vassilvitskii model

0
•We focus on class MRC


Our settings

•We assume to work with dense graph m = n, 1+c
for some constant c 0


Our settings

•We assume to work with dense graph m = n, 1+c
for some constant c 0

•Empirical evidences that social networks are dense
graphs
[Leskovec, Kleinberg and Faloutsos]


Dense graph motivation


•They study 9 different social networks





•They show that several graphs from different
1+c
domains have n edges





•They show that several graphs from different
1+c
domains have n edges

•Lowest value of c founded .08 and four graphs
have c .5


Our results


Results

Constant rounds algorithms for
•Maximal matching

•Minimum cut

•8-approx for maximum weighted matching

•2-approx for vertex cover

•3/2-approx for edge cover


Notation

• G = (V, E) : Input graph

• n: number of nodes
• m : number of edges
• η : memory available on each machine

• N : input size

Filtering

•Part of the input is dropped or filtered on the first
stage in parallel


Filtering

stage in parallel

•Next some computation is performed on the filtered
input


Filtering

stage in parallel

•Next some computation is performed on the filtered
input

•Finally some patchwork is done to ensure a proper
solution


Filtering


Filtering

FILTER


Filtering

FILTER

COMPUTE
SOLUTION


Filtering

FILTER

COMPUTE
SOLUTION

PATCHWORK
ON REST OF THE
GRAPH


Warm-up: Compute the minimum
spanning tree

m=n 1+c


spanning tree

m=n 1+c

PARTITION
EDGES
nc/2


spanning tree

m=n 1+c

PARTITION
EDGES
nc/2

n1+c/2 edges


spanning tree
n1+c/2 edges


spanning tree
n1+c/2 edges

COMPUTE
MST
nc/2


spanning tree
n1+c/2 edges

COMPUTE
MST
nc/2

SECOND
ROUND
nc/2

n1+c/2 edges


Show that the algorithm is in MRC0

•The algorithm is correct




•No edge in the final solution is discarded in partial
solution




solution

•The algorithm runs in two rounds




solution

c
•No more than O n 2 machines are used




solution

c
•No more than O n 2 machines are used

•No machine uses memory greater than O(n
1+c/2
)


Maximal matching

•Algorithmic difficulty is that each machine can only
see edges assigned to the machine


Maximal matching

•Algorithmic difficulty is that each machine can only
see edges assigned to the machine

•Is a partitioning based algorithm feasible?


Partitioning algorithm



PARTITIONING



PARTITIONING

COMBINE
MATCHINGS



PARTITIONING

COMBINE
MATCHINGS

It is impossible to build a maximal matching using only
red edges!!

Algorithmic insight

•Consider any subset of the edges E


Algorithmic insight


•Let M be a maximal matching on
G[E ]


Algorithmic insight


•Let M be a maximal matching on
G[E ]

•The unmatched vertices form a
independent set


Algorithmic insight

•We pick each edge with probability p


Algorithmic insight


•Find a matching on a sample and then find a
matching on unmatched vertices


Algorithmic insight



•For dense portions of the graph, many
edges should be sampled


Algorithmic insight



•For dense portions of the graph, many
edges should be sampled

•Sparse portions of the graph are small
and can be placed on a single machine


Algorithm

•Sample each edge independently with probability
10 log n
p=
nc/2


Algorithm

10 log n
p=
nc/2

•Find a matching on the sample


Algorithm

10 log n
p=
nc/2


•Consider the induced subgraph on unmatched
vertices


Algorithm

10 log n
p=
nc/2


•Consider the induced subgraph on unmatched
vertices

•Find a matching on this graph and output the union


Algorithm


Algorithm

SAMPLE


Algorithm

SAMPLE

FIRST
MATCHING


Algorithm

SAMPLE

FIRST
MATCHING

MATCHING ON
UNMATCHED NODES


Algorithm

SAMPLE

FIRST
MATCHING

MATCHING ON
UNMATCHED NODES

UNION


Correctness

•Consider the last step of the algorithm


Correctness


•All unmatched vertices are placed on a machine


Correctness


•All unmatched vertices are placed on a machine

•All unmatched vertices or are matched at the last
step or have only matched neighbors


Bounding the rounds

Three rounds:

•Sampling the edges


Bounding the rounds

Three rounds:


•Find a matching on a single machine for the sampled
edges


Bounding the rounds

Three rounds:


•Find a matching on a single machine for the sampled
edges

•Find a matching for the unmatched vertices


Bounding the memory

10 log n
•Each edge was sampled with probability p =
nc/2


Bounding the memory

10 log n
nc/2
˜ 1+c/2 )
•Using Chernoff the sampled graph has size O(n
with high probability


Bounding the memory

10 log n
nc/2
˜ 1+c/2 )
•Using Chernoff the sampled graph has size O(n
with high probability

•Can we bound the size of the induced subgraph on
the unmatched vertices?


Bounding the memory

1+c/2
•For a fixed induced subgraph with at least n
edges the probability an edge is not sampled is:
n1+c/2
10 log n
1− ≤ exp(−10n log n)
nc/2


Bounding the memory

1+c/2
•For a fixed induced subgraph with at least n
edges the probability an edge is not sampled is:
n1+c/2
10 log n
1− ≤ exp(−10n log n)
nc/2

•Union bounding over all 2 induced subgraphs shows n

that at least one edge is sampled from every dense
subgraph with probability
1
1 − exp(−10 log n) ≥ 1 − 10
n


Using less memory

˜ 1+c/2 )
•Can we run the algorithm with less then O(n
memory?


Using less memory

˜ 1+c/2 )
•Can we run the algorithm with less then O(n
memory?

•We can amplify our sampling technique!


Using less memory

•Sample as many edges as possible that fits in memory


Using less memory


•Find a matching


Using less memory


•Find a matching

•If the edges between unmatched vertices fit onto a
single machine, find a matching on those vertices


Using less memory


•Find a matching


•Otherwise, recurse on the unmatched nodes


Using less memory


•Find a matching


•Otherwise, recurse on the unmatched nodes

1+
•With n memory each iteration a factor of n edges
are removed resulting in O(c/) rounds

Parallel computation power

•Maximal matching algorithm does not use
parallelization



parallelization

•We use a single machine in every step



parallelization

•We use a single machine in every step

•When parallelization is used?


Maximum weighted matching
8

2 2
7
1

4
3
4 2
2


8

2 2
7
1

4
3
4 2
2

SPLIT THE GRAPH

8

2 2
7
1
4 3
2 4
2


8

2 2
7
1
4 3
2 4
2


8

2 2
7
1
4 3
2 4
2

MATCHINGS

2
7

3
2 4
2



2
7

3
2 4
2



2
7

3
2 4
2

SCAN THE EDGE
SEQUENTIALLY

2
7

3
4 2
2


Bounding the rounds and memory

Four rounds:

•Splitting the graph and running the maximal
matching:
˜ 1+c/2 ) memory
3 rounds and O(n

•Compute the final solution:
1 round and O(n log n) memory


Approximation guarantee (sketch)

•An edge in the solution can block at most 2 edges in
each subgraph of smaller weight
8

2
7

3
2 4
2


Approximation guarantee (sketch)

•An edge in the solution can block at most 2 edges in
each subgraph of smaller weight
8

2
7

3
2 4
2

•We loose a factor or 2 because we do not consider
the weight


Other algorithms

Based on the same intuition:

•2-approx for vertex cover

•3/2-approx for edge cover


Minimum cut

•Partition does not work, because we loose structural
informations


Minimum cut

informations

•Sampling does not seem to work either


Minimum cut

informations


•We can use the first steps of Karger algorithm as a
filtering technique


Minimum cut

informations


•We can use the first steps of Karger algorithm as a
filtering technique

•The random choices made in the early rounds
succeed with high probability, whereas the later rounds
have a much lower probability of success

Minimum cut

1


Minimum cut

RANDOM
WEIGHT
n δ1
3 1 5
1 2 3 2 2 3 1 1 3
4 4
4 3 4
4 3
1 3 2


Minimum cut

RANDOM
WEIGHT
n δ1
3 1 5
1 2 3 2 2 3 1 1 3
4 4
4 3 4
4 3
1 3 2

COMPRESS
EDGES WITH
SMALL WEIGTH

2
2 2
2
2 2 2


Minimum cut

RANDOM
WEIGHT
n δ1
3 1 5
1 2 3 2 2 3 1 1 3
4 4
4 3 4
4 3
1 3 2

COMPRESS
EDGES WITH
SMALL WEIGTH

2
2 2
2
2 2 2

From Karger at least one graph will contain a min cut w.h.p.

Minimum cut
2
2 2
2
2 2 2


Minimum cut
2
2 2
2
2 2 2

COMPUTE
MINIMUM
CUT

2
2 2
2
2 2 2

We will find the minimum cut w.h.p.


Empirical result (matching)
%#!

'()(**+*,*-.)/012
30)+(2/4-
%!!
!#$%'(%)#*+,'

$#!

$!!

#!

!
!!!$ !!!# !!$ !!% !!# !$ $
-.%/0'1234.4)0)*5'(/,'


Open problems


Open problems 1

•Maximum matching

•Shortest path

•Dynamic programming


Open problems 2

•Algorithms for sparse graph

•Does connected components require more than 2
rounds?

•Lower bounds


Thank you!


Filtering: a method for solving graph problems in map reduce

More Related Content

Similar to Filtering: a method for solving graph problems in map reduce (20)

Recently uploaded (20)

Filtering: a method for solving graph problems in map reduce