a

r
X
i
v
:
0
9
0
2
.
0
3
5
3
v
1


[
c
s
.
C
C
]


2

F
e
b

2
0
0
9
Non-monotone Submodular Maximization under
Matroid and Knapsack Constraints
Jon Lee

Vahab S. Mirrokni

Viswanath Nagarajan

Maxim Sviridenko

February 2, 2009
Abstract
Submodular function maximization is a central problem in combinatorial optimization, gener-
alizing many important problems including Max Cut in directed/undirected graphs and in hyper-
graphs, certain constraint satisfaction problems, maximum entropy sampling, and maximum facility
location problems. Unlike submodular minimization, submodular maximization is NP-hard. In this
paper, we give the rst constant-factor approximation algorithm for maximizing any non-negative
submodular function subject to multiple matroid or knapsack constraints. We emphasize that our
results are for non-monotone submodular functions. In particular, for any constant k, we present a
_
1
k+2+
1
k
+
_
-approximation for the submodular maximization problem under k matroid constraints,
and a
_
1
5

_
-approximation algorithm for this problem subject to k knapsack constraints ( > 0 is
any constant). We improve the approximation guarantee of our algorithm to
1
k+1+
1
k1
+
for k 2
partition matroid constraints. This idea also gives a
_
1
k+
_
-approximation for maximizing a mono-
tone submodular function subject to k 2 partition matroids, which improves over the previously
best known guarantee of
1
k+1
.

IBM T.J. Watson Research Center, [email protected]

Google Research, NYC [email protected]

Carnegie Mellon University, [email protected]

IBM T.J. Watson Research Center, [email protected]

1 Introduction
In this paper, we consider the problem of maximizing a nonnegative submodular function f, dened
on a ground set V , subject to matroid constraints or knapsack constraints. A function f : 2
V
R is
submodular if for all S, T V , f(S T) +f(S T) f(S) +f(T). Throughout, we assume that our
submodular function f is given by a value oracle; i.e., for a given set S V , an algorithm can query
an oracle to nd its value f(S). Furthermore, all submodular functions we deal with are assumed to be
non-negative. We also denote the ground set V = [n] = {1, 2, , n}.
We emphasize that our focus is on submodular functions that are not required to be monotone (i.e.,
we do not require that f(X) f(Y ) for X Y V ). Non-monotone submodular functions appear
in several places including cut functions in weighted directed or undirected graphs or even hypergraphs,
maximum facility location, maximum entropy sampling, and certain constraint satisfaction problems.
Given a weight vector w for the ground set V , and a knapsack of capacity C, the associated knapsack
constraint is that the sum of weights of elements in the solution S should not exceed the capacity C,
i.e,

jS
w
j
C. In our usage, we consider k knapsack constraints dened by weight vectors w
i
and
capacities C
i
, for i = 1, . . . , k.
We assume some familiarity with matroids [40] and associated algorithmics [45]. Briey, for a ma-
troid M, we denote the ground set of M by E(M), its set of independent sets by I(M), and its set of
bases by B(M). For a given matroid M, the associated matroid constraint is S I(M) and the asso-
ciated matroid base constraint is S B(M). In our usage, we deal with k matroids M
1
, . . . , M
k
on
the common ground set V := E(M
1
) = = E(M
k
) (which is also the ground set of our submodular
function f), and we let I
i
:= I(M
i
) for i = 1, . . . , k.
Background. Optimizing submodular functions is a central subject in operations research and com-
binatorial optimization [36]. This problem appears in many important optimization problems includ-
ing cuts in graphs [19, 41, 26], rank function of matroids [12, 16], set covering problems [13], plant
location problems [9, 10, 11, 2], and certain satisability problems [25, 14], and maximum entropy
sampling [32, 33]. Other than many heuristics that have been developed for optimizing these func-
tions [20, 21, 27, 43, 31], many exact and constant-factor approximation algorithms are also known for
this problem [38, 39, 44, 26, 15, 48, 18]. In some settings such as set covering or matroid optimization,
the relevant submodular functions are monotone. Here, we are more interested in the general case where
f(S) is not necessarily monotone.
Unlike submodular minimization [44, 26], submodular function maximization is NP-hard as it gen-
eralizes many NP-hard problems, like Max-Cut [19, 14] and maximum facility location [9, 10, 2]. Other
than generalizing combinatorial optimization problems like Max Cut [19], Max Directed Cut [4, 22],
hypergraph cut problems, maximum facility location [2, 9, 10], and certain restricted satisability prob-
lems [25, 14], maximizing non-monotone submodular functions have applications in a variety of prob-
lems, e.g, computing the core value of supermodular games [46], and optimal marketing for revenue
maximization over social networks [23]. As an example, we describe one important application in the
statistical design of experiments. The maximum entropy sampling problem is as follows: Let A be the
n-by-n covariance matrix of a set of Gaussian random variables indexed by [n]. For S [n], let A[S]
denote the principal submatrix of A indexed by S. It is well known that (up to constants depending on
|S|), log det A[S] is the entropy of the random variables indexed by S. Furthermore, log det A[S] is
submodular on [n]. In applications of locating environmental monitoring stations, it is desired to choose
s locations from [n] so as to maximize the entropy of the associated random variables, so that problem
is precisely one of maximizing a non-monotone submodular function subject to a cardinality constraint.
Of course a cardinality constraint is just a matroid base constraint for a uniform matroid. We note that
the entropy function is not even approximately monotone (see [30]). The maximum entropy sampling
problem has mostly been studied from a computational point of view, focusing on calculating optimal
solutions for moderate-sized instances (say n < 200) using mathematical programming methodolo-
1
gies (e.g, see [32, 33, 34, 29, 6, 5]), and our results provide the rst set of algorithms with provable
constant-factor approximation guarantee.
Recently, a
2
5
-approximation was developed for maximizing non-negative non-monotone submod-
ular functions without any side constraints [15]. This algorithm also provides a tight
1
2
-approximation
algorithm for maximizing a symmetric
1
submodular function [15]. However, the algorithms developed
in [15] for non-monotone submodular maximization do not handle any extra constraints.
For the problem of maximizing a monotone submodular function subject to a matroid or multiple
knapsack constraints, tight
_
1
1
e
_
-approximation are known [38, 7, 49, 47, 28]. Maximizing monotone
submodular functions over k matroid constraints was considered in [39], where a
_
1
k+1
_
-approximation
was obtained. This bound is currently the best known ratio, even in the special case of partition matroid
constraints. However, none of these results generalize to non-monotone submodular functions.
Better results are known either for specic submodular functions or for special classes of matroids.
A
1
k
-approximation algorithm using local search was designed in [42] for the problem of maximizing
a linear function subject to k matroid constraints. Constant factor approximation algorithms are known
for the problem of maximizing directed cut [1] or hypergraph cut [3] subject to a uniform matroid (i.e.
cardinality) constraint.
Hardness of approximation results are known even for the special case of maximizing a linear func-
tion subject to k partition matroid constraints. The best known lower bound is an (
k
log k
) hardness
of approximation [24]. Moreover, for the unconstrained maximization of non-monotone submodular
functions, it has been shown that achieving a factor better than
1
2
cannot be done using a subexponential
number of value queries [15].
Our Results. In this paper, we give the rst constant-factor approximation algorithms for maximiz-
ing a non-monotone submodular function subject to multiple matroid constraints, or multiple knapsack
constraints. More specically, we give the following new results (below > 0 is any constant).
(1) For every constant k 1, we present a
_
1
k+2+
1
k
+
_
-approximation algorithm for maximizing any
non-negative submodular function subject to k matroid constraints (Section 2). This implies a
_
1
4+
_
-
approximation algorithm for maximizing non-monotone submodular functions subject to a single ma-
troid constraint. Moreover, this algorithm is a
_
1
k+2+
_
-approximation in the case of symmetric submod-
ular functions. Asymptotically, this result is nearly best possible because there is an (
k
log k
) hardness
of approximation, even in the monotone case [24].
(2) For every constant k 1, we present a
_
1
5

_
-approximation algorithm for maximizing any non-
negative submodular function subject to a k-dimensional knapsack constraint (Section 3). To achieve
the approximation guarantee, we rst give a
_
1
4

_
-approximation algorithm for a fractional relaxation
(similar to the fractional relaxation used in [49]). We then use a simple randomized rounding tech-
nique to convert a fractional solution to an integral one. A similar method was recently used in [28] for
maximizing a monotone submodular function over knapsack constraints, but neither their algorithm for
the fractional relaxation, nor their rounding method is directly applicable to non-monotone submodular
functions.
(3) For submodular maximization under k 2 partition matroid constraints, we obtain improved ap-
proximation guarantees (Section 4). We give a
_
1
k+1+
1
k1
+
_
-approximation algorithm for maximizing
non-monotone submodular functions subject to k partition matroids. Moreover, our idea gives a
_
1
k+
_
-
approximation algorithm for maximizing a monotone submodular function subject to k 2 partition
matroid constraints. This is an improvement over the previously best known bound of
1
k+1
from [39].
1
The function f : 2
V
R is symmetric if for all S V , f(S) = f(V \ S). For example, cut functions in undirected
graphs are well-known examples of symmetric (non-monotone) submodular functions
2
(4) Finally, we study submodular maximization subject to a matroid base constraint in Appendix E. We
give a
_
1
3

_
-approximation in the case of symmetric submodular functions. Our result for general
submodular functions only holds for special matroids: we obtain a (
1
6
)-approximation when the ma-
troid contains two disjoint bases. In particular, this implies a
_
1
6

_
-approximation for the problem of
maximizing any non-negative submodular function subject to an exact cardinality constraint. Previously,
only special cases of directed cut [1] or hypergraph cut [3] subject to an exact cardinality constraint were
considered.
Our main technique for the above results is local search. Our local search algorithms are different
from the previously used variant of local search for unconstrained maximization of a non-negative sub-
modular function [15], or the local search algorithms used for Max Directed Cut [4, 22]. In the design
of our algorithms, we also use structural properties of matroids, a fractional relaxation of submodular
functions, and a randomized rounding technique.
2 Matroid Constraints
In this section, we give an approximation algorithm for submodular maximization subject to k matroid
constraints. The problem is as follows: Let f be a non-negative submodular function dened on ground
set V . Let M
1
, , M
k
be k arbitrary matroids on the common ground set V . For each matroid M
j
(with j [k]) we denote the set of its independent sets by I
j
. We consider the following problem:
max
_
f(S) : S
k
j=1
I
j
_
. (1)
We give an approximation algorithm for this problem using value queries to f that runs in time n
O(k)
.
The starting point is the following local search algorithm. Starting with S = , repeatedly perform one
of the following local improvements:
Delete operation. If e S such that f(S \ {e}) > f(S), then S S \ {e}.
Exchange operation. If d V \S and e
i
S{} (for 1 i k) are such that (S\{e
i
}){d}
I
i
for all i [k] and f((S \ {e
1
, , e
k
}) {d}) > f(S), then S (S \ {e
1
, , e
k
}) {d}.
When dealing with a single matroid constraint (k = 1), the local operations correspond to: delete an
element, add an element (i.e. an exchange when no element is dropped), swap a pair of elements (i.e.
an element from outside the current set is exchanged with an element from the set). With k 2 matroid
constraints, we permit more general exchange operations, involving adding one element and dropping
up to k elements.
Note that the size of any local neighborhood is at most n
k+1
, which implies that each local step can
be performed in polynomial time for a constant k. Let S denote a locally optimal solution. Next we prove
a key lemma for this local search algorithm, which is used in analyzing our algorithm. Before presenting
the lemma, we state a useful exchange property of matroids (see [45]). Intuitively, this property states
that for any two independent sets I and J, we can add any element of J to the set I, and kick out at most
one element from I while keeping the set independent. Moreover, each element of I is allowed to be
kicked out by at most one element of J. For completeness, a proof is given in Appendix A.
Theorem 1 Let M be a matroid and I, J I(M) be two independent sets. Then there is a mapping
: J \ I (I \ J) {} such that:
1. (I \ (b)) {b} I(M) for all b J \ I.
2. |
1
(e)| 1 for all e I \ J.
3
Lemma 2 For a local optimal solution S and any C
k
j=1
I
j
, (k+1)f(S) f(SC)+kf(SC).
Additionally for k = 1, if S I
1
is any locally optimal solution under only the swap operation, and
C I
1
with |S| = |C|, then 2 f(S) f(S C) +f(S C).
Proof: The following proof is due to Jan Vondr ak [50]. Our original proof [35] was more complicated
we thank Jan for letting us present this simplied proof.
For each matroid M
j
(j [k]), because both C, S I
j
are independent sets, Theorem 1 implies a
mapping
j
: C \ S (S \ C) {} such that:
1. (S \
j
(b)) {b} I
j
for all b C \ S.
2. |
1
j
(e)| 1 for all e S \ C.
When k = 1 and |S| = |C|, Corollary 39.12a from [45] implies the stronger condition that
1
: C\S
S \ C is in fact a bijection.
For each b C\S, let A
b
= {
1
(b),
k
(b)}. Note that (S\A
b
){b}
k
j=1
I
j
for all b C\S.
Hence (S \ A
b
) {b} is in the local neighborhood of S, and by local optimality under exchanges:
f(S) f ((S \ A
b
) {b}) , b C \ S. (2)
In the case k = 1 with |S| = |C|, these are only swap operations (because
1
is a bijection here).
By the property of mappings {
j
}
k
j=1
, each element i S \ C is contained in n
i
k of the sets
{A
b
| b C\S}; and elements of SC are contained in none of these sets. So the following inequalities
are implied by local optimality of S under deletions.
(k n
i
) f(S) (k n
i
) f(S \ {i}), i S \ C. (3)
Note that these inequalities are not required when k = 1 and |S| = |C| (then n
i
= k for all i S \ C).
For any b C \ S, we have (below, the rst inequality is submodularity and the second is from (2)):
f(S {b}) f(S) f ((S \ A
b
) {b}) f (S \ A
b
) f(S) f (S \ A
b
)
Adding this inequality over all b C \ S and using submodularity,
f(S C) f(S)

bC\S
[f(S {b}) f(S)]

bC\S
[f(S) f (S \ A
b
)]
Adding to this, the inequalities (3), i.e. 0 (k n
i
) [f(S) f(S \ {i})] for all i S \ C,
f(S C) f(S)

bC\S
[f(S) f (S \ A
b
)] +

iS\C
(k n
i
) [f(S) f(S \ {i})]
=

l=1
[f(S) f (S \ T
l
)] (4)
where = |C \ S| +

iS\C
(k n
i
) and {T
l
}

l=1
is some collection of subsets of S \ C such that
each i S \ C appears in exactly k of these subsets. Let s = |S| and |S C| = c; number the elements
of S as {1, 2, , s} = [s] such that S C = {1, 2, , c} = [c]. Then for any T S \ C, by
submodularity: f(S) f(S \ T)

pT
[f([p]) f([p 1])]. Using this in (4), we obtain:
f(SC)f(S)

l=1

pT
l
[f([p]) f([p 1])] = k
s

i=c+1
[f([i]) f([i 1])] = k(f(S) f(S C))
4
Approximate Local Search Procedure B:
Input: Ground set X of elements and value oracle access to submodular function f.
1. Let {v} be a singleton set with the maximum value f({v}) and let S = {v}.
2. While there exists the following delete or exchange local operation that increases the value of
f(S) by a factor of at least 1 +

n
4
, then apply the local operation and update S accordingly.
Delete operation on S. If e S such that f(S \ {e}) (1 +

n
4
)f(S), then S S \ {e}.
Exchange operation on S. If d X \ S and e
i
S {} (for 1 i k) are such that
(S \ {e
i
}) {d} I
i
for all i [k] and f((S \ {e
1
, , e
k
}) {d}) > (1 +

n
4
)f(S),
then S (S \ {e
1
, , e
k
}) {d}.
Figure 1: The approximate local search procedure.
AlgorithmA:
1. Set V
1
= V .
2. For i = 1, , k + 1, do:
(a) Apply the approximate local search procedure B on the ground set V
i
to obtain a solution
S
i
V
i
corresponding to the problem:
max{f(S) : S
k
j=1
I
j
, S V
i
}. (5)
(b) Set V
i+1
= V
i
\ S
i
.
3. Return the solution corresponding to max{f(S
1
), , f(S
k+1
)}.
Figure 2: Approximation algorithm for submodular maximization under k matroid constraints.
The second last equality follows from S \ C = {c + 1, , s} and the fact that each element of S \ C
appears in exactly k of the sets {T
l
}

l=1
. The last equality is due to a telescoping summation. Thus we
obtain (k + 1) f(S) f(S C) +k f(S C), giving the claim.
Observe that when k = 1 and |S| = |C|, we only used the inequalities (2) from the local search,
which are only swap operations. Hence in this case, the statement also holds for any solution S that
is locally optimal under only swap operations. In the general case, we use both inequalities (2) (from
exchange operations) and inequalities (3) (from deletion operations).
A simple consequence of Lemma 2 implies bounds analogous to known approximation factors [39,
42] in the cases when the submodular function f has additional structure.
Corollary 3 For a locally optimal solution S and any C
k
j=1
I
j
the following inequalities hold:
1. f(S) f(C)/(k + 1) if function f is monotone,
2. f(S) f(C)/k if function f is linear.
The local search algorithm dened above could run for an exponential amount of time until it reaches
a locally optimal solution. The standard approach is to consider an approximate local search. In Ap-
pendix A, we show an inequality (Lemma 16) analogous to Lemma 2 for approximate local optimum.
5
Theorem 4 Algorithm A in Figure 2 is a
_
1
(1+)(k+2+
1
k
)
_
-approximation algorithm for maximizing a
non-negative submodular function subject to any k matroid constraints, running in time n
O(k)
.
Proof: Bounding the running time of Algorithm A is easy and we leave it to Appendix A. Here, we
prove the performance guarantee of Algorithm A. Let C denote the optimal solution to the original
problem max{f(S) : S
k
j=1
I
j
, S V }. Let C
i
= C V
i
for each i [k + 1]; so C
1
= C.
Observe that C
i
is a feasible solution to the problem (5) solved in the ith iteration. Applying Lemma 16
to problem (5) using the local optimum S
i
and solution C
i
, we obtain:
(1 +)(k + 1) f(S
i
) f(S
i
C
i
) +k f(S
i
C
i
) 1 i k + 1, (6)
Using f(S) max
k+1
i=1
f(S
i
), we add these k + 1 inequalities and simplify inductively as follows.
Claim 5 For any 1 l k + 1, we have:
(1 +)(k + 1)
2
f(S) (l 1) f(C) +f(
l
p=1
S
p
C
1
) +
k+1

i=l+1
f(S
i
C
i
)
+
l1

p=1
(k l +p) f(S
p
C
p
) +k
k+1

i=l
f(S
i
C
i
).
Proof: We argue by induction on l. The base case l = 1 is trivial, by just considering the sum of the
k + 1 inequalities in statement (6) above. Assuming the statement for some value 1 l < k + 1, we
prove the corresponding statement for l + 1.
(1 +)(k + 1)
2
f(S) (l 1) f(C) +f(
l
p=1
S
p
C
1
)
+
k+1

i=l+1
f(S
i
C
i
) +
l1

p=1
(k l +p)f(S
p
C
p
) +k
k+1

i=l
f(S
i
C
i
)
= (l 1) f(C) +f(
l
p=1
S
p
C
1
) +f(S
l+1
C
l+1
)
+
k+1

i=l+2
f(S
i
C
i
) +
l1

p=1
(k l +p)f(S
p
C
p
) +k
k+1

i=l
f(S
i
C
i
)
(l 1) f(C) +f(
l+1
p=1
S
p
C
1
) +f(C
l+1
)
+
k+1

i=l+2
f(S
i
C
i
) +
l1

p=1
(k l +p)f(S
p
C
p
) +k
k+1

i=l
f(S
i
C
i
)
= (l 1) f(C) +f(
l+1
p=1
S
p
C
1
) +f(C
l+1
) +
l

p=1
f(S
p
C
p
)
+
k+1

i=l+2
f(S
i
C
i
) +
l

p=1
(k l 1 +p)f(S
p
C
p
) +k
k+1

i=l+1
f(S
i
C
i
)
l f(C) +f(
l+1
p=1
S
p
C
1
)
+
k+1

i=l+2
f(S
i
C
i
) +
l

p=1
(k l 1 +p)f(S
p
C
p
) +k
k+1

i=l+1
f(S
i
C
i
).
The rst inequality is the induction hypothesis, and the next two inequalities follow from submodu-
larity using
_

l
p=1
S
p
C
p
_
C
l+1
= C.
6
Using the statement of Claim 5 when l = k + 1, we obtain (1 +)(k + 1)
2
f(S) k f(C).
Finally, we give an improved approximation algorithm for symmetric submodular functions f, that
satisfy f(S) = f(S) for all S V . Symmetric submodular functions have been considered widely
in the literature [17, 41], and it appears that symmetry allows for better approximation results and thus
deserves separate attention.
Theorem 6 There is a
_
1
(1+)(k+2)
_
-approximation algorithm for maximizing a non-negative symmetric
submodular functions subject to k matroid constraints.
Proof: The algorithm for symmetric submodular functions is simpler. In this case, we only need to
perform one iteration of the approximate local search procedure B (as opposed to k + 1 in Theorem 4).
Let C denote the optimal solution, and S
1
the result of the local search (on V ). Then Lemma 2 implies:
(1 +)(k + 1) f(S
1
) f(S
1
C) +k f(S
1
C) f(S
1
C) +f(S
1
C).
Because f is symmetric, we also have f(S
1
) = f(S
1
). Adding these two,
(1 +)(k + 2) f(S
1
) f(S
1
) +f(S
1
C) +f(S
1
C) f(C \ S
1
) +f(S
1
C) f(C) .
Thus we have the desired approximation guarantee.
3 Knapsack constraints
Let f : 2
V
R
+
be a submodular function, and w
1
, , w
k
be k weight-vectors corresponding to
knapsacks having capacities C
1
, , C
k
respectively. The problem we consider in this section is:
max{f(S) :

jS
w
i
j
C
i
, 1 i k, S V }. (7)
By scaling each knapsack, we assume that C
i
= 1 for all i [k]. We denote f
max
= max{f(v) : v
V }. We assume without loss of generality that for every i V , the singleton solution {i} is feasible
for all the knapsacks (otherwise such elements can be dropped from consideration). To solve the above
problem, we rst dene a fractional relaxation of the submodular function, and give an approximation
algorithm for this fractional relaxation. Then, we show how to design an approximation algorithm for
the original integral problem using the solution for the fractional relaxation. Let F : [0, 1]
n
R
+
, the
fractional relaxation of f, be the extension-by-expectation of f, i.e.,
F(x) =

SV
f(S)
iS
x
i

jS
(1 x
j
).
Note that F is a multi-linear polynomial in variables x
1
, , x
n
, and has continuous derivatives of all
orders. Furthermore, as shown in Vondr ak [49], for all i, j V ,

2
x
j
x
i
F 0 everywhere on [0, 1]
n
; we
refer to this condition as continuous submodularity.
Due to lack of space, many proofs of this section appear in Appendix B.
Extending function f on scaled ground sets. Let s
i
Z
+
be arbitrary values for each i V .
Dene a new ground-set U that contains s
i
copies of each element i V ; so the total number of
elements in U is

iV
s
i
. We will denote any subset T of U as T =
iV
T
i
where each T
i
consists of all
copies of element i V from T. Now dene function g : 2
U
R
+
as g(
iV
T
i
) = F( ,
|T
i
|
s
i
, ).
Our goal is to prove the useful lemma that g is submodular. In preparation for that, we rst establish
a couple of claims. The rst claim is standard, but we give a proof for the sake of completeness.
7
Claim 7 Suppose l : D R has continuous partial derivatives everywhere on convex D R
n
with
l
x
i
(y) 0 for all y D and i V . Then for any a
1
, a
2
D with a
1
a
2
coordinate-wise, we have
l(a
1
) l(a
2
).
Next, we establish the following property of the fractional relaxation F.
Claim 8 For any a, q, d [0, 1]
n
with q +d [0, 1]
n
and a q coordinate-wise, we have F(a +d)
F(a) F(q +d) F(q).
Using the above claims, we are now ready to state and prove the lemma.
Lemma 9 Set function g is a submodular function on ground set U.
Solving the fractional relaxation. We now argue how to obtain a near-optimal fractional fea-
sible solution for maximizing a non-negative submodular function over k knapsack constraints. Let
w
1
, , w
k
denote the weight-vectors in each of the k knapsacks such that all knapsacks have capacity
1. The problem we consider here has additional upper bounds {u
i
[0, 1]}
n
i=1
on variables:
max{F(y) : w
s
y 1 s [k], 0 y
i
u
i
i V }. (8)
Denote the region U := {y : 0 y
i
u
i
i V }. We dene a local search procedure to solve
problem (8). We only consider values for each variable from a discrete set of values in [0, 1], namely
G = {p : p N, 0 p
1

} where =
1
8n
4
. Let > 0 be a parameter to be xed later. At any
step with current solution y [0, 1]
n
, the following local moves are considered:
Let A, D [n] with |A|, |D| k. Decrease the variables y(D) to any values in G and increase
variables y(A) to any values in G such that the resulting solution y

still satises all knapsacks and

y

U. If F(y

) > (1 +) F(y) then set y y

.
Note that the size of each local neighborhood is n
O(k)
. Let a be the index corresponding to max{u
i

f({i}) : i V }. We start the local search procedure with the solution y
0
having y
0
(a) = u
a
and zero
otherwise. Observe that for any x U
n
, F(x)

n
i=1
u
i
f({i}) n u
a
f({a}) = n F(y
0
). Hence
the number of iterations of local search is O(
1

log n), and the entire procedure terminates in polynomial

time. Let y denote a local optimal solution. We rst prove the following based on the discretization G.
Claim 10 Suppose , [0, 1]
n
are such that each has at most k positive coordinates, y

:= y+
U, and y

satises all knapsacks. Then F(y

) (1 +) F(y) +
1
4n
2
f
max
.
For any x, y R
n
, we dene xy and xy by (xy)
j
:= max(x
j
, y
j
) and (xy)
j
:= min(x
j
, y
j
)
for j [n].
Lemma 11 For local optimal y U G
n
and any x U satisfying the knapsack constraints, we have
(2 + 2n ) F(y) F(y x) +F(y x)
1
2n
f
max
.
Proof: For the sake of this proof, we assume that each knapsack s [k] has a dummy element (which
has no effect on function f) of weight 1 in knapsack s (and zero in all other knapsacks), and upper-bound
of 1. So any fractional solution can be augmented to another of the same F-value, while satisfying all
knapsacks at equality. We augment y and x using dummy elements so that both satisfy all knapsacks at
equality: this does not change any of the values F(y), F(y x) and F(y x). Let y

= y (y x) and
x

= x (y x). Note that for all s [k], w

s
y

= w
s
x

and let c
s
= w
s
x

. We will decompose y

and x

into an equal number of terms as y

t
and x

t
such that the s and s have small
support, and w
s

t
= w
s

t
for all t and s [k].
1. Initialize t 1, 1, x

, y

.
2. While > 0, do:
8
(a) Consider LP
x
:= {z 0 : z w
s
= c
s
, s [k]} where the variables are restricted to indices
i [n] with x

i
> 0. Similarly LP
y
:= {z 0 : z w
s
= c
s
, s [k]} where the variables are
restricted to indices i [n] with y

i
> 0. Let u LP
x
and v LP
y
be extreme points.
(b) Set
1
= max{ : u x

}
2
= max{ : v y

}, and = min{
1
,
2
}.
(c) Set
t
u,
t
v, , x

t
, and y

t
.
(d) Set t t + 1.
We rst show that this procedure is well-dened. In every iteration, > 0, and by induction
w
s
x

= w
s
y

= c
s
for all s [k]. Thus in step 2a, LP
x
(resp. LP
y
) is non-empty: x

/ (resp.
y

/) is a feasible solution. From the denition of LP

x
and LP
y
it also follows that > 0 in step 2b and
at least one coordinate of x

or y

is zeroed out in step 2c. This implies that the decomposition procedure
terminates in r 2n steps. At the end of the procedure, we have decompositions x

r
t=1

t
and
y

r
t=1

t
. Furthermore, each
t
(resp.
t
) corresponds to an extreme point of LP
y
(resp. LP
x
)
in some iteration: hence the number of positive components in any of {
t
,
t
}
r
t=1
is at most k. Finally
note that for all t [r], w
s

t
= w
s

t
for all knapsacks s [k].
Observe that Claim 10 applies to y,
t
and
t
(any t [r]) because each of
t
,
t
has support-size
k, and y
t
+
t
U and satises all knapsacks with equality. Strictly speaking, Claim 10 requires the
original local optimal y, which is not augmented with dummy elements. However even y
t
+
t
U
and satises all knapsacks (possibly not at equality), and the claim does apply. This gives:
F(y
t
+
t
) (1 +) F(y) +
f
max
4n
2
t [r] . (9)
Let M Z
+
be large enough so that M
t
and M
t
are integral for all t [r]. In the rest of the
proof, we consider a scaled ground-set U containing M copies of each element in V . We dene function
g : 2
U
R
+
as g(
iV
T
i
) = F( ,
|T
i
|
M
, ) where each T
i
consists of copies of element i V .
Lemma 9 implies that g is submodular. Corresponding to y we have a set P =
iV
P
i
consisting of the
rst |P
i
| = M y
i
copies of each element i V . Similarly, x corresponds to set Q =
iV
Q
i
consisting
of the rst |Q
i
| = M x
i
copies of each element i V . Hence P Q (resp. P Q) corresponds to
x y (resp. x y) scaled by M. Again, P \ Q (resp. Q\ P) corresponds to y

(resp. x

) scaled by M.
The decomposition of y

from above suggests disjoint sets {A

t
}
r
t=1
such that
t
A
t
= P \ Q; i.e. each
A
t
corresponds to
t
scaled by M. Similarly there are disjoint sets {B
t
}
r
t=1
such that
t
B
t
= Q \ P.
Observe also that g((P \ A
t
) B
t
) = F(y
t
+
t
), so (9) corresponds to:
g((P \ A
t
) B
t
) (1 +) g(P) +
f
max
4n
2
t [r] . (10)
Adding all these r inequalities to g(P) = g(P), we obtain (r + r + 1)g(P) +
r
4n
2
f
max
g(P) +

r
t=1
g((P \A
t
)B
t
). Using submodularity of g and the disjointness of families {A
t
}
r
t=1
and {B
t
}
r
t=1
,
we obtain (r + r + 1) g(P) +
r
4n
2
f
max
(r 1) g(P) + g(P Q) + g(P Q). Hence
(2 + r) g(P) g(P Q) +g(P Q)
r
4n
2
f
max
. This implies the lemma because r 2n.
Theorem 12 For any constant > 0, there exists a (
1
4
)-approximation algorithm for problem (8)
with all upper bounds u
i
= 1 (for all i V ).
Rounding the fractional knapsack. In order to solve the (non-fractional) submodular maximization
subject to k knapsack constraints, we partition the elements into two subsets. For a constant parameter
, we say that element e V is heavy if w
s
(e) for some knapsack s [k]. All other elements
are called light. Since the number of heavy elements in any feasible solution is bounded by
k

, we can
enumerate over all possible sets of heavy elements and obtain the optimal prot for these elements. For
light elements, we show a simple randomized rounding procedure (applied to the fractional solution
from (8)) that gives a (
1
4
)-approximation for the light elements. Combining the enumeration method
with the randomized rounding method, we get a (
1
5
)-approximation for submodular maximization
subject to k knapsack constraints. The details and proofs of this result are left to Appendix C.
9
4 Improved Bounds under Partition Matroids
The improved algorithm for partition matroids is again based on local search. In the exchange local
move of the general case (Section 2), the algorithm only attempts to include one new element at a time
(while dropping upto k elements). Here we generalize that step to allow including p new elements
while dropping up to (k 1) p elements, for some xed constant p 1. We show that this yields an
improvement under partition matroid constraints. Given a current solution S
k
j=1
I
j
, the local moves
we consider are:
Delete operation. If e S such that f(S \ {e}) > f(S), then S S \ {e}.
Exchange operation. For some q p, if d
1
, , d
q
V \ S and e
i
S {} (for 1 i
(k 1) q) are such that: (i) S

= (S \ {e
i
: 1 i (k 1)q}) {d
1
, , d
q
} I
j
for all
j [k], and (ii) f(S

) > f(S), then S S

.
The main idea here is a strengthening of Lemma 2. Missing proofs of this section are in Appendix D.
Lemma 13 For a local optimal solution S and any C
k
j=1
I
j
, we have k f(S) (1
1
p
) f(S
C) + (k 1) f(S C).
Proof: We use an exchange property (see Schrijver [45]), which implies for any partition matroid M
and C, S I(M) the existence of a map : C \ S (S \ C) {} such that
1. (S \ {(b) : b T}) T I(M) for all T C \ S.
2. |
1
(e)| 1 for all e S \ C.
Let
j
denote the mapping under partition matroid M
j
(for 1 j k).
Combining partition matroids M
1
and M
2
. We use
1
and
2
to construct a multigraph G on vertex
set C \ S and edge-set labeled by E =
1
(C \ S)
2
(C \ S) S \ C with an edge labeled a E
between e, f C \ S iff
1
(e) =
2
(f) = a or
2
(e) =
1
(f) = a. Each edge in G has a unique
label because there is exactly one edge (e, f) corresponding to any a E. Note that the maximum
degree in G is at most 2. Hence G is a union of disjoint cycles and paths. We index elements of C \ S
in such a way that elements along any path or cycle in G are consecutive. For any q {0, , p 1},
let R
q
denote the elements of C \ S having an index that is not q modulo p. It is clear that the induced
graph G[R
q
] for any q [p] consists of disjoint paths/cycles, each of length at most p. Furthermore each
element of C \ S appears in exactly p 1 sets among {R
q
}
p1
q=0
.
Claim 14 For any q {0, , p 1}, k f(S) f(S R
q
) + (k 1) f(S C).
Adding the p inequalities given by Claim 14, we get pkf(S)

p1
q=0
f(SR
q
)+p(k1)f(SC).
Note that each element of C\S is missing in exactly 1 set {SR
q
}
p1
q=0
, and elements of SC are missing
in none of them. Hence an identical simplication as in Lemma 2 gives

p1
q=0
[f(SC)f(SR
q
)]
f(S C) f(S). Thus,
(pk 1) f(S) (p 1) f(S C) +p(k 1) f(S C),
which implies k f(S) (1
1
p
) f(S C) + (k 1) f(S C), giving the lemma.
Theorem 15 For any k 2 and xed constant > 0, there exists a
1
k+1+
1
k1
+
-approximation al-
gorithm for maximizing a non-negative submodular function over k partition matroids. This bound
improves to
1
k+
for monotone submodular functions.
Acknowledgment: The proof of Lemma 2 presented in this paper is due to Jan Vondr ak. Our original
proof [35] was more complicated we thank Jan for letting us present this simplied proof.
10
References
[1] A. Ageev, R. Hassin and M. Sviridenko, An 0.5-approximation algorithm for MAX DICUT with given sizes
of parts. SIAM J. Discrete Math. 14 (2001), no. 2, 246255 (electronic).
[2] A. Ageev and M. Sviridenko. An 0.828 Approximation algorithm for the uncapacitated facility location prob-
lem, Discrete Applied Mathematics 93(2-3): 149156 (1999).
[3] A. Ageev and M. Sviridenko, Pipage rounding: a new method of constructing algorithms with proven perfor-
mance guarantee. J. Comb. Optim. 8 (2004), no. 3, 307328.
[4] P. Alimonti. Non-oblivious local search for MAX 2-CCSP with application to MAX DICUT, In Proceedings
of the 23rd International Workshop on Graph-theoretic Concepts in Computer Science, 1997.
[5] K. M. Anstreicher, M. Fampa, J. Lee and J. Williams. Using continuous nonlinear relaxations to solve con-
strained maximum-entropy sampling problems. Mathematical Programming, Series A, 85:221-240, 1999.
[6] S. Burer and J. Lee. Solving maximum-entropy sampling problems using factored masks. Mathematical Pro-
gramming, Volume 109, Numbers 2-3, 263-281, 2007
[7] G. Calinescu, C. Chekuri, M. P al and J. Vondr ak. Maximizing a monotone submodular function under a
matroid constraint, IPCO 2007.
[8] V. Cherenin. Solving some combinatorial problems of optimal planning by the method of successive calcula-
tions, Proc. of the Conference of Experiences and Perspectives of the Applications of Mathematical Methods
and Electronic Computers in Planning (in Russian), Mimeograph, Novosibirsk (1962).
[9] G. Cornu ejols, M. Fischer and G. Nemhauser. Location of bank accounts to optimize oat: An analytic study
of exact and approximation algorithms, Management Science, 23 (1977), 789810.
[10] G. Cornu ejols, M. Fischer and G. Nemhauser. On the uncapacitated location problem, Annals of Discrete
Math 1 (1977), 163178.
[11] G. P. Cornu ejols, G. L. Nemhauser and L. A. Wolsey. The uncapacitated facility location problem. In Discrete
Location Theory (1990), 119171.
[12] J. Edmonds. Matroids, submodular functions, and certain polyhedra, Combinatorial Structures and Their
Applications (1970), 6987.
[13] U. Feige. A threshold of ln n for approximating set cover. Journal of ACM 45 (1998), 634652.
[14] U. Feige and M. X. Goemans. Approximating the value of two-prover systems, with applications to MAX-
2SAT and MAX-DICUT. Proc. of the 3rd Israel Symposium on Theory and Computing Systems, Tel Aviv
(1995), 182189.
[15] U. Feige, V. Mirrokni and J. Vondr ak. Maximizing non-monotone submodular functions, FOCS 2007.
[16] A. Frank. Matroids and submodular functions, Annotated Biblographies in Combinatorial Optimization
(1997), 6580.
[17] S. Fujishige. Canonical decompositions of symmetric submodular systems, Discrete Applied Mathematics
5 (1983), 175190.
[18] M. Goemans, N. Harvey, S. Iwata, V. Mirrokni. Approximating submodular functions everywhere. In SODA
2009.
[19] M. X. Goemans and D. P. Williamson. Improved approximation algorithms for maximum cut and satisa-
bility problems using semidenite programming, Journal of ACM 42 (1995), 11151145.
[20] B. Goldengorin, G. Sierksma, G. Tijsssen and M. Tso. The data correcting algorithm for the minimization
of supermodular functions, Management Science, 45:11 (1999), 15391551.
[21] B. Goldengorin, G. Tijsssen and M. Tso. The maximization of submodular Functions: Old and new proofs
for the correctness of the dichotomy algorithm, SOM Report, University of Groningen (1999).
[22] E. Halperin and U. Zwick. Combinatorial approximation algorithms for the maximum directed cut problem.
Proc. of 12th SODA (2001), 17.
11
[23] J. Hartline, V. Mirrokni and M. Sundararajan. Optimal marketing strategies over social networks, World
Wide Web Conference (WWW), 2008, 189198.
[24] E. Hazan, S. Safra and O. Schwartz. On the complexity of approximating k-set packing. Computational
Complexity, 15(1), 2039, 2006.
[25] J. H astad. Some optimal inapproximability results. Journal of ACM 48 (2001): 798869.
[26] S. Iwata, L. Fleischer and S. Fujishige. A combinatorial, strongly polynomial-time algorithmfor minimizing
submodular functions, Journal of ACM 48:4 (2001), 761777.
[27] V. R. Khachaturov, Mathematical Methods of Regional Programming, Nauka, Moscow (in Russian), 1989.
[28] A. Kulik, H. Shachnai and T. Tamir. Maximizing submodular functions subject to multiple linear constraints.
Proc. of SODA, 2009.
[29] C.-W. Ko, J. Lee and M. Queyranne. An exact algorithm for maximum entropy sampling. Operations Re-
search 43(4):684-691, 1995.
[30] A. Krause, A. Singh and C. Guestrin. Near-optimal sensor placements in Gaussian processes: Theory, ef-
cient algorithms and empirical studies, Journal of Machine Learning Research 9 (2008) 235284.
[31] H. Lee, G. Nemhauser and Y. Wang. Maximizing a submodular function by integer programming: Polyhedral
results for the quadratic case, European Journal of Operational Research 94, 154166.
[32] J. Lee. Maximum entropy sampling. In: A.H. El-Shaarawi and W.W. Piegorsch, editors, Encyclopedia of
Environmetrics. Wiley, 2001.
[33] J. Lee. Semidenite programming in experimental design. In: H. Wolkowicz, R. Saigal and L. Vanden-
berghe, editors, Handbook of Semidenite Programming, International Series in Operations Research and
Management Science, Vol. 27, Kluwer, 2000.
[34] J. Lee. Constrained maximum-entropy sampling. Operations Research, 46:655-664, 1998.
[35] J. Lee, V. Mirrokni, V. Nagarajan and M. Sviridenko. Maximizing Non-Monotone Submodular Functions
under Matroid and Knapsack Constraints. IBM Research Report RC24679. 2008.
[36] L. Lov asz. Submodular functions and convexity. In: A. Bachem et. al., eds, Mathematical Programmming:
The State of the Art, 235257.
[37] R. Motwani and P. Raghavan. Randomized Algorithms, Cambridge University Press, 1995.
[38] G. L. Nemhauser, L. A. Wolsey and M. L. Fisher. An analysis of approximations for maximizing submodular
set functions I. Mathematical Programming 14 (1978), 265294.
[39] G. L. Nemhauser, L. A. Wolsey and M. L. Fisher. An analysis of approximations for maximizing submodular
set functions II. Mathematical Programming Study 8 (1978), 7387.
[40] J. G. Oxley, Matroid theory, Oxford Science Publications. The Clarendon Press, Oxford University Press,
New York, 1992.
[41] M. Queyranne. A combinatorial algorithm for minimizing symmetric submodular functions, ACM-SIAM
Symposium on Discrete Algorithms (1995), 98101.
[42] J. Reichel and M. Skutella, Evolutionary algorithms and matroid optimization problems, in Proceedings of
the 9th Genetic and Evolutionary Computation Conference (GECCO07), 947954, 2007.
[43] T. Robertazzi and S. Schwartz, An accelated sequential algorithm for producing D-optimal designs. SIAM
Journal on Scientic and Statistical Computing 10, 341359.
[44] A. Schrijver. A combinatorial algorithm minimizing submodular functions in strongly polynomial time,
Journal of Combinatorial Theory, Series B 80 (2000), 346355.
[45] A. Schrijver. Combinatorial Optimization, Volumes A-C. Springer-Verlag, Berlin, 2003.
[46] A. Schulz, N. Uhan, Encouraging Cooperation in Sharing Supermodular Costs, APPROX-RANDOM 2007:
271285
12
[47] M. Sviridenko. A note on maximizing a submodular set function subject to knapsack constraint. Operations
Research Letters 32 (2004), 4143.
[48] Z. Svitkina and L. Fleischer. Submodular approximation: Sampling-based algorithms and lower bounds. In
FOCS 2008.
[49] J. Vondr ak. Optimal approximation for the submodular welfare problem in the value oracle model. In STOC,
2008.
[50] J. Vondr ak, Personal communication, 2008.
A Missing proofs from Section 2
Proof of Theorem 1: We proceed by induction on t = |J \ I|. If t = 0, there is nothing to prove.
Suppose there is an element b J \ I with I {b} I(M). In this case we apply induction on
I and J

= J \ {b} (where |J

\ I| = t 1 < t). Because I \ J

= I \ J, we obtain a map

: J

\ I (I \ J) {} satisfying the two conditions. The desired map is then (b) = and
(b

) =

(b

) for all b

J \ I \ {b} = J

\ I.
Now we may assume that I is a maximal independent set in I J. Let M

Mdenote the matroid

M truncated to I J; so I is a base in M

. We augment J to some base

J J in M

(because any
maximal independent set in M

is a base). Then we have two bases I and

J in M

. Theorem 39.12
from [45] implies the existence of elements b

J \ I and e I \

J such that both (

J \ b) {e} and
(I \ e) {b} are bases in M

. Note that J

= (J \ {b}) {e} (

J \ {b}) {e} M. By induction on
I and J

(because |J

\I| = t 1 < t) we obtain map

: J

\I I \J

satisfying the two conditions.

The map is then (b) = e and (b

) =

(b

) for all b

J \ I \ {b} = J

\ I. The rst condition on

is satised by induction (for elements J \ I \ {b}) and because (I \ e) {b} M (see above). The
second condition on is satised by induction and the fact that e I \ J

.
Lemma 16 For an approximately locally optimal solution S (in procedure B) and any C
k
j=1
I
j
,
(1 + )(k + 1) f(S) f(S C) + k f(S C) where > 0 a parameter dened in the algorithm
description. Additionally for k = 1, if S I
1
is any locally optimal solution under only the swap
operation, and C I
1
with |S| = |C|, then 2(1 +) f(S) f(S C) +f(S C).
Proof: The proof of this lemma is almost identical to the proof of the Lemma 2 the only difference
is that left-hand sides of inequalities (2) and inequalities (3) are multiplied by 1 +

n
4
. Therefore, after
following the steps in Lemma 2, we obtain the inequality:
(k + 1 +

n
4
) f(S) f(S C) +k f(S C).
Since (k + 1)n (see Lemma 2) and we may assume that n
4
>> (k + 1)n, we obtain the lemma.
Running time of Algorithm A(Theorem 4) Here we describe a missing part of the proof of Theorem 4
about the running of Algorithm A. The parameter > 0 in Procedure B is any value such that
1

is
at most a polynomial in n. Note that using approximate local operations in the local search procedure
B (in Figure 1) makes the running time of the algorithm polynomial. The reason is as follows: one
can easily show that for any ground set X of elements, the value of the initial set S = {v} is at least
Opt(X)/n, where Opt(X) is the optimal value of problem (1) restricted to X. Each local operation
in procedure B increases the value of the function by a factor 1 +

n
4
. Therefore, the number of local
operations for procedure B is at most log
1+

n
4
Opt(X)
Opt(X)
n
= O(
1

n
4
log n), and thus the running time of the
whole procedure is
1

n
O(k)
. Moreover, the number of procedure calls of Algorithm A for procedure B
is polynomial, and thus the running time of Algorithm A is also polynomial.
13
B Missing Proofs from Section 3
Proof of Claim 7: Consider the line froma
1
to a
2
parameterized by t [0, 1] as y(t) := a
1
+t(a
2
a
1
).
Observe that all points on this line are in D (because D is a convex set). At any t [0, 1], we have:
l(y(t))
t
=
n

j=1
l(y(t))
x
j

y
j
(t)
t
=
n

j=1
l(y(t))
x
j
(a
2
(j) a
1
(j)) 0.
Above, the rst equality follows from the chain rule because l is differentiable, and the last inequality
uses the fact that a
2
a
1
0 coordinate-wise. This completes the proof of the claim.
Proof of Claim 8: Let D = {y [0, 1]
n
: y + d [0, 1]
n
}. Dene function h : D R
+
as
h(x) := F(x + d) F(x), which is a multi-linear polynomial. We will show that
h
x
i
() 0 for all
i V , at every point D. This combined with Claim 7 below would imply h(a) h(q) because
a q coordinate-wise, which gives the claim.
In the following, x an i V and denote F

i
(y) =
F
x
i
(y) for any y [0, 1]
n
. To show
h
x
i
() 0
for D, it sufces to have F

i
( + d) F

i
() 0. From the continuous submodularity of F, for
every j V we have
F

i
x
j
(y) =

2
F
x
j
x
i
(y) 0 for all y [0, 1]
n
. Then applying Claim 7 to F

i
(a
multi-linear polynomial) implies that F

i
( +d) F

i
() 0. This completes the proof of Claim 8.
Proof of Lemma 9: To show submodularity of g, consider any two subsets P =
iV
P
i
and Q =

iV
Q
i
of U, where each P
i
(resp., Q
i
) are copies of element i V . We have P Q =
iV
(P
i
Q
i
)
and P Q =
iV
(P
i
Q
i
). Dene vectors p, q, a, b [0, 1]
n
as follows:
p
i
=
|P
i
|
s
i
, q
i
=
|Q
i
|
s
i
, a
i
=
|P
i
Q
i
|
s
i
, b
i
=
|P
i
Q
i
|
s
i
i V.
It is clear that p + q = a + b and d := p a 0. Submodularity condition on g at P, Q requires
g(P)+g(Q) g(P Q)+g(P Q). But by the denition of g, this is equivalent to F(a+d)F(a)
F(q +d) F(q), which is true by Claim 8. Thus we have established the lemma.
Proof of Claim 10: Let z y

be the point in U G
n
that minimizes

n
i=1
(y

i
z
i
). Note that z is a
feasible local move from y: it lies in G
n
, satises all knapsacks and the upper-bounds, and is obtainable
fromy by reducing k variables and increasing k others. Hence by local optimality F(z) (1+)F(y).
By the choice of z, it follows that |z
i
y

i
| for all i V . Suppose B is an upper bound on all
rst partial derivatives of function F on U: i.e.

F(x)
x
i
|
x

B for all i V and x U. Then because

F has continuous derivatives, we obtain
|F(z) F(y

)|
n

i=1
B |z
i
y

i
| nB 2n
2
f
max

f
max
4n
2
.
Above f
max
= max{f(v) : v V }. The last inequality uses =
1
8n
4
, and the second to last inequality
uses B 2n f
max
which we show next. Consider any x [0, 1]
n
and i V . We have

F(x)
x
i
|
x

S[n]\{i}
[f(S {i}) f(S)]
aS
x
a

bS
c
\i
(1 x
b
)

max
S[n]\{i}
[f(S {i}) +f(S)] 2n f
max
.
Thus we have F(y

) (1 +) F(y) +
1
4n
2
f
max
.
Proof of Theorem 12: Because each singleton solution {i} is feasible for the knapsacks and upper
bounds are 1, we have a feasible solution of value f
max
. Choose =

n
2
. The algorithm runs the
14
fractional local search algorithm (with all upper bounds 1) to get locally optimal solution y
1
[0, 1]
n
.
Then we run another fractional local search, this time with each variable i V having upper bound
u
i
= 1 y
1
(i); let y
2
denote the local optimum obtained here. The algorithm outputs the better of the
solutions y
1
, y
2
, and value f
max
.
Let x denote the globally optimal fractional solution to (8), where upper bounds are 1. We will
show (2 + ) (F(y
1
) + F(y
2
)) F(x) f
max
/n, which would prove the theorem. Observe that
x

= x (x y
1
) is a feasible solution to the second local search. Lemma 11 implies the following for
the two local optima:
(2 +) F(y
1
) F(x y
1
) +F(x y
1
)
f
max
2n
,
(2 +) F(y
2
) F(x

y
2
) +F(x

y
2
)
f
max
2n
.
We show that F(xy
1
) +F(xy
1
) +F(x

y
2
) F(x), which sufces to prove the theorem. For
this inequality, we again consider a scaled ground-set U having M copies of each element in V (where
M Z
+
is large enough so that Mx, My
1
, My
2
are all integral). Dene function g : 2
U
R
+
as
g(
iV
T
i
) = F( ,
|T
i
|
M
, ) where each T
i
consists of copies of element i V . Lemma 9 implies
that g is submodular. Also dene the following subsets of U: A (representing y
1
) consists of the rst
My
1
(i) copies of each element i V , C (representing x) consists of the rst Mx(i) copies of each
element i V , and B (representing y
2
) consists of My
2
(i) copies of each element i V (namely the
copies numbered My
1
(i) + 1 through My
1
(i) +My
2
(i)) so that A B = . Note that we can indeed
pick such sets because y
1
+ y
2
1 coordinate-wise. Also we have the following correspondences via
scaling:
A C x y
1
, A C x y
1
, (C \ A) B x

y
2
.
Thus it sufces to show g(A C) + g(A C) + g((C \ A) B) g(C). But this follows from
submodularity and non-negativity of g:
g(A C) +g(A C) +g((C \ A) B) g(A C) +g(C \ A) +g(C A B) g(C).
Hence we have the desired approximation for the fractional problem (8).
C Rounding the fractional solution under knapsack constraints
Fix a constant > 0 and let c =
16

. We give a (
1
5
)-approximation for submodular maximization over
k knapsack constraints, which is problem (7). Dene parameter =
1
4c
3
k
4
. We call an element e V
heavy if w
i
(e) for some knapsack i [k]. All other elements are called light. Let H and L denote
the heavy and light elements in an optimal integral solution. Note that |H| k/. Hence enumerating
over all possible sets of heavy elements, we can obtain prot at least f(H) in n
O(k/)
time, which is
polynomial for xed k. We now focus only on light elements and show how to obtain prot at least
1
4
f(L). Later we show how these can be combined into an approximation algorithm for problem (7).
Let Opt f(L) denote the optimal value of the knapsack constrained problem, restricted to only light
elements.
Algorithm for light elements. Restricted to light elements, the algorithm rst solves the fractional
relaxation (8) with all upper bounds 1, to obtain solution x with F(x) (
1
4


2
) Opt, as described in
the previous subsection (see Theorem 12). Again by adding dummy light elements for each knapsack,
we assume that fractional solution x satises all knapsacks with equality. Fix a parameter =
1
ck
, and
pick each element e into solution S independently with probability (1 ) x
e
. We declare failure if S
violates any knapsack and claim zero prot in this case (output the empty set as solution). Clearly this
algorithm always outputs a feasible solution. In the following we lower bound the expected prot. Let
(S) := max{w
i
(S) : i [k]}.
15
Claim 17 For any a 1, Pr[(S) a] k e
cak
2
.
Proof: Fixing a knapsack i [k], we will bound Pr[w
i
(S) a]. Let X
e
denote the binary random
variable which is set to 1 iff e S, and let Y
e
=
w
i
(e)

X
e
. Because we only deal with light elements, each
Y
e
is a [0, 1] random variable. Let Z
i
:=

e
Y
e
, then E[Z
i
] =
1

. By scaling, it sufces to upper bound

Pr [Z
i
a(1 +)E[Z
i
]]. Because the Y
e
are independent [0, 1] random variables, Chernoff bounds [37]
imply:
Pr [Z
i
a(1 +)E[Z
i
]] e
E[Z
i
]a
2
/2
e
a
2
/4
= e
cak
2
.
Finally by a union bound, we obtain Pr[(S) a]

k
i=1
Pr[w
i
(S) a] k e
cak
2
.
Claim 18 For any a 1, max{f(S) : (S) a + 1} 2(1 +)k(a + 1) Opt.
Proof: We will show that for any set S with (S) a+1, f(S) 2(1+)k(a+1)Opt, which implies
the claim. Consider partitioning set S into a number of smaller parts each of which satises all knapsacks
as follows. As long as there are remaining elements in S, form a group by greedily adding S-elements
until no more addition is possible, then continue to form the next group. Except for the last group formed,
every other group must have lled up some knapsack to extent 1 (otherwise another light element
can be added). Thus the number of groups partitioning S is at most
k(a+1)
1
+ 1 2k(a + 1)(1 + ).
Because each of these groups is a feasible solution, the claim follows by the subadditivity of f.
Lemma 19 The algorithm for light elements obtains expected value at least (
1
4
) Opt.
Proof: Dene the following disjoint events: A
0
:= {(S) 1}, and A
l
:= {l < (S) 1 + l} for
any l N. Note that the expected value of the algorithm is ALG = E[f(S) | A
0
] Pr[A
0
]. We can
write:
F(x) = E[f(S)] = E[f(S) | A
0
]Pr[A
0
]+

l1
E[f(S) | A
l
]Pr[A
l
] = ALG+

l1
E[f(S) | A
l
]Pr[A
l
].
For any l 1, from Claim 17 we have Pr[A
l
] Pr[(S) > l] k e
clk
2
. From Claim 18 we have
E[f(S) | A
l
] 2(1 +)k(l + 1) Opt. So,
E[f(S) | A
l
] Pr[A
l
] k e
clk
2
2(1 +)k(l + 1) Opt 8 Opt lk
2
e
clk
2
.
Consider the expression

l1
lk
2
e
clk
2

t1
t e
ct

1
c
, for large enough constant c. Thus:
ALG = F(x)

l1
E[f(S) | A
l
] Pr[A
l
] F(x) 8 Opt

l1
lk e
clk
F(x)
8
c
Opt.
Because =
16
c
and F(x) (
1
4


2
) Opt from Theorem 12, we obtain the lemma.
Theorem 20 For any constant > 0, there is a (
1
5
)-approximation algorithm for maximizing a
non-negative submodular function over k knapsack constraints.
Proof: As mentioned in the beginning of this subsection, let H and L denote the heavy and light
elements in an optimal integral solution. The enumeration algorithm for heavy elements produces a
solution of value at least f(H). Lemma 19 implies that the rounding algorithm for light elements
produces a solution of expected value at least (
1
4
) f(L). By subadditivity, the optimal value
f(H L) f(H) +f(L). The better of the two solutions (over heavy and light elements respectively)
found by our algorithm has value:
max{f(H), (
1
4
) f(L)}
1
5
f(H) +
4
5
(
1
4
) f(L) (
1
5
) f(H L).
This implies the desired approximation guarantee.
16
D Missing Proofs from Section 4
Proof of Claim 14: The following arguments hold for any q [p], and for notational simplicity we
denote R = R
q
C\S. Let {D
l
}
t
l=1
denote the vertices in connected components of G[R], which form
a partition of R. As mentioned above, |D
l
| p for all l [t]. For any l [t], let E
l
denote the labels
of edges in G incident to vertices D
l
. Because {D
l
}
t
l=1
are distinct connected components in G[R],
{E
l
}
t
l=1
are disjoint subsets of E S\C. Consider any l [t]: we claim S
l
= (S\E
l
) D
l
I
1
I
2
.
Note that E
l
{
1
(b) : b D
l
} and E
l
{
2
(b) : b D
l
}. Hence S
l
(S\{
i
(b) : b D
l
})D
l
for i = 1, 2. But from the property of mapping
i
(where i = 1, 2), (S \ {
i
(D
l
)) D
l
I
i
. This
proves that S
l
I
1
I
2
for all l [t].
From the properties of the maps
j
for each partition matroid M
j
, we have (S \
j
(D
l
)) D
l
I
j
for each 3 j k. Thus the following sets are independent in all matroids M
1
, , M
k
:
_
S \ (
k
j=3

j
(D
l
) E
l
)
_
D
l
l [t].
Additionally, because |D
l
| p and |(
k
j=3

j
(D
l
) E
l
| (k 1) p, each of the above sets are in
the local neighborhood of S. But local optimality of S implies:
f(S) f(
_
S \ (
k
j=3

j
(D
l
) E
l
)
_
D
l
) l [t]. (11)
Recall that {E
l
} are disjoint subsets of S \ C. Also each element i S \ C is missing in the right-hand
side of n
i
k 1 terms (the
j
s are matchings onto S \ C). Using local optimality under deletions,
we have the inequalities:
(k 1 n
i
) f(S) (k 1 n
i
) f(S \ {i}) i S \ C. (12)
Now, proceeding as in the simplication done in Lemma 2 (using disjointness of {D
l
}
t
l=1
), we obtain:
f
_
S
_

t
l=1
D
l
__
f(S) (k 1) (f(S) f(S C))
Noting that
t
l=1
D
l
= R, we have the claim.
Proof of Theorem 15: We set p = 1 +
2k

. The algorithm for the monotone case is just the local

search procedure with p-exchanges. Lemma 13 applied to local optimal S and the global optimal C
implies f(S) (
1
k

1
pk
) f(S C) (
1
k

1
pk
) f(C) (by non-negativity and monotonicity). From
the setting of p, S is a k + approximate solution.
For the non-monotone case, the algorithm repeats the p-exchange local search k times as in Theo-
rem 4. If C denotes a global optimum, an identical analysis yields
_
1 +
1
p1
_
k
2
f(S) (k1)f(C).
This uses the inequalities
_
p
p 1
_
k f(S
i
) f(S
i
C
i
) + (k 1) f(S
i
C
i
) 1 i k,
where S
i
denotes the local optimal solution in iteration i {1, , k} and C
i
= C \
i1
j=1
S
j
. Using
the value of p, S is a
_
k + 1 +
1
k1
+
_
-approximate solution. Observe that the algorithm has running
time n
O(k/)
.
We note that the result for monotone submodular functions is the rst improvement over the greedy
1
k+1
-approximation algorithm [39], even for the special case of partition matroids. It is easy to see that
the greedy algorithm is a
1
k
-approximation for modular functions. But it is only a
1
k+1
-approximation
for monotone submodular functions. The following example shows that this bound is tight for every
k 1. The submodular function f is the coverage function dened on a family F of sets. Consider a
17
ground set E = {e : 0 e p(k +1) +1} of natural numbers (for p 2 arbitrarily large); we dene
a family F = {S
i
: 0 i k} {T
1
, T
2
} of k + 3 subsets of E. We have S
0
= {e : 0 e p},
T
1
= {e : 0 e p 1}, T
2
= {p}, and for each 1 i k, S
i
= {e : p i + 1 e p (i + 1)}.
For any subset S F, f(S) equals the number of elements in E covered by S; f is clearly monotone
submodular. We now dene k partition matroids over F: for 1 j k, the j
th
partition has {S
0
, S
j
}
in one group and all other sets in singleton groups. In other words, the partition constraints require that
for every 1 j k, at most one of S
0
and S
j
be chosen. Observe that {S
i
: 1 i k} {T
1
, T
2
} is
a feasible solution of value |E| = p(k + 1) + 1. However the greedy algorithm picks S
0
rst (because
it has maximum size), and gets only value p + 1.
E Matroid Base Constraints
A base in a matroid is any maximal independent set. In this section, we consider the problem of maxi-
mizing a non-negative submodular function over bases of some matroid M.
max {f(S) : S B(M)} . (13)
We rst consider the case of symmetric submodular functions.
Theorem 21 There is a (
1
3
)-approximation algorithm for maximizing a non-negative symmetric
submodular function over bases of any matroid.
Proof: We use the natural local search algorithm based only on swap operations. The algorithm starts
with any maximal independent set and performs improving swaps until none is possible. From the
second statement of Lemma 2, if S is a local optimum and C is the optimal base, we have 2 f(S)
f(S C) + f(S C). Adding to this inequality, the fact f(S) = f(S) using symmetry, we obtain
3 f(S) f(S C) +f(S) +f(S C) f(C \ S) +f(S C) f(C). Using an approximate local
search procedure to make the running time polynomial, we obtain the theorem.
However, the approximation guarantee of this algorithm can be arbitrarily bad if the function f is
not symmetric. An example is the directed-cut function in a digraph with a vertex bipartition (U, V )
with |U| = |V | = n, having t 1 edges from each U-vertex to V and 1 edge from each V -vertex to
U. The matroid in this example is just the uniform matroid with rank n. It is clear that the optimal base
is U; on the other hand V is a local optimum under swaps.
We are not aware of a constant approximation for the problem of maximizing a submodular function
subject to an arbitrary matroid base constraint. For a special class of matroids we obtain the following.
Theorem 22 There is a (
1
6
)-approximation algorithm for maximizing any non-negative submodular
function over bases of matroid M, when Mcontains at least two disjoint bases.
Proof: Let C denote the optimal base. The algorithm here rst runs the local search algorithm using
only swaps to obtain a base S
1
that satises 2 f(S
1
) f(S
1
C) +f(S
1
C), from Lemma 2. Then
the algorithm runs a local search on V \S
1
using both exchanges and deletions to obtain an independent
set S
2
V \ S
1
satisfying 2 f(S
2
) f(S
2
(C \ S
1
)) + f(S
2
(C \ S
1
)). Consider the matroid
M

obtained by contracting S
2
in M. Our assumption implies that M

also has two disjoint bases, say

B
1
and B
2
(which can also be computed in polynomial time). Note that S
2
B
1
and S
2
B
2
are bases
in the original matroid M. The algorithm outputs solution S which is the better of the three bases: S
1
,
S
2
B
1
and S
2
B
2
. We have
6f(S) 2f(S
1
) + 2 (f(S
2
B
1
) +f(S
2
B
2
)) 2f(S
1
) + 2f(S
2
)
f(S
1
C) +f(S
1
C) +f(S
2
(C \ S
1
)) f(C).
The second inequality uses the disjointness of B
1
and B
2
.
A consequence of this result is the following.
18
Corollary 23 Given any non-negative submodular function f : 2
V
R
+
and an integer 0 c |V |,
there is a (
1
6
)-approximation algorithm for the problem max{f(S) : S V, |S| = c}.
Proof: If c |V |/2 then the assumption in Theorem (22) holds for the rank c uniform matroid, and the
theorem follows. We show that c |V |/2 can be ensured without loss of generality. Dene function
g : 2
V
R
+
as g(T) = f(V \ T) for all T V . Because f is non-negative and submodular, so is g.
Furthermore, max{f(S) : S V, |S| = c} = max{g(T) : T V, |T| = |V | c}. Clearly one of
c and |V | c is at most |V |/2, and we can apply Theorem 22 to the corresponding problem.
19