Sketching Valuation Functions

Ashwinkumar Badanidiyuru

Shahar Dobzinski

Hu Fu

Robert Kleinberg

Noam Nisan

Tim Roughgarden

October 4, 2011
Abstract
Motivated by the problem of querying and communi-
cating bidders valuations in combinatorial auctions, we
study how well dierent classes of set functions can be
sketched. More formally, let f be a function mapping
subsets of some ground set [n] to the non-negative real
numbers. We say that f

is an -sketch of f if for every

set S, the value f

(S) lies between f(S)/ and f(S),

and f

can be specied by poly(n) bits.

We show that for every subadditive function f there
exists an -sketch where = n
1/2
O(polylog(n)).
Furthermore, we provide an algorithm that nds these
sketches with a polynomial number of demand queries.
This is essentially the best we can hope for since:
1. We show that there exist subadditive functions (in
fact, XOS functions) that do not admit an o(n
1/2
)
sketch. (Balcan and Harvey [3] previously showed
that there exist functions belonging to the class of
substitutes valuations that do not admit an O(n
1/3
)
sketch.)
2. We prove that every deterministic algorithm that
accesses the function via value queries only cannot
guarantee a sketching ratio better than n
1
.
We also show that coverage functions, an interesting
subclass of submodular functions, admit arbitrarily
good sketches.

Department of Computer Science, Cornell University. Re-

search supported by NSF awards CCF-0643934, IIS-0905467,
and AF-0910940, AFOSR grant FA9550-09-1-0100, a Mi-
crosoft Research New Faculty Fellowship, a Google Re-
search Grant, and an Alfred P. Sloan Foundation Fellowship.
Email: {ashwin85,hufu,rdk,shahar}@cs.cornell.edu

School of Computer Science and Engineering, Hebrew Univer-

sity of Jerusalem. Supported by a grant from the Israeli Science
Foundation (ISF), and by the Google Inter-university center for
Electronic Markets and Auctions. Email: [email protected]

Department of Computer Science, Stanford University, 462

Gates Building, 353 Serra Mall, Stanford, CA 94305. Supported
in part by NSF grant CCF-1016885, an ONR PECASE Award,
and an AFOSR MURI grant. Email: [email protected].
Finally, we show an interesting connection between
sketching and learning. We show that for every class
of valuations, if the class admits an -sketch, then it
can be -approximately learned in the PMAC model
of Balcan and Harvey. The bounds we prove are only
information-theoretic and do not imply the existence of
computationally ecient learning algorithms in general.
1 Introduction
On a nite set N, where [N[ = n, a set function
f : 2
N
R
+
is said to be subadditive if f(S) +f(T)
f(S T) for all sets S, T. In this paper we consider
functions that are monotone, i.e., f(T) f(S) for all
S T, and normalized, f() = 0.
Subadditive functions arise naturally in economics
as they capture the notion of complement freeness in a
fairly general sense. For example, a buyer facing multi-
ple items has a subadditive valuation function if having
two sets of items simultaneously does not generate ex-
tra value for him. Combinatorial auctions with this type
of valuation functions have been extensively studied; see
[5, 9, 12] for example. In [16], a hierarchy of complement
free functions was dened, with subadditive functions
being the most general class. In particular, it strictly
contains submodular functions.
Submodular functions correspond to the economic
concept of diminishing returns. Formally, a set
function f is submodular if f(S j) f(S)
f(T j) f(T), for all S T and j N
T. Such functions arise extensively in combinatorial
optimization [17], economics [21], social networks [18],
and recently machine learning [14, 15].
It is often of interest to communicate such set
functions among dierent parties; however, set functions
in general need 2
n
values to describe a value for
each possible bundle. Since the property of complement
freeness entails restrictions among the function values,
one may naturally ask if a reasonable estimation of such
a function can be obtained with much less information
from the function, at the loss of some exactitude. We
call such an estimation a sketch.
More formally, we say that g : 2
N
R
+
is an -
approximation of f : 2
N
R
+
if for every set S we have
that,
f(S)

g(S) f(S). We say that g is an -sketch

if in addition it can be represented by poly(n) bits
1
. Of
course, we are not interested only in proving existence of
sketches that provide a good approximation ratio, but
would also like to construct these sketches eciently.
Goemans et al. [13] showed that, when f is submod-
ular, an

O(

n)-sketch of f can be obtained by querying

fs value at polynomially many subsets of S (i.e., using
poly(n) value queries). Their construction is essentially
tight: they showed an almost matching lower bound
for sketching submodular functions with polynomially
many value queries. Another lower bound is implied by
the work of Balcan and Harvey [3]. They showed that
there are certain matroid rank functions, a subclass of
submodular valuations, for which every sketch fails to
provide an approximation ratio better than n
1
3
. Notice
that this bound is unconditional in the sense that it
holds even if we have unlimited computational power.
Can we obtain good sketches for the more general
class of subadditive functions? Subadditive functions
are signicantly harder to handle than submodular
functions. For example, Dobzinski et al. [9] showed
that there is no polynomial time O(1)-approximation
for the problem of maximizing subadditive functions
subject to a cardinality constraint by value queries,
whereas the classical greedy algorithm [20] gives a
e
e1
-
approximation for submodular functions. As another
example, in Appendix 6 we show that there are simple
subadditive functions for which no submodular function
provides a better than n
1/2
-approximation. Given sub-
additive functions looser structures, and the fact that
[13] used substantial techniques specic to submodular
functions, it is unclear whether one can obtain good
sketches for subadditive functions.
Our rst main result shows that for every subaddi-
tive function there exists a

O(

n)-sketch that can be

found with only polynomially many queries, albeit with
a form of queries called demand queries, which are more
powerful than value queries. Demand queries are moti-
vated in economic settings where an agent with a cer-
tain valuation function facing a set of items, each with
a price tag, is required to report a subset of items that
maximizes his utility. In mathematical terms, given a
function f : 2
N
R
+
, a demand query on f presents
a price vector p R
N
+
and gets as an answer a bundle
S arg max
TN
f(T)

jT
p
j
. Demand queries
have been broadly used and studied in the literature of
1
For simplicity we assume that for each set S, f(S)
{0, 1, 2, ..., poly(n)}. All results can be generalized to the case
where f takes arbitrary real values.
algorithmic game theory (see, for example, [7]), and are
known to be strictly more powerful than value queries in
the sense that one can simulate a value query by poly-
nomially many demand queries, but the converse is not
true [8].
We prove that the last result is essentially the
best one can hope for. First, we show that the ap-
proximation ratio is essentially tight in the sense that
there are XOS functions (a subclass of subadditive func-
tions) that do not admit an o(

n)-sketch. Second, we
prove that value queries are not powerful enough to ob-
tain good sketches: a deterministic algorithm that al-
ways nds O(n
1
)-sketches must use superpolynomi-
ally many value queries, for any > 0. This shows that
one must use stronger queries in order to obtain good
sketches.
Whereas for subadditive functions we show that
there are ecient algorithms using demand queries that
always produce sketches whose size matches the infor-
mation theoretic bound, we show that this is not always
the case for other valuation classes. We consider the
class of OXS functions, a subclass of submodular func-
tions that admits a 1-sketch [6] (i.e., the function can
be fully described in polynomial space). However, we
show that obtaining an n
1
2

-sketch requires exponen-

tially many value queries. Moreover, since OXS func-
tions belong to the class of gross substitutes valuations
for which a demand query can be simulated by poly-
nomially many value queries, we have that obtaining
an n
1
2

sketch requires exponentially many demand

queries.
We then consider another well-studied subclass of
submodular functions, coverage functions. We show
that coverage valuations admit short sketches of arbi-
trary precision. A coverage function f is dened on a
set N, each element of which corresponds to a subset
of some universe of points with non-negative weights,
and the value of f on S N is the weight of the points
in covered by the sets corresponding to elements of S.
We show that by appropriately sampling and reweight-
ing points from the universe one can obtain a (1 + )-
sketch which can be described in poly(n,
1

) space. It is
an open question to obtain this sketch with either value
queries or demand queries.
Finally we show an interesting connection between
sketching and learning. Balcan and Harvey [3] introduce
the problem of learning submodular functions: given
bundles S
1
, ..., S
poly(n)
sampled i.i.d. from some distri-
bution D and their values f(S
1
), ..., f(S
poly(n)
) accord-
ing to some unknown function f, can we nd an al-
most correct sketch of f for subsequent samples drawn
from D? They coin the term PMAC learning (probably
mostly approximately correct) to refer to this type of
approximation guarantee, in which the sketch may fail
to be an -approximation for certain bundles, but with
high probability a subsequent sample from D will not
belong to this exceptional set of bundles. We show that
PMAC learning can be done for every class of valua-
tions: if a class of valuations admits an -sketch, then
it can be learned. Using the results in the paper this
implies that a

O(

n)-sketch of subadditive functions

can be learned, as well as arbitrarily good sketches for
coverage and OXS valuations. However, the bounds we
prove are only information theoretic ones and we do not
show the existence of a computationally ecient learn-
ing algorithm for this problem.
Independently of our work, Balcan et al. [2] ob-
tained related, but largely complementary, results on
learning valuation functions. They give computation-
ally ecient algorithms for PMAC learning a

O(

n)-
sketch of a subadditive function using polynomially
many value queries. In comparison, our sketching al-
gorithm satises a stronger (i.e. pointwise) approxima-
tion guarantee but uses demand queries, and therefore
is not computationally ecient in general. Balcan et
al. [2] also present improved PMAC guarantees for cer-
tain classes of functions, such as XOS functions rep-
resented by a polynomial number of clauses, and they
prove hardness results for learning XOS and OXS valu-
ations that are analogous to the sketching lower bounds
we present here.
1.1 Sketching Subadditive Valuations: a Brief
Overview Let us sketch why every submodular func-
tion has an O(

n)-sketch [13]. Every submodular func-

tion f denes a polymatroid P
f
: x R
|S|
+
[ x(T)
f(T), T S, where x(T) =

jT
x
j
. The basic
idea is to show the existence of an ellipsoid E such
that
1

n
P
f
E P
f
. If we have such ellipsoid
E we can use it as our sketch, since for every S we
have that f(S) = max(1
S
)
T
x[x P
f
. Towards
this end, we consider the symmetrized version of P
f
,

P
f

= x R
n
[ [x[ P
f
. The renowned Johns
theorem states that for any centrally symmetric con-
vex body P in R
n
, there exists an ellipsoid E such that
1

n
P E P. We get our sketch by applying Johns
theorem to P
f
. Notice that this proves the existence of
an O(

n)-sketch, but not how to eciently nd it.

We now want to show that every subadditive func-
tion has an

O(

n)-sketch. As a rst step, we show

that every XOS function (a.k.a. as fractionally sub-
additive) has an O(

n)-sketch. A function is XOS if

there exists additive valuations
2
a
1
, . . . , a
t
such that
2
A valuation v is additive if for every set S we have that
v(S) =
jS
v({j}).
v(S) = max
i
a
i
(S). It is known that XOS functions
are a proper superclass of submodular functions and a
proper subclass of subadditive functions. The key obser-
vation here is that the class of XOS functions is exactly
the class for which f(S) = max(1
S
)
T
x[x P
f
, for
every S, by taking x
j
to be a
i
(j) for all j S, where
i

= arg max
i
a
i
(S). Hence every XOS function admits
an O(

n)-sketch.
Now we extend this result to subadditive functions.
We use a result in [10] that shows that for every sub-
additive function there exists an XOS function that
O(log n)-approximates it. This shows that every subad-
ditive function has an O(log n

n)-sketch: take the XOS

function that O(log n) approximates it, and provide the
O(

n)-sketch that was obtained using the ellipsoidal

approach. We are left with showing that such a sketch
can indeed obtained algorithmically.
A crucial insight of [13] is that the problem can be
reduced to the problem of nding a point in P
f
that is
far from a given ellipsoid, which in turn is equivalent
to the following optimization problem for any c R
n
+
:
max

i
c
2
i
x
2
i
s.t. x P
f
where c
i
s are coecients specifying an ellipsoid

i
c
2
i
x
2
i
1. A -approximation for this problem
will give a

n-approximation for P
f
, and hence a
O(

nlog n)-approximation for a subadditive f.

For submodular functions, Goemans et al. showed
that: (1) under certain conditions, a scaled polyma-
troid (corresponding to heterogeneous c
i
s) can be ap-
proximated by an unscaled polymatroid (correspond-
ing to the same c
i
s) within an O(log n) factor, and (2)
the classical greedy algorithm for maximizing submod-
ular functions subject to a cardinality constraint [20]
provides a (1
1
e
)
2
approximation for the unscaled case.
Notably, (2) requires only value queries. Unfortunately,
the approach of [13] fails in the case of subadditive func-
tions.
Therefore we develop new machinery to handle sub-
additive functions. En route, we signicantly simplify
the rst step for more general polytopes while avoid-
ing proving a structural theorem a la [13]. We start
by observing that for any elements i, j in S, if f(i)
is signicantly larger than f(j), then for any T S
that contains both i and j, the value of f(T) will not
change much if we ignore the contribution of j, a con-
sequence of subadditivity. Our goal now is to provide
a set of ellipsoids that will approximate f in dierent
magnitudes of values. This enables us to reduce the
problem to the still-challenging problem of approximat-
ing the quadratic program where all c
i
s are equal. In
addressing this, we discover an interesting substructure
for subadditive functions, which we call universal se-
quences.
Definition 1.1. Let f : 2
N
R
+
be a function. A
sequence = T
0
T
1
T
n
= N is called a -
universal sequence of f if, for any set S we have that
f(S) f(T
|S|
).
Note that each T
i+1
has one more element than T
i
.
In plain words, a universal sequence is an ordering of
the elements in N such that for any k n, the rst
k elements in this ordering provide a -approximation
for the maximization problem subject to cardinality
constraint of k. The greedy algorithm for submodular
functions, for example, shows that there is a
e
e1
-
universal sequence for any submodular function. We
show that any subadditive function admits a 4-universal
sequence. Then we construct a vector in P
f
using such
a sequence and show that the vector is an O(log
2
n)-
approximation for the quadratic program by exploiting
the symmetry and convexity of the objective function.
2 Approximating Subadditive Functions using
Demand Queries
In this section we describe an algorithm that outputs
an

O(

n) approximation to any subadditive function

f : 2
[n]
R
+
using polynomially many demand queries.
We follow the approach of ellipsoidal approximation
introduced in [13]. An ellipsoid E
A
R
n
denes a
function L by mapping each S [n] to max
xE
A
1
T
S
x,
where 1
S
is the indicator vector for S. In the following
we often use the term ellipsoid to also refer to the
function it denes. Recall that a function f : 2
[n]
R
+
denes a polytope P
f
= x(S) f(S), S [n] [
x R
n
+
, where x(S) =

iS
x
i
. In [13] the following
lemma is proven:
Lemma 2.1. ([13]) Let f be a function and P
f
be the
polytope it denes. If we have an algorithm A that
provides a -approximation for the quadratic program
max

i
c
2
i
x
2
i
s.t. x P
f
for every i, 1 c
i
f(i)

n + 1, then we can nd an
ellipsoid that provides a O(

n)-approximation in time
polynomial in the running time of A.
Unlike [13] we do not show how to solve this
quadratic program for all c
i
s. Nevertheless, we show
that to obtain an

O(

n)-sketch for subadditive func-

tions it suces to be able to solve the case when all c
i
s
are 1.
Lemma 2.2. Let f be a subadditive function and P
f
be
the polytope it denes. If we have an algorithm A that
provides a -approximation for the quadratic program
max

i
x
2
i
s.t. x P
f
then we can nd an O(

n log n)-sketch for f in time

polynomial in the running time of A.
Proof. We will reduce the problem in Lemma 2.1 to
the current problem with a loss of a log n factor in
the solution. Without loss of generality, we assume the
items in N are ordered such that f(1) f(2)
. . . f(n).
Definition 2.1. For each element i [n], dene its
vicinity to be V
i
= j [ i j n, nf(j) f(i).
For a subset S [n], let f[
S
be f[
S
(T) = f(T
S) for any T [n]. It is easy to see that f[
Vi
is still a subadditive function. The algorithm will
compute n ellipsoids such that each ellipsoid is an

O(

n) approximation for f[
Vi
, respectively. For a
subset S = i
1
, i
2
, . . . , i
m
[ i
1
< i
2
< . . . < i
m
, we
use the ellipsoid corresponding to f[
Vi
1
to approximate
f(S). If the corresponding ellipsoid is an

O(

n)
approximation for f[
Vi
1
, then since
f(S) f[
Vi
1
(S) +n n
1
f(i
1
) 2f[
Vi
1
(S),
we will have obtained an

O(

n) approximation for
f(S) itself. From this point on we will assume that
f(1) f(2) . . . f(n)
f({1})
n
.
Recall that 1 c
i
f(i)

n + 1. By the
assumption that f(1) . . . f(n)
f({1})
n
, the
ratio
maxi ci
minj cj
is polynomially bounded. We reduce the
problem by rst rounding each c
i
down to the largest
power of 2, and thereby grouping them into O(log n)
bins B
1
, . . . , B
k
; then we solve the optimization problem
max
iBi

x
2
i
s.t. x P
f
for each B
i
. It is easy to see that the best solution
x P
f
obtained will be an O(log
2
n) approximation for
the problem with dierent c
i
s.
Hence, it is left to show that one can approximate
the optimization problem
max

i
x
2
i
s.t. x P
f
within an O(log
4
n) factor using polynomially demand
queries. We rst introduce the notion of universal
sequences both for the ease of presentation and for its
own interest:
Definition 2.2. Let f : 2
N
R
+
be a function. A
sequence = T
0
T
1
. . . T
n
= N is called a -
universal sequence of f if, for any set S we have that
f(S) f(T
|S|
).
In particular, for submodular functions the greedy
algorithm produces a
e
e1
-universal sequence. For
subadditive functions we show that there exists a 4-
universal sequences. This will be a by product of the
following algorithm that nds a poly logarithmic ap-
proximation to the quadratic program. The algorithm
makes use of the following notion:
Definition 2.3. ([11, 10]) Let f : 2
N
R
+
and
S N. For each j S, let p
j
be a non-negative real
number. We say that the p
j
s are -supporting prices
for S if
j
p
j
f(S) and T S,
jT
p
j
f(T).
In the algorithm we assume without loss of gener-
ality that n is a power of 2 we can always add more
items with zero value):
1. Let T
0
= .
2. For each k. k = 1, 2, . . . , log
2
n:
(a) Find a set S, [S[ = 2
k
such that for every T,
[T[ = 2
k
we have that f(S) f(T).
(b) Partition S to S
1
and S
2
, [S
1
[ = [S
2
[. Let
S
k
= arg max
S{S1,S2}
(f(S)).
(c) Let T
k
= T
k1
S
k
.
(d) Let x
|T
k1
|+1
, . . . , x
|T
k
|
be O(log n)-
supporting prices of the set T
k
T
k1
.
3. Let x =
x
2 log n
.
Step (2a) is the problem of optimizing a subad-
ditive function subject to a cardinality constraint. A
2-approximation for this problem that uses only poly-
nomially many demand queries was provided in [1]
( = 2). Step (2d) can be implemented using the al-
gorithm of [10] that nds O(log n) supporting prices
using polynomially many demand queries
3
. Hence we
have that the algorithm can be implemented using only
polynomially many demand queries.
3
It is known that for general subadditive functions, O(log n)-
supporting prices are the best that one can hope for; however,
with XOS functions, for which 1-supporting prices exist, it
remains open whether one can use polynomially many demand
queries to get better than O(log n)-supporting prices.
We rst show that the algorithm nds a 4-universal
sequence. This will later be used to show that x is an
approximate solution to the quadratic problem.
Proposition 2.1. Let f be a subadditive function. The
algorithm nds a 4-universal sequence.
Proof. The proof relies on the following claim:
Claim 2.1. For every k, k log
2
n, and every T,
[T[ = 2
k
, we have that 2 f(T
k
) f(T).
Proof. Let S be the bundle we obtained in the kth
iteration of Step 2a. Let T be some bundle of size 2
k
. By
subadditivity and the guaranteed approximation ratio
we have that (f(S
1
) + f(S
2
)) f(T). We took S
k
to
be the larger of the two, and hence f(S
k
)
1
2
f(T).
Now by construction T
k
S
k
, and by monotonicity we
have proved the claim.
By ordering the elements in each S
k
arbitrarily and
then add them one by one to a sequence of sets, we
obtain a full sequence = U
0
U
1
U
2
. . . U
n
= N.
Note that every T
k
occurs in this sequence. We can
now prove the proposition: For any set S N, we
consider T
k
where k is log
2
[S[|. From Claim 2.1 we
have f(T
k
)
1
2
max
TN,|T|=2
k f(T)
1
4
f(S), where
the last inequality comes from subadditivity.
Lemma 2.3. x produced by the algorithm is an
O(log
4
n) approximation for the quadratic program.
The proof consists of the following claims.
Claim 2.2. x is in P
f
.
Proof. By denition of P
f
, it suces to show that
S N, x(S) f(S). We note that
x(S) =
x(S)
2 log n
max
i
x(S (T
i
T
i1
))
max
i
f(S (T
i
T
i1
))
f(S)
The rst inequality is valid because there are only
(log
2
n| + 1) dierent S
i
s, and the second inequality
follows from the fact that x is dened as a supporting
price of f(T
i
T
i1
).
Claim 2.3. k log
2
n, x(T
k
)
f(T
k
)
O(log n)
.
Proof. We prove this by induction. For k = 1, we have
x(T
1
) =
f(T1)
O(log n)
by denition of supporting prices. Now
suppose the claim is true for k, then by subadditivity
f(T
k+1
) f(T
k+1
T
k
) +f(T
k
)
= O(log n)(x(T
k+1
T
k
) +x(T
k
))
= O(log n)x(T
k+1
)
The equality follows from the denition of supporting
prices and the induction hypothesis.
Now using x, we dene a symmetric
4
submodular
function g : 2
N
R. For S N, we let g(S) =
max
TN,|T|=|S|
x(S). It is easy to verify that g is indeed
a submodular function. Let P
g
be the polymatroid it
denes, then we have
Claim 2.4. P
g

P
f
O(log
2
n)
.
5
Proof. It suces to show that for any y P
f
and
S N,
y(S)
O(log
2
n)
g(S). Let k be log
2
[S[|, then
by Claim 2.1 and Claim 2.3, we have
f(S) 4f(T
k
)
= O(log n)x(T
k
)
= O(log
2
n) x(T
k
)
O(log
2
n)g(T
k
)
O(log
2
n)g(S)
Since the vertices of P
g
are permutations of the
coordinates of x, we know that x is an optimal solution
to the optimization problem
max

x
2
i
s.t. x P
g
By Claim 2.4 and Claim 2.2, we have that x is a feasible
solution and gives an O(log
4
n) approximation for the
optimization problem
max

x
2
i
s.t. x P
f
3 Lower bounds
We have seen that there is a deterministic algorithm
to compute a

O(

n)-sketch of any subadditive function

using demand queries. In this section, we show that this
result is essentially the best possible, in two respects.
First, for any xed > 0, it is not the case that
every subadditive function admits a O(n
1/2
)-sketch
4
By symmetric we mean a function depending only on the
cardinality.
5
We note that we do not prove that Pg P
f
, and this is not
generally true.
of polynomial size. In fact our bound even holds for
XOS valuations. Previously, Balcan and Harvey [3]
showed that every polynomial-size sketch of submodular
functions cannot have an approximation ratio better
than n
1
3
. Second, for deterministic algorithms that are
limited to value queries, it is impossible to obtain a
O(n
1
)-sketch in polynomial query complexity.
We also consider the class of OXS functions. This
class was dened in [16] and is equivalent to the to the
class of weighted rank function of a transversal matroid
(and hence is a subclass of the class of submodular val-
uations). This class can be represented in O(n
2
) space
[6]. In contrast, we show that algorithmically obtaining
such a sketch via queries is hard: an O(n
1/2
)-sketch
requires an exponential number of value queries. More-
over, for this class a demand query can be implemented
via a polynomial number of demand queries [4]. Hence
we have that an O(n
1/2
)-sketch requires an exponen-
tial number of demand queries.
3.1 A Tight Lower Bound for Sketching XOS
Valuations We show that XOS functions do not admit
n
1
2

-sketches. This slightly improves over the result

of [3] that showed there are no n
1
3

-sketches if the
valuation function is submodular (in fact, even gross
substitutes).
Definition 3.1. A family C of subsets of 1, ..., n is
(h, )-good if
1. For each S C, [S[ h.
2. For each S, T C and S ,= T, [S T[ .
Lemma 3.1. If some C is (h, )-good then approximat-
ing XOS valuations to within a factor of better than h/
requires representation length of at least [C[.
Proof. For every subset D C we dene an XOS
valuation: v
D
(S) = max
TD
[T S[. Now we claim
that for every D ,= D

C there exists a subset S such

that v
D
(S)/v
D
(S) h/ or v
D
(S)/v
D
(S) h/. This
will be true exactly for S (D D

) (D

D). The
bound on sketching length is implied since any sketching
scheme with approximation ratio better than h/ cannot
give any two Ds the same sketch.
Lemma 3.2. For h =
1
2

n, for every , there exists a

family that is (h, )-good of size n
()
.
Proof. Choose any prime p such that n/4 p
2
n, and
identify a subset of N with the set F
2
p
, where F
p
denotes
the eld of integers modulo p. For each univariate
polynomial P of degree at most over F
p
, add the set
S
P
= (x, P(x)) [ x F
p
to the collection C. Each
set in C has cardinality p
1
2

n, and the intersection

of any two sets S
P
, S
Q
C has cardinality at most
because the equation P(x) = Q(x) is satised by at
most values of x. Finally, a polynomial of degree at
most is uniquely determined by a sequence of + 1
coecients in F
p
, so the cardinality of C is p
+1
= n
()
.
Theorem 3.1. Polynomial-size sketches cannot ap-
proximate XOS to within a factor of o(

n). Sub-
exponential-size sketches cannot approximate XOS to
within a factor better than O(n
1/2
).
3.2 Inapproximability of Subadditive Functions
using Deterministic Value Queries The following
theorem is a lower bound for deterministic sketching of
subadditive functions using value queries. The proof
shows that polynomially many value queries cannot
possibly provide enough evidence to distinguish the
subadditive function f(S) = [S[
1
from a dierent
subadditive function that takes the value n

on at least
one set of size n
1
.
Theorem 3.2. If a deterministic algorithm can com-
pute an -sketch of every subadditive function using
O(poly(n)) value queries, then = (n
1
) for every
xed > 0.
To prove the theorem, we begin with the following
characterization of subadditive set functions.
Lemma 3.3. Let (A
1
, c
1
), . . . , (A
k
, c
k
) 2
N
R
+
be a
sequence of pairs consisting of a subset of N and a non-
negative cost for that subset. Suppose that N =

k
i=1
A
i
.
The set function
g(U) = min
_
_
_

jJ
c
j

_
jJ
A
j
U
_
_
_
,
called the min-cost-cover function of (A
i
, c
i
)
k
i=1
, is a
non-negative, monotone, subadditive set function. Fur-
thermore, every non-negative, monotone, subadditive
set function can be represented as the min-cost-cover
function of a suitably dened sequence (A
i
, c
i
)
q
i=1
.
Proof. Non-negativity and monotonicity of g are clear
from the denition. Subadditivity follows from the
observation that if

jJ
A
j
U and

jJ
A
j
U

then

jJJ
A
j
UU

while c(J J

) c(J)+c(J

).
(Here we are using the notation c(J) to denote

jJ
c
j
.)
Conversely, if f is non-negative, monotone, and
subadditive, then f is equal to the min-cost-cover
function of the set of ordered pairs (A, f(A)) where A
ranges over all subsets of N. The min-cost-cover of U is
less than or equal to f(U) because the set U covers itself,
and it is not strictly less because f satises monotonicity
and subadditivity.
We will also make use of the following form of the
Cherno bound.
Lemma 3.4. Let T be a random subset of an n-element
set N, obtained by selecting every element independently
with probability p, and let U be any other, xed, subset
of N. For any 1 n,
Pr ([T U[ max, 2p[U[) < e
/12
.
We are ready now to proceed with the proof of
Theorem 3.2.
Proof. [Theorem 3.2] Let = /3, and assume without
loss of generality that < 1/2. Consider the sequence
of queries and responses that take place when we run
the algorithm using the subadditive function f(S) =
[S[
1
. Denote this sequence by (S
1
, v
1
), . . . , (S
q
, v
q
),
where v
i
= [S
i
[
1
for 1 i q. We will construct
a subadditive function g such that g(S
i
) = v
i
for
all i 1, . . . , q, but g(T) n

for some other

set T of cardinality at least k = ,n
1
|. When
the algorithm observes the sequence of queries and
responses (S
1
, v
1
), . . . , (S
q
, v
q
), its output f

must be
an -sketch of both f and g, hence f(T)/ f

(T)
g(T). This implies that f(T)/g(T) n
(1)
2

>
n
1
.
To construct the function g we use the probabilistic
method. Let T be a random subset of N obtained by
sampling each element independently with probability
p = 2k/n, and let g be the min-cost-cover function of the
sequence (S
1
, v
1
), (S
2
, v
2
), . . . , (S
q
, v
q
), (T, n

). Cheby-
shevs inequality implies that with probability at least
1
2
, [T[ 2k

n k. We will complete the proof
by showing that Pr(i s.t. g(S
i
) ,= f(S
i
)) is less than
1
2
for large enough n. In fact, we will show that the
event i s.t. g(S
i
) ,= f(S
i
) is contained in the union
of the events [T (S S

)[ maxn

, p[S S

[,
where S, S

range over all pairs of sets in the col-

lection , S
1
, S
2
, . . . , S
q
. By the union bound and
Lemma 3.4, the probability that there exists i such that
g(S
i
) ,= f(S
i
) is at most O(q
2
) exp(n

/12) and this is

less than
1
2
for suciently large n.
To nish the proof, we must show that the assump-
tion that g(S
i
) ,= f(S
i
) = [S
i
[
1
implies the existence
of two sets S, S

in the collection , S
1
, S
2
, . . . , S
q
such
that
(3.1) [T (S S

)[ maxn

, 2p[S S

[.
Clearly, by construction, g(S
i
) [S
i
[
1
, so assume
that the inequality is strict. It means that there is an
index set J [q] such that either
c(J) < [S
i
[
1
and
_
jJ
S
j
S
i
,
or
n

+c(J) < [S
i
[
1
and T
_
_
_
jJ
S
j
_
_
S
i
.
The rst alternative is not possible, since the set func-
tion f(S) = [S[
1
is subadditive. So assume the second
alternative holds. Letting V = (T S
i
)
_

jJ
S
j
_
, we
have S
i
V
_

jJ
S
j
_
, and by the subadditivity of
f(S) = [S[
1
this implies that
(3.2) [V [
1
[S
i
[
1

jJ
[S
j
[
1
> n

.
The set S
i
V is contained in

jJ
S
j
. Partition
S
i
V arbitrarily into disjoint subsets W
j

jJ
such
that W
j
S
j
for all j J. For each x S
i
dene
w(x) =
_
[W
j
[

if x W
j
for some j J
0 if x V .
Note that the cases are mutually exclusive and exhaus-
tive, so w(x) is well-dened for all x S
i
. We have
(3.3)

xSi
w(x) =

jJ
[W
j
[[W
j
[

jJ
[S
j
[
1
= c(J) < [S
i
[
1
,
so
(3.4)
1
[S
i
[

xSi
w(x) < [S
i
[

.
The argument now splits into two cases. If [W
j
[
1
2
[S
i
[
for all j J then we have w(x) 2

[S
i
[

for all
x S
i
V , hence, by (3.3),
2

[S
i
[

[S
i
V [ < [S
i
[
1
[S
i
V [ < 2

[S
i
[
[V [ >
_
1 2

_
[S
i
[ >

2
[S
i
[, (3.5)
using the fact that 1 2
x
> x/2 for all 0 < x < 1.
For suciently large n, the right side is greater than
2p[S
i
[, so combining (3.5) with (3.2), we obtain [V [
maxn

, 2p[S
i
[, which conrms (3.1) with S = S
i
, S

=
, since T S
i
V .
The remaining case is that [W
j
[ >
1
2
[S
i
[ for some
j J. In that case, the average value of w(x) over
x W
j
is [W
j
[

, which is greater than [S

i
[

, whereas
the average value of w(x) over x S
i
is strictly less
than [S
i
[

by (3.4). Consequently the average value of

w(x) over x S
i
W
j
must be strictly less than [S
i
[

.
Whenever x S
i
W
j
and w(x) > 0, then x belongs
to some W
j
such that [W
j
[ <
1
2
[S
i
[, and consequently
w(x) > 2

[S
i
[

. Arguing as in case 1, this implies that

(3.6) [V [ >

2
[S
i
W
j
[

2
[S
i
S
j
[.
Combining (3.6) with (3.2), we obtain [V [
maxn

, 2p[S
i
S
j
[, which conrms (3.1) with S =
S
i
, S

= S
j
, since T (S
i
S
j
) V .
3.3 A Lower Bound for OXS Functions
Theorem 3.3. Let f be an OXS valuation and let A be
an algorithm that that provides an n
1
2

-sketch, using
value queries and demand queries. A does not make a
polynomial number of such queries.
Proof. Since demand queries for OXS valuations can be
simulated using a polynomial number of value queries,
we assume henceforth that A makes only value queries.
Start with the complete bipartite graph K
k,n
where
k = n. Pick a random subset B of n nodes on the
RHS, and an arbitrary subset A of n nodes on the
LHS. (We will x , later to be (
_
(log n/n).) Delete
all edges between B and A
c
(where A
c
is the intersection
of the complement of A and the vertices in the LHS).
For a subset S of nodes on the RHS, let v(S) denote the
maximal matching size that only matches RHS nodes in
S. Observe that we can write
v(S) = minn, [S B
c
[ + minn, [S B[
(Since nodes in B can only contribute n, and because
there are only n LHS nodes.) Notice that v is a rank
function of a matching matroid, hence it is indeed an
OXS valuation.
It is enough to prove that, for every S, with high
probability over the choice of B, v(S) = min[S[, n.
(Then you learn nothing about B from any of your value
queries.) This will show that we cannot distinguish with
polynomially many value queries between v(B) = n
and v(B) = n, and an approximation lower bound of
1/ will therefore follow.
Assume we choose , so that n = (log n). If
[S[ = O(n) then v(S) = [S[. Otherwise, by Cherno
we have [S B[ and [S B
c
[ concentrated around their
expectations of [S[ and (1)[S[, respectively. If [S[ =
O(n) then [S B[ = O(n) and v(S) = min[S[, n.
If [S[ = (n) then [S B
c
[ = ((1 )n), which is
(n) provided /(1 ). In this case, v(S) = n.
Finally, we choose = 2 = (
_
log n/n) and get a
lower bound of (
_
n/ log n).
4 Coverage Functions Admit (1+)-Sketches
A set function f : 2
N
R
+
is called a coverage
function if there exists a nite set , a weight function
w : R
+
, and a function g : N 2

such that
f(S) = w
_
iS
g(i)
_
for all S N.
We will present a sampling algorithm which pro-
duces a coverage function

f on a set of points

with
[

[
27 n
2

2
, such that

f is a (1+)-sketch of f with high
probability. We will assume without loss of generality
that 0 < < 1 and that =

iN
g(i).
Algorithm: Let t =
27 n
2

2
. Dene q(x) =
max
_
w(x)
f({i})

i N, x g(i)
_
for x . Let p(x) =
q(x)

y
q(y)
. The algorithm draws t independent random
samples x
1
, . . . , x
t
from the distribution on specied
by p. It denes

= x
1
, . . . , x
t
and sets g

(i) =

g(i) for all i N. To dene the weight of an ele-

ment x

, we let m(x) denote the number of times

x occurs in the sequence and set w

(x) = m(x)
w(x)
tp(x)
.
Finally, dene

f to be the coverage function specied
by

, w

, and g

. The algorithm outputs the function

_
1 +

3
_
1

f.
4.1 Proof of (1 +)-approximation We will prove
that this algorithm outputs a (1 + )-sketch of f. We
need the following simple form of the Cherno bound,
which is well known [19] in the special case m = , and
whose general case follows from that special case by a
trivial scaling argument.
Lemma 4.1. Suppose Y
1
, . . . , Y
t
are mutually indepen-
dent random variables satisfying E
_

t
i=1
Y
i
_
= , and
suppose that each of the variables Y
i
is supported in the
interval [0, /m] for some m > 0. Then for 0 < < 1,
we have
Pr
_
(1 )
t

i=1
Y
i
(1 +)
_
> 1 2e
(
2
/3)m
.
Remark. The multiplicative form of the Cherno
bound is usually stated as two separate bounds, one
for Pr((1 ) >

Y
i
) and another for Pr(

Y
i
>
(1+)). The version stated above follows by summing
these two bounds and subtracting from 1, then using
the fact that e
(1+) ln(1+)
> e

2
/3
for 0 < < 1.
Fix an arbitrary set S N, let U =

iS
g(i), and
dene random variables Y
1
, . . . , Y
t
by
Y
i
=
_
w(xi)
tp(xi)
if x
i
U
0 otherwise.
Note that

f(S) =

t
i=1
Y
i
, and that the random
variables Y
1
, . . . , Y
t
are mutually independent. To apply
the Cherno bound, we will need the following pair of
claims.
Claim 4.1. For every S N, E[

f(S)] = f(S).
Proof. For each i we have
E[Y
i
] =

xU
_
p(x)
w(x)
t p(x)
_
=

xU
w(x)
t
=
1
t
w(U)
=
1
t
f(S)
The claim follows by summing over i = 1, . . . , t.
Claim 4.2. For every x and every S N, if
x

iS
g(i) then
w(x)
tp(x)


2
27 n
f(S).
Proof. The key observation is that

y
q(y) n.
To prove this, dene for each i N a set h(i) =
_
y g(i) [ q(y) =
w(y)
f({i})
_
. Note that every y
belongs to at least one of the sets h(i). Therefore

y
q(y)

iN

yh(i)
q(y)
=

iN

yh(i)
w(y)
f(i)
=

iN

yh(i)
w(y)

yg(i)
w(y)
n
Now, for any x

iS
g(i), we have
w(x)
t p(x)
=

2
w(x)
27 n
2

y
q(y)
q(x)

2
27 n

w(x)
q(x)
=

2
27 n
minf(i) [ i N, x g(i)

2
27 n
f(S)
Combining Claims 4.1 and 4.2 with Lemma 4.1 and
using = /3, m =
27 n

2
in that lemma, we obtain the
result that
Pr
_
[

f(S) f(S)[

3
f(S)
_
> 1 2e
(
2
/27 )(27 n/
2
)
= 1 2e
n
Taking the union bound over all sets S N, we
now see that with probability at least 1 2
n+1
e
n
,
the function
_
1 +

3
_
1

f is an -sketch of f where
=
_
1 +

3
___
1

3
_
< 1 +, where the last inequality
follows from our assumption that < 1.
5 Learning via Sketching
Balcan and Harvey [3] introduce the problem of learning
submodular functions. We are given poly(n) bundles
that are drawn i.i.d. from some distribution and their
values according to some unknown submodular function
f. The goal is to output a sketch of the functions, i.e.,
its learned valuation.
We will show that if a class of valuations admits
an -sketch
6
, then there exists a learning algorithm for
this class. The caveat is that we only focus at the num-
ber of allowed queries and the information that can be
learned from this. In particular we completely ignore
the actual algorithmic challenges and assume a compu-
tationally unbounded learner whose only constraints are
the access to the valuation function. This is in complete
contrast to most work in learning theory that focuses on
the algorithmic issues and for which the informational
questions are trivial.
We stress while sketching may require arbitrarily
complicated and numerous queries, we show that learn-
ing can always be done by value queries as long as
we allow some probability of error.
Definition 5.1. (Balcan and Harvey [3]) A
learning algorithm sees a sequence of labeled examples
(S
i
, v(S
i
)) where S
i
is drawn according to distribution
D on subsets of N, and then is asked a query S
drawn according to the same D and needs to output an
approximation v(S). We say it is an -approximating
PMAC learning (with parameters , ) for C if for every
distribution D and every valuation v C we have that
with probability of at least 1 (over the choice of S
i

from D), Pr
S
[v(S) v(S) v(S)] 1 .
Lemma 5.1. If a class of valuations V can be -
approximately sketched with length l(n) then it can be
-PMAC-learned from O((l(n) + log
1
)/) samples.
Proof. We use the usual principle in learning theory
stating that compression implies prediction. Let s :
V 0, 1

be the sketching function that takes a

valuation as input and outputs an -sketch of it (of
size l(n) = poly(n)). For each possible value q of length
l(n, t) let us dene v
q
(S) = min
v|s(v)=q
v(S), and thus
6
We assume only existence of a sketch and do not care how it
can be obtained nor with what type of queries.
for all v and all S we have that v
s(v)
(S) v(S)
v
s(v)
(S).
Let us use r = O((l(n, t) + log
1
)/) samples
(S
i
, v(S
i
)) and nd some q such that v
q
(S
i
) v(S
i
)
v
q
(S
i
) for all samples S
i
. Such q must exist simply
since q = s(v) for the real v is such. (Notice that this
may be algorithmically hard, but since we only count
queries, it can be done in terms of information available
to the mechanism after seeing the r samples.) We will
use v
q
as our learned valuation, i.e., on input S reply
with v
q
(S).
Now let us calculate the probability that this v
q
is
not a proper approximation for v, i.e. that on at least
-fraction of Ss (weighted as in D) we do not get an
-approximation. Fix a single q for which we do not
have the required approximation and let us calculate the
probability that all for all S
i
s we did get the required
approximation: it is at most (1 )
r
. Now we use
the union bound over all possible qs of length l(n) to
conclude that the probability that we output some bad
q is at most 2
l(n)
(1 )
r
, which is bounded by using
the choice of parameters we gave.
Using the upper bounds in this paper we have that:
Corollary 5.1. The following statements are true:
The class of subadditive valuations can be

O(

n)-
PMAC-learned.
The class of coverage valuations can be (1 + )-
PMAC-learned.
The class of OXS valuations can be (1+)-PMAC-
learned.
6 A gap between XOS and Submodular
Functions
In this section we give an example of XOS function for
which no submodular function is an O(

n) approxima-
tion.
We dene an XOS function f : 2
S
R
+
, [S[ =
n, as follows. Partition S evenly into

n subsets,
T
1
, , T

n
, each of size

n. Let f
i
be an additive
function on T
i
: f
i
(T) = [T S
i
[, T S. Then let f be
the maximum of these functions, i.e., f = max
i
f
i
. By
denition, f is XOS.
Theorem 6.1. If g is a submodular function such that
f(T)

g(T) f(T) for all T S, then

n
2
.
Proof. We construct a sequence of elements
x
1
, . . . , x

n
S inductively using g. For x S,
let (x) be the i such that x T
i
. Dene
x
1
= argmax
x
g(x) [ x S
Let M
i
=
i1
j=1
T
(xj)
and G
i
= x
1
, . . . , x
i1
. Dene
x
i
= argmax
x
g(x [ G
i
) [ x SM
i
, i = 2, 3, ,

n
where g(x [ T) is g(x T) g(T), the marginal value
of x given T. In words, x
1
, . . . , x

n
are a sequence
produced by a greedy algorithm that maximizes the
marginal utility at each step, subject to the constraint
that each element chosen is from a dierent subset in
the partition.
By considering the value of f on these elements, we
have
1 = f(x
1
, . . . , x

n
)
g(x
1
, . . . , x

n
)
= g(x
1
) +

i=2
g(x
i
[x
1
, . . . , x
i1
)
By submodularity and the procedure we selected
the sequence, the terms on the right hand side are
in a nonincreasing order, and therefore b

= g(x

n
[
x
1
, . . . , x

n1
)
1

n
. On the other hand,

n = f(x
1
, . . . , x

n1
T

n
)
g(x
1
, . . . , x

n1
T

n
)
(1 + (

n 1)b) 2
The theorem immediately follows.
References
[1] Ashwinkumar Badanidiyuru, Shahar Dobzinski, and
Sigal Oren. Optimization with demand oracles. CoRR,
abs/1107.2869, 2011.
[2] Maria Florina Balcan, Florin Constantin, Satoru
Iwata, and Lei Wang. Learning valuation functions.
CoRR, abs/1108.5669, 2011.
[3] Maria-Florina Balcan and Nicholas J. A. Harvey.
Learning submodular functions. In Lance Fortnow and
Salil P. Vadhan, editors, STOC, pages 793802. ACM,
2011.
[4] Alejandro Bertelsen. Substitutes valuations and m

-
concavity. M.Sc. Thesis, The Hebrew University of
Jerusalem, 2005.
[5] Kshipra Bhawalkar and Tim Roughgarden. Welfare
guarantees for combinatorial auctions with item bid-
ding. In Dana Randall, editor, SODA, pages 700709.
SIAM, 2011.
[6] Liad Blumrosen. Information and communication in
mechanism design. PhD Thesis, The Hebrew Univer-
sity of Jerusalem, 2006.
[7] Liad Blumrosen and Noam Nisan. Combinatorial
auctions. In Noam Nisan, Tim Roughgarden,

Eva
Tardos, and Vijay Vazirani, editors, Algorithmic Game
Theory. Cambridge University Press, 2007.
[8] Liad Blumrosen and Noam Nisan. On the computa-
tional power of demand queries. SIAM J. Comput.,
39(4):13721391, 2009.
[9] S. Dobzinski, N. Nisan, and M. Schapira. Approx-
imation algorithms for combinatorial auctions with
complement-free bidders. Mathematics of Operations
Research, 35:113, 2010.
[10] Shahar Dobzinski. Two randomized mechanisms for
combinatorial auctions. In APPROX, 2007.
[11] Shahar Dobzinski, Noam Nisan, and Michael Schapira.
Approximation algorithms for combinatorial auctions
with complement-free bidders. In Proceedings of the
37th Annual ACM Symposium on Theory of Computing
(STOC), 2005.
[12] Uriel Feige. On maximizing welfare when utility func-
tions are subadditive. Siam Journal on Computing,
39:4150, 2006.
[13] Michel X. Goemans, Nicholas J. A. Harvey, Satoru
Iwata, and Vahab S. Mirrokni. Approximating sub-
modular functions everywhere. In Claire Mathieu, ed-
itor, SODA, pages 535544. SIAM, 2009.
[14] A. Krause and C. Guestrin. Beyond convex-
ity: Submodularity in machine learning, 2008.
http://submodularity.org.
[15] A. Krause and C. Guestrin. Intelligent information
gathering and submodular function optimization, 2009.
http://submodularity.org/ijcai09/index.html.
[16] Benny Lehmann, Daniel Lehmann, and Noam Nisan.
Combinatorial auctions with decreasing marginal util-
ities. Games and Economic Behavior, 55(2):270296,
2006.
[17] L. Lov asz. Submodular functions and convexity. In
Mathemtical Programming: The State of The Art,
pages 235257. Bonn, 1982.
[18] Elchanan Mossel and Sebastien Roch. Submodularity
of inuence in social networks: From local to global.
SIAM J. Comput., 39(6):21762188, 2010.
[19] Rajeev Motwani and Prabhakar Raghavan. Random-
ized algorithms. Cambridge University Press, Cam-
bridge, 1995.
[20] G. L. Nemhauser, L. A. Wolsey, and M. L. Fisher. An
analysis of approximations for maximizing submodular
set functions i. Mathematical Programming, 14, 1978.
[21] D.M. Topkis. Supermodularity and Complementarity.
Princeton University Press, 1998.