Lecture 2: Convex sets

August 28, 2008

 Lecture 2

Outline

Review basic topology in Rn

Open Set and Interior
Closed Set and Closure
Dual Cone
Convex set
Cones
Affine sets
Half-Spaces, Hyperplanes, Polyhedra
Ellipsoids and Norm Cones
Convex, Conical, and Affine Hulls
Simplex
Verifying Convexity

Convex Optimization 1
 Lecture 2

Topology Review

Let {xk } be a sequence of vectors in Rn

Def. The sequence {xk } Rn converges to a vector x Rn when
kxk xk tends to 0 as k

Notation: When {xk } converges to a vector x, we write xk x

The sequence {xk } converges x Rn if and only if for each component

i: the i-th components of xk converge to the i-th component of x
|xik xi| tends to 0 as k

Convex Optimization 2
 Lecture 2

Open Set and Interior

Let X Rn be a nonempty set
Def. The set X is open if for every x X there is an open ball B(x, r) that
entirely lies in the set X , i.e.,
for each x X there is r > 0 s.th. for all z with kz xk < r, we have z X
Def. A vector x0 is an interior point of the set X , if there is a ball B(x0, r)
contained entirely in the set X
Def. The interior of the set X is the set of all interior points of X , denoted
R
by (X)
R
How is (X) related to X ?
Example X = {x R2 | x1 0, x2 > 0}
(X) = {x R2 | x1 > 0, x2 > 0}
R

(S) of a probability simplex S = {x Rn | x  0, e0x = 1}

R

R
Th. For a convex set X , the interior (X) is also convex

Convex Optimization 3
 Lecture 2

Closed Set

Def. The complement of a given set X Rn is the set of all vectors that do
not belong to X :
the complement of X = {x Rn | x / X} = Rn \ X
Def. The set X is closed if its complement Rn \ X is open
Examples: Rn and (both are open and closed)
{x R2 | x1 0, x2 > 0} is open or closed?
hyperplane, half-space, simplex, polyhedral sets?
The intersection of any family of closed set is closed
The union of a finite family of closed set is closed
The sum of two closed sets is not necessarily closed
Example: C1 = {(x1, x2) | x1 = 0, x2 R}
C2 = {(x1, x2) | x1x2 1, x1 0}
C1 + C2 is not closed!
Fact: The sum of a compact set and a closed set is closed

Convex Optimization 4
 Lecture 2

Closure

Let X Rn be a nonempty set

Def. A vector x is a closure point of a set X if there exists a sequence
{xk } X such that xk x

Closure points of X = {(1)n/n | n = 1, 2, . . .}, X = {1 x | x X}?

Notation: The set of closure points of X is denoted by cl(X)

What is relation between X and cl(X)?

Th. A set is closed if and only if it contains its closure points, i.e.,
X is closed iff cl(X) X

Th. For a convex set, the closure cl(X) is convex

Convex Optimization 5
 Lecture 2

Boundary

Let X Rn be a nonempty set

Def. The boundary of the set X is the set of closure points for both the set
X and its complement Rn \ X , i.e.,

bd(X) = cl(X) cl(Rn \ X)

Example X = {x Rn | g1(x) 0, . . . , gm(x) 0}. Is X closed?

What constitutes the boundary of X ?

Convex Optimization 6
 Lecture 2

Dual Cone

Let K be a nonempty cone in Rn

Def. The dual cone of K is the set K defined by

K = {z | z 0x 0 for all x K}

The dual cone K is a closed convex cone even when K is neither closed
nor convex
Let S be a subspace. Then, S = S .
Let C be a closed convex cone. Then, (C ) = C .
For an arbitrary cone K , we have (K ) = cl(conv(K)).

Convex Optimization 7
 Lecture 2

Convex set

A line segment defined by vectors x and y is

the set of points of the form x + (1 )y for [0, 1]

A set C Rn is convex when, with any two vectors x and y that belong
to the set C , the line segment connecting x and y also belongs to C

Convex Optimization 8
 Lecture 2

Examples

Which of the following sets are convex?

The space Rn
A line through two given vectors x and y
l(x, y) = {z | z = x + t(y x), t R}
A ray defined by a vector x
{z | z = x, 0}
The positive orthant {x Rn | x  0} ( componentwise inequality)
The set {x R2 | x1 > 0, x2 0}
The set {x R2 | x1x2 = 0}

Convex Optimization 9
 Lecture 2

Cone

A set C Rn is a cone when, with every vector x C , the ray {x | 0}

belongs to the set C

A cone may or may not be convex

Examples: {x Rn | x  0} {x R2 | x1x2 0}
For a two sets C and S , the sum C + S is defined by
C + S = {z | z = x + y, x C, y S} (the order does nor matter)
Convex Cone Lemma: A cone C is convex if and only if C + C C
Proof: Pick any x and y in C , and any [0, 1]. Then, x and (1 )y
belong to C because... . Using C + C C , it follows that ... Reverse: Let
C be convex cone, and pick any x, y C . Consider 1/2(x + y)...

Convex Optimization 10
 Lecture 2

Affine Set

A set C Rn is a affine when, with every two distinct vectors x, y C ,

the line {x + t(y x) | t R} belongs to the set C
An affine set is always convex
A subspace is an affine set

A set C is affine if and only if C is a translated subspace, i.e.,

C = S + x0 for some subspace S and some x0 C

Dimension of an affine set C is the dimension of the subspace S

Convex Optimization 11
 Lecture 2

Hyperplanes and Half-spaces

Hyperplane is a set of the form {x | a0x = b} for a nonzero vector a

Half-space is a set of the form {x | a0x b} with a nonzero vector a

The vector a is referred to as the normal vector
A hyperplane in Rn divides the space into two half-spaces
{x | a0x b} and {x | a0x b}
Half-spaces are convex
Hyperplanes are convex and affine

Convex Optimization 12
 Lecture 2

Polyhedral Sets
A polyhedral set is given by finitely many linear inequalities
C = {x | Ax  b} where A is an m n matrix

Every polyhedral set is convex

Linear Problem
minimize c0 x
subject to Bx b, Dx = d

The constraint set {x | Bx b, Dx = d} is polyhedral.

Convex Optimization 13
 Lecture 2

Ellipsoids

Let A be a square (n n) matrix.

A is positive semidefinite when x0Ax 0 for all x Rn
A is positive definite when x0Ax > 0 for all x Rn, x 6= 0
An ellipsoid is a set of the form
E = {x | (x x0)0P 1(x x0) 1}
where P is symmetric and positive definite

x0 is the center of the ellipsoid E

A ball {x | kx x0k r} is a special case of the ellipsoid (P = r2I )
Ellipsoids are convex

Convex Optimization 14
 Lecture 2

Norm Cones
A norm cone is the set of the form
C = {(x, t) Rn R | kxk t}

The norm k k can be any norm in Rn

The norm cone for Euclidean norm is also known as ice-cream cone
Any norm cone is convex

Convex Optimization 15
 Lecture 2

Convex and Conical Hulls

A convex combination of vectors x1, . . . , xm is a vector of the form
Pm
1 x1 + . . . + m xm i 0 for all i and i=1 i =1
The convex hull of a set X is the set of all convex combinations of the
vectors in X , denoted conv(X)

A conical combination of vectors x1, . . . , xm is a vector of the form

1x1 + . . . + mxm with i 0 for all i
The conical hull of a set X is the set of all conical combinations of the
vectors in X , denoted by cone(X)

Convex Optimization 16
 Lecture 2

Affine Hull

An affine combination of vectors x1, . . . , xm is a vector of the form

Pm
t1x1 + . . . + tmxm with i=1 ti = 1, ti R for all i
The affine hull of a set X is the set of all affine combinations of the vectors
in X , denoted aff (X)

The dimension of a set X is the dimension of the affine hull of X

dim(X) = dim(aff (X))

Convex Optimization 17
 Lecture 2

Simplex
A simplex is a set given as a convex combination of a finite collection of
vectors v0, v1, . . . , vm:
C = conv{v0, v1 . . . , vm}
The dimension of the simplex C is equal to the maximum number of linearly
independent vectors among v1 v0, . . . , vm v0.
Examples
Unit simplex {x Rn | x  0, e0x 1}, e = (1, . . . , 1), dim -?
Probability simplex {x Rn | x  0, e0x = 1}, dim -?

Convex Optimization 18
 Lecture 2

Practical Methods for Establishing Convexity of a Set

Establish the convexity of a given set X

The set is one of the recognizable (simple) convex sets such as
polyhedral, simplex, norm cone, etc

Prove the convexity by directly applying the definition

For every x, y X and (0, 1), show that x + (1 )y is also in X

Show that the set is obtained from one of the simple convex sets through
an operation that preserves convexity

Convex Optimization 19
 Lecture 2

Operations Preserving Convexity

Let C Rn, C1 Rn, C2 Rn, and K Rm be convex sets. Then, the

following sets are also convex:
The intersection C1 C2 = {x | x C1 and x C2}
The sum C1 + C2 of two convex sets
The translated set C + a
The scaled set tC = {tx | x C} for any t R
The Cartesian product C1 C2 = {(x1, x2) | x1 C1, x2 C2}
The coordinate projection {x1 | (x1, x2) C for some x2}
The image AC under a linear transformation A : Rn 7 Rm:
AC = {y Rm | y = Ax for some x C}
The inverse image A1K under a linear transformation A : Rn 7 Rm:
A1K = {x Rn | Ax K}

Convex Optimization 20