2.

1 Convex Set

Yipeng Liu

School of Electronic Engineering / Center for Robotics / Center for Information in Medicine
University of Electronic Science and Technology of China (UESTC)

[email protected]

October 9, 2016

1 / 27
 Overview

convexity is of paramount importance in optimization, albeit difficult

to detect for the untrained eye.

1. definition

2. example

3. property

4. sets’ separating

5. generalized inequality

6. minimum and minimal elements

2 / 27
 Basic topology

Let E be a normed vector space, that is a space with norm k•k. (e.g. RN , with
the Euclidean norm)

• open ball centered in x and of radius r > 0:

B(x, r) = {y ∈ K : ky − xk < r}
• closed ball: B[x, r] = {y ∈ K : ky − xk 6 r}

Q ⊆ E is open if ∀x ∈ Q there is an open ball B(x, rx ) ⊆ E

Q ⊆ E is closed if E\Q is open.
Q ⊆ E is bounded if Q is contained in some open ball.
Q ⊆ E is compact if every sequence {xn }n=0,1,2,··· has at least one limit point,
and all its limit points are in Q

Theorem (Compact Closed Bounded)

if E = RN , compact is equivalent to closed and bounded.

3 / 27
 Basic topology

Let E be a normed vector space, that is a space with norm k•k. (e.g. RN , with
the Euclidean norm)

let Q ⊆ E

the closure of Q or cl(Q) is the smallest closed set containing Q, i.e.

x ∈ cl(Q) ⇔ ∀ε > 0, K ∩ B(x, ε) 6= ∅

the interior of Q or int(Q) is the largest open set contained in Q, i.e.

x ∈ int(Q) ⇔ ∀ε > 0, B(x, ε) ⊆ Q

4 / 27
 Affine set

line through x1, x2 : all points

x = θx1 + (1 − θ)x2 , θ ∈ R

example: solution set of linear

equations {x|Ax = b};
conversely, every affine set can be
expressed as solution set of system of
linear equations.
affine set: contains the line through any two distinct points in the set.
difference between affine set and linear set?

5 / 27
 Convex set

line segment between x1 and x2 : all points

x = θx1 + (1 − θ)x2

with 0 6 θ 6 1.
convex set: contains line segment between any two points in the set

x1 , x2 ∈ C, 0 6 θ 6 1 ⇒ θx1 + (1 − θ)x2 ∈ C

examples (one convex, two nonconvex sets)

6 / 27
 Convex hull

convex combination of x1 , · · · , xK : any point x of the form

x = θ1 x1 + θ2 x2 + · · · + θK xK

with θ1 + θ2 + · · · + θK = 1, θk > 0.
convex hull: conv S: set of all convex combinations of points in S

x1 , x2 ∈ C, 0 6 θ 6 1 ⇒ θx1 + (1 − θ)x2 ∈ C

7 / 27
 Convex cone
K is a cone if x ∈ K, θ > 0 implies θx ∈ K

conic (nonnegative) combination of x1 and x2 : any point of the form

x = θ1 x1 + θ2 x2

with θ1 > 0, θ2 > 0.

convex cone: convexity + cone

i.e. set that contains all conic combinations of points in the set

8 / 27
 Hyperplanes and halfspaces

hyperplane: set of the form halfspace: set of the form

{x|aT x = b}, (a 6= 0) {x|aT x 6 b}, (a 6= 0)

• a is the normal vector.

• hyperplanes are affine and convex; halfspaces are convex.

9 / 27
 Euclidean ball and Ellipsoids
(Euclidean) ball with center xc and radius r:
 
B(xc , r) = x kx − xc k2 6 r = xc + ru kuk2 6 1

ellipsoid: set of the form

n o
x (x − xc )T P−1 (x − xc ) 6 1

with P ∈ Sn
++ (i.e., P symmetric positive definite)


other representation: kxc + Auk kuk2 6 1 with A square & nonsingular

10 / 27
 Norm balls and norm cones

norm a function k • k that satisfies:

• kxk > 0; kxk = 0 if and only if x = 0 (separates points)

• ktxk = |t| kxk for t ∈ R (absolute homogeneity or absolute scalability)

• kx + yk 6 kxk + kyk (triangle inequality or subadditivity)

notation: k • k is general (unspecified) norm; k•ksymb is a particular norm

norm ball with center xc and radius r: {x |kx − xc k 6 r }

norm cone: {(x, t) |kxk 6 t }

Euclidean norm cone is called second
order cone.

norm balls and cones are convex.

11 / 27
 Polyhedra

solution set of finitely many linear inequalities and equalities

Ax  b, Cx = d

A ∈ RM ×N , C ∈ RP ×N ,  is componentwise inequality

polyhedron is intersection of finite number of halfspaces and hyperplanes.

12 / 27
 Positive semidefinite cone
notation:

• SN is set of symmetric N × N matrices

• SN N

+ = X ∈ S |X  0 : positive semidefinite N × N matrices

X ∈ SN T
+ ⇔ z Xz > 0 for all z

SN
+ is a convex cone

• SN N

++ = X ∈ S |X  0 : positive definite N × N matrices

" #
x y
example: ∈ S2+
y z

13 / 27
 Operations that preserve convexity

practical methods for establishing convexity of a set C

1. apply definition

x1 , x2 ∈ C, 0 6 θ 6 1 ⇒ θx1 + (1 − θ)x2 ∈ C
2. show that C is obtained from simple convex sets (hyperplanes, halfspaces,
norm balls, ... ) by operations that preserve convexity
• intersection
• affine functions
• perspective function
• linear-fractional functions

14 / 27
 Intersection

the intersection of (any number of) convex sets is convex

example: n o
S = x ∈ RM | |p(t)| 6 1 for |t| 6 π/3

where p(t) = x1 cos t + x2 cos(2t) + · · · + xM cos(M t): inverse cosine series

for M = 2:

the intersection of infinite number of convex sets (half spaces)

15 / 27
 Affine function

suppose f : RN → RM is affine (f (x) = Ax + b with A ∈ RM ×N , b ∈ RM )

• the image of a convex set under f is convex

S ∈ RN convex ⇒ f (S) = {f (x) |x ∈ S } convex

• the inverse image f −1 (C) of a convex set under f is convex

n o
C ∈ RM convex ⇒ f −1 (C) = x ∈ RN |f (x) ∈ C convex

examples

• scaling, translation, projection

• set of linear matrix inequality { x |x1 A1 + · · · + xM AM  B } (with

Am , B ∈ SN )
n 2
o
• hyperbolic cone x xT Px 6 (cT x) , cT x > 0 (with P ∈ SN
+)

16 / 27
 Perspective and linear-fractional function

perspective function P : RN +1 → RN

P (x, t) = x/t, dom P = {(x, t) |t > 0 }

images and inverse images of convex sets under perspective are convex.

linear-fractional function f : RN → RM

Ax + b n
T
o
f (x) = , dom f = x c x + d > 0
cT x + d
images and inverse images of convex sets under linear-fractional functions are
convex.

17 / 27
 Perspective and linear-fractional function

example of a linear-fractional function

1
f (x) = x
x1 + x2 + 1

18 / 27
 Separating hyperplane theorem

if C and D are disjoint convex sets, then there exists a 6= 0, b such that

aT x 6 b for x ∈ C, aT x > b for x ∈ D

the hyperplane H = {aT x = b} separates C and D

strict separation requires additional assumptions (e.g., C is closed, D is a
singleton)

19 / 27
 Supporting hyperplane theorem
supporting hyperplane to set C at boundary point x0 :

{x aT x = aT x0 }

where a 6= 0 and aT x 6 aT x0 for all x ∈ C

supporting hyperplane theorem: if C is convex, then there exists a supporting

hyperplane at every boundary point of C.
20 / 27
 Generalized inequalities

a convex cone K ⊆ RN is a proper cone if

• K is closed (contains its boundary)

• K is solid (has nonempty interior)

• K is is pointed (contains no line)

examples

• nonnegative orthant K = RN N

+ = x ∈ R |xn > 0

• positive semidefinite cone K = SN

+

• nonnegative polynomials on [0, 1]:

n o
K = x ∈ RN x1 + x2 t + x3 t2 + · · · + xN tN −1 > 0 for t ∈ [0, 1]

21 / 27
 Generalized inequalities

generalized inequality defined by a proper cone K:

x K y ⇔ y − x ∈ K, x ≺K y ⇔ y − x ∈ int K

examples

• componentwise inequality (K = RN
+)

x RN y ⇔ xn 6 yn , n = 1, 2, · · · , N
+

• matrix inequality (K = SN
+)

X KN Y, Y − X is positive semidefinite

+

these two types are so common that we drop the subscript in K .

properties: many properties of K are similar to 6 on R, e.g.,

x ≺K y, x K y ⇔ x + u K y + v

22 / 27
 Dual cones and generalized inequalities

dual cone of a cone K:

n o
K∗ = y yT x > 0 for all x ∈ K

examples
∗
• K = RN N
+ : K = R+
∗
• K = SN N
+ : K = S+

: K∗ = (x, t) kxk2 6 t
 
• K = (x, t) kxk2 6 t

these three examples are self-dual cones

dual cones of proper cones are proper, hence define generalized inequalities:

y K∗ 0 ⇔ yT x > 0 for all x K 0

23 / 27
 Minimum and minimal elements

x ∈ S is the minimum element of S with respect to K if

y ∈ S ⇒ x K y

x ∈ S is a minimal element of S with respect to K if

y ∈ S, y K x ⇒ y = x

example:
K = R2+
x1 is the minimum element of S1 .
x2 is the minimal element of S2 .

24 / 27
 Minimum and minimal elements via dual inequalities

minimum element w.r.t. K

x is minimum element of S iff for all
λK∗ 0, x is the unique minimizer of
λT z over S.

minimal element w.r.t. K

• if x minimizes λT z over S for some λK∗ 0, then x is minimal.

• if x is a minimal element of a convex set S, then there exists a nonzero

λK∗ 0 such that x minimizes λT z over S.

25 / 27
 Minimum and minimal elements via dual inequalities

optimal production frontier

• different production methods use different amounts of resources x ∈ RN

• production set P: resource vectors x for all possible production methods

• efficient (Pareto optimal) methods correspond to resource vectors x that

are minimal w.r.t. RN
+

example (N = 2)
x1 , x2 , x3 are efficient;
x4 , x5 are not

26 / 27
 Infimum and minimum

let f : K ∈ RN → R

the infimum of f on K is the largest t, such that f (x) > t for every x ∈ K.
example: inf x>0 1/x = 0

the infimum is called a minimum if the infimum is attained:

∃x0 ∈ K such that f (x0 ) = inf f (x)

x∈K

example: a minimizer is such a point x0 :

minimize cos(x) = −1
x∈R

the set of minimizers being {(2k + 1)π : K ∈ Z}

The supremum, maximum and maximizers are defined similarly.

27 / 27