EE5121: Optimization Methods for Signal Processing and

Communication
Partial Solutions to Assignment 1 (As done in tutorial session)

1. Find the supremum and the infimum of the following sets:

(a) A1 = { 1n ; n = 1, 2, 3, . . .}
(b) A2 = {21/n , n = 1, 2, . . .}
(c) A3 = (−3, 4) ∪ [6, 10]
(d) A4 = (−3, 4) ∩ (2, 6)
(e) A5 = (−3, 4) ∩ [6, 10]
x
(f) A6 = { 1+x : x ∈ R, x ≥ 0}
Solution:
(a) supremum : 1
infimum : 0
(b) supremum : 2
infimum : 1
(c) supremum : 10
infimum : -3
(d) supremum : 4
infimum : 2
(e) supremum : −∞
infimum : +∞
By convention.
(f) supremum : 1
infimum : 0
2. Let E be a nonempty subset of an ordered set; suppose α is a lower bound of E and β is an upper
bound of E. Prove that α ≤ β .
Solution: Hint: The following holds ∀x ∈ E: α ≤ x ≤ β . Use the definition of ordered set to complete
the proof.
3. Let A be a nonempty set of real numbers which is bounded below. Let −A be the set of all numbers
−x, where x ∈ A. Prove that

inf A = − sup(−A)

Solution: Hint: Follows directly from the definitions of the supremum and the infimum.
4. Given A ⊂ R, can sup(A) = inf(A)? If so, characterize all such A.
Solution: sup(A) = inf(A) if and only if A is a singleton set. Hint: Assume that A is a singleton set,
and show that sup(A) = inf(A). Next, assume that sup(A) = inf(A) holds, and infer that this can be
true only if A is a singleton set.
5. Intuitively, for what class of subsets of R is sup(A) = max(A) true?
Solution: sup(A) ∈ A, that is, A is “closed from the right side.” Note that this is trivially true for
closed sets and compact sets.
6. For x ∈ R and y ∈ R, define
(a) d1 (x, y) = (x − y)2
p
(b) d2 (x, y) = |x − y|

1
 (c) d3 (x, y) = |x2 − y2 |
(d) d4 (x, y) = |x − 2y|
Check if each of the above is a metric.
Solution:
(a) d1 (x, y) = (x − y)2 is not a valid metric. Produce x, y, z ∈ R such that the triangle inequality is
violated.
p
(b) d2 (x, y) = |x − y| is√indeed a valid metric. Hint: Proving the triangle inequality is equivalent
√ √
to proving x + y ≤ x + y ∀x, y ≥ 0.
(c) d3 (x, y) = |x2 − y2 | is not a valid metric. (Why?)
(d) d4 (x, y) = |x − 2y| is not a valid metric. (Why?)
7. Let X be an infinite set, For p ∈ X and q ∈ X, define
(
1 if (p 6= q)
d(p, q) =
0 if (p = q).

Prove that this is a metric. Which subsets of the resulting metric space are open? Which are closed?
Which are compact?
Solution:
(a) i. From the definition, it is obvious that d(p, q) ≥ 0 ∀p, q ∈ X and d(p, q) = 0 if and only if
p = q.
ii. Similarly, it follows from the definition that d(p, q) = d(q, p).
iii. To show the triangle inequality d(p, z) ≤ d(p, q) + d(q, z), note that, for p, q, z ∈ X, there
are two cases: (i) p = z, and (ii) p 6= z. Case (ii) further has three sub-cases (a) p = q and
q 6= z, (b) p 6= q and q = z, and (c) p 6= q and q 6= z. For each of these cases, we see that
d(p, z) ≤ d(p, q) + d(q, z) holds.
Thus, d is a valid metric
(b) For a fixed p ∈ X, the neighborhood
(
{p}, if 0 ≤ r < 1,
Nr (p) = {q ∈ Z | d(q, p) < r} =
X, if r ≥ 1.

Hence, for each p ∈ X, when 0 ≤ r < 1, Nr (p) ⊆ {p}. Therefore, every singleton in X is open
with respect to the discrete metric. Next, note that any C ⊆ X can be written as

C = ∪ p∈C {p}.

Hence, every subset of X is open with respect to the discrete metric.

(c) Each subset D ⊆ X can be written as D = Cc , for some C ⊆ X. Hence, every subset of X is
closed with respect to the discrete metric.

8. Is every point of every open set E ⊂ R2 a limit point of E? Answer the same question for closed sets
in R2 .
Solution:
(a) Yes. Every point of an open set E is a limit point of E.
To see this, consider x ∈ E. Since this is an interior point, ∃ r0 > 0 such that Nr0 (x) ⊆ E. Hence,
for all r ≥ 0, Nr (x) \ {x} ∩ E 6= φ . Hence, it is also a limit point of E.
(b) No. Every point of a closed set E need not be a limit point of E. For example, a singleton set is
closed. However, it has no limit points.

9. Check if the following set is a closed subset of R2 and provide reasons for your claim.

A1 = {(x, y) : x > 0, y > 0, 3x + 2y ≤ 1}.

Solution: A1 is not closed. For instance, the origin (0, 0) ∈

/ A1 is a limit point of A1 .

2
10. Let f be a continuous real function on a metric space X. Let Z( f ) (the zero set of f ) be the set of all
p ∈ X at which f (p) = 0. Prove that Z( f ) is closed.
Solution: Let X,Y be metric spaces. Then, a function f : X → Y is continuous if and only if f −1 (A)
is open (respectively, closed) in X for every A open (respectively, closed) in Y .
Note that {0} is closed in R. Hence, Z( f ) = f −1 ({0}) is closed.

11. Let f (x) and g(x) be two continuous functions from Rd to R. Also g(x) 6= 0. Is max{ f (x), g(x)} a
continuous function? Provide justification.
Solution: Given that f , g : Rd → R are continuous. Also, h(x) = max( f (x), g(x)) ∀ x ∈ Rd .
Fix a ∈ Rd . Then, there are two possible cases:
(i) | f (a) − g(a)| > 0 or
(ii) f (a) = g(a).
For case (i), due to the strict inequality, there exists a neighborhood around the point a, where either
f > g or g > f . Hence, h = f or h = g in this neighborhood. Since both f and g are continuous, so is
h.
For case (ii), note that we have f (a) = g(a) = h(a). Fix ε > 0.
Since f is continuous, ∃ δ1 > 0, such that whenever |x − a| < δ1 , | f (x) − f (a)| < ε.
Since g is continuous, ∃ δ2 > 0, such that whenever |x − a| < δ2 , |g(x) − g(a)| < ε.
To check if h(x) is continuous, we compute:

|h(x) − h(a)| = | max( f (x) − g(x)) − h(a)|,

≤ max(| f (x) − h(a)|, |g(x) − h(a)|),
= max(| f (x) − f (a)|, |g(x) − g(a)|).

Define δ = min(δ1 , δ2 ). Whenever |x − a| < δ , we have

|h(x) − h(a)| < ε.

Therefore, h(x) = max( f (x), g(x)) is also a continuous function.

12. Are the following sets compact: provide some arguments
(a) A1 = [5, 10] ∪ {0}
(b) A2 = {x; x ∈ Rd , kxkα ≤ 1}
(c) A3 = {x; x ∈ Rd , Ax ≥ b}, A is invertible d × d matrix and b 6= 0

Solution:
(a) Yes, the given set is compact.
(b) The unit norm ball is compact ∀ 1 ≤ α ≤ ∞.
(c) The given half space is not compact.
13. Plot the following subsets of R2 :
(a) kxk1 ≤ 1, x ∈ R2
(b) kxk2 ≤ 1, x ∈ R2
(c) kxk∞ ≤ 1, x ∈ R2

Solution:

3
14. Let A be a m × n real matrix. For any x ∈ Rn , define

kxkA = kAxk2 .

Is kxkA a norm in Rm for any matrix A?. What properties of A will make k.kA a norm? (Hint: Check
when the norm becomes zero, and relate it to the Null space of the matrix. If you have not heard
of null space, it will be a good time to review basic linear algebra, especially solutions of linear
equations.)
Solution: kxkA is a norm if and only if A has a trivial null space. (Why?)

15. Let V be a vector space. Show that the sum of two inner products on V is an inner product on V .
Is the difference of two inner products an inner product? Show that a positive multiple of an inner
product is an inner product.
Solution: The sum of two inner products and a positive multiple of an inner product are valid inner
products. However, the difference of two inner products is not a valid inner product.
16. Show that if two subspaces V and W of an inner product space are orthogonal, then they are inde-
pendent.
Solution: Let x ∈ V and y ∈ W , x, y 6= 0. Then orthogonality implies hx, yi = 0.
Assume that ∃ c1 and c2 s.t c1 x + c2 y = 0.
∴ hc1 x + c2 y, xi = 0
⇒ c1 hx, xi + c2 hx, yi = 0
⇒ c1 hx, xi = 0. But as x 6= 0, hx, xi > 0. ∴ c1 = 0. Similarly, it can be shown that c2 = 0.
⇒ x and y are independent. Hence, the subspaces V and W are independent.