75033 Linear analysis 1999

Elements of mathematical analysis

Harald Hanche-Olsen
[email protected]

Abstract. These notes were written as a supplement to a course on partial

differential equations (PDEs), but have since been adapted for use in a course
on linear analysis.
This material is covered in many books. The presentation in this note is
quite terse, but I hope the motivated reader will not have any serious difficulty
reading it.
If you find misprints or other mistakes or shortcomings of these notes, I
would like to hear about it — preferably by email.

Introduction
in which the author tries to explain why studying this note is useful, and
gives fatherly advice on how to do so.
In a sense, mathematical analysis can be said to be about continuity.
The epsilon–delta arguments that you meet in a typical calculus course
represent the beginnings of mathematical analysis. Unfortunately, too
often these definitions are briefly presented, then hardly used at all
and soon forgotten in the interest of not losing too many students and
because, frankly, it is not that important in elementary calculus. As
mathematics becomes more abstract, however, there is no way to pro-
ceed without a firm grounding in the basics. Most PDEs, for example,
do not admit any solution by formulas. Therefore, emphasis is on differ-
ent questions: Does a solution exist? If so, is it unique? And if so, does
it depend on the data in a continuous manner? When you cannot write
up a simple formula for the solution of a PDE, you must resort to other
methods to prove existence. Quite commonly, some iterative method is
used to construct a sequence which is then shown to converge to a solu-
tion. This requires careful estimates and a thorough understanding of
the underlying issues. Similarly, the question of continuous dependence
on the data is not a trivial task when all you have to work with is the
existence of a solution and some of its properties.

How to read these notes. The way to read these notes is slowly.
Because the presentation is so brief, you may be tempted to read too

Version 2004–01–15
Elements of mathematical analysis 2

much at a time, and you get confused because you have not properly
absorbed the previous material. If you get stuck, backtrack a bit and
see if that helps.
The core material is contained in the first four sections — on met-
ric spaces, completeness, compactness, and continuity. These sections
should be read in sequence, more or less. The final two sections, one
on ordinary differential equations and one on the implicit and inverse
function theorems, are independent of each other.
The end of a proof is marked with in the right margin. Sometimes,
you see the statement of a theorem, proposition etc. ended with such
a box. If so, that means the proof is either contained in the previous
text or left as an exercise (sometimes trivial, sometimes not — but
always doable, I hope). If a proof is not complete, then this is probably
intentional — the idea is for you to complete it yourself.

Metric spaces
in which the basic objects of study are introduced, and their elementary
properties are established.
Most of mathematical analysis happens in some sort of metric space.
This is a set in which we are given some way to measure the distance
d(x, y) between two points x and y. The distance function (metric) d
has to satisfy some simple axioms in order to be useful for our purposes.
Later, we shall see that a metric space is the proper space on which to
define continuity of functions (actually, there is a more general concept
— that of a topological space — that is even more appropriate, but we
shall not need that level of abstraction here).

1 Definition. A metric on a set X is a real-valued function d : X ×

X → R satisfying:

d(x, x) = 0
d(x, y) > 0 if x 6= y
d(x, y) = d(y, x) (symmetry)
d(x, z) ≤ d(x, y) + d(y, z) (the triangle inequality)

A metric space is a pair (X, d) where X is a set and d a metric on X

(however we often speak of the metric space X, where d is understood).

Version 2004–01–15
3 Elements of mathematical analysis

Before moving on to the examples, we shall note that the triangle in-
equality can easily be generalised to more than three elements. For
example, two applications of the triangle inequality yields the inequal-
ity
d(x, w) ≤ d(x, y) + d(y, w) ≤ d(x, y) + d(y, z) + d(z, w).
In fact, it is not difficult to prove the general inequality
n
X
d(x0 , xn ) ≤ d(xk−1 , xk )
k=1

by induction on n. This is sometimes called the generalised triangle

inequality, but we shall simply call this the triangle inequality as well.
We shall resist the temptation to call it the polygonal inequality. While
the original triangle inequality corresponds to the fact that the sum
of two sides in a triangle is at least as large as the third, the above
inequality corresponds to a similar statement about the sides of an
n + 1-gon.

2 Examples.
R or C with d(x, y) = |x − y|.
qP
n
Rn or Cn with d(x, y) = kx − yk (where kxk = j=1 |xj |2 ).
Any set X with d(x, x) = 0 and d(x, y) = 1 whenever x 6= y. This is
called a discrete metric space.

3 Definition. A norm on a real or complex vector space X is a map

k · k : X → R satisfying:
kxk > 0 if x 6= 0
kaxk = |a| · kxk for every scalar a and x ∈ X
kx + yk ≤ kxk + kyk (the triangle inequality)
A normed space is a vector space with a norm on it. Such a space is
also a metric space with d(x, y) = kx − yk.

Version 2004–01–15
Elements of mathematical analysis 4

4 Examples.
qP
n
Rn or Cn with kxk = j=1 |xj |2 .
Pn
Rn or Cn with kxk = j=1 |xj |.
n n
R or C with kxk = max{|xj | : 1 ≤ j ≤ n}.
The space l∞ consisting of all bounded sequences x = (x1 , x2 , . . .) of
real (or complex) numbers, with kxk = max{|xj | : 1 ≤ j ≤ ∞}.

We shall often need to consider subsets of a metric space as metric

spaces in their own right. Thus, if (X, d) is a metric space and A is a
subset of X, then (A, d|A×A ) is a metric space. (The notation f |S is
often used to denote the restriction of a function f to the subset S,
in the sense that f |S (x) = f (x) when x ∈ S but f |S (x) is not defined
when x ∈ / S.) We say that A is given the metric inherited, or induced
from X. Later, we shall define what it means for a metric space to be
compact. Then we shall say that a subset A ⊆ X is compact if it is
compact when given the inherited metric, and we can do similarly with
any other concept relating to metric spaces.

5 Definition. The open ε-neighbourhood, also called the open ε-ball

(where ε > 0) of a point x in a metric space X is

Bε (x) = {ξ ∈ X : d(x, ξ) < ε}

The corresponding closed ε-neighbourhood (or –ball) is

B̄ε (x) = {ξ ∈ X : d(x, ξ) ≤ ε}

A neighbourhood of a point in a metric space is a subset containing an

ε-ball for some ε > 0. A set is open if it is a neighbourhood of every
one of its points. The interior of a subset A of X is the set of all points
for which A is a neighbourhood.

Note that x ∈ Bε (x), and that Bε (x) may consist of no points other
than x itself (if, for example, X is given the discrete metric and ε ≤ 1).
It should also be noted that B̄ε (x) is not necessarily the closure of Bε (x)
(for example, with the discrete metric, B1 (x) = {x} while B̄1 (x) = X).

Version 2004–01–15
5 Elements of mathematical analysis

6 Exercise. Use the triangle inequality to show that an open ε-ball is

in fact open. Prove that the interior of any set is open, and that the
interior of A is in fact the largest open subset of A. Finally, show that
the complement of a closed ε-ball, that is a set of the form X \ B̄ε (x),
is open.

7 Definition. A sequence (xn )∞ n=1 in a metric space X is said to con-

verge to a limit x ∈ X (and we write xn → x) if, for each ε > 0, there
is an index N so that d(xn , x) < ε whenever n ≥ N . A part (xn )∞ n=N
is called a tail of the sequence. Thus, the sequence (xn ) converges to
the limit x if and only if every neighbourhood of x contains some tail
of the sequence. A sequence is called convergent if it converges to some
limit.

8 Exercise. Show that no sequence can have more than one limit.
As a result of the above exercise, we can talk about the limit of a
convergent sequence, and write limn→∞ xn for the limit. Though a se-
quence can have only one limit, a non-convergent sequence can have
many limit points:

9 Definition. A limit point of a sequence (xn ) is a point x ∈ X such

that, for every ε > 0 and every N , there is some n ≥ N so that
d(xn , x) < ε. Equivalently, every neighbourhood of x contains at least
one point (and therefore infinitely many points) from every tail of the
sequence.

10 Exercise. A sequence (yk )∞ k=1 is called a subsequence of a sequence

(xn )∞
n=1 if it is possible to find n1 < n2 < · · · so that yk = xnk for all
k. Show that x is a limit point of the sequence (xn ) if and only if some
subsequence of (xn ) converges to x.

11 Definition. A subset F of a metric space X is called closed if,

whenever a sequence contained in F converges to a point in X, the
limit is in F .

Version 2004–01–15
Elements of mathematical analysis 6

12 Proposition. For a subset F of a metric space X the following are

equivalent:

(a) F is closed,
(b) the complement X \ F is open,
(c) for every x ∈ X, if every neighbourhood of x has a nonempty
intersection with F , then x ∈ F ,
(d) F contains every limit point of every sequence contained in F .

13 Definition. The closure of a subset A of the metric space X is the

set Ā of limits of all convergent sequences in A.

14 Proposition. The closure of a subset A of a metric space X is a

closed set containing A, and is in fact the smallest closed set containing
A. It also consists of all limit points of sequences in A. Finally, a point
x belongs to Ā if and only if every neighbourhood of x has a nonempty
intersection with A.

15 Proposition. The union of an arbitrary family of open sets is open,

and the intersection of an arbitrary family of closed sets is closed. The
intersection of a finite family of open sets is open, and the union of a
finite family of closed sets is closed.

For the sake of completeness, I include the definition of a topological space

here. This is a set X together with a family O of subsets of X satisfying the
following requirements:
∅ ∈ O and X ∈ O,
If U ∈ O and V ∈ O then U ∩ V ∈ O,
The union of an arbitrary subfamily of O is in O.
The members of O are called open, and their complements are called closed.
A neighbourhood of a point is a set containing an open set containing the
point.
Clearly, a metric space together with its open sets is a topological space.
In these notes you may notice that notions like continuity and compactness
have equivalent formulations in terms of open or closed sets (or neighbour-
hoods). These notions, then, can be generalised to topological spaces.

Version 2004–01–15
7 Elements of mathematical analysis

Completeness
in which we already encounter our first Theorem.

16 Definition. A sequence (xn ) is said to be a Cauchy sequence if, for

each ε > 0 there is some N so that d(xm , xn ) < ε whenever m, n ≥ N .

It is easy to see that any convergent sequence is Cauchy (exercise), but

the converse is not true, as can be seen from the metric space Q (all
√ the usual metric d(x, y) = |x − y|). A sequence
rational numbers, with
in Q converging to 2 in R is convergent in R and hence Cauchy, but
is not convergent in Q.

17 Definition. A metric space in which every Cauchy sequence con-

verges is called complete. A complete normed space is called a Banach
space.

One reason to be interested in complete spaces is that one can often

prove existence theorems by somehow constructing a Cauchy sequence
and considering its limit: Proving that a sequence converges may be
difficult, as you need to know the limit before you can use the definition
of convergence; by comparison, showing that a sequence is Cauchy may
be much easier. As an example, we state and prove the Banach fixed
point theorem, also known as the contraction principle.

18 Definition. A contraction on a metric space X is a function f from

X to itself so that there is a constant K < 1, such that

d(f (x), f (y)) ≤ Kd(x, y)

for every x, y ∈ X. A fixed point of a map f : X → X is a point x ∈ X

so that f (x) = x.

19 Theorem. (Banach’s fixed point theorem) A contraction on a

nonempty complete metric space has one, and only one, fixed point.

Proof: Let f : X → X be a contraction, and 0 < K < 1 such that

d(f (x), f (y)) ≤ Kd(x, y) whenever x, y ∈ X. Let x0 ∈ X be any point,
and define the sequence (xn ) by xn+1 = f (xn ), n = 0, 1, 2, . . .. It is

Version 2004–01–15
Elements of mathematical analysis 8

easy to prove by induction that d(xn+1 , xn ) ≤ K n d(x1 , x0 ) and hence,

by repeated use of the triangle inequality, whenever 1 ≤ m < n we get
n−1
X n−1
X
d(xn , xm ) ≤ d(xk+1 , xk ) ≤ K k d(x1 , x0 )
k=m k=m
∞
X Km
< d(x1 , x0 ) K k = d(x1 , x0 )
1−K
k=m

and since 0 < K < 1 it is then clear that (xn ) is a Cauchy sequence,
and hence convergent since X is complete. Let x be the limit.
We need to prove that x is a fixed point. We know that xn is an ap-
proximate fixed point when n is large, in the sense that d(f (xn ), xn ) →
0 when n → ∞ (because d(f (xn ), xn ) = d(xn+1 , xn ) ≤ K n d(x1 , x0 )).
We perform a standard gymnastic exercise using the triangle inequality:
f (x) is close to f (xn ) = xn+1 which is close to x. More precisely:

d(f (x), x) ≤ d(f (x), f (xn )) + d(xn+1 , x)

≤ Kd(x, xn ) + d(xn+1 , x) → 0 (n → ∞)

Thus, for any ε > 0 we can use the above inequality with a sufficiently
large n to obtain d(f (x), x) < ε, and since ε > 0 was arbitrary, we must
have d(f (x), x) = 0. Thus f (x) = x, and x is indeed a fixed point of f .
It remains to prove the uniqueness of the fixed point. So, assume x
and y are fixed points, that is, f (x) = x and f (y) = y. Then

d(x, y) = d(f (x), f (y)) ≤ Kd(x, y)

and since 0 < K < 1 while d(x, y) ≥ 0, this is only possible if d(x, y) =
0. Thus x = y, and the proof is complete.
In many applications of the fixed point theorem, we are given a
function which is not a contraction on the entire space, but which is
so locally. In this case, we need some other condition to ensure the
existence of a fixed point. In the following very useful case, it turns out
that the proof of the Banach fixed point theorem can be adapted.

20 Corollary. Assume X is a complete metric space, that x0 ∈ X,

and that f : B̄r (x0 ) → X is a continuous function. Assume further that

Version 2004–01–15
9 Elements of mathematical analysis

K < 1 is so that

d(f (x), f (y)) ≤ Kd(x, y) (x, y ∈ B̄r (x0 )),

d(f (x0 ), x0 ) ≤ (1 − K)r.

Then f has a unique fixed point in B̄r (x0 ).

Proof: The uniqueness proof is just as in the theorem. Next, let x1 =

f (x0 ), and more generally xn+1 = f (xn ) whenever xn is defined and
xn ∈ B̄r (x0 ). Just as in the proof of the theorem, we find

Km
d(xn , xm ) < d(x1 , x0 )
1−K
provided x0 , . . . , xn are defined. With m = 0, this becomes
1
d(xn , x0 ) < d(f (x0 ), x0 ) ≤r
1−K

using the assumption. Thus xn ∈ B̄r (x0 ), and therefore we can define
xn+1 . By induction, then, xn is defined and in B̄r (x0 ) for all n. The
proof that this sequence converges to a limit which is a fixed point is
just like before.

21 Proposition. A subset of a complete space is complete if and only

if it is closed.

Proof: Let X be a complete metric space, and A a subset of X.

First, assume that A is closed. To show that A is complete, consider
a Cauchy sequence in A. Then this sequence is also a Cauchy sequence
in X. But because X is complete, the sequence has a limit in X. Since
A is closed and the original sequence was contained in A, the limit
belongs to A. Thus the sequence converges in A, and we have proved
that A is complete.
Second, assume that A is complete. To show that A is closed in X,
consider a sequence in A converging to some point x ∈ X. Since A is
complete, this sequence also has a limit in A. But no sequence can have
more than one limit, so the latter limit must be x, which therefore must
belong to A. Thus A is closed.

Version 2004–01–15
Elements of mathematical analysis 10

22 Definition. The diameter of any subset A of a metric space is

diam A = sup{d(x, y) : x, y ∈ A}

23 Proposition. A metric space X is complete if and only if whenever

F1 ⊇ F2 ⊇ F3 ⊇ · · ·Tare closed nonempty subsets of X with diam Fn →
∞
0, the intersection n=1 Fn is nonempty.
T∞
Clearly n=1 Fn , if nonempty, has diameter 0, and so contains only
a single point.
Proof: First, assume X is complete, and let F1 ⊇ F2 ⊇ F3 ⊇ · · · be
closed nonempty subsets of X. Pick xn ∈ Fn for each n. If diam Fn → 0
the sequence (xn ) is Cauchy and hence convergent. The limit x belongs
to each FnTbecause xj ∈ Fn whenever j ≥ n, and because Fn is closed.
∞
Thus x ∈ n=1 Fn .
To prove the converse, T∞ let (xn ) be a Cauchy sequence, and let Fn =
{x
T∞ n , xn+1 , . . .}. Then n=1 Fn is the set of limit points of (xn ). Since
n=1 Fn 6= ∅, thus (xn ) has a limit point, which must be a limit of
(xn ).

24 Exercise. The above proof has several gaps and details left out.
Identify these, and fill them in.

Compactness
in which we define a most useful property of metric spaces such as closed and
bounded intervals.

25 Definition. A metric space is called compact if every sequence in

the space has at least one limit point (and hence a convergent subse-
quence).
Note that R is not compact, since the sequence xn = n has no limit
point.

26 Exercise. Prove that any closed subset of a compact metric space

is compact. Also prove that every compact subset of any metric space
is closed.

Version 2004–01–15
11 Elements of mathematical analysis

27 Definition. A metric space X is called totally bounded if, for every

ε > 0, there exist a finite number of points x1 , . . . , xn ∈ X so that, for
every y ∈ X, one of the points xi satisfies d(xi , y) < ε. In other words,
X is a finite union of ε-balls for every ε > 0.

28 Definition. A cover of X is a set of subsets of X whose union is

all of X. An open cover is a cover consisting of open sets.

29 Definition. A set F of subsets of X has the finite intersection

property if every finite subset of F has nonempty intersection; i.e., if
F1 , . . . , Fn ∈ F , then F1 ∩ · · · ∩ Fn 6= ∅.

30 Theorem. For a metric space X, the following are equivalent:

(a) X is compact,
(b) every open cover of X contains a finite cover of X,
(c) every set of closed subsets of X with the finite intersection prop-
erty has nonempty intersection,
(d) X is totally bounded and complete.

Proof: We prove (d) ⇒ (c) ⇒ (a) ⇒ (d). The proof of the equivalence
(b) ⇔ (c) will be left as an exercise. (Hint: F is a set of closed sets
with the finite intersection property but with empty intersection if and
only if {X \ F : F ∈ F } is an open cover with no finite subset which is
also a cover.)
(d) ⇒ (c): Assume X is totally bounded and complete, and let F
be a set of closed subsets of X with the finite intersection
Sn property. If
ε > 0, by the total boundedness we may write X = k=1 B̄ε (xk ). Let
Gk = {F ∩ B̄ε (xk ) : F ∈ F }. At least one of the families G1 , . . . , Gn has
the finite intersection property (exercise: prove this). Clearly each set
in Gk has diameter at most 2ε.
So far we have proved: Every set F of closed sets with the finite
intersection property has a refinement G (by which we mean a family
of closed sets, also with the finite intersection property, so that for every
F ∈ F there exists some G ∈ G with G ⊆ F ), each of whose members
has diameter no larger than some prescribed positive number.
Let now εk & 0. Let F0 = F and, for k = 1, 2, . . . let Fk be a
refinement of Fk−1 , each of whose members has diameter at most εk .

Version 2004–01–15
Elements of mathematical analysis 12

Next let F0 = X and, for k = 1, 2, . . . let Fk ∈ Fk with Fk ⊆ T Fk−1 . Now

∞
apply Proposition 23 to see that, by the completeness of X, k=1 Fk =
{x} for some x ∈ X.
Now let G ∈ F . For each k, since Fk refines F there is some Gk∈ Fk
with Gk ⊆ G. Since Fk has the finite intersection property, Fk ∩Gk 6= ∅,
so let xk ∈ Fk ∩ Gk . Since xk ∈ Fk for each k, x = limk→∞ xk . But
xk ∈ G and G is closed, and therefore T x ∈ G. Since G was an arbitrary
member of F , we have shown x ∈ F , so that F does indeed have a
nonempty intersection.
(c) ⇒ (a): Let (xn ) be a sequence in X, and let Fn = {xn , xn+1 , . . .}.
Clearly, F =T{F1 , F2 , . . .} has the finite intersection property, and its
∞
intersection n=1 Fn consists of all limit points of (xn ). If (c) holds
there is therefore at least one limit point.
(a) ⇒ (d): Assume X is compact. Clearly it is complete, for if a
Cauchy sequence has at least one limit point, that limit point is unique
and the sequence converges to that point (exercise).
Assume X is not totally bounded. Then there is some ε > 0 so that
no finite number of ε-balls covers SnX. Let x1 ∈ X be arbitrary, and for
n = 1, 2, 3, . . . pick xn+1 ∈ X \ k=1 Bε (xk ). Then (xn ) is a sequence
in X so that d(xm , xn ) ≥ ε whenever m 6= n. Such a sequence can have
no limit point, since no open ε/2-ball can contain more than one point
from the sequence. This contradicts the compactness of X.

The real numbers

in which we prove the Heine–Borel theorem and the completeness of the field
of real numbers.
We shall take the following fundamental property of R for granted:
Every nonempty subset S ⊆ R which has an upper bound has a least
upper bound a = sup S. More precisely, that a is an upper bound of
S means that x ≤ a for every x ∈ S. That a is a least upper bound
means that it is an upper bound for S, such that a ≤ b whenever b is
an upper bound for S. Clearly, the least upper bound is unique. For
completeness, we set sup ∅ = −∞, and sup S = +∞ if S has no upper
bound. The greatest lower bound inf S is defined similarly, but with
all the inequalities reversed. The existence of the greatest lower bound
can be deduced from the existence of the least upper bound by taking

Version 2004–01–15
13 Elements of mathematical analysis

negatives; in fact inf S = − sup{−s : s ∈ S}. For completeness, we set

inf ∅ = +∞, and inf S = −∞ if S has no lower bound.

31 Lemma. For any closed and bounded set F ⊆ R, sup F ∈ F .

Proof: Let a = sup F . If a ∈/ F then, since F is closed, there is some
ε > 0 so that F ∩ Bε (a) = ∅. But if x ∈ F then x ≤ a because a is an
upper bound for F , and so x ≤ a − ε (since otherwise |x − a| < ε). Thus
a − ε is an upper bound for F , which contradicts the definition of a as
the least upper bound for F . This contradiction shows that a ∈ F .

32 Theorem. (Heine–Borel) Every bounded and closed set of real

numbers is compact.
Historically, the version of compactness to be proven below is called
Cantor’s intersection theorem, while it is the open covering version that
is properly called the Heine–Borel theorem. The fact that any bounded
sequence of real numbers has a limit point is known as the Bolzano–
Weierstrass theorem.
Proof: Let K ⊆ R be a bounded and closed set, and let F be a family
of closed subsets of K, with the finite intersection property. Define F 0
to be the set of all intersections of finite subsets of F , and finally let

ω = inf{sup F : F ∈ F 0 }.

The claim is that ω ∈ F . Thus let F ∈ F . We need to prove that

T
ω ∈ F . Since F is closed, we only need to prove – for any ε > 0 – that
F ∩ Bε (ω) 6= ∅.
By the definition of ω, there is some G ∈ F 0 so that sup G < ω + ε.
Since F ∩ G ∈ F 0 , we have sup(F ∩ G) ≥ ω, and of course sup(F ∩ G) ≤
sup G < ω + ε. Hence sup(F ∩ G) belongs to Bε (ω). By Lemma 31,
sup(F ∩ G) ∈ F , so sup(F ∩ G) ∈ F ∩ Bε (ω), which proves the claim.

33 Corollary. Every bounded sequence of real numbers has a limit

point.

34 Corollary. R is complete.

Version 2004–01–15
Elements of mathematical analysis 14

Proof: A Cauchy sequence is certainly bounded. Thus it has a limit

point. But any limit point of a Cauchy sequence is a limit of the se-
quence, which is therefore convergent.

Continuity
in which we, at last, study the continuous functions, without which the study
of metric spaces would be a fruitless and boring activity. As an application, we
consider the problem of moving a differentiation operator under the integral
sign.

35 Definition. Let (X, d) and (Y, ρ) be metric spaces, and let f : X →

Y be a function. f is said to be continuous at x ∈ X if, for every ε > 0,
there exists some δ > 0 so that, whenever ξ ∈ X,

d(ξ, x) < δ ⇒ ρ(f (ξ), f (x)) < ε.

36 Exercise. For some fixed y ∈ X, let f : X → R be the function

defined by f (x) = d(x, y). Show that
f is continuous. Also, define a
metric ρ on X × X by ρ (x, y), (ξ, η) = d(x, ξ) + d(y, η). Show that ρ
is in fact a metric, and that d : X × X → R is continuous when X × X
is given this metric.

37 Definition. If V ⊆ Y we write f −1 [V ] = {x ∈ X : f (x) ∈ V } (even

if f has no inverse function f −1 ).

38 Proposition. Let X and Y be metric spaces, f : X → Y a function,

and x ∈ X. Then the following are equivalent:
(a) f is continuous at x;
(b) for each neighbourhood V of f (x), f −1 [V ] is a neighbourhood of
x;
(c) f (xn ) → f (x) whenever xn → x.
Proof: The equivalence of any two of these is easy to prove directly.
We prove a cycle of implications.
(a) ⇒ (b): If V is a neighbourhood of f (x), then Bε (f (x)) ⊆ V
for some ε > 0. Then, by continuity, for some δ > 0 we get Bδ (x) ⊆
f −1 [Bε (f (x))] ⊆ f −1 [V ], so f −1 [V ] is a neighbourhood of x.

Version 2004–01–15
15 Elements of mathematical analysis

(b) ⇒ (c): Let xn → x. Assume V is a neighbourhood of f (x). Then,

since f −1 [V ] is a neighbourhood of x, some tail of the sequence (xn )
is contained in f −1 [V ], and so the corresponding tail of the sequence
(f (xn )) is contained in V . Hence f (xn ) → f (x).
(c) ⇒ (a): Assume f is not continuous at x. Then for some ε > 0 and
every δ > 0 there is some ξ ∈ X with d(ξ, x) < δ but ρ(f (ξ), f (x)) ≥
ε. Let δk & 0, and for each k, let xk ∈ X with d(xk , x) < δk and
ρ(f (xk ), f (x)) ≥ ε. Then xk → x but f (xk ) 6→ f (x).

39 Definition. A function f : X → Y is said to be continuous if it is

continuous at every point in X.

40 Proposition. Let X and Y be metric spaces and f : X → Y a

function. Then the following are equivalent:

(a) f is continuous;
(b) for each open subset V ⊆ Y , f −1 [V ] is open;
(c) for each closed subset F ⊆ Y , f −1 [F ] is closed.

41 Theorem. Let X and Y be metric spaces and f : X → Y a contin-

uous function. If X is compact then f [X] = {f (x) : x ∈ X} is compact.

Proof: It is instructive to give several proofs of this fact.

First, let (yn ) be a sequence in f [X]. Since yn ∈ f [X] we can write
yn = f (xn ). By compactness the sequence (xn ) has a convergent sub-
sequence (xnk ). Since f is continuous, then the subsequence given by
ynk = f (xnk ) converges. Thus f [X] is compact.
Second, let U be an open cover of f [X]. Then {f −1 (U ) : U ∈ U }
is an open cover of X, and so by the compactness Sn of X there is a
finite
Sn number of sets U1 , . . . , U n ∈ U so that k=1 f −1
[Uk ] = X. Then
k=1 U k = f [X], and hence f [X] is compact.
Third, consider a family of closed subsets of f [X], with the finite
intersection property. The proof that the family has a nonempty inter-
section is left as an exercise for the reader.

42 Corollary. A continuous real-valued function on a compact space

is bounded, and achieves its maximum as well as its minimum.

Version 2004–01–15
Elements of mathematical analysis 16

Proof: A compact subset of R is bounded, and its infimum and supre-

mum are members of the set because it is closed.

43 Exercise. If A be a subset of some metric space X, the distance

from a point x ∈ X to A is the number

dist(x, A) = inf{d(x, a) : a ∈ A}.

Show that x 7→ dist(x, A) is a continuous function. Assume now that

K is a compact subset of X and that K ∩ Ā = ∅. Show that there is
some ε > 0 so that d(x, a) ≥ ε whenever x ∈ K and a ∈ A.

44 Definition. Let (X, d) and (Y, ρ) be metric spaces. A function

f : X → Y is called uniformly continuous if, whenever ε > 0, there
exists some δ > 0 so that ρ(f (ξ), f (x)) < ε whenever x, ξ ∈ X and
d(ξ, x) < δ.

45 Exercise. Show that the real functions x 7→ x2 and x 7→ 1/x (with

x > 0) are not uniformly continuous. Show that arctan is uniformly
continuous.

46 Proposition. A continuous function f : X → Y where X, Y are

metric spaces and X is compact, is uniformly continuous.

Proof: Let ε > 0. For every x ∈ X there is some δ(x) > 0 so that
ρ(f (ξ), f (x)) < ε whenever d(ξ, x) < δ(x).SBy the compactness of X
n
there are x1 , . . . , xn ∈ X so that X = j=1 Bδ(xj )/2 (xj ). Let δ =
min{δ(x1 ), . . . , δ(xn )}/2.
Now, if ξ, x ∈ X, then x ∈ Bδ(xj )/2 (xj ) for some j. If furthermore
d(ξ, x) < δ then ξ ∈ Bδ(xj ) (xj ) as well, and so

ρ(f (ξ), f (x)) ≤ ρ(f (ξ), f (xj )) + ρ(f (xj ), f (x)) < ε + ε = 2ε.

Hence f is uniformly continuous.

The above theorem and the notion of uniform continuity has many uses.
A simple application is the following result on differentiating under the
integral.

Version 2004–01–15
17 Elements of mathematical analysis

47 Proposition. Let f be a real function on some open subset U ⊂

R2 . Let a, b, and x0 be real numbers so that (x0 , y) ∈ U whenever
a ≤ y ≤ b. Assume that ∂f (x, y)/∂x exists and is continuous for each
Rb
(x, y) ∈ U . Then the function x 7→ a f (x, y) dy is differentiable at x0 ,
with derivative
 Z b Z b
d  ∂f
f (x, y) dy = (x0 , y) dy.
dx x=x0 a

a ∂x

Proof: First, since the compact set {(x0 , y) : a ≤ y ≤ b} is contained in

the open set U , there is some δ1 > 0 so that the, likewise compact, set
{(x, y) : |x − x0 | ≤ δ1 , a ≤ y ≤ b} is contained in U (exercise: prove this
using exercise 43 with A = X \ U ). Let ε > 0. By uniform continuity
of ∂f /∂x on this compact set, there is some δ > 0 so that
 
 ∂f
 (x, y) − ∂f (x0 , y) < ε

 ∂x ∂x 

whenever |x − x0 | < δ and a ≤ y ≤ b (clearly, by picking δ ≤ δ1 we

make sure that (x, y) ∈ U at the same time). Now
Z b Z b Z b Z x
∂f
f (x, y) dy − f (x0 , y) dy = (ξ, y) dξ dy
a a a x0 ∂x

and
 ! Z 
 1 Z b Z x b
∂f ∂f 
(ξ, y) dξ dy − (x0 , y) dy 
 
 x − x0 x0 ∂x a ∂x

a 
 
 1 Z b Z x  
∂f ∂f 
= (ξ, y) − (x0 , y) dξ dy 
 
 x − x0 a x0 ∂x ∂x 
Z bZ x 
1  (ξ, y) − ∂f (x0 , y) dξ dy
 ∂f 
≤
|x − x0 | a x0  ∂x ∂x 
Z bZ x
1
< ε dξ dy
|x − x0 | a x0
= |b − a|ε

Version 2004–01–15
Elements of mathematical analysis 18

which completes the proof.

The above result has many useful generalisations, in particular replac-
ing the integral by multiple integrals or surface integrals. As long as we
integrate over a compact set, essentially the same proof will work. How-
ever, for improper integrals uniform continuity will not work anymore.
For example, to prove a formula of the type
 Z ∞ Z ∞
d  ∂f
f (x, y) dy = (x0 , y) dy
dx x=x0 −∞ −∞ ∂x

uniform continuity is not enough, but if you can show, for every ε > 0,
the existence of some δ > 0 so that
 
 ∂f
 (x, y) − ∂f (x0 , y) < εg(y)

 ∂x ∂x 

wheneverR |x − x0 | < δ, and where the function g (independent of ε)

∞
satisfies −∞ g(x, y) dy < ∞, you can get the desired formula just as in
the above proof.

48 Exercise. Complete the above argument.

The following result is sometimes called Fubini’s theorem, but that is

misleading – Fubini’s theorem is much more general, and deals with
Lebesgue integrable functions. The conclusion is the same, however.
We include this simple special case because it is easy to prove with the
tools at hand.

49 Proposition. Assume that f is a continuous function on [a, b] ×

[c, d]. Then
Z b Z d Z d Z b
f (x, y) dy dx = f (x, y) dx dy.
a c c a

Proof: Replace b in both integrals by a variable ξ. Clearly, the integrals

are equal when ξ = a. If we can show that both sides are differentiable
with respect to ξ, with the same derivative, then they must have the
same value for all ξ, including ξ = b.

Version 2004–01–15
19 Elements of mathematical analysis

First, note that f is uniformly continuous. A direct computation

Rd
shows that c f (x, y) dy is a continuous function of x. Hence the fun-
damental theorem of calculus shows that
Z ξZ d Z d
d
f (x, y) dy dx = f (ξ, y) dy.
dξ a c c

Second, use the previous proposition to show that

Z dZ ξ Z d Z ξ Z d
d ∂
f (x, y) dx dy = f (x, y) dx dy = f (ξ, y) dy.
dξ c a c ∂ξ a c

Ordinary differential equations

in which we state and prove the fundamental existence and uniqueness the-
orem.
In elementary calculus courses (the so-called “advanced calculus”), when
discussing ordinary differential equations, the emphasis is on solutions
by formula. Sure, the uniqueness of solutions for the initial value prob-
lem is often proved, but this is usually a result of the particular struc-
ture of the equation. In this section we will be concerned with the
questions of existence and uniqueness in a more general setting.
The elementary theory can still throw some light on the general
problem, and hint at what can and cannot be expected to hold true.
We shall be concerned with initial value problems — that is, prob-
lems of the form
ẋ(t) = f (t, x(t))
(1)
x(0) = x0
with given function f and initial value x0 . (We might, more generally,
consider a given initial value x(t0 ) = x0 at some time t0 , but this gen-
eralisation is trivial. We shall always think of the independent variable
t as time, though of course this would be misleading in many applica-
tions.)
Consider, for example, the equation ẋ = x2 . This separable equation
is typically solved by formally rewriting it as dx/x2 = dt and inte-
grating, with the result x = 1/(τ − t). (This method misses the trivial

Version 2004–01–15
Elements of mathematical analysis 20

solution x(t) = 0, though.) If we are given an initial value x(0) = x0 ,

the integration constant must be given by τ = 1/x0 . Thus, if x0 > 0
then the solution goes to infinity (or “blows up”) at time t = τ = 1/x0 .
Therefore, we cannot expect a general existence theorem to give
global results, but we must settle for a local result instead: existence of
a solution x(t) for t in a neighbourhood of 0.
Another example is the equation ẋ = x1/3 , with the general solution
x = (2/3(t − τ ))3/2 in addition to the trivial solution x(t) = 0. We note
that the general solution is only valid for t > τ ; however, we can make a
solution valid everywhere by joining the general solution and the trivial
one as follows: (
0 t ≤ τ,
x(t) = 2 3/2
( /3(t − τ )) t>τ
However, this is an example of the breakdown of uniqueness, for the
initial value problem with initial value x(0) = 0 has infinitely many
solutions: Any solution like the one above with τ ≥ 0 will do. The
problem lies with the right hand side x1/3 , which is too singular for
uniqueness to hold.
The key to an existence and uniqueness result for the initial value
problem (1) is to note that a function x defined on an interval sur-
rounding 0 solves (1) if, and only if, it is continuous and satisfies the
integral equation
Z t
x(t) = x0 + f (τ, x(τ )) dτ (2)
0
for each t in the given interval.
We might imagine solving (2) by picking an arbitrary initial function
x1 (t) and proceeding to iterate:
Z t
xn+1 (t) = x0 + f (τ, xn (τ )) dτ (3)
0
and then hoping that xn will converge to the desired solution function
x as n → ∞.
This is known as Picard’s method and it does indeed work. We shall
use Banach’s fixed point theorem to show this. To carry out this pro-
gram, then, we must first define a suitable complete metric space to be
populated by functions x(t).

Version 2004–01–15
21 Elements of mathematical analysis

To this end, we may replace the interval around 0 by an arbitrary

metric space X. First, let l∞ (X) consist of all bounded, real-valued
functions on X. We use the norm

kf k∞ = sup{|f (x)| : x ∈ X}

on this space. A sequence in l∞ (X) which converges in this norm is

called uniformly convergent. In contrast, a sequence for which f (x)
converges for every x ∈ X is called pointwise convergent.

50 Exercise. Show that a uniformly convergent sequence is pointwise

convergent. Show that, given x, the map f 7→ f (x) is a continuous map
from l∞ (X) to R. What is the connection between these two state-
ments?
Finally, show that a pointwise convergent sequence need not be uni-
formly convergent (let f (x) = x/(1 + x2 ) and consider the sequence fn ,
where fn (x) = f (nx)).

51 Proposition. l∞ (X) is a Banach space.

Proof: Clearly, it is a normed space. We must show it is complete.

Let (fn ) be a Cauchy sequence in l∞ (X). For each x ∈ X, (fn (x)) is a
Cauchy sequence in R, and so has a limit which we will denote f (x).
To show that fn → f uniformly, let ε > 0. There is some N so that
kfn − fm k < ε whenever m, n ≥ N . By the definition of the norm, that
translates into |fn (x) − fm (x)| < ε whenever m, n ≥ N and x ∈ X.
Letting m → ∞ in this inequality, we conclude |fn (x) − f (x)| ≤ ε
whenever n ≥ N and x ∈ X. But, again by the definition of the norm,
this means kfn − f k ≤ ε whenever n ≥ N . In other words, fn → f
uniformly.
The continuous functions in l∞ (X) form a subspace which we shall
call Cb (X).

52 Proposition. Cb (X) is a Banach space.

Proof: By Proposition 21, we only need to show that Cb (X) is closed.

In other words, we must show that a uniform limit of continuous func-
tions is continuous.

Version 2004–01–15
Elements of mathematical analysis 22

So let now f1 , f2 , . . . be continuous real-valued functions on X, and

assume that fn → f uniformly. We must show that f is continuous.
So let x ∈ X, and ε > 0. By uniform convergence, there is some
N so that n > N implies kfn − f k∞ < ε. Pick any n > N . Since
fn is continuous, there exists some δ > 0 so that |fn (y) − fn (x)| < ε
whenever y ∈ Bδ (x). Now, for any such y we find

|f (y) − f (x)| ≤ |f (y) − fn (y)| + |fn (y) − fn (x)| + |fn (x) − f (x)| < 3ε

since |f (y)−fn (y)| ≤ kf −fn k∞ < ε (and similarly, |fn (x)−f (x)| < ε).

53 Definition. A real function f defined on R is said to be Lipschitz

continuous with Lipschitz constant L if |f (x) − f (y)| ≤ L|x − y| for
all x, y (this definition has an immediate generalisation to functions
between arbitrary metric spaces, of course). Similarly, a function f of
two variables is said to be uniformly Lipschitz continuous in the second
variable if it satisfies |f (t, x) − f (t, y)| ≤ L|x − y| for all t, x, y.

54 Theorem. (Picard–Lindelöf ) Consider the initial value problem

(1) where the right hand side f is defined and uniformly Lipschitz
continuous in the second variable on a neighbourhood of (t, x) = (0, x0 ).
Then (1) has a unique solution on some neighbourhood of t = 0.

Proof: The simple idea of the proof is to use the Banach fixed point
theorem on the function
Z t
Φ(x)(t) = x0 + f (τ, x(τ )) dτ
0

since a fixed point of this function is a solution of (1), because (1)

is equivalent to (2). The proof is somewhat complicated by the fact
that Φ(x) may be undefined for some functions x, namely those x for
which x(t) is sometimes outside the domain of definition of f . So we
use Corollary 20 instead.

Version 2004–01–15
23 Elements of mathematical analysis

First note that, given two functions x and y, we find

Z t 



| Φ(x) − Φ(y) (t)| =  f (τ, x(τ )) − f (τ, y(τ )) dτ 
0
Z t
≤ |f (τ, x(τ )) − f (τ, y(τ ))| dτ
0
Z t
≤ L|x(τ ) − y(τ )| dτ
0
≤ L|t| · kx − yk

where L is the Lipschitz constant of f (in the second variable). Thus, to

make Φ a contraction, we might restrict it to functions on the interval
[−T, T ] where T < 1/L. Further, to use Corollary 20 we must have

kΦ(x0 ) − x0 k∞ ≤ (1 − LT )r.

This norm is easily estimated:

Z t 
 
kΦ(x0 ) − x0 k∞ = sup 
 f (τ, x(τ )) dτ  ≤ M T
|t|≤T 0

where M is the maximum value of f on [−T, T ] × B̄ε (x0 ). By making

T smaller if necessary (which does not increase M ) we can achieve the
inequality M T ≤ (1 − LT )r which is exactly what we need to complete
the existence proof.
Uniqueness also follows, at least locally, from which one can patch
together a global uniqueness proof. It is better, however to use Grön-
wall’s lemma to show uniqueness (but we shall not do so here).
We have stated and proved the Picard–Lindelöf theorem for scalar
initial value problems only. However the same proof works, without
modification, for systems of first order equations: Just think of the
unknown function as mapping an interval around 0 into Rn , and f as
defined on a suitable open subset of R × Rn . It can in fact be seen to
work in a yet more general setting, replacing Rn by a Banach space.
Furthermore, higher order equations are handled by reducing them
to first order equations: For example, given a second order equation of
the form ẍ = f (t, x, ẋ) we put y = ẋ and so arrive at the equivalent
system ẏ = f (t, x, y), ẋ = y.

Version 2004–01–15
Elements of mathematical analysis 24

A first order system of equations ẋ = f (t, x) where f is Lipschitz defines

a function φ by writing the solution to the system satisfying the initial con-
dition x(s) = ξ as φ(ξ, s, t); thus φ(ξ, s, t) = ξ and ∂φ/∂t = f (φ). It turns
out (but we shall not prove it) that if f is a C 1 function then φ is also C 1 . In
other words, the solution of the system is a continuously differentiable func-
tion, not only of its parameter, but also of the initial conditions. We have
now arrived at the beginnings of the theory of dynamical systems, and this
is where we leave that theory.

The implicit and inverse function theorems

in which we state some conditions under which equations may be solved, and
some properties of the solution.
The implicit function theorem concerns situations in which one can
guarantee that an equation of the form F (x, y) = 0 defines y as a
function of x; that is, when we can find a function g so that F (x, g(x)) =
0 for all x in some neighbourhood of a given point x0 (and, moreover,
we want some form of uniqueness, so that F (x, y) = 0 has no solution
y 6= g(x), at least not locally).
The inverse function theorem concerns the existence of a local in-
verse of a given function f . Since the defining equation of the inverse,
f (g(x)) = x, can be written as F (x, g(x)) = 0 where F (x, y) = f (y)−x,
it should be clear that the inverse function theorem will be a special case
of the implicit function theorem. Thus we concentrate on the latter.
We shall need some definitions before we start.
The space of linear maps from Rn to Rm will be written L(Rn , Rm ).
Of course we know that this can be identified with the space of all
m × n-matrices Rm×n , but mostly, we shall prefer the more abstract
approach. On this space we use the operator norm defined by

kAxk
kAk = sup .
x∈Rn \{0} kxk

(Much of what follows works just as well if the Euclidean spaces Rn are
replaced by Banach spaces, and L(X, Y ) is the space of bounded linear
operators from a Banach space X to a Banach space Y .)

55 Definition. Let f be a function defined on a neighbourhood of a

Version 2004–01–15
25 Elements of mathematical analysis

point x ∈ Rn and with values in Rm . f is said to be (Fréchet) differen-

tiable at x if there is some A ∈ L(Rn , Rm ) so that

kf (x + ξ) − f (x) − Aξk
lim = 0.
ξ→0 kξk

The operator A is uniquely determined (exercise!), and is called the

derivative of f at x, written A = Df (x).

This is more conveniently written as the first order Taylor’s formula:

f (x + ξ) = f (x) + Df (x)ξ + o(kξk) (ξ → 0)

where o(kξk) is taken to mean some function r(ξ) so that r(ξ)/kξk → 0

as ξ → 0.

56 Exercise. Show that, if f is differentiable at x with Df (x) = A,

then all first order partial derivatives of f exist at x, and when A is
interpreted as a matrix, we have

∂fi
Aij = (x).
∂xj

Show that the converse does not hold, for example by considering the
function  x x
p 1 2 x 6= 0,
f (x) = x21 + x22
0 x = 0.


Now, consider a function f defined in an open subset U of Rn . The

function is said to be C 1 if it is differentiable at each point of U , and
the function Df : U → L(Rn , Rm ) is continuous.

57 Proposition. A function f : U → Rm is C 1 in the open set U ⊆ Rn

if and only if it has first order partial derivatives at each point of U ,
and those partial derivatives are continuous in U .

Proof: We prove only the hard part, leaving the rest as an exercise.
What we shall prove is the following: If f has continuous first order

Version 2004–01–15
Elements of mathematical analysis 26

partial derivatives in a neighbourhood of 0, then f is differentiable

there.
For brevity, we write gj = ∂f /∂xj . Write ej for the vector whose
j component is 1, while all thePothers are 0 (so e1 , . . . , en is the stan-
n
dard basis of Rn ). Then x = j=1 xj ej . We estimate f (x) − f (0) by
integrating the appropriate partial derivatives of f along the path con-
segments from 0 via x1 e1 , x1 e1 + x2P
sisting of straight lineP e2 , and so
k−1 k−1
on to x: Let γk (t) = j=1 xj ej + txk ek so that γk (0) = j=1 xj ej ,
Pk
γk (1) = j=1 xj ej , and γk0 (t) = xk ek . Then
 ! !
n
X k
X k−1
X
f (x) − f (0) = f xj ej −f xj ej 
k=1 j=1 j=1
n Z 1
X d
= f (γk (t)) dt
dt0
k=1
Xn Z 1
= xk gk (γk (t)) dt
k=1 0

Xn n
X Z 1
= xk gk (0) + xk [gk (γk (t)) − gk (0)] dt
k=1 k=1 0

The first sum on the last line is the desired linear function of x; it is
Df (0)x. To show that this is really the Fréchet derivative, we must
show the second sum is o(kxk). But if ε > 0 is given we can find δ > 0
so that kgj (x) − gj (0)k < ε whenever δ > 0. Clearly, for such an x and
0 ≤ t ≤ 1 we have kγk (t)k ≤ kxk < δ, and so each of the integralsP in
the second sum has norm < ε; hence their sum has norm < ε |xj |
and so it is o(kxk) as kxk → 0.
We leave the rest of the proof as an exercise. To show that f is C 1 ,
note that once we know that f is differentiable, the matrix elements
of Df are the partial derivatives of the components of f (in fact the
columns of Df are the functions gj ).

Version 2004–01–15
27 Elements of mathematical analysis

58 Proposition. (The chain rule) Let g map a neighbourhood of

x ∈ Rp into Rn and f map a neighbourhood of g(x) into Rm . If g is
differentiable at x and f is differentiable at g(x), then the composition
f ◦ g is differentiable at x, with derivative


D(f ◦ g)(x) = Df g(x) ◦ Dg(x)

When A and B are composable linear maps, it is more common

to write their composition as BA rather than B ◦ A. Thus the above
formula is more commonly written as D(f ◦ g)(x) = Df (g(x))Dg(x).
Proof: Simply write



f ◦ g(ξ) − f ◦ g(x) = Df g(x) g(ξ) − g(x) + o(kg(ξ) − g(x)k)



= Df g(x) Dg(x)(ξ − x) + o(kξ − xk)


+ o Dg(x)(ξ − x) + o(kξ − xk)



= Df g(x) Dg(x)(ξ − x) + o(kξ − xk)

and the proof is complete. (Exercise: Write the argument out more
carefully, dealing properly with all the epsilons and deltas.)
We can now state and prove the implicit function theorem. First,
however, let us consider the simple case of single variables. Clearly, the
curve in Figure 1 is not the graph of a function. Nevertheless, some
part surrounding the point (x0 , y0 ) is the graph of a function y = g(x).
This illustrates the fact that we can only expect to be able to show the
existence of a function g with F (x, g(x)) = 0 locally, that is, in some
neighbourhood of x0 . The trouble spots seem to be where the curve has
a vertical tangent or, equivalently, a horizontal normal. A normal vector
is given by ∇F = (∂F/∂x, ∂F/∂y), so the trouble spots are recognised
by ∂F/∂y = 0.
We return to the general case of functions of several variables. If
x ∈ Rm and y ∈ Rn , we may write the vector (x1 , . . . , xm , y1 , . . . , yn ) ∈
Rm+n as (x, y). If F is a function of (x, y), we write Dy F (x, y) ∈
L(Rn , Rn ) as Dy F (x, y)η = DF (x, y)(0, η). We then note that the
chain rule applied to the equation F (x, g(x)) = 0 yields Dx F (x, y) +
Dy F (x, g(x))Dg(x) = 0, so if Dy F (x, g(x)) is invertible, we find

Dg(x) = −Dy F (x, g(x))−1 Dx F (x, y).

Version 2004–01–15
Elements of mathematical analysis 28

Figure 1: The curve F (x, y) = 0 with a point (x0 , y0 ) on it.

It turns out that this invertibility condition, which gives us a unique

value of Dg(x), is sufficient to define the function g in a neighbourhood
of x.

59 Theorem. (Implicit function theorem) Assume given an Rn -

valued C 1 function F on a neighbourhood of (x0 , y0 ) ∈ Rm × Rn .
Assume that F (x0 , y0 ) = 0, and that Dy F (x0 , y0 ) is invertible. Then
there is a neighbourhood U of x0 and a C 1 function g : U → Rn with
g(x0 ) = y0 and F (x, g(x)) = 0 for all x ∈ U .

Proof: By replacing F (x, y) by F (x − x0 , y − y0 ) we may assume

x0 = 0 and y0 = 0. If A is an invertible n × n matrix, F (x, y) = 0
is equivalent to AF (x, y) = 0; hence we may replace F by AF . If we
let A = Dy F (0, 0)−1 this means we may, and indeed shall, assume
Dy F (0, 0) = I.
Write F (x, y) = y − H(x, y); thus F (x, y) = 0 ⇔ y = H(x, y), and
moreover Dy H(0, 0) = 0. Given x, we propose to solve the equation
y = H(x, y) by the iteration yn+1 = H(x, yn ) with y0 = 0. We must

Version 2004–01–15
29 Elements of mathematical analysis

show that H(x, y), as a function of y, is a contraction. But

Z 1
∂
H(x, y) − H(x, z) = H(x, ty + (1 − t)z) dt
0 ∂t
Z 1
= Dy H(x, ty + (1 − t)z) dt · (y − z)
0

and since Dy H(0, 0) = 0 and H is C 1 , there is some ε > 0 so that,

whenever kxk ≤ ε and kyk ≤ ε, we have kDy H(x, y)k ≤ 1/2. For such
x and y, then, the above equality implies
1
kH(x, y) − H(x, z)k ≤ ky − zk.
2
Clearly, then, for fixed x with kxk < ε, the map y 7→ H(x, y) is a
contraction of Bε (0) into Rn . This ball may however not be invariant
for all x (that is, the map y 7→ H(x, y) may not map the ball into itself).
However, if x is small enough, we shall see that Corollary 20 comes to
the rescue. In fact, all that remains is to satisfy the second inequality
in that Corollary, with K = 21 and r = ε. In our current setting, that
inequality becomes simply
1
|H(x, 0)| ≤ ε.
2
Since H is continuous and H(0, 0) = 0, we can find a δ > 0 so that
the above inequality holds whenever |x| < δ. By Corollary 20, then,
there is therefore a unique solution y = g(x) ∈ B̄ε (0) of the equation
y = H(x, y) wehenever x < δ, and thus of the equation F (x, y) = 0.
It only remains to establish the C 1 nature of g. To this end, consider



0 = F ξ, g(ξ) − F x, g(x)
Z 1
d

= F tξ + (1 − t)x, tg(ξ) + (1 − t)g(x) dt
0 dt
Z 1



= DF tξ + (1 − t)x, tg(ξ) + (1 − t)g(x) dt · ξ − x, g(ξ) − g(x)
0
Z 1 Z 1


= Dx F dt ·(ξ − x) + Dy F dt · g(ξ) − g(x)
|0 {z } | 0 {z }
A B

Version 2004–01–15
Elements of mathematical analysis 30

where the matrices A and B satisfy inequalities of the form kAk < C
and kI −Bk < 1/2 (the latter comes from the inequality kDy H(x, y)k <
1/2, and the former is just the boundedness of Dx F in a neighbour-
P∞ of 0). But
hood then B is invertible with kB −1 k < 2 (because B −1 =
k
k=0 (I − B) ), and we have

g(ξ) − g(x) = −B −1 A(ξ − x).

In the limit ξ → x, we find A → Dx F (x, g(x)) while B → Dy F (x, g(x))

from which the differentiability of g is easily shown. Furthermore,

Dg(x) = −Dy F (x, g(x))−1 Dx F (x, g(x))

which is continuous, so that g is C 1 .

60 Corollary. (Inverse function theorem) Assume there is given

an Rn -valued C 1 function f on a neighbourhood of y0 ∈ Rn , and that
Df (y0 ) is invertible. Then there is a neighbourhood U of f (y0 ) and a
neighbourhood V of y0 so that f maps V invertibly onto U , and the
inverse map g : U → V is C 1 .
Proof: Write F (x, y) = f (y) − x and apply the implicit function
theorem to F at the point (x0 , y0 ) where x0 = f (y0 ). Thus there is a
neighbourhood U0 of x0 and a C 1 function g on U0 so that f (g(x)) = x
whenever x ∈ U0 .
By the chain rule, Df (y0 )Dg(x0 ) = I so that Dg(x0 ) is invertible,
and by what we just proved (applied to g instead of to f ) there is
a neighbourhood V of y0 and a C 1 function h : V → Rn so that
g(h(y)) = y whenever y ∈ V .
Let U = g −1 [V ]. It is not hard to show that, in fact, h(y) = f (y)
whenever y ∈ V , and the restrictions f |V and g|U are each other’s
inverses.

Version 2004–01–15
31 Elements of mathematical analysis

Index

ball, 4 implicit function theorem, 28

Banach space, 7 induced metric, 4
Banach’s fixed point theorem, 7 inherited metric, 4
Bolzano–Weierstrass theorem, 13 interior, 4
Cantor’s intersection theorem, 13 inverse function theorem, 30
Cauchy sequence, 7 least upper bound, 12
closed, 5 limit, 5
closure, 6 limit point, 5
compact, 10 Lipschitz continuous, 22
complete, 7 metric, 2
continuous, 14 neighbourhood, 4
contraction, 7 norm, 3
convergence, 5 open, 4
cover, 11 Picard’s method, 21
derivative, 25 Picard–Lindelöf, 22
diameter, 10 pointwise convergence, 21
differentiable, 25 restriction, 4
discrete, 3 subsequence, 5
finite intersection property, 11 tail, 5
fixed point, 7 topological space, 6
Fubini’s theorem, 18 totally bounded, 11
generalised triangle inequality, 3 triangle inequality, 2, 3
greatest lower bound, 12 uniform convergence, 21
Heine–Borel theorem, 13 uniformly continuous, 16

Version information
Content-wise, this document is identical to the version labeled 1999–
04–30. But the source file has been upgraded to newer LATEX standards
in order to allow a LATEX rerun so I can take advantage of Type 1 fonts
and, most importantly, pdfTEX. Line and page breaking has changed a
little bit as a result.

Version 2004–01–15