Taylor Series

Linear approximation: Linear approximation is

to approximate a general function using a lin-
ear function. Given a differentiable scalar func-
tion f (x) of the real variable x, the linear ap-
proximation of the function at point a, as shown
in the Figure below, is obtained by
f (x) ≈ f (a) + f ′(a)(x − a)
(x)
where f ′ (a) = dfdx |x=a. The expression on
the right-hand side is just the equation for the
tangent line to the graph f (x) at point a. The
above is true when x is close to a.
f(x)

tangent line at a

a x

1
Example: Suppose f (x) = x2 − 5x + 7 is to be
linearised at x = 5.

f (5) = 25 − 25 + 7 = 7

f ′(x) = 2x − 5

f ′(5) = 10 − 5 = 5
Thus

f (x) ≈ f (5) + (x − 5) ∗ f (5)

= 7 + (x − 5) ∗ 5

= 5x − 18
Hence can also say that g(x) = 5x−18 is equa-
tion of tangent to curve f (x) = x2 − 5x + 7 at
x = 5.

2
Linear approximation of a multivariate func-
tion can be obtained similarly with the deriva-
tive at a point replaced by the partial deriva-
tives. Given a differentiable scalar multivari-
ate function f (x), where x = [x1, x2]T , the
linear approximation of the function at point
a = [a1, a2]T is
∂
f (x) ≈ f (a) + f (x)|x=a (x1 − a1)
∂x1
∂
+ f (x)|x=a (x2 − a2)
∂x2

= f (a) + (x − a)T ∇f (x)|x=a

where
∂ f (x)
 

∇f (x) =  ∂x1
∂ f (x)

∂x2
Example: Suppose

f (x) = x2
1 + 2x 2 − 5x − 8x + 7
2 1 2
is to be linearised at x = [5 4]T .
3
f ([5 4]T ) = 25 + 2 × 16 − 5 × 5 − 8 × 4 + 7 = 7

∂
f (x) = 2x1 − 5
∂x1

∂
f (x)|x=[5 4]T = 10 − 5 = 5
∂x1

∂
f (x) = 4x2 − 8
∂x2

∂
f (x)|x=[5 4]T = 16 − 8 = 8
∂x2
Thus

f (x) ≈ 7 + (x1 − 5) ∗ 5 + (x2 − 4) ∗ 8

= 5x1 + 8x2 − 50
Hence can also say that g(x) = 5x1 + 8x2 − 50
is equation of tangent to surface f (x) = x2
1+
2x22 − 5x 1 − 8x 2 + 7 at x = [5 4]T.

4
Taylor series: Given a differentiable scalar func-
tion f (x) of the real variable x, its Taylor series
reads
1
f (x) = f (a) + f ′ (a)(x − a) + f ′′(a)(x − a)2 + ...
2!
∞
1 (n)
f (a)(x − a)k
X
=
n=0 n!
′′ d2 f (x) (n) dn f (x)
where f (a) = dx2 |x=a, f (a) = dxn |x=a.

• In engineering, the Taylor series of a func-

tion at a certain point up to a finite order
n is used to approximate the function.

• If n = 1, this reduces to the linear approx-

imation.

Example: let f (x) = x2ex. Show that the Tay-

lor expansion of f (x) around 0, and up to the
4th order, is
5
 2 3 1 4
f (x) = x + x + x + . . .
2

f (0) = 02e0 = 0

f ′(x) = 2xex + x2ex = (2x + x2)ex

f ′(0) = 0

f ′′(x) = (2 + 2x)ex + (2x + x2)ex

= (2 + 4x + x2)ex

f ′′(0) = 2

f (3)(x) = (4 + 2x)ex + (2 + 4x + x2)ex

= (6 + 6x + x2)ex

f (3) (0) = 6

f (4)(x) = (6 + 2x)ex + (6 + 6x + x2)ex

= (12 + 8x + x2)ex
6
 f (4) (0) = 12
Thus
2 2 6 3 12 4
f (x) = x + x + x + ...
2! 3! 4!
1
= x2 + x3 + x4 + . . .
2
The Taylor series expansion can be general-
ized to the function of m variables as in the
following.

Taylor expansion of multivariate functions:

Given a differentiable scalar multivariate func-

tion f (x), where x = [x1, ..., xm]T , the Taylor
series of the function at point a = [a1, ..., am]T
∂
f (x) = f (a) + f (x)|x=a (x1 − a1) + ...
∂x1
∂
+ f (x)|x=a (xm − am)
∂xm
7
 1 ∂2 2 + ...
+ f (x)| x= a (x 1 − a1 )
2 ∂x2
1

1 ∂2 2
+ 2
f (x)| x= a (x m − am ) + ...
2 ∂xm

= f (a) + (x − a)T ∇f (x)|x=a

1
+ (x − a)T H(x − a)T + ...
2
where
∂ f (x)
 

∇f (x) =  ∂x1
∂ f (x)

∂x2

∂ 2f (x)
H= 2
|x=a
∂x
Note that the second term is an inner product
of a gradient of f (x) with the vector x − a and
the third term is a quadratic form with the
2
Hessian matrix ∂∂ xf2 .

8
Example:let f (x) = (x2
1 +2x 2 −5x −8x +7)ex1 .
2 1 2
Show that the Taylor expansion of f (x) around
0, and up to the 2nd order,is
1 2
f (x) = 7 + 2x1 − 8x2 − x1 − 8x1x2 + 2x2
2+...
2

f ([0 0]T ) = 7

∂
f (x) = (2x1−5)ex1 +(x2 2
1+2x2−5x1−8x2+7)e
x1
∂x1

= (x2 2
1 + 2x2 − 3x1 − 8x2 + 2)e
x1

∂
f (x)|x=[0 0]T = 2
∂x1

∂
f (x) = (4x2 − 8)ex1
∂x2

∂
f (x)|x=[0 0]T = −8
∂x2

9
∂2 x1 2 2 x1
f (x) = (2x 1 −3)e +(x 1 +2x 2 −3x 1 −8x 2 +2)e
∂x2
1

= (x2 2
1 + 2x2 − x1 − 8x2 − 1)e
x1

∂2
2
f (x)|x=[0 0]T = −1
∂x1

∂2
f (x) = (4x2 − 8)ex1
∂x1∂x2

∂2
f (x)|x=[0 0]T = −8
∂x1∂x2

∂2
f (x) = (4x2 − 8)ex1
∂x2∂x1

10
 ∂2
f (x)|x=[0 0]T = −8
∂x2∂x1

∂2 x1
f (x) = 4e
∂x2
2

∂2
2
f (x)x=[0 0]T = 4
∂x2
Thus

f (x) ≈ 7 + [x1 x2] × [2 − 8]T

!
1 −1 −8
+ [x1 x2] × × [x1 x2]T
2 −8 4

11
Similarly for a scalar function of a matrix vari-
able:
∂f T
f (X) = f (A) + trace[( ) (X − A)]) + . . .
∂X
The above formula shows 2 terms of Taylor
expansion. It uses the extension of definition
of an inner product to matrices
m X
X m
AB = aij bij
i=1 j=1
but
m m X
m
trace(AT B) = (AT B)ii =
X X
aij bij
i=1 i=1 j=1

12