Control of Nonlinear Systems.

S. T. Glad
Department of Electrical Engineering
Linkoping University
SE-581 83 Linkoping, Sweden

October 27, 2009

.

2
Contents

1 Introduction. 6
1.1 Some examples of nonlinear systems . . . . . . . . . . . . . . . . 6
1.2 Discussion of the examples . . . . . . . . . . . . . . . . . . . . . . 8
1.3 Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2 Nonlinear dierential equations. 10

2.1 Existence and uniqueness of solutions. . . . . . . . . . . . . . . . 10
2.2 Continuity and dierentiability with respect to initial values and
parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.3 Series expansions . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.4 Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

3 Canonical forms and exact linearization. 19

3.1 The relative degree of a nonlinear system . . . . . . . . . . . . . 19
3.2 A canonical form . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.3 The zero dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.4 Controller canonical form. . . . . . . . . . . . . . . . . . . . . . . 29
3.4.1 Construction of the coordinate transformation. . . . . . . 32
3.5 Exact linearization. . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.6 Exact linearization the multi-input case . . . . . . . . . . . . . 35
3.7 Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

4 Nonlinear observers 39
4.1 A straightforward observer . . . . . . . . . . . . . . . . . . . . . . 39
4.2 Observers based on dierentiation . . . . . . . . . . . . . . . . . 40
4.3 High gain observers . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.4 Observers based on state transformations . . . . . . . . . . . . . 45

3
 4.5 Observers based on state and output transformations . . . . . . . 53
4.6 Observers for systems with input signals . . . . . . . . . . . . . . 56
4.7 Closing the loop . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
4.8 Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

5 Stability and Lyapunov theory. 60

5.1 Lyapunov functions. . . . . . . . . . . . . . . . . . . . . . . . . . 60
5.2 Non-autonomous systems . . . . . . . . . . . . . . . . . . . . . . 63
5.3 Construction of Lyapunov functions . . . . . . . . . . . . . . . . 64
5.4 Lyapunov functions and frequency domain criteria. . . . . . . . . 67
5.5 Lyapunov functions and input output stability . . . . . . . . . . 73
5.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
5.7 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

6 Lyapunov based controller design 84

6.1 Control Lyapunov functions . . . . . . . . . . . . . . . . . . . . . 84
6.2 Backstepping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
6.3 Forwarding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
6.4 Stability of observers . . . . . . . . . . . . . . . . . . . . . . . . . 90
6.5 Rejection of disturbances . . . . . . . . . . . . . . . . . . . . . . 94
6.6 Passivity based control . . . . . . . . . . . . . . . . . . . . . . . . 96
6.7 An example from adaptive control . . . . . . . . . . . . . . . . . 100
6.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

7 Nonlinear optimal control. 105

7.1 The optimal control problem. . . . . . . . . . . . . . . . . . . . . 105
7.2 Innite horizon optimal control. . . . . . . . . . . . . . . . . . . . 109
7.3 Calculation of optimal control. . . . . . . . . . . . . . . . . . . . 110
7.4 Nonlinear robustness techniques . . . . . . . . . . . . . . . . . . . 113
7.5 Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

8 Harmonic analysis of nonlinear systems 117

8.1 Describing functions . . . . . . . . . . . . . . . . . . . . . . . . . 120
8.1.1 The classical describing function . . . . . . . . . . . . . . 121
8.1.2 Describing functions for bias plus sinusoid . . . . . . . . . 124

4
 8.2 Analysis of systems using the describing function . . . . . . . . . 125
8.3 Some design ideas based on the describing function . . . . . . . . 128
8.3.1 Modied integrators . . . . . . . . . . . . . . . . . . . . . 128
8.3.2 Dither techniques . . . . . . . . . . . . . . . . . . . . . . . 129
8.3.3 Compensation of nonlinearities . . . . . . . . . . . . . . . 131
8.4 Accuracy of the describing function method . . . . . . . . . . . . 132
8.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

9 Tracking and disturbance rejection 140

9.1 A simple nonlinear servo problem . . . . . . . . . . . . . . . . . . 140
9.2 General problem formulation . . . . . . . . . . . . . . . . . . . . 141
9.3 Center manifold theory . . . . . . . . . . . . . . . . . . . . . . . 143
9.4 State feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
9.5 Error feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
9.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

10 Physics based control 155

10.1 Lagrangian physics . . . . . . . . . . . . . . . . . . . . . . . . . . 155
10.2 Interconnected systems . . . . . . . . . . . . . . . . . . . . . . . . 160
10.3 Port controlled Hamiltonian systems . . . . . . . . . . . . . . . . 161

5
Chapter 1

Introduction.

1.1 Some examples of nonlinear systems

Our subject is the control of nonlinear systems. To get a feeling for it, let us
consider some examples.

Example 1.1 Consider a very simplied model for velocity control of an air-
craft. If the velocity is x1 and the mass normalized to 1, then

x 1 = x2 f (x1 ) (1.1)

where x2 is the engine thrust and f (x1 ) is the aerodynamic drag. A simplied
engine model is just a time constant from pilot command u to engine thrust:

x 2 = x2 + u (1.2)

Together (1.1) and (1.2) form a model of the aircraft velocity control.

Example 1.2 Consider the heat exchanger described in gure 1.1. A uid
which initially has the temperature T0 ows with the ow rate q through the
heat exchanger, which is surrounded by a medium with temperature Th . It is
assumed that very good mixing takes place so that one can assume the same
temperature T at every point in the heat exchanger. If the heat capacity of the
uid is c per unit volume and C for the whole heat exchanger, and if the heat
transfer coecient of the walls is , then a heat balance gives
d
(CT ) = qcT0 qcT + (Th T )
dt
Assume that the ow rate is controlled around a nominal ow q0 so that

q = q0 u

Then, using the numerical values

c/C = 1, /C = 1, Th = q0 = T0 = 1

6
 Th

q - T - q
T0

Figure 1.1: A simple heat exchanger model

gives the model

T = 2T + uT + u (1.3)
where the temperature T is a state variable and the ow change u is the input.
(Note that a positive u means a decrease in ow.)

Example 1.3 A pendulum where the length d is varied, is described by

+ 2d/d
+ (g/d) sin = 0

Dening x1 = and x2 = we get the equations

x 1 = x2
(1.4)
x 2 = (2x2 d g sin x1 )/d

Example 1.4 Consider a rigid body rotating freely in space. Let xi be the
angular velocity along the i:th principal axis. Let the external torque be a
vector whose components along the principal axes are u1 , u2 and u3 . The
equations are then
x 1 = a1 x2 x3 + u1
x 2 = a2 x1 x3 + u2 (1.5)
x 3 = a3 x1 x2 + u3
where
I2 I3 I3 I1 I1 I2
a1 = , a2 = , a3 =
I1 I2 I3
with the Ii being the moments of inertia. Here there are three state variables
(x1 , x2 and x3 ) and three input variables (u1 , u2 and u3 ).

7
Example 1.5 Consider the following electrical circuit
g
I
C
e

The resistive element is assumed to have a voltage drop g(I) which is a possibly
nonlinear function of the current. If the voltage across the capacitor is v, the
system is described by the equations
C v = I
(1.6)
e = v + g(I)

If the function g has an inverse g 1 we get the following description

v = g 1 (e v)/C (1.7)

1.2 Discussion of the examples

Is there a common description for all the examples? We see that (1.1,1.2), (1.3),
(1.4), (1.5) and (1.7) are all special cases of the description
. . . , u() )
x = f (x, u, u, (1.8)
where x is a vector of state variables and u is a vector values external signal,
often called input signal or control signal. If we exclude (1.4) we can write
x = f (x, u) (1.9)
This is often considered to be the standard description of a nonlinear system.
If we also exclude (1.7) then all the examples can be written in the form
x = f (x) + g(x) u (1.10)
where the right hand side is an ane function of u.
It should be noted that many control problems, which are not of the form
(1.10), can be put into that form by the introduction of extra variables and a
redenition of the input. In (1.4) for instance we could regard d as a state and
d as an input: x3 = d, u = d giving the description
x 1 = x2
x 2 = (2x2 u g sin x1 )/x3 (1.11)
x 3 = u
which is of the form (1.10). Similarly in (1.7) we could regard the derivative of
the voltage e as the input giving
v = g 1 (e v)/C
(1.12)
e = u

8
which is also of the form (1.10).
Note that the system (1.3) is a very special case of the system description (1.10),
since f is linear and g is ane. Such a system is called bilinear. The general
form is
m
x = Ax + uj Dj x + Bu (1.13)
j=1

where A and Dj are n n matrices and B is an n m matrix (where n is the

dimension of x and m is the dimension of u).
Finally we should note that the most general description we have discussed so
far, (1.8) is not general enough to cover all physical descriptions that might be of
interest. The description (1.7), for instance, was arrived at using the assumption
that the function g is invertible. If that is not the case, then we are stuck with the
description (1.6) which is a combination of algebraic and dierential equations.
Therefore it would be more natural to regard descriptions of the form

. . . , x(r) , u, u,
gi (x, x, . . . , u() ) = 0, i = 1, . . . , N (1.14)

as standard descriptions. The theory for such systems is however in the early
stages of development. Therefore we will concentrate on system descriptions of
the form (1.8), (1.9) and (1.10).

1.3 Exercises.
1.1 Show that the description (1.8) is no more general than (1.9) for linear
systems, i. e. show that there exists a transformation

z = Tx + Si u(i)

with nonsingular T , which transforms

x = Ax + B0 u + B1 u + + Br u(r)

into the standard description

z = F z + Gu

9
Chapter 2

Nonlinear dierential
equations.

In the previous chapter we discussed the desription of nonlinear systems and

arrived at the equation
. . . , u() )
x = f (x, u, u, (2.1)
and its sligthly more specialized form
x = f (x, u) (2.2)
In this chapter some basic properties of dierential equations are discussed. The
rst and obvious question is: Suppose we put a continuous input signal u = u(t)
into the right hand side. Can we then be sure that there exists a solution of the
dierential equation? If we x the time function u(t), then both (2.1) and (2.2)
can be seen as special cases of a dierential equation with time varying right
side:
x = f (t, x)

2.1 Existence and uniqueness of solutions.

Consider a nonlinear dierential equation
x = f (t, x)
(2.3)
x(t0 ) = x0
where x is an n-vector, t is a real number ( the time in almost all control theory
applications ) and where f is a continuous function. It turns out that it is more
convenient to analyze the equivalent integral equation
 t
 
x(t) = x0 + f , x( ) d (2.4)
t0

Initially only the local properties close to the starting point of (2.3) are investi-
gated. Consider therefore t- and x-values satisfying
t0 t t0 + a, |x x0 | b (2.5)

10
Assume that for all t and x satisfying (2.5) it is true that

|f (t, x)| ba1 (2.6)

|f (t, x1 ) f (t, x2 )| |x1 x2 | (2.7)

and that
= a < 1 (2.8)
In (2.6) and (2.7) the vertical bars denote the Euclidian vector norm. The
inequality (2.7) is usually called a Lipschitz condition on f .

Remark 2.1 Obviously (2.6) and (2.8) can always be satised if a is chosen
small enough, i.e. if the time interval is small enough.

For a continuous function v we dene

||v|| = max |v(t)| (2.9)

t0 tt0 +a

We can now state the following existence and uniqueness result.

Theorem 2.1 A dierential equation (2.3) which satises the conditions (2.5)
- (2.8) has a unique solution on the interval t0 t t0 + a.

Proof. We will construct the solution, using the iteration

 t
 
xn+1 = x0 + f , xn ( ) d
t0

and starting with the constant function

x0 (t) = x0

a) We show that
|xn (t) x0 | b; t0 t t0 + a
Obviously this is true for n = 0. Suppose it is known for all integers up to n.
Then  t 
b t
|xn+1 (t) x0 | |f (, xn ( ))|d d b
t0 a t0
The result follows from induction.

b) We show that xn converges to a limit. Having shown a) we know that we

can apply (2.6) - (2.8) to all the xn . Now consider the dierence between two
iterates
 t
|xn+1 (t) xn (t)| |f (, xn ( )) f (, xn1 ( ))| d
t0
 t
|xn ( ) xn1 ( )| d a||xn xn1 || = ||xn xn1 || (2.10)
t0

11
Using this estimate repeatedly we get

||xn+1 xn || ||xn xn1 || . . . n ||x1 x0 ||

If m > n then

||xm xn || ||xm xm1 || + + ||xn+1 xn || (m1 + . . . + n ) ||x1 x0 ||

n
||x1 x0 ||
1
This expression converges to zero as n goes to innity and {xn } is thus a Cauchy
sequence. In particular, xn (t), for xed t, is a Cauchy sequence of real numbers.
It then has to converge to some value x(t). Since this holds for all t in the
chosen interval, we have shown that

xn (t) x(t), t0 t t0 + a,

for some function x(t).

c) Show that x is continuous and satises (2.4). Since

|x(t + h) x(t)| |x(t + h) xn (t + h)| + |xn (t + h) xn (t)| + |xn (t) x(t)|

2||x xn || + |xn (t + h) xn (t)|

and each xn is continuous, it follows that x is a continuous function.
Consider
 
t   t
|xn (t)x0 f , x( ) d | |f (, xn1 ( ))f (, x( ))| d ||xn1 x||
t0 t0

It follows that  t  
xn (t) x0 + f , x( ) d
t0

as n . As xn x it follows that x satises (2.4).

d) Show that x is a unique solution. Suppose there are two solutions x and z.
Then using the same reasoning as in (2.10),

||x z|| ||x z||

Since < 1, this implies that ||x z|| = 0 and consequently that x = z.

Remark 2.2 If f is continuous but does not satisfy the Lipschitz condition
(2.7), then one can still prove existence but the solution is not necessarily unique,
as shown by the dierential equation
d
x = x, x(0) = 0
dt
which has the solutions
t2
x = 0, x=
4

12
Remark 2.3 Theorem 2.1 guarantees only local existence, since the time in-
terval might have to be chosen small as explained in Remark 2.1. If a is too
large there might not exist a solution over the whole time interval as shown by
the dierential equation.
d
x = x2 , x(0) = 1
dt
The solution is
1
x=
1t
which only exists for t < 1.

The phenomenon of Remark 2.3 is called explosion or nite escape time.

An interesting fact is that this is the only way in which global existence can be
lost. This is formalized by the following theorem.

Theorem 2.2 Let f (t, x) be continuous and satisfy (2.7) in the set

M = {(t, x) : t0 t t1 , |x| A}

Let the starting point satisfy |x0 | A. Then either there is a solution dened
on the whole time interval t0 t t1 or else there is a solution on t0 t
te , (te < t2 ), with |x(te )| = A . In other words the solution leaves the set M
at t = te .

Remark 2.4 Often it is possible to give an upper bound on the solution x ,

showing that it can not leave the set M of Theorem 2.2. It then follows that
there exists a solution on the whole time interval.

2.2 Continuity and dierentiability with respect

to initial values and parameters.
We will now consider a whole family of solutions with dierent initial conditions.
We write
x = f (x)
(2.11)
x(0) = y
To simplify the notation, we have assumed that f does not depend explicitly
on t. Then there is no loss in generality in assuming that the initial time is
t = 0. The results of this section are easily extended to the case where f is time
dependent however.
Our task will be to nd out how the solutions vary when the initial condition y
is varied. To do that, the following lemma is needed.

Lemma 2.1 (Gronwalls lemma) If the continuous and nonnegative functions

m and g satisfy
 t
m(t) C + g( )m( ) d, 0 t T (2.12)
0

13
for a positive constant C, then
 t 
m(t) C exp g( )d , 0tT (2.13)
0

Proof. Dene  t
h(t) = C + g( )m( ) d
0
Dierentiating gives

h(t) = m(t)g(t) h(t)g(t)
showing that

h(t)
g(t)
h(t)
Integrating both sides then gives the desired result.

We now dene the function

F (t, y)
as the solution of (2.11) at time t. The solution is thus regarded as a function of
two variables: the time t and the initial state y. First it will be shown that the
solution depends continuously ( in fact Lipschitz continuously ) on the initial
condition.

Theorem 2.3 Let the dierential equation (2.11) and the point x0 be given.
Asume that the conditions (2.6)-(2.8) are satised. Then there is a neighborhood
V of x0 and an  > 0 such that for every y V there is a unique solution of
(2.11) on [0, ]. Furthermore

|F (t, z) F (t, y)| et |z y| (2.14)

Proof. The rst part of the lemma follows directly from the proof of the
existence theorem. To show (2.14) dene

(t) = |F (t, z) F (t, y)|

Then
 t  t
(t) = |z y + (f (F (s, z)) f (F (s, y)))ds| |z y| + (s)ds
0 0

Using Gronwalls lemma immediately gives

(t) et |z y|

We can now go one step further and ask ourselves if the solution can be dif-
ferentiated with respect to the initial condition. The following notation will be
used. For a vector valued function f (x), fx (x) will denote the matrix whose
i, j-element is
fi
xj

14
For the function F (t, y)
Ft (t, y), Fy (t, y)
will denote the derivatives with respect to t and y respectively, the st being an
n- vector and the second an n by n matrix. Since F (t, y) is the solution of the
dierential equation, we have
Ft (t, y) = f (F (t, y))
Assuming that F is continuously dierentiable with respect to y, we get
Ft,y (t, y) = fx (F (t, y))Fy (t, y)
Since obviously
Fy (0, y) = I
we see that the derivative Fy if it exists must be a solution of the linear dier-
ential equation

t (t, y) = fx (F (t, y))(t, y) (2.15)
(0, y) = I
called the variational equation. In fact we have

Theorem 2.4 Let f in (2.11) be continuously dierentiable. Then F (t, y) is

continuously dierentiable with respect to y with the derivative satisfying (2.15).

Proof. Dene
(t, h) = F (t, y + h) F (t, y)
We have
 t
(t, h) (t, h)h = (f (F (s, y + h)) f (F (s, y))) ds
0
 t  t
fx (F (s, y))(s, y)h ds = fx (F (s, y))((s, h) (s, y)h) ds+
0 0
 t
+ ( f (F (s, y + h)) f (F (s, y)) fx (F (s, y))(s, h) ) ds
0
Take an arbitrary  > 0. Sincef is dierentiable there exists a > 0 such that,
for |h| < , the last integral is bounded by
 t
|F (s, y + h) F (s, y)|ds C|h|
0

for some constant C. Consequently we get the estimate

 t
|(t, h) (t, h)h| fx (F (s, y))|(s, h) (s, h)h| ds + C|h|
0

Gronwalls lemma then gives

|(t, h) (t, h)h| C|h|
From the denition of dierentiability it then follows that
for a new constant C.
Fy (t, y) = (t, y)

15
2.3 Series expansions
Consider again the dierential equation

x(t)
= f (x(t)), x(0) = x0 (2.16)

The solution x(t) is continuously dierentiable as a function of t, since it satises

(2.16). Let us assume that f is continuously dierentiable. Then the right hand
side is a continuously dierentiable function of t, which means that the left hand
side is, which means that x(t) is in fact twice continuously dierentiable with
respect to time. Dierentiating using the chain rule gives

x
(t) = fx (x(t))x(t)
= fx (x(t))f (x(t))

Dening the function

f (1) (x) = fx (x)f (x)
we can write
(t) = f (1) (x(t))
x
Let us now assume that f is twice continuously dierentiable. Then the function
f (1) (x) is once continuously dierentiable. It follows that x
(t) is continuously
dierentiable with
x(3) (t) = f (2) (x(t))
where
f (2) (x) = fx(1) (x)f (x)
Continuing in this fashion we have in fact proved

Theorem 2.5 Let f in (2.16) be k times continuously dierentiable. Then the

solution x(t) is k + 1 times continuously dierentiable with

x(j+1) (t) = f (j) (x(t)), j = 0, . . . , k (2.17)

where f (j) is dened recursively by

f (j) (x) = fx(j1) (x)f (x), j = 1, . . . , k, f (0) (x) = f (x) (2.18)

Corollary 2.1 Let f in (2.16) be k times continuously dierentiable. Then the

solution x(t) is given by

t2 (1) tk tk+1
x(t) = x0 + tf (x0 ) + f (x0 ) + + f (k1) (x0 ) + f (k) (x()) (2.19)
2 k! (k + 1)!

where 0 < < t.

Proof. Follows from a Taylor expansion.

Example 2.1 Consider the pendulum equation

 
x 1 = x2 0
, x0 =
x 2 = sin x1 a

16
We get  
x2
f (0) (x) = f (x) =
sin x1
    
0 1 x2 sin x1
f (1) (x) = =
cos x1 0 sin x1 x2 cos x1
    
(2) cos x1 0 x2 x2 cos x1
f (x) = =
x2 sin x1 cos x1 sin x1 x22 sin x1 + cos x1 sin x1
This gives
     
a (1) 0 (2) a
f (x0 ) = , f (x0 ) = , f (x0 ) =
0 a 0

with the series representation

 
at at3 /6
x(t) = + O(t4 )
a at2

If f is in fact analytic, then one can show the stronger result

Theorem 2.6 Let f in (2.16) be analytic. Then x is an analytic function of t,

given by
k
 t (k1)
x(t) = x0 + f (x0 ) (2.20)
k!
k=1

in a neighborhood of t = 0.

Example 2.2 Consider the scalar example

x = x2 , x(0) = x0

In this case we get

f (n) (x) = (n + 1)!xn+2
with the solution

 x0
x(t) = x0 + tn xn+1
0 =
n=1
1 x0 t

2.4 Exercises.
2.1 Show that the Riccati equation
d
P = AP + P AT + Q P C T R1 CP, P (0) = P0
dt
has a solution on the time interval [0, ] if  is small enough.

17
 2.2 Consider the scalar Riccati equation

p = 1 + p2 , p(0) = 0

Does it have a global solution ?

2.3 Use Theorem 2.2 to show that the scalar Riccati equation

p = 2ap + q p2 /r, p(0) = p0

has a solution on [0, t1 ] for any t1 , if q 0, r > 0.

2.4 Consider the dierential equation

d
x = x2 , x(0) = 1
dt
Suppose the initial condition is changed to x(0) = 1 + . Compute the change
in x(0.999) to rst order in , using

a. the variational equation.

b. the exact solution of the dierential equation for x(0) = 1 + .

2.5 Consider the dierential equation

d
x = 1 x2 , x(0) = 0
dt
Suppose the initial condition is changed to x(0) = . Compute the change in
x(1000) to rst order in , using

a. the variational equation.

b. the exact solution of the dierential equation for x(0) = .

2.6 Consider the dierential equation

x = f (x, p), x(0) = x0

where f is dierentiable. Prove that the solution x(t, p) is dierentiable with

respect to the parameter p and compute an expression for the derivative.
Hint: Rewrite the problem so that p becomes an initial state, by introducing
extra state variables.

18
Chapter 3

Canonical forms and exact

linearization.

In linear system theory it is well known that controller design is particularly easy
when the system is described by a controller canonical form. It is then natural
to ask if a similar canonical form can be achieved for a nonlinear system. We
will approach this question by rst looking at the input-output properties. This
will lead to the denition of the relative degree of a nonlinear system.

3.1 The relative degree of a nonlinear system

Consider a nonlinear system of the form

x = f (x) + g(x)u (3.1)

y = h(x) (3.2)

where x is an n-vector, u and y are m-vectors, and f , g and h are innitely

dierentiable functions. Here g is an n m matrix of dierentiable functions
and h an m-vector of dierentiable functions. Let g1 ,..,gm denote the column
vectors of g.
An important structural question for such a system is the extent to which inputs
directly aect outputs or their derivatives. Since we have assumed that the right
hand side of (3.2) does not depend on u, the output is not directly dependent
on any of the input signals. Dierentiating the i:th output we get, using the
chain rule
y i = hi,x (x)(f (x) + g(x)u) (3.3)

where hi,x is the row vector with elements

 
hi (x) hi (x)
hi,x (x) = ,...,
x1 xn

19
Let us introduce the operators

n

n

Lf = fi , Lgk = gik (3.4)
i=1
xi i=1
xi

We refer to Lf as the Lie derivative in the direction f . We can then rewrite

(3.3) as

m
y i = Lf hi + uk Lgk hi (3.5)
k=1

Before proceeding further let us consider some of the properties of the Lie deriva-
tive. As shown by (3.4) it is a rst order derivative operator. Applying one Lie
derivative to another gives an operator involving second order derivatives:
    gj   2
Lf Lg = fi gj == fi + fi gj
i
xi j xj i,j
xi xj i,j
xi xj

However, if we take Lf Lg Lg Lf , then the second order derivatives will cancel

out, and we get a new rst order operator, that can be interpreted as a Lie
derivative. 
  gj fj
Lf Lg Lg Lf = fi gi
j i
xi xi xj

The expression within parenthesis is called the Lie bracket of f and g and is
denoted [f, g]:
 gj fj
[f, g] = gx f fx g = fi gi
i
xi xi
We have thus shown the following formula

Lf Lg Lg Lf = L[f,g] (3.6)

Sometimes it is useful to consider repeated Lie brackets of the form

[f, g], [f, [f, g]], [f, [f, [f, g]]], . . .

The j times repeated Lie bracket of this form is denoted (adj f, g), for instance:

(ad3 f, g) = [f, [f, [f, g]]]

After this parenthesis about Lie derivatives we go back to (3.5). If Lgk hi is not
identically zero, then obviously the control uk will directly inuence y i , at lest
for some x. Suppose Lgk hi = 0 for all k and all x. Then we have y i = Lf hi and
we can continue dierentiating.

m
yi = L2f hi + uk Lgk Lf hi
k=1

As before we can see that uk inuences yi directly if Lgk Lf hi is not identically

zero. If Lgk Lf hi = 0 for all k and x, we continue the dierentiations. There
are two possibilities: either no time derivative of yi is directly dependent on the

20
 (i )
output, or else there is a smallest integer i such that yi depends directly on
some uk . In the latter case we have

Lgk Ljf hi 0, k = 1, . . . , m, j = 0, . . . , i 2
(3.7)
Lgk Lfi 1 hi 0, some k, 1 k m

Suppose that we have done the above procedure for all output signals and
obtained numbers 1 , . . . , m , satisfying (3.7). We can then write the input-
output relationship as
( )
y1 1
.
. = d(x) + R(x)u (3.8)
.
( )
ym m
where d and R are given by

Lf1 h(x) Lg1 Lf1 1 h1 ... Lgm Lf1 1 h1
.. .. ..
d(x) = . , R(x) = . . (3.9)
Lg1 Lfm 1 hm . . . Lgm Lfm 1 hm
m
Lf h(x)

From the denition of the i it follows that each row of R has at least one
element which is not identically zero. If R is nonsingular calculations become
especially simple as we shall see below.

Denition 3.1 We say that the system (3.1,3.2) has vector relative degree
(1 , . . . , m ) at x0 if (3.7) is satised and R(x0 ) is nonsingular

The main point in all the calculations we have made lies in the following fact.
Suppose the system has a relative degree. Then we can use the state feedback

u = R(x)1 (
u d(x)) (3.10)

where we regard u
as a new input signal. The resulting dynamics is

( )
y1 1
.
. =u
. (3.11)
( )
ym m

This is a decoupled linear relation from the new input to the output. Using
linear design techniques it is now possible to get any dynamics from input to
output. Our calculations can be summarized in the following theorem:

Theorem 3.1 Input-output linearization. A system having relative degree

can be given linear dynamics from input to output by using state feedback.

Proof. Follows directly from (3.8) and (3.10).

Example 3.1 Consider the rigid body of Example 1.4 with external moments
along only two of the principal axes. We consider these moments to be input

21
signals.

x 1 = a1 x2 x3 (3.12)
x 2 = a2 x1 x3 + u1 (3.13)
x 3 = a3 x1 x2 + u2 (3.14)

Assume that the outputs are the rst and second angular velocities:

y1 = x1 , y2 = x2 (3.15)

Since

Lg1 = , Lg2 =
x2 x3

Lf = a1 x2 x3 + a2 x1 x3 + a3 x1 x2
x1 x2 x3
we have
Lg1 h1 = Lg1 x1 = 0, Lg2 h1 = Lg2 x1 = 0
and

Lg1 Lf h1 = Lg1 a1 x2 x3 = a1 x3 , Lg2 Lf h1 = Lg2 a1 x2 x3 = a1 x2

showing that 1 = 2 if a1 = 0. Also we have

Lg1 h2 = Lg1 x2 = 1, Lg2 h2 = Lg2 x2 = 0

so that 2 = 1 and  
a1 x3 a1 x2
R(x) =
1 0
If a1 = 0, i.e. if the moments of inertia around the second and third axes are
not equal, then we see that the system has vector relative degree (2, 1) if x2 = 0.
If on the other hand a1 = 0, then all time derivatives of y1 are zero, so there is
no relative degree.

A major problem with the input-output linearization comes from the fact that
the linear input-output relationship (3.11) might not show all the dynamics of
the closed loop system. This is the case if the order of the system (3.11), i.e.
1 + + m is less than n, the number of state variables. There must then be
some dynamics which is unobservable from the output. If this hidden dynamics
is unstable it will not be possible to use the input-output linearization. To
investigate this question it is necessary to transform the system into a canonical
form with all state variables present.

3.2 A canonical form

Our discussion of relative degree and the example of the previous section suggest
that it might be natural to regard the outputs and their derivatives as state

22
variables. Therefore we dene new variables in the following way.
( 2) (2 2) ( 2) T
= (y1 , y 1 , . . . , y1 1 , y2 , y 2 , . . . , y2 , . . . , ym , y m , . . . , ym m )
( 1) ( 1)
= (y1 1 , . . . , ym m )T
= q(x)
(3.16)
or equivalently
( 2) (m 2)
= (h1 , Lf h1 , . . . , Lf 1 h1 , . . . , hm , Lf hm , . . . , Lf hm )T
( 1) ( 1)
= (Lf 1 h1 , . . . , Lf m hm )T (3.17)
= q(x)
where q is some function which is unspecied so far. If the relation between
these variables and x is invertible, then the new variables can be used as state
variables. We then get an interesting canonical form for the system.

Theorem 3.2 If the system (3.1), (3.2) has relative degree (1 , . . . , m ) at x0 ,

then q(x) in the variable change (3.17) can be chosen so that (3.17) is invertible
in a neighborhood of x0 . In the new variables the state space description is
= M + N
= 1 (, , ) + R((, , ))u
(3.18)
= 2 ((, , ) + 3 (, , )u
y = H 1 + H2
where M is a block diagonal matrix, where the i:th block has dimension (i
1) (i 1):

0 1 0 ... 0
M11 0 ... 0 0
0 0 1 0
M22 . . . 0




M = . . , with Mii = ... ..
.
..
.
.. ..
0 0 0 ... 1
0 Mmm
0 0 0 ... 0
N is also a block diagonal matrix with m blocks. Each block has i 1 elements
and is of the form
0
..

Nii = .
0
1
The matrices H1 and H2 are implicitly dened by (3.17). The function is the
inverse of (3.17)

To prove this theorem it is necessary to do some preliminary work. Introduce

the matrices
 T
S(x) = h1,x , . . . , (Lf1 1 h1 )x , . . . , hm,x , . . . , (Lfm 1 hm )x (3.19)
 
T (x) = g1 , . . . , gm , [f, g1 ], . . . , [f, gm ], . . . , (adr1 f, g1 ), . . . , (adr1 f, gm )
(3.20)
where r = max(1 , . . . , m ). We have the following fundamental fact

23
Proposition 3.1 If the system (3.1), (3.2) has relative degree (1 , . . . , m ) at
x0 , then the matrix S(x0 )T (x0 ) has rank 1 + + m (= number of rows).

Proof. Consider a case where m = 2, 1 = 2, 2 = 1. Then

h1,x
ST = (Lf h1 )x (g1 , g2 , [f, g1 ], [f, g2 ]) =
h2,x

Lg1 h1 Lg2 h1 L[f,g1 ] h1 L[f,g2 ] h1
Lg1 Lf h1 Lg2 Lf h1
Lg1 h2 Lg2 h2
where denotes elements whose values are unimportant. Now, since 1 = 2, we
have Lg1 h1 = 0, Lg2 h1 = 0. Using (3.6) we have

L[f,g1 ] h1 = Lf Lg1 h1 Lg1 Lf h1 = Lg1 Lf h1

L[f,g2 ] h1 = Lf Lg2 h1 Lg2 Lf h1 = Lg2 Lf h1

so that

0 0 Lg1 Lf h1 Lg2 Lf h1
S(x0 )T (x0 ) = Lg1 Lf h1 Lg2 Lf h1
Lg1 h2 Lg2 h2

The 2 2 block matrix to the lower left is exactly R(x0 ) and is therefore non-
singular. The two elements to the upper right constitute the rst row of R(x0 ),
so they can not both be zero. Consequently S(x0 )T (x0 ) has full rank. This
case (m = 2, 1 = 2, 2 = 1) can easily be extended to the general one. By
repeated use of (3.6) and possibly permutation of the columns, the matrix can
be brought into block triangular form, where the blocks on the diagonal consist
of rows of R(x0 ). The matrix must then have full rank.

From this proposition we can draw the following conclusions.

Proposition 3.2 If the system (3.1), (3.2) has relative degree (1 , . . . , m ) at

x0 , then

1. 1 + + m n
2. S(x0 ) has linearly independent rows
3. there are at least 1 + + m independent columns in T (x0 )

Proof. From Proposition 3.1 and the properties of the matrix rank, it follows
that

1 + + m = rank S(x0 )T (x0 ) min(rank S(x0 ), rank T (x0 ))

All three statements follow immediately.

We can now prove the main result.

24
Proof.(of Theorem 3.2) The Jacobian (at x0 ) of the coordinate change (3.17)
is (after some reordering of the rows)
 
S(x0 )
qx (x0 )

We know from Proposition 3.2 that S(x0 ) has linearly independent rows. Since
we are free to choose q, we can choose it so that the rows of qx are linearly
independent of those of S. The Jacobian is then nonsingular and it follows,
from the implicit function theorem, that the variable change (3.17) is invertible
close to x0 .

The canonical form (3.18) suggests an interesting choice of controller. Suppose

v is an m-vector of reference signals for the output y. Then the control law

u = k(, , , v) = R((, , ))1 (1 (, , ) + 1 (, ) + 2 (, )v) (3.21)

gives the dynamics

= M + N
= 1 (, ) + 2 (, )v (3.22)
y = H
from reference signal to output. Notice that the variable is not included in
this dynamics. The functions 1 and 2 can be chosen arbitrarily. There are
several interesting possibilities.

Noninteracting control

If the functions 1 and 2 in the control law (3.21) are chosen so that the i:th
component is of the form
(i 1)
(1 (, ))i = ai (yi , y i , . . . , yi )

(i 1)
(2 (, )v)i = bi (yi , y i , . . . , yi ) vi
then the relation between v and y breaks down into m relations of the form
(i ) (i 1) (i 1)
yi = ai (yi , y i , . . . , yi ) + bi (yi , y i , . . . , yi ) vi

We have then achieved noninteracting control. The i:th reference signal aects
only the i:th output.

Linear reference-output relation

Let the functions 1 and 2 in the control law (3.21) be chosen in the following
way.
+N
1 (, ) = M

2 (, ) = R

25
 v - = - y
=

?
- =

Figure 3.1: Structure of the system

where M , N and R are constant matrices of appropriate dimensions. The

resulting dynamics is then

= M + N
= M
+N + Rv
(3.23)
y = H
which is linear from v to y.

3.3 The zero dynamics

The controllers discussed in the previous section have a peculiar property. If we
consider equation (3.22), we see that the variable of (3.18) does not eect the
referenceoutput relationship. Writing down all of the dynamics gives

= M + N
= 1 (, ) + 2 (, )v
(3.24)
= 2 ((, , ) + 3 (, , )k(, , , v)
y = H

Graphically the situation described by (3.24) is shown in gure 3.1. We see that
the dynamics acts as a sink. It does not aect the output put is (possibly)
aected by v, and . This is something that we recognize from linear system
theory part of the dynamics is unobservable.
Let us consider the situation where v = 0, so that the desired output is y 0.
It is then natural to choose 1 so that
1 (0, 0) = 0
Suppose the system is initialized with
(i 1)
yi (0) = 0, y i (0) = 0, . . . , yi (0) = 0, i = 1, . . . , m
We will then have
0, 0

26
 = 2 (0, 0, ) + 3 (0, 0, )k(0, 0, , 0) (3.25)

Denition 3.2 The dynamics described by (3.25) is called the zero dynamics
of the system.

The term zero dynamics is motivated by the linear case.

Example 3.2 Consider the system (observer canonical form)

a1 1 0 0
x = a2 0 1 x + b2 u
a3 0 0 b3
 
y= 1 0 0
with transfer function
b2 s + b3
G(s) =
s3 + a 1 s2 + a 2 s + a 3

We assume that b2 = 0. Then y does not depend directly on u, but y does, so

the relative degree is 1 = 2. Introducing

1 = x1 , 1 = a1 x1 + x2 , = x3

we have the canonical form

1 = 1
1 = a1 1 a2 1 + 3 + b2 u
= a3 1 + b3 u

The zero dynamics corresponds to the control that keeps 1 and 1 identically
zero, i. e. u = 3 /b2 . The zero dynamics is then

b3
3 = 3
b2

The eigenvalue of the zero dynamics (b3 /b2 ) is thus equal to the zero of the
transfer function.

This example is easily extended to single-input-single-output linear systems of

arbitrary order and relative degree. The conclusion is the same: the eigenvalues
of the zero dynamics are the zeroes of the transfer function. By analogy with
the linear case the term zero dynamics is used also in the nonlinear case. The
linear analogy also explains the following denition.

Denition 3.3 A nonlinear system is called minimum phase if the origin is an

asymptotically stable point of the zero dynamics.

To see the signicance of the minimum phase property, we consider another

example.

27
Example 3.3 Consider the system

x 1 = x22 + u
x 2 = u (3.26)
y = x1

This system already has the canonical form. To keep y identically zero, one has
to use the control law
u = x22

The zero dynamics is then

x 2 = x22

which is clearly unstable, so the system is non minimum phase.

This example clearly shows that control of non minimum phase systems can be
tricky. A control that keeps the output identically zero is unacceptable since it
makes certain internal states (and the control signal) grow without bound.
The problems with the zero dynamics dissappear if the relative degree satises

1 + + m = n

In this case the vector has dimension zero, so there is no zero dynamics. Our
earlier example 3.1 is such a case.

Example 3.4 Consider again example 3.1. The coordinate change

1 = x1 , 1 = a1 x2 x3 , 2 = x2

gives the dynamics

1 = 1
a 2
1 = a2 1 121 + 1
2 u1 + a1 a3 1 22 + a1 2 u2
2
2 = a2 1 1
a1 2 + u1

The feedback
a2 1 1
u1 = + v2
a1 2
 
1 1
u2 = v2 a1 a3 1 22 + v1
a1 2 2
gives the decoupled closed loop dynamics

y1 = v1 , y 2 = v2

Further linear feedback can then give arbitrary closed loop poles. There is no
hidden dynamics.

28
3.4 Controller canonical form.
The last section showed us that there are great advantages with systems where
the relative degree satises

1 + + m = n (3.27)

so that there is no zero dynamics. Suppose one starts with a system

x = f (x) + g(x)u (3.28)

Is it then possible to choose an output so that the relative degree satises (3.27)?
To answer that question we will specialize to the case m = 1, i.e. the single-
input-single-output case.
From (3.7) it follows that we require a function h(x) satisfying

Lg Ljf h 0, j = 0, . . . , n 2
(3.29)
Lg Ln1
f h 0

This can be rewritten using (3.6):

L[f,g] h = Lf Lg h Lg Lf h = 0

L[f,[f,g]] h = Lf L[f,g] h L[f,g] Lf h = Lf Lg Lf h + Lg L2f h = 0

and so on. We nd that (3.29) is equivalent to

L(adk f,g) h = 0, k = 0, 1, . . . , n 2 (3.30)

L(adn1 f,g) h = 0 (3.31)

This is a system of partial dierential equations that has to be solved. It is
helpful to look at its geometric interpretation. For n = 3 we have the conditions

Lg h = 0, L[f,g] h = 0

They are satised if we can nd a family of surfaces of the form h(x) = c such
that g and [f, g] are tangent to the surfaces at every point, see gure 3.2. It
turns out that this construction can only be carried out if the vectors satisfy
certain conditions. To discuss this we need the following denition.

Denition 3.4 A collection of vector elds f1 (x), . . . , fp (x) is called involutive

if all the Lie brackets are linear combinations of the fi , i. e.


p
[fi , fj ](x) = k (x)fk (x), i, j = 1, . . . , p
k=1

where the k are innitely dierentiable scalar functions.

We also need the following fact about Lie brackets.

29
 [f, g]

h(x) = c

Figure 3.2: Geometric interpretation of the conditions Lg h = 0, L[f,g] h = 0.

Proposition 3.3 Let the solution of

x = f (x), x(0) = x0

after t units of time be xa , let the solution of

x = g(x), x(0) = xa

at t = h be xb , let the solution of

x = f (x), x(0) = xb

at t = h be xc and let nally the solution of

x = g(x), x(0) = xc

at t = h be xd (see Figure 3.3). Then

f
xc xb

xd f
xa
x0

Figure 3.3: Geometric construction to interpret the Lie bracket.

xd = x0 + h2 [f, g](x0 ) + O(h3 )

30
Proof. A Taylor expansion gives

h2
xa = x0 + hx(0)
+ (0) + O(h3 )
x
2
Since x = f and x
= fx x = fx f this can be written

h2
xa = x0 + hf (x0 ) + fx (x0 )f (x0 ) + O(h3 )
2
Analogous calculations give

h2
xb = xa + hg(xa ) + gx (xa )g(xa ) + O(h3 )
2
h2
xc = xb hf (xb ) + fx (xb )f (xb ) + O(h3 )
2
h2
xd = xc hg(xc ) + gx (xc )g(xc ) + O(h3 )
2
The right hand sides of these expressions can be evaluated at x0 . For instance

g(xa ) = g(x0 ) + gx (x0 )(xa x0 ) + O(h2 ) = g(x0 ) + hgx (x0 )f (x0 ) + O(h2 )

Performing analogous calculations for xc and xd gives

h2
xa = x0 + hf + fx f + O(h3 )
2
h2
xb = xa + hg + h2 gx f + gx g + O(h3 )
2
h2
xc = xb hf h2 fx (f + g) + fx f + O(h3 )
2
h2
xd = xc hg h gx (f + g f ) + gx g + O(h3 )
2
2
where the right hand side is evaluated at x0 . Substituting each equation into
the one below nally gives

xd = x0 + h2 (gx f fx g) + O(h3 ) = x0 + h2 [f, g] + O(h3

This result has immediate consequences for the construction of Figure 3.2. Con-
sider the construction of Proposition 3.3 but with f and g replaced by g and
[f, g]. If g and [f, g] are both tangent to h(x) = c at every point, then all so-
lutions to x = g and x = [f, g] lie in h(x) = c. From Proposition 3.3 the Lie
bracket [g, [f, g]] must then be tangent to h(x) = c. It will then be a linear com-
bination of g and [f, g] (with x-dependent coecients). The geometric picture
of Figure 3.2 thus implies that g and [f, g] are involutive.

Theorem 3.3 The system (3.30) with the condition (3.31) has a solution in a
neighborhood of a point x0 if and only if

31
 1. the vectors
(adk f, g)(x0 ), k = 0, . . . , n 1 (3.32)
are linearly independent
2. the vector elds
(adk f, g), k = 0, . . . , n 2 (3.33)
are involutive in a neighborhood of x0 .

Proof. (sketch) First we note that the necessity of condition 1 follows from the
third statement of Proposition 3.2.
For the other part of the theorem it is helpful to consider again the geometric
interpretation of gure 3.2. In the illustrated three-dimensional case g and [f, g]
have to be tangent to h(x) = c. For a general n the surface h(x) = c has to be
tangent to g, [f, g],...,(adn2 f, g) at all points.
Now consider what happens if a curve is generated which is tangent to g, then to
[f, g], then to g and nally to [f, g]. Since all these vector elds are tangent
to the surface h(x) = c, the curve will remain in the surface if it starts there.
On the other hand it follows from Proposition 3.3 that the resulting movement
will be in the direction [g, [f, g]]. Consequently [g, [f, g]] also has to be tangent
to h(x) = c. This argument can be extended to show that all brackets formed
among g, [f, g],...,(adn2 f, g) have to be tangent, showing that these vector
elds have to be involutive. We have thus sketched the necessity part of the
theorem.
That our conditions are also sucient follows from a celebrated theorem by
Frobenius, forming one of the cornerstones of dierential geometry.

After calculation of the function h we can introduce new state variables accord-
ing to (3.17). In this special case we get
 
zk = Lk1f h (x), k = 1, . . . , n (3.34)

with the state space description

z1 = z2
z2 = z3
.. (3.35)
.
zn = Lnf h + (1)n1 L(adn1 f,g) h u
We will call this form controller canonical form in analogy with the linear case.
In our discussion of relative degree we assumed that the output is y = z1 . How-
ever, the controller canonical form can be useful no matter what the output is.
In many cases we can regard h(x) as a ctitious output, used as an intermediate
step in the calculations.

3.4.1 Construction of the coordinate transformation.

We now turn to the actual computation of the function h whose existence is

guaranteed by Theorem 3.3. The procedure is also illustrated in gure 3.4.

32
 g

[f, g]

h(x) = c

g
0

Figure 3.4: Construction of coordinate transformation.

Take an arbitrary vector eld g(x) which is linearly independent of the vectors
in (3.33) . Let the solution of x = g(x) with initial value x0 be denoted 1 (t, x0 ).
Also let
2 (t, x0 ), . . . , n (t, x0 )
denote the solutions of

x = (adn2 f, g), . . . , x = [f.g], x = g(x)

respectively. Consider the following procedure. Starting at x = 0, follow the

solution of x = g for t1 units of time to the point x = 1 (t1 , 0). Then follow
the solutions of x = (adn2 f, g), x = (adn3 f, g) etc. for t2 , t3 , . . . units of time
respectively. Finally after following the solution curve of x = g for tn units of
time, the point
  

x = (t1 , . . . , tn ) = n tn , n1 tn1 , . . . 1 (t1 , 0) (3.36)

is arrived at. It is easy to see that the Jacobian of evaluated at ti = 0 is

simply  
g (adn2 f, g) . . . [f, g] g
Since the columns are linearly independent, the Jacobian of is nonsingular at
the origin. It follows from the inverse function theorem that there is a neigh-
borhood of the origin where is invertible. Then it is possible to solve the
equation (t1 , . . . , tn ) = x for t1 , giving

t1 = h(x)

To show the procedure, a simple example is worked out.

Example 3.5 The following system is given.

x 1 = x2 + x2 u
x 2 = x2 + u

33
The vectors of (3.32) are then
   
x2 x2 1
g= [f, g] =
1 1

which are linearly independent if x2 = 1/2. Taking g = (1 0)T gives

 
t1 + t22 /2
(t1 , t2 ) =
t2

and solving for t1 gives

h(x) = x1 x22 /2
From (3.34) the coordinate change is then

z1 = x1 x22 /2
z2 = x2 + x22

or equivalently
 
x1 = 1/4 + z1 + z2 /2 z2 + 1/4 2

x2 = 1/2 + z2 + 1/4

The canonical form shown by (3.35) is then

z1 = z2  
z2 = 1/2 2z2 + z2 + 1/4 + 2 z2 + 1/4 u

Clearly the relationship between x2 and z2 is only invertible if x2 > 1/2, so

the transformation is not a global one.

3.5 Exact linearization.

We saw earlier (equation (3.23)) that it is possible to get a linear relationship
between reference and output. For the special case we are discussing now we
get the following result.

Theorem 3.4 Suppose the system (3.28) with a scalar input satises the con-
ditions of Theorem 3.3. Then there exists a nonlinear transformation

z = T (x)

and a nonlinear feedback

u = k1 (x) + k2 (x)v
such that in the new variables the system has the form

z = Az + Bv

i.e. it is a linear system.

34
Proof. First transform the system to controller canonical form (3.35) as de-
scribed in the previous section. Then apply the nonlinear feedback
1
u= ((x) + v)
(x)
(The resulting system has only pure integrators. This is sometimes called the
Bronowsky canonical form. Of course it is possible to get any pole placement
by adding suitable terms.)

3.6 Exact linearization the multi-input case

The results for the single input case can be generalized to the case where u in
x = f (x) + g(x) u (3.37)
is an m-vector. One then has to consider
G0 = span(g1 , . . . , gm )
G1 = span(g1 , . . . , gm , [f, g1 ], . . . , [f, gm ])
..
.
Gn1 = span((adk f, gj ), k = 0, . . . , n 1, j = 1, . . . , m)

Theorem 3.5 Assume that g(x0 ) has rank m. For the system (3.37) it is
possible to introduce an output y = h(x) so that the relative degree satises
1 + + m = n in a neighborhood of x0 if and only if

1. all Gi have constant rank near x0

2. Gn1 has rank n.
3. G0 , . . . , Gn2 are involutive.

We illustrate the theorem with an example.

Example 3.6 Consider the control of a missile, gure 3.5. The components
of the velocity along the x-, y- and z-axes are u, v and w respectively. The
angular velocities are p, q and r. The forward velocity usually changes fairly
slowly and is therefore assumed to be constant. The force equations along the
y- and z-axes then give
Fy = m (v + ru pw gy ) (3.38)
Fz = m (w qu + pv gz ) (3.39)
where gy and gz are the gravitational components and Fy , Fz the aerodynamic
force components. The rigid body rotational equations are
Mx = Ixx p (3.40)
My = Iyy q (Izz Ixx )rp (3.41)
Mz = Izz r (Ixx Iyy )pq (3.42)

35
 x

y
z

Figure 3.5: Coordinate system of missile

where Mx , My and Mz are the aerodynamic torques and Ixx , Iyy and Izz the
moments of inertia. Introducing the state variables

x1 = w/u, x2 = v/u, x3 = p, x4 = q, x5 = r (3.43)

neglecting gravitation and assuming Iyy = Izz = I gives

x 1 = x4 x3 x2 + Cz (3.44)
x 2 = x5 + x3 x1 + Cy (3.45)
x 3 = C (3.46)
x 4 = Iq x3 x5 + Cm (3.47)
x 5 = Iq x3 x4 + Cn (3.48)

where
Fy Fz
CY = , CZ =
mu mu
Mx My Mz I Ixx
C = , Cm = , Cn = , Iq =
Ixx Iyy Izz I
Introduce
 T
C = CZ , CY , C , Cm .Cn

The elements of C depend on the state variables and control signals. We assume
that the missile is controlled by the three control signals

u 1 = e , u 2 = r , u 3 = a

corresponding to elevator, rudder and ailerons respectively. We assume for

simplicity that the control acts linearly to give

C = F (x) + g(x)u

36
Here F and g depend mainly on x1 and x2 , while the dependence on the angular
velocities x3 , x4 , x5 is much weaker. A typical structure for g is

g11 (x) 0 0
0 g22 (x) 0

g(x) =
0 0 g
33 (x) (3.49)
g41 (x) 0 0
0 g52 (x) 0

with g11 much smaller than g41 and g22 much smaller than g52 . The model of
the missile has the form
x = f (x) + g(x)u
with f given by
x4 x3 x2 + F1 (x)
x5 + x3 x1 + F2 (x)

f (x) =
F3 (x)
(3.50)
Iq x3 x5 + F4 (x)
Iq x3 x4 + F5 (x)
The simplest situation is when g(x) is actually constant. Then the transforma-
tion

z1 = x1 1 x4 (3.51)
z2 = x2 2 x5 (3.52)
z3 = x3 (3.53)
z4 = f1 (x) 1 f4 (x) (3.54)
z5 = f2 (x) 2 f5 (x) (3.55)

with 1 = g11 /g41 and 2 = g22 /g52 gives

z1 = z4 (3.56)
z2 = z5 (3.57)
z3 = f3 (x) + g33 u3 (3.58)
z4 = (f1 (x) 1 f4 (x))x (f (x) + gu) (3.59)
z5 = (f2 (x) 2 f5 (x))x (f (x) + gu) (3.60)

With z1 , z2 and z3 regarded as outputs, this system has relative degree (2, 2, 1)
and feedback linearization is possible. If g depends on x the situation is more
complicated. It is then necessary that g1 , g2 and g3 depend on x in such a way
that they are involutive.

3.7 Exercises.
3.1 Suppose the system of Exercise 3.5 has the output

y = 1 + x2

What is the relative degree? What is the zero dynamics?

37
3.2 Consider the bilinear system

x 1 = x1 + u
x 2 = 2x2 + x1 u
y = x1 + x2

What is the relative degree? What is the zero dynamics?

3.3 Consider the heat exchanger of example 1.2.

d
(CT ) = qcT0 qcT + (Th T )
dt
Let the state variables be x1 = T , x2 = q and Th = x3 . Suppose x2 and x3
are controlled from the inputs u1 and u2 with some lag due to time constants
in the control actuators. If T0 = 0, c/C = 1 and /C = 1, then the system is
described by
x 1 = x1 + x3 x2 x1
x 2 = x2 + u1
x 3 = x3 + u2
y1 = x1
y2 = x2
What is the relative degree? Describe a controller that linearizes and decouples
the system.

3.4 Transform the system

x 1 = x21 + x2
x 2 = x2 + u

into controller form. Compute a state feedback which gives linear dynamics
with poles in -1 and -2.

3.5 Transform the system

x 1 = 1 1 + x
1+u
x 2 = 1 + x1 1 + x2

into controller form. Compute a state feedback giving linear dynamics with
poles in -2 and -3.

3.6 Transform the system

x 1 = sin x2
x 2 = sin x3
x 3 = u

to controller form. In what region of the state space is the transformation well
dened ?

38
Chapter 4

Nonlinear observers

The use of exact feedback linearization described in the previous chapter requires
state feedback. This is also true of many other techniques that we will consider
in the coming chapters. Since all state variables are seldom measured, there is
a need to reconstruct the state from some measured output y. A device which
does this is called an observer. We will assume that the system has the form

x = f (x, u), y = h(x) (4.1)

where x, u and y are vectors with dimensions n, m and p respectively.

4.1 A straightforward observer

Recall that a linear system

x = Ax + Bu, y = Cx

has a natural observer of the form

= A
x x + Bu + K (y C x
) (4.2)

=xx
The observer error x satises

= (A KC)
x x (4.3)

and for an observable pair A, C the observer gain K can be chosen to give
A KC arbitrary eigenvalues.
A natural generalization of this observer to the nonlinear case (4.1) is to use

= f (
x x)(y h(
x, u) + K( x)) (4.4)

There are many strategies for choosing the gain K. Let the system (4.1) have
an equilibrium at the origin:

f (0, 0) = 0, h(0) = 0 (4.5)

39
with linearization
A = fx (0, 0), C = h(0) (4.6)
where the pair A, C is detectable. Then K can be chosen so that A K(0)C
has eigenvalues in the left half plane, guaranteeing an observer which converges
in a neighborhood of the origin.
A popular approach is the extended Kalman lter. Here the system (4.1) is
linearized around x
at each instance of time and K is chosen from a Kalman
lter design based on that linearization. The extended Kalman lter works
well in many applications, but there are few hard results on convergence and
performance.

4.2 Observers based on dierentiation

Consider to begin with a system without inputs.

x = f (x), y = h(x) (4.7)

If p = n, i.e. the number of outputs equals the number of states, then one can
construct a static observer by solving the nonlinear system of equations

h(x) = y (4.8)

with respect to x, at each time instant. The question of solvability of this

system of equations then arises. The basic mathematical fact that is used is the
following:

Theorem 4.1 The Implicit Function Theorem. Consider the system of

equations
F (x, y) = 0 (4.9)
where x Rn , y Rm and F is continuously dierentiable, Rn+m Rn . Let
xo , yo be such that

F (xo , yo ) = 0, Fx (xo , yo ) nonsingular

Then for all y in a neighborhood of yo (4.9) has a solution

x = (y), xo = (yo )

for some continuously dierentiable function .

From the implicit function theorem it follows that (4.8) can be solved at least
locally if the Jacobian hx is nonsingular.
Now consider the case of (4.7) under the more realistic assumption that p < n.
One can then get a system of equations with as many equations as unknowns
by considering also derivatives
(i 1)
y i = Lf hi (x), yi = L2f hi (x), . . . , yi = Lf i 1 hi (x)

40
Suppose we can nd integers 1 ,...,p , 1 + 2 + + p = n such that the row
vectors

h1,x , (Lf h1 )x , . . . (Lf 1 1 h1 )x

.. ..
. .
1
hp,x , (Lf hp )x , . . . (Lf p hp )x

are linearly independent for some xo . Then by the implicit function theorem we
can solve the corresponding system of equations

y 1 = h1 , y 1 = Lf h1 , . . . y (1 1) = Lf 1 1 h1
....
.. (4.10)
1
y p = hp , y p = Lf hp , . . . y (p 1) = Lf p hp

in a neighborhood of that xo . Often there are many ways of choosing the num-
bers 1 ,..,p that satisfy the linear independence requirements. However, one
usually wants to dierentiate physical signals as few times as possible. There-
fore it is natural to make the choice so that the highest i becomes as small as
possible. If this leaves some freedom of choice the second highest i should be
as small as possible, and so on. If the choice is made in this way, the numbers
1 .,,p are called the observability indices of the system.
We assume that every index satises i 1, that is we assume that every output
signal is used (otherwise we could delete one signal and regard the system as
one with fewer outputs). By reordering the output variables we can always get
the indices in decreasing order:

1 2 p 1 (4.11)

The indices i give the lengths of the rows in the array (4.10). The same
information is given by instead listing the lengths of the columns. We dene

i = number of outputs that are dierentiated i times (4.12)

Since we assume that all outputs are used, we have 0 = p. If the i are ordered
in decreasing order we have

p = 0 1 1 1 > 0 (4.13)

(Some authors call the i the observability indices.)

An observer based on dierentiation performs the calculations

1. Compute approximations of the derivatives

y1 , y 1 , . . . y (1 1)
.. ..
. .
yp , y p , . . . y (p 1)

41
 This can for instance be done using dierentiating lters of the form
s
1 + sT
where the lter time constant T has to be chosen as a compromise between
the desire to get an accurate derivative and the need to lter out high fre-
quency disturbances. Often the output is measured at discrete points of
time and then the problem of approximating the derivatives can be ap-
proached through numerical dierentiation, based on dierence schemes.

2. Solve the equations (4.10). There is no general way of performing this

step. For many physical systems the structure is such that an explicit
solution is possible. Otherwise a numerical solution has to be sought.
The non-singularity of the Jacobian guarantees that a Newton iteration
converges if the initial guess is good enough. If the equations consist of
polynomials, then Gr obner basis techniques might be used.

Dierentiation and input signals

So far we have looked at systems without inputs. If we include an input

x = f (x) + b(x)u, y = h(x) (4.14)

the situation becomes more complex, since the possibility of calculating x from
u and y will in general depend on the choice of input. Take for example the
system

x 1 = x2 u
x 2 = 0
y = x1

It is clear that if u = 0 there is no way to determine x2 , while for a nonzero u,

x1 and x2 can be calculated from

x1 = y, x2 = y/u

To study this problem in more detail we look at single-input-single-output sys-

tems of the form (4.14). To make the study easier we introduce the variables

z1 = h(x), z2 = (Lf h)(x), . . . , zn = (Ln1

f h)(x) (4.15)

or in vector notation
h(x)
(Lf h)(x)

z= .. = (x) (4.16)
.
(Ln1
f h)(x)

Proposition 4.1 If the Jacobian x (xo ) is nonsigular then the variable change
(4.16) is invertible in a neighborhood of xo .

42
Proof. Follows directly from the implicit function theorem.

In the new variables the dynamics becomes simple.

z1 = Lf h + Lb h u = z2 + g1 (z)u, where g1 (z) = Lb h

z2 = L2f h + Lb Lf h u = z3 + g2 (z)u, where g2 (z) = Lb Lf h
..
.
zn = Lnf h + Lb Ln1
f h u = (z) + gn (z)u, where (x) = Lnf h, gn (z) = Lb Ln1
f h

The system description in the z-variables is thus

z2 g1 (z)
.. ..

z = . + . u (4.17)
zn gn1 (z)
(z) gn (z)

This form has a number of interesting properties.

(n1)
Proposition 4.2 If u = 0 then z in (4.17) can be computed from y, y,..,y
.

Proof. Equation (4.17) directly gives

z1 = y, . . . zn = y (n1)
z2 = y,

If (4.17) has a special structure then it is possible to compute z for any choice
of u (provided u is known).

Proposition 4.3 Let (4.17) have the form

z2 g1 (z1 )
z3 g2 (z1 , z2 )

.. .
z = . + .. u (4.18)

zn gn1 (z1 , . . . , zn1 )
(z) gn (z)

Then z can be computed from u, y and their derivatives no matter how u is

chosen.

Proof. From (4.18) it follows that, for each j

zj = zj1 gj1 (z1 , . . . , zj1 )u

Since each zj only depends on the variables with lower indices, the zj can be
calculated recursively, starting with z1 = y.

43
4.3 High gain observers
In the previous section z was computed using derivatives of y and u (or their
approximations). With the structure (4.18) there is also a possibility of using
the observer structure (4.4) to avoid explicit calculations of derivatives. If we
introduce the matrices

0 1 0 ... 0
0 0 1 0 0
..
.. .
. . 0 , B =  
A= . . , C = 1 0 . . . 0 (4.19)
. 0
.. 1 1
0 0 0 ... 0

the system can be written

z = Az + B(z) + g(z)u, y = Cz (4.20)

The natural observer is then

z = A
z + B( z )u + K(y C z)
z ) + g( (4.21)

The observer error z = z z then satises

z = (A KC)
z + B((z) (
z )) + (g(z) g(
z )) u (4.22)

where the matrix

A KC
has the structure
k1 1 0 ... 0
k2 0 1 0

.. ..
A KC =
. . 0
(4.23)
.
.. 1
kn 0 0 ... 0
To simplify things we consider the situation when u = 0. Using the notation
= ((z) (
z )) we can write

z = (A KC)
z + B

We see that the observer can be regarded as a linear system with the contribution
from the nonlinearity as an external signal. The transfer function from to z
is given by

1
k1 + s
1
(sI A + KC)1 B = n .
s + k1 sn1 + + kn ..
sn1 + k1 sn2 + + kn1
(4.24)
Now suppose K is chosen in the following way

k1 = 1 /, k2 = 2 /2 , . . . , kn = n /n

44
where 1 ,..,n are chosen such that
sn + 1 sn1 + + n = 0
has roots strictly in the left half plane. Using these relations in (4.24) gives the
following tranfer function from to z.

(sI A + KC)1 B =

n
 n1
(1 + (s))
1
.. (4.25)
(s) + 1 (s)n1 + + n
n .
((s)n1 + 1 (s)n2 + + n1 )
Since all the transfer functions go to zero as  tends to zero the inuence of
the nonlinear terms on the estimation error becomes negligible. Intuitively this
observer should therefore work well for small  irrespective of the form of the
nonlinearity. To get precise results we need tools from stability theory that will
be presented in the following chapters. We will then return to the question of
stability of the high gain observer.
There are some obvious disadvantages of the high gain observer:

The measurement signal y is multiplied by the coecients ki = i /i that

tend to innity as  tends to zero. Any measurement error will therefore
be amplied by a factor that can become very large.
Since s in (4.25) always occurs as s the parameter  will act as a frequency
scaling so that the bandwidths of the transfer functions go to innity as 
goes to zero. The observer could therefore be sensitive to modeling errors
or noise at high frequencies.

4.4 Observers based on state transformations

There is one form of the system equations that is nice for proving convergence
of the observer, namely
x = Ax + f (u, y), y = Cx (4.26)
With the observer
= A
x x + f (u, y) + K(y C x
) (4.27)
the estimation error x = x x satises
= (A KC)
x x (4.28)
It is thus possible to get global convergence of the observer error, provided
the pair A, C is detectable. The form (4.26) is very special, but there is the
possibility of transforming a wider class of systems into that form. To begin
with we will look at systems without inputs and try to nd a transformation
x = X() which transforms the system
x = f (x), y = h(x) (4.29)

45
into the form

= f(),
y = h() (4.30)
f() = A + b(y), h() = C (4.31)

Let Y denote the inverse transformation: = Y (x) = X 1 (x). We assume

that the system (4.29) is characterized by the observability indices i and the
parameters i satisfying (4.11), (4.12), (4.13). We use the notation = 1 .
Introduce the observability matrix

h1,x
..
.

h0 ,x

(Lf h1 )x

..
.

Q= (4.32)

(Lf h1 )
..
.

(L h1 )x
f
.
.
.
(Lf h1 )x

It follows from the denition of the observability indices that Q is nonsingular

in a neighborhood of some point xo .
To simplify the calculations it is desirable to pick A and C matrices which are
simple. In the single output case every observable pair A, C can be transformed
into observer canonical form

a1 1 0 ... 0
a2 0 1 0
 
.. .. ,
A= . . C= 1 0 0 ... 0

an1 0 . . . 0 1
an 0 ... 0 0

If the linear terms depending on 1 = y are included in b(y) the system (4.30)
then takes the form
1 = b1 (1 ) + 2
2 = b2 (1 ) + 3
.. (4.33)
.
n = bn (1 )
y = 1

which can be regarded as a nonlinear observer form.

46
In the multi-output case an observable pair A, C can be transformed into

E1 0
0 E2
 
..
A= . , C = C1 0 0 (4.34)

0 E1
0

where the block in position i, j has dimensions i1 j1 . Ei consists of the
rst i1 rows and the rst i columns of an identity matrix, C1 is an inverible
matrix. The blocks marked depend only on the rst 0 states, i.e. on y. If
they are included in b(y), then the linear part is described by

0 E1 0
0 0 E2
 
..
A= . , C = C1 0 0 (4.35)

0 E1
0 0

The pair A, C then constitutes a so called condensed Brunovsky canonical form.

For example, if 1 = 3 and 2 = 1, then 0 = 2, 1 = 1, 2 = 1 and

0 0 1 0  
0 0 0 0 0 0
A=
0
, C=
0 0 1 0 0
0 0 0 0

If C1 = I the canonical form is called (dual) Brunovsky form. We will see below
that this case gives (fairly) simple calculations. Unfortunately some systems
can only be transformed into (4.30) by choosing a C1 dierent from the identity
matrix.
Now consider the properties of the transformation x = X(). Dene the Jaco-
bian
J() = X () (4.36)

By dierentiating the relation x = X() we get

x = J() = J()f() (4.37)

Since we also have x = f (x), we have the following relation between the right
hand sides of the equations

f (X()) = J()f() (4.38)

Now suppose we have a dierent pair of vector elds that are transformed in
the same way:
g(X()) = J() g () (4.39)

We have the following relation between their Lie brackets.

47
Proposition 4.4 Let the vector elds f , g be transformed as in (4.38), (4.39).
Then their Lie Brackets are related as

[f, g] = J[f, g] (4.40)

Here the Lie bracket in the left hand side is evaluated using dierentiation with
respect to x, while the bracket in the right hand side uses dierentiation with
respect to .

Proof. Exercise.

If g is a matrix with columns gi , g = (g1 , . . . , gk ) we can interpret the Lie bracket

columnwise:
[f, g] = [f, (g1 , . . . , gk )] = ([f, g1 ], . . . , [f, gk ]) (4.41)
Proposition 4.4 then remains true. We can now use this formalism to describe
the transformation from (4.29) to (4.30). Take g to be the unit matrix in the
coordinates. We then have

[f, g] = J[f, g] = J[f, I] = J f = J(A + b ) (4.42)

Since g(X()) = J()

g () = J() we have, rewriting in the x-coordinates

[f (x), J(Y (x))] = J(A + b (Y (x))) (4.43)

Now partition J into blocks J = (J1 , J2 , . . . , J ) where the number of columns

in each block is 0 , 1 , ..1 respectively. Interpreting (4.43) for each block
column gives

[f, J1 ] = Jb1 (4.44)

[f, Jk ] = Jk1 Ek1 , k = 2, . . . , (4.45)

where b1 is the Jacobian of b with respect to the rst 0 = p components of

(since b depends only on the outputs, it depends only on those variables). We
get further relations by considering the output equations

y = h(x),
y = h() = C

Dierentiating the output we have

h(x) = y = h()

(Lf h)(x) = y = (L h)()f
..
.
(L1
f h)(x) = y (1 ) = (L1
f
h)()

Dierentiating with respect to gives

hx
h
(L h)
(Lf h)x
f
..J =
..

. .
1 1
(Lf h)x (Lf h)

48
Evaluating the right hand side nally gives

hx C1 0 0 ... 0
(Lf h)x C1 E1 0 . . . 0

.. J = .. . (4.46)
. . . .
1
(Lf h)x ... C1 E1 E1

where the elements marked * depend on b. From (4.46) and (4.45) it is possible
to calculate the desired coordinate change if it exists. First we take a look at
the single variable case.

Observer form for single output systems

For p = 1 we can pick out the last column of equation (4.46) to get

hx 0
(Lf h)x ..

.. Jn = . (4.47)
. 0
(L1
f h)x 1

For the single output case the matrix to the left is just Q, which we assumed
to be nonsingular. This equation can therefore be solved to give Jn . The other
columns of J can then be calculated successively from (4.45) which for the single
output case becomes

Jk1 = [f, Jk ], k = n, . . . , 2 (4.48)

After solving for J we can calculate the transformation from

Y (x) = J 1 (x) (4.49)

provided this system is integrable, that is if J 1 (x) actually is the Jacobian of

some vector valued function. We have the following theorem.

Theorem 4.2 The problem of transforming (4.29) into (4.30) can be solved for
the scalar output case if and only if

1. The matrix Q is nonsingular.

2. Yx (x) = J 1 (x) with the columns of J given by (4.47), (4.48) is integrable.

The solution is obtained by computing = Y (x) from Y (x) = J 1 (x).

Proof. Our calculations above have shown that the conditions 1. and 2. are
necessary. The suciency follows as a special case of the multivariable case
discussed below.

To illustrate the importance of the integrability condition we look at two exam-

ples.

49
Example 4.1 Consider the system

x 1 = x2
x 2 = x1 x2
y = x1

Since Q = I we immediately get J2 = (0 1)T . J1 is then calculated from

    
0 1 0 1
J1 = [f, J2 ] = fx J2 = =
x2 x1 1 x1

giving    
1 0 1 1 0
J= , J =
x1 1 x1 1
The rst part of the coordinate change is already given by y = x1 = 1 . The
second part has to satisfy
2
= x1
x1
2
=1
x2

The rst equation gives

x21
1 = + (x2 )
2
where is an arbitrary function. Dierentiating and using the second equation
gives
 (x2 ) = 1 2 (x2 ) = x2 + k
for some constant k. Picking k = 0 gives the coordinate transformation

1 = x1
x21
2 = + x2
2
The system description in the new coordinates becomes

2
1 = 1 + 2
2
2 = 0
y = 1

Example 4.2 Now consider the system

x 1 = x2
x 2 = x21 + x22
y = x1

50
Performing calculations similar to those of the previous example we get
   
1 0 1 1 0
J= , J =
2x2 1 2x2 1
The second -coordinate then has to satisfy
2
= 2x2
x1
2
=1
x2

Solving the rst equation gives

2 = 2x1 x2 + (x2 )
for some arbitrary function 2 . Dierentiating with respect to x2 and using the
second equation gives
2
= 2x1 +  (x2 ) = 1
x2
This relation can not be satised for any choice of depending only on x2 and
the problem is thus not solvable.

The integrability can be checked without actually computing the transformation

x = X().

Proposition 4.5 A necessary and sucient condition for integrability of Yx (x) =

J 1 (x) is that
[(adj f, Jn ), (adk f, Jn )] = 0, j = 1, . . . , n 1, k = 1, . . . , n 1 (4.50)

Proof. We give only the necessary part of the proof. We have Yx (x)J(x) = I.
It follows that Yx Jk = ek , where ek the k:th column of the identity matrix.
From Proposition 4.4 we get Yx [Ji , Jk ] = [ei , ek ] = 0 implying [Ji , Jk ] = 0.
Since Jn1 = (ad1 f, Jn ), Jn2 = (ad2 f, Jn ) etc., the result follows.

Remark Since Lie brackets satisfy the Jacobi identity:

[a, [b, c]] + [b, [c, a]] + [c, [a, b]] = 0 (4.51)
it is not necessary to check all the brackets of (4.50). Some of them can be
deduced from the others.

Observer form for multi-output systems with C1 = I

For multivariable systems we rst consider the case where the transformation
can actually be made to dual Brunovsky canonical form so that C1 = I. Equa-
tion (4.46) then takes the form

hx I 0 0 ... 0
(Lf h)x E 1 0 . . . 0

.. J = .. .. (4.52)
. . .
(L1 h)x
f ... 1
E

51
where E i is a p i matrix whose rst i rows form a unit matrix. If we look
at the rows corresponding to Ljf h we have the equation
   
(Ljf ht j)x I
j Jj = (4.53)
(Lf hbj )x 0

where htj contains the rst j rows of h and hbj the remaining ones. From the
denition of the observability indices it follows that (Ljf hbj )x must be linearly
dependent on the elements above it in the matrix forming the left hand side of
(4.52) (otherwise it would be possible to lower one of the i ). However, from
(4.53) it follows that (Ljf hbj )x can not be linearly dependent on (Ljf htj )x . We
then have
(LJf hbj )x span{hx , . . . , (Lj1
f h)x } (4.54)
One can also show the reverse: when this criterion is satised and it is possible
to achieve the condensed Brunovsky form, then it is actually possible to achieve
C1 = I, that is the dual Brunovsky form. Assuming (4.54) to be true we can
proceed with the solution of (4.52). Introduce

0
hx ..
.. =
Qj = . ,
E j . (4.55)
j1
0
(Lf h)x
Ej1

Take a block structure J = (J1 , . . . , J ) which is compatible with the blocks of

the right hand side of (4.52). If we pick out those equations in (4.52) that have
known right hand sides, (i.e. independent of b) and combine them with (4.45)
we get the system

Q J = E (4.56)
Q J = E, Jj Ej = [f, Jj+1 ], j = 1, . . . , 1 (4.57)
j j j

This system can be solved using generalized inverses.

Lemma 4.1 (Rao and Mitra, 1971) The individually consistent matrix equa-
tions DY = E and Y F = H have a common solution if and only if EF = DH.
The general solution is then

Y = D E + HF D DHF + (I D D)Z(I F F ) (4.58)

where D , F are arbitrary generalized inverses (i.e. satisfying DD D = D,

F F F = F ) and Z is an arbitrary matrix of appropriate dimension.

Using this lemma it is easy to show the following result.

Lemma 4.2 Assume rank Q = n. The set of equations (4.56), (4.57) are
solvable if (4.54) is satised and the solution is given by

J = Q
E, Jj = [f, Jj+1 ]EjT + Q j Ej EjT )
j E(I

+ (I Q
j Qj )Zj (I Ei Ei ) (4.59)
T

52
Proof. Some Lie derivative calculations show that the condition EF = DH
of Lemma 4.1 is satised by (4.57). The formula (4.59) for the solution then
follows from that lemma on noting that Ej = (EjT Ej )1 EjT = EjT .

Observer form for multi-output systems with general C1

For a system in the form (4.35) it is possible to get the dual Brunovsky form
by introducing a transformed output y = C11 y. Of course C1 is not known
before the transformation x = X() has been computed. It is however possible
to deduce C1 from the condition that the transformed output equation
y = C11 y = C11 h(x)
should satisfy (4.54). If for instance
(Ljf hbj )x = M (Ljf htj )x + span{hx , . . . , (Lj1
f h)x}

then  
I 0
C11 = (4.60)
M I
removes the dependence on (Ljf htj )x . Working through j = 1, . . . , one can
try to satisfy (4.54) by forming C1 as a product of matrices of the form (4.60).
It turns out that this is possible precisely when
(Ljf hbj )x spanR (Ljf htj )x + span{hx , . . . , (Lj1
f h)x}, j = 0, . . . , 1 (4.61)
where span denotes linear combinations with coecients that are functions of
x and spanR denotes linear combinations with constant coecients. We have
then nally arrived at the general result.

Theorem 4.3 The problem of transforming (4.29) into (4.30) is solvable if and
only if
(a) There exist observability indices satisfying (4.11) with 1 + + p = n such
that Q has rank n.
(b) (4.61) is satised.
(c) The matrix J computed from Lemma 4.2 is integrable.

The solution is then given by X = J.

Proof. (sketch of suciency) Using transformations of the form (4.60) the

problem is reduced to the case C1 = I. Lemma 4.2 then shows that (4.56),
(4.57) can be solved. Some further calculations then show that the starred
elements of (4.52) come out right (so that b can be chosen). If computed J is
integrable, the required transformation x = X() can then be computed.

4.5 Observers based on state and output trans-

formations
Above we discussed the relation between the two Brunovsky forms by intro-
ducing a constant output transformation y = C11 y. Of course this idea can be

53
extended to include general transformations of the outputs of the form y = ( y ).
This gives us additional degrees of freedom and should make it possible to satisfy
the very restrictive integrability conditions of Theorem 4.3 more often. Com-
puting the conditions that should satsfy turns out to be very cumbersome.
To simplify matters somewhat we make an initial state space transformation of
the system. Assuming the matrix Q to be nonsingular, we can introduce the
new variables
(1 1)
z11 = y1 , z12 = y 1 , . . . , z1,1 = y1
.. (4.62)
.
zp1 = yp , zp2 = y p , . . . , zp,p = yp(p 1)
and get a locally invertible coordinate change. In these coordinates the state
variable description becomes
y1 = z11 , . . . , yp = zp1
z11 = z12 , . . . , zp1 = zp2
.. ..
. . (4.63)
z11 = f1 (z) zpp = fp (z)
where fi = Lf i hi .
This form is sometimes referred to as observable form or
observability form. Let be the Jacobian of and dene
yi
ij = (4.64)
yj
The fundamental result is then

Theorem 4.4 Consider a system in observable form (4.63). If it can be trans-

formed into (4.30) with A, C in dual Brunovsky form then has to satisfy
ij = 0, i > j (4.65)
ij 1 
p
2 fi
= kj (4.66)
y i zi zk2
k=1

In particular the quantities

2 fi
zi zk2
have to depend only on the output.

Proof. We consider only the very special case p = 1, n = 2. The general case
involves similar but messier calculations. Comparing
z1 = z2
z2 = f (z1 , z2 )
y = z1
with
1 = b1 (1 ) + 2
2 = b2 (1 )
y = 1

54
shows that
z2 = y = y = (b1 (1 ) + 2 ) (4.67)
Dierentiating this expression with respect to time gives

d  
f (z1 , z2 ) = 1 (1 ) + 2 ) + b1 (1 )(b1 (1 ) + 2 ) + b2 (1 )
y(b
dy

Substituting from (4.67) gives

d 1 2  
f (z1 , z2 ) = z2 + b1 (1 )1 z2 + b2 (1 )
dy

The key observation is now that, since there is a direct relationship between y
and y, 1 depends only on z1 and not on z2 . Dierentiating twice with respect
to z2 then gives
2 f (z1 , z2 ) d 1
=2
z22 dy
which is (4.66) for the case n = 2, p = 1.

We illustrate the procedure with the rocket example.

Example 4.3 Consider a vehicle that is accelerated with a unit force and has
an aerodynamic drag proportional to the square of the velocity. If x is the
position and v is the velocity the system description is

x = v
v = 1 v2
y = x

where it is assumed that the position is measured. Equation (4.66) becomes

d
=
d

with the solution e . Thus

d d
y
= e , or = e
d
y d

showing that
y = e
We now have the task of transforming the system

x = v
v = 1 v2
y = ex

according to the methods of Theorem 4.4. Since

hx = ex (1 0)
(Lf h)x = ex (v 1)

55
we have  
0
J2 = ex
1
Then  
1
[f, J2 ] = ex
v
and [g, [f, g]] = 0 showing that (4.50) is satised. From (4.66) the coordinate
change is  
1 x 1 0
J =e
v 1
We already knew from the relation between the outputs that

z1 = ex

which is the rst row of the above realtion. The second row gives

2
= vex
x
2
= ex
v
with the solution 2 = vex . The coordinate change is thus given by

1 = ex x = ln 1
2 = vex v = 2 /1

In the new variables the system description is then

1 = 2
2 = 1
y = 1

4.6 Observers for systems with input signals

The methods for transforming systems without inputs into observer form can
be extended to systems with inputs in several ways. Let the system dynamics
be described by

x = f (x, u)
(4.68)
y = h(x)

Then we can pick a constant control signal u0 and compute the variable trans-
formations for the system

x = f (x, u0 )
(4.69)
y = h(x)

56
using the technique of the previous section (if possible). The resulting transfor-
mation can then be applied to the system (4.68) to give

= b(C) + A + (, u)
(4.70)
y = C

where (, u0 ) = 0. If it turns out that has the form

(, u) = (C, u) (4.71)

then the observer

x = b(
i y) + A + (
y C )
y, u) + K( (4.72)

will have linear error dynamics.

Example 4.4 Suppose that we add the control signal in Example 4.3 to get

x = v
v = 1 v 2 + u
y = x

= u 1 g. If we apply the varaible transformation

where u

xi1 = ex x = ln 1
2 = vex v = 2 /z1

which was computed for u = 1 + g, the result is

1 = 2
2 = 1 + 1 u

which is of the form (4.70), (4.71).

4.7 Closing the loop

Often observers are used in connection with control. Typically one has a system
to be controlled
x = f (x, u), y = h(x) (4.73)
and designs a controller u = k(x) so that the closed loop system

x = f (x, k(x)) (4.74)

has desirable properties. If the state x can not be measured an observer

= f (
x x, u) + K(y h(
x)) (4.75)

is constructed and the control u = k(

x) is used. The closed loop system then
becomes
x = f (x, k(
x))
(4.76)
x
= f ( x)) + K(h(x) h(
x, k( x))

57
It would be nice if a good design of the state feedback system (4.74) and a
good design of the observer (4.75) would automatically give a good closed loop
design (4.76). For linear systems there are such results, often called separation
theorems, but for nonlinear systems it has turned out to be very dicult to
get such results that are practically useful. This means that it is necessary to
analyze (4.76) directly. One possibility is to use Lyapunov theory which will be
dealt with in the next chapters.

4.8 Exercises.
4.1 Consider the following variant of the aircraft speed dynamics (Example
1.1). Let v be the deviation from the speed of lowest drag and let T be the
thrust of the jet engine. If the control signal u is the desired thrust, then the
following equations hold under certain operating conditions.

T = T + u
v = v 2 + T
y = v

where we have assumed that the speed is measured.

a. Construct an observer based on dierentiation. b. Construct an observer
with linear error dynamics (if possible).

4.2 Consider the system

x 1 = x2
x 2 = x2 x3
x 3 = x2
y1 = x1
y2 = x3

Construct an observer with linear error dynamics (if possible).

4.3 A ship that moves with a constant speed in a straight line is observed with
a radar that measures distance only. Let px and py be the position of the ship
in rectangular coordinates, v its speed through water, and its heading angle.
If y is the radar measurement, the dynamical equations are

p x = v cos
p y = v sin
v = 0
= 0
y = p2x + p2y

a. Construct an observer based on dierentiation. b. Construct an observer

with linear error dynamics (if possible). c. Let there be a second measurement
y2 = . Construct an observer with linear error dynamics (if possible).

58
4.4 The roll dynamics of a certain missile is described by

=
= sin 4 + d
d = d + u

where is the roll angle, the angular velocity, d the aileron angle and u the
control input. The sin-term is caused by aerodynamic vortices. The roll angle
is observed.
a. Construct an observer based on dierentiation. b. Construct an observer
with linear error dynamics (if possible).

4.5 Consider Example 4.4. If the acceleration is a constant unknown, the

system equations are
x = v
v = a v 2
a = 0
y = x
Construct an observer with linear error dynamics ( if possible ).

4.6 Let a system be described in obsevable form by

x = f (x) + g(x)u
y = h(z)

and in observer form by

= b(C) + A + ()u
y = C

If we want an observer with linear error dynamics, then () must be a function

of C = 1 only. What conditions must g(x) then satisfy?

59
Chapter 5

Stability and Lyapunov

theory.

Stability is probably the most important property of a control system. If a

system is unstable , then it becomes meaningless to discuss properties like sta-
tionary errors, sensitivity or disturbance rejection. For nonlinear systems the
denition of stability requires care. For instance it is quite possible that a
system converges to a desired equilibrium after a small disturbance, but breaks
into oscillation after a large one. The best tool for investigating stability related
questions for nonlinear systems is Lyapunov theory. In fact, it is essentially the
only tool.

5.1 Lyapunov functions.

Consider a nonlinear system described by

x = f (x) (5.1)

where x is an n-vector and f is a continuously dierentiable function. From

Chapter 2 it follows that for every starting point there is a unique solution ( at
least on a small time interval ). Let (t, x) denote the solution of (5.1) at time
t ,when the initial value at t = 0 is x, ( i.e. (0, x) = x ).
To study the behavior of solutions it is convenient to introduce a scalar function
which is analogous to the potential of physics.

Denition 5.1 A continuously dierentiable scalar function V is called a Lya-

punov function of (5.1) on G if Vx (x)f (x) 0 for x G. (G is any open subset
of Rn ).

From the denition the following fundamental property of Lyapunov functions

immediately follows.

60
Proposition 5.1 If V is a Lyapunov function on G , then for t 0 and as long
as a trajectory remains in G, one has

V ((t, x)) V (x) (5.2)

Proof.
 t  t
d
V ((t, x)) V (x) = V ((, x))d = Vx ((, x))f ((, x))d 0
0 d 0

Now consider sets of the form

Bd = {x : V (x) d}, where d > 0 (5.3)

What one tries to do is to nd a Lyapunov function such that for some d > 0
the set Bd lies in G. If this is possible one knows immediately that solutions
starting in Bd never leave it.

Proposition 5.2 If V is a Lyapunov function on G and for some d > 0 the set
Bd lies in G, then for all x Bd one has (t, x) Bd , for all t > 0.

Proof. Follows immediately from Proposition 5.1.

With good luck one might nd that Bd is bounded. Then the following stability
result follows.

Proposition 5.3 Let the conditions of Proposition 5.2 hold and assume that
Bd is bounded. Then all solutions of (5.1) starting in Bd exist for all t and are
bounded.

Proof. Follows from the previous proposition and Theorem 2.2.

One way of ensuring that the sets Bd are bounded is to use a function V which
is radially unbounded i.e. satises

V (x) as |x|

If it is possible to use Proposition 5.3 to show that solutions are bounded, then
the next step is to nd out where they go as t goes to innity. This is formalized
by the concept of a limit set

Denition 5.2 Let the conditions of Proposition 5.2 hold and let x be a point
in Bd . A point y in Bd belongs to the limit set (x) if there is a sequence {ti }
such that ti and (ti , x) y. In other words, solutions starting at x come
arbitrarily close to the set (x).

When a suitable Lyapunov function is known, it becomes possible to nd out

where the limit set is, as shown by the following theorem.

61
Theorem 5.1 ( Lyapunov, LaSalle ) Let the assumptions of Proposition 5.3
be satised and dene
E = {x Bd : Vx (x)f (x) = 0}
Let M be the largest set in E with the property that solutions of (5.1) starting
in M remain in M for all t > 0. (Such a set is called an invariant set ). Then
for each x Bd there exists a c such that (x) M V 1 (c).

Proof. Let x0 be an arbitrary point in Bd . V is continuous on the compact

set Bd and hence bounded below on that set. Since it is also decreasing along
solutions of (5.1), (Proposition 5.1 ) , there exists a limit : limt V ((t, x0 )) =
c. Let y be an arbitrary point in (x0 ). Then there exists a sequence {ti },
ti , such that (ti , x0 ) y. Since V is continuous, it follows that V (y) = c.
Hence (x0 ) V 1 (c). From the relationship
(t, y) = lim (t, (ti , x0 )) = lim (t + ti , x0 )
ti ti

it follows that (t, y) (x0 ) for all t > 0. Then c = V ((t, y)) so that
d
0= V ((t, y)) = Vx ((t, y))f ((t, y))
dt
which shows that (x0 ) M .

The following example shows the kind of situation that can occur.

Example 5.1 Let the dierential equation be

x 1 = x1 + x2 x31 x1 x22
(5.4)
x 2 = x1 + x2 x21 x2 x32

Dening r2 = x21 + x22 and the function V (x) = r4 2r2 gives

Vx (x)f (x) = 4r2 (r2 1)2 0
so V is a Lyapunov function everywhere. Also the set Bd is bounded for any
d > 0. Since
M = E = {(0, 0)} {x21 + x22 = 1}
it follows that the limit set is either the origin or the unit circle.

The situation that is of most interest in applications is described by the following

Corollary.

Corollary 5.1 Assume that the conditions of Theorem 5.1 are satised and
that M consists of a single point {xm }. Then all solutions starting in Bd con-
verge to xm , which is necessarily a singular point of (5.1).

Proof. Since (x0 ) {xm } ,it follows that all solutions converge to xm . Since
solutions starting in {xm } remain in {xm }, it follows that f (xm ) = 0.

The corollary can be illustrated by a harmonic oscillator with nonlinear damp-

ing.

62
Example 5.2 Consider the system
x 1 = x2
(5.5)
x 2 = x1 x32
and the function V = x21 + x22 . Dierentiation gives Vx f = 2x42 0 so V is
a Lyapunov function where E is the x1 -axis . Since a solution remaining in E
must satisfy x2 0, it follows that also x1 = 0, so M = {(0, 0)}. Since Bd
is bounded no matter how large d is, all solutions of the dierential equation
approach the origin.

5.2 Non-autonomous systems

Often one encounters nonlinear systems where the right hand side depends ex-
plicitly on the time variable, so called non-autonomous systems:
x = f (t, x) (5.6)
We will look at the stability of equilibrium points for such a system. To simplify
things, we assume that the equilibrium is x = 0. The following denitions are
straightforward extensions of those that are used for time-invariant systems.

Denition 5.3 The system (5.6) with the equilibrium x = 0 is

uniformly stable if for each  > 0 there is a () > 0, independent of to ,

such that
|x(to )| < () |x(t)| < , all t to
uniformly asymptotically stable if it is uniformly stable, and there exists a
o > 0 such that
|x(to )| < o |x(t)| 0, t
and the convergence is uniform in to .
globally uniformly asymptotically stable if the denition above holds for
any o > 0.

Stability for non-autonomous systems can be proved using Lyapunov functions.

Since the dierential equation is time-varying it is however natural to allow time-
varying Lyapunov functions. To handle the time-variability we will assume that
the Lyapunov function is shut in between two positive denite functions. A
function W is called positive denite if it satises W (0) = 0, W (x) > 0 for
x = 0.

Theorem 5.2 Let x = 0 be an equilibrium of (5.6). Assume that there is an

open connected set around the origin where the following relations hold for all
t:
W1 (x) V (t, x) W2 (x) (5.7)
d
V (t, x) = Vt + Vx f (t, x) W3 (x) (5.8)
dt

63
Here V is a continuously dierentiable function, and the Wi are continuous
positive denite functions. Then x = 0 is uniformly asymptotically stable.

Proof. Let the set where the conditions of the theorem hold be . Dene
r = {x : |x| r} and take an r such that r . Take a such that
< min W1 (x)
|x|=r

and dene the sets

V = {x r : V (t, x) }, i = {x r : Wi (, x) }, i = 1, 2
Note that V is time dependent. From the construction it follows that

2 V 1 r
Now assume x(to ) 2 . Then x(to ) V and consequently V (to , x(to )) .
Since V has a negative derivative, it is decreasing and consequently V (t, x(t))
holds for all t. It follows that x(t) 1 r for all t to . Since this holds
for arbitrarily (small enough) r > 0, uniform stability is clear. To show that
solutions actually converge to the origin, we use the fact that any continuous
positive denite function W (x) can be bounded from below and above as

m (|x|) W (x) M (|x|), x r

where m , M are strictly increasing functions with m (0) = M (0) = 0. We
can thus write
1 (|x|) V (t, x) 2 (|x|), V 3 (|x|)

where the i are strictly increasing functions with i (0) = 0. We now get
V 3 (|x|) 3 (1
2 (V ))

V is thus bounded above by the solution to the dierential equation z =

3 (12 (z)) whose solution must converge to the origin. It follows that V
decreases to zero. From 1 (|x|) V (t, x) it follows that x converges to the
origin.

There is a corollary about global stability.

Corollary 5.2 Let the assumptions of Theorem 5.2 be satised for all x and
assume that W1 is radially unbounded. Then x = 0 is globally uniformly asymp-
totically stable.

Proof. If W1 is radially unbounded so is W2 . It follows that r in the proof can

be taken arbitrarily large.

5.3 Construction of Lyapunov functions

In most situations one has a system with a known critical point, i.e. the point
xm of Corollary 5.1 is known, and the problem is to nd a Lyapunov function V

64
which proves that solutions of (5.1) converge to xm . To get an idea of the prob-
lems involved, it is instructive to investigate how this might be done for linear
systems. To simplify the notation we assume that xm is the origin. Consider
the n-th order linear system
x = Ax (5.9)
where A has all its eigenvalues strictly in the left half plane and try to nd a pos-
itive denite Lyapunov function, which is a degree m homogeneous polynomial
in x. We can then write it in the form

V (x) = vi xk1i1 xk2i2 xknin (5.10)

where the sum is taken over all kij such that ki1 + ki2 + + kin = m. Now let
W (x) be a given positive denite homogeneous polynomial of degree m. Then
it is natural to try to solve

Vx (x)Ax = W (x) (5.11)

If (5.11) is solved, then the conditions of Corollary 5.1 are automatically satised
with xm = 0 and Bd can be made as large as desired.

Theorem 5.3 ( Lyapunov ) If A has all its eigenvalues strictly in the left half
plane, then (5.11) can always be solved with respect to V .

Proof. Vx consists of polynomials of degree m 1, since the dierentiation

lowers the degree by one. Multiplying by Ax gives a degree m homogeneous
polynomial again. By identifying the coecients of the monomials in the left
and right hand sides of (5.11) we get a set of linear equations for the coecients
of V . By using e. g. Kronecker products one can show that this system is
nonsingular whenever A has all its eigenvalues strictly in the left half plane.

Remark 5.1 When V and W are quadratic they are usually written in the
form
V (x) = xT Sx, W (x) = xT Qx
and equation (5.11) takes the form

AT S + SA = Q

called Lyapunovs equation.

Example 5.3 Let the system be given by

 
1 1
x =
0 2

and let the desired W be x41 + x42 . Writing V in the form

V (x) = v1 x41 + v2 x31 x2 + v3 x21 x22 + v4 x1 x32 + v5 x42

the left hand side of (5.11) becomes

4v1 x41 + (4v1 5v2 )x31 x2 + 3(v2 2v3 )x21 x22 + (2v3 7v4 )x1 x32 + (v4 8v5 )x42

65
An identication of coecients gives

v1 = 1/4, v2 = 1/5, v3 = 1/10, v4 = 2/70, v5 = 9/70

Now consider a general system

x = f (x)

Assume that f is given by a series expansion

f (x) = f (1) (x) + f (2) (x) + f (3) (x) +

where f (k) (x) is a homogeneous polynomial of degree k. Consider the problem

of nding a Lyapunov function V such that

Vx (x)f (x) = W (x) (5.12)

where W is a given positive denite function. Assume that V and W can be

written as expansions

V (x) = V (2) (x) + V (3) (x) +

W (x) = W (2) (x) + W (3) (x) +

Substituting into (5.12) and identifying coecients gives
(2)
Vx (x)f (1) (x) = W (2) (x)
(3)
Vx (x)f (1) (x) = W (3) (x) V (2) (x)f (2) (x)
(4) (5.13)
Vx (x)f (1) (x) = W (2) (x) V (3) (x)f (2) (x) V (2) (x)f (3) (x)
..
.

Here the linear part, f (1) (x) can be written in the more familiar form

f (1) (x) = Ax (5.14)

where A is an n by n matrix. An examination of this system of equations leads

to the following result.

Theorem 5.4 If f and W are given real analytic functions and if A of (5.14) has
all its eigenvalues strictly in the left half plane, then it is possible to nd a real
analytic function V which solves (5.12). The coecients of a series expansion
for V can be computed from (5.13).

Proof. Since A has all its eigenvalues strictly in the left half plane, we can use
Theorem 5.3, which tells us that any equation in (5.13) can be solved ( if its
right hand side is known ). One can then solve the rst equation of (5.13) to get
V (2) . Substituting V (2) into the right hand side of the second equation, one can
solve for V (3) . If V (2) and V (3) are substituted into the third equation, V (4) can
be computed, and so on. The resulting series can be shown to be convergent in
some neighborhood of the origin, so that V is well dened there.

66
5.4 Lyapunov functions and frequency domain
criteria.
A very common structure for nonlinear systems is the one given in Figure 5.1.
The system can be separated into a linear block with transfer function G(s) and

u y
G(s)

Figure 5.1: Nonlinear system separated into a linear and a nonlinear part.

a nonlinear one, arranged in a feedback loop. We will consider the case of a

static but possibly time-varying nonlinearity

u = f (t, y) (5.15)

The stability of such a conguration is a classical problem in control and its

study goes back at least to the second world war and work by Lure. We assume
that the linear part has m inputs and m outputs and that it has a minimal
n-dimensional realization

x = Ax + Bu, y = Cx, G(s) = C(sI A)1 B (5.16)

Lures work was inspired by applications where the nonlinearity was not pre-
cisely known, so he only assumed that it satised certain inequalities. We will
assume that the inequality is of the form

f (t, y)T (f (t, y) Ky) 0, all t, all y (5.17)

where K is a positive denite matrix. In the scalar case, m = 1, this inequality

has a simple interpretation, shown in gure 5.2. The nonlinear function is bound
by the horizontal axis and a line with slope K.
We will try to construct a quadratic Lyapunov function

V (x) = xT P x (5.18)

where P is a positive denite matrix. Computing the time derivative we get

V = (Ax Bf )T P x + xT P (Ax Bf ) (5.19)

Subtracting the negative quantity given by the left hand side of (5.17) we get

V xT (AP + P A)x 2xT P Bf 2f T (f Ky) (5.20)

67
 f ((t, y)
Ky

Figure 5.2: Bounds on the nonlinearity in the scalar case.

Completing the square we get

V xT (AP + P A + LT L)x ( 2f Lx)T ( 2f Lx) (5.21)

where L = (1/ 2)(KC B T P ). Suppose now that we can nd a P such that

AT P + P A = LT L P (5.22)

P B = C T K 2LT (5.23)

Then we get
V xT P x ( 2f Lx)T ( 2f Lx) (5.24)
which is negative denite and we can use Theorem 5.2 to prove stability. The
solvability of (5.22),(5.23) is characterized by a classical result.

Lemma 5.1 The Kalman-Yakubovich-Popov lemma. Assume that A has

eigenvalues with strictly negative real parts and that A, C is observable, A, B
controllable. Then the matrix equations

AT P + P A = LT L P
P B = C T LT W
W T W = D + DT

can be solved for L, W , P > 0,  > 0 if and only if the transfer function

C(sI A)1 B + D

is strictly positive real.

Proof. See the Appendix.

In this context we use the following denition of positive real.

Denition 5.4 A square rational transfer function matrix G is called positive

real if

all elements of G(s) are analytic for Re s > 0,

68
 any pure imaginary pole is a simple pole with positive semidenite residue
matrix,

for all real the matrix G(i) + GT (i) is positive semidenite.

G(s) is called strictly positive real if G(s ) is positive real for some  > 0.

It follows that an input-output stable SISO system is positive real when its
Nyquist curve lies in the closed right half plane.
We are now ready to state the basic stability result.

Theorem 5.5 Let the system (5.16) have all its poles strictly in the left half
plane and let the nonlinear feedback (5.15) satisfy the inequality (5.17). Then
the closed loop system is globally asymptotically stable if I + KG(s) is strictly
positive real.

Proof. If I + KG(s) is strictly positive real it follows from the Kalman-

Yakubovich-Popov lemma (with D = I and C replaced by KC) that (5.22),
(5.23) is solvable. V = xT P x is then a Lyapunov function satisfying the condi-
tions of Theorem 5.2.

The theorem can be reformulated using a standard trick called pole shifting, see
Figure 5.3. Adding and subtracting the linear block K1 does not alter the loop.

+
G(s)
-

K1

K1

+
f

Figure 5.3: Pole shifting by adding and subtracting a linear block.

However, the new diagram can be interpreted in a dierent way, as a linear

system
G = G(I + K1 G)1

with the nonlinear feedback

f(t, y) = f (t, y) K1 y

69
Suppose the nonlinearity satises the condition

(f (t, y) K1 y)T (f (t, y) K2 y) 0, all t, all y (5.25)

then the modied nonlinearity satises

f(t, y)(f(t, y) (K2 K1 )y) 0

and we get directly the corollary.

Corollary 5.3 Let the system (5.16) have a nonlinear feedback (5.15) satisfying
the inequality (5.25), with K2 K1 positive denite. Then the closed loop system
is globally uniformly asymptotically stable if

= G(I + K1 G)1
G

has all its poles strictly in the left half plane and

(I + K2 G(s)(I + K1 G(s))1

is strictly positive real.

Proof. As shown above the nonlinearity f satises (5.17) with K = K2 K1 .

From Theorem 5.5 we have to check the strict positive realness of

I + (K2 K1 )G(s) = I + (K2 K1 )G(s)(I + K1 G(s))1 =
= (I + K2 G(s))(I + K1 G(s))1

For the scalar case m = 1 this corollary can be formulated geometrically in a

well-known way.

Corollary 5.4 The Circle Criterion. Let u and y of the linear system (5.16)
be scalars. Assume that G has no poles in the right half plane and let the
nonlinearity satisfy

k1 y 2 yf (t, y) k2 y 2 , all t, all y (5.26)

Then a sucient condition for the closed loop system to be globally asymptot-
ically stable is that the Nyquist curve of G does not encircle or enter the circle
whose diameter lies on the real axis between 1/k1 and 1/k2 .

The geometric interpretation of the circle criterion is given in Figure 5.4.

Proof. The condition k1 y 2 yf (t, y) k2 y 2 can also be rewritten (f k1 y)(f
k2 y) 0. The stability condition is from Corollary 5.3 that G/(1 + k1 G) has
all its poles in the left half plane, and that

1 + k2 G
1 + k1 G

70
 f (y) k2 y
k11 k12
k1 y

G(i)

Figure 5.4: The circle criterion. Bounds on the nonlinearity to the left and
bounds on the Nyquist curve to the right.

is positive real. Since the rational function

1 + k2 z
1 + k1 z
maps the interior of the circle going through 1/k1 , 1/k2 onto the left half
plane, the positive realness condition is equivalent to the condition that the
Nyquist curve does not enter the circle. The stability of G/(1 + k1 G) follows if
the Nyquist curve does not enclose the circle.

There is a variation on the results above that uses a slightly dierent Lyapunov
function for the time-invariant case, namely
 y
V = xT P x + 2 f T ()K d (5.27)
0

where 0 is a free parameter. It is assumed that f T K is the gradient of a

scalar function so that the integral is path-independent (this is automatically
satised in the scalar case). To get a reasonable V , it is assumed that
 y
f T ()K d 0, all y (5.28)
0

The nonlinearity is assumed to satisfy

f T (y)(f (y) Ky) 0, all y (5.29)

Dierentiating (5.27) gives

V = xT (AT P + P A)x 2xT P Bf + 2f T KC(Ax Bf )

Adding the nonnegative quantity 2f T (f Ky) gives

V xT (AT P + P A)x f T (2I + KCB + B T C T K)f +

+ 2xT (P B + AT C T K + C T K)f

If we factorize according to

2I + KCB + B T C T K = W T W (5.30)

71
then we can complete the squares to get
V xT (AT P + P A + LT L)x (W f Lx)T (W f Lx)
provided L satises
P B = C T K + AT C T K LT W (5.31)
If we nally can satisfy
AT P + P A = LT L P (5.32)
for some  > 0, then we have
V xT P x (5.33)
The equations (5.32), (5.31), (5.30) are of the form occuring in the Kalman-
Yakubovich-Popov lemma and we get the following result.

Theorem 5.6 The multivariable Popov criterion. Let the system (5.16)
have all its poles strictly in the left half plane and let the nonlinear time invariant
feedback u = f (y) satisfy the inequality (5.29). Also assume that f T K is the
gradient of a scalar function and that (5.28) is satised. Then the closed loop
system is globally asymptotically stable if there is an 0 such that 1/ is
not an eigenvalue of A and
I + (1 + s)KG(s)
is strictly positive real.

Proof. Applying the Kalman-Yakubovich-Popov lemma with C T replaced by

C T K + AT C T K and D = I + KCB shows that we have to test for positive
realness of

I + KCB + (KC + KCA)(sI A)1 B =

I + KC(I + (sI A)1 + A(sI A)1 )B =
I + KC((sI A) + I + A)(sI A)1 B =
I + (1 + s)KG(s)
The condition that 1/ is not an eigenvalue of A guarantees that the pair
C +CA, A is observable if C, A is (we have assumed in the Kalman-Yakubovich-
Popov lemma that the system is controllable and observable).

There is a simple geometric interpretation in the SISO case.

Corollary 5.5 The classical Popov criterion. Let the conditions of The-
orem 5.6 be satised for a single-input-single-output system. Then a sucient
condition for global asymptotic stability is that
1
+ Re G(i) Im G(i) > 0, all (5.34)
K
Graphically this means that the so called Popov plot (ImG(i) versus ReG(i))
has to lie to the right of a line through 1/K with slope 1/ for some value of
, see Figure 5.5.

72
Proof. In the scalar case the positive deniteness condition is that
Re (1 + (1 + i)KG(i)) > 0
Taking the real part of the expression within parentheses gives the result.

Im G(i)

1/K
Re G(i)

slope 1/

Figure 5.5: The classical Popov criterion. If the Popov plot lies to the right of
the line, global asymptotic stability is guaranteed.

Example 5.4 Consider the linear system

2
G(s) =
(s + 1)(s + 2)
in feedback with a saturation nonlinearity


Ky |y| 1/K
f (y) = 1 y > 1/K


1 y < 1/K
In the left part of Figure 5.6 the Nyquist curve is shown together with a vertical
line through 1/9. It is clear that K 9 is the highest value for which the
circle criterion guarantees stability in this case. In the right hand part of the
gure the Popov curve is shown together with a line through 1/100, so the
Popov criterion shows stability for K = 100. In fact it is clear that the line
can intersect the real axis arbitrarily close to the origin, so the Popov criterion
proves stability for any K > 0 (which is the stability result what one intuitively
would guess).

5.5 Lyapunov functions and input output sta-

bility
Now consider a dynamic system
x = f (x) + g(x)u, y = h(x) (5.35)
We make the following assumptions
f (0) = 0, h(0) = 0, |g(x)| c1 , |h(x)| c2 |x| (5.36)

73
 0 0

0.1 0.1

0.2 0.2

0.3 0.3

0.4 0.4

0.5 0.5

0.6 0.6

0.7 0.7
0.2 0 0.2 0.4 0.6 0.8 1 1.2 0.2 0 0.2 0.4 0.6 0.8 1 1.2

Figure 5.6: Comparison of circle and Popov criterion.

So far we have only studied the uncontrolled system

x = f (x) (5.37)

One might wonder if asymptotic stability of (5.37) also implies a nice input
output behavior for (5.35). For linear systems it is well known that such a
relationship exists. We will prove a similar result for the nonlinear case. To
do that we assume that we have found a positive denite Lyapunov function V
such that
V (0) = 0, Vx (x)f (x) c3 |x|2 , |Vx (x)| c4 |x| (5.38)
Then we can show the following

Theorem 5.7 Suppose the system (5.35) satises (5.36) and that a Lyapunov
function exists with the properties (5.38). Then there exists a positive constant
k and a function (x) with (0) = 0, such that for any T > 0 and any input u
 T  T
2
y (t)y(t)dt k
T
uT (t)u(t)dt + (x0 ) (5.39)
0 0

where x0 is the initial state.

Proof. Using (5.36) and (5.38) we get

 T  T
0 V (x(T )) = V (x(t)) dt + V (x0 ) = Vx (x(t)) (f (x(t)) + g(x(t))u(t)) dt
0 0

 T
+V (x0 ) (c3 |x(t)|2 + c1 c4 |x(t)||u(t)| dt + V (x0 ) =
0
    2
c3 T
2
T
c3 c1 c4 c21 c24 2
|x(t)| dt |x(t)| |u(t)| + |u(t)| dt + V (x0 )
2 0 0 2 c3 2c3

showing that
 
c3 T
c21 c24 T
|x(t)|2 dt |u(t)|2 dt + V (x0 )
2 0 2c3 0

74
Since
|y(t)| c2 |x(t)|
from (5.36), it follows that (5.39) is satised with

c1 c2 c4 2c22
k= , (x0 ) =
c3 c3 V (x0 )

Remark 5.2 The inequality ((5.39) can often be given an energy interpreta-
tion: the energy in the output is less than some multiple of the energy in the
input plus the stored energy.

Example 5.5 Consider the mechanical system

x2
-
m - u
e e

x1
-
A mass m moving with velocity x2 , is pulled by an external force u. The mass
is connected to a nonlinear spring giving the force

F = k1 x1 + k2 x31

and to a linear damper giving the force

F = bx2

We consider the velocity x2 to be the output. The equations of motion are then

x 1 = x2
mx 2 = u k1 x1 k2 x31 bx2 (5.40)
y = x2

Now consider an energy balance for the time interval 0 t T . We have

energy delivered to the system:
 T
uydt
0

increase in kinetic energy:

m 
x2 (T )2 x2 (0)2
2
increase in potential energy of the spring:
 T
k1 k2 k1 k2
x 1 (k1 x1 + k2 x31 )dt = x1 (T )2 + x1 (T )4 x1 (0)2 x1 (0)4
0 2 4 2 4

75
energy lost in damper:
 T
b y 2 dt
0
Summing the various contributions we get
 T
 
by 2 + uy dt + (x(0)) = (x(T )) (5.41)
0

where
k1 2 k2 4 m 2
(x) =
x + x1 + x2
2 1 4 2
Since is a nonnegative function, we can also write
 T
 
by 2 + uy dt + (x(0)) 0 (5.42)
0

Note that the function , representing the energy stored in the system, is also
a Lyapunov function for the system with u = 0, since

= (x)x x = bx22 0

Motivated by this example one can consider general systems with an equilibrium
at the origin

x = f (x, u), y = h(x, u), x(0) = x0 , f (0, 0) = 0, h(0, 0) = 0 (5.43)

and dene a supply rate

w(u, y) = y T Qy + 2y T Su + uT Ru (5.44)

which is a quadratic form in the input and output. Motivated by (5.42) we

make the following denition.

Denition 5.5 A system (5.43) is called dissipative if there exists a function

w of the form (5.44) and a function , (0) = 0 such that
 T
w(u(t), y(t)) dt + (x0 ) 0 (5.45)
0

for all T 0 and all control signals u, where y is the output of (5.43) for the
initial condition x(0) = x0 .

We see that equation (5.39) implies that the system is dissipative (w = y T y +

k 2 uT u). This case is much studied in the control literature and motivates the
following denition.

Denition 5.6 A dissipative system with

w(u, y) = y T y + k 2 uT u

is said to have nite gain.

76
For an electric circuit where the u is the voltage and y the current (or vice
versa), the product uy is the supplied power. This has motivated the following
denition.

Denition 5.7 A dissipative system is called passive if w has the form

w(u, y) = uT y

and strictly passive if w is of the form

w(u, y) = uT y uT u

for some  > 0.

Let us now dene the storage function V as

 T
V (x0 ) = inf w(u(t), y(t)) dt (5.46)
u,T 0

where y is the solution of (5.43) with x(0) = x0 . The inmum is taken over
all inputs u and all times T 0. We now have the following generalization of
(5.42).

Proposition 5.4 Assume that (5.45) holds for a system (5.43). Then the stor-
age function V dened by (5.46) satises

0 V (x) (x), V (0) = 0

and  T
V (x0 ) + w(u(t), y(t)) dt V (xT ), T 0 (5.47)
0
where y is the output corresponding to u with the initial condition x0 and xT
is the state reached at t = T .

Proof. Since T = 0 is allowed in (5.46), it is clear that V 0. From (5.45) it

follows that V . Now for any choice of u, T 0, T1 T we have
 T  T1
V (x0 ) w(u, y)dt w(u, y)dt
0 T

The inequality is then also true if the last integral is replaced by its inmum,
giving
 T
V (x0 ) w(u, y)dt + V (xT )
0

An immediate consequence of the proposition is that storage functions and

Lyapunov functions are closely related.

Proposition 5.5 Consider a dissipative system where Q 0. Then the storage

function V (5.46) is a Lyapunov function for the uncontrolled system (u = 0).

77
Proof. Setting u = 0 in (5.47) gives
 T
V (xT ) V (x0 ) + y T Qy dt
0

which shows that V is decreasing along trajectories.

A problem is that the denition (5.46) does not give any guarantee that V is
a smooth function. Suppose however that V is a continuously dierentiable
function. Then we can dierentiate (5.47) to get

V = Vx (c)f (x, u) w(y, u) (5.48)

Let us specialize to systems of the form

x = f (x) + g(x)u, y = h(x), f (0) = 0, h(0) = 0 (5.49)

Equation (5.48) then becomes, for the case S = 0

Vx f Vx gu + hT Qh + uT Ru (5.50)

If R > 0 we can complete the square to get

1 1
Vx f (u R1 g T VxT )T R(u R1 g T VxT )+
2 2
1
+hT Qh Vx gR1 g T VxT (5.51)
4
From this inequality we get

Theorem 5.8 Consider a system (5.49) which is dissipative with S = 0, R > 0.

Assume that the storage function V of (5.46) is continuously dierentiable. Then
it satises the Hamilton-Jacobi inequality.
1
Vx f + Vx gR1 g T VxT hT Qh 0, V (0) = 0 (5.52)
4

Proof. Since (5.51) must be satised for any u, it is in particular true for
u = 12 R1 g T VxT , which gives (5.52).

There is a converse theorem.

Theorem 5.9 Consider a system of the form (5.49). Suppose the Hamilton-
Jacobi inequality (5.52) has a dierentiable, nonnegative solution for matrices
Q, R, R > 0. The the system is dissipative with supply rate

y T Qy + uT Ru

Proof. Since R > 0 we can add a term

1 1
(u R1 g T VxT )T R(u R1 g T VxT )
2 2
to the right hand side of (5.52) to get (5.51). Going through (5.50), (5.48) in
reverse we get (5.47), from which (5.45) follows.

78
5.6 Exercises
5.1 Rotating machinery is often described by the equation

x 1 = x2 (5.53)
x 2 = d x2 f (x1 ) (5.54)

where x1 is the angle of rotation and x2 is the angular velocity. The constant
d > 0 represents some type of viscous friction or damping and f a restoring
moment, f (0) = 0. Show that
 x1
1
V (x) = x22 + f (u) du
2 0

is a Lyapunov function and discuss under what conditions the state variables
converge to the origin.

5.2 Consider the system described in the previous exercise. For an electric
generator f has the form
f (x) = sin x
Compute the Lyapunov function and sketch the area of the phase plane where
solutions will converge to the origin.

5.3 Use the function

x21 + x22 x1 x32
V (x) =
2(1 x1 x2 )
to estimate the domain of attraction of the origin for

x 1 = x1 + 2x21 x2 (5.55)
x 2 = x2 (5.56)

5.4 Let f be a decoupled nonlinearity

f1 (t, y1 )
..
f (t, y) = .
fm (t, ym )

where each component satises a sector condition of the form shown in the left
part of Figure 5.4, i. e.
i yi2 yi fi (t, yi ) ki yi2
Rewrite these conditions in the form (5.25).

5.5 Suppose the nonlinearity f satises the condition

|f (t, y) Ly| |y|

for all t and y, where the vertical bars denote the Euclidean norm. Rewrite this
condition in the form (5.25).

79
5.6 A DC motor with axis resonances has the transfer function
4
G(s) =
s(s + 1)(s2 + 0.1s + 4)

It is controlled by a saturated P-controller. For what gain of the P-controller

can stability be guaranteed?

5.7 The system

10
G(s) =
(s + 1)(s + 2)(s + 10)
is controlled by a P-controller with dead-zone:


K(y 1) y > 1
u= 0 |y| 1


K(y + 1) y < 1

For what K-values can one guarantee stability?

5.8 The system  2 3 

G(s) = s+1 s+2
1 1
s+1 s+1

is controlled using
u1 = f1 (y1 ), u2 = f2 (y2 )
where
0 y1 f1 (y1 ) k1 y12 , 0 y2 f2 (y2 ) k2 y22
For what values of k1 and k2 can stability be guaranteed?

5.9 What is the Hamilton-Jacobi inequality for a linear system if one assumes
a quadratic function V ?

80
5.7 Appendix
Proof. The kalman-Yakubovich-Popov lemma. We will only give an
outline of the proof.
Positive realness implies solvability of the matrix equations. Suppose
that G(s) is strictly positive real. Then there is some  > 0 such that G(s /2)
is positive real. Since A has all its poles strictly in the left half plane, it is
possible to choose  so that A
= A + 2
I also has all its poles there.
Step 1. Factorize

G(s /2) + GT (s /2) = V T (s)V (s) (5.57)

where V is a transfer matrix with all its poles in the left half plane. For scalar
transfer functions it is fairly easy to see that this can be done, since the left hand
side is symmetric with respect to sign changes in s. Poles and zeros then are
symmetricly placed with respect to the imaginary axis and V (s), V (s) can be
formed by picking poles and zeros from the correct half plane. The multivariable
case is due to a classical result by Youla.
Step 2. Make a state space realization of the left hand side of (5.57) using the
realization of G. This gives
   
A
0 B  
x = x+ u, y = C B T x + (D + DT )u (5.58)
0 AT
CT

by parallell connection.
Step 3. Make a state space realization of the right hand side of (5.57) using a
minimal realization of V , with matrices F , G, H, J. Since V T (s)V (s) can be
regarded as a series connection of this system and one with matrices F T , H T ,
GT , J T , we get
   
F 0 G  
z = z + u, y = J T H GT z + J T Ju (5.59)
H H F
T T T
H J

Step 4. Make (5.59) into block diagonal form by using the transformation
 
I 0
z= z
K I

where K satises
KF + F T K + H T H = 0 (5.60)
This gives
   
F 0 G  
z = z+ u, y = J T H + GT K GT z + J T Ju
0 F T H T J + KG
(5.61)
Step 5. Show that (5.58) and (5.61) are minimal realizations of the left and
right hand sides of (5.57). This is an exercise in the PBH test.

81
Step 6. Since (5.58) and (5.61) are both minimal realizations of the same
transfer function there is a transformation
 
T T12
z = 11 x
T21 T22

between them. Consider T12 . Transforming between (5.58) and (5.61) shows
that
F T12 + T12 AT
= 0
T
Multiplying with eF t from the left and eA t from the right shows that
d  Ft T

e T12 eA t = 0
dt
which gives
T
T12 = eF t T12 eA t
Since the right hand side goes to zero, it follows that T12 = 0. In a similar
manner we can show that T21 = 0. The transformation is thus block diagonal.
Step 7. Writing out the tranformation between (5.58) and (5.61) gives

J T J = D + DT
1
F = T11 A
T11
G = T11 B
1
J T H + GT K = CT11

Step 8. Take

W =J
L = HT11
T
P = T11 KT11

Plugging these values into the equations of Step 7 and using (5.60) shows that
the matrix equations of the Kalman-Yakubovich-Popov lemma are satised.
Solvability of the matrix equations implies positive realness.
This is a straightforward but tedious calculation. Since it is not needed for the
circle and Popov theorems, we omit it.

82
.

83
Chapter 6

Lyapunov based controller

design

Since stability is such a central issue for control, it is natural that many controller
design methods use Lyapunov theory.

6.1 Control Lyapunov functions

A Lyapunov function is dened for a nonlinear system x = f (x) where the
right hand side is given. However, when doing control design the system is
x = f (x, u) where the control u is to be chosen. It is then natural to dene
a control Lyapunov function V to be a positive denite, radially unbounded
function such that

x = 0 Vx (x)f (x, u) < 0, for some u (6.1)

If a control Lyapunov function exists, then it is natural to try to pick a feed-

back control u = k(x) such that Vx (x)f (x, k(x)) < 0, which would guarantee
asymptotic stability. It is clear that such a choice can be made, but it is not
clear that it can be done without discontinuities in k(x). In particular, if the
system has the form
x = f (x) + g(x)u (6.2)

then V being a control Lyapunov function implies that

Vx g = 0 Vx f < 0 (6.3)

It is then natural to use a control law of the form

u = (x)(Vx (x)g(x))T (6.4)

where is some positive scalar function.

84
Open loop stable systems

Consider a system of the form (6.2) which is open loop stable but not necessarily
asymptotically stable. Suppose we know a function V such that
Vx f (x) 0, all x
If we have Vx g = 0 whenever Vx f = 0 for x = 0, then we have a control
Lyapunov function. We could choose (x) = 1 in (6.4) to achieve a smooth
control law which gives a negative denite V .

6.2 Backstepping
Backstepping is a method to extend a Lyapunov function and an associated
control law from part of the system to all the system by moving backwards
through the system. Typically one starts with a system
x = f (x) + g(x)z (6.5)
for which a control law z = k(x) and a Lyapunov function V are known, that is
V = Vx (f (x) + g(x)k(x)) = W (x) (6.6)
where W is some positive denite function. Now suppose this system is really
part of a larger system, so that z is not directly available but instead the system
description is
x = f (x) + g(x)z (6.7)
z = a(x, z) + b(x, z)u (6.8)
At this stage we assume for simplicity that z is a scalar. Rewriting the rst
equation gives
x = f (x) + g(x)k(x) + g(x)(z k(x))
z = a(x, z) + b(x, z)u
Let us try to extend the given Lyapunov function by writing
1
Ve (x, z) = V (x) + (z k(x))2 (6.9)
2
The time derivative then becomes (omitting arguments)

V e = Vx (f + gk) + Vx g (z k) + (z k)(a + bu kx (f + gz)) =

W + (Vx g + a + bu kx (f + gz))(z k)
We see that we can get
V e = W (z k)2
which is negative denite, if u is chosen as
1
u= (kx (f + gz) a Vx g (z k)) (6.10)
b
We have thus dened a control law for the extended system such that Ve becomes
a Lyapunov function.

85
Example 6.1 Consider a version of the mechanical system described in Exam-
ple 6.5 where there is no damping and the spring force is x31 . The system is
then described by

x 1 = x2
x 2 = x31 + u

if we neglect the disturbance w. By using the feedback

u = x1 x2

we get the same system as we used in Example 6.5. We thus know the Lyapunov
function
3 x4
V = x21 + x1 x2 + x22 + 1 ,
2 2
for that closed loop system. Now suppose that we apply the force through an
actuator that has a certain time constant so that we have

x 1 = x2
x 2 = x31 + z
z = z + u

Using (6.10) we get

u = (1 + )x1 (3 + )x2 z + x31 (6.11)

In Figure 6.1 a simulation is shown for the case = 1. As a comparison the

response is also shown when only the linear part of the control law is used, i.e.

u = (1 + )x1 (3 + )x2 z (6.12)

It is clear that the nonlinear term is needed to get a good damping.

5 450

4 400

3 350

2 300

1 250

0 200

1 150

2 100

3 50

4 0
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10

Figure 6.1: Response of mechanical system with backstepping controller (solid)

compared to linear controller (dashed). The left diagram shows x1 the right one
shows the Lyapunov function.

86
The backstepping approach is easily extended to vector valued inputs. Suppose
z and u in (6.7), (6.8) are vectors of length m. Then it is natural to replace
(6.9) by
1
Ve (x, z) = V (x) + (z k(x))T (z k(x)) (6.13)
2
and we get
V e = W (z k)T (z k)
by taking
u = b1 (kx (f + gz) a (Vx g)T (z k)) (6.14)

It is also possible to handle the system structure

x = f (x, z)
z = a(x, z) + b(x, z)u

provided the system x = f (x, u) has a stabilizing control law u = k(x) and a
Lyapunov function V (x) with

Vx (x)f (x, k(x)) = W (x), W positive denite

The idea is still to take the extended Lyapunov function

1
Ve (x, z) = V (x) + (z k(x))2
2
and choose u to give
V e = W (x) (z k(x))2
The formulas become messier because the expression for u will contain a division
with the factor (z k(x) which can not be explicitly cancelled.

Example 6.2 Consider the system

x 1 = arctan x2
x 2 = u

Take to begin with the system

x 1 = arctan u
1 2
It can be stabilized with the control u = x1 , as shown by taking V = 2 x1 ,
giving
V = x1 arctan x1
The extended Lyapunov function then becomes
1 2 1
Ve = x + (x1 + x2 )2
2 1 2
with
V e = x1 arctan x2 + (x1 + x2 )(u + arctan x2 )
We get
V e = x1 arctan x1 (x1 + x2 )2

87
by taking
x1 (arctan x1 + arctan x2 )
u = arctan x2 (x1 + x2 )
x1 + x2

Of course the backstepping approach can be used repeatedly. In this way sys-
tems with the structure

x = f (x, z1 )
z1 = a1 (x, z1 ) + b1 (x, z1 )z2
z2 = a2 (x, z1 , z2 ) + b2 (x, z1 , z2 )z3
..
.
zn = an (x, z1 , . . . , zn ) + b1 (x, z1 , . . . , zn )u

can be handled, provided a control law and corresponding Lyapunov function

is known for the system x = f (x, u).

6.3 Forwarding
There is also a method for extension of a Lyapunov function forwards, across a
nonlinear integrator. The system structure is assumed to be

z = a(x) + b(x)u (6.15)

x = f (x) + g(x)u (6.16)

There is assumed to be an equilibrium at the origin so that a(0) = 0, f (0) = 0.

We assume that we have a positive denite and radially unbounded function V ,
together with a control law k(x), such that

V = Vx (x)(f (x) + g(x)k(x)) (6.17)

is negative denite. We also assume that k is such that the asymptotic conver-
gence is actually exponential. We consider the system

z = a(x) + b(x)k(x) (6.18)

x = f (x) + g(x)k(x) (6.19)

Dene h(x) = a(x) + b(x)k(x) and let x = (t, xo ) be the solution of (6.19) with
initial condition xo . Then z is given by
 t
z(t) = z(0) + h((s, x(0))) ds
0

Since (t, x(0)) converges to zero exponentially, the integral converges. We can
thus dene the variable

=z+ h((s, x)) ds (6.20)
0

88
Using x and as state variables, (6.18), (6.19) can be written

= 0 (6.21)
x = f (x) + g(x)k(x) (6.22)

Using the Lyapunov function

1
Ve (x, ) = V (x) + 2 (6.23)
2

it is clear that V e = V 0. There is no asymptotic stability, however, since

x = 0 is an equilibrium regardless of . To get asymptotic stability one can write
the control in the form u = k(x) + u so that the system description becomes

= b(x)
u (6.24)
x = f (x) + g(x)k(x) + g(x)
u (6.25)

where b = b + (/x) g. We can then use (6.23) as a control Lyapunov function

for u
. This gives
V e = Vx (f + gk) + (Vx g + b)
u
We can then use the control

= (Vx g + b)
u

to stabilize the whole system.

Example 6.3 Consider the system

z = u
x = x + x3 + u

The subsystem x = x + x3 + u can be globally asymptotically stabilized using

u = k(x) = x3 and V = 12 x2 . With this feedback the system becomes

z = x3
x = x

The new variable becomes


x3
=z+ (x3 e3t ) dt = z
0 3

Using x, as state variables and writing u = x3 + u gives

= (1 x2 )
u
x = x + u

with the control Lyapunov function

x2 2
Ve = +
2 2

89
Since
V e = x2 + (x + x2 )
u

we can take
x3
= (x + x2 ) = x (z
u )(1 x2 )
3
In Figure 6.2 the system is simulated. A comparison is made with the controller
which has only linear terms, i.e.

= u = x z
u

In this case the linear controller does not stabilize the system for the initial
condition z(0) = 3, and we see an example of nite escape time.

3 5

4.5

2
4

3.5
1

0 2.5

1
1.5

1
2

0.5

3 0
0 5 10 15 0 5 10 15

Figure 6.2: System with forwarding controller (solid) compared to a linear con-
troller (dashed). The initial state is x(0) = 0, z(0) = 3. To the left are the
states, to the right the Lyapunov function.

6.4 Stability of observers

Let us now return to the question of stability of the observers discussed in
Section 4.3. There we had a system

z = Az + B(z) + g(z)u, y = Cz (6.26)

where

0 1 0 ... 0
0 0 1 0 0
..
.. ..  
A=
. . 0
, B = . , C= 1 0 ... 0 (6.27)
. 0
.. 1 1
0 0 0 ... 0

90
and
g1 (x1 )
gx (x1 , x2 )

..
g(x) = . (6.28)

gn1 (x1 , . . . , xn1 )
gn (x)
Using the natural observer

= A
x x + B( x)u + K(y C x
x) + g( ) (6.29)

then gives an observer error x = x x that satises

= (AKC)
x x+B((x)(
x ))+(g(x)g(
x))u = (AKC)
x+L(x, x
, u) (6.30)

where
x)) + (g(x) g(
L(x, x, u) = B((x) ( x)) u
T S x
Using the Lyapunov function V = x gives

V = x
T ((A KC)T S + S(A KC)) xT SL(x, x
x + 2 , u)

We now choose the gain as K = S 1 C T which gives

V = x
T (AT S + SA 2C T C) xT SL(x, x
x + 2 , u)

Finally we dene S to be the solution of the equation

AT S + SA C T C = S (6.31)

where is a parameter that is to be chosen suciently large. The resulting

expression for V becomes

V = )2 + 2
(C x
xT S x , u)
xT SL(x, x xT S x xT SL(x, x
+ 2 , u) (6.32)

Let |x| denote the ordinary Euclidian norm of x and use the notation |x|S =
|S 1/2 x| = (xT Sx)1/2 . Then (6.32) becomes

d
x|2S ) |
(| x|2S + 2|
x|S |L(x, x
, u)|S
dt
which can be simplied to

d
x|S ) |
(| x|S + |L(x, x, u)|S (6.33)
dt 2
To go further it is necessary to consider in detail the solution of (6.31). We
begin by looking at an example.

Example 6.4 Consider an observer for a two-dimensional system. Then

   
0 1   s s12
A= , C = 1 0 , S = 11
0 0 s12 s22

91
and equation (6.31) becomes
   
1 s11 s s12
= 11
s11 2s12 s12 s22

with the solution

   
1 2 2
S= , K = S 1 C T = (6.34)
2 23 2

Now consider the term L(x, x

, u). We have
   
L1 (g1 (x1 ) g1 (
x1 ))u
L= =
L2 (x) (x) + (g2 (x) g2 (
x))u

Let us make the following assumptions

|u(t)| uo , all t (6.35)

|(g1 (x1 ) g1 ( 1x
x1 ))| |x 1 | (6.36)
|(g2 (x) g2 (
x))| 2 |x x
| (6.37)
(x) ( 3 |x x
x) | (6.38)

Conditions (6.36) (6.38) are required to hold for all values of x and x . This
means that we require the functions to satisfy a global Lipschitz condition. When
this holds together with the bound on u it follows immediately that

1 , u)| 1 |
|L1 (x1 , x x1 |
|L2 (x, x
, u)| 2 |
x|

for some constants 1 and 2 . It is now possible to estimate the size of the
term |L|S in (6.33):

|L|S = (s11 L21 + 2s12 L1 L2 + s22 L22 )1/2

(1 21 |
x1 | + 22 1 2 | x| + 23 22 |
x1 || x|2 )1/2
(c2 2 + 2c1 c2 1 2 + 2c22 22 )1/2 |
x|S
%1 1 &' (
c3

where we have used the inequalities

x1 | c1 1/2 |
| x|S , x| c2 3/2 |
| x|S

that are satised for some constants c1 and c2 . It follows that (6.33) can be
rewritten as
d
x|S ) |
(| x|S + c3 |
x|S
dt 2
Since the right hand side is negative if is large enough it follows that the
observer error goes to zero for arbitrary initial conditions.

The calculations of the example are easily generalized to the general case which
gives the following theorem

92
Theorem 6.1 Consider the observer (6.29). Assume that there is a constant
uo so that the control signal satises u(t) uo for all t. Also assume that
and the functions gj (x1 , .., xj ) satisfy global Lipschitz conditions and that the
observer gain is K = S 1 C T with S given by (6.31). Then for each which is
suciently large there exists a constant C() such that
(t)| C()et/3 |x(0) x
|x(t) x (0)| (6.39)
i.e the observer error converges to zero with an arbitrarily fast exponential
convergence rate.

Proof. It is easy to see that the structure of (6.31) makes it possible to calculate
the elements of S recursively, starting with s11 = 1 . One can also show that
this matrix is always positive denite. It is also easy to see that the recursive
structure means that an element of S will have the form
soij
sij = i+j1

for some constant soij . This means that the estimate of the size of the term |L|S
that was done in Example 6.4 can be done in a completely analogous way for a
general system. One can therefore obtain the estimate
d
x|S ) |
(| x|S + c3 |
x|S
dt 2
from that example also in the general case. Choosing 6c3 gives the estimate
d
x|S ) |
(| x|S
dt 3
which directly gives (6.39).

This theorem gives a strong global convergence result for a general class of
observers. It is however worth noting that it depends on a number of conditions.

The result is global: it holds for any x(0) and any x

(0). However it requires
the system to be in the special form (6.26). In section 4.2 we discussed
how a general system could be brought into this form ba a variable change.
For the convergence of the observer to be relevant in the original physical
variables this coordinate change has to be global.
The functions (x) and gi (x) have to satisfy global Lipschitz conditions
which is fairly restrictive.
The observer gains grow to innity as grows to innity. This observer
design therefore usually requires a measurement of y with very low noise
level.

The observer and the closed loop system

In control design an observer is usually used to compute and estimate of the

state that is used in a controller based on state feedback. Typically the system
dynamics is given by
x = f (x) + g(x)u (6.40)

93
and we nd a control law u = k(x) and a Lyapunov function V such that

V = Vx (x)(f (x) + g(x)k(x)) q(x) 0 (6.41)

Then an observer is designed that gives a state estimate x and the controller

x) = k(x x
u = k( ) (6.42)

is used. Suppose a Lyapunov function Ve ( x) is known for the estimation error

x|2S in Theorem 6.1) with an inequality
(like |

V e qe (
x) 0 (6.43)

Then it is natural to try W = V + Ve as a Lyapunov function candidate. One

gets
W q(x) qe (
x) + Vx (x)(k(x x) k(x)) (6.44)
% &' (

For a specic design it is often possible to show that the term is suciently
small so that the overall expression becomes negative. It is however dicult to
give general conditions that guarantee that this will happen.
One might think that the high gain observer described by Theorem 6.1 would
guarantee stability also for the closed loop system, since according to (6.39),
after an arbitrarily short time the dierence between x and x becomes negligible.
The catch is that there might be very large transients during this short time,
due to the large gain in the observer. These large transients could, via the
control law destabilize the closed loop system beyond recovery. A high gain
observer therefore has to be combined with some scheme for handling these
initial transients. Provided this is done it is indeed possible to prove closed loop
stability for some classes of systems where nonlinear state feedback is combined
with high gain observers.

6.5 Rejection of disturbances

Suppose we have a system

x = f (t, x) + g(t, x)u (6.45)

for which we know a Lyapunov function V (t, x) which guarantees stability for
the open loop system by satisfying the conditions of Theorem 5.2. In particular
there is a positive denite function W such that

Vt + Vx f (t, x) W (x) (6.46)

Now we consider disturbances and uncertainties that can be viewed as additive

to the input:

94
 w

u
x = f + gu

Thus we can write

x = f (t, x) + g(t, x)(u + w(t, x, u)) (6.47)

where we have emphasized that the disturbance is allowed to depend on both

the state and the input as well as being time-varying. Disturbances that enter
as in (6.47) are sometimes said to satisfy a matching condition. If w were known
it would of course be possible to eliminate it completely by using feed-forward.
We will assume that w is completely unknown except for an upper bound:

|w(t, x, u)| (t, x) + |u| (6.48)

Computing the derivative of V gives

V = Vt + Vx (f + gu + gw) W + Vx gu + |Vx g|( + |u|)

Choose u to always give a negative contribution:

u = k(t, x) sign (Vx g)

It follows that
V = W + |Vx g|(k + + k) W
if k is chosen as /(1 ). The chosen control is thus

(t, x)
u= sign (Vx g) (6.49)
1

and it achieves that the Lyapunov function decreases at least as fast in the pres-
ence of the unknown disturbance as for the open loop undisturbed system. In
particular we achieve stability for the closed loop system according to Theorem
5.2. The price we have to pay is that the control law is discontinuous (unless
Vx g = 0 implies = 0) and that there is no guarantee that it will decrease to
zero as the equilibrium is approached.

Example 6.5 Consider the mechanical system of gure 6.3. A unit mass with
position x1 and velocity x2 is connected to a nonlinear spring and a damper.
The control signal u is a force and there is also an external force disturbance w.
The spring force is assumed to be x1 x31 and the damper force is supposed
to be x2 . The system equations are then

x 1 = x2
x 2 = x1 x31 x2 + u + w

95
 x2
w
u

x1

Figure 6.3: Mechanical system with nonlinear spring and force disturbance.

A Lyapunov function for the undisturbed, uncontrolled system, u = 0, w = 0,

is
3 x4
V = x21 + x1 x2 + x22 + 1 , V = x21 x22 x41
2 2
Suppose we know that |w| 1. Then (6.49) gives

u = sign (Vx g) = sign (x1 + 2x2 )

In Figure 6.4 the response of this controller is shown. The disturbance and
control signals are shown in Figure 6.5. Note that the control signal switches
very rapidly between its extreme values. In Figure 6.6 a low pass ltered version
of u is compared to the disturbance. Note that the average of u is close to w,
which accounts for the disturbance rejection.
5 3

4.5 2.5

4 2

3.5 1.5

3 1

2.5 0.5

2 0

1.5 0.5

1 1

0.5 1.5

0 2
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10

Figure 6.4: Response of the mechanical system. To the left is V (t), to the right
x1 (t). Uncontrolled system without disturbance: dotted, uncontrolled system
with disturbance: dashed, controlled system with disturbance: solid.

6.6 Passivity based control

In the previous chapter (section 5.5) we discussed passive systems. Passive
systems have several interesting properties. One of them is that passivity is
preserved under a feedback of the form shown in Figure 6.7.

96
 1 1

0.8 0.8

0.6 0.6

0.4 0.4

0.2 0.2

0 0

0.2 0.2

0.4 0.4

0.6 0.6

0.8 0.8

1 1
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10

Figure 6.5: Control of mechanical system. Disturbance signal w to the left and
control signal u to the right.

1.5

0.5

0.5

1.5
0 1 2 3 4 5 6 7 8 9 10

Figure 6.6: Control of mechanical system. Filtered u (solid) and w (dashed).

+ u1 y1 y
u S1

y2 u2
S2

Figure 6.7: Feedback loop with two passive systems.

97
Proposition 6.1 Consider the feedback system of Figure 6.7 where the systems
S1 and S2 are both passive. Then the closed-loop system with input u and
output y is also passive.

Proof. Since the systems S1 and S2 are passive the denition of passivity gives
 T
uTi yi dt + i (xoi ) 0, i = 1, 2
0

for some funktions 1 and 2 , where xo1 , xo2 are the initial states of S1 and S2 .
Using that
u 1 = u y2 , u 2 = y1 = y
one gets
 T  T
uT y dt = (uT1 y1 + y2T u2 ) dt 1 (xo1 ) 2 (xo2 )
0 0

which shows that the system with input u and output y is passive with =
1 + 2 .

In passivity based control one tries to control systems that are already passive
in a way that preserves passivity. The proposition above shows that one way
of doing this is to use a controller which is in itself passive. Usually this is too
restrictive however. Looking at Example 5.5 we see that equation (5.41) can be
rewritten as  
T T
uy dt + (x(0)) (x(T )) = by 2 dt
0 0

In passivity-based control one assumes this structure for a (possibly multivari-

able) system:
 T
uT y dt + (x(0)) (x(T )) = d 0 (6.50)
0
where represents stored energy and d is a dissipation term. One then considers
state feedback of the form
u = k(x) + v (6.51)
Substituting into (6.50) one obtains
 T  T
v y dt
T
k(x)T y dt + (x(0)) (x(T )) = d
0 0

If the feedback k can be chosen so that

 T
k(x)T y dt = (x(T )) (x(0))
0

for some function , then the closed loop system with v as new input will be
passive. Its energy storage function will be + . The idea is now to choose
so that the new energy function has a minimum at the desired equilibrium
of the system. If the natural dissipation d is not sucient to drive the system
towards the equilibrium fast enough, then further terms are introduced into the
controller to increase the dissipation.

98
This approach has been used extensively for mechanical systems whose stored
energy has the form
1
= qT I(q)q + V (q) (6.52)
2
where q is a vector of generalized coordinates (usually linear displacements and
angles), I is an inertia matrix and V is the potential energy. Using the feedback
k(x) = Vq + K(q qr ) (6.53)
where K is a positive denite matrix and qr is the desired position, gives
 T  T
k T y dt = (Vq + K(q qr ))q dt =
0 0
 T
d 1
(V + (q qr )T K(q qr )) dt = (x(T )) (x(0))
0 dt 2
1
where = V + 2 (q qr )T K(g qr ). The stored energy will be modied to
1 T 1
+= q I(q)q + (q qr )T K(q qr )
2 2
This function has a minimum at q = qr , q = 0, which is the equilibrium the
system will converge to if there is enough dissipation.

Example 6.6 Consider again Example 5.5, but let the damping be a nonlinear
function b(x2 ) of velocity.
x 1 = x2
mx 2 = u k1 x1 k2 x31 b(x2 ) (6.54)
y = x2
The stored energy is as before
k1 2 k2 4 m 2
(x) = x + x1 + x2
2 1 4 2
k1 2
Comparing with (6.52) we see that q = x1 , q = x2 , I(q) = m and V = 2 x1 +
k2 4
4 x1 . The control law (6.51), (6.53) then becomes

u = v + k1 x1 + k2 x31 K(x1 r)
where r is the reference value for the position. Note that this control law is
similar to exact linearization because the spring force is subtracted away. Note
also that it is dierent in not cancelling the damping term b(x2 ). The closed
loop system will be
x 1 = x2
mx 2 = v K(x1 r) b(x2 ) (6.55)
y = x2
and its stored energy function
m 2 K
V = x + (x1 r)2
2 2 2

99
For v = 0 and r = constant the closed-loop system has an equilibrium at x1 = r.
Using V as a Lyapunov function candidate gives

V = x2 (K(x1 r) b(x2 )) + K(x1 r)x2 = x2 b(x2 )

If the damping satises x2 b(x2 ) 0, then the equlilibrium x1 = r is stable. If

x2 b(x2 ) > 0 for x2 = 0 it is asymptotically stable. If the term x2 b(x2 ) does
not give fast enough convergence towards the equilibrium, then an additional,
x2 -dependent term in the controller might be useful.

6.7 An example from adaptive control

An area where Lyapunov theory has been used extensively is in the design of
adaptive controllers. The subject is large and we will just look at a simple
example to show the type of reasoning that can be used. Consider a linear
system controlled by a P-regulator when the set point is zero and there is an
external disturbance w, Figure 6.8. Let the relative degree of G (the dierence

y
G(s)

Figure 6.8: Linear system with P-controller and external disturbance.

between the degrees of denominator and numerator) be one, and assume that
all zeros are strictly in the left half plane. Then it is easy to see from a root
locus argument that G will always be stabilized if k is big enough. Also the
inuence of a constant w can be made arbitrarily small if k is large enough.
However, since G and w are unknown, we do not know how to pick k. A simple
idea is to use the adaptation rule

k = y 2 , k(0) 0

which lets k increase as long as there is a control error. This scheme can be
shown to work in an ideal noise free situation. In a realistic situation, however,
y does not decrease to zero due to disturbances and measurement noise, which
means that k will increase without bound. It is clear that some modication is
needed, such as
k = y 2 + f (k), k(0) 0 (6.56)
where k has to be determined. We assume that f (k) is chosen so that k always
remains positive. To get an idea how the choice should be made, we will try

100
to construct a Lyapunov function. Let n be the order of the linear system G.
Since the relative degree is one, there is a state space realization of the form

x = Ax + by (6.57)
y = cx dy + g(u + w) (6.58)

with g = 0, where x is an (n 1)-dimensional vector. We assume for simplicity

that the problem is scaled so that g = 1. Note that it is possible to keep y
identically zero by choosing u = cx + dy (for the case w = 0). The remaining
dynamics is then the zero dynamics, which is given by x = Ax. It is easy to see
that the eigenvalues of A are the zeros of G. Since we assumed G to have its
zeros strictly in the left half plane, A has all its eigenvalues there. We will now
consider a Lyapunov function candidate of the form

V = xT P x + y 2 + (k ko )2 (6.59)

where ko is a constant to be determined. The analysis will be made for the case
of a constant but unknown w. Dierentiating we get

V = (Ax+by)T P x+xT P (Ax+by)+2y(cxdyky+w)+2(kko)(y 2 +f (k)) =

  
  AT P + P A P b cT x
= xT y + 2wy + 2(k ko )f (k)
bT P + c 2(d + ko ) y
% &' (
Q

We see that there is no hope of making V negative all the time due to the
term 2wy. However, we can do the following. Choose P as the solution of the
Lyapunov equation
AT P + P A = I
This is always possible since A has all its eigenvalues strictly in the left half
plane. Next choose ko such that
 
I P b cT
Q= T
b P + c 2(d + ko )

is negative denite. Further choose f (k) such that

f (k) k, >0 (6.60)

We then have the following fact.

Proposition 6.2 For the Lyapunov function candidate (6.59) and an adapta-
tion rule satisfying (6.60) there are constants V0 > 0 and  > 0 such that in the
set
{(x, y, k) : V (x, y, k) V0 } (6.61)
we have
V (xT x + y 2 + k 2 )

Proof. Since Q is negative denite, there is a constant 1 > 0 such that

Q 21 I. For k > ko we have

V 21 (xT x + y 2 ) 2k 2 + 2kko + 2wy

101
Taking  = min(1 , ) we get
V (xT x + y 2 + k 2 )
if (xT x + y 2 + k 2 ) is large enough, which is the case if V (x) V0 with Vo large
enough.

The proposition shows that V will decrease in the set V V0 so that eventually
the set V (x, y, k) < V0 is reached. The adaptation will thus work in the sense
that all variables reach this bounded set. A closer look at the estimates is taken
in Exercise 6.10.

6.8 Exercises
6.1 Compute a backstepping controller for the system
x 1 = x21 + x2
x 2 = x2 + x3
x 3 = u

6.2 Consider the heat exchanger of example 1.2.

d
(CT ) = qcT0 qcT + (Th T )
dt
Let the state variables be x1 = T , x2 = q and Th = x3 . Suppose x2 and x3
are controlled from the inputs u1 and u2 with some lag due to time constants
in the control actuators. If T0 = 0, c/C = 1 and /C = 1, then the system is
described by
x 1 = x1 + x3 x2 x1
x 2 = x2 + u1
x 3 = x3 + u2

Let the control be of the form u1 = 1+ u

1, u2 = 1+ u
2. Compute the equilibrium
corresponding to u 1 = 0, u2 = 0. Then compute a Lyapunov based feedback
that makes that equilibrium asymptotically stable. Is it possible to achieve
global asymptotic stability (global in the sense of physically reasonable values
of the state variables)?

6.3 The Kokotovic benchmark problem.

x 1 = x2 + (x2 x3 )2
x 2 = x3
x 3 = u
a. Show that the system is not feedback linearizable.
b. Give a control law with associated Lyapunov function that achieves global
asymptotic stability.

102
6.4 Compute a controller for
x 1 = sin x2
x 2 = u
with the largest possible stability region (guaranteed by a Lyapunov function).

6.5 Extend the results of the previous problem to

x 1 = sin x2
x 2 = sin x3
x 3 = u

6.6 Solve (6.31) for a third order system and compute the observer gain K.
Alos compute the eigenvalues of A KC.

6.7 Consider the system and controller

x = f (x, u), u = k(x)
with the Lyapunov function V satisfying
c1 |x|2 V (x) c2 |x|2
Vx (x)f (x, k(x)) c3 |x|2 , |Vx | c4 |x|
for some positive constants ci . Now suppose there is a disturbance signal w
x = f (x, k(x)) + w, |w| c5
Show that x will eventually satisfy a bound
|x| c6
Compute an estimate of c6 based on the other ci .

6.8 A dissipative system is called strictly input passive if

w(u, y) = uT y uT u, >0
and strictly output passive if
w(u, y) = uT y y T y, >0
Consider the feedback system of Figure 6.7, where S1 and S2 are passive. State
a condition on strict input or output passivity for the individual systems which
guarantees strict output passivity for the closed loop system.

6.9 Simulate the adaptive scheme of Section 6.7 with f (k) = k for dierent
systems G and dierent, not necessarily constant, disturbances w.

6.10 Consider the adaptive scheme of Section 6.7 with f (k) = k. Let
< 1/max (P ). Show that ko can be chosen such that
V 2V + 2ko2 + 2|w|
What conclusions can be drawn about the properties of the adaptive scheme?

103
.

104
Chapter 7

Nonlinear optimal control.

In linear system theory many design methods for controllers are based on solu-
tions to the linear quadratic control problem:
 t1
 T 
min x Qx + uT Ru dt + x(t1 )T Q0 x(t1 ) (7.1)
0

for the system

x = Ax + Bu (7.2)
This chapter deals with the extension of the linear quadratic control ideas to
nonlinear systems and non-quadratic criteria.

7.1 The optimal control problem.

We generalize the linear quadratic problem (7.1), (7.2) in the following way.
Consider a nonlinear system

d
x = f (t, x, u), uU (7.3)
dt
where x is an n-vector and u an m-vector. The control is assumed to be con-
strained in such a way that u(t) U for all t, where U is some set in Rm .
Let
(t, x0 , u(.))
be the solution of (7.3) with the initial condition x(0) = x0 when the control
function u(.) is applied. Consider the problem of controlling the system from a
given point until a certain condition is met:

x(t0 ) = x0 (7.4)

(t1 , (t1 , x0 , u(.))) M (7.5)

where M is some set in R R . Typically M might be a point, in which case
n

one would want to reach a certain state at a certain time. As another example,

105
M might be a line parallel to the time axis, in which case one wants to reach
a certain point at an unspecied time. Also we want to nd the best control
satisfying (7.5), so we consider minimizing
 t1
J= L(t, x(t), u(t))dt + (t1 , x(t1 )) (7.6)
t0

where we have used the shorthand notation x(t) = (t, x0 , u(.)). The problem is
thus to transfer the state from the point x0 to the set M , using controls that lie
in U at each instant of time, in such a way that the criterion (7.6) is minimized.
Note that the nal time t1 is not specied, except for the conditions imposed
by M .
Since we would like the control expressed as a feedback law, we want to solve
the problem for all starting times and all starting points. Then J is a function
of these quantities as well as the control function:
J = J(t0 , x0 , u(.))
Suppose a minimizing u exists and dene
V (t, x) = min J(t, x, u(.)) (7.7)
u(.)

(where the minimization is carried out over those functions u that satisfy u(t)
U and give a trajectory satisfying (t1 , x(t1 )) M ). Thus V (t, x), the optimal
return, shows the cost of starting from x at time t when the control is chosen
optimally.

If we only consider trajectories that reach M once, then from the denition it
follows that
V (t, x) = (t, x), (t, x) M (7.8)

The basis for the analysis of the optimal control problem is the following simple
relation, sometimes denoted the principle of optimality.

Theorem 7.1 Let V be dened by (7.7) . Then

 t+h
V (t, x) L(t, (t, x, u(.)), u(t))dt + V (t + h, (t + h, x, u(.))) (7.9)
t

for all choices of u( ), t t + h satisfying u( ) U . Equality is obtained

precisely when u is optimal.

Proof. Suppose that an arbitrary u U is used from t to t + h and the optimal

one from t + h to t1 . Then, letting denote optimal values
 t+h
J(t, x, u(.)) = L(t, (t, x, u(.)), u(t))dt+
t
 t1
L(t, x (t), u (t))dt + (t1 , x (t1 )) =
t+h
 t+h
L(t, (t, x, u(.)), u(t))dt + V (t + h, (t + h, x, u(.)))
t

106
where x (t) = (t, (t, x, u(.)), u (.)). Since V (t, x) J(t, x, u(.)) with equality
when u is optimal, the theorem follows.

If the optimal return function is smooth it is possible to go one step further.

Theorem 7.2 If V is continuously dierentiable, then it satises the following

partial dierential equation.

0 = min(Vt (t, x) + Vx (t, x)f (t, x, u) + L(t, x, u)) (7.10)

uU

the so called Hamilton-Jacobi equation .

Proof. Letting h tend to zero in Theorem 1 gives

d
V (t, x(t)) + L(t, x(t), u(t)) 0
dt

with equality when u is chosen optimally. Using the dierentiation rule

d
V (t, x(t)) = Vt (t, x(t)) + Vx (t, x(t))f (t, x(t), u(t))
dt

then proves the theorem.

The converse of Theorem can also be shown.

Theorem 7.3 Let W be a continuously dierentiable function solving the prob-

lem

0 = minuU (Wt (t, x) + Wx (t, x)f (t, x, u) + L(t, x, u))

(7.11)
W (t, x) = (t, x), (t, x) M

Also assume that the minimizing u is given by a function

u = k(t, x) (7.12)

which is continuous and gives a trajectory satisfying (t1 , x(t1 )) M .

Then u = k(t, x) is optimal and

W (t, x) = V (t, x)

Proof. Dene x as the solution of

d
x = f (t, x, k(t, x))
dt

x(t0 ) = x0

107
and let u (t) = k(t, x (t)). Then
 t1

J(t0 , x0 , u (.)) = L(t, x (t), u (t))dt + (t1 , x (t1 )) =
t0
 t1
L(t, x (t), u (t))dt + W (t1 , x (t1 )) =
t0
 t1  
d
L(t, x (t), u (t)) + W (t, x (t)) dt + W (t0 , x(t0 )) =
t0 dt
 t1  
L(t, x (t), u (t)) + Wx (t, x (t))f (t, x (t), u (t)) + Wt (t, x (t)) dt+
t0
W (t0 , x(t0 )) = W (t0 , x(t0 ))
 t1
[L(t, x(t), u(t)) + Wt (t, x(t)) + Wx (t, x(t))f (t, x(t), u(t))] dt+
t0
 t1
W (t0 , x(t0 )) = L(t, x(t), u(t))dt + W (t1 , x(t1 )) =
t0
 t1
L(t, x(t), u(t))dt + (t1 , x(t1 )) = J(t0 , x0 , u(.))
t0

where u is an arbitrary control signal satisfying u(t) U and (t1 , x(t1 )) M .

Consequently u is optimal. Since J(t0 , x0 , u (.)) = W (t0 , x(t0 )) and u is
optimal, V = W .

This theorem shows that if we can solve (7.11) and if the resulting solution W
is smooth, then the optimal control problem dened by (7.6) and (7.3) is solved,
and the solution is in the feedback form (7.12). Unfortunately the function V
dened by (7.7) has a discontinuous gradient in many cases. The theory then
becomes much more complicated, since a generalized concept of derivatives is
needed to interpret the expression Vx f . Even when there is a smooth solution of
(7.11), it can seldom be expressed analytically. A simple example where (7.11)
can solved explicitly is the following.

Example 7.1 Let the system be

x = u

with the criterion  1

u4 x(1)4
J= dt +
t0 4 4
i.e. the nal time is specied to be t1 = 1. There is no restriction on the nal
state or on the control. The Hamilton-Jacobi equation becomes

u4
0 = min(Vt + + Vx u)
u 4
For x M , i.e. for t1 = 1 we have

V (1, x) = x4 /4

108
This gives the solution
x4
V (t, x) =
4(2 t)3
and the feedback law
x
u=
2t

7.2 Innite horizon optimal control.

In the linear quadratic control problem one often considers control over an
innite time horizon.

 T 
min x Qx + uT Ru dt (7.13)
0

Analogously it is natural to replace the criterion (7.6) with


J= L(x(t), u(t))dt (7.14)
0

Here we have assumed that there is no explicit time dependence in the integrand
L. It is then no restriction to assume that the initial time is 0. We also assume
that the system is time invariant and described by

d
x = f (x, u), uU (7.15)
dt

The optimal return must then also be time invariant: V (t, x) = V (x). The
Hamilton-Jacobi equation then reduces to

0 = min (L(x, u) + Vx (x)f (x, u)) (7.16)

uU

We can now reformulate Theorem 7.7 as

Theorem 7.4 Let W be a continuously dierentiable function solving the prob-

lem
0 = minuU (Wx (x)f (x, u) + L(x, u)) (7.17)

Also assume that the minimizing u is given by a continuously dierentiable

function
u = k(x) (7.18)

that drives the state to the origin as time goes to innity. Then this control
is optimal among all controls that drive the state to the origin, and W is the
corresponding optimal return function.

109
Proof. Consider a control u driving the corresponding x to the origin as t goes
to innity. Then

J(x0 , u(.)) = L(x(t), u(t))dt =
0
 
d
(L(x(t), u(t)) + Wx (x(t))f (x(t), u(t))) dt W (x(t)(t))dt =
0 0 dt

= (L(x(t), u(t)) + Wx (x(t))f (x(t), u(t))) dt + W (x0 ) W (x0 )
0

where the last inequality follows from (7.17). It follows that u = k(x) is mini-
mizing and that W is the optimal return.

Example 7.2 Consider the system

x = u, |u| 1

with the criterion 

(x2 + u2 )dt
0
The Hamilton-Jacobi equation becomes

0 = min (x2 + u2 + Vx u)
|u|1

which has the solution

)
x2 , |x| 1
V (x) =
x3 /3 + x 1/3, |x| > 1

with the feedback )

x, |x| 1
u=
sgn(x), |x| > 1

7.3 Calculation of optimal control.

Example 7.2 is atypical because it allows an explicit closed form solution. This
is not possible to achieve in the general case. We will look att some methods of
calculating the optimal control numerically.

Series expansion

If the functions L and f in (7.16) are real analytic (i.e. given by convergent series
expansions), then it is possible to compute the solution as a series expansion.
To show this we assume that the optimal control problem is given by
  
1 d
min l(x) + uT Ru dt, x = a(x) + b(x)u (7.19)
0 2 dt

110
where the functions a, b and l are real analytic. We thus assume that the control
appears linearly in the dynamics and quadratically in the criterion. This is not
necessary for the calculations that follow, but make them very much simpler.
The Hamilton-Jacobi equation takes the form
1
0 = l(x) + Vx (x)a(x) Vx (x)b(x)R1 b(x)T Vx (x)T (7.20)
2
and the optimal control is given by

u = k(x) = R1 b(x)T Vx (x)T (7.21)

Writing
1 T
l(x) = x Qx + lh (x)
2
a(x) = Ax + ah (x)
(7.22)
b(x) = B + bh (x)
1
V (x) = xT Sx + Vh (x)
2
where lh ,ah ,bh and Vh contain higher order terms (beginning with degrees 3,2,1
and 3 respectively), the Hamilton-Jacobi equation splits into two equations.

0 =Q + AT S + SA SBR1 B T S (7.23)
1
0 =Vhx (x)Ac x + lh (x) + Vx (x)ah (x) Vhx (x)BR1 B T Vhx (x)T (7.24)
2
1
Vx (x)wh (x)Vx (x) T
(7.25)
2
where

Ac = A BR1 B T S (7.26)
1 1
wh (x) = b(x)R b(x) BR
T
B T
(7.27)

(so that wh contains terms of degree 1 and higher). Equation (7.23) is the
ordinary Riccati equation of linear quadratic control. If we assume that

Q 0, R > 0, (Q, A) observable, (R, A)controllable (7.28)

then the theory of linear quadratic control tells us that (7.23) has a unique
positive denite solution. Letting superscript (m) denote m:th order terms,
equation (7.25) can be written

(V (m) )x Ac x = l(m) (x)+

 (m)
+ Vx (x)ah (x) 2 Vhx (x)BR1 B T Vhx (x)T 12 Vx (x)wh (x)Vx (x)T
1

(7.29)
The right hand side contains only (m 1):th, (m 2):th,.. order terms of
V . Equation (7.29) therefore denes a linear system of equations for the m:th
order coecients with a right hand side which is known if lower order terms have
been computed. Using the same arguments that was used in the calculation of
Lyapunov functions, it can be shown that this system of equations is nonsingular

111
as soon as Ac is a stable matrix. Since Ac represents the linearized closed
loop dynamics, this is guaranteed by linear quadratic control theory if (7.28)
holds. After solving (7.23), the 3rd, 4th, 5th,.. order coecients can then be
computed successively. It can be shown that the resulting series representation
of V converges and denes a real analytic function in some neighborhood of the
origin.

Model predictive control

Consider again the innite horizon optimal control problem (7.14), (7.15). If
the optimal return V is known it is easy to see that (7.14) is equivalent to the
nite horizon optimal control problem
 T
J= L(x(t), u(t))dt + V (x(T )) (7.30)
0

This formula is only of theoretical interest, since V is usually unknown. There

might however be some function , which approximates V . One then has the
approximating nite horizon problem
 T
J = L(x(t), u(t))dt + (x(T )) (7.31)
0

Now suppose that we do the minimization over a restricted class of control

functions u, namely those that are piecewise constant:

u(t) = uk , kh t < (k + 1)h, k = 0, . . . N 1 (7.32)

Here we assumed that T is a multiple of h, T = N h. Now J becomes a function

of N real numbers, u0 , . . ., uN 1 . To minimize J is thus a problem in nonlin-
ear programming that can be solved using commercial software. The problem
is dicult for several reasons. The computation of J for given values of ui
involves solving x = f (x, u) numerically and then calculating the integral in
(7.31) numerically. The problem is a constrained one, since the constraints on u
in (7.15) translates into constraints on the ui . Since the problem in the general
case in non-convex, there is no guarantee that a global minimum is found by a
numerical algorithm. However, in situations where the nonlinear programming
problem can be solved it forms the basis for nonlinear MPC (model predictive
control). In nonlinear MPC one solves the optimization problem on line with
the current value of x as the initial state to give u0 ,..,uN 1 . Then u0 is used
during a time interval of length h. After that the problem is resolved, regarding
the current time as t = 0 and using the actual measured x as a new initial
value of the optimization problem. Again only u0 is used and then the problem
is resolved. Since the current, measured, state x is used as the initial state
in the minimization of J, this gives a feedback controller. Since at each time
(7.31) is solved for a time interval which is T units forward from the present
time, this approach is also referred to as a receding horizon optimal control. It
is intuitively clear that nonlinear MPC will give a good approximation of the
solution to the original optimal control problem (7.30) if T is large enough, h
small enough and a good enough approximation of V .

112
The MPC technique has been particularly successful for the constrained linear
quadratic problem

J= (xT Qx + uT Ru) dt
0
x = Ax + Bu, ubi ui uti , i = 1, . . . , m
Nx + Mu 0
The receding horizon problem then has a criterion of the form
 T

J= (xT Qx + uT Ru) dt + x(T )T Qo x(T )
0

The nite dimensional optimization problem for u0 ,..,uN 1 is still quadratic

with linear constraints. It can be solved using ecient quadratic programming
algorithms.

7.4 Nonlinear robustness techniques

As in the linear case, many robustness problems can be formulated in the fol-
lowing framework:

d z
- -

u x
-

k 

The external signal d could be a disturbance or a reference signal. The output

z is some measure of the control error and should be small. The feedback k
then has to be chosen so that the gain from d to z is as small as possible.
Mathematically we formulate the problem as follows. Consider the system
x = f (x) + g(x)u + b(x)d
(7.33)
z = h(x)
where x is an n-vector, u an m-vector, y a p-vector and d a q-vector. We will
look at state feedback
u = k(x) (7.34)
so the resulting closed loop system becomes
x = f (x) + g(x)k(x) + b(x)d
   
z h(x) (7.35)
y= =
u k(x)
We will look at the following problems

113
Denition 7.1 The state feedback gain reduction problem is to nd a state
feedback (7.34) such that the gain from d to y of the closed loop system (7.35)
is less than some given value .

Denition 7.2 The optimal state feedback gain reduction problem is to nd

the smallest value , such that the state feedback gain reduction problem is
solvable for all > .

To compute the gain from d to y we can use the Hamilton-Jacobi inequality

(5.52). We get
1
Vx (f + gk) + Vx bbT VxT + hT h + k T k 0 (7.36)
4 2
Completing squares we can also write
1 1 1 1
Vx f + 2
Vx bbT VxT +hT h+(k+ g T VxT )T (k+ g T VxT ) Vx gg T VxT 0 (7.37)
4 2 2 4
We get the following result.

Theorem 7.5 Let be given. If the Hamilton-Jacobi inequality

 
1 1 T
Vx f + Vx bb gg T
VxT + hT h 0, V (x0 ) = 0 (7.38)
4 2

has a nonnegative solution V , then the control law

1
u = k(x), k(x) = g(x)T Vx (x)T (7.39)
2
gives a gain from d to y which is less than or equal to , that is
 T  T
(z T z + uT u) dt 2 dT d dt (7.40)
0 0

for all solutions of the closed loop system starting at x0 .

Solving the Hamilton-Jacobi inequality leads to the same type of problems as

the solution of the Hamilton-Jacobi equation. We give a simple example where
there is an explicit solution.

Example 7.3 (Van der Schafts system). Consider the system

x = u + (arctan x)d (7.41)

 
x
y= (7.42)
u

The Hamilton-Jacobi inequality becomes

 2
1 arctan x
1 Vx2 + x2 0
4

114
A global solution requires that

| arctan x| <

that is > /2. A feedback giving gain is then

x
u = 
1 (arctan x/)2

7.5 Exercises.
7.1 What is the optimal feedback for the system

x = u

with the criterion 

1 2
(x + x4 + u2 )dt
0 2
What is the optimal cost?

7.2 What are the optimal feedback and the optimal return for the system

x = u

with the cost 

1 4
(x + u2 )dt
0 2
Is the optimal return function real analytic?

7.3 Compute the optimal feedback up to third order terms for the system and
criterion which are given by

x 1 = x2 , x 2 = u

 2 
J= x1 + x41 + u2 dt
0
Hint: Maple, Mathematica.

7.4 Consider a constrained linear quadratic control problem with a scalar u:


J= (xT Qx + u2 ) dt
0
x = Ax + Bu, |u| 1

Let u = Lx be the solution to linear quadratic problem without the constraint

|u| 1. Then it is natural to try the control law

u = sat Lx

115
where sat is the function given by


1 z>1
sat z = z |z| 1


1 z < 1

a. Show that this control law is indeed optimal if x is a scalar.

b. Show that the control law is non-optimal in general.

116
Chapter 8

Harmonic analysis of
nonlinear systems

Linear systems are often analyzed by considering their response to sinusoidal

signals. For nonlinear systems such an analysis is more complicated since there
is no superposition principle. However it is still possible to get some feeling for
the system properties from such an analysis. We begin by considering a simple
example.

Example 8.1 Consider the following nonlinear control system.

r -
+ m e- u- 1 y
()3 -
(s + 1)2
6

where the nonlinearity u = e3 is followed by the linear system

1
G(s) =
(s + 1)2

In gure 8.1 is shown the step response for a unit step

r(t) = 1, t0

in the reference signal. We see that there is a large steady state error due to
the low gain of the cubic nonlinearity at low amplitudes. If the reference signal
is instead
r(t) = 1 + 1.5 sin 10t, t 0 (8.1)

the response is the one shown in gure 8.2. We see that the steady state error
has decreased dramatically due to the presence of a high frequency component
in r. Can we explain this phenomenon?

117
 1

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0
0 5 10 15 20 25 30

Figure 8.1: Output y(t) for a unit step in the reference.

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0
0 5 10 15 20 25 30

Figure 8.2: Output y(t) for step plus sinusoid.

118
One approach would be to assume that e is a sum of a constant and a sinusoid.
Using a complex representation we write
e(t) = e0 + e1 eit + e1 eit (8.2)
Since e(t) is real we have e1 = e1 . We then get
 3
u(t) = e0 + e1 eit + e1 eit = u0 + u1 eit + u 1 eit + u2 ei2t + u
2 ei2t +
3 ei3t
+u3 ei3t + u
where
u0 = e30 + 6e0 |e1 |2 , u1 = 3e20 e1 + 3e1 |e1 |2 , u2 = 3e0 e21 , u3 = e31
The output is then given by
u1 eit + G(2i)u2 ei2t +
y(t) = G(0)u0 + G(i)u1 eit + G(i)
u2ei2t + G(3i)u3 ei3t + G(3i)
+G(2i) u3 ei3t
Since e = r y, the signal e will contain terms with frequencies 2 and 3. Our
assumption (8.2) is then false. However, if is large enough, then the absolute
values of G(2i) and G(3i) will be small. The assumption (8.2) will then be
approximately true. Writing the reference signal in the form
r(t) = r0 + r1 eit + r1 eit
we get
e0 = r0 G(0)u0 , e1 = r1 G(i)u1
It is convenient to introduce the gains
u0 u1
Y0 (e0 , e1 ) = = e20 + 6|e1 |2 , Y1 (e0 , e1 ) = = 3e20 + 3e21 (8.3)
e0 e1
We then get
r0 r1
e0 = , e1 = (8.4)
1 + Y0 (e0 , e1 )G(0) 1 + Y1 (e0 , e1 )G(i)
These formulas look supercially like the usual formulas for the gain from refer-
ence signal to error signal for a linear system. The big dierence lies in the fact
that Y0 and Y1 are amplitude dependent gains. Note that each gain depends on
both e0 and e1 . It is this fact which is the clue to the behavior of the system.
Substituting (8.3) into (8.4) gives the following system of equations.
e0 (1 + G(0)(e20 + 6|e1 |2 )) = r0
(8.5)
e1 (1 + 3G(i)(e20 + e21 )) = r1
Since |G(i)| = |G(10i)| 0.01 we get approximately
e1 r1
and substituting into the rst equation
r0 r0
e0 2
=
1 + 6G(0)|r1 | 1 + 6|r1 |2
Since we had r0 = 1 and r1 = 0.75 in (8.1), we get
e0 = 0.23
in good agreement with Figure (8.2).

119
8.1 Describing functions
Generalizing the methodology of Example 8.1 we consider the following situa-
tion.

c(t)- z(t)-
N

A nonlinear system N has an input c(t) and an output z(t). Assume that the
input is a sum of sinusoidal functions:

c(t) = C1 sin(1 t + 1 ) + + CN sin(N t + N )

Using a complex representation we can write


N
 ij t 
c(t) = cj e + cj eij t (8.6)
j=1

If the output is written in the form


N
 ij t 
z(t) = zj e + zj eij t + w(t) (8.7)
j=1

where w(t) contains frequencies other than 1 , . . . , N we can dene the gain
for the frequency j :
zj
Yj (c1 , . . . , cN , 1 , . . . , N ) = (8.8)
cj
The function Yj is sometimes called the j:th describing function for N . Using
the gain Yj it is in principle possible to do block diagram calculations in the
same manner that one does for linear systems. We saw this in Example 8.1.
There are two main diculties with this approach.

In contrast to the linear case, w will in general be nonzero in steady state,

that is new frequencies are generated. If N is part of a feedback loop,
these new frequencies will come back at the input. If the new frequency
components are included in c, still new frequencies will be generated and
so on. Exact calculations will thus require that c contains innitely many
frequencies. To get around this problem one usually assumes that there
are linear elements that dampen the frequencies one does not want to
consider. In Example 8.1 we did this for the frequencies 2 and 3.
In general each Yj depends on all the frequencies j and all the amplitudes
cj . If N is a static nonlinearity the dependence on the frequencies will
dissappear as in Example 8.1. However the dependence on all amplitudes
remains. This means that signal components at dierent frequencies are
coupled, which complicates the calculations.

120
In practice describing function calculations have been conned to the following
special cases. They cover many applications however.

1. N is in fact linear with transfer function G(s). Then trivially

Yj = G(ij )

2. The signal c(t) has zero mean and contains a single frequency, that is
C it C it
c(t) = C sin t = e e
2i 2i
If the nonlinearity is static, the single describing function Y depends only
on C. This is the classical approach and sometimes the term describing
function method is used for this special case only.
3. The input signal c(t) has nonzero mean and contains a single frequency
(apart from = 0), that is
C it C it
c(t) = B + C sin t = B + e e
2i 2i
This case is sometimes referred to as the BSDF (bias-plus-sinusoid de-
scribing function) case.
4. The signal c(t) has zero mean and contains two frequencies 1 and 2 ,
that is
c(t) = C1 sin 1 t + C2 sin(2 t + )
This is sometimes referred to as the SSDF case (sinusoid-plus-sinusoid
describing function)

8.1.1 The classical describing function

As discussed above, the classical approach assumes a single sinusoid without

bias. Usually the nonlinearity is assumed to be static so that

z(t) = f (c(t)) (8.9)

The classical approach also assumes an odd nonlinearity, that is

f (x) = f (x) (8.10)

The reason is that nonlinearities without this property usually produce a bias
at the output. If the nonlinearity is part of a feedback system, this bias is fed
back to the input, making the assumption of zero bias invalid. If the static
nonlinearity (8.9) is used with the input
C it C it
c(t) = C sin t = e e (8.11)
2i 2i
the output becomes a periodic function with a Fourier expansion



z(t) = zk eikt = zk eik

121
where = t. The Fourier coecient of the bias term is

z0 = f (C sin )d = 0

because of (8.10). The coecient of the -component is

  
1 1 1
z1 = f (C sin )ei d = f (C sin ) cos di f (C sin ) sin d
2 2 2

Since the cosine integral is zero because of the symmetry (8.10), the describing
function becomes

z1 1
Y (C) = C = f (C sin ) sin d
( 2i ) C

which is real valued. Since each quarter of the interval [, ] makes the same
contribution in the integral, we can also write
 /2
4
Y (C) = f (C sin ) sin d (8.12)
C 0

This formula can be rewritten in many ways. Making the substitution C sin =
x, for instance gives
 C
4 xf (x)
Y (C) = dx (8.13)
C 2 0 C 2 x2
Using a partial integration, this can then be written
 C 
4
Y (C) = f  (x) C 2 x2 dx (8.14)
C 2 0

Example 8.2 The describing function for a relay with dead zone

6
H

D D -

is 
4 C  4H  2
Y (C) = H (x D) C 2 x2 dx = C D2
C 2 0 C 2

Nonlinearities with hysteresis

A nice feature of the classical describing function method is that it is one of the
few methods that can handle nonlinearities that are not single valued. Consider

122
 2 (x) 6
f'

- x

f%
1 (x)

Figure 8.3: Nonlinearity with hysteresis

for instance a nonlinearity with hysteresis as in Figure 8.3. We still consider

nonlinearities with an odd symmetry:

f1 (x) = f2 (x) (8.15)

The describing function is then given by

*  /2 +
/2
i i i
Y (C) = f1 (C sin )e d + f2 (C sin )e d (8.16)
C /2 /2

Introducing the mean value

f1 (x) + f2 (x)
f0 (x) =
2
and half the dierence
f2 (x) f1 (x)
(x) = = f2 (x) f0 (x) = f0 (x) f1 (x)
2
we see that f0 and  are odd and even functions respectively:

f0 (x) = f0 (x), (x) = (x) (8.17)

Replacing f1 and f2 in (8.16) by f0 and , it is easy to see that the describing

function is given by
*  /2 +
/2
4
Y (C) = f0 (C sin ) sin d i (C sin ) cos d (8.18)
C 0 0

Making again the substitution x = C sin we get the alternative expression

*  C +
4 C 
 2 2
Y (C) = f0 (x) C x dx i (x) dx (8.19)
C 2 0 0

Note that the imaginary part of Y is just

A
C 2
where A is the area enclosed by the hysteresis loop.

Example 8.3 Consider a relay with hysteresis:

123
 6
H

D -
D
H

The function f0 equals the nonlinearity of Example 8.2 and the area enclosed
by the hysteresis loop is 4HD. The describing function is thus
4H  2 4HD
Y (C) = C D2 i
C 2 C 2

8.1.2 Describing functions for bias plus sinusoid

Now consider an input signal of the form

c(t) = B + C sin t = B + C sin

and a system N described by a nonlinear function f (not necessarily possessing

any symmetries). The output signal is then

z(t) = f (B + C sin ) = z0 + z1 ei + z1 ei +

The describing function has two components


z0 1
Y0 (B, C) = = f (B + C sin ) d (8.20)
B 2B
and

z1 1
Y1 (B, C) = C
= f (B + C sin )(sin + i cos ) d (8.21)
2i
C

If we want to cover systems with hysteresis as in Figure 8.3 we can still dene
f1 (x) + f2 (x)
f0 (x) =
2
f2 (x) f1 (x)
(x) = = f2 (x) f0 (x) = f0 (x) f1 (x)
2
However, since we do not assume any symmetry of f1 and f2 , we will no longer
obtain the symmetries (8.17). Partitioning the integration intervals of (8.20)
and (8.21) in a suitable manner we get
 /2
1
Y0 (B, C) = f0 (B + C sin ) d (8.22)
B /2
  /2
/2
2
Y1 (B, C) = f0 (B + C sin ) sin d i (B + C sin ) cos d
C /2 /2
(8.23)

124
Making the same variable changes as in (8.12,8.13) we get the alternative for-
mulas  C
1 f0 (B + x)
Y0 (B, C) = dx (8.24)
B C C 2 x2
  C
C
2 xf0 (B + x)
Y1 (B, C) = dx i (B + x) dx (8.25)
C 2 C C 2 x2 C

or
 
2 C  C
Y1 (B, C) = f0 (B + x) C 2 x2 dx i (B + x) dx (8.26)
C 2 C C

Example 8.4 Consider the relay with hysteresis of Example 8.3. Let the input
have a bias, that is
c(t) = B + C sin t
with C |B| > D (this ensures that the whole hysteresis loop is covered). We
get from (8.24)
  C
BD
1 H H
Y0 (B, C) = dx + dx =
B C C 2 D2 B+D C 2 D2
    
H D+B DB
= arcsin arcsin
B C C
Since
f0 (x) = H((x + D) + (x D))
we get from (8.26)
,
 2 ,  2
2H D+B DB
Re Y1 (B, C) = 1 + 1
C C C

 B+D
2 4HD
Im Y1 (B, C) = H dx =
C 2 BD C 2

8.2 Analysis of systems using the describing func-

tion
In Example 8.1 we already saw the basic idea of describing function analysis.
The various signals are traced around the loop, resulting in a system of equa-
tions. We give another example.

Example 8.5 Consider the temperature control system of Figure 8.4. The
control signal is the heating power u which can either be switched on (with
power 2P ) or o (zero power) by a relay, depending on the dierence between
the setpoint temperature r and the actual temperature y. The heated object

125
r -e - 6 2P u - 20 y

+  D D- (s + 1)2

6

Figure 8.4: Temperature control system

r - e - D 6 -P ?

+
u - 20 y
-
+  P D +  (s + 1)2

6

Figure 8.5: Modied block diagram

126
 0

-0.2

-0.4

-0.6
C =5 C =1
-0.8

-1

-1.2 =3
-1.4

-1.6

-1.8

-2
-5 -4.5 -4 -3.5 -3 -2.5 -2 -1.5 -1 -0.5 0

Figure 8.6: 1/Y1 as a function of C for B = 0 (straight line) and B = 0.5

(curved line), together with the Nyquist curve of 1/(s + 1)2

is assumed to consist of two equal time constants. To get a symmetric relay

we can rewrite the system with an external signal as in Figure 8.5. We assume
that the reference signal is constant r = r0 , but because of the relay we assume
that an oscillation exists in the loop. As usual we hope that harmonics will be
damped by the linear system. We therefore try
C it C it
e = B + C sin t = B + e e
2i 2i
We then get, tracing the signals around the loop.
B = r0 G(0)(P + Y0 (B, C)B)
C C
= 0 G(i) Y1 (B, C)
2i 2i
We thus get the following system of equations
B + Y0 (B, C)G(0)B = r0 G(0)P
(8.27)
Y1 (B, C)G(i) = 1
Solving this system of equations (if possible) we will get the approximate average
level, as well as oscillation amplitude and frequency. The second equation of
(8.27) can be interpreted as the intersection of 1/Y1 and the Nyquist curve
G(i). This is shown in Figure 8.6 for the case P = 1, D = 1. We see that the
frequency and amplitude of the oscillation do not vary drastically for reasonable
changes in B. Since
    
1 1+B 1B 2
Y0 = arcsin arcsin
B C C C2 1
for small values of B and C 1.8 we get Y0 0.4, and consequently
B 0.1(r0 20), C 1.8, 3.5 (8.28)
for values of r0 that are not too far from 20.

127
 D 6
D- 

?
c - - 1 z-

+  s

Figure 8.7: Limited integrator

8.3 Some design ideas based on the describing

function
The describing function can be used to design regulators in a manner similar
to classical Bode-Nyquist-Nichols techniques. This is because the describing
function of the nonlinearity is treated as a transfer function (although amplitude
dependent). Looking at the describing function as a parameter that lies between
certain bounds, it is also possible to use robust design methods. We will here
look at some examples where the describing function has inspired nonlinear
compensation techniques.

8.3.1 Modied integrators

Most controllers contain integral action. The integrator gives high gain at low
frequencies at the cost of a negative phase shift. In applications there is a limit
to the signal level from the integrator which is needed or useful. If the output
of the integrator is run through a saturation, the negative phase shift remains.
Therefor a circuit like the one in Figure 8.7 is sometimes used. If the gain of
the linear part of the dead zone is high enough, the output of the integrator will
never rise much above D. If the input is

c(t) = C sin t

the output will then be

C
(1 cos ) D 0 0
z(t) = (8.29)


D 0 <

where = t and 0 is given by

2D
cos 0 = 1
C

128
 3

log(abs(Y))
0

-1

-2

-3
-3
-2
-1
0 0
0.5
1
1
2
1.5
3 2
log(w) C

Figure 8.8: log |Y | as a function of C and log() for a limited integrator.

The expression (8.29) is valid if

C D
If C < D there is no limitation and the output is simply
C
z(t) = cos

A calculation of the Fourier coecients of (8.29) gives the describing function
Y (C, )
1
= i , if < D

1
= (4(2D ) cos 0 + 3 + cos 20
2C
if D
i (4(2D ) sin 0 + 20 + sin 20 ))
(8.30)
where = C/. The amplitude and phase diagrams of the describing function
(8.30) are shown in Figures 8.8 and 8.9 for D = 1.

8.3.2 Dither techniques

Looking at Example 8.5, equation (8.28), we see that the control system actually
acts like a linear system in B for small values of the bias. This is despite the fact
that the relay is a highly nonlinear component. The reason is the presence of
the oscillation with amplitude C. Its eect is to produce an average of the relay
gain around the hysteresis loop, which turns out to be almost linear for the bias
component. In the temperature control system the oscillation was produced as
a by-product of the relay action in the control loop. However it is also possible
to achieve similar eects by introducing external signals. Consider the following
situation

129
 0

-0.5

arg(Y)
-1

-1.5

-2
-3
-2
-1
0 0
0.5
1
1
2
1.5
3 2
log(w) C

Figure 8.9: arg(Y ) as a function of C and log() for a limited integrator.

c -?
+
- f -
+ 

The signal d is periodic with a frequency which is high compared to the frequency
content of c. If d is a sine:
d(t) = C sin t

and c is slowly varying compared with d, then we can regard c as a constant

and use the analysis of section 8.1.2. The output will then be of the form

z(t) = Y0 (c(t), C)c(t) + w(t)

where Y0 is a describing function of the nonlinearity f . The signal w will

typically contain the frequencies , 2,... If the rest of the system acts as a low
pass lter, those frequencies will dissappear and the signal d has in fact changed
the nonlinearity from
z(t) = f (c(t))

to
z(t) = Y0 (c(t), C)c(t)

The signal d is often called a dither signal.

Example 8.6 Let the nonlinearity be a dead zone

130
 6
D -
D

If the slopes of the linear parts are 1, then (8.24) gives

  
C 2 2
Y0 = 1 + 1 1 + + arcsin + + arcsin
B

where = (B D)/C and + = (B + D)/C. If C is large enough, then

2D
Y0 1
C

which is independent of B, so that the nonlinearity is linearized by the dither

signal.

8.3.3 Compensation of nonlinearities

Consider a linear control system

r -
+ m e- u- y
F (s) G(s) -

6

where F is the controller and G is the controlled physical system. Suppose that
the physical system actually contains a nonlinearity and is described by

- G1 (s) - N - G2 (s) -

where G1 G2 = G. An old idea for compensating the nonlinearity is to feed

back a model of the system inside the regulator. The regulator structure then
becomes

131
 e -
+ m - - u
F (s)

6

G2 
N  G1 

Ideally the nonlinearity N is such that

=1
N +N (8.31)
Assume that there is no bias and that higher harmonics are suciently damped
and N can be replaced by single
by the linear parts. Then the nonlinearities N
describing functions Y and Y respectively. The regulator is then given by
F
F =
1 + F GY
and the transfer function from r to y becomes
F GY F GY
Gc = = (8.32)
1 + F GY 1 + F G(Y + Y )

If it is possible to achieve (8.31), then Y + Y = 1 and we get

FG
Gc = Y
1 + FG
which is the closed loop transfer function of the linear system in series with Y .
In particular the stability properties (determined by 1 + F G) are not aected
by the nonlinearity in this ideal case. In the more realistic case when it is not
possible to achieve (8.31) exactly, the eects can be analyzed using (8.32).

8.4 Accuracy of the describing function method

Since the describing function method is based on an approximation there is
always some uncertainty concerning conclusions that are based on it. There
are some possibilities of being precise about the uncertainty by estimating the
inuence of the higher order harmonics. Consider the situation described in
gure 8.10. There is no external input and we use the describing function to
try and predict the presence of a self-sustained oscillation. Assuming
e = C sin t (8.33)
and following the signals around the loop we get
G(i)Yf (C) = 1 (8.34)
where Yf is the describing function of f , computed from one of the formulas
(8.12) (8.14). In reality e would have the form
e = C sin t + eh (8.35)

132
 + m e-
0 - u- y
f () G(s) -

6

Figure 8.10: System structure for estimation of describing function accuracy.

where eh contains the higher harmonics. Our approximation lies in neglecting

eh when computing Yf . We will analyze the approximation under the following
assumptions.

1. The nonlinearity f is odd:

f (e) = f (e) (8.36)

2. The nonlinearity f is single-valued and lies between lines with slopes

and :
e2 ef (e) e2 (8.37)

3. An oscillation has only odd harmonics. (This is natural considering the

conditions on f .)

From the condition (8.37) on f it is easy to see that the describing function
satises
Yf (C) (8.38)
We dene the average gain, f0 , and the deviation from the average gain, r:
+
f0 = , r= (8.39)
2 2
Since our intuitive motivation for the describing function technique is based on
the assumption that the harmonics are attenuated by G, it is natural to try
to estimate the extent of the attenuation. This can be done by calculating the
gains
G(3i), G(5i), G(7i), . . .
for a given ) (remember that we only consider oscillations with odd harmonics).
Dene the quantities
- -
- 1 -
-
k () = - + f0 -- , k = 3, 5, 7, . . . (8.40)
G(ki)
() = inf(3 (), 5 (), . . .) (8.41)

For those values of for which () > r we dene

r2
() = (8.42)
() r

It is now possible to state conditions for not having an oscillation.

133
Theorem 8.1 Consider a system with the structure given by Figure 8.10 whose
nonlinearity satises (8.36) and (8.37). If
- -
- 1 -
- -
- G(ki) + f0 - > r, k = 1, 3, 5, . . . (8.43)

then there is no periodic solution with fundamental frequency and only odd
harmonics.
If the distance from every point of Yf (C) to 1/G(i) is greater than (),
then there is no periodic solution with fundamental frequency and only odd
harmonics.

Proof. See Khalil, [?].

The conditions have a graphical interpretation. Equation (8.34) can be rewritten

1
+ Yf (C) = 0 (8.44)
G(i)
so the describing function solution is obtained as the crossing of the inverse
Nyquist plot 1/G(i) by the describing function locus Yf (C). For the class of
functions we are considering the locus lies on the negative real axis between
and . Figure 8.11 shows the two situations where no oscillation can take place
for a certain . To get conditions that guarantee an oscillation to take place,

1/G(i)

f0 r Yf
3

5 1/G(i)

Figure 8.11: Two situations when no oscillation can take place for a certain .

one has to look at the uncertainty band around 1/G(i) dened by the circles
of radius (). It is the possible intersection of this band with the Yf -locus
that is important. The geometrical situation is shown in Figure 8.12. Let C1
and C2 be the amplitudes for which the uncertainty bounds intersect Yf (C).
Let 1 and 2 be the frequencies corresponding to uncertainty circles which are
tangent to Yf from above and below. Together they dene an uncertainty
rectangle in the amplitude-frequency space:

= {(, C) : 1 < < 2 , C1 < C < C2 } (8.45)

We also dene o and Co to be the frequency and amplitude for the nominal
intersection between 1/G and Y , i.e. the values that the describing function
method would give us. The basic result is then as follows.

134
 2
C2
C1

(o , Co )
1/G(i)

Figure 8.12: Intersection of 1/G and its uncertainty band with the locus of Yf .

Theorem 8.2 Consider a system with the structure given by Figure 8.10 whose
nonlinearity satises (8.36) and (8.37). Let there be an uncertainty band which
intersects the locus of Yf (C) in such a way that the set of (8.45) is well
dened. Let there be a unique intersection of 1/G(i) and Yf (C) in at o ,
Co . Assume that
dYf dImG(i)
(Co ) = 0, (io ) = 0
dC d
Then there exists a periodic solution, having odd harmonics only, of the form

e(t) = C sin t + eh (t)

and
where , C belong to
  2
2/ ()C
eh (t)2 dt
0 r

Proof. See Khalil, [?].

Example 8.7 Consider the system

+ m e-
0 - u- 3 y
f -
s(s + 1)2
6

where f is a saturation:

e |e| 1
f (e) = 1 e>1


1 e < 1
The describing function is
*
2 1 1
(arcsin C + C 1 C 2 ) C > 1
Yf (C) =
1 C 1

135
 0.6

0.4

0.2

0.2

0.4

0.6

0.8
1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0

Figure 8.13: 1/G(i) for = 0.7, 0.75, 0.8, . . . , 1.5 with error circles. The solid
line is Yf .

In Figure 8.13 the inverse Nyquist curve is plotted together with Yf and some
error circles. The nominal solution is o = 1 and Co = 1.8. We see that 1
0.95 and that 2 is slightly less than 1.05. The uncertainty bounds intersect
the real axis at 0.59 and 0.72 corresponding to C1 = 1.6 and C2 = 2.1. We
conclude that there exists a periodic solution whose fundamental component
has

0.95 1.05, 1.6 C 2.1

The nominal solution is o = 1, Co = 1.8. A simulation is shown in Figure 8.14.

8.5 Exercises

8.1 Compute the describing functions for bias plus sinusoid for a piecewise
linear system:

136
 2

1.5

0.5

0.5

1.5

2
0 5 10 15 20 25 30 35 40 45 50

Figure 8.14: Simulation of periodic solution.

6
-




Assume the slopes to be k1 and k2 respectively.

8.2 Compute the describing function for a sinusoid without bias when the
nonlinearity is a relay with dead zone and hysteresis:

6
H

D1 D2 -
D2 D1
H

8.3 Consider the temperature control system of Example 8.5. Suppose the
nonlinearity is a relay with dead zone and hysteresis as described in the previ-
ous exercise. In which way is the system behavior altered? What happens if
additional time constants are introduced in the linear system?

8.4 Consider the system

137
 r -
+ m e- u- 1 y
f -
s(s + 1)
6

where f is a saturation:

6
-

Let r be a sinusoid without bias. Compute the gain from r to e as a function

of frequency and amplitude. What happens if r has a bias?

8.5 Consider the system

+ m e-
0 - u- y
f 2 -
s(s2 +s+1)

6

where f is a saturation. Compute the amplitude and frequency of any periodic

solution together with error bounds.

8.6 Consider a nonlinearity with dither signal:

c -?
+
- f -
+ 

Suppose the dither signal is a sawtooth signal and the nonlinearity an ideal relay
(that is without dead zone or hysteresis). Show that the eective nonlinearity is
a saturation. How do the parameters of the saturation depend on the parameters
of the sawtooth signal?

138
.

139
Chapter 9

Tracking and disturbance

rejection

To follow a reference signal and reject disturbances is typically the main task
of a control system. When nonlinearities are present this requires some extra
care.

9.1 A simple nonlinear servo problem

Consider the simple system of Figure 9.1. If we assume the reference signal to
be constant, the control system is described by the following equations

x = x3 + Ke (9.1)
r = 0 (9.2)
e=rx (9.3)

We see that the description contains three parts, a model of the system with
controller (9.1), a model for the reference signal (9.2) and a description of the
error signal (9.3). We would like the error to be zero, at least in the steady
state. However we get
(r e)3 = Ke (9.4)

r -
+ m e- u- x-
K x = x3 + u

6

Figure 9.1: Control system with P-controller

140
 - ()3

r - e ?
+ m-
K - mu- x = x3 + u x-

6

Figure 9.2: Control system with P-controller and feed-forward.

for the steady state. If K is large enough we get

r3
e (9.5)
K
It is not surprising that we get a steady state error, since that is what we would
get also in the linear case. The nonlinearity makes itself felt from the fact that
the error is proportional to the reference signal cubed, rather than the reference
signal itself. If we want to remove the steady state error, we can introduce a
feed-forward compensation as in gure 9.2. The system equations are now
x = x3 + Ke + r3 (9.6)
r = 0 (9.7)
e=rx (9.8)
In equilibrium (x = 0) we have
0 = (r e)3 + Ke + r3 (9.9)
which reduces to
(K + 3r2 )e 3re2 + e3 = 0 (9.10)
The only solution is e = 0, so the tracking error is zero in steady state.

9.2 General problem formulation

Let us now place the problem mentioned in the preceding section in a general
framework. Suppose we have a control problem where there are both reference
signals to track and disturbance signals to reject. We can group these signals
together in a vector v(t) (dimension q). We assume that we have a model of the
reference and disturbance signals:
w = g(w) (9.11)
This model is often referred to as an exosystem. (In our example of the previous
section the model was just r = 0.) The system to be controlled is given by a
state space description
x = f (x, u, w) (9.12)

141
with input u (m-vector) and state x (n-vector). We assume that the control
objective can be expressed by an error signal e (p-vector):

e = h(x, w) (9.13)

The total system description is then of the form

x = f (x, u, w)
w = g(w) (9.14)
e = h(x, w)

Compare this with (9.1), (9.2), (9.3). To simplify calculations, we assume all
the functions f , g and h to be innitely dierentiable. In addition we postulate
that
f (0, 0, 0) = 0, g(0) = 0, h(0, 0) = 0 (9.15)
that is, the origin is an equilibrium point of the system for u = 0.
We want a control conguration that achieves the following objectives, the
Global Tracking Problem.

The closed loop system is globally exponentially stable when w = 0.

The error goes to zero:

lim e(t) = 0, for all x(0), w(0) (9.16)

t

However we will have to relax the requirement and be content with solutions
that are guaranteed to work locally. There are many possible controller cong-
urations, but we will look at the following two typical cases.
Pure state feedback
u = k(x, w) (9.17)
Note that the feedback is from all states, including w. This means that we
are actually using a controller that has feed-forward from the reference and
disturbance signals.
Pure error feedback
u = k(), = m(, e) (9.18)
Here we assume that the error is the input to a dynamic controller, which is the
classical control conguration.
What assumptions should we make about the exosystem? In linear control
theory one often assumes that the exosystem has all its eigenvalues on the
imaginary axis or in the right half plane. This is because modes of the exosystem
that decay to zero are trivial, since they do not contribute to the error as time
goes to innity. On the other hand it is natural to allow instability, since
one might be interested in tracking signals that are growing, like ramps. For
nonlinear systems however, unbounded signals are dicult to handle, so we will
assume that the exosystem is stable. To ensure that the exosystem is dicult
enough to require some kind of control action asymptotically, we will introduce
the following notion.

142
Denition 9.1 Consider a dynamic system
w = g(w), w(0) = w0 (9.19)
with solution w(t) = (t, w0 ). An initial value w0 is said to be Poisson stable
if, for every neighborhood U of w0 and every T > 0 there exist t1 > T and
t2 < T such that (t1 , w0 ) U and (t2 , w0 ) U .

Poisson stability of a state thus means that the solution comes back almost to
the same point, innitely many times. Summarizing, we make the following
assumption.
Assumption I. The origin is a stable equilibrium of the exosystem, and there
is a neighborhood of the origin, where every w is Poisson stable.
A system satisfying Assumption I is sometimes said to be neutrally stable. Let
us now try to solve the problem for the feedback case, by looking at the lin-
earizations at the origin. The local description of the system is
x = Ax + Bu + F w + 1 (x, u, w)
w = Gw + (w) (9.20)
e = Cx + Dw + 2 (x, w)
where 1 , 2 and are functions that vanish at the origin, together with their
rst order derivatives, and
A = fx (0, 0, 0), B = fu (0, 0, 0), F = fw (0, 0, 0), G = gw (0) (9.21)
C = hx (0, 0), D = hw (0, 0) (9.22)
If we assume a feedback
u = k(x, w) = Kx + Lw + 3 (x, w) (9.23)
then the dynamics of the closed loop system becomes
    
d x A + BK BL + F x
= + (x, w) (9.24)
dt w 0 G w
where is a function that vanishes at the origin together with its Jacobian.
Let us consider the eigenvalues of the linear part. If the linearized system is
stabilizable we can choose K so that A + BK has all its eigenvalues in the left
half plane. The stability of the exosystem implies that G can have no eigen-
values with positive real part. The Poisson stability means that there can be
no eigenvalues with negative real part. The matrix G thus has all its eigen-
values on the imaginary axis. Systems with the property that the linearization
has some eigenvalues on the imaginary axis have received considerable intention
and there exists a collection of results, known as center manifold theory. We
will consider it in the next section.

9.3 Center manifold theory

Consider a system
x = Ax + f (x), f (0) = 0, fx (0) = 0 (9.25)

143
where x is an n-vector and f is a k times (k 2) continuously dierentiable
function. The linearization is then

y = Ay (9.26)

Let Es , Eu and Ec be the vector spaces spanned by eigenvectors of A corre-

sponding to eigenvalues with negative, positive and zero real parts respectively.
The space Es thus contains solutions of (9.26) converging to the origin, while
Eu contains unstable solutions of (9.26) that start at the origin. We expect so-
lutions of (9.25) to be in some sense similar to those of the linearization (9.26),
at least close to the origin. One can formalize this by looking at invariant man-
ifolds. A manifold is a subset that locally looks like an open subset of R for
d

some value of d. A manifold is invariant if solutions of (9.25) that start in the

manifold remain there, when going backwards or forwards in time. Typically a
manifold is described by an equation

g(x) = 0 (9.27)

where g is a function from R to R whose Jacobian is nonsingular. The

n m

dimension of the manifold will then be n m. With this terminology we can

describe the relation between (9.25) and (9.26) as follows.

Theorem 9.1 For the system (9.25) there exist invariant manifolds Ws (the
stable manifold), Wu (the unstable manifold) and Wc (the center manifold)
that are tangent to Es , Eu and Ec respectively at the origin.

In our control theory application of tracking and disturbance rejection we are

only interested in systems without any unstable manifold. If we assume the
linear part to be block diagonal we can write

y = Gy + g(y, z)
(9.28)
z = Hz + h(y, z)

where G is a matrix having eigenvalues with strictly negative real parts, and H
is a matrix whose eigenvalues all lie on the imaginary axis. The functions g and
h are zero at the origin together with their Jacobians. The center manifold can
then locally be described by a function

y = (z) (9.29)

Since a center manifold is tangent to Ec at the origin we get

(0) = 0, (0) = 0 (9.30)
z

The invariance gives the relation y = z z.
From (9.28) we then get

(z)
(Hz + h((z), z)) = G(z) + g((z), z) (9.31)
z
which is a partial dierential equation for from which we can in principle
calculate the center manifold.

144
 1

0.8

0.6

0.4

0.2

y
-0.2

-0.4

-0.6

-0.8

-1
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
x

Figure 9.3: Phase portrait for (9.32). The stable and center manifolds are solid
while various trajectories are dash-dotted.

Example 9.1 Consider the system

x = x + y 2
(9.32)
y = y 3 + x3

The subspaces Es and Ec are the x-axis and y-axis respectively. In Figure 9.3
the phase plane is shown. The stable manifold is shown as a solid curve and
is tangent to the x-axis as expected. All trajectories that do not start on the
stable manifold tend to approach the same curve asymptotically. This curve,
which is tangent to the y-axis is a center manifold. The equation for the center
manifold (9.31) becomes

 (y)(y 3 + (y)3 ) = (y) + y 2 (9.33)

From (9.30) it follows that close to the origin we have

(y) = y 2 + y 3 + O(y 4 ) (9.34)

Substituting this expression into (9.33) gives

(y) = y 2 + O(y 4 ) (9.35)

The center manifold is thus locally a parabola, symmetric about the positive
x-axis. Substituting the expression for into the system equations gives

y = y 3 + y 6 + O(y 8 ) (9.36)

as the dynamics on the center manifold. Solutions that start on the manifold
close enough to the origin will thus converge slowly to the origin.

In the example solutions seem to approach the center manifold rapidly. This
behavior is typical.

145
Theorem 9.2 Suppose y = (z) is a center manifold of (9.28) at the origin.
There is a neighborhood U of the origin and positive numbers K and C such
that, if y(0), z(0) is in U then

|y(t) (z(t))| CeKt |y(0) (z(0))| (9.37)

as long as (y(t), z(t)) is in U .

Denition 9.2 We call a system

x = f (x), f (x0 ) = 0, A = fx (0) (9.38)

exponentially stable at x0 if all eigenvalues of A have strictly negative real parts.

9.4 State feedback

We now return to the state feedback problem. We begin by a more careful
formulation of what we want to achieve. Consider the system

x = f (x, u, w)
w = g(w) (9.39)
e = h(x, w)

Denition 9.3 We will look at a local version of the global tracking problem
presented in section 9.2. The local state feedback tracking problem consists of
nding a controller
u = k(x, w) (9.40)
such that

1. The closed loop system

x = f (x, k(x, 0), 0) (9.41)

is locally (at the origin) exponentially stable.

2. All solutions of
x = f (x, k(x, w), w)
w = g(w) (9.42)
e = h(x, w)

starting suciently close to the origin, are such that

lim e(t) = 0 (9.43)

t

Now consider (9.24). We realize that there must exist a center manifold

x = (w) (9.44)

146
Since solutions starting on the center manifold remain there, we get

x = w w (9.45)

Substituting the dierential equations satised by x and w, we get

w (w) g(w) = f ((w), k((w), w), w) (9.46)

We can then show the following result.

Theorem 9.3 If the local state feedback tracking problem is solvable, then the
pair A, B is stabilizable and there exist functions c(w) and (w) satisfying

w (w) g(w) = f ((w), c(w), w) (9.47)

h((w), w) = 0 (9.48)

Proof. Equation (9.47) was shown in (9.46). To show (9.48) consider a point
(w0 , (w0 ) on the center manifold. For any  > 0 and any T > 0 we can, by
Assumption I, nd a t > T such that |w(t) w0 | < . Since is smooth, we
can actually nd a t > T such that

|w(t) w0 | < , |(w(t)) (w0 )| <  (9.49)

For any point on the center manifold the solution will thus come back arbitrarily
close to that point innitely many times. The only way of achieving the require-
ment e(t) 0 is then to have the error equal to zero on the center manifold,
that is
h((w), w) = 0 (9.50)
To get exponential stability we see from (9.24) that there has to be a K such
that A+BK has all its eigenvalues strictly in the left half plane. This is precisely
the denition of stabilizability for the pair A, B.

The conditions of the theorem are also sucient as shown by the construction
of the following theorem.

Theorem 9.4 Suppose there exist continuously dierentiable functions and

c, with (0) = 0, c(0) = 0 such that (9.47), (9.48) hold and assume that A, B is

stabilizable. Choose a function k(x) such that A + B kx (0) has all its eigenvalues
strictly in the left half plane. Then the control law
(w))
u = k(x, w) = c(w) + k(x (9.51)

solves the local state feedback tracking problem.

Proof. The rst property of Denition 9.3 is trivially satised since

(0)) = k(x)
k(x, 0) = c(0) + k(x (9.52)

The linearization at the origin of f (x, k(x, 0), 0) is (A + B kx (0), showing the
exponential stability. The closed loop system will then be given by
(w)), w)
x = f (x, c(w) + k(x
(9.53)
w = g(w)

147
The right hand side has the linearization
  
A + B kx (0) x
(9.54)
0 G w
(w)), w) gives f ((w), c(w), w), so
Replacing x with (w) in f (x, c(w) + k(x
the center manifold equation for (9.53) is precisely the rst equation of (9.47).
Equation (9.48) shows that the error is zero on the center manifold. Since Theo-
rem 9.2 shows exponential convergence to the center manifold we conclude that
the error decays exponentially, so property 2 of Denition 9.3 is also satised.

Example 9.2 Consider the simple control system of Section 9.1. The system
dynamics without the controller is
x = x3 + u
r = 0
e=rx
The center manifold equations are
0 = (r)3 + c(r), 0 = r (r) (9.55)
giving
(r) = r, c(r) = r3 (9.56)
Modifying the P-controller according to (9.51) gives
u = r3 + K(r x) (9.57)
which is precisely the compensation shown in Figure 9.2.

9.5 Error feedback

We will now dene the error feedback problem for the system
x = f (x, u, w)
w = g(w) (9.58)
e = h(x, w)
more precisely. We will assume that the regulator is a general nonlinear system
with state , input e and output u:
u = k()
(9.59)
= m(, e)
We then have the following problem to solve.

Denition 9.4 The local error feedback tracking problem consists of nding a
controller
u = k()
(9.60)
= m(, e)
such that

148
 1. The closed loop system
x = f (x, k(), 0)
(9.61)
= m(, h(x, 0))
is exponentially stable at the origin.
2. All solutions of
x = f (x, k(), w)
w = g(w)
(9.62)
= m(, h(x, w))
e = h(x, w)
starting suciently close to the origin, are such that
lim e(t) = 0 (9.63)
t

To formulate the conditions for solving the local error feedback tracking problem
it is useful to have the following concept.

Denition 9.5 The system

x = f (x) (9.64)
y = h(x) (9.65)
is immersed into the system
= f(
x x) (9.66)
x)
y = h( (9.67)
if there exists a smooth mapping x
= (x) such that
x f (x) = f( (x)) (9.68)
(x))
h(x) = h( (9.69)
for all x.

d
Equation (9.68) in the denition is another way of saying that dt (x) = f( (x)),
for a solution to (9.64) i.e. that solutions to the x-system are carried into
solutions to the x-system. Equation (9.69) shows that this is done in such a
way that the output is the same. The denition thus says that any output that
can be produced by (9.64), (9.65) can also be produced by (9.66), (9.67) by
taking x(0) = (x(0)).
We can now state the basic result.

Theorem 9.5 Consider the local error feedback tracking problem with assump-
tion I satised. It is solvable if and only if there exist functions (w) and c(w)
with (0) = 0, c(0) = 0 such that
w (w) g(w) = f ((w), c(w), w)
(9.70)
h((w), w) = 0

149
and such that the system
w = g(w)
(9.71)
u = c(w)

with output u is immersed into a system

= ()
(9.72)
u = ()

in which (0) = 0, (0) = 0, M = (0), K = (0) and where the pair

   
A 0 B
, (9.73)
NC M 0

is stabilizable for some N and the pair

 
A BK  
, C 0 (9.74)
0 M

detectable.

Proof. Necessity. If the problem is solvable then there exists a controller

= m(, e), u = k() (9.75)

such that for solutions of (9.62) the error goes to zero at least locally at the
origin. Dene M = m (0, 0), N = me (0, 0), K = k (0). Linearizing (9.62) at
the origin we get

x A BK F x
d
= N C M N D + (x, , w) (9.76)
dt
w 0 0 G w

where is a remainder term which is zero at the origin together with its Jaco-
bian. From the center manifold theorem it follows analogously to the proof of
Theorem 9.3 that there exist mappings x = (w), = (w) such that

w g(w) = f ((w), k((w)), w) (9.77)

w g(w) = m((w), 0) (9.78)

Dening c(w) = k((w)) we get the rst equation of (9.70) from (9.77). The
second equation of (9.70) is shown using the same argument as in the proof of
Theorem 9.3. Dening () = m(, 0) and () = k() is is seen from (9.78)
that (9.71) is immersed into (9.72).
From (9.76) it follows that the matrix
 
A BK
NC M

has all its eigenvalues strictly in the left half plane. Since
     
A BK A 0 B  
= + K 0
NC M NC M 0

150
it follows that    
A 0 B
,
NC M 0
is stabilizable and since
     
A BK A BK 0  
= + C 0
NC M 0 M N

it follows that  
A BK  
, C 0
0 M
is detectable.
Suciency. Since    
A 0 B
,
NC M 0
is stabilizable it follows that
   
A BK B
,
NC M 0

is stabilizable, and since  

A BK  
, C 0
0 M
is detectable, so is  
A BK  
, C 0
NC M
It follows that the linear system
   
A BK B  
z = z+ u, y= C 0 x
NC M 0

is both stabilizable and detectable. Consequently there is a dynamic output

feedback controller
+ By,
= A
u = C
which gives a closed loop system with all eigenvalues strictly in the left half
plane. (The controller could for instance be based on an observer and feedback
from the observer states.) The matrix

A BK B C
J = N C M 0 (9.79)

BC 0 A

then has all its eigenvalues strictly in the left half plane.
Now construct the following controller

= () + N e
+ Be
= A (9.80)
+ ()
u = C

151
The closed loop system is then described by

x A BK B C x
d
= N C M 0
+ higher order terms
dt BC
0 A
w 0 0 0 G w
The block matrix in the upper left hand corner is identical to the matrix J of
(9.79) which has all its eigenvalues strictly in the left half plane. The exponential
stability condition is thus satised. Let be the mapping from (9.71) to (9.72)
dened by the immersion. Then x = (w), = (w) denes the center manifold.
It follows from the second equation of (9.70) that the error is zero on the center
manifold. From Theorem 9.2 it follows that the error goes to zero as t goes to
innity.

Remark 9.1 The condition that (9.71) is immersed in (9.72) means that the
controller contains a model of the exosystem. This is clearly seen in the con-
troller construction (9.80). Sometimes this fact is referred to as the internal
model principle.

Remark 9.2 Our controller (9.80) is linear except for the terms and . This
is sucient to show that the tracking problem is solved locally. In practice
one would probably want to try a nonlinear controller to extend the region of
convergence.

Remark 9.3 It is easy to see that the pair (9.73) is stabilizable only when
A, B. Similarly detectability of (9.74) requires detectability of A, C. A necessary
condition for solving the local error feedback tracking problem is therefore that
the controlled system has a linearization which is stabilizable and detectable.

Example 9.3 Consider again the simple control system equations

x = x3 + u (9.81)
r = 0 (9.82)
e=rx (9.83)
where we now want to use error feedback. The center manifold equations (9.70)
are
0 = (r)3 + c(r), 0 = r (r) (9.84)
The solution for and c is
(r) = r, c(r) = r3 (9.85)
For e = 0, the control should thus be generated by
= 0, u = 3
We thus get M = 0, K = 0. It follows that the pair
   
A BK 0 0    
= , C 0 = 1 0
0 M 0 0
is not detectable. The problem is thus not solvable.

152
Example 9.4 Modify the previous example so that

x = x x3 + u (9.86)
r = 0 (9.87)
e=rx (9.88)

The center manifold equations (9.70) are

0 = (r) (r)3 + c(r), 0 = r (r) (9.89)

The solution for and c is

(r) = r, c(r) = r + r3 (9.90)

For e = 0, the control should thus be generated by

= 0, u = + 3

We thus get M = 0, K = 1. If we take N = 1, then the matrix

   
A BK 1 1
=
NC M 1 0

has already its eigenvalues in the left half plane, so we can take the controller

= e, u = + 3

9.6 Exercises
9.1 Calculate the center manifold of the system

x = x + y + y 2
(9.91)
y = x y

Is the dynamics on the center manifold stable or unstable?

9.2 A classical example of a system having a non-unique center manifold is

x = x2
(9.92)
y = y

Show that there are in fact innitely many center manifolds and compute them
all.

9.3 Consider Example 9.4 and take the linear part of the controller, i.e.

= e, u=

Compare the performance with the nonlinear controller, especially for large steps
in the reference signal. Discuss the result.

153
 9.4 For a linear plant with linear exosystem the conditions for solving the
tracking problem are a set of linear matrix equations. What are they?

9.5 Use (9.47) to give some examples of tracking/disturbance rejection prob-

lems that are impossible to solve.

9.6 Consider the system

x 1 = u
x 2 = x1 + v 3

where v is a sinusoidal disturbance, whose angular frequency is known but whose

amplitude and phase are unknown. It is desired to have x2 equal to zero, at
least asymptotically. Construct a controller.

154
Chapter 10

Physics based control

10.1 Lagrangian physics

The Lagrangian methos of modelling has its origin in mechanics but can be
extended to other systems, e.g. electrical and electro-mechanical ones. The
state vector is assumed to have the form
 
q
x=
q

for some set of variables q. In mechanics q typically consists of distances and

angles, so that q consists of velocities and angular velocities. The Langrangian
modeling technique also postulates the existence of a function L of the form

L(q, q) V (q)
= T (q, q) (10.1)

satisfying the equation

d T
L LTq = Q (10.2)
dt q
In mechanics T is the kinetic energy and V is the potential energy. We will
assume that the kinetic energy has the form
1 T
T (q, q)
= q D(q)q,
D(q) = DT (q) > 0 (10.3)
2
The vector Q is called the generalized force. In mechanics it consists of ordi-
nary forces (corresponding to components of q that are distances) and torques
(corresponding to angles in q). In control applications Q typically has the form

Q = F (q)
+ d + Bu (10.4)

where F is some generalized force generated by the motion (often some type
of friction), d is an external disturbance and u is the control. The generalized
force F is assumed to satisfy

0,
qT F (q) F (0) = 0 (10.5)

155
For control purposes (10.2) therefore takes the form
d T
L LTq = F (q) + d + Bu (10.6)
dt q
If the vectors u and q have the same size and B is nonsingular the system is said
to be fully actuated, while the case of fewer components in u than q is called
under-actuated.

Example 10.1 Consider a pendulum hanging from a cart.

u
M


q2
q1
m

The system is described by the position of the cart (q1 ) and the angle of the
pendulum (q2 ). The kinetic energy is
  
1  M + m m cos q2 q1
T = q1 q2
2 m cos q2 m2 q2
Here we have assumed the pendulum to be a point mass m at the end of the
rod. The potential energy is given by
V = mg cos q2
If we assume that F has the form
 
b 0
F = 1 q
0 b2
then the generalized force is
   
b1 0 1
Q= q + u
0 b2 0
Note that this is an under-actuated system.

To show that the Lagrangian formalism can be used also for non-mechanical
systems we consider an electrical circuit.

Example 10.2 A series circuuit is given by the following diagram

R1 (x2 ) L1

x2
u C1

156
Let q = x1 be the charge on the capacitor. Then q = x2 is the current into the
capacitor which is also the common current all through the series circuit. Now
T becomes the energy of the inductor
L1 2
T = q
2
while V is the energy in the capacitor
1 2
V = q
2C1
The Lagrangian is then
L1 2 1 2
L= q q
2 2C1
The generalized forces will correspond to voltages, composed of external signals
and dissipation, i.e. voltage drops across resistors. In this case one gets

Q = u R1 (q)

so that the dynamics is given by

d 1
(L1 q)
+ q = u R1 (q)

dt C1
and the state equations become

x 1 = x2
1
x 2 = (u x1 /C1 R1 (x2 ))
L1
in accordance with circuit theory.

Equilibria

An equilibrium of a Lagrangian system is solution of (10.6) for d = 0, u = 0

where q is constant, and consequently q is identically zero. It follows that T is
also identically zero so that (10.6) reduces to

Vq (q) = 0 (10.7)

The equilibria are therefore the stationary points of the potential function.

Passivity and stability

Systems satisfying Lagranges equations have several nice properties that are
related to the following result.

Theorem 10.1 For a system satisfying

d T
LTq (q, q)
L (q, q) = F (q)
+ Bu (10.8)
dt q

157
the following relation holds

d
= qT F (q)
H(q, q) + qT Bu (10.9)
dt
where H(q, q)
= T (q, q)
+ V (q) can be interpreted as the total energy.

Proof. Using the fact that Lq q = 2T one has

d d d d
H = (T + V ) = (2T T + V ) = (Lq q L) =
dt dt  dt  dt 
d d
= Lq q + Lq q Lq q Lq q = Lq Lq q = qT (F (q)
+ Bu)
dt dt

This result has a number of consequences.

Corollary 10.1 Dene the output to be y = B T q.

Then the system described
by (10.8) is passive.

Proof. Since V is assumed bounded from below, this is true also for H so that
H c for some constant c. From (10.9) then follows that
 T
+ H(x(T )) c
y T udt + H(x(0)) = qT F (q)
0

so that  T
y T udt + (x(0)) 0
0

with = H c.

Corollary 10.2 For a system (10.8) without external signals (u = 0) H is a

Lyapunov function.

Proof. From (10.9) it follows that d

dt H(q, q)
= qT F (q)
0

Corollary 10.3 For a system (10.8) without external signals (u = 0) assume

that the following holds, for some constants 1 > 0, 2 > 0.

The potential V is radially unbounded.

The kinteic energy satises T 1 qT q.

For the potential V there is precisely one point qo such that Vq (qo ) = 0

2 qT q.
The dissipative force satises qT F (q)

Then the equilibrioum q = qo , q = 0 is globally asymptotically stable.

158
Proof. From the rst two conditions follows that H is radially unbounded.
Using (10.9) gives
d
= qT F (q)
H(q, q) 0
dt
with equality only for q = 0. If q = 0 along a trajectory, then from (10.8) one
must have Vq (q) = 0 which implies that q = q0 . Theorem 5.1 is then applicable.

Control of fully actuated Lagrangian systems

For a fully actuated system the matrix B is square and invertible. It is then no
restriction to assume that B is a unit matrix so that the system dynamics is

d T
LTq (q, q)
L (q, q) = F (q)
+u (10.10)
dt q

Suppose one wants to control a Lagrangian system around a constant set point
qr . This means that qr has to be an equilibrium point for the closed loop system.
If one wants to keep the Lagrangian structure this means (according to (10.7))
that the potential has to be changed so that qr becomes a stationary point. This
can be done by dening a new potential W with a minimum at qr and using
the control
u = VqT (q) WqT (q)

The system dynamics then becomes

d T
L (q, q)
LTq (q, q)
= F (q)

dt q

where L = T W . A simple choice of W is W = 12 (qr q)T Kp (qr q) for some

positive denite matrix Kp . The control is then

u = VqT (q) + Kp (qr q)

To aect the convergence to the equilibrium the control can be extended to

u = VqT (q) + Kp (qr q) Kd q (10.11)

where Kd is a positive denite matrix. The closed loop dynamics is then

d T
L (q, q)
LTq (q, q)
= (F (q)
+ Kd q)

dt q

The eect of the Kd q-term

can thus be interpreted as an addition to the natural
friction term F . Since KD is positive denite the conditions of Corollary 10.3
will be met so that qr is a globally asymptotically stable equilibrium. Note that
the controller (10.11) that achieves this is a multivariable PD-controller with a
feedforward from the potential energy term.

159
10.2 Interconnected systems
Many engineering systems consist of interconneted simple systems. This fact is
reected by modeling concepts like bond graphs and is used in object oriented
modeling languages like Modelica. The Hamiltonian modeling techniques of
classical physics can be adapted to cover systems of this type and also give a
framework for design of controllers.

Storage elements

A basic idea is that energy is stored in simple elements that are linked together.
Each storage element is assumed to have the following properties.

There is a stored quantity x (e.g.electric charge in a capacitor).

The time derivative of the stored quantity is regarded as a ow f = x into
the component (e.g. electric current).
1 2
The stored energy is a function H(x) of the stored quantity (e.g. 2C x
for a capacitor)
There is dened an eort variable e = dH
dx (e.g. the voltage of a capacitor)

The power absorbed into the system is thus d

dt H(x) = dH
dx x
= ef

These properties are consistent with those of a C-element in bond graph theory.
For an I-element the roles of eort and ow are reversed.
Now assume that there are n storage elements and that each element is described
by a stored variable xi , a ow fi , an energy storage function Hi (xi ) and en eort
ei = dHi /dxi . We introduce the vectors

x1 f1 e1
.. .. ..
x = . , f = . , e = .
xn fn en

For simplicity it is assumed that the total energy H is just the sum of the enrgies
stored in the individual components (i.e. there are no phenomena like mutual
incuctances between components).

H(x) = H1 (x1 ) + + Hn (xn ) (10.12)

The connection of the dierent components is assumed to give a linear relation

f = Me (10.13)

for some matrix M . This is true in circuit theory (Kirchos laws) and for bond
graphs (p- and s-junctions). It is assumed that the interconnection itself does
not store, dissipate or generate energy. This means that the total power eT f
going into the interconnection has to be zero so that, for all e,

0 = eT f = eT M e = eT M T e eT (M + M T )e = 0 M = M T

160
i. e. the matrix M has to be skew-symmetric.
The interconnected systems we have described are described by
H H
x = f = M e = M Hx (x)T , Hx = ( ,..., ) (10.14)
x1 xn
with M skew-symmetric. Systems of this form are called Hamiltonian with
Hamilton function H. Since H = Hx x = Hx M HxT = 0 the total energy is
constant.
Now assume that some eorts and ows are not connected to storage elements
but are inputs and outputs. Partition the vectors and M as
     
ex fx Mxx Mxu
e= , f= , M=
eu fy Mux
T
0
where ex , fx are connected to storage elements, eu = u is the input and fy = y
is the output. The system description is then

x = Mxx ex + Mxu eu = Mxx HxT (x) + Mxu u (10.15)

y = fy = T
Mux HxT (x) (10.16)

10.3 Port controlled Hamiltonian systems

The systems described by (10.15) (10.16) are a special case of so called port
controlled Hamiltonian systems. In general they are systems that casn be written
in the following form.

x = J(x)HxT (x) + g(x)u

(10.17)
y = g T (x)HxT (x)

where J(x) is skew-symmetric. This system satises

d
H(x) = Hx x = Hx J(x)HxT (x) + Hx g(x)u = y T u
dt
showing that H is constant as long as u = 0. This means that systems having an
internal dissipation of energy can not be modelled. One way to model dissipation
is to consider a port controlled Hamiltonian system with two sets of inputs and
outputs:

x = J(x)HxT (x) + g(x)u + gR (x)uR

y = g T (x)HxT (x)
T
yR = gR (x)HxT (x)

The input uR and the output yR are then connected by some mathematical

relation uR = (x, yR ). Assuming this relation to have the form uR = R(x)y R

with R 0, the model is then

x = (J(x) R(x))HxT (x) + g(x)u

(10.18)
y = g T (x)HxT (x)

161

where R(x) = g(x)R(x)g T
(x) 0. A model of the form (10.18), where J is a
skew symmteric matrix and R a non-negative denite matrix, is called a port
controlled Hamiltonian system with dissipation. The structure of the system
immediately gives the following result.

Proposition 10.1 A system described by (10.18) is passive with the following

energy balance.
 T  T
y u dt + H(x(0)) H(x(T )) =
T
Hx RHxT dt 0 (10.19)
0 0

Proof. Follows from an integration of the equation

d
H = Hx x = Hx JHxT Hx RHxT + Hx gu = Hx RHxT + y T u
dt

Example 10.3 Consider again the electrical system of Example 10.2. If x1

denotes the charge on the capacitor, and x2 the magnetic ow in the inductor,
the energy functions are
x21 x22
H1 (x1 ) = , H2 (x2 ) =
2C1 2L1
The ows are
f1 = x 1 = current into capacitor, f2 = x 2 = voltage over inductor
while the eorts are
dH1 x1 dH2 x2
e1 = = , e2 = =
dx1 C1 dx2 L1
If we assume that the voltage drop over the resistor has the form r(e2 )e2 , the
relations between ows and eorts can be written
f1 = e2 , f2 = e1 r(e2 )e2 + u
leading to the system equations

       
0 1 0 0 Hx 1 0
x = + u
1 0 0 r H x2 1
% &' ( % &' (
J R

Example 10.4 Consider the mechanical system

v
u

x1

162
where the spring is nonlinear with spring force e1 = x1 + x31 and the damping is
linear with force bv. If the state variable x1 is the position, the spring energy is
x21 x4
H1 (x1 ) = + 1
2 4
Let the mass be m and dene x2 = mv. Then the kinetic energy is
x22 x2
H2 (x2 ) = , e 2 = Hx 2 = =v
2m m
The ow variables are

f1 = x 1 = v, f2 = x 2 = total force

with the relations

f1 = e2
f2 = u e1 be2

From these relations the following model can be deduced

           
d x1 0 1 0 0 x1 + x31 0   x1 + x31
= + u, y = 0 1
1 0
x2 x2
dt x2 0 b 1
% &' ( % &' ( % &' m
( m

J(x) R(x) HxT

where H = 12 x21 + 14 x41 + 12 x22

Using the Hamiltonian structure for control

There are basically two things you can do, using state feedback, if you want to
keep the Hamiltonian structure. The rst one is to change the energy function
H, the second one is to change the damping given by R. To change H one can
use a control u = k(x) + v satifying
T = g(x)k(x)
(J(x) R(x))H x (10.20)

The dynamics of (10.18) is then changed into

T + g(x)v
x = (J(x) R(x))(H + H)x

where the Hamiltonian is changed from H to H + H. To change R (or J) one

can use a state feedback u = k(x) + v, where k satises
R(x))H
(J(x) T
x = g(x)k(x) (10.21)

The system dynamics is then changed into:

R(x) R(x))H
x = (J(x) + J(x) T
x + g(x)v

The structures in equations (10.20) and (10.21) impose restrictions on what can
be achieved by state feedback while keeping the Hamiltonian structure. This is
seen in the following example.

163
Example 10.5 Consider again Example 10.4. Equation (10.20) becomes
x2 = 0
H
x1 = k(x)
H

showing that H has to depend only on x1 . It is also clear from the second
equation that the x1 -dependence of H can be chosen arbitrarily by choosing k
suitably. It is therefore possible to place a stable equilibrium (a minimum of H)
at any x1 -value. From (10.21) it is seen that R can be chosen to be of the form
 
0 0
0 b

giving the relation

x2
k(x) = b
m
The total control then has the form
x1 b x2
k(x) = H
m

164