MASSACHUSETTS INSTITUTE OF TECHNOLOGY

DEPARTMENT OF MECHANICAL ENGINEERING

2.151 Advanced System Dynamics and Control

Discrete Time Observers and LQG Control 1

The ZOH Plant Model: We consider a continuous LTI plant with state equations

x = Ax + Bu
y = Cx (1)

that is to be regulated by a discrete-time controller with sampling and command updates at

intervals T . In addition we assume that a zero-order hold (ZOH) at the controller output will
produce a piecewise constant command u(t) = u(kT ) for kT t < (k + 1)T , k = 0, 1, . . . .
The response of the system at time T will be
Z T
AT
x(T ) = e x(0) + eA(T ) Bu(0)d
0
" Z T #
AT AT A
= e x(0) + e e d B u(0)
0

= x(0) + u(0) (2)

R
where = eAT and = eAT 0T eA d are constant matrices. Equation (2) predicts the
response of the system at time T from the initial state and the zero-order input. This
solution may be repeated piecewise on each interval, using the prior solution as the initial
state, generating the difference equation description of the system:

x(k + 1) = x(k) + u(k)

y(k) = Cx(k) (3)

where the sampling interval T has been made implicit. The block diagram representation
is shown in Fig. 1. We note that, for a zero-order input with updates at intervals T , the
response of the discrete-time system in Eq. (3) will be identical to the continuous plant of
Eq. (1) at the sampling times.

u x x y
k + k + 1 k k
G z -1 C
+

Figure 1: ZOH discrete-time equivalent system

Taking the z transform of Eqs. (3) Gives the pulse transfer-function matrix

H(z) = C (zI )1 (4)

1
D. Rowell 12/13/04

1
Discrete-Time Controller Design: The properties of controllability and observability
transfer between the discrete and continuous representations. The discrete-time model may
therefore be used to design controllers for a controllable system described by Eqs. (1). For
example in a pole-placement regulator with control law u(n) = Kx(n) the closed-loop
state equations will be
x(n + 1) = ( K)x(n) (5)
and the eigenvalues may be placed arbitrarily in the z-plane using the same techniques as
for the continuous design. (Note that the effect of pole placement in the z-plane must be
interpreted differently from the s-plane).

Discrete-Time Full Observer: The discrete-time model of Eqs. (3) may be used to
generate an observer based controller for the continuous plant, with the assumption of zero-
order command. We assume that the system is observable, and that we desire estimates
x(k) of the state x(k) based on observation of the output y alone. As in the case of the
continuous observer, we use feedback of the estimator error y(k) Cx(k) as a correction
term
x(k + 1|k) = x(k|k 1) + u(k) + Ke [y(k) Cx(k|k 1)] (6)
where Ke is the observer gain matrix, and the notation x(k + 1|k) denotes that the estimate
(or prediction) of x(k + 1) is made using measurements available at time k.
Define the estimation error x as
x = x x, (7)
subtraction of Eq. (3) from Eq. (6) gives the error difference equation:

x(k + 1|k) = x(k|k 1) Ke [y(k) Cx(k|k 1)]

= [ Ke C] x(k|k 1) (8)

which allows arbitrary z-plane placement of the observer poles, as in the continuous case.
The observer described by Eq. (6) is known as a prediction observer, based on the fact
that it predicts the state values at the next time step from past measurements. This means
that control based on x(k) does not depend on the most current measurements, which might
lead to performance degradation. it is useful to construct an alternative observer formulation
that provides a current estimator x(k) based on the most recent measurements of y. Modify
Eq. (6) to estimate
x(k) = x(k 1) + u(k 1) (9)
and
x(k|k) = x(k) + Ke [y(k) Cx(k|k 1)] (10)
where x(k) is the estimate based on the prediction from the previous time step. Substitution
of Eq. (10) into Eq. (9) and advancing one time-step gives

x(k + 1|k) = x(k|k 1) + u(k) + Ke [y(k) Cx(k|k 1)] (11)

In this case the error dynamics are described by

x(k + 1|k) = [ Ke C] x(k|k 1) (12)

The block diagram of the current observer is shown in Fig. 3. The question then becomes
which of the two estimators should be used in closed-loop feedback design? In general,

2
 y k

u x x +
k + k + 1 k -
G z -1 C
+

F
x +
k
K e
+

Figure 2: Current discrete observer

because it is based on the most recent sample, one should use the current estimate because
it provides the fastest response to unknown disturbances or measurement errors.
The use of the current observer in a state-feedback controller with a continuous plant is
shown in Fig. 3.

u k = -K x k u (t)= u (k T ) .
P la n t y (t) y k = y (k T )
D A C x = A x + B u A D C
y = C x

C o n tin u o u s

x x +
+ k + 1 k -
G z -1 C
+

F
x k +
K K e
+

Figure 3: Discrete-time state feedback control of a continuous plant using a current discrete
observer

Optimal Estimation The Kalman-Filter: Optimal estimation provides an alternative

rationale for the choice of observer gains in the current estimator. Instead of arguments
based on the pole placement, the optimal estimator is based on observer performance in the
presence of process noise and measurement errors. Suppose the discrete plant model of Eq.
(3) is extended
x(k + 1) = x(k) + u(k) + 1 w(k)
y(k) = Cx(k) + v(k) (13)
where the process noise w(k) and measurement noise v(k) are white Gaussian random se-
quences with zero mean, that is
E {w(k)} = 0 and E {v(k)} = 0
where E {} denotes the statistical expectation, and zero correlation (white)
n o n o
E w(i)wT (j) = 0 and E v(i)vT (j) = 0 for i 6= j

3
and have covariances defined by
n o n o
E w(k)wT (k) = Rw and E v(k)vT (k) = Rv

The optimization task is to determine a set of observer gains Ke to minimize the variance
of the estimation error, which is denoted P(k):
n o
P(k) = E (x(k) x(k))(x(k) x(k))T ,

in the presence of the process and measurement noise. The derivation is beyond the scope
of this handout, we simply state that the solution is a structure the same as the full-state
current observer with set of time-varying gains Ko

x(k) = x(k) + Ko (k)(y(k) Cx(k)) (14)

where
Ko (k) = P(k)CT R1
v (15)
and where P(k) is found from

P(k) = M(k) M(k)CT (CM(k)CT + Rv )1 CM(k). (16)

The matrix M(k) is the covariance of the state estimates x(k) before the measurement. The
estimate x(k) is found from x(k 1) using Eq. (13) with w(k 1) = 0 because we know that
this is the expected value since the expected value of the plant noise is zero. Thus

x(k) = x(k 1) + u(k 1) (17)

This equation is known as the time update, whereas the change in the estimate from x(k)
to x(k) after measuring y(k) in Eq. (14) is known as the measurement update. The matrix
M (k) is updated as
M(k + 1) = P(k)T + 1 Rw T1 (18)
The solution starts with assumed initial values for M(0) and x(0). At each time step the
filter is updated in two steps:

At the measurement time: k + 1 k, sample y(k)

x(k) = x(k) + Ko (k)(y(k) Cx(k))

Between updates: Prepare for the next update

x(k + 1) = x(k) + u(k)

M(k + 1) = P(k)T + 1 Rw T1
P(k + 1) = M(k + 1) M(k + 1)CT (CM(k + 1)CT + Rv )1 CM(k + 1)
Ko (k + 1) = P(k + 1)CT R1
v

4
 w (t) v (t)

u (t)= u (k T ) y k = y (k T )
u = -K x P la n t y (t)
k k .
D A C x = A x + B u A D C
y = C x

C o n tin u o u s

x x +
+ k + 1 k -
G z -1 C
+

C o n tr o l g a in s F K a lm a n filte r g a in s
x k +
K K o (k )
+

Figure 4: Control of a continuous plant with a Kalman-filter based observer

A block diagram for the closed-loop control of a continuous plant using a Kalman-filter
estimator is shown in Fig. 4.
In an actual design problem, meaningful values can be assigned to Rv , which is based on
sensor noise which can often be found from the specifications. The same cannot be said of
the process noise, which is often an mathematical artifice that is used to expedite the opti-
mization. Physically, Rw is used to account for unknown disturbances and uncertainties in
the plant model. The disturbance noise model should be chosen to approximate any known
disturbances, but the designer is often forced to use acceptable values based on simulation
studies.

LCG Control the Steady-State Kalman-Filter: In practice, the time-varying

Kalman gains tend to steady-state values as k increases. In a control system that runs
for a very long time, the limiting gains may be used to define a so-called linear quadratic
gaussian (LQG) regulator. The structure is the same as the current observer based controller,
but the difference is that instead of the design being based on observer pole placement, it
is based upon minimization of estimate errors in the presence of process and measurement
noise. Again the derivation is beyond the scope of this handout, suffice it to say that the
steady-state Kalman-filter gains K are

K = M CT (CM CT + Rv )1 (19)

where M is derived from the stable eigenvectors of the estimator Hamiltonian

" #
T + CT Rv 1 C1 1 Rw T1 HT Rv 1 C1
He =
1 1 Rw T1 1

and
M = X1
where " #
X

5
are the stable eigenvectors.

LQG Design Example Using Matlab: Consider the coupled-mass system in Fig. (5)
Under the influence of applied force u(t). The system has equations of motion:
+ K(y d) = u
M y + B(y d)
md + B(d y) + K(d y) = 0;

Figure 5: Compliant mass example

If we assume that the measurable outputs are the displacements d and y, the state and
output equations are:

0 1 0 0 0
K/m B/m K/m B/m 0

x = x + u
0 0 0 1 0
K/M B/M K/M B/M 1/M
" #
1 0 0 0
y = x
0 0 1 0
h iT
where x = d d y y . The following parameters are used: M = 1, m = 0.1, K = 0.091,
B = 0.0036. A sampling period of T = 0.4 s. is assumed. With these values the state
equations are:

0 1 0 0 0
0.91 0.036 0.91 0.036 0

x = x + u
0 0 0 1 0
0.091 0.0036 0.091 0.0036 1

and after converting to a ZOH discrete model we have:

0.9285 0.3876 0.0715 0.0124 0.0013
0.3516 0.9146 0.3516 0.0854 0.0124

x(k + 1) = x(k) + u(k)
0.0071 0.0012 0.9929 0.3988 0.0799
0.0352 0.0085 0.0352 0.9915 0.3988

The following Matlab script takes the plant model, discretizes it using a ZOH discrete
model, chooses LQR gains, forms a Kalman-filter observer for state estimation, combines

6
the plant, controller, and observer models and plots some initial condition responses of the
closed-loop system:

% 2.151: LQG regulator for coupled masses

% (See handout on discrete observers).
%---------------------------------------------------------
% Enter the continuous plant model
%
M = 1; m = 0.1; k = 0.091; b = 0.0036;
A = [ 0 1 0 0
-k/m -b/m k/m b/m
0 0 0 1
k/M b/M -k/M -b/M];
B = [0; 0; 0; 1/M];
C = [1 0 0 0
0 0 1 0
0 0 0 0]; % get d, y and u as outputs
D = [0; 0; 1];
%----------------------------------------------------------
% Create the LTI continuous model
%
plant = ss(A, B, C, D, InputName, Force F (N), OutputName, ...
{Displacement d (m), Displacement y (m), Force F (N)});
set(plant, StateName, {d, d_dot, y, y_dot})
%----------------------------------------------------------
% Create the Discrete model with a sampling time of T=0.4 s.
%
T = 0.4;
ZOH = c2d(plant, T,zoh);
% Retrieve the matrices
[Phi, Gamma, Cd, Dd] = ssdata(ZOH);
%----------------------------------------------------------
% Design the regulator by computing the LQR Gain matrix K
% Set R = [1] arbitrarily and scale the Q matrix accordingly.
% Make Q diagonal
%
R = 1;
Q11 = input(LQR controller Q_11: );
Q22 = input(LQR controller Q_22: );
Q = diag([Q11 0 Q22 0]);
K = dlqr(Phi, Gamma, Q, R);
%----------------------------------------------------------
% Compute the Kalman filter gains
% Assume rms noise of 1% on each sensor channel
Rv = 0.01^2 * eye(2);
% Input Rw

7
 Rw = input(Enter estimator Rw: );
sensors = [1, 2]; % d and y are sensed
known = [1]; % force u
P = ss(Phi, [Gamma Gamma], C, [D D], T);
[Observer, Ko] = kalman(P, Rw, Rv, [], sensors, known);
%-----------------------------------------------------------
% Create the regulator and the closed-loop system
%
lqg_reg = lqgreg(Observer, K, current);
feedin = [1]; % force u
feedout = [1, 2]; % d and y
Gcl = feedback(ZOH, lqg_reg, feedin, feedout, +1);
%-----------------------------------------------------------
% Compute and plot the initial condition response
% Set x_1(0) = 1, all others to zero.
x0 = zeros(8,1);
x0(1) = 1;
figure(1), clf
initial(Gcl, x0)
% Plot all states and compare actual values with Kalman estimates
[y, t, x] = initial(Gcl, x0, 10);
figure(2), clf
subplot(2,2,1), stairs(t,x(:, [1 5])), grid, legend(x_1=d, x_1hat, 0)
title(Response of states and predictive estimates to x_1(0) = 1), ...
xlabel(Time (s))
subplot(2,2,2), stairs(t,x(:, [2 6])), grid, legend(x_2=ddot, x_2hat, 0)
xlabel(Time (s))
subplot(2,2,3), stairs(t,x(:, [3 7])), grid, legend(x_3=y, x_3hat, 0)
xlabel(Time (s))
subplot(2,2,4), stairs(t,x(:, [4 8])), grid, legend(x_4=ydot, x_4hat, 0)
xlabel(Time (s))
%---------------------------------------------------------------

References:

1. Franklin, G.F., Powell, J. D., Workman, M.L. Digital Control of Dynamic Systems
(3rd Ed.), Addison Wesley

2. Philips, C.L., Nagle, H.T. Digital Control System Analysis and Design (2nd Ed.),
Prentice Hall

3. Astrom, K.J., Wittenmark, B. Computer-Controlled Systems, Prentice Hall