elements of

modern
control
theory
*• r-'A
 Digitized by the Internet Archive
in 2019 with funding from
Kahle/Austin Foundation

https://archive.org/details/controltheoryOOOOgree
Elements of Modern

Control Theory
 dedicated
to the memory of
Max Greensite
and
Sholom Smilowitz
 CONTROL THEORY: VOLUME I

Elements of Modern
/ Control Theory

Arthur L. Greensite

Design Specialist, Flight Control Systems

Convair Division of General Dynamics
San Diego, California

SPARTAN BOOKS

NEW YORK WASHINGTON

ELEMENTS OF MODERN CONTROL THEORY

Copyright © 1970 by Spartan Books.

All rights reserved. This book or parts thereof may not be
reproduced in any form without permission from the publisher.
Library of Congress Catalog Card Number 68-23969
Standard Book Number 87671-553-6

Printed in the United States of America.

Sole distributor in Great Britain, the British Commonwealth,

and the Continent of Europe:
Macmillan & Co. Ltd.
4 Little Essex Street
London, W.C. 2
 I l—

^ i-f
. u /

PREFACE

The emergence of modem control theory as a major engineering

discipline has been accompanied by the publication of many textbooks
with varying degrees of emphasis as regards theory and practical
application. The present work is intended to fill the gap from the point
of view of the aerospace engineer who is concerned with the design of
automatic control systems for space vehicles.
In order to present a coherent and self-contained exposition
proceeding from first principles, it was decided to treat the theory and
applications in two separate volumes. The present book is therefore
concerned primarily with basic theory, and a companion volume*
contains a detailed treatment of applications to realistic aerospace
vehicles.
In any work of this nature, the desire to be both comprehensive and
rigorous strains the available resources of time and space, as well as the
mathematical maturity expected of the student. However, because of the
fact that many sophisticated control concepts are rapidly blossoming
into hardware reality, one cannot safely neglect these without seriously
handicapping the training of the aerospace control engineer.
A brief glance at the Table of Contents shows that every major facet
of modem control theory is covered, beginning with the elements of
linear feedback systems up to, and including, recent developments in
learning theory. Obviously, for so ambitious a scope, some compromises
are necessary. These reflect the analytical maturity expected of
senior-level or first-year graduate students in the aeronautical or
aerospace engineering sciences. Thus, we presuppose a knowledge of
differential equations, matrices, vectors, complex variables, and
elementary dynamics. Various digressions into such topics as the calculus
of variations, probability theory, pattern recognition, theory of
functions, etc., are developed “ab initio. ” Our treatment of this subject

*Greensite, A., Analysis and Design of Space Vehicle Flight Control Systems, New York, Spar¬
tan Books, 1970.

v
 PREFACE
VI

matter leans heavily on heuristic and plausibility arguments in which the

derivations are, therefore, purely formal. Nevertheless, in the author s
opinion, they make accessible to the average student many sophisticated
results, which might not otherwise be within his grasp. On the other
hand, the well-established and basic linear and nonlinear theory is
developed more carefully. Here again, in order to keep the length within
reasonable limits, it has been necessary to assume a working knowledge
of Laplace transforms and transfer functions on the part of the student.
When this is not the case, it may readily be supplied by the instructor or
else the student may easily acquire the necessary rudiments by collateral
reading. In all cases, the instructor and practicing engineer will find
appropriate references to the open literature where more detailed and
rigorous arguments are available.
The author has used the material in this book for introductory and
advanced control theory courses in the graduate school of aerospace
engineering at San Diego State College. For a four-hour- a-week,
one-semester introductory course, emphasis may be placed on Chapters
One, Two, and Three. Depending on the inclination or interests of the
students or the instructor, one or more of the remaining chapters may be
studied in varying degrees of depth. In a two-semester course all the
chapters may be covered together with selected topics from the
companion volume. This approach has been used by the author and was
well received by his classes.
Most of the material presented here, and in the companion volume on
applications to aerospace vehicles, was originally prepared for the
National Aeronautics and Space Administration under Contract Number
NAS8-11494. The purpose was to document and illustrate the methods
used for design and analysis of space vehicle flight control systems. The
author is grateful to Clyde D. Baker, Billy G. Davis, and Fred W. Swift of
the Aero Astro Dynamics Laboratory, Marshall Space Flight Center,
Huntsville, Alabama, who in their capacity as NASA project mana¬
gers for this task contributed invaluable guidance, comments, and
suggestions.
I am also indebted to Mr. Ray Bohling, Chief of Guidance and
Control, NASA, Washington, D.C., whose advice and constructive
criticism enhanced this effort considerably.
Many improvements in structure and content of the book resulted
from the author’s discussions with Prof. T. J. Higgins of the University of
Wisconsin, who read the original manuscript in its entirety. I ac¬
knowledge his contributions with appreciation and gratitude.
PREFACE VII

My wife, Carole, provided the much needed understanding and

inspiration during this endeavor, as did my children, Jeffrey, Fred, and
Marcia. For this I am deeply grateful.

Arthur L. Greensite
San Diego, California
 CONTENTS

Chapter One. Theory of Linear Systems

1.1. Stability Analysis-Continuous Systems . 3
1.2. Stability Analysis-Sampled Data Systems . 57
1.3. Compensation Techniques . 88
1.4. State Variable Method . 121
1.5. Observability and Controllability . 146
1.6. The Invariance Principle . 167

Chapter Two. Theory of Nonlinear Systems

2.1. Phase Plane Analysis . 216
2.2. Describing Functions . 245
2.3. Lyapunov’s Direct Method. . 298
2.4. On-Off Controllers . 339
2.5. The Popov Method . 347

Chapter Three. Sensitivity Analysis

3.1. Measures of Sensitivity . 372
3.2. Design for Sensitivity . 406
3.3. Sensitivity and Optimal Control . 431

Chapter Four. Stochastic Effects

4.1. Mathematical Preliminaries . 447
4.2. The Wiener Theory . 472
4.3. The Kalman Theory . 481
4.4. Stochastic Effects in System Design . 536

Chapter Five. Optimal Control

5.1. Mathematical Concepts . 572
5.2. Standard Solutions . 656
5.3. Aerospace Applications . 679

Chapter Six. Adaptive Control

6.1. Model Reference Techniques . 754
6.2. Virtual Slope Methods . 767
6.3. Digital Adaptive Filter . 775
6.4. Notch Filter Methods . 798
6.5. Frequency Independent Signal Processing . 818

IX
X CONTENTS

Chapter Seven. Learning Control Systems

7.1. Statement of the Problem . 835
7.2. Pattern Classification . 836
7.3. Application to the Time Optimal Problem . 852
7.4. Time Optimal Reorientation Maneuver . 866

Index . 883
 Chapter One
THEORY OF LINEAR SYSTEMS

The basic theory of linear systems goes back to Newton and includes
many refinements and extensions (Laplace transforms, linear operators)
developed over the years. In the period following World War II, the
rapid development of the theory of automatic control (feedback principle)
led to the need for special methods specifically suited to linear feedback
control systems. There emerged the frequency-response techniques
(Nyquist, Bode) and root locus (Evans). These permitted a rapid and
efficient analysis of what might be called “conventional” control systems.
Multiple loop, high order systems presented extreme difficulties for
“paper-and-pencil” analysis. The advent of the modern high-speed digital
computer has, for the most part, eliminated this problem, and the design
of linear automatic control systems has become virtually routine. Con¬
sequently, research studies in recent years have been in the areas of
nonlinear, adaptive, and optimal control. However, the well has not
run dry in linear theory. New ideas do appear.
A case in point is the theory of observability and controllability. One
important result of this theory is a new insight into the relationship be¬
tween transfer functions and the state variable representation. This has
clarified the phenomenon of “hidden oscillations” as well as the condi¬
tions under which a conventional transfer function is an accurate repre¬
sentation of a given dynamic system. The theory is perhaps most powerful
when applied to multivariable, i.e., multiple input/multiple output, sys¬
tems. There are many instances in the literature where the neglect of this
idea has led to erroneous results.
Another recent development of some importance is the Principle of
Invariance, whose basic ideas have been extensively advanced by the
Russians. Here, the fundamental idea, while simple and elegant, en¬
counters severe difficulties in practical application. However, the basic
approach is novel and will no doubt prove useful in the design of control
systems as current obstacles are progressively resolved.
1
2 ELEMENTS OF MODERN CONTROL THEORY

Various refinements in such standard tools as root locus and frequency-

response methods appear periodically. Many of these are discussed in
this chapter, especially if they are potentially useful in space vehicle
control systems.
This exposition also takes into account the marked trend in recent
years toward the state variable method of analysis. This method is
considered here in some detail for the stationary case, since at present
this is the only type for which computational aspects are not over¬
whelming.
An extensive list of references that the reader may consult for greater
detail in the topics discussed is provided at the end of this chapter.
In Sect. 1.1, the fundamental techniques for the analysis of linear
continuous systems are developed. These are discussed in their order of
chronological development, beginning with the classical methods of
determining whether the characteristic equation for a given system con¬
tains roots with positive real parts. This is followed by a discussion of
the frequency-response methods due to Bode and Nyquist, which permit
a more complete determination of the response properties in terms of
frequency, relative damping, etc. Since this method is one of the most
widely used at the present time, the treatment is more detailed, and
applications to aerospace systems are discussed. Next, the prominent
facets of the root locus method are considered, with special emphasis on
the qualitative features of the system response (transient and steady-
state) that are obtainable by this method.
Finally, the recent developments for the stability analysis of systems
expressed in the state variable format are summarized.
All the above methods deal with what is essentially the same problem;
namely, how to determine the response properties of a linear system in
some efficient and enlightening manner. The crudest are the classical
methods (Routh-Hurwitz), since the only information obtained is whether
the system is stable or unstable. The frequency-response method, as the
name implies, yields results (in terms of a frequency-magnitude plot) on
how fast the system responds to various command signals (and distur¬
bances), together with indications of how quickly the transient motions
die out. The method is especially adapted for application to feedback-
type systems.
The special virtue of root locus is that the transient and the steady-state
properties of the system are simultaneously available.
In the final sections, two recent theoretical developments that enhance
the use of the above methods are discussed. The concepts of observability
and controllability (due to Kalman) help clarify the relationship between
THEORY OF LINEAR SYSTEMS 3

a system expressed in state variable form and a system expressed in

transfer function form. This is especially useful in analyzing multi-
variable systems.
The Invariance Principle, discussed in Sect. 1.6, is a novel method for
rendering a system insensitive to extraneous disturbances. Various
problems connected with its specific implementation to a launch vehicle
autopilot are discussed.

1. 1 STABILITY ANALYSIS-CONTINUOUS SYSTEMS

The input-output relation for any linear system may be expressed

in the form

X a k Dn~k x(t) = y(t)

k=0

(1)

dt*

where the a* are constants. The complete solution of this equation

withy(t) identically zero is

Xi(t) = X Ct-eV (2)

i=l

where A, are the roots of the characteristic equation

X ak\n~k= 0. (3)
k=0

The complete solution of Eq. (1), when y(t) ^ 0, is given by

x(t) = Xi(t) + x2(t) (4)

where x2(t) is a particular solution of Eq. (1). These results are classical,
and details may be found in any standard text on differential equations.
Although, in principle at least, linear systems may be solved in a
straightforward manner, a multitude of highly specialized techniques
have been developed to yield results more efficiently and for specific
needs. Basically, these methods bypass the problem of solving Eq. (1)
repetitively for each new set of system parameters, yielding instead such
4 ELEMENTS OF MODERN CONTROL THEORY

information as simple stability and relative stability, and dealing with

such terms as damping, resonant frequency and gain, and phase margin.
In discussing these methods, we will at all times be mindful of appli¬
cation to typical space vehicle control problems. For this purpose, a
fairly complete description of an attitude control system is contained in
Appendix A. This will be used repeatedly in examples serving the dual
purpose of clarifying the topic under discussion and highlighting promi-
ment facets of the attitude control system.

1.1.1 Routh-Hurwitz Method

Instead of a complete solution of Eq. (1), it is often necessary merely

to know whether the solution is stable. This is assured if all the roots
of the characteristic Eq. (3) have negative real parts.
The classical Routh-Hurwitz criterion is contained in the following.!7.8)

Theorem A: The roots of the nth degree polynomial equation with

real coefficients

f(X) = X a A"-* = 0 (5)

k=0

have negative real parts if, and only if, all the determinants

ai a3 a6 ....
ao a2 3<4 .... &2k—2
0 ax a3 • • • • . &Vc~Z
ao 32
0 ai

ao
0

0 0 0 . a,

are positive for k = 1,2,3, . . ., n.

Any term a,-, whose subscript j > n, is set equal to zero.
The following is a simple illustration of the method.

Example 1: Consider the system characterized by the transfer

function of Eq. (A23) with K7 set equal to zero. According to (A26),
 THEORY OF LINEAR SYSTEMS

the characteristic equation is given by*

s3 + ais2 + a2s + a3 = 0

where

ai = K„
a2 = KaK0Krh„ ~ Ma
a3 = Kc(K4p0 — na).

Using the parameter values

= 2.5 Ac = 2
K c = 5 Aa ~ 1
Ivr = 1

we find

SLi = 5
a2 = 24
a3 = 20.

Therefore

Hi ax = 5

ai a3
H2
a0 a2

ai a3 0
ao a2 0 2000.
0 ai a3

Since all the H* are positive, the characteristic equation has no roots
with positive real parts, and the system is stable.
Theorem A may be reformulated in a manner that does not require
the evaluation of determinants. The essential result is contained in the
following.

*It is immaterial whether we use s (the Laplace operator) or X, since for present purposes this is
merely a dummy variable.
6 ELEMENTS OF MODERN CONTROL THEORY

Theorem B: Consider the polynomial equation

f(X) = L a*X“-*= 0 (7)

k=0

where the coefficients, a*, are all real.

Form the array of numbers:

ao a2 a4 a6

ai a3 a5 a7

bi b2 b3
Cl c2 C3

di d2
ei e2

fi

where

aia2 —

a0a3 aia4 — a0a6

bi = )
b2
ai ai

bia3 —

axb2 bia5 — aib3

Cl = C2
bi bi

cib2 —

bic2 Cib3 — bic3

di = J
d2
Cl Cl

dic2 —
cid2 dic3 — Cid3
ex = e2
di di

eid2 —

die2
fi =
ei

etc.
Notice that two terms in the first column are used in each calculation.
As the term to be calculated shifts to the right, the additional two terms
in the formula also shift to the right. The formula for calculation of
terms in any given row uses only those terms in the two rows immediately
above. The process is contained for (n + 1) rows.
The number of changes in sign of the terms in the first column of the
above array is equal to the number of roots with positive real parts.
THEORY OF LINEAR SYSTEMS 7

Furthermore, if the a* are not all of the same sign, or if any a* is zero,
then some roots are either pure imaginaries or else have positive real
parts.
It may happen that the first column term in any row is zero but the
remaining terms in this row are not all zero. In this case, replace the
zero term by an arbitrarily small constant, e, and proceed as usual.
If all the coefficients of any row are zero, this indicates a pair of
complex roots with zero a real part.
The above results are classical and constitute the earliest attempts to
study the stability of linear systems in some rational manner. In recent
years, these criteria have been generalized in various ways. Perhaps the
most significant is the extension of the method to determine “relative
stability.” In this case, one derives, on the coefficients, a*, conditions
that ensure that all the roots of the characteristic equation lie to the
left of the shaded lines shown in Fig. 1.1.
Now any complex root pair may be expressed as

— =b j \/l — fB2]

where cen represents an undamped natural frequency and is the relative

damping factor. (See Fig. 1.1.) It is evident, therefore, that the locus of
all roots having the same damping factor is two straight lines extending

Figure 1.1. Region of Relative Stability.

8 ELEMENTS OF MODERN CONTROL THEORY

from the origin into the left-half plane and making equal angles with
both halves of the imaginary axis. According to Fig. 1.1, the relative
damping factor defined by the shaded lines is given in terms of 6 by

cos d — y/l — fill (8)

where is the relative damping factor of a point lying on the shaded

line. By substituting

X = ze ie (9)

into Eq. (5), we obtain the new polynomial

fi(z) = a^e ,(" k)ezn~k = 0 (10)

k=0

whose roots are identical to those of Eq. (5), except that they are rotated
counterclockwise by d degrees. Consequently, any root of Eq. (5) that
is located in the sector between the shaded line and the negative imagi¬
nary axis will appear as a root in the right-half plane of Eq. (10). The
difficulty now is that the coefficients ofEq. (10) are (in general) complex.
What is needed, therefore, is a “Routh-Hurwitz” type criterion that is
valid for polynomials with complex coefficients. Various criteria of this
type have indeed been developed, and the simplest, perhaps, is the
Bilharz-Frank theorem given by Marden(9) in the following form.

Theorem C: Given the polynomial equation

fi(z) = J2 (A* + \Bk)zn~k= 0 (11)

*=>0
where the Ak and B* are real, with*

Ao — 1
Bo = 0.

*There is obviously no loss of generality in assuming that the leading coefficient is unity.
THEORY OF LINEAR SYSTEMS 9

Form the determinants

Ai A3 Ab A-2k-l -b2 B4* - • ‘ _ B2<;_2

1 A2 a4 A-Uc-2 -Bx B3 —B2A-3
0 Ai A3 A-2k-3 0 b2 .
•
1 A2 B, .
0 Ai 0 .
1-
0-
Ak 0 0 - Ak 0- . . B*_i
0 b2 b4 B2ifc_2 Ai A3 . A.2k—3
0 Bi b3 B2i_3 1 A2 . A2i_4
0 b2 0 Ai .
1.
0

0 B* 0 Afc_i

where k = 1,2,3, • • •, n.
The number of roots of Eq. (11) having positive real parts is equal to
the number of changes of sign in the sequence, Ai, A2, • • •, A„.
Any term A^or B, whose subscript i > n is set equal to zero.
When the B* are all zero, i.e., Fx(z) is a real polynomial, then Ax = Hx
and A*, = H*, Ht_i where H* is the determinant defined by Eq. (6),
with a0 = 1. Then, since sgn (AXA2)= sgn A2, and sgn (A* Ak+1) = sgn
(Hh Ht+]) for k = 2,3, •••, n — 1, the theorem reduces to the con¬
ventional Routh-Hurwitz form.
The necessity of evaluating high order determinants is often awkward,
even though there are a large number of zero elements. An alternate
approach that permits the use of the simplified criteria of Theorem B is
the following/10) Substitute

X = ze* (12)

into Eq. (5), yielding the polynomial

f2(z) = 23 akei(nk)9znk = 0. (13)

k=> 0

The roots of this polynomial are identical to those of Eq. (5), except
that they are rotated clockwise by B degrees. Forming the product
10 ELEMENTS OF MODERN CONTROL THEORY

fc(z) = fi(z)f2(z)= 52 aaa^ey(a 13)9

z(2n a /3) (14)
a=0 /3=0

results in a new polynomial having the property that any complex root
pair of Eq. (5) lying in the sector between the shaded lines and the
imaginary axis now appears as a complex root pair in the right-half
plane of Eq. (14). The coefficients of Eq. (14) are all real. In fact, for
any specific pair of indices, a = p and /3 = q, we have a term of the
form

dip 2Lg6 — (P— z(2n —p—r)

while for a = q and /3 = p, the term appears as

apaqeiip~r)9z^2n~p~r)
.

Summing these last two yields a term of the form

2a/>a9z(2n—?—r)cos (p — r)6.

Collecting coefficients of like powers of z results in the expression

2n

fc(z) = 52 biZ"-*. (15)

k—0

The coefficients b*; are evaluated below for polynomials in X of up to

the tenth order.

b0 = a02
bi = 2a0ai cos 8
b2 = ai2 -f- 2a0a2 cos 28
b3 = 2aia2 cos 6 + 2a0a3 cos 3 8
b 4 = a22 + 2aia3 cos ‘28 + 2aoa4 cos 4 8
b5 = 2a2a3 cos 8 -f- 2aia4 cos 38 -(- 2aoa3 cos 58
b6 = a32 + 2a2a4 cos 28 + 2axa6 cos 4 8 + 2a0ae cos 68
b7 = 2a3a4 cos 8 2a2a6 cos 38 + 2axa6 cos 58 + 2a0a7 cos 78
b8 = a42 + 2a3a6 cos 28 + 2a2a6 cos -id + 2axa7 cos 68 -f 2a0a8 cos 88
b2 2a4a6 cos 8 4- 2a3a6 cos 36 -)- 2a2a7 cos 58 T 2axas cos 78
+ 2a0a9 cos 9^
bio = a6 2 + 2a4a6 cos 28 + 2a3a7 cos 4 8 + 2a2a8 cos 68 + 2aiag cos 88
+ 2a0ai0 cos 100
THEORY OF LINEAR SYSTEMS 11

bn = 2a5a6icos 0 + 2a4a7 cos 30 + 2a3a8 cos 5 6 + 2a2a9 cos 76

+ 2aiaio cos 9 6
bi2 = a62 + 2a&a7 cos 26 + 2a4a8 cos 40 + 2a3a9 cos 60 + 2a2ai0 cos 80
bi3 = 2a6a7 cos 0 + 2a5a8 cos 30 + 2a4a9 cos 50 + 2a3ai0 cos 70
b« = a72 + 2a6a8 cos 20 + 2a5a9 cos 40 + 2a4ai0 cos 60
bis = 2a7a8 cos 0 + 2a6a9 cos 30 + 2a6ai0 cos 50
bie = a82 + 2a7a9 cos 20 + 2a6ai0 cos 40
bn = 2a8a9 cos 0 + 2a7ai0 cos 30
bi8 — a92 + 2a8ai0 cos 20
bi9 = 2a9ai0 cos 0
b2o — aio".

A simple application of these results is given below.

Example 2: In Example 1 it was shown that the characteristic

equation

s3 + 5s2 4- 24s + 20 = 0 (a)

has no roots in the right-half plane. We now seek to determine if there is

any root having a relative damping factor less than 0.5. From Eq. (8) we
find that the sector under consideration is defined by 0 = 30°.
The problem will be solved by direct application of Theorem C and
also by the criteria of Theorem B, using Eq. (15). In the present case,
the polynomial, fi(z), becomes

fi(z) = e~3,0z3 + 5e 2,0z2 + 24e ,sz + 20

= z3 + 5e'V + 24e2)Sz + 20e3’6

(b)
0

where

A0 = 1 Bo = 0
Ai = 4.33 Bj = 2.5
A2 = 12 B2 = 20.78
A3 = 0 B3 = 20.

The determinants, A*, are

12 ELEMENTS OF MODERN CONTROL THEORY

4.33 0 0 20.78 0
1 12 0 -2.5 -20
A3 — 0 4.33 0 0 -20.78
0 20.78 0 4.33 0
0 2.5 20 1 12

=
-24,994

4.33 0 -20.78
A2 = 1 12 -2.5
0 20.78 4.33

= 18.02

Ai — Ai — 4.33.

Since there is one change of sign in the sequence Ai, A2, A3,Theorem C
indicates that Eq. (b) has one root in the right-half plane, which, in turn,
means that Eq. (a) has a complex root pair whose relative damping
factor is less than 0.5.
To obtain the same result via another route, we calculate the bt of
Eq. ( 15), using the ak coefficients of Eq. (a). The result is

b6 = 400 b3 = 208
b5 = 831 b2 = 49
b4 = 676 bi = 8.66
bo = 1.

The number array of Theorem B becomes

1 49 676
8.66 208 831
25 580 400
7 692
-1891 400
692
400.

There are two changes of sign in the first column, which indicates
that the equation
6

fc(z) = £ b*z6-* = 0
k=0
THEORY OF LINEAR SYSTEMS 13

has two roots in the right-half plane; therefore Eq. (a) has a complex
root pair with a relative damping factor less than 0.5.

1.1.2 Frequency- Response

The methods discussed in this section are concerned with determining

the stability properties of the feedback system shown schematically in
Fig. 1.2. The notation is the one most widely used in the control
literature.

R(s) = Laplace transform of the reference signal

C(s) = Laplace transfrom of the controlled variable
E(s) = Laplace transform of the error signal
B(s) = Laplace transform of the feedback signal
G(s) = forward loop transfer function
H(s) = feedback loop transfer function
K = open-loop gain
s = Laplace operator

The following quantities are of fundamental importance.

Closed-Loop Transfer Function:

C(s) _ KG(s)
(16)
R(s) 1 + KG(s)H(s)

Open-Loop Transfer Function:

= KG(s)H(s) (17)
E(s)

R(s) __ E(s) C(s)

KG(s)
V
_ i

B(s)

H(s)

Figure 1.2. Schematic of Feedback Control System.

14 ELEMENTS OF MODERN CONTROL THEORY

The open-loop transfer function may be expressed in either of the

following two forms.

B(s)
K.h(^+«n(i>^+i)
i=l j=l\COj OJj /
(18)
E(s) 8 T / S2 2tl \
B-ner**
4=1
+ dii( —
2+—
fs+ ]) " '

m P

K I] (s + a<) II(s2 + 2fyC0yS

+ «/)
B(s) = ^_ (19)
E(s) g r
s" IT (s + bi) H(s2 + 2ffcofs+ wf2)
4=1 £ =1

It will be shown later (Sect. 1.3.2) that K„ provides a measure of

steady-state error.
Let us write Eq. (18) in the form

B(s) = KnAi(s)
E(s) s"A2(s)' 1

Then the equation of motion for the system, in Laplace transform

notation, becomes

[snA2(s) + I\„Ai(s)]C(s) = Ks„A2(s)G(s)R(s). (21)

The values of the roots of the characteristic equation

s"A2(s) + KnAi(s) = 0 (22)

determine the stability of the system. If the system is to be stable, then

Eq. (22) must not have any roots in the right-half s plane.
The Nyquist criterionU B determines the number of roots of Eq. (22)
in the right-half s plane from a frequency-response plot of the open-loop
transfer function. Unlike the Routh test, the Nyquist method also yields
information on “relative stability,” which is important from a control
point of view. This idea will be made precise later.
The Nyquist criterion is classical. It is discussed in every standard text
on control theory. However, most authors, in an attempt to avoid the
THEORY OF LINEAR SYSTEMS 15

use of complex variables, construct an awkward, burdensome, and some¬

times questionable “proof” of the criterion, with the result that the
reader is more often mystified than enlightened.
Since the Nyquist criterion is basic in linear control theory and its
derivation is simple and straightforward, it seems appropriate to develop
it here. With this as a foundation, some of the more complicated Nyquist
diagrams may be interpreted with ease and assurance.
Let

(23)

denote a rational function of s. In accordance with common usage, a

root of Di(s) is called a zero , while a root of D2(s) is called a pole of the
function Y(s).
Draw the closed contour Ti in the s plane such that it encloses all the
poles and zeros of Y(s). Then there exists a closed curve, r2, in the Y
plane, which results from mapping each point of ri onto the Y plane.
(See Fig. 1.3.) We say that a closed contour is described in a positive
sense if the interior of the contour is always to the left as the point
moves along the contour.
If none of the poles or zeros lie on the contour, then, if the contour,
I\, encloses in a positive sense Z zeros and P poles of Y(s) (this takes
account of multiplicity of poles and zeros), the corresponding contour,
r2, in the Y plane encircles the origin

N = Z - P (24)

times in a positive sense.*

Positive encirclement about a point, p0, is defined as follows. Consider
a radial line drawn from p0 to a representative point on the closed contour.
As the point on the contour proceeds around the contour in a positive
sense, the radial line sweeps out an angle 2ttN, where N is a positive
integer. The point, p0, is then said to be encircled N times in a positive
sense.
Consider now the special contour, rx, shown in Fig. 1.4. We say that
(loosely speaking) p is very small and Q is very large. This will enclose all

*This result, which follows from a simple application of Cauchy’s Residue theorem, is proved in
any standard text on complex variables.
16 ELEMENTS OF MODERN CONTROL THEORY

jOJ

Im

Figure 1.3. Mapping of Closed Contour in s Plane

to Closed Contour in Y Plane.

the finite poles and zeros of Y(s). The small semicircle about the origin
is drawn so that a pole of Y(s) at the origin is not on the contour.
Similarly, arbitrarily small semicircles are drawn on the imaginary axis
to avoid purely imaginary poles and zeros of Y(s).
We now investigate the form of r2 when a point on I\ approaches
zero on the positive jo> axis, i.e., with p —> 0. If Y(s) has an nth order
pole at the origin, then it may be written as

Y am_ism 1 "T . T" ao)

" sn(brsr + br_1sr~1+ . + bo)' ("5)

For small s, this may be approximated by

Y(s) (26)
THEORY OF LINEAR SYSTEMS 17

Now if si is any point on the small semicircle, it may be expressed as

si = pe,e.

The corresponding point on the r2 contour is approximated by Eq. (26).

/

— e-»'»9 = Y(si)
Pnbo

It follows that if the small semicircle about the origin is described in

the sense shown in Fig. 1.4, the corresponding portion of the r2 contour
describes n large semicircles in a counterclockwise direction.
Let us now assume that Y(s) has the special form

Y(s)= -L + L(s) (27)

Kn

where K„ is a positive constant. The mapping of the JY contour of Fig.

1.4 onto the L plane can be obtained by shifting the corresponding map
on the Y plane to the left by an amount 1/Kn.

jw

Figure 1.4. Nyquist Contour in s Plane.

18 ELEMENTS OF MODERN CONTROL THEORY

It follows that contour Tiof Fig. 1.4, described in a positive sense, will
map into a contour, r, in the L plane, that encircles the point ( —1/K„, jO)
in a positive sense N = Z — P times.
This is the Nyquist Stability Criterion. Note that if L(s) is expressed as

Ai(s)
L(s)
s"A2(s)

where Ai(s) is the quantity that multiplies K» in the numerator of Eq.

(18), and A2(s)is the quantity that multiplies s’1in the denominator, then

skA2(s) + K^A^s)
Y(s) (28)
K„snA2(s)

It is obvious that the roots of the numerator (zeros) of Eq. (28) are
also the roots of the characteristic Eq. (22).
Since the poles of L(s) are also the poles of Y(s), it follows that Eq.
(22) has

Z = N + P (29)

roots in the right-half plane, where P is the number of poles of L(s) in the
right-half plane and N is the number of positive encirclements of the
( —1/Kn, jO) point in the L plane.

Remark: In most textbooks, one considers

L(s) = K”Al(s)
s"A2(s)
and stability is described in terms of the (—1, jO) point. However, it
is simpler to move the point 1/K„ than to redraw the KnAi(s)/s"A2(s)
locus for every new value of K„. Note also that if K„ is a negative
quantity, all the previous results hold except that the critical point
is (1/K„, jO) instead of ( —1/K„, jO).
The value of the exponent, n, in the above expression indicates the
so-called “system type.” (See Sect. 1.3.1.) It provides a measure of
the steady-state error in response to particular input signals (step,
ramp, etc.).
Table 1.1 shows frequency-response and root locus plots for some
typical open-loop transfer functions. For the cases shown:
THEORY OF LINEAR SYSTEMS 19

Table 1.1. Frequency Response and Root Locus Plots of Some

Common Open-Loop Transfer Functions

Case 1

Nyquist Plot

KG(s) H(s) = —- -2— - -

s^s+l) (T2s+1)

Root Locus Nichols Plot

20 ELEMENTS OF MODERN CONTROL THEORY

Table 1.1. Frequency Response and Root Locus Plots of Some

Common Open-Loop Transfer Functions ( Cont’d)

Case 2

Nyquist Plot Bode Plot

K (r s+l)(rs+l)
KG(s)H(s) = - --
' sC1s+l)C2s+l)(r3s+l)Cr4s+l)

Root Locus Nichols Plot

THEORY OF LINEAR SYSTEMS 21

Table 1.1. Frequency Response and Root Locus Plots of Some

Common Open-Loop Transfer Functions ( Cont’d)

Case 3

Nyquist Plot Bode Plot

K (T s+1)
KG(s) H(s) = -
s Cr1s+1)(r2s+1)

Root Locus Nichols Plot

22 ELEMENTS OF MODERN CONTROL THEORY

1. K„ and r are positive constants.

2. Arrows on the Nyquist plots indicate the direction of increasing
frequency.
3. The symbols N, P, and Z on the Nyquist plots have the meaning
defined by Eq. (29).
4. Ri on the root locus plot denotes the operating point; i.e., it is the
closed-loop pole.

Note that when there is an nth order pole at the origin, the point o> = 0+
is connected to w = 0_ by n large counterclockwise semicircles. Further¬
more, an encirclement of the —1 point is positive if, in tracing the
Nyquist diagram as co varies from + °o to — <*>,the net encirclement is in a
counterclockwise direction. Otherwise, the encirclement is negative.
This follows from the discussions related to Figs. 1.3 and 1.4.
Two important figures of merit that may be obtained from the Nyquist
plot are the phase margin and gain margin. These are defined as follows.
Gain margin is the factor by which the gain, K„, must be multiplied to
make the locus pass through the ( —1, jO) point.
Phase margin is the amount of phase shift needed at unity gain to make
the locus pass through the (—1, jO) point.
These concepts may be clarified by considering the Nyquist plot of

R-tT'TaS “f- 1)
KG(s)H(s) =

which is shown in Fig. 1.5. It is readily ascertained that in this case,

P = 1 and N = —1. This means that there are no closed-loop poles in
the right-half plane, i.e., the system is stable.
Inspection of Fig. 1.5 indicates that the system will be unstable if the
gain, K„, is raised or lowered a sufficient amount. The relevant upper
and lower gain margins* are Ai = 1/hi and X2= l/h2. Also the system will
be unstable if a phase lag of 71 degrees or a phase lead of 72 degrees is
added to the system. 71 and 72 are thus the appropriate phase margins.
The usual specifications for acceptable design are as follows.

gain margin: 6 db (minimum)

phase margin: 30° (minimum)

*Gain margin is generally expressed in decibels. The decibel equivalent, N<rt, of a number, N, is
N<26 = 20 logio N.
THEORY OF LINEAR SYSTEMS 23

The gain margin of 6 db means that the open-loop gain may be

increased by a factor of 2 before instability occurs. A precise specifmation
of this quantity depends on the degree of accuracy with which the
mathematical model represents the physical system; it is, in effect, a
24 ELEMENTS OF MODERN CONTROL THEORY

“margin of safety” in design. In other words, while a nominal value of

gain may yield acceptable steady-state and transient response, if a
relatively small change in gain results in drastic changes in system
properties, the design cannot be considered adequate. The specification
of gain margin is intimately related to such factors as measures of per¬
formance (see Sect. 1.3.1) and system sensitivity.*
The specification of phase margin reflects the amount of additional
phase lag (or lead) that will cause instability. It is one measure of per¬
formance quality (which is considered in greater detail in Sect. 1.3.1).
The values given above are representative of current aerospace design.
An insight into the significance of phase margin may be obtained by
considering a second order system. Thus, in Fig. 1.2, let

2
Wl
KG(s) (30)
s(s *F 2fiwi)

and

H(s) = 1.

Then obviously

(s2 + 2fi«is + cox2)C(s) = wi2R(s).

The Nyquist plot of the open-loop transfer function, Eq. (30), is

obtained by replacing s with jw; viz.

KG(ju) = - — - = M/V
—co2+ 2jfiwiw

where M is the magnitude and <p is the phase angle of the complex
number KG(jco). It follows immediately that

if = TT + 6

where

2f ion
tan 6 =
co

*See Chapter Three.

THEORY OF LINEAR SYSTEMS 25

Also, the value of w that corresponds to M = 1 is given by

co02= C0i2[v-/1+ 4fi4 — ‘2fi'2]

while the value of Qcorresponding to M = 1 is

tan d0
Wo

By definition, the phase margin, 7, is equal to 60. Therefore, the

phase margin is given by

2fi
7 = tan-
-(V 1 + 4f? - 2f12)1/2J

This relation is very nearly linear for 0 < 7 < 50 0 and may be written as

7^110fi. (31)

For second order systems, the phase margin is therefore directly re¬
lated to the relative damping factor. The response of most systems of
engineering interest is, in fact, governed by a dominant pair of complex
poles. Consequently, in such cases, the phase margin is a measure of how
oscillatory the system is. (See also Sect. 1.3.1.)
It is often convenient to make use of the Simplified Nyquist Criterion,
which may be stated as follows.
If the open-loop transfer function, KG(s) H(s), contains no poles in
the right-hand s plane, and if the Nyquist locus does not encircle the
( —1, jO) point, the system is stable.
It is also possible to derive a kind of generalized Nyquist criterion as
follows. Instead of the s plane contour, Ti, of Fig. 1.4, consider the
contour, Ti, shown in Fig. 1.6. As before, we apply this to the transfer
function, Y(s), defined by Eq. (27).
In the conventional case, the Nyquist locus is obtained by replacing
s in L(s) by jw, and letting « vary from zero to infinity. To describe the
contour of Fig. 1.6, however, we must replace s by ( —f 0« + jco\Ai — f02),
where f0 is a prescribed constant. Now if the contour, TJ, is described in
a positive sense, then the corresponding contour, r', in the L plane will
encircle the point, ( —1/Ka, jO), N = Z - P times in a positive sense,
where Z and P are the Y(s) zeros and poles, respectively, within contour
T'i. Since the poles of L(s)and Y(s)are identical, this serves to determine
 26 ELEMENTS OF MODERN CONTROL THEORY

jW

the number of closed loop transfer function poles that have either
positive real parts or a relative damping factor less than f 0-
This constitutes the Generalized Nyquist Criterion.
The computation of L(s0), where s0 = —f0co + jco\/l — for
0 < co< °° , may be simplified materially by using the relation

s* = (-l)*[T*(f) - K/rTr72U*G-)]co* (32)

T*( ) = Tchebychev polynomial of first kind of order k

U*( ) = Tchebychev polynomial of second kind of order k
which is proved in Appendix B.
Thus, a polynomial in s, of the form

F(s) = E
k— 0

becomes

F(s) = E (-l)4a*conT,(r) - jvT^UfcO-)] (33)

k= 0

after substituting Eq. (32).

THEORY OF LINEAR SYSTEMS 27

It is often convenient to display a frequency-response plot in a form

other than polar coordinates (which is the medium of plotting the
Nyquist locus).
The most general case of a rational transfer function in factored form
is shown in Eq. (18). This consists of terms of the type, s±n, (ts + l)±x,

and(—
\w„2
+ ^
a>„
s + 1
Each of these terms, with s replaced by jco, results in a complex number,
M/i£, having a distinctive form when plotted for M in decibels and « on a
logarithmic scale. This is shown in Figs. 1.7-1. 9. A frequency-response
curve, when plotted in the coordinate scale shown, is called a Bode plot.
The fundamental advantage of this representation is that the general
form of the frequency response can be quickly visualized and displayed
with a minimum of effort, since addition of basic forms rather than
multiplication is required. This is, of course, due to the logarithmic,
rather than numeric, representation of magnitude.

Note: The Bode plot for (jw)n is obtained by taking the mirror image
of the above lines about w = 1. In this case, 0 =nir/2.

Figure 1.7. Bode Plot of (1/jco)".

 ELEMENTS OF MODERN CONTROL THEORY
28

(deg)
0

Note: The Bode plot for (1 + j wt) is obtained by taking the mirror
image of the above curves about M = 0.

Figure 1.8. Bode Plot of (1 + jeer)-1.

Figure 1.9. Bode Plot of 1 /(I 4- 2jfu + (ju)2].

THEORY OF LINEAR SYSTEMS 29

By making the abscissa the phase angle, <p,instead of u, we obtain the

Nichols plot. A typical curve obtained in this manner is shown in Fig.
1.10.

Figure 1.10. Nichols Loci for 1/|1 + 2jfu + (ju)2l.

In either the Nichols or Bode plots, a variation in gain is reflected in a

shift of the magnitude curve either up or down. Also, the effect of adding
a particular network is easily apparent in the resulting variation in gain
and phase on the overall transfer function. This property is particularly
useful in synthesis and the determination of compensating networks to
modify the closed-loop performance. This subject will be treated briefly
in Sect. 1.3.
30 ELEMENTS OF MODERN CONTROL THEORY

Remark: In general, the determination of stability from the frequency-

response plots of complicated or nonminimum phase* open-
loop transfer functions can be done with assurance only from
a Nyquist diagram. This is because net encirclement of the
critical point is not clearly defined in terms of Bode or Nichols
plots except in simple cases. Consequently, one can deal with
phase and gain margins on a Bode or Nichols plot only when
these quantities have been correctly related to a Nyquist locus.

1.1.3 Root Locus

Consider again the feedback system shown in Fig. 1.2, for which the
closed-loop transfer function is given by Eq. (16) (repeated here for
convenience).

C(s) _ KG(s) (34)

R(s) 1 + IvG(s)H(s)

For a specified driving function, R(s), it is a straightforward pro¬

cedure to determine the response, c(t), by taking the inverse Laplace
transform of Eq. (34). To do this with a minimum of effort requires that
the roots of [1 -f KG(s) H(s)] be known. The root locus method^3) is a
systematic graphical procedure for obtaining these roots as a function of
Iv when quantity G(s) H(s) is expressed as a product of factors of the
form (s + p) and (s2 + 2fcos + u2).
The root locus method is distinguished by the fact that the roots of
[1 + KG(s) H(s)] are also the roots of the characteristic equation of the
system. (See Sect. 1.1.) Thus, all the properties of the system response
(transient and steady-state) are immediately available. This is not the
case for the frequency-response methods, where considerable additional
effort is required to obtain the features of the transient response.
In order to determine the roots of the equation

1 + KG(s)H(s) = 0 (35)

we must find the values of s that satisfy the two conditions

KG(s)H(s)| = 1 (36)

*A transfer function is minimum phase if it has no poles or zeros in the right-half s plane.
THEORY OF LINEAR SYSTEMS 31

/KG(s)H(s) = 180 ° (37)

Systematic rules for doing this are developed in Appendix D.

The root locus method is most efficient when the poles and zeros of
IvG(s) H(s) are available by inspection. In any case, a locus of roots of
Eq. (35) may be determined as a function of K in the s plane, and each
specific value of Iv corresponds to a specific set of roots of Eq. (35).*
As K varies from 0 to °°, the loci of roots are plotted in the s plane.
The root locus gives a particularly clear indication of how the closed-
loop poles shift with changes in open-loop gain, K. In fact, any system
parameter may be used as the “root locus variable” if Eq. (35) can be
rearranged such that this system parameter appears as the coefficient of
G(s) H(s).
For complex multiloop systems where the particular parameter to be
varied cannot be isolated [such as K in Eq. (35)] , the root locus is still a
powerful tool if a digital computer is used. The system equations may
be fed in as raw data, and the computer programmed to solve the
equations for discrete values of any parameter, thus permitting a root
locus to be plotted for this parameter.
A proper evaluation of the root locus is, therefore, of fundamental
importance in system design. Accordingly, this aspect will be emphasized
in the discussion that follows.
Let us note, first of all, that G(s) and H(s) are, in general, expressed in
fractional form as follows.

(38)

(39)

where Gi(s), G2(s), Hi(s) and H2(s) are polynomials ins. For a physically
realizable system, G2(s) H2(s) is of equal or higher order in s than
Gi(s) Hi(s) .
Substituting Eqs. (38) and (39) in Eq. (34) results in

C(s) _ KGi(s)H2(s) (40)

R(s) G2(s)H2(s) T KGi(s)Hi(s)

The number of separate root loci is equal to the order of the characteristic Eq. (35).
32 ELEMENTS OF MODERN CONTROL THEORY

The roots of the denominator of this expression, i.e., the closed-loop

poles, are obtained from the root locus once K is specified. The roots of
the numerator are simply the zeros of G(s) and the poles of H(s),
respectively. We may, therefore, write Expression (40) as

C(s) = KQx(s)
R(s) Q2(s)

Kjl (s - z<)

n (s - P;)
7=1

The transfer function for a unit step input becomes, therefore

Kli (s - Zi)
C(s) = — - . (42)

*11 (S - P;)
1=1

It is often asserted that the response c(t) = £-1[C(s)] is governed by a

few dominant poles, p>, in Eq. (42), requiring little or no computation to
ascertain the general features of c(t). We propose to examine this idea in
detail.
Expressing Eq. (42) in partial fractions (assuming that all the poles are
simple)

(43)
s ;=i * - Pi

Here*

Kli (-z i)
K„ = - 1=1 (44)

n (-Pi)
j=i

*In the terminology of complex variable theory, the Ko and Kt are the residues, at the
respective poles, of C(s).
THEORY OF LINEAR SYSTEMS 33

Kjj (s - z,•)
=

V
(45)
sir (s - py)

The inverse Laplace transform of Eq. (43) is, therefore

c(t) = K0 + H K i&vi
l. (46)

If a pole, p£, is complex, its corresponding residue, K£, is also complex.

Furthermore, in this case, there also exists a term of the form K£ept‘,
where the bar denotes complex conjugate. We then have

Iv£ep*‘ + K iept‘ = 2|K£|e<Tticos (co£t + <p{) (47)

where

Suppose there exists a pair of dominant poles,* px and p2, such that
| p, | ^ | Pi | for i = 2, 3, • • •. In order to ascertain if the poles p;
(i = 2, 3, •••) have a negligible influence on the response, c(t), it is
necessary to examine first the magnitude of the term K, ep*‘ compared to
2 | Ki | e<r*1
cos (coit + ifi), and second, the influence of pion Ki and ipi.
For definiteness, let us consider the configuration shown in Fig. 1.1.
Equation (42), for this case, is written as

K
C(s) = (48)
s(s - Pi) (s - p2)(s - p3)
where

Pi = -ai + j/h

P2 - ai - j/3i

*A dominant pole in the s plane is defined as that which has the smallest absolute value and
therefore has the predominant influence on the time response.
34 ELEMENTS OF MODERN CONTROL THEORY

Figure 1.11. Configuration of s Plane for Dominant Polar Pair with

Additional Real Pole.

and

K = — pip2p3. (49)

We note that via the Final Value theorem of Laplace transforms*

lim [c(t)] = lim [sC(s)] = 1. (50)

t—* » 8—>0

Now

* A
C(s) = Z (51)
i= 1 s - p<

where

A. = [(s - p.) C(s)]s=,pi.

The At may be obtained graphically from Fig. 1.1 1, as follows.

Assuming the system is stable.

THEORY OF LINEAR SYSTEMS 35

Ai = —/ -7, - 90 ° - 7,
abc "

A2 = ~/yx + 90 ° + 7,
abc

A3= —/180 °
c2d-
where

a = |px —p0| = |p2 - po|

b = |pi — p2|

c = |pi - p3| = |p2 - p3|

d = |p»|.

But

K = -pip2p3

= ~(ai2 + /?i2)p3

= a2d.

Therefore, A0 = 1. Furthermore

K _ ad
abc be

Now if d » a, then c« d and

a
Ai
b

which indicates that the influence of p3 on the magnitude of A3 becomes

vanishingly small. Furthermore, in this case, y2 —> 0, which indicates
that the phase contribution also becomes negligible.
36 ELEMENTS OF MODERN CONTROL THEORY

Also

A3 _K — —* 0 for c 2> a
c2d

and

qp3‘ = e~dt —> 0 for large d.

A widely used empirical rule is that p, may be neglected if | p» | > 6 | pi |

where pi is a dominant pole. The above analysis obviously holds for p,
complex.
Several general conclusions may be drawn regarding the influence of a
pole, p3, on the negative real axis on the time response of the system
when p3 may not be neglected. First, we note that A3 is a negative
quantity. This means that the term A3 ep3( subtracts from the time
response. This, in turn, means that the additional pole on the negative
real axis tends to make the system more sluggish. The overall effect is
to make the system behave as if the relative damping factor and natural
frequency were decreased.
Consider now the influence of a zero on a dominant pole pair. In
this case, we write

K(s - zi)
C(s) (52)
s(s - px) (s - p2)

where pi and p2 are as before and

P1P2
K =
Zl

This choice of K yields a steady-state unit step response of one. The

pole-zero configuration is now as shown in Fig. 1.12. The partial
fraction expansion of Eq. (52) is

C(s) = E -Al_.
i—0 S-Pi

An evaluation of the residues gives

THEORY OF LINEAR SYSTEMS 37

Im

Figure 1.12. Pole-zero Configuration for Dominant Pole Pair and

Additional Zero.

Kd
Aq
a2

Ax —/y, - 90 ° - 7i
ab

A2 = — /- t2 + 90 ° + 7x.
ab

Also

K = PtP2 _
Zi d

and

Ao — 1

while

cK a c
Ai A2I
ab b d
38 ELEMENTS OF MODERN CONTROL THEORY

For d> a, obviously, c « d, 72 —>0, and the influence of the zero on

the system response becomes vanishingly small as in the case of the
additional pole. However, when the zero zj is not negligible, it adds a
phase lead of t2 degrees rather than a phase lag (which was the case for
the pole). As a result, the first maximum in the system response is
reached sooner. The overall effect is, therefore, a system with a faster
response time (than the system without the zero) and an apparent
increase in natural frequency and relative damping factor.
The influence of additional poles and zeros on the response of a
system having a dominant complex pole pair may be exhibited qualita¬
tively as shown in Fig. 1.13.
In order to complete the discussion, it is necessary to consider a
dominant complex pole pair with an additional small dipole (nearly
coincident pole-zero pair) anywhere in the s plane. The effect of this
dipole is virtually negligible, since the vectors drawn from the dominant
pole to the dipole are nearly equal and the phase contribution from the
zero cancels that due to the pole. Furthermore, the term, K* ep;‘, due to
the pole in the dipole is very small, since K, is small because of the
nearby zero.
When the additional poles and zeros have appreciable influence on the
system response, the general qualitative features of this response may
be quickly determined via the graphical methods used here. However,
various nondimensionalized unit step responses obtained via computer are
shown for reference in Table 1.2. These are useful for preliminary
design studies.

c (t)

Figure 1.13. Unit Step Response Showing Influence of Additional Pole or Zero.
THEORY OF LINEAR SYSTEMS 39

Poles
Complex
of
Pair
2:
Case
Real
1:
Two
Configuration
Pole-Zero
Selected
for
Response
Step
Unit
1.2.
Table
Q)

(})0 ‘98U0dB9H UI9}eXg

 ELEMENTS OF MODERN CONTROL THEORY
40

C and
Poles
Complex
of
Zero
One
Pair
4:
3:
Case
(Pole-Zero
Configuration
Selected
for
Response
Step
Unit
1.2.ont’d)
Table

ft)o ‘osuodsoH raojsXs

(l)o 'osuodsay uiojsAg

THEORY OF LINEAR SYSTEMS 41

Pole
Real
One
and
Poles
Complex
of
5:
Pair
6:
Case

(])o ‘esnodssH cnejs^s

 ELEMENTS OF MODERN CONTROL THEORY
42

Re

C with
Poles
Complex
of
8:
Case
Parr
Pair
7:
(Pole-Zero
Configuration
Selected
for
Response
Step
Unit
1.2.ont’d)
Table

(j)o ‘ssnodseH inajsAg

THEORY OF LINEAR SYSTEMS 43

The foregoing ideas provide a clear qualitative description of the

system response, given the closed-loop pole-zero configuration. Because
both the transient and steady-state features of the motion are readily
apparent from the root locus, this technique exhibits marked superiority
over the classical frequency-response methods. However, the two
approaches tend to supplement rather than conflict with one another.
The addition of poles and zeros to a root locus diagram, for example,
generally requires that the loci be redrawn and new operating points
(closed-loop poles) obtained for specified gain. In a Bode or Nichols
plot, the new gain and phase margins are obtained virtually by in¬
spection, with little or no rework required.

Example 3: Consider the pitch plane autopilot of Fig. 1.A3 (in

Appendix A). Bending, slosh, and gyro dynamics are neglected, with
the result that the open-loop transfer function may be expressed (see
Fig. 1.14) as

KaKcKs^c
(s+K,)
(.+£)
I\G(s)H(s) = (53)
T

S(S2 - Ma) (S + Kc)

or

k + V<K*S
/
+ *>
KG(s)H(s) = (54)

The following parameter values are used.

Kj = 0.20 sec~ r = 0.04 sec

K« = 0.333 sec tic = 6.45 sec :

Kc = 15 sec-1 na = 2.14 sec-2

Ka = 3 (nondimensional)
44 ELEMENTS OF MODERN CONTROL THEORY

CJ

0>
>

Q.
O
3
<
O
c

£
"O

s
a

§
£3

1
£
THEORY OF LINEAR SYSTEMS 45

Replacing s by jco in Eq. (35) and letting u>vary from zero to infinity,
we obtain the Nyquist plot of Fig. 1.15.
Since there is one open-loop pole in the right-half plane (P = 1), and
one clockwise encirclement of the (-1, jO) point as w varies from- « to
+ °°(N = —1), we find from Eq. (29) that Z = 0; hence the system is
stable.
An inspection of the diagram indicates^that the upper and lower gain
margins are Xi = 1/0.23 = 4.35 = 12.8 db and \2 = 1/6.32 = 0.158 =
- 16.2 db, respectively. Consequently, if KA is assumed to be the only
adjustable parameter, instability occurs for Kj > 3 X 4.35 = 13.1 or
KA < 3 X 0.158 = 0.474. The phase margin is found to be 26 degrees.
The Bode and Nichols plots for Eq. (35) are shown in Figs. 1.16 and
1.17, respectively. Also indicated are the gain and phase margins noted
above.
The Nichols plot is used extensively by many aerospace agencies to
represent complicated open-loop transfer functions. In order to discuss
the general features of such a diagram, we will include the effect of three
bending and two sloshing modes added to the present example. A
Nichols plot of the general form shown in Fig. 1.18 is then obtained.
The terminology in this figure is that in general use in the aerospace
industry for autopilot control of launch vehicles. The gain and phase
margins related to specific modes are immediately apparent. If additional
phase lag exceeding yi degrees is introduced at unity gain, the rigid body
mode will become unstable. Similarly, and 72 represent the factors
that, if exceeded either by an increase or decrease in gain, will make
the rigid body mode become unstable. Also, the addition of a phase
lead of 73 degrees (at unity gain) will cause the first bending mode to
become unstable.
Autopilot performance specifications are often expressed in terms of
gain and phase margins for specific modes. The ease with which these
quantities are determined on a Nichols plot makes this type of specifi¬
cation particularly attractive.
We may also analyze the system of Fig. 1.14 using root locus. The
poles and zeros of the open-loop transfer function are available from
inspection of Eq. (53), and the root locus is shown in Fig. 1.19.
Certain correlation with the results obtained via frequency-response
are immediately evident. Thus, it was found from the frequency-
response analysis that instability occurs for IC < 0.474 or IC > 13.1.
According to Fig. 1.19, there will be closed-loop poles in the right-half
plane if KA < 0.47 or KA > 13.2. That the agreement is quite good is not
surprising, since a lack of agreement would simply indicate an error
somewhere in the analysis.
46 ELEMENTS OF MODERN CONTROL THEORY

Figure 1.15. Nyquist Plot for Example 3.

THEORY OF LINEAR SYSTEMS 47

(ggp) 0)

Exampl
for
Plot
Bode
1.16.
Figure
(rad/
Frequen
sec)
48 ELEMENTS OF MODERN CONTROL THEORY

Figure 1.17. Nichols Plot for Example 3.

For = 3, the closed-loop poles and zeros, obtained from Fig. 1.19,
are shown in Fig. 1.20, where the pole at the origin represents a unit step
input. In calculating the response for this case, the dipole at (-0.2,
-0.36) and the pole at -29.9 are very nearly negligible. Therefore, from
Eqs. (46) and (47)

c(t) = 0(t) = K0 + 2|Ki|e<n* cos (wit + <Pi) +

The pole locations are

Pi — or T- j&>i— —2.42 + j 6. 1

P2 — — 4.93.
THEORY OF LINEAR SYSTEMS 49

Slosh
and
Include
Bendin
Modes
3with
Examp
for
Plot
Nichol
1.18.
Figure

<P
(deg)
 ELEMENTS OF MODERN CONTROL THEORY
50

Im
THEORY OF LINEAR SYSTEMS 51

3.
Examp
for
Config
Pole-Z
1.20.
Figure
Loop
Closed
52 ELEMENTS OF MODERN CONTROL THEORY

From this it is apparent that the step response is governed primarily

by a constant, a damped oscillation, and a time decaying term. In fact,
we find immediately

oscillation frequency: an = 6.1 rad/sec

(j 2 42
relative damping factor: ft = - = - 1- = 0.37
vV + C0X2 V 6.12 + 2.422

time constant of decaying exponential: r = — = - = 0.203.

p2 4.93

As before, we find that the unit step response in the steady-state is

unity, which means that K0 = 1. Therefore, from Eqs. (44) and (45)

Kzx
K0 — - = 1
Pi P2

kJ — KA3
lm|
A0AxA2

K(p2 — zx)
k2 =
p2A22
Ao = |px = 6.56
Ax = 2wx = 12.2

A2 = |Pi — p2| = 6.6

A3 = Pi —zx| = 6.12
Zx = -3.

These values are measured directly on Fig. 1.20. We have, finally

THEORY OF LINEAR SYSTEMS 53

The first of these expressions shows how the amplitude factor of the
oscillatory response is influenced by the presence of the additional pole
and zero.
The significant fact about K2 is that the exponentially decaying term
K2ep2‘ will be positive or negative, depending upon whether the zero or
the pole is nearer the imaginary axis. All the essential features of the
response are thereby delineated. y
We will conclude the discussions of this section by considering various
refinements in root locus technique developed in recent years. The usual
method of obtaining the root locus is by graphical trial and error, aided
by various systematized procedures. However, by analyzing the root
locus as a continuous function, certain features that are obscured by the
graphical approach become evident.

1.1.4 Analytic Root Loci

According to Eq. (35), the root locus is the function that satisfies

Y(s) = KG(s)H(s) = — 1. (55)

We may write this as

G(s)H(s)
= - I. (56)

Since K is always real, any value of s for which G(s) H(s) is real will
be a solution of Eq. (56). Thus, the root locus is defined by the con¬
dition*

Im [G(s)H(s)] = 0

or equivalently
Im [KG(s)H(s)] = 0. (57)

Equation (57) is valid for positive or negative Iv, which means that this
yields the root locus for - oo < K < <». We may add the relations

Re [G(s)H(s)] < 0 for K > 0

Re [G(s)H(s)] > 0 for K < 0

&fIn( ) = imaginary part of ( );Re( )= real part of ( ).

 ELEMENTS OF MODERN CONTROL THEORY
54

to distinguish the loci for positive or negative K, but this is not necessary
for present purposes.
We will, therefore, refer to Eq. (57) as the equation of the root locus,
regardless of the sign of K.
Usually, Y(s) may be written as a ratio of polynomials in s, viz.

P(s)
Y(s) = KG(s)H(s) = (58)
Q(s)'

In this case, Eq. (57) may be expressed as

-P(s) Im P(s) Re Q(s) — Re P(s) Im Q(s) _ Q

Im
-Q(s)J [Re Q(s)]2 + [Im Q(s)]2

which reduces to

Im P(s) Re Q(s) — Re P(s) Im Q(s) = 0. (59)

This is the equation of the root locus. If P(s)= a real constant, then Eq.
(59) simplifies to

Im Q(s) = 0. (60)

Two features of the root locus may be derived immediately from Eq.
(59). First, replacing s by a + jo, setting « equal to zero, and then
solving the resulting equation for a yields the “breakaway points” for
the real axis. Also, if s is replaced by a + jo and <j is then set equal to
zero, the solution of the resulting equation for o will yield the points
of intersection with the imaginary axis.
 THEORY OF LINEAR SYSTEMS
55

In certain simple cases Eq. (59) represents some well-known ele¬

mentary curve. Consider the case

K(s -f- ai)

Y(s)
(s + aiy + ft2'

From Eq. (58) and (59), we have, after replacing s by a + jw

(a + a2)2 + co2 = (ai — a2)2 + ft2.

This is the equation of a circle of radius R = [(ai - «2)2 -ft ft2]1'2 and
center at ( —a2, jO). The root locus is shown in Fig. 1.21.
In similar manner, one finds that the root locus for

Y(g\ _ K[(s + q2)2 -f- ft2]

[(s + ai)2 + ft2]

shown in Fig. 1.22 is also a circle, centered at the point a0 on the figure.
It is apparent that only in simplified cases is it possible to obtain
useful information with relatively modest effort. When s is of high
order, the resulting algebraic complexity makes this approach quite
unattractive.
In certain cases, however, it is possible to construct complex root
loci from simpler ones by making use of the following theorem due to
Steiglitz.C1 7>
“Let Ti be the root locus associated with Gfts), and let T2 be the
locus associated with-G2(s). Then the intersections of Ti and T2 are on
the root locus associated with Gfts) G2(s).”
This theorem is useful when the total open-loop transfer function can
be broken up into a product of two other transfer functions whose
root loci can be drawn immediately. The concept of an arbitrarily
located coincident pole-zero pair in combination with the above theorem
greatly facilitates the construction of various types of root loci. We will
illustrate the application of these ideas in the construction of the root
locus for
56 ELEMENTS OF MODERN CONTROL THEORY

Im

K(s + a-i)
Figure 1.21. Root Locus for Y(s) =
(s + ai)2 + Pi2
THEORY OF LINEAR SYSTEMS
57

(s T ai)[(s + C*2)2
+ ^2 ]

The pole configuration for this case is shown in Fig. 1.23a, which also
shows a coincident pole-zero pair at ( —7, jO). This arrangement may be
broken up into the two separate pole-zero configurations shown in Fig.
1.23b and c. For each of the two latter cases, the root locus may be
drawn immediately as shown. Superimposing these two loci yields points
of intersection that define a point on the root locus for the given transfer
function. Putting the coincident pole-zero pair at another location along
the real axis again yields two further points on the root locus of the
original function. By taking a sufficient number of locations for the
coincident pole-zero pair, a number of points on the root locus for the
given transfer function will be obtained so as to completely define the
curve. Figure 1.24 shows the final form of the root locus constructed in
this manner.

1.2 STABILITY ANALYSIS-SAMPLED DATA SYSTEMS

The analysis of sampled data (variously called discrete, or digital)

control systems is based mainly on the development of specialized tools
that permit the application of the usual linear methods (Nyquist criterion,
root locus, etc.) to the problem at hand. In this respect, the Z transform
and its extension, the modified Z transform, play a fundamental role.
The manner in which these concepts are treated in most standard texts
leaves much to be desired. Representing a sampler output as a sum of
terms, each of which is a constant multiplied by a unit impulse, is not
without its mystical overtones. In fact, without a corrective term
(omitted in most texts), such a representation is wrong.
It would seem appropriate, therefore, to discuss the underlying ideas
of the mathematical model of the sampling process in a plausible and
logical manner. It is neither necessary nor desirable to assume that a
pulse width is “infinitely small.” One may develop the concept of a Z
transform without appealing to this assumption, which does violence to
physical intuition, and which, therefore, casts some doubt on the validity
of the results.
Having laid a firm foundation (hopefully) for the fundamentals, the
prominent aspects of the theory follow readily. These include the
essentials of Z transform algebra, application to feedback systems, sample
and hold, transport lag, etc. An application of the methods to a problem
in attitude control of a launch vehicle will conclude the discussions.
 ELEMENTS OF MODERN CONTROL THEORY
58

Im

Im

Figure 1.23. Root Locus Construction by Combination of Simpler Loci.

 59
THEORY OF LINEAR SYSTEMS

Im

D
O

Root
Final
1.24.
Figure
 60 ELEMENTS OF MODERN CONTROL THEORY

1.2.1 The Z Transform

The fundamental sampling operation will be discussed with reference

to Fig. 1.25. It is assumed that the sampling rate is constant, with period
T, and that the sampler is closed for an interval of length 7. The output,
r*(t), may, therefore, be expressed as

r*(t) = h(t) r(t) (61)

where h(t)is a time function of the form shown in Fig. 1.26. By taking
the Laplace transform of both sides of this equation, we find

R*(s) = H(s) & R(s) (62)

where the symbol & denotes complex convolution,* and

1 - e-7s
H(s) - £[h(t)] = (63)
s(l — e~TS)

Noting that

we find that the only poles of H(s) are the zeros of

(1 - e~TS).

Consequently, we find that the poles of H(s) are given by

Sn JIlC05

n — 0, =t 1, =t 2, • • •

where

CO

*Cf. Ref. 3, p. 275.

THEORY OF LINEAR SYSTEMS 61

r(t)

(b) Sampler
r*(t)

(c) Output Function

Figure 1.25. Sampler Operation.

If we denote the residue of H(s) at the pole, s„, by k„(s„), then we have

1 — e-/"“sY

27rnj
n = 0, ±1, ±2, ••%
62 ELEMENTS OF MODERN CONTROL THEORY

h(t)

Figure 1.26. Modulating Time Function.

Making use of a well-known theorem in Laplace transforms,* we may

then express Eq. (62) in the form

1 — e~inu°y
R*(s) = £ R(s — s„) (64)
2irnj

n = 0, rbl, ±2,

Since

lim
1 — e~J'"“vy 1
n— >0 2rrnj T

we may write Eq. (64) as

1 _ e-jno>sy
R*(s) = 1 R(s) + E R(s - s„) (65)
-L n 2xnj

II — rbl, rb2, • • • .

Now by taking the inverse Laplace transform, we find

r*(t) = r(t) + r(t) E — [sin nwst — sin ncos(t — 7)] (66)

T n II7T

n = 1, 2, 3, •••

*Cf. Ref. 3, p. 277.

THEORY OF LINEAR SYSTEMS 63

This result displays the output of the sampler as an attenuation of the

input plus an infinite number of higher harmonics of decreasing
amplitude.
Equation (66) is exact; no approximations of any kind have as yet
been introduced. Let us now assume that y is a small quantity (small in
the sense that second and higher order terms in the series expansion for
/

Q-jnusy are negligible). Then

(67)

and Eq. (64) reduces to

R*(s)
« ^ X R(s- jmo,)
n
(68)

n = 0, ±1, ±2, ••• .

But 1/T is precisely the residue of Ar(s) at the pole sn, where*

5r(t) = X 8(t — nT)

n

n = 0, 1, 2, • • •

Ar(s) = £[Sr(t)] = 1 + e-*1* + e-2r« + •••

(69)

It follows, therefore, that Eq. (68) is equivalent to

R*(s) = yAr(s) & R(s)

which, after taking the inverse Laplace transform, becomes

r*(t) = yr(t)5r(t). (70)

5(t) = unit impulse function.

64 ELEMENTS OF MODERN CONTROL THEORY

Thus, in fact, it does appear that the sampler output is a series of unit
impulses, with each unit impulse, 5(t — nT) , being multiplied by the
factor 7r(t - nT). The purpose of the preceding development has been to
emphasize the fact that the duration of switch closure, 7, need not be
infinitesimal. For the mathematical representation of Eq. (70) to be
valid, it is merely necessary that higher order terms in the expansion for
e-,n^y be negligible. This situation prevails when (7/T)2 is small
compared to 7/T.
It is also true that many authors assume, ab initio, that the equation

r*(t) = r(t)5r(t) = r(nT)5(t - nT) (71)

71=0

represents the sampler output. This is wrong. The multiplying factor, 7,

which represents the switch contact duration, must be included as shown
in Eq. (70).
We may, however, take the point of view that Eq. (71) represents the
sampler output provided the factor, 7, is absorbed in the gain of the
transfer function following the sampler. This approach does indeed
afford a degree of convenience in the development that follows, and we
will therefore adopt it. Consequently, we will take Eq. (71) to represent
the sampler output, subject to the aforementioned understanding. In this
case, Eq. (68) becomes

R*(s) = ~ Z R(s - jnco,) (72)

n

n = 0, ±1, ±2, %• %.

With these preliminaries concluded, we may proceed to a formal

development of the Z transform and its application to the analysis of
feedback control systems.
We now seek an expression for R*(s) when R(s), the Laplace
transform of r(t), is a rational algebraic function of s. From Eq. (71), we
Find that R*(s) may be expressed in the form of a complex convolution
as follows.*

R*(s) = £[r(t)5r(t)] = R(s) & Ar(s) (73)

*Cf. Ref. 3,p. 275.

 THEORY OF LINEAR SYSTEMS
65

where Ar(s) is defined by Eq. (69). But if

A(s)
R(s) (74)
B(s)

where A(s) and B(s) are polynomials in s, and the roots of B(s) are all
simple, then* y

A(sjO
R*(s) E A r(s S*) (75)
k= 1 B'(st)

where

B'(st)
=^B(S)]s=ii
q = order of polynomial B(s)
Si = a pole of R(s) ; i.e., a root of B(s) = 0.

Making use of Eq. (69), this last equation may be expressed as

R*(s) y-' A(si) _i _ (76)

h B'(si)Li Q—T(s — SA)_

We now define the Z transform of a function, r(t), as the Laplace

transform of the sampled function r*(t) with esT replaced by z.
When r(s) has the form (74) and all of its poles are simple

y-' A(s k) z
Z[r(t)] = R(z) = (77)
k=i B'(st) Lz - eT3k_

If R(s) has a repeated pole of order (m + 1), it can be handled by the

Z transform operation by noting that

1 1 d™ / 1 \
- t- = (_l)m . (- ).
(s + a)m + 1 m! dam \s + a/

For example, if

l
Ri(s)
(s + a)2

*Cf. Ref. 3, p. 277.

66 ELEMENTS OF MODERN CONTROL THEORY

then*

Z[ir(t)] = Z[Ri(s)] = Ri(Z)

= Z
1
U
.(s + a)2_ da _ s +f aJ

The following illustrates the use of the Z transform in a typical case.

Example 4: Let it be required to find the Z transform of the function

whose Laplace transform is

R(s) - 18(3+2) .
(s + 3)(s + 6)2

We may express this in partial fractions as follows.

24 2
R(s) +
(s + 6) 2 (s + 6) (s + 3)

Now, by letting

2
Ri(s)
s + 6

2
R2(s)
s + 3

we find at once that

2z
Ri(z) =
z — e~bT

2z
R2(z) = _ p— 3T ’
z — e

Also, if

24
R*(s)
(s + 6) 2

The Z transform operation is expressed by any one of the three equivalent symbols shown.
THEORY OF LINEAR SYSTEMS 67

then

R3(z)

or

R3(z) 24zTe_6r
(z - e_6r)2'

The desired Z transform is, therefore

x 2z 2z 24zTe-6 T
R(z) = - — - 1- - - .
z — e-6T z — e~3T (z — e-6T)2

A short list of Z transforms is given in Table 1.3. Various

mathematical properties satisfied by the Z transform may be found in
the literature. (18’19)

Table 1.3. Z Transforms of Elementary Functions

R(s) r(t) R(z)

-nTs -n
e 6(t - nT) z

1 z
u(t)
s z - 1

i Tz
t
2 2
s (z - 1)

i -at z
e
s + a -aT
z - e

b z sin b T
sin bt
s
2 +
u2
b z2 - 2 z cos bT + 1

-aT
b “at * ht z e sin b T
e sm bt
2 -aT , ^ ^ -2aT
(s +a)2+b2 z - 2 ze cos bT + e
 ELEMENTS OF MODERN CONTROL THEORY
68

In order to carry over the block diagram concepts of Laplace

transform methods into Z transform analysis, one further relation must
be established. As a preliminary to this, we must show that R*(s) is a
periodic function.
For a particular value of s, we have, from Eq. (72)

R*(si) = - X R(si - jnw.)

T n

n = 0, ± 1, ±2, ••• .

Therefore

R*(si - jws)
1_ R(si — jnw, — j«.)
T n

=^ L R[si
- + n)].
By letting m = 1 + n, and noting that

m = — °° when n = — »

m = -f- co when n = -|- 00

we may write the above equation as

R*(si - j«.) = i X R(si ~ jmw,)

T m

m = 0, ±1, ±2, ••• .

But the right-hand side of this equation is R*(si) by definition.

Therefore

R*(si - jws) = R*(si). (78)

Consider now the simple open-loop system of Fig. 1.27. It follows

directly from Laplace transform block diagram algebra that

C(s) = G(s)R*(s). (79)

THEORY OF LINEAR SYSTEMS 69

* o- -
c*(t)
-%º

r(t) r*(t) 1 c(t)

G(s)

Figure 1.27. Simplified Open-Loop Sampled System.

If c(t) were sampled, then the Laplace transform of the sampled

function would satisfy Eq. (72); viz.

C*(s) = i Z C(s - jncos)

-L n

n = 0, =1=1, ±2, •••

Using Eq. (79), this becomes

C*(s) = - X) G(s)- jnco,)R*(s- jnco.).

T n

Noting that R*(s) satisfies the periodic property Eq. (78), this further
reduces to

C*(s) = R*(s) X G(s - jnwj)

n = 0, ±1, =b2, %%% .

But the term inside the brackets is G*(s) by definition. Consequently

C*(s) = R*(s)G*(s). (80)

If now we replace eaT by z, this equation may be written in terms of Z

transforms, as follows.

C(z) = R(z)G(z) (81)

This is the fundamental equation that permits us to express Z

transforms in the block diagram notation analogous to Laplace trans¬
forms.
70 ELEMENTS OF MODERN CONTROL THEORY

It should be emphasized that C(z) implies a function that is sampled;

this is not the case for c(t) in Fig. 1.27. The concept of a Z transform for
a continuous function is, however, very useful, and this requires that
C(z) be properly interpreted. We may think of c*(t) as the output of a
fictitious sampler, synchronized with the system sampler, as shown by
the dotted lines in Fig. 1.27. Consequently, the output function derived
by Z transform analysis yields values of the time function only at the
sampling instants.
It should also be pointed out that the factor, y, of Eq. (70) is now
assumed to be absorbed in the transfer function, G(s), in order that the Z
transform method yield valid results.
The results obtained thus far may be extended to permit the analysis
of analog-to-digital converters in the control loop. These devices have the
property that a continuous (analog) input produces a discrete output of
the form shown in Fig. 1.28. This suggests that the operation may be
represented schematically by the diagram of Fig. 1.29, which also
contains a transport lag (pure time delay). The switch closure time is
assumed negligibly small compared with T. A schematic of this type is

Figure 1.28. Output of Sample and Hold.

THEORY OF LINEAR SYSTEMS 71

widely used to represent the dynamics of a digital computer in a control

loop, since the output of the computer consists of fixed quantities whose
value is changed in discrete steps at regular intervals. The transport lag is
included to account for the fact that an output signal is delayed for a
finite interval after an input is applied.

Hold Circuit Transport Lag

r(t) r*(t) C2<1>

G (s> Gtl' (s)'
H K'

Figure 1.29. Sampler Followed by Hold and Transport Lag.

It follows readily from Fig. 1.28 that the output may be represented
as*

ci(t) = E r(nT)[u(t - nT) - u(t - nT - T)].

n= 0

Taking the Laplace transform

0 —nT s 0 — (n-Fl )Ts

t'i(s) = E r(nT)
n= 0
Is S

ce

= E r(nT)e_nrs
n= 0

However, from Eq. (71)

R*(s) = E r(nT)5(t - nT).

n= 0

It follows that

Ci(s) = R*(s)G„(s) (82)

where

0,(8) = 1 ~ e~- (83)

s

u(t) = unit step function.

72 ELEMENTS OF MODERN CONTROL THEORY

is the transfer function of the “holding circuit.” The transfer function of

the transport lag is simply

Gl(s) = eh! (84)

where TL is lag duration.

In the foregoing development, the analysis proceeded directly from
the form of the output shown in Fig. 1.28. This led to the result that the
operation could be interpreted as an impulse input applied to a device
having a transfer function of form (83). The effect of a small switch
closure duration is reflected mainly in a slightly distorted “corner” in the
output curve shape of Fig. 1.28. Neglecting this, the result given by Eq.
(82) is exact. Consequently, when a hold circuit, given by Eq. (83),
follows the sampler, the factor, y, does not appear, and the discussion
following Eq. (70) does not apply. Stated another way, when a hold
circuit follows the sampler, Eq. (71) rather than Eq. (70) is used to
represent the sampler. The gain of the transfer function following the
sample and hold is not modified by the factor, y.
There is now sufficient information to proceed with the Z transform
analysis of sampled data feedback control systems.
However, in order to consider the influence of transport lag without
requiring a further digression, we will introduce the concept of the
Modified Z Transform at this point. If, in the diagram of Fig. 1.27, we
introduce a fictitious delay AT as shown in Fig. 1.30, we can write a new
function of G(s) as follows.

Z*[G(s)] = G(z,m) = £[g(t - AT)«r(t)] (85)

where

£[g(t - AT)] = G(s)e-Ars

m = 1 — A.

r(t) c(t)
0(8) *> At
T

Figure 1.30. Sampled Data System with Fictitious Delay.

THEORY OF LINEAR SYSTEMS 73

The output, c(t), in Figs. 1.28 and 1.30 is continuous, but the Z
transform of the output yields values only at the sampling instants.
However, by using the modified Z transform defined by Eq. (85), the
actual output can be obtained by varying A between 0 and T. One can,
in fact, develop a complete theoryU9) based on the modified rather than
conventional Z transform, but this is Jbeyond the scope of the present
discussion. The modified Z transform will be used only to facilitate the
analysis when transport lag is included in the control loop.
We note only that if G(s) may be expressed as

A(s)
G(s)
B(s)

where A(s) and B(s) are polynomials in s (the latter of order q) and all
the roots of B(s) are simple, then— analogous to Eq. (77)

1 s lT
r, , \ v' Ms*) (86)
G(z,m) = z 1 2^ 777- t .1 - e-7’(,-s/t)J
k=1 B (s^

The following equation relates the conventional to the modified Z

transform.

G(z) = lim zG(z,m) v J

m —>0

A short list of modified Z transforms is given in Table 1.4.

In Fig. 1.31 are summarized the basic properties of Z transform
algebra. These are based on the fundamental relation, Eq. (81). Parts (a)
and (b) of Fig. 1.31 are straightforward, while the relation given in part
(c) is derived as follows. By virtue of Eq. (79) we have

Ri(s) = Gi(s)R*(s)

C(s) = G2(s)R* (s).

But by Eq. (80)

R?(s) = Gi*(s)R?(s).
Combining the last two equations
C(s) = G2(s)Gi*(s) R*(s).
74 ELEMENTS OF MODERN CONTROL THEORY

Table 1.4. Modified Z Transforms of Elementary Functions

R(s) r(t) R(z , m)

-nTs m-l-n
e 6 (t - nT) z

1 i
u(t)
s z - 1

i m T T
t
s
2 z - 1 ' ,(z - 1)
2
-amT
i -at e

s + a -aT
z - e

b z sin mbT + sin (1 - m) b T

sin bt
2 2 2
s + b z - 2 z cos bT + 1

. . _ -aT . , , ^1 -amT
b -at . . . |zsinmbT +e sin(l-m)bT|e
e sin bt
2 -aT . _ -2aT
(s +a)2+b2 z - 2 z e cos b T + e

The sampled function, C*(s), satisfies Eq. (72); viz.

C*(s) = i £ C(s - jnw„)

-L n

n — 0, =bl, i2, ••• .

Therefore

C*(s) = — £ G2(s — jncos)Gf(s — jnojs)R*(s — jncos).

n

Utilizing property (78), this reduces to

C*(s) = R*(s) G*(s) ^ Z G2(s

- 4
- jno)s)
n

However, by Eq. (72), the quantity in the brackets is simply G2*(s).

Therefore

C*(s) = R*(s) Gf(s) G*(s).

(a)Sim
Sam
Dat
Sys
THEORY

R(s)
OF LINEAR

C(z) = R(z) G(z)

-
SYSTEMS

22L G(s)
- -a
V'
I | o -

C(s)
—
75

c<z>
%º

o-

C(z)
r—** » o— %º
1
R(s) . R(z) .
1 C(s)
G (s) G (si
2V

C(z)
=R(z)Z[Gx(s)G2(s)j= R(z)
GJG,,(z)
(b) Sampled Data System With Cascaded Elements

C(z)= R(z) Gx(z) G2 (z)

(c) Sampled Data System With Synchronized Samplers

Figure 1.31. Z Transform Algebra.

If now, esT in this equation is replaced by z, we obtain finally

76 ELEMENTS OF MODERN CONTROL THEORY

It is important to emphasize that in general

Gi(z)G2(z) ^ GiG2(z)

where, in accordance with common usage

Gi(z)G2(z) = Z[Gi(s)]Z[G2(s)]

G!G2(z) = Z[G1(s)G2(s)].

Using the relations shown in Fig. 1.31, the equations for the feedback
systems shown in Fig. 1.32 may be derived without difficulty.
When a pure lag appears in the control loop, the open-loop transfer
function may be written as

G(s) = e-*’D«G0(s) (88)

where To is the duration of the lag. If we write

T0 = XT

where T is the sampling period, then three cases may be distinguished.

1. 0 < X < 1.

2. x > l.

3. X = k = integer.

For Case 1, Eq. (88) becomes

G(z) = Z[G(s)] = G0(z,m)]m=i_x. (89)

For Case 2

G(z) = z-fG0(z,m)]m=i_x' (90)

where

X — l -)- X'

l = integer

0 < X' < 1.

THEORY OF LINEAR SYSTEMS
77

.—Pf'o-E®*-

Figure 1.32. Selected Feedback Configurations for Sampled Data Systems.

78 ELEMENTS OF MODERN CONTROL THEORY

Finally, for Case 3

G(z) = z-*G0(ss). (91)

Having derived the properties of Z transforms and the rules for

manipulating them, we must now show how this procedure yields
information on the stability and response of sampled data systems.
The key fact is that the Laplace transform of a sampled time function
contains terms of the form esT (which has an infinite number of roots).
We pass to the Z transform by the change of variable

z = esT (92)

In doing this, the left-half s plane is transformed into the interior of

the unit circle in the Z plane.* (See Fig. 1.33.) Consequently, for a
sampled system to be stable, it is necessary and sufficient that the poles
of the Z transform of the overall system lie within the unit circle.
It is also of interest to examine the manner in which a constant
damping line in the s plane is transformed into the Z plane. Referring to

Figure 1.33. Mapping of Left Half of s Plane into Z Plane.

*An analytical proof of this result is contained in standard texts.*18'18'

THEORY OF LINEAR SYSTEMS 79

Fig. 1.34, we note that the constant damping line is described by

w = — cr tan 7

where

S = — <7 JCO )/

7 = cos -1fi

and fi is the relative damping factor. Therefore

2 — qsT _ gf(-( T 4- jco)

— q—(TT(1+ j tan y)

This is the equation of a logarithmic spiral in the Z plane. (See Fig.

1.34.)
In a similar manner, it is easy to show that lines of constant co and
constant a in the s plane map into radial lines and circles, respectively, in
the Z plane, as shown in Fig. 1.34.
In describing the response characteristics of a sampled function, it is
convenient to map the circles, w„2 = constant, in the s plane into the Z
plane. For this case, the circle in the s plane is described by

S = <7+ j (a,! — a2)112

so that

= qst == + 7(^2 — a2)1/2J

= e •’ cos 2ir
(I -wn2 a2\12 .
) + i sin
fa>n2
2ir[ --
—cr2Y/2
I
V w„2 / \ ws2 / _

where

2 7T
= "TTr-
T

Figure 1.35 shows this mapping for values of u, from wn - «,/ 2 to w,

= ws/36. Lines of constant damping from f =0.05 to f = 0.9 are also
shown.
80 ELEMENTS OF MODERN CONTROL THEORY

Im

Figure 1.34. Transformation of Constant Damping Line.

THEORY OF LINEAR SYSTEMS 81
82 ELEMENTS OF MODERN CONTROL THEORY

The use of such a diagram in connection with a Z plane root locus will
be discussed subsequently. Note that a different diagram is required for
each different value of sampling period, T.
In order to investigate the stability properties of a sampled data
feedback control system of the types shown in Fig. 1.32, it is necessary
to determine the location of the closed-loop poles of the system. In Fig.
1.32a, for example, one must determine the roots of

1 + GH(z) = 0.

The root locus formalism developed for continuous systems carries

over directly in the present case. However, recalling that the system
output is given at the sampling instants only by the Z transform method,
the locations of the closed-loop poles in the Z plane are interpreted as
follows.

Location of Closed-Loop Pole Mode of Transient Behavior

I. Outside the Unit Circle Unstable Operation

II. Inside the Unit Circle Stable Operation
(a) Real Pole in the Right-Half of Decaying Output Sequence
the Unit Circle
(b) Real Pole in the Left-Half of Alternating Output Sequences of Di¬
the Unit Circle minishing Amplitude
(c) Conjugate Complex Poles in Damped Oscillatory Output Sequence
the Unit Circle

The application of the methods considered thus far will be illustrated

by the following examples.

Example 5: A very elementary sampled data feedback system is

shown in Fig. 1.36. The transfer function, G(s), is given by
K
G(s) =
s(s + 1)

The sampling period, T, is unity. Taking the Z transform of the

open-loop transfer function, we find

(1 - esr) K 0.368K(z + 0-72)

s s(s + 1). (z - 1) (z - 0.368)'
 THEORY OF LINEAR SYSTEMS
83

Figure 1.36. Sampled Data System for Example 5.

The pole-zero configuration in the Z plane is shown in Fig. 1.37,

along with the resulting root locus, which is obtained in a conventional
manner. From the diagram, it is apparent that the system is unstable for
K >2.43.

Im

Example 6: In certain launch vehicle control systems, attitude

information is derived from the guidance computer instead of a displace¬
ment gyro. The autopilot configuration is then a hybrid combination of
analog and digital signals. Figure 1.38 is a schematic of such a system
wherein the effects of guidance dynamics and digital computer characteris¬
tics are incorporated in simplified form. A detailed discussion of the
philosophy leading to this configuration is contained in Ref. 20.
84 ELEMENTS OF MODERN CONTROL THEORY
THEORY OF LINEAR SYSTEMS 85

The open-loop transfer function is readily obtained from Fig. 1.38, as

follows.

Ka>V e~»V(l ~ e-^)(s2 + q>B2)

G(s)
s2(s2 + 2tAwAs + uA2)

where

“3 — KxKcKj/jc

2 nio HcOiT
COb — -
OM Tc

f = 1, / Ke
" 2\ KaKr»c •

The symbols have their meanings defined in Appendix A.

Assuming that TD = T, the Z transform of G(s) is obtained after a
tedious but straightforward calculation, viz.

G(z) = 1^5?"] (z2 + Diz + D0)

_ K„ J z(z —l)(z2—Riz + R0)

where

I) = T — Co{l — e~aT[2 cos bT — cos (bT — (p) sec <p\j

Dj=^ {(l- e~2ar)"K'-S

- 2e-
cos bT — cos (bT — <p) sec <p

Do =
C
Id
0
i)
c ./
e~2aT — e~aT cos (bT — <p) sec <p

2G
Go —
Ua

Ci=u~2 +4^2- 0
( 0 \03b '
 ELEMENTS OF MODERN CONTROL THEORY
86

a —

b = aWl -

*- ,an“(iLr^‘)
Rj = 2e~aT cos bT

R0 = e~2aT.

Using the following numerical values, which represent a typical booster

vehicle

KA = 1.4 aT = 67.1 ft/sec2

Kfl = 0.485 sec-1 tp = 40.2 ft

K0 = 12.5 sec-1 T„ = 81,400 lb

m0 = 1256 slugs T =1.25 sec

D = 1.04 Rx = 6.7 X 10-4

Dx = 0.202 R0 = 16.2 X 10-8.

Do = -49.8 X 10~6

With these values, we see that the numerator polynomial of G(z) has
roots at -0.202 and essentially zero, while the roots of the denominator
quadratic are both essentially zero. Consequently, for purposes of
drawing the root locus, G(z) may be written as

T.04Ka (z + 0-202)
G-(z) =
. 0.485 z2(z — 1)

The root locus for this case is shown in Fig. 1.39. Choosing Kw= 0.2
yields closed-loop poles as indicated by the small squares. The response
is governed by the complex pole pair shown, which has an undamped
natural frequency equal to cos/8 or 0.63 rad/sec and a relative damping
factor of 0.35 as determined by an overlay on Fig. 1.35. Instability
occurs for Kw > 0.37.
THEORY OF LINEAR SYSTEMS 87

1.39.
Figure
Plane
Z
6.
Example
for
Locus
Root
88 ELEMENTS OF MODERN CONTROL THEORY

Figure 1.40 is the schematic of a realistic launch vehicle control system

wherein attitude commands are derived from a guidance computer. The
vehicle dynamics for this case are as given in Appendix A. Bending and
other high frequency dynamics are neglected, but fuel sloshing is taken
into account.
The determination of the Z transform of the open-loop transfer
function for a system of this complexity is not feasible except with
computer assistance. The case considered is derived from Ref. 21,
where a Z transform computer routine!22) is used to obtain the poles and
zeros of the Z transform of the open-loop function. This, in conjunction
with any root locus program, yields the Z plane root locus shown in
Fig. 1.41. Numbered points on the locus are the guidance gain, Kw. In
the present case, the sloshing loci are nothing more than coincident
pole-zero pairs within the stable region, i.e., inside the unit circle. The
superimposed curves of constant damping ratio and undamped natural
frequency give a rapid indication of the nature of the system response
for any prescribed value of Ku. The tendency of the poles and zeros to
cluster at the origin and the point z = 1 is also shown. A similar pheno¬
menon was evident in the previous example.
Remark: No effort has been made in the preceding exposition to consider
a variety of detailed and specialized topics related to sampled
data systems. For a more complete treatment of such areas,
which include finite-pulse-width samplers, multiple samplers
with different periods, compensation techniques, etc., we must
of necessity refer the reader to standard texts.(i8,i9)

1.3 COMPENSATION TECHNIQUES

The need for compensation arises when the conflicting requirements on

steady-state and transient response cannot be satisfied by simple adjust¬
ment of open-loop gain. Compensating networks will usually be required
when:
a. The transient response is satisfactory but the steady-state error is too
high.
b. The steady-state error is within allowable limits but the transient
response is unsatisfactory.
c. Both the steady-state error and the transient response are unsatis¬
factory.
d. Specific performance criteria are not satisfied.
THEORY OF LINEAR SYSTEMS 89
 ELEMENTS OF MODERN CONTROL THEORY
90

1.41.
Figure
Plane
Z
for
Locus
Root
6.
Examp

=0.25
K
Point
Operating
NOTE:
THEORY OF LINEAR SYSTEMS 91

Before discussing specialized compensation techniques, it is appropriate

to consider the criteria that provide a measure of performance quality.
This is done in the following section.

1.3.1 Measures of Performance

From a controls point of view, the performance quality of a system

may be evaluated in terms of:

a. Stability.
b. Sensitivity.
c. Noise.
d. Transient Response.
e. Static Accuracy.

It goes without saying that, first and foremost, the system must be
stable. Measures of stability are important from a practical point of
view and will be considered shortly. Sensitivity is usually measured in
terms of variation of system performance as a function of variations in
prescribed parameters. Since this is the subject of a separate chapter, it
will not be considered here. Similarly, the behavior of the system in the
presence of noise is analyzed in Chapter Four and will be dismissed from
further consideration here. We shall, therefore, be concerned exclusively
with stability, transient response, and static accuracy.
The classical interpretation of degree of stability stems from the use
of the Nyquist plot of the open-loop transfer function as the medium of
analysis. Since stability is related to encirclement of the 1” point, it is
natural to relate degree of stability to proximity of the Nyquist curve to
this point. The terms phase margin and gain margin were defined in Sect.
1.1.2. These represent the phase lag (or, in some cases, phase lead) that
can be added to the system (at unity gain) before instability occurs; and
the factor by which the gain can be increased (or decreased) before
instability occurs. Thus, one may interpret each of these quantities as a
“factor of safety” in the system.
In addition, phase margin is a measure of the response characteristics
of the closed-loop system to sinusoidal inputs. In Sect. 1.1.2, it was
shown that, for systems whose response is governed by a dominant pair
of complex poles, the phase margin is intimately related to the relative
damping factor. To examine the general case, in frequency-response
terms, consider the two Nyquist loci depicted in Fig. 1.42. In Fig. 1.42a,
the phase margin is relatively large, while in Fig. 1.42b it is small.
Consider now the closed-loop frequency response, viz.
 ELEMENTS OF MODERN CONTROL THEORY

R(8) C(s)
KG(s)
+ n

Figure 1.42. Phase Margin and Closed- Loop Response.

C(jq>)= KG(jco) = M <p

R(jw) 1 + KG(jw)

Im^ (jw)
<p = tan = tan_V

(i«)
THEORY OF LINEAR SYSTEMS 93

M =
KG(jcd)
1 + KG(jco)

It is apparent that locus a (large gain margin) has a smaller peak, M,

than locus b (small gain margin). Systematic procedures (the M and N
circles of classical control theory) for determining peak M are discussed
in elementary texts on control theory and will not be considered here.
Our present purpose is merely to qualitatively exhibit the influence of
gain margin on the characteristics of the closed-loop frequency response,
a typical plot of which has the form shown in Fig. 1.43. In this case,
peak M is denoted by M™, and the corresponding frequency by com. The
latter is often referred to as the resonant frequency of the closed-loop
system.
An additional figure of merit is available from the response curve of
Fig. 1.43. The system bandwidth is defined as the frequency range in
which the attenuation (in the present case, quantity M) is greater than
- 3 db. Thus, in Fig. 1.43, the bandwidth is 0 — ub. The figure of - 3 db
is somewhat arbitrary. Some authors define bandwidth in terms of 0 db.
Bandwidth requirements are generally dictated by the particular system.
It is usually desired to pass all signals within some prescribed frequency
spectrum and to suppress all others. In general, while large bandwidths
correspond to two static errors, they also lead to higher sensitivity to
extraneous noise inputs.
System performance is sometimes specified in terms of time-response
rather than frequency-response characteristics, although the two are
related.!1)

Figure 1.43. Closed-Loop Frequency Response.

 94 ELEMENTS OF MODERN CONTROL THEORY

Figure 1.44. Typical Step-Input Response.

Figure 1.44 shows a typical response to a step input signal. Various

figures of merit are defined as follows. Rise time is the time required to
reach the first overshoot (tr in Fig. 1.44). The percent overshoot is
defined at tr in the manner shown in the figure. The time following
which the errors are less than 5 percent of the steady-state value is called
the settling time (ts in the figure). The (step input) steady-state error is
the difference between the steady-state value of the response and the
magnitude of the step input (unity in Fig. 1.44). For second-order
systems, it is an elementary exercise to relate the above quantities to the
undamped natural frequency and relative damping factor. Consequently,
when a dominant complex pole pair exists, these definitions have a useful
interpretation.
Other measures of performance quality are based mainly on an integral
of the error (difference between input and feedback signals). Among
these are

I =

and simdar types.(66) However, these are rarely used and will not be
discussed here.
THEORY OF LINEAR SYSTEMS 95

When prescribed performance, as expressed in any of the above forms,

cannot be obtained with simple adjustment of open-loop gain, some
form of compensation must be employed. The conventional networks
commonly used for this purpose are the lag, lead, or lead-lag types, with
frequency-response methods adopted as the medium of analysis. Since
this approach is amply discussed in standard texts, in what follows we
shall analyze the compensation problem exclusively from the root locus
point of view.

1.3.2 Continuous Systems

The discussion of this section will focus on the control system depicted
in Fig. 1.45. Here Gi(s) represents the fixed plant that has a transfer
function of the general form
M

K II (s - z»)
Gi(s) = — — - (93)
N

s" II (s - p3)
i—l

and Gc(s) is a compensator whose form is as yet unspecified, By defining

the quantities
Kp = lim G(s) (94)
$—%º0

K„ = lim sG(s) (95)

s— >0

(96)
K0 = lim s2G(s)
s->0

which are known as the position, velocity, and acceleration error coeffi¬
cients, respectively, and where

G(s) = Gc(s)Gi(s)

Figure 1.45. Unity Feedback Control System.

 ELEMENTS OF MODERN CONTROL THEORY
96

we may systematically examine the steady-state response of the system

under prescribed inputs. More specifically, we shall be concerned with
the steady-state error in response to a unit step input. Using the Final
Value theorem, it follows from Fig. 1.45 that
sR(s)
lim [e(t)] = lim
t— >oo s— >0 _1 + G(s)_

which, after putting R(s) = 1/s, becomes

1
lim[e(t)] = lim (97)
t—>CG 3— >0 _1 + G(s)_

Assuming for the moment that Gc(s) = 1, i.e., no compensation, we

find that for a Type 1 system [n = 1 in Eq. (93)], the steady-state
response to a unit step input is zero. However, if n = 0, then

1
lim[e(t)] = (98)
t-tx 1 + Kp

The position error coefficient is thus a direct measure of the steady-

state error in response to a unit step input for a Type 0 system.* We shall
hereafter be concerned only with plants of the form
M

K IJ (s — z.)
i= 1

Gi(s) =- - (99)
n (s - pj)
;% =l

i.e., only Type 0 systems. We note that K and Kpare related by

M

k n (-z.)
KP = — —- . (100)
N

n (-py)
y-i

The general design principles involved in lead or lag compensation

will be discussed with reference to a specific case. We assume that the
transfer function of the fixed plant in Fig. 1.45 is given by

*i.e., n = 0 in Eq. (93).

THEORY OF LINEAR SYSTEMS 97

Gi(s) = - (101)
(s - Pi)(s - p2)(s - p3)

and the root locus (assuming no compensation) is as shown in Fig. 1.46.

If it is desired that the relative damping ratio of the closed-loop poles
equal 0.3., then the open-loop gain, K = 165. In this case, we find from
Eq. (100) that the position error coefficient, Ivp = 2.95. Via Eq. (98) we
find, therefore, that

lim [e(t)] = 0.253.

t—* oo

In other words, for a unit step input, the final steady-state value of
the output is 1 —U.253 = U.747. This error may be reduced by increasing
K, which, in turn, decreases the relative damping factor and which, for
K sufficiently large, leads to instability. We seek to increase the open-
loop gain (and therefore decrease the steady-state error) without de¬
creasing the relative damping factor. For this purpose we introduce the
compensating lag network

A(TiS -(- 1) A (s z c)
(102)
(aTiS +1) a (s pc)
where

= — > 1
Pc

and A represents the gain (as yet unspecified) of an amplifier associated

with the lag network.
The open-loop transfer function is now

KA (s - zc)
G(s) = — (103)
a (s - pc)(s - pi)(s - p2)(s - p3)

Observe that for the uncompensated case, the gain, K, at point s0 is

readily computed from

K = n ISo- Vi (104)
 ELEMENTS OF MODERN CONTROL THEORY
98

Im

Figure 1.46. Root Locus for Eq. (101).

THEORY OF LINEAR SYSTEMS 99

In the compensated case, if s0 is still a point on the root locus, then we

must have

3
KA So ~ Pd
II ISo- P. (105)
a
So zc i=l

The root locus in the vicinity of s0 will be essentially unaltered if

s0 — z„| |s0 — pc| (106)

and

< (s0 - zc) « (s0 - pc). (107)

Under these conditions, it follows that

A = a.

The new position error coefficient is, therefore

KA _ ( —z c)_
K V AKP. (108)
« (-Pc)(-pi)(-p2)(-p3)

In other words, the position error coefficient has been increased by a

factor of A(= a) without essentially altering the position of the dominant
closed-loop poles. Thus, the new steady-state error is

1
lim [e(t)] =
t— *co 1 AK„

If we take a = A - 10, then K/ = 10 x 2.95 = 29.5, which means that

the steady-state error in the compensated case is

1
lim [e(t)J = 0.0328
1—>CG 1 + 29.5

compared with 0.253 for the uncompensated case.

In order to satisfy conditions (106) and (107), we take pc = 0.01 and
z„ = 0.10. The exact location is not crucial. However, the dipole must
 100 ELEMENTS OF MODERN CONTROL THEORY

be in the vicinity of the origin in order that the influence on the dominant
pole transient response be negligible.
The undamped natural frequency for the dominant mode, i.e., corre¬
sponding to the closed-loop pole at s0, is 4.65 rad/sec. Suppose now that
we wish to increase the response time of the system, say by increasing
the undamped natural frequency to 6.5 rad/sec without changing the
relative damping factor. In other words, it is desired to have the dominant
closed-loop pole at si in Fig. 1.46. It is readily ascertained that this may
be accomplished by adding a network which contributes 33° of phase
lead at si. There are an infinity of networks that will do this, but we add
the requirement that the position error coefficient have a prescribed
value.
The given plant now has the form

Gt'(s) = iA _ + °-10>_ (109)

a (s + 0.01)(s + 2)(s + 4)(s + 7)

We introduce the lead compensator

G '(s) = + 1) _ B(s — zg)

(110)
(XTis +1) (s - pg)

where

A = ^<1
pe

and B is the gain of the amplifier associated with the network.

Consequently, the new open-loop transfer function is

BKA (s - Zg) _ (s - Z0)

G'(s) =
(s - pt) (111)
n (s Pj)
J“0
THEORY OF LINEAR SYSTEMS
101

where

z0 s zc = 0.10
Po = pc = 0.01
Pi = -2
P2 = -4
P3 = —7
A = a = 10
K = 165.

If si is a point on the root locus for the compensated system, Eq.

(Ill), then we must have

II ISl - Pi
BKA _ |si —p£| 3'=0
a |Si —Z£| |Si — Z0|

The position error coefficient is therefore given by

BKA ( —z£)( —zp)

Kp

(~P£) II (~Pi)
1=0
3

II |si - Pil
Si — p£| Iz£[ kl 1=0
(113)
Si — Z£| |pfI |si - z0|
II IP1
1=0

z£, pf, and B must be chosen such that si is a point on the root locus
for the system and the corresponding position error coefficient, K„" =
45.
This may be done as follows. Referring to Fig. 1.47, we write K/'as

(114)
a
 ELEMENTS OF MODERN CONTROL THEORY
102

Im

Figure 1.47. Calculation of Lead Network Parameters.

where

X =
Ini c

IpeI d

Si — ze| = 8j

|Si —pe| = b

|z0| II
J=0
I —

si - z0| %n iPi1
THEORY OF LINEAR SYSTEMS 103

The quantity, L, may be calculated from the fixed poles and zeros and
is, therefore, a known quantity.
From the geometry of the figure, we have

yi
tan
C — Xi

yi
tan y
d - X!

<P = d — 7

tan /8 — tan 7
tan <p = tan (/? — 7)
1 + tan j8 tan 7

This last relation reduces to

yi + (xi — c) tan <p

X d Wl
(115)

yi - ^Xi- —Jtanv
Applying the law of sines to triangle pe zi si, we have

a _ b
sin 7 sin d

or

b _ sin d _ sin /J
a sin 7 sin (/3 — 95)

_ yi _ (116)
yi cos <p— (c — Xi) sin ^

Substituting Eqs. (115) and ( 1 16) in Eq. (114) and solving for c yields

^ _ coi2E" sin ip_ (117)

yiL — Kp(yi cos <p— x 1 sin <p)

All the quantities on the right-hand side of this equation are known.
Having c, we then determine d from Eq. (115).
 ELEMENTS OF MODERN CONTROL THEORY
104

In the present case we have the values

L = 57.9 Kp = 45
toi = 6.5 xx = 2.0
v = 33 0 yi = 6.2.

We find, therefore

c = 5.96
d = 15.60
X = 0.382

which means that

pe = -15.60
z£ = —5.96.

Using these numerical values we find, from Eq. (112)

BKA
660.
a

Since A = a

B 660 _ 660
K ~ 165

The required parameters have thus been completely determined. Note

that the amplifier gains, A and B, may be absorbed in the plant gain, K,
if the latter is adjustable. The disposition of necessary amplifiers depends
on the actual hardware configuration of the system, i.e., the input im¬
pedance of the compensating network must be low (ideally zero), and
the impedance that loads the output of the network must be high (ideally
infinite).
In the case just considered, the parameters of the lead network have
been chosen to satisfy a constraint on the position error coefficient.
Sometimes these parameters must be chosen so that the ratio of time
constants, X, is a maximum. This will result in a minimum value for the
gain of the additional amplifier, B, and, therefore, a minimum value for
the bandwidth of the resulting system. Such a requirement is often
dictated by noise or saturation constraints.
 THEORY OF LINEAR SYSTEMS
105

Again the open-loop transfer function is as shown in Eq. (1 1 1), and it

is required that si be a point on the root locus. (See Fig. 1.46.) The
values of xi, yi, and <p are as before. Referring to Eq. (1 15), we seek the
value of c that maximizes X. Via elementary calculus, we find that this
value is given by

o>i(sin 6 — sin <p)

(118)
sin (0 — tp)
where

and

0 > <p.

Using the given values, we find

c = 4.18
X = 0.414
d = 10.1
PC = —10.1
z£ = —4.18.

From Eq. (112)

BKA
502
a

or

502 _ 502 3.04.

K ~~ 165

The position error coefficient is obtained from Eq. (1 13) as

K" = 37.1.

The form of Eq. (118) permits one to derive a simple geometrical

construction for the determination of z^ and pe for maximum X. By
 ELEMENTS OF MODERN CONTROL THEORY
106

substituting c from Eq. ( 11 8) into the relation

tan /3 = % y —
C — Xj

we find, after some straightforward reduction

„ sin (e ~ v)
tan /8 = - -- r-
1 — cos (6 — <p)

From this it follows that

1 — cos (0 — <p)
cos /3 = / I=.
V2[l — cos (0 — <p)]

Letting 8 = DO0-^, and making use of some elementary trigonometric

identities, we find that the above relationship simplifies to

8 = 1 (e- *). (119)

As shown in Fig. 1.47, the required pole and zero are then found after
a trivial geometric construction.
The discussions thus far have been concerned with Type 0 systems
and the position error coefficient. The procedure is, however, identical
for Type 1 and Type 2 systems where the parameters of interest are the
velocity and acceleration error coefficients.
The simple networks thus far discussed are not effective in all situations.
In the case just considered, the open-loop poles of the fixed plant were
on the real axis. This permitted considerable latitude in choosing a lead
network that would shift the closed-loop poles. In Fig. 1.48, the original
root locus, before compensation, is shown by dotted lines, and the root
locus of the compensated system is shown by the solid lines. (This figure
is not to scale.) The general qualitative effects are, however, apparent.
The compensator zero effectively cancels the pole, p2, which enables the
main locus to be shifted to the left by an appreciable amount.
THEORY OF LINEAR SYSTEMS 107

Im

Re

Figure 1.48. Comparison of Root Loci for Compensated and Uncompensated Systems.

Consider now the situation depicted in Fig. 1.49, where the fixed
plant has a pair of complex open-loop poles and one pole, p3, on the
real axis. The uncompensated root locus is shown by the dotted lines. A
simple lead compensator of the form (109) cannot alter the system
significantly, in the way of increasing either the relative damping factor
or the undamped natural frequency. This becomes evident by observing
a typical “compensated” root locus as shown by the solid lines in Fig.
1.49. A problem of this type is precisely the one encountered in launch
vehicle autopilots where the lightly damped pole pair is due to the vehicle
bending mode. Here a more general type of compensation is required.
A suitable compensating network in this case is given by

Gc(s) = (120)

where u, < The complex zero pair, zj, is chosen to effectively cancel
the complex pole pair, pi, and the complex pole pair, pe, is moved far to
108 ELEMENTS OF MODERN CONTROL THEORY

Figure 1.49. Simple Lead Compensation of a System Having a Complex

Open-Loop Pole Pair.

the left. The resulting situation is depicted in Fig. 1.50. Figure 1.51
shows a passive linear network for realizing the transfer function, Eq.
(120).

Im

Figure 1.50. Compensation via Complex Lead Network.

THEORY OF LINEAR SYSTEMS 109

R R
3 4
„ T T 1 + s + (T + T)s+1
E 12 R (R + R + R ) ' 1 27
out 1 2 3 47 J
In
R 3 (R 2 + R 4 )7 R
2
T T 1 + s + T 1 (1 + R
~) 7 + T 2
1 2 R (R + R + R )
12 3 4

T =

For a fixed, lightly damped pole pair, this type of compensation is

effective. However, in typical launch vehicle autopilots, the bending
mode poles vary with time, and a more sophisticated approach is required.
These are discussed in Chapter Six.

1.3.3 Sampled Data Systems

In the discussion relating to compensation of sampled data control

systems, we will consider the general configuration shown in Fig. 1.52.
The transfer function of the fixed plant is Gi(s) and its Z transform is
Gi(z). The Z transform of the compensating network is Gc(z), and the
open-loop transfer function has a Z transform given by

G(z) = G<!(z)G1(z). (121)

We define the position, velocity, and acceleration error coefficients by

K* = lim [G(z)] (122)

z—> 1

K* = lim [(z — l)G(z)] (123)

Z— > 1

K* = lim [(z — l)2G(z)]. (124)

110 ELEMENTS OF MODERN CONTROL THEORY

We say that G(z) represents a Type 0 system if it has no poles at z = 1 ;

G(z) is a Type 1 system if it has one pole at z = 1, etc. The definitions are
completely analogous to those for the continuous case discussed in Sect.
1.3.2, with obvious modifications for Z plane rather than s plane analysis.
Using the Final Value theorem for Z transforms/19) we find that

(z - l)R(z)\
lim [e*(t)] = lim
^—%ºoo 2— %º1 z[l + G(z)]j '

For a unit step input

R(z) = -L-
z — 1

which means that

1
lim [e*(t)] = lim (125)
t—> oo z— %º1 _1 + G(z)_

Consequently, the error coefficients defined by Eqs. ( 122)—( 124) pro¬

vide a measure of the steady-state error of the sampled signal in the same
manner as the error coefficients, Eqs. (94)-(96), do for a continuous
system.
The compensation techniques for a sampled data system using a pulsed
compensator of the form shown in Fig. 1.52 follow a pattern analogous
to that used for continuous systems. To see this, consider the Z trans¬
form of the fixed plant whose most general form is
THEORY OF LINEAR SYSTEMS 111

k n (z - zo
Gi(z) = - —- . (126)

(z - l)n II (z ~ P>)
1=1

A Z plane root locus may be drawn for this in a manner analogous to

that used in obtaining Fig. 1.39 or 1.41; i.e., normal root locus pro¬
cedures apply. Interpretation of these loci is facilitated by the use of a
diagram of constant damping and natural frequency of the type shown
in Fig. 1.35.
In order to reshape the root locus to alter either the undamped
natural frequency, relative damping factor, or steady-state error, a com¬
pensation network is generally required. For purposes of illustrating the
principles involved, we will consider only the simplest type given by

Gc(z) = (127)
(z - pc)

This compensator is a lead or lag network, depending on the relative

location of z„ and pc in the Z plane. Typical cases are shown in Fig.
1.53. In general, the compensator network is of the phase lead type if
the pole lies to the left of the zero and of the phase lag type if the reverse
is true. It is apparent from Fig. 1.53 that more phase lead is obtainable
from a compensator of this type if the zero is placed in the right half and
the pole in the left half of the unit circle.
As is the case with continuous-type compensation networks, physical
realizability places certain constraints on permissible locations of the
poles and zeros. The compensator of the pulsed-type shown schematically
in Fig. 1.52 may be implemented by digital programming/23) delay-line
networks, (24) or sampled RC networks. We will consider only the last of
these, which in general may be exhibited in the two basic forms shown in
Figs. 1.54 and 1.55. The transfer function, G„(s), represents the zero
order hold given by Eq. (83), while GA(s) and HA(s)are physically realiz¬
able RC networks.
Consider first the series-type compensator of Fig. 1.54. We have

Ge(z) = = Z[Gff(s)Gx(s)] s G„Ga(z).

E.(z)
112 ELEMENTS OF MODERN CONTROL THEORY

Figure 1.53. Pole- Zero Configuration in Z plane for Compensating Network, Eq. (127).

Gc(Z)

Figure 1.54. Series-Type Sampled Data Compensator.

THEORY OF LINEAR SYSTEMS 113

UsingG^(s), given by Eq. (83), we find

'G.(s)'
G„(z) = (1 - z_1)Z
L s J

or

G^s) G „(z)
s J (1 — z x)

Substituting for Gc(z) from Eq. (127)

Ga(s) (1 - ZCZ X)
s J (1 - Z !) (1 - PcZ 0

=( 1 ~ ZA 1 , (Zc- pA 1
\1 - pJ (1 - Z-1) \1 - pc/ (1 - PcZ-1)
Taking the inverse Z transform and solving for G^(s)

8/(1 Zc)
S +
1 - Pc (128)
G,(b) =
(s + a)
114 ELEMENTS OF MODERN CONTROL THEORY

where

a (129)

T = sampling period, in seconds.

If the network described by Eq. (128) is to be physically realizable

by passive elements, pc and zc must satisfy*

0 < pc < 1 (130)

0 < zc < 1. (131)

Thus, with the series type of compensator, only the configuration

shown in Fig. 1.53b can be realized. The network is phase lead if
zc > pc and phase lag if zc < pc.
A somewhat greater latitude in pole-zero configuration is afforded by
using the parallel-type compensator shown in Fig. 1.55.
Here we find

Eo(z) 1
Gc(z)
E;(z) 1 + Gi/Hx(z)

Ha(s)~
1 + (1 - z-!)Z
s

or

Ha(s) 1 - G c(z)
(132)
s (1 - z_1)Gc(z)

Substituting for Gc(z) from Eq. (127)

Ha(s) (Zc- Pc)z^ = (z, - Pc)

(1 - Z-0(1 -ZcZ-1) (1 - Zc)

1_ 1

_(1 - Z-1) (1 - ZcZ-1)-

*If zc > 1, then the network (128) is nonminimum phase.

THEORY OF LINEAR SYSTEMS 115

Taking the inverse Z transform and solving for ha(s)

(zc - pc)b
H.4(s)
(1 - zc)(s + b)

where

b = (133)

In order for the network in Eq. (132) to be physically realizable by

passive elements, we must have

0 < zc < 1 (134)

but there is no restriction on pc other than pc ^ zc. Note that if the

quantity (zc — pc) in Eq. (132) is negative, i.e., pc > zc, the negative sign
at the summing junction in Fig. 1.55 is replaced by a plus sign. Note
also, that if the transfer function of the plant contains a hold circuit, viz.

Gi(s) = G*(s)Gj(s)

then one hold circuit may be eliminated by feeding the output of GH(s)
in Fig. 1.55 directly in G((s).
Given now the open-loop transfer function whose Z transform is de¬
scribed by Eq. (126), one may shape the resulting Z plane root locus,
using the compensator (127), to satisfy a variety of criteria. Nothing
essentially new in the way of technique is involved (other than inter¬
pretation), since the usual root locus rules carry over directly into the Z
plane. One may, for example, require a quicker rise time, increased
damping, and/or a decrease in steady-state error. The root locus shaping
methods are completely analogous to those for the continuous case,
except that results are interpreted via a diagram of the type shown in
Fig. 1.35. There is, consequently, more work involved, but no difference
in principle. For purposes of illustrating the basic approach, we will
take the transfer function of the fixed plant as

_ Ki _
s(0.1s + l)(0.05s + 1)

which is preceded by a hold circuit of the type described by Eq. (83).

116 ELEMENTS OF MODERN CONTROL THEORY

Therefore, the Z transform of the fixed plant is

_ 0.0164Ki(z + 0.12)(z + 1-93)

"l(Z ~ (z - 1) (z - 0.368)(z - 0.135)

where the sampling period, T = 0. 1 sec.

It is readily found that there is a zero steady-state error in response to
a step input. However, for a unit ramp input

Tz
R(z)
(z - l)2

there is a finite steady-state error.

The Z plane root locus for the uncompensated system is shown in
Fig. 1.56. For a relative damping factor of f = 0.7, the corresponding
gain is found to be Ki = 2.6. The associated velocity error coefficient is

K* = 0.1 X Kx = 0.26

via Eq. (123).

Suppose it is required that K*be increased to a minimum of 1.5 while
the relative damping factor is kept the same. Obviously, this cannot be

Im

Figure 1.56. Z Plane Root Locus for Uncompensated System.

THEORY OF LINEAR SYSTEMS 117

accomplished merely by increasing Ki. Using a sampled network of the

form in Eq. (127), one may derive various analytical or geometric pro¬
cedures to accomplish this in a manner completely analogous to that of
Sect. 1.3.2. However, in the present case, a few quick cut-and-try ap¬
proaches show that a phase lead compensator of the form

z - 0.368
Ge(z)
z + 0.950

yields satisfactory results. In effect, the zero in the compensator at

zc = 0.368 cancels the corresponding pole in Gi(z), and pc is then located
to ensure that the required closed-loop pole on the constant f = 0.7 line
satisfies the angle criterion of 180°. The Z plane root locus of the
compensated system then appears as shown in Fig. 1.57. Here we find
that at the new closed-loop pole, Ki is 17.1, and the new value of K*
is 17.1. Note that the compensated system has a greater gain margin;
Ki >61.8 for instability, while for the uncompensated system, instability
occurs for Ki > 13.2. This is a typical result obtained when using phase
lead networks.
The block diagram for the complete system is shown in Fig. 1.58.
Note that the parallel type of compensator has been used, since a pole in
the left-half plane was required.
We have discussed only one type of compensator, namely a passive
network employing samplers. It is also possible to use so-called continuous

Im

Figure 1.57. Z Plane Root Locus for Compensated System.

118 ELEMENTS OF MODERN CONTROL THEORY

Figure 1.58. Block Diagram of Compensated System.

compensation wherein the sampler following G„(s) in Fig. 1.52 does not
appear. Here, frequency-response methods are generally employed, and
these are more elaborate and complicated than the techniques discussed
above. Detailed expositions are available in the literature.!19)
It was noted in connection with continuous compensation methods
(Sect. 1.3.2) that a perennial problem in launch vehicle autopilots is the
presence of a lightly damped, complex pole pair adjacent to the imaginary
axis. When there is a digital computer in the loop, as depicted in Fig.
1.59, the Z plane root locus takes the form shown in Fig. 1.60. Here the
lightly damped pole pair is located adjacent to the unit circle, and the
corresponding locus is such that relatively small values of open-loop
gain will result in closed-loop poles outside the unit circle. The methods
of resolving this problem are similar to that used in the continuous case,
namely, a sampled data equivalent of the notch filter, Eq. (120). This
takes the form

A(z) _ aiz2 + a2z + a3

(135)
E(z) z2 + b2z + b3

The zeros of Ge(z) are located in the immediate vicinity of the

bending poles, while the poles of Gc(z) are placed farther away from the
unit circle. The resulting loci for the compensated case are shown by the
dotted lines in Fig. 1.60. It is apparent that the compensated system has
a substantially higher gain margin.
The sampled compensator of Eq. (135) may best be implemented by
utilizing the digital computer directly in the loop. For example, the
transfer function in z is replaced by the equivalent difference equation

ajEn + a2 E„_i -|- a3En_2 b2An_i — b3An_2

leading to a straightforward digital computation.

THEORY OF LINEAR SYSTEMS 119

Digital
Computer
120 ELEMENTS OF MODERN CONTROL THEORY

_ Uncompensated

Figure 1.60. Z Plane Root Locus for Launch Vehicle Autopilot.

It was noted in the previous section that the difficulties in this problem
are compounded because the bending pole locations vary with flight
time, thereby seriously limiting the usefulness of the above approach.
More sophisticated techniques are discussed in Chapter Six.
In the current generation of large launch vehicles (Atlas, Titan, Saturn),
the approach taken for this problem involves programmed gains and
filters. At specific times of flight, one switches to a different value of
open-loop gain (in the summing amplifier and/or rate gyro). This is
usually accompanied by the switching in or out of a prescribed filter.
This, of course, presupposes reasonably good data on the vehicle inertial
and bending mode properties as a function of flight time. For certain
complex vehicles, switching may be inadequate, or else the bending mode
data (especially for higher modes) may not be known with sufficient
precision. In this case, the use of one of the various adaptive control
techniques may be necessary. Needless to say, such a step must be
weighed against the added cost and complexity and the decreased
reliability that adaptive control entails.
THEORY OF LINEAR SYSTEMS 121

1 .4 STATE VARIABLE METHOD

1.4.1 Continuous Systems

The traditional methods for the analysis and synthesis of linear feed¬
back control systems make extensive use of the concept of a “transfer
function,” whether in the frequency domain (Nyquist) or in the time
domain (root locus). In the rapidly developing fields of optimal and
adaptive control, however, it has been found more appropriate to deal
with linear systems via the “state variable” approach, in which matrix
techniques play a fundamental role. This concept of the “state of a
system” has also served to clarify various aspects of the transfer function
method, especially when applied to multivariable input-output systems.
The literature contains many incorrect results for multivariable systems
because various subleties involved in dealing with a so-called matrix
transfer function have been neglected. This subject will be discussed at
length in Sect. 1.5.
Actually, the terms “state variable” and “state space techniques” are
really new names for a body of ideas that have long been used in such
areas as analytical dynamics, quantum mechanics, ordinary differential
equations, and stability theory. These ideas were first introduced in
control theory by the Russians, mainly in the development of the
Lyapunov stability theory. In the United States, the work of Kalman
and Bellman (among others) contributed to the widespread adoption of
state variable methods in control system design.
The basis for all the subsequent discussions is contained in the follow¬
ing pair of matrix equations, which describe the motion of a given
dynamic system.

x = Ax + Bu (136)

v = Cx + Du (137)

Here

x = n dimensional state vector

u = p dimensional input vector
v = q dimensional output vector
A = n X n constant (system) matrix
B = n X p constant (input) matrix
C = q X n constant (output) matrix
122 ELEMENTS OF MODERN CONTROL THEORY

D = q X p constant matrix
d
(') - ( )
dt
n = order of the system.

To see how an input-output relation, given in the form of a conventional

transfer function, may be expressed in the format of Eqs. (136) and
(137), consider

X
P(b) _ ?=°
(138)
5(s)
X aiSn_l
i=0

Where s represents the Laplace operator and where, without loss of

generality, we may take «0 = 1 ; the and are constants.*
Using a variant of the so-called “m methods,”!48) we make the change
of variable

e = 5 + Trx (139)
ao

o _ /3°
@n <2n
<2o

a _ 00
Pn—1 Oin—
1 (HO)
a 0

a Po
Pi — - <*1
«0

Then the system of equations

Xi = x2

x2 = x3

(141)
Xn— i xn

1 r,
Xn [V 0!iXn —1 &n—1X2 ^nXi]
Ot 0

*We make the usual assumption that the initial conditions are zero.
THEORY OF LINEAR SYSTEMS 123

together with the transformation defined by Eq. (139) is completely

equivalent to Eq. (138).
In this case, the matrices for the system, Eq. (136) and (137), are
given by

0 1 0 . o
/

ooi :

A = (142)
0

«72 “72-1 £*i

<20 <*0 <20 ,

0
C = I, the unit matrix
B = (143)
0 D = 0, the null matrix
1

and

u == u„ = — = a scalar.
£20

Note that numerator dynamics have been effectively eliminated; there

are no derivatives of the forcing function, 8, in the system (141).
The matrix, A, for this particular case has a special form known as
the companion (standard) form.
It should be noted that the state-space representation for a system is
not unique but depends upon the particular transformation utilized.
With reference to the system described by Eqs. (136) and (137), we
shall be concerned with three fundamental questions:

a. Is the system stable?

b. What is the form of the solution forx(t)?
c. What is the system transfer function?

The answer to the first question is contained in the following.(25)

124 ELEMENTS OF MODERN CONTROL THEORY

“A necessary and sufficient condition for the stability of the system

represented by Eqs. (136) and (137) is that the real part of each eigen¬
value of the matrix, A, be negative.”
With regard to the second question, the solution of Eq. (136) is given
by(25>

X(t) = <f>(t— to)x(to) + <j)Bu(cr)d<7 (144)

where

«f(t)
At y AH* (145)
h k!
A° = I, the unit matrix.

Finally, in order to derive the transfer function that represents the

system, (136) and (137), we have, after taking the Laplace transform

X(s) = (Is — A) x[13U(s) + x(to)]

V(s) = CX(s) + DU(s).

If the initial condition vector, x(t0), is zero, then combining the above
two equations yields

V(s) = L(s)U(s) (146)

where L(s) is the matrix transfer function, given by

L(s) = C(Is - A)-1B + D. (147)

Whde succinct answers have been provided to the three basic questions
posed in relation to the system, (136) and (137), a variety of details
remain to be filled in for purposes of design and analysis. In the first
place, the solution for x(t), as given by Eq. (144), requires the evaluation
of the transition matrix, <t>(t). Evaluating this quantity via Eq. (145) is
not the most efficient method, because of error uncertainty in the
truncation process. A practical and theoretically exact expression for
4>(t) is( 38)
THEORY OF LINEAR SYSTEMS 125

_ \l£

3X2<

<t>(t) = M M-1
(148)

„X„(

where the X, are the distinct eigenvalues* of the matrix, A, and M is the
corresponding modal matrix. In other words, X,- is a root of the character¬
istic equation

|xi - a| = o

and for each value of Xv, the matrix equation

[XJ — A] Ci = 0

yields a value for c<, the eigenvector, which is determined to within an

arbitrary constant. One way of normalizing the eigenvector is to divide
every component of the vector by the component of smallest magnitude.
Thus every eigenvector will have at least one component equal to unity.
However, this is more a matter of convenience then necessity. The
matrix formed by these eigenvectors is termed the modal matrix.
For the case of distinct eigenvalues, the columns of the modal matrix can
be taken equal or proportional to any nonzero column of Adj[X,I — A],
Each choice of X; specifies only one column of the modal matrix, since
the columns of Adj[X,I — A] are linearly related for a given Detailed
accounts of the foregoing techniques are available in standard texts. (39)
Various special forms of the matrices in Eqs. (136) and (137) are
useful in particular cases, and it is convenient to denote these by special
symbols. For example, A is used, as shown in Eq. (136), to denote the
most general form of the system matrix. If all the components, a,y, of
A except
a.n& oc 1,2, , n

a,.,-+i i = 1, 2, • • •, (n - 1)

*When there are multiple eigenvalues, the expression for <f>(t) is much more complicated. For a
complete treatment in this case, see Ref. 25 .
 126 ELEMENTS OF MODERN CONTROL THEORY

are zero, then we say that the system matrix is in companion standard
form, and we denote it by F. It was noted earlier that the system matrix
for the set of equations, (141), is of this form.
If the input matrix, B, is of dimension n X 1 (i.e., an n vector), we
represent it by the symbol, f. In the special case when the nth component
of f is unity and all the other components of f are zero, we denote it by
the symbol, h. The system represented by F and h in the state-variable
form has a special name, the phase variable ( canonical) form.
We will also use the lambda and Vandermonde matrices, which are
defined, respectively, as follows.

A = (149)

T = (150)

Various transformations for simplifying a given system for design or

analysis are available. Consider first the system described by

x = Ax + fui (151)
where

Ui = a scalar.

The transformation

x = Jz (152)

converts Eq. (151) to the canonical (phase variable) form(28)

z = Fz + hui (153)
THEORY OF LINEAR SYSTEMS 127

where*

J = MR_1T_1 (154)

in which M and T are the modal and Vandermonde matrices, respectively,

for A, and where R is an n X n diagonal matrix obtained from

RM-1f = T_1h. (155)

Various simplified procedures are available for computing the inverse

Vandermonde matrix. (26,27)
An alternate transformation that converts Eq. (151) to the phase
variable (canonical) form is(45)

x = 't'z ( 156)

where the transformation matrix, is formed as follows. First calculate

the controllability matrix

E = [f Af A2f . An_1f] (157)

which must be checked to see if it is of rank one.^ (Otherwise this

method is not valid.) Then, after writing the characteristic equation of
the system as

det[A - XI] = X" - X f Ai_1 = 0 (158)

i=l

we form the matrix

—fs fa f2. fn 1
-fa f4. -fn 1 0
f4 :

! -fn 1 0
-fn 1 0
1 0 0

*It is assumed that the eigenvalues of A are distinct.

fThis condition is directly related to the controllability of the pair (A, f). See Sect. 1.5.
128 ELEMENTS OF MODERN CONTROL THEORY

The transformation matrix is then given by

* = Eft. (159)

The importance of the canonical (phase variable) form of the system

equation, as represented by Eq. (153), is that this matrix equation may
be readily reduced to a single scalar equation involving derivatives of up
to nth order of the dependent variable.
The transformation

x = My (160)

where, as already noted, M is the modal matrix for A, converts the

system, Eqs. (136) and (137), to the normal form (31)

y = Ay + Pu (161)

v = Ty + Du. (162)

Here

P = M-lB = normal input matrix (163)

T = CM - normal output matrix ( 164)

and A is defined by Eq. (149).

The n components of the vector y are termed the normal coordinates.
Note that the form of Eq. (161) is such that these are completely un¬
coupled.
In terms of the normal matrices, the matrix transfer function, L(s),
may be readily expressed as a partial fraction expansion. To do this, let
r\ represent the ith column of r, and P< the itL row of P. Then

L(s) = C(Is - A)~lB + D

= T(Is - A)-»P + D. (165)

Noting that
THEORY OF LINEAR SYSTEMS 129

s —

s — \2
(Is - A)-1 =
(166)

• 1

S — X„

the expression forL(s) reduces to*

L(s)- i-1± S - X*
+ D (167)

where

K, = r,P,, (168)

Quantity K, is a q X p matrix of rank one, since it is formed by

taking the outer product of two vectors.
We now illustrate the application of the above ideas in the following
examples.

Example 7: We are given the transfer function

9(s) = 3s2 + 4s + 1
5(s) s2 + 5s + 2
It is required to express this in the state variable matrix form, Eqs.
(136) and (137).
From a comparison with Eq. (138), we have

ao = 1 Po — 3

ai = 5 Pi = 4

£*2 = 2 Pi = 1.

We are still using the assumption that the eigenvalues, X,, are distinct.
130 ELEMENTS OF MODERN CONTROL THEORY

The state variable equations are

Xl = x2
(a)
x2 = — 2xi —5x2 + 5

using Eq. (141). Also, from Eq. (140)

"-5 '
7 = L—H_
Therefore, the transformation, Eq. (139), becomes

e = 35 - 5xi - 1 lx2. (b)

To see how the transfer function format is recovered from Eqs. (a)
and (b), note that from the former

xi = — 2xi — 5xi + 5

or

(s2 —
|—5s —
f- 2)xi — 5. (c)

From Eq. (b)

d = 35 — 5xx — llx2

= 35 — 5xi — llxi

or

(11s + 5)xi = 35 - 6. (d)

Eliminating xi between Eqs. (c) and (d) yields the original transfer
function.

Example 8: We are given the dynamical system described by

x = Ax + fui
THEORY OF LINEAR SYSTEMS 131

where

_2 -1 r r
1 0 1 f = 1
-1 0 i_ _i_

and ui is a scalar. /
It is required to express the system in canonical (phase variable) form.
The characteristic equation is

|IA - A| = A3+ X2- 2 = 0

which yields the eigenvalues

Ai = 1

V = —1 + j
X3 = -1 - j.

From this we obtain, for the modal matrix

5
M = •(3 + 4j) -(3 - 4j)
(2 + j) (2 - j) .

whose inverse is

-2j 2j 8j
M-1 - — : % (1- j) -1 1
10j 1 -1
(1 + j)

From Eq. (150) we obtain, for the Vandermonde matrix

1 1 1
T = 1 -(1-j) -d + j)
1 —2j 2j

The inverse is

4j 4j 2j
T"1 (1 + 3j) (1 - 2j) -(2 + j)
10j -(1 - 3j) -(1 + 2j) (2 - j).
132 ELEMENTS OF MODERN CONTROL THEORY

The transformation matrix for this problem is given by Eq. (154),

where the matrix, R, is calculated from Eq. (155).
Since

8
M-T = -d - j)
10j
(1 + J')_

and

2j
T-'h = -(2 + j)
10j
(2 - j)_

where hr = [0 0 1 ] , we find after substituting the above quantities in Eq.

(155)

_1J-H
fl o o 8j 2j
0 r22 0 -(1 - j) =
“(2 + j)
_0 0 r33_ _ (1 + j)_ (2 - j)_

From this we obtain

1 0 0
0 2(1 + 3j) 0
0 0 2(1 - 3j)

whose inverse is

20 0 0
R-1 i 0 (1 — 3j) 0
Lo 0 (1 + 3j)

Substituting in Eq. (154) yields

J = MR_1T_I

0 -1 1
0 3 1
2 1 1
THEORY OF LINEAR SYSTEMS 133

This is the required transformation matrix that converts the given

system to the canonical (phase variable) form in the manner indicated
by Eqs. (152) and (153).
As a check on the result, we calculate

-1 2
' 1 0
1 0

0 1
J-*AJ 0 0 5= F
2 0

J-T

The same result is obtained by using the transformation defined by

Eq. (156). A simple calculation yields

1 -2 2
E = [f Af A2f] 1 2 -2
1 0 2

The coefficients in the characteristic equation, (158), are

ti = 2, f, = U, f, = - 1

from which we form

0 1 1
ft = 1 1 0
1 0 0

Therefore, by Eq. ( 159)

6 -l 1
* = E ft = 0 3 1
2 1 1
 ELEMENTS OF MODERN CONTROL THEORY
134

As a check, we calculate

N^-l -1
-1

and note that

0 1 0
0 0 1 = F
2 0 -1

and

0
'F~1f = 0 h.
1

Example 9: We are given the system

x = Ax + Bu
v = Cx + Du

where

"-2 -1 1
A = 1 0 1 B =
-1 0 1

2 0 1
C
0 3 0

We seek the solution for v(t) when the initial condition vector, x(0), is
zero, and u(t)is a prescribed function of time.
This can be done in either of two ways:

a. By solving for x(t) from Eq. ( 144).

b. By obtaining the matrix transfer function, Eq. (147), and taking the
inverse Laplace transform of Eq. (146).

In the first case, the crucial step is the calculation of the transition
matrix, <t>(t). The system matrix, A, is identical to that of Example 8.
THEORY OF LINEAR SYSTEMS 135

Consequently, the eigenvalues and modal matrix, M, are as given in

Example 8. We then calculate $(t) using Eq. (148) as follows.

0 5 5
1 (-3 - 4j) (-3+ 4j)
1 (2 + j) (2 - j) _
~eXl‘ 0 0 -2j 2j 8j
0 eXs< 0 ( —1 + j) —1 1
0 0 eXs' _(1 +j) 1 —1

Carrying out the matrix multiplication, we find that the components

of $(t) are:

$11(t) = e~‘(cos t — sin t)

$i2(t) = —e~‘ sin t
$13(t) = e_‘ sin t

$2i(t) = — i [e‘ — e_< (cos t + 7 sin t)]

o

$22(t) = l [e! + e~‘ (4 cos t + 3 sin t)]

5

$23(t) = l [4e‘ — e_‘ (4 cos t + 3 sin t)]

o
$3i(t) = — i [e‘ — e~‘ (cos t — 3 sin t)l
5
$32(t) = i [e‘ — e~‘ (cos t + 2 sin t)]
o
$33(t) = \ [4e‘ + e~‘ (cos t + 2 sin t)].
o

Assume now that the vector, u(t), is composed of a unit step and unit
ramp as follows.

1
u(t)
t

The second quantity on the right-hand side of Eq. (144), which repre¬
sents the particular solution, is a three-dimensional vector whose com¬
ponents thus become

[ [4$n(t — c) 4" t$i2 (t — a) 4- 3t$!3(t — <j)]dcr

^0
136 ELEMENTS OF MODERN CONTROL THEORY

f [4'J)2i(t — o) -p t<J>22(t— o’) -f- 3t€>23(t — a)]dcr

Jo

f [4$3i(t — o) T t4>32
(t — cr) -)- 3t4>33(t — a)]da.
*’o

Using the f>iy(t) obtained above, these components are readily evaluated
by elementary methods. For example, a lengthy but straightforward inte¬
gration yields, for the first of the above components

j' — o) -)- t4>i2(t— c) -F 3t4>i3(t — o’)]da

Jo

= [ {4e“((_<r,[cos (t — o’) — sin (t — a)] — te-(!-<T) sin (t — o)

Jo

-F 3te“(‘_<r) sin (t — o)}do'

= t F e~'[4 cos t — t (sin t -f- cos t)].

Having thus obtained x(t), we find v(t) directly from

v = Cx + Du.

To obtain v(t) via the matrix transfer function approach, we use Eq.
(167). This requires the values of r and P. These are readily calculated
as follows.

0 5 5
"2 o r
r = CM = 1 ( —3 — 4j) ( —3 + 4j)
0 3 0
J (2 + j) (2 - j) _

1 (12 + j) (12 - j)
.3 -3(3 + 4j) -3(3 - 4j)

-2j c7 <—»
•
GO 4 0
P = M-‘B = — -(1 - j) i i 0 1
10j
_ (1 + j) 1 -i _ _io CO

-4 13
1
2(1 + j) -j
5
_2(1 ~ j) j
THEORY OF LINEAR SYSTEMS
137

Therefore

1 V [-4 13] 1 ‘-4 13"

5 _3_ 5 .-12 39

K2 — r 2P 2 1 (12 + j) [2(1 + j) -j]

5 -3(3 -H4j)_

1 2(11 + 13j) (1 - 12j)

5 6(1 - 7j) -3(4 - 3j)_

1 (12 - j) " [2(1 - j) j]

h-3 — r 3P 3
5 -3(3 - 4j)_

1 2(11 - 13j) (1 + 12j)

5 Le(l + 7j) -3(4 + 3j)
We have, finally

K,
L(s) = D + z
(s - X.)

"5 (f '-4 13"

, 1
.0 4_ 5(s - 1) —12 39_

+ 1 [2(11+ 13j) (1 - 12j)

5(s + 1 —j) [6(1 - 7j) -3(4 - 3j)

1 (1 + 12j)
+
2(11 - 13j)
5(s + 1 + j) L6(l + 7j) -3(4 + 3j)_

Since the last two matrices are complex conjugates, the above reduces
to

"5 0" 1 '-4 13"

L(s)
_0 4_ 5(s - 1) — 12 39_

~2(lls - 2) (s + 13)
+
5(s2 + 2s + 2) [6(s + 8) —3 (4s + 7)

Now
138 ELEMENTS OF MODERN CONTROL THEORY

1
s
U(s) = £[u(t)] = £
1
S2

We obtain, therefore

V(s) = L(s)U(s)

1 ' "1
s , 1 1
hP*- 1—1_
1CO s
V (s)
1 5(s - 1) -12 39_ 1
_ S 2_ _ S2_

1
s
2(lls - 2) (s + 13)
+
5(s2 + 2s + 2) _6(s + 8) —3 (4s + 7) 1
S2

Taking the inverse Laplace transform of this expression gives v(t).

Remark: It is apparent that the operations involving multi-input/multi¬

output systems are considerably more complex than those for
single-input/single-output systems. As a matter of fact, purely
formal manipulations in the multi-dimensional case may con¬
ceal various subtleties, the neglect of which may invalidate the
final results. These questions will be taken up in Sect. 1.5.

1.4.2 Sampled Data Systems

Let us now consider the case where the input vector, u(t), is sampled
periodically and constrained to be constant over each interval, i.e.

u[(k + £)t] = u(kr)

0 < £ < 1

k = 0, 1, 2, ••• (169)

and r is the (constant) sampling period. The situation may be depicted

in the schematic form shown in Fig. 1.61. The system dynamics are
described by Eqs. (136) and (137).
THEORY OF LINEAR SYSTEMS 139

u(t) Sample v(t)

System Dynamics
and Hold

Figure 1.61. Sampled Data System.

For any prescribed interval, we have/from Eq. (144)

x[(k + |)t] = <t>(^r)x(kr)+ / <f>(kr+ £r — <r)Bu(kr)d(7. (170)

JkT

If we make the change of variable

i (ar) = kr + £r — a (171)

Eq. (170) becomes

rir
x[(k + £)t] = $(£r)x(kr) + / <t>(^)Bu(kr)di/'. (172)

In particular, for £ = 1

x*+i = Gx* + Hu* (173)

v*+i = Cxjt+i + Du^i (174)

where

G= <t>(r)
= eAT (175)

H= f $(*)Bd^ (176)

and

x*+i = x[(k + 1)t], etc.

xi+1 may be expressed as a function of the initial state and the input
sequence, m, by successive iteration of Eq. (173), viz.

x*4.i = G*+1xo +
(177)

k = 0, 1, 2, ••• .
140 ELEMENTS OF MODERN CONTROL THEORY

We may consider Eqs. (173) and (174) as the state equations of a

discrete time system whose input, state, and output are, respectively,
specified by the vector sequences u*, xk, and vk. This case will be referred
to as the discrete time system. Since A, B,'Ci, and D are constant matrices,
it is easy to verify that G and H are constant matrices.
It should be emphasized that xk 1is the value of the state vector at the
sampling instant only. This point of view leads to the usual methods of
Z transform analysis. Furthermore, the use of Eq. (172) gives the values
of the state vector between the sampling instants and is analogous to the
modified Z transform theory.
For purposes of investigating the system stability, it is sufficient to
consider the homogeneous form of Eq. (177), viz.

x* = G*x0. (178)

Using the Sylvester Expansion theorem, (39) we may write G* as

G — ml
G* =i^n (179)
i= 1 ;=1 -M* Mi-

where m»is a (distinct) eigenvalue of G. Now

G = eAT = MEM-1 (180)

f eXlT 0
E = eX2T
(181)

from Eqs. (175) and (148). The m. are obtained as the roots of

|g - Mi| = o. (182)

However

M_1(G - mI)M = E - fil.

Therefore
THEORY OF LINEAR SYSTEMS 141

Since M is nonsingular, | M 1| and | M | are nonzero, which means that

|G - MI| = |E - MI| = 0.

In other words, G and E have the same eigenvalues. Therefore

a. = eV. (183)

Substituting Eq. (179) in (178), we have

n k
G - ml
x* = z Mi*n X0.
i=l 3=1 lm. Aj
3^ i

It follows that for xk to be bounded as k —> °o , it is necessary and

sufficient that

kl <1. (184)

This is the stability condition for the discrete time system, Eqs. (173)
and (174).
We now seek to express the discrete time system, Eqs. (173) and
(174), in terms of Z transforms analogous to the Laplace transform
representation, Eqs. (146) and (147), of the continuous system, Eqs.
(136) and (137).
The sampled state vector, x*(t), may be written as

x*(t) = Z x(kr)6(t — kr) (185)

k=0

which is the vector equivalent of Eq. (71).* We have, therefore

£[x*(t)] = Z x(kr)£[5(t — kr)] = ^2 x(kr)e kTs= X*(s)

k= 0 k—0

or

Z[x(t)] = X(z) = Z x(kr)z“*. (186)

lc*= 0

*The influence of the multiplying factor, y, in the discussion following Eq. (71), is not relevant
in the present case, since we are dealing with a sample-and-hold operation. See the discussion
following Eq. (84).
142 ELEMENTS OF MODERN CONTROL THEORY

Multiplying Eq. (173) by z~k and taking the summation from k = 0

to oo

2] x[(k+ 1)t] = G ^ x(kr)z~*

+ H ^ u(kr)z-A
4= 0 4=0 4=0

= GX(z) + HU(z) (187)

by virtue of Eq. (186). Also via (186)

00 CO

X(z) = x(kr)z~k = x(0) + 2Z x{kr)z~ki

4=0 4=1

Making use of the Real Translation theorem/3) this becomes

X(z) = x(0) + z-1 £ x[(k + l)T]z~k

k= 0

or

£ x[(k + 1)r] = Z[x(t + r)] = z[X(z) - x(0)]. (188)

k—0

Continuing in this fashion, it is easy to show that, in general

oo m— 1

2Z x[(k + m)r] = Z[x(t + mr)] = zm [X(z) — 2Z x(kx)z_t] (189)

4=0 4=0

where m is an integer.
Substituting Eq. (188) in (187), we find

zX(z) - zx(0) = GX(z) + HU(z). (190)

Similarly, by taking the Z transform of Eq. (174), we find

V(z) = CX(z) + DU(z). (191)

If the initial condition vector, x(0), is zero, then by eliminating X(z)

from the above two equations, we obtain

V(z) = W(z)U(z) (192)

W(z) = C(zl - G)-1H + D. (193)

THEORY OF LINEAR SYSTEMS 143

We say that W(z) is the sampled data matrix transfer function for the
discrete time system, Eqs. (173) and (174).
The transformation

x* = My, (194)

where (as before) M is the modal matrix for A, converts the system, Eqs.
(173) and (174), to the normal form

y,+i = Ey* + Qu* (195)

v, = Ty, + Du* (196)

where

Q = M-1H (197)

and E and r are defined by Eqs. (181) and (164), respectively.

Consequently, the sampled data matrix transfer function, Eq. (193),
may be expressed in terms of the normal matrices as follows.

W(z) = T(zl - E)-!Q + D (198)

This may be put in the form of a partial fraction expansion in a manner

completely identical to that for the continuous matrix transfer function,
Eq. (165).

Example 10: Consider the discrete time system shown in Fig. 1.62.
We seek to obtain the solution for xi(t) at the sampling instants, first by
the state variable methods and then by the usual Z transform technique.
In terms of state variables, we may express the system dynamics by

Xi = x2

x2 = — 2xi — 3x2 + u(t)

Sample and Hold

Figure 1.62. Discrete Time System for Example 10.

144 ELEMENTS OF MODERN CONTROL THEORY

where u(t) satisfies Eq. (169). This is in the form

x = Ax + Bu

= Cx + D

where

0
A = B =
-2

C = [1 0] D = 0.

The characteristic equation is

|XI - A| = X2 + 3X + 2 = 0

which yields the eigenvalues

Xj = -1

X2 = —2.

Consequently, the modal matrix for A is

1 -1
M
-1 2

and its inverse is given by

M->
[? ;]•
Also

E
, =
fex>T 0
0 exA

while

<I>(t) = MEM 1= G

(2e T — e 2T) (e-r — e~2T)

—2(e-T — e-2T) —(e-T — 2e_2T)
THEORY OF LINEAR SYSTEMS 145

For t = 1 , this becomes

0.6005
-0.4652
0.2326
-0.0973 :]
and from Eq. (176)

0.1998
0.2325

These computations permit the system to be expressed in the matrix

difference equation format of Eqs. (173) and (174). The solution in this
case is given by Eq. (177).
To derive the Z transform of the system from the state variable repre¬
sentation, we calculate

r = CM = [1 -1]

and use Eq. (198), viz.

W (z) = T(zl - E)_1Q

0.6321 0.4323 0.2(z + 0.368)

z - e-i z - e"2 (z - 0.368) (z - 0.135)

As a check on this, we calculate the Z transform of the system directly

from Fig. 1.62 as follows.

(1 - e~”) 1
W(z) = Z
s (s2 + 3s + 2)_

= (1 - z-')Z
Ls(s2 + 3s + 2)J
146 ELEMENTS OF MODERN CONTROL THEORY

z 0.5z

(z - e !) (z - e 2)_

0.2(z + 0.368)
(z - 0.368) (z - 0.135)

1.5 OBSERVABILITY AND CONTROLLABILITY

Much of the recent literature in control theory is based on the de¬

scription of a given system by vector differential (or difference) equations.
The classical approach, however, represents the physical system by
transfer functions, i.e., the Laplace transforms of the differential equa¬
tions relating the output to the input. There has always been the implicit
assumption that these two methods are essentially the same and will
yield identical results. Apparently, clear and rigorous demonstration of
the fact that such an equivalence is true only under carefully stipulated
conditions was first given by Kalman, (42> who introduced the concepts
of controllability and observability. Before proceeding with a formal
development of these ideas, it is instructive to consider an example that
illustrates the basic problem.
Consider the feedback-control system shown in Fig. 1.63. An ele¬
mentary calculation shows that

V(s) _ s — 1
U(s) s + 5

which indicates that the system is stable. However, notice that a pole
and zero have been cancelled in the right-half plane. This operation,
which appears theoretically valid, must be carefully examined, as we will
show.
If we now choose state variables as shown in Fig. 1.63, we may
describe the system by the dynamic equations

xi = — 4xi 4- 5x2 — 5u

*2 = Xl + U

v = xi — x2 -F u
THEORY OF LINEAR SYSTEMS
147

Figure 1.63. Feedback Control System.

or, equivalently, by

x = Ax + Bu

v = Cx -f Du

where

C = [ 1 -1] D = l.

The eigenvalues for the system matrix, A, are readily obtained; viz.

Xi = —5

X* = 1

which indicates instability.

It appears that in passing from the state variable to the transfer
function approach, a vital feature of the system response has been lost.
This transition is even more subtle (and hazardous) in the multidimen¬
sional case. In the analysis of sampled data systems, the problem of
“hidden oscillations”!43) is of this general type. The above example
shows that hidden oscillations may occur even in continuous-type
systems.
148 ELEMENTS OF MODERN CONTROL THEORY

a. Under what conditions is it permissible to represent a given dynamical

system by a matrix transfer function?
b. Given a matrix transfer function, what is the equivalent dynamic
system?

In answering the above questions we shall invoke the concepts of

observability and controllability, which will be developed ab initio. It
will be shown that a disregard of the first question can lead to erroneous
results; indeed, much of the extensive literature on multivariable control
systems is suspect for this reason. (34-37) The second question is not at all
trivial. It is very easy to underestimate or overestimate the order of a
system from an examination of the matrix transfer function. Some
examples later in this section will illustrate the difficulties encountered.
We begin with some basic definitions and theorems. These will relate
to the dynamic system described by Eqs. (136) and (137), which are
repeated here for convenience.

x = Ax + Bu (199)

v = Cx + Du (200)

The normal (or diagonal) form of these equations is given by

y = Ay + Pu (201)

v = Ty + Du (202)

which are merely Eqs. (161) and (162) of Sect. 1.4.1.

Definition 1: A system is said to be completely state controllable if,

for any t0, each initial state, x(t0), can be transferred to
any final state, x(t/), in a finite time, t/ > t0.

The word “completely” is used here to conform to Kalman’s termi¬

nology. He defines the concept of a controllable state and uses the word
“completely” to emphasize that every initial state is controllable. Since
controllability is here defined as a property of the system, the word
“completely” is somewhat redundant. Consequently, as used in this
section, the words, “complete controllability (or observability)” will be
used interchangeably with “controllability (or observability).”
THEORY OF LINEAR SYSTEMS 149

Definition 2: A system is said to be completely output controllable if,

for given t0 and tf, any final output, v(t/), can be attained
starting with arbitrary initial conditions in the system at
t — to-

Definition 3: An unforced system is said to be completely observable

if, for given t0 and t/, every state, x(t0), can be determined
from the knowledge ofv(t)in the interval (t0, t/).

It is easy to showd) that any system, S, may always be partitioned

into four possible subsystems (shown in Fig. 1.64) as follows.

a. A system. S'4, that is state controllable and observable.

b. A system, SB, each of whose normal coordinates are observable and
uncontrollable.
c. A system, Sc, each of whose normal coordinates are controllable and
unobservable.
d. A system, SD, each of whose normal coordinates are uncontrollable
and unobservable.

According to this decomposition, the only subsystem that has to do

with the relationship of v to u is S'4. The observable subsystem, SB,
only adds a disturbance to the controlled part of the output. It is
apparent that any analysis that neglects subsystems, SB, Sc, and S'0, will
be erroneous and possibly catastrophic, especially if the state variables
associated with these subsystems become large.
Consider now the transfer function matrix for the system, S, of Eqs.
(199) and (200), which is given by Eq. (164), viz.

L(s) = C(Is - A)-1B + D

= T(Is - A)-‘P + D. (203)

With respect to this transfer function and its relation to system S as

represented by its partitioned subsystems, the following theorem is of
fundamental importance.

Theorem I (Kalman-Gilbert): The transfer function matrix, L(s),

represents, S'4, the state controllable and observable subsystem of S.
An immediate consequence of this theorem is that unless the system,
S, is completely state controllable and observable (i.e., S = S'4), the
150 ELEMENTS OF MODERN CONTROL THEORY

Figure 1.64. Partition of Dynamic System.

transfer function matrix will not represent all the dynamic modes of the
system. If any of these modes is unstable, “hidden oscillations” will
occur.
It is important, therefore, to establish criteria for determining the
controllability and observability of given dynamic systems. The fol¬
lowing theorems serve this purpose.

Theorem II: The system, S, represented by Eqs. (199) and (200),

is completely output controllable if, and only if, the composite
q x (n + 1) p matrix

[CB : CAB • CA2B | . | CA-^B j D]

is of rank q.
For complete state controllability, the above matrix, with C = I and
D = 0, must be of rank n.
Theorem III: The system, S, represented by Eqs. (199) and (200),
is completely observable if the composite n X nq matrix
[Cr ; (a^C1, ; . ; (Ar)n-icr]
is of rank n.
If the eigenvalues of the system matrix, A, are distinct, then the
following theorem due to Gilbert!31) is perhaps simpler to apply.
THEORY OF LINEAR SYSTEMS 151

Theorem IV: Let the system, S, be represented by its normal form,

Eqs. (201) and (202). Assume that the eigenvalues of the system matrix,
A, are distinct. Then the system, S, is completely state controllable if P
has no rows that are zero; it is also completely observable if r has no
columns that are zero.

Example 1 1: Consider the case introduced at the beginning of this

section and shown in Fig. 1.63.

We found that

-4 5
A = B =
1 0_
C = 1 -1] D = 1

\i = -5 n = 2

X2 = 1 q= 1

Some simple computations yield

CB = -6 CAB = 30

' 25' '-5"

kTCT -
—5_ 5_

Substituting in the matrix of Theorem II

[CB i CAB i D] = [-6 30 1]

= rank 1 = q

which means that the system is completely output controllable.

However

-5 25
[B i AB] 1 —5

= rank 1 < n

so that the system is not completely state controllable.

152 ELEMENTS OF MODERN CONTROL THEORY

Furthermore

[Cr | ArCr] = _ 1 5

= rank 1 < n

so that by Theorem III, the system is not completely observable.

To check these results by Theorem IV, we compute

1
M = modal matrix for A
1

1 1
M_1= k o 1 5

1
p = M-1B =
0

r = CM = [-6 0].

Since P has a zero row and r has a zero column, the system is neither
completely state controllable nor observable.
We would expect, therefore, that the transfer function calculated by
Eq. (203) would not contain all the dynamic modes of the system. We
obtain, in fact

L(s) = r(Is - A)_1P + D

_ 1 0
= [-6 0] s + 5 1 + 1
0 s — 1

6 s - 1
+ 1
s + 5 s + 5'

Example 12: The system considered is described by Eqs. (199) and

(200), with

u 1 0 ‘ 0
A = 5 0 2 B = 0
-2 0 _2 _0.5_

C = [-2 1 0] D = 0.
THEORY OF LINEAR SYSTEMS
153

We readily compute

X» = - 1

X2 = -3

X3 = 2

0.5 0.5
M = -1.5 1
1 -0.25.

-0.625 0.625 1.25

M-1 = 2.25 -0.75 1.5
30
4 2 1

0.625
P = M-1B = 0.75
30
0.50

T = CM = [3 -2.5 0],

Therefore, according to Theorem IV, the system is completely state

controllable, since P has no zero rows; however, it is not completely
observable, because there is a zero column in r. We would, therefore,
expect the transfer function, L(s), to have order two instead of three. A
straightforward computation shows that

L(sj = T(Is — A)-1P

' 1 0 0
r 0.625~1
s + 1 1 0
[3 -2.5 0] 1 0.75
0 s -f- 3
0 0 s - 2
0.50 _

(s + l)(s + 3)

The transfer function does not contain the unstable mode corre¬
sponding to the eigenvalue equal to two.
Checking for the controllability and observability of large complex
systems may be quite tedious. The use of the following three theo-
rems(31> reduces the workload somewhat.
154 ELEMENTS OF MODERN CONTROL THEORY

Figure 1.65. Parallel Connection of Two Systems.

Theorem V: Let the parallel connection of systems Si and S2 form

a composite system, S. (See Fig. 1.65.) Then a necessary and sufficient
condition that S be completely state controllable (observable) is that
both Si and S2 be completely state controllable (observable).

Theorem VI: Let the series connection of system Si followed by S2

form a composite system, S. (See Fig. 1.66.) Then a necessary (though
not sufficient) condition for the complete state controllability (observa¬
bility) of S is that both Si and S2 be completely state controllable
(observable).

Figure 1.66. Series Connection of Two Systems.

If Si and S2 are both completely state controllable (observable), any

uncontrollable (unobservable) coordinates of S must originate, when
designated according to eigenvalue, in S2 (Si).

Theorem VII: Let Si and S2 be the systems in the forward and

return paths, respectively, of a feedback system, S. (See Fig. 1.67.)
Let S„ denote the series connection of Si followed by S2. Also, let the
series connection of S2 followed by Sj be denoted by S0. Then a neces¬
sary and sufficient condition that S be completely state controllable
(observable) is that Sc (S0) be completely state controllable (observable).
Furthermore, if S2 is static, i.e., contains only pure gain elements, S
is completely state controllable and observable if Si is completely state
controllable and observable.
THEORY OF LINEAR SYSTEMS 155

Theorem VII is important because closed-loop controllability and

observability can be ascertained from the open-loop systems, Scand S0.
Thus, one is not forced to deal with intricate closed-loop equations.
The discussion thus far has been limited to continuous-type systems.
Extensions to the discrete time case have been made by KalmanGD and
Sarachik and Kreindler.(4°) The definitions and theorems that follow
relate to the discrete time system described by Eqs. (173) and (174),
which are repeated here.

x*+i — Gx* + Hu* (204)

v* = Cx* + Du* (205)

For this case, the relevant sampled data matrix transfer function is
given by Eq. (193) or Eq. (198), viz.

W (z) = C(zl - G)_1H + D

(206)
= T(zl — E)_1Q + D.

Definitions 1, 2, and 3 for complete state controllability, complete

output controllability, and complete observability carry over completely
to the discrete time case, with obvious minor modifications.
Also, Theorem II holds for the discrete time case if the test matrix is
replaced by

[CG • CGH - CG2H j . . CG^H | Dj. (207)

Similarly, Theorem III is valid for the discrete time case if the test
matrix is replaced by

[Cr ; Grcr i : (Qr)»-iCr]. (208)

156 ELEMENTS OF MODERN CONTROL THEORY

In other words, Theorems II and III are true for discrete time systems
of the type described by Eqs. (204) and (205), if A is replaced by G, and
B is replaced by H.
A natural question presents itself at this point. Suppose that the
system represented by Eqs. (199) and (200) is completely state control¬
lable and observable. How are these properties affected by the intro¬
duction of sampling? An answer to this question is contained in the
following^41)

Theorem VIII: Suppose that the system, Eqs. (199) and (200), is
completely state controllable and observable in the absence of sampling.
Then, if u(t) is constant over each sampling interval, r, a sufficient con¬
dition for the complete state controllability and observability of the
sampled system is

27rm
Im(Ai - Ay)

whenever

Re(Ai - Ay) = 0

where A,-and Ay are eigenvalues of A, and m = ±1, ±2, • • •.

If u(t) is a scalar, this condition is necessary as well.

Example 13: Consider the system, Eqs. (199) and (200), where

0 0
A = 0 1
- (a2 + tt2)

1
B = 0

7T

C = [1 1 0J

D = 0.

It is assumed that u(t) satisfies Eq. ( 169). In this case, we may represent
the system in the discrete-time form, Eqs. (204) and (205), proceeding as
follows.
THEORY OF LINEAR SYSTEMS
157

|XI - A| = 0

Xi = —7

X2 = —a + ]7r

X3 — — a — J"-

0 0
M = 1 1
— (a — j 7r) •(a + jjr).

2j 7T 0 O'
M-1 = —1^
0 (a + j7r) 1
2j ir 0 — (a — j 7r) -1

1
P = -0.5j
0.5j

r = [i 1 i]

Taking r = 1, we find that the only nonzero components of 4>(r) are

<t>n(l) = e-7

$22(i) = —e~a

4*33(1)= —a

which means that

0 0
— e~a 0
0 -e~“

while

" (1 - e-7)
y Hi

tt (e-“ - 1)
H = = h2
a2 + 7r2
0 _0 _
158 ELEMENTS OF MODERN CONTROL THEORY

hi (1 - e~7)
7

7r (a + j7r)(e~a — 1)
a2 tt2

a( —a + j7r)(e~a — 1)
a2 + 7r2

Using Theorem IV for the continuous system we find that the system
is completely state controllable and completely observable, since P has
no zero rows and r has no zero columns.
However, by applying Theorem II to the discrete time system, we find

[H ; GH j G2H]

"Hi e_7H1 e-27H1~

= H2 -e~“H2 e-2“H2
_ 0 0 0

which is of rank 2 (< n = 3), so that the discrete time system is not
completely state controllable. Furthermore, using the discrete time
version of Theorem III

[Cr ; Grcr ; (GT)2cr]

~1 e~7 e-27'
= 1 -e“« e-2"
.0 0 0

which is also of rank two. Therefore, the discrete time system is not
completely observable.
The same result could have been obtained directly from Theorem
VIII, since

Im(X2 — X3) = 27r

when He (X2 — X3) = 0.

As a result of this analysis, we would expect that the Z transform of
the system would not contain all the dynamic modes. In fact we find,
from Eq. (206)
THEORY OF LINEAR SYSTEMS 159

W(z) = T(zl - E)_1Q

0 0
z — e 27T,jHl
1
= [1 1 1] 0 0 (a + j7r)H2
z - eXz, 2ir]
( —a + j7r)H2_
0
z — e

(1 - e~y) 7r(e-a + 1)
y(z — e y) (a2 + 7r2)(z + e~a)

As a check on this, we may calculate the Z transform from the

schematic representation of the system shown in Fig. 1.68. From this
we obtain the system transfer function as

V(s) 1
+
U(s) J Ls + 7 (s + a)2 -(- 7r2_

Taking the Z transform, we find

W(z) = (1~e~Tr) +
t7(z — l)(z — e yT) (a 2 -f- 7r2) .(z - 1)

ze~aT sec 6 cos (ttt — 6)

2ze aT cos ttt + e 2aT J >

where

- .an-(f)

Figure 1.68. Schematic of System of Example 13.

160 ELEMENTS OF MODERN CONTROL THEORY

For t = 1 , this reduces to the value of W(z) obtained above.

Inspection of the G matrix for this example indicates that there are
three modes: one corresponding to the eigenvalue, mi = e-7, and the
other two corresponding to the multiple eigenvalue (of order 2), m2 = —e~a.
In the Z transform, W(z), only a single, rather than a multiple, eigenvalue
is apparent for —e~“. This is because the observability and controllability
conditions are not satisfied. Furthermore, this is precisely the reason
why some sampled data systems exhibit the phenomenon of “hidden
oscillations.”
We now turn our attention to the second main problem posed at the
beginning of this section: “Given a matrix transfer function, what is
the equivalent dynamic system?”
It should be pointed out, to begin with, that the state-space representa¬
tion for a given transfer function is not unique, because the choice of
state variables is, to some degree, arbitrary. However, the differential
equation representation is unique (assuming zero initial conditions) if the
system is controllable and observable. In this case, it can always be
reduced to an appropriate state variable form. However, in some cases, a
pole and zero of the system may cancel, in which case the transfer
function will not represent the actual system and hidden oscillations may
occur. Furthermore, if the system is not controllable, it cannot be
reduced to phase variable form, and some of the transformation matrices
will be singular.
To provide some motivation for the following discussion, consider the
matrix transfer function

1 2

(s + 1) (s + 1)
L(s) (209)
-1 _ 1_
(s + l)(s + 2) (s + 2)

This may be written in the form of a partial fraction expansion as

follows.

K,-
L(s) = z
(s - s.)

where K, is the ith residue matrix given by

K,: = lim [(s — Si)L(s)]

THEORY OF LINEAR SYSTEMS 161

and the st- are the poles of the matrix elements in L(s). For the special
case of Eq. (209), we have

Si = — 1 s2 = -2

and

1 2
Ki
-1 0
(210)
0 0
K2 =
1 1

Consequently, it would appear that the weighting function matrix

corresponding toL(s) is

£ ^(s)] = Kiesi‘+ K2es2‘

which implies that the dynamic system corresponding to L(s), given by

Eq. (209), is of second order. It will be shown later that L(s) actually
corresponds to a third order system.
A method for constructing the state variable representation of a given
matrix transfer function of correct order is contained in the following
important theorem due to Gilbert/31)

Theorem IX: Given a rational matrix transfer function, L(s), whose

elements have a finite number of simple poles, s,-, i = 1,2,..., m. Let
the partial fraction expansion of L(s)be

m K
L(s) = E + D (211)
i= 1 S - s,

where

K, = lim [(a - s,)L(s)] (212)

8— *8 i

D = lim L(s). (213)

8 — %º«

Let the rank of matrix Ki be denoted by r,-. Then L(s) can be repre¬
sented by a system of differential equations, Eqs. (199) and (200) or
 162 ELEMENTS OF MODERN CONTROL THEORY

(201) and (202), whose order is

n= Ir, (214)
i=l

Applying this theorem to the matrix transfer function given by Eq.

(209), we note that since the corresponding residue matrices, K2 and K2,
given by Eq. (210), have rank two and one, respectively, the correspond¬
ing system is third order.
It is apparent from Theorem IX that the system order, n, is equal to the
number of distinct poles, m, in the elements of L(s) only when the rank
of every K; matrix is one. The partial fraction expansion given by Eqs.
(167) and (168) for L(s) in the special form, Eq. (165), ensures that the
resulting K, all have rank one. In general, however, L(s) is in some
arbitrary form as a result of compensation and design manipulation. In
this case, Theorem IX must be used to check the correct order of the
system.
Gilbert!31) has also given a constructive procedure for determining the
state variable representation from the matrix transfer function. This
may be summarized as follows.
Since K, is of rank ri} there are r ; linearly independent columns in K;.
Form a matrix, />„ consisting of these r, linearly independent columns.
We may then write

Ki = PSi (215)

where /3; is to be determined. A simple manipulation shows that*

Pi = {piTPi)~lPiTK„ (216)

The corresponding state variable representation is given by Eqs. (201)

and (202), where

X2I2 (217)
**XmIn

*The matrix (p;r Pi), is nonsingular, since Gram determinant [pi7 p<| is nonzero if the columns of
pi are linearly independent.
THEORY OF LINEAR SYSTEMS 163

0i
02
P = (218)

0 m.

P [plP2* Pm]
(219)
I, = unit matrix of order r,

and\i =
Note that is an eigenvalue of multiplicity r,: and that the order of
m

system is n = X! in.
i= 1

Example 14: Given the matrix transfer function shown in Fig.

1.69, we seek to derive the corresponding differential equation repre¬
sentation. In the absence of a sound systematic procedure, it would be
a formidable task to obtain the equivalent state variable representation of
correct order. Using the method outlined above, the procedure is quite
straightforward. The poles of the elements in L(s) are obtained by
inspection, viz.

51 — — 1

52 = —2

53 = —3

s4 = —4

s4 = —5.

Consequently, from Eq. (212)

Ki = lim [(s — Si)L(s)]

8—*81

"8 0 0 0
0 0 4 1
3 1 0 3

This is obviously of rank three, so that

164 ELEMENTS OF MODERN CONTROL THEORY

+
CO

<N

+
<N
CQ
lO

+
CO

+
CQ
CM
CO +

.1
ti¬
Tt<
iO +
+ CQ

CQ
+
CM
00
+
CO

+ +

CQ CM

ll

CQ
THEORY OF LINEAR SYSTEMS 165

8 0 0
Pi = 0 0 4
3 1 0

and

1 ' —3 0
(pirpi) 1 -3 73 0
64
0 0 4

From Eq. (216) we find

1 0 0 0
Pi = 0 1 0 3
0 0 1 0.25

In similar fashion, we obtain

-4.5 -3 0 1
K2 0 0 -6 0 = rank two
0 0 0 0

-3 0
0 -6
0 0

1.5 1 0 -0.333
& =
0 0 1 0

0 0 1 f
k3 1 1 2 2 = rank two
-3 -3 1 1

0 1
1 2
-3 1

110 0
0 0 11

-0.5 9 1 0
K 0 0 0 0 = rank one
0 0 0 0
166 ELEMENTS OF MODERN CONTROL THEORY

P4 = 0
_0_

/?4 = 0.5 9 1 01
-

0 0 0 o'
k5 = -
1 0 0 -3 s= rank one
2 0 0 6_
-

o'
P5 =
—
1

2_

$5 = [1 0 0 3].

Using Eqs. (2 17)—(2 19), we find

-Is
-21,
A = %3U
-41,
-51,

s o o -3 0 0 1 1
r = 0 0 4 0 -6 1 2 0
3 1 0 0 0 -3 1 0

1 0 0 0
0 1 0 3
0 0 1 0.25
1.5 1 0 0.333
P = 0 0 1 0
1 1 0 0
0 0 1 1
0.5 9 1 0
0 0 3

The system is obviously of order nine, with

s, = X, = —1 = pole of order three

«2 = X2 = —2 = pole of order two

s., = X;i = —3 = pole of order two

THEORY OF LINEAR SYSTEMS
167

s4 = \4 = —4s simple pole.

Ss = X5 = —5s simple pole.

Having A, P, and r, the state variable form is given directly by Eqs.
(201) and (202). That the input vector, u(t), is of dimension four is not
apparent from inspection of the given L(s).

1 .6 THE INVARIANCE PRINCIPLE

In virtually every realistic control system, the controlled variable is

sensitive to some type of external disturbance. Assuming that only one
external disturbance is significant, the design of a feedback control
system reduces to that of determining the transfer functions

~~ = T*(s) (220)
R(s)

— = TjXs) (221)
D(s)
where

R(s) = Laplace transform of reference input

-D(s) = Laplace transform of disturbance input

C(s) = Laplace transform of controlled variable.

For purposes of obtaining independent control of TB(s) and Tc/(s), two

separate compensation networks are required (the so-called “two-degree-
of-freedom” system).
For definiteness, consider the control system shown in Fig. 1.70. Here
G2(s) represents the transfer function of the fixed plant, and Gi(s) and
D(S)

Figure 1.70. Two-Degree-of-Freedom Feedback Control System.

168 ELEMENTS OF MODERN CONTROL THEORY

H(s) are compensation networks that are to be designed such that the
transfer functions of Eqs. (220) and (221) are realized. It is immediately
evident from Fig. 1.70 that

C(s)= G1(s)G2(s) = T (222)

R(s) 1 + Gi(s)G2(s)H(s)

C(s) _ G2(s)
D(s) 1 + Gi(s)G2(s)H(s)

1 C(s) {ZZJJ
= T£7(s).
~ Gi(s) Lr(s)J

It is apparent that a suitable choice of G2(s) and H(s) permits T*(s)

and T(,(s) to be realized independently. For example, Gi(s) may be
selected from the requirement

C(s) = TVs) = T ^s)

D(s) Gx(s)

H(s) may then be chosen such that Eq. (222) is satisfied.

Ideally, one may require that Tt/(s) = 0. In principle, this may be
achieved by putting Gx(s) = K, where K is theoretically infinite. Then,
from Eq. (223), it follows that T„(s) « 0, while Eq. (222) reduces to
Ti;(s) « 1/H(s).
In practice, in addition to the obviously unrealistic requirement for
infinite open-loop gain, the above approach is severely limited by the
fact that the compensation networks often require an excess of zeros
over poles, leading to intolerable problems of noise and physical reali¬
zability. However, this technique is useful and feasible when reasonable
constraints are placed on the TH(s) and T(7(s) transfer functions. Various
schemes based on these ideas are treated in the literature/1 >4M7)
In this section, we will consider a slightly different approach to the
problem of making a control system insensitive to external disturbances.
This concept, which was originated and developed to a high degree by
Russian scientists, has been termed the “Invariance Principle. ” The
present exposition leans heavily on a survey paper by Preminger and
Rootenberg,(44) which also contains extensive references to the Russian
literature.
The basic idea is extremely simple. Suppose that the transfer function
relating the controlled variable to an external disturbance is given by
THEORY OF LINEAR SYSTEMS 169

C(s) _ B(s)
(224)
D(s) A(s)

where A(s) and B(s) are polynomials in s.

We say that there is “absolute invariance” relative to c(t) if

B(s) = 0 for d(t) ^ 0. (225)

A system has “conditional invariance” relative to c(t) if

B(s)D(s) = 0 when B(s) 3^ 0

(226)
and d(t) 5^ 0.

Obviously, conditional invariance is dependent on the form of the

disturbance. In this case, absolute invariance is realized for only one
form of disturbance.
One may also have “steady-state invariance” relative to c(t), a condi¬
tion that occurs when the influence of d(t) on the steady-state value of
c(t) is cancelled. This situation is realized by zeroing certain coefficients
in the B(s) polynomial. For example, by zeroing the constant term in B(s),
one obtains a zero steady-state error in response to step disturbances. By
zeroing other coefficients in B(s), steady-state invariance for other types
of disturbances may be achieved.
The essential premise in the principle of invariance is that the dis¬
turbance itself is used to generate a signal that will cancel the influence
of the disturbance on the controlled variable. To make these ideas
precise, consider the system shown schematically in Fig. 1.71. A simple
calculation shows that

— } = G2(s)[L1(s)- L2(s)Gi(s)]. (227)

D(s)

Therefore, in order to achieve absolute invariance, the transfer func¬

tion, L2(s), must be

(228)

By suitable design of L2(s), conditional or steady-state invariance can

also be achieved.
170 ELEMENTS OF MODERN CONTROL THEORY

Figure 1.71. Absolute Invariance via Additional Path from Disturbance.

The problems of stability or dynamic behavior of the system are not

relevant for cases in which absolute invariance is realized, since no
transients appear at all with changes in the disturbance. Transients will
appear in systems with only conditional or steady-state invariance, but
the additional path from the disturbance does not affect stability of the
controlled system, provided elements G2(s), Lr(s), Gi(s), and L2(s) are all
stable.
In some cases, it is simpler to sense a variable that is dependent on the
disturbance rather than sensing the disturbance itself. Consider, for
example, the system indicated schematically in Fig. 1.72, where the
multiple input-output block represents the matrix equation

"Y(s)“ Gn(s) Gi2(s) ‘D(s)“

(229)
_C(s)_ s) G22(s)_ _X(s)_
Here, one feeds back Y(s),
providing a feedback loop from D(s) directly. It is readily determined
that for this case
C(s) _ G2i(s) T Gi(s)L(s)[Gi2(s)G2i(s) — Gn(s)G22(s)] (230)
D(s) “ 1 + G12(s)G1(s)L(s)

Figure 1.72. Variance via an Additional Path from an Internal Variable.

THEORY OF LINEAR SYSTEMS 171

Consequently, in order to realize absolute invariance, we must have

_ G2i(s)_
L(s) (231)
Gi(s)[Gii(s)G22(s) — Gi2(s)G2i(s)]

In contrast to the previous case, wherein the disturbance signal was fed
back directly, the use of a disturbance-dependent variable as an added
loop definitely affects system stability.
From the sensitivity point of view, transfer function G2i(s) and Gi^s)
behave open-loop with regard to parameter variations.!48) However,
sensitivity to parameter variation in Gn(s), G22(s), Cn(s), and L(s) is re¬
duced, because these systems behave “closed-loop” in this respect.
Nevertheless, parameter variations anywhere in the system affect the
invariance condition adversely.
While the use of the invariance principle is theoretically attractive, its
practical application is limited by four main problems:

a. The requirement for differentiating networks in the feedback path

from the disturbance or disturbance-dependent variable.
b. The extreme sensitivity of the system to parameter changes.
c. The lack of accurate instrumentation to sense the disturbance.
d. The appearance of additional disturbances, apart from the one for
which the system was originally designed.

In spite of these difficulties, a substantial improvement in performance

quality can be obtained if invariance is employed in conjunction with a
feedback system. Figure 1.73 shows a feedback system that incorporates

D(s)

Figure 1.73. Feedback System with Absolute Invariance via Disturbance Feedback.
172 ELEMENTS OF MODERN CONTROL THEORY

Figure 1.74. Feedback System with Absolute Invariance via Disturbance-Dependent

Variable Network.

a path from the disturbance directly. Figure 1.74 shows the use of an
additional loop from a disturbance-dependent variable. The main ad¬
vantage in using invariance with a feedback system is the reduced sensi¬
tivity to parameter variations. While in practice it is not feasible to achieve
a theoretically exact absolute invariance, it is possible to design high
quality control systems that have a high degree of “approximate invari¬
ance.”
In the case of direct disturbance feedback, as illustrated in Fig. 1.73,
stability is not affected by the additional path.* Here we find that

C(s) = G2(s)[L1(s)- L2(s)G1(s)]

D(s) 1 + Gi(s)G2(s)H(s) 1
which means that for absolute invariance, we must have

L2(s) = (233)
Gi(s)

One of the main difficulties encountered in this case is the difficulty

of sensing D(s) accurately.
If we use the disturbance-dependent variable feedback as shown in
Fig. 1.74, then

c(b) = g21(s)- g^sIlc^g^GmCs) - gu(s)g22(s)1 (234)

D(s) 1 + Gi(s)Gu(s)L(s) + G22(s)H(s)

Provided, of course, that L2(s) is stable.

theory of linear systems
173

The condition for absolute invariance becomes

_ G2l(s) _
L(s) (235)
Gl(s)[Gi2(s)G2l(s) — Gii(s)G22(s)]

It is tempting to investigate the possible application of the invariance

principle to launch vehicle autopilots, where wind gusts take the role of
external disturbances. The simplified equations, obtained from Appendix
A, are as follows.

mU (a ' - 6) = TCS - Laa (236)

6 = ij,c5 + naa (237)

a = a1 + aw (238)

, w
Y = - (239)
U

5c = K A(ec - KRe - e) (240)

8 -\- Kc5 = KCSC (241)

Quantity aw represents the external wind gust, i.e. , it is the ratio of

wind velocity normal to the vehicle to vehicle forward velocity. It is
generally not feasible to measure aw directly. However, an accelerometer
can sense w, which is, therefore, a measure ofa'(assuming U is constant).
Consequently, we will takea'as the disturbance-dependent variable.
With reference to Fig. 1.74, we identify the notation of the present
problem as follows.

R(s) = 0C(s)

C(s) = d( s)

X(s) * «(s)

Y(s) S « '(B)

D(s) = aw(s)
174 ELEMENTS OF MODERN CONTROL THEORY

Also

K,KC (242)
Gi(s)
s + Kc

H(s) = Kffis + 1. (243)

Eliminating a from Eqs. (236) and (237) via Eq. (238), we obtain,
after some reduction

/
a Gn(s) Gl2(s) aw

(244)
j _G2l(s) G22(s)_ J

where

Gn(s) (245)

(246)
Gl2(s)

G2l(s)
(247)

G22(s) (248)

Absolute invariance of d with respect to aw is assured if L(s) satisfies

Eq. (235). In the present case, using Eqs. (245)-(248), this reduces to
THEORY OF LINEAR SYSTEMS 175

L(s)
mU(s
+Kc)s (249)
T,K,K,
(l+|)
The result thus obtained, while theoretically attractive, suffers from
obvious practical limitations. The zeros in the numerator of L(s) intro¬
duce serious problems of noise and physical realizability. Furthermore, a
simplied mathematical model of the actual vehicle was used to obtain the
result, and it may be expected that a more complete representation of
the vehicle would introduce substantial complexities. Finally, the actual
vehicle parameters are time varying, and this would tend to vitiate the
invariance properties.
However, one succinct feature of the compensating network, L(s), is
quite apparent: the need to cancel the actuator dynamics and use a
precisely defined gain in the network. The use of acceleration feedback
for purposes of load reduction is, of course, not new. Nevertheless, the
present approach to the problem is novel, and it would appear that
attempting to achieve a kind of approximate invariance in the manner
outlined would be worthwhile. To our knowledge, no investigation along
these lines has been made.
The discussions thus far have been in terms of system transfer func¬
tions, a procedure that has the virtue of familiarity, since United States
control engineers have traditionally used this approach. Much of the
Russian literature on the invariance principle is, however, expressed in
the state variable format, leading to results that are sometimes more
convenient for special purposes.
We may formulate one type of invariance problem as follows. Suppose
we are given the system
x = Ax -f- fu (250)

v = Cx (25 1)
which is expressed in the state variable format of Sect. 1 .4, i.e.

x = n dimensional state vector

u =e input; a scalar

v = n dimensional output vector

A = n X n (constant) matrix
f = n dimensional (constant) vector

C = n dimensional (constant) row vector.

 176 ELEMENTS OF MODERN CONTROL THEORY

What are the conditions that ensure that v is absolutely invariant with
respect to u? This problem has been solved by Rozenoer,(45> who
showed that a necessary and sufficient condition for this invariance is
that the relations*

CrAT = 0
(252)
k = 1, 2, - (n - 1)

be satisfied.
This result is typical of those derived in the Russian literature, which
nevertheless is of a mostly theoretical nature concerned with questions of
existence and realizability.
A simple application of the above result is contained in the following.

Example 15: Consider a second order version of the system, Eqs.

(250) and (251), with

-3 o
A =
-4 a22

C = [2 1]

Parameters a22 and fi are to be determined such that v is absolutely

invariant with respect to u.
In the present case, condition (252) reduces to

ficj -T f2c2 — 0

fi(cian + c2a2i) + f2(ciai2 + c2a22) = 0.

Substituting numerical values

2f, + 10 = 0

fi(— 6 ~ 4) + 10(10 + a22) = 0.

We find, therefore

fi — o

a22 = — 15.

*The superscript T, as usual, denotes transpose, while k is an exponent.

THEORY OF LINEAR SYSTEMS 177

As a check on stability, we obtain for the eigenvalues of A

Xj = —5

X2 = -13

which shows that the system is stable. '

 APPENDIX A

A PITCH PLANE ATTITUDE CONTROL

SYSTEM FOR A LAUNCH VEHICLE

NOMENCLATURE

lyy = moment of inertia of reduced vehicle (i.e. , excluding slosh masses)

about pitch axis
= servoamplifier gain
Kc = engine servo gain
K / = integrator gain
= rate gyro gain
t = length parameter along vehicle; positive in aft direction
tc - distance from origin of body axis system to engine swivel point
(P - distance from origin of body axis system to accelerometer
iR = distance from c.g. of rocket engine to engine swivel point
l Pi = distance from origin of body axis system to ith slosh pendulum
hinge point
la. ~ distance from origin of body axis system to center of pressure
L Pi = Length of i411 slosh pendulum
La = aerodynamic load
m0 = reduced mass of vehicle = mT — X mp,-
i

mr = total mass of vehicle

mpi = mass of ith slosh pendulum
mB = mass of rocket engine
M(<) = generalized mass of ith bending mode
q(i) = generalized coordinate of ith bending mode
s = Laplace operator
t - time
Tc = control thrust
u = bending deflection
U0 = forward velocity of vehicle
w = normal velocity of vehicle
a = angle of attack
oiT - thrust acceleration
7 = flight path angle
178
THEORY OF LINEAR SYSTEMS 179

r, = angle of ith slosh pendulum

8 = rocket engine deflection angle
8C = command signal to rocket engine
9 = attitude angle
dB = error signal
9F = feedback signal
dc = command angle '
Ac IcA/Iyy
At*

Vi — mpiaT/\yy
(0 =
u negative slope of ith bending mode = —
(0 =
<f> normalized mode shape function for ith bending mode
f(i), w(i); a),; fc, fpi) wpi = relative damping factor and
undamped natural frequen¬
cy for: ith bending mode;
rate gyro; engine actuator;
i111slosh pendulum

The following equations describe the motion of a launch vehicle in

the pitch plane, using a conventional autopilot. The underlying assump¬
tions and limitations are discussed in detail in Volume Two.*

(Al)
s20 = S2 + Ac 8 + fjact — y. viT i
XT /
(A2)
(s2 + 2r(i)co<i>s + [w(i,]2)q(t> = —— (m*£*s2 + Tc)5
M(l)

(s3 + 2£cwcs2 + a>c2s + Kca>c2)5 = K c c

28,
c
(A3)

eF = e + 2>pJ0q<o +
Kffico«2s
[0 + (A4)
(s2 2£rCi)RS-f- A)ft2)

Note that this formulation provides for the possibility that the rate
and position gyros are not placed at the same location along the vehicle.

(A5)
5c= Kx(s+ K-^G,(s)04

*A. Greensite, Analysis and Design of Space Vehicle Flight Control Systems, 1970 .
180 ELEMENTS OF MODERN CONTROL THEORY

9e — dc dp (A6)

m0(w — Uo0) = Tc5 — Laa + E rnp,Thar (A7)

The elastic displacement is given by

u E q(i)(t (A9)

while

— = - E q(1)(t)«r<*>(*) (A 10)
dl i

where

(All)

The vehicle geometry is depicted in Fig. 1.A1, and the sign convention
for bending parameters is shown in Fig. 1.A2. Figure 1 .A3 is a block
diagram of the pitch plane autopilot; it is assumed here that only the
lowest bending mode is significant.
The complete set of equations, (A1)-(A8), is too formidable for
pencil-and-paper analysis. By introducing appropriate simplifications,
the various essential features of the control system are placed in evidence.
Ultimately, a simulation of the complete system via computer serves
merely to refine the results obtained with the approximate analysis.
We assume first that perturbations in flight path angle are negligible.
This enables us to write

Furthermore, it is assumed that the rate and position gyros are located
sufficiently close together that

(*) (*)
(A12)
 THEORY OF LINEAR SYSTEMS
181

If sloshing is neglected and only the lowest bending mode is con¬

sidered, we find

6F A-i) (s2 + 2{nuRs + c^)(s2 + CiS + c0) (s2 + 1P2)

5 (s2 - MJ(s2 + 2f*a,*s + w*2)(s2 + 2f«V1>s + [co^]2)
(A13)

where

^(1>TC (A 14)
Ai —
182 ELEMENTS OF MODERN CONTROL THEORY

Figure 1.A2. Sign Convention for Bending Parameters.

[co(1)]2+ fiaAi (A 15)

Co =
1 Ai

2f<1)co(1) (A 16)
Cl
1 - A!

Te (A 17)
r =
nii

f/t — (A 18)
THEORY OF LINEAR SYSTEMS 183

+
184 ELEMENTS OF MODERN CONTROL THEORY

The quantity (s2 + 2U + w*2) may be factored as:

s2 + 2^co,tS + toK2= «s2(Kis + l)(r*s + 1)

where

k; = MU - V (UY - i)]-1

TR = Wr(U + V (fffi)2 — 1)]_1.

In the range of values usually considered, (uR > 120), tr is negligibly

small, while

- V (U)2- 1 = tR

Consequently

2U 2U
K* + K* K,
OiR COft

Therefore

(A 19)
S2 + 2fflWSS + Ur2 — KsO)fl2

From Eqs. (A3) and (A5) we have

_ _ KaKcojc2(s + Kj)Gf(s) (A20)

Be ~ s(s3 + 2fc0)cs2+ coc2s+ IW)'

Combining Eqs. (A 13), (A 19), and (A20) yields

dF E/t KcKr [ic N(s) (A21)

E V D(s)
THEORY OF LINEAR SYSTEMS 185

where

D(s) = s(s2 — Ma)(s3 + 2fcwcs2 + 0)C2S+ Kcwc2)(s2 + 2^r(x>rS

+ o)fl2)(s2+ (1,w(1)s+ [w'1’]2).

/

The open-loop transfer function of Eq. (A21) is in a form that permits

the use of either root locus or frequency-response methods for stability
analysis. This transfer function may be further simplified in various ways.
For example, uR, \p, and u>c are generally quite large compared to the
dominant or control frequencies. Therefore, a transfer function valid in
the low frequency range is given by

- = [KxKcK*Mc(1 - AO]
s(s2 - mJ(s + Kc) (s2 + 2r(1)«(^s + [co^>]2)
(A22)

where it has been assumed that GP(s) = 1.

The influence of bending is negligible if wa) is large of if o-o(1) 0. In
this case

- = KaKcKrhc
S(S2 — Aa)(s + Kc)

With the inclusion of a simple lag filter in the loop, this becomes

(A23)

If it is further assumed that

(s + KO
1
s
186 ELEMENTS OF MODERN CONTROL THEORY

(in other words, Kz is very nearly zero), then

0F
Y^aKcKruA s + i)
(A24)
dE

t(s2 — Ma)(s + K j(s+;)

The simplest possible form of the open-loop transfer function is
obtained by assuming that Kc and 1/r are large compared to the domin¬
ant time constants of the system; viz.

dF (A25)
6b

This last form is useful mainly in determining crude order-of-magnitude

estimates for the open-loop gains.
Note that in all cases, for any one of Eqs. (A21)-(A25), the character¬
istic equation of the system is given by

dF (A26)
1 + -L = 0.
6e
 APPENDIX B

RELATIVE DAMPING FACTOR IN TERMS

OF TCHEBYCHEV POLYNOMIALS

Let a point in the complex plane be described by

s = «[-r + jVi - r2]- (Bi)

This has the usual meaning where « is the undamped natural frequency
and r is the relative damping factor. We form successive powers of s as
follows.
s2 = w2[(2f2 - i) - 2jrvr=T2]

s3 = u3[(-4r3 + 3D + j(4D - i)Vi - DJ

s4 = co4[(8f4 - 8f* + 1) + j(-8f* + 4D-S/1 - f2]

It is now observed that the sequence

-r

(2D - 1)

(-4D + 3D

(8f4 - 8f2 + 1)

is precisely T, (—D, T2 (-D, T3 (—D, T4 (-D, where T* ( ) is the

Tchebychev polynomial of the first kind of order k.
Furthermore, the sequence
1

-2f

(4D - 1)

(-8D + 4D
187
188 ELEMENTS OF MODERN CONTROL THEORY

is simply Ui (—f), U2 ( —f), U3 (— f), U4 (—f), where U* ( ) is the

Tchebychev polynomial of the second kind of order k.
This suggests the representation

s* = «*[T*(-r) + jvT=-?u*(-r)]. (B2)

Making use of the standard relations

Tt(-f) = (-1)*T*(T)

u* (—r) = (-i)*u*(r)

we write this as

s* = (-l)*«*[T*(r) - jvT=“f5U*(r)] (B3)

which is Eq. (32) of Sect. 1.1.2. We proceed to prove this by induction.

Multiplying Eq. (B3) by Eq. (Bl) yields

s*+i = - (i - r2)u,(r)]

-jVr=T2[T,(f) + ru*(r)]}. (B4)

Making use of the following relations for the Tchebychev polynomials

T*+2(r) = 2fT«:+i(f) — Ti(f) (B5)

U*+2(f) = 2fU*+i(r) - U*(f) (B6)

Two-) = ru*+i(r) - u4(r)- (B7>

Equation (B4) reduces to

= (-i)w«w[Ta) - ivr^u4+1(r)]

Q.E.D.
Consequently, if Eq. (B3) is substituted into the polynomial

f(s) = J2 a*s4
k= 0
THEORY OF LINEAR SYSTEMS 189

the latter becomes

f(s) = Z (-l)*a*co*[T»(f) - jVl - r2ut(f)].

k= 0

This result was apparently first obtained by Siljak,(15> though in a

somewhat more cumbersome manner than that given here.
 APPENDIX C

A NOTE ON LAPLACE TRANSLORMS

Most pedagogic expositions of the Laplace transform begin with the

definition

(Cl)
0

following which the elegant means of solving linear time invariant differ¬
ential equations via Eq. (Cl) are demonstrated. However, since no
motivation is given for choosing the peculiar form of (Cl) to begin
with, the procedure has the air of some esoteric ritual being conducted
by a sorcerer.
We will attempt, therefore, in what follows, to show that the function
defined by Eq. (Cl) arises quite naturally when one seeks to solve a
linear differential equation in a somewhat unconventional fashion. Our
line of departure is, of course, the original route taken by Laplace
almost 200 years ago—a route that has been almost completely sub¬
merged by the mathematical refinements developed subsequently.
The equation to be solved is

(C2)
y, a,D*x = f(t)

where the a < are known constants and D ! is the linear operator defined by

(C3)

i = 1,2, ••• n

and the initial conditions are given as

190
THEORY OF LINEAR SYSTEMS
191

X0 = x(0)

dx
]

(C4)

We may also write Eq. (C2) as

*>(D)x = f(t) (C5)

where

<p(D) = (anD" + a7l_iD',-‘ + + aiD1 -f- ao). (C6)

It is known from the elementary theory of differential equations that

the solution of Eq. (C2) involves terms of the form e_s(. As a first step,
therefore, let us investigate the properties of the equation that results
from multiplying (C2) by e_,‘ and integrating from zero to infinity, viz.

(C7)

Here we assume that s is a positive real constant. In the more general

theory, s is allowed to be complex, but for present purposes this is
neither necessary nor desirable. With s taken as real, all the usual results
are derived with a minimum of effort; at the same time, some of the
mathematical subtleties connected with complex s are bypassed.
Analyzing the terms on the left-hand side of Eq. (C7) sequentially, we
have, to begin with
 ELEMENTS OF MODERN CONTROL THEORY
192

via integration by parts. If we assume that

lim [e~s‘x] = 0 (C8)

/— &fco

then the above reduces to

/•“ dx f" (C9)

/ e~s<— dt = —x0 + s / e_8!xdt.
J0 dt J0

Similarly

r d2x , dx r dx ,
/ e~st — dt = + s / e~s<— dt
dt2 dt. L dt

Assuming that

dx (CIO)
lim = 0
/% —&fCo dt.

and using Eq. (C9), this reduces to

d2x f°° (Cl 1)

/ e~si— r dt = — (xi + sx0) + s2 / e~stxdt.
Jo dt Jo

In similar fashion, we find that, in general

foo drx
/ e_s( ——dt = — (sr_1x0 + s^xi + •
Jn dtr

(Cl 2)
+ SXr_2 + Xr_; /oo e- ‘xdt

under the assumption that

d’x
lim = 0
dt’

i = 0, 1, 2, %• • , n- 1. (Cl 3)

For simplicity, assume that all the initial conditions are zero; i.e.

x,- = 0 for i = 0, 1, 2, • %• , n — 1.
THEORY OF LINEAR SYSTEMS 193

Then, in place of Eq. (C5), we have

(Cl 4)
/ooo
/•oo
e_s‘xdt = / e~s(f(t)dt.
^0

Now by using definition (Cl), we may write this as

F(s) (Cl 5)
X(s) =
*p(s)

with the result that

'F(s) ^ (C16)
x(t) = £ HX(s)] £_1
-*P(
s)-
Consequently, the solution of the given equation is expressed in terms
of the inverse form of Eq. (Cl), requiring only that one tabulate a suit¬
able set of paired functions, g(t) <->G(s), in order to write the solution
virtually by inspection.
For example, if Eq. (Cl 5) is written as

F(s) _ A,
X(s)
(p(s) i=1 s — s,

then it is sufficient to note that

/“ 0
e_"e5»'dt A
= -— -—
S — Si

from which the solution may be written directly.

 APPENDIX D

RULES FOR CONSTRUCTING THE ROOT LOCUS

The root locus method was first devised by W. R. Evans in 1948.

Since that time, it has become a very useful method of analysis and
synthesis of control systems.
The root locus is a plot of the roots of the characteristic equation as a
function of gain in the complex s plane. The poles of the transient
response mode C(s)/R(s) are related to the zeros and poles of the open-
loop transfer function B(s)/E(s) and the gain K. The relationship is
shown below, where G(s) and H(s) are expressed as the ratio of two
polynomial equations.
Ni(s) (Dl)
G(s) =
Di(s)

N,(s) (D2)
H(s) =
D2(b)

B(s) _ nxn2 (D3)

G(s)H(s) =
E(s) DiD2

C(s) _ G(s) Nj/Di (D4)

R(s) 1 + G(s)H(s) 1 NiN2
DiD-2

The roots of Ni N2 = 0 in Eq. (D3) are called the zeros of the open-
loop transfer function G(s) H(s) and the roots of D,D2 = 0 are called
the poles of the open-loop transfer function G(s) H(s). Similarly, the
roots of 1 + NiN2/DiD2 = 0 are called the poles of the closed-loop
transfer function C(s)/R(s) and determine the operating root locations
corresponding to the system time constants. The numerator Ni/Di of
the control ratio C(s)/R(s) merely modifies the coefficients of the tran¬
sient components.
The characteristic equation of the system is from Eq. (D4)

1 + G(s)H(s) = 0 (D5)

194
THEORY OF LINEAR SYSTEMS 195

or

G(s)H(s) = - 1. (D6)

The location of the roots of the characteristic equation vary as the

gain K is varied. The plot of these roots as a function of the gain K,
where K varies from zero to infinity, is called the root locus.
To establish two important criteria that are useful in plotting the root
locus, it is convenient to express the transfer function Ci(s) H(s) as the
product of linear factors in the following form.

K(s - ZQ(8 - Z2)(s - Z,)---(s - Z„)

G(s)H(s) (D7)
S*(S - Pi)(s - P,)(S — P3)' • -(8 — Pn)’
In the factored form shown above, the Z’s denote zeros and P’s denote
poles, and pairs of these quantities may be complex conjugates. The
quantity K is the gain factor of G(s) H(s). For most physical systems,
the order of S in the denominator is n + N, and is greater than or equal
to the order of the numerator m.
Each linear factor of Eq. (D7) can be represented by a magnitude and
phase angle and is therefore called a phasor. The s-plane representation
of the phasors, shown in Fig. 1.D1, is seen to originate on the poles and
zeros, and terminates at the point s.

K(s - Zi)(s - Z2)

G(s)H(s)
sN(s — Pi)(s — P2)(s — P3)
196 ELEMENTS OF MODERN CONTROL THEORY

The magnitude of a particular phasor, say (s — Pi), where s = —a + jw

and Pi = —ctPi + jwPl, will be

[pi = |s — Pi] — + \/ (—a -p a Pi)2+ (a) — ojP, (D8)

and the corresponding phase angle will be

</>/>,=/s ~ Pi = tan i (V—wa~+ “pi \

aPj/
(D9)

The linear factors in rectangular form can then be related to the polar
form as follows.
i<f>*i
s — Zi — rZ|e

s —Z2= rZ!e7^z2

s» = = 1>0Ne’N<t>Po (DIO)

s - Pj = rPley0pi
s - P2 = rP,e

Upon substituting Eq. (DIO) into Eq. (D7), the entire G(s) H(s)
transfer function can be written in polar form.

K(rZle^zi)(r22e^22)...(rZme^)
J*Z„
G(s)H (s) =
(Dll)
(rp^e,w0po)
'po (rPle^pi). . .(rP ey^p»)

_ KrZlrz. • %•i'z i (4>zl+. . .+4>Zm)

- (.N<t>p0+<f>p1+
. . ,+4>pn)
rp0 rPi"'fp„

= Kre^

where

%-rZm
r ( D 1 2)
r n r r
rPo rPl • • %rPn
THEORY OF LINEAR SYSTEMS 197

and

'j/ = (0Z, + • • • + <t>Zrn)— (N <t>P,+ 4>p1+ • %• + 4>Pn)- (D13)

The phase angles, (</>),are measured positive counterclockwise from a

line parallel to the cr-axis.
/

From Eqs. (D6) and (Dll) then

G(s)H(s) = Kre*> = - 1 (D14)

where

ei<f>= cos ip + j sin \p.

The magnitude of G(s) Ii(s) is

|G(s)H(s)| = |Kr (cos \p + j sin \p)\ = |Kr| = 1. (D15)

The gain factor, K, may be positive or negative. For K > 0

G(s)H(s) = Kr (cos i/' + j sin \p) = —1 (D16)

= cos = — 1

since Kr = 1 and j = 0 for G(s) H(s) to be a real number. From Eq. (D16)
the phase angle of G(s)H(s) becomes odd multiples of r

\p = ± 7T, ± 37T, ± 5x, • • ’

(D17)
= (2k T 1) 7r, k = 0 ± 1, =b 2, • • • .

ForK < 0

G(s)H(s) = — Kr (cos \p + j sin \p) = —1

(D18)
= — cos ^ = — 1

and the phase angle becomes even multiples of w

\p = 0, ± 2t, ± 47r, • • •
(D19)
= 2k7r, k = 0, ± 1, ± 2, • • •.
198 ELEMENTS OF MODERN CONTROL THEORY

In summary, the two criteria that are required in plotting the root
loci of a system are the phase criterion and magnitude criterion.

Phase Criterion:

H 4>zt— (N<£p0+ H 4>Pi)= (2k + 1) 7r, K > 0

-i (D20)
= 2k7r, K < 0

where

k — 0, zb 1, zb 2, • • • .

Magnitude Criterion:

K II rzj
i—1
- = 1 (D21)
n

rPon rPi
i—l

where II denotes the product of the r’s.

The root locus is plotted by finding all points in the s plane which
satisfy the phase criterion. After the locus is completely plotted, the
magnitude criterion is used to scale it in terms of the values of gain K
that correspond to particular roots along the locus. The following
procedure is helpful in constructing the root locus.

1. Obtain the open-loop transfer function and put it in the form

K(s — Zi)(s — Z,)- %• (s — Zm) (D22)

G(s)H(s) =
s"(s - P0(s - P2)---(s - P„)

and plot the zeros Z, and poles P, in the s plane, using the
same scale for both the real axis and imaginary axis. Continu¬
ous curves start at each pole of G(s)H(s) for K = 0 and termi¬
nate on the zeros of G(s)H(s) for I\ = oo.

2. Draw the loci along those sections of the real axis which are to
the left of an odd number of real poles and zeros when K is
positive. When K is negative, the locus lies to the left of an
THEORY OF LINEAR SYSTEMS 199

even number of real poles and zeros. This rule is a result of

applying the phase angle criterion to a point s as it is moved
along the a axis. The net phase angle contribution of phasors
originating from complex conjugate poles and zeros to the
point s on the a axis is zero and may be neglected. For the
particular pole zero configuration shown in Fig. 1.D2, the real
axis portion of the root locus Is shown by application of this
rule.

3. The real axis intercepts of the asymptotes are located at the

centroid of the poles and zeros or

N-\-n m

Ip.-Ez.
i=l i=J

centeroid = ac — — -- (D23)
(N + n) - (m)

The number of asymptotes can be found by inspection of

G(s)H(s) and is equal to the number of poles minus the number
of zeros or

number of asymptotes = (N + n) — (m). (D24)

The angle at which the asymptotes intercept the a axis is

found from
, (2K + 1)7r (D25)
9 asymptote = — —- r- 7 t'
(N + n) - (m)
200 ELEMENTS OF MODERN CONTROL THEORY

For example, let

G(s)H(s) = - K(S ~ ^ - .
s2(s + l)(s + 2 - j2)(s + 2 + j2)

Then, the number of asymptotes = (2 + 3) — (1) = 4.

C—1 — 2 + j2 — 2 — j2) — (+1) _ ,/0

^asymptotes = ± —, ± — k = 0, ±1, — 2.
4 4

4. The breakaway point between two poles or zeros located on

the cr axis is shown in Fig. 1.D4. The loci originate at the
poles for K = 0 and coalesce at the breakaway point <rb and
enter the complex region. For two zeros located on the a

K(s - 1)
G(s)H(s) =
THEORY OF LINEAR SYSTEMS 201

+jw +Jw

—X a o 1 o
K=0 K=0
K =0O| K - »

Figure 1.D4. Breakaway Points.

axis, the locus enters the real axis at the breakaway point and
terminates on the zero at I( = «. In both cases the loci are
perpendicular to the real axis at the breakaway point. The
breakaway point can be found by assuming values of a,, be¬
tween the poles or zeros until the following equality is satisfied.

T - - _ + T r_ 1_ 7 = T t_ -_ T + £ _ i _ T
cri, + Pf ah + Zr,-1 \0h + ZfJ |cr6 —
(- Pr,|
(D26)

where

Pf = ith pole to the left of trial point

Pr, = ith pole to the right of trial point
Zf = i,h zero to the left of trial point
Zr, = ith zero to the right of trial point.

5. The angle of departure at which the locus leaves a complex

pole or enters a complex zero can be determined by adding up
all the phase angle contributions to the pole or zero in question.
Subtracting this sum from 180° gives the required direction.
For example, the angle of departure for the locus leaving pole
P2 in Fig. 1 .D5 is found from

4>z, — y4>p„+ — 180

or

4>P., = ISO 0 -p (4>p0+ 4>P\ T <i>ps) ‘t’zv

202 ELEMENTS OF MODERN CONTROL THEORY

6. The preceding five steps provide enough information to con¬

struct only a portion of the root locus and to determine the
manner in which the root locus will behave. The remaining
branches of the root locus located off the real axis can be
found by starting at a breakaway point and successively select¬
ing trial points immediately above the breakaway point until
the phase angle criterion is satisfied, or starting at a pole
located off the real axis and selecting trial points along the
tangent defined by the angle of departure until the phase
angle criterion is satisfied. Once the phase angle criterion for
this point is satisfied, a new trial point is chosen and adjusted
until the locus terminates on a finite zero, becomes tangent
to an asymptote, or enters a short distance into the right-half
s plane. Only the upper portion of the root locus need be
determined since it is symmetric with respect to the real axis.
The lower portion of the root locus can be constructed by
symmetry. Generally, the important part of the root locus lies
in finite portion of the s plane around the origin.
The use of a spirule greatly facilitates the search of the
roots satisfying the phase angle criterion. It is essentially a
device for measuring the net phase angle of the phasors drawn
from poles or zeros to any point s.

7. Once the locus is completely drawn, it is calibrated in terms of

the gain K using the magnitude criterion of Eq. (D21).

To show the relationship between the root locus and the correspond¬
ing time solution, consider the following second order differential equa¬
tion relating the attitude, <p, to the commanded attitude,

ip T- 2£ci)ntp
+ u„2tp= w„2<pc (D27)
THEORY OF LINEAR SYSTEMS
203

a) Underdamped - stable, £ < 1

b) Critically damped - stable, £ = 1

c) Overdamped - stable, £ > 1

Figure 1.D6. Relationship between Roots in the s Plane and the Corresponding Transient
Solution in the Time Domain.
204 ELEMENTS OF MODERN CONTROL THEORY

The transfer function of this second order system is

<p{
s) = _ u„2 (D28)
<pc{
s) s2 + 2fw„s + w„2

and the corresponding transient response to a step input on <pcis

*(t) = Cie*i‘ + C2e*2‘ + 1 (D29)

where the roots obtained from the characteristic equation are

Si,2 = — ± jco„\/ f2 — 1 = — a ± jo>d, 0 < f <1

Si ,2 = — C0n, f = 1

Si ,2 = — f<0n ± (j)n 2 — 1, f > 1

si, 2 — ±jw?i, r — o.

A plot of the roots sh s2 in the s plane and the corresponding transient

solution of <pfor a step input of <pcis shown in Fig. 1 .D4.

When the roots Si, S2 are located to the left of the j« axis, the system
is stable, and the transient part of the response decays to zero. When
the roots are located on the jw axis, the system contains no damping, and
sustained oscillations result. As the roots move to the right of the jw
axis, the amplitude of the oscillations increases with time, and the
system is unstable. The above statements are true regardless of the
order of the system. In the complex s plane, horizontal lines represent
constant damped frequency, vertical lines represent a constant rate
of decay, <r; radial lines through the origin represent constant damping
ratio, and circles concentric about the origin represent constant
natural frequency, w„.

REFERENCES

1. J. G. Truxall, Automatic Feedback Control System Synthesis, New

York, McGraw-Hill Book Company, 1955.
2. C. J. Savant, Basic Feedback Control System Design, New York,
McGraw-Hill Book Company, 1958.
THEORY OF LINEAR SYSTEMS 205

3. M. F. Gardner and J. L. Barnes, Transients in Linear Systems, New

York, John Wiley & Sons, Inc., 1942.
4. A. F. Schmitt and others, Approximate Transfer Functions for
Flexible Booster and Autopilot Analysis, WADD TR-61-93, April
1961.
5. C. D. Johnson and W. M. Wonham, “Another Note on the Trans¬
formation to Canonical (Phase Variable) Form,” IEEE Trans, on
Automatic Control, July 1966.
6. W. C. Schultz and V. C. Rideout, “Control System Performance
Measures: Past, Present, and Future,” IRE Trans, on Automatic
Control, February 1961, p. 22.
7. E. J. Routh, Stability of a Given State of Motion, London, Mac¬
Millan & Co. Ltd., 1877.
8. A. Hurwitz, “Concerning the Conditions Under Which an Equation
has only Roots with Negative Real Parts,” Math. Ann., Vol. 46,
1895, p. 273 (in German).
9. M. Marden, “The Geometry of the Zeros,” Am. Math. Soc., 1949.
10. T. Usher, “A New Application of the Routh-Hurwitz Stability
Criteria,” A IEE Trans., Vol. 72, Part II, 1953, p. 291.
11. H. Nyquist, “Regeneration Theory,” Bell System Tech. Journ., Vol.
II, No. 126, 1932.
12. H. W. Bode, Network Analysis and Feedback Amplifier Design, New
York, D. Van Nostrand Company, Inc., 1945.
13. W. R. Evans, “Graphical Analysis of Control Systems,” Trans. AIEE,
Vol. 67, Part 1, 1948, p. 547.
14. H. M. James, N. B. Nichols and R. S. Phillips, Theory of Servo¬
mechanisms, New York, McGraw-Hill Book Company, 1947.
15. D. D. Siljak, “Generalization of Mitrovic’s Method,” IEEE Trans, on
Appl. & Ind., Vol. 83, 1964, p. 314.
16. W. C. Wojcik, “Analytical Representation of the Root Locus,”
ASME Trans., J. Basic Eng., March, 1964, p. 37.
17. K. Steiglitz, “An Analytical Approach to Root Loci,” IRE Trans, on
Automatic Control, September 1961, p. 326.
18. J. T. Tou, Digital and Sampled Data Control Systems, New York,
McGraw-Hill Book Company, 1959.
19. E. I. Jury, Sampled Data Control Systems,- New York, John Wiley &
Sons, Inc., 1958.
20. A. Greensite, Analysis of Atlas/Centaur Autopilot With Guidance
Feedback, General Dynamics Convair Report AE69-0365, May 1,
1960.
 206 ELEMENTS OF MODERN CONTROL THEORY

21 . K. C. Bonine, Flight Dynamics and Control Analysis of the Centaur

Vehicle, General Dynamics Convair Report GD/A-DDE65-004,
January 1965.
22. R. F. Ringland, New Digital Routine for Determination of Z Trans¬
forms, General Dynamics Convair Memorandum IGM6 1-004, March
10, 1961.
23. J. M. Salzer, “The Frequency Analysis of Digital Computers Operat¬
ing in Real Time,” Proc. IRE, Vol. 42, No. 2, p. 457.
24. A. R. Bergen and J. R. Ragazzini, “Sampled Data Processing Tech¬
niques for Feedback Control Systems,” Trans. AIEE, Vol. 73,
Part II, p. 236.
' 25. L. A. Zadeh and C. A. Desoer, Linear System Theory, New York,
McGraw-Hill Book Company, 1963.
26. J. T. Tou, “Determination of the Inverse Vandermonde Matrix,”
IEEE Trans, on Automatic Control, July, 1964, p. 314.
27. H. J. Wertz, “On the Numerical Inversion of a Recurrent Problem:
The Vandermonde Matrix,” IEEE Trans, on Automatic Control,
October, 1965, p. 492.
28. C. D. Johnson and W. M. Wonham, “A Note on the Transformation
to Canonical (Phase Variable) Form,” IEEE Trans, on Automatic
Control, July, 1964, p. 312.
29. E. J. Putzer, “Avoiding the Jordan Canonical Form in the Discussion
of Linear Systems with Constant Coefficients, ’Mm. Math. Monthly,
January, 1966, p. 2.
30. R. E. Kalman, “Mathematical Description of Linear Dynamical
Systems,” Journ. SIAM (Control), Ser. A., Vol. 1, No. 2, 1963, p.
152.
31. E. G. Gilbert, “Controllability and Observability in Multivariable
Control Systems,” Journ. SIAM ( Control), Ser. A, Vol. 1, No. 2,
1963, p. 128.
32. E. Kreindler and P. E. Sarachik, “On the Concepts of Controllability
and Observability in Linear Systems,” IEEE Trans, on Automatic
Control, April, 1964, p. 129.
33. R. W. Brockett, “Poles, Zeros, and Feedback: State Space Interpre¬
tation,” IEEE Trans, on Automatic Control, April, 1965, p. 129.
34. H. Freeman, “Stability and Physical Realizability Considerations in
the Synthesis of Multipole Control System,” Trans. AIEE, Vol. 77,
Part II, 1958, p. 1.
35. I. M. Horowitz, “Synthesis of Linear Multivariable Feedback Con¬
trol Systems,” Trans. IRE, Vol. AC-5, No. 2, 1960, p. 94.
THEORY OF LINEAR SYSTEMS 207

36. E. V. Bohn, “Stabilization of Linear Multivariable Feedback Con¬

trol Systems,” Trans. IRE, Vol. AC-5, No. 4, 1960, p. 321.
37. K. Chen, R. A. Mathias and D. M. Sauter, “Design of Noninteracting
Control Systems Using Bode Diagrams,” Trans. AIEE, Vol. 80,
Part II, 1961, p. 336.
38. R. Bellman, Introduction to Matrix Analysis, New York, McGraw-
Hill Book Company, 1960.
39. R. A. Frazer, W. J. Duncan and A. R. Collar , Elementary Matrices,
London, Cambridge University Press, 1938.
40. P. E. Sarachik and E. Kreindler, “Controllability and Observability
of Linear Discrete-time Systems,” Int. Journ. of Control, Vol. I,
No. 5, 1965, p. 419.
41. R. E. Kalman, Y. C. Ho and K. S. Narendra, “Controllability of
Linear Dynamical Systems,” Contributions to Differential Equations,
Vol. I, New York, Interscience Publishers, Inc., 1963.
42. R. E. Kalman, “On the General Theory of Control Systems,” Proc.
Int. Congress on Automatic Control, Moscow, 1960, London, But-
terworth Scientific Publications, Vol. 1, 1961, p. 481.
43. E. I. Jury, “Hidden Oscillations in Sampled Data Control Systems,”
Trans. AIEE, Vol. 75, Part II, 1956, p. 391.
44. J. Preminger and J. Rootenberg, “Some Considerations Relating to
Control Systems Employing the Invariance Principle,” IEEE Trans,
on Automatic Control, July, 1964, p. 209.
45. L. I. Rozenoer, “A Variational Approach to the Problem of In¬
variance of Automatic Control Systems,” Avtomatika i Telemek-
hanika, Vol. 24, No. 6, June, 1963 (in Russian).
46. I. M. Horowitz, “Design of Multiple Loop Feedback Control
Systems,” IRE Trans, on Automatic Control, April, 1962, p. 47.
47. S. J. Merchav, “Compatibility of a Two Degree of Freedom System
with a Set of Independent Specifications,” IRE Trans, on Automatic
Control, January, 1962, p. 67.
48. G. W. Smith and R. C. 'Hood, Principles of Analog Computation,
New York, McGraw-Hill Book Company, 1959.

PROBLEMS AND EXERCISES FOR CHAPTER ONE

1. In each of the following characteristic equations, use Routh’s test

to investigate the presence of roots having positive real parts.

a. s4 + 2s3 + 10s2 + 5s + 1 = 0.
 208 ELEMENTS OF MODERN CONTROL THEORY

b. s6 + 4s6 + 20s3 + 3s2 + 7s + 30 = 0.

c. s6 + 21s4 + 5s3 + 8s2 + 14s + 3 = 0.

d. s4 + 16s3 + 42s2 + 12s + 5 = 0.

e. s6 + 3s4 + 16s3 + 40s2 + 52s + 75 = 0.

2. Determine if any of the following characteristic equations have

roots whose relative damping factor is less than 0.40. Make use
of the generalized Routh test.

a. s4 + 2s3 + 8s2 + 4s + 3 = 0.

b. s6 + 27s4 + 268s3 + 708s2 + 1108s + 409 = 0.

c. s4 + 10s3 + 36s2 + 70s + 75 = 0.

3. In Fig. 1.2, suppose that H(s) = 1 and

G(s)= - \- .
S(riS + 1)(t2S + 1)

What inequality, involving K, n, and r2, must be satisfied in order

that the system be stable?

4. Referring to Fig. 1.2, let H(s) = 1. For each of the following

values of G(s) sketch the Nyquist curve and root locus.

1
a. G(s)
s(s + 4)'

b.
(s + 3)
G(s)
s(s + 5)

c. G(s)
s(s + l)(s + 2)

d. (s + 1)
G(s) =
s2(s2 + 6s + 25)
 THEORY OF LINEAR SYSTEMS 209

e. G(s) = - --
(s + l)(s + 2)

For the cases a, b, and e, determine the gain and phase margins
when K = 10. In cases c, d, and f, determine the range of values of
K for which the system is stable.

5. In Fig. 1.2, suppose thatH(s) = (s + 1) and

G(s) =
s(s2 + 8s + 32)

Find that value of K for which the closed-loop pole has a relative
damping factor of 0.30. By making a sketch of the Nyquist curve,
determine the phase margin for this value of K.

6. Show that the characteristic equation for the system of Problem 3

may be written as

1 + Tl = 0.

Taking T2 — 0.25 and K — 5, sketch the root locus as a function of

ti, and determine the value of n above which the system is unstable.
(Note that there is an open-loop pole at minus infinity.)

7. Show that the portion of the root locus which is off the real axis is
a hyperbola in the case of the unity feedback system for which

KG (a) = - -
s(s + ai)(s + a2)

where ai and a2 are positive constants.

8. In Fig. 1.32, parts a and d, take

210 ELEMENTS OF MODERN CONTROL THEORY

K
G(s)
s(s + 1)

_ 1_
H(s)
(s + 2)

where the sampling period, T - 0.2 sec.

Determine the maximum value of K for which the system is stable

in each of the two configurations.

9. For the sampled data system of Fig. 1.36, let

G(s) =

By means of a Z plane root locus, investigate the stability properties

of the system when the sampling period T is 0.5, 1, and 2 seconds.
Which of the three values of T is best as far as giving the maximum
value for K before instability occurs?

10. In Fig. 1,32a, let

K
G(s)
s(s + 2)

and

H(s) = e~rDs.

Let the sampling period T equal unity. Investigate the stability

properties of the system when Tn = 0.5 and 2.0. Determine which
value of T„ gives superior results from a stability point of view and
substantiate your answer by appropriate Z plane root loci.

1 1. For each of the cases of Problem 4, determine the steady-state error

in response to
(a) a unit step input.
(b) a unit ramp input.
 THEORY OF LINEAR SYSTEMS 211

Consider a unity feedback system for which

KG (s) =
s(s + l)(s + 5)

Determine the value of K which gives a value of 0.5 for the relative
damping factor of the closed-loop pole. Using this value of K,
calculate the steady-state error in response to a unit step inpui.
Suppose this steady-state error must be reduced by a factor of at
least five, without materially altering the features of the closed-loop
response. Show how this can be done by using a lag circuit in
cascade.

13. In the previous problem, suppose that the value of K, which gives
the closed-loop pole a relative damping factor of 0.5, also results in
an unacceptably low value for the undamped natural frequency of
this closed-loop pole. It is required to double this undamped natural
frequency without materially changing the relative damping factor.
Design a suitable compensator for doing this.

14. A unit feedback system has an open-loop transfer function

K
K(Ks) =
s2(s + 5)

It is required to have the closed-loop poles at -0.75 ± j‘2. Choosing

a lead compensator of the form of Eq. (110), choose zt and p/ such
that the ratio z(/pf is a maximum, thereby yielding a minimum
bandwidth.

15. In the sampled data system of Fig. 1.36, suppose that

K
G(s) =
(s + 2)

and the sampling period, T - 1.25 sec. The compensator given by

Eq. (127) may have any of the four pole-zero configurations shown
in Fig. 1.53. Using a Z plane root locus, evaluate qualitatively the
influence of each of the four types on the closed-loop response.
Sketch the appropriate root locus in each case, and use Fig. 1.35 to
aid your evaluation.
212 ELEMENTS OF MODERN CONTROL THEORY

16. Express each of the following systems in matrix form by defining

appropriate state variables, and such that derivatives of the forcing
function do not appear.

a. 0_ -(- 3 0 T 4 0 -f~ 60 = 5 4“ 25

b. 1+ 50 + 20 + 100 = 35 + 45 + 6-

17. Find the state variable matrix representation of the system

a + 2a = 5 -f- 3r/

0 + 40 + 50 = 25 + 4t? + r).

18. Consider the system described by

Xi = x2 + 2u

x2 = — 2xj — 3x2 + u

y = xi + 2u.

By means of a suitable transformation of coordinates, express this

system in normal form. Find the matrix transfer function for the
original system.

19. Calculate the transition matrix for the system of Problem 17, and
find the response to a step input assuming zero initial conditions.

20. Express the system of Example 5 (Sect. 1.2.1) in state variable

matrix form. Calculate the transition matrix and find the response
to a unit step input assuming zero initial conditions. Take K = 2.0.

21. For the discrete time system represented by Eqs. (173) and (174),
suppose that
~-l 0 0" "l 0
-1 -2 0 H = 1 1
-1 0 —3_ 1 0_

0 1 0" "0 (f
D =
-1 0 —3_ _1 4_

and let the sampling period equal unity.

22.
Conside
the
transfe
functi
a.casc
—
=
Gi(
24
In
Fi
1.7
su
th
THEORY OF LINEAR SYSTEMS 213

Using a suitable transformation of coordinates, express the system

in normal form.
What is the sampled data matrix transfer function for this case?

s + 1
Gi(s) = , G2(s)
s + 3 s —
f- 3

Suppose these are connected in

5(s)
b. parallel: = Gj(s) + G2(s).
5(s)

c. feedback: — = - —^ - .
S(s) 1 -f- Gi(s)G2(s)

Discuss the observability and controllability in each case.

23. For the system described by Eqs. (199) and (200) take
0 1 (f --T
A = 5 0 2 B = 1
— 2 0 —2_ _-l_

C = -2 1 0] D = 0.

Investigate the observability and controllability in this case.

1
Gx(s) = G2(s)
s + 1 s + 2

Li(s) = K o = constant.

What must be the form of L2(s) to achieve

a. absolute invariance.
b. steady state invariance.
c. conditional invariance.
214 ELEMENTS OF MODERN CONTROL THEORY

What can be said about the problems of physical realizability in

each case?

25. We seek to achieve an absolute invariance for a given feedback

system by using a disturbance-dependent feedback. Using the
scheme shown in Fig. 1.74 with

H(s) = K2(s)

Gu(s) = - Gi2(s) — 1
s

1
G2i(s) —1 G22(s)
S'

determine the required form of L(s) and discuss physical realiza¬

bility. How are the stability features of the system affected by the
use of the disturbance-dependent feedback loop?
 Chapter Two
THEORY OF NONLINEAR SYSTEMS

In this chapter, we discuss a variety of techniques available for the

analysis of nonlinear control systems. While there is voluminous tech¬
nical literature on this subject, it is, for the most part, of a theoretical
nature beset with serious computational difficulties in application.
It is the primary purpose of this chapter to consider those techniques
that have proven most useful in the analysis of space vehicle control
systems. Well-established techniques such as the phase plane and des¬
cribing functions will be treated from first principles together with
extensive application to problems of flight control. Various refinements
described in the recent literature will be treated.
A substantial portion of the following sections deals with the Lyapunov
theory of stability with emphasis on flight control applications. While
this technique is still under active development, its broad features are
well established, and it has proven useful in providing insight to previously
obscure areas.
It is safe to say that this particular branch of control theory will
never reach the definitive form which characterizes linear control. This
is, of course, due to the fact that a nonlinear system is best defined in a
negative sense; that is, a nonlinear system is merely one that is not linear.
Because of this, we must sharply restrict the scope of the study in order
to yield useful results. The most general theory available is that of
Lyapunov. The analytical elegance of this concept is accompanied by
serious difficulties in application due mainly to the broad generality of
the theory; in short, its chief virtue is also its fundamental handicap.
Much progress has, however, been obtained by delineating specialized
cases which, of course, yield specialized results. The best example in this
area is the Lur’e technique. There is no doubt that further results of
theoretical and practical interest will be forthcoming in the future.
Of the specialized methods presently available for pragmatic design,
the most prominent are the phase plane and describing functions. Recent
215
216 ELEMENTS OF MODERN CONTROL THEORY

developments in the former have been concerned with improving the

methods for constructing the phase plane portraits and extending their
application. The generality of the describing function technique has been
extended by the introduction of the “Dual Input Describing Function.”
The fundamental restriction in the phase plane method is the limitation
to second order systems, while the use of describing functions involves
the assumption of low band pass properties of the system being analyzed.
However, the formulation of any mathematical model involves some
compromise with reality. Ultimately, the experience and judgment of the
control engineer provide the best guide for the proper application of
these methods together with their proper interpretation.
The principal value in the proper application of nonlinear techniques
is the generation of qualitative features of a control system, which are
essentially unaffected by inclusion of higher order dynamic effects and
thereby permit a rational interpretation of results obtained by computer
simulation of a complex system.

2. 1 PHASE PLANE ANALYSIS

The phase plane method is concerned with determining the general

features of the solution of the differential equation

X + f(x, x)x + g(x, x) = 0. (1)

The phase plane is a plot of x versus x and represents the trajectory of

the system for any given set of initial conditions, x (o) and x (o). By
classifying and studying the general form of these trajectories, one may
draw conclusions on the system behavior without having to solve the
equation for every set of initial conditions. While these methods are
both elegant and powerful, their application to the analysis of nonlinear
flight control problems is severely limited by three restrictions:

1. The phase plane is applicable only to second order systems.

2. Only systems that are not externally excited may be studied.
3. The range of admissable nonlinearities is limited.

Each of these limitations can to some extent be circumvented. Ex¬

tensions of the method to a phase space of order higher than two have
been developed by Ku/U Grayson and Mishkin/2) and Kuba and
Kazda/2) among others. There is, however, a substantial increase in
THEORY OF NONLINEAR SYSTEMS 217

complexity both at the computational level and in physical insight. As

a result, these methods have not been used extensively in practical
problems.
The second limitation may be partially removed to the extent that
step and ramp inputs may be treated. To see this, let Eq. (1) be written
as
/

X + f(x, x)x + g(x, x) = a + bt (2)

where a and b are constants and t > 0.

Making the change of variable, z = x —a —bt, Eq. (2) becomes

z + f(z + b, z + a + bt)(z + b) + z = 0. (3)

This equation is autonomous* if either b = 0 (step function alone) or

f(x, x) S f(x).
The third limitation is inherent in the form of the differential equation,
(1); namely, only signal-dependent nonlinearities are allowed. The co¬
efficients of x and x may not contain the independent variable explicitly.
In spite of the rather limited range of systems thus amenable to
analysis, the phase plane method is valuable in that it yields qualitative
results for systems of higher order and is exact for those cases that are
adequately described by second order equations. In the latter case, a
wealth of insight into phenomena which cannot be predicted even ap¬
proximately by the linear theory is obtained, and this often provides a
framework for predicting and explaining in qualitative fashion those
control systems in which are included higher order dynamic effects.
The phase plane method is based on three essential concepts:

1. The phase plane representation.

2. Singular points.
3. Limit cycles.

These will be discussed in the following sections.

2.1.1 Phase Portrait

By defining a new variable, y, Eq. (1) may be written as

i.e., t does not appear explicitly.

218 ELEMENTS OF MODERN CONTROL THEORY

X = y

y = -f(x, y) y - g(x, y). (4)

Actually, the method to be discussed is applicable to the slightly

more general system described by

x = P(x, y)
(5)
y = Q(x, y).

It is easy to eliminate t completely from this set of equations; viz.

dy = Q(x, y)
dx P(x, y)

A variety of specialized techniques has been developed for determining,

graphically, the phase plane trajectories of systems represented by Eq.
(1) or Eq. (6). They differ mainly in the specific form of the equation
treated. Since these methods are dealt with at length in standard texts, (4)
only two of these will be described briefly.

2. 1.1.1 Graphical Methods

A variety of graphical methods is available to solve Eq. (6), depending

on the specific form of this equation. Perhaps the simplest technique is
the so-called “Isocline Method,” which is applicable if P(x, y) = y in
Eq. (6). This method is perhaps best explained by a specific example.
Consider the equation*

x — 0.2 (1 — x2) x + x = 0. (7)

This may be written in the form

dy = 0.2(1 - x2)y - X (g)

dx y

where y = x. A series of lines are drawn for different constant values

of dy/dx = m. These are shown in Fig. 2.1 for m = 0, ±0.5, ±1. Now

*The knowledgeable reader will recognize this as the van der Pol equation, the workhorse of
nonlinear theory.
THEORY OF NONLINEAR SYSTEMS 219

m = 0

Figure 2.1. Graphical Construction of Phase Plane Trajectory Using the Isocline
Method.

let A be a point on the solution curve. (This may represent the initial
state x [o], y [o].) The slope at point A is equal to 1 since it lies on the
curve for m = 1. Point B on the system trajectory may be determined as
follows. We note first of all that the trajectory proceeds in a clockwise
direction, as may be readily verified by considering the relation between
x and y. Now since B is on the m = 0.5 curve, its slope equals 0.5, and
the average slope between A and B is, therefore, 0.75. Drawing a line of
slope = 0.75 through point A consequently determines point B. Pro¬
ceeding in this fashion, the entire trajectory is formed as shown in Fig.
2.1.
The accuracy of the method can be made as high as desired by using a
large number of isoclines (lines of constant slope).
Another method, which is somewhat more direct, proceeds by con¬
structing the system trajectory by means of small circular arcs. This
approach, known as the “Delta Method,” is applicable to equations of
220 ELEMENTS OF MODERN CONTROL THEORY

the form

x + V (x, x) = 0. (9)

By properly defining a variable, 8 , we may write this as

x + 8 + x = 0. (10)

Again using y = x, we may write Eq. (10) as

dy _ x + 3
(ID
dx y

where

5 = <p(x,y) - x. (12)

For small changes in the variables x and y, 5 remains essentially con¬

stant and, in this case, Eq. (11) may be integrated to yield

y2 + (x + 5)2 = R2 (13)

where R2 is a constant. This is the equation of a circle of radius R whose

center is at x = — 8, y = 0. Thus, for a suitably small increment, the
system trajectory is a circular arc of the form shown by Eq. (13). The
construction proceeds as follows. The initial point is located atx(o), y(o),
as shown in Fig. 2.2. These values of x and y determine the value of 8,

Figure 2.2. Basic Construction of the Delta Method.

THEORY OF NONLINEAR SYSTEMS 221

which thereby locates the center of the circular arc and determines the
radius R automatically. A short circular arc is then drawn to locate the
new point, xj and yx. The procedure is then repeated until the entire
trajectory is formed. Figure 2.3 illustrates the trajectory obtained for the
system

x + 25(1 + 0. fx2)x = 0

with the initial conditions

x = —1.8

x = y = —1.6.

A typical circular arc construction is shown for the point x = 3,

y r' 0, for which the value of <5equals 2.7.

Remark: Graphical methods, by themselves, are of limited usefulness in

establishing the essential properties of the nonlinear equation
as represented by Eq. (5) or Eq. (6). Such questions as
stability, boundedness of solutions, etc. are determined in
rigorous and elegant fashion by invoking the concepts of singu¬
lar points and limit cycles, which are discussed in Sects. 2.1.2

Figure 2.3. Complete Trajectory Constructed

Using the Delta Method.
222 ELEMENTS OF MODERN CONTROL THEORY

and 2.1.3. A powerful technique for piecewise linear systems

is treated in Sect. 2.1.4. As a preliminary to this, the phase
plane solutions for second order linear systems are developed
in the next section.

2. 1 . 1 .2 Direct Solution of Equations

In certain cases, the trajectory of the system in the phase plane may
be obtained by integrating Eq. (6) directly. For example, consider the
equation

e + 2 fw0e + 0Jo2e= aj0*ec

ec = A, t > 0 (14)

= 0, t < 0

which represents a linear system with a step input.

Writing

x — e — A

T = W0t

this may be put in the form

x" + 2fx' + x = 0 (15)

where primes denote differentiation with respect to r. Letting y = x',

and dividing through by y, Eq. (15) may be expressed as

dy = _ 2fr + x (16)
dx y

If f = 0, then this integrates immediately to

x2 + y2 = c2 (17)

which is the equation of a circle centered at the origin; the constant c is

determined by the initial conditions.
THEORY OF NONLINEAR SYSTEMS 223

If f = 0 and the sign in front of the x term in Eq. (15) were minus
instead of plus, then Eq. (15) would integrate to

y2 — x2 = c2 (18)

or

x2 y2 — c2 (19)
/

depending on the value of the initial conditions. These curves, which

are rectangular hyperbolas, are shown in Figs. 2.4 and 2.5.
Turning now to the general case, we consider the equations

x = y

y + 2fcoy + w2x = 0 (20)

which may be expressed as

dy = _ (2ftoy + oj2x) (21)
dx y

after eliminating t. Performing a change of variable

y = vx (22)

Eq. (21) becomes

v + 2fo) T" dx = — xdv. (23)

v J

This may now be integrated by separation of variables to yield

y2 + 2fcoxy + w2x2 = (24)

_2f_ / y + fqjx \
<®(x,y) tan"
V\ - f2 \u)X\/l — fv

when |f| < 1.

For the case of |f| > 1 , Eq. (23) integrates to

(y — Aix)*l = C2(y — a2x)x2 (25)

\i = — fw — wV f2 — l

\2 — fa; + w y/ f 2 — 1•
224 ELEMENTS OF MODERN CONTROL THEORY

Typical trajectories corresponding to Eq. (24), the underdamped case,

are shown in Figs. 2.6 and 2.7, for « = 1 and f = 0.3 and 0.5, respectively.
THEORY OF NONLINEAR SYSTEMS 225

Figure 2.7. Phase Portrait of the Damped Second Order System

x 4- 2£x 4- x = 0; £ = 0.5.

From Eq. (21), observe that the slope changes sign aty = - x/2f. This
line is also shown in the figure.
226 ELEMENTS OF MODERN CONTROL THEORY

The trajectories corresponding to Eq. (25), the overdamped case, are

shown in Figs. 2.8 and 2.9, where u = 1 and f = 1.4 and 2.0, respectively.
The phase portrait is characterized by principal directions and the final
portion of any trajectory is along a straight line through the origin. The
principal directions are determined as follows. Suppose that it is possible
to choose a constant, X, such that y/x = X is a portion of the trajectory.
For this to be true, we find from Eq. (21) that the following condition
must be satisfied.
X2 + 2fwX + w2 = 0

This is, in fact, the characteristic equation of the system. In other

words, when X is a root of the system characteristic equation, then there
is a solution in the phase plane given by y = Xx. These are two such
paths, corresponding to X! and X2. These paths are straight lines through
the origin and they define principal directions of the phase portrait. Just
as the time behavior of the overdamped second order system is primarily
determined, initially, by the larger of the two roots, and finally by the
smaller one, so a typical trajectory in the phase plane starts parallel to
the principal direction with the larger slope (say x2) and ends parallel to
the direction with the smaller slope (Xi). These features of the phase
portrait for overdamped systems are evident in the trajectories shown in
Figs. 2.8 and 2.9.
The phase portraits of Eqs. (24) and (25) may be further simplified by
an appropriate change of variable. Consider first Eq. (24). If new
variables are defined by

u = (coVi — b2)x

v = y + fwx

then Eq. (24) may be expressed as

u2 + v2 = C'iev<

With a change to polar coordinates

u = p cos 8

v = p sin 6
THEORY OF NONLINEAR SYSTEMS 227

l\ I < Ix21
Figure 2.8. Phase Portrait of the Damped Second Order System
x 4- 2£x + x = 0; £ = 1.4 .

Ujl < |x2|

Figure 2.9. Phase Portrait of the Damped Second Order System
x + 2ix + x = 0; ? = 2.0 .
228 ELEMENTS OF MODERN CONTROL THEORY

and we obtain

P — C3ee

which is the equation of a logarithmic spiral. Turning to Eq. (25), if one

adopts the coordinate transformation

u = y — X2x

v = y - Xix

it is found that Eq. (25) reduces to

v = C4uXaA‘

which is the equation of a family of parabolic-like curves centered on

the v axis.

Remark: The importance of the results derived in this section lies not
in their neatness and elegance but rather in the fact that they
are basic to the study of the piecewise linear systems considered
in Sect. 2.1.4. What is significant is that the phase portraits
are distinctive, depending upon whether the system is under¬
damped, overdamped, or undamped. These qualitative charac¬
teristics, taken in conjunction with properties related to singular
points and limit cycles, provide one of the most valuable tools
available for predicting the qualitative features of nonlinear
phenomena. These topics are taken up next.

2.1.2 Singular Points

The slope given by Eq. (6) becomes indeterminate when, simulta¬

neously

P(x, y) = 0
(26)
Q(x, y) = o.

The point at which this occurs is termed a “singular point.” An analysis

of the nature of singular points is important from the point of view of
equilibrium states of the system; whether trajectories converge to or
 THEORY OF NONLINEAR SYSTEMS 229

diverge from this state, or in some cases, yielding information on the

existence of closed paths in the phase plane (periodic phenomena).
If a singular point exists at x = a and y = b, then a Taylor expansion of
Eq. (5) about this point yields

x = ai(x — a) + a2(y — b) -f higher order terms

/

y = bi(x — a) + b2(y — b) + higher order terms.

In a sufficiently small region about the singular point, the higher order
terms are negligible and the system behaves linearly; viz.

x = ai(x — a) + a2(y — b) (27a)

y = bi(x - a) + b2(y - b). (27b)

For this case, the characteristic equation of the system is found to be

s1 2 3 4 5 — (ai T" b2)s -f- (aib2 — a2bi) = 0. (28)

The nature of the roots of this equation determines the behavior of

the system trajectories in the phase plane. Denoting these roots by Xx
and X2, six cases may be distinguished.

1. Xi and X2 are both real and negative. This case is termed a stable
node and all trajectories converge to the singular point, as shown
in Fig. 2.10a.
2. Xi and X2 are complex conjugate with negative real part. This
condition gives rise to a stable focus, with all trajectories con¬
verging to the singular point in the manner shown in Fig. 2.10b.
3. Xi and X2 are pure imaginary. For this case the motion is simple
harmonic, with the oscillation amplitude dependent on the initial
conditions. The trajectories are closed paths around the singular
point which in this case is termed a center. The phase plane
portrait is shown in Fig. 2.10c.
4. Xi and X2are both real and positive. This gives rise to an unstable
node with all trajectories diverging from the singular point, as
shown in Fig. 2.10d.
5. Xi and X2 are complex conjugate with positive real part. This case
results in oscillatory motion but with divergent amplitude; all tra¬
jectories leave the singular point, as shown in Fig. 2.10e. The
singular point here is termed an unstable focus.
230 ELEMENTS OF MODERN CONTROL THEORY

b. Stable Focus - The roots are complex e. Unstable Focus - Roots are complex
conjugates with negative conjugates with positive
real parts real parts

c. Center - The roots are complex conju- f. Saddle Point - The roots are both real
gates with zero real parts and of opposite sign

Figure 2.10. Typical Phase Plane Trajectories for Singular Points.

6. Aj and X2 are both real with one positive and one negative. The
singular point here is termed a saddle point. The trajectories are
depicted in Fig. 2.1 Of.

The type of singular point encountered is essentially dependent on the

coefficients in Eq. (28). If we write
THEORY OF NONLINEAR SYSTEMS 231

/u — aib2 — a2bi

v = — (ai -f- b2)

then a concise tabulation of the singular points is as depicted in Fig. 2.1 1,

where the line of demarcation between the nodes and foci is given by
the parabola '

v2 = 4/u.

The actual equations of the phase plane trajectories are obtained by

substituting Eqs. (27) for P(x. y) and Q(x, y) in Eq. (5) and writing the
resulting set in the form of Eq. (6) after eliminating t. Equation (6)
then has a solution in closed form in the manner discussed in Sect.
V

SADDLE
POINT

Figure 2.11. Singular Points as a Function of> and v.

232 ELEMENTS OF MODERN CONTROL THEORY

2. 1.1. 2. Note that while the results obtained there were valid for the
whole phase plane, the case discussed here is limited to the immediate
vicinity of the singular point because of the linearizing process involved
in deriving Eqs. (27).

2.1.3 Limit Cycles

The determination of the nature of the singular points, together with

the use of the graphical methods described previously, is sufficient to in¬
dicate the general nature of the system trajectories. However, for purposes
of stability analysis, a knowledge of the possible existence of limit cycles
is essential. A limit cycle is an isolated closed path in the phase plane.
The limit cycle may be stable or unstable depending on whether the
paths in the neighborhood converge to the limit cycle or diverge away
from it. From a practical point of view, only the former is important.
The two types of limit cycles are illustrated in Fig. 2.12.
Unfortunately, there is no completely general method for determining
the limit cycles of any given system. The available methods are summa¬
rized below in four theorems which are stated without proof.*

*The proofs may be found in Ref. 5.

THEORY OF NONLINEAR SYSTEMS 233

Theorem I: If outside a circle, Ci, in the phase plane all paths are
converging (the radial distance to the point moving along the path is
decreasing, the radial distance being measured to the center of the circle,
Ci), and inside a smaller circle, C2, with the same center as Ci, the
paths are diverging, then a limit cycle must exist within the region
bounded by Ci and C2.
/

Theorem II (Bendixson’s First theorem): WithP(x, y) and Q(x, y)

defined as in Eq. (5), then no limit cycle can exist within any region in

Theorem III (Bendixson’s Second theorem): If a path stays inside a

finite region, D, and does not approach a singular point, then it must
either be a limit cycle or approach a limit cycle asymptotically.

Theorem IV (Poincare): Within any limit cycle, the number of

node, focus, and center types of singularity must exceed the number of
saddle points by one.

Remark: Most systems of interest, which can be put in the form (5), may
be analyzed by invoking the concepts of singular points and
limit cycles, together with appropriate graphical methods. The
form of the phase portrait thus determined yields valuable
information relating to stability, points of equilibrium, bound¬
edness, or periodicity of the solution, etc. These phenomena
often cannot be predicted by linearized techniques alone. The
application of these ideas will be illustrated by two simple
examples.

Example 1: We consider the van der Pol equation

x — e(l — x2)x + x = 0.

Writing this in the form

x = y = P(x, y)

y = e(l - x2)y - X = Q(x, y)

it is readily found that a singular point exists at x = y = 0 and that this

234 ELEMENTS OF MODERN CONTROL THEORY

singular point represents an unstable node. Furthermore

dP dQ
= e(l - x2).
dx dy

By Theorem II, a limit cycle exists, since this quantity changes sign at
x = ±1. The phase portrait for this system is shown in Fig. 2.13. It is
worthy of note that this limit cycle is reached whatever the form of the
initial conditions. Linear systems do not display this phenomenon.

Figure 2.13. Phase Portrait for Example 1.

Example 2: The system described by

x + o)02 x — h2x3 = 0

closely approximates the motion of a pendulum with large deflection

angles. Writing this as

* = y = P(x, y)

y = — (&>o2— h2x2 ) x = Q(x, y)

we see that there are three singular points as follows.

THEORY OF NONLINEAR SYSTEMS 235

a. A center at x = y = 0.
b. A saddle at x = w0h,y = 0.
c. A saddle at x = — &>0h,y = 0.

This information is sufficient to construct the phase portrait shown in

Fig. 2.14. Note that, depending on the initial conditions, there is either
a stable limit cycle or a divergent motiop. This is again in contrast with
linear systems where the stability is not affected by the initial conditions.
Various other cases of phase plane analysis are treated in the refer¬
ences. The work of DavisC6) is especially noteworthy in its analysis of
some complex situations where the nonlinearities are analytic and differ¬
entiable. Many cases in flight control systems are characterized by
piecewise linear conditions, viz., relays, saturation, dead zone, hysteresis,
etc. In other cases, the nonlinearity may be approximated by piecewise
linear segments. The analysis of this type of system is particularly simple
and elegant, yielding considerable information on the properties of the
motion with comparatively little effort. This approach, which is due to
Kalman/7) is described next.

2.1.4 Piecewise Linear Systems

For a linear system, a determination of the nature of the critical point

is sufficient to characterize the complete behavior of the system for all

Figure 1.14. Phase Portrait for Example 2.

236 ELEMENTS OF MODERN CONTROL THEORY

time. This problem was discussed in Sect. 2.1 .1.2. Figure 2.10 depicted
the types of phase plane trajectories which characterize each type of
singular point. In Sect. 3.1.2, these trajectories are said to indicate the
form of the phase portrait in the vicinity of the singular point, since the
relevant equation was linearized about this point. However, if the system
is linear in the entire region, then these paths are valid in the entire
region. These considerations suggest that the phase plane be decomposed
into regions, in each of which the system is linear, then combining the
various paths in order to completely describe the motion. At all times,
the qualitative aspects of the system response will be kept in the fore¬
ground. The aim is to obtain the greatest possible degree of insight into
the problem with a minimum of labor. While exact numerical results may
be obtained with some additional effort, this is not our primary ob¬
jective at the moment. As will be seen, a wealth of information on
properties of the system motion will be derived with little computational
effort.
The approach will be described by applying the method to several
problems in the field of flight control. A few preliminary remarks are in
order. For any region in the plane where the system is linear, the motion
is governed by

9 + 2 + OT0 = w20* (29)

where 6* represents an external forcing function. For phase plane

analysis, this may be either a step or ramp function, or zero. For
definiteness, we will consider only the step input, in which case, the
change of variable

x = e* - e

converts Eq. (29) to

x T 2fo>x T a>2x = 0. (30)

Putting this in the standard form

x = y = P

y = — 2fcoy — a)2x = Q (31)

we find that the singular point is at x = y = 0, and is characterized by

THEORY OF NONLINEAR SYSTEMS 237

Stable Node: r > 1, w > 0

Stable Focus: 0<f<l,co>0
Center: f = 0, CO2> 0 (32)
Unstable Node: f < — 1, co > 0
Unstable Focus: - 1 < f < 0, co > 0
Saddle: CO2< 1.

In the examples to follow, it will beTnore convenient to work with

the 6 — 6 plane rather than the x — y plane. The foregoing discussion
indicates that all that is required is a translation of the origin. Con¬
sequently, one may work with Eq. (29) directly in the# — 6 plane, if one
considers that in this case the singular point is at 6 = 6*, 6 = 0.
We now consider the problem of a simplified pitch plane vehicle
control system incorporating various types of nonlinearities.

Example 3: The system to be analyzed is shown in Fig. 2.15, with

the control loop as depicted in Fig. 2.16. The symbols used have the
following meaning.

I = moment of inertia of vehicle, slug-ft2

JZA = servoamplifier gain, N. D.
K* = rate gyrogain, sec
le = control thrust moment arm, ft
la = aerodynamic load moment arm, ft
La = aerodynamic load, lb/rad

Figure 2.15. Vehicle Geometry for Example 3.

238 ELEMENTS OF MODERN CONTROL THEORY

Figure 2.16. Control Loop for Example 3.

s = Laplace operator, sec-1

T c = control thrust, lb
t = time, sec
50 = engine command, deg
5 = engine angle, deg
9 = pitch angle, deg
9b = error signal, deg
9C = input signal, deg

He = T ctc/I = control engine effectiveness parameter, sec-2

Ha = La^a/I = aerodynamic effectiveness parameter, sec- 2

The equations defining the motion are

9 — fic8 + na9 (33)

8C = — (Kfls + 1)6] (34)

9C = A, t > 0
(35)

The nonlinearity is a saturation characterized by*

8 = B , 8 c > Scm

8c> 8cm < 8C < 8cm (36)

= 13 , 8C <C. 8cm %

*Scm = saturation limit of

THEORY OF NONLINEAR SYSTEMS 239

Dividing the 8C — 8C phase plane into three regions, as shown in Fig.

2.1 7, we may write the relevant equations as follows.

Region ®:8C - fia8c = (37)

Region ®:SC + ncKAKR'8c + — na)8c

= (HrKA— Ha)8* (38)

Region ®:SC — ^a8c = na8% (39)

Here

* K^(/iaA + a cB)
o l = -
Mar

** _ h«KaA (40)
O 2 — — -
(MoK^j — Ha)

*
8 3
Ma

The 8* are the singular points for their respective regions. Note
that a singular point need not be located inside its own region. When
240 ELEMENTS OF MODERN CONTROL THEORY

indeed it is located outside its own region it is known as a virtual

singular point.
Let us assume that the system parameters are such that 5* and 8* are
saddle points while S*2is a stable focus. This situation exists when the
parameters A, B, KA, K«, /*„, are all positive, (ncKA - y.a) > 0, and the
IvAand Kfl are chosen appropriately. Then 5* is located in Region ©, as
are also Stands*. Typical trajectories for each region then have the form
shown in Fig. 2.17. The initial conditions are 8C = KaA, 8c = 0; i.e.,
the trajectories all start on the positive <$,axis. Now if KaA > 8CM,then
the system diverges; a typical form of the trajectory is as depicted by
Path I. If K,4A < 8cm, but with the magnitude of the step command such
that Path II is followed, then on entering Region ®, the trajectory follows
a divergent path, and the system is unstable. With the magnitude of the
step input, A, reduced still further, a path such as III is followed and a
point of stable equilibrium is reached with 8C taking a steady-state value
equal to 5*- There are thus only two possible types of response, one of
which results in a state equilibrium, and the other an unbounded di¬
vergence. For given values of the system parameters, this is solely a
function of the magnitude of the step input. For the case discussed
herein (aerodynamically unstable vehicle), catastrophic instability is a
potential problem.
In the case of an aerodynamically stable vehicle, the parameter is
negative and the singular points, 8 * and S*, are not centers. These will
still be located within Region © and the paths for Regions® and ® will
be as shown in Fig. 2.18. Depending on the magnitude of the step input,
the paths may or may not enter Region ®. In all events, an equilibrium
condition will be reached with 5 = S* (which is now positive, since na is
assumed negative).

Example 4: Taking again the case of the pitch plane autopilot, we

investigate the influence of dead zone in the position gyro. The control
loop is shown in Fig. 2.19a and the characteristics of the nonlinearity
are depicted in Fig. 2.19b. The relevant equations are:

0 V'tyO— /Jic,8

8 = Ka(0c - K Re - f)

f = o - |0| > e0

= o,- e0 '<e '<e0.

THEORY OF NONLINEAR SYSTEMS 241

O
O

2.18.
Figure
Stable
3(Aerod
Examp
for
Portrait
Phase
Case).
242 ELEMENTS OF MODERN CONTROL THEORY

b.

Figure 2.19. Characteristics of Nonlinear Element for Example 4.

dc is again taken to be a step input of magnitude, A. We may write,

therefore

6 + MoK4KR0 + (MJ<„ - MJ0 = (hcKa - ,00*! (a)

when |0| > 0O,while

6 + ncKA\\Rd — Ma0 = — (b)

if |0| < 6„,where

THEORY OF NONLINEAR SYSTEMS 243

$* — AcK^(A T do)
(pcKa — na)

AcK^A
V2 = - -
Ma

The phase plane is again divided into,, three regions, as shown in Fig.
2.20. (9* is the critical point for Regions © and © while (9*is the critical
point for Region ©. When the quantity (ncKA — na) is positive and the
system gains are appropriately chosen, 6\ represents a stable focus andd%
is a saddle point. The trajectory starts at the origin (0 = 0= 0 initially)
and, on reaching the region boundary, takes a path corresponding to
the stable focus in Region ®. The critical point represents a point of
stable equilibrium. Note that in the present case, with positive step
input of magnitude A, 6 is never negative.

Example 5: As a final illustration, consider the system shown in

Fig. 2.21. The approach here is as follows. Damping must be provided
to make the system stable for small errors; however, the response for

Figure 2.20. Phase Portrait for Example 4.

244 ELEMENTS OF MODERN CONTROL THEORY

large errors should be as fast as possible. This is accomplished by opera¬

ting on the damping rather than the system gain. The damping, in fact,
is made to depend on the system error rather than the output rate. The
governing equations are as follows.

5 = KAdE - id

8 — na6 = nc5

Oe = 0e - d

f = Kakr |0®|< da

= o |0*| > do

In the dE — dE plane we have

dB + ^cKaKh^b + — fia)dE = (mcKj —

for \dE\< d0and

8E + (/ttKj — Ma)0E = (/icKj — Ha)d*

for \eB\> d„ where

THEORY OF NONLINEAR SYSTEMS 245

Figure 2.22. Phase Portrait for Example 5.

We have again used a step input of magnitude A. The location of the

singular point is the same for all three regions, as shown in Fig. 2.22.
Assuming that the quantity (ncl\A — m<*)is positive, the singular point for
Regions ® and ® is a center, while for Region © the singular point is
either a stable focus or a stable node, depending on the system gain. For
a sufficiently large step input, 6% is located in Region ©, as shown, and
motion starts at the point dB = A, dB = 0. In Regions® and ®, the tra¬
jectories are ellipses with center 6%, while in Region ©, the trajectory is a
spiral with the same center. There is a point of unstable equilibrium at
eB = o, eE = — 60. A slight disturbance will cause the motion to follow
the trajectory of Path I. It is obvious that for smaller values of step input
such that 9% is located in Region ©, the motion will reach a point of
stable equilibrium.

2.2 DESCRIBING FUNCTIONS

Because of the fact that linear methods of analysis are so powerful and
universally applicable, it is natural to seek to “linearize” the nonlinear
systems encountered in control analysis. One well-established technique
involves the derivation of the (linear) equations characterizing small
perturbations about steady-state conditions. When the motions are not
small, this approach is invalid. The method of “describing functions,”
246 ELEMENTS OF MODERN CONTROL THEORY

developed in this country by Kochenburger/8) may be used to analyze

control systems incorporating one nonlinearity if the following conditions
are satisfied.

a. The input to the nonlinear element is sinusoidal.

b. The output of the nonlinear element is periodic and of the same
fundamental frequency as the input signal.
c. Only the fundamental of the output wave need be considered in
frequency-response analysis.
d. The nonlinear element is not time varying.

Under these conditions, the method yields an “equivalent linear gain”

for the nonlinear element. Using frequency-response methods, one is
then able to determine the stability properties of the system, the fre¬
quency and amplitude of limit cycles if they exist, and whether these
limit cycles are stable or unstable.
In this section, no attempt will be made to develop the method in rigor
or depth, since this topic is treated extensively in standard texts.(9) We
will instead emphasize the basic principles and fundamental assumptions,
and develop some very general forms for the describing function. Some
recent developments such as the dual input describing function (DIDF)
and methods for multiple nonlinearities will be discussed. Potential
applications to launch vehicle control problems will be stressed.

2.2.1 Definition of the Describing Function

The conventional definition of a transfer function of a linear dynamic

element is based on the response of the element to an input signal of the
form

x = R„ sin cot. (41)

The output is then

y = Ro sin (cot + <p) (42)

where, in general, R„ is different from R„ and <pis the angle of phase lag.
The amplitude ratio, R0'/R0, and phase lag, <p,are functions of the input
frequency but not of the input amplitude. If the dynamic element is not
linear, then the response to a sinusoidal input is periodic but not sinusoi¬
dal. In seeking to define an equivalent transfer function for a nonlinear
THEORY OF NONLINEAR SYSTEMS 247

element, one is led to consider the frequency spectrum of the output and
to assume that only the first harmonic is significant.
Let the input to the nonlinear element be given by Eq. (41). Then the
output has the property

y(t + T) = y(t) (43)

where T = 2tt/o> . The output y(t) may be expanded in a Fourier series

as follows.

. . A„ v-' . 27rnt , v' tj • 27rnt (44)

y(t) = — + l_, A„ cos — - i LB, sin ——-
2 n=1 1 n=l T

where

2
ra + T
/ 27rnt^ (45)
_ T /Ja y(t) cos\ T~ )) dt

^27rnt\ (46)
2
_ T % [ Ty(t)
sin( ) dt

and a is any constant.

We note also

27rnt , „ 27rnt „ . 27rnt ,

A„ cos - \- Bn sin - = sin - (- co„
T T T

~ 2xnt . „ 27rnt .
= Ln sin - cos in„ + Ln cos - - sin
T T

Equating coefficients of like sine and cosine terms, we obtain

= Cn cos ipn

An = C* sin <pn.

Solving this for C* and <pngives

C„ = Va7 + b„2 (47)

ipn = tan 1— . (48)

248 ELEMENTS OF MODERN CONTROL THEORY

Hence, Eq. (44) may be written in the form

If we put

a = -L

T = 2L

then we have

A„= i J y(t)cos dt

Bn= ~J y(t)sin dt.

We see, therefore, that if y(t) is an even function, i.e., y(t) = y( —t), then

B„ = 0

and

A’~lL y(t) (f-‘)dt

while if y(t) is an odd function, i.e., y( —t) = —y(t), then

An = 0

and
THEORY OF NONLINEAR SYSTEMS 249

For the special case of

y(cot) = y(cot + 2 it) (51)

we have

00
(52)
y(«t) = A + La: cos ncot/+ X sin ncot
^ n—1 n=l

where

An = — y(cot) [cos ncot] d (cot) (53)

7T Ja

1 ra^~27r
B„ = - / y(cot) [sin ncot]d (cot). (54)
7T *'«

If

y(wt) = y(-«t)

then

B, = 0

2 rir (55)
A„ = — / y(cot) [cos ncot] d (cot).
7r Jo

If

y( —cot) = - y(cot)

An = 0

(56)
Bn = — / y(cot) [sin ncot] d (cot).
7T Jn

We now define the describing function as the complex ratio

250 ELEMENTS OF MODERN CONTROL THEORY

This may be expressed in the equivalent form

= ge(Ro) + jbe(R0). (58)

As is obvious from the definition, the only significant quantity at the

output of the nonlinear element is taken to be the fundamental harmonic
of the Fourier spectrum. In this way, an equivalent linear transfer
function (the describing function) is obtained. The validity of this ap¬
proximation is strongly dependent on the low band-pass properties of the
system considered. This condition is fulfilled in most systems of engi¬
neering interest and is, in fact, more accurate the higher the order of the
system.

2.2.2 Describing Functions for Various Nonlinear Elements

In this section, the describing function for various common types of

nonlinearities will be developed. Following this, a very general des¬
cribing function will be derived that is applicable to a wide class of
piecewise linear elements. The section will conclude with an extensive
tabulation of describing functions of the type most generally encountered
in practice.

Combined Saturation and Dead Zone. This nonlinearity is illustrated

in Fig. 2.23a, and the output waveform is shown in Fig. 2.23b. We have

y(wt) = 0 0 < cot < coti

— R<>(sin cot sin coti), coti ^ cot cot2

where

sin coti = D/R„ and sin cot2 = S/R„.

Therefore

R„ (sin cot — sin coti) sin cot d(cot)

THEORY OF NONLINEAR SYSTEMS
251

Figure 2.23. Combined Dead Zone and Saturation.

4 r*
4— I R0 (sin wt2 — sin wti) sin wt d(cot)
ir Jtt

2R„
ic
cot2 — o»ti -)- sin2cot2
2
sin2wtiJ 2
252 ELEMENTS OF MODERN CONTROL THEORY

and

Ax = 0.

Hence

r Bi 2 sin 2o>t2 sin 2cot ;

UD — — - ~ Ct)t2 — C0tl + (59)
Ro 7T

The following special cases may be noted.

For saturation only

wti = 0

so that

2f sin 2wt2 (60)

Gc — wt2 + - --
rr L 2

For dead zone only

so that

2 Tw_ ^ sin 2wtx (61)

Gd
^ l_2 " 1 2

The describing function is shown graphically in Fig. 2.24.

Coulomb Friction. This case is shown in Fig. 2.25. We see that

Y(«t) = - K ,|| = — K sgn R

or

y(«t) = — K R > 0

= K R < 0.
THEORY OF NONLINEAR SYSTEMS 253

and
Saturat
for
Functio
Zone.
Dead
Describ
2.24.
Figure
254 ELEMENTS OF MODERN CONTROL THEORY

R(t>

Figure 2.25. Coulomb Friction.

Hence

4 f 4K
B, = - / (—K)sin«td (wt)= - —
7T " o IT

Ai = 0

so that

Bt 4K (62)
R/0 7TR()
THEORY OF NONLINEAR SYSTEMS 255

Exponential Nonlinearity. In this case, (Fig. 2.26), the nonlinearity is de¬

fined by

Since y is an odd function

Figure 2.26. Exponential Nonlinearity.

256 ELEMENTS OF MODERN CONTROL THEORY

A. = 0

2 fn
Bi = — / (R0 sin cot)" sin cot d (cot)
7T Jo

where r ( ) = Gamma Function. Hence

^ J d —
2R0”~
-
1r( 22) 7=- - 7- '
(63)
VTr(£+3)
Relay with Dead Zone and Hysteresis. The essential elements of this
nonlinearity are depicted in Fig. 2.27. Here we find that

TZ r'TT — a>/3 |Z r 277

2JT —
— 01/3
cop
A0 = K
— / d (cot)- / d(cot) =
77 •'COO 77 "77+coa

277— 01/3
K K f
= —- / cos cot d (cot) — — I cos cot d (cot)
77 •'oia 77 77+coa

2K . . .
= —— (sin coa — sin cop)

r 77-COp 1-2T7-COS
B, = — / sin cot d (cot) — — / sin cot d (cot)
77 77 TT+GIOC

2K
(cos coa + cos co/3)

where

A T h
sin coa
2R
THEORY OF NONLINEAR SYSTEMS 257

a. NONUNEAR ELEMENT

Figure 2.27. Relay with Dead Zone and Hysteresis.

A —h
sin oj/3
2R0 '

Now

Ci = VAi2 + B)2
258 ELEMENTS OF MODERN CONTROL THEORY

and

Ai
f>\ tan-1
Bi

Putting

9 = coj3

we find

(64)

Figures 2.28 through 2.30 depict this describing function graphically.

In these figures, a normalized ordinate is used, viz., (Gp) = A/KGD.
This reduces to

a. No hysteresis (h = 0)

(65)
Go =

b. No dead zone (A = 0)

4K (66)
Go / ~ *
ttR„

8 =

c. No hysteresis or dead zone (h ~ A = 0)

THEORY OF NONLINEAR SYSTEMS 259

Hysteresis. The following relations are evident from Fig. 2.31 .

y(wt) = R0 sin «t 0 < cot < -

R„ - I < cot < 7T — cotl

260 ELEMENTS OF MODERN CONTROL THEORY

J3|<j
00 (M «X>

Fun
De
Re
2.2
An
Fig
Pha

O 00 (M CO O t^QOCMCOO t*<
• I t-* •-* (Si 03 CO ri* t)*
I I I I I I I 1 I

(Sap) ch
THEORY OF NONLINEAR SYSTEMS 261

-4b -44 -40 -36 -32 -28 -24 -20 -16 -12 -8 -4 0

<P1
(deg)
Figure 2.30. Relay Describing Function, Amplitude versus Phase.

3ir
= R0 sin cut + 7T — cutl < Cut <C —

-
< cut < 2ir — cotl

= R0 sin wt 2 tt — cuti < cut < 2n

where
262 ELEMENTS OF MODERN CONTROL THEORY

Figure 2.31. Hysteresis.

RP — a
sin wti -
Ro

Therefore

Ai

Ro j"n~utl a i
H- - cos cot d (wt) - / cos cot d (cot)
* J 27rKn
THEORY OF NONLINEAR SYSTEMS 263

Rc r3*h . a
+ — / sin wt cos wt d (cot) -
7r TT
—OJf 2?r

2TT
ITT- — COt
Ro 1 a

7T
(
J3t/2
cos cot d (cot) H-
27T

R0 r2* . a
d- / sin cot cos cot d (cot) cos cot d (cot)
7T •'27T— 2tt
1 wtl

Rq rvli a r7^2.
Bi = — 1 / sin2 cot d (cot) — — / sin cot d (cot)
i ' 2tt j0

TT —
^ 1,— ^ ^
—- / sin cot d (cot) — — / sin cot d (cot)
,r t{2 27r "r/2

R »3ir/2 a r3ir/2
d- - sin2 cot d (cot) ~h~ sin cot d (cot)
7T " TT-COf 27T JjT —COt
1 1

£ 27T-CO*! a 2TT
—OH1
- - / sin cot d (cot) + — / sin cot d (cot)
TT •'2t/9 2tt •'o-./o

R r27r a f27r
-f- —:- / sin2 cot d (cot) - / sin cot d (cot)
7T 007T—OH 2ir *2Tr — Ci)t

=^ [5+0,11
+1~(r0)cos
a,ti]
TT — COt

A0= - f R„sincot—-1 d (cot)+ —f R0— — d (cot)

7Too 2J 7T7T/2
J L 2J
2TT—COt
37T/2 iTT—cot^
+ - f R0sincot+ -1 d (cot)
——f R0— d (cot)
ir ** L 2J tt J L 2J
TT—cot Stt/2
264 ELEMENTS OF MODERN CONTROL THEORY

27 r

a
R0 sin cot — d (cot) = 0
2.
2 TT— Olti

and we have

VaTTb? -1Ai
Gd tan
R„ Bj

1/2
2 2
— —2 ) —|—( — —
|—coti -(- sin coti cos coti ) tan_1/3
rT2 R0 / \2 /
(68)

where

ii~2)
p = - .

R°^ +coti
+sincoti
coscoti
^
+ General Describing Function for Piecewise Linear Elements. Most
of the describing functions derived thus far are special cases of a more
general form (10) shown in Fig. 2.32. Applying Eqs. (53), (54), and
(58) to this case, we find

g(R°) = -x Ro (— #1 + 202 +03 — 04 — 0s)

itK0

R0 .
+ — (sin 20i — 2 sin 202 — sin 203 + sin 204 + sin 206)
2

+ 2a(— cos 0i + 2 cos 02 — cos 06) + 2d (cos 03 — cos 04)1

(69)
U2
+ Ro (Ob— Od + — (sin 203 — sin 205)
71R, 2

+ 2k2 (cos 06 — cos 03)

+ ^-[Ro(ir— 202) +
7T IV 0
R0 sin 202 — 4k3 cos 02]

and
THEORY OF NONLINEAR SYSTEMS 265

2.32.
Figure
Nonlin
Linear
Piecew
Genera
A
266 ELEMENTS OF MODERN CONTROL THEORY

b(R0) = -5L. —° (cos 20i T- cos 203 — cos 204

7rR0 L 2

cos 206) + 2a (sin 0i — sin 06) + 2d (sin 03 — sin 04)] (70)

n2 R
° (— cos 203 -f cos 206) + 2k2 (sin 05 — sin 03)
ttR0 L 2
where

Dc — Fe
k2 =
D - F

k3
n3

„ . , a . , b _ . _. c
0i = sin-1 — 02 = sin 1— 03 = sin 1 —
R0 R0 Ro

d g
04 = sin-1 — 06 = sin-1 — .
R0 R0

Note that the 0f ’s have been defined such that they all lie in the first
quadrant. A variety of special cases may be deduced from Eqs. (69) and
(70). An extensive tabulation of these is given in Table 2.1., which is
adapted from Ref. 10.

A General Polynomial Type Nonlinearity. The input-output character¬

istic of this type of element may be expressed by the equation

y = cnx"+ c„_ixn-2|x|+ cn_2x"-2

+ c„_3x"-4|x|+ .
••• + C2x|x| + CiX.

There is no loss of generality in assuming that n is odd. The absolute

value signs are used to ensure that y is an odd function of x. Proceeding
in a manner completely analogous to that used in deriving Eq. (63), we
find

(71)
THEORY OF NONLINEAR SYSTEMS 267

Remark: All of the describing functions encountered thus far are inde¬
pendent of frequency. Occasionally, a situation arises in which
the describing function is dependent on input frequency. This
may occur, for example, when there is a velocity saturation.
The analysis in this case is extremely complex and the use of a
computer is mandatory. Various problems of this type are
discussed by Kochenburger.(U)

2.2.3 Application to Stability Analysis

The describing functions, as developed in the previous section, may

be used to study the existence of self-sustained oscillations (limit cycles)
in feedback control systems. It is important to bear in mind the major
assumptions on which this analysis is based, namely

a. The system is autonomous, i.e., unforced and time invariant.

b. The nonlinearity is frequency independent.
c. The linear transfer function of the system contains sufficient low
pass filtering to warrant excluding from consideration all harmonics
of the nonlinear element output other than the fundamental.

For purposes of analysis, the system under consideration may be

depicted as shown in Fig. 2.33, with r = 0. We assume that the input to
the nonlinearity may be represented as

x = R0 sin cot.

We now seek to determine sufficient conditions to ensure that this

oscillation be maintained indefinitely. If the system under discussion
were linear (Gz> = 1 in Fig. 2.33), then a sufficient condition for the
existence of sustained oscillations would be

Figure 2.33. Use of Describing Function in Stability Analysis.

268 ELEMENTS OF MODERN CONTROL THEORY

Table 2.1. The Describing Functions for Twenty-three Common

Nonlinear Elements

Nonlinear devices with memory

THEORY OF NONLINEAR SYSTEMS 269

Table 2.1 . The Describing Functions for Twenty-three Common

Nonlinear Elements ( Cont’d)

Nonlinear devices with memory

270 ELEMENTS OF MODERN CONTROL THEORY

Table 2. 1. The Describing Functions for Twenty-three Common

Nonlinear Elements (Cont’d)

Nonlinear devices with memory

THEORY OF NONLINEAR SYSTEMS 271

Table 2. 1. The Describing Functions for Twenty-three Common

Nonlinear Elements (Cont'd)

Nonlinear devices with memory

272 ELEMENTS OF MODERN CONTROL THEORY

Table 2. 1. The Describing Functions for Twenty-three Common

Nonlinear Elements ( Cont’d)

Nonlinear devices with memory

THEORY OF NONLINEAR SYSTEMS 273

Table 2. 1. The Describing Functions for Twenty-three Common

Nonlinear Elements ( Cont’d)

Nonlinear devices with memory

be o
| i
£%
a a
S
§ B
s 3
Characteristic Describing function

g(Ro>= o
[*% Ro(262+e3' V +~T (- 2 8ln262

- sin 2 0+ sin 2 0 ) + 2a (2 cos 0 - cos 0

n3 3 5 2 3

-C08V]
+iilo[Ro(e5-e3)
+^(8ln2e3
- sin 2 0 ) + 2k (cos 0 - cos 0 )
5 2 5 3

+~ 1r ( it - 2 0 ) + R sin 2 0„
ttR [ o 2 o 2
o

*4k300862]
"l f Ro
b(Ro)
=^ [~f (C0S
63' C°82V +28(8l“®3
o
'"‘“VI+ik [~T(-cos
20g o

+ cos 2 0^_)+ 2k^ (sin 0^ - sin 0^)

274 ELEMENTS OF MODERN CONTROL THEORY

Table 2.1 . The Describing Functions for Twenty-three Common

Nonlinear Elements ( Cont’d)

Nonlinear devices with memory

d
b£> O
$ 2
£ 8
a a
§ «
a 3
Characteristic Describing function

g(Ro>
=^r[ V2 Ve3-e5) +!^(-28ln2e2
o *

- sin 2 0+ sin 2 0 ) + 2a ( 2 cos 0 - cos 0

3 0 6 0

- COS
0g) o(e5-e3)+T-(8ln2e3

- sin 2 0^) + 2k^( - cos 0^ + cos 0^)

4M .
—— cos 0„
irR 2
o

b(R0>
= 1~r (coe
2e3•cos
o
2e5)
-2a(sin03+sin©J +^ [-f- (-cos20g
3 o
+ cos 2 0 ) + 2k (sin 0 - sin 0 )
oc 5 o
THEORY OF NONLINEAR SYSTEMS 275

Table 2.1. The Describing Functions for Twenty-three Common

Nonlinear Elements (Cont’d)

Nonlinear devices with memory

276 ELEMENTS OF MODERN CONTROL THEORY

Table 2.1. The Describing Functions for Twenty-three Common

Nonlinear Elements (Cont’d)

Nonlinear devices with memory

THEORY OF NONLINEAR SYSTEMS 277

Table 2. 1. The Describing Functions for Twenty-three Common

Nonlinear Elements ( Cont’d)

Nonlinear devices with memory

278 ELEMENTS OF MODERN CONTROL THEORY

Table 2. 1. The Describing Functions for Twenty-three Common

Nonlinear Elements (Cont’d)

Nonlinear devices with memory

Characteristic Describing function

g(Ro>
- 7r (2V- VV+V81”26! o

15
- sin 2 02)+4a (cos8^ - cos S^J

n3 r
+ —— R ( ir - 2 0J + R sin 2 0„
irR l o 2' o 2

- 4k cos 0„
3 2

b(R ) = 0
o

g(Ro>
= I2V-V«2> + Ro(8in2ei
o
“i/i M - sin 2 02>+4a (cos8^ - cos 0^)j
16
ab
4M
^ —r— cos 0
»rR 2
o

b(R ) = 0
o
THEORY OF NONLINEAR SYSTEMS 279

Table 2. 1. The Describing Functions for Twenty-three Common

Nonlinear Elements (Cont’d)
280 ELEMENTS OF MODERN CONTROL THEORY

Table 2.1 . The Describing Functions for Twenty-three Common

Nonlinear Elements (Cont’d)

Nonlinear devices with memory

THEORY OF NONLINEAR SYSTEMS 281

GH
K

In the present case, this condition becomes

KGH =

Furthermore, in the linear case, the simplified Nyquist criterion*

states that if the critical point, —1/K, is to the left of the G(jw) H(j«)
locus when the latter is traced out in the direction of increasing fre¬
quencies, then the system is stable. In the nonlinear case, when the
describing function is used to represent the nonlinearity, the above
statement remains valid if —1/K is replaced by — (1/GDR0).
To examine the implication of these concepts, we plot the KGH and
— (1/Gd(R0) curves in the Nyquist plane, a typical case being as shown
in Fig. 2.34. Here, the arrows on KGH and — [1/GD(R0)] indicate the

Im

Re

K GH K GH
2

Figure 2.34. Nyquist Curves and Describing Function.

*The simplified Nyquist criterion is valid when all the poles of the open-loop transfer function
he in the left-hand plane.
282 ELEMENTS OF MODERN CONTROL THEORY

direction of increasing « and R0, respectively. For a sufficiently low

open-loop gain (the Ki GH curve), there is no intersection of the curves,
and no limit cycles exist. For a higher open-loop gain (the K2 GH curve),
a point of intersection is located at p2, and a limit cycle is indicated
having an amplitude and frequency corresponding to the values of R0 and
w at the point p2. It remains to determine whether this limit cycle is
stable or unstable. We note that when the value of R0 is such that
— [1/Gd(R0)] is to the right of p2, then instability is indicated by the
simplified Nyquist criterion, causing R0 to increase and thereby moving
the point — [1/GP(R0)] to the left. When R0 is such that the point
— [1/Gd(R0)] is to the left of p2, then the Nyquist criterion indicates
stability. Consequently, R0 decreases, moving the point — [1/GD(R0)]
to the right. It is obvious, therefore, that p2 is a stable point of equili¬
brium, and that p2 is a stable limit cycle.
The amplitude and frequency of this limit cycle are independent of
the initial conditions, a phenomenon not exhibited by linear systems.
When there is a phase lag associated with the describing function, the
— [1/Gd(R0)1 curve may appear as in Fig. 2.35. This particular repre¬
sentation is typical of a relay having hysteresis and dead zone. It may be
shown that the Nyquist criterion still applies, even though the critical
point is not on the negative real axis. Thus, if R0 is such that the operating
point is to the right ofp2, the system is unstable, R0 increases, and the
operating point moves along the — [l/G0(Ro)] curve toward p2. Reasoning
as before, we find that p2 represents a point of stable equilibrium, i.e., a
stable limit cycle.
Figure 2.36 indicates a situation with increased dead zone. The result
is that there is no intersection of the curves and no limit cycle can exist.
Another possibility is shown in Fig. 2.37, where the curves intersect at
two points. To determine the stable limit cycle we may reason as follows.
If R0 is such that the operating point on — [1/GD(R0)1 is located between
Pi and p2, the system is stable, R0 decreases, and the operating point
moves to pi. The motion is stable and there is no limit cycle. If the
operating point on the — [1/GD(R0)] curve is between p2 and p3, the
motion is unstable, R0 increases, and the operating point moves top3.
Also, if the operating point is to the left of p3, the motion is stable, and
the decreasing R0 causes the operating point to move toward p3. Thus,
p3 represents a stable limit cycle and p2 is an unstable limit cycle. Sum¬
marizing, we find that any disturbance that causes an R0 whose operating
point on the — [1/GD(R0)] curve is to the left of p2 will result in a motion
which damps out completely, while larger amplitudes of R„ will result in
a limit cycle of frequency and amplitude determined by point p3.
THEORY OF NONLINEAR SYSTEMS 283

Figure 2.35. Nyquist Curve and Describing Function with Memory— Limit
Cycle Shown.

Figure 2.36. Nyquist Curve and Describing Function with Memory-No

Limit Cycle.

2.2.4 Multiple Nonlinearities

The analyses of the previous sections have been based on the assump¬
tion that there is only one nonlinearity in the system. For multiple
nonlinearities, various extensions of the describing function technique
284 ELEMENTS OF MODERN CONTROL THEORY

Im

Figure 2.37. Nyquist Curve and Describing Function with Memory-Two

Limit Cycles Shown.

have been described in the literature. (13-15) Perhaps the most straight¬
forward approach is to develop a composite describing function when
the nonlinear elements are in series. This problem has been studied by
Gronnerd15) If the nonlinearities are separated by linear dynamic
elements, the analysis is much more difficult. Even for relatively simple
systems some sort of computer assistance is required.
While a detailed computational approach is tedious, the broad outlines
of the method of analysis are relatively simple and afford a degree of
THEORY OF NONLINEAR SYSTEMS 285

insight to the results generated by computer. We will here describe a

technique due to Gran and Rimer,!1 21 which is applicable to multiple
nonlinearities and can be put in the form shown in Figs. 2.38 or 2.39.
Certain restrictions must be imposed on the linear elements, Gi(s), for
the describing function method to be applicable. In Fig. 2.38, there is
no restriction as long as the input signal, x, is sinusoidal, which will be
the case if the system is low pass. For the form of Fig. 2.39, however,
the output of the linear dynamic elements must be sinusoidal. There¬
fore, all the Gi(s) must be low pass.
The method to be described is valid for various modifications or
combinations of Figs. 2.38 and 2.39. The essential condition that must
be satisfied is that the input to the nonlinear elements be sinusoidal;
tliis requirement serves to indicate which parts of the system must be
low pass.
Consider now the system of Fig. 2.38. The input, x, is assumed to be
of the form

x = R0 sin cot.

Consequently, the input to the ith nonlinearity is

R0|G,(jai)|sin [«t + /Gi(j<*) ].

Figure 2.38. Nonlinear Elements in Parallel.

Figure 2.39. Nonlinear Elements in Series.

286 ELEMENTS OF MODERN CONTROL THEORY

The describing function of the ith nonlinearity, denoted byGD(,) (R0,co),

is therefore, a function of the amplitude, R0, and the input frequency, co.
If these describing functions are used to replace the nonlinearities in the
system block diagram, the result is a linear system.
In Fig. 2.39, the input to the first nonlinearity is

R oIGi(jco)| sin [cot+ /Gi(jco) ].

The input to the second nonlinearity becomes

R0|G,(jco)[
|G2(jco)|GD(1)(R0,
«) sin (cot+ Z9l + ZZA

and this is used to determine the describing function for the second
nonlinearity, GD(2)(R0,w). This procedure is continued until all the
describing functions are formulated. Although these are, in general,
quite complex, they depend only on R0 and co. For either form, a set of
values of R0 and co will determine a linear system whose operating gain
and pole-zero locations are determined for these values of R0 and co.
The characteristic equation for the entire system is now written as

II [s + Zi(R0, co)]
1 + K0(R0> co) — - =0. (72)
M

XT Is T Pi(Ro>w)]
t=l

For a specified value of R0 and co, a pole-zero configuration is deter¬

mined along with an operating gain, K0. A limit cycle is indicated at that
point on the root locus where the latter intersects the imaginary axis.
However, a change in R0 (or co) also changes the pole-zero configuration
as well as the operating gain. Therefore, in general, a separate root locus
is required for every value of R0 and co. The general procedure is as
follows.

a. Select a value of R0 and co (say R0(1) and coi). Determine the root
locus and operating gain, K0(1).
b. Select a new value of R„ (say R^2)),and with the same value of coi.
A new pole-zero configuration and a new root locus result, along
with a new value for operating gain, K0(2).
c. On a separate diagram, connect all the operating points, Ko(0, re¬
sulting in an “input-dependent locus.” The limit cycle amplitude and
THEORY OF NONLINEAR SYSTEMS 287

frequency are determined by the intersection of this locus with the

ja> axis. If the frequency at crossover and the original assumed
frequency, «i, do not correspond, then a new input-dependent locus
for a different w must be determined and the process continued until
the assumed frequency and crossover frequency correspond. This is
the predicted limit cycle.

The above procedure simplifies materially for the case of Fig. 2.38, since
the pole-zero locations and operating gains do not depend on frequency.
In any event, the input-dependent locus generated by this method may
be interpreted as follows.

a. If the locus remains in the right-hand plane, the system is unstable;

if it remains in the left-hand plane, the system is stable.
b. If an increasing R0 causes the locus to cross the imaginary axis from
the right-hand to the left-hand plane, a stable limit cycle exists.
c. If increasing R0 causes the locus to cross the imaginary axis from
the left-hand to the right-hand plane, there is an unstable limit cycle.

The proof of the above assertions follows the conventional arguments for
root locus stability. As an example of the above approach, we consider
the following.

Example 6: The block diagram of the system to be investigated is

shown in Fig. 2.40. This may be rearranged to the form shown in Fig.
2.41, which is of the parallel form of Fig. 2.38. Now by putting x =
R0sin wt, the open-loop transfer function becomes

K0(R0)
(s + l)2(s + l)

Figure 2.40. Control Loop for Example 6.

288 ELEMENTS OF MODERN CONTROL THEORY

Figure 2.41. Equivalent Control Loop for Example 6.

where

}= 100Gp(1)(Ro)
10 G d(2)(R0) + 1

1 = 100 G g(2)(R0) + 1
" 10 G d(2)(R0) + 1

G d(1)(R0)= 2^1 sin-1— — -A \/R02—b; 4MX

+ — VR02 - b2
TT R„ Rn2 ttR 2

4Mj
GD'2>(R0)
ttRq

ni = slope of saturation nonlinearity.

In this example

b = 1

Mi = 1000

M2 = 10.

A typical root locus for a particular value of Rc is shown in Fig. 2.42,

while Fig. 2.43 depicts the input-dependent locus for this problem. The
pole locations for specified values of Re are summarized in the following
table.
THEORY OF NONLINEAR SYSTEMS 289

Figure 2.42. Root Locus for Example 6.

Figure 2.43. Gain Locus for Example 6.

290 ELEMENTS OF MODERN CONTROL THEORY

For R0 = 0, the point on the input-dependent locus corresponds to the

double pole at —1, and for R0-^> «, the locus returns to this same point.
In accordance with the criteria listed above, the point 2 in Fig. 2.43
corresponds to an unstable limit cycle, while point 7 corresponds to a
stable limit cycle. The latter occurs at the values R0= 1000 and w = 3.6
rad/sec.

2.2.5 Dual Input Describing Function (DIDF)

The method is based on consideration of two sinusoids being applied at

the input of a nonlinear element. In certain cases, a high frequency low
amplitude signal (dither) is introduced externally to modify the character¬
istics of the nonlinearity. This is the so-called “dynamic lubrication”
method, which often has the effect of linearizing the nonlinear element,
and tends to stabilize the system.
In the present case, however, the high frequency signal is taken to be
the system limit cycle, while the slowly varying signal is derived from
input commands to the system. The concept of a DIDF was first intro¬
duced by West(42) and is discussed in a standard text by Gibson. (43) While
useful in certain cases, this approach is extremely tedious and lengthy.
An extension of the method by Gelbe and Vander VeldeG6) affords a
crucial simplification in that a limit cycling system is treated as a control
system whose signal transmission properties can be readily derived under
certain mild restrictions on the input command signal. The discussion
will be limited to feedback control systems of the form shown in Fig.
2.44.
If a system which exhibits a limit cycle in the unforced state is sub¬
jected to a slowly varying input, the output follows this input, on the
average, to within some following error. A typical situation is shown in
Fig. 2.45. Over any limit cycle period, the system error may be approxi¬
mately modeled as a sinusoid plus d-c bias. The sinusoid is associated
with the limit cycle and the d-c bias with the error. This suggests that

Figure 2.44. Control Loop for DIDF Analysis.

THEORY OF NONLINEAR SYSTEMS 291

Figure 2.45. Input and Output Waveforms.

the input to the nonlinear element be taken as

x = Ri + R0sin (i>0t. (73)

Making the same assumptions as for conventional describing functions,

the output from the nonlinear element becomes

y = Ri + R',sin (coat + <pi). (74)

In other words, the output from the nonlinearity is a different d-c bias
plus a sinusoid of different amplitude and with some phase angle, <pi,
with respect to the input. The quantity, w0>represents the limit cycle
frequency. We now define the following.

Limit Cycle DIDF = N0(R0, Ri) = — e,Vl (75)

R0

Signal DIDF = Ns(R0,Ri) = (76)

Ri

The nonlinearity in a limit cycling control system is, therefore, char¬

acterized by two DIDF’s. The first of these, the limit cycle DIDF, is used
to predict the system limit cycle in a manner identical to that in con¬
ventional describing function analysis; i.e., the limit cycle is obtained as
the solution to the equation
292 ELEMENTS OF MODERN CONTROL THEORY

N0(R0, Ri) G(jo>) = -1

as described in Sect. 2.2.3.
The signal DIDF will be considered as a linear gain for some range of
(Ri/R0),and will, therefore, lead to a linear system description from the
point of view of command inputs. This situation is depicted in Fig. 2.46,
and represents the approximate mathematical model for the study of the
input-output dynamics of the given nonlinear system.
It is clear that the equivalent representation is valid whenever the
DIDF formulations are themselves valid. This means that, in addition to
the usual conditions imposed on the conventional describing function,
two additional requirements must be satisfied. These are:

Ri <2
Rn 3
(77)
1

COo 3

where ws is the frequency of the input signal, x(t), and a>0is the limit
cycle frequency. A thorough discussion of these conditions is contained
in Gelb and Vander Velde. (16)
The requirement imposed on the frequency ratio stems from the as¬
sumption that the error (between input and output) be approximated by
a d-c bias over one limit cycle period. The requirement on amplitude
ratio is necessary to ensure that higher harmonics in the DIDF approxi¬
mation be negligible compared to the fundamental. Gelb and Vander
Velde show that when conditions, Eq. (77), are satisfied, the approxi¬
mate DIDF is within 5 percent error referred to the untruncated
expression for the DIDF.
The above ideas can be clarified by considering a particular case.

R sin cot
o

Figure 2.46. Equivalent Control Loop for DIDF Analysis.

THEORY OF NONLINEAR SYSTEMS 293

Example 7: The system to be investigated is shown in Fig. 2.47.

We begin by deriving the relevant DIDF’s for the relay. We use

x = Ri + R0 sin &>0t.

From Eq. (75)

where 9 - a>0r. Also

fir+di /•27T — dl /&f2tt

Nt - 1 me - / Dd0+ / Dd0
27rKi Jo J2tv — 0\

-
l
irRi

where 0i Note that for R! “small”

4D
N0
7rR0

which is precisely the conventional describing function for a relay, and

2D
Ns
ttRo
294 ELEMENTS OF MODERN CONTROL THEORY

The aim of the present analysis is to determine the conditions under

which a representation of the form of Fig. 2.46 is valid. For in that case,
conventional linear techniques may be employed to determine the input-
output- system dynamics. It is not difficult to show that in order to
satisfy the relations in Eq. (77) it is necessary that the dominant closed-
loop poles of G(s) in Fig. 2.46 lie within a circle of radius K<u0(with
center at the origin) in the complex plane.
The limit cycle frequency, o>0, and amplitude, R0, may be determined
in the usual manner for the system of Fig. 2.47. The results are displayed
in the root locus of Fig. 2.48. For this simple case, it is more convenient
to work with the equation of the root locus, which is

3ct2 — a>2 + 4£i coi <x = 0

s = <T+ jw.

This curve crosses the imaginary axis, {a = 0), when u>= on, which means
that the limit cycle frequency, o>0= an.
Now by substituting s = janin

1+ N0G(S) = 0

Im

Figure 2.48. Root Locus for Example 7.

THEORY OF NONLINEAR SYSTEMS 295

we find

N0 — 2£i coi.

But since N0 - the limit cycle amplitude is determined as

ttRo

7r|l 0)^

The value of Ns in Fig. 2.46 is, in the present case

N. « ^ &on.
2

In order to determine the value of the closed-loop pole on the real axis,
we put

S = a

N s = |l wi

in the system characteristic equation

1 + Ns G(s) = 0.

Time

Time

Figure 2.49. Response to Step Input for System of Example 7.

b.
Sa
El
(
F
c.
Re
H2.
(F
2i
296 ELEMENTS OF MODERN CONTROL THEORY

This yields a « — |i wi if |i < 0.1. The closed-loop poles are indicated in

Fig. 2.48. It is seen that the dominant closed-loop pole is within the circle
of radius (1/3) a>0, indicating that the equivalent representation of Fig.
2.46 is valid. A typical response to a step input is shown in Fig. 2.49 for
the actual system and its equivalent linear representation. The agreement
between the dominant modes in the two cases is seen to be remarkably
good.

2.2.5. 1 DIDF for Typical Nonlinearities

The DIDF for some of the more common types of nonlinearities are
listed below.

a. Relay with Dead Zone (Fig. 2.50a):

THEORY OF NONLINEAR SYSTEMS 297

-6

1_
6

1 1 -D

a. Relay with Dead Zone

c. Hysteresis

Figure 2.50. Typical Nonlinear Characteristics.

d. Polynomial Type Nonlinearity. This nonlinearity is expressible in the

form

v = C„ x” + C*_ix"-2|x| + . + C2x|x| + Cix.

There is no loss of generality in assuming that n is odd. We have

298 ELEMENTS OF MODERN CONTROL THEORY

C-^)
n0=^E k
c*n!R0"-*-
(n — k) ! k !
*Ri* V

C-^)
In (^1
\ 2
N.= Ctn!-TRon_*Rifc_1
k (n — k) !k!
odd

where r ( ) denotes the gamma function.

2.3 LYAPUNOV S DIRECT METHOD

The general theory of stability of motion as developed by A. Lyapu¬

nov/17) also known as Lyapunov’s Direct Method or the Second Method
of Lyapunov, is now recognized as the most powerful theoretical tech¬
nique available for the study of nonlinear control systems. Indeed, the
extreme generality of the theory also accounts for the frustrating diffi¬
culties experienced in seeking to apply the theory to systems of even
moderate complexity. It is important to recognize that the Lyapunov
approach provides merely a framework for analysis rather than a detailed
computational algorithm. This is perhaps to be expected, since nonlinear
systems are best defined in a negative sense; that is, a nonlinear system is
merely one which is not linear.
The Lyapunov method was virtually unknown in this country as little
as ten years ago. However, in recent years, a prodigious technical litera¬
ture has been developed which has served to stimulate interest in this
subject among control engineers. References 18 through 21 are concerned
with the basic theory and contain extensive bibliographies.
For present purposes, the exposition will be limited to a careful and
precise statement of the principal theorems in language that is familiar to
control engineers. While this will somewhat compromise the rigor of
presentation, it will hopefully be compensated by the added degree of
physical insight obtained. Extensive use will be made of illustrative
examples and physical applications. Of the principal directions in which
the theory has developed in recent years, only those which are potentially
useful in control system analysis will be stressed. Few proofs will be
THEORY OF NONLINEAR SYSTEMS 299

given and little attention will be paid to mathematical niceties. These

will be disposed of with reference to the pertinent literature. All impor¬
tant results will, however, be stated as precisely and completely as
possible, with due regard for inherent limitations and conditions to be
satisfied. In this way, it is felt, the material can be put in a form which is
most useful for purposes of practical application.
/

2.3.1 Symbols and Definitions

The presentation of the material in the following sections is greatly

facilitated by the use of matrix methods and a uniform terminology. It
will be assumed that the reader is familiar with the basic operations
involving vectors and matrices. Throughout Sect. 2.3, the following
conventions will be adopted.

a. Capital letters will denote matrices.

b. Lower case English letters will denote vectors.
c. Lower case Greek letters will denote scalars.
d. Subscripted English and Greek lower case letters will denote scalars.
e. The following exceptions are noted.

1. V, e, g, h, r, t will denote scalars.

2. i, j, k, n will denote integers.

Also

AT = transpose of matrix A
A-1 = inverse of matrix A
a •b = scalar product of vectors
Va = gradient of a
C) = d/dt( ).

Various terms will be used repeatedly in the following discussions and

for clarity, these are listed and defined below.

Definition 1: A system is said to be linear if it may be represented as

x = Ax + f(t) (78)

where A is a constant matrix. If the coefficients of the matrix, A, are

functions of the independent variable, t, then the system is said to be
300 ELEMENTS OF MODERN CONTROL THEORY

linear time varying. Any dynamical system that cannot be expressed in

the form (78) is called nonlinear.

Definition 2: The variables, xi, x2,. ,xn, which are the components
of the state vector, x, are referred to as the state variables, since at any
given instant of time they represent the condition or state of the system.

Definition 3: The n-dimensional Euclidean space which has axes labeled

xi, x2, . , x„ is called the state space. Each point in this state space
represents a particular set of values that the state variables may assume.
In other words, each point represents a particular state of the system.
Given the initial state of the system, (t = 0), then for t > 0 the state
variables describe a trajectory in state space.

Definition 4: A nonlinear system, described in general by

X = g(x, t) + (79)

and the linear system, given by Eq. (78), are said to be forced if f and f
are not zero.

Definition 5: The systems described by Eqs. (78) and (79) are said to be
free if f and f are zero.

Definition 6: The systems described by Eqs. (78) and (79) are said to be
stationary if the elements of A and g are not functions of the independent
variable, t; otherwise, they are called nonstationary.

Definition 7; A system that is both free and stationary is called auto¬

nomous.

Definition 8: Associated with every vector, x, is a scalar called the norm,

which is written as 11x ||. A norm satisfies the followingrelations.

1. 11x|| > 0 for all x 0.

2. | |x|| = 0 if x = 0.
3. ||x + y11 I|x|I + ||y||.
4. 11«x|| = |a| •11x||, a = anyscalar.
THEORY OF NONLINEAR SYSTEMS 301

The following are some commonly used norms.

1. ||x|| = (xrx)1/2= s/J'. x,-2.

X

2. ||x|| = J2 |x,-|.
i

3. ||x|| = max |x,-|/

i

Only the first of these, which is the standard concept of length of a

vector, will be used in the following discussions.

Definition 9: The scalar defined by

n = xrBx

is called a quadratic form in the state variables xi, x2, . , x». The
n X n matrix, B, is symmetric.

Definition 10: The quadratic form, n, will be called positive definite if,
for all x X 0 and such that m = 0 for x = 0

n > 0

and negative definite if

n < 0 .

Furthermore, m is said to be positive semidefinite if

a ^ 0

and negative semidefinite if

m <0 .

A (symmetric) matrix, B, is said to be positive definite if the relation

xrBx >0

is satisfied. Similar definitions hold for negative definite and positive and
negative semidefinite matrices.
302 ELEMENTS OF MODERN CONTROL THEORY

Example (a):

m(x1, X2) = Xi2 + x22 .

This is positive definite for all x.

Example (b):

/t(xi, X2) = Xi2 + x22 - Xi3 .

This is positive definite for all x with ||x|| < 1.

Example (c):

ju(xi, x2) = (xi - x2)2 .

Since m = 0 whenever xj = x2, this is positive semidefinite.

Example (d):

fi(X1,X2,X3)= XX2+ X32.

Here n = 0 whenever xi = x2 = 0 and x3 is finite. Therefore, m is semidefinite.

Definition 11: If the system described by

x = f(x, t) (80)

has the property that

f(a, t) = 0 , t ;> 0

then a is called an equilibrium point of the system. Whenever an

equilibrium point exists, a simple transformation of variables, z = x — a,
transfers this point to the origin. Consequently, there is no loss of
generality in assuming that the equilibrium point is always located at the
origin.

Definition 12: We consider the autonomous system

x = f(x) (81)
THEORY OF NONLINEAR SYSTEMS
303

which has an equilibrium point at the origin, viz.

x(o) = 0. (82)

Let be a region in the state space for which ||x|| < a, and letR(2)
be a similar region for which ||x|| < p. We may envision these as hyper¬
spheres centered at the origin with radii a and p, respectively. It is
assumed that a > p. Further, let the initial state of the system at time
t = 0 be x = b ^ 0. We say that the system is:

1. Stable, if for every b contained in R<2>there is a region R(3) defined by

INI < 7, where a > y > p , such that lim [x (t)] < a. In other words,
t - >00

the trajectory of the system never leaves the region R(3).

2. Asymptotically stable, if for any b in R<2>, lim [x (t)] —> 0.

t — > 00

3. Asymptotically stable in the large (completely stable), if it is asymp¬

totically stable and the region R<2>encompasses the entire state space.

4. Unstable, if for some b contained in region R<2>with p arbitrarily

small, the lim [x (t)] > a.
t— >00

It will be noted that the concept of stability is not as sharply delineated

as in the linear case. The type of stability is generally a function of the
initial state. As a rule, a stable system in an engineering sense is either
characterized by a limit cycle of “small” amplitude or else a convergent
trajectory to the equilibrium point. When a system is asymptotically
stable, it is desirable to know the region of asymptotic stability. Further¬
more, it is important to determine the conditions that ensure complete
stability. The basic theorems which provide an answer to some of these
questions are contained in the following section.

Remark: There is some compromise with mathematical rigor in the above

definitions. A more rigorous formulation could not be attempt¬
ed without requiring a firmer theoretical foundation. The
definitions are, however, eminently satisfactory for engineering
purposes— and this is the theme of the present exposition.
Sharper definitions, as well as a more extended discussion of
various types of stability, together with various existence
theorems, may be found in the literature/18)
 304 ELEMENTS OF MODERN CONTROL THEORY

2.3.2 Basic Theorems

The very active research in recent years in the area of nonlinear

stability has produced a rich variety of results for determining the
stability of nonlinear systems. Much of this work is of a highly theoreti¬
cal nature with no immediate application to practical systems. In keeping
with the aim of the present exposition, we will present only those results
which have proven useful in the study of modern nonlinear control
systems. The discussions will be, wherever possible, in the engineering
vernacular, and most of the formal presentations will be supplemented
with practical (albeit simplified) examples.
We consider an nth order automonous nonlinear system described by

x = f(x) (83)

with an equilibrium point given by

x(o) = 0. (84)

In what follows, we shall be concerned with a real valued scalar func¬

tion (the Lyapunov function ) having one or more of the following
properties.

1. All the first partial derivatives of V(x), with respect to the components,
Xi, of x, exist and are continuous for all x.

2. V(x) is positive definite; i.e.

V(x) >0 x^O.

V(o) = 0

3. The quantity V(x) = — — = VV-x

i=i dXj dt

is negative definite; i.e.

V(x) <0x^0.
V(o) = 0

4. The quantity V(x) is negative semidefinite; i.e.

V(x) <0x^0.
THEORY OF NONLINEAR SYSTEMS 305

5. V(x) —» ooas 11x|| —> oo.

6. V(x) > 0.

The following theorems refer to the system described in Eqs. (83) and
(84) and the properties of an appropriate Lyapunov function.

Theorem I: If there exists a Lyapunov function, V(x), satisfying 1,

2, and 4, then the system (83) is stable in the vicinity of the origin.

Theorem II: If there exists a Lyapunov function, V(x), satisfying 1,

2, and 3, then the origin in (83) is asymptotically stable.

Theorem III: If V(x) satisfies 1, 2, 3, and 5, then the origin is

asymptotically stable in the large.

Theorem IV: If V(x) satisfies, 1, 2, 4, and 5, and if in addition V(x)

is not identically zero along any solution of (83) other than the origin,
then the origin is asymptotically stable in the large.

Theorem V: If V(x) satisfies 1, 2, and 6, then the origin of (83) is

unstable.

Remark: Quite often it is difficult to assure or confirm asymptotic

stability in the large. One can establish only that the origin is
stable, or else asymptotically stable. These are only local con¬
cepts. The previous theorems (I and II) give no information
on the extent of stability or asymptotic stability. We know
only that if the perturbations are not “too large,” the system
tends to return to the equilibrium state, but we know nothing
about what “too large” means. The following theorem due to
La Sallet24) helps clarify the situation.

Theorem VI: Let R(r) denote the region where V(x) < r. Within
this region, assume that V(x) satisfies 1 and 2 and that the region R(r) is
bounded. Then any solution of Eq. (83) which starts in R(r) is:

a. stable, and remains in R (r) if V(x) satisfies 4 in this region.

b. asymptotically stable, if V(x) satisfies 3 in R(r).

The theorems presented in this section represent only a small fraction

of the available theorems relating to Lyapunov’s second method. They
306 ELEMENTS OF MODERN CONTROL THEORY

do include, however, most of those of direct interest to the control

engineer. Various additional results and generalizations, applicable in
special circumstances, will be presented in the following sections.
A few general observations are in order at this point. We note first of
all that the conditions for stability given thus far are sufficient; they may
not be necessary. Consequently, failure to satisfy the conditions of these
theorems does not mean that the system being analyzed is necessarily
unstable. Furthermore, a Lyapunov function for a given system is not,
in general, unique. The determination of that Lyapunov function yielding
the least restrictive conditions on stability is often a difficult problem.
Finally, the most obvious limitation in the practical application of the
direct method is the lack of a general systematic procedure for generating
Lyapunov functions. A variety of specialized techniques, applicable in
particular circumstances, will be discussed in the sections to follow. The
use of the theorems presented thus far will be illustrated in the following
examples.

Example 8:
Xi = x2 — atXi(xx2 + x22)
X2 = - X! - ax2 (xx2 + x22)

where a = positive constant.

If we take

V = Xl2+ x22
then

V = — 2a (Xi2 + X22)2

which is obviously negative definite for all x. Furthermore, we note that

V —» * as | |x| | —» oo.

Consequently, by Theorem III, the system is asymptotically stable in

the large.

Example 9 :
Xi = x2

x2 = — a(l + x2)2x2 — Xi

a = positive constant.
THEORY OF NONLINEAR SYSTEMS 307

Again take

V = Xj2 + x22.

It follows that

V = - 2a(l + x2)2x22
/
which is negative semidefinite.* Also

We will now show that V = 0 is not a trajectory of the system. In fact,

V = 0 implies that x2 equals zero or —1. The slope of the system trajectory
is

2
= — «(1 + X2) _ 5l
dX;

But for x2 = 0 this slope is infinite, which means that the xi axis cannot be
a trajectory of the system. Furthermore, forx2 =— lwe have

which is never zero except at the origin. Therefore, the line x2 =— 1 can¬
not be a system trajectory.
Since we have now fulfilled the conditions of Theorem IV, we conclude
that the origin of the given system is asymptotically stable in the large.

Example 10:

«-h (1 — |e|) e-F e —0.

Putting t = xj, we write this as

Xi = x2

x2 = (1 — |xi|) X2- Xi.

Since V= 0 for x2 = 0 and xj arbitrarily.

308 ELEMENTS OF MODERN CONTROL THEORY

Again taking
V = xi2 + x22

we find

V = - 2x22 (1 - |x x| ) .

Consider now the region in the state space which is bounded by the

curve

xi2 + x22 = 1.

Within this region, V is positive definite and V is negative definite.

Therefore, by Theorem IV, this region is asymptotically stable.

Example 1 1:
xi + Xi T x i3 == 0 .

We write this as

Xj = x2

X2 = — X2 — Xj3.

Here, if we take V =x!2 + x22as before, we can obtain no conclusive

results, However, if we use

1 , 1 ,
V = iX3 +2X2
then there follows

V = — x22

which is negative semidefinite. Reasoning analogous to that of Example

9, shows that V = 0 cannot be a trajectory of the system. And since
V —> oo as ||x|| —> oo, Theorem IV enables us to conclude that the origin of
the system is asymptotically stable in the large.

Example 12: Some of the examples considered thus far can be

analyzed from a more unified point of viewd19) Consider the dynamical
system

« + g(e)« + h(e) = 0.
 (

THEORY OF NONLINEAR SYSTEMS

309

We assume that g and h are polynomials in e, with g even and h odd. It is

also assumed that h(o) = 0. Define

Note that <p is odd and even, and that

<p(o) = iZ'(o) = 0.

Now write X! = «, and express the system in the equivalent form

Xi = x2 — ^(Xi)

x2 = - h(xi).

Note that under the foregoing assumptions, the origin is an equilibrium

point of the system.
As a Lyapunov function, we take

V = i x22 + vKxx).

It follows that

V = - h(xi)^(xj).

We now suppose that there exist two positive constants, a and f), such that

h(x1)<p(xi)
> 0 for |xi| < a, xx ^ 0
iZ'(xi)< /3 implies|xi| - a.
If these two relations are satisfied, then the region defined by V < /3 is
bounded, and within this region V < 0 . Furthermore, V = 0 in this region
only when x, =0. But the x2 axis is not a trajectory of the system since
the slope

dx2 _ _ h(xj)
dxi x2 — <p(xi)
310 ELEMENTS OF MODERN CONTROL THEORY

is finite for x2 = 0. By Theorem VI, we conclude that every solution

initiating in the interior of the region defined above is asymptotically
stable.
As an example of this approach consider the system

e + M (1 - e2) * + « = 0

where m = positive constant. In this case, with « = xj, we have

Xi = X2 — <p(Xi)
x2 = — h(xj)

with

g(Xi) = M(1 - X!2)

h(xi) = Xi

<p(xi) = —

^(Xl)
=^-Xi2.
Taking

V= ~x22
+ iA(xi)
=\ (X22
+ Xj2)
we find

V = )
which is negative semidefinite for |xi| < \/3. Consequently, we take
a = V3. Now if iA(xi) < 0 is to imply that |xi| < V%, we must have
0 = 3/2. It follows, therefore, that the region bounded by x,2 + x22 = 3
is a region of asymptotic stability.
THEORY OF NONLINEAR SYSTEMS 311

2.3.3 Lur’e Method

The earliest successful attempt to evolve a systematic method for

generating Lyapunov functions is due to Lur’e.l20) This technique is the
subject of a book by Letovt21) which discusses many applications in
great detail, but is overly burdened by an awkward notation. As a matter
of fact, it is possible to exhibit all the essential features of the method
quite clearly and succinctly by adopting matrix notation. Various dis¬
tinctions considered by Letov and other Russian investigators are shown
to be irrelevant. The emphasis in the present discussion will be on
adapting the basic results to realistic control systems without, however,
compromising the analytical development. The power of the Letov
method lies in its ability to analyze systems of moderate order. There
are, nevertheless, serious restrictions on the type of nonlinear system that
may be treated. To motivate the discussion, we consider the system
shown fn Fig. 2.51. The nonlinear function, <p(<r),satisfies the conditions

<7

(85)

*>(o) = 0.

a 0 «J) 6
&f - 1 N.L. G(s)

Figure 2.5 1. Representation of Nonlinear System.

G(s) is a linear transfer function of the form

i= 0

G(s) = - (86)
<P n

We assume that the denominator of this expression contains no multiple

or zero roots and that each of the roots has a negative real part. It may
also be assumed, without loss of generality, that the coefficient of s» in
312 ELEMENTS OF MODERN CONTROL THEORY

the denominator is unity. We also add the restriction that the degree of
the numerator is at least one less than that of the denominator. In
summary, it is stipulated that

«0 = 10„=O. (87)

Under these conditions, it is always possible to write the input-output

dynamics of the linear element in the form(25>

x = Ax + b(p(<r) (88)
cr = — Xi

where x and b are n vectors and A is a constant matrix determined from

d = xi

Xf = x<+i+ b;<p (89)

i = 1,2, . ,i-l

Xn y] z-t-
1 Xi -j- bn^3 (90)
i=l

bQ = po = 0

bi = Pi — X ai-jbj (91)
i=o

i = 1, 2. . , n.

This method is illustrated by a simple example, as follows.

G(s) = - = - + g2
<P S2 + aiS + a2

From Eq. (91), we find

bi = (h — axb0 = ft
b2 = /32 — a2b0 — aibi = /32 — ai/3i.

Also, using Eqs. (89) and (90)

Xi — x2 +

X2 = a2Xi «1X2 T (/32 — aifii)<p.

THEORY OF NONLINEAR SYSTEMS 313

Expressed in matrix notation

Xi 0 1 Xi bi
=
+
_*2_ _ — a2 — «i_ _x2__ Xi 1—

0 — xi.

Notice that in the format of Eq. (88), the numerator dynamics (deriva¬
tives of the driving function) are effectively eliminated. In the analysis
to follow, we will consider a slightly generalized version of Eq. (88),
namely

x = Ax + h(p{<j) (92)
T
(J — V X

(v 5=a constant n vector).

Lur’e considered the dynamical system

x = Ax + b<p(<r)
\ = <p{o) (93)
a = vrx + y£.

He distinguished between indirect control (when 7^0) and direct

control (7 = 0). We will, in fact, show that these distinctions are
irrelevant and that Eqs. (93) are completely equivalent to Eqs. (92).
Suppose that 7^0. Differentiating the third of Eqs. (93) and
making use of the first two, we find

a%= vTx + 7£

= vrAx + (vrb + y)<p.

There is indeed no loss of generality in assuming that 7 = 0. For if

7^0, and we write, instead of Eqs. (93)

x = Ax -f- b<^>
T i(p

a = vTx

where f is a constant vector defined by

7 = vrf
314 ELEMENTS OF MODERN CONTROL THEORY

one finds that

or — VrX

— vr(Ax T hip -f- fip)

= vrAx + (vrb + y )ip.

Consequently, a nonzero y need not be considered explicitly if the b

vector is appropriately modified.
All of the analysis to follow will deal with the system of Eqs. (92),
with the nonlinearity satisfying conditions (85). By virtue of the stipu¬
lations placed on the linear element, U(s), the constant matrix, A, satisfies
the following.

1. All the eigenvalues of A have negative real parts.

2. No eigenvalue of A equals zero.
3. All the eigenvalues of A are distinct.

Some of these restrictions will be relaxed in Sect. 2.3.4.

The system (92), which is in the phase variable (canonical) form, may
be expressed in diagonal form by making the transformation

x = Ty (94)

where

1 1 . 1
Xi X2 . X„
^l2 ^22 . X„2
T = : : : (95)

n — 1 1 n — 1
X1 x2a~ \
n

This matrix is known as the Vandermonde matrix; the X, are the eigen¬
values of A. Asa result of this operation, we find

y = Fy + ni*>(<r)
(96)
T
<T = C y
THEORY OF NONLINEAR SYSTEMS 315

where

£ O
F = ^2 = T_1AT (97)
.0

m = T-!b (98)
/

c = Tr V. (99)

For the Lyapunov function, we take*

cr
j
V = yrPy + /
= <p(a)da. (100)

Taking the derivative with respect to time using Leibnitz’s formula, and
making use of Eq. (96), with a = cry, one obtains

V= yrPy+ yrPy+ *>(*)

—
dt
+ f ^dt
da
(101)
= yrPy + yrPy +

= — yrQy + <p(2mrP + crF)y + <^2crm

where

Q = - (FrP + PF). (102)

In seeking to establish asymptotic stability in the large, we must have

(by Theorem III)

V > 0

V < 0

V —» ooas IMl - > co.

If P is positive definite, and if Eq. (85) holds, the first and third conditions
are automatically satisfied. To satisfy the second condition, Q must be

*P is a constant symmetric matrix whose elements are, for the moment, undetermined.
316 ELEMENTS OF MODERN CONTROL THEORY

positive definite. It is knownO9) that if all the eigenvalues of F have

negative real parts, then in Eq. (102), if P is positive definite, so is Q,
and vice versa. One may, therefore, choose either P or Q in some appro¬
priate manner.
Following Lur’e, we let

Q = aar. (103)

The components of the vector, a, are chosen as follows. If X< = where

()* is used to denote complex conjugate, then a,- = a f. Otherwise, the
a/s are simple constants (as yet undetermined). Writing

r = — crm (104)

and substituting Eq. (103) into Eq. (101), and then completing the
square, results in

V = — [ary — <py/F]2+ y[2mrP + crF + 2ar\/r]y- (105)

If r > 0 and <p(o) = 0, then the above expression is negative definite if

2mrP + crF + 2arv/r = 0. (106)

Consequently, if a vector, a, can be found to satisfy this equation, we

may assert that the system (92) is asymptotically stable in the large (via
Theorem III).
To solve Eq. (106), it is necessary to solve Eq. (102) for P, where Q is
given by Eq. (103). This solution is( 2 7)

P = f eFT‘QeFldt.
Jo
(107)
Now

3Xlt
e* = 3X2t
0

It follows, therefore, that the (ij)th component of the matrix

eFTtqeFt
THEORY OF NONLINEAR SYSTEMS 317

IS

(X i+\ j)t

Furthermore

(X i+XyH dt 3j i&j
a,aJ
' (V + Xy)

Therefore, the (ij)th component of the P matrix is given by

P = _ a<a7 (108)
(Xj + Xy)

Various additional stability criteria may be developed by taking modi¬

fied forms of the Lyapunov function, Eq. (100), and placing additional
restrictions on the nonlinearity. The reader is referred to Letov<21) for
details. We now illustrate the application of this method by a simple
example.

Example 13: The system block diagram is as shown in Fig. 2.51,

and the nonlinearity satisfies Eq. (85). The linear transfer function is

(s + 4)
G(s)
(s + l)(s + 2) <p

s + 4
s2 + 3s + 2

Using relations (89)— (9 1), we obtain

x = Ax + b<p

where

A =

b =
318 ELEMENTS OF MODERN CONTROL THEORY

and

a — vTx = — xi = —&

where

The eigenvalues of A are, in general, obtained from (I = identity

matrix)

|A - AI| = 0.

However, in the present case, these eigenvalues are the poles of the
open-loop transfer function, G(s), viz.

Ai = -1

A2 = — 2.

Calculating the Vandermonde matrix

and its inverse

-1

we then have

from Eq. (97), while

3
m

-2
THEORY OF NONLINEAR SYSTEMS 319

1
c =

from Eqs. (98) and (99), respectively. Furthermore, via Eq. (104), we find

r = — c7'm = 1

and

6a !2 4aia2
P
12
4aia2 3a 22_

using Eq. (108).

Finally, in order to satisfy Eq. (106), and, therefore, to establish that
the system is completely stable (asymptotically stable in the large), there
must exist real constants, ax and a2, satisfying

ai2 -f - (2 — 3a2)a! + I =0
3 3

ai2 — 2(a! -(- l) a2 — 2 — 0.

This is indeed the case, since we find that ai = -0.98 and a2 = -1 .41 .
For higher-order systems, it becomes increasingly more awkward to
solve Eq. (106) for the vector a. For an nth order system, the n com¬
ponents of the vector, a, are to be obtained by solving n simultaneous
algebraic equations. This presents serious difficulties at the computa¬
tional level. An approach offering some advantages in this respect is the
following.
We assume that

7i
Q = (109)
0

Here the t.-’s are positive constants. Substituting this expression for Q in
Eq. (101), and completing the square, yields

—V = (y + <pQ-1u)rQ (y + <pQ-1u) + (r - u^Q-' u) <p2 (110)

320 ELEMENTS OF MODERN CONTROL THEORY

where

(111)
u = — (Pm + - Fc)

and r is as defined by Eq. (104). Since Q as defined by Eq. (109) is

obviously positive definite, the right-hand side of Eq. (1 10) is positive if

r ^ ur Q-1 u . (112)

This condition replaces Eq. (106) as the requirement for complete

stability. The essential difference is that Eq. (1 12) is a scalar inequality
rather than a set of simultaneous algebraic equations.
A slight complication arises when some of the eigenvalues of the state
matrix, A, are complex. For y < in Eq. (109) positive, this will result in
some of the components of P, as calculated from Eq. (102), being com¬
plex, with the result that some of the components of u, obtained from
Eq. (Ill), are also complex. This simply means that in Eq. (112) one
simply uses the Hermitian form, u*Q-1u, rather than urQ_1u. Nothing
else is changed.
The following example illustrates the application of the method.

Example 14: We consider the system whose nonlinearity satisfies

Eq. (85) and which is described by

X = Ax -f bip(a)

a = VrX

where

0 1 0
A 0 0 1
-30 -31 -10

b =

-1
v -1
0
THEORY OF NONLINEAR SYSTEMS 321

The eigenvalues of A are

Xi = -2

X2 = -3

X3 = -5

so that

1 1 1
T = -2 -3 -5
4 9 25

30 16 2
1
T"1 = -30 -21 -3
6
6 5 1

and

8
1
m = T lb = -9
6
1

r = — crm = 8.

Taking Q as

0
72

7s.

where the y-s are (as yet undetermined) positive constants, we solve for
322 ELEMENTS OF MODERN CONTROL THEORY

P either from Eq. (107) or Eq. (102). The result is

0
l 7i
~2Xl

P =
1
~ 2 X2

The components of the vector, u, are now obtained from Eq. (11 1);
viz.

1
di = ~ (mt- Ti + X,)2

i = 1, 2, 3

where m, denotes the ith component of the vector, m.

Consequently, the inequality, Eq. (112), becomes

v (7! + 3y (72 - 6y (73 + 100)2

8> 9yx + 16y2 + 100y3
One set of positive 7/s which satisfies this inequality is

71 = 1

72 = 6

73 = 50 .

We conclude, therefore, that the given system is completely stable.

The Lur’e method represents an important step in the development of
a fairly general theory of stability for nonlinear systems.
The technique can be applied to moderately high order systems des¬
cribed by the format of Eqs. (92) with the nonlinearity satisfying Eq.
(85). A more detailed description of the nonlinear element is not re¬
quired. On the other hand, phase plane methods are limited to second
THEORY OF NONLINEAR SYSTEMS 323

order systems, while the describing function technique requires a fairly

precise description of the nonlinearity.
There are several severe limitations, however. It is possible that the
criteria of Eqs. (106) or (112) will fail to be satisfied even if the system
is stable. In this case various modifications of the assumed Lyapunov
function, Eq. (100), may be taken, together with the application of
additional restrictions on the nonlinearity. This will yield slightly dif¬
ferent stability criteria, which may be satisfied where others might not.
Several such approaches are considered by Letov.G1)
Nevertheless, there are restrictions of a fundamental nature such that
stable systems will always be rejected by the Lur’e method. Consider,
for example, a linear element described by

<p = ktr

(0 < k < oo) .

This obviously satisfies Eq. (85). It is apparent, therefore, that in the

case of a linearized system, the Lur’e stability criteria would select as
stable only those systems that are stable for all positive values of the
open-loop gain. If the root locus of the open-loop transfer function,
G(s), is not confined to the left half of the s plane, a linearized system
will, for some positive value of the open-loop gain, be unstable. Hence,
the stability criteria developed in this section will reject all those systems
of the type shown in Fig. 2.52.
It must be emphasized, however, that the fact that the linear portion
of a system with a single nonlinear gain element is confined to the left
half of the s plane does not imply that the system is stable. Conversely,
the fact that the linear portion of a system with a single nonlinear gain
element is not confined to the left half of the s plane does not imply that
the system is unstable.
Based on the foregoing considerations and the assumptions inherent in
the Lur’e method, we may summarize the reasons why stable systems will
be rejected by this method as follows.

a. The root locus of the linear part of the system is not confined to the
left half of the s plane.
b. There are open-loop poles at the origin of the s plane.
c. The open-loop function has multiple poles.
d. The constant, r, in Eq. ( 106) or Eq. ( 1 12) is negative.
324 ELEMENTS OF MODERN CONTROL THEORY

a. Stable for low values of gain

Im

Im

o. Stable for intermediate values of gain

Figure 2.52. Root Loci for Certain Linear Systems.

An approach which broadens the scope of the Lur’e method con¬

siderably, and which overcomes most of the above limitations, is de¬
scribed in the following section.

2.3.4 Pole and Zero Shifting

Some of the most serious limitations in the Lur’e method may be
removed by adopting an approach developed by Rekasius and Gibson/22)
THEORY OF NONLINEAR SYSTEMS 325

The pole shifting technique may be used to:

a. Analyze a system whose open-loop gain does not fall below a certain
minimum value.
b. Separate multiple open-loop poles.
c. Analyze systems with open-loop poles in the right-half plane.
/

Essentially, what is involved is an appropriate change of variable such

that the transformed system satisfies the restrictions of the Lur’e method.
We will consider the system depicted in Fig. 2.51 and described by Eq.
(92). It is assumed that the nonlinearity satisfies the inequality

a(f > ac2 (113)

where a is a positive constant.

In other words, the nonlinearity is confined to the first and third
quadrants such that |^| always exceeds a |<r|. (See Fig. 2.53.) Now define
a new variable, <p', by

<pr = <p — acr . (114)

Substituting this in Eq. (1 13) yields the new condition

(up1 > 0 (115)

Figure 2.53. Characteristics of Nonlinear Element.

326 ELEMENTS OF MODERN CONTROL THEORY

Figure 2.54. Revised Block Diagram in Pole Shifting Method.

which obviously satisfies Eq. (85). The revised block diagram is shown
in Fig. 2.54. If we now deal with the new nonlinear parameter, <p', de¬
fined by Eq. (114) and the new linear transfer function

G(s) (116)
G(s) ~ 1 + aG(s)

it is found that all of the stipulations of the Lur’e method may be

satisfied. Specifically, we note that if

N(s)
G(s) =
D(s)

where N(s) and D(s) are polynomials in s, then

-G(s) _~ D(s)
N(s)
+ <*N(s)'

It is obvious that an arbitrarily small value of a will separate multiple

poles and that increasing values of a will, in most cases of interest, shift
a pole in the right-half plane into the left-half plane. Consequently the
pole shifting technique may be used to prove the stability of systems
where the Lur’e method fails.
The zero shifting technique may be employed to study the stability
of systems whose open-loop gain does not exceed some specified maxi¬
mum. In particular, suppose that the nonlinearity satisfies the inequality

0 dip < |3cr2 (117)

where is a positive constant. (See Fig. 2.55.)

THEORY OF NONLINEAR SYSTEMS 327

Figure 2.55. Characteristics of Nonlinear Element.

Defining a new variable by

a (118)

and substituting back in Eq. (117) yields

trV > o .

However, the Lur’e method cannot be applied directly, since the

input to the nonlinear element, a', is no longer given by the second of
Eqs. (92). (See Fig. 2.56.) A Lyapunov function other than Eq. (100)
must be tried. If we select

V = yrPy

yO <s)

I - 1

Figure 2.56. Revised Block Diagram in Zero Shifting Method.

328 ELEMENTS OF MODERN CONTROL THEORY

where y represents the canonic state vector in the system described by

Eq. (96), and P is a constant symmetric matrix, then we find

V = - yrQy + 2ymTPy

where Q is given by Eq. (102). As in the Lur’e method, we take Q to be

of the form Eq. (103). Completing the square in V then yields

V = — (ary)2 + <p(2mrP) y .

The resulting stability equation becomes

mTP = 0 . (119)

By using pole shifting combined with zero shifting, an even wider

range of systems may be analyzed. Reference 22 contains a variety of
examples together with several modified Lyapunov functions.

2.3.5 The Variable Gradient Method

The usual methods of Lyapunov stability analysis begin with an as¬

sumption of a tentative Lyapunov function, from which an attempt is
made to determine those conditions ensuring that the system is, in some
sense, stable. The selection of a suitable Lyapunov function, appropriate
to a given system, is a task that generally taxes the ingenuity of the
analyst. In this section, we consider methods for generating Lyapunov
functions systematically. Various studies in this connection have been
reported by Szego(28> and Ingwerson/29) but the Variable Gradient
method, developed by Schultz and Gilbson,(23) is by far the most power¬
ful. We will consider this method in some detail.
It is known that if a system is stable in state space, then a Lyapunov
function, V, exists for this system. (18> Furthermore, if V exists, then its
gradient, W, also exists. Given this gradient, both V and V may be cal¬
culated. We note that when V is given one may write

dV dxj

where forx,-, we insert the system equations. Now this may be expressed
as

V = (VV)rx . (120)
THEORY OF NONLINEAR SYSTEMS
329

To find V from VV, we write

rX fX
V = J/0 (VV)Tdx
= J0/ VV-dx. (121)

This is a line integral to an arbitrary jaoint in the state space and is inde¬
pendent of the path of integration. The simplest such path is the fol¬
lowing.

• •, 0)d7!+ f V2(x!,72,0, . , 0)dy2

•'o

(122)
n

(^i> ^2? . > “yTij&yn

Jo

In this expression, V,-( ) stands for the ith component of the vector, VV.
To obtain a unique scalar function, V, by a line integration of a vector
function, VV, it is necessary that(3°)

V X VV = 0 . (123)

The core of the Variable Gradient method is contained in the following

assumption for the form of the vector, VV.

aiiXx + 01x2X2+ . + oi„x„

<221X1+ 022X2 + . + 02 nx*
w = .
(124)

ajiiXi + 0*2X2 + . + annxn

The coefficients, 0,7, are assumed to be made up of a constant portion

and a portion which is a function of the state variables, viz.

ai j <Xijk~b a (x) . (125)

Several significant properties of the an emerge upon examination of Eq.

(124). First of all, the solution of a given problem may require that the
ith component of VV, i.e.

Vi = Oil Xx 4* 0,2 X2 + ~b a in X*
330 ELEMENTS OF MODERN CONTROL THEORY

contain terms that have more than one state variable as factors. Evidently
such terms may be determined from terms such as ai3 (x) x;; therefore,
the a Unix) may be taken without loss of generality as aiill{xl). Further¬
more, for V to be positive definite in the neighborhood of the origin, aiik
must always be positive. Also, if the relation

lim V(x) - > oo

INI - * °°

is to be satisfied, then must be an even function of x< and greater

than zero for large x*. In case aiik = 0, then, for the same reason, we
must have at-iM(x;) even and greater than zero for all x,-. Additional re¬
strictions on the an are imposed by the curl equations, (123). Finally,
the an must be so chosen as to make V at least negative semidefinite.
In practice, the procedure is often simpler to apply than the above
exposition would indicate. To begin with, one may assume the a,/s are
constants, using state-dependent ai3's only if needed to satisfy the curl
equations, or to make V negative semidefinite.
The formal application may be summarized as follows.

a. Assume VV of the form shown in Eq. ( 124).

b. From VV determine V via Eq. (120) and the state equations of the
system.
c. Constrain V to be at least negative semidefinite.
d. Determine the remaining coefficients, aih via Eq. (123).
e. Recheck V, since Step d may have altered the V determined in Step c.
f. Determine V via Eq. (122).
g. Check system stability by any of the theorems of Sect. 2.3.2.

We illustrate this procedure by a simple example.

Example 15: The system to be analyzed is shown in Fig. 2.57. The

nonlinear element is described by

<p = <r\p(a)

where ^(<r) is some arbitrary function of <j. We seek to determine the

restrictions on ^(o-) and the zero, p, of the linear transfer function, such
that the system is asymptotically stable in the large. This particular
problem is treated by Schultz and Gibson, (23) and among their derived
stability conditions is that the slope of the nonlinearity must be positive.
THEORY OF NONLINEAR SYSTEMS 331

Figure 2.57. Control Loop for Example 15.

It will be shown here that this requirement is, in fact, unnecessary. The
differing results stem from the fact that the equations of Schultz and
Gibson contain derivatives in the forcing function, which may be elimi¬
nated by making the transformation of variables described in Sect. 2.3.3.
Proceeding in this way, we write the state equations as

xi = x2 — h(xO xi

x2 = — x2 — (0 — 1) h (xj.) X!

where

6 — Xi = — a

h(xi) - - \K-xi) %

From Eq. (124), we have

auXi + ai2x2
VV
c*2iXi T 2x2

where we have arbitrarily set a22 = 2. Experience has shown that this
particular selection for a »* is appropriate for most systems of practical
interest. From Eq. ( 120), we find

V = [an — a2i — i/'(o)ai2 — 2 i/f (cr)(/3 1)] Xi X2

—iA (a) [an + (/3 — 1) 021! Xi2 — (2 — ai2) X22%

To satisfy Eq. (123), we must have

332 ELEMENTS OF MODERN CONTROL THEORY

or

a 12 = <*21

If we put

<212 = 0

and

an = 203 - 1) * to

then the expression for V becomes

V = - 2x22 - 2(j8 - 1) tA2(a)x!2

which is obviously negative definite if 0 > 1. We now have

'2*to(l8 - l)xi'
VV =

2x2

Determining V via Eq. (122), it is found that

V= x22+ 2(0- 1)Jo[ h(xi)xidxi.

Now for V > 0 and V < 0, it is sufficient that

0 > 1

h(xi)xi > 0 .

Furthermore, the condition

lim V(x)— > oo

X — > oo

is obviously satisfied. Theorem Ill of Sect. 2.3.2 enables us to conclude,

therefore, that the given system is asymptotically stable in the large if

P > 1
THEORY OF NONLINEAR SYSTEMS 333

and

h(xi)xi > 0 .

2.3.6 Miscellaneous Methods

The specific means of applying the" Lyapunov theory discussed in the

previous sections are perhaps the simplest and most useful presently
available. There exists, however, an extensive literature dealing with
specialized Lyapunov functions applicable to particular systems. Even an
abbreviated discussion of all of these techniques is beyond the scope of
the present study. A few are nevertheless worthy of passing notice. The
Zubov methods l) is an elegant means of constructing Lyapunov func¬
tions; this method is, however, dependent on solving certain partial
differential equations whose complexity depends on the system being
analyzed. It is, therefore, useful only for simple systems of low order.
Puri and Weygandt(32) developed a method based on analogy with Routh’s
canonical form for linear systems. A concept of some merit, due to Reiss
and Geiss,(32) involves the construction of Lyapunov functions, from an
assumed V, by integrating by parts. A recent study by Brockett(34)
develops new stability criteria for linear, stationary systems with a non¬
linear gain feedback. These criteria involve only the frequency response
of the linear portion and the maximum value of the nonlinear gain.
In this section we will conclude the study of Lyapunov stability theory
by considering two cases of special interest. The first of these is a very
general result which may be derived with very little effort. We consider
the system

x = f (x)
(126)
f(o) = 0 .

As a Lyapunov function, we take

V = xrQx (127)

where the components of Q are constants. Then we have

V = frQx + xrQf . (128)

334 ELEMENTS OF MODERN CONTROL THEORY

By carrying out the computation for — [f(ax)] it may be verified that

f(x) = f J(ax)xda (129)

Jo

where J(x) is the Jacobian matrix of f(x). Substituting this in Eq. (128),
we find

V= f xr[Jr(ax)Q
+ QJ(ax)]xda. (130)
Therefore, by Theorem III of Sect. 2.3.2, we may conclude that the
system (126) is asymptotically stable in the large if for some positive
definite matrix Q, the matrix [JT (x)Q + QJ(x)] is negative definite for
all x ^ 0.
As usual, the selection of a suitable matrix, Q, for the problem at hand
is the core of the matter, requiring a judicious blend of experience and
ingenuity.
We conclude our discussion of the Lyapunov theory by taking note of
some deceptive facets of the stability problem for linear nonstationary
systems. It is common practice, in the design of autopilots for launch
vehicles, to take so-called “time slices”; in effect, a linear, time varying
system is analyzed by assuming that it is valid to consider the system as
non-time varying during preselected small time intervals. Essentially, the
assumption may be stated as follows. If, in the system

x = A(t)x (131)

the eigenvalues of A(t) have negative real parts for all t, then the system
may be presumed stable. That this premise is not true in general may be
demonstrated by the following counter-example due to Zubov/35) For
the system described by

Xi = (12 sin 6t cos 6t — 9 cos2 6t — l)xi

+ (12 cos2 6t + 9 sin 6t cos 6t)x2

x2 = (9 sin 6t cos 6t — 12 sin2 6t)xx

— (12 sin 6t cos 6t + 9 sin2 6t + l)x2 .

The eigenvalues are — 1 and —10 for all t. Yet the solution is
THEORY OF NONLINEAR SYCTEMS 335

Xi = ai exp (2t) (cos 6t + 2 sin 6t)

+ as exp ( —13t) (sin 6t —2 cos 6t)

x2 = ai exp(2t) (2 cos 6t — sin 6t)

+ ct2 exp( —13t) (2 sin 6t + cos 6t)

which is obviously unstable. ,

The Lyapunov theory may be applied in straightforward fashion to the
system described by Eq. (131). LetC(t)be a positive definite matrix for
all t. As a Lyapunov function, we take

V = xr C(t)x . (132)

Taking the derivative with respect to time

V = x^ (ArC + CA + C) x = xrBx . (133)

To prove stability, one must select a positive definite matrix C(t) such
that V, as given by Eq. (133), is negative definite. In certain simple
cases, some useful results may be obtained as follows. Let the system of
Eq. (131) be of second order with the matrix, A(t), given by

0 1

A(t) =
ai a*.

where ai and a2 are functions to t. Assume that the matrix C(t)has the
form

1 0
C(t) =
0 c22(t)

It is required that C22(t) > 0 for all t. It follows that

V = 2(1 + ai c22) xi x2 -j- (2a2 c22 T c22)x22.

This expression will be negative semidefinite if

1 aic22 — 0 (134)
336 ELEMENTS OF MODERN CONTROL THEORY

and

2a2C22 T 622 < 0 . (135)

From Eq. (134) we obtain

1 (136)
C22 = - %
ai

Substituting this in Eq. (135) yields

2a2 ai (137a)
— + — 2 < 0.
ai ai

Furthermore, the requirement that C22(t) be positive for all t means that

(137b)
— > 0 .
ai

From the stipulations imposed on C(t), we have

lim [V(x)j —> =0 . (138)

llxll —* 00

We will now show that V f 0 along any solution of the given system
other than the origin. In fact

(2a2 ai\
- — + — ) x2‘
ax a /I

which is zero only along the xi axis. But the slope of the system tra¬
jectory is

dx2
— ai - i aj
dx! x2

which is infinite for x2 = 0. Therefore, the xi axis cannot be a trajectory of

the system. This proves the assertion. Invoking Theorem IV of Sect.
2.3.2, we may conclude that if the inequalities

1 (139)
- — > 0
THEORY OF NONLINEAR SYSTEMS 337

and

_2a2+ ^<o (140)

ai ax2

are satisfied, then the system is asymptotically stable in the large.

A result of considerable interest for launch vehicle control systems
can be derived from the above analysis. We take a simplified version of
an autopilot-controlled space launch vehicle, whose geometry is depicted
in Fig. 2.58, with the control loop schematic, as shown in Fig. 2.59.

Figure 2.58. Geometry for Simplified Attitude Control System.

Figure 2.59. Control Loop for Simplified Attitude Control System.

 ELEMENTS OF MODERN CONTROL THEORY
338

The symbols have the following meanings.

I = vehicle moment of inertia, slug-ft2

Ka = servo amp. gain, N. D.
KR = rate gyro gain, sec
4 = thrust moment arm, ft
4 = aerodynamic moment arm, ft
La = aerodynamic load, lb/rad
s = Laplace operator, sec- 1
Tc = thrust, lb
5 = thrust angle
6 = pitch angle
Hc = Tc 4/1
/xa - La fa /I

All of the quantities Tc, La, I, 4, and 4 are known functions of time.
From Fig. 2.59, we find (writing xi = 0)

Xi — X2

x2 = ai (t) xi + a2 (t) x2

where

ai(t) — Ma(t) K-x Mc(t)

a2(t) = KaKb Mc(t) .

The foregoing analysis is directly applicable. We find, in fact, that the

stability requirements are:

K aMc — fia > 0

2K AK R)Xc>
(KaHc — Ha)

The first of these is the usual condition derived from linear stationary
analysis. The second condition displays quantitatively the destabilizing
influence of rapid increase in/i«.
THEORY OF NONLINEAR SYSTEMS 339

2.4 ON-OFF CONTROLLERS

One important class of nonlinear control systems is characterized by

the fact that the nonlinear element switches discontinuously between
extremes. These systems are variously called on-off, bang-bang, or relay
type. They find wide application in such areas as roll control for
boosters, attitude control for satellites-and re-entry vehicles, etc.
The use of phase plane methods is appropriate for the analysis of these
systems, and the techniques of Sect. 2.1 are directly applicable. How¬
ever, there are many properties of on-off controllers meriting their study
as a separate topic. This will be done in the present section.
A very general type of on-off control system is shown in Fig. 2.60.
The properties of the relay are shown in Fig. 2.61. We will proceed to
the analysis of this system from its simplest form, adding various com¬
plexities progressively and finally giving a solution for the complete
system. In this way, the manner in which each of the system parameters

TIME
DELAY LOAD

Figure 2.60. A General Form of On-Off Control System.

f #3)

Figure 2.61. Relay Characteristics.

340 ELEMENTS OF MODERN CONTROL THEORY

(such as lead filter, time delay, dead zone, and hysteresis) affect the total
performance will be made evident.

2.4.1 Ideal Relay

The simplest form of the system of Fig. 2.60 is with no lead filter or
time delay and with no dead zone or hysteresis in the relay. In other
words, Ti = r = h = ju = 0. We will also assume that there is no input,
i.e., dc = 0. The system equation then takes the form*

Ie + M sgn e = 0 (HI)
and the initial conditions are

«(o) = fQ

e(o) = <r0.

Whene > 0, Eq. (141) becomes

M (142)
e
I 1

Integrating once, we obtain

M (143)
- y (t - ai)
where

Performing a second integration yields

M (144)
e - a2 = - y (t - ai)

where

* sgn t = 1 if e > 0
= 1 if e < 0 .
THEORY OF NONLINEAR SYSTEMS 341

Eliminating (t - ai) between Eqs. (143) and (144) results in

2M
<2= ~ ~Y (* “ a^- (145)

This equation represents the system trajectory in the phase plane. It is a

parabola with vertex at e = a2, <!= 0, and focus at e = a2 - i = 0.

When 6 < 0, the motion is governed by

M
e = 7-

Proceeding in the same manner as before, we find, for the equation of

the system trajectory in this case

2M
(e-a'2) (146)

where

Equation ( 146) represents a parabola similar to Eq. (145), but of opposite

concavity.
The complete trajectory of the system appears as shown in Fig. 2.62.
It is merely a limit cycle whose amplitude depends on the initial condi¬
tions.

2.4.2 Relay with Dead Zone

If the relay has a dead zone of magnitude 2n, then the trajectory, Eq.
(145), is valid as long as /3 > ^. When —n<p<n,M = 0, and the tra¬
jectory in the phase plane is described by « = h, where a is the value of
6 at Point 0 in Fig. 2.63. Since at this time e = ht + eh it is inferred that
e increases for positive h and vice versa. There is now no difficulty in
completing the phase plane representation for this case, as shown in Fig.
2.63. Again the magnitude of the limit cycle depends on the initial
conditions.
 ELEMENTS OF MODERN CONTROL THEORY
342

Figure 2.62. Phase Portriat for Ideal Relay Control System.

Figure 2.63. Phase Portrait Showing Influence of Dead Zone.

2.4.3 Ideal Relay with Lead Circuit

In this case, n = h = r = 0, and Tx is finite. We have

f = M sgn (Ti e + «).

THEORY OF NONLINEAR SYSTEMS 343

Consequently, when (Ti t + «) is positive, the motion follows Eq. (145).

M changes sign when

Ti e + « = 0 (147)

after which the motion proceeds per Eq. (146). In this way, the phase
plane trajectory of Fig. 2.64 is constructed. When the trajectory reaches
Point CD in this diagram, an ambiguity arises. The motion tends to con¬
tinue in that portion of the phase plane to the right of the switching line.
However, the tangent to the trajectory parabola has a slope whose mag¬
nitude is less than that of the switching line. The parabola representing
motion to the right of the switching line appears to the left of the
switching line. This is shown by the dotted portion in Fig. 2.64. There
is apparently no way to describe the motion beyond this point (which is
obviously not a point of equilibrium). This paradox is, in fact, a result
of the assumption of a perfect relay, where it is implicitly assumed that
changes of state may take place at infinite speed. By introducing a small
time lag, it may be shown tnat the trajectory converges to the origin

Figure 2.64. Phase Portrait Showing Effect of Lead Circuit.

344 ELEMENTS OF MODERN CONTROL THEORY

exhibiting the well-known high frequency, low amplitude oscillation

(chatter). This will be done in the next section.

2.4.4 Relay with Lead Circuit and Time Delay

The situation to be considered here is the same as that in the previous

section but with the addition of a finite time delay, r. To construct the
phase plane portrait for this case, we need to develop the equations of
the “delayed” switching lines.
Equation (147) gives the ideal switching line when r = 0. However,
when r is finite, actual switching occurs at some later time. Let time,
ti , denote the time corresponding to ideal switching and write

«i — e(ti) (148)

<=i— <(ti) . (149)

We may determine ti as follows. From Eqs. (144) and (143)*

(150)

M
(151)

Since these values of a and «i must satisfy Eq. (147), we find

(152)

The positive sign is used in front of the radical since tx is positive. Now
the time at which actual switching occurs is given by

t2 — ti T T. (153)
We write

«2 — *(^2)

«2 — *(^2) •

For simplicity we have assumed that e 0 = 0.

THEORY OF NONLINEAR SYSTEMS 345

Substituting in Eq. (150)

M
^ ~ ~ 21

M
~ ~ 21 ^ r^2 c° '
/

Using the value of ti given by Eq. (152), we find

( Mr2\
1 Y*+Tr) / 2i Y/2 (154)
M(Ta- t) + 11“ V1 + M to) %
From Eq. (151)

«2 — — — (ti + r)

M / 21 \1/2 “I
I (Tl+ m'")+TJ-
—Ti + ( T x2+

Using Eq. (154) this becomes

1 Mr (155)
«2 = «2+ OT (2Tl — t)
(Tj - r) [_' 1 21

Equation (155) represents the delayed switching line when M changes

sign from positive to negative. To obtain the equation for the delayed
switching line when M changes sign from negative to positive, we proceed
in identical fashion using Eq. (146) instead of Eq. (145) to obtain an
equation analogous to Eq. (150). The procedure is otherwise the same.
We obtain finally

_ i_ Mr (156)
«2 62 - (2Tl
(Ti - t)

Figure 2.65 depicts the character of the phase plane trajectory for this
case. We have indicated by U and the delayed switching lines corres¬
ponding to Eqs. (155) and (156), respectively. The case shown is typical
for the condition, Tx^> r. When r is on the same order of magnitude as
Ti, a limit cycle of the form shown in Fig. 2.66 is produced.
 ELEMENTS OF MODERN CONTROL THEORY
346

DELAYED
SWITCHING

Figure 2.66. Limit Cycle Produced by Large Time Lag.

2.4.5 Relay with Dead Zone and Hysteresis

The characteristics of the relay were shown in Fig. 2.61 , and we assume
that the lead time constant is finite. For the present we assume that t = 0.
THEORY OF NONLINEAR SYSTEMS 347

Let (Ti € + e) be positive and decreasing. Then the motion follows Eq.
(144). When

Tlt + 6 = M (157)

f switches to zero and the trajectory is described by e = e%,where ti is

the time at which the switching occurs. ^When

Tie + e = - h (158)

f switches to —M. Continuing in this way, we construct the phase plane

portrait of Fig. 2.67. Here, lines h and h represent the switching lines,
Eqs. (157) and (158), respectively.
Using the approach developed in the previous section it is now a
straightforward procedure to include the effect of finite delay, t.
Figure 2.68 shows the construction of the phase plane portrait, and Fig.
2.69 depicts the terminal limit cycle for this case.

2.5 THE POPOV METHOD

For many years, most of the research studies in the area of nonlinear
controls have been concerned with extending the basic ideas of the

Figure 2.67. Phase Portrait for Relay with Dead Zone and Hysteresis.
 ELEMENTS OF MODERN CONTROL THEORY
348

Delayed
switching
lines

Figure 2.68. Construction of Phase Portrait for Relay with Dead Zone,
Hysteresis, and Pure Time Lag.

Slope =

Figure 2.69. Final Limit Cycle Resulting From Construction of Fig. 2.68.

Lyapunov theory and fitting it to the needs of modern automatic control

systems. In Sect. 2.3, we summarized some of the salient features of the
current state of the theory. As already noted, one of the chief problems
in practical application is the discouragingly complex algebraic manipu¬
lations required when the order of the system being analyzed is moder¬
ately high.
THEORY OF NONLINEAR SYSTEMS 349

In what is apparently one of the first papers to leave the Lyapunov

“fold” in studying nonlinear systems, Popov!36) derived stability criteria
in terms of frequency response. He considered systems which contain a
linear part and one nonlinear element subject to rather mild restrictions.
His results are otherwise very general.
The ch ef virtue in the stability criteria developed by Popov is that
useful results are easy to obtain even, for high order systems. Using a
computer, one may analyze systems of up to twentieth order— much in
the same way as linear systems.
In general, the systems which may be analyzed using the Popov
method encompass a much broader spectrum than those in the Lur’e
method. While they are not as completely general as the Lyapunov
theory, they are nevertheless of much greater utility in the areas where
they apply.
The system to be considered is described by Eqs. (92), which are
repeated here for reference.

X = Ax -f b<p{(r) (159)

cr = VrX (160)

It is assumed that all the eigenvalues of the constant matrix, A, have

negative real parts. This restriction will be relaxed in later sections.
Furthermore, <p(<r)is a continuous scalar function satisfying the condition

(161)
a

In what follows, we shall need an explicit expression for the transfer

function of the linear portions of Eqs. (159) and (160). This may be
derived as follows. Taking the Laplace transform of Eq. (159) yields

s x(s) = Ax(s) + b<p(s)

or

x(s) = (Is — A) 1 W>(s) .

Using Eq. (160 ),a = vrx

G<*>
- -^ - vr<ls
- A)_‘b- (162)
 350 ELEMENTS OF MODERN CONTROL THEORY

Popov’s result is stated in the following theorem.

Theorem: If, in the system described by Eqs. (159) and (160), the
following conditions are satisfied:

a. All the eigenvalues of A have negative real parts

<p((r)
b. 0 t < k

c. There exists a nonnegative real number q such that*

Re [(1 + jqo>)G(j«)] + y > 0 (163)

for all co > 0, where G is defined by Eq. (162),

then the origin, x = 0, is asymptotically stable in the large. t
The inequality, Eq. (163), may be expressed in several simplified
forms. Since the linear transfer function may be written in general as
the ratio of two polynomials in s, viz.

P(s)
G(s)
Q(s)

we have

_ Pl(<«))
+ jP2(to)
(JWj ~ Qi(co) + jQ2(co)

where the subscripts 1 and 2 are used to denote real and imaginary parts,
respectively. This reduces to

G(j«) — Gi(u) -f jG2(a>) (164)

where

PiQi + P2Q2
(165)
1_ Ql2+Q22

P2Q1 — P1Q2
(166)
2= Qi2+ Q22 '

*The symbol, Re [ ] , denotes “real part of.”

t Also called complete or absolute stability.
THEORY OF NONLINEAR SYSTEMS
351

As a result, the inequality, Eq. (163), becomes

Gj(w) — qco G2(w) > 0 . (167)

k

Further manipulations are possible in order to analyze complex high

order systems in a rather routine fashion. A discussion of these will be
deferred in order to show the application of the results presented thus
far in two simple examples.

Example 16: We consider a second order version of Eqs. (159) and

(160) with

0 1"
A =

-10 -7

b =

v =

A direct application of Eq. (162) yields

—<x(s) s + 3
G(s) = *>(s) = s2 + 7s + 10’

Figure 2.70 is a schematic of this system. We seek to establish an upper

Figure 2.70. Control Loop for Example 16.

352 ELEMENTS OF MODERN CONTROL THEORY

limit on k in

(T

which ensures complete stability of the system. From Eqs. (165) and
(166) we find

4a>2 + 30
Gl = a,4 + 29o2 + 100

—co(w2+11)
°2 = «4 + 29+ + 100

Substituting in Eq. (167)

4a>2+ 30 + qw2 (<o2+ 11) + l (w4 + 29o>2+ 100) > 0

k

we see that this inequality is satisfied for all w > 0, for any nonnegative
q, and 0 < k < °o. Consequently, this system is completely stable as long
as the nonlinear function is restricted to the first and third quadrants.

Example 17 : Consider a third order version of Eqs. (159) and ( 160)

where

1 0
A = 0 1
-29 -10

0
b = 0
1

v =

For third and higher order systems, the matrix inversion to be performed
in Eq. (162) becomes extremely awkward, and for all practical purposes
impossible, for only moderately high order systems. The essential dif¬
ficulty stems from the need to carry polynomials in s through the maze
THEORY OF NONLINEAR SYSTEMS 353

of the inversion process. We may circumvent this predicament by first

expressing the given system in canonical form in the manner described
in Sect. 2.3.3. Thus, by writing

x = Ty

where T is the Vandermonde matrix given by Eq. (95), we have, in the

present case
\i = —1

X2 = —4

X3 = 5

1 1 1
T = -1 -4 -5
1 16 25_

~-20 -9 -1
T-1 = - — 20 24 4
12
-12 -15 -3

and we find

y = Fy + rrup

a = cry

where

m = T-1 b

1
c = Trv = 1
1

Since (Is - F) is now a diagonal matrix, the inversion is trivial, and we

find from Eq. (162)
354 ELEMENTS OF MODERN CONTROL THEORY

G(s) = - cr(Is — F)_1m

_ _ 1_

(s + l)(s + 4) (s + 5)

From Eqs. (165) and (166)

Gi = 20(1 - w2) [400(1 - co2)2+ «2(29 - w2)2]-1

G2 = - co(29 - a>2)[400(1 - w2)2 + co2(29 - co2)2]-1.

Substituting in Eq. (167), we obtain, after some reduction

i +(f -qy+(f+ ^ - »> +(f +2°)>

°-
As in Example 16, we seek to determine an upper limit on k for which
the system is completely stable. Thus k is obviously dependent on the
choice of q. If q = 0, then the above inequality reduces to k < 41/20. A
brief inspection of the above inequality indicates, however, that an
optimum upper limit for k may be obtained by taking q = 342/k, in
which case we find k < 498.
It is instructive to compare this value with that for the corresponding
linear system obtained by replacing <p(<r)with <p = ko-. This situation is
depicted in Fig. 2.71, with its root locus shown in Fig. 2.72. Via con¬
ventional linear techniques we find that neutral stability occurs for
k = 560.
In the two examples here considered, it was easy to choose the param¬
eter, q, such that we obtained the maximum upper bound on k for the
system to be stable.
For systems of higher order, a somewhat greater effort is necessary to
do this but the procedure is still relatively straightforward. To aid in

Figure 2.71. Linearized Version for System of Example 17.

THEORY OF NONLINEAR SYSTEMS
355

doing this, the inequality, Eq. (167), is manipulated a little further. We

define

U = Gi(u) (168)

W = - coG2(co). (169)

The inequality, Eq. (167), may, therefore, be written as

U + qW + i > 0
(170)
q ;> 0, k > 0 .

If we replace the inequality in Eq. (170) by an equal sign

U + qW + - = 0 (171)
k

then it is easy to show that Eq. (170) represents those points in the U-W
plane which are to the right of the line, Eq. (171).
The procedure now consists of making a plot of G(jco) in the U-W plane
for 0 ^ w < oo. Popov<36) calls this the “modified phase amplitude char¬
acteristic” (MPAC). The line, Eq. (171), is now fitted to this curve such
that the curve remains wholly to the right of the line while the abscissa
356 ELEMENTS OF MODERN CONTROL THEORY

— 1/k is made as small as possible. Figure 2.73 illustrates the method for
a typical G(j«) locus.
It is apparent that the usefulness of the method is not seriously com¬
promised by taking systems of arbitrarily high order. The limiting factor
is computer time and storage, much the same as for linear system
analysis.
In the remaining sections devoted to the Popov method, we will relax
various restrictions and discuss some generalizations.

Figure 2.73. A Typical MPAC Locus Showing Determination of Stability Boundary.

THEORY OF NONLINEAR SYSTEMS 357

2.5.1 Singular Cases

In the discussion of the Popov method presented in the previous

section, it was specifically stipulated that the eigenvalues of the system
matrix must have negative real parts. Those cases in which the system
matrix had simple or multiple eigenvalues on the imaginary axis were
referred to by Popov as singular. Some of these singular cases have been
analyzed by Popov(37) and Yakubovitchd41) However, a simple and
elegant result has been obtained by Aizerman and Gantmacher,(39) which
includes the results obtained by these previous investigators as special
cases.
We consider again the system described by Eqs. (159) and (160). The
matrix, A, is permitted to have simple or multiple eigenvalues on the
imaginary axis. However, instead of Eq. (161), the nonlinearity is
assumed to satisfy

(172)
a

where e is an arbitrarily small positive number.* The basic result is the

following. If

a. Inequality ( 172) is satisfied.

b. The linear system obtained by substituting <p— e<ris stable.
c. There is a real, nonnegative number q such that

Re[(l + j«q) G(jco)]+ ^ > 0

holds for all w > 0.

Then the origin x = 0 of the system is completely stable.
Further analysis proceeds in a manner completely analogous to that
described in the previous section.

2.5.2 Forced Systems

In all the systems considered thus far, there were no external forces
applied. As a matter of fact, there are very few results available on the

* Essentially this means that the slope of the nonlinearity is not permitted to be zero at the
origin in the a — ip plane.
358 ELEMENTS OF MODERN CONTROL THEORY

stability of nonlinear systems excited by external forces. Consequently,

a recent study by Naumov and Tsypkin,(4°) which adopts the Popov
approach in the case where there are externally applied forces, is especially
significant. The results obtained are also characterized by a greater
generality in that the restrictions on the linear portion of the system are
greatly relaxed.
The criteria developed by Naumov and Tsypkin may be presented in a
fairly simple manner, and this will be the subject of the present section.
The system to be considered is of the form shown in Fig. 2.74. The basic
result is contained in the following theorem.

Theorem: The system of Fig. 2.74 is completely stable if the fol¬

lowing conditions are satisfied.

a. The applied force f(t) is bounded; that is, lim f(t) < <*>.
t - %º00

b. There exists a nonnegative number, r, such that

G(j«)
(173)
1 + rG(jo;)

for all w > 0.

^ d<p(a)
r + e * “ckT - e (174)

where « is an arbitrarily small positive number, and k is a positive

constant which satisfies Eq. (173).

Note that the poles of G(s) are not restricted to lie in the left-half plane.
This constitutes a significant generalization over the results given in the
previous two sections.

Figure 2.74. Nonlinear System with Forcing Function.

THEORY OF NONLINEAR SYSTEMS 359

In the case where all the poles ofG(s)do lie in the left-half plane, then
we may take r = 0. A further notable difference between the forced and
unforced systems is that the stability criteria for the former involve the
derivative of the nonlinear element rather than the function itself.
For purposes of practical application, the condition in Eq. (173) may
be manipulated to a form in which frequency response techniques may
be employed in a familiar manner. Defining a constant, B = k/r, we may
write Eq. (173) as

k G(jq>)
+ ^ 0
B + k G(jw) (175)

0 <C to < 00 .

Now let

k G(j«) = M(w) + j N(co) . (176)

Substituting this in Eq. (175) and replacing the inequality by an equal

sign, we find

[M(co) + ~ (B + 1)]* + N’(eo) = \ (B — ly . (177)

It is easy to show that this defines a family of circles passing through

the point, ( —1, jO), having a radius 1/2 (B — 1), and situated to the left
of the line, M(«) = —1. (See Fig. 2.75.) Via simple geometric arguments
it may be established that inequality, Eq. (175), is satisfied outside the
B circles.
Thus, for any given k, if the kG(jco) locus in the M-N plane is found
to be tangent to a certain B circle, say Bi, this determines a value of r,
which, in turn, defines the limits for d<p(cr)/do-in order to ensure complete
stability. The procedure is suggestive of the use of compensating filters
to shape the k G(jco) locus in the vicinity of the B circles in order to
enhance the stability properties of the system. Conventional linear
techniques are appropriate for this purpose.
360 ELEMENTS OF MODERN CONTROL THEORY

REFERENCES

1.Y. H. Ku, Analysis and Control of Nonlinear Systems, New York,

Ronald Press Company, 1958.
2. L. P. Grayson and E. Mishkin, Three Dimensional Phase Space
Analysis, Polytechnic Institute of Brooklyn, Microwave Research
Lab., Report R-741-59, PIB-669, May, 1959.
3. R. E, Kuba and L. F. Kazda, “A Phase Space Method for the
Synthesis of Nonlinear Servomechanisms,” AIEE Trans., Vol. 75,
Part II, November, 1965, p. 282.
THEORY OF NONLINEAR SYSTEMS 361

4. W. J. Cunningham, Introduction to Nonlinear Analysis, New York,

McGraw-Hill Book Company, 1958.
5. N. Minorsky, Introduction to Nonlinear Mechanics, J. W. Edwards,
Inc., Ann Arbor, 1947.
6. H. T. Davis, Introduction to Nonlinear Differential and Integral
Equations, New York, Dover Publications Inc., 1962.
7. R. E. Kalman, “Phase Plane Analysis of Automatic Control Systems
With Nonlinear Gain Elements,” AIEE Trans., Vol. 73, Part II,
1954, p. 383.
8. R. J. Kochenburger, “Frequency Response Method for Analyzing
and Synthesizing Contactor Servomechanisms,” AIEE Trans., Vol.
69, Part 1, 1950, p. 270.
9. J. G. Truxal, Control System Synthesis, New York, McGraw-Hill
Book Company, 1955.
10. R. Sridhar, “A General Method for Deriving the Describing Func¬
tions for a Certain Class of Nonlinearities,” IRE Trans, on Auto¬
matic Control, Vol. AC-5, 1960, p. 135.
11. R. J. Kochenburger, “Limiting in Feedback Control Systems,”
AIEE Trans., Vol. 72, Part II, 1953, p. 180.
12. R. Gran and M. Rimer, “Stability Analysis of Systems with Multiple
Nonlinearities,” IEEE Trans, on Automatic Control, 1965, p. 94.
13. H. G. Jud, “Limit Cycle Determination for Parallel Linear and Non¬
linear Elements,” IEEE Trans, on Automatic Control, 1964, p. 183.
14. R. V. Halstenberg, “Combined Hysteresis and Nonlinear Gains in
Complex Control Systems,” IRE Trans, on Automatic Control,
1958, p. 51.
15. A. D. Gronner, “The Describing Function of Backlash Followed by
a Deadzone,” AIEE Trans., Applications & Industry, Vol. 77,
1958, p. 403.
16. A. Gelb and W. E. Vander Velde, “On Limit Cycling Control
Systems,” IEEE Trans, on Automatic Control, Vol. AC-8, 1963,
p. 142.
17. A. Lyapunov, “Probleme General de la Stabilite du Mouvement,”
Annals of Mathematics Study No. 17, Princeton, Princeton Uni¬
versity Press, (This is a French translation of Lyapunov’s original
paper which was published in Russia in 1902.)
18. R. E. Kalman and J. E. Bertram, “Control System Analysis via the
Second Method of Lyapunov; I. Continuous Time Systems, II.
Discrete Time Systems,” Trans. ASME, Vol. 82, Series D, 1960,
p. 371.
362 ELEMENTS OF MODERN CONTROL THEORY

19. J. P. LaSalle and S. Lefschetz, Stability by Lyapunov’s Direct

Method with Applications, New York, Academic Press, 1961.
20. A. I. Lur’e, Certain Nonlinear Problems in the Theory of Auto¬
matic Control, Gosteknizdat, 1951 (Russian).
21. A. M. Letov, Stability in Nonlinear Control Systems, Princeton,
Princeton University Press, 1961 (a translation of the 1955 Russian
edition).
22. Z. V. Rekasium and J. E. Gibson, “Stability Analysis of Nonlinear
Control Systems by the Second Method of Lyapunov,” IRE Trans,
on Automatic Control, Vol. AC-7, 1962, p. 3.
23. D. G. Schultz and J. E. Gibson, “The Variable Gradient Method for
Generating Lyapunov Functions,” AIEE Trans., Vol. 81, Part II,
1962, p. 203.
24. J. P. Lasalle, “Some Extensions of Lyapunov’s Second Method,”
IRE Trans., Vol. CT-7, 1960, p. 520.
25. J. H. Laning and R. H. Battin, Random Processes in Automatic
Control, New York, McGraw-Hill Book Company, 1956.
26. J. T. Tou, An Introduction to Modern Control Theory, New York,
McGraw-Hill Book Company, 1964.
27. R. Bellman, Introduction to Matrix Analysis, New York, McGraw-
Hill Book Company, 1960.
28. G. P. Szego, “A Contribution to Lyapunov’s Second Method:
Nonlinear Autonomous Systems,” Trans. ASME, J. of Basic Eng.,
1962, p. 571.
29. D. R. Ingwerson, “A Modified Lyapunov Method for Nonlinear
Stability Analysis,” IRE Trans, on Automatic Control, Vol. AC-6,
May, 1961 .
30. H. Lass, Vector and Tensor Analysis, New York, McGraw-Hill Book
Company, 1950.
31. S. G. Margolis and W. G. Vogt, “Control Engineering Applications
of Zubov’s Construction Procedure for Lyapunov Functions,”
IEEE Trans, on Automatic Control, 1963, p. 104.
32. N. Puri and C. Weygandt, “Second Method of Lyapunov and Routh’s
Canonical Form,”/, of Franklin Inst., Vol. 276, 1963, p. 365.
33. R. Reiss and G. Geiss, “The Construction of Lyapunov Functions,”
IEEE Trans, on Automatic Controls, 1963, p. 382.
34. R. W. Brockett, “On the Stability of Nonlinear Feedback Systems,”
Proc. Joint Automatic Control Conf, Stanford, Calif., 1964, p. 288.
35. V. Zubov, Mathematical Methods for the Study of Automatic
Control Systems, New York, Pergamon Press, 1962.
THEORY OF NONLINEAR SYSTEMS 363

36. V. M. Popov, “Absolute Stability of Nonlinear Systems of Auto¬

matic Control,” Automation arid Remote Control , Vol. 22, 1961,
p. 857.
37. V. M. Popov, “A Critical Case of Absolute Stability,” Automation
and Remote Control, Vol. 23, 1962, p. 1.
38. E. N. Rozenvasser, “The Absolute Stability of Nonlinear Systems,”
Automation and Remote Control Vol. 24, 1963, p. 283.
39. M. A. Aizerman and F. R. Gantmacher, “On Critical Cases in the
Theory of Absolute Stability of Controlled Systems,” Automation
and Remote Control, Vol. 24, 1963, p. 669.
40. B. N. Naumov and Y. Z. Tsypkin, “A Frequency Criterion for
Absolute Process Stability in Nonlinear Automatic Control Systems,”
Automation and Remote Control, Vol. 25, 1964, p. 765.
41. V. A. Yakubovitch, “Absolute Stability of Nonlinear Control
Systems in Critical Cases,” Automation and Remote Control, Vol.
24, 1963, pp. 273 and 665.
42. J. C. West and others, “The Dual Input Describing Function and Its
Use in the Analysis of Nonlinear Feedback Systems,” Proc. IEE,
Vol. 103, Part B, 1956, p. 463.
43. J. E. Gibson, Nonlinear Automatic Control, New York, McGraw-
Hill Book Company, 1963.

PROBLEMS AND EXERCISES FOR CHAPTER TWO

1. Using either the isocline or the delta method, sketch the phase plane
trajectories for

2x
a. x 4- j—j + 9x = 0
|*|

x(0) = 0, x(0) = 3.

b. x + (1 + 4x2)x = 0

x(0) = 0, x(0) = 2.

c. x + ^x|x| + 2x|x| = 0

x(0) = 0, x(0) = 2.
364 ELEMENTS OF MODERN CONTROL THEORY

d. X- (1- | X2)X
+ X= 0
x(0) = 1, x(0) = 0.

e. x + 5 sin x = 0

x(0) = 0, x(0) = 0.75.

2. In each of the systems of Problem 1, locate the singularities and

determine the nature of the phase plane trajectory in the vicinity of
each singularity.

3. A certain feedback system is described by the following equations.

c + c + 4c = 4u

e = r — c

where r and c are the input and output signals, respectively. The
relation between u and e is given by

u = e —3 < e < + 3

= 3 e > 3

= - 3 e < -3

Assuming that r is a step input, determine the location and nature

of all singularities in the system and discuss the properties of the
transient response for large and small values of r.

4. Volterra’s equations of species survival are given by

x = aix -f a2xy

y = — a3y + a4-xy.

Determine the location and type of the phase plane singularities.

With ai = 3, a2 = 2, a3 = 5, a4 = 1, sketch some typical trajectories.
Reverse the sign of a3and indicate the nature of the solution in this
case.
 THEORY OF NONLINEAR SYSTEMS 365

5. A certain type of oscillator is described by the equation

x + 2f x + 16 x = 0

where

f = 10 -2 < x < 2

= 0 |x| > 2.

If x(0) = 0, discuss the nature of the transient response for large and
small initial deflections.

6. In the feedback system of Fig. 2.33, suppose that

G«=srhy H<s>
-1
where the nonlinear element is an on-off device of the type shown
in Item 23, Table 2.1. Determine the amplitude and frequency of
the steady-state limit cycle using the describing function method.
Take M = 1.

7. In Fig. 2.33, assume that

32
= s2 - 2.4s + 16 ' = s + 1

where the nonlinearity is a saturation device of the type shown in

Item 20, Table 2.1. Let M = 1 and m = 2. Employing describing
function arguments, discuss the nature of the steady-state solution.
Sketch the appropriate Nyquist loci as required. Are the general
features of the steady-state solution affected by the size of the
initial disturbances?

8. Suppose that the output, u, of a nonlinear element is related to its

input, e, by

u = e3+ e e2 .

Derive the describing function for this nonlinearity. (Hint: Proceed

from first principles using the Fourier series of the output in
response to a sinusoidal input.)
11.
In
Fig
2.5
12
Csu
th
tsy
dbh
366 ELEMENTS OF MODERN CONTROL THEORY

9. In the system of Fig. 2.33, take

_ 1
H(s) = 1.
= s(s2 + 1.2s + 9) ’

The nonlinearity is a relay with dead zone (Item 18, Table 2.1).
With the aid of appropriate loci in the Nyquist plane discuss

a. the possible existence of limit cycles.

b. the amplitude and frequency of the limit cycles as a function
of open-loop gain, dead zone, and magnitude of relay output.

10. The Nyquist curves in Fig. 2.34 are representative of a unity feed¬
back system where the linear transfer function has the form

_ K_
^^ (s + a)(s + b)(s + c)

and the nonlinearity is a relay with dead zone. Show how the
conclusions relative to limit cycle properties derived from Fig.
2.34, may also be ascertained from a root locus plot.

K
G(s)
s(s + 1)

and

(f = <72for a > 0

- — cr2 for a < 0.

Using the Lyapunov theory, determine the necessary conditions for

stability.

y f (1 + 0.5y)y + y = 0.

Determine the region of stability via the Lyapunov method.

 THEORY OF NONLINEAR SYSTEMS 367

13. Determine the region of stability for the system

x = —2x + xy2

y = -y + yx2.

14. The simplified equations of motion for attitude control of a satellite

are

I*xi — (I„ — Ljx2x3 = M,

IyX2 — (L — I*)xix3 = Mj,

IzX3 (lx IJxxX2 — Mz

where I*, Iy, lz, and M*, M„, Mz, are the moments of inertia and ex¬
ternal torques, respectively, about the indicated axes. The quantities
xi, x2, and x3 denote the angular velocities about the x, y, and z axes,
respectively. In order to eliminate the tumbling motion of the
satellite, it is proposed to use the control laws

Mj = — kiXj

M„ = — k2X2

Mz = — k3x3 .

Investigate the system stability using the Lyapunov method. (Hint:

Try a Lyapunov function of the form, V = I*xi2 -f I„x22+ Lx32.)

15. A single axis attitude control system for a satellite subject to gravity
gradient effects may be described by

6 = — {6 T K S') -I- sin 2d.

Using a Lyapunov function of the form

V = - (02 + 02) + 1 - cos 26

U!

investigate the stability properties of the system as a function of K.

368 ELEMENTS OF MODERN CONTROL THEORY

16. Show that the system of Fig. 2.51 with

K
G(s) = 7— ,— . , .
(s + a) (s + b)

where the nonlinearity satisfies the conditions of Eq. (85), is globally

asymptotically stable for all positive values of K, a, and b.

17. Referring to Fig. 2.5 1, let

s + 2
G(s) =
s2(s + 5)

and

ip = 100(1 + lOtrV.

Investigate the stability of the system using the Lur’e method in

conjunction with pole shifting.

18. Let

1
G(s)
s(s + 1)

and

(p = cr3

in Fig. 2.5 1.

Determine the stability properties of the system using the Variable

Gradient method.

19. For the system described in Example 14, Sect. 2.3.3, suppose that
the nonlinear element satisfies the condition

0 < < k.
a

Using the Popov method, derive an upper limit for k which ensures
absolute stability.
THEORY OF NONLINEAR SYSTEMS
369

20. In Fig. 2.74, suppose that

5(s + 1)
G(s)
s2(s + 3)

and

p = a3.

Discuss the stability features of this system in response to a step

input.
 Chapter Three
SENSITIVITY ANALYSIS

At the risk of belaboring the obvious, we may point out that a system
design and its hardware implementation are never completely identical.
Design values, when translated into equipment specifications, usually
include permissible deviations (tolerances) that reflect the degree of per¬
formance considered acceptable. Tolerances that are too “tight” are
generally wasteful of money and effort, while those that are too liberal
could compromise performance.
In control systems, once nominal values have been established for
component gains and time constants, there remains the problem of
determining the effect of deviations from them. A brute force approach
consists of what we may euphemistically call a “parameter study,” in
which all combinations of selected parameter values are examined with
respect to resultant system performance. When a large number of para¬
meters are involved, such a task becomes unmanageably large.
A rational approach to the problem is contained in the methods of
sensitivity theory. The concept of sensitivity is an old one in control
theory, and, indeed, one of the primary virtues of the principle of feed¬
back was that it provided a degree of insensitivity to parameter variations
in the forward loop. This idea is expanded upon in Sect. 3.1.1.
The modern theory of sensitivity is concerned with determining the
effects of parameter variations on system performance in a broad sense.
It embraces the usual frequency-response methods for single-input/single-
output systems, as well as state variable representations and sampled-
data, multivariable, and optimal control systems. The fact that significant
theoretical contributions still appear is indicative of the formative state
of the theory. Nevertheless, the foundation is sufficiently firm to be
extremely useful in system design.
An intelligent use of these methods provides a framework for the
rational specification of system component tolerances, and a quantitative
evaluation of how the system is affected by parameter deviations.
370
SENSITIVITY ANALYSIS 371

The inherent ability of a feedback system to minimize the effect of

parameter deviations has long been recognized. In fact, this reduction in
system sensitivity was originally employed as a quantitative measure of
the advantages of feedback. (2) The measure of sensitivity, as first de¬
fined by Bode(2) (and expressed in frequency-response terms), had re¬
mained unchanged for almost a decade. It is still a sound tool for
evaluating the performance quality of a-' feedback control system in the
light of sensitivity requirements.
With the emergence of some of the newer ideas in control theory
(root locus, state variable, sampled data, multivariable systems, etc.), a
need developed for an expanded notion of the classical concept of
sensitivity. Studies by Ur,(3> Huang,<4) and RungG) developed measures
of sensitivity for the closed-loop poles of a system in terms of open-loop
gain and of open-loop poles and zeros. Questions of sensitivity for
sampled data systems were considered by Lindorff,(14) and measures of
sensitivity for multivariable systems were developed by Cruzd13)
A basic contribution was made by Horowitz, (15,19) wh0 showed that
sensitivity analyses need not be restricted to small parameter variations.
A natural consequence of this result— namely, that a conventional linear
system could be designed to cope with large parameter variations—
inevitably posed the question of whether (in many cases) an adaptive
system is really superior to a well-designed conventional system. Two
studies(20.2i) in the literature apply Horowitz’s technique to control
problems that were thought to be incapable of solution by conventional
linear methods.
With the questions of control system sensitivity in virtually definitive
form at present, attention is being focused on sensitivity considerations
in optimal control systems, here the basic problem is essentially the
following. Given is a system for which an optimal control function has
been calculated. The control is optimal in the sense that a prescribed
function has been minimized (or maximized). The question arises, “How
do variations in system parameters affect the performance function?”
Very little work of any significance has been done on this problem.
Dorato(29) proposed a measure of sensitivity for this case in which he
also outlined a computational procedure. The numerical difficulties,
however, are formidable, and computer solutions are mandatory for any
except the most trivial cases. Pagurek(9,i2) considered the general prob¬
lem from a more unified point of view and defined a sensitivity function
as the derivative of the optimal performance function with respect to the
variable system parameter. His treatment has a theoretical elegance that
is potentially very useful. However, it also exhibits a formidable
372 ELEMENTS OF MODERN CONTROL THEORY

computational complexity, and, perhaps equally important, his sensi¬

tivity measure is such that a parameter variation may actually improve
the performance function. This would appear to be a semantic and
perhaps logical deficiency that is not in harmony with the usual inter¬
pretation of a sensitivity function. In this respect, the definition pro¬
posed by Rohrer and Sobral(28) is superior, in that any system deviations
lead to degradation of the performance function.
In short, sensitivity theory for optimal control systems is still in its
formative stages and has not yet reached the definitive form that char¬
acterizes feedback control systems.

3. 1 MEASURES OF SENSITIVITY

In general terms, the basic problem is to provide a quantitative

measure of the deviation of a system function when the elements that
comprise this function vary in some prescribed manner. This degree of
dependence, or sensitivity, is conveniently expressed in terms of the
ratio of percentage change in the function to percentage change in the
parameter, viz.

(1)

The form of this expression suggests the alternate definition

o y =
(2)
d(Aix)

Historically, this representation has been used to indicate the ability of

a feedback loop to decrease the sensitivity of the overall transfer function
to variations in the parameters of the open-loop function. However, the
basic idea is useful in studying the dependence of any system function
on any parameter. This requires a slightly generalized version of the
classical concept of sensitivity. In Sects. 3. 1.1-3. 1.5, various specialized
sensitivity functions, together with their respective interpretations and
areas of application, are developed.
SENSITIVITY ANALYSIS 373

3.1 .1 System Transfer Function

The classical feedback configuration is shown in Fig. 3.1. Let

T(s) = = G(s) (3)

R(s) 1 + G(s)H(s)

denote the overall system transfer function.

We are interested in ascertaining the change in T(s) due to small
changes in G(s). Using the sensitivity function defined by Eq. (1), we
have*

s T _ G dT 1 _ 1
G T aG ~ 1 + GH " 1 + L' (4)

A well-designed system will exhibit a low sensitivity over some pre¬

determined frequency range. Obviously, SGr —>0 for |GH| —> oo (a great
deal of feedback), while Sj —» 1 for |GH| —>0. With no feedback (H = 0)
S/ = 1.
In the case where SGr > 1, the system without feedback is superior to
one with feedback, as far as sensitivity to parameter changes is con¬
cerned. In other words, the use of feedback is no assurance in itself
that sensitivity to parameter variations is diminished. This idea is
intimately related to the fact that over a frequency range of interest
there may indeed be positive feedback. BodeO) has shown thatf

/ tn[ SGr(joj)]dw
=0
Jo

*The argument, s, will be dropped whenever it is convenient and where no ambiguity results,
f Assuming that the open-loop transfer function has at least two more poles than zeros, or,
equivalently, if | GH | -> 0 at greater than 6 db/octave.
374 ELEMENTS OF MODERN CONTROL THEORY

which means that in any practical system there is as much positive

feedback as there is negative feedback. The problem is, therefore, one
of specifying G and H such that, in the frequency band of interest,
purely negative feedback is obtained.
Consider, for example, the system of Fig. 3.1 with

K
G(s) = H(s) = 1 (5)
s(s + a)

and where a and K are positive constants. We readily find that

Sr _ 1 _ s(s + a)
(6)
1 + G(s) s(s + a) + K

In terms of frequency response, we have

o2(co2+ a2)
isGT(jco)i
= (7)
K2 - 2Kco2 + arV + a2)

a aco
4 S/(j«) = tan 1 — tan-1 — - — (8)
—Cl) (K — to2)

Thus

ISg7"
(jco)|< 1 for co2< —

Scf(jco)l> 1 for co2> — •

Obviously, the greater the value of K, the greater the frequency range
in which feedback is effective for reducing the sensitivity parameter
variations.
Evaluation of SGT(jw)is most conveniently accomplished from a Nyquist
plot of the open-loop transfer function. From Eq. (4)

1
SGr(jco) (9)
1 + L(jco)

The vector [1 -f L(jco)] is shown in the Nyquist plot of Fig. 3.2. It is

apparent that in this case, |i -(- L(jw)| > 1 for frequencies less than coi.
SENSITIVITY ANALYSIS 375

Figure 3.2. Nyquist Plot of Open-Loop Transfer Function.

This, in turn, means that |SGr(jw)| < 1 for < wi. Therefore, a well-
designed system, in terms of low sensitivity, will have a bandwidth less
than ui.
If a plot of the closed-loop frequency response is available, SGr(jcu)may
be read directly off the diagram, as shown in Fig. 3.3.
376 ELEMENTS OF MODERN CONTROL THEORY

Im

For computational purposes, if we denote the perturbed values of

the open- and closed-loop transfer functions by G*(s) and T*(s), then
with

AG(s) = G*(s) - G(s)

(10)
AT(s) = T*(s) - T(s)

we have

G aT
(11)
tag
SENSITIVITY ANALYSIS 377

so that

T* = T + AT = T 1 + — SJ (12)
L G .

Consequently, if the nominal and perturbed transfer functions are

specified, the sensitivity SGr may be used directly to calculate the new
transfer function, T(s).
The ideas discussed above constitute the classical notions of sensitivity,
which, because of historical precedence, were necessarily expressed in
terms of frequency response. Modern control theory, which deals with
such concepts as root locus, state variables, etc., would, therefore, seem
to require an expanded approach to the definitions and use of sensitivity.
This is considered in the following sections.

3.1.2 Closed-Loop Poles

Referring to the unity feedback system shown in Fig. 3.4, we assume

that the open-loop transfer function is expressed in terms of its poles and
zeros as
n

k n (s + zj)
Ka(s)
(13)
0(s)
n (s + p<)

A deviation in G(s) may be due to a variation in open-loop gain, K;

the zero, z,-; the pole, p<; or any combination of these.
Obviously, a change in G(s) manifests itself in a shift of the closed-
loop poles of the system; i.e., a shift in the poles of

C(s)
&f G(s)
+_ a

Figure 3.4. Unity Feedback System.

378 ELEMENTS OF MODERN CONTROL THEORY

K II (s + zy)
7—1
C(s) G(s) Ka(s)
T(s) (14)
R(s) 1 + G(s) <3(s) + Ka(s) m-f-n

II (s + q»)
1=1

assuming m > 1.
Conventional root locus techniques may be used to determine how
the closed-loop poles are affected by changes in loop gain only. For
present purposes, we seek to determine the sensitivity of the closed-loop
poles to variations in K, zy, and p<. This particular sensitivity may be
defined in various ways. UrO takes

where x may be the open-loop gain, a pole, or a zero. On the other hand,
HuangO) uses

s . = x_ dq,
* q » dx

which is closest in form to the classical definition of Eq. (1) or (2).

In the ensuing discussion, we will adopt the type defined by McRuer
and Stapleford/O viz.

S*f = (15)

(16)

dq» .
S (17)
8py

Definitions ( 15)—( 17) are particularly convenient, since (as will be

shown subsequently) they satisfy a number of very useful relationships
with the open-loop poles and zeros and with the residues of the closed-
loop transfer function. These enable the sensitivity properties of the
system to be expressed in a very simple and enlightening form.
SENSITIVITY ANALYSIS 379

We now write the open-loop transfer function as

G = G(s, IC, Zj, p<) (18)

to emphasize that it is a function of not only s and K, but also the pole
and zero locations. Forming the total differential, we have

ip 6G , I dG <3G,
dG = — ds + —-dK + E — dzy + E- — dp, . (19)
6s dK ,=1 dzj ,=i dpy

Since — q, is a root of

1 + G(s) = 0 (20)

the total differential of Eq. (19) must be zero for s = —qi. Setting
dG = 0 and s = —qi in Eq. (19), we obtain, after rearranging terms

/dG

dq, =
(§' dK +
.E (f)
' dZ; + E dp, \dPi
(21)
« ,°G\
(f). s = -qi
=Gt)J s=-q.
ds ) s = -q,-

Since q, is itself a function of K, zy, and p, an alternate expression for

dq, may be written, viz.

q. = q.(K, zh p,) . (22)

Taking the total differential

T, aq.-dK , v' 6q, , , a (23)

dq,- = K — - h E — dz, + E — dp,.
dK K ,=i dz, ,=! dp.

By virtue of Eqs. ( 15)-( 17), this may be written as

1 12" 71 771+71

dq, = S; — + E S,•/ dz, + E S,/ dp,. (24)

3=1 3=1

After equating like coefficients in Eqs. (21) and (24), we find

380 ELEMENTS OF MODERN CONTROL THEORY

(26)

(27)
(?)
Note that the gain sensitivity is based on a fractional change in K,
while the pole and zero sensitivities are based on absolute shifts in p, and
z y. This apparent lack of harmony in the definitions is of little concern,
since these definitions lead to simple and instructive relations for the
respective sensitivities.
Recalling that G( —q,) = —1, while

3G = G
dK I\
dG G
dZj (s + Zj)
dG G

dP / (S + P;)

from Eq. (13), the sensitivity relations, Eqs. (25)—(27), reduce to

1
S
(28)

(29)

(30)

If we now let Q, denote the residue of T(s) at the pole, -q;, then
SENSITIVITY ANALYSIS 381

This relation is readily proved as follows. From Eq. (14)

T(s)
G(s) _ Q< (32)
1 + G(s)~ £ (s + q<)
where(t6)

Q< = (s + q»)G(s)~l (33)

. 1 + G(s) J

assuming that all the poles of T(s) are simple.

Let

(s + q,-)G(s)
*<(s) (34)
1 + G(s)

Then, by definition

= Q< % (35)

Writing Eq. (34) in the form

[1 + G(s)] <t>,(s) = (s + q<) G(s)

and differentiating both sides with respect to s

(1 + G)3><+ 4>,G' = G + (s + q,)G '

where

Solving for <t\-

G(s) + (s + q»)G,(s) - [1 + G(s)]$'(s)

$.(s)
G'(s)

But since G(-q,) = -l,we obtain

382 ELEMENTS OF MODERN CONTROL THEORY

*<( q<) G'(s)J = Qi = S,

s — <3t

Q.E.D.
Another important relation satisfied by the pole and zero sensitivities
is

n m-\-n

Es,/ + E s l. (36)
;=i 7=1

This is, in fact, a direct consequence of Eqs. (28)-(30) and the form of
G(s). We note that

(37)

But

i
/
a
s + Zj

m+n 1
P' = pE
l
s + Pi %

Therefore
1
SK' =
m-fn i (38)
E + E
7=1 -q* j=i q«— Pi

or, equivalently

Q * m+n C i

E — +L — = l .
zi - q« % q«-- Pi

Using Eqs. (29) and (30) leads to Eq. (36).

SENSITIVITY ANALYSIS 383

The gain sensitivity, S«' (which is, in general, a complex number), has
a simple physical interpretation in the s plane. It is a vector, tangent to
the root locus at s = —q,-, and oriented in a sense opposite to increasing
K. This follows trom the fact that a root locus is a plot of the roots of

1 + G(s) = 0.

The total derivative is

<3G dG
dG = — ds + — dK .
os dk

But along the locus, dG = 0. Therefore

s=-<u

Since dK/K is a real number, the direction of d s along the locus for
positive dK/K is given by —SK4.
The classical and gain sensitivities are related in a very simple way.
From Eq. (4), with H(s) = 1

„ 1 G
Sg = 1 + G = 1 “ 1 + G ‘

By virtue of Eqs. (31) and (32), this becomes

m-j-n
S/c7
Sg r _
— i - £ (40)
i=l
(s + <!«)

Thus, the classical sensitivity is interpreted as a weighted sum of the

gain sensitivities.
The basic problem of this section is the determination of the shift of
the closed-loop poles for prescribed (small) variations in gain and open-
loop poles and zeros. In view of the foregoing discussions, the main
relation is given by
dK " dz j dp,-
(41)
dq, = K + “ (z>- qt) + ~i (<1<
_ P>)-

which is obtained by combining Eqs. (24), (29), and (30).

384 ELEMENTS OF MODERN CONTROL THEORY

In a given situation, the nominal values of K, q*, z and p> are known.
When the variation of G(s) is expressed in terms of dK, dzy, and dpy, one
may determine dqy if the gain sensitivity, Sr , is known. Various methods
of calculating this quantity are considered next.

3.1 .2.1 Calculation of Gain Sensitivity

The most accurate calculation of the gain sensitivity is by the Direct

Method, viz.

m
SV — (42)
(s) + KV(s)J

This relation follows directly from Eqs. (13) and (28), making use of
the fact that G( —q.) = —1. For high order systems, the evaluation of
SK' via Eq. (42) becomes quite laborious. One may then use a graphical
approach (slightly less accurate), the simplest of which is the so-called
Gain Perturbation Method. Here we make use of the incremental ap¬
proximation to Eq. (15), i.e.

S (43)

To employ this relation, it is necessary to have available the location

of two closed-loop poles for two slightly different values of open-loop
gain. The situation is depicted in Fig. 3.5. Having K and AK, we measure
the magnitude of Aq» directly off the figure. Obviously, since K and AI\
are real numbers, the phase angle associated with SKlis the same as the
phase angle of Aq* (also measured off the figure).
One may devise a more accurate graphical method that is a direct
consequence of the fact that the residue of a function of a complex
variable at a given singularity can be expressed in terms of vectors to the
poles and zeros of the function in the s plane. Since the gain sensitivity
at —q< is indeed the residue ofT(s)at the pole— q,-, we have, by virtue
of Eqs. (31) and (33)

(s + q<) G(s)~
1 + G(s)
SENSITIVITY ANALYSIS 385

Gain = K
s =
-qi

Figure 3.5. Calculation of Gain Sensitivity by Gain

Perturbation Method.

Using Eq. (14), this becomes

K II (zy - q.)
7=1

SU = (44)
m-\-n

II (qy - q<)
7=1

We may call the approach using Eq. (44) the Vector Method. To use
this, it is necessary to have a complete set of closed-loop poles. The gain
sensitivity is then expressed in terms of a product of vectors from the
open-loop zeros divided by the product of vectors from the closed-loop
poles. The accuracy is limited only by the accuracy of the graphical plot.
At this point, it is instructive to consider the application of the above
ideas in a specific case.

Example 1 : The schematic of the system is shown in Fig. 3.4. We take

K(s 5) Ka(s)
G(s) = (45)
(s + l)(s + 2.75) 0(s)
386 ELEMENTS OF MODERN CONTROL THEORY

The corresponding root locus is depicted in Fig. 3.6, which also shows
the closed-loop poles corresponding to K = 7.25. It is required to deter¬
mine the shift in these closed-loop poles due to prescribed variations in
gain and open-loop poles and zero.
Applying Eq. (41) to this case, we have

dK dzi Api Ap2

dqi = S / 77 + + + (46)
(z! - qi) (qi - pi) (qi - p2) _

By direct measurement of the vector quantities involved (See Fig. 3.7);

Zl - qi = 3,00 /100°

qi — pi = 5.38 /~~33°

qi — p2 = 4.03 /~47°

— qi = —5.5 + j2.96 .

It remains to determine SKl. This will be calculated by each of the

three methods described in the previous section.
Using the Direct method, we have, from Eqs. (42) and (45)

"s2 + 3.75s + 2.75

Sk = - r 0 7 = 3.68 /10°
_(8'+ Ka‘. _2s + 3.75 + 7.25_
s = -q1

By the Gain Perturbation method via Eq. (43):

KAqi 7.25 X 0.5 /13

S Ji = 3.62 //13° .
AK

Finally, using the Vector method and Eq. (44):

n ! Iv(zi - qi) 7.25 X 3.0 /too0

3.67 /10° .
K “ (q2— qi) ~ 5.92 /90°

As expected, when the root locus is drawn accurately, the Vector

method yields a value virtually identical to that obtained by the Direct
method.
Assume now that
SENSITIVITY ANALYSIS 387

o o in o
LO O LO o 10
<M <N CO
c4 1-H © I I I

T n- i- 1- r T n- 1- 1

1.
Exam
for
Locus
Root
3.6.
Figure

7.25

20
388 ELEMENTS OF MODERN CONTROL THEORY

<u

3.7
Fig
Ve
for
Qu
De
of
Se

.25

21
SENSITIVITY ANALYSIS
389

AK = 1 Api = 0.25

Ap2 = -0.5 Azi = 0.75.

Then, from Eq. (46)

dqi = 3.68 /10 °

7.25

0.75 0.25
"r 3.0 /T00° ^ 5.38 X -33°

0.5 1
~ 4.03 /~47°

= (3.68 /10°) (0.3154 /-81°)

= 1.16 /~71° .

The new closed-loop pole, —q*, is then obtained by vector addition, as

shown in Fig. 3.8.

Figure 3.8. Shift of Closed-Loop Pole in

Example 1.
390 ELEMENTS OF MODERN CONTROL THEORY

As a check on the method, we calculate the closed-loop poles for

8.‘25(s + 5.75)
G(s) =
(s + 1.25) (s + 2.25) '

We find

— q* = -5.875 + j 3.968 .

In view of the fact that the parameter deviations were not at all
infinitesimal, the agreement is excellent.

Remark: One of the main virtues of the method herein described is that
useful qualitative sensitivity features may be obtained with
little effort. An examination of Eq. (41) indicates that the
sensitivity of a closed-loop pole to variations in a particular
open-loop pole or zero diminishes with increasing distance
between the two. Thus, variations in open-loop poles or zeros
far removed from the closed-loop pole in question have a
minor influence on the latter. This is perhaps intuitively evi¬
dent, but Eq. (41) expresses this condition in precise fashion.

3.1.3 Eigenvalues and Eigenvectors

We consider a linear stationary system expressed in the state variable

format as follows.

x = Ax + w (47)

x(0) = c (48)

where x is an n-dimensional state vector (n x 1 matrix), A is a constant

n X n matrix whose typical element will be denoted by a,7, and w is a
vector-forcing function.
It is known that the solution for the system, Eqs. (47) and (48), is
given by

e
At
c + / eA(l~
a,w (<r) d<7 (49)

where eAt is the transition matrix, which may be computed in several

SENSITIVITY ANALYSIS 391

ways.* The usual representation is

e^u = M M-1 . (50)

Here the A, denote the eigenvaluest of the matrix A, and M is the

modal matrix for A.
Thus, the distinctive feature of the state variable representation is
that the response is governed by the eigenvalues of the system matrix,
A. Using an alternate representation** for the transition matrix

(51)

where

uy = eigenvector (n X 1 matrix) corresponding to the eigenvalue, A,

(u<is the ith column of the modal matrix M)

vz- = row vector (1 X n matrix) whose elements are the ith row of
M"1.

The response is expressed as a weighted sum of the system eigen¬

vectors (modes), which clearly indicates the relative contribution of each
mode to the total response.
Now the deviation of any system parameter from its nominal value is
reflected in a change in one or more of the elements of the system
matrix, A. We investigate the problem of determining the change in the
eigenvalue, Ay (or eigenvector, uy), due to a change in an element &k( of
the matrix, A. One measure of sensitivity is the ratio of the (small)
increment in Ay to the (small) increment in a kt We, therefore, define

*See Chapter One.

j Throughout the ensuing discussions, these eigenvalues are assumed to be distinct.
**See Appendix A.
392 ELEMENTS OF MODERN CONTROL THEORY

the eigenvalue sensitivity

(52)
da*/

The problem is now one of finding a suitable expression for S*/’ in

terms of given system parameters. Following Laughton/11) we let

A* = the matrix obtained by replacing the (k/)th element in A by

a*/ + Aa*/

F*/ = the cofactors of theCkT)11* element of (A — XI).

Then, expanding the determinant of (A* — XI) by the elements and

cofactors of the kth row, we find

det [A* — XI] = a*i F*i (X) + a*2 F*2 (X) T- .

+ (a*/ + Aa*/) F*/ (X) + . + akn^ knW (53)

= det [A — XI] + Aa*/ F*/(X) .

Thus, the eigenvalues of A* are given by the roots of

det [A — XI] + Aa*/ F*/(X) — 0 . (54)

We may, therefore, write Eq. (53) in the form

XT (X» — X) — XI (X*— X) + Aa*/ F*/(X) (55)

where X? denotes the ith eigenvalue (assumed distinct) of A*.

Replacing X by Xy, Eq. (55) reduces to

AXy F*/(Xy)
Aa*/ n

IX (X? - Xy) (56)

i^i

where

AXy = Xy - Xy . (57)
SENSITIVITY ANALYSIS 393

Therefore, in the limit

Sk ’ = d\; _ E^(Xy)
" d&kl F«>(Xy) (58)

F(1)(Xy) = — Idet [A - XI]) \=m\j % (59)

dX '

Equation (58) is an explicit relation for the eigenvalue sensitivity,

S It- It is possible to obtain a simpler and more elegant expression as
follows. For this purpose, we adopt the notation

j.L (7 Ay
S‘ - the matrix whose (kf)‘ element is ~—
da k(

L(X) = adj [A - XI] .

Then the matrix equivalent of Eq. (58) is

(60)
F(1)(Xj)'
We have also*

L(Xy) = UyVy (61)

(Xy) = VyUy (62)

where uy and vy are the eigenvectors and the row vector previously
defined. Using this and Eq. (A5) of Appendix A, we find that the
eigenvalue sensitivity matrix of Eq. (60) may be expressed as

S3 = Vyr Uyr . (63)

This is the basic result of the analysis. Equation (63) shows that a
knowledge of the eigenvalues and modal matrix for A yields all the infor¬
mation concerning the system sensitivity, in addition to that necessary
for determining the system response.

These relations are derived in Chapter Three of Reference 18.

394 ELEMENTS OF MODERN CONTROL THEORY

In most practical situations, one system parameter usually appears in

several elements of the matrix, A. The variation of the jth eigenvalue is
then given by

AXj= H S4/Aa*/ (64)

kX

where the summation is taken over all perturbed values, &k(.

Proceeding in a completely analogous manner, we find, for the eigen¬
vector sensitivity

duy ^ , i
/ j b.k( Uy
d&hf i=l (65)

where hk(l is the (k/)th element of the matrix

H* = VA u,-r (66)
(Xy- X,-)'

As pointed out by Laughton/11) the above ideas may be profitably

applied to the evaluation of analog computer simulations when time
and amplitude scaling is involved. This is important because sensitivity
features, as determined by system-parameter changes on the computer
simulation, often serve as basic guidelines for design. It will be shown
that amplitude scaling materially alters the eigenvalue sensitivity matrix,
while time scaling has no effect on this sensitivity.
To show this, we define a new independent variable by

t = j8t, /3 = positive constant.

Then the free motion of Eq. (47) takes the form

whose solution is

= 12 eX‘7UyViC
= X eX,T(S']

using (A 14) and Eq. (63).

SENSITIVITY ANALYSIS 395

The eigenvector sensitivity matrix is thus unaffected by a change in

time scale.
Consider now a change in amplitude scale. This may be simply repre¬
sented by the coordinate transformation

x = Ty

where T is a diagonal matrix. Thus, instead of

x = Ax

we have

Y = By

where

B = T-1 AT . (67)

Two matrices, A and B, related by a transformation of type (67), are

said to be connected by a collineatory transformation A18) It is easy to
show that, in this case, A and B have the same eigenvalues. We have

T-1 AT - XI = T~> (A - XI) T

and, therefore

det [T-1 AT - XI] = (det T^MdetfA - XI]) • (det T) = det[A - XI]

Q.E.D.
But A may be written as

A=±\imT (68)
i=l

by virtue of Eqs. (A9) and (63).

Consequently, using Eq. (67) and the fact that A and B have the
same eigenvalues, we find

B = Z MS/f (69)
i=l
396 ELEMENTS OF MODERN CONTROL THEORY

where

s; = T-1 [Sf T. (70)

It follows, therefore, that the sensitivity properties of the transformed

system (amplitude scaled computer simulation) may differ markedly
from the actual system sensitivity.

Example 2: We consider the system described by

x = Ax

where

-2 -1 1
A 1 0 1
-1 0 1

The eigenvalues are

Xi = 1.0

X2 = —1 + j

X3 = — 1 — j .

The modal matrix is

5 5
—(3 + 4j) - (3 - 4j )
(2 + j) (2 - j)
and its inverse

M-1 = (1 +
(1 - j)
Therefore

0 5
Ui = 1 u2 = -(3 + 4j)
_ 1 _ (2 + j) _

5
u3 =
1_ Tv
CO,
04- 4j)
- j) J
SENSITIVITY ANALYSIS 397

and

Vi = -1

v2 - (1 + j)
10

v3 = (1 - J) -j
10

Assume now that element a23 in matrix A changes from 1 to 0.2. We

are interested in determining the shift in the complex eigenvalue, X2.
From Eq. (64)

AX2 — S23 Aa23 .

Now

aa23 - —0.8

while, by Eq. (63)

1
(1 + j)' [5 - (3 + 4j) (2 + j)]
S2 = v2r u2r
10
j
L_j
5(1 + j) (1 - 7j) (1 + 3j)
1
5j (4 - 3j) -(1 - 2j)
To (1 - 2j)_
_-5j -(4 - 3j)

which means that

(1 ~ 2j)
s23a= 10

Consequently

(1 - 2j)
AX2 (-0.8) 0.08 - 0.1 6j .
10
398 ELEMENTS OF MODERN CONTROL THEORY

Thus, using Eq. (57), we find that the new value of X2is

A* = (-1 + j) + (0.08 - 0.16j) = -0.92 + 0.84 j .

It is instructive to compare this with the exact value of X2obtained by

calculating the eigenvalues of the perturbed matrix

_2 -1 1
A* = 1 0 0.2
-1 0 1

After a simple calculation, we find

Xf = 0.815

X* = -0.9075 + 0.81 j

X* = -0.9075 - 0.81 j

which indicates very good agreement.

Remark: The particularly attractive feature of the sensitivity measures

developed in this section is that these eigenvalue sensitivity
matrices are available almost by inspection after the basic re¬
sponse data (modal matrix, transition matrix, etc.) has been
calculated for the given system. It is, therefore, possible to
determine quickly, and with little effort, the shifts in the
eigenvalues for prescribed variations in system parameters. The
elements of the eigenvalue sensitivity matrices provide quick,
qualitative measures of how the eigenvalues are affected by
specific parameter variations.

3.1.4 Sampled-Data Systems

The sensitivity measures considered thus far carry over with simple
and obvious modifications to sampled data systems.
Employing Z transforms, Eq. (4) becomes

(71)
SENSITIVITY ANALYSIS 399

which may be called the classical sensitivity for sampled data systems.
It may be evaluated in the manner shown in Figs. 3.2 and 3.3, except
that in the present case, we plot L(z) with* z = e’“T, and co runs from
zero to 7r instead of from zero to infinity. In other words, the usual
frequency-response methods for sampled data systems apply.
Furthermore, the discussion of Sect. 3.1.2 applies directly, except
that we deal with the z plane instead of s plane poles and zeros.
It is also easy to show that the eigenvalue sensitivities discussed in
Sect. 3.1.3 have a direct equivalent for discrete systems.
Consider, for example, the discrete system represented in state
variable form by

Xjfc+1= Ax* (72)

x0 = c = initial value vector. (73)

Xfcis an n-dimensional state vector, and subscript k indicates that this

is the value at time t = t*. For simplicity, the sampling interval has
been normalized to unity. A is a constant n X n matrix.
It is known that the solution to the system, Eqs. (72) and (73), is given
by(l 7)

xt = A4c . (74)

To put this in a more convenient form, we note that

A = M-1 A M (75)

where M is the modal matrix for A and

Xl

A =

The quantities, X,-, denote the (distinct) eigenvalues of A. As in Sect.

3.1.3, we write

*T = sampling period.
 ELEMENTS OF MODERN CONTROL THEORY
400

M = [ui u2

Vi

V2

M-1 =

Then, after noting that

A* = MA* M“ (76)

we may write Eq. (74) as

X* = [UiU2 Vi

V2

'% \n Vn

which reduces to

(77)
X*= X ^ UiVi
c .

Defining the eigenvalue sensitivity as

o > _ dA; (78)

= - —

O&kt

where a,k( is the (kf)th element of A, an argument completely analogous

to that of Sect. 3.1 .3 leads to

S' = v jr u jT (79)

which is completely equivalent to Eq. (63).

SENSITIVITY ANALYSIS 401

3.1.5 Multivariable Systems

A multivariable feedback control system may be viewed as a generaliza¬

tion of the schematic of Fig. 3.1 in which R(s) and C(s) are vectors, and,
therefore, the quantities G(s) and H(s) are matrices of transfer functions
(or simply matrix transfer functions). In tins case, it is not immediately
clear how to formulate a matrix equivalent of the sensitivity function
defined by Eq. (4). It is apparent, however, that such a matrix sensitivity
function should have two primary characteristics: (a) it should provide a
quantitative measure of the sensitivity of the closed- loop system to
parameter variations as compared with the sensitivity of the open- loop
system to these same variations; and (b) it should reduce to the scalar
equation (4) for the single-input/single-output case. It has been shown
by Cruz and Perkins that both requirements are exhibited by the matrix
relating the output errors due to parameter variations in a feedback
system to the output errors due to parameter variations in a correspond¬
ing open-loop system. This point of view will be developed in what
follows. For this purpose, we consider the open- and closed-loop multi-
variable systems shown in Figs. 3.9 and 3.10. Here

R(s) = p-dimensional input vector

U (s) = m-dimensional vector

C(s) = n-dimensional output vector

Gc(s) = m X p matrix

Figure 3.9. Multivariable Open-Loop Control System.

Figure 3.10. Multivariable Feedback Control System.

402 ELEMENTS OF MODERN CONTROL THEORY

Gi(s) = m X p matrix

G(s) = n X m (plant) matrix

H(s) = p X n matrix.

Subscripts o and c on vectors U(s) and C(s) refer to the vectors in the
open- and closed-loop systems, respectively. We define also

AG(s) = G*(s) - G(s) (80)

where G*(s) represents the plant matrix when the parameters differ from
nominal.
Referring to Figs. 3.9 and 3.10, we find

Co(s) = G(s) U0(s) (81)

Uo(s) = Ge(s) R(s) (82)

Cc(s) = G(s) Ue(s) (83)

Ue(s) = Gi(s) [R(s) - H(s) Ce(s)] . (84)

If the plant parameters differ from nominal, then we distinguish the

vector signals for this case by the starred quantities

C*(s) = G*(s) Uo(s) (85)

C*(s) = G*(s) U?(s) (86)

U*(s) = Gr(s)[R(s) - H(s) C*(s)] . (87)

Note that U0(s) remains the same.

Now define the Laplace transform of the vector output errors as

follows.

K(s) = Oo(s) - C*(s) (88)

Ec(s) = Cc(s) - C?(s) (89)

SENSITIVITY ANALYSIS 403

It will now be shown that the matrix relating E0(s) to Ec(s) has all the
properties of a sensitivity matrix. In this context, it is assumed that
Gc(s), Gi(s), and H(s)are such that Cc(s) = G0(s) when there are no plant
variations.
From Eqs. (83) and (84), we obtain (dropping the argument s for
simplicity)
/

Cc = (I + GGiH)-1 GGxR . (90)

Similarly, from Eqs. (86) and (87)

C* = (I + G*G1H)~1 G*GjR . (91)

Substituting Eqs. (80), (90), and (91) into Eq. (89), we obtain, after
some reduction

Ec = (I + G*GiH)_1 [AGGi(HT - I)] R (92)

where

T = (I + GGiH)-1 GGi . (93)

Now Eq. (90) can be written as

Co = TR . (94)

Combining this with Eq. (84)

Uc = Gx(I - HT)R . (95)

But since Cc = CQ by assumption, we have

U0 = Uc = Gi(I - HT)R . (96)

Also

E0 = (G - G*)U0 = -AGU0 = AGGRHT - I)R . (97)

Substituting this in Eq. (92), the latter becomes

Ec = (I + G*GiH)_1 E0 . (98)
404 ELEMENTS OF MODERN CONTROL THEORY

We define

S = (I + G*G1H)~1 (99)

as the sensitivity matrix for the multivariable system.

Note that for a single-input/single-output system with small plant
variations such that G* is approximately equal to G, S is equal to the
classical sensitivity as defined by Eq. (4). It will also be observed that
while matrices G, G1; Gc, and H are in general not square, matrix S is
always square.
In certain special cases, S may be expressed in several interesting
ways. For example, if the overall closed-loop transfer matrix is denoted
by Tc with a nominal plant G, and by T? with plant G*, with analogous
meanings for T0 and T*, then

Ec = Cc — C* = (Tc - T*)R

E0 = C0 - C* = (T0 - T*)R .

If (T0 — T*) is a square nonsingular matrix, we can eliminate R from

the above relations, obtaining

Ec = (Tc - T*)(T0 - T*)-1 E0

Comparing this with Eq. (98), we see that

S = (Tc - T*)(To - T*)-1 . (100)

Now

T0 = GGc (101)

and

(T0 — T*) = (G — G*)Gc . (102)

By virtue of the assumption that the open- and closed-loop systems

of Figs. 3.9 and 3.10 are equivalent for nominal parameters, we have
Tc = T„. If it is further assumed that matrices G, To, Gc, and (G - G*)
are square and nonsingular, then, from Eqs. (101) and (102)
SENSITIVITY ANALYSIS 405

(T0 - T*)-i = Gr1 (G - G*)-1 = To1 G(G - G*)-1 =

Tr1 G(G - G*)"1

Substituting this in Eq. (100) yields

S = ATcTr‘ GAG-1 (103)

/

where

atc = t* - Tc . (104)

Note the striking similarity in form between the sensitivity matrix,

Eq. (103), and the scalar equivalent, Eq. (11). It is apparent that the
analysis could have proceeded with S defined by (103) and subsequently
interpreted as the matrix relating Ec to E0.
We now seek to provide a criterion that compares the sensitivity of the
closed-loop system to that of the open-loop system for prescribed varia¬
tions in the plant parameters. Following Cruz and Perkins,!1 3> we take,
as an index of performance

I.P. = (105)

e(t) = £ HE(s)]

where E(s) is either Ec(s) or E0(s). In practical situations, tf may be taken

as four or five times the largest time constant of the system. Now for the
feedback system to be superior to the corresponding open-loop system,
the inequality

r‘f {%‘f
/ e cr(t)ec(t) dt < / e0r(t)e0(t) dt (106)
\ **0

must be satisfied. It may be shown!13) that this condition is ensured if

[S( — jco)]r[S(jco)] < I . (107)

For the scalar case, it was shown in Sect. 3.1.1 that |S(j«)| < 1 was
sufficient to ensure that the feedback system has better sensitivity
properties than the open-loop system. The matrix equivalent of this is
exhibited in Eq. (107). For a well-designed system, it is, therefore,
necessary to satisfy Eq. (107) over the frequency band of interest.
406 ELEMENTS OF MODERN CONTROL THEORY

3.2 DESIGN FOR SENSITIVITY

The methods of Sect. 3.1 permit the performance quality of a given

system to be evaluated in terms of its sensitivity to parameter variations.
This is the analysis problem. It is generally taken as an afterthought,
if considered at all. In one sense, this omission is not often serious,
since a feedback system has “built-in” sensitivity features that are
adequate in most designs. However, one may take the point of view that
a prescribed sensitivity is a fundamental design parameter and may deter¬
mine appropriate compensation networks accordingly. Techniques for
accomplishing this are not nearly as well-developed as for conventional
compensation (to ensure relative stability, time or frequency response,
etc.). One method, useful in certain cases, is described in Sect. 3.2.1.
Another aspect of the sensitivity problem, first pointed out by Horo¬
witz/19) is that, in certain instances, one mistakenly treats what is
basically a sensitivity problem as one of adaptive control. He introduces
the concept of “sensitivity in the large,” and shows that, in many cases, a
suitably compensated “high gain” system exhibits the features generally
thought to be obtainable only by adaptive methods. This approach will
be discussed in Sect. 3.2.2.

3.2.1 Compensation Networks

One design method for feedback system compensation is based on the

assumption that the system-response specifications can be expressed in
terms of desired locations of dominant closed-loop roots. Lead or lag
networks are usually employed to ensure that the dominant roots are at
desired locations, and the steady-state error for prescribed inputs (ramp,
step) is less than a given value.
If only the first of the above conditions is to be satisfied, there will be
an infinity of solutions. With both conditions, the solution is unique
(when the form of the compensation network is prescribed).
For the problem to be treated here, the first of the above conditions is
retained, but the second is replaced by a specification concerning the
root sensitivity. This new condition may take different forms (depending
on the particular problem), such as sensitivity of the relative damping
factor or natural frequency. The method is due to Rung and ThalerW
and is developed in the following.
SENSITIVITY ANALYSIS 407

3.2. 1.1 The U Circle

We consider the sensitivity of the closed-loop poles in the manner

discussed in Sect. 3.1.2. The pertinent definitions, Eqs. (29) and (30),
are repeated here for convenience.

c ’— sk‘,
(108)
Z’ (zy - Qi

Sk'
V = (109)
(q< - Py)

These satisfy relation Eq. (36), which may be written as

m-f-n
1
Z + z (110)
7=1 7=1 (q* - py)

The above three relations suggest a simple graphical procedure for

determining the root sensitivities. From the closed-loop pole, —qi5 a
vector can be drawn toward each open-loop pole and away from each
open-loop zero. The magnitude of each vector must be equal to the
inverse of the distance from —q,- to the pole or zero involved. (See Fig.
3.11.) By adding all these vectors (see diagram b of Fig. 3.11), we
obtain a vector sum, U, which by virtue of Eq. (110) is precisely the
vector 1/S/ It is easy to see, therefore, that the sensitivity of —qt to each
pole or zero is equal to the vector just drawn to the particular pole (or
away from the particular zero) divided by the vector U. A given con¬
figuration of open-loop poles and zeros thus implies a unique vector U
associated with the system.
Consider now a compensation procedure whereby a pole and zero are
added to the negative real axis. If the prime purpose is to obtain a pre¬
scribed form of response characteristics, the location of the compen¬
sating pole and zero is, in general, not unique. Elowever, if we add the
stipulation that the closed-loop root sensitivity is to be (in some sense)
minimized, then a unique solution is obtained. One type of sensitivity
is expressed by the requirement that the U vector be maximized. Prior
to investigating this question in detail, we derive a fundamental property
of the U vector locus when a compensating pole and zero are added.
Suppose that —q = —c ± jd represents the desired location of the
dominant closed-loop roots. Suppose further that this requires a phase
 ELEMENTS OF MODERN CONTROL THEORY
408

Im

NOTE: Scale in diagram (b) is

2-1/2 times that in
diagram (a).

Figure 3.1 1. Determination of U Vector.

shift of <pdegrees, which is to be contributed by the compensating pole

and zero placed along the negative real axis in the s plane. If <p> 0, a
lead network is required, while if<p < 0, a lag network is required. In this
way, the desired root location will he on the revised root locus.
SENSITIVITY ANALYSIS 409

1m

Figure 3.12 shows the pole-zero configuration for an uncompensated

system, together with a nominal closed-loop pole (solid square) and the
desired dominant pole (Point Q). Let <pdenote the phase lead that must
be contributed by the compensating pole and zero. Vector QI represents
the uncompensated U vector, that is, before the compensating pole and
zero are added. Then

As the location of the compensating pole and zero is varied,

subject to the restriction that <pdegrees of phase shift be con¬
tributed at point Q, the locus of the U vector is a circle with
center at I and radius R = (1/d) sin <p. This circle will be called
the U circle.

To prove this, we focus attention on the sensitivity vectors for the

compensating pole and zero. In Fig. 3.13, these are denoted by QM and
VQ, respectively. It will be shown first that V and M move along a circle
with radius r = l/2d. Applying the law of cosines, we have

e2 = — [cosVi + cosV2 — 2 cos<p!cos<£>2

cos^l .
cP

Using some elementary trigonometric identities, and the fact that

410 ELEMENTS OF MODERN CONTROL THEORY

v - <pi+ <pi, we reduce this to

sin2 <p .
e2
^ d2

We have also

sin2 <p
e2 = 2r2 — 2r2 cos 2 =
~ d2

Solving for r

r =
2d

which means that we may express e as

e = 2r sin <p = constant.

SENSITIVITY ANALYSIS 411

An elementary geometric theorem then shows that Q, M, and V always

lie on a fixed circle (for fixed d and <p), which is here referred to as the
M circle.
The sum of the pole and zero sensitivity vectors is QN. This must be
added to the uncompensated U vector, QI, of Fig. 3.12, to obtain the U
vector for the compensated system. Since the magnitude of QN is
constant, the tip of this final U vector describes a circle of radius

sin
e 2r sin <p =
~d

Q.E.L).

3. 2. 1.2 U Circle Limits

The fact that the compensating pole and zero are restricted to lie on
the negative real axis in the s plane means that the U vector locus will
traverse only a portion of the U circle. To determine these limits, con¬
sider Fig. 3.14a, in which the compensating zero is located at the origin.
To obtain the resulting U vector, draw IV parallel to QO. Now locate
point Jr by constructing VIJr = <p/2. From Jr, draw a line that passes
through the center, S2, of the M circle. Finally, draw IUr perpendicular
to JrS2. Point Ur on the U circle represents the right-hand limit of the U
vector locus.
The left-hand limit is obtained from Fig. 3.14b. Here the compen¬
sating pole is located at minus infinity. Locate point on the M circle
by constructing A I&Q = <p. Draw the line, Ifi, that intersects the U
circle at point U^. Point U/ represents the left-hand limit for the U vector
locus.
Thus, as the compensating pole and zero move from the extreme right
position (where the compensating zero is at the origin) to the extreme
left position (where the compensating pole is at minus infinity), the U
vector follows that portion of the U circle that is shown solid in Fig.
3.14b.
These results are easily established by elementary geometry.

3. 2. 1.3 Design Techniques

It will now be shown how the U circle may be used to satisfy stipulated
specifications on the sensitivity of the desired root location. The required
sensitivity may be expressed in one of several ways:
412 ELEMENTS OF MODERN CONTROL THEORY

Im

Im

(b) Left Hand Limit: Compensating Pole at Minus Infinity

Figure 3.14. U Circle Limits.

SENSITIVITY ANALYSIS 413

a. The sensitivity of the root at Q is to be a minimum with respect to

variations in the open-loop poles and zeros. This means that the U
vector of the system must be a maximum. In Fig. 3.14b, for example,
the U vector of maximum magnitude is given by QUr, i.e., with a
compensating pole at infinity.
b. The relative damping must be constant when the open-loop gain
fluctuates. Since the U vector is tangent to the compensated root
locus at point Q, this condition requires that the U vector (extended
if necessary) pass through the origin of the s plane.
c. The relative damping factor must be constant when a particular pole,
say p0, varies. A simple analysis shows that, in this case, the U vector
must be chosen so that the phase of Sp^’ is equal to the phase of the
desired dominant root,— q,-(or the phase of— q{ plus 180°).

Each of the above specifications requires that the compensating pole

and zero be found after the U vector is selected. This is merely a con¬
verse of the procedures already discussed. Consider, for example, the
situation shown in Fig. 3.15. Suppose that QUC represents the desired
U vector, determined in accordance with some prescribed procedure.

Im

Figure 3.15. Determination of Compensating Pole and Zero Location.

414 ELEMENTS OF MODERN CONTROL THEORY

If the compensation is to introduce a phase lead of <pdegrees at Q, then

the compensating zero is to the right of the compensating pole. To
locate the compensating zero, one proceeds as follows. Through ft (the
center of the M circle) draw a line perpendicular to IUC and intersecting
the M circle at J. Obtain points V and M such that 4- MftJ = 4- = <p.
Then, through Q, draw a line parallel to IV. The intersection of this line
with the real axis locates the compensating zero. The compensating pole
is then found by drawing a line through Q and parallel to IM.
Each of the three sensitivity specifications listed above leads to the
prescribed U vector, from which the locations of the compensating pole
and zero are determined. The general procedure is perhaps best illustrated
in terms of a specific example.

Example 3: Given is the open-loop transfer function

s(s + 1) (s T 4)

The system dynamic specifications lead to the requirement that the

dominant roots be located at —1 ±j. Figure 3.16 shows the open-loop
pole configuration in the s plane, where Q is the desired location of the
closed-loop pole. It is readily found that a compensating network that
contributes <p= 63° of phase lead at Q is necessary. Via a direct appli¬
cation of the methods discussed above, we first construct the compen¬
sated U vector, QI, and then find, for the radii of the M and U circles,
respectively

1 1
r “ 2d = 2
e = 2r sin <p = 0.89 .

The results are shown in Fig. 3.16, in which the limit points on the
U circle have also been determined.
We now seek to obtain a unique location for the compensating pole
and zero by considering, in turn, three types of sensitivity specification.
We first take the case where the sensitivity of the root at Q is to be a
minimum with respect to variations in the open-loop poles. As noted
earlier, this simply means that the vector from Q to the U circle, i.e., the
U vector, must be a maximum. An inspection of Fig. 3.16 indicates
immediately that the maximum U vector is given by QUr, i.e., at the
SENSITIVITY ANALYSIS 415

3.
Example
for
Circles
U
and
M
3.16.
Figure
416 ELEMENTS OF MODERN CONTROL THEORY

right limit point of the U circle. Consequently, the compensator for

this case has a zero at the origin and a pole at —1.30 (the latter deter¬
mined by the initial requirement that 63° of phase lead be contributed
at point Q).
As an alternate sensitivity specification, we may require that the
relative damping factor for the dominant root be constant for variations
in open-loop gain. Since the U vector is tangent to the compensated
root locus at Q, this would require that the U vector be in the direction
of line Q0 (constant relative damping). However, we note from Fig.
3.16 that the limiting position of QU is given by QU^(i.e., the left limit
point of the U circle). This is, therefore, the best that can be obtained.
Since this location requires a pole at infinity, the compensating zero is
located on the negative real axis such that <p = 63° of phase lead is
contributed at Q. This corresponds to a zero at —1.50.
Finally, we consider the requirement that the relative damping factor
for the closed-loop pole be constant with respect to variations of pole
—p2. As previously noted, this implies that the phase of S^' be equal to
the phase of the closed-loop root at— q, (denoted in Fig. 3.16 by point
Q). From the definition of the root sensitivity, Eq. (109)

q i SfC* _ _ 1_
* (q<- pO /_U
\Sjc7 (q» - Pa)

In Fig. 3.17, the vector from point Q to the U circle represents

1/SK\ Vector QA represents q* - p2. Furthermore, the magnitude and
phase angle of the closed-loop pole, —q<, is depicted by the vector 0Q.
We may, therefore, write

QU X QA
and

AS* = - AQU - AQA = AOQ .

Consequently, the phase angle of QU (the U vector) is given by

AQU = - AQA - AOQ = -270° - 135° = -405° = -45° .

In other words, the U vector must be aligned with 0Q. This is the
same result obtained for constant relative damping factor with respect to
SENSITIVITY ANALYSIS 417

for
Constr
3.17.
Figure
Damp
Relati
Consta
418 ELEMENTS OF MODERN CONTROL THEORY

open-loop gain variations. In the present case, the result is intuitively

plausible, since it implies that the compensating zero is to be placed as
close as possible to pole p2. This has the effect of minimizing the
influence of p2 on the transient response of the root at Q.

3.2.2 Sensitivity in the Large

The discussions thus far have emphasized either implicitly or explicitly

that the sensitivity techniques developed are valid only for small excur¬
sions of a particular parameter from nominal. This is, in fact, a direct
consequence of how sensitivity is defined. The basic definition, Eq. (1),
is expressed in terms of differential quantities, from which it is inferred
that an equation such as Eq. (4) cannot be used for large parameter
excursions. It can be shown quite easily, however, that the restriction to
small deviations is easily removed.
In the feedback control system of Fig. 3.18, suppose that the plant
transfer function, P(s), varies because of fluctuations in the plant param¬
eters. Let*

AP = P - P0 (111)

AT = T - To (112)

where the zero subscripts indicate values for nominal plant parameters
and T is the closed-loop transfer function

T = (113)
1 + L

Figure 3.18. Conventional Feedback Control System.

For ease of writing, the argument, s, in the transfer functions will hereafter be dropped.
SENSITIVITY ANALYSIS 419

L = GHP . (114)

The A quantities are assumed finite and not necessarily small.

Now define the sensitivity of T to variations in P as

T AT P
(115)
P AP T '

By substituting Eqs. (Ill )—( 113) in Eq. (115), we find, after some
reduction

S/ = — ]— . (116)
1 + L0

Here we have not used the relation

ar = G
ap (1 + GHP)2

which was used to derive Eq. (4). Therefore, Eq. (116) is valid what¬
ever the magnitude of AP.
It was first pointed out by HorowitzG5/9) that a failure to appreciate
the far-ranging significance of the sensitivity relation for large parameter
variations, Eq. (116), has led to the formulation of ambiguous and
superficial “adaptive” systems that are, in fact, less effective than con¬
ventional linear types. He quotes several motivations in the literature to
justify the needs for adaptive methods:

... it is generally taken for granted that the dynamic char¬

acteristics of the process will change only slightly under any
operating conditions encountered during the lifetime of the
control system. Such slight changes are foreseen and are
usually counteracted by using feedback. Should the changes
become large, the control equipment as originally designed
may fail to meet performance specifications/23)
The use of feedback in the classical sense . . . may be con¬
sidered passive adaptation. . . . For more complex system or
systems whose environment is more severe, simple passive
adaptation may not be sufficient/22)
If, however, the parameter variations are extremely large, the
gain required to maintain the specified system performance
420 ELEMENTS OF MODERN CONTROL THEORY

may become so high as to be unobtainable because of noise or

saturation limits.!24)
For systems with more complicated dynamics, it is usually
not possible to “swamp out” the fixed element characteristics
and still obtain the desired performance. . . . The use of condi¬
tional feedback, sometimes referred to as the “model ap¬
proach,” allows the designer to minimize these effects. . . .
Even the best of linear designs breaks down, however, when
the parameters vary over too wide a range.!25)
The adaptive approach would aim to maintain a prescribed
sensitivity or performance criterion in the face of process
changes. . . . The adaptive viewpoint would be especially suited
to the design of controllers for processes whose dynamics are
not completely known in advance.!26)
Conventional control systems are designed to meet certain
specifications under certain given conditions of the environ¬
ment and the system parameters, but should these conditions
change the performance will change as a result.!27)

To examine these arguments critically, we consider the classical feed¬

back configuration in which G(s) = 1 in Fig. 3.18.
The sensitivity is

1 + HP0

while the desired closed-loop transfer function is

To = P0S
o
1 + HP0

For this configuration it is impossible to realize S/ independently of

T0. Consequently, one must compromise either the sensitivity or the
desired closed-loop properties of the system. This fundamental property
of the system is directly related to the fact that there is only one “degree
of freedom ” for this configuration; i.e., only one free compensating
network is available.
When G(s) ^ 1 in Fig. 3.18, then G and H may be chosen so thatS/
and T are realized independently of one another. For example, from
Eq. (116)
SENSITIVITY ANALYSIS 421

L0 = y - 1. (117)
bp

Thus H may be selected to satisfy the sensitivity constraint while G

may be chosen to yield the desired closed-loop requirement from

T0 = GP° = GP0S/ . (118)

1 + Lo

Here there are two “degrees of freedom. ” For this case, “it appears
that it is theoretically possible to design an ordinary type of feedback
system to have a chosen sensitivity to any amount of plant variation.”!19)
Many different types of two-degree-of-freedom configurations are pos¬
sible/15) The essential point is that they are all essentially equivalent in
having the capability to realize T and SPTindependently. With this point
in mind, it becomes fruitless to search for new and exotic configurations
(i.e., model feedback, reference model, etc.) when what is really needed
is merely an additional degree of freedom.
As is generally the case, added benefits are obtained at a price. In the
situation considered above, the desired insensitivity of T to large varia¬
tions in P is obtained at the cost of large open-loop gain, which effectively
means compensating networks with large bandwidth. The design philoso¬
phy of the method will be explained in terms of familiar root locus
concepts.

3.2.2. 1 Design in the s Plane

A typical problem in control system design is usually expressed as the

need to maintain a few dominant closed-loop roots relatively invariant
while system parameters vary over wide ranges. For definiteness, let us
consider the plant transfer function given by

P = _ __ . (119)
s(s2 + 2fpw„s + cop2)

It is assumed that K may vary by a factor of 4, while and are

such that the poles of P may be anywhere in the rectangles ABCD,
ABC D of Fig. 3.19. It is desired that the dominant closed-loop pole
pair be located within a circle of radius 1.2, centered at -10 ± j 10
(R, R in Fig. 3.19).
422 ELEMENTS OF MODERN CONTROL THEORY

Im

Figure 3.19. Design Philosophy for Invariance of Dominant Closed-Loop Poles.

One way of achieving this is to place a complex zero pair (zh zi) in the
vicinity of (R, R) and to use a high open-loop gain. There are two funda¬
mental problems:

a. Where to place the complex zero pair.

b. Where to place the poles associated with the zeros (since the compen¬
sator must be physically realizable).

The use of high open-loop gain means that ultimately the root loci
associated with the compensator poles are such that a closed loop may
be in the vicinity of the dominant poles or else in the right-half plane
(unstable) if the gain is sufficiently high. Consequently, these compen¬
sator poles must be “sufficiently far” to the left. The further they are to
the left (and the higher the gain), the greater the bandwidth of the
system. Usually, one stipulates that the compensator closed-loop poles
should be to the left of some arbitrary line, UVY'U', in the s plane.
There is some freedom in choosing this line, depending on the particular
SENSITIVITY ANALYSIS 423

application; however, this choice is an integral part of the design pro¬

cedure, and once made, serves as a primary constraint to be satisified.
It is required, therefore, to

a. Choose (zi, zi) such that the desired T is obtained.

b. Choose the compensator pole pair (y, y) such that the desired sensi¬
tivity is achieved.
/

A simple analysis shows that a two-degree-of-freedom structure is

required. That is, if the compensator poles and zeros were contained
within G or H alone (with the other identically one), then zi and y could
not be manipulated to control both sensitivity and desired root location
independently.
Furthermore, of the infinity of possible compensator pole locations,
we seek to select the one that results in minimum bandwidth. The
design thus proceeds in two stages:

a. Locate the compensator zero pair (zi, zi).

b. Locate the compensator pole pair (y, y).

We consider these in turn.

3. 2. 2.2 Compensator Zero Location

The location of the compensator zeros is influenced by the variation

in open-loop gain, K, and by the drift of the open-loop poles of P.
Consider first the variation in K. Suppose that E is a root off + L = 0
when K takes a particular value. Then (see Fig. 3.20)

II (p;E)
i

K = = -
n we)
;

which is merely the usual expression for open-loop gain as the product
of pole vectors to the closed-loop pole divided by the product of zero
vectors to the same closed-loop pole. When K is increased to K', the
closed-loop pole moves to point E', and we have

= II (PiEQ
XI WE') '
424 ELEMENTS OF MODERN CONTROL THEORY

Figure 3.20. Variation of Closed-Loop Pole with Variation in K.

The conditions of the problem are such that the drift in E is very
small; therefore, p<E piE' and zyE'« zyE' except for the zero, zh near
E. Consequently

K' (siE)
(120)
K ~ (zxEO‘

To find the change in the dominant root of 1 -f L = 0 due to change

in the position of the complex pole of P, let pi, pi and pi, pi, respectively
denote the old and new complex pole positions of L. Also, let E, E',
denote the old and new positions of the dominant root of 1 + L = 0.
Then

(PlE)(piE) II (P*E) (plE')(Pi'E') II (p,E0

<5*1

(ZlE) II (z,-E) (zaE') II (zyE')

i*l i*i

(plE'KplEO II (p.E)
t^l

(ziEO n (zjE)
1*1
SENSITIVITY ANALYSIS 425

Therefore

(ziEQ^ (plEQ(pjEQ
(ziE) ~ (PlE)(PlE)
This approximation is valid whenever |PiE| ^ |p<E'| except possibly
for i = 1, and when |z*E| « |ZjE',1 except for j = 1. As noted earlier,
the design constrains the drift in E td be small, which means that these
conditions are generally satisfied. In this case, it is also permissible to
assume that piE'^piE andpiE'^piE. We then have

(ziEQ = (pjEKpjE) (121)

(ziE) (PiE)(PlE) '

Equations (120) and (121) are used to find the approximate shape
and orientation of the region of variation of the dominant roots of
1 + L = 0 due to variations in gain and the poles of L. This is done as
follows.
In Fig. 3.21, Point O corresponds to the location of the compensating
zero, and X corresponds to the nominal location of the dominant root.
These two points may be selected at will, since only their relative location
determines the scale and orientation of Fig. 3.21. It is assumed in the
following that Kmin represents the nominal value of K. As drawn in Fig.
3.21, OX = 1.0 /0°.
Now suppose that K changes from its nominal value to K' = 4Kmin.
The new position of the dominant root (denoted by X') is given by Eq.
(120); viz.

_ ziX__ OX =
K ~ zjX' OX'

This serves to locate X' in Fig. 3.21; i.e., OX' = 0.25 OX. Thus,
X'X is the approximate locus of the dominant roots as K changes from
Emin to 4 Kmin.
With the open-loop gain equal to its nominal value, Kmin, we now
seek to determine the locus of the dominant root as the open-loop pole
traverses rectangle ADBC of Fig. 3.19. Consider first the effect of
the plant poles moving from A to D. We use Eq. (121)* with (see
Fig. 3.19)

Keeping in mind that E(E') in Fig. 3.20 or Eq. (121) corresponds to X(X') of Fig. 3.21.
426 ELEMENTS OF MODERN CONTROL THEORY

piE 4
piE « —4 + j20

p,'E sa - 10
piE « -10 + j20 .

In these relations, E and (E') is assumed located at R in Fig. 3.19

(which also accounts for the “approximately equal to” symbol being
used). Therefore

(zxEQ = OM = ( —10)(—10 + j 20)

(ZlE) OX ( —4)( —4 + j 20)

and

OM = 2.70 ,/15°

since OX = 1.0 /®° by definition. This locates point M in Fig. 3.21.

With K = 4 Kmin and the plant pole still at D, Eq. (120) is used to
determine M', viz.

z(E = (M
ZjE' OM'

In other words, MM' is the approximate locus of the dominant root

at K varies for Kmin to 4 Kmin when the plant pole is at D.
Continuing in this fashion, we obtain the diagram of Fig. 3.21. The
quadrilateral, XMJN, represents the locus of the dominant root when
the plant pole traverses rectangle ADBC in the s plane (Fig. 3.19) and
K = Kmin. The two other quadrilaterals represent the dominant root
loci for the other values of gain shown.
The general shape of the region of variation of the dominant root is
now available and is used to determine the required scale of Fig. 3.21
and its orientation in the s plane. The dominant poles of T (roots of
1 + L = 0) lie inside region X'M'MJNN'X'. By trial and error, the
circle of minimum radius that contains this region is found (center R
and radius RM in Fig. 3.21). The specifications dictate that the domi¬
nant roots of 1 + L =0 must lie inside a circle of radius 1.2 centered
at —10 + jlO. The magnitude of RM is, therefore, 1.2, and R must
correspond to point —10 + jlO. It is found that OR, which is the
distance of zi from —10 + jlO, is also 1.2. It is clear from Fig. 3.21 that
SENSITIVITY ANALYSIS 427

o
O /

-30

Figur
3.21.
of
Regio
Varia
Domi
Roots
428 ELEMENTS OF MODERN CONTROL THEORY

point R is a root of 1 -f- L = 0 when K = Kmin and when the plant poles
are at approximately —5 ± lOj. The latter is obtained by noting that in
Fig. 3.21, R lies on lineXM, which corresponds to line AD in Fig. 3.19.
By measuring on Fig. 3.21, it is found that XR ;^(0.18)XM. Since
(0. 18) (AD) « 1, point —5 + jlO is thus determined.
With the above information, the required location of the zeros, zh zi,
of Fig. 3.19 can be obtained fairly closely. It is known that the net
angle of the vectors from the poles and zeros of L to —10 + jlO must be
180°. Since the far-off poles of L are not known as yet, a few degrees
may be assigned for their contribution, or, as a first approximation, they
may be neglected altogether. Therefore (see Fig. 3.19), 180° = ZOR +
ZpR + ZpR — ZziR — ZziR. Since ZziR ^ 90°, this leads to ZziR ^
146°. It has previously been determined that |ziR| = 1.2. Since R is at
—10 + j 10, this locates zi at —9 + j 9.3. With this information, it is pos¬
sible to turn to the problem of locating the far-off poles of L.

3. 2. 2. 3 Location of Far-off Poles

The design specifications require that the far-off poles be located

sufficiently to the left of line UVV'U' (Fig. 3.19) that no closed-loop
pole is to the right of the line for maximum open-loop gain. Suppose
that, for K = 4 Kmin, a closed-loop pole pair is located at (W, W) in Fig.
3.22, i.e., at —30 ± j 180. At W, the totality of poles and zeros near the
origin appears as a single pole at the origin. The phase contribution of
this pole at the origin to the point W is 100°. Consequently, the two
far-off poles must contribute a total of 180 - 100 = 80° to W. It is easy
to see that the locus of pole positions that contribute 80° to W is a seg¬
ment of a circle passing through W. Three points on this circle are quickly
located. One must be a point on the negative real axis such that a double
pole placed here contributes 2 X 40 = 80° to W. (See Fig. 3.22.) The
other two points are on horizontal lines through W and W such that the
first contributes 0° and the second contributes 80° (N and N in Fig.
3.22). This enables one to draw the circle shown.
Suppose now that the two far-off poles are located at (Q, Q). Since a
closed-loop pole is to appear at W for K = 4 Kmin, we must have

4 Kmin= |(QW)(QW)(OW)|
. (122)
However, for a closed-loop pole to appear at R (Fig. 3.19), we must
have in addition
SENSITIVITY ANALYSIS 429

Figure 3.22. Location of Far-off Poles.

(PlR)(pill)(01l)(QR)(QR) (123)
(ziRXzrR)
430 ELEMENTS OF MODERN CONTROL THEORY

If the Kmin calculated by Eq. (122) is substantially equal to the Kmin

calculated by Eq. (123), then (Q, Q) is a permissible location for the
poles. If not, another pole location along the circle is selected, and the
process is repeated until the two Kmin are in good agreement.
The Kmin thus determined is not necessarily the lowest possible, and,
these far-off pole locations are, therefore, not the best, in the sense of
smallest bandwidth. A new W point along UVVTJ' is chosen, and the
process is repeated until the smallest Kmin (and, therefore, the minimum
bandwidth) is found.
This completes the design procedure.

Remark: The concept of sensitivity in the large appears to shed new

light on the question of exotic adaptive techniques for plants
that exhibit wide parameter variations. The methods discussed
above have been applied to aircraft and to re-entry vehicles!2 h2°)
and have pointedly challenged the capability of adaptive
methods to yield a superior system.
As previously mentioned, the price paid for producing a
prescribed insensitivity is the large bandwidth of the resulting
system. In some cases, an enormous gain-bandwidth require¬
ment may be substantially reduced by first employing minor
feedback loops around the plant.
There are possible limitations in the above approach due to
noise and signal saturation. In this respect, it should be noted
that the amount of plant saturation caused by the useful signal
input is determined by the desired system performance and is
completely independent of how the performance is to be ob¬
tained, whether by ordinary feedback or by means of an
adaptive system. The signal level in the plant is determined by
the desired output; therefore, any two systems with the same
plant, the same desired system transfer function, and the same
output must have exactly the same plant-saturation problems
with respect to the useful signals, or to the noise that enters at
the same point as the useful signal.
Another serious limitation of ordinary feedback (and one
that has not received much attention in the adaptive literature)
is the inability to realize, even theoretically, any desired loop
gain-bandwidth when the plant is nonminimum-phase (actually
if it has at least two zeros or poles in the right-half plane).
Therefore, it may be impossible to obtain the desired benefits
of feedback by means of ordinary feedback structures. In this
SENSITIVITY ANALYSIS 431

regard, Horowitz!19) states, “It would be a genuine and im¬

portant contribution if it could be shown how these desired
benefits ( insensitivity, disturbance rejection, etc.) may be ob¬
tained by adaptive systems. However, once again, while the
above has been rather vaguely cited in the literature as a
justification for departing from ordinary feedback, there has
not been any corresponding demonstration that the adaptive
systems can do any better. One searches in vain in the adaptive
literature for a clear-cut quantitative statement of a problem
which is shown to be intractable by ordinary feedback, but
amenable to an adaptive design.”

3.3 SENSITIVITY AND OPTIMAL CONTROL

A general formulation of the optimal control problem is the following.

Given a system described by the vector differential equation

x = f(t, x, u, a) (124)

x(t0) = c (125)

where x is a state vector, u is the control vector, and a is a vector repre¬

senting a set of m plant parameters. The nominal value of the ith com¬
ponent of a will be denoted by a,0- The problem is to select the control
vector, u, such that the index of performance

/vL(t, x, u, a0) dt + G[t/, x(t/)] (126)

is a minimum (or maximum). Note that the index of performance is

based on the nominal value of a. Suppose that by one of the usual
optimization methods, the optimal control is found to be*

u*(t) = SP'jt,x(t), a0] . (127)

We denote the corresponding index of performance by

J*(to, X, a0) = Min J (t0, x, u, a0) . (128)

u

*The optimal control shown is closed-loop; that is, it depends on the current state of the system.
For open-loop control, c would replace x(t) in Eq. (127).
432 ELEMENTS OF MODERN CONTROL THEORY

If the actual parameter vector is used instead of the nominal a0, the
increment in performance index is given by

AJ = J*(t0, x, a) — J*(t0, x, a0) . (129)

AJ may be positive or negative, depending on how the system param¬

eters vary. In other words, there may indeed be some combination of
system parameters that yields an improved index of performance since
the latter was optimized with respect to u only. Expanding J*(t0, x, a)
in a Taylor series about the nominal system parameters and discarding
all but first-order terms reduces Eq. (129) to

(130)

— &>iQ •

Following Pagurek/12) we define the performance index sensitivity

function as

c j _ dJ*(tp, X, a)
(131)
f da i

In certain cases, this sensitivity function may be expressed directly in

terms of the given parameters of the system. One such case is the
optimal control problem for the linear system

x = A0x + B0u (132)

y = Gpx (133)

x(t0) = c

with a quadratic performance index

r‘f
j (to, X, u, a0) = / (yrQy + uTRu)dt + xr(t/)Mx(t/) . (134)
SENSITIVITY ANALYSIS 433

Here, A, B, G, Q, M, and R are constant matrices,* and the zero sub¬

scripts denote nominal values of the parameter vector, a.
It is known!30) that for this problem, the optimal control vector is
given by

u*(t) = -R-1 B0rPo(t) x (t) (135)

/

where P0(t) satisfies the matrix Riccati equation

Po + P„Ao + A0tPo - PoBoR"1B0rPo + GorQGo = 0 (136)

Po(t/) = M (137)

and the optimal performance index is

J*(t, x, a0) = xrP0x . (138)

These results are also derived in Chapter Five.

Now when the control, Eq. ( 135), is used with the actual (or perturbed)
system parameter vector, a, instead of the nominal, a0, the motion is
described by

x = Ax - BR-1 B0r P0x = F(t)x (139)

where

F(t) = A - BR-1 B0r P0 . (140)

The solution of Eq. (139) is

x(t) = <t>(t,t0)c (141)

where <t>(t,t0) is the transition matrix!10) for the system, Eq. (139).
Substituting Eq. (135) and (141) in Eq. (126), we obtain

J*(t0, x, a) = cT J W(t,t0)Qi<t>(t,
t0)dt+ &T(tf,
to)M4>(t/,
to) (142)

*We also make the usual assumption that the weighting matrices, Q, R, and M, are symmetric,
positive definite. Actually it is sufficient for Q and M to be positive semidefinite.
434 ELEMENTS OF MODERN CONTROL THEORY

where

Qi = GrQG + PoBoR-1B0TP0 . (143)

Equation (142) is of the form

J*(t, x, a) = x^Px (144)

where

r‘f
P(t) = Jt/ 4>r(r,
t)Q1(r)#(r,
t)dr + 4>r(t,,
t)M$(t,,t) . (145)

Noting that 4>(t, r) = I (the unit matrix) for all r, we find the boundary
condition

P(t/) = M . (146)

If we differentiate Eq. (145) with respect to t using Leibnitz’s rule,

we have

rpi/d<J>r(r,
_ t) t)
P =J - dt
—-Qi(t)4>(t,
t)dr+ vi/'’
dt
M«f>(t/,
t)
(147)

+ /Jt t)Qi(r)
—T-
dt
t}-dr
+ 4>nt/,
t)M dt
- Q,(t).
Noting that the transition matrix satisfies the relations*:10)

d4>r(r, t)
= - FT(t)<t>r(r,t) (148)
dt

d<t>(r,t)
= - *(t, t)F(t) (149)
dt

we obtain the following matrix equation for P.

P + FrP + PF + Q, = 0 (150)

P(t,) = M . (151)
SENSITIVITY ANALYSIS 435

It is easy to see that if we put a = a0 (i.e., A = A0, B = B„, etc.), then

P(t) Po(t) (152)

S - Sq

By virtue of Eq. (144), the sensitivity, function, Eq. (131), becomes

SJ = cr —
* 3a i
c
(153)
= ai0

But from Eq. (150)

*-(**)
3at- \dt /
+ ft**+
d&i
^!p
da. t-
+ p^
dat
(154)
+ A F + *9?- 0 .
d&i d&i

We now define

3P
P<(t) = (155)
3af

and assume that

_3/dP\ = d/ap\
3a,\dt / dt\3at/

an operation that is valid under mild restrictions on P.

Then the performance index sensitivity function is given by

S 4 = crPjC (156)

where (after putting a = a0 in Eq. [154] and making use of Eq. [152])

dt
Pt- + FtF , + PiF + Qs = 0 (157)

P.CV) = 0 (158)
436 ELEMENTS OF MODERN CONTROL THEORY

and

(159)
d&i _ a-to

Thus, having the sensitivity function, Eq. (156), the increment in the
performance function due to prescribed increments 5at in the system
parameter is obtained from Eq. ( 130) as

AJ = E S/5a; . (160)

Remark: For the situation analyzed above, the calculation of the per¬
formance index sensitivity function is conceptually simple.
However, it is readily apparent that, in order to obtain numeri¬
cal results, the use of a computer is virtually mandatory. The
theory may also be generalized to include nonlinear systems.
In this case, it can be shown!9) that quantities J*(t, x, a) and
J*(t, x, a0) of Eq. (129) each satisfy a type of Hamilton Jacobi
equation. However, as may be expected, the computational
aspects become overwhelming.
We may also note that the sensitivity function defined in
this section is not the only one that could be used. Rohrer
and SobraK2^) suggest the use of a relative sensitivity function.
This is defined as follows.
At the plant parameter, a, the relative sensitivity for the
control, u(t), is defined as the difference between the actual
value of the performance index and the value that would be
obtained if the control were the optimal for the plant param¬
eters, a (divided by the optimal performance index for nor¬
malization).

a j _ J(a, u) - J(a, u*)

VJ aa (161)

Among the obvious advantages of this definition is that S/ is

always a positive number. Moreover, the relative sensitivity
reduces to zero at the value of the plant parameters for which
the control, u(t), is optimal. System performance is always
compared with an attainable value. This eliminates a discon¬
certing element in the Pagurek theory above; namely, that
off-nominal values of the system parameters may actually
improve the performance index. Further studies are necessary
to establish the superiority of one method over the other.
 APPENDIX A

LINEAR SYSTEM RESPONSE IN TERMS

OF EIGENVECTORS
£

Consider the unforced dynamic system described by

x = Ax (Al)

x(0) = c (A2)

where x is the state vector (n X 1 matrix), A is a constant n X n matrix,

and c is the initial condition vector.
If the ith eigenvalue of A is denoted by A,• (assumed distinct) and the
corresponding eigenvector by u<, then

A u i = \i uf
(A3)
= 1,2,-

and

ufru, = u/Ui = 0 whenever i X j . (A4)

Equations (A3) and (A4) are well-known properties of eigenvectors.

The modal matrix for A may be written as

M = [U] us . u,] .

After calculating the inverse, we express it in the form

Vl

v2

M"1 =

437
438 ELEMENTS OF MODERN CONTROL THEORY

where v, is a row vector (1 X n matrix). By virtue of the relation

M-1M = I

it follows that*

VjUy = bij (A5)

where 8a is the Dirac delta function.

Now any vector, x, can be represented as
n

X= X Ui (A6)
i= 1

where the £,% are appropriate scalars.

Premultiplying Eq. (A6) by v*

Vkx = Vi X U V*U;
i=l

and summing over all k

n n n n n

X vAx = X b X v4 u£ = X it = Xf*
^=1 i=l &=1 i=l /c=l

by virtue of Eq. (A5).

It follows that

£* = VfcX (A7)

and, therefore
n

X = Xv,xu«. (A8)
i= 1

Using this result, we may write

Ax = A X V,-x Uj .

*In the language of formal matrix theory, the set of vectors juj) is said to form a basis, and the
set of vectors |v,-j is the reciprocal basis.
SENSITIVITY ANALYSIS 439

But since (v,x) is a scalar, we may transpose to

Ax = J2 (Vix) Aut-= J2 (vi x) Xt-ui

t=l 1=1

after applying Eq. (A3). Finally

Ax = ^2 \ Uj Vix
i=l

which leads to

A = £ XiUi Vi (A9)

Now the solution of Eq. (Al) may be written in the form

n
(A 10)
X= a*(t)Ui
1=1

where the a,(t) are scalar functions of time to be determined, In view

of Eqs. (A2) and Eq. (A10), we have

C = J2 a*(0)Ui
i=1
or

a»(0) = ViC (All)

using Eq. (A7).

Substituting Eq-. (A 10) in (Al), we find

OLiUi A /*1 OCi

Uj l OCiA Ui ^^ Xi U{
1=1 1=1 1=1 1=1

using Eq. (A3).

Therefore

on = \i a i . (A12)
440 ELEMENTS OF MODERN CONTROL THEORY

The solution of this equation, subject to the initial condition Eq.

(All), is

oti{t) = ex<<v,c . (A 13)

Consequently, after substituting Eq. (A13) in Eq. (A10), we obtain

X = X eXi<Vi C u,-
i= 1

or, equivalently

(A 14)
x = X eXi' u,-v,-c .
i=l

This is the result sought. It expresses the response of the system as a

weighted sum of the individual modes.
Since the solution to Eq. (Al) may be expressed in the equivalent form

x = e^'c (A 15)

a comparison of Eqs. (A 14) and (A 15) shows that

n
(A 16)
i=l

REFERENCES

1. J. G. Truxal, Automatic Feedback Control System Synthesis, New

York, McGraw-Hill Book Company, Inc., 1955.
2. H. W. Bode, Network A nalysis and Feedback Amplifier Design, New
York, Van Nostrand Company, Inc., 1945.
3. H. Ur, “Root Locus Properties and Sensitivity Relations in Control
Systems,” IRE Trans, on Automatic Control, January, 1960, pp.
57-65.
4. R. V. Huang, “The Sensitivity of the Poles of Linear Closed Loop
Systems,” AIEE Trans., Part II, Appl. & Ind., Vol. 77, May, 1958,
pp. 182-187.
5. W. M. Mazer, “Specification of the Linear Feedback Sensitivity
Function,” IRE Trans, on Automatic Control, June, 1960, pp.
85-93.
SENSITIVITY ANALYSIS 441

6. D. T. McRuer and R. L. Stapleford, Sensitivity and Modal Response

for Single Loop and Multiloop Systems, Technical Report ASD-
TDR-62-812, Air Force Systems Command, Wright-Patterson Air
Force Base, Ohio, January, 1963.
7. I. M. Horowitz, Synthesis of Feedback Systems, New York, Aca¬
demic Press, Inc., 1963.
8. B. T. Rung and G. J. Thaler, “Feedback Control Systems: Design
with Regard to Sensitivity,” IEEE Trans. Appl. & Ind., Vol. 83,
No. 75, November, 1964, pp. 360-366.
9. B. Pagurek, “Sensitivity of the Performance of Optimal Control
Systems to Plant Parameter Variations,” IEEE Trans, on Auto¬
matic Control, April, 1965, pp. 178-180.
10. L. A. Zadeh and C. A. Desoer, Linear System Theory, New York,
McGraw-Hill Book Company, Inc., 1963.
11. M. A. Laughton, “Sensitivity in Dynamic System Analysis,” J.
Elec. & Control, Vol. 17, 1964, pp. 577-591.
12. B. Pagurek, “Sensitivity of the Performance of Optimal Linear
Control Systems to Parameter Variations,” Int. J. of Control, Vol.
1, 1965, pp. 33-45.
13. J. B. Cruz and W. R. Perkins, “A New Approach to the Sensitivity
Problem in Multivariable Feedback System Design,” IEEE Trans,
on Automatic Control, July, 1964, pp. 216-223.
14. D. L. Lindorff, “Sensitivity in Sampled Data Systems,” IRE Trans,
on Automatic Control, April, 1963, pp. 120-125.
15. I. M. Horowitz, “Fundamental Theory of Automatic Linear Feed¬
back Control Systems,” IRE Trans, on Automatic Control, Decem¬
ber, 1959, pp. 5-19.
16. M. F. Gardner and J. L. Barnes, Transients in Linear Systems, New
York, John Wiley & Sons, Inc., 1942.
17. R. Tomovic, Sensitivity Analysis of Dynamic Systems, New York,
McGraw-Hill Book Company, 1963.
18. R. A. Frazer, W. J. Duncan and A. R. Collar , Elementary Matrices,
London, Cambridge University Press, 1938.
19. I. M. Horowitz, “Plant Adaptive Systems Versus Ordinary Feed¬
back Systems,” IRE Trans, on Automatic Control, January, 1962,
pp. 48-56.
20. I. M. Horowitz, “Linear Adaptive Flight Control Design for Re-
Entry Vehicles,” IEEE Trans, on Automatic Control, January,
1964, pp. 90-97.
21 . R. H. LaBountry and C. H. Houpis, “Root Locus Analysis of a High
Gain Linear System with Variable Coefficients: Application of
442 ELEMENTS OF MODERN CONTROL THEORY

Horowitz’s Method,” IEEE Trans, on Automatic Control, April,

1966, pp. 255-263.
22. M. Margolis and C. T. Leondes, “On the Philosophy of Adaptive
Control for Process Adaptive Systems,” Proc. NEC, Vol. 15, 1959,
pp. 27-33.
23. R. E. Kalman, “Design of a Self-optimizing Control System,” Trans.
ASME, Vol. 80, No. 2, February, 1958, pp. 468-478.
24. R. Staffin and J. G. Truxal, Executive-controlled Adaptive Systems,
Research Report R-688-58, PIB-616, New York, Microwave Re¬
search Institute, September, 1958.
25. G. W. Anderson and others, “A Self-adjusting System for Optimum
Dynamic Performance,” IRE Nat. Convention Record, Part 4,
1958, pp. 182-190.
26. E. Mishkin and R. A. Haddak, “Identification and Command
Problems in Adaptive Systems,” IRE WESCON Convention Record,
Part 4, 1959, pp. 125-135.
27. J. E. Gibson and E. S. McVey, “Multidimensional Adaptive Control,”
Proc. NEC, Vol. 15, 1959, pp. 17-26.
28. R. A. Rohrer and M. Sobral, “Sensitivity Considerations in Optimal
System Design,” IEEE Trans, on Automatic Control, January,
1965, pp. 43-48.
29. P. Dorato, “On Sensitivity in Optimal Control Systems,” IEEE
Trans, on Automatic Control, July, 1963, pp. 256-257.
30. R. E. Kalman, The Theory of Optimal Control and the Calculus of
Variations, Report TR61-3, Baltimore, Research Institute for Ad¬
vanced Studies, 1961.

PROBLEMS AND EXERCISES FOR CHAPTER THREE

1. In Fig. 3.1, let H(s) = 1, and

G(s)= K(s+ a) .
(s + b)(s + c)

We seek to determine the change in the closed-loop transfer func¬

tion, T(s), due to small changes in b. Derive an appropriate
sensitivity function for this case. Is T(s) more sensitive to positive
or negative increments in b?

2. Consider a unity feedback system in which the open-loop transfer

function has the form
 SENSITIVITY ANALYSIS 443

G(s)= — -(s-+1) .
s2 + 3s + 3.25

Construct the root locus for this system. Verify that for K = 1.0,
there are closed-loop poles at —2 ± j 0.5. Suppose now that the
open-loop poles are shifted to —1.75 ±j 0.85 and the zero shifts to
—1.20. Using the appropriate sensitivity relations, calculate the
new positions of the closed-loop poles.

3. In a certain feedback control system, the open-loop transfer func¬

tion is given by

_ 4_
G(s)
s(s + a)(s + 5)

The nominal value of the parameter, a, is 1.0. For this case, the
closed-loop poles are located at —0.40 ± j 0.80 and —5.1. Manu¬
facturing tolerances on the equipment require that permissible
limits be specified for the parameter, a. If it is required that
deviations in the parameter, a, should be in the direction of
increasing relative damping factor for the complex closed-loop poles,
should the permissible variation in a be positive or negative? Sub¬
stantiate your answer.

4. Given is the system, x = Ax, where

0 1 0
A = 0 0 1
-20 -29 -10

Calculate the eigenvalues and the corresponding eigenvalue sensi¬

tivity matrix. Use this to determine the new eigenvalues when the
element —10 changes to —15. Compare your result with the values
obtained by a conventional computation of eigenvalues.

5. The open-loop transfer function for a certain control system is given

by
444 ELEMENTS OF MODERN CONTROL THEORY

The transient response specifications require that a closed-loop pole

pair be located at —3 ± j 3. Determine a simple lead network
which will do this subject to the added stipulation that the relative
.damping factor at this closed-loop pole remain essentially constant
for small variations in open-loop gain.
 Chapter Four
STOCHASTIC EFFECTS

A comprehensive treatment of the control problem for space vehicles

requires a consideration of the various random effects that influence
overall performance. In this category, we include instrumentation noise,
parameter uncertainty, extraneous disturbances, etc. Underlying all these
cases is the fact that one is dealing with phenomena that are not pre¬
dictable in a deterministic sense. Statistical methods must, therefore, be
employed in some rational manner that yields useful results.
In this chapter, we are mainly concerned with random (hereafter called
“stochastic”) effects insofar as they influence the design of space vehicle
control systems. Attention will, therefore, be focused on three areas
where stochastic control principles have been employed to enhance
system performance.
The first of these deals with instrumentation noise, the word noise
being used in a generic sense to denote extraneous effects that contami¬
nate a given signal. The classical solution to this problem involves the
design of filters to suppress unwanted signals. The design of these filters
presupposes that the desired and parasitic signals can be characterized by
well-defined frequency bands. But if both the signal and noise are de¬
scribed only in a statistical sense, then a more sophisticated approach is
required. The modern approaches to this problem all stem from the
classic work of Wiener, (6) who laid the foundation for the optimal design
of filters that minimize the influence of random noise. The features and
use of some of these techniques are described in this chapter, especially
as they pertain to instrumentation noise commonly encountered in space
vehicle control systems.
A second and related problem is that of separating signal from noise in
a wider sense, i.e., when instrumentation noise is not the sole source of
445
446 ELEMENTS OF MODERN CONTROL THEORY

extraneous signals. It may be desired, for example, to control a system

optimally when there are extraneous disturbances whose general features
can be described only in a statistical manner. This would include the
design of a launch vehicle autopilot subjected to winds whose description
is given by suitable probability distributions.
Finally, it may be desired to determine various control system param¬
eters to minimize some designated error function when parasitic signals
are present.
All these methods are presently being used or are potentially useful in
the design of space vehicle control systems. Various applications are
described in this chapter.
So that this chapter will be reasonably self-contained, the exposition
proceeds from first principles. Derivations have been included, primarily
as an aid to understanding the basic principles and limitations of the
method. These derivations are characterized by extreme brevity and
generally phrased in the engineering vernacular, a procedure that has a
high degree of plausibility and intuitive appeal. The mathematical purist
may, therefore, find distress in such steps as unhesitatingly interchanging
the order of integration and assuming that various limiting processes are
valid. In all instances, however, suitable reference is made to rigorous
demonstrations in the literature. Because the development is, in many
instances, simpler (albeit less rigorous) than that available in standard
texts, it was deemed justified to include it here, especially as it contri¬
butes to the understanding of what is generally a difficult subject.
The period immediately following publication of Wiener’s classic
world6) on linear filters and stationary time series witnessed the appear¬
ance of a broad range of studies dealing with various extensions and
generalizations. The newer developments enlarged the scope of the
theory to include nonstationary time series and finite memory (time
varying) linear filters as well as some special results for nonlinear
filters.!? 4 2> These results were, for the most part, highly theoretical,
and the sophisticated mathematics made them inaccessible to most engi¬
neers. This condition was remedied somewhat by the appearance of
several texts(5,47) aimed specifically for control engineers.
Until only a few years ago, engineering application of the theory was
very limited. Wiener’s original problem related to the design of a realiz¬
able linear filter that minimized the mean square error when the filter
input was composed of a signal corrupted by additive noise and both the
signal and noise had well-defined statistical properties. A related study
by Phillipst47) showed how to vary some parameters of a fixed filter in
order to achieve mean square error minimization.
STOCHASTIC EFFECTS 447

The wide scope of the Wiener theory, however, stimulated fresh ap¬
proaches to related problems. Press and Houbolt(49) showed how some
of Wiener’s equations could be applied to gust loads on airplanes in the
sense of determining output power spectra from random inputs expressed
in power spectral form.
Perhaps the most significant new result since Wiener was obtained by
Kalman/13) who expressed Wiener’s formulation in state transition con¬
cepts in the time domain (rather than frequency) and thereby simplified
the mathematical structure immensely; more important, he greatly en¬
larged the area of application. Most of the engineering literature in the
last three or four years in stochastic control has been expressed in
Kalman’s format. The method enables one to treat problems of noise
and disturbance minimization in a highly systematic manner; it has been
used to design optimal filters and control functions as well as to identify
unknown system parameters. Bryson and Johansen(29) extended Kal¬
man’s technique to include the case of “colored noise.”
The theory of Wiener-Kalman filtering (as it is now called) has become
virtually classical in a very short time. The basic results are firmly and
rigorously established, and the theory has assumed a prominent role in
modern control system design. Recent studies have demonstrated that
an intimate relationship exists between the Wiener-Kalman theory and
related statistical optimization methods such as linear regression/53)
maximum likelihood/54) and dynamic programming.!25) A summary of
these ideas is contained in a stimulating paper by Smith/55) A judicious
blend of advances in related fields will no doubt further enlarge the
scope and application of stochastic control concepts.

4.1 MATHEMATICAL PRELIMINARIES

This section presents some of the essential aspects of the theory of

probability and random processes. The treatment is extremely brief,
highlighting definitions and results pertinent to the subject matter. In
keeping with the theme of the presentation, we emphasize physical
understanding rather than mathematical abstraction, with the hope that
the underlying ideas can be made intuitively plausible. We do not seek,
however, to disparage the merits of a rigorous treatment, which is neces¬
sary for some of the more sophisticated applications of the theory. For
this, we must, of necessity, refer the reader to standard references/1*3)
448 ELEMENTS OF MODERN CONTROL THEORY

4.1.1 Random Variables

The concept of a random variable is fundamental to all that follows.

A random variable is defined as a function whose value depends on the
outcome of a chance event. Generally, in any chance event, certain
outcomes are “more likely” than others. This idea is made precise by
defining a distribution function F(x), as follows.

F(x) = Prob (R < x)

In words, F(x) is the probability that the random variable, R, takes on

a value equal to or less than x.
Since the probability of an event must lie between zero (impossibility)
and one (certainty), it follows that F(x) must satisfy the following.

F(a) < F(b) if a < b (1)

IT II o (2)

F(oo) = 1 (3)

If F(x) is differentiable, as will be assumed for the most part here-

after, then we define the probability density function by

d F(x)
f«- a*
Since F(x) is nondecreasing, it follows that

f(x) > 0 for all x . (4)

Furthermore, if a < b, then

= Prob (R < b) - Prob (R < a)

= Prob (a < R < b) . (5)

Also
STOCHASTIC EFFECTS 449

f f(x)
^ — ao
dx = F(x) - F(— oo) = F(x) (6)

[ f(x) dx = F(oo) — F(— oo) = 1 . (7)

Associated with each probability density function, f(x), of a random

variable, R, are the following. /

Mean:

E(R) -- m (8)

Mean Square:

E(R2) (9)

Variance:

E[(R — m)2] = a2 = v — m2
(10)

= / (x — m)2f(x) dx .
J — oo

Quantity a is called the standard deviation.

The moments of the density function are defined by

an = E(Rn) = f x"f(x)dx (11)

* — ac

while the central moments are defined by

m*= E[(R—m)n]= f (x —m)"f(x)dx . (12)

J — ao

Note that

ao = 1 Mo = 1

oti — m Mi = 0

Oi2 = v M2 = <72

7 2
a1 = ct2 — a\
450 ELEMENTS OF MODERN CONTROL THEORY

A knowledge of the density function serves in defining the properties

of the random variable completely. If the density function is not known,
then the statistical properties of the random variable are described in
terms of its first and higher moments. The more of these that are known,
the more complete the description of the random variable. In most cases
of interest, a knowledge of the first two moments is sufficient to describe
the properties of the random variable. Some of the more common
probability density functions are listed below.

Binomial:

n! (13)
f(x) p‘(l - p)n~x .
x!(n — x) !

f(x) is the probability that an event will occur x times in n trials, where
p is the probability that the event will occur in a given trial.
Here n and x are positive integers such that 0 < x < n, and p is a real
number between zero and one. It follows readily that

m = np (14)

<r2= np(l — p) . (15)

Poisson:

Ax
-A
f(x) xl e

(16)
x = 0, 1, 2,

A > 0 .

In this case

m = cr2 = A . (17)

Gaussian:

(x — m)2 (18)
f(x)
2 a2
STOCHASTIC EFFECTS 451

The above results are readily generalized to the case of n random

variables, Ri, R2, . , R*. For example, the joint distribution func¬
tion, F(xi, x2, . , x„) is defined by

F(xi, x2, . , xn) = Prob (Ri < xi, R2 < x2, . (19)
. j Rn ^ X„] .

In words, F(xx, x2, . , x„) is the probability that R< takes on a value
equal to or less than x, for i = 1, 2, • • • •, n. All the inequalities must be
satisfied simultaneously.
The joint probability density function is defined by

dnF(xi, x2, . x,) (20)

f(xi, x2,
dxi dx2 . dxn

The quantities of interest are the following.

Mean:

m, = E(R») . (21)

Variance and Covariance:

i = j ; variance
an E[(R, - m,)(Ry - my)] (22)
i ^ j ; covariance

Correlation:

pij = Gi}(a<i ajj) 1/2 % (23)

Of special interest in the subsequent analyses is the multivariate

Gaussian ( normal) density function given by

1 — — (24)
f(xi, x2, . , x,) = [(2x)»det V]-1'2 exp[— —(x - m)^V-(x - m)].

Here x and ni are n vectors and V is an n X n matrix whose typical

element is an] viz.

V = E[(R - m)(R - m)*1] . (25)

452 ELEMENTS OF MODERN CONTROL THEORY

In the sections that follow, we shall deal extensively with probability

distributions of two variables. The following definitions define the
parameters of interest in this case.

Moments:

(26)

Central Moments:

Hjk = E[(Ri — mi)J'(R2 — m2)'c]

(27)

Note that, in particular

«io = E(Ri) = mi

«oi = E(R2) = m2

M20= E[(Ri — mi)2]

= a20 — mo2 -- variance of Ri

M02= E[(R2 — m2)2]

= a 02 — aoi2 = variance of R2

Mu = E[(Rj — mj)(R2 — m2)]

= an — am a0i = covariance of Ri and R2 .

Note also the following equivalence.

0T2 — Mil

(Til — M20

<r22 — M 02
STOCHASTIC EFFECTS 453

The quantity

P12 — 0'12(o’ll<722)1,2

= AUi(m20
M02)^^2

is called the correlation coefficient of Ri and R2.

The fundamental quantities of interest in relation to a random variable
are the mean and the variance of the probability density function. To
obtain some physical insight into the significance of these quantities,
consider the Gaussian (or normal) density function shown in Fig. 4.1.
When the mean, m, equals zero, the curve f(x) vs x is symmetrical about
the x = 0 axis as shown in Fig. 4.1a. For any nonzero value of m, the
curve is merely shifted parallel to the f(x) axis in the manner shown in
Fig. 4.1b. Parameter a gives a measure of the rapidity with which the
curve drops off. Figure 4.1 c shows two Gaussian density functions where
<72> <7i. It is apparent that the smaller the value of <7,the steeper the
curve.
As an illustration of the use of this curve and the order of magnitude
of numerical quantities involved, let it be required to determine the prob¬
ability that a random variable, R (whose density function is Gaussian),
assumes a value in the range (m — b) to (m + b) for a given value of b.
From Eq. (5)

Prob [m — b<R<m + b]= / f(x) dx

m—b

where

In order to use available tables for the Gaussian density function, we

convert to “standard” form via the substitution x = W + m. This leads
to

Prob [m — b<R<m + b] = F(m + b) — F(m — b)

454 ELEMENTS OF MODERN CONTROL THEORY

f(x)

Figure 4.1. The Gaussian Density Function.

Using standard tables/2) we find

Prob [m — b < R < m + b] = 0.6827 if b = a

STOCHASTIC EFFECTS 455

= 0.9545 if b = 2a

= 0.9973 if b = 3a .

In other words, the smaller the value of a, the greater the tendency of
the random variable to assume a value close to the mean.

4.1.2 Random Processes '

By a random process we shall mean a collection or ensemble of func¬

tions of time having certain statistical properties. A typical case is shown
in Fig. 4.2. The random process is characterized by probability distri¬
bution functions defined as follows. Let x(t) denote a representative
member of the ensemble |x(t)). Then x(ti) is a random variable in the
sense defined in Sect. 4.1.1. A probability distribution function, Fi(xi, ti)
is defined by

Fi(xi, ti) = Prob [x(ti) < Xi] (28)

where xt is a prescribed number. The probability density function is

dF(xi, ti) (29)

fl(Xi, ti)
dXi

Time

Figure 4.2. An Ensemble of Random Functions of Time.

456 ELEMENTS OF MODERN CONTROL THEORY

It follows that

Prob [x, < x(ti) < Xi + dx^ = fi (xi, ti) dxi . (30)

Similarly, the probability that simultaneously x(ti) < xj and x(t2) < x2
is given by the second probability distribution function

F2(xi, ti ; x2, t2) = Prob [x(ti) < Xi; x(t2) < x2] . (31)

The corresponding probability density function is given by

d2F2(xi, ti; x2, t2) (32)

f2(xi, ti ; x2, t2)
dxi dx2

Similarly

F,(xi, ti; x2, t2; x3, t3) = Prob [x(ti) < xj; x(t2) < x2.; x(t3) < x3

etc. Each fB implies all previous f* for k < n by the relation

(33)
fn-i(xi, tj ; /oo
fn(Xl, tl,
— OO
xK, t„)dx„

As in Sect. 4. 1 . 1 , we define the mean by

rrii = E[x(t»)] (34)

and the covariance by

aij = E{[x(t») - mj [x(ty) - m,-]J . (35)

It is important to note the distinction between the definitions for a

random variable and for a random process.
The covariance may be written in expanded form as follows.

cuj = E[x(ti) x(ty)] — m,my

The first term on the right-hand side of this expression plays a funda¬
mental role in stochastic control theory, and it is accorded a separate
name and symbol.

<Pxx{ti,ty) = E[x(tf) X(ty)] (36)

STOCHASTIC EFFECTS 457

We call this the autocorrelation function of x, the subscript xx indi¬

cating correlation of x(t) with itself as distinguished from cross correla¬
tions to be discussed later.
The set of density functions fi, f2, . , f„ describes the random
process in ever increasing detail. In addition to the mean and covariance
already defined, higher order moments may also be defined in a manner
analogous to that of Sect. 4.1.1. However, these will not be required
for the topics to be dealt with and they are, therefore, not pursued
further.
It is often required to consider the statistical relationships between
two random processes, {x(t) } and {y(t)). For this case, we define a
general joint distribution function as follows.

Fm„(c)(xi, ti, . , xm, tm; yi, t{, . . y„, t;)

= Prob [x(ti) < xi, . , x(tm) < xm; y(tj;) < yi,
. , y(t*) < yn]

The associated joint density function is

fmn(c)(Xl, tl, *, XTO,t TO,yi, ti, % y*,

, tn)
(37)
dm+nF ‘

3xi •dxm dyi % dyn

In practical applications, only the joint density function, fu(c)(xn ti:

ylf t/), is used extensively. Of the moments associated with these joint
density functions, the only one that will be used subsequently is

(38)
E[x(ti) y(t2)] f
j — 00
f x(ti)y(ts)
J — CO
f^Cxi,
ti; yi, t2) dxi dyi .

This quantity is called the cross correlation function and is denoted by

the symbol

ipxyiti, t2) = E[x(ti) y(t2)] . (39)

If x {(t) } and {y(t) } are statistically independent— that is, if the value of
x(t) does not depend in any way on the value of y(t)-then

E[x(ti) y(t2)] = E[x(ti)] E(y(ts)] . (40)

458 ELEMENTS OF MODERN CONTROL THEORY

We sometimes deal with a signal of the form

z(t) = x(t) + y(t)

where {x(t)| and (y(t)} are two given random processes. In this case, we
find, directly from the definitions

<pzz(
tl, t2) = t2) + (Pxy(tl, t2) + t2) + tl, t2) . (41)

From Eq. (39), it is obvious that in general, <pxy(ti,t2) ^ t2).

Note that if (x(t) j and (y(t)} are statistically independent, then

<fxJ/(tl, t2) — 4>y

i(tl, t2) — 0 (42)

if one or both of the random processes has a zero mean value. In the
particular case when Eqs. (40) and (42) hold, then Eq. (41) reduces to

<Pzz(
tl, t2) = <pxx(tl,t2) + <Pyy(tl,t2) • (43)

A random process is said to be stationary if its statistical properties do

not vary with time. An important consequence of this property is that

E[x(t + r)] = E[x(t)] .

The particular choice r — —t gives

E[x(0)] = E[x(t)] .

The autocorrelation function for a stationary random process satisfies

<Pxx{
tl + T, t2 T" t) = <Pxx(
tl, t2)

for all T. In particular, if T = —ti.then

<Pxx(t>l,
t2) — PxxiO, t2 tj) .

In other words, the autocorrelation function for a stationary random

process depends only on the time interval, not the specific values of
time. It is appropriate to denote the autocorrelation function for a
stationary random process by a special symbol; thus

r**(r) — ^ix(0, t2 ti) (44)

STOCHASTIC EFFECTS 459

where r = t2 — ti. In similar manner, for the cross correlation function

IVv(t) = “PatyCO,
t2 tl) . (45)

Note that

r*y(r) = E[x(t) y(t + t)\ . (46)

The autocorrelation function satisfies^the following

r,x(r) = r «(-T) (47)

ITxt(t)| < r„(0) (48)

If x(t)does not contain any periodic component, then

lim Yxt{t) —* 0 . (49)

A stationary random process is often assumed to be ergodic. Generally

speaking, this property permits one to equate ensemble averages and
averages with respect to time performed on a single representative
member of the ensemble. More precisely, a stationary random process is
said to be ergodic if every member, x(t), of the ensemble {x(t) } satisfies

E{V[x(t)]} = lim — (50)

+ r)] dt
’ r— 2T

where V[x(t)] is any random variable associated with x(t);e.g., V = x(t),

x2(t), x(ti) x(t*).
When the ergodic property holds, we have

(51)
E[x(t)] = lim —
2T
[1 x(t)dt
J _ rp

E[x2(t)] - lim —
T—>» 2T
f% x2(t)
J _ rp
dt (52)

Yxz(t) = E[x(t) x(t + r)]

1 fT (53)
- lim — /x(t) x(t + r) dt .
r->« 2T J-T
460 ELEMENTS OF MODERN CONTROL THEORY

The ergodic property is important because it eliminates the need for

dealing with a large ensemble in order to calculate probability density
functions. For example, if the random process is stationary, then

r**(r) = ail = / XiX2f2(xi,x2, r)dxi dx2 .

However, if the process is also ergodic, then the two foregoing

quantities are given in much simpler form by Eqs. (51) and (53) which
do not involve probability density functions. This results in a crucial
simplification for computational purposes.

4.1.3 Random Signals and Linear Filters

Given a random stationary signal characterized by appropriate statisti¬

cal parameters, we may pose the question, “If this signal is applied at the
input of a linear filter, what is the output?” A linear filter is best de¬
scribed by its transfer function, which requires that the input signal be
decomposed into its harmonic components if the signal is periodic. The
action on each harmonic component is then determined and the results
superimposed.
When the signal is not periodic, it cannot be decomposed into discrete
harmonic components; but if it has a Fourier (or Laplace) transform,
then it has a continuous frequency spectrum that can be treated mathe¬
matically as a spectrum of harmonics. It is natural, therefore, to investi¬
gate the properties of the Fourier transform of x(t), viz.

An immediate difficulty is encountered. If x(t) is stationary, then its

statistical properties do not vary with time; however, x(t) can wander
randomly ad infinitum with respect to time. In short, the integral in the
above expression may not converge, in which case its Fourier transform
does not exist. However, while an expression such as
STOCHASTIC EFFECTS 461

may not converge, it happens that the quantity

1 rT (54)
E[x*(t)] = Urn-L x2(t) dt ^ '
2T J -t

which represents the mean square* of x(t), is finite for nearly all random

phenomena of interest. It is natural to think of / x2(t) dt as being a

measure of power, in which case Eq. (54) represents average power in

the interval ( —T, T).
We now have a foothold on a method of converting a quantity of
interest, expressed in the time domain, to an equivalent formulation in
terms of frequency spectrum. The procedure is as follows. Define

xr(t) = x(t) —T < t < T

= 0 otherwise. (55)

The Fourier transform of xT(t) exists and is given by

/oo
Xr(t)e-,“‘
- oo
dt

(56)

(57)
:r(t)-±.[ Xr(&»)e,'a* da> .

The quantity defined by

Xr(co)|2 (58)
: lim
2T

is called the power spectral density, f We will show that this quantity
exists and is finite if the mean square value ofx(t) is finite.

*See Eq. (52).

f Some authors call this the power density spectrum, spectral density, power spectrum, etc.
462 ELEMENTS OF MODERN CONTROL THEORY

We have

f |Xr(co)|2daj
= f Xf(co)Xy(—oj)do
" — oo " — 00

/oo — 0O
XT(co)dw / X]p(t)e'"‘dt
*
/*oo
— 00

/oo — CD V
/*oo
xj-(t) dt / Xr(w)e'“‘ dco
- 00

/oo
xr*(t)
- CO
dt .

Therefore

lim— f lXr-(-^2
dw= lim— [ xj(t)dt = lim— [ x2(t)
dt.
r-*.2w•'-« 2T r-.. 2T•'-» r-»«2T
J-r
This last quantity is the mean square value of x(t), which is finite by
assumption. Consequently, the power spectral density defined by Eq.
(58) exists, since its integral converges.
It may be shown that the autocorrelation function is the Fourier
transform of the power spectral density. To do this, define

Cr(r)= — f
2T J-t
x(t)x(t+ r) dt

which is merely Eq. (53) with T finite. Note that

lim C t(t) = IT*(t) .

t-> »
The Fourier transform ofCr(r) is

lim[ Cr(r)e~;“T
T - %º
oo J — 00
dr

= lim —
r-- 2T
/OO /•oo
e_)<JT
—00
dr / xr(t) xr(t + r) dt
j —00

= lim —
2T
/OO
dt /i»0O
-
[xr(t)eJ’“(][xT(t
00 " — oo
+ r)e->'“(,+T)]
d7
STOCHASTIC EFFECTS 463

= lim —
T—*« 2T
/OO
- CO
poo
xT(t)eia‘ dt / xr(ti)e_J'“‘i d^
J — ao

= lim — Xr( Ct))

X.T’(co)
r— 2T

= lim|Xr(a,)l2
2T
= G**(w)

by Eq. (58).
This means that

(59)
G**(co) = / rxx(r) e )lut dr

and

(60)
e'“T do .

Since both ria:(r) and GIX(co) are real-valued even functions of their
respective arguments, we may write

(61)
Gxx(co) — 21 r xx(r) cos cor dr
Jo

1 f*3 (62)
r«(r) = — / Gxx(co) cos cor dco .
•7r " n

These are known as the Wiener-Khinchin equations.

Remark: There is a distressing lack of uniformity in the literature, re¬

garding the definitions of Gxx(co) and the Fourier transform.
The former is variously defined as

|Xr(co)|»|Xr(co)|2etc.
2ttT

while the Fourier transform pairs are defined by

464 ELEMENTS OF MODERN CONTROL THEORY

00

F(w) e,u‘ dco

(Ref. 5)

or

(Ref. 48)

The Fourier transform pair, Eqs. (56) and (57), is apparently the most
widely adopted.* This form will be used consistently in this chapter.
There even seems to be some inconsistency by the same author. Laning
and Battin (Ref. 5, p. 123) use

F(w)= —U f f(t) dt
to define the Fourier transform, while in deriving the Wiener-Khinchin
equations (Ref. 5, p. 132), they write

Needless to say, great care must be exercised in comparing results

derived by different authors.
We now return to the question posed at the beginning of this section,
namely, how does a linear filter affect a random signal? To answer this
we proceed as follows.
If x(t) is any signal applied at the input of a linear filter whose
weighting function is h(t), then the output is given by1

*e.g., Refs. 12, 46, 47.

f See Appendix B.
STOCHASTIC EFFECTS 465

/oo
0
h(u) x(t — u) du .

The autocorrelation function of y(t)is

rw(r) = limj- [ y(t) y(t + r) dt

r—» 2T •'-j, '

fT roo t* oo

= lim
/ T
J — J'
*0
/ h(u) x(t — u) du
J
/ h(v) x(t + r — v) dv dt .
0

By interchanging the order of integration, we have

1 f
Tm(r)
= r /”h(u)
h<v)lim — J x(t —u) x(t + r —v) dt du dv

If we make the change of variable, ti = t - u, inside the bracketed

quantity, then

lim — I x( t — u) x(t + r — v) dt
r-..2T J-T

1 (T
= lim — / x(ti) x(ti + u + r - v) dti = ria:(u — v + r)
9T J _
2T

via Eq. (53).

Therefore, r w(r) may be written as

/OO /i»00
v j0
h(u) h(v) rM(u — v + r) du dv .

Taking the Fourier transform of both sides and making use of Eqs.
(56) and (59), we find

/OOJ/'CO
/»
C O
J h(u)h(v)rM(u—v -(-r)dudv e~} dt

After a change in the order of integration, this becomes

466 ELEMENTS OF MODERN CONTROL THEORY

roo noo r oo

h(u) h(v) / rix(u — v + r) e~ia,Tdr du dv .

J „ J n j — 00
0 0

Now by making the change of variable, f = u — v + r, this simplifies to

Gj,(u) = / h(u) e’wudu / h(v) dv / rIX(f) e_i“f df .

Jq \ —00

However, by the definition of the Laplace transform

/co
h(t) e-“ dt

S = a + jcj

while

/ r**(0 e-J'"Sdf = Gxx(w)

» — oo

by Eq. (59).
We have, finally

Gy3,(w)= H(—jw) H(jto) G

= |H(j«)|*Gx,(«) . (63)

In the usual terminology, H(jw) is the transfer function of the linear

filter.
Equation (63) is of fundamental importance. It shows how the power
spectral density of a random signal is altered when the signal passes
through a linear filter. This relation will be used often in the following
sections.
The discussion thus far has not been limited to any specific type of
random process. It turns out that much random phenomena of interest
can be described in terms of normal or Gaussian distributions. More
specifically, a random process is termed Gaussian ( normal) if it is char¬
acterized by the n-dimensional probability density function/1 7)

f«(xi, ti; ; x n> t»)

= [(2-tt)” det M]~1/2 exp[— - (x — m)r M-1(x ni)] % (64)

STOCHASTIC EFFECTS 467

Here x is an n vector whose typical element is x,-, while m is the mean

vector, a typical element of which is

m < = E[x(t,)] . (65)

M is the covariance matrix whose ijth component is given by

a a = E{[x(t,-)- mj [x(ty) — my]}

= ty) - m.niy . (66)

If the process is stationary, then m, = m; = m and

o ij = riI(riy) — m2 (67)

where rty = t,- — ty. The Gaussian random process is completely deter¬
mined from its joint density function f2 (xx, t»-; x2 ty) because the auto¬
correlation function is expressed in terms of this density function by

/oo/I*
co
x(ty)
—00 j - <30
x(ty) f2(xj, t,; x2, ty) dxx dx2 . /Z"0\
The Gaussian random process has the important property that it
remains Gaussian after passing through a linear filter.

4.1.4 Practical Considerations

A fundamental problem in the analysis of random (stochastic) pro¬

cesses is the determination of the statistical properties that characterize
the process. In the general case, the problem is very formidable. For
purposes of computational and theoretical expediency, one is compelled
to make various assumptions that are valid in greater or lesser degree in
practical situations. In the first place, most random phenomena of
interest are stationary; that is, in general, the statistical properties are
invariant with respect to translation along the time axis. This property
affords crucial simplifications in both theoretical analysis and computa¬
tional methods. One often assumes also that the process under considera¬
tion is ergodic. This assumption enables one to compute the statistical
properties from a single long record rather than an ensemble of the type
shown in Fig. 4.2. Hereafter, unless otherwise noted, we shall treat
only those random processes that are both stationary and ergodic.
468 ELEMENTS OF MODERN CONTROL THEORY

Consider now a single noise record of the type shown in Fig. 4.3. This
is assumed to be a typical member of an ensemble representing a station¬
ary, ergodic, random process. We are interested in calculating the mean
and autocorrelation function. One may proceed as follows. Divide the
trace of Fig. 4.3 into n equally spaced points a distance, d, apart. With
the kth point is associated an ordinate, a*, that may be positive or nega¬
tive. The mean is simply determined by using the discrete version of Eq.
(51), viz.

(69)
E[x(t)] = - a* •
0 i=l

The autocorrelation function is obtained from the discrete form of

Eq. (53), viz.
n
1 (70)
Txx(t) at ai+*
n i— 1

where r = kd. The curve of r**(r) vs t is plotted in Fig. 4.4. The general
shape of this curve is typical of a wide variety of random phenomena of
interest, such as vacuum tube noise, radar fading records, and atmospheric
turbulence. It may be very closely approximated by

Txx(t) = A e_fe|rl cos c r (71)

where A, k, and c are positive constants. The power spectral density may
be obtained from Eq. (61), viz.

oi2 + (k2 + c2) (72)

Gxx(oj) = 2 Ak
+ 2(k2 - c2)a>2+ (k2 + c2)5 ]•
The shape of this curve depends on the quantity (3c2 — k2) and is de¬
picted in Fig. 4.5. Consider now a special case of Eq. (71) in which
A = k/2 and c = 0. It is not difficult to show that for these parameters

lim - ke~ft|T| =0 r ^ 0
k—
»°°L2 J
and

k r°
lim - / e-A,T|dr = 1 .
k—>00 2 •'-CO
STOCHASTIC EFFECTS 469

c
d

8
oi
<D
.52

Z
n>
Tb
c

rn

a)

.1
tL

ci
470 ELEMENTS OF MODERN CONTROL THEORY

Consequently, with Yxx(r) defined by

r»(r) = — e~A,T'

it follows that

lim [r**(r)] = 5(r)

A:—+oo

where 5(r) is the Dirac delta function.* This last relation simply states
that successive values of the random function are completely uncorre¬
lated, no matter how small the interval between successive samples. This
is, of course, a mathematical abstraction that is never realized completely
in practice. Nevertheless, this concept is a highly useful theoretical tool
and closely approximates many important random phenomena (e.g.,
Brownian motion). The idea of a purely random process, often called
white noise, may be arrived at in many ways. Some of these are dis¬
cussed in Appendix A. However, in general, we will say that a random
process is white if its autocorrelation function is given by

r„(T) = B8(t) (73)

where B is a positive constant. The power spectral density for Eq. (73)
is obtained by direct application of Eq. (62), viz.

*See Appendix A.
 STOCHASTIC EFFECTS 471

Gxx(“)

a. 3C2- K2 < 0

G
XX
(W)

2 2
b. 3C -K >0

Figure 4.5. Curves of Power Spectral Density.

G„(«) = B . (74)

Thus, the power spectral density is a constant, which, by analogy with

the spectrum for visible light, accounts for the name “white noise.”
We now proceed to a discussion of the Wiener theory and some of its
ramifications.
472 ELEMENTS OF MODERN CONTROL THEORY

4.2 THE WIENER THEORY

The transmission of information, whether by electrical, mechanical,

social, or, indeed, biological channels, often has the effect of introducing
extraneous signals that materially corrupt the basic signal. To fix ideas,
we consider a signal, f<(t), made up of two parts, as follows.

fi(t) = f.(t) + fn(t) (75)

Quantity f,(t) represents the uncorrupted signal, while f*(t) is the

extraneous information. In the real world, we have available only f*(t),
and it is natural to investigate means of extracting the true signal, fs(t),
from its “noisy” environment. The term noise will be used in a generic
sense to describe any extraneous and undesirable signals that corrupt
useful information.
Simple means of achieving this are well known and elementary. For
example, if f,(t) is known to be concentrated in one frequency band
while the noise is generally contained in another (often much higher)
frequency band, simple passive filtering will serve to eliminate virtually
all of f„(t) without disturbing f,(t). Again, if the noise is known to be
restricted to one frequency or a very narrow range of frequencies (e.g.,
the 60-cycle hum in radio receivers), a narrow band attenuator or notch
filter will effectively “clean up” the signal.
The problem becomes substantially more difficult when one considers
not single signals, but whole classes of signals; both the basic signal and
the noise can be described only in some statistical sense. Furthermore,
fs(t) can be separated from f„(t) only if their statistical descriptions con¬
tain some distinguishing features. In addition, there exists the problem
of adopting some criterion of how well this separation is accomplished.
Posed in this fashion, the problem is very formidable. By making
three basic assumptions, Wiener(6) obtained an elegant solution and laid
the groundwork for all subsequent research in stochastic control theory.
These assumptions are:

a. The signal, f,(t), and the noise, f„(t), are each members of a stationary
random process.
b. The device used to operate on f,(t) is a physically realizable linear
filter.*

*A linear filter is said to be physically realizable if its transfer function has no poles in the right-
half plane or on the imaginary axis.
STOCHASTIC EFFECTS 473

c. The criterion to be used in selecting the “best possible” linear filter

is the rms difference between the actual and desired outputs.

By restricting the permissible operations to linear filters, we immedi¬

ately have, for the filter output

(76)
fo(t)
= j h(r)f,-£t
- t)dr
where h(r) is the weighting function* of the filter. Since this filter is to
be physically realizable, we must have

h(r) =0 for r < 0 (77)

since a real system cannot respond before an input is applied. In this

case, nothing is changed if instead of Eq. (76) we write

(78)
fo(t) = h(r) f,(t - r) dr .

what follows, it will be more convenient to use Eq. (78) than Eq.
In
(76). If we denote the desired signal by fd(t), then the rms error is given
by

(79)
fd(t)p dt .

The desired signal, fd(t), may be any one of a multitude of functions

of f,(t). Usually we take

fd(t) = f.(t)

which constitutes the filter problem. If we let

fd(t) = f.(t + a)

we have the filtering and prediction problem. In the absence of noise,

the latter is called simply pure prediction.

See Appendix B.
474 ELEMENTS OF MODERN CONTROL THEORY

By substituting Eq. (78) in Eq. (79) and making use of the fact* that

1 fT (80)
ro6(r) lim — /
r->«°2T ^-T
f0(t) f6(t + r) dt

we obtain ^

/OO
h(r)dr Jpoo
h(cr)
Ta(r—a)d<r
(81)
- 2 / h(r) rM(r) dr + r*,(0) •

Via conventional application of the variational calculus, it is found

that the value of h(r) that minimizes Eq. (81) is given by the solution
of the integral equation

Tid(r) — f h (<r) r«(r — a) dcr = 0, r > 0 .

J — OO

This is the well-known Wiener-Hopf equation. This equation could

be solved in routine fashion were it not for the constraint, Eq. (77).
As a matter of fact, if this latter constraint were neglected, one would
obtain

Gras(tu)
H(s) Hrf(s) (83)
Gss(co) + Gnn(w)

where

H(s) = £[h(t)]

'Hd(s) = 1 if f„(t) = f,(t)

= eaa if fd(t) = fi(t + a) (84)

and G„(g>) and Gnn(w) are the power spectral densities for f,(t) and f„(t),
respectively.

*See Sect. 4.1.2.

fit has been assumed here that the signal and noise are uncorrelated; i.e., r5n(r) = 0.
b.
Form
the
func
STOCHASTIC EFFECTS 475

The solution of Eq. (82), which takes account of condition Eq. (77),
is summarized here as follows.*

a. Given the power spectral densities, G„(w) and G»»(w), for the signal
and noise, respectively, replace « by s/j and form the sum

Gi,-(s) = Gss(s) + Gnn(s) . (85)

/

Factor this quantity as follows.

G„(s) = Gi(s) Gi(-s) = |Gi(s)|2 (86)

W»(s) = Gaa(s)Hd(s). (87)

Gi(-s)
Here, Hd(s) is the transfer function relating fd(t) to fs(t). For
example, if fd(t) = f,(t), then Hd(s) = 1. This is the case of simple
filtering. If fd(t) = f,(t + a), i.e., filtering and prediction or pure
prediction, then Hd(s) = eos. The right-hand side of Eq. (87) is ex¬
panded in partial fractions, and the inverse Faplace transformation of
the physically realizable^ terms is calculated. As a result of this
operation, we have

hi(t) = 0 t < 0

= hi»>(t) t > 0 . (88)

c. By taking the Faplace transform of Eq. (88), we obtain

Hx(8) = JBfluCt)] . (89)

d. The transfer function of the required physically realizable linear

filter is then given by

Hx(s) (90)
H(s)
Gi(s) '

The method is illustrated in the following examples.

* Details of the analysis leading to the result given here may be found in Refs. 5 and 6. The
method as outlined here appears to be substantially simpler than that given in most references.
fThat is, those terms having poles in the left-half plane only.
476 ELEMENTS OF MODERN CONTROL THEORY

Example 1: Given
1
Gss(co)
1 + CO2

Gnn(<o) = k2

we seek to determine the optimum realizable filter for a prediction time

interval, a.
The following steps are a direct application of the procedure outlined
above and require no further explanation.

1 + k2 - k2 s2
Gij(s) + k2
1 — s2

= G,(s) Gx(-s)

where

\T + k2 + ks
Gi(s)
1 + s

Now

1 1 - s
Hd0Hs)
1 - s2 Vl + k2 - ks

Ai A2
ea“
S + 1 vl + k2 - ks

where

_ 1_
Ai
k + X/1 + k2

k
A2
k + x/1 + k2 '

Only the first term inside the bracket represents a physically realizable
filter. Taking the Laplace transform of this yields
STOCHASTIC EFFECTS 477

hi(t) = 0 t < 0

= Aie~<'+a) t > 0 .

Consequently

Hi(s) = <£[Aie~((+a)]
/
Aie~tt
s 4- 1 '

We have, finally

H(s) - SW . _ Aier" .
Gi(s) ks + Vl + k2

This is the transfer function of the required optimal linear filter.

Example 2: The given power spectral densities are

Gss(o>) 6 _ 1_
ir (“2 +qT

2 (a>2+ 2)
G
r (o>4+ 4)

As in the previous example, it is required to determine the optimum

filter for a prediction time interval, a.
Proceeding as before

( 2 (2 - s2)
G«(s) = —
vr (f - S’) 7T (4 + S4)

_ 8 (s2 + 2.264s + 2,031) (s2 - 2.264s + 2.031)

‘ * (f + S)(f - s)(s* + 2s + 2)(s* - 2s + 2)

We have therefore

Gi(s) (s2+ 2.264s+ 2.031)

(s -f |)(s2 + 2s + 2)
478 ELEMENTS OF MODERN CONTROL THEORY

Hi<°>(s) = Hd(s)
G,(-s)

where Hd(s) = eos. This leads to

(s2 — 2s T- 2)ea8
HiW(s) = —
2tt
(s + i)(s2 -2.264s + 2.031)

Expanding this in partial fractions and taking the Laplace transform

of the physically realizable portion results in

hx(t) = 0 t < 0

(V) X09444
e_1'6('+o) t > 0 .

Therefore

2.004 e_1-8a
Hi(s)
V 7T
(S + |)
and finally

Hi(s)
H(s)
Gx(s)

which, in expanded form, becomes

0.7085 e~1,6° (s2 + 2s + 2)

H(s)
(s2 + 2.264s + 2.031)

Example 3: The given spectral densities are

Gss(w) = - —
4 + w2

25
Gnft(co)
25 + co2

We seek to determine an optimal realizable filter (no prediction in the

present case).
STOCHASTIC EFFECTS 479

The following steps are again self-explanatory.

G„(s) —
4 — s2

25
Gn^(s) =
25 - s2

and

r, , ,
G;,(s) = -
1 ,h 25
4 - s2 25 - s2

'5.0990
(s+ 2.1926)1
|"5.0990
(2,1926
- s)l
. (s+ 2)(s+ 5) J L (2- s)(5- s) J

5.0990(s + 2.1926)
GM =
(s + 2) (s + 5)

Hj^Cs) - 1 %X (2 S)(5 S)
(4 - s2) 5.0990(2.1926 - s)

= 0.1961
[Hggi
L(s+2)
+ °-6!9-6 1
(2.1926—s) J

hi(t) = 0 t < 0

= 0.3274 e_2‘

0.3274
Hi(s) =
s + 2

and finally

0.3274 (s + 2) (s + 5)
(s + 2) X 5.0990(s+ 2.1926)

0.0642(s + 5)
(s + 2.1926)
480 ELEMENTS OF MODERN CONTROL THEORY

Remark: It is pertinent at this point to review the significance and

interpretation of the results thus far obtained. What has been
done essentially is to consider a class or ensemble of signals
about which only limited statistical data is available, namely
the autocorrelation function.* The signals are corrupted by
additive noise for which the autocorrelation function is also
known. What has been done is to derive the form of a physi¬
cally realizable linear filter that is optimum in the sense that
the rms error between the actual and desired output is mini¬
mized.
Let us suppose that in Example 1, for instance, there is no
noise present. We find, in this case, that the optimum filter is
simply an attenuator of the form H(s) = e-a, where “a” is the
prediction interval. This result is somewhat surprising, since
it states that the “best” prediction of the signal is merely a
constant (less than unity) times the present value of the signal.
The usefulness of this result may be seriously questioned. It is,
however, the solution to the problem under the conditions
stated. It is certainly the most that can be expected if the
available statistical information of the signal is limited to the
correlation function (or power spectral density), and the opti¬
mum operator is to be a linear filter, optimal in the least square
sense. In short, limited data can yield only limited information.
We note in passing that, in the case of pure filtering (and no
noise), the optimum filter reduces to H(s) = 1, which, of course,
is in accord with physical intuition.

The methods discussed here have been extended in various ways. In

the cases thus far considered, it has been assumed that an infinitely long
record of the signal has been available. This assumption simplified the
mathematics leading to Eq. (82), the Wiener-Hopf equation. If the inte¬
gration limits are replaced by finite values, the mathematical complexities
are increased considerably. Furthermore, if the signal is not of the sta¬
tionary type (i.e., if its autocorrelation function is not invariant under a
time translation), then the previous results are not applicable. These
extensions are treated in the literature. CM 2) The analyses are extremely
complex, and it is difficult to retain a physical grasp of the situation,
which is so necessary for practical application. In this respect, a

*The power spectral density gives the same information, since these are related by Eqs. (61) and
(62).
STOCHASTIC EFFECTS 481

significant breakthrough has been made by Kalman, (13,14) who has

interpreted the Wiener problem in terms of conditional expectations
and state transition concepts. Thus, he has not only simplified the
mathematical aspects, but has au,o displayed the results in a manner
permitting application to a wider class of problems and reducing the
problems of finite data and nonstationary effects to manageable pro¬
portions.
This approach will be considered in tbe following sections.

4.3 THE KALMAN THEORY

The Wiener problem is concerned with finding a linear filter that

minimizes the mean square error between actual and desired output
when information on the signal and noise is specified in terms of power
spectral densities (or correlation functions). As noted in the preceding
section, the analysis leads to an integral equation whose solution yields
the impulse response of the required filter. Serious complications are
introduced in the case of finite memory or nonstationary processes.
The approach adopted by Kalman(13-t4) yields a much simplified
mathematical structure that includes all the situations mentioned above
as special cases. Furthermore, the form of the solution permits a wider
range of application; the result is not merely an optimal filter, but an
algorithm for obtaining optimal measurements of (deterministic) dynamic
systems corrupted by additive noise. This noise may act on the dynamic
system, the measurement, or both. The fundamental premises in Kalman’s
analysis are:

a. Arbitrary random signals are represented as the output of a linear

dynamic system excited by white noise.
b. The concepts of state and state transition are used throughout.
c. If the minimum mean square error is adopted as the optimality
criterion, then the best estimate of a signal is given by the conditional
expectation

E[xi(t*)| y(t0), y(ti), . , y(t*)] .

We shall discuss these in turn. The first premise involves the concept
of a “shaping filter which has been considered in the previous section
in a somewhat disguised form. By definition, a shaping filter for an
arbitrary random process with a power spectral density, Gyy(u), has the
482 ELEMENTS OF MODERN CONTROL THEORY

property that a white noise input generates a random process having the
same power spectral density. Referring to Eq. (63), and letting G**(w) = k2
(i.e., white noise), we see that the shaping filter, Y(j«), is defined by

Gw(co)= k2|Y(jco)|2
. (91)

When the power density spectrum, G „„(«), is a real-valued, even, non¬

negative function of u for all real values of «, then Eq. (91) may be
solved for Y(j«), which is in fact physically realizable (i.e., there are no
poles or zeros in the right-half plane).
The simplest means of doing this has already been indicated in Eq.
(86), namely: replace « by s/j in GB!/(«) and form the expression*

Gw(s) = Yx(s) Yh-s) . (92)

The term Yi(s) is the shaping filter for Gw(w).

The following examples illustrate the calculation of the shaping filter
for given power spectral densities.

Example 4:

1
G „(&))
1 + w2

G„(s)

Yi(s) = — L. .
1+ s

Example 5:
25 + a)2
G»s(co)
4 + w4

25 - s2 5 + s 5 — s
Gss(s)
4 + s4 Ls2+ 2s + 2. Ls2- 2s + 2J

*We have assumed that the white noise, Gxx(ai) = 1 .

STOCHASTIC EFFECTS 483

and

5 + s
Yi(s)
s2 + 2s + 2

Example 6:

169 + o>2
Gss(co)
w4 + 238w2 + 1692

G„(s)
13 + s 1_1CO co 1- 1

Ls2+ 24s+ 169JLs2 - 24s + 169J

s + 13
Yds)
s2 + 24s + 169

Kalman’s second premise involves the use of the concepts of state and
state transition. Thus, instead of expressing the transfer function in terms
of the complex variable, s, it is used merely to relate the output-input
function in state transition terms in the time domain. In general, the
shaping filter, Yi(s), may be written as

/3n_iSl
V /• N 0(s) i=0 ^
Yi(s) = — = - ,m > n . (93)
^?(s) m
^ *m—jS
i= o

It was shown in Chapter One that the input-output relation expressed

by this transfer function may be written in terms of a vector matrix
differential equation as follows.

x = Ax + b^> (94)

Here A is a constant n X n matrix and x and b are n vectors. The compo¬

nents of the state vector x are linear functions of d and <p, while the
components of the b vector are constants. In what follows, the form of
Eq. (94) will be used in preference to Eq. (93).
The third premise adopted by Kalman is that conditional expectation
rather than variational calculus may be used to solve the problem of
optimal filtering and prediction. This observation requires closer scrutiny,
484 ELEMENTS OF MODERN CONTROL THEORY

involving the concept of a conditional probability distribution, which is

defined as follows. (5)

Prob[x(t„) < xn | x(ti) = xj; x(t2) = x2; . ; x(t*_i) = xn_i]

/
J - OO
tij x2, t2j . , xnj tn) dx„
= ^(*0 = (95)
hi—i(xi, ti ; x2, t2; , xrt—i, tn_i)

where f*( ) has the meaning defined by Eqs. (32) and (33). In other
words, ^(x*) is the probability that x(t„) takes on a value less than or
equal to x», given that x(ti), x(t2), . , x(t,_i) have taken on the
values, xi, x2; ••• xn_i, respectively. The conditional mean or expectation
is defined by

E[x(t„)| x(ti) = xi; x(t2) = x2;. ; x(t„_i) = x„_i]

(96)
/ x(t„) d'k„(xm)
.
V - CO

Suppose now that we are given a signal x(t) and a noise n(t). Only the
sum y(t) = x(t) + n(t) can be observed. Assume that we have observed
and know the values of y(ti), y(t2), . y(t„_i) exactly. What can be
inferred about the value of x(t)? If we know the conditional probability
distribution

'k(xu) = Prob[x(t„) < x* | y(tx) = yq . ; y(t*_i) = yK_d (97)

then this conveys all the information derived by the measurements y(tx),
. y(tn-i). Any statistical estimate of x(ti), denoted by X(ti), will
then be some function of ^(xn). In general, the value of X(tn) will be
different from the actual (but unknown) value of x(t„). A rational
method of determining some optimal estimate, X(tn), is to define a func¬
tion of the estimation error, e, and try to minimize this. Let a loss
function be defined by

L(e) = L( —e)

L(0) = 0 (98)

L(e2) > L(ei) when e2 > ei > 0

STOCHASTIC EFFECTS 485

where

e = x(t„) - X(t„) .

An optimal estimate may then be determined in straightforward

fashion, using the following result due to Sherman. O 6)

Theorem I: Assume that L(e) satisfies Eq. (98) and that the condi¬
tional distribution function, 'k (x„), defined by Eq. (97), is such that

(A) ^(xn — x„) = 1 — ^(xn - x„)

(B) + (1 - X)x„<2>]< X^(xnd>) + (1 - X) *(xn<2>)

for all x„(1), xre(2) < xn and 0 < X < 1, where xn = E [x*].
Then the estimate X(tR) of x(t„), which minimizes E[L(e)], is the
conditional expectation

E[x(tj|y(ti) = vi; . ; y(tn_i) = yn_i] .

In short, if the conditional distribution function is known to satisfy

certain conditions, then the optimal estimate is merely the conditional
expectation. In certain cases, even these restrictions may be removed.
According to a theorem by Doob,0) if L(e) = e2, then Theorem I holds
without restrictions (A) and (B). Furthermore, if random processes
x(t) and n(t) are Gaussian, then Theorem I holds, since quantity y(t) must
also be Gaussian.
We have shown that, under certain conditions, an optimal estimate of
a signal corrupted by additive noise may be determined via conditional
expectation. But, in general, the conditional distribution function is not
known, nor is it known what general class of such functions satisfies (A)
and (B) of Theorem I. Some progress can be made only if one can
justifiably assume that the random processes considered are Gaussian.
Now if the random processes are Gaussian, it is knownU?) that the
mean square error criterion yields a linear optimal estimate. Also, since
the calculation of this estimate involves only the means and covariances
of the Gaussian process, then this estimate is also optimal for any
random process with the same means and variances if the optimal esti¬
mate is required to be linear.

This, then, forms the basis for formulating the problem in the following
manner.
486 ELEMENTS OF MODERN CONTROL THEORY

Statement of the Problem

I. Continuous Case

Given is the dynamic system

dx
— = A(t) x -f B(t) w(t) . (99)

The observed signal is

z(t) = M(t) x -T v(t) (100)

where A(t) and B(t) are n X n matrices, x is an n vector, z(t) is an m

vector, and M(t) is an m X n matrix; w(t) and v(t) are n and m vectors,
respectively, representing independent Gaussian random processes with
zero means and covariance matrices*
(101)
E[w(t) wt(t)] = Q(t) «(t - r)
E[v(t) vr(r)] = R(t) 5(t - r) (102)

E[v(t) wr(r)] = 0
(103)
where 8 is the Dirac delta function and Q(t) and R(t) are symmetric non¬
negative definite matrices continuously differentiable in t.
It is assumed that the measurement of z(t) starts at some fixed instant,
t0 (which may be — °o), at which time P(t0) = E[x(t0) xr(t0)] is known.
The optimal estimation problem is then formulated as follows. Given
the known values of z(r) in the interval t0 < r < ti, find an estimate
x(ti|t) of x(ti) that minimizes the function, E(«2), where

€ = x(ti) - x(ti|t) .
If ti < t this is called the smoothing problem. If ti = t this is called
the filtering problem, while for ti > t we have the prediction problem.

II. Discrete Case

Given is the dynamic system

x(t + 1) = <t»(t+ 1, t) x(t) + r(t + 1, t) w(t) . (104)

The observed signal is

z(t) = M(t) x(t) + v(t) (105)

*The superscript T denotes matrix transpose.
STOCHASTIC EFFECTS 487

where <t>(t+ 1, t) and r(t + 1, t) are transition matrices and the other
symbols have the same meaning as in the continuous case, except that in
Eqs. (101)-(103) the numbers t and r are integers. In other words, w(t)
and v(t) are constant during the sampling interval (which in the present
case has been normalized to unity).
The optimal estimation problem is as stated in the continuous case
except that instead of being given z(r) in the interval t0 < r < t, we are
given the sequence of measurements, z(0), z(l), . , z(t). As before,
we are also given the initial covariance of x(0); namely, P(0) = E[x(0)
xr(0)].

Remark: In the problem statement, Eqs. (89) and (104) constitute

models of the message process; that is, the statistical properties
of the message are represented as the output of a linear system
excited by white noise. If the white noise is Gaussian, then the
system output is Gaussian. Furthermore, given any random
process with known first and second order statistical moments
(averages), one can find a unique Gaussian process with the
same moments. In line with the earlier discussion, if one seeks
an optimal linear estimate that minimizes an rms error, the
same result will be obtained if the process is assumed Gaussian;
for then, under the same optimality criteria, the result will be
an optimal estimate that is linear. It should be emphasized
that these observations are valid only when one considers
statistical moments no greater than second order.

Given now the first and second statistical moments, how does one
obtain Eqs. (99) and (104)? This is largely an unsolved problem, and
we are compelled to start with these models and consider the question
of how to obtain them as a separate problem. There is one exception.
If the statistics of the message are known in the form of a power spectral
density or an autocorrelation function (of the stationary type), then
one may obtain Eq. (99) from the shaping filter for the message in the
manner described by Eqs. (92)—(94).
Having stated the basic problem, how does one obtain the solution?
As noted earlier, Kalman showed that the optimal estimate may be
obtained by calculating conditional probabilities; the details of the pro¬
cedure may be found in Refs. (13) and (14). The analysis relies heavily
on the methods of abstract probability theory, which are not readily
accessible to control engineers. It has been shown by GreensiteG6) that
Kalman’s solution may be derived from first principles in dynamic
488 ELEMENTS OF MODERN CONTROL THEORY

programming theory.* We will adopt this approach in obtaining the

solution to the problem. t
We consider first the discrete case as given by Eqs. (104) and (105).
For ease of writing, we will use the following abbreviations.

x(t) = X,

z(t) = z,

etc. Also

4>(t + 1, t) ss <f>(

r(t + l, t) = Tf

M(t) = M,.

By virtue of Bayes law, we have (where the g quantities denote proba¬

bility density functions)

gXf+l|2f [x0, . , X(+1|Z0, . , z (]

gZf|X(+i [Zp, Z(|x0> . , Xt+l] gXf+1 [x0, Xt+l]

•(106)
gz< [z0, . , Zt]

We note that

gz,|x,+i [z0, . , Z(|x0, . , X(+i]

(107)
= gz<lx' [z0, . , Zi|X0, . , Xt]

since the probability of the sequence z0,. , zt is independent of x(+i.

Furthermore

gz<|x<[zo, . , Z,|x0, . , xt] = n gv- [zi - M,x,l (108)

i=0

via Eq. (105) and the independence of v,.

We have also<19)

gx<+i (xo, . , X(+1] = go [x0] gi [xi|x0] go [x2|xi, x0] • • •

. g(+l [X(+l|X(, . , x0] .

*The elements of dynamic programming are given in Chapter Five,

t An outline of Kalman’s approach is contained in Appendix E.
STOCHASTIC EFFECTS 489

But since the random disturbance is independent, it follows that

gi [xt-|Xi_i,. , x0] = gX,1Xt-1[x,-|x,-_i].

Consequently, Eq. (106) may be written as

gx,+i|z, [x0, . , X(+i|*o,. , Z(j

< t+1

n g1'' (z<~ X<1

go [x0]II gx,|x,_i[Xi|x,_il
_™ ^ _ (109)
gz< [zo, . , Z(]

In accordance with the previous discussions, we assume that x0 is

Gaussian with mean

E(x0) = M (110)

and covariance

E[(x0 — /u)(x0 — m)t] = Po • (111)

But since

“I T

Sx(jx(_i[x(|xt_i]= f( exp J r,_i (x, — <t>(_ix,_i) Q<-i r,_i (x,

1 -1
- 4>(_!
x,_i)lj
=f,exp
£ - (x, -
2
x,_i)r U (_i (x, - $,_i xt_i)

where

u, = r, Q( rv (112)

and

gvt [z, — M, x,] = t t exp [ M, x,)T Rf1 (z t

Eq. (109) may be expressed in the form

490 ELEMENTS OF MODERN CONTROL THEORY

gX!+l|Z([X0, • , X£_f_l
| Zq> • , z,J = 0t exp {“ 5<x”
_ m)1

Po-‘(x0 (zi — M< Xi)T R<_1(z«' — M, x,)

" i=0 -

+ (xj+i - Xi)T Ur‘(x<+i ~ Xi)]| (113)

where $t depends only on the sequence, z0, . ,zt, and known con¬
stants.
Since the random effects are white and Gaussian, the least-square
estimate and the maximum likelihood!1 9) estimate are identical/20) The
problem is then reduced to one of choosing the sequence x0, . , x, that
maximizes the quantity in the braces of Eq. (113) or minimizing the
quantity

•1(+i — (xo — f*)T Po Kxo — m) + E |"(z*

i=o
- Mf x{)T R,-_1(z» ~
L
Mi xO

+ (xi+1 - 4>iXi)^ UrTxi+i - 4>,-Xi) (114)

This is precisely a multistage decision process to which the methods of

dynamic programming are directly applicable.
Let us define

f(+i(x(+1) = Min <(x0 - m)t Po_1(x0 - n)

x0, . , x, 1

+ E (115)
(zi — MiX,)r Ri_1(zi ~ MiXi)

+ (xi+1 - <t>iXi)TUrhxi+i - «]} •

Via the Principle of Optimality,!21) we have

f(+i(x<+i) = Min [(z, - MfXdMtrhzi — Mtx,)

x(
|T (x, + i <Gxt) rU ( hx* + i — ‘t’fXi) -f- f,(x,)l

which, after some rearrangement, may be written as

STOCHASTIC EFFECTS 491

ft+i(x,+i) = Min [x(r(M,rRr1M< + $irUr^i)X|

x(
- 2(z,*-R(-im, + \tT+i Ur14>e)xt + z(Rr‘z,
+ x tT+i Ur‘x(+i + f«(x *)] •

We now assume a solution of the form*

/

ft(xi) = x,tA(X, - 2b<rx< -f c, (117)

where A < is a symmetric n X n matrix, b, is an n vector, and ct is a scalar,

all of which are for the moment unknown. Substituting this back in Eq.
(116), we obtain, after some reduction

f,+i(x ,+i) = Min [xtrL,x* — 2a,Txf + z/rR71zt

x(
4- X,+Ir Ur1 X(+1 + C(J (118)

where

L( = MjRr'Mt + $(rUr1$, + A, (H9)

«r, = b, + QjVr'xt+r + M/R^z, . (120)

The value of x, that minimizes the quantity in the brackets in Eq.

(1 18) is readily found to be

x( = L t V< . (121)

Substituting this back in Eq. (118) and simplifying, we obtain

(122)

where

A«+i = Ur1 (I - ^Lr'R'Ur1) (123)

b,+i = Ur1 LrHbt + M(T R,_1Z<) (124)

&fThis particular form of b(x<) is suggested by the fact that the criterion function Eq. (104) is a
quadratic form, and the state equations are linear.
492 ELEMENTS OF MODERN CONTROL THEORY

cf+i = z/Rr'd - Mi l r1 M|T Rr^zi

- 2b L(-1 Mi'Rr1 z< - biTLrx b( - c< . (125)

The function ft + i(x( + x) may be interpreted as follows. Let “a” be

any value of xt + x. Then —f, + i(a) is a measure of the likelihood of the
most probable sequence of states, x0, . , x, + h in which xt + i takes on
the particular value, “a”, given the observed sequence, z0, . , zt and
the a priori distribution on x0. An optimal choice of “a” is that which
maximizes the likelihood function and which, for white Gaussian statistics,
is also the one which minimizes the mean-square error. From Eq. (122),
the optimal value of xt + i is found to be

xf+i = Ai+r’bi+i . (126)

Substituting Eqs. (119), (123), and (124), we find, after some lengthy
reduction

xT+i — $t( I 4" Hf 1 <t>irUi 1 <t>i)(H| -(- <t>irUi 1 $i) 1

(127)
(bi + M,t Rr1 z()

where

Hi = A, + MiT Rr1 Mi . (128)

A crucial simplification of Eq. (127) may be obtained by making use

of the following relation.!22)

(C! - c2 ca~l C4)-1 = Cr1 + c r 1 c2 (c, - c4 Cr1 c2)-[ c4 Crl ( 129)

Ci = m X m matrix (nonsingular)

C2 = m X n matrix

Ci = n X n matrix (nonsingular)

C4 = n X m matrix

Using the above relation, Eq. (127) reduces to

xT+i = 4>iAi xbi - 4>,A,-'MiT(Mi Af1 Mfr + R,)-» MiAr'bi

+ Af1 M tT (Mi AC Mir + R|)-i z i . (130)
STOCHASTIC EFFECTS 493

We now seek to obtain the physical significance of the matrix A<. For
t = —1, we find, from Eq. (115)

fo(x0) = (x0 - n)T Po_1(xo - a)

= x0r P<rl x0 - 2ht P0_1 x0 + m7-Po-V

while from Eq. (1 22) '

fo(x0) — Xo Ao xo — 2b0r x0 + C( .

This permits us to equate terms as follows.

Ao = Po"1

b0 = Po~V

c0 = Po~V

Consequently, we may interpret Ar1 as the covariance matrix, P,.

Noting further that Eq. (126) enables us to write x? for P,b<, we
find that Eq. (130) reduces to

x(t —
|—1 t) = 4>(t + 1, t) x(t | t — 1)

+ K(t)[z(t) - M(t) x(t | t — 1)] (131)

K(t) = *(t + 1, t) P(t | t — l)MT(t)[M(t) P(t | t — l)Mr(t)
+ R(t)]-1 . (132)

Here we have reverted to the original notation of the problem, and

have written x(t + 1 | t) for x*4J to conform with the common notation
in the literature.
It remains to determine the recurrence relation for the covariance
matrix, P(t + 1 | t). This may be obtained from Eq. (123) by taking the
inverse, applying Eq. (129), and then substituting Eqs. (112) and (119).
The end result is

P(t + 1 | t) = <t>(t
+ 1, t){P(t | t — 1) — P(t | t — l)Mr(t)
[M(t) P(t | t — l)Mr(t) + R(t)]-1 M(t) P(t [ t — 1)} (133)

<t>r(t+ i, t) + r(t + l, t)Q(t)rr(t + l, t) .

494 ELEMENTS OF MODERN CONTROL THEORY

Here, obviously, P(t0|t0 - 1) = P(t0) = P0, which is assumed given.

This represents a type of initial condition. Equations (1 3 1)-( 133) repre¬
sent the solution to the problem in the discrete case. It can be shown!23)
that

P(t|t - 1) = E[x(t|t - 1) xr(t|t - 1)] (134)

where

x(t|t — 1) = x(t) — x(t|t — 1) . (135)

Figure 4.6 depicts both the dynamic system of Eqs. (104) and (105),
i.e., the message, and the optimal filter described by Eqs. (131 )—( 133).
If the unit interval for the discrete case solution is allowed to approach
zero, we obtain the solution for the continuous case, Eqs. (99) and (100),
viz.

dx(t|t) = A(t) x(t|t) + K(t)[z(t) - M(t) x(t|t)], optimalestimate ^3^

dt
K(t) = P(t)Mr(t) R_1(t), optimal gain (137)

dP(t) = A(t) P(t) + P(t)AT(t) - P(t)MT(t) R-‘(t)M(t)P(t) (138)

dt
4- B(t)Q(t)B7'(t), variance equation.

The limiting process is a matter of some delicacy because of the

presence of Dirac delta functions. However, the procedure can be made
mathematically legitimate by a sufficiently sophisticated analysis. A
typically “engineering type” of proof (which, therefore, involves some
compromise with. rigor) is contained in Ref. 23.
Equation (136) is the optimal estimate for the filtering case. For
optimal filtering and prediction, we add the relation

x(ti|t) = 4>(ti,t) x(t|t)

Figure 4.7 is a block diagram for the message and optimal filter for the
continuous case.
STOCHASTIC EFFECTS 495

>

4.6.
of
Case.
Discrete
Filter
Optimal
Figure
Model
and
Message
the
496 ELEMENTS OF MODERN CONTROL THEORY

4.7.
Figure
of
Optim
Mode
and
Messa
the
Filter
Contin
Case.
STOCHASTIC EFFECTS 497

The optimal filter is in general nonstationary, because A(t), 4>(t + 1, t),

and M(t) are time-dependent. If, in fact, these quantities are constant,
then as t —> <», the system, Eqs. ( 136)—( 138), reduces to the classical
Wiener solution.
It is worthy of note that the derivations given here are not affected
materially by the fact that the system is nonstationary or that the
available data is finite. The analysis leads directly to the form of the
optimal filter without the extraordinary mathematical complications
that characterize the classical approach.

4.3.1 Interpretation of the Kalman Filter

Before considering some direct applications of the Kalman theory, it

is instructive to examine the physical significance of the operations
described by the estimation equations. In order to simplify the situation
without ehminating the essential features, consider the problem of esti¬
mating the components of a constant n vector, x. The only available
measurement is of a linear combination of the components of x which is
contaminated by white noise. Mathematically, this corresponds to the
system, Eqs. (104) and (105), with <t>(t+ 1, t) = I, w(t) = 0, and
M(t) = M = constant. In the present case, M is a known m X n matrix
and we are given the initial covariance, P0. For simplicity, it is assumed
that R(t) = I
The optimal estimation equations, (131 )—( 133), may, therefore, be
written as

X<+1 = x« + K,(z, - Mx() (14°)

K, = P(Mr(M P(Mr+ I)-1 (141)

p(+1 = p( - P,Mf(M P(Mr + I)-‘M P( (142)

where we are again using the simplified notation.

Making use of the relation in Eq. (129), Eq. (142) simplifies to

Pt+f 1 = P rl + MrM . (143)

But Eq. (142) can also be written as

P1+1(P*+f 1 - P r1) = P,Mt(M P(Mr + I)-lM

498 ELEMENTS OF MODERN CONTROL THEORY

which, by virtue of Eq. ( 143), becomes

P,+1Mr = P«Mr(MP«MT + I)-1 . (144)

Repeated application of this equation yields

P(Mt = P t_iM r(M P i_iM T + I)-1

= P t_2Mr(2M P mM t + I)"1

= P0Mr(tMP<>Mr + I)"1 •

For t sufficiently large, this becomes

P(Mr = 7 P0Mr(M P0Mr)-i . (145)

u

Substituting Eqs. (141) and (145) in Eq. (140), we find

X(+l — x< + P0Mr(M P0M7')-'(zjMxi) . (146)

This equation is in the form of a multidimensional stochastic approxi¬

mation/24.25) The correspondence between the two concepts is intrigu¬
ing and suggestive of the underlying unity between statistical estimation
procedures.
Let us now assume that M is an n X n matrix, which is nonsingular.
This corresponds to the usual observability condition/27) Equation (146)
then simplifies to

Xf+i = — -— x, + — -— (x + M-1 v() .

t + 1 t + 1

By repeated application of this equation we have

xe+i= ^ | x,_i+ —1 [2x+ M-1(v<

+ vt_i)]
t + 1 t + 1

2 X*—
2+ 1 [3x+ M_1(v,+ vt—i
+ vt—
2)]
t+ 1 t + 1

t - j - , 1
- x,_,- H- (j + l)x + M-i E v,-
t + 1 t + 1 i= t— j -1
STOCHASTIC EFFECTS 499

For j = t, this becomes

X(+l
1

t + 1
[ (t + l)x + E M-‘
1=0
(147)
t+ i
1
E M-1
t + 1

which is precisely the weak law of large numbers.!2) Another way of

writing Eq. (147) is

1 *^
X«+1= X H- — E M-1 V,-.
t + lH

In other words, for sufficiently large t, there is virtual certainty that

x(+i is the true value of x. Note that the variance matrix, Eq. (145),
becomes (for Mann x n matrix)

P, = i(MrM)_1
b
.

For sufficiently large t, this approaches zero as expected.

4.3.2 Calculation of the Kalman Filter

In this section, we will illustrate the determination of the optimal

filter, via the Kalman equations, in two simple cases. More realistic appli¬
cations of practical importance will be considered in Sect. 4.3.4.

Example 7: The given data is the same as that of Example 1 in Sect.

4.2. In order to use the Kalman theory, it is necessary to express the
power spectral density of the message as the output of a linear dynamic
system excited by- white noise. Thus, the shaping filter for

is readily found to be
500 ELEMENTS OF MODERN CONTROL THEORY

Expressed in the time domain, in the form analogous to Eq. (99), this
becomes

x = —x + w

where w is white noise having a power spectral density of one, which

means that

E[w(t)wr(r)] = 5(t — t)

The observed signal is

z = x + v

where the power spectral density of v has been given as G^w) = k2;
consequently*

E[v(t)vr(r) = k2 5(t— r) .

Figure 4.8 is the schematic of this system. The solution for the optimal
filter is given by Eqs. ( 136)-( 138), where, in the present case

A(t) = 1 Q(t) = 1

B(t) = 1 R(t) = k2 .

M(t) = 1

Figure 4.8. Schematic of Message Model for Example 7.

*It is assumed that E[w(t)] = E[v(t)| = 0.

STOCHASTIC EFFECTS 501

Each of these quantities is scalar. We have, therefore

dx(t|t)
— dt — x(t|t) + K[z(t) —x(t|t)l
P
K =
R

P '
0 =
-2P-i + Q.

The quantity dP/dt has been set equal to zero in Eq. (138), since we
are seeking the steady-state (time invariant) form of the optimal filter.
From the above equations, we find that this filter has the form shown
in Fig. 4.9. The corresponding transfer function is

x(t|t) _ K .
z s+l + K

Solving for P in the quadratic equation given previously, and substitut¬

ing in K, we find that this solution corresponds with that given in Example
1 of Sect. 4.2. The inclusion of the prediction interval, a, requires that
we use Eq. (139), where, in the present case

<t»(ti,t) = e (!i " = e~°

where ti —t = a is the prediction interval.

Figure 4.9. Schematic of Optimal Filter for Example 7.

Example 8: It is sometimes required to design an optimal realizable

filter for two related messages, the measurement of each of which is
corrupted by additive noise. For example, suppose that the power
spectral density of a velocity signal is hj/w* while for the position signal
it is hu’/&j4. Measurements of velocity and position are corrupted by
additive white noise. Some problems of this type have been treated by
502 ELEMENTS OF MODERN CONTROL THEORY

Bendat(28> using the classical Wiener theory. The analysis is characterized

by some intricate mathematics in which it is difficult to obtain physical
insight. Using the Kalman approach, however, a simple and physically
plausible solution is obtained in a straightforward manner. From the
power spectral densities given above, it is easy to obtain the shaping
filters which, in turn, lead to the message model shown in Fig. 4.10.
Here x2 and xx represent the velocity and position, respectively, while z2
and zi are the respective measurements. The equations of the message
model are given by

Xi = x2

X2 = Wi

and for the measurements

zi = hn xi -f vi

z2 = h22 x2 T v2 .

These equations may be written in the matrix form corresponding to

Eqs. (99) and (100), noting the following equivalence.

0 1 0
A = B =
O O _1_

Figure 4.10. Model of Messages for Example 8.

STOCHASTIC EFFECTS 503

Ji o
M =
_0 h.22_
qn 0 rn 0
Q = R =
o o _0 T22

The optimal filter is given by Eq. (136), viz.

dx(t]t}-= [A - k(t)M] x(t|t) + k(t)z(t) .

Figure 4.11 is the schematic for this filter. Note that this is time
varying, since, in the general case, only finite data is available. A time
invariant form results when t —> » , since, for this case, P becomes a
constant (corresponding to the classical assumption that an infinitely
long past record of the signal is available).

Figure 4.1 1. Model of Optimal Filter for Example 8.

504 ELEMENTS OF MODERN CONTROL THEORY

To solve the problem in the case of finite data one must be given the
quantity* P(0), which is written in the form

<711
(0) <712(0)
P(0) =
(72l(0) (722(0)_

where <7i2(t) = <721

(t), since P(t) is a symmetric matrix.
From Eq. (138), we find

hn2 an2 h222

22 (7 12
(711 — 2(712 —
Til T22

h,i2 (711 (712 ll222

22 <712 <722
(712 — (722 —
Til T22

hll2 (7122 h222 <722

2
<722 — — + qn
Til T22

The optimal gains are given by Eq. (137), viz.

hn <7ii
kn =
Til

111 (712
k2i —
Til

I22 <712
ki2 —
T22

122 <722
k22 —
r22

In order to obtain some numerical results, let us assume that

hn =1 Qn = 1 <7n(0) = 1

h22 = 2 Tn = 16 <712(0) = 0

r22 — 1 <722(0) = 0 .

*See Eq. (Ill) and the discussion following Eq. (133).

STOCHASTIC EFFECTS 505

Figure 4.12. Solution of the Variance Equations of Example 8.

0.12

0.10

0.08

cq

0.06

0.04

0.02

Figure 4.13. Optimal Gains in Example 8.

The solution for the variance and optimal gains is shown in Figs. 4.12
and 4.13, respectively. Kalman!14) shows that undei mild restrictions, a
steady state is reached that is equivalent to the time invariant optimal
filter. This steady state is apparent in the present case. Note that the
initial values of the time varying parameters of the filter are strongly
dependent on the assumed values for <rt>. However, the final steady state
is independent of the assumed values, which is plausible, since with
increased information available, the initial assumptions play a vanishingly
decreasing role in determining the form of the filter. Note that the
506 ELEMENTS OF MODERN CONTROL THEORY

steady-state values for ktJ and could have been obtained directly from
the a equations by assuming = 0 and solving the resulting set of alge¬
braic equations. For high order systems, however, this is a difficult
process, and it is much simpler to integrate the variance equations until a
steady state is reached. The speed with which one converges to this
steady state is, of course, dependent on how good the initial estimate is.

4.3.3 The Problem of “Colored” Noise

The Kalman filter for the continuous case, as described by Eqs. ( 136)-
(138), is based on the fact that the measurement noise, v(t) in Eq. (100),
is white and nonzero. If indeed v(t) were identically zero (i.e., the
measurement were perfect), then the variance equation, (138), would be
singular, since R(t) would be a null matrix. Furthermore, the analysis
leading to Eqs. ( 136)—( 138) is based on the assumption that v(t) is white,
i.e., successive values for small intervals are essentially uncorrelated. The
case where v(t)has a power spectral density that is not a constant cannot
be treated within the framework of the theory thus far presented. This
gap has been filled in a recent paper by Bryson and Johansen, (29) which
treats the general case of measurements corrupted by colored noise, white
noise, no noise, or any combination thereof. The details of the general
procedure are quite lengthy, and we will accordingly describe only one
special, though important, case. The reader is referred to the afore¬
mentioned reference for a complete treatment.
It is assumed that the process (message) is described by

Xi — Fn xi + ui (148)

where xj and ui are r vectors, and Fn is an r X r matrix, which, in general,

is time-varying. Also

E[Ul(t)] = Efxi(t)] = 0 (149)

E[ui(t) Uir(r)] = Qj(t) 5(t - r) (150)

E[Xl(0) xp(0)] = Pj(0) . (151)

The measurements are given by

y = Ci xi + m (152)
 STOCHASTIC EFFECTS 507

where y is a p vector, Ci a p X r matrix, and map vector representing

colored noise.
As might be expected, the procedure involves the representation of
the statistical properties of m as the output of a linear system excited by
white noise. Specifically, in the present case, we assume that

m = Am + Bu2 (153)
/
where A and B are p X p matrices, u2 is a p vector, and

E[u2(t)] = E[ui(t) u2T(r)] = 0 (154)

E[u2(t) U2r(r)] = Q2(t) «(t - r) (155)

E[m(0)] = E[m(t) xir(t)] = 0 (156)

E[m(0)mr(0)] = N(0) . (157)

The optimal filter is then given by*

Xi(t|t) = xf(t|t) + Ky (158)

xf(t t) = Fh xi(t|t) - K[Ay + Hxi(t|t)] - Ky (159)

Iv = (PiHT + Qi CiT)R_1 (160)

Pi = Fu Pi + P]F] / + Qi - (PiHT + Qi Cir)R-> (Ci Qi + H Pi) (161)

and

H == Ci + Ci Fn — AC i (162)

R = Ci Qi CiT -(- BQ2 B (163)

The initial conditions are

N)-* 1 Pi(0+) = {Pi - Pi CiT(CiPi CxT+ CxP,}(„0 (164>

xi(0+) = {Pi Cir(Ci Pi Cir + N)->)l=0y(0). (165)

*See Ref. 29 for the derivation. The result given here is a slightly generalized version of Example
1 in that paper.
508 ELEMENTS OF MODERN CONTROL THEORY

The discontinuities at the initial time are due essentially to the fact
that an initial estimate and an exact measurement are simultaneously
available at the start.
We illustrate the application of these ideas in the following.

Example 9: Consider the problem stated in Example 3 and solved

there by the Wiener method. The same problem will be solved here by
the techniques presented in this section.

The shaping filters for the message and noise are found to be

1
Gi«(s) =
s + 2

5
G1(*)(s) =
s + 5 .

Consequently, the message can be represented as

Xi — — 2xi Ui

with

E[ui(t) Uir(r)] = 5(t - r)

The measurement is

y = xi + m

where

m = —5m -f- 5u2

E[u2(t) U2r(r)] = 5(t — t) .

Comparison of these equations with the set, Eqs. ( 148-1 55), indicates
the following equivalence.

Eii = —2 A = —5

Qi = 1 R = 5

Ci = 1 Q2 — 1
STOCHASTIC EFFECTS 509

Since we are seeking the steady-state solution (time invariant filter),

K and Pi are constants, with the latter obtained from Eq. (161) with
Pi=0.
Substituting known values, we find from Eqs. (160-163)

R = 26.0

H = 3.0

Pi = 0.2232

K = 0.0642 .

The equations of the optimal filter are then given by Eqs. (158) and
(159), viz.

Xi(t|t) — x*(t[t) + Kv
x?(t|t) = —2xi(t|t) — K[—5y + 3xi(t|t)] .

Eliminating xf between these equations results in

xx(t|t) _ 0.0642(s + 5)
y ~ (s + 2.1926)

which is identical to the result obtained in Example 3 by the Wiener

method.

4.3.4 Aerospace Applications

Direct applications of the Wiener theory in aerospace problems have

been rather limited. For any but the simplest type of systems, the
mathematical structure becomes too unwieldy. On the other hand,
Kalman’s approach has the virtue of exhibiting immediately the form of
the optimal filter and makes use of either finite or infinite data with
equal facility. This makes it very attractive for use in systems where
optimal estimates based on noisy measurements are to be improved as
additional information becomes available. The computational algorithms
are easily adapted for computer processing in real time for operational
systems. It is required only that the system be linear and that initial
variance estimates be available. The latter restriction is not too serious,
510 ELEMENTS OF MODERN CONTROL THEORY

since the effect of poor initial estimates diminishes as more and more
measurements become available. The linearity stipulation is more funda¬
mental; usually this is satisfied by treating the estimation error (which
may be taken as the state variable) as a linear expansion about some refer¬
ence condition. The prospects are, therefore, very favorable for applying
the theory in order to obtain significant improvements in system per¬
formance for a wide variety of realistic situations in aerospace guidance
and control. A sampling of the literature in this area is contained in Refs.
30 through 36.
In what follows, we will discuss three problems which are fairly
typical of present applications of the theory.

4.3.4. 1 Optimal Estimation of Position and Velocity^30)

The problem to be considered involves the in-flight determination of

the position and velocity of a space vehicle for purposes of midcourse
guidance. It is presumed that a reference trajectory is known, but because
of various random effects, the vehicle will never be precisely on this
reference trajectory. Consequently, it is proposed to make a series of
measurements that will, in fact, give the actual position and velocity of
the vehicle and that will thereby permit the necessary guidance corrections
to be applied. However, the measurements obtained are contaminated
by noise, so that again, it is not possible to know the position and velo¬
city precisely. The problem reduces to one of making optimum use of the
measurements for purposes of guidance correction.
In order to apply the Kalman theory, certain conditions must be
satisfied. First of all, the system must be linear. The equations describing
the vehicle motion are, however, highly nonlinear.
Consider the situation depicted in Fig. 4.14. The motion of the
vehicle is derived on the basis of including the gravitational effects of the
earth, moon, and sun. For a vehicle on a lunar mission, this accounts for
the predominant effects influencing the motion. The sun and moon are
assumed spherical and homogeneous, while the gravitational field of the
earth is modified to take account of surface oblateness.
An inertial geocentric coordinate frame* is adopted where the Z axis
lies along the earth’s polar axis and is positive to the north; X and Y are
in the equatorial plane with the positive X axis pointed in the direction
of the first point of Aries. The Y axis is oriented to complete a right-
handed orthogonal system.

*See Ref. 57.

STOCHASTIC EFFECTS 511

(Xs, Yg, Zs)

Figure 4.14. Geocentric Coordinate System.

The vehicle equations of motion then take the following form. (37)

Ae X 5Z2 Mm(X Xm)

X = - 1 +
T3
(166)

m M«(X - X.)
= fi(X, Y, Z)
rm3 A.3

m»(Y - Ym)
Y = -
r3
(167)

mYm m.(Y - Y.)

= f»(X, Y, Z)
rm3 A,3

Z = -
r3 1+J(!) ( ^Um(Z Z m)

(168)

M»(Z — Z„)
- f.(X, Y, Z)
r1 m ^ A.3
512 ELEMENTS OF MODERN CONTROL THEORY

where

r = (X2 + Y2 + Z2)1'2

rm = (Xra2 + Ym2 + ZJ)W

rs = (Xs2+ Ys2+ Zs)V2

Am = [(X - Xm)2 + (Y - Ym)2 + (Z - Z,)2]1'2

As = [(X - Xs)2 + (Y - Ys)2 + (Z - Z,)2]1'2

fie = 1.407683 X 1016ft3/sec2

fim = 1.729774 X 1014ft3/sec2

Us = 4.68023 X 1021 ft3/sec2

a = 20.9258 X 106 ft = radius of earth at equator

J = 1.6246 X 10-3 .

In order to linearize the set of Eqs. ( 166)—( 168), we expand each in a

Taylor series about the reference trajectory, viz.

X = fi(XB, Y«, ZB) + (X - XB) + (Y - Y,)

+ ^ (Z—ZB)+ higher
O/j
order
terms
with similar equations for Y and Z.
If we now define a vector, x, whose components are

Xi = X - xfi

X2 = Y - yr

X3 = Z - Z*

x4 = X - x« = Xi (169)

X8 = Y - Yb = X2

X6 = Z - Zr = Xs
STOCHASTIC EFFECTS 513

then, in view of the preceding results, we have

x = A(t)x (170)

0 I
A(t) =
_ A,(t); o

where

afi dU dfi
dX dY dZ
Ai(t) df2 df2 dft
ax dY dZ
df3 df3 dt.
_dX dY dZ.
1=3X3 unit matrix

0 = 3x3 null matrix.

All the partial derivatives are evaluated along the reference trajectory.
Since measurements are to be made at discrete intervals, it is con¬
venient to express Eq. (170) in the form

x(t + At) = <t>(t+ At, t) x(t) (171)

where <*>(t+ At, t) is the transition matrix!1 8) for the system in Eq.(170).
It is assumed that

E[x(t0)] = 0

and that the covariance matrix

P0 = E[x(to) Xr(t0)]

is given, where t0 is the time at the start of the estimation scheme. In

other words, at the start of the process, it is “expected” that the
deviation vector, x, is zero (i.e., the vehicle is on the reference trajectory),
and our confidence in this assertion is expressed quantitatively by the
initial value of the covariance matrix, P0. As noted earlier, the estimation
scheme is not too sensitive to P0 in the long run. A large P0 merely
514 ELEMENTS OF MODERN CONTROL THEORY

means that more measurements must be taken to ensure thatP(t) ulti¬

mately is reduced to below a preselected level. Some care must be taken,
however, that P0 not be too large, since, in some instances, the numerical
processing may produce a negative definite P(t) which will invalidate the
computation. The question of uncertainty in the a priori estimates of
covariance has been clarified in recent papers by Soong(38> and
Nishimura.t39)
It now remains to consider the instrumentation to be employed and
the means of incorporating this in the estimation scheme. For present
purposes, it will be assumed that one can measure the angles, a, /3, and y,
as shown in Fig. 4.15. From the geometry depicted here, one can readily
derive the equations which relate these angles to the vehicle position; viz.

z
a = sin-1 — (172)
r

Y
P — sin 1 (173)
(X2 + Y2)

7 = sin-1 —• (174)
r

The instrumentation output is of the form

OC-m OCact4“ Vj

* @act -F V2

ym y act + V3

where the subscript “act” denotes actual value, and the v, represents
noise. It will be assumed that the noise has zero mean and is uncorre¬
lated between successive measuring instants, i.e., white.
We now define

A(Xm OCm Otriom

Aftm @m ^nom

Ay m ' y m y nom

where the subscript “nom” means nominal value— a known quantity if

the reference trajectory is known.
STOCHASTIC EFFECTS 515

Earth

Figure 4.15. Measurement Scheme.

If we now let

Ad? Oiact Ot-nom

and similarly for A/3 and A7, we may write

A am = Aa + Vi (175)

A/3m = A/3 T V2 (176)

A7„, = A7 + v3 . (177)

The quantities A«, A/3, and A7 represent deviations from a nominal or

reference value, and if these are assumed small, then a Taylor series
516 ELEMENTS OF MODERN CONTROL THEORY

expansion about this nominal results in

da da da
Aa dX dY dZ Xl

dp dp dp
AP = dX dY dZ x2

dy dy dy
_Ay_ _dX dY dZ. ,X3.

By combining this with the previous results, we may write

z(t) = M(t)x + v(t) (178)

where v(t) is the noise vector whose components are vi, v2, v3; x is the
state vector defined by Eq. (169); and

Zi(t) A am

z(t) = z2(t) = APm

_Zs(t)_ _Aym_

M(t) - Mi(t) • 0

where

da da da
dX dY dZ

dp dp dp
dX dY dZ

dy dy dy
dX dY dZ

and 0 is a 3 X 3 null matrix.

The estimation scheme is now in the form of Eqs. (104) and (105),
where, in the present case, w(t) = 0. Consequently, the optimal esti¬
mates are given by Eqs. ( 13 1)-( 133) as follows.
STOCHASTIC EFFECTS 517

x(t + At|t) = <t>(t+ At, t) x(t|t - At)

(179)
+ K(t) [z(t) — M(t) x(t|t — At)]

K(t) = <t»(t+ At, t) P(t|t - At)MT(t)

(180)
[M(t)-P(t|t - At)Mr(t) + R(t)]-1

P(t + At11) = <t>(t+ At, t) |P(t|t - At) - P(t|t - At)Mr(t)

[M(t) P(t|t - At)Mr(t) + R(t)]-1 (181)
M(t) P(t|t - At)} <t>T(t
+ At, t)

Recall that x(t) is a vector representing deviations from the reference

trajectory. Therefore, x(t + At|t) is an optimal estimate of deviations
from nominal. This, in turn, permits one to estimate the actual trajectory.
Regarding the actual implementation of this procedure in an on-board
digital computer, the following points may be noted. First of all, as far
as calculating the actual trajectory is concerned, the set of Eqs. ( 166)—
(168) could be used, given initial values of position and velocity rather
than the linearized version, Eq. (171).
In other words, the estimated values of position and velocity can be
used to calculate future position and velocity based on the precise
equations of motion, rather than linearized versions, which are inherently
less accurate.
However, the computations involving P and K still require the linearized
approach because of the manner in which Eqs. (132) and (133) were
derived. The matrix, R, which represents the covariance matrix of the
noise vector, v(t), is constant and can be initially stored in the computer.
Theoretically, one can also store <t>(t+ At, t) and M(t), since these are
known for all values of t, given the reference trajectory. However, this
may lead to computer storage capacity problems, and also to less flexi¬
bility in choosing measurement times and intervals. There is the additional
disadvantage that the linearizations are about a reference trajectory that
is now less accurate than the new estimated trajectory. Consequently, it
appears preferable to linearize about the estimated rather than a reference
trajectory, from the point of view of minimizing error buildup. This is
clearly the correct procedure, since P has to do with the difference
between the estimate and the true state; the estimate is, on the average,
closer to the true state than is the reference. Since <t>(t+ At, t) and M(t)
are to be recomputed between successive measurements, it is, of course,
518 ELEMENTS OF MODERN CONTROL THEORY

necessary that the computation time on the computer be less than the
time interval, At, between successive observations.
Some results of a digital computer study of this problem are given by
Smith and others.!30)

4. 3. 4. 2 Optimum Stellar Inertial Navigation System!34)

A general discussion of the dynamic properties of an inertial navigation

system is presented in Chapter Four of Ref. 40. As discussed here, the
system dynamics depend upon the mode and duration of operation. The
problem to be analyzed in this section is concerned with an inertial
system operating in conjunction with a star tracker (stellar-inertial mode)
and represents a specialized case of the more general dynamics.
The basic problem derives from the fact that, over a period of time,
the drift rates of the gyros in the inertial platform introduce significant
error in the navigation system. There are various ways of alleviating this
problem.* The approach to be considered here makes use of the Kalman
theory with the result that significant improvement over previous methods
can be demonstrated.!34)
To begin the discussion, we define three coordinate systems in the
nomenclature of Ref. 40.

I. <p is the vector angle relating the platform coordinate system to a

true coordinate system (attitude error).
II. 56 is the vector angle relating the computer coordinate system to a
true coordinate system (position error).
III. i is the vector angle relating the platform coordinate system to the
computer coordinate system.

The vector angles are related by

<P= t + 56 . (182)

We shall be especially concerned with the vector angle, i, that can be

shown!40) to satisfy the equation

= e (183)
STOCHASTIC EFFECTS 519

in the stellar-inertial mode of operation. Here e represents the gyro drift

rate vector whose effects on the system must somehow be compensated
for. The information available on e is generally of a statistical nature, and
the problem is one of making an optimal estimate of i based on some
reasonable stochastic description of e.
In what follows, we shall assume that e is composed of two parts

e = 6c -Y er (184)

the first of which is an unknown constant vector (gyro bias) and the
second a zero mean random vector whose power spectral density is
known. Specifically, we assume that each component er» of er, corre¬
sponding to each of the three platform gyros, has a power spectral density
given by

(185)

The corresponding autocorrelation function is

r.(T) = (186)

It is assumed that the gyros are uncorrelated, i.e.

E[e„-(t) 6„-(t)] = 0 i ^ j •

Now according to the definition of a shaping filter,* if white noise

having a power spectral density given by

G„ <«>(«) = 2 (187)

is passed through the linear (shaping) filter having the transfer function

s + p

then the output power spectral density is equal to that of Eq. (185).
Accordingly, we may represent the statistical properties of er, by the

See Eq. (91).

520 ELEMENTS OF MODERN CONTROL THEORY

dynamic system

<r< = —0ieTi 4- wf (188)

where

E[w,-(t) w»(r)] = 2ai 2/3, 5(t — r) . (189)

This may be written in matrix form as

er = Her + W . (190)

The equation for the unknown bias constant is

6C= 0. (191)

Combining Eqs. ( 183), ( 1 84), ( 190), and ( 19 1), we have

X = Ax + Rw (192)

where

r i

X = «r

- .

o : I : I

A = o • H jO

o i o : o

B I

Note that Eq. (192) is expressed in terms of partitioned matrices-not

scalars.
STOCHASTIC EFFECTS 521

We seek to estimate the state vector, x, making optimal use of appro¬

priate measurements. The present situation is distinguished from that of
Sect. 4.3.4. 1 in that both correlated noise (the er vector) and an unknown
constant (the bias vector, ec) have been incorporated in an augmented
state vector. In other words, these additional complications may be
treated in straightforward fashion at the cost of dealing with a higher
order system. Since there is no associated difficulty of a theoretical
nature, the limitations are primarily computer time and storage require¬
ments.
We now turn to a consideration of the measurement procedure. A star
tracker that is physically mounted on the stable platform can be driven
in both azimuth and elevation. Tracking is accomplished by selecting a
star from the catalog stored in the system computer. The computer then
automatically computes the telescope pointing angles, i.e., the azimuth
angle, a, and the elevation angle, 7, through which the telescope must
turn to point at the selected star. Neglecting errors in the telescope drives,
it would be possible to point the telescope directly at the star if the com¬
puter coordinate system and the platform coordinate system were coinci¬
dent. The platform system, however, is rotated from the computer
system by the vector angle, W That is to say, the vector angle, i , is the
telescope-pointing error, or the error in pointing a platform-mounted
telescope at a star.
We seek, therefore, to determine the relationship between the errors
in the pointing angles, a and 7, and the vector angle \p.
The position vector of the telescope relative to the platform coordinate
system is given by (see Fig. 4. 16)

cos 7 COS a
Sr cos 7 sin a
sin 7

This vector has components in the computer coordinate system as

follows.

S, = ASP

1
A =
522 ELEMENTS OF MODERN CONTROL THEORY

Figure 4.16. Orientation of Telescope in Computer Coordinate System.

or*

C7Ca + i/'zC7Sa — 1/787

So — ipzcyca + C7Sa + 1/787
ipyCyCa — ipxCysa + sy

This may be written as

So = Sp + ASp

where

i/'2SaC7 — \J/yS
7
ASp — \pzCaCy + \pxsy (193)
\pv(iacy — xpxSacy

The increment in SP, due to increments A« and A7, is

0(7 T" A7) c(a + Aa) 07 Ca

ASp — 0(7 + A7) s(a + Aa) —

C7 Sa

s(y + A7) _ ST _

*Wc have used the abbreviations, Sa, ca, for sin a, cos a, etc.
STOCHASTIC EFFECTS 523

which reduces to

— Aa Gy Sa — Ay S7 Ca
Aa Ca cy — A7 Sa S7 (194)
Ay cy

By equating the expressions (193) and (194) we obtain the relations

between the \p vector and the errors in pointing angles, viz.
/

Aa = \f/x cosa tan7 -f \{/vsina tan7 — 1pz

Ay = — ipx sina + \]/y cosa .

We now define the measurement vector as

Zl Aa

M8!^ -f V (195)
_z 2_ A7_

where

taii7 cosa tan7 sina —1

M, =
—sina cosa 0_

and v is a two-dimensional white-noise vector. In order to be compatible

with Eq. (192), we write Eq. (195) as

z 0 x -f v (196)

where 0 is a 2 X 3 null matrix.

We will consider the discrete version of Eq. (192), which may be
written as

x(t 4- 1) = <t>(t-t- 1, t) x(t) + r(t + 1, t) w(t) (197)

where $(t +1, t) is the transition matrix for the system, and

r(t + 1 , t) = / 4>(t +
Jt
1, r) Bdr .
524 ELEMENTS OF MODERN CONTROL THEORY

Equations (197) and (196) are in the form of Eqs. (104) and (105),
so that the optimal estimation equations are given by Eqs. (131 HI 33).
The optimal estimates thus obtained are used to correct the system in
some appropriate fashion. In the paper by Bona and Hutchinson!34) this
information is used to “reset” the inertial system to compensate for
gyro drift rates. Typical results obtained are shown in Figs. 4.17 and 4.18.
The upper curve in each figure indicates the accuracy obtainable by
conventional methods.* The runs were made with the system operated
as described above, employing optimal estimation followed by corrections
for a period of six hours, after which a conventional correction procedure
was used. The improvement in accuracy achieved by employing optimal
estimation procedures is quite dramatic.

4.3. 4. 3 Optimal Estimation of Local Vertical and Orbital Parameters!4 1)

In this section, we discuss the application of the Kalman theory to the

problem of determining the direction of the normal to the earth’s sur¬
face from aboard an earth satellite making use of horizon sensor measure¬
ments that are contaminated by noise. In addition to providing an
optimal estimate of the local vertical, the estimation scheme corrects for
errors in the assumed values of the elliptical orbit parameters that
determine the motion in the orbital plane.
Much of the simplicity of the scheme results from the assumption that
all measurements are taken in the orbital plane and that the vehicle
rotation is about an axis normal to the plane. After injection into orbit,
the vehicle attitude control system removes rotations about all vehicle
axes except one. The attitude control system is also used to align this
axis with the normal to the orbital plane. A good estimate of the orbital
parameters (which are constant if orbital perturbations are neglected) is
available. The estimation scheme will refine these values as measurements
are processed.
If a vehicle fixed reference direction is chosen in the plane of rotation,
each horizon sensor measurement may be interpreted as an angle between
local vertical and this reference direction. The simplest situation to con¬
sider is when the reference is in the direction of local vertical at perigee
and the satellite is rotating at the mean motion of the orbit, (o> = o>s= 27t/t ,
where r is the period). At any time, t, after perigee passage, the angle, a,
between local vertical and reference is simply the difference between the

*That is, the so-called “damped inertial mode.” Cf. Ref. 40, p. 146.
STOCHASTIC EFFECTS 525

feet)
(thousand
Longitude
Error
Latitude

Figure 4.17. Longitude Error Comparison.

526 ELEMENTS OF MODERN CONTROL THEORY

true anomaly, <p, and the mean anomaly, 7, of the elliptic orbit. (See
Fig. 4.19.)

a(t) = «p(t) - 7(t) = f(t) (198)

More generally, for any constant rate of rotation and an arbitrary

injection point, (t = 0), a(t) is given by

a(t) = 13+ f(t) - f(0) - CO (199)

The quantity 0 is simply a bias angle depending on the arbitrary

choice of the reference direction. Following injection, a(t) depends upon
the behavior of f(t) as compared to f(0), and the difference between the
actual rotational rate and the mean motion of the orbit.
For orbits of small eccentricity, e, f(t) may be expressed!42) as a series
in e and7(t).

„ . 1 5 107 \
f(t) = l 2e - - e3 + 96
— e6 +
“ ' 4608
e7jsin 7(t)
(200)
47273
+ %+ e7 sin 7 7(f)
32256

Also

9ir
7(t) = — (T + t) (201)
T

where T is the (virtual) time from perigee passage to the injection point,
and

r = Ca3/2 (202)

which is simply Kepler’s third law. Here, “a” is the semimajor axis of
the ellipse, and C is a known constant.
Consequently, a(t) may be expressed as a function of five independent
parameters and time, t, viz.

a(t) == f(e, a, T, w, /8, t) (203)

STOCHASTIC EFFECTS
527
528 ELEMENTS OF MODERN CONTROL THEORY

The orbital parameters are e, a, and T, while « and /3 relate to the

orientation of the vehicle.
The measured angle, «m(t ,-), determined from a noisy horizon sensor
measurement of the local vertical at time, t,:, is

= a(t,') + V(t{) (204)

where v(t,-) is random white Gaussian noise whose statistical properties

are given by

E[v(t<)l = 0 (205)

E[v2(t,-)1 = <r2 (206)

E[v(t,-) v (ty)] = 0 i ^ j . (207)

If a nominal or calculated value of a(t,-) is denoted by anom(ti), then

4oim(t*) ^7n(ti) Oinomi^i). (208)

In calculating anom(ti), the best current estimates of e, a, T, w, and /3 are

used. At time t = 0, an initial estimate of these quantities is available.
From Eqs. (204) and (208)

Aa„(t<) = Aa(t,0 + v(t;) (209)

where

Aa(ti) = a(ti) - anom(t,-) . (210)

If the error parameters

Ae,- = e — e,-

Aa j ~~ a &i

AT,- = T — Tt- (211)

Aco, = w — oji

A/3, = 0 - p{
STOCHASTIC EFFECTS 529

are small, then Aa(t ,•) may be approximated by*

5a (t,-) 5a (t,) 5a (tj)

Aa(t,-) Ae + Aa + AT
_ 3e _ _ 5a _ . 5T _
(212)
5a (t,) 5a (t ,-)
+ Atu T~ A/3
_ dcj _ 5/3 _
/
which is obtained via a Taylor series expansion of Eq. (203) and dropping
higher order terms. In Eq. (211), e,- denotes the updated nominal value
of e calculated after measurement at time t,-, and similarly for a,-, T,-, etc.
From Eqs. (209) and (212)

Zi = M,-Xi 4- Vf (213)

where

Zi = z(t<) = Aam(t 0

V,- = v(t,-)

5a (t,) 5a (t,) 5a (tQ

M, =
5e 5a 5/3 _

a 1 X 5 matrix

“A e<
Aa,
x< = AT<
A 03i

M i -

Equation (2 13). is in the form of Eq. (105). In the companion

equation (104), w(t) = 0 in the present case, while $(t + 1, t) is the
unit matrix, since the state vector, x,, is constant. Consequently, the
associated estimation scheme, Eqs. (131 )—( 133), reduces to

Xvfi = x; + K,[z, - Mix,] (214)

K, = PtM/[MfPfM/ + Ri)~' (215)

*The partial derivatives are evaluated using nominal parameter values.

530 ELEMENTS OF MODERN CONTROL THEORY

Pitl = Pi - P,M/[MT\M/ + Ri]-1 MiPi (216)

where we have used the abbreviated notation. Here

R, = a2

and we are given

P0 = E[x0x0r]

and

E[x0J = 0 .

The computational procedure is as follows. At t = 0, we are given a

set of nominal values for the parameters e, a, T, co, and /3. Using Eq.
(203), we calculate anom(0). Furthermore, we have x0 = 0, and the given
value of P0- This permits us to calculate Pi and xx from Eqs. (214)-
(216), which in turn yields a revised set of values for e, a, T, «, and /3.
With these, we calculate aKOTn(ti),an improved estimate ofa(t). The esti¬
mation scheme then proceeds recursively, obtaining more refined values
of a( t) and the parameters e, a, T, a>,and /3.
A more complete discussion of the problem and the results of some
computer studies are contained in the paper by Knoll and Edelstein.C41)

4.3.5 Use of Quasilinearization

In applying the Kalman estimation procedure, it is essential that the

system under consideration be linear. When this is not the case, a
linearizing procedure may be employed by the usual perturbation
methods. Thus, by considering only small deviations from a nominal
trajectory, a linear system is obtained. This procedure was illustrated in
the previous examples.
A somewhat less restrictive procedure is that of quasilineariza¬
tion/43. 44) which essentially replaces a nonlinear system by a sequence
of linear equations, whose solution approaches that of the original non¬
linear system. The main advantage of quasilinearization over perturbation
methods is that a nominal or reference trajectory need not be specified a
priori.
STOCHASTIC EFFECTS 531

4.3.5. 1 The Quasilinearization Method

Consider the vector differential equation

x = f(x, r) (217)

where x is an n-dinrensional state vector,, The boundary conditions are

x,(t<) = ai; i = 1, 2, . ,n (218)

where the t,- are, in general, not all equal; i.e., we are considering a
multipoint boundary value problem.
Let x(0)(r) be an initial approximation to the solution of Eq. (217).
The initial approximation may be any function that satisfies Eq. (218).
Then the (k + l)th approximation is determined from the kth approxima¬
tion by the linear differential equation

x(*+n(T) = f<«(t) x<*+»(t) + u<*>(r) (219)

where F(fc)(r) is an n X n matrix whose ijth component is

df .-(x<*>,t)
F..l‘>(r)
i; v '
= (220)
dXj

i, j = 1, 2,

and

u<i5(r) = f(x«>, r) — F(t)(r) x(t)(r) . (221)

The maximum value of k should satisfy some error criterion, for

example

|x(*+d—x(A:)|< e (222)

where e is a predetermined error vector.

If x(*+1)(r) is sufficiently close to x(k)(r), then the differential equation
(219) is a sufficiently close approximation to the nonlinear equation
(217).
Kalabal43) showed that the sequence of functions, x(0)(t), x(1)(r), . ,
x(h-i)(t)5 converges quadratically to x(r), the solution of Eq. (217), if
532 ELEMENTS OF MODERN CONTROL THEORY

f(x, t) is a strictly convex function, and the off-diagonal terms of the

Jacobian of f(x, r) are all positive.
The general solution of Eq. (219) is

X(fc+1)(T)= H<*+1>(t) c<t+1) + p(*+D(r) (223)

where

JJ (*-Hl)(r) = F<*>(t) H<*+1>(t) (224)

H<H-i)(ro) = I the unit matrix

p(*+D(r) = F<«(t) p(*+0(r) + u(fc+1)(r) (225)

p(m)(To) = o

c(w-i) = initial condition vector

T > T0 .

The n components of cc*+1) are found by writing the expression for

the jth component of x<a+1)(t), viz.

x/*'+»(t) = E Ha<*+1>(t)
c,<A
+i>+ p/t +1)(r) .
q=l

Using the n boundary values given by Eq. (218), this may be written
as

a a = E H,-9(i+1)(r<) + pyt^Hr,-) . (226)

Q= 1

An application of this method to the solution of a simple nonlinear

boundary value problem is contained in Appendix C.
The main disadvantage in the use of quasilinearization, aside from the
fact that it is more complicated then perturbation methods, is the require¬
ment that f(x, t) be strictly convex and that the off-diagonal terms of the
Jacobian of f(x, t) be positive. However, these conditions are sufficient;
they may not be necessary. Ohap and StubberiuK45) analyze a specific
problem in orbit estimation where the quasilinearization method con¬
verges even though f(x, r) do not satisfy the aforementioned conditions.
STOCHASTIC EFFECTS 533

A detailed procedure for combining Kalman filtering and quasilineariza¬

tion as a general estimation technique is outlined in the following section.

4. 3. 5. 2 Combination with Kalman Filtering

The nonlinear system to be considered is given by Eq. (217), and its

quasilinear equivalent is described by Eq. (219). We will first transform
the latter into a linear difference equation as follows. For simplicity, we
will drop the superscripts in the ensuing discussion. Letting r = t and
t + 1 in Eq. (223), we have

x(t) = H(t) c + p(t)

(227)
x(t + 1) = H(t + 1) c + p(t + 1) .

Eliminating c. from these equations yields

x(t + 1) = $(t + 1, t) x(t) -I- w(t) (228)

where

*(t + 1, t) = H(t + l)H~l(t) (229)

w(t) = p(t + 1) - -t>(t + 1, t) p(t) . (230)

Equation (228) is identical to Eq. (104), except that r(t + 1, t) = I

and w(t) is a nonrandom vector in the present case. Consequently

E[w(t)] = w(t)

and*

Q(t) = 0 .

In this special case, the optimal estimation equations, (131 HI 33),

become

x(t + l|t) = <t>(t+ 1, t) x(t|t — 1)

(231)
+ K(t) [z(t) - M(t) x(t|t - 1)] + w(t)

*See Eq. (101).

534 ELEMENTS OF MODERN CONTROL THEORY

K(t) = 4>(t+ 1, t) P(t|t - l)Mr(t) fM(t) P(t|t - l)Mr(t)

(232)
+ R(t)]-1
P(t + l|t) = €>(t+ 1, t) {P(t|t - 1)
- P(t|t - l)Mr(t) [M(t) P(t|t - l)Mr(t) (233)
+ R(t)]-1 M(t) P(t|t - 1)} <t>T(t
+ 1, t) .

The general procedure requires that a sufficient number of measure¬

ments be taken such that the n boundary values necessary for the iterative
solution of Eq. (219) can be determined. The measurements are of the
form analogous to Eq. (105), viz.

z(t) = M(t)x(t) 4- v(t) (234)

where z(t) is an m vector.

In order to start the process, we take h measurements, z(0), z(l), . ,
z(h — 1), where h is the first positive integer that is greater than the ratio
n/iu. This then yields hm (>n) scalar equations,* the first n of which are
a set of simultaneous algebraic equations used to determine the initial
vector, c. The random vector, v(t), in Eq. (234) is assumed zero for the
moment. In other words, the initial vector, c, is only approximate, since
the noise effect has been discarded. Specifically, we have

z(0) = M(0) x(0)

z(l) = M(l) x(l)

z(h — 1) = M(h — 1) x(h — 1) .

*The(hm-n) excess equations axe merely discarded, since they are redundant for present purposes.
STOCHASTIC EFFECTS 535

Replacing x(t) by Eq. (223)

z(0) = M(0) H(0) c + M(0) p(0)

z(1) = M(l) H(l) c + M(l) p(l)

z(h - 1) = M(h - 1) H(h - 1) c + M(h - 1) p(l) .

These equations are solved for c. We now determine the (2h — 1)

optimal estimates x(l|0), x(2|l), . , x(2h - l| 2h - 2), and the cor¬
responding covariance matrices, P(l|0), P(2[ 1), . , P(2h — l| 2h —
2) from Eqs. (23 1)—(233), where P(0) is assumed given and the initial
value of the state vector is taken as

x(0) = x<*«>(0) = H(i+1(0) c + p (0) . (235)

We have included the superscripts to emphasize that the final iterated

value of the quasilinearization is used, i.e., when the condition, Eq.
(222), has been satisfied. The quantities $>(t + 1, t) and w(t) in Eqs.
(231 )—(233) are calculated from Eqs. (229) and (230).
The procedure then continues as follows.

1. Compute the optimal estimates of the next set of observations,

z(h|h — 1), z(h + l|h), . , z(2h — 112h —2),where

z(t + l|t) = M(t + 1) x(t + l|t) . (236)

2. Make the next set of (h - 1) measurements, z(h), z(h + 1), . ,

z(2h - 2).

3. Determine the prediction errors, z(h|h — 1), z(h + l|h), . ,

z(2h - 2|2h - 3), where

z(t + l|t) = z(t + 1) — z(t + l|t) . (237)

536 ELEMENTS OF MODERN CONTROL THEORY

4. Compute the (h - 1) weighted observed quantities z*(h), z*(h + 1),

. , z*(2h - 2), where

z*(t) = z(t) + Az(t|t — 1) . (238)

The quantity A is an m X m weighting matrix to be discussed subse¬

quently. These (h - 1) values of z*, together with z(h — l|h - 2), are
used to determine a new initial condition vector, c, for the quasilineari¬
zation of Eq. (217) over the interval (h — 1) < r < (3h — 3).

5. Compute the (2h — 1) optimal estimates x(2h|2h — 1), x(2h + l|2h),

. , x(3h - 2|3h - 3), and the corresponding covariance
matrices, P(2h|2h — 1), P(2h + 112h), . , P(3h —2|3h —3)
using the new initial condition vector and the previously determined
optimal estimates, x(2h — 112h) and P(2h — l| 2h) .

6. Continue the process by returning to Step 1 and incrementing all of

the arguments by (h — 1).

The presence of the weighting matrix in Eq. (238) is necessary in order

to ensure some degree of stability in the computations. For example,
if A = 0, then the new information provided by the optimal estimates is
not fed back to help improve the accuracy of the computed transition
matrices. If the transition matrices are computed using only estimates of
the observations, i.e., with A = —I, then the computations may diverge
with time, since the estimate z(t + 111) is really composed of two quantities;
one is the true optimal estimate of M(t + 1) x(t + 1), and the other is an
error due to the fact that the computed transition matrix is not the true
transition matrix. Hence, the computational errors would be cumulative.
It is sufficient to take A = —1/2 I to ensure computational stability.
The paper by Ohap and StubbenuK45) gives some numerical results for
an orbit estimation problem.

4.4 STOCHASTIC EFFECTS IN SYSTEM DESIGN

The discussion of the Wiener and Kalman methods contained in Sect.

4.2 and 4.3 does not exhaust the stochastic techniques available for
control system application. Of the many design tools that have evolved
in recent years for the rational design of control systems subject to
stochastic effects, we have chosen two examples that are especially useful
in aerospace applications. These are discussed in the following sections.
STOCHASTIC EFFECTS 537

4.4.1 Minimization of Error

Whereas the Wiener method is normally concerned with the design of

a physically realizable filter to minimize a mean square error criterion,
one often has a fixed transfer function where only a few parameters may
be varied to achieve this minimization. In one of the earliest practical
applications of the Wiener approach, Phillips!4?) considered a radar
tracking device where certain “equalizer”, parameters could be varied to
achieve a minimum mean square error. In order to describe the general
approach, wherein the basic ideas are not drowned in a tidal wave of
mathematics, we will consider a highly simplified version.
A message consisting of a signal, x(t), and additive noise, n(t), is applied
at the input of a device having a transfer function, L(s). If the output
signal is denoted byy(t), then we have, in Laplace transform notation

Y(s) = L(s) [X(s) + N(s)] . (239)

If we define the error by

e(t) = y(t) — x(t)

or, equivalently

E(s) = Y(s) - Xfs) (240)

then it follows that

E(s) = [L(s) - 1] X(s) + L(s) N(s) . (241)

We are given the power spectral densities for the signal and noise as
follows.

2 (T2ft
G zz(co) (242)
ft2 A O)2

Gnn(co) = k2 (243)

where k, a , and ft are positive constants.

The transfer function, L(s), is of the form

1
L(s) = (244)
Ts + 1 '
538 ELEMENTS OF MODERN CONTROL THEORY

It is required to determine the value of T such that the mean-square

error is minimized.
Using Eqs. (52), (53), and (60), we find that

E[e2(t)
] = ? = I\.(0)= —/ G„(»)
dco. (245)
Also, via Eqs. (63) and (241)

Gee(co)
= |L(jco)- l|2 G„(«) + IL(jto)|2G„(w). (246)
Combining the last two equations, we have

e2= — f°|L(jco)
- l|2G„(a)dco+ — [ |L(jco)|2
Gn„(w)
dco. (247)

Taking the first integral on the right-hand side of this equation, we have

— f |L(jco)—l|2Gii(co)
do;
2k L„

——f [L(jco)
—1][L(—
jw)—1]
G**(co)
dco
2k•Loo
j_ r r icoT i r_jMT_i 2^
2kloo L 1 + jcoTJ Ll - jcoTJ (/32
+ co2)
1 r _ 2jcr2 /3T2 co2dco_
27rjloo (1+ jcoT)03
+ jco)(l—jcoT)(/3
—jco)
This may be evaluated by the method given in Appendix D. The final
result is

o-2j3T
l|2G,*(co)dco
1 + T|8

Similarly, the second integral in Eq. (247) is found to be

2- /|L(jco)|2an(co)
2k Loo
dco
= 21
.
Therefore

o-2/3T
e-
(248)
1 + 0T
STOCHASTIC EFFECTS 539

In order that this expression be a minimum, we must have

k
T = (249)
<jy/20— k/3

When more complex transfer functions are considered, it is often

found that the value of a parameter that minimizes the mean square
error results in a system with unsatisfactory performance from a con¬
ventional point of view. This is generally in the form of a highly
oscillatory response, since the mean square error criterion places a heavy
weight on large errors. Consequently, results obtained by this analysis
must be interpreted with due regard for other factors.

4.4.2 Wind Loads on a Launch Vehicle

The short period dynamics of an autopilot controlled launch vehicle in

the pitch plane may be described by*

mU(a — 9) = Tc5 — La(a T- aw) (250)

IS — TJC8 T Lafa(<X 4" Ciw) (251)

5 = — Kx(0 + Ks0) (252)

a w (253)

The vehicle geometry is shown in Fig. 4.20; the symbols have the fol¬
lowing meaning.
I = vehicle moment of inertia

Ka = servoamplifier gain

Kff = rate gyro gain

^a = aerodynamic moment arm

lc = control thrust moment arm

La = aerodynamic load coefficient

m = vehicle mass

These equations are derived in Chapter One, “Short Period Dynamics,” in the companion volume,
Analysis and Design of Space Vehicle Flight Control Systems, Spartan Books, 1970..
540 ELEMENTS OF MODERN CONTROL THEORY

Tc = control thrust

U = forward velocity of vehicle

W„ = wind velocity

a = angle of attack

aw = defined by Eq. (253)

5 = control thrust deflection angle

6 = pitch angle

For simplicity, all higher order effects, such as bending, sloshing,

engine inertia, and instrumentation dynamics, have been neglected. This
is valid if one is interested mainly in crude approximations of vehicle
response to wind loads. More specifically, we shall be concerned with
the bending moments induced in the vehicle due to wind loads.
The bending moment at some station, j, (see Fig. 4.20) is given by

Mj = —Laly(a + «„)-£ m^U(d - 9) (254)

i

where ti is the distance from the ith mass to station j , and the summation
is taken over all discrete masses forward of station j.
By combining Eqs. (250)-(254), we obtain the transfer function
relating the output M,(t) to wind input W„(t) as follows.

JC[My(t)] = G(s) £[W„(t)] (255)

STOCHASTIC EFFECTS 541

where

Lq/A (s6 + a4s4 + a3s3 + a2s2 + ats + a0)

G(b) (256)
. U / (s6 + b4s4 + b3s3 + b2s2 + ths + b0)

and the a’s and b’s are known constants.

If the power spectral density of the wind is given, one may calculate
the power spectral density of the bending/noment response via Eq. (63).
Such calculations have been made by Press and Houbolh49) for aircraft.
However, the implied assumption (other than system linearity) is that
the wind statistics may be represented by a stationary time series. For a
vertically rising launch vehicle, it is known that the wind statistics are
decidedly nonstationary. Consequently, a somewhat expanded effort is
required in order to obtain meaningful results.
To begin with, the basic quantity influencing the bending moment
response is W„(t). In the present case this is assumed to be a random
variable. Normally, W„ is taken to be a function of altitude (the wind
profile). However, for a launch vehicle, the mission profile is known; we
can, therefore, relate altitude to time (based on some initially prescribed
time reference).
We now assume that the wind velocity satisfies a Gaussian probability
density function, viz.

f(xi, x2- , xn) = [(27r)"det A(W„)] 1/2exp { — ^ [x — ji(Ww)]TA 1(Ww)

•[x - m(W.)]} (257)

where

A(W„) = E{[W„ - jz(W„)][W.- p(W„)]r} (258)

"mi(W„)" "W„(ti)_

M2(W„) W w(t2)
P(W„) = w. =

JX*(W„)_ _w„(to_
542 ELEMENTS OF MODERN CONTROL THEORY

X)

X2

X =

m( W„) = E[W„(ti)] (259)

and the i|th component of A(W„) is given by

cr.-XWJ = E([W.(tO - M<(W.)][W.(ty) - My(W«)]}• (260)

This assumption is less restrictive than that of stationary wind statistics.

It is, in fact, supported by certain empirical evidence. If the wind
statistics are Gaussian, then so is the bending moment response. Con¬
sequently, one need only compute the mean and covariance in order to
define the probability distribution of the bending moment response.
In order to compute the means, m>(Ww), and variances, <ro-(W„), assume
that N samples* of the wind profile are taken. As noted earlier, the wind
profile is usually expressed as wind velocity versus altitude. However,
with each value of altitude we will associate a corresponding value of
time that is related to the specific mission profile.
We then calculate

M<(W.)
= ^N P=1
Z W„W(t,) (261)
and

*«(W.) = i £ [Ww(p)(t,-)
- M,'(WJ][W„,(p)(ty)
- w(W„)] (262)
IN p=l

where Wjp,(tt) denotes a measurement of the pth sample.

*N should be a fairly large number, perhaps a few hundred, for the results to be meaningful.
STOCHASTIC EFFECTS 543

Strictly speaking, this procedure is not correct, since the true means
and variances are given by

Mi(
Ww)=[. " - 00
f xi f(xi,x2,.
J - 00
, x„)dxxdx2. dxn (263)
and

=f. J — CO
[ [xf- m(Wv)][Xi
j — 00
- #i,-(W„)]
f(Xl,
x2,
. , xn) dxi . dx„ . (264)

However, for large values of N, the sample means and variances cal¬
culated from Eqs. (261) and (262) will be a close approximation to the
true values.
From Eq. (255), we find
t

Mr(My)
= E[My(tr)]= f g(tr —r) ftT(Ww)d
t (265)
^0

where

g(t) = £-'[G(s)] .

Equation (265) gives the mean value of the bending moment at station
j at time, tr.
The bending moment variance is calculated from

<r,.(My) = E {[My(tr) - Mr(My)] [My(t.) - /».(My)]} . (266)

In order to obtain this result in terms of wind variance, we proceed as

follows.
t

My(tr)
- Mr(My)
= f g(tr- t) W.(r)dr

- J0[ g(tr- r)Ht(Ww)

dr
t

= [ g(tr—t) fW.(r)
—Mr(W„)]
dr
Jo
544 ELEMENTS OF MODERN CONTROL THEORY

Similarly
a

My(t.) - „.(My) [ g(t. - o rvvM(r)- Mr(w.)]dr .

Therefore

[My(tr) - /ir(My)] [My(t.) M.(My)]

t

t)[W.(t)
- HT(WW)}
dr [g(t.- f)[W..(f)
- Mf(W.)]
dr
*0

t) g(t. - f) [W.W - Mr(W.)] [W.(f)

- dr dr .

Taking the expectation of both sides of this equation results in

t t

o-r.(My)
= f J g(tr- t)g(t,- f)oT^(Ww)
drdr. (267)
Thus, the mean and variance of the bending moment response is given
in terms of the mean and variance of the wind by Eqs. (265) and (267).
Furthermore, since the wind statistics are Gaussian, so is the bending
moment response; viz.

f(yi, y«, . , y») = [(2*-)" det A (My)]-1'2 exp { - \ [y - p(My)]*’

• A_1(M,)[y- p(My)]} (268)

where

A (My) = E {[My - H(My)] [My - m(My)]T}

TVty(tO ' " Ml(My)'

My(t2) M.(My)

My = A(My) =

. My(t„) . _ Mn(My)_
STOCHASTIC EFFECTS 545

yi
ys

y=

yn j

Since there are structural design limits for any launch vehicle, it is
essential that one calculate the probability of exceeding a given value
of bending moment, M? , at various values of station j. Using the pre¬
vious results, we find that the probability of not exceeding the value
M* at any time of flight, tlt t2, . , tn, is given by

, y«) dy, dy2 dy„ .

If we are interested only in the probability that M? is not exceeded

at a particular time, tr, then

[yr - Mr(M,)]2(.
Prob [My(tr) <
2cr
rr2(My)j Jr
(269)

which is a substantially simpler computation. Quantities Mr(My) and

crrr(My) are given by Eqs. (265) and (267), respectively.
Let us write M? in the form

M* = Mr(My) + ao>r(My) (270)

where a is a positive constant.

Then if we make the change of variable

yr = <7rr(My)zr + Mr(My) (271)

Equation (269) becomes

Prob (My(tr) < My] (272)

546 ELEMENTS OF MODERN CONTROL THEORY

This integral may be evaluated from standard tables (e.g., Ref. 2, p.

132). We find

Prob [M;(tr) < M*] = 0.841 if a = 1

= 0.999 if a = 3 .

For the case under consideration, jur(My) and o>r(My) are available from
Eqs. (265) and (267), respectively; therefore, for a given M*, the appro¬
priate value of a is obtained from Eq. (270), thus permitting the evalua¬
tion of the required probability from Eq. (272).
The scheme herein presented, while having a certain rational and
intuitive appeal, has several drawbacks in practice. First of all, the
available evidence that the wind statistics satisfy a Gaussian distribution
is far from conclusive. Also, the validity of Eq. (265) is based on the
assumption that the system under consideration is linear and stationary,
which, of course, is not strictly true, except for very short time intervals.
Finally, the computations become extremely cumbersome for systems of
only moderate order. The method is useful, therefore, only for obtaining
crude estimates, which must be interpreted with caution.
In actual practice, the complete nonlinear time varying system dynamics
are used to simulate the vehicle on a computer, and a large number of
runs are made for different wind profiles. Time histories of critical param¬
eters, such as bending moment, angle of attack, and engine deflection,
then become available. Various criteria, such as not exceeding a limiting
value of a parameter 95 or 99 percent of the time, are used to evaluate
the system performance.
 APPENDIX A

THE DELTA FUNCTION

The delta function is defined by the properties

8 (x) = 0 x 0 (Al)

/ f 8 (x)dx = 1 . (A2)
" a

This is more precise than the common engineering definition

8 (x) = 0 x ^ 0

5 (x) = oo x = 0 .

The delta function is not a mathematical function in the strict sense.

In all legitimate applications, this function is visualized as the result of a
limiting process involving a function, 5 (x, e), which satisfies the following
conditions

8 (x, e) > 0 — CO < X < oo (A3)

0 < 6 < oo

f 8 (x, e) dx = 1 0 < e < go (A4)

J — oo

lim 8 (x, e) = 0 x ^ 0 (A5)

e-0

Examples of such functions are

1. 5 (x, e) = -«<x<£

= 0 otherwise .

547
548 ELEMENTS OF MODERN CONTROL THEORY

1
2. 5 (x, e) x > 0
e

= 0 x < 0 .

3. 5 (x e) = e (l/e)2 .
«V 7T

e sin2(x/f)
4. 5 (x, e) =
7T x2

The most useful relation involving delta functions is

/oo
g(x) 5(x - x0) dx = g(x0) (A6)
—00

where

g(x) is continuous at x = x0.

A rigorous treatment of the delta function is contained in Ref. 52.

 APPENDIX B

THE WEIGHTING FUNCTION

The response of a linear filter to a, unit impulse (delta function)

applied at time t = 0 is called the weighting function. It follows that
h(t) = 0 for t < 0 in the case of physically realizable filters. The response
to an impulse input at time t1; $(t — ti), is given by h(t — ti).
In order to develop an expression for the output, f„(t), in terms of the
weighting function and an arbitrary input, f,(t), one may proceed as
follows.
Let the input signal, f,(t), have the form shown in Fig. 4.B1. The
response to a differential input, f*(ti) 5(t — ti) Ati, is given by f,-(ti)
h(t — ti) Ati. By virtue of linearity, the response to an arbitrary input,
fv-(t), may be approximated by

fo(t) = ^2 f<(t„)h(t — t„) At*. (Bl)

n

where the input signal is approximated by the sequence of impulses

depicted in Fig. 4.B1.

h (t)

Figure 4.B1. Form of Arbitrary Input Signal.

549
 550 ELEMENTS OF MODERN CONTROL THEORY

In the limit, as At* —> 0, we obtain

f0(t)= Jof h(t- n)f<(rO

dn. (B2)
An alternate derivation of this expression may be obtained via Laplace
transform theory. If F,(s)and F0(s) denote the Laplace transforms of f,-(t)
andfo(t), respectively, then

F0(s) = H(s) Fi(s) (B3)

where H(s) is the transfer function relating the output to the input.
Applying the Complex Multiplication Theorem* to Eq. (B3) yields

f0(t)= £~l[F0(s)]
= [ h(t- n) f»-(n)
dr,
Jo
(B4)
Via the change of variables, r = t — tu we find

fo(t)= [ h(r)f,-(t- r)dr. (B5)

It is sometimes convenient to write this as

f0(t)= Jo[ h(r)f,-(t- r)dr (B6)

which has the same value as Eq. (B5), since

f»(r i) = 0 for rj < 0.

The weighting function may, therefore, be interpreted as the inverse

Laplace transform of the transfer function.

*Ref. 46, p. 228.

 APPENDIX C

A SIMPLE APPLICATION OF QUASI LINEARIZATION

In order to clarify the basic ideas of the quasilinearization method

discussed in Sect. 4.3.5. 1, we consider the following nonlinear equation.

Xi — e*1 = 0 (Cl)

with the boundary conditions

xi(0) = xi(l) = 0 . (C2)

According to the discussion of Sect. 4.3.5. 1, we may write this as

Xi = x2 = fi
(C3)
x2 = e*1 s f2

with

xi(0) = 0 = a0i
(C4)
xi(l) = 0 = au .

From Eq. (220)

Fuw(t) = 0 F./^r) = 1
(*)
(C5)
F„lS)(r) = e1' F2!W(t)= 0

Therefore, from Eq. (221)

u ,W(r) = 0
<k)
(C6)
u2(i)(r) = (I - x,(i))e‘
551
552 ELEMENTS OF MODERN CONTROL THEORY

As an initial approximation, we assume

1

o'
I_ X 0
(C7)
x<0)(r)
_x2(0)(r)_ _ 0 _

Then

F„(0)(r) = 0 F12(0)(t)= 1

F21(0>(t)= 1 F220)(t) = 0 .

From Eq. (224)

Hn(1)(r) = H21(1)(r)

H.2(1>(t)= H22(1)(r)

H2i(r)(x)= Hn(1)(r)

H22(1)(t)= H12(1)(t)

Hn(1)(0) = H22^(0) = 1

Ht2(1)(0)= H21(,)(0)= 0 .

The solution of this system is given by

H„(1)(r)
=H22(1>(t)
=~(e-+e-)
(C8)

H12(1)(r)
=H21(1)(r)
=1(ex- e-x^)
.
Also,
fromEqs.
(225),
(C5),
and
(C6)
P,roW - p,“»

p 2U)(x)= p i(1)(r) + 1

P iO,(0) = p2(1)(0) = 0 .
STOCHASTIC EFFECTS 553

The solution to this set of equations is given by

The first component of the vector equation (223) is

x ,(1)(r) = H„(1)(r) c ,(lj + H12(1)(t)c ,(1)+ p !(1)(r) .

Substituting known values for r = 0 and r = 1 yields two equations for
the two unknowns cj(1)and c^. These turn out to bec{1)= 0 and cl1*=
— 0.46212. Therefore, the first quasilinear approximation to Eq. (Cl) is

Xi»>(r) = - 0.46212 Hu«>(r) + Pi(1)(r) (CIO)

where H12(1)(r)
and Pi(1)(f)are given by Eqs. (C8) and (C9), respectively.
The calculations of successive iterations are straightforward. The
following table summarizes the results obtained after two iterations.

T Xl«» xd1) Xi(2) Xl actual

0 0 0 0 0
0.1 0 -0.04128 -0.04144 -0.04144
0.2 0 -0.07297 -0.07327 -0.07327
0.3 0 -0.09539 -0.09580 -0.09580
0.4 0 -0.10874 -0.10924 -0.10924
0.5 0 -0.11318 -0.11370 -0.11370
0.6 0 -0.10874 -0.10924 -0.10924
0.7 0 -0.09539 -0.09580 -0.09580
0.8 0 -0.07297 -0.07327 -0.07327
0.9 0 -0.04128 -0.04144 -0.04144
1.0 0 0 0 0

The last column is calculated from the closed form solution of Eq.
(Cl), which is known to be

x(t) = — In 2 + 2 lnja sec [0.5 a(r — 0.5)] }

where a is the root of

a sec (0.25a) — y/2 — 0

or a = 1.33606 .
 APPENDIX D

THE PHILLIPS INTEGRAL

In calculating mean square error in the manner of that in Sect. 4.4. 1 ,

one is required to evaluate integrals of the form

-±r *-w~ d, (Dl)

2nj J-»h „(x) hn(—x)
where

h„(x) = a0xn + aixn_1 + . + a* (D2)

g*(x) = b0x2n-2 + bjx2"-4 + . + bn_j

(D3)
j = v - 1•

Note that the order of the numerator is at least two less then the order
of the denominator. Also, it is required that the roots of h(x) be in the
upper-half plane. Any function, f(x), which contains only even powers of
x can be factored in the required form, since if x0 is a root of f(x), so is
(~x0).
The value of the integral of Eq. (Dl) is tabulated in Ref. 47 for n = 1,
2, . ,7.
The values of In for n = 1, 2, 3, 4 are given below

2 Bo Bi

h = -
2 Bo Bi

- a, b0 + a, b, -

I
2 Bo (Bo B3 — Bi B2)
554
STOCHASTIC EFFECTS 555

b0(a2 a 3 — aj a4) — ao a3 bi -t- ao ai b2 -f- (ao a3 — ai a2)

I, =
2a0(a0 a32 -f ap a$ — ai a2 a3)

The evaluation of the Phillips integral is based on Parseval’s theorem.

See Ref. 56 for a more complete treatment.
 APPENDIX E

KALMAN’S FILTER AND PREDICTION EQUATIONS

As noted in Sect. 4.3, Kalman’s derivation of the optimal filter equa¬

tions makes use of some sophisticated mathematical concepts. In order
to exhibit the basic ideas in Kalman’s development of the main results,
we shall outline the essential steps of his derivation in simplified form. In
doing this, we will use the following properties of Gaussian random
vectors.
Let xi and x2 be Gaussian random vectors with

Mean:

m = E(xi)

M2 = E(x2)

Covariance:

(711= E[(xi — Ml)(Xl - Mi)r] = cov(xi)

<722= E[(x2 — M2)(x2 — M2)r] = COv(x2)

(712= E[(Xi — Ml)(X2 - M2)r] = cov(xx, x2) .

Then

(El)

If we let

xi = E(xi|x2)
X Xi - Xi .

Then

E(x xt) = 0 . (E2)

556
 STOCHASTIC EFFECTS 557

Finally, if xi, x2, and x3 are Gaussian random vectors, and the pair
x2, x3are independent, then

E(xi|x2,x3) = E(xi|x2) + E(xi|x3). (E3)

Properties in Eqs. (El), (E2), and (E3) are derived in Ref. 57.
We now consider the discrete form of the optimal filter problem as
stated in Sect. 4.3 and repeated here for -convenience. Given the dynamic
system

x(t + 1) = <t>(t+ 1, t) x(t) + T(t + 1, t) w(t) (E4)

z(t) = M(t) x(t) F v(t) . (E5)

The random vectors w(t) and v(t) are Gaussian with means and covari¬
ance given by

E[w(t) ] = E fv(t) ] = 0 (E6)

E[w(t) wr(r ) ] = cov [w(t)l = Q(t) t < t < t + 1

(E7)
= 0 otherwise

Elv(t) vr(r)] = cov [v(t) ] = R(t) t < r < t + 1

(E8)
= 0 otherwise

E[v(t) wr(r)] = 0 .

We are given the observed values, z(0), z(l), . , z(t). Kalman shows
that the optimal estimate of x(t + 1), given that z(0 ), z(l), . ,z(t) have
occurred, is merely the conditional mean

x(t + l|t) = E[x(t + l)|z(0), z(l), . , z (t) 1

= Efxft + l)|z(t)] . (E9)

The problem is then reduced to that of calculating Eq. (E9). For

this purpose, it is assumed that x(0) is a Gaussian random vector of zero
mean with a known covariance. It then follows that x(0), x(l), . , x(t)
is a sequence of Gaussian random vectors with zero mean. Furthermore,
558 ELEMENTS OF MODERN CONTROL THEORY

it follows from Eq. (E5) that z(0), z(l), . . z(t) is also a sequence of
Gaussian random variables with zero mean.
It is convenient to adopt the following notation, in a manner similar to
Eq. (E9).

z(t + l|t) = Efz(t + 1)|Z(t)] (E10)

x(t + l|t) = x(t + 1) — x(t + 111) (Ell)

and similarly for z(t + l|t), etc.

The fundamental problem, as previously noted, involves the computa¬
tion of the conditional expectation in Eq. (E9), which, because of the
Gaussian statistical properties of the system, may be expressed in terms
of the given means and variances. To begin with, we note that Eq. (E9)
may be expressed as

x(t + l|t) = E[x(t + l)|z(t), Z(t - 1)] . (El 2)

But

z(t) = z(t|t - 1 + z(t|t - 1) (El 3)

by definition, while by Eq. (El),z(t|t — 1) is a linear function of Z(t — 1).

It follows, therefore, that Eq. (El 2) may be written as

x(t + l|t) = E[x(t + l)|z(t|t - 1), Z(t - 1)] . (El 4)

Furthermore, z(t|t — l)and Z(t — 1) are independent random vectors

by Eq. (E2). Consequently, via Eq. (E3), we obtain

x(t + l|t) = E[x(t + l)|z(t|t - 1)] + E[x(t + 1)|Z(t - 1)]

= E[x(t + l)|z(t|t -1)] + x(t + l|t - 1) . (E15)

From Eq. (E5), we have

z(t|t — 1) = M(t) x(t|t - 1) + v (t 11 — 1)

= M(t) x(t 11 - 1) (E16)

since v(t) is uncorrelated for successive time instants. Substituting this in

Eq. (El 3) yields
 STOCHASTIC EFFECTS
559

Z(t|t - 1) = z(t) - M(t) x(t|t - 1)

= M(t) x(t) + v(t) - M(t) x(t|t - 1)

= M(t) x(t|t - 1) + v(t) . (El 7)

Also from Eq. (E4)

x(t + l|t - 1) = <t>(t+ 1, t) 2(t|t - 1) + r(t + 1, t) w(t|t - 1)

= <t>(t+ 1, t) x(t 11 — 1)

which becomes

£(t + l|t) = <t>(t+ 1, t) x(t| t - 1) + E[x(t + l)|z(t|t - 1)]

via Eq. (El 5). Applying Eq. (El), this may be written as

x(t + 111) = <t>(t+ 1, t) x(t|t — 1) + {cov[x(t+ 1), z(t|t — 1)]}

*{cov[z(t|t - 1)] }-‘z(t|t - 1) . (E18)

Also

cov [z(t|t - 1)] = E{[z(t|t - 1)] [z(t|t - l)Jr|

= E{[M(t) x(t|t - 1) + v(t)] [xr(t|t - l)Mr(t) + vr(t)]J

= E{M(t) x(t|t — l)xT(t|t - l)Mr(t) + v(t) vT(t)|

since x(t) and v(t) are independent.

Defining

P(t|t - 1) = E[x(t|t - l)xT(t|t - 1)] (El 9)

we have

cov[z(t|t - 1)] = M(t) P(t|t - l)Mr(t) + R(t) . (E20)

Finally
560 ELEMENTS OF MODERN CONTROL THEORY

cov[x(t + 1), z(t|t — 1)J = E{[‘i>(t+ 1, t) x(t) + r(t + 1, t) w(t)]

•[M(t) x(t|t - 1) + v(t)]r}

= E{ + 1, t) x(t)] [M(t) x(t|t - 1)]T}

= E{[$(t + 1, t) x(t|t - 1) + <*>(t

+ 1, t) x(t|t - 1)]

-IM(t) x(t|t - l)]T}

= El<t>(t+ 1, t) x(t11 - 1) xr(t|t - l)Mr(t)l

= $(t + 1, t) P(tlt - l)Mr(t) .

(E21)

Here we have used the fact that

a. x(t), w(t), and v(t) are all independent of each other.

b. x(t|t — 1) and x(t|t — 1) are independent of each other by Eq. (E2).

Substituting Eqs. (E20) and (E21) in (El 8), and making use of Eq.
(El 7) yields

x(t + 111) = <i>(t+ 1, t) x(t|t — 1) + K(t)[z(t) — M(t) x(t|t — 1)] (E22)

K(t) = <t>(t+ 1, t) P(t|t - l)NH(t)[M(t) P(t|t - l)Mr(t) + R(t)]-1 (E23)

while

P(t + l|t) = cov[x(t + 111)] = cov[x(t + 1) — x(t + 111.)]

= cov[x(t + 1) — <t*(t+ 1, t) x(t|t — 1) — K(t) z(t|t — 1)]

using Eqs. (E22) and (El 7).

Expanding this

P(t + l|t) = cov[<f>(t + 1, t) x(t) + r(t 4- 1, t) w(t) — <I>(t4- 1, t)

x(t|t — 1) — K(t) z(t|t — 1)]

= cov[€>(t 4- 1, t) x(t|t — 1) + <t>(t+ 1, t) x(t|t — 1)

STOCHASTIC EFFECTS
561

+ r(t + 1, t) w(t) — $(t + 1, t) x(t|t — l)

- K(t) z(t|t - 1)]

= cov{[$(t + 1, t) - K(t) M(t)] x(t|t - 1)

+ r(t + 1, t) w(t) -^K(t) v(t)}

via Eq. (El 7). Performing the expectation operation, we have

P(t + l|t) = [4>(t + 1, t) - K(t) M(t)] P(t|t - l)[d>(t + 1, t)

- K(t) M(t)]r + r(t + i, t) Q(t) rr(t + i, t)

+ K(t) R(t) Kr(t)

using the fact the x(t) w(t), and v(t) are all independent of each other.
Rearranging, simplifying, and eliminating K(t) via Eq. (E23), we obtain

P(t + l|t) = <t>(t+ 1, t){P(t|t - 1) - P(t|t - l)MT(t)[M(t)

•P(t|t- l)Mr(t) + R(t)]-1M(t) P(t|t - 1)}0>T(t

H-1, t)
+ r(t + 1, t)Q(t) rr(t + 1, t) . (E24)

The solution to the problem of obtaining the optimal filter is embodied

in Eqs. (E22)-(E24), which requires that the initial value of the covariance
matrix P(t0|t0 — 1) be given. Here obviously, P(t0|t0 — 1) = P(t0).

REFERENCES

1. J. L. Doob, Stochastic Processes, New York, John Wiley & Sons,

Inc., 1953.
2. W. Feller, An Introduction to Probability Theory and Its Appli¬
cations, New York, John Wiley & Sons, Inc., 1950.
3. H. Cramer, Mathematical Methods of Statistics, Princeton, Princeton
University Press, 1946.
4. E. C. Titchmarch, Introduction to the Theory of Fourier Integrals,
London, Oxford University Press, 1937.
562 ELEMENTS OF MODERN CONTROL THEORY

5. J. H. Laning and R. H. Battin, Random Processes in Automatic

Control, New York, McGraw-Hill Book Company, Inc., 1956.
6. N. Wiener, The Extrapolation, Interpolation, and Smoothing of
Stationary Time Series, New York, John Wiley & Sons, Inc., 1949.
7. L. Zadeh and J. Ragazzini, “An Extension of Wiener’s Theory of
Prediction,”/. Appl. Phys., Vol. 21, 1950, p. 645.
8. R. C. Booton, “An Optimization Theory for Time Varying Linear
Systems with Nonstationary Statistical Inputs,” Proc. IRE, Vol.
40, 1952, p. 977.
9. L. Zadeh, “Optimum Nonlinear Filters,” J. Appl. Phys., Vol. 24,
1953, p. 396.
10. R. C. Davis, “On the Theory of Prediction of Nonstationary Sto¬
chastic Processes,” /. Appl. Phys., Vol. 23, 1952, p. 1047.
11. C. L. Dolph and M. A. Woodbury, “On the Relation Between Green’s
Functions and Covariances of Certain Stochastic Processes and Its
Application to Unbiased Linear Prediction,” Trans. Am. Math. Soc.,
Vol. 72, 1952, p. 519.
12. J. S. Bendat, “A General Theory for Linear Prediction and Filtering,”
J. Soc. Ind. & Appl. Math., Vol. 4, 1956, p. 131.
13. R. Kalman, “A New Approach to Linear Filtering and Prediction
Problems,”/. Basic Eng., Vol. 82, Part D, 1960, p. 35.
14. R. Kalman and R. Bucy, “New Results in Linear Filtering and
Prediction Theory,”/. Basic Eng., Vol. 83, Part D, 1961, p. 95.
15. E. B. Stear, “A Critical Look at Vehicle Control Techniques,”
Astronautics and Aerospace Eng., August, 1963, pp. 80-83.
16. S. Sherman, “Non Mean Square Error Criteria,” IRE Trans., IT-4,
1958, p. 125.
17. W. B. Davenport and W. L. Root, An Introduction to the Theory
of Random Signals and Noise, New York, McGraw-Hill Book
Company, Inc., 1958.
18. J. T. Tou, Modern Control Theory, New York, McGraw-Hill Book
Company, Inc., 1964.
19. A. M. Mood, Introduction to the Theory of Statistics, New York,
McGraw-Hill Book Company, Inc., 1950.
20. M. J. Levin, “Optimum Estimation of Impulse Response in the
Presence of Noise,” IRE Trans, on Circuit Theory, 1960, p. 50.
21 . R. Bellman, Dynamic Programming, Princeton, Princeton University
Press, 1957.
22. E. Bodewig, Matrix Calculus, New York, Interscience Publishers,
1959, 2nd ed.
STOCHASTIC EFFECTS 563

23. A. Greensite, Optimization Theory for Linear Stochastic Control

Systems, General Dynamics Convair Report ERR-AN-427, Novem¬
ber 15, 1963.
24. H. Robbins and S. Monro, “A Stochastic Approximation Model,”
Ann. Math. Stat., Vol. 22, 1951, p. 400.
25. J. Blum, “Multidimensional Stochastic Approximation Method,”
Ann. Math. Stat., Vol. 25, 1954, p. 737.
26. A. Greensite, “Dynamic Programming and the Kalman Filter
Theory,” Proc. Natl Elec. Conf, Vol. 20, 1964, p. 601.
27. R. Kalman, “On the General Theory of Control Systems,” Proc.
1st Int. Cong, on Automatic Contr., London, Butterworth Scientific
Publications, Vol. 1, 1961, p. 481.
28. J. S. Bendat, “Optimum Filters for Independent Measurements of
Two Related Perturbed Messages,” Trans. IRE, Vol. CT-4, March,
1957, p. 14.
29. A. E. Bryson and D. E. Johansen, “Linear Filtering for Time Varying
Systems Using Measurements Containing Colored Noise,” IEEE
Trans, on Automatic Control, January, 1965, p. 4.
30. G. L. Smith, S. F. Schmidt, and L. A. McGee, Application of
Statistical Filter Theory to the Optimal Estimation of Position and
Velocity on Board a Circumlunar Vehicle, NASA TR R-135, 1962.
31. S. F. Schmidt, “State Space Techniques Applied to the Design of a
Space Navigation System,” Proc. Joint Automatic Control Conf,
paper No. 1 1-3, 1962.
32. J. L. Farrell, “Simulation of a Minimum Variance Orbital Navigation
System,” Proc. AIAA/ION Third Manned Space Flight Meeting,
1964, p. 46.
33. J. C. Wilcox, “Self Contained Orbital Navigation Systems with Cor¬
related Measurement Errors,” Proc. AIAA/ION Guidance & Contr.
Conf, 1965, p. 231.
34. B. E. Bona and C. E. Hutchinson, “An Optimum Stellar-Inertial
Navigation System,” J. of Inst, of Navigation, Vol. 12, No. 2,
1965, p. 171.
35. T. L. Gunkel, “Orbit Determination Using Kalman’s Method,” J. of
Inst, of Navigation, Vol. 10, No. 3, 1963, p. 273.
36. A. L. Knoll and M. M. Edelstein, “Estimation of Local Vertical and
Orbital Parameters for an Earth Satellite Using Horizon Sensor
Measurements,” AIAA Journ., Vol. 3, No. 2, 1965, p. 338.
37. R. M. Baker and M. W. Makemson, An Introduction to Astro-
dynamics, New York, Academic Press, Inc., 1960.
564 ELEMENTS OF MODERN CONTROL THEORY

38. T. T. Soong, “On A Priori Statistics in Minimum Variance Estimation

Problems,” Proc. Joint Automatic Control Conf, 1964, p. 338.
39. T. Nishimura, “On the A Priori Information in Sequential Estima¬
tion Problems,” Proc. Nat’l. Elec. Conf., 1965, p. 511.
40. C. T. Leondes, Guidance and Control of Aerospace Vehicles, New
York, McGraw-Hill Book Company, Inc., 1963.
41. A. L. Knoll and M. M. Edelstein, “Estimation of Local Vertical and
Orbital Parameters for An Earth Satellite Using Horizon Sensor
Measurements,” AIAA Journ., Vol. 3, No. 2, 1965, p. 338.
42. D. Brouwer and B. Clemence, Methods of Celestial Mechanics, New
York, Academic Press, 1961.
43. R. Kalaba, “On Nonlinear Differential Equations; The Maximum
Operation and Monotone Convergence,” J. Math., Mech., Vol. 8,
1959, p. 519.
44. R. Bellman, H. Kagiwada, and R. Kalaba, “Orbit Determination as a
Multipoint Boundary Value Problem and Quasilinearization,” Proc.
Nat. Acad. Sci., Vol. 48, 1962, p. 1327.
45. R. F. Ohap and A. R. Stubberud, “A Technique for Estimating the
State of a Nonlinear System,” IEEE Trans, on Automatic Control,
1965, p. 150.
46. M. F. Gardner and J. L. Barnes, Transients in Linear Systems, New
York, John Wiley & Sons, Inc., 1942.
47. H. M. James, N. B. Nichols, and R. S. Phillips, Theory of Servo¬
mechanisms, New York, McGraw-Hill Book Company, Inc., 1947.
48. Y. W. Lee, Statistical Theory of Communication, New York, John
Wiley & Sons, Inc., 1960.
49. H. Press and J. C. Houbolt, “Some Applications of Generalized
Harmonic Analysis to Gust Loads on Airplanes,” J. Aero. Sci., Vol.
22, 1955, p. 17.
50. J. J. Freeman, Principles of Noise, New York, John Wiley & Sons,
Inc., 1958.
51. S. O. Rice, “Mathematical Analysis of Random Noise,” Bell System
Tech. J., Vol. 23, p. 282, and Vol. 24, p. 46.
52. B. Friedman, Principles and Techniques of Applied Mathematics,
New York, John Wiley & Sons, Inc., 1956.
53. S. L. Eagin, “Recursive Linear Regression Theory, Optimal Filter
Theory, and Error Analysis of Optimal Systems,” IEEE Int. Conv.,
March, 1964.
54. H. E. Rauch and others, “Maximum Likelihood Estimates of Linear
Dynamic Systems,” AIAA Journ., Vol. 3, No. 8, 1965, p. 1445.
 STOCHASTIC EFFECTS 565

55. G. L. Smith, “The Scientific Inferential Relationships Between

Statistical Estimation, Decision Theory, and Modern Filter Theory,”
Proc. Joint Automatic Contr. Conf, 1965, p. 350.
56. G. C. Newton, L. A. Gould, and J. F. Kaiser, Analytical Design of
Linear Feedback Controls, New York, John Wiley & Sons, Inc.,
1957.
57. E. Anderson, An Introduction to Multivariate Statistical Analysis,
New York, John Wiley & Sons, Ipc., 1959.

PROBLEMS AND EXERCISES FOR CHAPTER FOUR

1. Derive the autocorrelation function for the sine wave, x(t) =

A sin(w0t + <p), where A, and are constants. [Hint: Use Eq.
(53) and note that since the integrand is periodic, one may replace
the integral and the limiting operation by an integral over one period
and division by the period. Note also that the change of variable,
y = w0t + <p,simplifies the integration materially.]

2. Band limited white noise refers to a random signal whose power

spectral density is flat over a certain frequency range and zero else¬
where, i.e.

Gxx(w)= K, 0>i< |«| < 0>2

= 0, elsewhere.

Show that the corresponding autocorrelation function is given by

rM(r) — (sin o>2t — sin on r) .

7rr

3. The power spectral density for a certain random signal is given by

2(72j
3
Gxx(w)
O)24-

where o-2and p are positive constants. Show that, for this case, the
autocorrelation function is

r xx(t) = cr2e~^lTl.
566 ELEMENTS OF MODERN CONTROL THEORY

4. It was stated in Sect. 4.1.4 that white noise is not physically realiz¬
able since it implies that successive values of the random function
are completely uncorrelated no matter how small the interval
between successive samples. To examine the physical impossibility
from another point of view, calculate the mean square value for a
white-noise signal and interpret the result.

5. Is the random signal having a power spectral density, GM(«) = 1/w2,

physically realizable? Why?

6. A random signal whose power spectral density is = 4/(co2 + 16)

is applied as the input to a filter whose transfer function is l/(as + 1)
where a is a positive constant. Determine the power spectral density
of the output.

7. Let x(t) represent the input signal to a stable filter whose transfer
function is H(s), and let y(t) denote the output of the filter. Ifx(t)
is white noise of unit amplitude, show that

Tri/r) = h(r)

where ILi/t) is the cross correlation function for the signals x(t) and
y(t), and h(r) is the weighting function of the filter. Consequently,
this is one way of determining the transfer function of a filter.

8. Suppose that the band-limited white noise of Problem 2, with K = 1

and wi = —u2, is applied as the input to a stable filter whose transfer
function is l/(as + 1). Show that the mean-square value of the out¬
put isl/Va tan_1(w2a). What is the mean-square value of the output
when the input is white noise? What conclusions can be drawn from
assuming that band-limited white noise can be closely approximated
by white noise?

9. Let [x(t) + y(t)] be the input to a transfer function which is merely

a gain, K. Both x(t) and y(t) are band limited white-noise signals
with power spectral densities

G‘a:x(a))
— 1 , |CO
| Ci)
o
otherwise
STOCHASTIC EFFECTS 567

Gi/i/cu)— 0.1 , | < IOojo

= 0 , otherwise.

Determine the value of K, which is optimal in the sense of least

square error.

10. The motion of a body in a foree-free field is given by x = 0.

Measurements of the displacement of the body from a reference
point are obtained as

Time (sec) x(ft)

0 0.95
1 2.10
2 3.05
3 3.98
4 5.02

It is known that the measurements are corrupted by additive noise

whose mean is zero and whose covariance is 0. 1 ft2 .
Suppose further that

E|x(0)] = E[x(0)] = E[x(0) x(0)] = 0

E[x2(0)] = 10 ft2
E[x2(0)] = 10 (ft/sec)2 .

Calculate the estimate and the associated covariance matrix of the

error after each observation.
 Chapter Five
OPTIMAL CONTROL

The usual performance criteria for control systems are expressed in

terms of undamped natural frequency, relative damping factor, gain and
phase margin, steady-state error, etc. At a higher level of sophistication,
one is often faced with the need to optimize the control system design
from the point of view of minimum fuel expenditure, minimum time,
maximum payload, or similar requirements. Optimal use of available
resources is an implicit goal of every engineering design. The perennial
demands for increased performance in aerospace control systems have
not only revitalized the classical mathematics, which provides the medium
and foundation for optimality, but have also stimulated further research
to improve available methods.
Certain problems in this area are well defined and can be expressed in
a mathematical format well suited to the application of the classical
variational calculus. Among these are the classical problems of program¬
ming rocket thrust to achieve maximum altitude(2°) and attain minimum
fuel transfer between circular orbits.C21) But a multitude of related
optimality criteria for launch or orbital trajectories, such as maximum
range, minimum fuel, and minimum time, lead to progressively more
difficult analytical formulations and strain the capabilities of the classical
variational methods. References 20 through 40 are only a representative
list of studies in optimal trajectories.
While optimization criteria can generally be clearly defined for space
trajectories, they are not so clearly delineated for control systems dealing
with “short period” dynamics. Here stability and constraints on the state
variables are generally the paramount considerations. It is often difficult
to relate natural mathematical optimality criteria with meaningful physical
parameters. Thus, for example, it is often expedient to use some quadratic
function of state or control variables as a mathematical measure of per¬
formance. However, it is not always possible to justify this choice by
568
OPTIMAL CONTROL 569

purely physical means. Indeed, any linear feedback system is optimal in

the sense that it minimizes the integral of a quadratic function of state
and control variables whose weighting factors are appropriately chosen.
Nevertheless, recent studies have indicated that meaningful criteria can
be established in terms of specified closed-loop response)14) or model
following/13)
The basic theories for control system and trajectory optimization are
identical; the difference is in fundamental time constants, which will
generally differ by several orders of magnitude. It is, therefore, un¬
necessary to distinguish between the two in a basic exposition. In
defining the scope of this chapter, we are guided first by the general
theme of this book, which is the presentation of established and proven
methods that have been shown to be useful in the design of aerospace
control systems. Second, it is intended to reflect the current state of
technical development. Subject to the limitations of space and mathe¬
matical sophistication expected of the reader, fundamental results will,
therefore, be presented concisely and (hopefully) clearly, with a minimum
of analytical abstractions. In addition to pedagogic examples, applications
to realistic aerospace control systems will be discussed. Limitations and
subtleties in the basic theory will be emphasized, especially as they affect
basic design.
An intelligent discussion of optimal control systems depends on de¬
fining an explicit measure of performance quality. Often this can be
done only qualitatively. For example, a launch vehicle autopilot or
satellite attitude control system may be required merely to exhibit “good”
dynamic response characteristics, such as frequency, relative damping,
and low sensitivity to extraneous disturbances. This is essentially a
problem in conventional design. If further demands are made on the
system— such as performing a stipulated mission in minimum time, or
with minimum fuel or energy expenditure— a more sophisticated analysis
is required.
There are essentially two aspects to the problem of optimal control.
One is the proper formulation of the problem (which is not always trivial);
the other is the development or application of an appropriate tool to
solve the problem.
The first is more an art than a science, since the formulation of a
mathematical model for a real system must be a compromise between
the needs for analytical tractability and for including all significant
dynamic features. The results of an analysis must, therefore, be inter¬
preted with due regard for these factors.
570 ELEMENTS OF MODERN CONTROL THEORY

This chapter is concerned primarily with the analytic tools for solving
optimization problems. The natural vehicle for this exposition is the
classical calculus of variations. Broadly speaking, the problem is formu¬
lated as follows. Assume a system whose dynamic properties are described
by a set of ordinary differential equations. Some type of initial and
terminal conditions are prescribed. One or more degrees of freedom are
available in the form of a control function that must be programmed such
that a stipulated function of the control and state variables is minimized
(or maximized).
In this fashion, we have the classical problem of Mayer* and the well-
established solution. Some qualifications are in order, however. First,
this “solution” is often in the form of a system of nonlinear differential
equations with two-point boundary conditions. The computational diffi¬
culties associated with this are far from trivial and often overwhelming.
Second, the classical formulation includes no constraints on the state or
control variables. This severely limits its usefulness, since, in the real
world, bounds on energy and magnitude of a control force are hard
realities. In some instances, a state variable must be contained within
prescribed bounds— such as limiting an angle of attack to ensure structural
integrity.
As a result of these limitations in the classical techniques, several new
approaches to the problem have been initiated in the last ten years. These
have led to the development of dynamic programming, the gradient
method, the maximum principle, and the generalized Newton-Raphson
technique. With this new body of theory, the range of problems that can
be resolved has been considerably enlarged. But as usual, few panaceas
are available.
Dynamic programming, which exhibits the greatest indifference to the
presence of “pathological” functions, is limited by computer storage
capacity.
The gradient method, one of the more powerful techniques, generally
yields only local extrema, primarily because small deviations from a
reference condition are analyzed.
The maximum principle, in which is included the effect of control
constraints, is a more elegant formulation of the classical Weierstrauss
condition in variational calculus. It is, however, completely identical to
the classical methods when the inequality constraints are included via the
Valentine condition.!")

*Also called the problem of Lagrange or Bolza, when expressed in slightly different fashion.
OPTIMAL CONTROL 571

The generalized Newton-Raphson technique is a powerful new tool

for solving nonlinear systems with two-point boundary conditions. It
has been remarkably successful in solving problems that strained the
capability of conventional methods. This, of course, has served to in¬
vigorate the classical approach, which is attractive because of its analytical
elegance. The weakness of the method is that only sufficient conditions
for convergence are known. Even these are hard to apply. It is known,
however, that the method works in cases where the sufficiency condition
is not satisfied. (In other words, this sufficient condition may not be
necessary.) Consequently, a certain amount of “cut-and-try” is necessary
in practice.
For any given problem, therefore, one may select one of a multitude
of superficially unrelated methods to achieve optimization. Generally,
the form of the problem or the type of results desired will indicate the
method to be used. A fuller understanding of the virtues and limitations
of each method requires recognition that all these methods are implicitly
(if not explicitly) related. We have, therefore, not derived any of these
results in the conventional manner but have shown (Sect. 5.1.5) that
each is a consequence of one general formulation.
The examples of application of the theory are an attempt to balance
simplicity of exposition with practical realism. These examples, together
with a sampling of the literature cited in the references, provide a fair
indication of the current state of the art.
The basic problem of optimal control can be stated generally as fol¬
lows. Assume a dynamic system described by some appropriate set of
differential equations. A parameter or set of parameters (controls) is to
be programmed in such a way that a prescribed measure of performance
takes on an extremal (maximum or minimum) value. Usually, there are
prescribed values for the initial and/or final state of the system. Further¬
more, there may be constraints, dictated by physical considerations, on
the control or state variables. Taking account of all these factors, we
seek to determine the control function form that ensures the measure of
performance is maximized, or minimized, as the case may be.
There are many aspects to this problem, depending on the complexity
of the system, the form of the constraints, and the type of performance
criterion adopted. Furthermore, a solution may be characterized as:
open-loop, wherein the control is obtained as a function of time and,
therefore, of the initial conditions; or closed-loop, in which case the
control is a function of the current state of the system.
All these considerations directly affect the mathematical complexity
and the ease of obtaining a solution. The following sections present the
572 ELEMENTS OF MODERN CONTROL THEORY

essential features of standard optimization techniques, beginning with the

simplest problems expressed in the classical variational format. This is
followed by a discussion of the classical theory limitations that motivated
some of the modern techniques, such as the maximum principle, gradient
methods, and dynamic programming.
Typical applications and standard solutions are then described. The
concluding sections deal with recent theoretical developments, together
with typical aerospace applications.

5.1 MATHEMATICAL CONCEPTS

The mathematical features of the basic optimization techniques are

described in Sects. 5. 1.1 -5. 1.4. In general, the treatment is concise but
explicit; the primary aim is to facilitate application of the theory to
practical engineering problems. Elaborate motivations and mathematical
abstractions are, therefore, avoided. Illustrative examples are used
liberally to enhance comprehension of the basic ideas.

5.1.1 Variational Calculus

For the solution of optimization problems of a more general character

than is possible via the elementary calculus, the classical tool is the cal¬
culus of variations. Most problems of interest to the aerospace controls
engineer can be formulated in terms of the so-called Mayer problems 134)
which includes a variety of related problems as special cases. It is,
therefore, of extreme generality and will, accordingly, be described in
some detail.
The Mayer problem is usually stated as follows. There are n functions,
x*(t), of an independent variable, t, that satisfy the differential constraints

^•(x, x, t) = 0 j = 1, 2, - , p(< n)
0)
x(t) = n vector

subject to boundary conditions of the type

>Pr(Xi, X/, t,', t/) = 0 r = 1, 2, - , q« 2n + 2) (2)

OPTIMAL CONTROL 573

where t,- and tf are the initial and final times, respectively, and*

x, = x(t<)

X/ = x(t/)

d_
(') = dt (' ) .

It is required to choose the functions, x*(t), such that the quantity

J = [G(x,t)]/ i

= G(x, t) | (=%
ty ~ Cr(x,t)| t=<4- (3)
is minimized subject to the constraints, Eqs. (1) and (2).
The problem thus formulated is more general than is superficially ap¬
parent. For example in the problem of Lagrange, the function to be
minimized is

J = f L(x,x, t) dt . (4)
J t.
I

If we define an auxiliary variable by

t

x„+1(t)
= J L(x,
x,t)dt (5)

with

XTl+l(tj) 0 (6)

then we reduce to a Mayer-type problem such that

J X„+l(t/) (7)

with the additional constraint

= x„+i(t) - L(x, x, t) = 0 (8)

*The subscripts i and f are used exclusively to denote initial and final value, respectively, not the
components of a vector. Symbols other than i and f will be used to denote vector components.
574 ELEMENTS OF MODERN CONTROL THEORY

In the problem of Bolza, the function to be minimized is

f
,] = [G(x, t)] + L(x, x, t) dt . (9)
'
t

We again reduce this to a Mayer-type problem by defining an auxiliary

variable in the manner shown above.

Inequality Constraints. If there exist inequality constraints of the

form*

Kx< x*< K2 (10)

then by introduction of a new variable, xn+i(t), defined by(")

(x* - K0(K2 - ±k) - xn+12 =0 (11)

the problem is put in the Mayer format, where J, defined by Eq. (3), is
minimized subject to the added differential constraint

<Pn+
1 = (Xi - K0(K2 - Xk) - xn+i2 = 0 . (12)

The Mayer format is, therefore, extremely general and finds wide
application in aerospace controls problems. A solution is obtained in the
following way. Form the augmented function

F=Zx^ (13)
j-i

where the \3 are time dependent functions called the Lagrange multipliers.
A necessary condition for the minimization of the criterion function
defined by Eq. (3) is that the Euler-Lagrange equations

d_ k = 1, 2, ••••, n (14)
dt

be satisfied.
The system of Eqs. (1) and (14) is subject to (2n + 2) boundary condi¬
tions. Of these, q are supplied by Eq. (2), while the remaining ones are

*The discussion also applies if the inequality constraint is on x* instead of x*.

OPTIMAL CONTROL 575

determined from the transversality condition*434)

A dF . dF_
5G + A - X/t (15)
*-i dxk dxk

where 5( ) is the variation operator.* Irr general

5G= ± «xt+ ^ St (16)

*- 1 dxk at

which means that Eq. (15) can be expressed as

(17)
%

This expression splits up into (2n + 2) subconditions of the form

(18)

(19)

(20)

(21)

If the boundary condition for a particular variable, say xkf, is not

prescribed, then 8xkf is arbitrary, which means that

See Appendix B.
576 ELEMENTS OF MODERN CONTROL THEORY

(f-
\dxk
+ f)
dxk) t=tf
= 0. (22)

Notice that boundary values cannot be assigned to nonderivated

variables. These are, in fact, a mathematical consequence of the con¬
straining equations, (1), and the Euler-Lagrange equations, (14).
If the augmented function, F, does not contain t explicitly, then it
can be shown that*

—F + Z —x* = C = constant . (23)

k—1 dXk

This means that Eq. ( 17) simplifies to

= 0 (24)

and the set of Eqs. ( 18)-(2 1) is simplified accordingly.

The solution of certain variations problems is characterized by the
fact that one or more of the derivatives, xk, are discontinuous. For
example, in a rocket vehicle, the acceleration is discontinuous at the point
where thrust is terminated or staging occurs. In this case, a mathematical
criterion is needed to join the portions of the extremal arc^ at points of
discontinuity. This is supplied by the Erdmann-Weierstrass corner con¬
ditions' Cl34)

(25)

F+ i: — x,] = F+ Z^x* (26)

l dx,t *=i dXk / +

where the negative and positive signs denote conditions immediately

before and after a point of discontinuity.
For the special case in which F does not depend explicitly on t, the
last relation reduces to

(C)_ = (C)+ (27)

*Ref. 134, p. 205.

fThe extremal arc is that curve in n space characterized by Xi(t), •••• xn(t), which extremizes J.
OPTIMAL CONTROL 577

by virtue of Eq. (23). In other words, the integration constant, C, has

the same value for all subarcs composing the extremal arc.
Satisfying the Euler-Lagrange equations, (14), merely ensures that
function J, defined by Eq. (3), is either a maximum or a minimum. More
precisely, it is a local extremum; in other words, it may be only one of
many extremal functions. This corresponds roughly to the case of a
curve in elementary calculus whose derivative vanishes at several points.
The fact that the slope is zero gives no information as to whether the
point on the curve is a maximum or minimum or whether it is indeed the
global maximum or minimum. In the variational calculus, the necessary
and/or sufficient conditions for type of extrema are not easily applied.
Perhaps the most useful for engineering purposes is the Legendre-Clebsch
condition ,(134) which states that the relation

(28)
k=1 j=l C^Xfc
SXj

must be satisfied if J is a minimum. If the derivative, xr, of function xr

does not appear explicitly in F, then in Eq. (28) we replace x, with xr.
The above condition is merely necessary (not sufficient) to ensure that
J is a minimum. Furthermore, only a local minimum is ensured.
Fortunately, in most cases of engineering interest, the type of extremal
arc obtained is determined from physical or numerical considerations,
and only rarely does an ambiguity exist between local and global extrema.
The methods discussed above are illustrated in the following examples.

Example 1 . The Sounding Rocket: This is one of the earliest aero¬

space problems treated by variational methods. (5,20,150) in the usual
formulation, we seek the thrust-time history such that a maximum height
is attained for a given propellant weight. Alternatively, we may seek the
minimum propellant weight to attain a specified altitude. The relevant
differential equations are

T - D = m(V + g) (29)

h = V (30)

T = a,* (31)

0 m . (32)
578 ELEMENTS OF MODERN CONTROL THEORY

Here

T ; - thrust

D s D (V, h) s drag

m = instantaneous mass

h = altitude

Y = velocity
g = gravity acceleration

fi = exit velocity of burned propellant

(3 - propellant mass flow

Physical considerations require that the mass flow satisfy the inequality

0 < j3 < pM (33)

where is a given constant.
The problem is easily formulated as a Mayer-type, as follows. The
differential constraints are

Vi = h - V = 0 (34)

^ Y + g + (D ~ M = 0 (35)
m

v>8= m p —0 (36)

v* = mtt - 0) -«2 = 0 . (37)

The last relation, which is in the form of Eq. (12), accounts for the
inequality constraint, Eq. (33). No special physical significance is
attached to the variable, a.
The problem variables are identified as follows.
xi = h

x2 = V

x„ = m

X4 = /3

X5 = a
OPTIMAL CONTROL 579

The augmented function is, therefore

F —Xi(h—V) -j-X2^V
+ g+ m + 0)
(38)
+ \<[0(0m ~ 0) ~ a2]
/
and the Euler-Lagrange equations become

X2 dD
X, = (39)
m dh

X
X’ -“ —X
*' 4-
+ m W (40)

(0H — D)X2
Xa — (41)
nr

° =“(mX2
“ Xa)
+X4(0*~2/3) (42)

0 = a\a . (43)

Since F does not contain t explicitly, we have, from Eq, (23), com¬
bined with the Euler-Lagrange equations, (42) and (43)

^iV
—^g+ —
^x2+ X2
— 0=c (44)

where C is an integration constant.

It now remains to formulate the criterion function, J,and the boundary
conditions, \f/T. If we seek to maximize the altitude attained, subject to
the boundary conditions

hj =0 ti - 0
V, = 0 V/ = 0 (45)
m, - m0 m7 =
then J becomes*

J = — h/ . (46)

Obviously, the problem of minimizing (— h/) is equivalent to that of maximizing h/.

580 ELEMENTS OF MODERN CONTROL THEORY

Since t, and h,are not prescribed, the transversality condition, Eq. (17),
yields

(47)

(48)

The first of these relations implies that

C = 0 (49)

since G(= —h) is independent of t and bif is arbitrary. Equation (48)

leads to*

— 1 + Xi/ = 0 . (50)

According to Eq. (43), an extremal arc is composed of subarcs along

which a = 0 or X4 = 0. The first of these implies that /3 = 0 or /? =
by virtue of Eq. (37). If we write

K = — X2 — X3 (51)
m

then the condition X4= 0 and Eq. (42) leads to

K = 0 and K = 0 . (52)

Taking the time derivative of Eq. (51) and combining with the Euler-
Lagrange equations, we obtain

dD \
m(D + mg) K = 0
^V+ D) K -f* Xi (V - M) D

(53)

+ ' v &v ~ mg i M

after making use of Eq. (44).

The notation Xi/ stands forXi (t/).

OPTIMAL CONTROL 581

Consequently, along a variable thrust subarc, the relation

(V- M)
D+ (v^ - mgjn=0 (54)
must be satisfied. This, therefore, determines the variable thrust param¬
eter, (8 = — rii. In order to ascertain the physical significance of this
equation, consider a drag function given by

D = Ci V2, Ci = positive constant. (55)

Combining this with Eq. (54), we find

Ci V2
m

This shows that a decrease in mass is accompanied by a decrease in

velocity; in other words, the acceleration is negative. Consequently, the
variable thrust arc is such that the vehicle is flown with thrust always less
than drag during this phase.
It now remains to determine how the subarcs for zero, maximum, and
variable thrust are connected such that the optimality conditions are
satisfied and in a manner that is consistent with the given boundary
conditions. To do this, we make use of the Weierstrass-Erdmann corner
conditions and the Legendre-Clebsch conditions. Equations (25) and
(26) show that at each corner point

(Xi)_ = (Xi)+
(Xj)—= (x2)+
(56)
(X8)_ = (x8)+
(C)_ = (C)+ .

That is, the above parameters are continuous at the corner points of the
extremal arc.
Now since C = 0 and since V and m are also continuous, Eq. (44)
shows that 8 may be discontinuous provided that

(K)_ = (K)+ = 0 (57)

which implies that

(K)_ = (K)+ = 0 . (58)
 ELEMENTS OF MODERN CONTROL THEORY
582

Furthermore, the Legendre-Clebsch condition, Eq. (28), leads to

—2A„[(5«)2 + m > 0 .

Therefore, when X4 j* 0, we have, by virtue of Eq. (42)

- 5^5 [(!“)!
+H 5 0
or, equivalently

K
> o . (59)
Pm - 2/3

It was noted earlier that \4 ^ 0 implies that /3 - 0 or /3 = Pm- Conse¬

quently, the above inequality shows that

p = Pm when K > 0

P — 0 when K < 0 (60)

0 < p < Pm when K = 0 .

The last relation follows from the fact that for \4 = 0, a ^ 0 which
means that p may assume an intermediate value as shown by Eq. (37).
The quantity P (= — m) is in fact determined from Eq. (54).
It now appears that the solution is completely determinate. The six
equations, (34)-(36) and (39H41), together with the six boundary con¬
ditions

h4 = 0 \i/ = 1 from Eq. (50)

V4 = 0 V/ = 0

m, = 0 ni/ = mp

may be integrated to yield m, V, and h as well as the Lagrange multipliers.

Quantity Iv, defined by Eq. (51), is calculated in the course of the
solution and determines the thrust switching points in accordance with
Eq. (60).
It is tempting to consider the problem as now solved. However, from
a practical point of view, several factors disturb this state of euphoria.
First of all, the difficulties associated with solving a system of nonlinear
OPTIMAL CONTROL 583

differential equations subject to two-point boundary conditions are too

well-known to be belabored here. In the present case, one must guess the
initial values of X1;X2, and X3 in Eqs. (39)—(41 ) and hope that the final
boundary conditions are satisfied. Various iterative procedures are
available for adjusting the initial guesses to ensure that one converges to
the stipulated final values. This is not always possible, since poor
guesses lead to numerical instabilities, with the result that the engineer is
often led to seeking other ways of making a living. Nevertheless, the
practical importance of this problem has stimulated extensive research
for improved methods. Some promising approaches are discussed in
Appendix A.
In the present problem, it is not necessary to solve the equations of
motion in order to determine the general features of the optimal thrust
program. In does require, however, that certain simplifications be intro¬
duced. Specifically, we assume that the drag is given by Eq. (55). Then,
since D is independent of h, Eq. (39) reduces to Xi = 0, which means that
Xi = 1 by virtue of Eq. (50). As a result, Eqs. (53) and (54) may be
written as

m(ci V2 + mg) K = ci /3V (2/u + V) K + Q (61)

(62)

We may plot the variable thrust condition, Eq. (62), in the m — V

plane as shown in Fig. 5.1 . The initial and final points on the trajectory
are here designated by ® and respectively. It is obvious that an
extremal arc begins with maximum thrust from Point ® (corresponding
to the initial conditions m< = m0, and Vi = 0). The rest of the figure
shows various possibilities of using maximum thrust, coasting, and variable
thrust to arrive at the terminal point, @, which is obviously approached
via a coasting arc (constant mass). We will show that assuming the
quadratic drag law given by Eq. (55) leads to a unique optimal thrust
program consisting of the three subarcs shown in Fig. 5.2. Consider first
the possible path, © - ® - ® - ©, shown in Fig. 5.1. Along the maxi¬
mum thrust arc, ® - ®, we must have K > 0, while along the coasting
arc, © - ©, K < 0 as required by the optimal switching conditions, Eq.
(60).
But since Eq. (57) shows that K is continuous at a switching point, it
follows that
584 ELEMENTS OF MODERN CONTROL THEORY

at Point © : K = 0, K < 0

at Point © : K = 0, K > 0 .

In this case, Eq. (61) indicates that Q < 0 at Point © and Q > 0 at
Point ©. Expressed in terms of Eq. (62)

v,!C~+') < msg at Point ©

**.(?+>) > m,g at Point © .

According to Fig. 5.1, Vr < V2, so that the above two conditions are
incompatible. By similar arguments, we eliminate the Subarcs © - © - ©
 OPTIMAL CONTROL 585

and ® - ® - ©. The optimal path must, therefore, be as shown in Fig.

5.2, composed of a maximum, followed by variable thrust, and then a
coasting arc.
If the final mass is sufficiently large, then there will be only a path of
maximum thrust followed by coasting, as shown by Path ® - © - ©
in Fig. 5.2.
It may be shown that the solution is not completely determinate in all
cases. Consider, for example, the situation shown in Fig. 5.3. The peak
and trough in the variable thrust curve, Eq. (54), may be due to the
peculiarities of the drag function near Mach 1. A variable thrust along
® - © is not admissible, since this corresponds to increasing mass.
Consequently, there must be some intermediate coasting path, © - ®.
However, the precise location of this path cannot be determined from
the present theory.
We have analyzed this example in some detail, since it highlights both
the virtues and the limitations of the classical variational calculus. The
586 ELEMENTS OF MODERN CONTROL THEORY

Figure 5.3. Possible Ambiguity in Thrust Program for General Drag Law.

theory is well adapted for solving relatively simple (textbook) problems

but suffers from severe limitations of a computational nature when
applied to moderately realistic practical problems. This motivated the
development of the so-called direct methods (gradients, dynamic program¬
ming), which generally exhibit marked computational advantages over
the variational approach. Before turning to these, we will first discuss
the Pontryagin maximum principle, in which the classical variational
calculus is highly systematized and is, therefore, much simpler to apply
in practice.

5.1.2 Maximum Principle

A wide class of optimum control problems can be formulated as

follows. Given the system
OPTIMAL CONTROL 587

X = f(x, u, t) (63)

where x is an n-dimensional state vector and u is an m-dimensional con¬

trol vector. The components of the latter are subject to constraints of
the form

Vi < uy < nj
(64)
j = 1, 2, - , m

where the v, and m are known constants. It is required to determine u(t)

in the interval, t< < t < tf, such that the criterion function

J = Cr X, (65)

is a minimum (maximum). Here c is a constant n vector that is known,

and the notation x, means x(ty). In other words, we seek to minimize
(maximize) a prescribed linear combination of the final value of the state
vector components.
As noted in Sect. 5.1.1, many types of criterion function can be re¬
duced to form Eq. (65). Thus, if it is required to minimize (maximize)
the integral

(66)

we define a new state variable by

(67)

with

Xn+i(t,-) = 0 (68)

and add the equation

xu+i = L(x, u, t) (69)

to the system in Eq. (63). With these modifications, the problem is

588 ELEMENTS OF MODERN CONTROL THEORY

reduced to that of minimizing (maximizing) the quantity

J = x„+i(t/) . (70)

Furthermore, the problem of minimizing (maximizing) some function

of the final state vector, f(x,), reduces to the same format if we define
an additional state variable

xn+i = f(x) (71)

with*

Xn+l(ti) = f(Xj) (72)

and add to the system, Eq. (63), the equation

Xn+i
= Vf d± (73)
U ’ **

The new problem is then reduced to that of minimizing (maximizing)

the quantity

^ X7!/^-i(t»/)
* (74)

In order to avoid any possible ambiguity due to the notation adopted,

we emphasize that the subscripts i and f are used exclusively to denote
initial and final values, respectively. They are not to be interpreted as
components of a vector. Vector components will be designated by sub¬
scripts j, k, etc. Thus
Xi(ti)
x2(tf)

-Xn(ti).

x, = jth component of the vector x

x,7 = Xy(t/), etc.

*The symbol x< means x (tj) .

OPTIMAL CONTROL 589

One frequently encounters the so-called “minimum time” problem, in

which it is required to transfer the system from a given initial to a given
final state in minimum time. Mathematically, this requirement is
expressed as that of minimizing the integral

i:dt . /
(75)

By introducing a new state variable such that

Xn+1 = 1 , xn+i(t<) = 0 (76)

Xn+1
= J‘if dt (77)

the problem is reduced to one of minimizing the final value of xre+1.

In what follows, it will be assumed that all necessary transformations
have been accomplished; we will deal exclusively with the set of Eqs.
(63M65).
We now define the components of a vector, A, as follows.*

X,= -
*-i dx> (78)

j = 1, 2,

We also define the function

H = Z V; = W % (79)
J— 1

One sometimes refers to A as the costate vector; the function, H, is

known as the hamiltonian.
The maximum principle of L. S. Pontryaginf4) is then stated as follows.
“In the system described by Eq. (63), if there exists a control vector,
u, which satisfies the constraints, Eq. (64), while minimizing (maximizing)
the criterion function, Eq. (65), then this control vector necessarily
maximizes (minimizes) the hamiltonian, Eq. (79).”

Our notation anticipates the interpretation of this quantity as the Lagrange multiplier.
590 ELEMENTS OF MODERN CONTROL THEORY

Making use of the definition of the hamiltonian, Eq. (79), we may

write Eqs. (63) and (78) as

(80)

j = 1, 2, - , n

(81)

j = 1, 2, ••••, n .

These are identical in form to the Hamilton canonical equations in

analytical mechanics (which accounts for the name accorded the function
H).
In application, the extremal control vector, u, is obtained as a function
of X after the minimizing (maximizing) operation on H is performed. A
completely determinate solution is then obtained by integrating the 2n
equations, (80) and (81). This, in turn, requires that 2n boundary condi¬
tions be specified. However, there are only n boundary conditions
immediately apparent, namely

Xy(ti) Xy{
(82)
i = 1, 2, - , n

The other boundary conditions depend on the constraints imposed on

the final value of the state variables and the form of the criterion
function, Eq. (65). We distinguish several cases.

I. Final Time Fixed; No Constraints on Final State Variables

In this case, the remaining n boundary values are

Ay(tr) = -c,
(83)
j = 1, 2, - , n

where the c, are the components of the vector c in Eq. (65).

OPTIMAL CONTROL 591

II. Final Time Fixed; Constraints Imposed on Final State Variables

One type of constraint imposed on the final state vector is that it lie
within a prescribed region of the state space, viz.

+ [x(t,)] < 0 . (84)

In this case, if the function \p is differentiable, we write

dip
by (85)
dXj t—t
1

and the boundary values for X are given by

\ (t/) = - Oy - 7?by[x(t/)]
(86)
j = 1, 2,

t [*(t/)] = 0 (87)

where y is a constant. There are (n + 1) equations in the set, Eqs. (86)

and (87) for the (n + 1) unknowns X,(t/), j = 1, 2, - , n, and the
constant y. Thus, one of the above equations may be used to eliminate y,
which in combination with the initial values in Eq. (82) yields the
required 2n boundary conditions for the set, Eqs. (80) and (81).
In the special case where q(< n) components of x,(t/) are prescribed
exactly

x;'(t/) = Xy
(88)
j = 1, 2, - , q(< n)

the remaining boundary values are given by

X*(t/) = — c*
(89)
k = (q + 1), - ,n .

III. Final Time Open

Since there is an additional degree of freedom in the system, an addi¬
tional equation is required to make the solution determinate. If tf does
592 ELEMENTS OF MODERN CONTROL THEORY

not appear in the criterion function, Eq. (65), then

H(t/) = X Xyfy (90)

j-i

is the additional relation required. If ty does appear in J, then the variable

xn+x is introduced such that

Xm-1 = 1

x„+i(ti) = 0 .

The final value of xn+i replaces ty in the criterion function, and the
added relation takes the form

»H-1

H(t/) = X Xyfy = 0 . (91)

y-i

The statement of the maximum principle, together with the boundary

conditions as specified above, contains all the information necessary to
solve the optimal control problem. There is, however, one possible
ambiguity, and this is discussed next.

Singular Solutions. For a certain class of problems, the control, u, is a

scalar which enters the hamiltonian in a linear manner, viz.*

H = I(x, X) + u K(x, X) (92)

where I and K are scalar functions that are independent of u. Here it has
been assumed that the explicit dependence of the system equations or
criterion function on time, t, has been removed by defining an additional
state variable xn+i such that

Xn+l = 1 , xB+1(0) = 0 .

It is possible that the switching function, K(x, X), is identically zero

over some finite time interval. In this case, the hamiltonian H ceases to

* Actually, the discussion that follows is applicable to any function of u as long as the form of Eq.
(92) remains.
OPTIMAL CONTROL 593

be an explicit function of the control variable u, and the maximum

principle is unable to provide information about the desired optimal
control. The usual procedure of selecting u so as to maximize H breaks
down. In some instances, it is possible to show that the condition
K(x, X) — 0 violates the constraints of the system (see the latter part of
Example 2). Thus no further analysis is necessary, i.e., the solution is not
singular. ,
In what follows, we will investigate the nature of singular solutions
based on the work of Johnson and Gibson. (174> It may be shown that*

H* = I + u* K = Max(I + u K) = 0
u _ _ (93)
0 < t < t, .

Therefore, the condition

I(X, x) = K(X, x) = 0 (94)

corresponds to the case of singular control. The problem is simplified

materially if the above condition reduces to a relation which is inde¬
pendent of X. To do this, we may use the equations

I = I = I = •••• = 0
(95)
K = K = K = •••• = 0

which follow directly from Eq. (94). If we now substitute^

dH(x, X, ul) (96)

dH(x, X, u*)
Xy = (97)
dx,

into Eq. (95), then it may happen that the optimal control in the singular
region reduces to a function involving the state variables only, viz.

n* = u*8(x).

*Here, and in the remainder of the chapter, the asterisk superscript will be used to denote tne
optimal value.
f The subscript s on u* is used to denote the optimal control in the singular region.
594 ELEMENTS OF MODERN CONTROL THEORY

A result of this type is obtained in Example 2. This is not always

possible, especially for high order systems. However, when a reduction of
this type can be achieved, the optimal control in the singular region is
available as a function of the current state of the system.
For a more detailed discussion of the problem, the reader is referred to
Johnson and Gibson.!174)
The application of the maximum principle to problems of optimal
control may be summarized as follows.

a. Form the hamiltonian, Eq. (79).

b. Derive the optimal control, u*(t), by choosing that u(t) which (sub¬
ject to the control constraints) maximizes the hamiltonian.
c. Investigate possible singular solutions.
d. Determine the optimal trajectory by integrating the 2n equations,
(80) and (81), using the optimal control u*(t) and the appropriate
boundary conditions.

It is instructive to compare the necessary conditions for an optimum,

Eq. (78), with the Euler-Lagrange equations, (14). If we write

<f>j= ~ fj = 0 (98)

then

F = Z A,(±y- fy) (99)

whereupon Eqs. (14) become

A*=-LXy^-. j OX/t
(100)
Comparing this with Eq. (78), we see that the choice of notation is
indeed justified.
It should be emphasized that the maximum principle provides only a
set of necessary conditions, much the same way as the Euler-Lagrange
equations and the Legendre-Clebsch condition provide only necessary
conditions in the variational calculus. Sufficient conditions are hard to
come by (except in the case of linear systems). Nevertheless, this is
usually a mathematical luxury. In practical situations, physical con¬
siderations will generally confirm uniqueness and sufficiency.
OPTIMAL CONTROL 595

We should note also that the maximum principle yields a solution in

the form of a set of (in general) nonlinear equations with two point
boundary conditions. Thus, there exist the same computational diffi¬
culties as with the variational calculus.
In the final analysis, it must be granted that the maximum principle
solves no problem that cannot also be solved by variational methods.
Its appeal is mainly in the elegance of ^its format and the ease with
which it can be applied. This, however, is a powerful argument in its
favor for purposes of practical application.
The results obtained thus far yield an optimal control that is a func¬
tion only of time and the initial conditions. In other words, the control
is open-loop. For most control systems it is desirable to have a control
which is a function of the current state of the system, i.e., closed-loop
control. The latter is generally hard to come by, except in special cases.
For linear systems with a quadratic performance function, it is possible
to obtain an optimal closed-loop control by deriving the Hamilton-
Jacobi equation of the system. This is done in Sect. 5.2.1.

Example 2. The Sounding Rocket. The problem formulated in Ex¬

ample 1 will here be solved via the maximum principle. We have

Xi = x2 = fi (101)

x, = (102)
x8

x8 = — 0 = ft . (103)

The boundary conditions are

Xi(ti) = 0 ti = 0

x2(t<) = 0 x2(t/) = 0 (104)

x,(t<) = 0 x»(t/) = mp

where xi = h, x2 = V, x8 = rn .
We seek to maximize the final altitude. Consequently, the criterion
function to be minimized is

j = -h, - -Xj, . (105)

596 ELEMENTS OF MODERN CONTROL THEORY

Equation (89) then supplies the additional boundary condition

\i(t/) = 1. (106)

We might just as easily have sought to maximize h,, in which case we

would have Xx(t,) = —1. The form above has been adopted in order to
permit ready comparison with the results of Example 1.
From Eqs. (78) or (81), we find

(107)
x3 dxi

X2 = — Xx + -~x2 (108)
x3 dx2

% V_ (0M - D) (109)
A 8 — - -- A2 .
X3

The hamiltonian is

H = XxX2+ X2 “ D) - gj - X,/3
(110)
=

[Xx (| +g)]+0 ~ ]•
x2~X2
This function is maximized if

when K > 0
(111)
0 = 0 when K < 0

where

TT X2M .
Iv — — Xg . (112)
x8

When K = 0, there is apparently no way to determine the optimal

control, since H is then independent of 0. This difficulty may be
resolved by employing the methods discussed in the section on “Singular
Solutions.” Using Eq. (95), the conditions corresponding to K = K = I =
0 lead to the three equations
OPTIMAL CONTROL 597

X2|u
— — x8 = o (113)
x8

/<3D D\
- x8\! + + - )x2= 0 (114)

Xix2—X2(— + gV - 0 . (115)

Combining the last two of these leads to

D + n(x2^ - x„ =0. (116)

Since x8 = —0, this relation yields the value for the optimal control in
the singular region. In other words, a variable thrust subarc (if it exists)
must satisfy Eq. (116). This result is identical to that obtained in
Example 1 by the variational calculus, i.e., Eq. (54).
The relation in Eq. (90) for determining t, is superfluous in the
present case, since physical reasoning indicates that the final subarc is a
coasting phase (/3 = 0), and the terminal condition is reached when
x2(t) = 0. However, Eq. (90) is satisfied for this value oft/ and provides
a check on the solution.
We should now like to investigate the possibility of a variable thrust
subarc in the absence of aerodynamics, i.e., D = 0. In this case, the
condition of Eq. (116) leads to mg = 0, an absurdity. Therefore, the
optimal control in a vacuum is of the bang-bang type.

Example 3. Minimum Time Control of a Nonlinear Process: Con¬

sider the second-order nonlinear system described by

Xi = xj, = fi (117)

x2 = —u2(u2 x2 T a Ui) = f2 . (118)

Here a is a positive constant, and the control functions, ui and u2,

satisfy the constraints

Ui| < 1 (119)

0 < 7 < U2 < 1 (120)

598 ELEMENTS OF MODERN CONTROL THEORY

7 = prescribed constant.

It is required to determine the form of ui(t) and u2(t) such that the
system is brought to the equilibrium position, xi = x2 = 0, in minimum
time from an arbitrary initial position

xi(tf) = a

x2(t<) = b .

In other words, the function to be minimized is of the form of Eq.

(75). Consequently, we define

x8 =

and

x8 = 1 = f8

x»(ti) = 0 .

The problem now reduces to that of minimizing the quantity

J = Xg(t/) .

The components of the costate vector, x, are given by Eq. (78), viz.

Xi = 0 (121)

X2 = — Xi + u22X2 (122)

X8 = 0 . (123)

The hamiltonian is, therefore, expressed as

H — Xj x2 X2 u2 (u2 x2 -(- «Ui) -p X8 . (124)

However, by Eq. (89)

x,(t,) = -1 .
OPTIMAL CONTROL 599

This relation, in conjunction with Eq. (123), shows that

Xa(t) = -1

which means that Eq. (124) reduces to

H = Xj x2 — X2u2 (u2 x2 + aUi) — 1 . (125)

/

It is immediately evident that H is maximized by*

ui = — sgn X2 . (126)

If we write Eq. (125) in the form

H=~1+XlX2
+X2[(S)
"X2
(Ua
+1~)] (127)

then we find that H is maximized if u2 satisfies the following.

1. For X2 < 0 (uj = 1)

a. If x2 > 0

then u2 = 1

b. If x2 < 0, then

u2 = 1 if |p| > 1

= P if 7 < |p| < 1

= 7- if |p| < 7 •

2. For X2 > 0 (ui = —1)

a. If x2 < 0

then u2 = 1

*sgn y = 1 if y > 0
= - 1 if y < 0.
600 ELEMENTS OF MODERN CONTROL THEORY

If x2 > 0, then

u2 = 1 if p > 1

= p <- IV
*4-» Q. IV r~H

= 7 ~o Al (128)

where

P 2x2 .

Conditions (126) and (128), together with the differential equations

(1 17), (1 18), (121), and (122) and boundary values

xx(t<) -- a xi(t/) - 0

x2(t.) - b x2(t/) = 0

with the added constraint

[Xi X2 — X2U2(u2X) + a Ui)] =1

t —tj

obtained from Eq. (91), are sufficient to determine the optimal mand u2.
For systems of moderately high order, the corresponding computa¬
tional tasks are not at all trivial, even with computer assistance. In the
present case, however, the phase plane representation may be utilized to
advantage in order to exhibit the salient features of the solution.
We observe parenthetically that the fundamental difficulty in starting
a solution is that the sign of X2is not known initially. However, the form
of Xi and X2is known, since Eqs. (121) and (122) are readily integrated;
viz.

X, =

X2
= eA{/32
—ft3le~A
dt (129)

where
OPTIMAL CONTROL 601

and /8i and /32 are constants of integration. The significant information
available from an inspection of Eq. (129) is that X2 cannot change sign
more than once.
Now for purposes of phase plane representation, we express Eqs. (117)
and (1 18) as

dxx = - - - . (130)
u2(u2 x2 a Ui)

From condition ( 126), we know that uj = 1, while Eq. ( 128) shows that

u2 = 1 whenever|x2| < “

regardless of the sign of X2. Consequently, in the phase plane, only two
trajectories can enter the origin. One trajectory, Li (see Fig. 5.4), is
obtained from the solution of Eq. (130) with uj = u2 = 1. The other
trajectory, Lj , is given by the solution of (130) with uj = -1 and u2 = 1.
We solve Eq. (130) by working backward; that is, we take xi(0) = x2(0) =0.
The resulting equations are

For L\: X2 < 0, x2 > 0

a
xi = — x2 — aln
x2 + a

ForL[ : X2 > 0, x2 < 0

Xl — x2 + aln
a — X2

We now observe, for instance, that a trajectory cannot leave Lx unless

X2 changes sign. Assume this happens at Point ©. Then ux switches sign
and becomes -1 , while u2 takes the value y, the trajectory continuing as
shown by the Path ® - © - ®. At Point ®, on line L2, u2 switches to

* a

satisfying this relation until the trajectory reaches Point ®, after which
= 1.
602 ELEMENTS OF MODERN CONTROL THEORY

Continuing the analysis along similar lines, we see that the phase
plane is divided into six semi-infinite regions as shown in Fig. 5.5. In
the half plane to the left of Li-Li , the control function, uj = -1 , while
in the other half plane, uj = 1. Since, as shown earlier, \2 can change
sign only once, a trajectory, once it hits L, — L[, will remain on it until
reaching the origin. Several typical trajectories from arbitrary initial
points are sketched in Fig. 5.5.
A complete solution for the case where

xi(t<) = -5.45, a = 1.2

x2(t,) = 0 7 = 0.5

is shown in Figs. 5.6 and 5.7.

OPTIMAL CONTROL 603

5.1.3 Gradient Method

The earliest studies in optimal flight control systems employed the

classical variational calculus, since this was the only tool available for
the problem at hand. While many important and significant results were
obtained, it became apparent that computational difficulties associated
with the solution of nonlinear two-point boundary value problems (gener¬
ated by the variational approach) severely limited the usefulness of the
method. Since many problems of interest were of sufficient complexity
to make the computational difficulties overwhelming, it was natural to
seek new ways of solving them. In this respect, one of the more
important concepts that emerged was the so-called “gradient method.”
The main virtue of this approach is that the numerical solution reduces to
604 ELEMENTS OF MODERN CONTROL THEORY

that of solving an initial value problem rather than a two-point boundary

type. This enhances the computational aspects no end.
The main ideas of the method evolved in stages rather than appearing
overnight in fully developed form. It appears that the fundamental con¬
cept is due to Courantd151) A systematic development specifically
oriented for aerospace applications is contained in the works of
Kelley(75>55) and Bryson. (75,106) Various refinements and related tech¬
niques were developed by H0/135) Knapp, (139) Dreyfus/1 52) and
Jazwinskid153)
As is Sects. 5.1.1 and 5.1.2, we will present only the main results
reflecting the current state of the art. Proceeding from first principles in
each area requires that a fairly elaborate groundwork be laid, which
entails a task beyond the scope of this chapter. It will be shown later
(Sect. 5.1.5) that the aforementioned results are quite readily derived
from a more general point of view within the framework of the theory
of dynamic programming.!16) The chief merit of this approach lies in its
conceptual plausibility and intuitive appeal. The Euler-Lagrange equations
and the maximum principle are found to be necessary consequences of
OPTIMAL CONTROL 605

the principle of optimality.O6) However, in order to discuss the gradient

method in this context, it is convenient to provide a motivational frame¬
work prior to presenting the main results.
The basic idea of the gradient method may be made plausible by
considering an analogous problem in ordinary calculus.

Steepest Descent in Ordinary Calculus. Consider a scalar function of n

variables

f = f(x) (131)
606 ELEMENTS OF MODERN CONTROL THEORY

where, as before, x is an n vector. It is required to determine the vector

x<0) such that the function f is stationary, i.e., is either a maximum or a
minimum.
A first change in f is given by

df = 22 — dx, = (V*f)rdx (132)

i—l dXj

where

df_
dxi

Vxf
(133)

df_
dXn _

which is the usual definition for the gradient of a function. It is known

that the gradient is the directional derivative in the direction of greatest
change. Therefore, for a prescribed increment, dx, the greatest change,
df , is produced if dx is taken in the gradient direction, viz.

dx = Kv»f (134)

where K is a constant that is chosen negative if a decrease in the value

of f is desired and positive if an increase is desired.
With this choice of dx, the change in f becomes*

df = K(V*f)2 . (135)

Note that since Eq. (132) is a linear approximation, dx must be

“sufficiently small” to ensure that the linearity restrictions are not
violated.
Suppose now that x(1) is a first guess for the minimizing vector. The
function, Eq. (131), takes the value f(x(1)). We now seek to apply a
fixed increment, dx, to x(1) such that the corresponding decrease inf(x(1))

*For any vector, a, a2 = ar a.

OPTIMAL CONTROL 607

is a maximum. As noted above, this is achieved by taking dx in the

negative gradient direction, i.e., taking as an improved estimate

x<*> = x«> - Kv*f

(136)
K = positive constant.

The value of K must be chosen such that

f(x<*>) < f(x«>) . (137)

If this relation is not satisfied, it implies that linearization was violated,

and a smaller value of K must be used. The process is continued until

|f(x<r+l>)
_ f(x(r))| < e
(137a)
e = small preassigned positive constant.

The vector x(r)is then the one that minimizes f(x) to the desired degree
of accuracy. If x(0; is the exact minimizing vector, then

Vxf(x(0>) = 0 . (138)

It should be emphasized that this approach (like the variational one)

ensures only a local extremum. In practice, global optimality is generally
verified by physical considerations.
The nature of the gradient method suggests the picturesque analog of
descending a hill in some efficient manner. At any given point, one
determines the steepest slope and then proceeds a short distance in that
direction.
This procedure, in the context of ordinary calculus, deals with incre¬
menting variables in order to optimize a function. In the variational
calculus, one seeks to optimize the function of a function, and here we
must vary a function rather than a variable. As a result, we are con¬
fronted with all sorts of disagreeable complications. Nevertheless, the
basic principle still applies and yields a powerful tool for optimization
problems in aerospace guidance and controls. It will be shown that the
gradient format for variational problems is conceptually identical with
that for the elementary calculus, except that Green’s functions replace
the partial derivatives and the vanishing of the gradient is intimately
related to the conditions expressed by the Euler-Lagrange equations.
These ideas will be developed next.
608 ELEMENTS OF MODERN CONTROL THEORY

Steepest Descent in Function Space. The essential features of the gradient

method, as applied to variational problems, may be exhibited by investi¬
gating the following. Given the integral

(139)

it is required to choose the vector, x(t), such that I is minimized (or

maximized). Note that there is an essential distinction between this and
the previous problem. Whereas x was an n-dimensional variable before,
x(t)is an n-dimensional function now.
Suppose now that and tf are fixed. By taking the variation* of I,
we find

(140)

Since the variation and differential operators are commutative^

(141)

Eq. (140) becomes

(142)

*See Appendix B.
fSee Appendix B.
OPTIMAL CONTROL 609

However

/ fd_ (ViL^fix dt = (ViL)rSx = 0 (143)

dt

since 5x(t,) = 5x(t<) = 0 because t, and t, are both specified. Therefore,

Eq. (142) reduces to

ftf d
51 /Jt. (5x)rVxL - - (V,L)
dt
(144)
%

Defining a generalized gradient by

[ ]=^-v-
x
(t45)
we may write

r‘f
51 = / (5x)»-[L],dt. (146)
J t.
%

The variation of I is thus expressed as an inner product in function

space,* compared with the differential of f, Eq. (132), which is expressed
as a conventional inner product of finite vectors. In addition, we note
the correspondence

df 51

dx <=> dx(t)

V*f L .

The operator [ ], defined by Eq. (145) may thus be identified as the

gradient of I in function space. Consequently, in analogy with Eq. (134),
we choose

5x = K[L], (147)

*See Appendix C.
610 ELEMENTS OF MODERN CONTROL THEORY

as the variation in x(t) that produces the greatest change in 51. Here, K is
a constant that is negative if it is desired to decrease I, and positive
otherwise. The gradient is evaluated along the nominal path x(1)(t). The
variation in I is expressed, therefore, as

(148)

in complete analogy with Eq. ( 135).

We recall that a necessary condition for f to be an extremum is that
the relation

V*f(x) = 0 (149)

be satisfied; i.e., that the gradient vanish at the local extremum.

Invoking a fundamental theorem of the calculus of variations^8) —
namely that 51 must vanish if I is stationary— an inspection of Eq. (146)
leads to the condition

(150)

since 5x is arbitrary between the end points, t< and tf. In other words, a
necessary condition for the extremum in function space is that the
generalized gradient vanish.
In order to relate this condition to the necessary conditions derived
via the conventional variational approach, we will solve the problem
using the methods of Sect. 5.1.1. To this end we define

(151)

and pose the equivalent problem of minimizing

J = Xn+i(t/) (152)

subject to the constraint

•P he Xn+1 - L(x, X, t) = 0 (153)

and initial condition

Xn+l(t<) = 0 . (154)
OPTIMAL CONTROL 611

Forming the augmented function*

F = A(xn+1 - L)

we find

dF dL
- = — X -
dXj dXj

dF _ _ a
dXj dx.

while

dF

dXn+l

dF
= 0 .
dxn+i

The Euler-Lagrange equations are then

d / dL \ dL
-j- ( X ) —X = 0
dt \ dxy/ dxj
j = 1, 2,

X = 0 .

The last relation shows that A is a constant, which means that the
preceding equations become

dL
d_
dt dx; (155)

J = T 2,

*See Eq. (13) et seq. Note that X as used here is a scalar.

612 ELEMENTS OF MODERN CONTROL THEORY

These are the Euler-Lagrange equations for the problem at hand. Written
as

(156)

we find by comparison with Eq. (150) that the vanishing of the gradient
in function space is another way of expressing the Euler-Lagrange equa¬
tions.
For purposes of numerical computation, the evaluation of the gradient
employing the operator defined by Eq. (145) involves the calculation of
higher order derivatives. This is never an advisable numerical procedure,
and other computational devices must be sought. It turns out that the
use of Green’s functions leads to an efficient algorithm for the numerical
computation of the gradient. We turn to a discussion of the basic ideas
involved in this approach.
Consider the system described by

Xy = fy(X, U, t)
(157)

where suitable boundary values are prescribed, and u is an m-dimensional

control vector. It is required to determine u(t) such that the function

J = J[x(ty)] (158)

is an extremal. As noted in the previous sections, a wide variety of per¬

formance functions can be expressed in this format. Suppose now that
one can select a control function, u(t), that satisfies the given boundary
conditions for Eq. (157), but that does not necessarily optimize Eq.
(158). We will refer to this control vector and the corresponding tra¬
jectory as the nominal u(t) and x(t), respectively. From this nominal
trajectory, we may calculate a value for J.
Suppose now that we adopt a new control, u(t) + 5u(t), which repre¬
sents a perturbed value of the nominal control. This will produce a
perturbed trajectory, x(t) + 5x(t). We seek to determine the form of the
perturbation that will produce the greatest improvement (increase or
OPTIMAL CONTROL 613

decrease, as the case may be) in the performance criterion, J. For 5u and
8x sufficiently small, we have, to first order

±j + 8Xj = fy(x, U, t) 4- 5fj(x, u, t)

(159)
(j = 1, %, n) •

By way of the Taylor expansion, (neglecting higher-order terms)

u,t) - ± 5x»+ ± 6u,

k=l dXk f=i

and noting that*

we have, from Eq. (159)

dfj
it (SXi> Z — <5
x* + J2 p- (160)
fc»=i
dXk £=\ oMf

(j = 1, 2, % n)
>

This may be written in vector matrix form as

4- (5x) = A(t)«x + B(t)«u (161)

dt

5x(tj) = 0 (162)

where

*See Appendix B.
614 ELEMENTS OF MODERN CONTROL THEORY

ah dh_ dii_
dxi dx2 dxn

af2 af.
dxx dx2
(163)
A(t) =

din di, dU
dxt dx2 dxn

afi dfi dii

dui du2 dum

af, df2
aux dU2
(164)
B(t) =

af, af«. af^

dUi du2 dum

All partial derivatives are evaluated along the nominal trajectory.

Equation (161) is a linear nonstationary system whose solution is
given by(155)

5x(t/) = f <t>(t;,
t) B(t)5u(t)dt (165)

where t*) is the state transition matrix for the system, Eq. (161). It
satisfies the relations
OPTIMAL CONTROL 615

5x(t) = t*) b; 5x(t0 = b (166)

-t: $(t, tO = A(t) <t>(t,tO (167)

^(tfc, tO = I, the unit matrix. (168)

Now /

5J = J[x(t/) + 5x(t/)] - J[x(t,)]

which reduces to

<5J = cr5x(t/) (169)

where

=0 =M (170)

Substituting Eq. (165) in Eq. (169), we have

5J= J (5u)r
Br(t)
f>r(t
/,t)c dt (171)

The evaluation of this expression is awkward, because the behavior of

5J depends on <t>(t/,t) as a function of t, its second argument, while
<t>(t/,t) is normally related to current and past state as a function of its
first argument; see Eq. (166). This predicament may be neatly resolved
by employing the theory of adjoint equations. Specifically, we write*

X = —A(t)X (172)

which is the vector matrix equation, adjoint to the system, Eq. (161). If
we take the initial conditions on this to be

X(t/) = c (173)

*The notation anticipates the interpretation of X as the vector Lagrange multiplier.

616 ELEMENTS OF MODERN CONTROL THEORY

where o is given by Eq. (170), then the solution of Eq. (172) is given by

\(t) = ^(t, t /) e (174)

where ddt, t*) is the state transition matrix for the system, Eq. (172), and
satisfies

l-¥(t,
dt
t*)= -A(t)¥(t,t*) (175)

'Kt*:, t*) = I, the unit matrix. (176)

Notice that Eq. (174) is integrated “backwards’’ from the given final
conditions. A(t) is, therefore, a known function (which, of course, de¬
pends implicitly on the reference trajectory).
Now it is known!155) from the theory of adjoint equations that

•Kt/, t) = ¥(t, t/) (177)

which means that Eq. (174) may be expressed as

X(t) = $r(t /, t) c . (178)

Substituting this in Eq. (171), we find

r*f
8J= J (8u)T
BT(t)X(t)dt . (179)
If we define

H = f7’A (180)

then

m
= V„H = BrX . (181)
du

Therefore, Eq. (179) becomes

5.J = (182)
OPTIMAL CONTROL 617

This now has the form of an inner product in function space, and, in
the light of the previous discussions, we interpret (dH/du) as the gradient
in function space with respect to the control vector, u(t). Since the latter
is arbitrary, the condition for the vanishing of 5,J(which thereby ensures
the J is an extremal) is that the gradient vanish; viz.

It will be shown later (Sect. 5.1.5) that this condition is precisely the
maximum principle discussed in the previous section. Thus, H, as defined
by Eq. (180), is indeed the hamiltonian of the system, and A is the vector
Lagrange multiplier. We note also that the quantities, dH/du^, are the
Green’s functions (sometimes called “influence functions”) for the sys¬
tem; these indicate how small perturbations, <5u,affect the performance
function, J.
It now follows that a maximum change in 6J is obtained when

Su = KV„H (184)

i.e., the change m 5u is along its gradient.

K is taken negative if a decrease in J is sought, and positive otherwise.
As a result*

5J = K (185)

in the same manner as before.

In order to limit the variation 5u, it is convenient to introduce the
constraint

(186)

where e is a prescribed constant. Therefore, in order to extremize 5J

subject to the constraint in Eq. (186), we introduce an additional
Lagrange multiplier, m, and write

*We recall that for some vector, b, the notation ba means brb.
 ELEMENTS OF MODERN CONTROL THEORY
618

5J =f' (Su)Tdt+IX
|\*- j' (6u)2
dtj (187)

= J (5u)r - - ]c
~ ldt+ ^ '
For an extremal 5J, the first variations must vanish; i.e.

S(«J) -I'M [@ - H * - °
which leads to

(188)

where the Lagrange multiplier, n, is evaluated from

The constant, /*, may be identified with constant K in Eq. (184).

In principle, the procedure is now straightforward. One selects a
nominal u(t) that satisfies the boundary conditions for the system and
that yields, therefore, a nominal trajectory, x(t), along with some value
for the performance criterion, J. In order to decrease (or increase, as the
case may be) this value of J, one uses a perturbed control function,
u(t) + Su(t), where 5u(t) is selected in accordance with Eqs. ( 184) or ( 188).
One then determines a new trajectory and a new value for J that is
improved (hopefully) over the preceding one. This procedure is continued
until the improvements in J are less than some predetermined amount,
or when 5J —» 0 .
In practice, a variety of complications preclude the formulation of a
simple cookbook procedure that will always work. Thus far, these
factors have been touched upon only lightly or not at all. Since a success¬
ful application of the gradient method depends upon a full understanding
of the various subtleties involved and of how to resolve them in particular
applications, we must consider these in a little greater detail.
OPTIMAL CONTROL 619

The Problem of Constraints. Having provided a degree of motivation

for the gradient technique, it remains to investigate the manner in which
various realistic constraints are accounted for in the general approach.
Three main types of constraint occur in physical applications.

1. Terminal:

*«[x(t/),t,J = 0 ' (190)

a = 1,2, - , p

0 (191)

/3 = p + 1, p + 2, - , q .

2. State Variable:

g*(x, u, t) < 0 (192)

k = 1, 2, - , r .

3. Control:

vt< H (193)

t =1,2, - ,m .

The final time, tf, is generally free (unspecified), and the initial condi¬
tions, xy(t») , may be specified or left open to optimization.
The simplest type of constraint, as far as the gradient method is con¬
cerned, is the inequality constraint, Eq. (193), on the control variables.
These are accounted for in the computational sequence, as follows.!55)
Assume that the nominal control vector unom(t) satisfies Eq. (193) for
all t.* The gradient technique then generates a new control

u”e“(t) = uoW(t) -I- 5u(t)

and it is observed whether at each instant of time, the relation

*Since the numerical procedure must of necessity deal with discrete instants of time, u"om(t)
satisfies Eq. (193) at all discrete time instants involved in the computation.
620 ELEMENTS OF MODERN CONTROL THEORY

vl < + <5u^(tjt)< m

{l = 1, 2, ••••, m)

is satisfied. If it is, we then use u"era(t) to generate a new trajectory in the

manner previously discussed. If not, we “trim” the control as follows.

V^t*) = h if

and

*r^k) = vt if u/e“(t.) < vt ,

To take account of the state variable constraint, Eq. (192), we define

new state variables

x„+*(t) G*(x, u, t) dt
(194)

k = 1 2 r

with

G*(x, u, t) = 0 if g* < 0 (195)

= g k~ if gfc > 0

k = 1,2, r

and add

= G*(x, U, t)
(196)
(k = 1, 2, r)

to the other state equations, together with the initial conditions

Xn+fc(t,) 0
(197)
k = 1, 2, - , r .
OPTIMAL CONTROL 621

We note that the derivatives of G* are everywhere continuous if the

derivatives of gk are continuous/153)
As a result of this operation, the state variable inequality constraint,
Eq. (192), is equivalent to

x„+*(t /) — 0
(198)

which is in the form of the terminal equality constraint, Eq. (190).

Consequently, we will dismiss state variable inequality constraints from
further consideration, assuming that they have been transformed to the
equivalent constraint, Eq. (190).
The problems that now remain are the treatment of the terminal
constraints, Eqs. (190) and (191), the selection of appropriate step sizes
and proportionality constants, and the finding of a nominal control
vector that yields a trajectory satisfying prescribed terminal and initial
values. These will be discussed within the context of the general solution,
which is considered next.

The General Solution. Given is the system

x = f(x, u, t) (199)

x = n-dimensional state vector

u = m-dimensional control vector

with the initial conditions

x,(t.) = a,
(200)
j = 1, 2, ••••, s(< n)

where the (n — s) initial conditions, which are not specified, are left open
to optimization. At the final time, t/; the constraints

<AJx(t /), t/] = 0

(201)
a = 1,2, - , p
622 ELEMENTS OF MODERN CONTROL THEORY

tplxitf), t,| < 0 (202)

= p+1, P + 2,

must be satisfied. In general, tf is unspecified.

The components of the control vector must satisfy the inequality
constraints

v{< < M/ (203)

t = 1, 2, - , m .

It is assumed that constraints on the state variable, as given by Eq.

(192), have been transformed to type (190) by the introduction of ad¬
ditional state variables in the manner indicated previously.
We now seek to determine the control vector, u(t), that minimizes
(maximizes) the performance index

J = JfxCt/), t/] (204)

such that the restraints, Eqs. (201)-(203), are satisfied, given the initial
conditions, Eq. (200).
The problem thus formulated is of wide generality. Many realistic
optimization problems in aerospace guidance and control can be ex¬
pressed in this format.
In presenting the solution to the problem, we will limit the discussion
to the main results. The complete derivations are quite lengthy, requiring
many digressions that are beyond the scope of the present discussion.
For a detailed treatment, the reader should consult the appropriate
references.^, 42,55, 75, 76,103,135, 153)
Suppose now that we arbitrarily select a nominal u(t) that satisfies the
constraint in Eq. (203) and that thereby yields a nominal trajectory, x(t),
satisfying the initial conditions of Eq. (200) but not necessarily all the
terminal constraints, Eqs. (201) and (202).
The problem is now taken as one of minimizing the function*
Q

P = K0J + £ K7Vfr72 (205)

The values of J and <py are, of course, obtained from the nominal trajectory.
OPTIMAL CONTROL 623

where

K0 = —1 if J is to be maximized

= + 1 if J is to be minimized

K7 = 0 if the nominal trajectory satisfies the

terminal constraint, given by Eqs.
(201) or (202f.

The basic idea here is that if the I\7 are large (compared with K0), any
control, u(t), that seriously violates terminal conditions of Eqs. (201) and
(202) will give rise to large values of K7^72, with the result that the next
iteration on u(t) will be biased in favor of satisfying the terminal con¬
straints rather than optimizing J. Consequently, the more closely the
terminal constraints are satisfied, the smaller the value of K7^72 (what¬
ever the size of K7), with the result that subsequent iterations on u(t)
become biased in favor of optimizing J. This notion, which is intuitively
plausible, is also mathematically legitimate. 056, 157)
Now since the final time, tf, is free, one of the \py = 0 may be singled
out as a stopping condition and is, therefore, not included in the sum¬
mation term of Eq. ( 205). Call this stopping condition

S2[x(t /) , t/] = 0 . (206)

At each pass, the integration of the trajectory equations is stopped when

ft = 0.
Defining the hamiltonian as before

H = f*\ (207)

where X satisfies

X = —A(t)X (208)

with A(t) given by Eq. (163), we take, for the initial conditions on X

Xy(t,) = E(t/)(M\ (209)

0(t,) \axy/(_t
Then, if the variation in the control vector is constrained by
624 ELEMENTS OF MODERN CONTROL THEORY

w^(t)[<5u^(t)]2dt (210)

e = prescribed constant

W/(t) > 0, a weighting factor

we take, as a new control

Unew(t) = UoW(t) + 5u(t) (211)

where u oW(t) represents the original nominal control and

5u^(t) =
_l _ dH
2/uw^(t) duf (212)

1 = 1, 2, • • • •, m

where m is an additional Lagrange multiplier calculated from

_L_ (213)
Vw^(t)

The weighting functions, wy(t), are introduced to permit a greater

flexibility in the numerical convergence procedure, especially in regions
of high sensitivity. For initial trials, one may take w^(t) = 1 for all L
Now by defining

(214)

we may write Eq. (213) as

(215)
V

Using this to eliminate p from Eq. (212), the latter becomes

e dH
5u^ = (216)
du(
OPTIMAL CONTROL 625

It can be shown that the change in P is then given byd58)

dP = eV (217)

which means that e must be chosen negative, since a minimum of P is

sought.
The (n — s) initial conditions, which are unspecified, are now modified
according to

(218)

j = s + 1, s + 2, - , n .

It is noted that the \y( t<) are the Green’s functions for the initial con¬
ditions.
The general computation procedure is now summarized as follows.

1. Select a nominal control vector, u(t), and integrate the equations

of motion, Eq. (199), using the initial conditions, Eq. (200). If
only s of the n initial conditions is given, choose arbitrary values
for the (n — s) unspecified initial conditions. These will then be
optimized in the course of the computational procedure. The
integration is terminated when the condition in Eq. (206) is
satisfied. This yields the nominal trajectory and the nominal
terminal time, t,. In general, not all the terminal constraints,
Eqs. (201) and (202), will be satisfied.

2. Calculate P from Eq. (205).

3. Integrate the adjoint Eqs. (208) “backward,” using the terminal

conditions given by Eq. (209). Note that the matrix, A(t), is
defined by Eq. (163).

4. Calculate the hamiltonian, Eq. (207), and the Green’s functions,

dH/dxif.

5. Choosing some appropriate values for the weighting functions,

w^(t), evaluate v2 by Eq. (214).

6. Using an appropriate step size, e (which must be a negative

number), calculate 8uf, with ^=1,2, - , m. The resulting
626 ELEMENTS OF MODERN CONTROL THEORY

u = uoU + 5u must be “trimmed” to satisfy the inequality

constraints, Eq. (203).

7. Using u»«w(t), obtain a new trajectory in the manner of Step 1.

The unspecified initial conditions are modified in accordance
with Eq. (218).

8. Calculate P from Eq. (205), and check whether this is less than
the values of P from the previous iteration. If not, reduce the
size of e and return to Step 6.

9. Stop the iteration when some suitable accuracy criterion is

satisfied. This may take the form

rj2< <j
I'Pa]< Pa
a = 1, 2, ' • • •, P

i0< o

0 = 1, 2,

where a and pa are prescribed constants.

Remark: Various arbitrary constants must be chosen in the computa¬

tional procedure described above. The values for these constants
will generally depend on the particular problem under investi¬
gation, and no one particular set of values is best in all cases.
Nevertheless, certain guidelines are generally valid. First, as
far as the K7 of Eq. (205) are concerned, a large value will
yield a new trajectory that will tend to satisfy the terminal
constraints to a greater extent than it optimizes the criterion
function, J. In short, the early iterations will seek to yield an
admissible trajectory that satisfies all the terminal constraints
before optimizing J. The value of K7 may be periodically
reduced in the course of the computation. This process indi¬
cates a willingness to accept mild excursions from the terminal
constraints in favor of optimizing J, which is the primary goal.
The proper step size e is difficult to determine initially,
since it reflects the permissible linearity range. Too low a
OPTIMAL CONTROL 627

value means long computing time, while too large a value

violates linearity and yields inaccurate results. A few prelimi¬
nary trials should yield an acceptable range of values.
The weighting functions, w^(t), are introduced mainly to
afford a greater degree of flexibility in the convergence pro¬
cedure. Normally one may take w^(t) = 1 for all i and t. It
may happen that, near the optimum, large changes in P result
from even small step sizes, e. In this region, the w/(t) may be
manipulated so that the <5u^may be varied with time. This
sometimes produces a smoother convergence to the optimum.

Example 4: We will now investigate the problem of maximizing the

payload for a single-stage boost vehicle that is required to attain a pre¬
scribed altitude, velocity, and flight path angle. It is assumed that: the
vehicle trajectory is contained in a plane passing through the center of
the earth; the earth is spherical; and the gravity forces obey the inverse
square law. In this case, the motion is described by (see Fig. 5.8)

mV = T cos a — D — mg sin 7 (219)

mV'v = T sin a + L — mg cos 7

Figure 5.8. Coordinate System for Example 4.

628 ELEMENTS OF MODERN CONTROL THEORY

— 2mVwjj cos <Pb (220)

To + h

h = V sin y (221)

R = V cos 7 (222)
r0 + h /

lh = —(8 (223)

with

T = V£/3 (224)

g = go (225)
ro + h /

The symbols have the following meaning.

c = speed of sound; a known function of altitude

D == drag; a known function of Mach number and angle of attack
g = gravity acceleration; a known function of altitude
g0 = gravity acceleration of earth’s surface
h = altitude
L = lift; a known function of Mach number and angle of attack
m = instantaneous mass of vehicle
M = Mach number = V/c
R = range measured along earth’s surface
r0 = radius of earth
T = thrust
V = velocity
Ve = velocity of exhaust gases in rocket engine; a known constant
a = angle of attack
8 as thrust variation parameter
y == flight path angle
<pE= angle of earth’s polar axis with perpendicular to plane of motion
angular velocity of earth about polar axis

The control variables are /3 and a, i.e., the thrust magnitude and
orientation.*

*Short period dynamics are neglected; therefore the thrust angle and angle of attack are equivalent.
OPTIMAL CONTROL 629

These are required to satisfy the inequality constraints

ai < a < £*2 (226)

0 < 8 < 8m . (227)

It is also required to limit the axial and normal loads as follows.

(T + L sin a — D cos a) /mg < LA (228)

(L cos a + D sin a)/mg < Ljv (229)

where Lx and L^are prescribed constants.

The following boundary conditions are specified.

V(t.) = 0
7(t<) = tt/2 V(t,) = Vr
R(t<) = 0 7(t/) = 7t (230)
h(t,) = 0 h(t/) = hr
m(t<) = m7
t< = 0 t / = free

We may, therefore, take, as a stopping condition

Q, = h(t/) — hr = 0 (231)

and the remaining terminal constraints become

h * V(t,) - Vr = 0 (232)

^2 = 7(t/) - 7r = 0 . (233)

The inequality constraints, Eqs. (228) and (229), involve state variables.
We, therefore, define

x6 = Gj dt (234)

*7 = G2 dt (235)
630 ELEMENTS OF MODERN CONTROL THEORY

where

Gi = 0 if gi < 0
(236)
= gi2 if gi > 0

G2 = 0 if g2 < 0
(237)
= g22 if g2 > 0

and

T + L sin a — D cos a T
gi - - la (238)
mg

L cos a + D sin a T
g2 GiV • (239)
mg

We add to the state equations

x6 = Gi (240)

x7 = G2 (241)

with initial conditions

x6(t<) = 0 (242)

x7(tf) = 0 . (243)

In this fashion, the state variable inequality constraints, Eqs. (228) and
(229), are transformed to the terminal constraints.

= xe(t/) = 0 (244)

- x7(t,) = 0 (245)

x2 = 7 u2 = /3
OPTIMAL CONTROL 631

x8 = k

x4 = R

x6 = m

then the problem may be expressed in the standard format as follows.

Given the system
2

V, u2 cos Ui — D (xi, x8, ui) / r0 \ . f

X! = - -- — — 1— - - go ( - ) sin x2 s= h (246)
x6 \r0 + x8/
2

... _ V, u2 sin ui + L (xi, x8, ux)

aj — %
g0 (
I ~
r„ *
\ .
I COS X2
Xi x6 Xi \r0 + x8/

+
Xi cos x2 (247)
2 cos <pe = t2
r0 + x8

x8 = Xi sin x2 = f8 (248)

(249)

(250)

(251)

(252)

where

Gi = 0 if gi < 0

= gi2 if gi > 0

G2 == 0 if g2 < 0

= g22 if g2 > o

V« u2 + L(xi, x8, Ui) sin ut D(xj, x8, Ui)

gi = (r0 + x8)2 - La
X6 go r02
632 ELEMENTS OF MODERN CONTROL THEORY

L(xi, x8, Ui) cos Ui + D(xi, x8, Ui) sin Ui

g2 = (r0 + x8)2 -
x6 go To2

with the terminal constraints

1Pi = xi(t/) - Vr = 0 (253)

\p2 — x2(t/) yT — 0 (254)

lAs --- x6(t/) = 0 (255)

^4 = X7(t/) = 0 (256)

stopping condition
= Xg(t/) — hT = 0 (257)

and initial conditions

xi(t.) = 0 xs(ti) = mi
x2(t<) = tt/2 XjCtt) = 0 (258)
x3(ti) = 0 xj(ti) = 0.
x4(ti) = 0

It is required to determine ui(t) and u2(t) such that

J = X„(t/) (259)

is a maximum, subject to the control variable constraints

a.\ < Ui < a2 (260)

0 < U2 < Pm • (261)

The computational procedure now continues in the manner outlined

previously.

5.1.4 Dynamic Programming

The theory of dynamic programmingO6) is one of those rarities— a

genuinely original contribution to a long outstanding problem. As is
OPTIMAL CONTROL 633

often the case with the appearance of a new idea, a flood of papers dealing
with extensions, ramifications, and applications has resulted. The theory
has helped influence the development of such diverse areas as manage¬
ment science, information theory, sequential analysis, optimum filtering,
adaptive control, learning theory, optimal trajectories, and variational
calculus. The passage of time, however, has had a sobering effect on
some of the overly optimistic claims made in the early stages of the
theory’s development. Space limitations preclude discussion of all facets
of the theory save those immediately relevant. We shall content ourselves
with the barest outline of the main ideas and discuss a variety of appli¬
cations that hopefully will clarify the method and exhibit its utility. In
the next section we will show how the basic premise of dynamic program¬
ming includes, as a special case, each of the optimization techniques thus
far considered.
Of fundamental concern in what follows will be the concept of a
multistage decision process. At any one stage in this process, one may
make a decision (select a control function), following which the next
stage is reached. Successive stages are related by a known transformation.
Each stage is characterized by a state vector, and each decision results in
some cost. These ideas may be made explicit as follows.
Let the state of a dynamic process be characterized by a state vector,
x. Suppose that successive states are related by

x<* » = T(x<«, u<«) (262)

where u0) is a decision (control) vector, and T is a known transformation.

Equation (262) is a statement of the fact that, when in stage (state) j,
a control uC;) is chosen, and then the new stage x(j+I) is reached, depending
on the explicit form of T(x<3), u0)). Assume there are N stages in the
process, and a performance index (cost; is given by

J = Z g(x<» u<») . (263)

j=i

We pose the problem of choosing the control sequence, u(1), u(2), • • • •,

uw, such that the function J is a minimum (maximum). For ease of
discussion, let us call any permissible control sequence, u(1>, u(2),
uw, a policy. If the initial state of the process is characterized by the
state vector, c, the value of J is obviously a function of c and the policy
(for a given value of N). Let us now define
634 ELEMENTS OF MODERN CONTROL THEORY

Wtf(c) = the value of J when the process starts in state c, having N

stages to go, and using an optimal policy.

An optimal policy, of course, is one that optimizes the cost function,

J. The fundamental premise of dynamic programming is embodied in
the following.

The Principle of Optimality .-d 6) An optimal policy has the

property that, whatever the initial state and the initial decision,
the remaining decisions must constitute an optimal policy with
regard to the state resulting from the first decision.

The principle of optimality is used to derive, for the Wy(c), recurrence

relations that yield both the optimal policy and the value of the optimal-
cost function. The reasoning proceeds as follows. We are in the initial
state, c, and have N stages to go. We pick an initial value of the control
vector, u<!>. As a result, we reach a new state

x«> = T(c, u«>) (264)

and incur a cost, g(c, u(1)), with (N — 1) stages left in the process. If we
proceed in an optimal fashion from this new state, the cost incurred is
WAr_x[T(c,u(1))], by definition. By way of the principle of optimality, we
see that for some specific value of u(1) we incur a cost

g(c, u«>) + W^flXc, u<i>)] .

But we wish to choose u(1) so that it is maximized.* Therefore

Wat(c) = Max g(c, u«>) + W^v-xflXc, u«>)] (265)

u<n

Consider now the function, Wx(c), i.e., the optimal value of J with
only one stage to go. It is easy to see that

Wi(c) = Max g(c, u(1)) .

(266)
ua>

*For definiteness, we consider the problem of maximizing J. The argument, of course, holds
equally well for minimization.
OPTIMAL CONTROL 635

For purposes of computation, one proceeds as follows. Assume that

u(1) may take any one of a prescribed finite set of values. Then calculate
Wi(c), using Eq. (266), for a range of values of c. This yields a table of
YVi(c), each with its corresponding optimal which is then utilized to
calculate W2(c) via Eq. (265) for a range of values of c, along with its
corresponding optimal u«>, etc. Proceeding in this fashion, we find
Wi(c), W2(c),----, W*(c), together with the optimal policy for each.
Note that this is a flooding technique in effect, we solve the specific
problem by imbedding it in a class of related problems. This is a highly
efficient technique but has one major limitation. It is what Bellman
refers to as the “curse of dimensionality.” Suppose that the vector,
x (or c), has n components, and assume that it is required to evaluate
100 different values for each component. Then to store Wy(c) in the
computer would require 100'“ storage cells. Since modern-day computers
have on the order of 30,000 storage cells, we see that N = 3 is a marginal
figure. Various means of overcoming this limitation are discussed in
Refs. 9, 10, 168, and 170. We will digress for a moment to consider an
elementary application.

Example 5: Given the system

X = X2 + u (267)

where x and u are both scalars, determine u(t) such that the function

(268)

is a minimum; the final time, tf, is given. The control function u(t) must
satisfy the inequality constraint

-2 < u < 2 . (269)

This simple, rather contrived problem illustrates the computational

procedure using dynamic programming and also exhibits both the power
and limitations of the method. The problem is, first of all, nonlinear,
having a cost function that is not everywhere continuous, and must also
satisfy inequality constraints on the control variable. These features
introduce distressing complications in the classical approach. The dy¬
namic programming approach, however, is completely indifferent to
analytic aberrations.
 ELEMENTS OF MODERN CONTROL THEORY
636

To apply dynamic programming, the problem must first be cast in the

form of a multistage decision process, which means that we must use the
discrete versions of Eqs. (267) and (268), viz.

dx = (x2 + u) dt (270)

J = 2 lx* —u,fc3|dt (271)

k=l

t / = N dt . (272)

This representation is valid as long as the increment, dt, is sufficiently

small. Successive states are then related by

xU+1) = x«> + [(x<«)* + ua>] A (273)

A = dt

which is the transformation law, Eq. (262), for this problem.

Now define

W at(c) = the value of J, Eq. (271), obtained by starting in state c,

having N stages to go, and using an optimal policy.

A direct application of the Principle of Optimality yields the recurrence

relation

W*(c) Min - u3|A + W tf—i[c + (c2 + u)A] (274)

u

This simply means that in state c, with N stages to go, if we choose a

particular (admissible) u, then we incur a cost |c — u3|Afor this first stage
and end in state [c + (c2 + u)A], Assuming that we proceed optimally
thereafter, the remaining cost is, by definition, W^_i [c + (c2 + u)A] for
the remaining (N — 1) stages. However, we choose not an arbitrary u for
the first stage, but the one that minimizes this total cost. Hence the form
of Eq. (274).
It is easy to see that when there is only one stage to go, we have

Wi(c) = Min|c —u3|a (275)

u
OPTIMAL CONTROL 637

and this represents the “'initial condition for the recurrence relation in
Eq. (274).
We now examine the computational sequence in some detail. To do
this, we must decide on the range of interest for the state variables and
the number of values of u in the range -- 2 < u < 2. For present pur-
poses, we will consider a painfully simplified version wherein
/

c = -5, -4, -3, -2, - 1, 0, 1, 2, 3, 4, 5

u = -2, -1, 0, 1, 2: A = 0.1 .

Then via Eq. (275), we calculate the following table.

c Wi(c) u

-5 0.3 -2
-4 0.3 -1
-3 0.2 -1
-2 0.1 -1
-1 0 -1
0 0 0
1 0 1
2 0.1 1
3 0.2 1
4 0.3 1
5 0.3 2

Normally, in realistic problems, this operation is performed on a com¬

puter, and the table is printed out. Now we calculate W2(c) from

W2(c) = Min{|c — u3|A + Wi [c + (c2 + u)A]}

u

for a range of values of c. For example, with c = 3, we calculate the

bracketed quantity for each of the five permissible values of u; we then
select the one that is a minimum as the value of W2(3). This yields, for
example

W2(3) = 0.50; u = 1 .

Proceeding iteratively, we obtain, ultimately, the tabular form

638 ELEMENTS OF MODERN CONTROL THEORY

C Wjv(c) U

which represents, along with the tabular forms forWy(c), j = 1, 2, - ,

N — 1 the solution to the problem. Note that we have the solution, not
only for one specific initial condition, c, but a whole range of possible
initial conditions. This is characteristic of the dynamic programming
method.
We have thus far considered only “running costs” of the type ex¬
emplified by Eq. (263). It may be desired to minimize (or maximize) a
function of the final state

J = g[x(t/)] . (276)
In this case, we define

Wjvr(c) = the value of g[x(t/)] starting in state c, having N stages to go,

and using an optimal policy.

A simple argument shows that the recurrence relations are

W„(c)= MinW*_![T(c,
u)] (2?7)

W0(c) = g(c) . (278)

In other words, with zero stages to go, the optimal W0(c) is merely the
value g[x(t/)], where c replaces x(t/).
For the functional equations thus far considered, it was possible to
write explicit initial conditions; that is, the number of stages in the
process was known beforehand. This is not always the case. To examine
this situation, consider the problem of driving the solution of

±1 = xs

x2 = h(xi, x2, ui)

xi(0) = Ci

x2(0) = c2
OPTIMAL CONTROL 639

to the equilibrium state, xi = x2 = 0. in minimum time, where the con¬

trol, ui, is constrained by

a < ui < b .

We consider the discrete version

xi(t + A) = xi(t) + xottiA

xs(t + A) = x2(t) + h[xi(t), x2(t), Ui(t)]A

A = dt

and instead of requiring that xi and x2 be simultaneously zero, it is suffi¬

cient to require that (xi2 + x22) < e, where t is some small positiye number.
If we define

W(ci, c2) = the time required to reach the equilibrium region (xi2 T
x22) < e, starting in state (ci, c2), and using an optimal
policy

then a simple application of the principle of optimality yields the func¬

tional equation

W(Cl, c2) Mini A + W[ci + c2A,c2 -f h(ci, c2, Ui)A] (279)

Ui l

The computational procedure for this equation is not immediately

apparent, since we are unable to set up “initial” conditions. This
equation is of the implicit type; that is, the unknown function appears
on both sides of the equation. It is not a recurrence relation of the
type previously considered.
We may, nevertheless, obtain a solution via the concept of approxi¬
mation in policy space, a powerful technique first discovered by Bell¬
man/1 6) It is crudely analogous to and predates the gradient technique
discussed in Sect. 5.1.3. One proceeds as follows. Select a policy, u(0)(t),
that brings the system to the equilibrium state, but that, in general, does
not result in minimum time. Let this time duration be denoted by
W(0)(ci, c2), which is a first approximation to the optimal return function,
W(ci, c2). Do this for a range of values of cj and c2. Calculate an improved
640 ELEMENTS OF MODERN CONTROL THEORY

return function, W(1)(ci, c2), via

W(1)(ci, c2) = MinjA + W(0)[ci + c2A, c2 + h(ci, c2, Ui)A]

ui '%

also for a range of values of cx and c2, storing the improved policy ui(1).
It can be shownO6) that

W(,)(ci, c2) < W<«>(oi, c2)

and the iteration stops when

|W(,)(cj,c2) - W(#-1)(Cl,
c2)|< v

i) = small positive constant.

The number and type of optimization problems that can be solved by

the simple methods outlined above is truly astonishing. References 16,
17, and 159 contain numerous examples, together with an extensive
bibliography. We now consider some simple, though fairly realistic,
aerospace applications.

Example 6: We return to the sounding rocket problem already

analyzed via the variational calculus (Example 1) and the maximum
principle (Example 2). In Example 1 it was found that the optimal
thrust control program may contain an ambiguity* for certain forms
of the drag function. (See Fig. 5.3.) This ambiguity is completely
resolved in the dynamic programming solution.

We now proceed to express the problem in the format of dynamic

programming, using the notation defined in Example 1. The state
transition equations are written in discrete form, as follows.

(280)

dh = VA (281)

A = dt

*This ambiguity also exists via the maximum principle.

OPTIMAL CONTROL 641

We have to take account of the fact that the control function satisfies
the relation

0 = —m (282)

and the constraint

0< 0< 0U (283)

where 0M is given. Let us suppose that admissible values of 0 may be

taken from the q + 1 distinct values

0 = 00, 01, • • • *, 0q (284)

where

00 = 0

0g = 0m, the permissible limit

and that

0i+ 1 — 0j = y, a prescribed positive constant.

Now a particular 0k corresponds to a particular mass increment (dm)fc,

determined from

_ (dm)*
Pk — - (285)
A

We now select A, y, and q such that

0k = ~k (286)

where k is a positive integer (or zero) that is interpreted as a number of

mass (i.e., fuel) increments. If, at time zero, the total mass is

m(0) = m0 + mF(0)!

where
642 ELEMENTS OF MODERN CONTROL THEORY

m0 = fixed mass (structure and payload)

mf = fuel mass

then we find the number of “stages” in the process as

m,(0) = N .

In other words, the size of the mass incrementshaving been established,

a particular fuel mass can be expressed as a specified number of these
mass increments. In particular, we denote the number of mass incre¬
ments available at the start of the process by N.
The problem is to determine a thrust control policy that maximizes
the final altitude, given a prescribed fuel mass. Define

Wjy(V, h) = the altitude attained starting with velocity V, at altitude

h, having available N fuel increments, and using an
optimal policy.

The functional equation is

/ (3ic[i — D
WN(V, h) = MaxlVA+ War-k V
Hk 1
+
\m0 + N ;)a,h+vaJJ. (287)

The reasoning is as follows. Starting in state (V, h), we acquire an

altitude increment, VA, and reach a new state

y + ~ D
\m0 + N

h + VA .

Proceeding optimally thereafter, the altitude acquired is

\ g <1
i

W N—k -d + > o
_ \m0 -j- N /

by definition. Note that there now remain (N — k) stages, since the

choice /3* means that k fuel increments were expended. But we may
choose (3k to maximize this return. Hence, the form of Eq. (287). The
OPTIMAL CONTROL 643

initial condition, W0(V, h), is easily obtained since, with zero stages to
go, the altitude acquired is simply the solution of the system

V =
D(V, h)
- g(h)
m0

h = V

where the initial conditions on V and h are merely the arguments of

W0(V, h), and the integration terminates when V = 0. Denoting the
altitude acquired during this coast by hr, we see that

W0(V, h) = hr(V, h)

the notation hr(V, h) emphasizing that the altitude acquired during

coast is a function of the starting values of V and h. If the altitude is
sufficiently high such that the drag is negligible, then

V2
w„(V, h) = -L .
2g

The computational procedure is now straightforward. Some care

must be exercised, however, in the choice of increments. One must
choose A = dt small enough, for example, such that Eqs. (280) and
(281) represent good approximations to the continuous case. If we
consider the numerical values

m(0) = 1000 slugs

m^O) = 900 slugs

n = 2000 ft/sec

= 25 slugs/sec

then we may take

A = dt = 0.1 sec

0 = 0, 0.1, 0.2, ••••,25

 ELEMENTS OF MODERN CONTROL THEORY
644

mass increment = 0. 1 slug

N = 9000 .

The solution time for this problem would be on the order of a few
minutes on a computer of the 7090 type.

Example 7: One of the earliest studies dealing with the application

of dynamic programming to trajectory optimization is the following. O60)

Let the motion of an aircraft in the vertical plane be described by

(see Fig. 5.9)

mV = T — DCV, h) — mg sin 6 (288)

h = V sin 6 (289)

where

V = velocity
D = drag
T = thrust (constant)
m = mass (constant)

' r
mg

Figure 5.9. Coordinate System for Example 7.

OPTIMAL CONTROL 645

g = gravity acceleration
h - altitude
d = angle of thrust vector with respect to horizontal reference.

We pose the problem of programming the thrust vector angle, 9, such

that the aircraft attains a prescribed velocity and altitude, starting from
some given initial velocity and altitude, in minimum time.
Taking the discrete version of Eqs. (28$) and (289)

dV =
T - D(V, h)
— g sin 6 A (290)
m

dh = VA sin 6 (291)

A = dt

and defining

W(V, h) = the time required to reach the prescribed terminal state

(V„ hr) starting in state (V, h) and using an optimal
policy

we obtain, via the Principle of Optimality

T - D
W(Y, h) = A + Min V +
6
( m
?)a,
— g sin 9 JA, h + VA sin 9 . (292)

This is an implicit type of functional equation that must be solved

iteratively by the method of approximation in policy space.
Viewing the problem in a different light, Cartaino and DreyfusO60)
obtained a recurrence relation for W(V, h) for which initial conditions
could be prescribed, thereby circumventing the need to solve an equation
of the explicit type. Their reasoning is based on the observation that
the given problem is equivalent to that of determining the minimum
time path in the h — V plane. From Eqs. (290) and (291), we see that

A =
K £> (293)

ex* - °)
w = mg .
646 ELEMENTS OF MODERN CONTROL THEORY

Imagine now that the h — V plane is subdivided into grids with incre¬
ments dh and dV. We suppose that the mass point representing the
airplane moves in discrete steps from one node to the next, either straight
up (V = constant) or horizontally to the right (h = constant). At each
node point, a decision must be made whether to proceed vertically up
or horizontally to the right. The criterion, of course, is that the grid
from the current (initial) V and h to the prescribed VT and hrbe tra¬
versed in minimum time. The “costs” associated with the respective
alternatives are
_ wdh for constant V
~ V(T - D)
(294)
wdV
- tor constant h
g(T - D)
via Eq. (293).
Let us note at the outset that the insensitive pragmatist might be
tempted to enumerate all possible paths and then select the minimum
time path by direct comparison. In the case of a p by q grid, the total
number of possible paths under the conditions stipulated is
(p + q) !
(p - 1) ! (q + 1) ! '

For a grid with p = q = 100, the number of paths to be enumerated is

on the order of 1069. Since this closely approaches the estimated number
of atoms in the galaxy, one is compelled to seek another approach.
As a matter of fact, via the Principle of Optimality, with W(V, h)
defined as before, we find

wdh
A. + W[V, h + dh]
V(T - D)

W(V, h) = Min (295)

wdV
B. + W[V + dV, h]
g(T - D)

This is arrived at as follows. If we are at the node in the grid whose

coordinates are V and h, we have the option of moving vertically upward
(constant V), in which case we incur the cost
wdh

V(T - D)
 OPTIMAL CONTROL 647

(i.e., this is the time consumed in going from h to h + dh) and arrive at
the new grid point, (V, h + dh). Proceeding optimally thereafter, the
cost is W(V, h + dh). This is alternative A. Alternative B is arrived at
by similar reasoning. Finally W(V, h) is taken as the smaller of the two
alternatives A and B. Since the terminal values of velocity and altitude,
Vr and hr, are specified, we see that
x wdV
W(Vr - dV, hr) (296)
g[T - D(V r, hr)]

wdh
W(Vr, hr - dh) (297)
V r[T - D(V r, h r)J '

These represent the initial conditions for the recurrence relation in

Eq. (295).
It is easy to show that the number of paths to be enumerated in this
approach is on the order of pq, compared with the astronomical number
of paths to be enumerated by the brute force approach.
Figures 5.10 and 5.11, abstracted from Ref. 160, show some typical
results. From Fig. 5.10, Eq. (289), and the relation

x = V cos 6

x = horizontal distance

we may plot the optimal path in the h — x plane, which is depicted in

Fig. 5.1 1. The constant Mach number and constant altitude subarcs for
the optimal path are well delineated in Fig. 5.10. The unique feature of
the present approach is that the complexity of the drag function presents
no difficulty whatever, whereas in the classical approach, one may en¬
counter all sorts of analytical obstacles.

5.1.5 Unifying Principles

In the methods presented thus far, only the main results have been
stated, together with various examples illustrating their application. The
development of detailed derivations in each case would require a docu¬
ment the size of a textbook rather than a chapter. Nevertheless, the
omission of derivations constitutes a pedagogic deficiency that precludes
complete comprehension. Furthermore, a superficial examination would
seem to indicate that the four optimization techniques considered thus far
are basically unrelated. It may, in fact, be shown that not only are these
648 ELEMENTS OF MODERN CONTROL THEORY

Figure 5.11. Altitude/Horizontal Distance Profile of Minimum Time Path.

OPTIMAL CONTROL 649

techniques intimately related, but that they all fall out as special cases
from a more general point of view—namely, they are all necessary conse¬
quences of the Principle of Optimality. We proceed to demonstrate this
in the following way.
Let the system dynamics be given by

x = f(x, u, « (298)

x(t,) = £ (299)

where, as before, x and u are the state and control vectors of dimension
n and r, respectively. It is required to choose u(t)such that a function of
the terminal state

J = J[x(t/), t,l (300)

is minimized. The control vector must satisfy the inequality constraints

Vj < \X, < fij

(301)
i = L 2,

and a stopping condition is given by

iA(x, t) = 0 . (302)

Note that both J and \p are scalars.

We define

W (x, t) = the value of J at the stopping condition \p = 0, when we

start the problem in state x at time t, and use an optimal
policy.

The Principle of Optimality yields the relation

W(x, t) = Min [W(x + dx, t + A)] (303)

u
where A = dt.
Via a Taylor series expansion, we have
650 ELEMENTS OF MODERN CONTROL THEORY

W(x + dx, t + A) = W(x, t) +

, ,
dx + -
aW A
A
at

after dropping higher order terms.

Substituting this in Eq. (303), dividing through by A, and letting
A —»o, we obtain

(304)

This is equivalent to the two equations

aw _ / aw \ (305)
at Vdx )

(306)

if it is assumed that there are no constraints on u. Carrying out the

partial differentiation operation in Eq. (306) leads to*

(307)

Here, af/au is an n x r matrix whose (kj)th component is af*/au,-.

If we define

X = - — (308)
dx

then Eqs. (305) and (307) can be written as

(309)

(310)

*See Appendix C,
OPTIMAL CONTROL 651

In principle, a solution may be obtained as follows. Solve Eq. (310)

for u obtaining

u = u(x, X, t) . (311)

By substituting this in Eq. (309), and solving for W, we have

/

W - YV(x, t) . (312)

Finally, eliminating W between Eqs. (311) and (312) yields the opti¬
mal u as follows.

u = u(x, t)

It is possible, however, to convert this problem involving the solution

of partial differential equations to that of ordinary differential equations
which are precisely the Euler-Lagrange equations of the calculus of
variation.
To do this, we take the partial derivative of Eq. (309) with respect to x
(keeping in mind that u is now considered a function of x and t), viz.

Noting that

this equation reduced to

Using Eq. (311), this further simplifies to

(313)
652 ELEMENTS OF MODERN CONTROL THEORY

But

d\ _ / dX\ f dX (314)
dt \ dx / dt
Also

/ ax \ _ d\k _ a / aw \ _ d / dW\ _ d\j

\ dx ) dXj dXj\ dxk) dxk\ dx, ) dxk
kj

In other words

(315)

By virtue of Eqs. (3 14) and (3 15), Eq. (3 13) becomes, finally

dX
+ 0 . (316)
dt

It is not difficult to show that this is the matrix form of the Euler-
Lagrange equations.
For this purpose, consider the augmented function defined by Eq.
(13), viz.

F = X*(x* - fO •
fc-1

We obtain directly

— - - t X,
dXj k=x \Xj

j = 1, 2, % ,n .

Consequently, the Euler-Lagrange equations

OPTIMAL CONTROL 653

_d
dt

j = 1, 2,

may be expressed as

*;+ i: - o
k=i dXj

j = 1, 2,

which is completely identical with Eq. (316).

This leads us to interpret Xy as the rate of change of the optimal
return function with respect to the state variable, x3.
Examining Eq. (304), we see that the optimal u minimizes the expres¬
sion

Now by defining

H = \rf

we have

~ xT
(™)
V dx J
f = Min -Xrf - Max Xrf Max H
u u u

which is precisely the maximum principle of Pontryagin.

We note that
654 ELEMENTS OF MODERN CONTROL THEORY

Consequently, Eqs. (298) and (316) may be written as

dx _ aH
dt ax

dX dH
dt d x

The solution to the problem is obtained by solving the 2n equations,

(298) and (316). Thus, we must have 2n boundary conditions; or, if the
final time is not prescribed, we need (2n + 1) boundary conditions.
We have available the n initial conditions of Eq. (299). Suppose that
we are also given the p terminal conditions

xy(t/) = 7 i

where p is less than n.

We may then derive an additional (n — p) terminal condition as fol¬
lows. Noting that

W(x, t,) = J

we have at the end point

dW = Z — dxy + — dt = 0 (317)
i dx,- at

and

#- £ 4t dx, + * dt - 0 . (318)
j dx, at
Here

j - P + 1, p + 2, . , n

since, if a generic x„ is prescribed at t = t,, the corresponding

OPTIMAL CONTROL 655

dxa = 0.

»“*/

The last two equations yield

dW = dXy = 0

j = p + 1, p + 2, • • • n .

Since this must hold for all arbitrary variations, dx>, it follows that at
the end point

/ a.J \
dW
= Xy(t/) =
dJ Ut hi
dXj dx, ( df N
\ dXj
)
(319)
V at ;
*"•/

j = p + 1, p + 2, . ,n .

If tf is not prescribed, one more boundary condition is required. We

obtain this in the following way. Eliminate one of the dx;, say dxa,
between Eqs. (317) and (318); this yields

Therefore
656 ELEMENTS OF MODERN CONTROL THEORY

fd J N
) dt
dW dJ
Adxa (320)
dt at ( d* N\ dt
\ dxa ,1

Note that

1. If J and ip do not contain time explicitly, when dW/dt = 0, which

means that Arf = constant for all t

2. In the minimum time problem, J = tf, so that if t does not depend

on t, dW/dt = 1 at the end point and

\n -1 = 0.

,=</

Equations (319) and (320) are sometimes called the transversality

conditions.
We have already shown (in Sect. 5.1.3) that the Euler-Lagrange
operator

d_
dt

may be interpreted as a generalized gradient. Furthermore, the condition

in Eq. (306) is an alternate expression for Eq. (183), wherein (dH/du) is
another type of gradient (the Green’s functions).
We have shown, therefore, that the basic results embodied in the
variational calculus, the gradient technique, and the maximum principle
are all necessary consequences of the principle of optimality in dynamic
programming.

5.2 STANDARD SOLUTIONS

In certain well-defined optimization problems, it is possible to obtain

closed-form solutions. If the format of these problems is sufficiently
general to include as special cases a wide variety of situations of practical
interest, it is tempting to refer to these as standard solutions. These
OPTIMAL CONTROL 657

results are useful not only because they solve specific problems, but also
because they may be taken as first approximations to more complex
problems. This theme will be further developed in Sect. 5.3.

5.2.1 Linear Problems

The most tractable optimization probjem, from an analytic point of

view, is one whose dynamics are governed by a set of linear differential
equations with a performance index of the quadratic type. This is ex¬
pressed mathematically by

x = Ax + Bu (321)

V = Gx (322)

x(0) = £ (323)

J = f (xr fij' QGx + ur Ru) dt + xJ’(ty) Mx(t/) (324)

•'O

where

x = n-dimensional state vector

u = m-dimensional control vector
y = q-dimensional measurement vector
£ = initial condition vector
J = performance index (cost); a scalar
A = n X n constant matrix
B = n X m constant matrix
G = q X n constam matrix
Q = q X q constant matrix (symmetric)
R == m X m constant matrix (symmetric)
M = n X r± constant matrix (symmetric).

The problem is usually stated in the following form. Given the system,
Eqs. (321) and (322), with the initial condition in Eq. (323), determine
the control vector, u(t), such that the performance index, Eq. (324), is a
minimum.*

The arguments are completely analogous if “maximum” replaces “minimum.”

658 ELEMENTS OF MODERN CONTROL THEORY

We define the optimal return function as follows.

W(x, t) = Min f (x^ Gr QGx + ur Ru)dt

Jt
u
(325)

+ XT(ty) Mx(t,)

Note that W(x, t) is a function of the current state and the current
time. Applying the Principle of Optimality, we find that W(x, t) satisfies
the recurrence relation

W(x, t) = Min
u
^xrGrQGx
+urRu^
A+W^x
+dx,
t +A^(326)
A = dt .

Expressing the second term in the brackets by its Taylor expansion,

we have

W^x
+dx,
t +A^=W(x,
t)+ dW (x, t)
dx
dx
dW

dt

dW (x, t) dW
W (x, t) + (Ax + Bu) A + — A
dx dt

retaining only first order terms in A. Substituting this in Eq. (326),

dividing through by A, and letting A —> 0, we obtain

dW
dt
= Min

u
( XT’ QGx -I- ur M?y (Ax + Ru) . (327)

From Eq. (325), we see that a boundary value is given by

W(x, ty) = Xr(t/) Mx(t/) . (328)

OPTIMAL CONTROL 659

Equation (327) is the Hamilton-J acobi equation for the system which
could also be derived from the maximum principle. Despite its formida¬
ble appearance, Eq. (327) has a closed-form solution that may be readily
obtained as follows. From Eq. (327), we write the two equations

xt Gr QGx+ ur Ru + ( — ) (Ax+ Bu) = - — (329)

V dx / . at

au
^xrGrQGx
+urRu (Ax -f- Bu) = 0 . (330)

Now Eq. (329) is satisfied only for that value of u obtained by solving
Eq. (330). Thus u, when substituted in Eq. (329), yields an equation
that is completely equivalent to Eq. (327). Performing the indicated
operations in Eq. (330), we find

_a_ = o
au

aw
2Ru + Br — =0
ax

or, finally

u
*
- — R_1 Bt (331)
2

We use the superscript ( )* to indicate that this is the optimal control

function (policy), which when substituted in the system (321) yields the
optimal trajectory x*(t), and is the one which minimizes (or maximizes)
the performance function in Eq. (324).
Substituting Eq. (331) in Eq. (329) yields

- — = x^ Gr QGx + BR-* Br (332)

at

This equation is equivalent to Eq. (327). We assume a solution of the

660 ELEMENTS OF MODERN CONTROL THEORY

form

W(x, t) = xT P(t) x (333)

where P(t) is a symmetric matrix, which is for the moment unknown.

Then

dW
= 2 P(t) x
dx

aw
— = xr P(t) X

and Eq. (332) becomes

-x*1 Px = xT(Gr QG + 2 PA - PBR-1 BrP)x . (334)

Since the matrix P is symmetric, it is convenient to have all the terms

inside the parentheses symmetric; in fact, the only one not symmetric is
2 PA, since A in general is not symmetric. It is known, however, that
any square matrix may be expressed as the sum of a symmetric and a
skew-symmetric matrix. In the present case

2 PA = (PA + A^ P) -+%(PA - AT P)

where

(PA + ArP) = symmetric

(PA — ArP) = skew-symmetric.

Now let

a = Ab

where a and b are arbitrary vectors. Then

&T = AT

and

br(PA - ArP)b = br PAb - b^ A^Pb = brPa - arPb = 0 .

OPTIMAL CONTROL 661

In other words, (PA — ArP) is identically zero. Consequently, Eq.

(334) becomes

xr[p _|_ pa + Ar P — PBR-1 Br P + G* QG]x = 0 .

This yields the nonhomogeneous matrix Riccati equation

P + PA + ArP — PBR-1 B*P = —Gr QG . (335)

From Eqs. (328) and (333), we see that

P(t/) = M. (336)

The solution of Eq. (335) may be obtained in the following way.(161>

Define an associated system of linear matrix equations as follows.

Y = AY - BR"1 BrZ (337)

Z = -G1" QGY - ArZ (338)

Then the solution of Eq. (335) is given by

P = ZY-1 . (339)

This is easily verified by differentiating the above expression and

substituting for Y and Z from Eqs. (337) and (338). Use is made of the
relation

Y-i = — y-i y Y"1

which follows by differentiating the identity

Y-i = Y'1 Y Y"1

with respect to time.

Writing Eqs. (337) and (338) in the form

V = FV (340)

where
662 ELEMENTS OF MODERN CONTROL THEORY

Z
V a
Y

—GT QG
F
-BR-1 Br A

we find

V(t) = eFt V(0) . (341)

If final rather than initial conditions are known, we write instead

V(t) = eF(t~tf) V(t/)

which is equivalent to Eq. (341) if we note that

e~Ftf V(t,) = V(0) .

Consequently, there is no loss of generality in dealing with V(0) as an

“initial condition” matrix.
Let Eq. (341) be partitioned as follows.

Z(t) ' ^n(t) V»l*(t)“ " Vi(0) %

Y(t) _ *P2l(t) <P22(t) _ V.(0) _

Then using Eq. (339), we find

P(t) = fai(t) vx(0) + v>12(t)v2(0)] [<%(t) v^o) + Mt) v2(0)]->

= Ki(t) + Vo(0)] te,(t) + ^22(t) Vo(O)]-1 (342)

where V0(0) replaces V2(0)Vr1(0). The matrix V0(0) is evaluated using the
boundary condition of Eq. (336).
Equation (342) is the solution of the matrix Riccati equation, (335).
Having P(t), we evaluate the optimal control vector, u*(t), from Eq.
(331), making use of Eq. (333), viz.

u*(t) = -R-1 Br P(t) x(t) . (343)

OPTIMAL CONTROL 663

An important special case arises when M == 0 and tf —> °o in Eq. (324).

In this case, W and P do not depend on t. The matrix Riccati equation,
(335), then reduces to

PA + ATP - PBR-1 Br P = —Gr QG (344)

which is merely an algebraic equation for the determination of P. For

large n, the computational solution of Eq. (344) is not trivial. The recom¬
mended procedure is to integrate Eq. (342) with an assumed value for
V0(0) , and stop when P(t) reaches a steady-state value, which is then taken
as the desired value of P. The speed of convergence depends, of course,
on the selected value of V0(0). This, however, is not too critical, since
convergence is generally very rapid.
The optimal control vector is now given by

u*(t) = —K x(t)
(345)
Iv = R_1 Br P, a constant matrix.

In other words, the optimal control is a linear function of the current

state. Thus, the optimal control is expressed as a closed-loop system
rather than an open-loop type, which has characterized the results ob¬
tained in previous sections.

5.2.2 Norm-Invariant Systems

Consider the nonlinear system

x = g(x, t) + u (346)

where, as before, x is an n-dimensional state vector and u is an n-

dimensional control vector. The latter satisfies a constraint of the form*

| |u| | < 7 . (347)

We pose the problem of determining the control vector, u(t), subject

to the above constraint, which drives the system from an arbitrary initial
state to the “origin,” x = 0, in minimum time.

See Appendix C for a discussion of norms.

664 ELEMENTS OF MODERN CONTROL THEORY

We suppose that the system is norm-invariant ; that is, the homogeneous

form of Eq. (346)

x = g(x, t) (348)

has the property

11x(t) 11 = | |x(0)11 for all t > 0 . (349)

In this case, Eq. (Cl 5) shows that

|*|| = ^ ^ = o
1 11 llxll IJxll

which leads to

xT g(x, t) = 0 . (350)

The results that follow were first obtained by Athans.(9°) His deriva¬
tions, however, relied heavily on function space methods and were not in
the “mainstream” of optimal control theory.
A simpler and more direct development is possible by application of
the basic concepts of dynamic programming, as follows.
Define

W (x) = the time required to transfer the system from state x

to the origin, x = 0, using an optimal policy.

Applying the Principle of Optimality, we obtain

W(x) = Min
u
A + W x + (g + u)A ]}' (351)

In other words, if we are in state x, then by applying a control, we

incur a “cost,” A = dt, and arrive in the new state, [x + (g -f u)A], Pro¬
ceeding optimally thereafter, the cost is W[x + (g + u)AJ by definition.
However, we select u to minimize this total cost. Hence, the form of
Eq. (351).
OPTIMAL CONTROL 665

Via a Taylor expansion

aw(x)
W[x + (g + u)A] = W(x) + (g + u)A + 0(A2)
dx

Substituting this in Eq. (351), dividing through by A, and letting

A —>0, we find '

0 = Min

u
1 + (S?)' (g + u) (352)

Minimizing the expression in the brackets is equivalent to minimizing

(353)

where u is constrained by Eq. (347). Since expression Eq. (353) is merely

the inner product of two vectors, one of which is arbitrary and the other
of which is constrained in magnitude, it is easy to see that Eq. (353) is
minimized when

(354)

Equation (352) is, therefore, equivalent to

(355)

The solution of this equation is

W(x) = - (xrx>) = - ||x| | (356)

7V / 7
as may be verified by direct substitution, viz.
666 ELEMENTS OF MODERN CONTROL THEORY

aw l
x
dx 7

which, when substituted in Eq. (355), yields

The middle term vanishes by virtue of Eq. (350), Q.E.D.

Using Eq. (356), the optimal control vector, u(t), may be written as

x(t)
u*(t) = — (357)
x(t)||

Thus, u*(t) is obtained as a function of the current state (feedback

principle), and the minimum time is obtained from Eq. (356), withx*(t)
representing the optimal trajectory; i.e., the integration of Eq. (346)
using the optimal control, Eq. (357).
A similar analysis shows that the control that minimizes

k u dt k = positive constant (358)

while driving the system from an arbitrary state, x, to x = 0, is also given

by Eq. (357). The terminal time, t/, is not fixed in advance. This may be
interpreted as a minimum fuel problem.
It may also be shown, via the same techniques, that the control that
minimizes

(um) dt (359)

while transferring the system from an arbitrary initial state, c, to x = 0,

is given by

u*(t)
HI . x(t) (360)
U ||x(t)||
x(0) = c

if = prescribed.
OPTIMAL CONTROL 667

The optimal trajectory. x*(t), is the solution of Eq. (346), with u*Ct)
given by Eq. (360). Note that now the optimal control depends explicitly
on the initial state and the prescribed time, t/, whereas previously the
optimal control was a function of the current state of the system only.
This last case, the minimization of the integral, Eq. (359), is a type of
minimum energy problem.
The above results are quite significant, since closed-form solutions for
nonlinear optimization problems are hard to come by. More important,
several cases of practical interest in aerospace control are indeed character¬
ized by norm-invariant properties. The latter is often a consequence of
conservation of momentum. In particular, two classes of norm-invariant
systems are the following.

1. All linear systems of the form

x(t) = A(t) x(t) + u(t) (361)

||u(t)|| < 7

A(t) = -A*(t) . (362)

All nonlinear systems of the form

x(t) = B[x(t), t] x(t) + u(t) (363)

||u(t)|| < y

B[x(t), tl = —Br[x(t), t] . (364)

Example 8: For purposes of analyzing the short period motion,

a satellite may be assumed suspended in a force-free field. Let E, I2, and
I8 denote the three moments of inertia of the body about the principal
axes that pass through the center of mass. Also, denote by xi, x2, and x3
the three angular velocities about the axes, 1, 2, and 3, respectively. The
motion is then described by the Euler equations

11 xi = (E — I3) x2 x3 + T! (365)

12 x2 = (I8 — Ii) x8 xi + T2 (366)

13 x3 = (Ii — E) xi x2 + T3 (367)

where Ti, T2, and T3 are the components of the torque vector T.
 ELEMENTS OF MODERN CONTROL THEORY
668

If we define new variables by

yi = Ii xi
y2 = I2 x2 / (368)
y8 = Is xa '

then the Euler equations take the form

yi = | a )y2y8+ Tj (369)

y2= | a ) y8yi+ T2 (370)

y» = | 4: | yiy2+ T8. (371)

Quantities yi, y2, and y3 are the components of the angular momentum
vector, y. It is easy to show that when Tt = T2 = Ts = 0

i ||y(t)||= 0.
dt
(372)
In fact, in the present case, this is merely a statement of the Principle of
Conservation of Angular Momentum. Now if the constraint on the
torque vector is of the form

11T (t) 11< 7 (373)

the theory of Sect. 5.2.2 is directly applicable. Therefore, the control law

y(t)
'J'* (374)
||y(t)||
will bring the system from an arbitrary angular momentum to zero
angular momentum in minimum time. This corresponds to stopping the
tumbling motions of the body in the shortest possible time.
Equation (374) shows that the torque vector takes on its maximum
absolute value and is directed opposite to the angular momentum vector.
OPTIMAL CONTROL 669

5.2.3 Optimal Trajectories

A simple form of optimal launch trajectory problem may be formulated

in the following way. It is assumed that the vehicle is a point mass in a
uniform gravitational field; aerodynamic effects are neglected, and the
flat earth approximation is used. The equations of motion then take the
form (see Fig. 5.12)

Aui cos u2 ,
Xi = = II (375)
X3

Aui sin u2
x2 = - g = f2 (376)
X3

XS = — aUl = f3 (377)

X4 = Xi - f4 (378)

X5 = x2 = f6 . (379)

Figure 5.12. Coordinate System for Optimal Trajectory Problem.

 ELEMENTS OF MODERN CONTROL THEORY
670

Here

xj(t) = horizontal component of velocity

x2(t) = vertical component of velocity

x4(t) - range

xs(t) = height
m(t)
x3(t) = nondimensional mass =
m(0)

T
ui(t) - nondimensional thrust = tjt
i v

u2(t) = inclination of the thrust vector with respect to the horizontal.

The thrust is given by

T = — ihju n = constant (380)

and satisfies the constraint

0 < T < Tmax (381)

which means that

0 < ui(t) < 1 .

Constants A and a are defined by

A =
m(0)

a =
m(0)ju

It is required to program the thrust magnitude and direction (m and u2,

respectively) such that a suitable index of performance is optimized. In
various guises this problem has been studied by many investigators.!18’67
70,71) Even so simplified a problem as this does not yield a closed-form
solution. However, certain general features of the optimal control
OPTIMAL CONTROL 671

functions can be obtained, and the complete solution requires that one
solve a set of differential equations with prescribed initial and terminal
boundary conditions.
The development that follows is based on the work of Isaevd64) and
includes the results of many other investigators as special cases.
We formulate the general problem as follows. Given the initial condi¬
tions

x*(0) = ay
(382)
i = 1, 2, ',5

find the control (ui, u2) that transfers the system described by Eqs.
(37 5)-(379) from the given initial state to a prescribed final state such
that the function

J = £ Cy
Xy(ty) (383)
j=i

is a minimum (maximum), where the final time, t/, is given.

As shown below, the problems of maximum range, altitude, final
velocity, and minimum fuel are special cases of this general formulation.
The prescribed final state and performance function will, of course,
differ in each case. Nevertheless, certain general features of the control
functions are common to all.
We will seek to obtain a solution via the maximum principle. The
components of the Lagrange multiplier (costate) vector are defined by
Eq. (78); viz.

L — (384)
fc=i dx.j

— X4 (385)

— x6 (386)

^( Xicos u2 + X2sin u2J (387)

x82V /

X4 = 0 (388)
672 ELEMENTS OF MODERN CONTROL THEORY

The hamiltonian is then given by

H = Z Xyfy
j-1
or, in expanded form

H = ux
-(
X3 \
Xxcos U2 -T X2sin u2 J — a\, )- - X2 g

(390)
+ X4 Xj + Xg x2 .

Minimizing (or maximizing) H with respect to ux and u2 is equivalent

to minimizing (or maximizing)

H = ux
-( Xx cos u2 +
)-
X2 sin u2 ) — a\
1

X
(391)

An elementary calculation shows that H is maximized by

Ux = 1 > 0

< 0 (392)

u2 = tan-1 — (393)
Xx

and minimized by

ux = 1 $2 > 0

= 0 <t>2< 0 (394)

u2 = tan-1 —
X,

, Xl 2T
= — tan 1- (395)
X2 2
OPTIMAL CONTROL 673

where

= J_VXi2+ x2s_ Xs (396)

x8 A

<i>2= - V/Xl2 + Xs2+ . (397)

x8 A

For definiteness, we will consider only the case of maximizing * the

performance index, J, of Eq. (383). In this case, the maximum principle
requires that we minimize H. Accordingly, the optimal control functions
are given by Eqs. (394) and (395).
Now Eqs. (388) and (389) yield, immediately

X4 — x40 (398)

X6 = X50 (399)

and, in turn

X) = — X4ot T Xio (400)

X2 = — Xsot -E X20 (401)

where Xi0, X20, X40,and X50are constants.

It follows that the optimal u2(t) is given by

u*(t) = X20~-5gt-. (402)

Xio — X40t

We have, therefore, the results that the optimal thrust inclination is a

bilinear function of time and the thrust magnitude is of the bang-bang
type. This is a very general feature of the optimal control, since no
stipulations have as yet been imposed on the performance index, J, or
the boundary conditions, x,(t/).
If we define

Sg 4>2 = 1, 4*2 > 0

(403)
= 0, 4>2< U

Minimizing J is equivalent to maximizing ( —J).

674 ELEMENTS OF MODERN CONTROL THEORY

then after substituting Eqs. (394) and (395) into the set of Eqs. (375)-
(379), (385M389), we obtain

.* AAi Sg *^2
(404)
Xl x8(Xi2
+ X22)1'2

.* A\2 Sg ^2
x, = — - -- — g (405)
X8(Xj2+ X22)1'2

X3 = — a Sg <t>2 (406)

X4 = Xi (407)

Xg = x2 (408)

v< II 1 (409)

•/<
w II 1
(410)

. A(Xi2 + X22)1'2Sg $2
A8 (411)
X82

v? II 0 (412)

v<0 II O (413)

The set of Eqs. (404)-(408) describes the optimal trajectory. It is easy

to show that the switching function, $2, can change sign no more than
twice. From Eq. (397)

4>2= — + Xt)
X8(Xi2+ X22)1'2

which is obtained by elementary differentiation and making use of Eqs.

(406) and (41 1). By virtue of Eqs. (409), (410), (412), and (413), this
may be written as

^ = —- - - (414)
Xg(t2 + 01 t + ft)1'2

where the /3, are constants. Since x8(t) is a bounded monotonic function,
OPTIMAL CONTROL 675

it follows that <t>2can vanish for only one value of t, say t0. Therefore,
by Rolle’s Theorem, $2 can vanish at no more than two points.
It follows that the optimal trajectory can have no more than two
powered phases.
We now consider several special cases.

1. Maximum Horizontal Velocity

/

Given

Xy(0) = a,
(415)
j = 1, 2,

x*(t/) = 0

x*(t/) = b8 t/ given (416)

x5(t/) = b6

maximize J = x^t,) .
Therefore, ci = 1

c2 — c8 — C4 = Cj = 0 .

Using the methods of Sect. 5.1.2 to determine the boundary conditions

for the \i, we find

Xi(t/) — —1 = X10
(417)
^(t/) = 0 = X40.

The solution of the problem is completely determined by integrating

the ten differential equations, (404)-(413), whose boundary conditions
are given by Eqs. (41 5)-(41 7), with $2 defined by Eq. (397).
Note that by virtue of Eqs. (417), the control law, Eq. (402), reduces
to

u2(t) — X501 — X2o . (418)

676 ELEMENTS OF MODERN CONTROL THEORY

2. Maximum Altitude

Given

xy(0) = ay
(419)
j = 1, 2,

Xs(ty) = b8

t/ given (420)

x4(t,) = b4

maximize J = x5(ty) .
Therefore, c8 = 1

Ci = C2 = c8 = c4 = 0

and

Xl(ty) = 0

\*(ty) = 0 (421)

x#(t /) — —1 .

In this case, the solution to the problem is obtained by integrating the

set of Eqs. (404)-(413), with the boundary conditions given by Eqs.
(419M421).
We note also that via, Eq. (421), the control law, Eq. (402), becomes

. 1
u2 (t) = — — = constant. (422)
X4o

3. Maximum Range

Given

xy(0) = a,-
(423)
OPTIMAL CONTROL 677

x*(t/) = bg

if given (424)

x*(.t/) = 0

maximize J = x4(t/)
'

C4 = 1

Cl = C2 = Cg = Cj = 0

Xi(ty) = 0 = Xio

Xg(t/) — 0 = X20 (425)

X4(t/) — 1 = x40.

Again the solution is obtained by integrating the set of Eqs. (404)-

(413), but with the boundary conditions, Eqs. (423)-(425).
The conditions in Eqs. (425) now yield the control law

u^(t) = —X60= constant. (426)

4. Minimum Fuel

Given

xy(0) = a>
(427)
j = 1, 2, -,5

X4(ty) = b4

tf given (428)

X8(ty) = 0

maximize J = —x8(ty)
 678 ELEMENTS OF MODERN CONTROL THEORY

Ci = C2 = C4 = C6 = 0

Ai(t/) — 0 = Aio

X2(t/) — 0 = A20 (429)

As before, the problem is solved by integrating Eqs. (404)-(413), but

for this case using the boundary conditions, Eqs. (427)-(429).
Using conditions (429), the optimal thrust inclination is again a con¬
stant; viz.

U2(t) = = constant. (430)

Remark: It has been shown that the optimal control for this problem is
characterized by the following properties.
. 1 The thrust magnitude is either zero or maximum (bang-
bang).
2. The thrust inclination is, in the general case, a bilinear
function of time. For specialized cases, it may be a con¬
stant or a linear function of time.
3. The optimal trajectory contains no more than two power¬
ed phases.
In order to obtain the complete solution (exact form of the
optimal trajectory), it is necessary to solve a set of nonlinear
differential equations with two point boundary conditions.
This is not a trivial computational task. Some promising tech¬
niques for doing this efficiently have been developed in recent
years; a description of these is contained in Appendix A.
It should also be noted that in the above analysis there was
the implicit assumption that a solution exists. While this may
be true in most cases of practical interest, it must be recognized
that not all combinations of conditions give a meaningful or
well-defined problem. Generally, however, if there exists a con¬
trol that satisfies the given boundary conditions, one may pro¬
ceed to calculate an optimal control with the assurance that this
is physically and mathematically meaningful.
OPTIMAL CONTROL 679

There is one further mathematical feature of the solution:

the control laws derived represent only necessary conditions
for the optimum. They may not be sufficient. Sufficiency
proofs are at present available only for linear systems. Again,
in most cases of practical interest, this is a mathematical
sophistry that does not seriously compromise the solution.

5.3 AEROSPACE APPLICATIONS

The literature on application of optimal control theory to guidance

and control of aerospace vehicles is very extensive. Even the large list of
references in this chapter is far from complete. One of the best survey
papers on this subject is the one by Paiewonsky,(!9) which reviews the
highlights and historical development of the theory and gives a critical
evaluation of basic contributions. Other basic sources are the books by
Lawden,(2) Chang/1) Leitmann/3) and MerriamU46) and the papers by
Miehle/18) Lawden,(70) and Ho.(135)
The choice of examples to illustrate “typical” applications is largely
subjective. Among the factors that may guide such a selection are
theoretical novelty, practical importance, or mathematical elegance. Since
the primary aim for the moment is pedagogic exposition, the examples
that follow were selected for their didactic value and practical interest.
The extensive detailed treatment that characterized practical design
problems of high order was avoided, since the details tend to obscure
the essence.

5.3.1 Lunar Soft Landing152)

A problem of some practical interest is the following. Assume that a

lunar vehicle has arrived in the vicinity of the moon’s surface. The
motion of the vehicle is vertically downward, and the only forces acting
are the thrust and lunar gravity (assumed constant). The thrust magnitude
may be throttled between zero and some prescribed maximum. It is
desired to calculate the thrust program that will result in a soft touch¬
down (altitude and altitude rate simultaneously zero) while minimizing
the fuel expenditure.
With these assumptions, the motion is governed by

(431)
680 ELEMENTS OF MODERN CONTROL THEORY

where

h = altitude of vehicle above lunar surface

in = instantaneous mass

g = lunar gravity acceleration.

The thrust is given by —cm, where c is a positive constant andrii < 0.

We also have the boundary values

h(0) = xio
(432)
h(0) = x2o

hit/) = 0
(433)
h(t/) = 0

and the thrust magnitude constraint

— a < m(t) < 0 (434)

where a is a positive constant.

Since the vehicle is assumed to be in the terminal descent phase,
physical considerations indicate that xi0 > 0 and x20 < 0.
Minimizing the fuel expenditure is equivalent to minimizing the
performance index

J = m(0) — m(t/) . (435)

This completes the mathematical statement of the problem. Before

proceeding with a formal solution, certain simplifications may be effected
in the following way. We have

rii
in

where £n( ) denotes natural log of.

Therefore, Eq. (431) may be written as

h= - c^ (thmi)
- g. (436)
OPTIMAL CONTROL 681

Integrating between the limits of 0 and t,

h(t) = - c^n -5^ - gt + h(0) .

m(0)

Now h(t/) = 0 if, and only if

c/n^ = ft(0)
- gt,.
m(0)
Solving for m(t,)

h(0) ~ gt,
m(t,) = m(0) exp (437)
c

Substituting this in Eq. (435)

J = m(0) 1 exp

It is apparent, therefore, that minimizing the fuel expenditure is com¬

pletely equivalent to minimizing the total time, t,. In other words, we
reduce to a minimum time problem.
We introduce the definitions

h = xi

Xi = x2

x8 = m

u = m .

Equation (431) is then represented by the system

Xi = x2 = fi (438)

c u — g = ff2
x2= — — (439)
X3

Xs = U = f3 (440)
682 ELEMENTS OF MODERN CONTROL THEORY

with the control constraint given by

-« < u(t) < 0 (441)

and boundary conditions

xi(0) = xio xi(t/) = 0

X2(0) = x20 x2(t,) = 0 (442)

x8(0) = x80 x8(t/) s free.

In accordance with the discussion of Sect. 5.2.1, the minimum time

problem

min

is converted to a problem of minimizing x4(t/) where

x« = f dt
* 0

X4 = 1 (443)

X4(0) = 0 . (444)

Proceeding via the maximum principle, we find, for the hamiltonian

H=Xi
x2—X2
^ +X8
u-fX4 (445)

where the Lagrange multipliers satisfy

Xi = 0 (446)

x2 = -Xr (447)

x8 = X2cu (448)
" ~^r

X4 = 0 . (449)
OPTIMAL CONTROL 683

In order to minimize x4(t/), we must maximize H. The optimal control

is, therefore, given by*

u*(t) = —a when <i> < 0

(450)
— 0 when $ > 0

0X2 '
<t> = X, (451)
x8

In Ref. 52, a rather elaborate analysis is used to show that <t>cannot

change sign more than once in the interval 0 < t < tf. It is possible,
in fact, to derive this result very easily, as follows.
Differentiating Eqs. (451) and using Eqs. (440), (447), and (448),
we find

(452)
x8

But Eq. (446) shows that Xi = constant, while x8 = m(t) is a positive

monotonic function of t. Therefore, <t>cannot change sign in the interval,
0 < t < tf, which, in turn, means that $ can change sign no more than
once in this interval. Consequently, once the full thrust is switched on,
it remains on until touchdown. We now seek to determine the relation
between xj(t) and x2(t), at which switching occurs. Let the interval over
the thrusting phase be denoted by (0, tT). Denote by xj0, x20,and X30the
altitude, altitude rate, and mass at the instant of switching to full thrust.
Then from Eq. (439)

x2(t) gt + Xm (453)

0 < t ^ 11 .

Using Eq. (438), we obtain

*A somewhat lengthy analysis shows that the singularity condition <t>= 0 is incompatible with
the physical constraints of the problem. See Ref. 52.
 ELEMENTS OF MODERN CONTROL THEORY
684

Xi(t) = [' x,(r)dr+Xxo

Jo
= (l - ^t)ln (l - -£t) a \ X30 / \ x30 '

+ ct — - gt2 + X20 t + x10

(454)
0 < t ^ 1 7> .

For a soft landing, we must have

xi(tr) = x2(tr) = u .

Therefore, Eqs. (453) and (454) become

0 = - c In ( 1 — t T) —gt T+ x2o (455)

V x30 /

(456)

+ X20 1 T T x10 .

These may be solved for x10 and x20 as follows.

x2o = (457)

Xl'0
= - 2M
a
in(l
\
- 4- tT)
x3n /
- ct
r - i
2
gtr^ (458)
If we eliminate tr between these two equations, we obtain a relation
of the form F(x10, x20) = 0, which determines the altitude and altitude
rate at which one switches to full thrust. A fairly simple form for F is
obtained by using the approximation

Oft t 1 / atjA
X30 2 \x3'0/
which, for

m(tr)
> 0.75 (459)
X30

is accurate to within 2 percent.

OPTIMAL CONTROL 685

In this case, Eqs. (457) and (458) become

xio = atr2 (460)

x20 = —2 atr — btT2 (461)

where

Ca - gx3'0
X30

b=-(—)•
2 \X30 /

Since the ratio of maximum thrust to initial mass must be greater than
g, we see that a > 0. Also, since x^ is positive, it is apparent that the
only value of tT that is physically meaningful is

tt — (462)

Substituting this into Eq. (460) yields

F^x10,
x20^
=^xio
+x2'0
+2a/j/^-• (463)
Physical considerations dictate that xi0 > 0 and x20 < 0, since altitude
is positive and the vehicle velocity is in the negative xi direction. Further¬
more, the inequality (459), together with the relation

x»(t) = m(t) = x3'0 — at

0 < t tr

leads to

X30
0 < tr < (464)
4a
686 ELEMENTS OF MODERN CONTROL THEORY

Using tr ] max = X30/4a , we find that the range of values of interest in

the xi - x 2 plane is given by

(465)

(466)

A plot of the switching function, Eq. (463), for the range of values
given by Eqs. (465) and (466) is shown in Fig. 5.13.
Assume now that the vehicle is at some initial altitude, xi0, and with an
initial altitude rate, x2o- The free fall trajectory is readily obtained from

Xl = X10
1_ (467)
2g

This is also plotted in Fig. 5.13.

Figure 5.13. Plot of Switching Function and Free Fall Trajectory.

OPTIMAL CONTROL 687

The form of the optimal trajectory is now apparent. If the initial

conditions are such that F(xi0, x20) > 0—i.e., the point (xi0, x2o) lies above
the switching curve as shown in Fig. 5.13— the vehicle is allowed to
continue in free fall until F(xi, x2) = 0. At this point, full thrust is
switched on and remains on until touchdown.

5.3.2 Optimal Control of Booster Vehicles

/

The fact that a complete solution is available for the problem of opti¬
mal control of a linear system with a quadratic performance index (Sect.
5.2.1) has motivated the effort to relate pragmatic design requirements
to this format. As a matter of fact, a far from trivial problem is that of
formulating a meaningful optimality criterion for a launch vehicle auto¬
pilot. One usually requires that the control system stabilize the unstable
airframe. In addition, it is desired that the deviations from desired atti¬
tude be small, and that the angle of attack be small in order to reduce
bending loads, engine angle deflections, etc. With the exception of
stability, none of these automatically results from the quadratic per¬
formance criterion.
Several recent studies have attempted to relate meaningful performance
requirements in booster autopilot systems to the quadratic performance
criterion. We shall consider in detail the papers by Tyler and Tuteur/13)
Fisher/14) Bailey/122) and Tyler. )136) Rather than discuss these indi¬
vidually, we will take a unified point of view. The results of particular
studies will be shown to be special cases of a general approach. Among
the problems investigated are the following.

1. Stabilizing a flexible booster using multiple sensors.)14)

2. A model-following system.)136)
3. Minimizing deviations from desired states.)122)
4. Relating the characteristic equation of the optimal system to the
weighting matrix elements in the performance index.)13)

These will be referred to as Problems 1-4. As a preliminary, the results

of Sect. 5.2.1 will be expressed in a variety of alternative forms to
facilitate treatment of individual problems.
In the notation of Sect. 5.2.1, the system considered takes the form

x = Ax + Bu (468)

y = Gx (469)
 ELEMENTS OF MODERN CONTROL THEORY
688

which are merely Eqs. (321) and (322) repeated here for convenience.
For the performance index, we take the following version of Eq. (324).

J = lim[ (xTGrQGx
+ UrRu) dt (470)
*/-»«Jo \ /

In this case, the optimal control is given by Eq. (345).

u*(t) = —Kx(t)
(471)
Iv = R-1 BT P

where R, B, and P are constant matrices, the latter obtained as the solu¬
tion of the steady-state Riccati equation

PA + ArP - PBR-1 BrP = -Gr QG . (472)

This is the simplest case; the optimal control, Eq. (471), is merely a
constant matrix times the current state of the system (feedback principle).
If we take for the performance criterion

J= (xtCTr
QGx+ urRu^dt (473)
then the form of the optimal control is still given by Eq. (471) except
that P, and, therefore, K, are now time varying, with P obtained as the
solution of

- P = PA + Arp - PBR-1 BrP + Gr QG . (474)

The above is merely a summary of the results obtained in Sect. 5.2.1.

Alternative forms of this solution have been obtained by KalmanG3)
and Mercian/146) and have been used in the studies by Tyler/1 36)
Bailey/14) and Tyler and Tuteur/13) which will be discussed here. It
will clarify the discussion to relate these results to the ones derived in
Sect. 5.2.1 .
To obtain Kalman’s equations, we first postmultiply Eq. (474) by x,
obtaining

—Px = PAx + ArPx — PBR-1 BrPx -I- GrQGx . (475)

OPTIMAL CONTROL 689

The costate (Lagrange multiplier) vector is obtained from the optimal

return function, W(x, t), via Eq. (309), viz.

<3W
X =
dx

which in the present case may be expressed as

X = — 2Px

using Eq. (333). However, Kalman’s performance index contains the

factor 1/2 in front of the integral of Eq. (470), which means that his
optimal return function is half of that defined by Eq. (325). Further¬
more, to obtain the minimum of the performance function, he minimizes
the hamiltonian defined by Eq. (310), whereas the conventional (maxi¬
mum principle) approach requires the maximization of the hamiltonian.
This means that, instead of X, as defined above, we must take

1 dW
X = Px (476)
2 dx

to conform with Kalman’s assumptions. Noting that

A = Px + Px (477)

we obtain, after substituting Eqs. (476) and (477) in (475)

A = -An - GrQGx (478)

where we have used Eqs. (468) and (471). Furthermore, Eqs. (468) and
(471) show that the optimal trajectory is described by

x = Ax - BR-1 BrX (479)

in terms of the costate vector, Eq. (476). The last two relations may be
written as

x A -BR"1 Br x

(480)
A -GrQG — AT X
690 ELEMENTS OF MODERN CONTROL THEORY

It is apparent that the costate vector, X, is the adjoint of x. Conse¬

quently, the eigenvalues (closed-loop roots) in the characteristic equation

(Is - A) BR-1 Br
= 0 (481)
G^QG (Is + A)

consist of the eigenvalues of the optimal system and their mirror images
about the imaginary axis in the s plane, which belong to the adjoint
system.
We note further that the optimal trajectory is also described by

x* = (A — BK) x (482)

using Eqs. (468) and (471). The characteristic Eq. (482), together with
its adjoint, is, therefore, given by

|Is - (A - BK)| •|Is - (A + BK)| = 0. (483)

Thus, the roots of Eq. (481) and (483) must be identical. This fact
will be used later to establish a relationship between the eigenvalues of
the optimal system and its feedback gains.
We turn now to a discussion of Merriam’s method. The performance
criterion is taken as

=!0 (y°~y)®(yo
~y)+(ud~u)r(u°- dt . (484)

Here y is the output vector given by Eq. (469), and yD is a reference or

desired output. Similarly, ufl is the desired or reference control, and u is
the actual system control vector. The matrices Q and R are assumed to
be diagonal matrices that may be time varying.
The optimal return function is defined in a manner similar to Eq. (325),
as follows.

W(x,
t)=Min
| - y^Q^yD
-
T

+
R^uc
— dt (485)
OPTIMAL CONTROL 691

Proceeding as in Sect. 5.2.1, we find

. J1 rp
dW
at
= Min

u
\Yd—yj Q^yc
—y^+ R^uD
—
(486)

+
(f ) (Ax
+Bu)
The optimal control is found to be

u* = uD
1
— — R-1
2
Br
(") (487)

Assuming a solution of the form

W(x, t) = p(t) - 2 22 Pa(t) xa(t) + 22 22 PjS7(t)X^(t)x7(t) (488)

a—) /3= 1 7=1

we find

<9\V n n n
— = P - 2 22 Pa Xa + 22 22 P/37 X7 (489)
at a=l /3=1 7=1

dW o , o V
— = ~ 2 pa + 2 2^ P/3a X/3• (490)
dxa 0=1

Substituting Eqs. (487M490) in Eq. (486), and assuming that p^7 = p7/3
leads to

n n

(P + F) - 2 22 Pat + Fa + 22 P olol “I”

01=1 a=i

+ 2Z Z P/37+k F/37+ 21 Xj3 X7 = 0 (491)

/3=1 7=1

The quantities F, Fa, and F^7 depend only on the components of the
matrices A, B, Q, R, and G. In order for Eq. (491) to be satisfied, each of
692 ELEMENTS OF MODERN CONTROL THEORY

the coefficients on the left-hand side of this equation must vanish

independently. This leads to

- P = F (492)

- Pa = Fa (493)

~ P/3-y
= \ (%
F|37
+ F7jSJ (494)

Yj 1)2, 1 • * *, n

Equations (492)-(494) are a set of [1/2 n(n — 1) + 2n + 1] differential

equations for determining the various p’s. Noting that

W(x, t/) = 0

via Eq. (485), we find, for the initial conditions

p(t/) = pa(t/) = P07(t/) = 0 . (495)

Having calculated the p’s, the optimal control is obtained directly

from Eq. (487), making use of Eq. (490).
We now consider in detail the four problems stated at the beginning
of this section.

Problem 1:

In order to investigate the possibility of stabilizing a flexible launch

vehicle using the theory developed here, we will consider the system
whose dynamics are expressed by

m Uo (a — 9) = Tc 8 — Laa — (mg cos do) 6 (496)

9 = iic 8 + y.a a (497)

qi + 2£i qi + coi2 qi = — —— 5 . (498)

Mi
OPTIMAL CONTROL 693

The symbols have the following meaning.*

g = gravity acceleration
I„ = moment of inertia of vehicle about pitch axis
lc = distance from mass center of vehicle to engine swivel point
(a = distance from mass center of vehicle to center of pressure
La = aerodynamic load per unit angle of attack
m = mass of vehicle '
Mi = generalized mass of first bending mode
qi = generalized displacement of first bending mode
T0 = control thrust
U0 = forward velocity of vehicle
a = angle of attack
5 = thrust angle deflection
6 = attitude angle
dD = steady-state attitude angle
Me = T/Tc I„

or

= relative damping factor for first bending mode

wi = undamped natural frequency of first bending mode

Conventional instrumentation whose output is a linear combination of

the state variables is described as

Rate Gyro:

duo = K b{6 + <T

0<l) qi) (499)

Position Gyro:

dPa — (6 -T qi) Ivp (500)

Accelerometer:

(501)

Angle of A ttack Sensor:

Qa - K«(« — aaa) 9i) • (502)

See Volume Two for a more complete account of this problem.

 ELEMENTS OF MODERN CONTROL THEORY
694

Here, KS) K0, Ka, and Kp are the gains associated with the respective
sensors; aT is the thrust acceleration, and the a(1) quantities are the
normalized bending mode slopes whose subscripts indicate the location
of the sensor along the vehicle.
If we define

xi = 0 x4 = qi

x2 = d x5 = qi (503)

Xg = a u = 5

then the system described by Eqs. (496)-(498) may be expressed as

x = Ax + Bu

with

0 1 0 0 0
0 0 Ma 0 0
A = g COS0o/Uo 1 —La/mU0 0 0
0 0 0 0 1

0 0 0 —coi2 — |iWi

0
Me
B = Tc/mU0
0

—T0/Mi_

This is equivalent to the system, Eqs. (468) and (469), with G = 1, the
unit matrix.
Taking a performance index of the form in Eq. (470), the optimal
control is given by Eq. (471), viz.

u*(t) = - Kx(t) . (504)

Note that, in the present case, R is a scalar, since u is a scalar and K in

the above expression is a 1 X n matrix (i.e., a row vector). The control
is optimal in the sense that the performance index, Eq. (470), is mini¬
mized; this criterion has not as yet been directly related to control
OPTIMAL CONTROL 695

system requirements, such as response speed, overshoot, and damping.

These may indeed be determined once K is calculated, since the closed-
loop poles of the system are then directly obtainable.
Obviously, Iv is a function of Q and R, and, therefore, the properties
of the “optimal system” are directly (though not simply) related to the
choice of Q and R. In Fisher’s paper, (14> R is taken as unity, and an
iterative process that will lead to a system of acceptable dynamic
qualities is performed on Q (starting with some arbitrary choice).
Basically, one guesses Q, determines K, and then calculates the closed-
loop poles of the resulting system. If this does not yield an acceptable
system from the point of view of dynamic response properties, the process
is repeated with a new choice of Q, etc. The method is completely cut-
and-try. Fisher gives no technique whereby successive iteration converges
to some desired form. Previous experience no doubt is a valuable guide
in obtaining meaningful results.
A basic property of the optimal control, Eq. (504), is that there is one
feedback loop for each control variable. It is known, however, that
acceptable systems may be designed with only one or two feedback loops.
Of course, the more feedback loops employed, the greater the capability
of achieving specified performance. It is instructive, therefore, to investi¬
gate the means whereby an nth order system may approximate the opti¬
mal control, Eq. (504), with fewer than n feedback loops. Let us assume
that m sensors are used and that the output of each sensor is given by

71

(505)

1, 2 m

We note, for example, that two rate gyros may be viewed as two
different sensors if their KB and <j0(1)are different. Thus, for the system,
Eqs. (496H498), the use of one position gyro and two rate gyros and
two accelerometers (having different gains) will provide five independent
measurements that are linear combinations of all the state variables.
If now we use

u' = Ec <zi (506)

i= 1

to denote the control function employing the sensors (the c* are as yet
undetermined constants), then Eqs. (504) and (506) will be equivalent if
696 ELEMENTS OF MODERN CONTROL THEORY

n m

22 k» x, = 22 c» z» 22 c» 22 x, . (507)
»=1 i— 1 1 ?•=!

This leads to

brc = -K «% (508)

b = m X n matrix

c = m X 1 matrix (i.e., m vector)

K = 1 X n matrix (i.e., n-dimensional row vector).

If m = n, then the components of c are uniquely determined from

Eq. (508). If m < n, then there is no unique solution for c. However,
for an approximate solution, we may seek the value of c that minimizes
the error criterion

E = ||bT c + I\T||2 = (brc + KT)r(brc + Kr) . (509)

This is easily found to be

— = 2(bbr c + bKr) = 0
dc

or

bbTc = -bKr (510)

The matrix equation, (510), is a system of m linear equations for the

determination of the m components of c.
It should be noted that the approximate solution will not be optimal
and that stability is not guaranteed. Thus, each particular solution must
be investigated individually to determine if performance is acceptable.

Problem 2:

The given system is again described by

x = Ax + Bu (511)

y = Gx . (512)
OPTIMAL CONTROL 697

It is desired that the system behave as the “model”

i = Ltj . (513)

In order to achieve this, we formulate the performance criterion as

J
/ 7[(y- Ly
)*%
Qtf- Ly)
+urRuJ
dt. (514)

Presumably, an appropriate selection of the weighting matrices, Q and

R, will yield the desired result. An analysis completely identical with
that of Sect. 5.2.1 shows that, in the present case, the optimal control
has the form

u(t) = - R-1 BT [P + GrQ (GA - LG)] x (515)

where P is the solution of the steady-state form of the matrix Riccati

equation

PA + ArP + 6rQG - PBR_1BrP = 0 (516)

and

R = Br Gr QGB + R (517)

A = A - BR-1 BT GT Q(GA - LG) (5 18)

d = Q - QGBR-1 Br Gr Q (519)

6 = GA - LG . (520)

It is convenient to write Eq. (5 15) in the form

u(t) = - (Kr + Km) x = - Kx (521)

where

Kr = R-1 BrP (522)

Km = R-1 Br Gr Q(GA - LG) . (523)

698 ELEMENTS OF MODERN CONTROL THEORY

The control is thus expressed as a sum of two terms, one of which

depends on the solution of the Riccati equation, and the other of which
depends on the properties of the model.
We now seek to obtain some insight into this result. More specifically,
what must be the stipulations on Q and R in order to have the given
system exhibit the dynamic properties of the model? For this purpose,
we consider the second order system described by

" 0 1 ~0
A = B =
_1 1
<1 1

1
B21 _

1 0
G =
0 1

with weighting matrices

Qn 0
Q = R = R11, a scalar
0 Q22

and the matrix describing the model

0 1
L =
— L21 — L2:

We then find

R11 — Q22B212T" R11

An = 0 A12 = 1

A21 B212Q22 (L21 — A21)

— A21 —
B212Q22 + Ru

A22 — — A21 —
B212Q22 (L22 — A22)
B212Q22 + Ru
OPTIMAL CONTROL 699

\w Q222
22
B212Q22 + R11

Qn — Qn

B21 Q 22
K, [(L21 — A21)(L22 — A22)]
B212Q22 + R11

The determination of Kr depends on the components of the P matrix,

which are found from Eq. (5 16), as follows.

P21 — P12 — -^21

R11, ["/A21
R21
\
b2i2 L\ b212/
1/2
Q22 R11
+
B2i2

A.22R11 A22 R11 \ . 2P21 Rn

P22 —
B212 L\ B212) B212
* * -i1/2
+ (La _ A*)>
B212 J

It is not necessary to calculate Pn, since u(t) depends on the product

of Br and P in Eq. (522), and the Bn element that would multiply P11
is zero.
Now, since we are interested primarily in minimizing the “error” term
(y — Ly) in Eq. (514), it is logical to take Q»R. In the present case,
it is sufficient to let Q22» Rn. This leads to

K„

while

Q22—»0

which means that

 ELEMENTS OF MODERN CONTROL THEORY
700

and, in turn

Kr re 0 .

Consequently, the optimal control reduces to

u(t) re — Kmx

Xl

' si [(L,‘
_A“)(L>'
_A”)] x2

(524)

=—
—[(l21
—A2i^Xi
+(l22
—A22^x2J
Substituting this in Eq. (511)

x = Ax + Bu

= (A - BKm)x

But

0 1 0
(A - BKm) = [(L21 — A2i)(L22 — A 22)]
—a22
B2i B21
- A2i

0 1

%L2i l2

Thus, the optimal system behaves exactly as the model. It is instruc¬

tive to examine these results in terms of conventional transfer functions
and feedback loops.
Figure 5.14 depicts the feedback loops introduced by the optimal
control function, Eq. (524). The system with the feedback loops, shown
in Part (a), reduces to the form shown in Part (b) after some elementary
simplifications. The distinctive feature of this result is that near-infinite
gains are not required to make the given plant behave as some prescribed
model. In the conventional model following scheme, shown in Fig. 5.15,
the overall transfer function is
OPTIMAL CONTROL 701

PLANT

(a)

(b)

Figure 5.14. A Model Following Optimal Control System.

PLANT

Figure 5.15. Conventional Model Following System.

c(s) _ G(s) [1 + KM (s)]

R(s) 1 + KG(s)
702 ELEMENTS OF MODERN CONTROL THEORY

+ G(s) M(s)

from which it is apparent that

M(s) for K -»
R(s)

Thus, the scheme generated by optimal control theory appears ex¬

tremely attractive. In practice, however, there are two fundamental limi¬
tations. First of all, there is no guarantee that response to external
disturbances is acceptable. Second, all the state variables must be accessi¬
ble for measurement. For moderately high order systems, the latter
condition may be difficult to realize. Nevertheless, the above methods
are a novel approach to an outstanding problem and may prove useful
in certain applications.

Problem 3:

Instead of including the model directly in the performance index, one

may attempt to match the system output to the model output. This
leads to a performance index of the form of Eq. (484), which is repeated
below.

r‘f
J = J [(yo - y)r Q(y*>- y) + (u*, - u)r R(uj> - u)] dt (525)

The distinction between this and the types previously considered is

that the upper limit of integration, tf, is finite. Consequently, it is
anticipated that the optimal control function will contain time varying
gains. Bailey(122) applied Merriam’s technique to the problem of opti¬
mizing the performance index in Eq. (525) for the yaw dynamics of a
launch vehicle whose motion is described by

(526)
OPTIMAL CONTROL 703

(527)

Here

I* = moment of inertia of vehicle about yaw axis

tc — distance from mass center of vehicle to engine swivel point
(p = distance from mass center of vehicle to center of pressure
Ljg = aerodynamic load per unit angle'of attack
m = mass of vehicle
Tc = control thrust
Uo = vehicle forward velocity
Y = normal displacement measured parallel to an inertial reference
8 = control engine deflection
\p = yaw attitude angle.
Via the definitions

xi = Y
x2 = Y

x8 = ^
x4 = i
u = 5
The pertinent system matrices become

1
(Tc + Lp)
m Uo m

1 0 0 0
A. =

hJi o o
h Uo
0 0 1

T,
m

0
B =

Tc 4
I,
0
704 ELEMENTS OF MODERN CONTROL THEORY

G = I, the unit matrix.

The optimal control in this case is given by Eq. (487), which, in

combination with Eq. (490) and the above value of B, reduces to

u = u0 ———F-(pi+ 23P0iX0
'j + -r( p34-23P^3
X(3>)1
(528)
RuLm\ 0=1 / lz\ 0=1 / J
where R = Ru is a scalar, since u is a scalar. The p’s are obtained from
Eqs. (492)-(494), which in the present case take the form*

• B 2
~~%Pl = Qll Y o 4~ All Pl + P2 + A31 P3 — —— Pl P11
Kn
(529)
B31Bn ( , \ B312
~ —- - 1 Pll P3 4“ Pl Pl3 ) — —- Pl3 P3
R11 \ / Rn

_ a — n v _ ®n2 „ B31Bn ( ^
P2 — Li22 J-D — p —‘ Pl Pl2 — - p - ( Pl2 P3 4“ P23 Pl J

(530)
B3i2
P23 P3
Rn

—P3—Q33
\pD
4*P4— Bn2
PlPl3—^31 ( P3 P13 4* Pl P; 3)
R11 Rn \
(531)
Bn2
P3 P33
Rn

B 2
P>4= Q44 'pD 4" Pl A14 4- P3 A34 - 2- pj p14
Rn
(532)
B31 Bn / . \ B312
r; - l P3 Pl4 4- Pl P34 1 — —- P3 P34
R11 \ / Rn

P11 — Q11 4- 2 An Pn + 2 pn 4- 2 A31 P13 — Dn2

(533)
2 B31 Bn B3i2 _ ,
- - - P11 P13 ~ —— P132
rtn R11

*WerecaUthat paj3 = p^.

OPTIMAL CONTROL 705

—Pli = All Pia + P22 + A31 p23 — — P12 Pn

Ru
(534)
B31 Bu /
P12 P13 + P23 Pi Pl3 P23
Rn \ / Ru

B 2
Pl3 = An P13 + P23 + A31 P33 + P14 — - Pn P13
' R11
(535)
B31 Bn
Ru
+ Pn P33 ) B^
Rn
P33

B 2
— P14 = All Pl4 + P24 + A31 P34 + Al4 Pll + A34 P13 — —— Pn Pl4
Rn
(536)
B31 Bn / \ B312
Pl4 Pl3 + Pll P34 P13 P34
Ru V / Ru

Bn2 2 B31 Bn
— P22 — Q22 — —— Pl22 — P12 P23 — •" - P23 (537)
Kn R11 R11

. _ Bn2 B31 Bn / , \
P23 — P24 — —— P12 P13 — ——- I P13 P23 + P12 P33 1
R11 R11 V /
(538)
>31
P23 P33
Ri

B31I Bn/
P24 = A14 P12 + A34 P23 — —— P12 Pl4
Bn2
Rn R II V
P23 Pl4 + Pl2 P34 )
(539)
P23 P34
Ri

— f>33— Q33 + 2 P34 Pl3"

2 Bsi Bi
Pl3 P33 —
B^ P332 (540)
Ri R11

—P34 — A14 P13 -f- A34 A33 + P44 — —- Pl3 Pl4

R11
(541)
_ B31Bn / \ B312
Rn V
P13 P34 + P33 P
U) Rn P33 P34
706 ELEMENTS OF MODERN CONTROL THEORY

Bn2
— P44 = Q44 + 2 Aj4 Pl4 + 2 A34 P34 Pir
Ri
(542)
2 B31 Bn
Pl4 P34
B312
d 2
R11 TrTP34
The weighting matrix Q has been assumed diagonal.

Q11 0
Qa
Q = Q33
Q44_

and the subscripted A and B terms are the components of the respective
matrices.
The set of Eqs. (530M542), when solved with the initial conditions

Pa(t/) = Pa0(t/) = 0
(543)
«, 0 = 1, 2, 3, 4

yield the p’s which, when substituted in Eq. (528), give a control function
of the form

u(t) — uD(t) T K^.(t)ip -f K*(t)£ -f- Kr(t)Y -(- Ky(t)Y • (544)

Baileyt122) gives some results for a specific selection of Ruand theQ.i

that represents a compromise between the desirability for “tight” control
and minimum drift. The superior results obtained by using time varying
gains in the feedback loops must be weighed against the added complexity
of the control system. Furthermore, the influence of the neglected higher
order dynamic effects, nonlinearities, and parameter uncertainties, in
addition to extraneous disturbances, remains to be evaluated.

Problem 4:

We again consider the system

x = Ax + Bu (545)

y = Gx (546)
OPTIMAL CONTROL 707

with the performance criterion

J = lim[ ' (xTGTQGx

+ urRu)dt . (547)
The optimal control has been found to be*

u(t) = - Kx(t) (548)

where K is a constant matrix. Substituting this in Eq. (545), we obtain

the equation of the optimal system

x = (A - BK)x . (549)

We seek to determine the relationship between the components of K

(feedback gains) and the components of the weighting matrices, Q and
R.
An alternative derivation has shown that the optimal system and its
adjoint satisfy^

A - BR-1 Br
(550)
Gr QG - Ar

The characteristic equation of the optimal system, together with its

adjoint, may, therefore, be obtained from Eq. (549) as

|ls - (A - BK) | -| Is - (A + BK)| = 0 (551)

or, from Eq. (550), as

Is - A BR-1 Br
= 0 . (552)
GT QG Is + AT

Since Eqs. (551) and (552) are the characteristic equations for the
same system, they must have the same roots. Therefore, by equating
coefficients of like powers of s, we obtain a set of equations that relates

*See Eq. (471).

fSee Eq. (480).
708 ELEMENTS OF MODERN CONTROL THEORY

the components of K to the components of Q and R. However, this

brute force approach soon runs into an avalanche of complicated algebra
that yields little insight and no general results.
Tyler and TuteurG3) show that if Q is a diagonal matrix and R is taken
as the unit matrix, then for large Q a, the optimal system exhibits the
properties of a Butterworth function. Their design approach involves the
expansion of the determinant equation, (552), which is the characteristic
equation of the optimal system, together with its adjoint. A root locus
is drawn for a specific Qti as the “system gain” and with all other Q;i set
equal to zero. After a Q matrix is determined in this fashion, the calcu¬
lation of the Iv matrix is straightforward.
Rather than explore this technique in detail, we will indicate a simpler
and more elegant approach that affords a higher degree of insight. This is
based on a crucial simplification of the characteristic Eq. (552), which
will now be developed. To do this, we will make use of the following
two relations between determinants.

= |a| • |5 — y a 1 /31 (553)

7

Mn — 7/31 = nn m |mIm— 0y\ (554)

where

a = m X m matrix (nonsingular)
P = m X n matrix
7 = n X m matrix
8 = n X n matrix
M = scalar
I< = i X i unit matrix.

The first of these is proven in Bodewig’s book!1 72) (p. 217), and the
second is due to Plotkin.073) Applying Eq. (553) to (552) yields

|s I, - A| • | (s I„ + Ar) - Gr QG (s I, - A)-' BR-1 Br| = 0 . (555)

But | s I„ Aj = 0 only for those values of s that correspond to the

open-loop poles. Consequently, the characteristic equation for the
closed-loop (optimal) system reduces to

| (s I„ + Ar) - Gr QG (s I* - A)-1 BR-1 Br| = 0. (556)

OPTIMAL CONTROL 709

Furthermore

(s I„ + AT) - GT QG (s I, - A)-1 BR'1 Br

= (s In + AT) [I -(8 1, + Ar)-1 GT QG (s I, - A)-1 BR~> Br]

which means that

I s In + Ar| • |I„ - (s I, + Ar)_1 Gr QG (s I„ - A)-1 BR-> Br| = 0 .

Now since [s I„ + Ar] = 0 only for those values of s that correspond

to the open-loop poles of the adjoint system, Eq. (556) is further
reduced to

11» - (s In + AT)-! Gr QG (s In - A)-1 BR'1 Br| = 0 . (557)

An application of Eq. (554) yields

|Im - R-1 Br (s I„ + Ar)-‘ Gr QG (s In - A)'1 B| = 0 . (558)

We now observe that the quantity

L(s) = G(Is - A)-1 B (559)

is the matrix transfer function* for the system, Eqs. (545) and (546).
Also

Lr (~s) = —Br(Is + A')-1 Gt . (560)

Therefore, Eq. (558) becomes

|I* + R-1 Lr (-s) QL(s)| = 0 . (561)

This is, in fact, a multidimensional form of Chang’s Root Square

Locus Method. (t) In principle, one could plot the loci for Q«/R» as a
parameter and determine the Q and R that yield acceptable dynamic
response. From this, the gain matrix K, and hence the optimal system,
could be determined.

*See Chapter One, Sect. 1.4.

 ELEMENTS OF MODERN CONTROL THEORY
710

Consider, for example, the single-input, single-output system for which

G = 1 X n matrix
B = n X 1 matrix
Q = Qn = scalar
R == Rn = scalar.

Then Eq. (561) takes the form

1 + Ql1L(—s) L(s) = 0 (562)

Rn

where

Y(s)_ TM ~ N(s) _
kfr
M (3~8J)
(563)
U(s) D(s) fr ( 8“ P»)

r < n

and z, and p< are the zeros and poles, respectively, of the open-loop
system, with k a known factor.
Defining

a = - s2

Mr = - z/

Vi = - Pi2

we express Eq. (562) as

0 = 1 +
k2 Qn U (a~w) = T(s) T ( —s) . (564)
Rn
n y®- Vi'j
Thus, standard root locus techniques can be used with k2 Qn/Rn as
the variable gain. A given value of this variable gain yields pairs of roots
12,, each pair representing a point in the s plane, together with its mirror
OPTIMAL CONTROL 711

image about the imaginary axis; i.e., a closed-loop pole for the optimal
and adjoint system. If these closed-loop roots are acceptable from a
dynamic response point of view, then the corresponding Qu and Ru are
used to determine the optimal gain matrix, K.
Note that the optimal system contains only those roots in the left-hand
s plane. It can be shown that the optimal system obtained by this pro¬
cedure is always stable. Therefore, the mirror image poles belong to the
adjoint of the optimal system. '
In simple situations, the components of K may be directly related to
Q11/R11. For the second order case described by

" Xi 0 1' Xi " 0 '

—

+
_ X2 _ _ ~b —a _ _ x2 _ _ c _

Xi

y = [1 0]
X*

we obtain, after equating Eq. (551) to (552)

[s2 + s(a + c K2) + (b + c Ki)] [s2 — s(a + c K2) + (b + c Ki)]

= (s2 + as + b)(s2 — as + b) + ^ c2 .
Ru

Equating coefficients of like powers of s and solving for Ki and Iv2 yields

Ki = b i/i + Or <r

c c V + Rub2

K2 = - a , ./a* + 2-Ki
c y c2 c
K = [Ki K*].

Thus, because the relationships between the dynamic characteristics

(such as closed-loop frequency and damping ratio) and the feedback
gains are known, these can be related directly to the components of the
Q and R matrices.
712 ELEMENTS OF MODERN CONTROL THEORY

5.3.3 Optimal Reentry From Orbit

The guidance and control problem for a manned spacecraft during

the atmospheric re-entry phase must take account of two primary
figures of merit: the surface heating rate and the accelerations exper¬
ienced by the crew. To formulate the problem mathematically, the
vehicle is assumed to be a point mass moving about a spherical, non¬
rotating earth with an inertial coordinate frame as shown in Fig. 5.16.
The motion is described by

h = — V -sin 7 (565)

v,
V = g sin 7 -
D (566)
m

g cos 7 V cos t L
7 = - V - (567)
te -T h mV

Figure 5.16. Coordinate System for Re-entry Problem.

OPTIMAL CONTROL 713

where

D = drag force
g = gravity acceleration
h = altitude above earth’s surface
L = lift force
m = mass of vehicle
rE = radius of earth
V = velocity of vehicle
7 = flight path angle.

The rate of heating due to atmospheric friction is given by

(568)

where K4 is a heating constant and p is the atmospheric density. The

expression (568) represents the heating rate per unit surface area, so
that, for a particular vehicle, Iv4 is a function of the surface area of
the vehicle nose region.
The acceleration sensed by the crew is due only to the aerodynamic
forces and is given by

(L2 + D2)1'2
(569)
m

The limit of human endurance is a function of the acceleration magni¬

tude and the length of time applied. Within the range of 5 to lOg’s,
this endurance limit is roughly a linear function of the acceleration
squared.
In view of these observations, a reasonable measure of performance
may be expressed as

(570)

where K7 is a relative weighting constant between the heating and

acceleration effects. It is convenient to define the additional state
variables
714 ELEMENTS OF MODERN CONTROL THEORY

r‘ (v + d2) (572)
X6
Jt m2

with

xi(t<) = 0 (573)

xs(t<) = 0 (574)

and add the equations

x* = K4 p1'2 Vs (575)

.6 L2 4- D2 (576)
m2

to the system, Eqs. (565H567).

The performance criterion becomes, therefore

J = X4(t/) + K7 x5(t/) (577)

For notational convenience, we define also

X! = h rs = Ki

X to II g = Ka

II
-> m = K3

We approximate the atmospheric density by the exponential model

P = Kr e*6*1 (578)

where K6 and K( are constants.

The aerodynamic forces are expressed as

(579)
L = — p X22 K10 Cl

D = — p Xa" K10 CB
(580)
OPTIMAL CONTROL 715

where Ki0 is a reference area and Ct and CD are the lift and drag coeffi¬
cients. The latter are approximated by

Ci = Kn sin u cos u (581)

CD = Ku + K1S sin2 u (582)

where Kn, Ki2, and Kis are appropriate /constants for the lift drag polar
and u is the angle of attack, which is taken as the control variable.
Combining all the above relations, the state equations for the system
become

Xi = — x2 sin x3 = h (583)

x2 = K2 sin x3 — Kiqex6ll 12 + K13 sin2 (584)

K3

cos x3 x2 cos x8
x3 K>
x2 Kj xi

_ K62Kiq Kn e KeXj
6 1 x2 sin u cos u = fa (585)
K3

±4 = K4 K6 e^®11 x23 = f4 (586)

^(K’gK“!)e“-.x,<[K„.,in,u cos2 u -f Ki:

+2Ki2
Knsin2
u+Ki32
sin4
uJ=f6 (587)

The problem of optimal control is now formulated as follows.

“Given the system described by Eqs. (583M587), calculate the angle
of attack history, u(t), that will minimize the function, Eq. (577).”
The boundary conditions are

xi(t.) = h0 xx(t/) = h /
x2(t.) = Vo x2(t/) = V/
x3(ti) = To Xg(t/) = y, (588)
x4(t.) = 0
x6(ti) = 0
716 ELEMENTS OF MODERN CONTROL THEORY

This problem was analyzed by Payne,!1 67) using the maximum princi¬
ple. We form the hamiltonian

H = Z Xi ft (589)
l

where the X<satisfy*

K6’Iv, Kiq^
Xl — X;
[( k3 ^eK&Xl
x22
^Ki2
+K13
sin2
u)]
f x2cos x3 / K62K6 Kio Kn \^ e216*1
Ki x2
K3
sin u cos u
]
-[( k4 k6 k6
)ei I Ktxl x^3
]
-,[( 2 K64 K6 K 7 -^^-10
k3
-
j 2^ga:i4 ( 17
1 e e Ax24 I Kn
2*9
sin2 u
9
cos2 u

+ K12 + 2 Ki2 Ki3 sin2 u + sin4 u

)] . (590)

X2 = Xi sin x3
+X22 Kl°
) e*6*1
x2
L\
(Kl2
+K13
sin2
u)JKa /

K2 cos x3 cos x3 , (/ K52

K52Iv10
K; \
K10Kii Kx
% TX3
x22 Ki + x4 T-^ - —- J e 6 sinu cosu ]
[
X4 3 K4 K5 tf^x,1 ]
(K„2
sir
+ K12 + 2 Ki2 Ki3 sin2 u K132 sin4
')] (591)

X3 — X] x2 cos x3 — X2 K2 cos x3 X;
[ K22 si
sin

x2
x3

(592)
x2 sin x3 "1
" Ki+ XlJ
See Sect. 5.1.2.
OPTIMAL CONTROL 717

X4 = 0 (593)

X6 = 0. (594)

In order to minimize J, Eq. (577), we must maximize H, Eq. (598),

with respect to u. If u is not bounded, it may be obtained from

dH (x, X, u)'
(595)
du

However, a brief examination of the system, Eqs. (583)-(594), indi¬

cates that this is not a simple task. Thus, it is necessary to use some
iterative procedure to determine u such that H is a maximum at all
points along the trajectory. This may be done, for example, by com¬
puting the optimal u such that the relation

Max H (x, X, u)
(596)
u

is satisfied for all t. It is sufficient to let u assume a range of discrete

values and calculate u by direct search. It is necessary simultaneously to
solve the two point boundary value problem represented by the ten
equations, (583M587) and (590M594). Eight boundary conditions are
specified by Eqs. (588), and the remaining two are given by*

X4(t,) = - 1 (597)

X5(t,)=— K7. (598)

Payne!167) uses a variant of the Neighboring Optimum method!119)

to solve this system. The optimal trajectory thus obtained is shown in
Figs. 5.17-5.21, which depict the time histories of each of the state
variables. The time history of the optimal control function is shown in
Fig. 5.22. In each case, the solid line represents the condition K7 = 0,
while the dashed line is the case for K7 = 200. The most pronounced
effect of including the acceleration in the criterion function is shown in
Fig. 5.21, which indicates a reduction of about 25 percent in the g
forces by using the weighting factor of Iv7 = 200. The heating rate (Fig.
5.20) is thereby increased, but not significantly.

See Eq. (85) and the discussion in Sect. 5.1.2.

718 ELEMENTS OF MODERN CONTROL THEORY

feet/sec)
feet)
of
(thousands
ALTITUDE
VELOCITY

Figure 5.18. Optimal Trajectories, Time-Velocity.

OPTIMAL CONTROL 719

(deg)
ANGLE
FLIGHT
-ft/sec)
HEATING
(lb
RATE

Figure 5.19. Optimal Trajectories, Time-Flight Angle.

Figure 5.20. Optimal Trajectories, Heating Rate.

720 ELEMENTS OF MODERN CONTROL THEORY

(g’s)
ACC
(deg
CON
Figure 5.22. Optimal Trajectories, Control Function.
OPTIMAL CONTROL 721

Figure 5.23 shows how the total heat and acceleration effects along
the optimal trajectory vary with parameter K7. This plot can be used to
determine the minimum amount of additional heating that the vehicle
must absorb in order to achieve a specified reduction in acceleration
effects. For example, to reduce the acceleration as shown in Fig. 5.21,
it is necessary that the vehicle absorb about 10 percent additional heat
during the re-entry maneuver.
The basic data for the problem is given below.

Kj = 2.09 X 107 ft
K2 = 32.2 ft/sec2
K3 = 250 lb sec2/ft
Iv4 = 1.0 X 10-4 (lb)1/2 sec
Kj = 0.052 (lb)1/2 sec/ft2
K* = -4.26 X 10-6 ft-1
K10 = 66.5 ft2
Ku — 1.2
Ki2 = 0.274
K13 = 1.8

Figure 5.23. Optimal Tradeoff Heat-Acceleration Effects.

722 ELEMENTS OF MODERN CONTROL THEORY

Initial conditions

Xl(ti) 400,000 ft

x2(t.) 36,000 ft/sec

x3(t.) 8.09 deg.

Final conditions

Xl(t/) 250,000 ft

X2(t/) 27,000 ft/sec

Xa(t/) 0 deg.
 APPENDIX A

COMPUTATIONAL SOLUTION OF TWO POINT

BOUNDARY VALUE PROBLEMS

As noted repeatedly in this chapter, the solution of an optimization

problem by variational methods or the maximum principle leads to the
requirement for solving a set of differential equations with two point
boundary conditions. This requirement is indeed one of the main
limitations of the classical approach, since the solution of boundary
value problems is a formidable computational task. Various solutions
have been obtained in special cases for particular problems,* but a reas¬
onably general theory was not available until fairly recently.
The basic idea in the method to be described here is that of “quasi¬
linearization,”!145) in which a nonlinear problem is transformed to a
sequence of linear ones by means of a “generalized Newton-Raphson
operator. ”<149) As the name implies, the method is closely akin to the
Newton-Raphson technique for obtaining the roots of transcendental
equations by successively replacing the arcs of a curve by its tangents.
Since the solution of the nonlinear problem is reduced to solving its
linear equivalent, the latter will be considered first.

Al. THE LINEAR CASE

Given is the matrix vector differential equation

y = A(t)y + h(t) (Al)

where y (t) and h(t) are n vectors and A(t) is an n X n matrix. It is required
to determine y(t) in the interval (0, t/) such that the boundary conditions

y>(0) = a j (A2)
j = 1, 2, — , r

*Cf. Refs. 64, 77, 110, 113, and 120.

723
724 ELEMENTS OF MODERN CONTROL THEORY

y>(t/) = by (A3)
j = r + 1, n

are satisfied.
A simple and direct method for obtaining y(t) is the following. (14?)
Integrate the homogeneous version of Eq. (Al)

u = A(t) u (A4)

(n — r) times. The initial conditions at the mth time are

u,(m)(0) = 0 j ^ r + m
(A5)
= 1 j = r -f m .

This yields the (n — r) vectors

u<m)(t) , 0 < t < t/ (A6)

m = 1, 2, • • •, (n - r)

Now integrate the equation

v = A(t)v + h(t) (A7)

using as initial conditions

Vy(0) = ay , j = 1, 2, •••, r
(A8)
= o, j = (r + 1), • • •, n .

Call this solution

v(t) , 0 < t < t/ . (A9)

The general solution of Eq. (Al) is then given by

y/(t) = ICP Uy<»>(t)+ v y(t) (A 10)

0=1

where the Cp are (scalar) constants determined by solving the (n - r)

linear algebraic equations
OPTIMAL CONTROL 725

by — 22 Cp u3(j>)(t/)-f v;(t/) (All)

p=i

j = (r + 1), • • •, n

The distinctive feature of this method is that all integrations are per¬
formed with initial conditions; it is not necessary to “guess” at final
values. The solution is exact (within roundoff error) after a finite
number of operations.

A2. THE NONLINEAR CASE

The vector equation to be solved now takes the form

x = f(x, t) (A1 2)

where x and f are n vectors, and the boundary conditions are given by

x/(0) = a, (A 13)
j = 1, 2, •••, r

Xy(t/) = by (A 14)
j = (r + 1), •••, n .

Let x(0)(t) be an initial guess for the solution of Eq. (A12). Successive
improvements are obtained from

£(H-D = f<*>(x(*+1> - x<*>) + f<*> (A15)

where f (*>==f(x<*>, t), and F(4> is the Jacobian matrix whose ijth com¬
ponent is given by

t) (A 16)
dXy

Equation (A 15) may also be written as

X<*+1>= F(*) x(4+l) + w(.k) (A17)

where

w(*> = f (* > _ F(*) x(t> _ (A18)

726 ELEMENTS OF MODERN CONTROL THEORY

Equation (A1 7) is a linear equation, which, together with the boundary

conditions, (A 13) and (A 14), is completely analogous to the system
(A1)-(A3). It may, therefore, be solved by the methods of the previous
section.
Note that the conventional Newton-Raphson method solves the scalar
equation g(x) = 0 via

g(xt)
Xt+l = Xi (A 19)
g'(Xi)

x0 = initial guess.

Writing Eq. (A 19) in the form

0 = g'(xi) [xi+i - x<] + g(xd (A20)

the analogy with Eq. (A15) is evident. The generalized Newton-Raphson

approach is conceptually identical to the scalar case in that a curve is
successively replaced by tangent lines. In other words, the nonlinear
problem is replaced by a sequence of linear problems.
A suitable “stopping” criterion is given by

|x(*+o_ x<*)|< e (A21)

where e is a predetermined error vector.
A relevant question at this point is: “What are the conditions that
ensure that this procedure converges to the solution of Eq. (A 12)?”
It has been shown(149465) that sufficient conditions for convergence
are:

1. fy(x, t) is strictly convex* for all t in the interval (0, t/).

2. 3f,(x, t )/dXj > 0 wheni ^ j.

*A function <p(x) is strictly convex in an interval (a,b) if, for xi, x2, X, with a < xi < x2< b
and 0 < X < li, we have

v>[Xxi— (1 — X)x2] < X<p(xi) -f (1 — X)v>(x2).

If <?" (x) exists, then p(x) is strictly convex if <?" (x) > 0.
OPTIMAL CONTROL 727

We emphasize the word “sufficient,” since these conditions may not

be necessary. This procedure has been found to converge, for example,
when none of the f/x) were convex. Each case must, therefore, be
investigated independently and convergence checked by a criterion of
the type (A21) unless the aforementioned conditions are satisfied before¬
hand. It is worthy of note that convergence (when it occurs) is quadratic;
that is, the number of correct digits approximately doubles at each
iteration. Furthermore, this convergence is monotonic, which means
that accurate solutions are obtained with relatively few iterations, even
with poor initial guesses.
Thus far, we have assumed that the final time, tf, was known before¬
hand. It sometimes happens that t/ is not known initially, and this
requires a modified treatment. One approach is described in Ref. 145,
which, however, entails numerical differentiation— a procedure to be
avoided if possible. A superior method, due to Long, (t 66) is the following.
Introduce a new independent variable, s, defined by

t = as

where a is a constant to be determined. The new end points are taken

as s = 0 and s = 1. Therefore, once a is determined, the value of tf is given
by tf = a.
Suppose that a typical equation of the system (A12) is x, = f,(x, t).
If we write

dx
= x
ds

then we have

Xj = af ,(x, as) .

The parameter a is treated as an additional state variable by writing

a' = 0 .

Thus, the new state vector of the system is of dimension (n + 1), and
we proceed as before. Note that one makes an initial guess on a (in x(0)(t)).
This is, therefore, constant during each iteration, but it varies from one
iteration to the next. Thus the determination of tf becomes an integral
part of the quasilinearization procedure.
 APPENDIX B

THE VARIATION OPERATOR

Consider the function

F = F(y,y, t) (Bl)

where the dot denotes derivative with respect to t, the independent

variable. For a fixed value of t, F depends only on y and y.
Suppose now that y is replaced by Y, where

Y = y + ei](t)
(B2)
e = a constant.

The change er?(t) in y(t) is called the variation of y(t) and is denoted by 5y

5y = e)j(t) . (B3)

We may further define the variation of y(t) as

5y = Y - y (B4)

which, by virtue of Eq. (B2), becomes

5y = eh • (B5)

However, from Eq. (B3)

it(Sy>
= % (B6)

Comparing Eqs. (B5) and (B6), we find

(B7)

In other words, the operators 8 and d/dt are commutative.

728
OPTIMAL CONTROL 729

With y replaced by Y of Eq. (B2), the change in F may be expressed as

AF = F(y + tv, y + erj, t) - F(y, y, t) .

Taking a Taylor expansion in which only linear terms are retained, this
reduces to

dF dF
AF = eri + eil •

After substituting Eqs. (B3) and (B5), the resulting expression is

defined as the (first) variation of F

dF dF (B8)
SF- ^ iy + » * %
For a complete analogy with the definition of a differential, one
might perhaps anticipate the definition

dF dF dF
5F * Ty Sy+ w s* + Tt st
However, t is not varied, so that 5t = 0. Hence the analogy is indeed
complete.
It is easy to verify from the above definitions that the variation opera¬
tor obeys the same laws as the differentiation operator. Thus

5 (Fi Fa) = Fi 5 F2 + F2 5 Fi (B9)

/ Fj \ F2 8 Fi — Fi 5 F2
5\ f2) “ F22 (BIO)

etc.

The essential distinction between the variation and differential opera¬

tors is the following. The differential of a function is a first order
approximation to the change in that function along a particular curve.
However, the variation of a function is a first order approximation to the
change from curve to curve.
For further details, we refer the reader to standard texts on the calculus
of variations/98-^)
 APPENDIX C

INNER PRODUCT IN FUNCTION SPACE

For any two vectors, a and b, the inner product * is defined by

a-b = arb = ^ a> by (Cl)

;=1

while the norm of a vector, a, is defined by

|Ja|| = (a-a)172. (C2)

Two vectors, a and b, are said to be orthogonal if

a-b = 0 (C3)

when both a and b are nonzero.

A set of vectors, {a (t)}, is termed an orthogonal set if

a(r)-a(s) = 0 for r 5^ s . (C4)

If, in addition, the relation

a(r) •a(r) = 1 for all r (C5)

is satisfied, then {a } is called an orthonormal set of vectors.

It may be shown that the inner product of two vectors satisfies the
Schwartz inequality

lla'b|| < ||a|| -||b|| (C6)

*Also called scalar or dot product.

730
 OPTIMAL CONTROL
731

and that for any two vectors, a and b, the following relations hold.

||a|| > 0 for all a ^ 0 (C7)

||a|| = 0 if, and only if, a = 0 (C8)

||a + b|| < ||a|| + (Jb|| (C9)

||/3a||= /3||a|j, /3= scalar (CIO)

Furthermore, if we denote the gradient operator by

d&i

then*

i (b'Aa)
=A*b (Cl 1)

d
77" (brAa) = Aa . (Cl 2)
OD

In particular

d
— (arAa) = 2Aa . (C13)

Also, the vector

As usual, the superscript^ denotes transpose.

732 ELEMENTS OF MODERN CONTROL THEORY

dy

da,i

dy

da,

dy_
dan

is called the gradient of the scalar y.

Now let u and v be n vectors, each of which is a function of an m
vector x. Then

d / du \T / dv \T
s(u'v)=(dr)v +(dr)u- <C14)
/ du \ _
dUi
Here I — 1 is an n X m matrix whose (ij)111component is —

Finally, if F(<p) is an n vector, <pan m vector, and z an r vector, then

dF / dF \ / d^ \
dz \y d (p /f y\ dz )
d«p

If the vector, a, is a function of a parameter, t, then

^ imi
= ini= Ti dt

4 (ara)_l/2 2 a<a<
z i=-i

which reduces to

11 n aTd (C15)
W " Hall•

Consider now two functions, f(t<) and gft,), which are defined for
discrete values of t<, in an interval, ti, t2, - , t„. Suppose we define
OPTIMAL CONTROL 733

a slightly generalized inner product by

J2 f(ti) g(t») At,

t— 1

where

At,' — t<4-1 ft .

Then, in the limit

lim J2 f(t.) g(t<) At< = (f,g)= f f(t)g(t)dt. (Cl6)

n —>oo
At< 0

The quantity, (f, g), is defined as the inner product of the functions,
f(t) and g(t), in function space. Loosely speaking, we say that Eq.
(C 16) gives the inner product of two vectors in an infinite dimensional
Euclidean space.
The norm of f(t) is defined by

llfll= VP • (Cl 7)

This satisfies a set of relations completely analogous to (C6)-(C10); viz.

lift g)ll < llfll • llsll (Cl 8)

||f|| > 0 for all f 5^ 0 (Cl 9)

||f|| = 0 if, and only if, f = 0 (C20)

Ilf + gll < llfll+ INI (C21)

||j8f|| = /3||f|| for any scalar 0 . (C22)

In analogy to Eq. (C4), we call a system of functions {f<}orthogonal

if for any two functions, fr and f.

(fr,f.)= / "h(t)
f.(t)dt= 0, r ^ s (C23)
734 ELEMENTS OF MODERN CONTROL THEORY

Furthermore, in analogy with Eq. (C5), the system of functions {f »•}is

orthonormal if

(f*, f*) = 1 for all k . (C24)

For a more comprehensive account of these ideas, the reader is referred

to standard texts.!1 54,162,1 63)

REFERENCES

1. S. S. L. Chang, Synthesis of Optimal Control Systems, New York,

McGraw-Hill Book Company, Inc., 1961.
2. D. F. Lawden, Optimal Trajectories for Space Navigation, London,
Butterworth Mathematical Texts, 1963.
3. G. Leitmann, Optimization Techniques with Applications to
Aerospace Systems, New York, Academic Press, Inc., 1962.
4. L. S. Pontryagin and others, The Mathematical Theory of Optimal
Processes, New York, John Wiley & Sons, Inc., 1962.
5. H. S. Tsien and R. C. Evans, “Optimum Thrust Programming for a
Sounding Rocket,” ARS Journ., 1951, pp. 99-107.
6. E. B. Lee, “Design of Optimum Multivariable Control Systems,”/.
Basic Eng., March, 1961, pp. 85-90.
7. A. Greensite, Optimal Transfer Between Coplanar Orbits— A
General Approach, General Dynamics Convair Report ERR-AN
229, November 19, 1962.
8. B. Friedland, “Optimum Control of an Unstable Booster with
Actuator Position and Rate Limits,” AIAA Journal, Vol. 3, No. 7,
1965, pp. 1268-1274.
9. R. E. Larsen, “Dynamic Programming with Reduced Computa¬
tional Requirements,” IEEE Trans, on Automatic Control, April,
1965, pp. 135-143.
10. S. J. Kahne, Feasible Control Computations Using Dynamic
Programming, Air Force Cambridge Research Laboratory Report
ARCRL-65-232, April, 1965.
11. A. Greensite, The Optimization of Missile Gust Response, General
Dynamics Convair Report ERR-AN-1 35, March 4, 1962.
12. A. Greensite, Optimal Control Subject to State Variable Inequality
Constraints with Application to the Booster Loads Problem,
General Dynamics Convair Report ERR-AN-490, March 13, 1964.
OPTIMAL CONTROL 735

13. J. S. Tyler and F. B. Tuteur, “The Use of a Quadratic Performance

Index to Design Multivariable Control Systems,” IEEE Trans, on
Automatic Control, January, 1966, pp. 84-92.
14. E. E. Fisher, “An Application of the Quadratic Penalty Function
Criterion to the Determination of a Linear Control for a Flexible
Vehicle,” AIAA Journ., Vol. 3, No. 7, 1965, pp. 1262-1267.
15. A. Greensite, The Theory of Optimal Control with Application to
Missiles and Space Vehicles, General Dynamics Convair Report
ERR-AN-13 1, March 21, 1962.
16. R. Bellman, Dynamic Programming, Princeton, Princeton Uni¬
versity Press, 1957.
17. R. Bellman and S. Dreyfus, Applied Dynamic Programming,
Princeton, Princeton University Press, 1962.
18. A. Miele, “General Variational Theory of the Flight Paths of
Rocket Powered Aircraft, Missiles, and Satellite Carriers,” Astro-
nautica Acta, Vol. IV, 1958, p. 264.
19. B. Paiewonsky, “Optimal Control: A Review of Theory and
Practice,” AIAA Journal, 1965, pp. 1985-2006.
20. G. Leitmann, “A Calculus of Variations Solution of Goddard’s
Problem,” Astronautica Acta, Vol. II, 1956, pp. 55-62.
21. W. Hohmann, Die Erreichbarkeit der Himmelskorper, Oldenbourg,
Munich, 1925; also NASA Translation TT-F -44, 1960.
22. R. B. Barrar, “An Analytic Proof That the Hohmann-type Transfer
is the True Minimum Two-impulse Transfer,” Astronautica Acta,
Vol. IX, No. 1, 1963.
23. D. F. Lawden, “Optimal Intermediate-Thrust Arcs in a Gravita¬
tional Field,” Astronautica Acta, Vol. VIII, No. 2, 1962, p. 106.
24. D. F. Lawden, “Minimal Trajectories,” J. Brit. Interplanet. Soc.,
Vol. 9, July, 1950, pp. 179-186.
25. D. F. Lawden, “The Determination of Minimal Orbits,” J. Brit.
Interplanet. Soc., Vol. II, September, 1952, pp. 216-224.
26. D. F. Lawden, “Minimal Rocket Trajectories,” ARS Journ., Vol.
23, 1953, pp. 360-365.
27. D. F. Lawden, “Stationary Rocket Trajectories,” Quart. J. Mech.
Appl. Mech., Vol. 7, December, 1954, pp. 488-504.
28. D. F. Lawden, “Optimal Programming of Rocket Thrust
Direction,” Astronautica Acta, Vol. I, 1955, pp. 41-56.
29. D. F. Lawden, “Optimum Launching of a Rocket into an Orbit
Around the Earth,” Astronautica Acta, Vol. I, 1955, pp. 185-190.
30. T. N. Edelbaum, “Some Extensions of the Hohmann Transfer
Maneuver,” ARS Journ., Vol. 29, 1959, p. 864.
736 ELEMENTS OF MODERN CONTROL THEORY

31. J. P. Carstens and T. N. Edelbaum, “Optimum Maneuvers for

Launching Satellites into Circular Orbits of Arbitrary Radius and
Inclination,” ARS Journ., Vol. 31, 1961, pp. 943-949.
32. H. Munick, R. McGill, and G. E. Taylor, “Minimization of
Characteristic Velocity for Two-impulse Orbital Transfer,” ARS
Journ., Vol. 30, 1960, pp. 638-639.
33. S. P. Altman and J. S. Pistiner, “Minimum Velocity Increment
Solution for Two-impulse Coplanar Orbital Transfer,” A1AA
Journ., Vol. I, 1963, pp. 435-442.
34. R. F. Hoelker and R. Silber, The Bi-elliptical Transfer Between
Circular Coplanar Orbits, Dept, of the Army Tech. Memo 2-59,
Army Ballistic Missile Agency, 1959.
35. W. R. Fimple, “Optimum Midcourse Plane Changes for Ballistic
Interplanetary Trajectories,” AIAA Journ., Vol. 1, 1963, pp.
430-434.
36. J. M. Homer, “Optimum Impulsive Orbital Transfer Between
Coplanar Orbits,” ARS Journ., Vol. 32, 1962, pp. 1082-1089.
37. L. Ting, “Optimum Orbital Transfer by Several Impulses,”
Astronautica Acta, Vol. VI, 1960, p. 256.
38. C. J. Wang, G. W. Anthony, and H. R. Lawrence, “Thrust
Optimization of a Nuclear Rocket of Variable Specific Impulse,”
ARS Journ., Vol. 29, 1959, p. 341.
39. W. G. Melbourne, “Three-dimensional Optimum Thrust Trajec¬
tories for Power-Limited Propulsion Systems,” ARS Journ., Vol.
31, 1961, p. 1723.
40. W. G. Melbourne and C. G. Sauer, Jr., “Optimum Thrust Programs
for Power- Limited Propulsion Systems,” Astronautica Acta, Vol.
VIII, 1962.
41. S. E. Dreyfus and J. R. Elliott, “An Optimal Linear Feedback
Guidance Scheme,” J. Math. Anal. Appl., Vol. 8, 1964, pp.
364-386.
42. H. J. Kelley, “An Optimal Guidance Approximation Theory,”
Inst. Elec., Electron. Engrs. Trans. Auto. Control, Vol. AC-9,
October, 1964, p. 375.
43. Z. V. Rekasius, “A General Performance Index for Analytical
Design of Control Systems,” Inst. Radio Engrs., Trans. Auto.
Control, Vol. AC-6, May, 1961.
44. J. Zaborsky, “The Development of Performance Criteria for
Automatic Control Systems,” Am. Inst. Elec. Engrs. Inst. Tech.
Groups Auto. Control, Vol. 1, No. 2, 1962.
OPTIMAL CONTROL 737

45. J. E. Gibson and others, “A Set of Standard Specifications for

Linear Automatnx Control Systems,” Am. Inst. Elec. Engrs.
Trans., Part II, 1961.
46. J. Walkovitch, R. E. Magdaleno, D. T. McRuer, F. D. Graham, and
J. D. McDonnell, Performance Criteria for Linear Constant-
Coefficient Systems with Deterministic Inputs, Wright-Patterson
Air Force Base Aeronautical Systems Div. Report TR 61-501,
1961.
47. P. A. Reynolds and E. G. Rynaski, “Application of Optimal Linear
Control Theory of the Design of Aerospace Vehicle Control
Systems,” Proceedings of the ASD Optimal System Synthesis
Conference, 1962.
48. E. G. Rynaski, P. A. Reynolds and W. H. Shed, Design of Linear
Flight Control Systems Using Optimal Control Theory, Wright
Patterson Air Force Base Aeronautical Systems Div. Report
ASD-TDR-63-376, April, 1964.
49. A. A. Goldstein, A. H. Greene, and A. J. Johnson, “Fuel
Optimization in Orbital Rendezvous,” in R. C. Langford and C. J.
Mundo, eds., AIAA Progress in Astronautics and Aeronautics:
Guidance and Control II, Vol. 13, New York, Academic Press,
1964, pp. 823-844.
50. H. K. Hinz, “Optimal Low-thrust Near-circular Orbital Transfer,”
AIAA Joum., Vol. 1, 1963, pp. 1367-1371.
51. L. W. Neustadt, “Synthesizing Time Optimal Control Systems,”/.
Math. Anal. Appl., Vol. 1, December, 1960, p. 4.
52. J. S. Meditch, “On the Problem of Optimal Thrust Programming
for a Lunar Soft Landing,” Inst. Elec., Electron. Engrs. Trans.
Auto. Control, Vol. AC-9, 1964, p. 233.
53. B. A. Hall, R. G. Dietrich, and K. E. Tieman, “A Minimum Fuel
Vertical Touchdown Lunar Landing Guidance,” in R. C. Langford
and C. J. Mundo, eds., AIAA Progress in Astronautics and
Aeronautics; Guidance and Control II, Vol. 13, New York,
Academic Press, 1964, pp. 965-994.
54. L. W. Neustadt, A General Theory of Minimum-fuel Space
Trajectories, University of Michigan Report TR06181-1-T,
November, 1964.
55. H. J. Kelley, “Successive Approximation Techniques for Tra¬
jectory Optimization,” Proceedings of the IAS Symposium on
Vehicle Systems Optimization, 1961, p. 10.
56. A. T. Fuller, “Relay Control Systems Optimized for Various
738 ELEMENTS OF MODERN CONTROL THEORY

Performance Criteria,” Proceedings of the International Federa¬

tion on Automatic Control Congress, London, Butterworth’s
Scientific Publications Ltd., 1960.
57. B. D. Fried, “On the Powered Flight Trajectory of an Earth
Satellite,” Jet Propulsion, Vol. 27, 1957, pp. 641-643.
58. H. J. Kelley, “An Investigation of Optimal Zoom-climb Techni¬
ques,” J. Aerospace Sci., Vol. 26, 1959, pp. 794- 802, 824.
59. A. Miehle, “On the Non-steady Climb of Turbo-jet Aircraft,” J.
Aeronaut. Sci., Vol. 21, November, 1954, pp. 781-783.
60. John S. MacKay, A Variational Method for the Optimization of
Interplanetary Round-trip Trajectories, NASA Report TND- 1660.
61. A. R. Hibbs, “Optimum Burning Program for Florizontal Flight,”
ARSJourn., Vol. 22, 1952, pp. 204-212.
62. A. E. Bryson and S. E. Ross, “Optimum Rocket Trajectories with
Aerodynamic Drag ” Jet Propulsion, Vol. 28, 1958, pp. 465-469.
63. S. Ross, “Composite Trajectories Yielding Maximum Coasting
Apogee Velocity,” ARS Journ., Vol. 29, 1959, pp. 843-848.
64. L. J. Kulakowski and R. L. Stancil, “Rocket Boost Trajectories for
Maximum Burnout Velocity,” ARS Journ., Vol. 30, 1960, pp.
612-618.
65. R. T. Stancil and L. J. Kulakowski, “Rocket Boost Vehicle
Mission Optimizations,” ARS Journ., Vol. 31, 1961, p. 935.
66. J. V. Breakwell, “The Optimization of Trajectories,” J. Soc. Ind.
Appl. Math., Vol. 7, June, 1959, pp. 215-247.
67. G. Leitmann, “Optimal Thrust Direction for Maximum Range,”/.
Brit. Interplanet. Soc., Vol. 16, 1958, pp. 503-507.
68. D. F. Lawden, “Optimal Program for Correctional Maneuvers,”
Astronautica Acta, Vol. IV, 1958, p. 264.
69. C. T. Striebel and J. V. Breakwell, “Minimum Effort Control in
Interplanetary Guidance,” IAS Paper, January, 1963, pp. 63-80.
70. D. F. Lawden, “Interplanetary Rocket Trajectories,” in F. I.
Ordway III, ed., Advances in Space Sciences, Vol. 1, New York,
Academic Press, Inc., 1959, p. 1053.
71. G. Leitmann, “The Optimization of Rocket Trajectories,” Progress
in the Astronautical Sciences, Vol. 1, Amsterdam, North Holland
Publishing Company, 1962.
72. B. Garfinkel, “Minimal Problems in Airplane Performance,” Quart.
Appl. Math., Vol. 9, No. 2, 1951.
73. P. Cicala and A. Miele, “Brachistochronic Maneuvers of a Variable
Mass Aircraft in a Vertical Plane,” J. Aeronaut. Sci., Vol. 22,
August, 1955, pp. 577-578.
OPTIMAL CONTROL 739

74. M. R. Hestenes, A General Problem in the Calculus of Variations

with Applications to Paths of Least Time, RAND RM 100, Rand
Corp., March, 1949.
75. H. J. Kelley, “Gradient Theory of Optimal Flight Paths,” ARS
Vol. 30, 1960, pp. 947-954.
76. A. E. Bryson, W. F. Denham, F. J. Carroll, and M. Mikami,
“Determination of Lift or Drag Programs to Minimize Re-entry
Heating,”/. Aerospace Sci., Vol. *19, April, 1962.
77. E. S. Levinsky, “Application of Inequality Constraints to Varia¬
tional Problems of Lifting Re-entry,” IAS Paper, No. 61-21,
January, 1961; also Wright Air Development Div. Report. TR
60-369, July, 1960.
78. I. Flugge-Lotz, Discontinuous Automatic Control, Princeton,
Princeton University Press, 1953.
79. L. Kazda, “Control System Optimization Using Computers as
Control System Elements,” Proceedings of Computer in Control
Systems Conference, American Institute of Electrical Engineers,
1958.
80. E. Kreindler, “Contributions to the Theory of Time-Optimal
Control,”/. Franklin Inst., April, 1963, p. 275.
81. R. Bellman, I. Glicksberg, and O. Gross, “On the Bang-Bang
Control Problem,” Quart. Appl. Math., Vol. 14, 1956, pp. 11-18.
82. J. P. LaSalle, “The Time Optimal Control Problem,” Contri¬
butions to the Theory of Nonlinear Oscillations, Vol. V,
Princeton, Princeton University Press, 1960.
83. R. V. Gamkrelidze, ‘The Theory of Time Optimal Processes in
Linear Systems,” Bull. Acad. Sci. USSR English Transl., Vol. 2,
1958, pp. 449-474.
84. L. I. Rozonoer, “L. S. Pontryagin’s Maximum Principle in the
Theory of Optimum Systems, I, II, III,” Avtomat. i Telemeh., Vol.
20, October, November, December, 1959, pp. 1320-1334, 1441,
1458, 1561-1578; Automation Remote Control Transl., Vol. 20,
June, July, August, 1960, pp. 1288-1302, 1405-1421, 1517-1532.
85. H. K. Weiss, “Analysis of Relay Servomechanisms,” /. Aeronaut.
Sci., Vol. 13, July, 1946, p. 364.
86. P. K. C. Wang, “Analytical Design of Electrohydraulic Servo¬
mechanisms with Near Time-Optimal Response,” Inst. Elec.,
Electron. Engrs., Trans. Auto. Control, Vol. AC-8, January, 1963.
87. I. Flugge-Lotz and H. Marbach, “The Optimal Control of Some
Attitude Control Systems for Different Performance Criteria,”
Proceedings of the Joint Automatic Control Conference, 1962.
740 ELEMENTS OF MODERN CONTROL THEORY

88. J. S. Meditch, “On Minimal-Fuel Satellite Attitude Controls,” Inst.

Elec., Electron. Engrs., Trans. Appl. Ind., No. 71, March, 1964, p.
120.
89. B. Friedland, “A Minimum Response-Time Controller for Ampli¬
tude and Energy Constraints,” Inst. Radio Engrs., Trans. Auto.
Control, Vol. AC-7, January, 1962.
90. M. Athans, P. L. Falb, and R. T. Lacoss, “Time-, Fuel-, and
Energy-Optimal Control of Nonlinear Norm-invariant Systems,”
Inst. Elec., Electron. Engrs., Trans. Auto. Control, Vol. AC-8,
July, 1963.
91. S. S. L. Chang, “Minimal Time Control with Multiple Saturation
Limits,” Inst. Elec., Electron. Engrs., Trans. Auto. Control, Vol.
AC-8, January, 1963.
92. H. K. Knudson, “An Iterative Procedure for Computing Time
Optimal Controls,” Inst. Elec., Electron. Engrs., Trans. Auto.
Control, Vol. AC-9, January, 1964, p. 23.
93. R. E. Kalman, “The Theory of Optimal Control and the Calculus
of Variations,” Research Institute for Advanced Studies, Report
TR 61-3, 1961.
94. E. B. Lee, “Geometric Properties and Optimal Controllers for
Linear Systems,” Inst. Elec., Electron. Engrs., Trans. Auto.
Control, Vol. AC-8, October, 1963, p. 379.
95. R. E. Kalman and R. W. Koepche, “Optimal Synthesis of Linear
Sampling Control Systems Using Generalized Performance
Indices,” Trans. Am. Soc. Mech. Engrs., November, 1958, p. 1 120.
96. S. Katz, “A Discrete Version of Pontryagin’s Maximum Principle,”
J. Electron. Control, Vol. XIII, August, 1962, p. 179.
97. M. R. Hestenes, “Variational Theory and Optimal Control
Theory,” Computing Methods in Optimization Problems (edited
by A. V. Balakrishnan and L. W. Neustadt), New York, Academic
Press, Inc., 1964.
98. O. Bolza, Lectures on the Calculus of Variations, New York,
Dover Publications, Inc., 1961.
99. F. A. Valentine, ‘The Problem of Lagrange with Differential
Inequalities as Added Side Conditions,” Contributions to the
Calculus of Variations, 1933-1937, Chicago, University of Chicago
Press, 1937.
100. L. D. Berkovitz, “On Control Problems with Bounded State
Variables,”/. Math. Anal. Appl., Vol. 5, December, 1962.
101. W. Kipiniak, Dynamic Optimization and Control: A Variational
OPTIMAL CONTROL 741

Approach, Cambridge, Mass., Technical Press, Massachusetts Insti¬

tute of Technology, and New York, John Wiley & Sons, Inc.,
1961.
102. R. V. Gamkrelidze, “Time Optimal Processes with Bounded Phase
Coordinates,” Dokl. Akad. Nauk, SSSR, Vol. 125, 1959, pp.
475-478.
103. S. Dreyfus, Variational Problems with State Variable Inequality
Constraints, Rand Corp. Report P-2605, July, 1962.
104. W. F. Denham, Steepest-Ascent Solution of Optimal Programming
Problems, Raytheon Co., Report BR-2393, April, 1963; also
Bryson, A. E., and Denham, W. F., J. Appl. Mech., Vol. 29, June,
1962.
105. A. E. Bryson, W. F. Denham, and S. E. Dreyfus, “Optimal
Programming Problems with Inequality Constraints I: Necessary
Conditions for Extremal Solutions,” AIAA J., Vol. 1, 1963, pp.
2544-2550.
106. W. F. Denham and A. E. Bryson, “Optimal Programming Problems
with Inequality Constraints II: Solution by Steepest-Ascent,”
AIAA J., Vol. 2, 1964, pp. 25-34.
107. C. A. Desoer, “The Bang-Bang Servo Problem Treated by
Variational Technique,” Inform. Control, Vol. 2, December, 1959,
pp. 333-348.
108. L. W. Neustadt, “Minimum Effort Control Systems,” Soc. Ind.
Appl. Math. J., Vol. 1, 1963.
109. J. H. Eaton, “An Iterative Solution to Time-Optimal Control,”/.
Math. Anal. Appl., Vol. 5, 1962, pp. 329-344.
110. Y. C. Ho, “Successive Approximation Technique for Optimal
Control Systems Subject to Input Saturation,” J. Basic Eng., Vol.
84, March, 1962, pp. 33-40.
111. C. Draper and Y. Li, “Principles of Optimizing Control Systems
and an Application to the Internal Combustion Engine,” Mech.
Eng., Vol. 74, February, 1952, p. 145.
112. N. J. Rose, “Optimum Switching Criteria for Discontinuous
Automatic Controls,” Institute of Radio Engineers Convention
Record, Part 4, 1956, p. 61.
113. E. I. Fadden and E. G. Gilbert, “Computational Aspects of the
Time-Optimal Control Program,” in A. V. Balakrishnan and L. W.
Neustadt, eds., Computing Methods in Optimization Problems,
New York, Academic Press, Inc., 1964.
114. R. Fletcher and M. J. D. Powell, “A Rapidly Convergent Descent
Method for Minimization,” Computer J., July, 1963.
742 ELEMENTS OF MODERN CONTROL THEORY

115. M. J. D. Powell, “An Iterative Method for Finding Stationary

Values of a Function of Several Variables,” Computer J., July,
1962.
116. B. H. Paiewonsky, P. Woodrow, F. Terkelsen, and J. McIntyre, A
Study of Synthesis Techniques for Optimal Control, Aeronautical
Systems Div. Report ASD-TDR-63-239, Wright-Patterson Air
Force Base, June, 1964.
117. B. Paiewonsky, P. Woodrow, W. Brunner, and P. Halbert,
“Synthesis of Optimal Controllers Using Hybrid Analog Digital
Computers,” in A. V. Balakrishnan and L. W. Neustadt, eds.,
Computing Methods in Optimization Problems, New York, Aca¬
demic Press, Inc., 1964.
118. L. E. Elsgolc, Calculus of Variation, Reading, Mass., Addison-
Wesley Publishing Company, Inc., 1962.
119. J. V. Breakwell, J. L. Speyer, and A. E. Bryson, “Optimization
and Control of Nonlinear Systems Using the Second Variation,”
SIAM J., Series A: Control 1, 1963, p. 193.
120. C. W. Merriam III, “An Algorithm for the Iterative Solution of a
Class of Two-Point Boundary Value Problems,” SIAM J., Series A:
Control 2, 1964, p. 1.
121. F. J. Ellert and C. W. Merriam III, “Synthesis of Feedback
Controls Using Optimization Theory— an Example,” Inst. Elec.,
Electron. Engrs. Trans. Auto. Control, Vol. AC-8, April, 1963.
122. F. B. Bailey, ‘The Application of the Parametric Expansion
Method of Control Optimization to the Guidance and Control
Problem of a Rigid Booster,” Inst. Elec., Electron. Engrs., Trans.
Auto. Control, Vol. 9, January, 1964.
123. L. Schwartz, Optimization Techniques— a Comparison, U.S. Air
Force Flight Dynamics Lab. Report TDR-64-21, Wright-Patterson
Air Force Base, March, 1964.
124. E. L. Peterson and F. X. Remond, Investigations of Dynamic
Programming Methods for Flight Control Problems, U.S. Air Force
Flight Dynamics Lab. Report TDR-64-11, Wright-Patterson Air
Force Base, March, 1964.
125. A. V. Balakrishnan and C. W. Neustadt, “Computing Methods in
Optimization Problems,” Proceedings of the January 1964 UCLA
Conference, New York, Academic Press, Inc., 1964.
126. H. B. Curry, “The Method of Steepest Descent for Nonlinear
Minimization Problems,” Quart. Appl. Math., Vol. II, October,
1944.
OPTIMAL CONTROL 743

127. K. Levenberg, “A Method for the Solution of Certain Nonlinear

Problems in Least Squares,” Quart. Appl. Math., Vol. II, July,
1944.
128. T. N. Edelbaum, “Theory of Maxima and Minima,” Optimization
Techniques (edited by G. Leitmann), New York, Academic Press,
Inc., 1962, Chap. I.
129. H. A. Spang III, “A Review of Minimization Techniques for
Nonlinear Functions,” SIAM Review, Vol. 4, October, 1962.
130. D. J. Wilde, Optimum Seeking Methods, Englewood Cliffs,
Prentice-Hall, Inc., 1964.
131. B. V. Shah, R. J. Buehler, and O. Kempthorne, “Some Algorithms
for Minimizing a Function of Several Variables,”/. SIAM, Vol. 12,
March, 1964, p. 74.
132. C. B. Tompkins, “Methods of Steep Descent,” in E. F. Becken-
bach, ed., Modem Mathematics for the Engineer, New York,
McGraw-Hill Book Company, Inc., 1956, Chap. 18.
133. R. Bellman, “On the Application of the Theory of Dynamic
Programming to the Study of Control Processes,” Proceedings of
the Symposium on Nonlinear Circuit Analysis, Brooklyn, Poly¬
technic Press, 1956, pp. 199-213.
134. G. A. Bliss, Lectures on the Calculus of Variations, Chicago,
University of Chicago Press, 1946, p. 296.
135. Y. C. Ho and P. B. Brentani, On Computing Optimal Control with
Inequality Constraints, Minneapolis-Honeywell Report 1529 TR-5,
March, 1962; also, SIAM J. Control, Vol. 1, 1963, p. 319.
136. J. S. Tyler, Jr., “The Characteristics of Model-following Systems as
Synthesized by Optimal Control,” Proceedings of the Joint
Automatic Control Conference, 1964, p. 41.
137. N. N. Krasovskii and A. M. Letov, “The Theory of Analytic Design
of Controllers,” Automation and Remote Control, Vol. 23, June,
1962.
138. Z. V. Rekasius and T. C. Hsia, “On an Inverse Problem in Optimal
Control,” Inst. Elec., Electron. Engrs., Trans. Auto. Control, Vol.
AC-9, October, 1964.
139. C. H. Knapp and P. A. Frost, “Determination of Optimal Control
and Trajectories Using the Maximum Principle in Association with
a Gradient Technique,” Proceedings of the Joint Automatic
Control Conference, 1964, pp. 222ff.
140. S. A. Jurovics and J. E. McIntyre, ‘The Adjoint Method and its
Application to Trajectory Optimization,” ARS J., Vol. 32, 1962,
p. 1354.
744 ELEMENTS OF MODERN CONTROL THEORY

141. R. R. Greenley, “Comments on ‘The Adjoint Method and its

Application to Trajectory Optimization,’ ” AIAA J., Vol. 1, 1963,
p. 1463.
142. B. H. Paiewonsky, A Study of Time Optimal Control, Aero¬
nautical Research Associates of Princeton, Report 33, June, 1961;
also, in J. P. LaSalle and S. Lefshetz, eds., Proceedings of
International Symposium on Nonlinear Differential Equations and
Nonlinear Mechanics, New York, Academic Press, Inc., 1963, pp.
333-365.
143. F. B. Smith, Jr. and J. A. Lovingood, “An Application of
Time-Optimal Control Theory to Launch Vehicle Regulation,”
Proceedings of the Optimum System Synthesis Conference,
September 11-13, 1962.
144. H. Kelley, “Optimization Techniques,” Method of Gradients
(edited by G. Leitman), New York, Academic Press, Inc., 1962,
Chap. 6.
145. R. McGill and P. Kenneth, “Solution of Variational Problems by
Means of a Generalized Newton-Raphson Operator,” AIAA J.,
Vol. 2, 1964, p. 1761.
146. C. W. Merriam, Optimization Theory and the Design of Feedback
Control Systems, New York, McGraw-Hill Book Company, Inc.,
1964.
147. T. R. Goodman and G. N. Lance, “The Numerical Integration of
Two Point Boundary Value Problems,” Math. Tables and Other
Aids to Computation, April, 1956, pp. 82-86.
148. R. S. Long, “Quasilinearization and Orbit Determination,” AIAA
Journ., Vol. 3, No. 10, 1965, pp. 1937-1940.
149. R. Kalaba, “On Nonlinear Differential Equations; the Maximum
Operation and Monotone Convergence,”/. Math. andMech., Vol.
8, 1959, pp. 519-574.
150. A. Miehle, General Variational Approach to the Optimum Thrust
Programming for the Vertical Flight of a Rocket, Report
AFOSR-TN-57, March, 1957.
151. R. Courant, “Variational Methods for the Solution of Problems of
Equilibrium and Vibrations,” Bull. Am. Math. Soc., Vol. 49, No.
1, 1943.
152. D. Dreyfus, “The Numerical Solution of Variational Problems,”/.
Math. Anal, and Appl., Vol. 5, 1962, pp. 30-45.
153. A. H. Jazwinski, “Inequality Constraints in Steepest Descent
Trajectory Optimization,” J. Aero. Sci., October, 1962, p. 1268.
OPTIMAL CONTROL 745

154. A. E. Taylor, Introduction to Functional Analysis, New York,

John Wiley & Sons, Inc., 1958.
155. L. A. Zadeh and C. A. Desoer, Linear System Theory, New York,
McGraw-Hill Book Company, Inc., 1963.
156. R. Courant, “Variational Methods for the Solution of Problems of
Equilibrium and Vibrations,” Bull. Am. Math. Soc., 1953, pp.
1-23.
157. T. Butler and A. V. Martin, “On a Method of Courant for
Minimizing Functionals,” J. Math, and Physics, Vol. 41, 1962, pp.
291-299.
158. A. H. Jazwinski, Steepest Descent Trajectory Optimization with
Inequality Constraints, General Dynamics Report ERR-AN-172,
June 6, 1962.
159. A. Greensite, Notes on Dynamic Programming, General Dynamics
Convair Report ERR-AN-217, October 25, 1962.
160. T. F. Cartaino and S. E. Dreyfus, “Application of Dynamic
Programming to the Airplane Minimum Time to Climb Problem,”
Aero. Eng. Review, 1957, pp. 74-77.
161. W. T. Reid, “A Matrix Differential Equation of the Riccati Type,”
Am. J. of Math., Vol. 68, 1946, pp. 237-246.
162. R. Bellman, Introduction to Matrix Analysis, New York, McGraw-
Hill Book Company, Inc., 1960.
163. R. A. Frazer, W. J. Duncan and A. R. Collar, Elementary Matrices,
London, Cambridge University Press, 1938.
164. V. K. Isaev, “Pontryagin’s Maximum Principle and Optimal
Programming of Rocket Thrust,” Automation and Remote
Control, Vol. 22, No. 8, 1961, pp. 986-1001.
165. R. McGill and P. Kenneth, “A Convergence Theorem on the
Iterative Solution of Nonlinear Two Point Boundary Value
Systems,” XIVth Int. Astronautical Fed. Congress, Paris, 1963.
166. R. S. Long, “Newton Raphson Operator; Problems with Undeter¬
mined End Points,” AIAA Joum., 1965, p. 1351.
167. J. A. Payne, Computational Methods in Optimal Control
Problems, Air Force Flight Dynamics Lab. Report TR-65-50,
August, 1965.
168. A. Greensite, A Functional Approximation Technique in N-Space
with Application to Dynamic Programming, General Dynamics
Convair Report ERR-AN-107, January 15, 1962.
169. A. Greensite, A Further Note on Functional Approximation in
N-Space, General Dynamics Convair Report ERR-AN-262,
January 9, 1963.
746 ELEMENTS OF MODERN CONTROL THEORY

170. R. Bellman and S. Dreyfus, “Functional Approximations and

Dynamic Programming,” Math Tables and Other Aids to Compu¬
tation, Vol. XIII, 1959, p. 247.
171. C. Lanczos, “Trigonometrical Interpolation of Empirical and
Analytical Functions,” Journ. of Math, and Physics, 1938, p. 123.
172. E. Bodewig, Matrix Calculus, Amsterdam, North Holland Pub¬
lishing Company, 1959, 2nd ed.
173. M. Plotkin, “Matrix Theorem with Applications Related to
Multivariable Control Systems,” IEEE Trans, on Automatic
Control, Vol. AC-9 1964, p. 120.
174. C. D. Johnson and J. E. Gibson, “Singular Solutions in Problems
of Optimal Control,” IEEE Trans, on Automatic Control, Vol.
AC-8, January, 1963, pp. 4-15.

PROBLEMS AND EXERCISES FOR CHAPTER FIVE

1. Given is the system whose equation of motion is

Xi = -Xj + u .

For a given tf, it is required to determine the control function, u,

which transfers the system from xdO) = a to xx(t/) = b, while mini¬
mizing the integral

Show that the optimal control is given by

(b — ae 7)
u = - — — — e‘ .
sinh tf

Solve the problem by (a) the maximum principle, (b) dynamic

programming.

2. Consider the system whose motion is described by

OPTIMAL CONTROL 747

Xj = x2

x2 - — ax2 — bxi + cu

where, a, h, and c, are positive constants. Show that the control

function, u, which minimizes the integral
/

for any arbitrary initial conditions, is given by

\J = -c(pia x1 4- Paa x2)

where

P12 - ~ C2 (1 ~ t)

a
P22 —
c-

Derive this result by

a. the maximum principle

b. dynamic programming.

3. In Problem 1, suppose that the control function must satisfy the

inequality constraint

|u| < M

where M is a positive constant. Show that the optimal u in this

case is given by

u = M sgn \i
 ELEMENTS OF MODERN CONTROL THEORY
748

where

Ai — Xi + 2xi , Ai(t/) — 0 .

Formulate precisely the two point boundary value problem which

must be solved in order to obtain an explicit solution for the opti¬
mal u.

4. For the system of Problem 2, let u satisfy the constraint

|u| < M , M = positive constant.

Consider the problem of calculating u such that the system is

driven from the initial state xi(0) = £i, x2(0) = £2, to the terminal
state, xx(t/) = x2(t/) = 0, such that t/ is a minimum. Show that the
optimal control is of the bang-bang type and formulate the two
point boundary value problem which must be solved in order to
obtain an explicit representation of the optimal u.

5. Consider the system

Xi = x2

x2 = u

|u| < 1 •

Show that the control, u, which drives the system from an arbitrary
initial state to xi(t/) = x2(t/) = 0, in minimum time can be expressed
in terms of the current state of the system; in other words, the
optimal control is closed loop. (Hint: Derive the equation of
motion in the xi x2 plane and find the expression for the switching
curve.)

6. Show that the time optimal control for the system

x(t) = h(x, t) + B(x, t) u(t)

[u(t)] < 7, a constant r vector

where x and h are n vectors, B an nxr matrix, and u an r vector, is

of the bang-bang type.
 OPTIMAL CONTROL 749

7. A positive quantity, c, is to be divided into n parts, such that the

product of the n parts is a maximum. Using dynamic programming,
let W„(c) be the maximum attainable product. Derive the appropriate
functional equation. Show by induction that

8. A general type of routing problem may be posed as follows. There

are N points numbered i = 1, 2, - , N. We let m,-,- denote the
cost incurred in going from point i to point j. Paths exist between
some (but not all) points; however, point N may be reached from
any other point by going to intermediate points, as necessary. It is
required to determine the path from point k to point N such that
the total cost is a minimum. Define

W(k) == the cost incurred in going from point k to point N, using an

optimal policy.

Using dynamic programming, derive the functional (recurrence)

equation for the problem. Give the necessary initial condition.

9. For the system of Problem 5, suppose that it is required to deter¬

mine u such that the integral

is a minimum. The final time, tf, is given. This is one type of fuel
optimal problem. Using the maximum principle, determine the
form of the optimal control in terms of the costate variables. Is it
possible for the optimal control to have the value zero during the
control interval?

10. Use the method of dynamic programming to derive a procedure that

eliminates the necessity for solving a two point boundary value
problem in Problem 9.

11. A simplified optimal trajectory problem may be formulated as

follows. The motion of a point mass is described by the following
 ELEMENTS OF MODERN CONTROL THEORY
750

differential equations

Xi = a cos u

x2 = b sin u — g

X3 = Xi

x4 = x2 .

Here xx, x2, x3, and x4 denote horizontal velocity, vertical velocity,
horizontal displacement, and vertical displacement, respectively.
These equations result from the assumption of flat earth, constant
thrust, constant mass, and negligible aerodynamic effects. Thus a,
b, and g are positive constants.
Given are the initial conditions

xi(0) = £x x3(0) = £3

x2(0) = £2 x4(0) = |4

and the terminal condition, x4 (t/) = h, where t/, the terminal time,
is prescribed. It is required to determine the form of the thrust
angle, u(t), such that the final horizontal velocity is a maximum.
Using the maximum principle, find the form of the optimal u in
terms of the costate variables.

12. Show how the optimal control, u, for Problem 1 1, could be deter¬
mined via dynamic programming methods. Derive the necessary
recurrence relation together with whatever initial conditions are
required to obtain a solution.

13. List and explain the advantages and disadvantages of dynamic pro¬
gramming vis a vis the maximum principle

a. theoretical
b. computational
c. in practical design problems.

How does the gradient method provide further aid in computational

procedures?
 Chapter Six
ADAPTIVE CONTROL

It has been said that to define adaptive control is to invite an argument.

Indeed, the phrase is fraught with emotional overtones both for those
to whom it represents a potential (if not present) panacea, and for those
to whom adaptive control is at best an ambiguous aspiration having no
cohesive structure within which a sound unified theory can be formulated.
Within the context of the present chapter, which is limited to adaptive
control of space launch vehicles, the term adaptive control will be applied
in a limited sense. It is generally agreed that adaptive control as a design
philosophy enters the picture when a control system cannot be adequately
designed a priori to cope with a changing or unknown environment.
There are two fundamental aspects to this problem: identification and
control function manipulation. The adaptive control of space launch
vehicles exhibits both of these features.
The question of adaptive control for a launch vehicle autopilot must
begin with a discussion of the problems of conventional control.
The optimum design of a space launch vehicle, from an overall perfor¬
mance point of view, maximizes payload and minimizes structural weight.
Minimum structural weight results in increased flexibility (i.e., the ability
of the vehicle to bend), which, in turn, has an adverse effect on the atti¬
tude control system.
The purpose of the attitude control system is to determine and
maintain the direction of vehicle travel. A boost vehicle is similar to a
long rod that bends in two ways when forces are placed upon it. The
first is a steady-state bend equivalent to the sag in a rod when it is
supported horizontally at both ends. The second is an oscillatory bend
by the rod if a weight is dropped on it. This oscillatory bending tends to
die out unless it is continually forced and excited as in the case of the
control system continually applying forces to the vehicle. The system
must be designed so that it does not increase bending beyond safety
margins. The vehicle attitude sensor, rigidly attached to the vehicle, also
751
752 ELEMENTS OF MODERN CONTROL THEORY

measures bending at the sensor location. The attitude control forces

computed from the sensor output are thus partially determined by the
bending magnitude at the sensor. Factors depending upon the relative
bending direction at the sensor locations and the attitude control force
point, plus delays in computing the attitude control force magnitude
from the sensor output, determine whether the applied force increases or
decreases any bending that may exist. The effects of computational
delays are directly dependent upon the oscillatory frequency of the
bending.
A normal control system for a flexible vehicle does not allow high
frequencies to pass through to the force point, thus eliminating the
reinforcing of high frequency oscillatory bending modes. Computational
delays are so designed that the low frequency oscillatory bending modes
are suppressed by the control forces rather than reinforced. However,
with very large and very flexible vehicles, a normal control system
design may be inadequate. For very flexible vehicles the bending oscilla¬
tory frequencies become low enough that even the higher modes cannot
be filtered out without detrimental effects upon attitude control. With
large vehicles it is impossible to predetermine the bending oscillatory
frequency to the accuracy required to adjust the computational delays
in a manner to guarantee control.
A detailed quantitative discussion of these factors is contained in
Volume Two. The main problems from a controls point of view may be
summarized as follows.

a. Bending mode frequencies are not known with sufficient precision.

b. Bending mode properties vary with flight time.
c. The lowest bending mode frequency may be of the same order of
magnitude as the control frequency.

Many schemes have been devised to cope with these problems. They
have been called “adaptive” since the control system parameters vary
(adapt) as a function of flight environment. A discussion of these con¬
cepts forms the subject matter for the present chapter.
A discussion of adaptive control cannot proceed without at least a
tentative definition. One widely accepted is that an adaptive control
system can monitor its own performance, evaluate the effectiveness of
its response, and then modify its parameters automatically to achieve
desired performance. When applied to a launch vehicle autopilot this
implies that the adaptive control system will identify an unstable bending
mode frequency, and activate some suitable compensation device to
ADAPTIVE CONTROL 753

stabilize the system. This compensation may take the form of cancel¬
lation, phase stabilization, or gain stabilization. The cancellation tech¬
nique, as typified by the model-reference or virtual slope methods, for
example, makes the sensor output behave as if the sensor were located on
an antinode (point of zero bending mode slope). Phase and gain stabili¬
zation techniques depend on an accurate identification of the bending
mode frequency since these modes are highly tuned (very low relative
damping factor). In some cases signal Shaping by conventional filtering
is inadequate, since the signals to be separated do not have a sufficiently
large frequency separation. The digital adaptive filter and frequency
independent signal processing techniques have been developed to cope
with this problem.
One thing is certain. Adaptive control has not suffered from a lack of
attention in the literature. Neither has Air Force nor NASA funding
been lacking for pursuing investigations of particular concepts. Yet
today, some ten years after adaptive control began to be studied in¬
tensively, very few adaptive systems have reached the flight test stage
(these mainly on aircraft), and none are operational. A partial answer
is that so-called conventional methods have been refined considerably,
and the point at which adaptive control becomes mandatory has never
been clearly defined. There is a considerable (and understandable) reser¬
vation in committing oneself to complicated and exotic control schemes
where multimillion dollar vehicles are concerned if their need has not
been clearly established.
At present, conventional and straightforward control methods appear
adequate. However, if past experience is any indication, one will as¬
suredly be confronted with problems that strain the capabilities of
current technologies. When this occurs, advanced schemes presently in
the “drawing board” stage will be evaluated under actual operating
conditions and thus serve both to refine and enlarge these concepts
presently called “adaptive control.”
The following sections describe the major bending mode adaptive
techniques. They also provide a critical evaluation of the virtues and
limitations of each type. To date, none of these schemes has been flight
tested, although many of them have been thoroughly simulated (via
computer) with varying degrees of complexity. Consequently, while
different government agencies and aerospace contractors have their pref¬
erences, there is no universal agreement that any one type represents a
definitive solution to the problem.
In order to simplify the presentation, and provide a common frame¬
work for analysis and comparison of the different methods, each will be
754 ELEMENTS OF MODERN CONTROL THEORY

discussed with reference to the mathematical model described in Appen¬

dix A.

6.1 MODEL-REFERENCE TECHNIQUES

The basic idea of employing a model to obtain improved system

response is apparently due to Lang and Ham.U) However, in its original
version, this was merely a disguised high gain system in which prefiltering
together with high open-loop gain were used to achieve relatively invari¬
ant response in the presence of parameter variations. The methodology
may nevertheless be applied in different ways in order to satisfy pre¬
scribed objectives. Two particular extensions of this philosophy to the
bending mode problem are described next.

6.1.1 Whitaker (MIT) System

The model-reference adaptive control concept was originally proposed

and investigated at the MIT Instrumentation Laboratory. It was evolved
to enable design of a control system that could adjust its own controllable
parameters in the presence of changing or poorly defined operating char¬
acteristics. The underlying philosophy has been to provide a control
system that will meet system specifications provided its variable param¬
eters can be adjusted to proper values. The function of the adaptive
system is then that of providing the proper parameter values.
Figure 6.1 is a functional diagram of a general model-reference adap¬
tive flight control system. The system specifications are incorporated
into a reference model that receives the same commands as the actual
system. A response error signal is obtained by comparing the response of
the model to that of the system. The controllable parameters are
adjusted so that the integral squared error between system and model
outputs is minimized. The index of performance is then the integral-
squared response error, and the criterion for successful adaptation is
that the integral-squared error (ISE) be the minimum value obtainable
with the parameter variation provided.

(1)

At the desired operating point the slope of the ISE as a function of the
ADAPTIVE CONTROL 755

BASIC FLIGHT CONTROL SYSTEM

Figure 6.1. Simplified Functional Diagram of a Model-Reference Adaptive Flight

Control System.

variable parameters Pn is zero, or

0P„aP,r
.,*P.(/(E),dt)-° <2>
If the limits of integration are independent of Pn and the integral of the
partial derivative of the function exists, the partial differentiation may
be carried out under the integral sign, resulting in the error quantity

(EQ). dt = E dt

(3)
= / W£(t)
(E)dt.
The error quantity is the integral of the error weighted by a function
such that the error quantity is indicative of the state of the system. The
756 ELEMENTS OF MODERN CONTROL THEORY

sign and magnitude of the error quantity indicate the direction and
amount that the variable parameter should be changed. A simple mech¬
anization results if the change of the parameter is made proportional to
the error quantity. The weighted error is then proportional to a time
rate of change of the parameter, which will result in setting the parameter
to the desired operating point. The integration of Eq. (3) may then be
carried out in the parameter adjustment device.
The weighting function is determined by performing a straightforward
partial differentiation of the differential equation for the error as a
function of the input quantities, or by considering the change in the
parameter as a disturbance entering the system after the parameter. The
error weighting function can be generated by taking the signal at the
input to the variable parameter and feeding it through a filter having the
same performance function as the system, cascaded with a filter that is
the reciprocal of some of the forward path components. It is not possible
to obtain a signal that exactly satisfies the equations, since the ill-defined
or unknown characteristics of the system are what lead to the require¬
ment for the adaptive system. An approximation to Eq. (3) is obtained
by substitution of the performance function of the model for the per¬
formance function of the system. In cases where a system model does
not exist explicitly, Eq. (3) may be approximated by using an approxi¬
mation for the dynamic characteristics of the optimum system.
The validity of this approximation depends upon the accuracy of the
approximation of the system by the model and weighting function
filter. In the neighborhood of optimum response, this approximation is
good. For system parameter settings far removed from optimum the
approximation is poor; however, if the algebraic sign of the weighting
function is correct, proper system operation is obtained. Even though
the weighted error has the incorrect sign instantaneously, satisfactory
results are obtained if the error quantity has the proper sign over the
evaluation period. The derivation of these equations is based on the
assumption of the constant coefficient linear system. When the param¬
eters are varied during the response to input signals, these equations are
in error by perturbation terms.
Because of the nonlinear nature of the system, operation can best be
studied by analog simulation. Simulation is also required to evaluate the
use of approximate dynamics in the weighting function filter. It is to be
expected that the operating state selected by the adaptive system will
not be the optimum operating state predicted by the exact Eq. (2), but
will have some error due to the approximations. It has been found that
these errors are usually small and the characteristics of the weighting
function filter usually are not critical.
ADAPTIVE CONTROL 757

In applying the above concepts to the elastic vehicle control problem,

we must begin by defining two quantities:

a. The performance index.

b. The parameter(s) to be varied.

In order to provide the adaptive capability, it is also necessary to define

a model that incorporates the system specifications and to obtain a
response error that is indicative of the state of the system. The most
desirable situation would be elimination of the bending, in which case
the “model” is zero and the response error is the bending itself. Thus
we may take as a performance index

where q(1) is the generalized coordinate of the first bending mode.* The
control system to be described is then adaptive only with respect to this
mode.
We must now choose the parameter to be varied in order to minimize
Eq. (4). For this purpose, KezeE4) takes the “effective bending” sensed
by the rate gyros. Such a signal may be obtained, for example, by
taking the difference between the outputs from two gyros located at
different positions along the vehicle; thus

{6a + crG8(1)
q<1J)- {6a + <rG,(1)
q»>) = <r*(1)q<J>.

The quantity aB(1) is selected as the parameter to be varied in order to

minimize Eq. (4). Proper operation of the system then requires that

^/[q<1>?dt
- hh* [q0,1‘
dt=2/q<1,( =0%<5)
The system is now made adaptive by setting the rate of change of
<jfi(1)proportional to

(6)

*The nomenclature used in the following discussion is defined in Appendix A.

758 ELEMENTS OF MODERN CONTROL THEORY

To do this we must have available both q(1) and dq (1)/d<rB(1).

The latter
may be determined directly from the equations of motion of the system.
Referring to Fig. 6.2 and Appendix A, we have*

(s2 — na) 0R = juCS (7)

(s2 + 2 r1 w1s + [o1]2)qa> = - ^7 5 (8)

(s + Kc) 8 = Kc 8C (9)

5c = K* (fi, - dF) (10)

— Or + <r<r(1)
Q(1)+ Ks (Or + ^oh1I)q(1))

+ K* [(0s + oak" q(1)) - (6* + ajl) q»>)]

= (K* s + 1)(0« + <rG(1)

q(1)) + Kb s as(1) q'1’ (11)

Figure 6.2. MIT Adaptive Control System.

It is assumed that Ki = 0 and a » 8 .

ADAPTIVE CONTROL 759

where

<Tna)—c0r1) — cr0}l)

and

<reRW= crG(1>

From Eqs. (7)—( 1 1), we find that q<l) is related to 9Cby

G„(s)
q(1)
=- ^ (s2
- dc (12)

where

G0(s) = [(s2 - Ma)(s + Kc) + Ka Kc txc(K* s + 1)] (s2 + 2f(1) coCl)

s + [coU)]2)

ka kc tc
[Kfl S (o-G(1>
+ 0>(1))+ cto^] (s2 — Ha) . (13)

(i)
Taking the partial derivative of Eq. (12) with respect to <rj , we find

„ , s aq(1> Ka Kc Kb Tc ,
Go(s) T- (jy - - (s2 - Ma) s q(1) = 0
dai

or

aq<D Ka Kc Ks Tc (s2 - Ma)

(l)
(14)
da M(1) Go (s)

If the transfer function for the weighting filter is taken as

K4 Kc Tc (s2 — Mq)
G, (s) = (15)
M(1) Go (s)

and if the input is Kscrtf(1)q(n, we see that the output of the weighting
filter is the quantity
760 ELEMENTS OF MODERN CONTROL THEORY

Furthermore, the output of the integrator is

fW1J q(1) (IV)

and it follows that the multiplier output is

i.e., proportional to the error quantity, Eq. (6). The signal is used to
adjust the sign and magnitude of the gain X. A steady-state value is
achieved when the multiplier output is zero, which indicates that the
first bending mode signal has been essentially eliminated from the
“blended” rate gyro feedback.
Some typical results of a computer simulation for this system are
shown in Fig. 6.3. The input 9C was a square pulse in each of the three
cases shown. In the first case, with the adaptive loop open, it is apparent
that the bending mode signal is poorly damped. The improvement in
response for the next two cases (where the adaptive system is employed)
is quite evident.

A PITCH
ANGLE

Figure 6.3. Response Characteristics of MIT Adaptive System.

ADAPTIVE CONTROL 761

Remark: The MIT adaptive system is an elegant and conceptually sound

approach to the bending mode problem, and has an obvious
appeal to the theoretically inclined. The system is fast acting,
and acts on either command or disturbance inputs with equal
effectiveness. Studies have shown that the multiplier may be
replaced by a relay, and that the integrator (for generating
q(1)) may be replaced by a low pass filter. It is thus relatively
simple to mechanize. '
On the other hand, the scheme is effective only when one
bending mode is significant and well separated in frequency
from the other modes.
It should be noted also that the weighting filter is of neces¬
sity an approximation since, as is apparent from Eqs. (15) and
(13), some estimate is necessary for the M(1), <j^\ and w(1)
values that are not known; hence the need for adaptive control.
Furthermore, the quantities nc and na are time varying, and
various higher order dynamic effects are neglected. These
factors are not usually crucial, however; it means essentially
that the error quantity, Eq. (6), will not attain a zero value.
Adaptive control is still exhibited in the sense that corrections
are applied (by varying X in Fig. 6.2) to reduce the error. How
well this is done is a function of how well the weighting filter
approximates the “optimum” filter.

6.1.2 Tutt and Waymeyer System

A “brute force” approach to the problem of separating rigid body

from bending mode signals is by using passive low pass linear filters on
the rate and displacement gyro outputs. It is well known that this ap¬
proach results in an unacceptable deterioration in rigid body response.
In order to recover rigid body motion signals, one could conceivably
employ predicted attitude and rate information from a model of the
rigid body dynamics of the vehicle. Such a scheme is shown in Fig. 6.4.
The engine angle signal 8 is fed to a model of the rigid body dynamics,
which requires a knowledge of the control and aerodynamic moment
effectiveness, ju? and /4, respectively (asterisks denote “model” param¬
eters). The attitude rate output from the model, dM, is then multiplied
by Ks, the rate gyro gain, fed through a high pass filter, and summed
with the actual rate gyro output after the latter was fed through a low
pass filter. This sum is used as the rate feedback to the system. Thus,
the rate gyro information is heavily filtered to reject bending mode
762 ELEMENTS OF MODERN CONTROL THEORY

Figure 6.4. Model-Reference Adaptive System.

signals, and a “clean” rigid body rate signal is obtained from a model of
the rigid body dynamics. A similar procedure is applied to the attitude
displacement information.
In accordance with this control philosophy, it would appear that one
could generate “model” attitude and rate information from a configura¬
tion of the form shown in Fig. 6.5. However, it is not possible for Ha to
operate only on computed information, since a closed-loop pole-zero pair
would appear in the right-half s plane, and these would cancel only if
Ha = Ha exactly. Therefore, the configuration shown in Fig. 6.4 is used
where h% is acted upon by a combination of computed and actual body
attitudes.
A crucial factor in the model-reference approach is the required ac¬
curacy in H* and ju* , of which the latter is the more critical. Computer
traces for attitude rate in response to step input commands in 9C are
shown in Figs. 6.6 and 6.7 for ju* 15 percent less than and 15 percent
greater than Ha, respectively. These traces indicate (and analytical studies
verify) that the tolerance on h1 (with respect to Ha) should be heavily
weighted toward the positive (h% > Ha)-
In Fig. 6.8 are shown the rate response traces due to a step wind
disturbance for both model feedback and conventional systems, each
ADAPTIVE CONTROL 763

Figure 6.5. Alternate Model Configuration.

t (sec)

Figure 6.6. Command Response with n', =

15 percent Less Than na .

Figure 6.7. Command Response with y.',, -

15 percent Greater Than ^a-

with the same system parameters. The very poor response for the model
feedback system was to be expected, since actual body motion is not
generated solely by engine deflection. In the configuration of Fig. 6.4,
the engine responds only to the heavily filtered real body loop.
764 ELEMENTS OF MODERN CONTROL THEORY

0 1 2 3 4 5

t (sec)

Figure 6.8. Rate Response to Step Disturbance.

Since for a realistic launch vehicle the wind inputs are often the
predominant inputs to the system, the model-reference configuration of
Fig. 6.4 is clearly inadequate. Various modifications are proposed by
Tutt and Waymeyer/5) of which all are based essentially on some form
of angle-of-attack sensing. The problem is to obtain some sort of dis¬
turbance information without reintroducing the bending previously re¬
jected.
To do this we may adopt the same control philosophy used in the
attitude and rate loops; namely, to use an actual value and a “model-
reference” value for angle of attack, which is combined and used as an
additional feedback loop for the control system. The system would
then take the form shown in Fig. 6.9. The schematic for the a model is
derived from Eqs: (Al) and (A2) of Appendix A, with

T„ fa W
N« = N„ oc M —
m U0 m Uo Uo‘

The degree of improvement in disturbance response for a typical case

is shown in Fig. 6.10, in which are superimposed the responses for
systems denoted as conventional, model feedback, and model feedback
with a sensing. Use of a sensing thus yields a significant improvement.
ADAPTIVE CONTROL 765

MODEL

Figure 6.9. Model-Reference Adaptive System with Disturbance Sensing.

Whether this improvement is good enough is something to be determined

for the specific mission considered.

Remark: The model-reference system is very effective when certain

rather stringent requirements are satisfied. First the rigid
body parameters (mass, moment of inertia, center-of-gravity
location, aerodynamic properties) must be accurately known
as a function of flight time. Second, and more important,
disturbance effects must be minimal. In other words, the
vehicle motion must be derived primarily from command in¬
puts. This condition is obviously not satisfied for ballistic
booster vehicles, and various additional complexities must be
introduced in the control system to take account of distur¬
bance inputs.
It may be noted in passing that if the rigid body model is
accurate, one may dispense with the rate gyro entirely.
Many aspects of the model-reference control scheme require
further investigation. A basic element of the system is the
rigid body model, and it is necessary to determine the sensitivity
of the system performance to variations on the rigid body
“model” parameters. In a realistic system the “model” is
essentially a filter with nonstationary parameters. For both
 ELEMENTS OF MODERN CONTROL THEORY
766

analytical and pragmatic reasons this is an undesirable situation,

and it would be important to determine if gain switching at
preselected intervals could replace continuous variations in ixt
and ix*. Another open question at present is the choice of
break frequencies in the high and low pass filters. This must
obviously be decided for each specific vehicle and depends
primarily on the frequency separation between rigid body and
first bending mode signals. When these are in fairly close
proximity, the effectiveness of the model-reference scheme
appears marginal.
It has also been argued that the model-reference scheme is
not an adaptive system at all. Since the control configuration
is predetermined and there is neither identification of system
parameters nor their control as a function of environment, it is
not reasonable to claim that this system is adaptive. However,
this could motivate an involved digression in semantics, and we
must resist the temptation to pursue it further.

t (sec)
Figure 6.10. Comparison of Response to Wind Inputs.
ADAPTIVE CONTROL 767

6.2 VIRTUAL SLOPE METHODS

If the vehicle bending modes were known precisely, then a simple

means of suppressing bending mode signals for a particular mode would
be to locate the gyros on an antinode (point of zero slope) for that mode.
However, even if the bending mode data were known precisely as a
function of flight time, the antinode location (and other bending mode
data) varies with time, so that one specific location for the gyro is not
satisfactory throughout the flight regime. Conceptually, one may visual¬
ize a gyro moving along the vehicle such that it constantly tracks the
desired antinode. This is obviously a fanciful notion, but it is possible
to formulate a control scheme that effectively performs this function by
other means. Two methods of achieving this are described next.

6.2.1 Gyro Blending

The essential idea for the gyro blending scheme is motivated by the
following considerations. Suppose two rate gyros located at different
points along the vehicle are used. The output of the rear gyro is*

(19)

and, similarly, for the forward rate gyro, viz.

Kfl )% (20)

If it is assumed that the only significant bending mode is i = 1, we may

proceed as follows. Multiply Eq. (19) by some number K, and multiply
Eq. (20) by (K - 1). Adding the result, we have

IvB 6r + K« [(1 — K.)<70R(l>+ Kero/1*] q(1) (21)

In order to obtain a signal free of bending mode information, we see

that K must satisfy the relation

(1 — K) <T(JB(1)
T" K<JOFl) — 0 . (22)

See Appendix A for definitions of nomenclature.

 ELEMENTS OF MODERN CONTROL THEORY
768

Under suitable conditions this operation may be made automatic, and

the resulting control system configuration has the general form shown in
Fig. 6.11. It remains to investigate the means whereby K is made to
satisfy Eq. (22) automatically.
Referring to Fig. 6.12, which shows the blender operation in detail,
suppose that the rate gyro signals attenuated by (1-K) and K, respectively,

Figure 6.1 1. Gyro Blender Adaptive Control System.

Figure 6.12. Detail of Gyro Blender Operation.

ADAPTIVE CONTROL 769

are passed through bandpass filters designed to reject all signals outside
the anticipated frequency range of the first bending mode. The signal
appearing at the input of the integrator is then

E = |(1 - K) «0RWq«)| - |K<r0,^ q«| .

It is apparent that a steady-state condition (E = 0) is reached when K

satisfies Eq. (22), which is precisely 'the condition required for the
blender output to be free of bending mode signals.
Since K, which is a potentiometer setting between zero and one, must
always be positive, a fundamental requirement for proper operation of
this system is that the mode slopes at the two gyro locations be of
opposite sign at all times.

Remark: This scheme is workable only if there is assurance that certain

prescribed conditions are satisfied. First of all, this method
requires that the bending mode slopes at the gyro locations
always be of opposite sign. Usually this condition prevails
only for the first bending mode. Furthermore, effective
operation of the bandpass filters requires that there be a
moderate frequency separation between rigid body mode and
bending mode signals on the one hand, and between first
bending mode and other dynamic modes of the system on the
other. If this condition does not prevail, the integrator will
not null properly, the correct value of K will not be estab¬
lished, and the rate feedback signal will, therefore, contain
bending mode information.

6.2.2 Phasor Cancellation

As in the previous system, the aim of the phasor cancellation concept

is to yield a feedback signal free of bending mode signals by an operation
that effectively locates the gyros at an antinode. The essential idea may
best be described by examining the Fourier decomposition of the feed¬
back signal, viz.

9/ = Z E, sin («<» t + <A<)

. (23)
i= 0

Each of the terms on the right-hand side of this equation is a phasor,

Ei, with amplitude E< and phase angle i/-*. We may associate E0 with the
770 ELEMENTS OF MODERN CONTROL THEORY

rigid body mode and the E< with the bending modes. It will again be
assumed that only the first bending mode is significant.
The adaptive system described here generates a phasor whose ampli¬
tude and phase angle are identical to Ex, and subtracts this from the
feedback control signal, thereby giving a signal free of first bending mode
information. The method is illustrated schematically in Fig. 6.13. In
Fig. 6.14 are shown the frequency response characteristics of the adjust¬
able bandpass filter. To analyze the operation of the system, assume that
is equal to w(1),the first bending mode frequency. Then the output of
the adjustable bandpass filter is a signal whose amplitude and phase are
equal to that of the first bending mode signal, with all other frequencies
sharply attenuated. Therefore, subtracting this from e'F gives a signal
essentially free of first bending mode information.
The function of the adaptive controller is to track the bending mode
frequency and set u, equal to u>(1)at all times. This may be accomplished
in the manner shown in Fig. 6.15. In general, wF will not be equal to co(1),
which means that the bending phasor, after passing through the adjust¬
able bandpass filter, will incur a phase shift, A^x (see Fig. 6.14). This
signal, after then passing through the secondary bandpass filter, will
incur another phase shift approximately equal to A^x if u, is not too far
from w'11(this means essentially that ax'1' is such that operation is along
the nearly linear portion of the phase shift curve in Fig. 6.14). Thus, A^x
is a measure of the deviation of wF from a>(1).

Figure 6.13. Phasor Cancellation Adaptive Control System.

ADAPTIVE CONTROL 771

Figure 6.14. Frequency Response Characteristics of Bandpass Filter.

FROM OUTPUT
TO ADJUSTABLE
OF ADJUSTABLE
BANDPASS FILTER
BANDPASS FILTER

LIMITER
NO. 1

SECONDARY
BANDPASS FILTER

2£ s
LIMITER
MULTIPLIER
NO. 2
^F S +^F
T
Figure 6.15. Detail of Adaptive Controller.
772 ELEMENTS OF MODERN CONTROL THEORY

Now the outputs from limiters No. 1 and No. 2 are

A sin w(1) t

and

A cos (w(1>t 4- A\pi)

respectively. The amplitude A is constant, due to the limiters.

The output of the multiplier is then

A2 sin w(1) t cos (co(1)t + Aipi) .

If A^i is small such that sin A^i & Afa, this last relation may be written as

The second term, which is a phasor of twice the bending mode

frequency, is sharply attenuated by the integrator. Thus, the output of
the multiplier is essentially proportional to Ai/'i. The integrator output
adjusts uF in both bandpass filters in such a way as to drive A^i to zero;
that is, until uF = u'-1'.
The results of an analog computer simulation of this system are shown
in Fig. 6.16.
The filter was tuned to 10 rad/sec with the bending frequency at 23
rad/sec. (This simulates a bending frequency increase of greater than
100 percent in a step fashion. The bending was excited by a one-degree
surface command.) The uF trace clearly demonstrates the tuning action.
Referring to the trace of the bending mode, q(l), it can be seen that the
bending is unstable with the filter not tuned to the bending frequency.
The bending magnitude reached a maximum peak of ±0. 09 foot while
the filter was being tuned to the bending frequency. The filter output
then cancelled the bending oscillation sensed by the rate gyro and dis¬
placement gyro.
The double bending frequency component of the multiplier output is
seen impressed on the uF trace. This term is greatly attenuated and does
not present a problem.
The effect of varying the integrator gain, k, is shown in Fig. 6.17.
With k set at 400, the adaptive loop was marginally stable. In the lower
trace, with k set at 150, the adaptive loop gain was too low, and,
ADAPTIVE CONTROL 773

- +5 deg - — —

v/ \ju Imff
H - 1
sec

— -- - - - V
o _

- +5 -

%
- ^10 rad/sec -

°F - 0-

• -10 •

%-20 % XU
k = 250

- +0.05 ft -

(1).

-0.05 -

Figure 6.16. Analog Computer Trace, Initially Unstable Bending Mode.

k = 400

- 20 RAD/SEC %
1
sec

k = 150

b. Low Adaptive Loop Gain

Figure 6.17. Effect of Variation in Integrator Gain, k.
774 ELEMENTS OF MODERN CONTROL THEORY

consequently, the tuning action was sluggish with considerable overshoot.

The value of k = 250 used in the traces of Fig. 6.16 represents a com¬
promise value, and results in a filter “lock on” time of approximately
three seconds.
The operation of the filter when the vehicle is exposed to a random
disturbance such as a gusts is shown in Fig. 6.18. White noise was
introduced into the a equation to simulate gust disturbances. The filter
was offset to 10 rad/sec, and the bending frequency was 22 rad/sec. The
filter tunes to the bending frequency and cancels the bending signal to
prevent an unstable condition.

Remark: The phasor cancellation scheme exhibits excellent bending

mode suppression characteristics when only one bending is
significant. However, its performance deteriorates sharply when
the bending mode frequency is very near the rigid body fre¬
quency. This results from the tendency of the filter to attenu¬
ate the short-period control frequencies as well as the bending
mode components.

- +5°-

- 20 RAD/SEC

Figure 6.18. Analog Computer Trace, Response to Random Disturbance.

ADAPTIVE CONTROL 775

6.3 DIGITAL ADAPTIVE FILTER

As has been pointed out repeatedly, the essential problem in auto¬

pilot control of launch vehicles is the fact that signals reflecting the
structural vibrations appear at the input to the control actuator. In
many cases the phase relationships are such that the feedback loops
amplify the magnitudes of these signals leading to catastrophic instability.
Because of the fact that the bending and rigid body (control) frequencies
are not well separated, conventional passive filtering is not always
effective. This is, in a sense, intuitively apparent, since if there are no
sharp distinguishing features between two groups of objects the separa¬
tion cannot be sharp; in short there is a large “gray” area.
As long as one thinks in terms of separating signals on a frequency
basis alone, this dilemma remains. However, in a launch vehicle control
system the bending and control frequencies, while in close proximity
in a frequency sense, are usually widely separated in relative damping
factor. Typically a control mode has f = 0.6 to 0.8 while for the
bending mode f = 0.005 to 0.02. This fact forms the basis for an ele¬
gant and sophisticated scheme due to Zaborsky,(6>7) which he termed
Digital Adaptive Filtering.
The main idea involves identifying the dynamic modes in a given
signal, after which the signal may be decomposed and expressed as a
sum of individual modes. The effectiveness of this procedure requires
that these modes be well separated in the s plane; frequency charac¬
teristics, as such, play no unique role. After the signal decomposition
has been accomplished, only predetermined modes may be passed on to
the control system with the result that there is theoretically perfect
suppression of undesired modes.
The crucial elements in this technique are the identification and de¬
composition features. It is appropriate, therefore, to motivate the dis¬
cussion by considering the general aspects of this problem.
We assume that the transfer function for the system under considera¬
tion has the form

V(s)
= G(s)
U(s)
k

12 a» 8s'
i=0

(24)
12 bi s*
<-0
776 ELEMENTS OF MODERN CONTROL THEORY

IT (s - zi)
i=0

= K -

n Cs- p<)

The output v(t) and its first (n — 1) derivatives have the initial values

d4 v(t)
tv - v,u (o)

i = 0, 1, 2, (n - 1) .

One may then write the Laplace transform of the output as

5Z ai s4 H V(,) (o) J2 bj s'-4-1

i=0 i=0 i= »+l
(25)
V(s) U(s) +
n

H bi s4 12 b<s4
t=0

The system input is written as

u(t) = 2Z Ri t4 (26)
«=0

or

U(s) tllRi = <+1

(27)
i= 0

Substituting this into Eq. (25) and taking a partial fraction expansion,
we find

n—o g+h+i r>

V(s) = z + T] — (28)
w=\ W=1 b

where g is the order of the singularity of G(s) at s = 0.

If there are y real poles and n — g — y complex poles, then the
inverse Laplace transform of Eq. (28) becomes
ADAPTIVE CONTROL 111

v(t) = £ K* ^ + E 2 |Kj «-«*»cus (fiat +/KS)

x=l z==3/-f-l

?+>*+l
D„, t”’-1
+ E (29)
W>=1
(w - 1) ! '

Here the second summation extends over the complex poles in the
upper half s plane. For present purposes it is presumed that the form of
v(t)as shown by Eq. (29) is known. However, the parameters

K* and p, x = 1, 2, • • •, y

K, = |K,| /Kx
> x = (y + 1), • • (n - g)
and p, = —ax + i/3, ;

D„ w = 1, 2, • • •, (g + h + 1)

are unknown. As shown above, these constitute a total of (2n - g + h + 1)

parameters.* In order to identify these, we may take a total of (2n - g +
h + 1) measurements of v(t), thereby giving the proper number of equa¬
tions that in principle may be solved for the above unknowns.
A more feasible procedure from a practical point of view is to identify
these parameters by means of a least-squares procedure.
Suppose for example we observe the M values (where M> 2n - g + h +
1)

Vrr (t N- M+l)

Vm (t N-M+'i)

Vm (W)

and define the quantity

F (tAT-M+l)= W (i) [vm(t.y_M+i) V (W-M+*)] (30)

*One may without loss of generality assume that bn = 1 in Eq. (24), in which case the number of
unknown parameters becomes 2n — g -f- h.
778 ELEMENTS OF MODERN CONTROL THEORY

where W(i) is an optional weighting factor. For example, it might be

desired to weigh recent measurements more heavily than others. Then
the condition for a least-squares fit becomes

Min J2 F2(V-jf+«) . (31)

r> K D 1=1

This leads to the (2n — g + h + 1) equations

~ (F • F) = 0 (32)

X = 1, 2, • • •, (n - g)

^;<F-F)=0 (33)

x = 1, 2, • • - , (n - g)

m. (F %F> = 0 (34)

w = 1, 2, • • •, (g + h + 1)

the solution of which yields the (2n — g + h + 1) unknown parameters,

p®, Iye, ±)w.
However, the set of Eqs. (32)-(34) is nonlinear, and their solution,
even by computer, is a formidable task. We may take the point of view
that identification is periodically repeated at judiciously selected inter¬
vals such that approximate values for p*, K*, and D„ are known. In this
case we write

P* = Pxat + Apx

K* = Kxn + AK* (35)

F)W = Dyjjy ADy,

where the pIJV, KIJV,DWJVare initial estimates and the A( ) quantities are
assumed small. The value of v(t) at some generic timet* is then given by
ADAPTIVE CONTROL 779

v(t*) = v*(t*) + E - v*(t*) Ap*

x

(36)

+ 22 V*(tfc)AK» + 22 ^5" V*(t*) AD,

where the asterisk signifies that the nominal (subscript N) parameters are
used. Using Eqs. (35) and (36), the set of Eqs. (32)-(34) is linear and
may be solved for Ap*, AKX, and AD„, thereby determining updated
values for px, Kx, and D„.
Once the present values of px, Kx, and Dro are known, Eq. (29) com¬
pletely determines the signal v(t). Furthermore, Eq. (29) determines this
signal as a sum of its individual components. Any component or any
group of components can then be separated from the signal v(t) by
computing in real time a partial sum which includes only the desired
components. This separation of the signal components can best be done
on a digital computer. The output of the digital computation can then
be converted, in line, to an “analog” form and used to replace the actual
v(t) for the purposes of control. By this process it is possible to restrict
the composition of the control signal to only the desired modes of
response; for example, the rigid body modes of a missile. All other
modes, such as the elastic modes, can be filtered out simply by leaving
them out of the summation of Eq. (29) as carried out on the computer.
Clearly this filtering operation should be effective for filtering any
modes for which the roots of the characteristic equation, px, are separated
by a large distance in the s plane. It is not required that the frequencies
be different; only the total distance counts. In fact, two modes of the
same frequency are effectively filtered if their damping is quite different.
The digital instrumentation of the operation described in the pre¬
ceding section is sketched in Fig. 6.19. An analog-to-digital converter
samples and digitizes the measured continuous signal v(t) every T seconds.
The present sample is v(NT) where N is the total number of samples
from the beginning of the operation. A fixed number M of the most
recent samples is stored in the digital computer. When a new sample
arrives, it is put at the first address, the other samples are shifted to the
left by one place, and the oldest sample in the memory is dropped. An
equal number of samples produced by the computer, which represents the
present fitted curve, is stored in the block below. These latter samples are
computed from the present best estimates of the parameters px, K*, and
 ELEMENTS OF MODERN CONTROL THEORY
780

D„ (denoted by pw, KXNand Dw) and jointly represent the sampled form
over the last r = MT seconds of the v(t) or v(t; px, Kx, Dw) signal of Eqs.
(29)—(36). The difference of signals v(t) and vm(t) over the last t = MT
seconds

<t>(t)= v(t) — vm(t) (37)

is used to determine through an inversion of Eqs. (32)-(34) the new

estimate pXAr+1,KXJV+1,andDw+iby Eq. (35). More precisely, the operation
is digital, involving the sampled form of $(t), the difference of the
samples in the two blocks of Fig. 6.19. These sets of samples can be
represented as vectors or column matrices

<t> = v — Vm (38)

with coordinates

<J>i= <t>[(N- M + i) T]

= v[(N - M + i) T]

- v„[(N - M + i) T] . (39)

Naturally, Eqs. (32)-(34) are also set up and inverted in a discrete

and digital version. Once the new estimate pxw+i, Kw+i, and Dw+i is
obtained by the computer from the discrete versions of Eqs. (32)—(34)
and (35), then the updated samples v[(N — M + i)T] for the lower
memory block in Fig. 6.19 can be obtained from Eq. (29) and the
digital computer is ready to take in the next sample of vm(NT) and repeat
the outlined cycle. Simultaneously, the computer has also obtained such
a partial sum of Eq. (29) that consists of those modes only which com¬
prise the desired filtered signal v/NT). This latter is fed through a
digital-to-analog converter directly to the control system and the air¬
frame. Note that, ideally, this operation completely suppresses the
undesirable components of the signal; the filtering is perfect.
The application of the above ideas to a typical launch vehicle auto¬
pilot is shown in Fig. 6.20.
While the complete signal decomposition and digital filtering process
can be successfully instrumented with high speed airborne computers, it
undoubtedly puts considerable demands on the computer. Thus, the
process should be simplified.
ADAPTIVE CONTROL 781

of
Schemat
6.19.
Figure
Filter.
Adaptiv
Digital
782 ELEMENTS OF MODERN CONTROL THEORY

o
33
a
o
3
<

%§
>

c
3

&
4
<

*§b
5

c
.o
o
—

a
D.
<

©
<N
d
a>
fc-

I
h
ADAPTIVE CONTROL 783

This simplification can be achieved if the plant output signal v(t)

contains a predominant desired mode or modes that are damped at a
much taster rate (approximately f = 0.7) than those signal components
to be suppressed (approximately f = 0.005).
Let it be assumed that pi,2 = —a. ± j/3 represents a damped oscillatory
(rigid body) mode that comprises v'(t), the desired part of the input
signal. Then from Eq. (29)

v'(t) = 2 |Ki| e~°“ cos (fit + /Kt)

(40)
= Ae~a‘ cos /3t + /3e~at sin /3t

where the constants A and B replace |KX| and /%. The total input signal
is

v(t) = v'(t) + v"(t) (41)

where v"(t) denotes the undesired portion ofv(t).

Assuming that Eq. (40) represents the predominant portion of v(t),
the problem reduces to one of identifying A, B, a, and fi by suitable
measurements and computer computations.* This operation is to be
performed each time a sample is received; the updated values are available
by the time the next sample arrives. It has been found, however, that a
and 13change very slowly and may be estimated at more widely separated
intervals than A and B. Accordingly, we will describe in detail the esti¬
mation procedures for A and B (a and /3 assumed known and invariant)
only. The four parameter estimation procedure is identical in principle,
involving merely a more complicated “bookkeeping.”
From Eqs. (36) and (40) we have

v'(t) « v(t) = A^e at cos /3t + ^3Ncat sin /3t

(42)
+ (e~at cos /?t) AA + (rat sin /3t) AB

where A*, and B^ are the current estimates of A and B, respectively.

Suppose we have available M measurements, denoted by the vector

* Actually it can be shown that the desired portion of v(t) can be readily identified even if the
undesired portion, v"(t), is not negligible, providing that the relative damping of the latter is
small compared to v'(t). See Ref. 14.
784 ELEMENTS OF MODERN CONTROL THEORY

Vm

Vm (W—if+z)

(43)

_ vm (tw) _

In the present case the set of Eqs. (32)—(34) becomes

M
d
X F(W+i) = 0 (44)
d(AA) i=l

d(AB)
f) F2(t*_*+i)
= 0. (45)
i= 1

These two equations, which are linear in AA and AB, may be readily
solved for AA and AB, giving the new estimate

A^ = A* + isA (46)

B*+i = B* + AB . (47)

When the next measurement, vm(ty+i), becomes available, the process

is repeated using the new estimates above and the measurements,
vm(ty—Af+2),Vm(W— jf+s), • • •, VOT(tiv+l).
In a typical launch vehicle application of the form shown in Fig. 6.20,
v(t) is the total error signal, containing the rigid body, bending, and other
modes. The digital filter output consists essentially of the rigid body
portion of v(t) as shown in Eq. (40). Subtracting this from the total
command error results in a signal that consists essentially of the signal
components associated with the elastic oscillations.
It is then feasible to close the loop for the vr signal through a compen¬
sating network, as shown in Fig. 6.21, that is designed for optimum
rigid body response while excluding for the purpose of this design the
elastic poles and zeros from G(s). Conversely, the loop is closed for ve
through a compensation designed for outimum damping at the bending
modes while disregarding for this design the rigid body poles and zeros in
G(s). If the ve control branch is omitted, the bending modes will oscillate
ADAPTIVE CONTROL 785

Figuie 6.21. Basic Configuration for Signal Decomposition by Digital Filtering.

open loop and attenuate with the small structural damping. In this
latter case the digital filter simply prevents closed-loop excitation of the
bending modes, which might make them unstable.
The operation of a typical launch vehicle autopilot employing the
configuration of Fig. 6.20 is shown in Fig. 6.22, which illustrates the
response to a one-degree step command. It is apparent that there is firm
control of the vehicle as long as there is a component in the error corre¬
sponding to the predominant rigid body modes; that is, as long as there
is a transient. As the transient signal dies out, however, the digital filter
output fades out and the control loop for the airframe opens. This
loss of control is especially objectionable when the airframe is aero-
dynamically unstable.
This feature represents the major problem in the application of the
digital adaptive filter to launch vehicle control systems. The difficulty is
that the digital adaptive filter in effect closes the loop for the rigid body
transient response but keeps the loop open for the body elastic oscil¬
lations and all other signals. This open-loop operation is the desired
condition as far as the body elastic oscillation is concerned. However,
there are desired signals, such as guidance commands and feedback
signals resulting from wind shear, that must be transmitted just as the
rigid body transients are transmitted. The frequency band of these
signals is usually significantly lower than the frequency of the rigid
body transient. In order for the control system to respond to these
desired signals, the signal rejected by the digital adaptive filter (residual)
may be passed through a secondary filter, as illustrated in Fig. 6.23. This
786 ELEMENTS OF MODERN CONTROL THEORY

TOTAL MEASURED ATTITUDE ANGLE, 0

CO
UJ
UJ
OC
'O
ll]
Q

W
a:
o
w
Q

TIME (sec)

Figure 6.22. Basic Transient Response to One-Degree Step Attitude Command.

filter will have a low cutoff frequency so that it will adequately pass
slowly varying command signals yet suppress the elastic oscillations.
Analysis indicates that a conventional linear filter may be suitable for
this secondary filtering operation. The combination of the digital
ADAPTIVE CONTROL 787

DIGITALADAPTIVEFILTER
I- 1FITTED

Figure 6.23. Block Diagram of Vehicle Control System with the Digital Adaptive
Filter and Secondary Linear Filter.

adaptive filter and secondary filter will then transmit the desired com¬
ponent of the total error signal but will reject body bending oscillations.
With a secondary filter added in the manner shown in Fig. 6.23, the
system whose response is illustrated in Fig. 6.22 now exhibits the response
shown in Fig. 6.24. It is apparent that there is no longer any loss of
control. For best operation the secondary filter is cut in when the digital
filter output reaches a predetermined level (at approximately two sec¬
onds in Fig. 6.24). This cut in time is not critical; there would be similar
results with a ± 50 percent tolerance on this value.
The response shown in Fig. 6.24 combines the speed and accuracy of
the digital filter control shown in Fig. 6.22 with the steady-state stability
of the secondary filter. A small amplitude residual oscillation with the
low predominant frequency of the sluggish secondary filter can be ob¬
served in Fig. 6.24, but the overall response is still nearly ideal.
In Figs. 6.22 and 6.24 the system, including the bending modes, was
initially at rest at time t = 0 when the step input was applied. Figure
6.25 shows the system response when the three bending modes are
already oscillating with a rather high amplitude at the time of applying
788 ELEMENTS OF MODERN CONTROL THEORY

TOTAL MEASURED ATTITUDE ANGLE, 9

TO!

DEGRE
0 5 1.0 1.5 2.0 2.5 3.0 3 5

ELASTIC BENDING ATTITUDE ANGLE

DIGITAL FILTER OUTPUT

nr rm mm
Iff!]
1.6
Ein np liii Hi 1 1ft liii jjlM II % I 1 i P in liii E *TTT
ml
"T
I 1 m ; i\ 1
j Hiiij Ii!I ittitr ttr if P 1m {rrl
....

I
....

ml
:i i :
' ;i igl tb ;[ l;i:

0.8
*4r
35
it::
liii
titii ft
1
du JliL
si |Hj ii ii SSI
m 1 Iliii f
r tt

£ Si
I S It;
1 HH j:t.
uT n:
• • •*

liii
Hi:
did. jf m: Y % Fm fr~ rfff P
5a .
1 tf8 j!i
* - i
»+J
ttr
1 1 H1SB IS
.... — -

0
Pji L
::::
i m hi;
4—
Tr
jitt
Hi 2 HI i{I] hit IfIi
—

yii st* UJ iLi-ii ::::

-0.8
ttr
I §gjf i fH: d; • tSf| tjf Tf j|: liii Hjf frtf frfi

-1.6 1i i | 11§ mm
m lip!Hii
Us U} Ii iiliiilil
Siti
HHiiiii:
ii j
f
1 IIi jjj jl
1 Jl |
11 i ]
s
c) 0.5 1.0 i .5 2 .n 2 !.5 3 .0 3.

TIME (sec)

Figure 6.24. Basic Transient Response as in Fig. 6.22 but Control Transferred to
Secondary Filters at t = 1.96 Sec.

the step input command. It can be observed in Fig. 6.25 that the digital
filter output curve regains its clear damped sinusoidal form after the first
quarter cycle, although in the total error, v(t) = 1 - 0(t), the rigid body
ADAPTIVE CONTROL 789

TOTAL MEASURED ATTITUDE ANGLE, 0

DEGREE RIGID BODY ATTITUDE ANGLE, 6

ELASTIC BENDING ATTITUDE ANGLE

DIGITAL FILTER OUTPUT

-0.8

-1.6

1.5 2.0

TIME (sec)

Figure 6.25. Basic Transient Response as in Fig. 6.22, but with Applied Step.

mode is nearly swamped by the presence of the elastic oscillations.

During the first quarter cycle of approximately 0.5 second the input
790 ELEMENTS OF MODERN CONTROL THEORY

data are insufficient for positive identification of the rigid body mode;
consequently, some of the bending signal component is seen in the
digital filter output during this period. Comparison of the digital filter
output in Figs. 6.22 and 6.24 reveals that the presence of body bending
oscillations does not cause significant deterioration in the rigid body
response. The bending modes still oscillate essentially with open-loop
frequency and damping. Different amplitude selections of the initial
bending oscillations will have varying effects on the late part of the sys¬
tem response, but the transient remains essentially unaffected as long as
the bending amplitude does not much exceed half the amplitude of the
step command.
What would happen in the case shown in Fig. 6.25 if no digital filtering
were incorporated in the control system is revealed by Fig. 6.26. Com¬
parison of Figs. 6.25 and 6.26 makes it possible to appreciate the effect
of the digital filter with signal decomposition in stabilizing the bending
modes.
Instead of using a secondary filter, another possible solution to the
loss-of-control problem can be the periodic restarting of the fitting
process. This simply consists of periodically resetting the length of the
fitting interval to M = 1 sample in the computer. Then the computer
raises this number one sample at a time until Mmax, the ultimate length of
the fitting interval, is reached. Such restarting will assure the continuous
supply of small transients because there always will be some small error
at the instant of restart. This initial error is then eliminated by the control
action. The resulting sequence of successive small transients will assure
continued stability. It also assures the stepwise following of any con¬
tinuous control signals dc that may be present. Discontinuities in the
command dc or its lower derivatives must be monitored. Such discon¬
tinuities mark the start of new transients and should occasion the
switching to M = 1 and the ensuing development of a new fitting
cycle.
The discussion in the preceding paragraphs assumes that the length of
the fitting interval is stationary; that is, the length of fitting interval does
not change from one sample to the next. This type of operation is only
justified if the entire fitting interval is filled with a signal branch of a
transient. Consequently, the length of the fitting interval must be made
variable when new transients start. Immediately after the start of a new
transient, the fitting interval will be only one sample. Then the fitting
interval will gradually increase until it reaches the length of the desired
stationary fitting interval. This transition is referred to as the fade-in
period.
ADAPTIVE CONTROL 791

TOTAL MEASURED ATTITUDE ANGLE, 0

DEGRE
0 0.5 1.0 1.5 2.0

ELASTIC BENDING ATTITUDE ANGLE

0 0.5 1.0 1.5 2.0

TIME (sec)

Figure 6.26. Transient Response as in Fig. 6.25, but Digital

Filter Replaced by Unity Gain.
792 ELEMENTS OF MODERN CONTROL THEORY

Two problems arise in conjunction with the fade-in period: (1) it is

necessary to somehow identify the start of a new transient branch and
(2) the quality of curve fitting during the fade-in period must be investi¬
gated.
For a detailed discussion of this, we must refer the reader to Zaborsky’s
report/6)
The performance of the Digital Adaptive Filter in a launch vehicle
control system was extensively investigated via computer simulation/6)
Some of the results are shown in Figs. 6.27 through 6.34. The control
configuration is as shown in Fig. 6.23 where the secondary filter is a
simple first order lag with a cutoff frequency equal to one-fourth of the
lowest rigid body frequency. The sampling interval in the Digital
Adaptive Filter is normalized to unity when the number of samples per
closed-loop rigid body cycle is equal to 25.
Figure 6.27 illustrates the system response to a unit step input when
the vehicle is completely rigid (no bending modes). Curve fitting in the
digital computer is based on two parameters, A and B. The response
consists of an almost pure second-order sinusoid. It is easy to see that
this case is handled smoothly and effectively by the Digital Adaptive
Filter. The amplitudes, A and B, decay to zero, of course, as the error
signal approaches zero.
The same case with one bending mode added is shown in Fig. 6.28.
The rigid body is still controlled in a well damped, second order mode
while the bending mode operates essentially open loop with a slight
structural damping, as shown by the command error signal curve. Note
that the rigid body control remains tight and stable even when the
bending oscillation greatly exceeds the rigid body component.
The response to an alternating step input is shown in Figs. 6.29 through
6.31. The time period between successive step inputs was varied from
100 percent to 50 percent of the closed-loop rigid body period. An
elastic mode oscillation of twice the closed-loop rigid body frequency
with 0.05 damping ratio and an initial condition of 0.2 times the ampli¬
tude of the command input signal was present in these runs. In Fig. 6.29
the period of the input is long enough so that the fitted curve is com¬
pletely determined. In Figs. 6.30 and 6.31 the curve fitting process is
interrupted by the alternating input. It may be seen that the rigid body
output response remains stable and tight. The noticeable increase in the
body bending component shown in the command error signal of Fig.
6.30 at the initiation of several of the new step inputs is attributed to
two facts:
ADAPTIVE CONTROL 793

TIME (Number of Samples)

TIME (Number of Samples)

Figure 6.27. Control System Error Response to a Unit Step Input.

a. The curve fitting process does not start until the fourth sample, thus
allowing a new unfiltered excitation to the bending mode.
b. The phase relationship between the bending frequency and the input
is less favorable to curve fitting as each succeeding step is applied.

Note that the succeeding steps of Fig. 6.31 do not produce the same trend.
Figure 6.32 shows the response to a ramp input. Since the system
used here has only one integration in the open loop, a steady-state error
794 ELEMENTS OF MODERN CONTROL THEORY

0.20

0.15

0.10

0.05

-0.05

-0.10

0 8 16 24 32 40 48 56

TIM E (Number of Samples)

Figure 6.28. Response to a Unit Step Input (Bending Mode Added).

will develop in the rigid body output response. This is clearly observed
in the figure. Also apparent is the fact that the rigid body output
ADAPTIVE CONTROL 795

LEGEND

- -RIGID BODY OUTPUT RESPONSE

- COMMAND ERROR SIGNAL

- INPUT COMMAND

AMPLITUDE
SIGNAL
ERROR
COMMAND
AND
CURVE,
FITTED
OUTPUT,
BODY
RIGID \
\

A
\\ / /

\j
/
\ /
\ /
\/

y ^ — • /
/V--
V
/ \

u
<y V

/
21 -

0
-

10 J-
-
L
- -

30 4ii
- -

50
-

60
-

70 75

TIME (Number of Samples)

Figure 6.29. Response to an Alternating Step Input with a Half-Period of Twenty-Five

Samples.

Figure 6.30. Response to an Alternating Step Input with a Half-Period

of Eighteen Samples.
796 ELEMENTS OF MODERN CONTROL THEORY

Figure 6.31. Response to an Alternating Step Input with a Half -Period of Twelve Samples.

increases smoothly in one cycle (twenty-five samples) to the steady-state

condition. This indicates that the digital filter separates the correct
“fitted curve” from a “command error.” Although the fitted curve goes
to zero, a residual error signal is transmitted through the secondary filter
so that the modified error signal is not zero and the system continues to
follow the ramp command.
The time required to establish a “fitted output” depends strongly on
the number of samples used per fitting interval. From the standpoint of
economy in the airborne computer it is desirable to reduce the number of
samples as much as is possible without deterioration in performance.
Figure 6.33 shows both the rigid body output response and the command
error signal for four runs in which the number of samples per rigid body
cycle is varied from 8 to 50. The response curves shown in Fig. 6.33 all
contain a body bending oscillation of twice the closed-loop rigid body
frequency and an initial amplitude of 0.2 times the command error
signal. The response with both twenty-five and fifty samples is good, and
very little difference may be noted between these runs. A moderate
departure in the response may be noted when twelve samples are used,
and a substantial departure may be noted when only eight samples are
used.
ADAPTIVE CONTROL 797

Figure 6.32. Control System Response to a Ramp Command Input.

It may be concluded that the number of samples per closed-loop rigid

body cycle can be held to approximately twelve if necessary, although a
greater number of samples is desirable if computation time is available in
the computer.
The most dramatic feature of the Digital Adaptive Filter is its capa¬
bility to separate the predominant rigid body mode from the bending
mode even when the two are equal in frequency. This condition is
illustrated in Fig. 6.34.

Remark: The Digital Adaptive Filter scheme is an ingenious and power¬

ful method for removing parasitic signals in a control system.
It is severely limited (at least as far as application to launch
vehicle autopilots is concerned) by the fact that its ability to
respond to disturbance (wind) inputs is questionable. In Ref.
6, the combined command and disturbance inputs have been
simulated and the performance appears to be adequate. How¬
ever, no simulation results with a pure disturbance input are
shown. As a matter of fact, NASA engineers, with access to
an as yet unpublished study on this problem, reported thab2>
798 ELEMENTS OF MODERN CONTROL THEORV

This method appears to be feasible only when the com¬

ponent of the gimbal input signal, due to the command
inputs, dominates the component due to the disturbances,
and the commands are of a specified form. Since wind
disturbances grossly override command inputs from the
guidance system, the digital adaptive filter does not appear
to be applicable for launch vehicle uses.

Thus, while the Digital Adaptive Filter system has provided an elegant
solution to the main problem, its application to launch vehicle auto¬
pilots is frustrated by difficulties that present no problem in conventional
approaches. Whether some bright control engineer will be blessed with
new insight to revitalize this method remains to be seen.

6.4 NOTCH FILTER METHODS

A simple and highly effective way to gain stabilize the bending modes
in a feedback control system is to employ notch filters whose resonant

W
Q
P
H
P

<
P
<
£
O
oo

a
O
cd

£
C
U

<
H

(X

>*
Q
O
CQ

a
o
5

TIME (Fractions of a Rigid Body Periods

Figure 6.33. Control System Response with Variation in Sampling Rate per
Rigid Body Cycle.
ADAPTIVE CONTROL 799

AMPLITUDE
SIGNAL
ERROR
COMMAND
AND
CURVE,
FITTED
OUTPUT,
BODY
RIGID

Figure 6.34. Control System Response with Body Bending Frequency Equal to Rigid
Body Frequency.

NOTCH NOTCH NOTCH

6c >—weE
COMPENSATION —
NO. 1 NO. 2 NO. 3

RATE GYRO
NO. 1
1 1 1
VEHICLE 1 1 16
ACTUATOR
DYNAMICS

<XH-
RATE
NO.
GYRO
2
LP 1 1 1

SPECTRAL
FILTERS

DISPLACEMENT
GYRO

Figure 6.35. Spectral Identification Adaptive Control System

800 ELEMENTS OF MODERN CONTROL THEORY

frequencies are set equal to the bending mode frequencies. There are
two fundamental difficulties in this approach; namely the bending mode
frequencies are not known a priori with sufficient accuracy, and they
vary with flight time due to variation in vehicle inertial properties.
It would appear that if one could track the bending mode frequencies
automatically during flight and use this information to vary the resonant
frequency of a notch filter accordingly, the problem would be solved.
This approach has indeed been studied extensively and the methods
proposed differ only in the means whereby the bending mode frequencies
are identified and in the manner of mechanizing the notch filter. The
two most promising techniques are discussed below.

6.4.1 Spectral Identification Filter

Because of the elastic oscillatory motion of the vehicle, the control

system sensors (angular displacement, rate, etc.) will contain signals at
the bending mode frequencies in addition to those reflecting rigid body
motion. The basic principle in the spectral identification method is that
if the power spectral density of the sensor output is measured, there will
be peaks at the bending mode frequencies. The problem is then one of
mechanizing some scheme whereby these peaks are identified, and using
this information to set the notch filter resonant frequency. A schematic
of the overall control system configuration is shown in Fig. 6.35. It is
assumed that only the first three bending modes are significant and,
therefore, three notch filters are used. Note that by using two rate
gyros (located at two different points along the vehicle) and subtracting
ADAPTIVE CONTROL 801

their outputs, a signal is obtained which is free of rigid body information.

Thus, the input to the spectral filters consists only of bending mode
signals. The output of the spectral filters (shown by the dotted lines in
Fig. 6.35) is used to set the resonant frequency of the notch filters. The
crucial element in this configuration is the spectral frequency identifi¬
cation system. This will now be examined in some detail.
A typical input to the spectral filter will have the form shown in Fig.
6.36. This signal is, in general, nonperiodic so it cannot be represented
as a summation of terms with discrete frequencies (Fourier series). In
fact, f(t) contains components at all frequencies. To obtain a represen¬
tation of f(t) as a function of frequency, co, we may proceed as follows.*
Define a function f r(t) such that

T T (48)
fr(t) = f(t), - - < t — = 0, otherwise.

This function is illustrated in Fig. 6.37. If the graph of fT(t) is repeated

every T seconds, we obtain a periodic function of period Tj, which is
expressible as a Fourier series. It would appear plausible that as T is
increased without limit, the Fourier series for fr(t) will approach that of
f(i).
Since fr(t) has been prolonged as a periodic function, its Fourier series
is obtained in the usual way, as follows.

where

(50)

(51)

If we let

*The mathematically sophisticated reader will recognize the following presentation as a heuristic
development of the basic results of Fourier Integral theory.
802 ELEMENTS OF MODERN CONTROL THEORY

2irn

then the preceding relations become

A 2 00 (52)
f r(t) = — + — X] cos cont + B„ sin co„ t)
i t „=1

f T/2
An = I f(t) COSCOrt,
t dt (53)
J -T/2

fT/2
B n= f(t) sin con t dt . (54)
•'-T/2
ADAPTIVE CONTROL 803

We now define the quantity

Actin = w,+i — wn

‘27r(n + 1) 27rn
T Y~
2ir y
~ Y '

By substituting this in Eq. (52), the latter becomes

\ 1 x°°' (55)
fr(t) = — H— (A, cos w„ t + B, sin w, t) Aw, .
TJ T n=J
,

However, since w0 = 0, we may write

Aq Ao AcOq
T 2ir

Ao cos coot
Acoq
2ir

After adding and subtracting A0/T in Eq. (55) we obtain

\ 1 00 (56)
f„(t) = — — + - 23 (A, cos w, t + B„ sin u, t) Aw, .
T7T »= 0

It seems reasonable to suppose that as T approaches infinity the

infinite summation above approaches an integral. This can indeed be
established rigorously.* Equation (56) then takes the form
/• OO /»<X

f(t) = / A(w) (cos wt) dw + / B(w) (sin wt) dw (57)

•'n •'n

where

cos wt) dt (58)

B(<•})= -[ f(t) (sin wt) dt . (59)

T J0

See any standard text on Fourier integrals.

804 ELEMENTS OF MODERN CONTROL THEORY

Equation (57) is one form of the Fourier Integral. The functions A(«)
and B(to) are the cosine and sine transforms, or Fourier transforms, of the
function f(t).
Note that in a Fourier series we get contributions from integral values
of wt/2 7T. In a Fourier integral there are contributions from every real
value of co.
For present purposes the importance of the representation in Eq. (57)
is that the total squared amplitude of f(t)at the frequency co4is

A2(<ot) + B2U) . (60)

A spectral filter, tuned to m, is merely a device that computes the

quantity of Eq. (60). In practice, the interval of integration in Eqs.
(58) and (59) is computed with sufficient accuracy for a time length
equivalent to approximately five periods of the spectral tuned frequency
03k-

For application to a control system configuration of the type shown

in Fig. 6.35, 24 spectral filters were used to span the expected frequency
range of the bending modes/8) Because the spectral filter output
amplitudes will be greatest when the tuned frequency of the spectral
filter is near a bending frequency, the three tuned frequencies associated
with the three spectral filters having the largest amplitudes are used to
set the resonant frequencies of the three notch filters.
With twenty-four spectral filters in the system the rapid computation
of the integrals, Eqs. (58) and (59), may impose excessive requirements
on the airborne computer. In Ref. 8 the approximate values of A(«) and
B(co) were computed by replacing the sine and cosine functions by a
square wave of amplitudes + 1 and - 1, with the sign change occurring
when the respective sine or cosine functions changed sign. This resulted
in a significant reduction in computer complexity without markedly
impairing system performance. For a thorough analysis of the system
the reader is referred to the Autonetics Technical Summary report/8)

Remark: The technical feasibility of the Spectral Identification Adaptive

Control system has been established by extensive computer
simulation. Applied to a launch vehicle of the Saturn class,
the computer requirements are a memory capacity of approxi¬
mately 2000 words (word length = 16 bits) and a 5-n sec add
time.
It has been found that if a bending mode is well stabilized
and its excitation is low, then the identification is poor. On
ADAPTIVE CONTROL 805

the other hand, when the bending mode stability is poor, the
bending energy is high and the bending frequencies are well
identified by the spectral system. The performance variation
is thus in the proper direction.
The same study181 also noted that the third mode identifi¬
cation was, in general, poorer than either the first or the
second mode identification. This was attributed to several
possible causes: '
a. There is an attenuation of high frequencies due to system
lags, and thus the third mode energy is in general lower
than the first and second mode energy.
b. The spectral filter accuracy decreases as the number of
samples per tuned frequency period decreases, thus the
accuracy of the higher frequency spectral filters is not as
high as the lower frequency spectral filters.
c. The actual frequency being identified is the closed-loop
bending frequency, which may shift more radically for
the third bending mode.
The general question of how close the lowest bending mode
frequency can be to the rigid body frequency without having
the notch filter phase lag deteriorate rigid body response to
unacceptable proportions has not been investigated.

6.4.2 Adaptive Tracking Filter

Instead of performing the frequency tracking and filtering functions

separately, it is possible to mechanize a device that tracks the bending
mode frequency continuously (instead of in discrete fashion as in Sect.
6.4.2) and sets the notch filter resonant frequency accordingly. This
concept has been mechanized in various ways/9'13)
The principle of operation of one type of frequency sensor may be
described with reference to Fig. 6.38. For simplicity we assume that
ein is a sinusoid of the form
ein = A sin (wt + a) .

Consequently, it satisfies the equation

ein + to2 ein = 0 .

This suggests a type of feedback system of the form shown. If the

output of the integrator is initially zero, a nonzero voltage will appear
806 ELEMENTS OF MODERN CONTROL THEORY

Figure 6.38. Frequency Sensor Principle of Operation.

at the output of the summing junction. This in turn produces a voltage

at the output of the integrator which causes ein wT2to be nonzero. When
the null position is reached (ee = 0), the output of the integrator, uT,
is exactly equal to w, the frequency of the input signal.
Of course the use of pure differentiation renders this scheme quite
impractical. A usable form of this principle is shown in Fig. 6.39. Here

Figure 6.39. A Practical Frequency Tracker.

ADAPTIVE CONTROL 807

the use of a double lag filter in conjunction with the differentiation

limits the high frequency (noise) amplification. An identical lag filter
is used in the parallel loop to preserve the phase relationship. In this
circuit it is assumed that the input signal ein is composed of several sinu¬
soids. A constant voltage coMis summed with the integrator output to
assure that only signals whose frequency exceeds will be sensed. The
function of the input filter is to assure that only that sinusoid of lowest
frequency (greater than coM) is operated upon. This input filter has a
transfer function of the form

2
C0j>

“j" y C01 S COip

If the integrator output is initially zero, then ccT = coM,and since ee is

positive cof increases. Denoting by the lowest frequency in e,„
greater than aiM, we see that when uT = uR the output from this filter,
denoted by e2, will contain the signal of frequency wR highly amplified
and all higher frequency signals sharply attenuated. Effectively then, ei
will contain only the signal of frequency «*, and in the null state u>T
will equal cos.
A simplified dynamic analysis of this scheme may be performed to
obtain some crude estimates of performance quality. For this purpose
we may neglect the influence of the input and double lag filters, in which
case it is readily found that cor satisfies the equation

(61)

If it is assumed that ei is a sinusoid of the form

ei = A sin cot

then it follows that

COy 2 (62)
- TT- j + WT
AK sin cot

Here A, K, and « are positive constants. The solution of Eq. (62) is

808 ELEMENTS OF MODERN CONTROL THEORY

— + tan /3
w
(63)

1 + — tan 0
u

where

co0= UT(0)

0=2 n + sin2 —
)

n = < >
7r

and < > denotes the maximum integer of the argument; e.g., <23/4> = 5.
It is noted that the system is inherently stable (in this simplified
version). Furthermore, the manner in which the input signal amplitude,
integrator gain, and initial setting of wT influence the speed with which
uiT approaches « is indicated explicitly.
A means whereby the signal frequency is tracked and the notch filter
resonant frequency set automatically is shown in the schematic of Fig.
6.40.
The demodulator is half wave and phase sensitive with a transistor
switch performing the demodulation function. To describe the operation,
we assume that e<„ is composed of a single signal of fixed frequency.
Now since e'in and ei differ in phase only by the contribution of the
(Krs2 + 1) term, we see that e,'„ is either in phase or 180° out of phase
with ei. Assuming that the resonant frequency of the notch filter is above
the frequency of the input signal ein, we note that ei„ and ei are in phase.
Consequently, the demodulator output is positive, which, in turn, acts to
increase KT, thereby causing the resonant frequency of the notch filter
to decrease. If the notch filter resonant frequency is below the input
signal frequency, then e,;n and ei are 180° out of phase and the reverse
occurs. When the two frequencies are equal, then the phase difference
ADAPTIVE CONTROL 809

Figure 6.40. An Adaptive Tracking Notch Filter.

between e,;„ and ei is 90° and the average value of ec is zero and eK nulls
to a fixed value that is a measure of the input signal frequency.
It is easy to see that, in the steady state, the overall transfer function
is

ei _ Krs2 + 1
&in K7.S2 -|- T2S-j- 1

where

1
Kr =

coa= frequency of -input sinusoid, ein.

The bandpass filters have the transfer function

t32s2 + s2
(raS + l)2 '

These are designed to attenuate low frequency (rigid body) signals. As a

further precaution against tracking low frequency signals, a lower limit
may be set on the integrator output, thereby preventing the notch filter
810 ELEMENTS OF MODERN CONTROL THEORY

resonant frequency from tuning to a rigid body frequency. Another

means of ensuring that rigid body signals are not tracked is to feed the
difference of two gyro signals into the unit, thereby effectively removing
the rigid body content from the frequency tracking input.
Figure 6.41 is a detailed schematic of the system shown in Fig. 6.40.
The amplifier gain K was varied by using in the amplifier feedback loop a
variable gain potentiometer driven by a servo motor whose input signal
was the integrator output. This system, which was built as a bread¬
board model, is discussed in detail in Refs. 9 and 11. A typical result
obtained with this unit is shown in Fig. 6.42.
The most extensive analysis to date of the tracking notch filter con¬
cept and its application to launch vehicle autopilots is the one by
Cunningham and SchaeperkoetterO4) which was performed in the course
of a NASA funded study. Included in this study were:

a. Time required to track the frequency as a function of this frequency.

b. Tracking error.
c. Attainable notch attenuation.
d. Demodulator properties.
e. Effect of higher harmonics.
f. Multiple frequency inputs.

The adaptive tracking filter analyzed by Cunningham and Schaeper-

koetter is shown in Fig. 6.43. This is a somewhat more generalized
version than the types considered thus far, and has the capability for a
variety of specialized operations. For a constant co„it is readily found
that

&out_ Xcs2 X/,Wps T \a cop2 (64)

e in S2 “l- p Q)p S 4“ CO
p2

This has the general form of a complex lag-lead network. The impor¬
tant feature here is that after the bending mode frequency has been
tracked the filter may be used for phase stabilization of the bending
mode rather than gain stabilization, which is achieved by the use of a
notch filter.
With phase stabilization the control system provides the proper gain
and phase characteristics at the bending mode frequency to obtain a
closed-loop damping of the mode greater than the open-loop damping.
Gain stabilization, on the other hand, provides enough attenuation at
the bending mode frequency to ensure system stability regardless of
ADAPTIVE CONTROL 811

e
1

R
7
vW
/

ek

Figure 6.41. Schematic of Adaptive Tracking Notch Filter.

b. OUTPUT SIGNAL, e

-
— - — —

'

in
c. INTEGRATOR (XJTPUT, e,
k

Figure 6.42. Typical Operation of Adaptive Tracking Filter.

812 ELEMENTS OF MODERN CONTROL THEORY

NOTCH FILTER

I_ _ _ _ _ ,_ _ _ _

Figure 6.43. Schematic of Adaptive Tracking Filter.

the bending mode phase. Depending on the specific application, one

type may exhibit superior features over the other.
To use the configuration of Fig. 6.43 as a notch filter, we simply put
Xb= 0, and \a-\c= 1; viz.

— = - — + “p2_ . (65)
e<n s2 + 2£p <jivs + Up2

Typical frequency response characteristics of the two types are shown

in Figs. 6.44 and 6.45.
In the Frequency Sensor part of Fig. 6.43 there exists the capability
for three types of reference signal er for the demodulator, depending on
the position of switches Si and S2.
We designate these as follows.

CASE SWITCH TYPE

Si s2
I Closed Open High Pass
II Open Closed Low Pass
III Closed Closed All Pass
ADAPTIVE CONTROL 813

AMP
RAT
(deg)
(dB)
PHA NORMALIZED
FREQUENCY,u/cjp
 ELEMENTS OF MODERN CONTROL THEORY
814

AM
RAT
(deg
(dB)
PHA
0.1 0.2 0.4 0.6 0.8 1 2 4 6 8 10

NORMALIZED
FREQUENCY,
<j/(jp
Figure 6.45. Frequency Response of Notch Filter, Eq. (66).
ADAPTIVE CONTROL 815

The respective er/ej„ transfer function and frequency-response char¬

acteristics are shown in Fig. 6.46. A selection of a particular type is
determined by the specific application. For a launch vehicle autopilot the
use of Case I is indicated, since it is desirable to filter out the rigid body
and propellant slosh frequencies as much as possible.

MAGNITUDE PHASE CASE

II

III

Figure 6.46. Reference/Input Signal Transfer Functions.

It remains to examine the operation of the demodulator. We will

investigate three types that are capable of detecting the phase angle
between the input and reference signals. These are the multiplier,
chopper, and double chopper demodulators.
For ease of exposition assume that the input signal contains a single
frequency, viz.

e,n = A sin ua t . (66)

If «a > up an examination of Fig. 6.46 shows that the reference signal

eT leads the input by an angle <p that is between zero and 90 degrees.
Therefore, we write

er — C sin (coat —
F <p)
(67)
0 < y? < 90° .
816 ELEMENTS OF MODERN CONTROL THEORY

The output for each of the three types of demodulators is shown in

Fig. 6.47. It is apparent that the average value of demodulator output
ed is positive in each case. For the multiplier type we have

1 (68)
= —AC cos ^ .

When = «0, that is, when <p= 90°, we see that the average value of
ed is zero, indicating that in the steady state the resonant frequency of
the filter is equal to the frequency of the input signal.
After expressing the output of the chopper demodulator in a Fourier
series, the fundamental (d.c.) component is found to be

2A COS ^

7T
(69)
fund

MULTIPLIER CHOPPER DOUBLE CHOPPER

3. ADAPTIVE TRACKING FILTER FREQUENCY

BELOW INPUT FREQUENCY (0 < <p < -|)

Figure 6.47. Operation of Various Types of Demodulators.

ADAPTIVE CONTROL 817

Similarly, for the double chopper type

ed (70)
fund

Thus, in each case the average output of the demodulator is zero

when <p = 90°. The effect of higheT harmonics is investigated in the
Cunningham and Schaeperkoetter report.
A similar analysis shows that when ua < uv (and, therefore, er leads e,„
by an angle <psuch that 90° < <p< 180°) the system tends to decrease o>p
and lead to the steady state <o= 90° with wp = ua.
A schematic of the complete autopilot employing two adaptive
tracking filters is shown in Fig. 6.48. Extensive simulation studies of this
system as applied to the Saturn IB vehicle are contained in Ref. 14.

Remark: Many studies have shown that the adaptive tracking filter con¬
cept is feasible and practical. Among the objections that may
be leveled against it are that the instrumentation is relatively
complex and, therefore, involves a compromise with reliability.
There are also lower limits in the ratio of allowable first
bending mode to rigid body frequency. Nevertheless, this
approach is a prime candidate for adaptive control of highly
flexible launch vehicles.

Figure 6.48. Control System Utilizing Adaptive Tracking Filters.

818 ELEMENTS OF MODERN CONTROL THEORY

6.5 FREQUENCY INDEPENDENT SIGNAL PROCESSING

The method to be described here is not, strictly speaking, an adaptive

technique, since it presupposes a knowledge of bending mode data, and
all gains and compensation are designed a priori. It is, however, a novel
solution to the problem of separating rigid body from bending mode
signals, whatever the ratio of the respective frequencies. Unlike passive
filtering methods the desired (rigid body) signal is generated free of any
phase lag or attenuation and (theoretically) free of all parasitic modes.
The method involves the use of “processing functions” and requires that
the number of sensing elements be equal to the number of modes of
motion considered. Rejection of parasitic mode signals is accomplished
by making the processing function associated with each sensing element
a prescribed function of modal slopes or displacements.
The advantages of this method are its simplicity and the absence of any
need for onboard computational capability. Since the modal data are
assumed known, the processing functions may be precomputed and no
further onboard computations are required. Also, because of the fact
that the method does not rely on the relative frequencies of the closed-
loop rigid body and elastic modes, the separation and rejection of bending
motion are accomplished even in situations where a bending frequency
coincides with a closed-loop rigid body frequency.
The main idea is due to Howard/15) The ensuing discussion essentially
parallels his presentation.
According to Eq. (A 10) the displacement gyro output is

0 — Or 4" q(j) .
;

For notational convenience in the following discussion we will write

this as

(71)
a a qlj>
i=i

where

6i = output of displacement gyro located at station i along the vehicle

aij = the bending mode slope of the jth bending mode at location i
n = number of bending modes.
ADAPTIVE CONTROL 819

If there are n + 1 gyros at n + 1 locations, then from Eq. (7 1) we have

01 % O' 10 0*11 G In Or

$2 021 • • • * 0*271 q CD

=
q(2)
(72)
. '

On O'tiO &7ll & %n

071+1- -On+1,0 O’n+l.l 0*n- 1-1. n - q(n)

where

o+o = 1 for i = 1, 2, • • •, n + 1 .

This may be expressed concisely in matrix notation as follows.

© = crq (73)

Solving for the column vector q

q = o--1© (74)

where

Zio z* z. z n+1,0

Zn z, z* z n+ 1,1

z. z. z n+1,2 (75)

Zi z. z- z n+\,n

D = det [c]

and Z a is the cofactor of the element in the matrix a.

Equation (74) gives the rigid body rotation dR and the n generalized
coordinates q(i) as functions of the measured outputs from the n + 1
820 ELEMENTS OF MODERN CONTROL THEORY

sensors and the known modal data. In particular

(76)

If we define

(77)

then Eq. (77) may be written as

n+l
(78)
X Pi(<r)Oi.
i=l

The quantity P<(<r)is called the attitude processing function, which is a

function of the modal data only (assumed known). Consequently, if n
bending modes are significant, then the use of n + 1 sensors, together with
the known attitude processing functions, Pi(o-), is sufficient to generate
the rigid body rotation signal dR free of all bending mode information.

n+l

0ft = X P»(°0 0j (79)

t=l

where 08-is the output of the rate gyro located at station i, and P<(cr) is as
defined by Eq. (77).
In the case of n = 2, for example (which presupposes that bending
modes higher than the second may be neglected), we find that

X P.fr) 0.

where
ADAPTIVE CONTROL 821

Zso
Ptto
D -5( 011022 012 021
)
D = ((72
1032 02203l) (oil O'32 — 01203l) T (oil 022 — (71202])

and similarly for the rate signals.

We see that Lhe attitude processing functions are “gains” to be used
with the gyro outputs (0i, 02, d3) to generate a signal free of bending mode
information. The scheme is implemented in the manner shown in Fig.
6.49.

Remark: Subject to the stipulation that bending mode data are known,
the above method is an elegant and attractive means of ob¬
taining a theoretically perfect filtering of bending mode signals,
which is in no way affected by the frequency separation be¬
tween the desired and parasitic signals. Extra sensors are the
only additional equipment required and there is no need for
“exotic” signal processing.
In application to real vehicles there are two obvious limita¬
tions: (1) there are inherent limits in the accuracy of bending
mode data, and (2) the bending mode properties vary with
flight time, due to varying properties of the vehicle as fuel is
expended.

Figure 6.49. Block Diagram of System for Processing Attitude and Attitude Rate Signals.
822 ELEMENTS OF MODERN CONTROL THEORY

Howard’s study<15) includes an extensive investigation of the

influence of errors in modal data on the closed-loop stability
of the system. For a typical launch vehicle it was found that
when the amplitude of the rigid body pitch attitude is greater
than or equal to the amplitude of the bending motion at the
nose of the vehicle, the error in the pitch attitude output will
always be less than 16 percent if the modal slope errors do not
exceed either + 15 percent or - 50 percent.
An analysis of the influence of modal errors on the stability
of the closed-loop system indicated that certain combinations
of modal errors tend to degrade stability of at least one of the
bending modes, whereas other combinations of modal errors
tend to enhance bending mode stability. For modal errors of
10 percent or less, and those combinations of modal errors
that tend to degrade stability, no instability occurred in either
mode when the loop was closed with nominal gain. The results
indicate, however, that, for the nominal range of gain consid¬
ered, the first bending mode is more sensitive to errors in the
modal data than the second, and, presumably, higher bending
modes.
It has been shown that when the error coefficients are
positive, stability is maintained without degradation. This
suggests that an attempt be made to bias the nominal value
of the modal slopes in such a way as to ensure that the error
coefficients are always positive. Examination of the tabidated
data for a typical vehicle reveals that it is possible to do this
for a range of modal error. This is equivalent to the use of
structural feedback to stabilize the bending modes in the
presence of errors in the modal data.
As far as the time varying properties of the modal data are
concerned, it would appear that gain switching in the attitude
processing functions at preselected intervals would be a plaus¬
ible solution. However, this must be investigated in detail for
each particular vehicle.
 APPENDIX A

VEHICLE DYNAMICS

The equations of motion and conventional control techniques for a

launch vehicle are described in Volume Two. For purposes of complete¬
ness, and pertinent to the needs of the present chapter, a simplified
summary of the equations that describe the system dynamics is presented
below.

6r = He 5 + Ha a (Al)

m(w — U0 dR) = Tc 5 — La a (A2)

(s2 + 2 r(<) «<« s + [«<»]*) q ^ S

(A3)
i = 1, 2, • • %

(s + Kc)5 = Kc 8C (A4)

s 8C = Kx(s + K,) dE (A5)

6e — 6c 6? (A6)

W
a — — T aw (A7)
U0

Ww
(A8)
a* ~ ~ u; •

Rate gyro output

6 = eR + X>c(<) q«> . (A9)

i

Displacement gyro output

6 = dR+ £> . (A10)

823
824 ELEMENTS OF MODERN CONTROL THEORY

Bending displacement

u = 2>c« (4) q(« (t) . (All)

The vehicle geometry and sign conventions are shown in Figs. 6. A 1 and
6.A2. Typical normalized mode shapes are shown in Fig. 6.A3. Nomen¬
clature is defined as follows.

I = moment of inertia of vehicle about pitch axis

Ka = servoamplifier gain
Kc = engine servo gain
K, = integrator gain
Ks = rate gyro gain

Figure 6.A1. Vehicle Geometry.

ADAPTIVE CONTROL

Figure 6.A2. Sign Convention for Bending Parameters.

I = length parameter along vehicle; positive in aft direction

to = distance from vehicle mass center to engine swivel point
4 = distance from vehicle mass center to center of pressure
= aerodynamic load per unit angle of attack
m = mass of vehicle
M<» = generalized mass of ith bending mode
q <») = generalized coordinate of ith bending mode
s = Laplace operator
t = time
Tc = control thrust
u = bending deflection
U0 = forward velocity of vehicle
826 ELEMENTS OF MODERN CONTROL THEORY

1. <p6) is normalized to unity at engine gimbal point.

Figure 6.A3. Typical Normalized Mode Shapes.

w = normal velocity of vehicle

a = angle of attack
5 = rocket engine deflection angle
&c = command signal to rocket engine
Or — rigid body attitude angle
0E = error signal
Op = feedback signal
0C = command angle
Me = Tc tjl
Ma fa/I

o(v) = negative slope of ith bending mode = —

d(
ADAPTIVE CONTROL 827

<to(i) = <r(,) (to) = value of cr(<) at gyro location

V>(i) = normalized mode shape function for i*h bending mode

f (l\ w(,) = relative damping factor and undamped natural frequency

fori* bending mode

Sloshing and engine inertial effects have been neglected. The engine
actuator has been represented by a' simple first order lag and gyro
dynamics have been neglected. This is a valid approximation for studying
the lower frequency elastic modes.
The above model is adequate for exhibiting the salient features of the
adaptive methods considered in this chapter.
In studying the stability properties of the system one may also assume
that

Kr 0

and

a 6r .

The definition of a given by Eq. (A7) is useful mainly in determining

response to wind inputs.

REFERENCES

1. J. F. Lee, “A Digital Adaptive Flight Control System for Flexible

Missiles,” Seventh Symp. on Ballistic Missile and Space Tech.,
1962, pp. 115-147.
2. J. C. Blair, J. A. Lovingood, and E. D. Geissler, “Advanced Control
Systems for Launch Vehicles,” Astronautics and Aeronautics,
August, 1966, pp. 30-39.
3. G. Lang and J. M. Ham, “Conditional Feedback Systems— A New
Approach to Feedback Controls,” AIEE Trans., Vol. 74, July, 1955.
4. A. Kezer, L. G. Hofmann, and A. G. Engel, “Application of Model
Reference Adaptive Control Techniques to Provide Improved
Bending Response of Large Flexible Missiles,” Sixth Symp. on
Ballistic Missile and Space Tech., New York, Academic Press, Inc.,
1961, pp. 113-151.
828 ELEMENTS OF MODERN CONTROL THEORY

5. G. E. Tutt and W. K. Waymeyer, “Model Feedback Applied to

Flexible Booster Control,” IRE Trans, on Automatic Control, May,
1961, pp. 135-142.
6. J. Zaborsky, W. J. Fuedde, and M. J. Wendl, “Control Through
Digital Filtering by Signal Decomposition with Application to
Highly Elastic Boosters,” IEEE Trans, on Appl. and Ind., March,
1964, pp. 87-98.
7. J. Zaborsky, W. J. Luedde, and M. J. Wendl, New Flight Control
Techniques for a Highly Elastic Booster, ASD TR 61-231, Septem¬
ber, 1961.
8. J. M. Johnson, R. K. Crow, and C. F. Lutes, Study of Structural
Bending Adaptive Control Techniques for Large Launch Vehicles,
Technical Summary Report, Contract No. NAS8-20056 (Auto-
netics, Inc.), March 16, 1966.
9. A. Greensite, An Advanced Autopilot for Elastic Booster Vehicles,
General Dynamics Convair Report No. ERR-AN-038, February,
1961.
10. G. W. Smith, “Synthesis of a Self Adaptive Autopilot for a Large
Elastic Booster,” IRE Trans, on Automatic Control, Vol. AC-5,
No. 3, August, 1961, p. 229.
11. A. Greensite, “A Frequency Sensor and Roving Notch Filter for
Control System Applications,” Proc. Nat’l. Elec. Conf, Vol. 21,
1965, pp. 558-563.
12. R. Gaylor, R. L. Schaeperkoetter, and D. C. Cunningham, “An
Adaptive Tracking Filter for the Stabilization of Bending Modes in
Flexible Vehicles,” AIAA/JACC Guid. and Contr. Conf, Seattle,
1966, pp. 441-447.
13. H. H. Hosenthien and M. T. Borelli, An Adaptive Tracking Notch
Filter for Suppression of Structural Bending Signals of Large Space
Vehicles, NASA TM X-53000, October 1, 1963.
14. D. C. Cunningham and R. L. Schaeperkoetter, Study of Applica¬
tions of a Tracking Filter to Stabilize Large Flexible Launch Vehicles,
Final Technical Report, Contract No. NAS8-20080 (Sperry Inc.),
May, 1966.
15. J. C. Howard, A Frequency Independent Technique for Extracting
the Rigid Body Motion from the Total Motion of a Large Flexible
Launch Vehicle, NASA TN D-3 109, November, 1965.
16. P. H. Whitaker, “Design Capabilities of Model Reference Adaptive
Systems,” Proc. Nat’l. Elec. Conf, Vol. 18, 1962, pp. 241-249.
17. R. E. Andeen, “Self Adaptive Autopilots,” Space /Aeronautics,
April, 1965, pp. 46-52.
ADAPTIVE CONTROL 829

18. R. E. Andeen, “Stabilizing Flexible Vehicles,” Astronautics and

Aeronautics, August, 1964, pp. 38-44.
19. E. B. Stear, “A Critical Look at Vehicle Control Techniques,”
Astronautics and Aerospace Eng., August, 1963, pp. 80-83.
20. R. K. Smyth and J. C. Davis, “A Self Adaptive Control System for
a Space Booster of the Saturn Class,” Proc. JACC, 1962.
21. R. E. Andeen and P. P. Shipley, “Digital Adaptive Flight Control
System for Aerospace Vehicles,” AIAA Journ., Vol. 1, No. 5, May,
1963, pp. 1105-1109.
22. R. Carney, “Design of a Digital Notch Filter with Tracking Require¬
ments,” IRE Trans, on Space Elec, and Telemetry, December, 1963,
pp. 109-114.
23. E. B. Stear and P. C. Gregory, “Capabilities and Limitations of
Some Adaptive Techniques,” Nat’l. Aerospace Elec. Conf, 1962,
pp. 644-660.
24. R. K. Smyth, R. M. DuPlessis, and L. K. Mattingly, “A Digital
Self Adaptive Body Bending Filter for Flexible Airframe Control,”
Nat’l. Aerospace Elec. Conf., 1960, pp. 369-376.
25. L. T. Prince, Design, Development, and Flight Research of an
Experimental Adaptive Control Technique for Advanced Booster
Systems, ASD-TDR-62-1 78, November, 1962.
26. R. R. Mitchell, Rapid Identification of the Elastic Mode Frequencies
of a Space Booster, General Dynamics Convair Report No. ERR-AN-
499, May 5, 1964.
27. G. G. Lendaris, “The Identification of Linear Systems,” AIEE Trans,
on Appl. & Ind., September, 1962, pp. 231-240.
28. R. Roy and K. W. Jenkins, Identification and Control of a Flexible
Launch Vehicle, NASA CR-551, August, 1966.

PROBLEMS AND EXERCISES FOR CHAPTER SIX

1. A certain system has a transfer function

C(s)
G(s) .
R(s)

We seek to devise a scheme whereby C(s)/R(s) will behave like

M(s), a given transfer function. Suppose we introduce a feedback
scheme such that
830 ELEMENTS OF MODERN CONTROL THEORY

C(s) = G(s) E(s)

E(s) = K [R(s) - B(s)]

B(s)

where K is a gain constant.

Show that, for large K, the transfer function for this system is
approximately

C(b)
M(s) .
R(s)

Consequently, in this case, variations in the plant transfer function,

G(s), have negligible influences on the closed-loop response.
What difficulties could be anticipated in the practical implemen¬
tation of this scheme?

2. As an alternate solution for Problem 1 , consider th& system

C(s) = G(s) E(s)

E(s) = R(s) - K [C(s) - B(s)]

B(s) = M(s) R(s)

where, again, M(s) represents the desired behavior of the closed-loop

response, C(s)/R(s). Calculate C(s)/R(s) in this case and investigate
the behavior for large K.
Are there any advantages of this scheme over that used in Prob¬
lem 1?

3. Consider a revised form of the problem discussed in Sect. 6.1.1.

Specifically, suppose that the vehicle were completely rigid. Thus
we suppress Eq. (8) and reduce Eq. (1 1) to the form

df = (Rjj s -f- 1) da .

Show how the MIT scheme could be applied to the problem of

coping with variations in Kx, the open-loop gain.
 ADAPTIVE CONTROL 831

4. It was mentioned in connection with the gyro blender scheme

(Sect. 6.2.1) that various theoretical and instrumentation imper¬
fections result in the fact that the expression given by Eq. (22) is
not exactly zero. We seek to investigate the influence on system
stability when expression (22) is

a. positive.
b. negative.
/

Use Eqs. (7)—( 10) as the description of the system with the rate
feedback as shown in Fig. 6.12. Take for the system parameters

Ma = 1 Kc = 15

Me = 4 Kx = 3

ad1* = 15 Kfi = 0.5

r(i) = o Tc = 340,000

M(1) = 1600

The quantities K,crGS(1), o-0f(1,are not known individually. All that we

need for the analysis is that expression (22) is either positive or
negative. Plot the poles and zeros of the open-loop transfer function
in the s plane. The essential criterion for determining the influence
of a nonzero expression, (22), on system stability is the behavior
of the angle of departure of the root locus from the (bending) pole
on the imaginary axis. Translate these conclusions into conditions
on <ros(1),a0}X), and K which do not degrade stability.

5. Using the same general approach as in Problem 4, analyze the

system stability features of the phasor cancellation scheme (Sect.
6.2.2) when uF < on and when uF > on.

6. The problem of identifying the dynamic modes of a given system

is fundamental in many adaptive control schemes. Suppose that a
system is described in the state variable format by

x = Ax + Bu
832 ELEMENTS OF MODERN CONTROL THEORY

where x is an n vector and, in the present case, u is a scalar (see

Sect. 1.4). If we assume that u is piecewise constant in intervals of
equal length, then, according to Sect. 1.4.2, we may express the
system in discrete form as follows.

Xx+l — CrXic T Hu*

It is further supposed the matrix G is in the canonical (phase vari¬

able) form. The methods described in Sect. 1.4. 1 can be employed
to accomplish this. Show that (n -f 1) measurements of the input
and output are then sufficient to determine the unknown elements
in G. With this, outline the computational steps required to cal¬
culate the eigenvalues (and therefore the dynamic modes) of the
original system.

7. A control system is described by the following equations (in Laplace

transform notation).

C(s) = jA(b)

A(s) = & E(s)

B(s) = Kb A(s) + C(s)

E(s) = KA [R(s) - B(s)]

where C(s), R(s), B(s), and E(s) represent the output, input, feed¬
back, and error signals, respectively. The quantities KA, Kfl, and
Meare constants.
Show that the closed-loop transfer function is

C(s) _ _ w2
_ (a)
R(s) s2 -f 2 f oi s + co2

where

u* = MeIvA

f = 2 V^mTKx•
ADAPTIVE CONTROL 833

The parameter, nc, is subject to variations from a nominal value,

Hco. It is required to maintain the undamped natural frequency at a
prescribed value, co0,in spite of variations in nc. To do this, the use
of a frequency tracker, which measures the actual co of the system,
is suggested. Then, forming the difference (w0— w), indicate
schematically how this would be used in conjunction with an
integrator (whose output operates on KA) to maintain the closed-
loop frequency at the prescribed value.

8. For the system of Problem 7, suppose that K* (instead of mc) is sub¬

ject to variations. We now seek to maintain f constant in spite of
variations in Ks.
For this purpose, let f = f0 + Af (where f0 is the desired value).
Substituting in Eq. (a) of Problem 7 (expressing the result in the
time domain) we have, after solving for Af

2 o30c Af = w02r — (c + 2 f0co0c + w02c) .

Using differentiators and multipliers as required, show how this may

be mechanized. (Neglect the lag effects which must necessarily be
associated with practical differentiators.) Flow would you modify
this scheme to obtain a signal whose sign depends on the sign of Af
only— not c?
Now having a signal which is proportional to Af, how must this
be used (in conjunction with a suitable integrator) to maintain the
actual f of the system essentially constant?
 Chapter Seven
LEARNING CONTROL SYSTEMS

One of the more exciting concepts that has arisen in recent years is that
of learning control. The basic idea is due to Rosenblatt and his co¬
workers at Cornell University, and stems from an attempt to model the
functioning of the human brain by a network of threshold logic units/4)
The initial experiments produced quite dramatic results and the tempta¬
tion to characterize this model in anthropomorphic terms was irresistible.
However, the passage of time has had a sobering effect on some of the
overly optimistic claims and predictions which were made in the early
days. It was found that the device which they constructed (and called
the Perceptron) was nothing more than a pattern classifier whose esoteric
mysteries were dispelled when viewed in the light of switching theory.
The idea of learning control nevertheless exerted a mesmerizing influence
on researchers in automatic control theory. A good summary of recent
developments in the field is contained in Refs. 23 through 25, which also
include extensive bibliographies.
As far as application to space vehicle control systems is concerned,
significant progress has been made mainly in one area, that of closed-loop
optimal control. It will be recalled from Chapter Five that, in general,
the algorithms of optimal control theory generate trajectories which are
open- loop; that is, they are a function of the initial state of the system.
In practical situations, we usually require closed- loop control, i.e., a
control which is a function of the current state of the system.
An attempt to achieve closed-loop optimal control via the concepts of
learning theory exploits the so-called “generalizing” feature of threshold
logic networks.
Basic studies in this area have been made by Smithd) and Mendel. (2)
The material in this chapter is based on an investigation of a space
vehicle reorientation problem conducted by the author/22)
In the following sections, we restrict ourselves to the time optimal
problem where it is known that the control is bang-bang. All of the tools
834
LEARNING CONTROL SYSTEMS 835

needed to solve the problem are developed from first principles. Our
problem is one of designing a time optimal controller in which the
controller output is a function of the current state of the system, i.e.,
closed loop.

7.1 STATEMENT OF THE PROBLEM

/

A wide class of nonlinear time varying systems may be described by

the following matrix differential equation of motion.

z(t) = h (z, t) + B (z, t) u(t)

z = M X 1 (state) vector
h = M X 1 vector (1)
B = M X r matrix
u = r X 1 (control) vector

The control vector satisfies a constraint of the form

|u| < Uo . (2)

We pose the problem of calculating the control u(t), which satisfies

inequality (2), and which drives the system from an arbitrary initial state,
z(0), = to the terminal state, z(tr) = 0, in minimum time. It is readily
found from the maximum principle that the hamiltonian is*

H = -1 + Xr (h + Bu) (3)

where

m
X (4)
dz

It follows, therefore, that the optimal control is given byt

u*(t) = u0 sgn (BrX) (5)

*The superscript T denotes transpose.

f The asterisk notation ( )* is used to denote the optimal quantity.
836 ELEMENTS OF MODERN CONTROL THEORY

which is obviously of the bang-bang type. As is well known, the calcu¬

lation of u*(t) is not a trivial problem, since initial values for X are not
known. Various iterative procedures are available(6>7> for computing the
optimal control. These result in a trajectory which is a function of the
prescribed initial state, f
However, since it is known that the optimal control is always either
one or the other of two known values, we may view the closed-loop
problem as one of “pattern classification.” In other words, given the
present state of the system, what criterion shall be used to achieve the
proper dichotomy?
In order to pursue this line of reasoning, it is necessary to summarize
those elements of the theory of pattern classification pertinent in this
context. This is done in Sect. 7.2.

7.2 PATTERN CLASSIFICATION

We shall assume that a pattern is an entity that can be characterized

by d numbers, which we denote by xi, x2, . , xd. The vector whose
components are (xi, x2, . , xd) will be called the pattern vector, and
will be denoted by X. It is further assumed that each pattern vector
belongs to one of R categories.* Then the process of pattern classification
may be represented schematically in the manner shown in Fig. 7.1. The
output of the pattern classifier, i0, is either 1, 2, 3, . , or R; i.e., the
category to which the input pattern vector belongs.
In order to exhibit further the structure of the pattern classifier, we
introduce the concept of a discriminant function. Let gi (X), g2(X), . ,

'

X1 »

xi fa
PATTERN

•
CLASSIFIER
•
•

Xd- - %º
PATTERN

Figure 7.1. Schematic of Pattern Classification Process.

"% Occasionally we shall write Xi to denote a specific class of pattern vectors, and X?1 for the
jth member of this class.
LEARNING CONTROL SYSTEMS 837

gs(X), be scalar and single valued functions of the pattern X. These

functions, which are called discriminant functions, are chosen such that
for allX in category j (j = 1, 2, • • • •, R), we have

gy(X) > g*(X) (6)

where

j, k = 1, 2, • R

j 7^ k .

In other words, if X is in category j, then the jth discriminant function

g;(X) has the largest value. Schematically, the use of discriminant func¬
tions enables one to depict the pattern classification process in the
slightly more detailed manner shown in Fig. 7.2.
The problem now reduces to one of specifying the discriminant func¬
tions and then implementing a suitable maximum selector.

Figure 7.2. Pattern Classification Using Discriminant Functions.

 ELEMENTS OF MODERN CONTROL THEORY
838

For present purposes, we shall be concerned only with classifying

patterns into two categories (R = 2) and with the special case of linear
discriminant functions. These stipulations will lead to consideration of a
special classification device called the threshold logic unit (TLU), which
forms the basic element of the closed-loop time optimal controller to be
developed later.

7.2.1 Linear Discriminant Functions

Suppose we have a set of N distinct patterns {X(1), X(2), . ,X(Ar)},

where each pattern belongs to either category A or category B. We
consider the linear discriminant function defined by
d

g(X) = H WiXi + wd+i (7)

where the w * are constants.

It is convenient to visualize each pattern vector, X0), as a point in a d
dimensional hyperspace. We say that the N patterns are linearly separable
if, and only if, there exists a linear discriminant function of the form (7)
such that
g(X) > 0 for all X in category A
(8)
g(X) <0 for all X in category B .

Clearly, the patterns are linearly separable if, and only if, there exists a
hyperplane such that all the points (patterns) belonging to category A
are on one side of the hyperplane, and all the points (patterns) belonging
to category B are on the other side of this hyperplane. If we define a d
dimensional vector W, whose components are wi, w2, . , wd, then the
equation of the hyperplane takes the form

W •X = — wd+i . (9)

A few simple properties of this hyperplane are immediately apparent:

a. If wd+i = 0, the hyperplane passes through the origin.

b. If w» = 0 for i = 1, 2, . . d, the hyperplane is parallel to theith
coordinate axis.
c. The orientation of the hyperplane depends on the values of wi, w2,
. , w d.
LEARNING CONTROL SYSTEMS 839

The pattern classification procedure summarized above can be mechan¬

ized by a very simple device, described next.

7.2.2 Threshold Logic Unit

The pattern classifier, employing the hyperplane (9), can be imple¬

mented as shown by the schematic of Jug. 7.3. This structure, which
consists of a set of weights, a summing device, and a threshold element,
is commonly called a threshold logic unit (TLU).
The threshold element responds with a + 1 signal if g(X) > 0, and
a — 1 signal if g(x) <0. We then associate a TLU output of + 1 with
pattern category A, and a TLU output of - 1 with pattern category B.
The TLU is the basic building block of a wide variety of so-called
“learning machines.” The terminology is unfortunate and misleading
since “learning” refers only to the well-defined interactive procedure for
calculating the W vector for the hyperplane, Eq. (9). Apparently, some
of the early investigators in the field could not resist the temptation to
characterize this technique as the mechanical analog of conditioned reflex
in sentient creatures. Pattern classification is, of course, a more accurate
and descriptive term.
It is clear from the above discussion that the fundamental problem of
pattern classification via TLU is the calculation of the W vector. This
subject is treated in the following section.

7.2.3 Dichotomization Techniques

In order to facilitate the subsequent discussion, we will reformulate

the basic problem in more convenient terms. We define an augmented
weight vector, a, of dimension (d + 1), whose first d components are the

Figure 7.3. Threshold Logic Unit.

 ELEMENTS OF MODERN CONTROL THEORY
840

same as W and whose (d + l)th component is wd+1. We also define an

augmented pattern vector Y, of dimension (d + 1), whose first d com¬
ponents are the same as X, and whose (d + l)th component is+ 1.
There is given a training set of N augmented pattern vectors where it
is known that the set

{Y0(1),Ya(2),. , Y.<*>} (10)

belong to category A, and the set

{Y6(1\Y6(2),. ,Y6<">} (11)

belong to category B. Since Eq. (9) and set (10) comprise the complete
training set, we have

N = L + M (12)

It is required to determine a such that

Y aU) %OL> 0
(13)
j = 1, 2, ') L

Ybw %a < 0
(14)
k = 1, 2, - , M .

If sets (10) and (11) are linearly separable, then a vector, 5, satisfying
(13) and (14) can always be found. These weights are then used in the
TLU of Fig. 7.3, which then acts as the pattern classifier. We turn to a
discussion of methods for calculatings.

7.2.4 Error Correction Procedures

The earliest method!4) for determining a was called a “training”

method, since the patterns were presented one at a time to the TLU and
an adjustment was made in the weights depending on whether the pattern
was correctly or incorrectly classified by the TLU. The patterns may be
presented in any order, but it is necessary that each pattern in the train¬
ing set be tried several times. The order of presentation is immaterial.
LEARNING CONTROL SYSTEMS 841

When the patterns are presented cyclically, each pass through of the
training set is called an iteration.
Before training begins, the TLU weights (i.e.,«) are set to any arbitrary
value, say zero. If the TLU correctly identifies a pattern, no adjustments
are made in a. Suppose now that the TLU responds incorrectly to an
augmented pattern vector Y. Then the current a is changed to a new
value, «, as follows. '
a = a T" pY

if Y belongs to category A
(15)
a1 = a — pY

if Y belongs to category B
where p is a scalar greater than zero.
If p is constant at each trial, this is known as the fixed increment rule.
As long as the patterns in the training set are linearly separable, the
above procedure will generate an a, which will correctly identify all the
patterns in the training set after a finite number of iterations.
One of the earliest proofs of convergence of the above procedure was
given by Block. (4) Many other investigators have derived alternate proofs.
There exist variations in the above procedure whereby p is modified
in the course of the iteration in order to speed up the convergence. A
full discussion of these is contained in Nilsson’s book. (3)

7.2.5 Theory of Inequalities

The anthropomorphic overtones of the manner of calculating a may be
dispelled by expressing the basic problem in terms of the theory of linear
inequalities.!9 dO) For this purpose, we define an N X (d + 1) matrix, D,
as follows.
[ Y„M ]*%
[ Ya(2) ]*%

D = [ Y,<*> ]t % (16)
- [ Y6(1) ]r
- [ Y(,(2) ]*•

- [ Y»<*>]r
842 ELEMENTS OF MODERN CONTROL THEORY

In this section, we shall write Y and a. without a bar to emphasize their

interpretation as (d + 1) X 1 matrices.
The problem now becomes one of finding a (d+ 1) X 1 matrix (column
vector), a, which satisfies the linear matrix inequality

D« > 0.

The error correction procedure of Sect. 7.2.4, when expressed in

these terms, would then take the form

a (i + 1) = a(i) + pDT {sgn [ |z(i)| ] - sgn [z(i)]| (18)

z(i) = Da(i) —7 (19)

where p is a positive constant, and 7 is an N dimensional column vector

whose components are fixed (but arbitrary) positive constants. A con¬
venient value for p and the components of 7 is unity. A suitable initial
value for a is «(0) = 0.
While convergence of this algorithm is guaranteed^) (as long as the
patterns are linearly separable), the convergence is usually very slow.
A method which is exponentially convergent has been developed by
Ho and Kashyap.O°) This is summarized by the equations:

«(o) = D# 0(o) (20)

y(i) = D «( i) - 0(i) (21)

«(i + 1) = «(i) + pD# [y(i) + |y(i)|] (22)

0(i+ 1) = j8(i) + p[y(i) + |y(i)|] (23)

0(o) > 0, but otherwise arbitrary. (24)

The condition

0 < p < 1 (25)

is sufficient to ensure convergence. In Eqs. (20) and (22), the symbol

# denotes pseudo inverse.*

*Also called generalized inverse. See Ref. 1 1 An elementary discussion of this topic is contained
in Appendix A.
LEARNING CONTROL SYSTEMS 843

The main difference in computational complexity between this and

the preceding algorithm is the need to calculate the pseudo inverse of D.
This in turn requires a conventional matrix inversion of (DrD), which is a
(d + l)X(d + 1) matrix. Since (d+ 1) is generally not excessively large
in most problems of interest, this is not too serious a liability.
An extremely important corollary of the Ho-Kashyap algorithm is the
built-in check on linear separability. Thirds stated as follows.

The occurrence of a nonpositive vector, y(D, at any stage indi¬

cates that the pattern vectors are not linearly separable, and,
therefore, that no solution vector exists for the problem.

It would be instructive at this point to apply the above methods to a

simple case.

An Example. Suppose that we are given the augmented pattern

vectors

0 1
0 0
y/> = Y (2) -
0 1A ~ 0
1 1

1 1
0 1
y,(3) = 1
li

0
1 1

which belong to category A, and the set

— - — - " “
—

0 0 1 0
1 0 1 1
v ,(2) - v (3) - v (4) _
* B — I B — 1 B —
1 1 1 0
1 1 1 1
-
-
_ _

which belong to category B. These eight pattern vectors constitute the

complete training set.
Using a TLU of the form shown in Fig. 7.3 (with d = 3), determine the
weights of(wi, w2, w3, w4)—i.e., the a vector— such that i„ =+ 1 if the pat¬
tern is in category A and i„ = — 1 if the pattern is in category B.
844 ELEMENTS OF MODERN CONTROL THEORY

The (unaugmented) pattern vectors, X, considered as points in three

dimensional space, are shown in Fig. 7.4. It is visually apparent that the
points in categories A and B may be separated by a plane. In the general
case, the determination of whether the patterns in a given set are linearly
separable is not a trivial task. There are various techniques available for
doing this.O2) This problem is taken up in detail in Sect. 7.3.1, especially
as it relates to TLU realizability for closed-loop time optimal control.
We now focus our attention on the calculation of a.
First let us use the error correction procedure of Sect. 7.2.4. Taking
p = 1 and an initial value for a of zero, we present the eight augmented
pattern vectors to the TLU sequentially, and adjust the weights according
to the rule given by Eq. (15). The procedure is summarized in Table 7.1.
We see that, at the fourth iteration, all patterns are identified correctly
and the corresponding weights are

— “ — —

Wi 3
W2 -2
w3 -3
w4 1
_ — _ _

To determine a solution vector, a , using the Ho-Kashyap method, we

form the D matrix of Eq. ( 16), as follows.

Q Denotes Category B

Figure 7.4. Geometric Representation of Pattern Vectors.

LEARNING CONTROL SYSTEMS 845

Table
1.Summary
7.
of
Correction
Error
Procedure
 ELEMENTS OF MODERN CONTROL THEORY
846

C
Procedure
Correction
Error
of
1.Summary
7.
Table
( ontinued)
LEARNING CONTROL SYSTEMS 847

0 0 0 1
1 0 0 1
1 0 1 1
1 1 0 1
0 -1 -1 -1
0 0 -1
1 -1 -1 -1
0 -1 0 -1

Using the method of Appendix A, the pseudo inverse is found to be

-1 1 1 1 1 1 -1 1
-1 -1 -1 1 -1 1 -1 -1

4 -1 -1 1 -1 -1 -1 -1 1
2 1 0 0 0 -1 1 -1

If we now take

/3r(o) = [1 1 1]

then from Eq. (20), we obtain

1
-1
«(0)
-1
.5

Consequently

.5
1.5
.5
.5
Da(o)
1.5
.5
.5
.5
 ELEMENTS OF MODERN CONTROL THEORY
848

Since all the components of this vector are positive, «(o) is a solution
vector.
Thus, the Ho-Kashyap method obtained a solution with only one
iteration. .
Finally, we seek to obtain the solution vector, a, by the algorithm of
Eqs. (18) and (19). For starting values we take

p — 1

a(o) - 0

7r = [1 1 1 1 1 1 1 1] •

Then we find, successively

ar(l) [4 -4 -4 0]
[Da(l)]r -
[0 4 0 0 0 4 4 4]
zr(l) [-1 3 -1 -1 —
1 3 3 3]
<xT(2) -

[8 -4 -4 4]
[Da(2)]r =
[4 12 8 8 4 0 -4 0]
zr(2) =
[3 11 7 7 3 - 1 -5 - 1]
«r(3) =
[6 -8 -8 -2]
[Da(3)]r =
[-2 4 -4 -4 18 10 12 1

zr(3) _
[-3 3 -5 -5 17 9 11 9]
ar(4) =
[10 -6 -6 4]
[Da(4)]r =
[4 14 8 8 8 2 -2 2]
zr(4) =
[3 13 7 7 7 1 -3 1]
«r( 5) =
[8 -8 -8 2]
[Da(5)F =
[2 10 2 2 14 6 6 6] .

The occurrence of a positive Da terminates the algorithm, and we see

that, in the present case, a (5) is the solution weight vector.
Note that each method yielded a different value fora. This is not at all
surprising since, if a separable hyperplane exists, there must exist an
infinity of these. An examination of Fig. 7.4 makes this intuitively
evident.

7.2.6 Generalization Properties

Given a training set of augmented pattern vectors, there are two

questions which arise immediately; namely, are the patterns linearly
LEARNING CONTROL SYSTEMS 849

separable, and, if so, how long will it take to determine a solution

vector, a (i.e., the separating hyperplane). These questions have been
treated briefly in Sects. 7.2.4 and 7.2.5.
The really crucial and fundamental problem, as far as application of
TLU pattern classifiers is concerned, may be stated as follows.

“How well will the separating hyperplanes separate patterns which

are not included in the training set?”

This is a difficult and largely unsolved problem. It seems reasonable

to assume that new patterns, which are “close” to the old patterns (i.e.,
those in the training set), will be correctly identified. Indeed, there is
extensive empirical evidence to substantiate this claim. (M) However, to
express this idea quantitatively in the general case is very difficult. It
would appear to be a function of the location and orientation of the
separating hyperplane relative to the points in pattern space. Any of
the methods for determining the separating hyperplane, as given in Sects.
7.2.4 and 7.2.5, will yield only one such hyperplane. Determining if
there is indeed a “best” separating hyperplane is a difficult problem. It
would seem intuitively plausible that if the number of training patterns
were an appreciable fraction of the total patterns to be encountered by
the TLU, then the probability of misclassification of a new pattern
would be small.
In order to examine these ideas quantitatively, we must introduce
some appropriate metric. Now the equation of the hyperplane is given
by Eq. (9); viz.

WX = - wd+i . (26)

It is easy to show that the normal distance from an arbitrary point X to

this hyperplane is

X-W + wd+1 (27)

|W|

If there are many points in the pattern set for which f is small, then it
seems plausible that new points (i.e., not in the training set) which are
near training set points having a small t have a significant probability of
being incorrectly classified by the TLU. For this reason, it appears
desirable to orient the separating hyperplane such that the minimum (
850 ELEMENTS OF MODERN CONTROL THEORY

for points in the training set be as large as possible. A means of doing

this is described by Gluckmand13)
There remains the more difficult problem of ascertaining quantitatively
the probability of misclassifying patterns which are not in the training
set. As mentioned previously, this question is crucial in practical appli¬
cations. In this respect, some partial, though significant, results have
been obtained by Cover. O4)
To begin with, we say that an augmented pattern vector, Y, not in the
training set, is ambiguous relative to a given set of hyperplanes if two of
these hyperplanes correctly classify the training patterns, but yield dif¬
ferent classifications for the new pattern vector. A simple illustration of
this idea is shown in Fig. 7.5. Lines A and A both correctly classify the
seven patterns in the training set. However, they yield different classifi¬
cations for the new (unaugmented) pattern X<2). Thus, X(2) is ambiguous
relative to this particular dichotomy of the set of training patterns.
However, “new” patterns X(1) and X(3) are not ambiguous relative to
this dichotomy.
Now if there are N pattern vectors in a d dimensional space, it is
known that the total number of dichotomies that can be achieved by
linear separation (i.e., by a TLU) is(3)

C(N, d) (28)

where the notation (*) denotes the binomial coefficient, viz.

><o
%¡

Figure 7.5. Ambiguous and Nonambiguous New Pattern Vectors.

LEARNING CONTROL SYSTEMS
851

(0 - inrry, ^
Cover has shown that:(H) “If each of the linearly separable dichot¬
omies of N patterns in d space is equally probable, then the probability,
A (N, d), that a new pattern X is ambiguous relative to a random linearly
separable dichotomy is

A(N, d) = > d_ 12 . (30)

C(N, d)

In control problem applications, N and d are large numbers and the

computation of A (N, d) by Eq. (30) becomes extremely awkward. It is
shown in Appendix B that a close approximation for A (N,d) is

A(N, d) gs — —- . (31)
N - d

This is valid if N > 2d, and if N, d, and (N - d) are appreciably greater

than unity. The error in (3 1) is on the order of

N
(32)
(N - 2d)2 '

Formula (31) was also derived by Cover in a manner completely dif¬

ferent from that shown in Appendix B. However, his derivation did not
yield error bounds, and the conditions under which it was valid were not
completely clear.
In order to obtain some feel for the type of numbers involved, consider
a control problem for a fourth order system where each of the four state
variables is quantized into twenty units. There are then 204 = 160,000
hypercubes in the state space. The dimension d of the pattern vector
would be 4 X 20 = 80. Assume that the correct control is known only
for N = 4,000 hypercubes (say by obtaining optimal open-loop trajectories
for only a few initial conditions). In short, the number of training pat¬
terns is only 4,000/160,000 = 2 1/2 percent of the total number of
possible patterns.
852 ELEMENTS OF MODERN CONTROL THEORY

For this case we find that a new pattern has a probability A (4000, 80)
« 0.020 of being ambiguous relative to a random dichotomy. Of course,
in actual practice the random stipulations of Cover’s result are not
strictly met. Nevertheless, this serves to substantiate in some degree the
extraordinary capability of a TLU to correctly identify new patterns,
when it has been “trained” on only a small fraction of the patterns
likely to be encountered.
Note that when a new pattern is equally likely to be on either side of
the separating hyperplane, the probability of misclassification is 1/2
A (N, dj.

7.3 APPLICATION TO THE TIME OPTIMAL PROBLEM

In order to apply the theory of Sect. 7.2 to control problems, we

must resolve two basic questions. First, how are state variables to be
encoded into pattern vectors compatible with operation via the TLU?
Second, what are the conditions which ensure that a given dichotomy
in state space (not pattern space) can be effected by a TLU? We con¬
sider these questions in turn.

7.3.1 Coding

In this section we consider the problem of representing analog signals

by pattern vectors. Suppose, for example, that the signal, z,-, which we
interpret as the ith component of a state vector, z, is known to be in
the range

a* < Zi < b< (33)

where a < and b, are known numbers. We may quantize this signal into
dj intervals of length, A<, in which case we have

b< — a,- + d,- A, . (34)

With each interval, we associate a unique set of numbers (x<1,x(J, . ,

x«). The vector, having these numbers as components, we denote by
X,:, as in Sect. 7.3. Each of the numbers, x< will have the value 0 or 1.
There are obviously many ways of doing this.
We now associate a pattern vector with z< in the following way.
LEARNING CONTROL SYSTEMS 853

interval
number z i in the interval Xir

1 ai to [aj + Aj [0 . 0 1]
2 [ai -f- A,-] to [a,- + 2Aj] [0 . Oil]

3 [aj + 2Ai] to [a,' + 3Aj] [0- . 0 111]

di [a,- + (di — l)Ai] to [ai + diAij ^11 • u

di components

By constructing the pattern vectors in this way, we are assured that they
constitute a linearly independent set. *
There are otner ways of constructing linearly independent pattern
vector sets. For instance, using the so-called “one out of n” code, we have

interval
T
number X<

1 [0 •••01]
2 [0 • -010]
3 [0 •0100]
4 [0 0 1 0 0 0]

di [1 0 0].

di components

*A set of m vectors, ab aj, . , am, is said to be linearly independent if

\l &1 “I- X2 &2 .... 4“ Xm 8m = 0

implies
Xj — X2 — — — Xm — 0.
854 ELEMENTS OF MODERN CONTROL THEORY

The above codes are not the most efficient in terms of using a mini¬
mum number of components in the pattern vector.
For example, if d< < 2", where M is a positive integer, then a simple
binary code would use only M components in the pattern vector. Thus,
if d<= 12, we could write the following.

interval interval
number X/ number x/

1 [0001] 7 [0111]
2 [0010] 8 [1000]
3 [0011] 9 [1001]
4 [0100] 10 [10101
5 [0101] 11 [1011]
6 [0110] 12 [1100]

However, the above pattern vectors are not linearly independent. Since
the property of linear independence is a fundamental requirement for the
TLU realizability theorems discussed in the next section, this type of
code will be employed exclusively in the remainder of the chapter.
Thus, the process of coding is nothing more than a transformation
from state space to pattern space. Physically this is accomplished by an
encoder. The scheme just described may be mechanized in the manner
shown in Fig. 7.6. This particular arrangement uses simple summing and
threshold elements and is, therefore, structurally analogous to a TLU.

7.3.2 Realizability Theorems

After applying conventional optimal control theory to a given problem

(for which the optimal control is of the bang-bang type), one generally
obtains an expression for the switching surface either explicitly or
implicitly. In the former case, the switching surface may be found as a
function of the state variables in such a way that

control = Max if f(zi, z2, . zM)

= f(z) > 0

control = Min if f(zi, z*, . , zw)

= f(z) < 0 .
LEARNING CONTROL SYSTEMS 855

Figure 7.6. Schematic of Encoder.

If f(z) is a simple function, the problem is essentially solved. It may

be, however, that the mechanization of f(z) by electronic components is
quite difficult and involves serious compromises with weight, noise,
reliability, etc. In this case, it is relevant to investigate the possibility of
approximating f(z) by a TLU network.
We may proceed as follows. Let each of the variables, z<, be quantized
and encoded into pattern vector Xi of dimension d,, in the manner
described in Sect. 7.3.1. Let

ii(zi) = Xi- Wi
(35)
i = 1, 2, M

and

f(z) = £ Xi-Wi (36)

t=l

where Wi is a weight vector of dimension dj.

856 ELEMENTS OF MODERN CONTROL THEORY

SmithdS) proves the following theorem. “If a linearly independent

code is used to encode each parameter, zf, and if the function desired is
of the form

f(z) = Yj U(Zi) (37)

t=l

(i.e., there are no cross product terms), then each fj(zt) in Eq. (35) can
exactly match the corresponding f<(z,-) except for quantization errors.”
The scheme is illustrated in Fig. 7.7. If we use the method of Sect. 7.3. 1
for generating the linearly independent pattern vectors for zt-, and if z<
then is quantized into ds intervals, the corresponding pattern vector X<
will have d, components. We let F,(3) denote the value of fi(zt) at the
midpoint of the jth interval. We also define the matrix

[x^r '

[X.W] r

(38)

[X i (d^]r

Figure 7.7. Function Generation by TLU Networks.

LEARNING CONTROL SYSTEMS
857

Then the weight vector, W is given by

W, = Pr1 F, (39)

where F, is the column vector whose components are F,(», j = 1,2. - , d*.
The inverse of Pt- always exists, since the row vectors are linearly
independent. We note that P, is in fact a d, X d< matrix of the form

all 1
zeros 1
P, = (40)
1 all
1 ones

Consequently, its inverse is

all -1 1
zeros 1
Pr1 -1 (41)
-1 1 all
1 zeros

The matrix inversion in Eq. (39) is, therefore, no problem.

To illustrate this method, suppose that it is required to generate the
function

fi(zi) = sinh (zi + 1) — 0.5

in the interval —1.0 < z1; < 2.5 using the TLU approximation technique
described above. Talcing d, = 7, which means that z< is quantized in
increments of 0.5, the computations and coding may be summarized as
follows.

Interval, /x Range of Zi Fi<A) T

1 -1 to -.5 -.247 [0 0 0 0 0 0 1]
2 -.5 to 0 .322 [0 0 0 0 0 1 1]
3 0 to .5 1.102 [0 0 0 0 1 1 1]
4 .5 to 1.0 2.290 [0 0 0 1 1 1 11
5 1 to 1.5 4.191 [0 0 1 1 1 1 1]
6 1.5 to 2.0 7.289 [0 1 1 1 1 1 1]
7 2.0 to 2.5 12.376 fl 1 1 1 1 1 1]
858 ELEMENTS OF MODERN CONTROL THEORY

From Eqs. (39) and (41), the weight vector, Wi, is found to be

0 0 0 0 0 -1 r ' - .247' 5.087

0 0 0 0 -1 1 0 .322 3.098
0 0 0 -1 1 0 0 1.102 1.901
0 0 -1 1 0 0 0 2.290 — 1.188
0 -1 1 0 0 0 0 4.191 .780
-1 1 0 0 0 0 0 7.289 .569
1 0 0 0 0 0 0 12.376 -.247

The scheme is shown implemented in Fig. 7.8, and a comparison between

the fi(zj) as generated by the TLU and the exact fj(zi) is illustrated in Fig.
7.9. In most optimal control problems of interest, the switching function
is not easily obtainable as a function of the state variables. The question
then arises of whether, in these circumstances, one can still construct a
TLU network that closely approximates the actual switching surface (as a
function of the state variables.
Formally, one may attempt to do this in the following way. After
quantizing the state space in some fashion, a series of optimal open-loop
trajectories is calculated for several randomly selected initial conditions.
In this way, a certain number of the hypercubes in state space will be
identified with the correct value of the control. Usually, these will repre¬
sent only a small fraction of the total number of hypercubes in the state
space. We may, however, train a TLU network (pattern classifier) on this

Figure 7.8. Function Generation by TLU.

LEARNING CONTROL SYSTEMS 859

(i «V

Figure 7.9. Comparison Between Approximate

and Exact Switching Curve.

set of training pattern vectors, and hope that “new” pattern vectors have
a high probability of being correctly identified. It will be recalied from
Sect. 7.2.6 that, with numbers typical of practical situations, a training
set constituting only 2 1/2 percent of the patterns in state space is suf¬
ficient to classify new patterns correctly with about 98 percent proba¬
bility. A TLU network constructed in this way could thus be expected
to be very effective and well within the usual engineering accuracy.
Such a procedure, however, implicitly assumes that a dichotomization
via TLU is possible for the switching surface, the form of which is not
known to begin with. In short, we must ascertain those properties of the
switching surface (the exact form of which is unknown) which ensure
that it is indeed “TLU realizable.” Significant results in this area have
been obtained by Berkovec.(16d7) They are broadly similar to those
derived by Smith (discussed earlier in this section), but are somewhat
more general. They are based on the concept of projectability, which may
be defined as:

“A surface in n space with a continuous boundary, symmetrical about

an (n — 1) dimension hyperplane, is pro jec table along a direction per¬
pendicular to the hyperplane if the boundary is single valued on one
side along any line normal to the hyperplane of symmetry.”
 ELEMENTS OF MODERN CONTROL THEORY
860

A surface f(zi, zz, . , zM) in M space with a monotonic boundary,

that is, one for which

i£>o or — < 0 (42)

dz i dz i

for all i = 1, 2, . . . , M, is projectable.

Bercovec’s theorem states that: “A surface which is projectable may
be approximated with arbitrary accuracy with a single linear threshold
element by using a linearly independent code for each of the state
variables.”
The condition of this theorem is sufficient. It may not be necessary.*
There still remains the question of how to ensure that a condition like
(42) is indeed satisfied if the form of F is not known a priori. No general
rule is available for doing this. However, in specific cases, this is often
possible. BercovecO?) shows that, if the system is of the form

z = Az + Bu

u < |u0|

where A = M X M constant matrix, B = M X 1 column vector, and the

control u is a scalar, then if the eigenvalues of A are distinct, real, and
negative, the projectability criterion, (42), is satisfied.
It can also be shown that the projectability criterion is satisfied for a
second order system with complex eigenvalues and a scalar control func¬
tion. Extensions to higher order cases become progressively more diffi¬
cult. However, in cases where the eigenvalues contain a dominant
complex pair, various approximations are possible to establish projecta¬
bility.
For systems containing a vector control function of r components, it
will generally be necessary to construct a TLU network for each of the
control function components. Thus, a single TLU cannot generate all r
components of the control vector, though it may be possible to generate
each component of the control vector with a single TLU. As a result, it
is necessary to use a minimum of r TLU’s to generate the switching
surface with arbitrary accuracy.
It should be emphasized that Bercovec’s theorem gives only sufficient
conditions for TLU realizability. A given system may fail to satisfy (42)
and yet be TLU realizable.

* Actually Bercovec specifies a “1 out of n” code. However, it is easy to show that any linearly
independent code satisfies the requirements of the theorem.
LEARNING CONTROL SYSTEMS 861

Unless there is reason to suspect anomolous behavior in the shape of

the switching surface, one may proceed to construct a TLU network
based on a training set obtained from a number of computed open-loop
optimal trajectories. If one finds a separating hyperplane for the patterns
in this training set (the Ho-Kashyap set described in Sect. 7.2.5 will
determine whether this is possible), then it is reasonable to assume that
this effectively dichotomizes all the hypercubes in the state space.
This procedure is outlined in detail in the next section.

7.3.3 Synthesis of Closed-Loop Controller

Consider the system shown in Fig. 7.10.

Suppose that through the action of a disturbance or through the
application of a new input command, the system is not in the desired
state, z = 0. It is required to design the controller such that the vector,
z, reaches the zero (equilibrium) state in minimum time. We assume
that the system dynamics are described by the matrix differential equation

z = h(z, t) + B(z, t) u (43)

which is precisely Eq. (1).

It is further supposed that an analysis of the problem along the lines
indicated in Sect. 7.1 discloses that the optimal u(t) is + u0 or — u0,
where u0 is a known quantity.
The design procedure for constructing the optimal closed-loop con¬
troller may be summarized:

a. Quantize each component, z < of the state vector, z, into d,- intervals.
M

This results in J][ d< hypercubes in the state space.

i= 1

b. Construct a linearly independent code, X,, for each z*. Define the
total pattern vector, X, by
Xr = [Xjr X2r . XMr) .

Figure 7.10. Typical Gosed-Loop Control System.

862 ELEMENTS OF MODERN CONTROL THEORY

A unique total pattern vector is thus associated with each hypercube

in the state space. Any specific hypercube may be conveniently
identified by an interval number set (mi, M2, . , vM) in which each
Mi is a number which locates the quantized interval in zf.
A simple illustration will clarify the procedure. Suppose that the
number of state variables M = 3, and that d3 = 8, d2 = 6, and di = 5.
The linearly independent pattern vectors may then be constructed for
each Zi, as follows.

interval
number, Mi X/

1 [0 0 0 0 0 0 0 1]
2 [0 0 0 0 0 0 1 1]
3 [0 0 0 0 0 1 1 1]
4 [0 0 0 0 1 1 1 1]
i\ 5 [0 0 0 1 1 1 1 1]
6 [0 0 1 1 1 1 1 1]
7 [0 1 1 1 1 1 1 1]
8 [1 1 1 1 1 1 1 1]

1 [0 0 0 0 0 1]
2 [0 0 0 0 1 1]
3 [0 0 0 1 1 1]
z2 4 [0 0 1 1 1 1]
5 [0 1 1 1 1 1]
6 [1 1 1 1 1 1]

1 [0 0 0 0 11
2 [0 0 0 1 1]
Z3 3 [0 0 1 1 1]
4 [0 1 1 1 1]
5 [1 1 1 1 1]

The total pattern vector associated with the hypercube with an interval
number set of (2, 5, 4), say, would be

Xr = [0 00000110111110111 1],

c. Using conventional methods, generate a series of optimal open-loop

trajectories with randomly selected initial states. The number of such
LEARNING CONTROL SYSTEMS 863

trajectories should be sufficient to span about 3-5 percent of the

total number of hypercubes in the state space. With each hypercube
that is contained within the calculated trajectories, there is now associ¬
ated the correct value of the optimal control vector, u. These hyper¬
cubes, together with the corresponding control vector, constitute the
set of training patterns for the determination of the TLU weights,
d. For each component, u,-, of the control vector, u, determine the set of
weights which correctly dichotomizes the control using the set of
training pattern vectors obtained in Step c.
The schematic for the controller, where the control vector, u, has
two components, is shown in Fig. 7.11. In accordance with the
notation used here, the weight vector

W,T = [wii, wi2, . WijJ

is associated with pattern vector, Xt .

The total weight vector is defined by

Wr = [Wir W/ . Wmt] .

With X and W defined in this way, any of the methods described in

Sect. 7.2 may be used to obtain W (and the threshold weight, wd+i).
This procedure is repeated for each component of the control vector.
All of the information needed to design the optimal controller is now
available.
It should be noted that the “generalizing” capability of the controller
is enhanced as the order of the system is increased and as the quantization
of the state variables is made finer (increasing d,).
For example, in the two dimensional case with dx = d2 = 10, and with
forty-five squares of the state space in the training set (N = 45, d = 20)
the probability of misclassification of any of the fifty-five remaining
squares in the state space is*

^ A(45,20)= 0.322
which is little better than tossing a coin, even though the training set
contained 45 percent of the squares in the complete state space.

Using the “exact” Eq. (30).

 ELEMENTS OF MODERN CONTROL THEORY
864

Figure 7.11. Detail of Controller.

On the other hand, consider a three dimensional case with di = d2 =,

da = 30, so that the entire state space is composed of 303 = 27,000
cubes. If there are 1,000 patterns in the training set (which constitute
3.7 percent of all the cubes in state space), then with N = 1,000 and d =
90, the probability of misclassifying a cube in the remaining state space
is*

Using Eq. (31).

LEARNING CONTROL SYSTEMS 865

(yA(1,000,90) 0.049 .

The increase in efficiency of the controller is quite dramatic.

This property of the TLU optimal controller is very attractive, since
for the higher order problems (where the effort required to generate
optimal open-loop trajectories is correspondingly greater) proportionately
fewer patterns are required for designing the controller.

7.3.4 Hardware Considerations

The TLU networks discussed thus far can be implemented in a variety

of ways. One simple transistor resistor arrangement is shown in Fig. 7.12.
With suitably chosen components and bias voltages, the input quantities
are voltages, and the resistor values, Rc, R0, Ri, R2, . , Rd, are related to
the weights. The voltage, Es, represents the sum

22 wf xf + wd+i
i=i

which is the linear discriminant function of Eq. (7), and the output
voltage, E0, would be the i„ of Fig. 7.3.
Many other mechanizations are possible. The study by Tanaka^S) con¬
tains a detailed description and analysis of the above scheme together
with five other possible TLU mechanizations, namely:

iR °

Figure 7.12. Resistor Transistor Implementation of

the TLU.
866 ELEMENTS OF MODERN CONTROL THEORY

a. Multiple Coil Relay Circuits.

b. Parametron Circuits.
c. Magnetic Core Circuits.
d. Resistor Tunnel Diode Circuits.
e. Double Anode Diode Circuits.

If the TLU controller is part of the control system on a space vehicle

where an onboard digital computer is available, then a portion of the
computer could be reserved for TLU logic computation.
In short, there is considerable flexibility in the physical implementation
of the TLU; most of the schemes make very modest demands in the way
of space, weight, and power requirements.

7.4 TIME OPTIMAL REORIENTATION MANEUVER

The following problem arose in a recent aerospace vehicle control

study. The angular motion of a nonsymmetric body is described by

I„v 6 — lyZ\J/ = Ly (44)

—lVz 9 + lzz \p = Lz (45)

where lvv, lzz, and lyz are the moments and product of inertia, respectively,
about the indicated axes. 9 and 4/, the pitch and yaw angles, respectively,
and Ly and Lz denote the control torques about the indicated axes. The
control torques satisfy a saturation constraint of the form

|Ly(t)| < L0
(46)
|L,(t)| < L0 .

It is required to program the control torques, L„ and Lz, in such a way

that the system is returned to the equilibrium position, 9 = 9 = 4/ =
4/ = 0, in minimum time from any arbitrary initial state.
This can be accomplished only by making Lv and Lz a function of the
present state of the system, i.e., closed-loop control. Using conventional
optimal control theory, it is possible to generate only one optimal tra¬
jectory, given one initial state. In short, the control is open loop.
However, in line with the main theme of this chapter, we may use a
series of these open-loop optimal trajectories to construct a TLU con¬
troller which is time optimal in a closed-loop sense.
LEARNING CONTROL SYSTEMS 867

Since the calculation of the open-loop optimal trajectories is funda¬

mental in this approach, a closer look at the computational aspects ap¬
pears warranted. Naturally, we can highlight only the salient points in
such a discussion, since the general subject is too broad for detailed
consideration here.
We begin by expressing the system equations in a more convenient
form. /

By defining the state variables

zi = 6 , 2,3 - 'P
(47)
z2 = 6 , z4 = i

we can express Eqs. (44) and (45) as

Zi - z2 (48)

z2 — <ii Ly -)- a3 L2 (49)

z3 = z4 (50)

z4 = a3 Ly -f- a2 L2 (51)

where

*d>l I zzj(lyv yz ) (52)

= Iyy/(IyyIz* ^-yz) (53)

813 Ity^/(J^-yy
Izz -1
-yz ) • (54)

Furthermore, with the change of variable

4(t)
Uy(t) = (55)

.. L.(t)
U,(t) = T (56)
-L'o

T = (L„a01/21 (57)
 ELEMENTS OF MODERN CONTROL THEORY
868

the system (48)-(51), and (46) can be written in the nondimensional

form

Z\ = z2 (58)

z2 = U!/ “b bi u. (59)

z3 = z4 (60)

z4 = bi uv + b2 uz (61)

3* VI (62)

|u,| < 1 (63)

where

bl lyz/bz (64)

b2 (65)

and the prime denotes derivation with respect to r.

In matrix form, the set of Eqs. (58)-(63) can be written as

z' = Az + Bu (66)
where

zT = [Zi z2 z3 z4]

ur = [u„ u«]

0 10 0
OOOO
0 0 0 1
0 0 0 0

0 0
B “ oo
bi b2
LEARNING CONTROL SYSTEMS 869

M < (67)

We now formulate the main problem as follows. Given system Eq. (66)
and the control function constraint, Eq. (67), determine the control
function, u(t), which satisfies Eq. (67), And which transfers the system
from the initial state

z(0) = £ (68)

to the terminal state

z (Tt) = 0 (69)

in minimum time, i.e., such that rr is a minimum.

We note first of all that system Eq. (66) is completely controllable.
This follows from the fact that the matrix

G = [B : AB j A2B : A3B]

is the rank 4. *
Thus, we know that there exists a control which can accomplish the
reorientation in finite time.
By invoking the maximum principle, it is found that the optimal
control must minimize the hamiltonian

H = - 1 + \r (Az + Bu) (70)

where

(71)

It is apparent that H is minimized by

u*(t) = sgn(BT\) (72)

or, in component form

As long as b2 9^ bA
 ELEMENTS OF MODERN CONTROL THEORY
870

u^(t) = sgn (X2+ bi \4) (73)

u2(r) = sgn (bi X2 + b2 X4). (74)

But

x; = 0 (75)

x; = -Xx (76)

X3 = 0 (77)

X4 = -X3 (78)

and

X4= X10 (79)

X2 = — Xjot +%X20 _ (80)

X3 = X30 (81)

X4 = — X30r X4o (82)

where X10,X20,X30, and X40are constants. Therefore

= sgn [— (X10+ bi X30)t -(- (X20+ bi X40)] (83)

u2 = sgn [—(bi X10T" b2 X30)r T (bi X2o b2 X4o)]. (84)

The actual computation of u*(t) now requires that we solve a two point
boundary value problem. In this respect, one of the more powerful
techniques available in the present case is Neustadt’s method. (6) How¬
ever, this requires that the system be “normal.” Loosely speaking,
normality implies that the system is controllable with respect to each
component of the control vector. Using a simple testU) for normality,
we conclude that since the matrices

Gx = [Bx ; ABx i A2Bi : A3Ba]

g2 = [B* ; ab2 ; a2b2 ; a3b2]

LEARNING CONTROL SYSTEMS 871

(where Bi and B2 denote the first and second columns of B, respectively)

are both singular, the present system is nonnormal. Thus, one powerful
numerical tool is “shot down.”
Furthermore, a direct consequence of nonnormality is the fact that
BT\ in Eq. (72) is zero over some finite interval. This, in turn, means
that the optimal u (t) is undefined in this interval. In short, the optimal
control problem is “singular.” '
It does not imply, however, that an optimal control does not exist.
It does follow that the optimal control is not unique. Thus, we are
faced with the distressing situation in which even the computation of
the optimal open-loop control is not a trivial or straightforward matter.
Taking account of the above considerations, it would appear that a
gradient method is appropriate for the above problem. Specifically, we
reformulate the problem such that it is required to minimize the error
from a fixed terminal state (the origin in state space) where the total
time is initially prescribed.
If the minimum error for this fixed total time is finite, repeat the
procedure with a new larger total time. Continue in this fashion until the
error from the fixed terminal state is as small as desired. The total time
in this final trajectory is the minimum time for the original time optimal
problem and the associated control is the corresponding time optimal
control.
This procedure too has some subtle computational nuances. It can be
shown that, due to the fact that the control appears linearly in Eq. (66),
the gradient technique will encounter convergence difficulties in the
vicinity of the optimum. U9) Various means of resolving this problem
are discussed by Johansen, (19) leading to a solution which is generally
acceptable for engineering purposes. Being thus forewarned of the
possible hazards and pitfalls in the calculation of the optimal open-loop
trajectories, one may reasonably expect to obtain these ultimately,
though not without some sweat and perhaps an occasional tear.
With these in hand, one may then proceed with the design of the TLU
closed-loop controller in the manner described in Sect. 7.3.3.
In order to provide a basis for comparison, it is instructive to investigate
the performance of a conventionally designed control system. For this
prupose, we stipulate a proportional type of control, as follows.

u = -Kz (85)

where
 ELEMENTS OF MODERN CONTROL THEORY
872

ki k2 0 0
K =
0 0 k3 Ic,

Substituting Eq. (85) in Eq. (66) yields

z' = (A - BK) z
(86)
= Fz

where

0 1 0 0
— ki — k2 — b]k3 — bik4
(87)
0 0 0 1

—bik4 — bika —b2k3 — b-jlc,

The characteristic equation for the matrix F is

|si — F| = s4 T a4 s3 T a2 s2 -4- a3s 4- a4 = 0 (88)

ai = k2 + b2k4

a2 = k2k4 (b2 — bi2) -(- ki T b2k3

(89)
a3 = (k2k3 T kik4)(b2 — bi2)

a4 = k4k3 (b2 — bj2) .

We assume that

b2 - bi2 > 0 . (90)

In this case, Routh’s criterion shows that Eq. (88) has no roots in the
R-H plane if each of thekf are positive and if the inequality

a42a4
aia2 — a3 > - (91)
B<3

is satisfied.
LEARNING CONTROL SYSTEMS 873

If k, = 0, then*

ai — k2

aj — ki -f- b2k3 (92)

a3 = kjk3 (b2 —' bi2)

&4 — kik3 (b2 — bi2) .

For this case, inequality (91) is automatically satisfied if ki, k2, and k3
are positive. In other words, the system is stable for all positive values of
k<.
We now write Eq. (88) as

_ k2s [s2 + k3 (b2 — W)] _

(93)
[s4 + (ki -f- b2k3) s2 + kik3 (b2 bi2)]

This permits the determination of a suitable k2 via root locus.

Equation (93) may be written as

k2s(s2+ <302) _ (94)

1 + (s2 + /3i2)(s2+ &2) _

where

02 = k3 (b* - bi2)

ft2 =
(ki b2k3)
/ 4 ktk3 (b2 - bt2)
(ki — b2k3)2 T 4 k]k3b2

A2 =
(ki + b2k3)
i - i/7T 4 kxk3 (b2 - bi2)
(ki — b2k3)2 + 4 kik3b2

It is apparent that when b2 — bf2 > 0, and ki and k3 are positive, then
the quantity inside the radical is always positive, and less than one. Thus,

&fSimilar results hold for k2 = 0.

874 ELEMENTS OF MODERN CONTROL THEORY

S,2, /3i2,and /322are positive numbers. Also, in this case

ft2 > 0<? > ft2 . (95)

The root locus for this case is shown in Fig. 7.13. Once can now pro¬
ceed along conventional lines to obtain a prescribed type of transient
response for the system. However, for a more precise analysis, one must
take account of the saturation constraint on u (via describing functions,
say). Since this is a routine (though cumbersome) procedure, it will not
be further pursued here.
One fact should be noted, however. Because of the nonlinearity in¬
troduced by the control saturation, the characteristics of the time response
will differ for different inputs or initial conditions. In a sense, one can
say that the conventional system can be “optimized” only for one (or a
small class of) inputs. This limitation does not exist with TLU optimal
controller.

I
m

Figure 7.13. Root Locus for Conventional Control.

 APPENDIX A

THE PSEUDO INVERSE OF A MATRIX

Consider an n X in matrix B and an'm X n matrix C. Throughout the

ensuing discussion, assume that m < n, and that B and C are both of rank
m. We define the pseudo inverse of B as

B# = (BrB)~l Br . (Al)

This is sometimes called the left inverse of B. We note that this

reduces to the ordinary inverse when n = m.
In a similar way, we define the pseudo inverse of C as

C# = Cr(CCr)-1 (A2)

which is also called the right inverse.

The motivation for the pseudo inverse stems from the following. If G
and H are column vectors of dimension n and m, respectively, thenGU

Theorem I: The solution of the matrix equation, BX = G, which

minimizes (BX —G)r (BX-G), is

X = (BrB)-1 BrG
(A3)
= B#G .

Theorem II: The solution of the matrix equation, CY = H, which

minimizes YrY is

Y = Cr(CCr)-1 H
(A4)
= C#H .

We further define the left identity matrix

I, = BB# (A5)

875
876 ELEMENTS OF MODERN CONTROL THEORY

and the right identity matrix

I* = C#C . (A6)

This is motivated by the fact that IL B = B and CIR = C.

Consider now any nonzero pXq matrix whose rank r may be less than
its smaller dimension, q. Such a matrix may be written as*

A = BC (A7)

where BispXr, and C is r X q. The pseudo inverse of A is given by

A# = C# B# . (A8)

It can be shown that(H)

a. The pseudo inverse of a matrix is unique.

b. For a nonsingular matrix, the pseudo inverse reduces to the conven¬
tional inverse.
c. (A*)* = A.
d. AA# = IL and A#A = I„.

*B is formed by taking the r independent columns of A. Then there exists an r X q matrix, C,

such that A = BC. Both B and C are of rank r since the rank of a product does not exceed the
rank of any factor.
 APPENDIX B

DERIVATION OF EQ. (31)

Given are p and q such that

0 < p < 1 (Bl)

q = 1 - p . (B2)

Let N and k be positive integers which satisfy the inequality

Np < k < N (B3)

and let*

(B4)
B,(k)
- t ( Nr
) P'<!-'
where the notation
ion(b) denotes the binomial coefficient, viz.

/ a\ _ a!
a|_ (B5)
\ b/ b! (a — b)!

Define the quantity

k + 1
A„(k) (k) PAqAr-*+1 k + 1 - (N + 1) p
(B6)

Bahadur(21 ) shows that

A^k) (B7)
1 - Mk) < l + v~2

*B*r(k) is in fact the cumulative binomial distribution function.

877
878 ELEMENTS OF MODERN CONTROL THEORY

where

k - Np (B8)
11 VpqN

Thus, for the special case of p = q = 1/2 and for tj sufficiently large, it
follows that

k + 1
sewo k + 1 - | (N+ 1)_
(B9)

Using the fact that the binomial coefficient satisfies the relation

O)-(.-b)
Eq. (B9) may be written as

N + 1 -d
IdH(f) L2
1
+ 1) -d J
(BIO)

where

d = N - k . (B 11)

Furthermore, if N, d, and (N-d) are appreciably greater than unity, Eq.

(BIO) further simplifies to

y ( N \ ~ ( N\ (N ~ d) (B 12)
h\ i ;~Vd^(N-2d)
Using this approximation for the C function of Eq. (28), we find that
A(N, d), as given by Eq. (30), becomes*

A(N, d) (B 13)

Again assuming that N, d, (N — d) > > 1.

LEARNING CONTROL SYSTEMS 879

By virtue of Eq. (B 11), condition (B3) may be expressed as

N > 2d > 0 (B14)

while

N - 2d (B 15)
V~ Vn

Consequently, as a result of Eq. (B7), the error in the approximate

relation (B13) is on the order of

1 N
(B 16)
rf (N - 2d)2 '

REFERENCES

1.F. B. Smith, Jr., A Logical Net Mechanization for Time Optimal

Regulation, NASA TN D-1678, December, 1962.
2. J. M. Mendel and J. J. Zapalac, Applications of Adaptive Com¬
puters to the Synthesis of Time and Fuel Optimal Controllers,
Douglas Report No. 4167, September, 1966.
3. N. J. Nilsson, Learning Machines, New York, McGraw-Hill Book
Company, Inc., 1965.
4. H. D. Block, “The Perceptron: A Model for Brain Functioning,”
Rev. Modern Physics, Vol. 34, 1962, pp. 123-135.
5. M. Athans and P. L. Falb, Optimal Control, New York, McGraw-
Hill Book Company, Inc., 1966.
6. L. W. Neustadt, “Synthesizing Time Optimal Control Systems,” J.
of Math. Anal, and Appl., 1960, pp. 484-493.
7. Y. C. Ho and P. B. Brentani, “On Computing Optimal Control with
Inequality Constraints,”/. SIAM Control, 1963, pp. 319-348.
8. A. Novikoff, “On Convergence Proofs for Perceptrons,” Proc. Symp.
on Math. Theory of Automata, Vol. XII, 1963, pp. 615-622.
9. S. Agmon, “The Relaxation Method for Linear Inequalities,”
Canad. J. Math., Vol. 6, 1954, pp. 382-392.
10. Y. C. Ho and R. L. Kashyap, “An Algorithm for Linear Inequalities
and its Applications,” IEEE Transactions on Electronic Computers,
October, 1965, pp. 683-688.
11. R. Penrose, “On the Generalized Inverse of a Matrix,” Proc.
Cambr. Phil. Soc., 1955, pp. 406-413.
380 ELEMENTS OF MODERN CONTROL THEORY

12. R. C. Singleton, “A Test for Linear Separability as Applied to Self

Organizing Machines,” in M. Yovits and others, eds., Self Organizing
Systems, New York, Spartan Books, 1962, pp. 504-524.
13. H. Gluckman, “On the Improvement of Linear Separation by
Extending the Adaptive Process with a Stricter Criterion,” IEEE
Transactions on Electronic Computers, December, 1966, pp. 941—
944.
14. T. M. Cover, “Geometrical and Statistical Properties of Systems of
Linear Inequalities with Applications in Pattern Recognition,” IEEE
Transactions on Electronic Computers, June, 1965, pp. 326-334.
15. F. W. Smith, “A Trainable Nonlinear Function Generator,” IEEE
Transactions on Automatic Control, April, 1966, pp. 212-218.
16. J. W. Bercovec, Time Optimal Control with Adaptive Networks,
Ph. D. Dissertation, State University of Iowa, June, 1964.
17. J. W. Bercovec, and D. L. Epley, On Time Optimal Control with
Threshold Logic Units, IEEE Wescon Conv. Rec., 1964.
18. R. J. Tanaka and others, Research on Automatic Computer Elec¬
tronics, Wright Patterson AFB Report No. RTD-TDR-63-4173, Vol.
II, February, 1964.
19. D. E. Johansen, “Convergence Properties of the Method of Gradi¬
ents,” Advances in Control Systems, Vol. 4, New York, Academic
Press, Inc., 1966.
20. T. N. Greville, “The Pseudoinverse of a Rectangular of Singular
Matrix and its Application to the Solutions of Systems of Linear
Equations,” SIAM Rev., Vol. l,No. 1, 1959, pp. 38-43.
21 . R. R. Bahadur, “Some Approximations to the Binomial Distribution
Function,” Ann. Math. Stat., Vol. 31, 1960, pp. 43-54.
22. A. Greensite, “Synthesis of Closed Loop Time Optimal Control
Systems Via Threshold Logic Networks,” Proc. AIAA 7th Aerospace
Sci. Meeting, New York, January, 1969, Paper No. 69-77 .
23. J. M. Mendel, Survey of Learning Control Systems for Space Vehicle
Applications, Joint Automatic Control Conf., Seattle, 1966, pp. 1-
11.
24. J. T. Tou and J. D. Hill, Steps Toward Learning Control, Joint
Automatic Control Conf., Seattle, 1966, pp. 12-26.
25. J. Sklansky, “Learning Systems for Automatic Control,” IEEE
Transactions on Automatic Control, Vol. AC-11, No. 1, 1966, pp.
6-19.
LEARNING CONTROL SYSTEMS 881

PROBLEMS AND EXERCISES FOR CHAPTER SEVEN

1. Given are a set of patterns {x(1), X(2), • %• -X(JV)}and a set of points

Pi, P2, •• •Pfl in d space. It is known that the patterns tend to
“cluster” about the points, P<. Suppose we decide to associate a
given pattern with that category i for which |X — Pt|2 is a minimum.
Derive a linear discriminant function "Which will do this.

2. Given are the augmented pattern vectors

1 0
0 0
Y^(2)
0 1
1 1

0 0
1 1
(3) YY4)
1 0
1 1

which belong to category A, and the set

1 0
0 0
Y*<2> =
1 0
0 1

1 1
1 1
Yb«> =
0 0
1 1
— _

which belong to category B.

Are these linearly separable? Why?

3. Equation (28) shows that a total of fourteen dichotomies can be

achieved with four pattern vectors in a plane (d = 2). Give a graphical
882 ELEMENTS OF MODERN CONTROL THEORY

illustration of this by constructing the seven lines which partition

the four points in all possible ways. The four points must be in
“general position” (i.e., no three points are collinear). Each partition
gives two different classifications, and hence there are fourteen
possible dichotomies.

4. Given are N “d dimensional” patterns and a TLU with (d + 1) ad¬

justable weights. If one of the possible dichotomies is selected at
random, show that the probability that it is realizable by a TLU is
given by

= 1 , N < d .

(Hint: There are 2N possible dichotomies of N patterns.)

Make a plot of P(N, d) vs X, where X is defined by N = X(d + 1).
Take X in the range 1. 0-5.0, and plot separate curves for d = 2, 15,
and 30.
Note the pronounced effect at X = 2 for large d. In short, for
large d, there is virtual certainty of being able to achieve any specific
dichotomy of fewer than 2(d 1) patterns. Conversely, it is almost
impossible to achieve a specific dichotomy of more than 2(d + 1)
patterns. This leads one to define the machine capacity by

C = 2(d + 1) .

5. Design a TLU network which will match the function

f (z) = |sin z

in the range 0 < z < Stt/2. Use a quantization level of ten equal
intervals.
 INDEX

Acceleration error coefficient, 95, 109 state, 146

Adaptive control Correlation, 451
digital filter, 775 coefficient, 453
model reference, 754 Costate vector, 589
notch filter, 798 Coulomb friction, 252
virtual slope, 767 Covariance, 451, 456
Adjoint, 616 Cross correlation function, 457
Approximation in policy space, 639
Attitude processing function, 820
Augmented function, 574 Damping
Autocorrelation function, 457 factor, 8, 79, 100
Autonomous system, 207 spiral, 79
Autopilot Dead zone, 250-51, 257
launch vehicle, 44, 83, 178 Delay time (see pure delay)
performance specifications, 22-24 Delta
function, 547
method, 219
Bandwidth, 93 Demodulator, 808
Binomial distribution function, 450 Density function, 448
Block diagrams (feedback), 13 Describing function, 245
Bede plot, 27 coulomb friction, 252
Bolza problem, 574 dead zone, 250, 251, 252
Boundary value problem, 723 dual input, 290
hysteresis, 257, 258-259, 262
multiple, 283
Calculus of variations, 572 piecewise linear, 264
Central moment, 449 saturation, 250, 251, 252
Characteristic equation, 3, 14, 125 Digital
Closed loop compensation, 109
frequency response, 93 control system, 57
poles, 194 programming, 111
transfer function, 13 Dipole compensation, 99
Coding, 852 Direct control, 313
Collineatory transformation, 395 Discontinuous control, (see on-off systems)
Colored noise, 506 Discriminant functions, 837, 838
Compensation techniques, 88 Distribution function, 448
continuous systems, 95 binomial, 450
sampled data systems, 109 gaussian, 450, 451
Conditional probability, 484 poisson, 450
Contactor, (see relay) Disturbance, 167
Constant damping line, 78 Dominant poles, 33
Constraints (in optimal control), 574, 619 Dynamic Programming, 632
Controllability, 146
continuous-type systems, 147, 155
discrete time systems, 155 Eigenvalue sensitivity, 393
output, 149 Encoder, 854

883
884 INDEX

Erdmann-Weierstrass conditions, 576 principle, 167

Ergodic property, 459 steady-state, 169
Error Isocline Method, 218
coefficient, 95
correction, 840
Euler-Lagrange equations, 574, 652, 656 Kalman theory, 481
External disturbances, 167

Lag
Feedback systems, 13 compensation, 97
Filter pure (see Transport lag)
compensation, 97, 99-101 Lagrange
Kalman, 481, 497, 499 multiplier, 589
notch, 810 problem, 573
tracking, 805 Laplace transform, 190
Flexible launch vehicle, 817 Launch vehicle dynamics, 178, 823
Focus Lead compensator, 100
stable, 229 Legendre-Clebsch conditions, 577
unstable, 229 Limit cycle, 232
Forced systems (nonlinear), 357 Linear
Fourier discriminant function, 838
series, 247 separability, 843
transform, 463-64 systems (in optimal control), 653
Frequency Linearly independent
response, 13, 91, 93 codes, 853
tracking, 805 vectors, 853
Function space, 730 Linearly separable, 838
Lyapunov function, 304
Lur’e method, 31 1
Gain
margin, 22-24
sensitivity, 380 Matrix
Gaussian distribution, 450-51, 466 companion, 123, 126
Gradient method, 603 canonical, 126
Green’s function, 617 modal, 125
sensitivity, 404
system, 121
Hamilton-Jacobi equation, 659 transition, 124
Hamiltonian, 589 Vandermonde, 126
Hidden oscillations, 147 Maximum principle, 586
Ho-Kashyap method, 842-43 Mayer problem, 572
Hold circuit, 72 Mean, 451, 456
Hysteresis, 256, 257, 258, 259, 262 Model-reference, 754
Modified Z transform, 72
Moments (probability), 452
Ideal relay, 340
Impulse
function (see Delta function) Nichols plot, 29
response (see Weighting function) Nodes, 229
Indirect control, 313 Noise
Inner product, 730 colored, 506
Invariance, white, 470
absolute, 169 Nonlinearities
conditional, 169 describing function, 245
INDEX 885

limit cycles, 232 Popov method, 347

Lyapunov method, 298 Position error coefficient 95, 109
piecewise linear, 235 Power spectral density, 461
phase plane, 216 Principle of Optimality, 634
relay, 339 Probability, elements of, 447
stability, 267, 298, 328 Projectability, 859
Norm, 730 Pseudo inverse, 875
Norm-invariant system, 663 Pulse
Normality, 870 input, 63
Notch filter, 805, 808-12 width, 57
Nyquist criterion, 14-15 Pure lag (see transport lag)
simplified, 25
generalized, 26
Quadratic forms, 301
Quasilinearization, 530-31, 551
Observability, 146, 148
On-off systems, 339
Open loop Random
gain, 13 process, 455
transfer function, 13, 195 variable, 448
Optimal Relative damping factor, 8, 187
control (booster vehicles), 687 Relay, 339
estimation, 510 Resonant frequency, 93
filter, (see Kalman filter) Riccati equation, 661
lunar landing, 679 Rise time, 94
policy, 633 Root locus
reentry, 712 analytic, 53
reorientation, 866 construction, 194
trajectories, 669 s plane, 31
Optimality, principle of, 634 Z plane, 79
Oscillations Routh-Hurwitz method, 4, 8
hidden, 147
sustained, (see limit cycle)
Overshoot response, 94 Saddle point, 230
Sample and hold, 70, 72
Sampler, 60
Pattern Sampling frequency, 60
classification, 836 Saturation, 252
vector, 836 Sensitivity
Performance index, 757 eigenvalue, 393
Phase function, 373
margin, 22, 24 gain, 380, 384
plane, 216 in the large, 418
portrait, 217 matrix, 404
Phasor cancellation, 769 pole and zero, 380
Phillips integral, 554 Shaping filter, 481
Piecewise linear systems, 235 Singular points, 228
Poisson distribution, 450 Sounding rocket, 577, 595, 640
Pole, 15 Stability
closed loop, 22, 194 continuous systems, 3, 124
open loop, 194 discrete systems, 57, 78, 141
sensitivity, 380 nonlinear, 298
shifting, 324 relative, 7
Pontryagin principle, 589 via frequency response, 13
via root locus, 30
886 INDEX

Standard deviation, 449 Van der Pol equation, 233

State Variable gradient method, 328
transition matrix, 124 Variance, 449
variables, 121 Variation operator, 728
Stationary random process, 458 Variational calculus, 572
Statistical design, 536 Vectors
Steepest descent, 605, 608 linearly independent, 853
Stellar navigation, 518 norm, 730
Sustained oscillations, (see limit cycle) orthogonal, 730
Velocity error coefficient, 95, 109
Virtual
Threshold logic unit, 839 singular point, 239
Time slope method, 767
optimal control, 597
response characteristics, 93 Weight vector, 839
Transfer function Weighting function, (see Density function)
closed loop, 13 Whitaker system, 754
matrix, 124, 143 White noise, 470
open loop, 13 Wiener
Transport lag, 72 -Hopf equation, 474
Transversality conditons, 656 -Khinchin equation, 463
Two point boundary value problems, 723 theory, 472
Wind loads, 539

U circle, 407 Z transform, 60

Undamped natural frequency, 7 Zero, 15
Unit open loop, 194
circle, 78 sensitivity, 380
impulse, (see Delta function) shifting, 324-325
 R02023 6b323

TL
3280
9780876715536
02/11/2019 15:42-2