THIRD EDITION

About the Author

M Gopal is presently Professor, Department of Electrical Engineering, Indian

Institute of Technology, Delhi. His teaching and research stints span over three
decades at prestigious institutes.
Dr Gopal is the author/co-author of six books. His Video Course on Control
Engineering is being transmitted periodically through EKLAVYA Technology
Channel of the IITs. He is author of an interactive web compatible multimedia
course on Control Engineering, available on the site: [email protected]
Dr Gopal has a large number of research publications to his credit. His current
research interests are in the areas of Machine Learning, Soft-Computing
Technologies, and Intelligent Control.
 THIRD EDITION

M GOPAL
Professor
Department of Electrical Engineering
Indian Institute of Technology
Delhi

Tata McGraw-Hill Publishing Company Limited

NEW DELHI
McGraw-Hill Offices
New Delhi New York St Louis San Francisco Auckland Bogotá Caracas
Kuala Lumpur Lisbon London Madrid Mexico City Milan Montreal
San Juan Santiago Singapore Sydney Tokyo Toronto
Published by the Tata McGraw-Hill Publishing Company Limited,
7 West Patel Nagar, New Delhi 110 008.

Copyright © 2009, by the Tata McGraw-Hill Publishing Company Limited.

No part of this publication may be reproduced or distributed in any form or by any means, electronic, mechanical,
photocopying, recording, or otherwise or stored in a database or retrieval system without the prior written permission of
the publishers. The program listings (if any) may be entered, stored and executed in a computer system, but they may
not be reproduced for publication.

This edition can be exported from India only by the publishers,

Tata McGraw-Hill Publishing Company Limited

ISBN-13: 978-0-07-0668805
ISBN-10: 0-07-0668809

Managing Director: Ajay Shukla

General Manager : Publishing—SEM & Tech Ed: Vibha Mahajan
Sponsoring Editor: Shukti Mukherjee
Editorial Executive: Suman Sen
Executive—Editorial Services: Sohini Mukherjee
Junior Manager—Production: Anjali Razdan

General Manager : Marketing—Higher Education & School: Michael J Cruz

Product Manager : SEM & Tech Ed : Biju Ganesan

Controller—Production: Rajender P Ghansela

Asst General Manager—Production: B L Dogra

Information contained in this work has been obtained by Tata McGraw-Hill, from sources believed to be reliable.
However, neither Tata McGraw-Hill nor its authors guarantee the accuracy or completeness of any information
published herein, and neither Tata McGraw-Hill nor its authors shall be responsible for any errors, omissions, or
damages arising out of use of this information. This work is published with the understanding that Tata McGraw-Hill
and its authors are supplying information but are not attempting to render professional services. If such services are
required, the assistance of an appropriate professional should be sought.

Typeset at Script Makers, 18, DDA Market, A-1B Block, Paschim Vihar,
New Delhi 110063 and printed at Unique Color Carton, 15/72, Punjabi Bagh, New Delhi-110 026.

Cover: Rashtriya Printers

RAXYYRYFDZZDZ
 Dedicated
with all my love to my
son Ashwani
and
daughter Anshu
 Contents

Preface XI
Part I Digital Control: Principles and Design in Transform-Domain 1
1. Introduction 3
1.1 Control System Terminology 3
1.2 Computer-Based Control: History and Trends 9
1.3 Control Theory: History and Trends 12
1.4 An Overview of the Classical Approach to Analog Controller Design 14
1.5 Scope and Organization of the Book 19
2. Signal Processing in Digital Control 21
2.1 Why Use Digital Control 21
2.2 Configuration of the Basic Digital Control Scheme 23
2.3 Principles of Signal Conversion 24
2.4 Basic Discrete-Time Signals 30
2.5 Time-Domain Models for Discrete-Time Systems 32
2.6 The z-Transform 41
2.7 Transfer Function Models 52
2.8 Frequency Response 58
2.9 Stability on the z-Plane and the Jury Stability Criterion 61
2.10 Sample-and-Hold Systems 69
2.11 Sampled Spectra and Aliasing 72
2.12 Reconstruction of Analog Signals 77
2.13 Practical Aspects of the Choice of Sampling Rate 80
2.14 Principles of Discretization 82
Review Examples 100
Problems 107
3. Models of Digital Control Devices and Systems 113
3.1 Introduction 113
3.2 z-Domain Description of Sampled Continuous-Time Plants 115
3.3 z-Domain Description of Systems with Dead-Time 122
3.4 Implementation of Digital Controllers 126
3.5 Tunable PID Controllers 133
3.6 Digital Temperature Control System 146
3.7 Digital Position Control System 151
3.8 Stepping Motors and their Control 157
3.9 Programmable Logic Controllers 163
Review Examples 180
Problems 183
 viii Contents
4. Design of Digital Control Algorithms 193
4.1 Introduction 193
4.2 z-Plane Specifications of Control System Design 196
4.3 Digital Compensator Design using Frequency Response Plots 212
4.4 Digital Compensator Design using Root Locus Plots 224
4.5 z-Plane Synthesis 237
Review Examples 242
Problems 246

Part II State Variable Methods in Automatic Control: Continuous-Time and

Sampled-Data Systems 253
5. Control System Analysis Using State Variable Methods 255
5.1 Introduction 255
5.2 Vectors and Matrices 256
5.3 State Variable Representation 266
5.4 Conversion of State Variable Models to Transfer Functions 276
5.5 Conversion of Transfer Functions to Canonical State Variable Models 282
5.6 Eigenvalues and Eigenvectors 293
5.7 Solution of State Equations 303
5.8 Concepts of Controllability and Observability 314
5.9 Equivalence between Transfer Function and State Variable Representations 325
5.10 Multivariable Systems 329
Review Examples 334
Problems 341
6. State Variable Analysis of Digital Control Systems 351
6.1 Introduction 351
6.2 State Descriptions of Digital Processors 352
6.3 State Description of Sampled Continuous-Time Plants 359
6.4 State Description of Systems with Dead-Time 364
6.5 Solution of State Difference Equations 367
6.6 Controllability and Observability 372
6.7 Multivariable Systems 376
Review Examples 379
Problems 386
7. Pole-Placement Design and State Observers 393
7.1 Introduction 393
7.2 Stability Improvement by State Feedback 394
7.3 Necessary and Sufficient Conditions for Arbitrary Pole-Placement 397
7.4 State Regulator Design 400
7.5 Design of State Observers 404
7.6 Compensator Design by the Separation Principle 413
7.7 Servo Design: Introduction of the Reference Input by Feedforward Control 418
7.8 State Feedback with Integral Control 420
 Contents ix
7.9 Digital Control Systems with State Feedback 422
7.10 Deadbeat Control by State Feedback and Deadbeat Observers 432
Review Examples 434
Problems 440
8. Linear Quadratic Optimal Control Through Lyapunov Synthesis 451
8.1 Introduction 451
8.2 The Concept of Lyapunov Stability 452
8.3 Lyapunov Functions for Linear Systems 454
8.4 Parameter Optimization and Optimal Control Problems 458
8.5 Quadratic Performance Index 461
8.6 Control Configurations 467
8.7 Optimal State Regulator 470
8.8 Optimal Digital Control Systems 479
8.9 Constrained State Feedback Control 484
Review Examples 488
Problems 495

Part III Nonlinear Control Systems: Conventional and Intelligent 501

9. Nonlinear Systems Analysis 505
9.1 Introduction 505
9.2 Some Common Nonlinear System Behaviours 507
9.3 Common Nonlinearities in Control Systems 508
9.4 Describing Function Fundamentals 511
9.5 Describing Functions of Common Nonlinearities 514
9.6 Stability Analysis by the Describing Function Method 519
9.7 Concepts of Phase Plane Analysis 527
9.8 Construction of Phase Portraits 529
9.9 System Analysis on the Phase Plane 535
9.10 Simple Variable Structure Systems 542
9.11 Lyapunov Stability Definitions 546
9.12 Lyapunov Stability Theorems 549
9.13 Lyapunov Functions for Nonlinear Systems 556
Review Examples 561
Problems 567
10. Nonlinear Control Structures 574
10.1 Introduction 574
10.2 Feedback Linearization 575
10.3 Model Reference Adaptive Control 581
10.4 System Identification and Generalized Predictive Control in Self-Tuning Mode 587
10.5 Sliding Mode Control 600
10.6 Reinforcement Learning Control 607
Problems 615
 x Contents
11. Neural Networks 620
11.1 Introduction 620
11.2 Neuron Models 624
11.3 Network Architectures 633
11.4 Learning in Neural Networks 640
11.5 Training the Multilayer Neural Network—Backpropagation Tuning 649
11.6 Function Approximation with Neural Networks 654
11.7 System Identification with Neural Networks 655
11.8 Control with Neural Networks 660
11.9 Support Vector Machines 665
Review Examples 679
Problems 684
12. Fuzzy Logic Models 688
12.1 Introduction 688
12.2 Fuzzy Quantification of Knowledge 695
12.3 Fuzzy Inference 706
12.4 Designing a Fuzzy Logic Controller (Mamdani Architecture) 709
12.5 Data Based Fuzzy Modeling (Sugeno Architecture) 719
12.6 System Identification and Control with Neuro-Fuzzy Systems 723
12.7 Genetic Algorithms 726
Review Examples 737
Problems 740
References 748
Answers to Problems 755
Appendix A: MATLAB Aided Control System Design: Conventional Control
URL: http://www.mhhe.com/gopal/dc3e
Appendix B: MATLAB Aided Control System Design: Intelligent Control
URL: http://www.mhhe.com/gopal/dc3e

Index 775
 Preface

The dramatic development of computer technology has radically changed the boundaries of practical control
system design options. It is now possible to employ very complicated, high-order digital controllers, and to
carry out the extensive calculations required for their design. These advances in implementation and design
capability can be achieved at low cost due to the widespread availability of inexpensive, powerful digital
computers and related devices. There is every indication that a high rate of growth in the capability and
application of digital computers will continue far into the future.
Fortunately, control theory has also developed substantially over the past 45 years. The classical design
methods have been greatly enhanced by the availability of low-cost computers for system analysis and
simulation. The graphical tools of classical design like root locus plots, Nyquist plots, Bode plots, and
Nichols chart can now be more easily used with computer graphics. Coupled with hardware developments
such as microprocessors and electro-optic measurement schemes, the classical control theory today provides
useful design tools to practising control engineers.
The modern control theory (which can’t be termed modern any longer) refers to the state-space based
methods. Modern control methods initially enjoyed a great deal of success in academic circles, but did not
perform very well in many areas of application. Modern control provided a lot of insight into system struc-
ture and properties; nevertheless, it masked other important feedback properties that could be studied, and
manipulated, using classical control. During the past three decades, a series of methods, which are a combi-
nation of modern state-space methods and classical frequency-domain methods, have emerged. These tech-
niques are commonly known as robust control.
The rapid development in digital computers and microelectronics has brought about drastic changes in
the approach to analysis, design, and implementation of control systems. The flourishing of digital control
has just begun for most industries and there is much to be gained by exploiting the full potential of new
technology.
Implementation of nonlinear robust control schemes—model reference adaptive control, self-tuning con-
trol, variable structure sliding mode control—has now become a relatively simple task. In the process of
understanding and emulating salient features of biological control functions, a new field called ‘intelligent
control’, has emerged.
Fuzzy logic, artificial neural networks, and genetic algorithms have grown into three distinct disciplines
with the aim of designing “intelligent” systems for scientific and engineering applications. The theory of
fuzzy logic provides a mathematical morphology to emulate certain perceptual and linguistic attributes,
associated with human cognition. It aims at modelling the inexact modes of reasoning and thought processes,
that play an essential role in the remarkable human ability to make rational decisions in an environment of
uncertainty and imprecision. Artificial neural networks attempt to emulate the architecture and information
representation schemes of the human brain. Genetic algorithms provide an adaptive, robust, parallel and
randomized searching technique where a population of solutions evolves over a sequence of generations, to
a globally optimal solution.
While the development of individual tools—fuzzy logic, artificial neural networks, genetic algorithms—
was in progress, a group of researchers felt the need of integrating them in order to enjoy the merits of
 xii Preface

different biologically inspired techniques into one system. The result is the development of several hybrid
paradigms, like neuro-fuzzy, neuro-genetic, fuzzy-genetic, and neuro-fuzzy-genetic. These hybrid paradigms
are suitable for solving complex real-world problems for which only one tool may not be adequate.
We are passing through a phase of rosy outlook for control technology: there has been a tendency to
portray neural network and/or a fuzzy-logic based controller as magical; a sort of black box that does
amazing things. Enthusiastic researchers have dreamed big dreams and made sweeping predictions on the
potential of intelligent control systems with neural processor/fuzzy logic chips in the control loop. The
growth of the new field during the past few years has been so explosive that the traditional model-based
control theory and the traditional op amp/computer-based implementations of controllers are under threat.
New design methods are required to prove themselves in actual practice, before they can displace well-
accepted techniques. Intelligent control technology is slowly gaining wider acceptance among academics
and industry. Even if there has been some over-enthusiastic description of the field, the scientific commu-
nity, and industry, are converging to the fact that there is something fundamentally significant about this
technology. Also, preparations are underway, to accord a warm welcome to integrated control technology–
integration of intelligent control theory with the traditional model-based theory, and integration of the VLSI
microprocessors with neuro-fuzzy processors to implement the controller.
This book includes a tutorial introduction to the knowledge-based tools for control system design. Rigor-
ous characterization of theoretical properties of intelligent control methodology will not be our aim in our
tutorial presentation; rather we focus on the development of systematic engineering procedures, which will
guide the design of the controller for a specific problem.
The arrangement of topics in the text, is a little different from the conventional one. Typically, a book on
digital control systems starts with transform-domain design and then carries over to state space. These books
give a detailed account of state variable methods for discrete-time systems. Since the state variable methods
for discrete-time systems run quite parallel to those for continuous-time systems, full-blown repetition is not
appreciated by the readers conversant with state variable methods for continuous-time systems. And for
readers with no background of this type, a natural way of introducing state variable methods, is to give the
treatment for continuous-time systems, followed by a brief parallel presentation for discrete-time systems.
To meet this objective, we have divided the treatment of transform-domain design and state-space design
in two parts of the book. Part I deals with digital control principles and design in transform domain, assum-
ing that the reader has had an introductory course in control engineering concentrating on the basic princi-
ples of feedback control and covering various classical analog methods of control system design. The mate-
rial presented in this part of the book is closely related to the material already familiar, but towards the end
a direction to wider horizons is indicated. Part II of the book deals with state variable methods in automatic
control. State variable analysis and design methods are usually not covered in an introductory course. It is
assumed that the reader is not exposed to the so-called modern control theory. Our approach is to first
discuss state variable methods for continuous-time systems, and then give a compact presentation of the
methods for discrete-time systems using the analogy with the continuous-time systems.
For the purpose of organizing different courses for students with different backgrounds, the sequencing
of chapters, and dependence of each chapter on previous chapters, has been properly designed in the text.
The typical engineering curriculum at the second-degree level includes core courses on ‘digital control
systems’, and ‘linear system theory’. Parts I and II of the book have been designed to fully meet the require-
ments of the two courses. In Part III of the book, a reasonable detailed account of nonlinear control schemes,
both the conventional and the intelligent, is given. The requirements of elective courses on ‘nonlinear
 Preface xiii
control systems’, and ‘intelligent control’, will be partially or fully (depending on the depth of coverage of
the courses) served by Part III of the book.
The typical engineering curriculum at the first-degree level includes a core course on feedback control
systems, with one or two elective courses on the subject. The book meets the requirements of elective courses
at the first-degree level.

WEBSITE URL
The Online Learning Center for the book:
http://www.mhhe.com/gopal/dc3e
· provides the students with the source codes of the MATLAB problems given in Appendix A and
Appendix B of the book; and
· provides the faculty with the Solution Manual for the exercise problems given at the end of all chapters
in the book. This part of the site is password protected, and will be made available by the Publisher to
the faculty on request.

M Gopal
[email protected]
 Part I
DIGITAL CONTROL: PRINCIPLES AND DESIGN IN
TRANSFORM-DOMAIN

Automatic control systems play a vital role in the (technological) progress of human civilization. These
control systems range from the very simple to the fairly complex in nature. Automatic washing machines,
refrigerators, and ovens are examples of some of the simpler systems used in the home. Aircraft auto-
matic pilots, robots used in manufacturing, and electric power generation and distribution systems repre-
sent complex control systems. Even such problems as inventory control, and socio-economic systems
control, may be approached from the theory of feedback control.
Our world is one of continuous-time variables type. Quantities like flow, temperature, voltage,
position, and velocity are not discrete-time variables but continuous-time ones. If we look back at the
development of automatic control, we find that mass-produced analog (electronic) controllers have been
available since about the 1940s. A first-level introduction to control engineering, provided in the
companion book ‘Control Systems: Principles and Design’, deals with basics of control, and covers
sufficient material to enable us to design analog (op amp based) controllers for many simple control loops
found in industry.
From the 1980s onward, we find microprocessor digital technology beginning to take over. Today,
most complex industrial processes are under computer control. A microprocessor determines the input to
manipulate the physical system, or plant; and this requires facilities to apply this input to the physical
world. In addition, the control strategy typically relies on measured values of the plant behaviour; and this
requires a mechanism to make these measured values available to the computing resources. The plant can
be viewed as changing continuously with time. The controller, however, has a discrete clock that governs
its behaviour and so its values change only at discrete points in time. To obtain deterministic behaviour
and ensure data integrity, the sensor must include a mechanism to sample continuous data at discrete
points in time, while the actuators need to produce a continuous value between the time points with
discrete-time data.
Computer interfacing for data acquisition, consists of analog-to-digital (A/D) conversion of the input
(to controller) analog signals. Prior to the conversion, the analog signal has to be conditioned to meet the
input requirements of the A/D converter. Signal conditioning consists of amplification (for sensors
generating very low power signals), filtering (to limit the amount of noise on the signal), and isolation
(to protect the sensors from interacting with one another and/or to protect the signals from possible
damaging inputs). Conversion of a digital signal to an analog signal (D/A) at the output (of controller), is
to be carried out to send this signal to an actuator which requires an analog signal. The signal has to be
amplified by a transistor or solid state relay or power amplifier. Most manufacturers of electronic instru-
mentation devices are producing signal conditioners as modules.
The immersion of the computing power into the physical world has changed the scene of control
system design. A comprehensive theory of digital “sampled” control has been developed. This theory
requires a sophisticated use of new concepts such as z-transform. It is, however, quite straightforward to
translate analog design concepts into digital equivalents. After taking a guided tour through the analog
design concepts and op amp technology, the reader will find in Part I of this book sufficient material to
enable him/her to design digital controllers for many simple control loops, and interfacing the controllers
to other sub-systems in the loop; thereby building complete feedback control systems.
 The broad space of digital control applications can be roughly divided into two categories: industrial
control and embedded control. Industrial control applications are those in which control is used as part of
the process of creating or producing an end product. The control system is not a part of the actual end
product itself. Examples include the manufacture of pharmaceuticals and the refining of oil. In the case of
industrial control, the control system must be robust and reliable, since the processes typically run
continuously for days, weeks or years.
Embedded control applications are those in which the control system is a component of the end prod-
uct itself. For example, Electronic Control Units (ECUs) are found in a wide variety of products including
automobiles, airplanes, and home applications. Most of these ECUs implement different feedback con-
trol tasks. For instance, engine control, traction control, anti-lock braking, active stability control, cruise
control, and climate control. While embedded control systems must also be reliable, cost is a more sig-
nificant factor, since the components of the control system contribute to the overall cost of manufacture
of the product. In this case, much more time and effort is usually spent in the design phase of the control
system to ensure reliable performance without requiring any unnecessary excess of processing power,
memory, sensors, actuators etc., in the digital control system. Our focus in this book will be on industrial
control applications.
Perhaps more than any other factor, the development of microprocessors has been responsible for the
explosive growth of the computer industry. While early micoprocessors required many additional
components in order to perform any useful task, the increasing use of large-scale integration (LSI) or
very large-scale integration (VLSI) semiconductor fabrication techniques has led to the production of
microcomputers, where all of the required circuitry is embedded on one or a small number of integrated
circuits. A further extension of the integration is the single chip microcontroller, which adds analog and
binary I/O, timers, and counters so as to be able to carry out real-time control functions with almost no
additional hardware. Examples of such microcontrollers are Intel 8051, 8096 and Motorola MCH
68HC11. These chips were developed largely, in response to the automotive industries’ desire for
computer-controlled ignition, emission control and anti-skid systems. They are now widely used in proc-
ess industries. This digital control practice, along with the theory of sampled-data systems will be dealt
with in Chapters 2–4 of the book.
 Introduction
1
1.1 CONTROL SYSTEM TERMINOLOGY

A Control System is an interconnection of components to provide a desired function. The portion of the
system to be controlled is given various names: process, plant, and controlled system being perhaps the most
common. The portion of the system that does the controlling is the controller. Often, a control system
designer has little or no design freedom with the plant; it is usually fixed. The designer’s task is, therefore, to
develop a controller that will control the given plant acceptably. When measurements of the plant response
are available to the controller (which in turn generates signals affecting the plant), the configuration is a
feedback control system.
A digital control system uses digital hardware, usually in the form of a programmed digital computer, as
the heart of the controller. In contrast, the controller in an analog control system is composed of analog
hardware; an electronic controller made of resistors, capacitors, and operational amplifiers is a typical
example. Digital controllers normally have analog devices at their periphery to interface with the plant; it is
the internal working of the controller that distinguishes digital from analog control.
The signals used in the description of control systems are classified as continuous-time and discrete-time.
Continuous-time signals are defined for all time, whereas discrete-time signals are defined only at discrete
instants of time, usually evenly spaced steps. The signals for which both time and amplitude are discrete, are
called digital signals. Because of the complexity of dealing with quantized (discrete-amplitude) signals,
digital control system design proceeds as if computer-generated signals were not of discrete amplitude. If
necessary, further analysis is then done, to determine if a proposed level of quantization is acceptable.
Systems and system components are termed continuous-time or discrete-time according to the type of
signals they involve. They are classified as being linear if signal components in them can be superim-
posed—any linear combination of signal components, applied to a system, produces the same linear combi-
nation of corresponding output components; otherwise a system is nonlinear. A system or component is
time-invariant if its properties do not change with time—any time shift of the inputs produces an equal time
shift of every corresponding signal. If a system is not time-invariant, then it is time-varying.
A typical topology of a computer-controlled system is sketched schematically in Fig. 1.1. In most cases,
the measuring transducer (sensor) and the actuator (final control element) are analog devices, requiring,
respectively, analog-to-digital (A/D) and digital-to-analog (D/A) conversion at the computer input and
 4 Digital Control and State Variable Methods
output. There are, of course, exceptions; sensors which combine the functions of the transducer and the A/D
converter, and actuators which combine the functions of the D/A converter and the final control element are
available. In most cases, however, our sensors will provide an analog voltage output, and our final control
elements will accept an analog voltage input.
Command input Disturbance Controlled
(Desired plant behaviour) inputs output

Final
A/D Computer D/A control Plant
element

Clock

Sensor

Fig. 1.1 Basic structure of a computer-controlled system

In the control scheme of Fig. 1.1, the A/D converter performs the sampling of the sensor signal (analog
feedback signal ) and produces its binary representation. The digital computer (control algorithm) generates
a digital control signal using the information on desired and actual plant behaviour. The digital control signal
is then converted to analog control signal via the D/A converter. A real-time clock synchronizes the actions
of the A/D and D/A converters, and the shift registers. The analog control signal is applied to the plant
actuator to control the plant’s behaviour.
The overall system in Fig. 1.1 is hybrid in nature; the signals are in the sampled form (discrete-time
signals) in the computer, and in a continuous form in the plant. Such systems have traditionally been called
sampled-data systems; we will use this term as a synonym for computer control systems/digital control
systems.
The word ‘servomechanism’ (or servosystem) is used for a command-following system wherein the con-
trolled output of the system is required to follow a given command. When the desired value of the controlled
outputs is more or less fixed, and the main problem is to reject disturbance effects, the control system is
sometimes called a regulator. The command input for a regulator becomes a constant and is called set-point,
which corresponds to the desired value of the controlled output. The set-point may however be changed in
time, from one constant value to another. In a tracking system, the controlled output is required to follow, or
track, a time-varying command input.
To make these definitions more concrete, let us consider some familiar examples of control systems.

Example 1.1 Servomechanism for Steering of Antenna

One of the earliest applications of radar tracking was for anti-aircraft fire control, first with guns and later
with missiles. Today, many civilian applications exist as well, such as satellite-tracking radars, navigation-
aiding radars, etc.
The radar scene includes the radar itself, a target, and the transmitted waveform that travels to the target
and back. Information about the target’s spatial position is first obtained by measuring the changes in the
back-scattered waveform relative to the transmitted waveform. The time shift provides information about
the target’s range, the frequency shift provides information about the target’s radial velocity, and the
 Introduction 5
received voltage magnitude and phase provide infor-
nna ax
ismation about the target’s angle1[1].
Ante In a typical radar application, it is necessary to
point the radar antenna toward the target and follow its
movements. The radar sensor detects the error be-
tween the antenna axis and the target, and directs the
antenna to follow the target. The servomechanism for
Elevation Azimuth steering the antenna in response to commands from
a b radar sensor, is considered here. The antenna is
designed for two independent angular motions, one
about the vertical axis in which the azimuth angle is
Fig. 1.2 Antenna configuration varied, and the other about the horizontal axis in
which the elevation angle is varied (Fig. 1.2). The
servomechanism for steering the antenna is described by two controlled variables—azimuth angle b and
elevation angle a. The desired values or commands are the azimuth angle br and the elevation angle ar of the
target. The feedback control problem involves error self-nulling, under conditions of disturbances beyond
our control (such as wind power).
The control system for steering antenna can be treated as two independent systems—the azimuth-angle
servomechanism, and the elevation-angle servomechanism. This is because the interaction effects are
usually small. The operational diagram of the azimuth-angle servomechanism is shown in Fig. 1.3.
The steering command from the radar sensor, which corresponds to target azimuth angle, is compared with
the azimuth angle of the antenna axis. The occurrence of the azimuth-angle error causes an error signal to pass
through the amplifier, which increases the angular velocity of the servomotor in a direction towards an error
reduction. In the scheme of Fig. 1.3, the measurement and processing of signals (calculation of control signal)
is digital in nature. The shaft-angle encoder combines the functions of transducer and A/D converter.

Fig. 1.3 Azimuthal servomechanism for steering of antenna

Figure 1.4 gives the functional block diagrams of the control system. A simple model of the load
(antenna) on the motor is shown in Fig. 1.4b. The moment of inertia J and the viscous friction coefficient B
are the parameters of the assumed model. Nominal load is included in the plant model for the control design.
1
The bracketed numbers coincide with the list of references given at the end of the book.
 6 Digital Control and State Variable Methods
The main disturbance inputs are the deviation of the load from the nominal estimated value as a result of
uncertainties in our estimate, effect of wind power, etc.
Load
disturbance

˜r Motor ˜ B
+ e Power
+ J
amplifier
– Load

(a) (b)
Load
disturbance

˜r + Motor ˜
e + Power
amplifier +
Load
– –

Tachogenerator
Rate
signal

(c)
Fig. 1.4 Functional block diagrams of azimuthal servomechanism

In the tracking system of Fig. 1.4a, the occurrence of error causes the motor to rotate in a direction
favouring the dissolution of error. The processing of the error signal (calculation of the control signal) is based
on the proportional control logic. Note that the components of our system cannot respond instantaneously,
since any real-world system cannot go from one energy level to another in zero time. Thus, in any real-world
system, there is some kind of dynamic lagging behaviour between input and output. In the servosystem of
Fig. 1.4a, the control action, on occurrence of the deviation of the controlled output from the desired value
(the occurrence of error), will be delayed by the cumulative dynamic lags of the shaft-angle encoder, digital
computer and digital-to-analog converter, power amplifier, and the servomotor with load. Eventually, how-
ever, the trend of the controlled variable deviation from the desired value, will be reversed by the action of
the amplifier output on the rotation of the motor, returning the controlled variable towards the desired value.
Now, if a strong correction (high amplifier gain) is applied (which is desirable from the point of view of
control system performance, e.g., strong correction improves the speed of response), the controlled variable
overshoots the desired value (the ‘run-out’ of the motor towards an error with the opposite rotation), causing
a reversal in the algebraic sign of the system error. Unfortunately, because of system dynamic lags, a reversal
of correction does not occur immediately, and the amplifier output (acting on ‘old’ information) is now
actually driving the controlled variable in the direction it was already heading, instead of opposing its excur-
sions, thus leading to a larger deviation. Eventually, the reversed error does cause a reversed correction, but
the controlled variable overshoots the desired value in the opposite direction and the correction is again in
the wrong direction. The controlled variable is thus driven, alternatively, in opposite directions before it
settles on to an equilibrium condition. This oscillatory state is unacceptable as the behaviour of antenna-
 Introduction 7
steering servomechanism. The considerable amplifier gain, which is necessary if high accuracies are to be
obtained, aggravates the described unfavourable phenomenon.
The occurrence of these oscillatory effects can be controlled by the application of special compensation
feedback. When a signal proportional to motor’s angular velocity (called the rate signal) is subtracted from
the error signal (Fig. 1.4c), the braking process starts sooner before the error reaches a zero value.
The ‘loop within a loop’ (velocity feedback system embedded within a position feedback system)
configuration utilized in this application, is a classical scheme called minor-loop feedback scheme.

Example 1.2 Variable Speed dc Drive

Many industrial applications require variable speed drives. For example, variable speed drives are used for
pumping duty to vary the flow rate or the pumping pressure, rolling mills, harbour cranes, rail traction, etc.
[2–4].
The variable speed dc drive is the most versatile drive available. The Silicon Controlled Rectifiers (SCR)
are almost universally used to control the speed of dc motors, because of considerable benefits that accrue
from the compact static controllers supplied directly from the ac mains.
Basically, all the dc systems involving the SCR controllers are similar but, with different configurations
of the devices, different characteristics may be obtained from the controller. Figure 1.5 shows a dc motor
driven by a full-wave rectified supply. Armature current of the dc motor is controlled by an SCR which is, in
turn, controlled by the pulses applied by the SCR trigger control circuit. The SCR controller thus combines
the functions of a D/A converter and a final control element.

Load

Speed Actual
reference + – speed

Tachogenerator

A /D Motor
Full Over-current
bridge protection
ac Digital
rectifier
computer
circuit
SCR SCR
trigger control
circuit

Fig. 1.5 Variable speed dc drive

Firing angle of the SCR controls the average armature current which, in turn, controls the speed of the dc
motor. The average armature current (speed) increases as the trigger circuit reduces the delay angle of firing of
the SCR, and the average armature current (speed) reduces as the delay angle of firing of the SCR is increased.
In the regulator system of Fig. 1.5, the reference voltage which corresponds to the desired speed of the dc
motor, is compared with the output voltage of tachogenerator, corresponding to the actual speed of the
motor. The occurrence of the error in speed, causes an error signal to pass through the trigger circuit, which
 8 Digital Control and State Variable Methods
controls the firing angle of the SCR in a direction towards an error reduction. When the processing of the
error signal (calculation of the control signal) is based on the proportional control logic, a steady-state error
between the actual speed and the desired speed exists. The occurrence of steady-state error can be elimi-
nated by generating the control signal with two components: one component proportional to the error signal,
and the other proportional to the integral of the error signal.

Example 1.3 Liquid-level Control System

This example describes the hardware features of the design of a PC-based liquid-level control system. The
plant of our control system is a cylindrical tank. Liquid is pumped into the tank from the sump (Fig. 1.6). The
inflow to the tank can be controlled by adjusting valve V1. The outflow from the tank goes back into the sump.
Valve V1 of our plant is a rotary valve; a stepping motor has been used to control the valve. The stepping
motor controller card, interfaced to the PC, converts the digital control signals into a series of pulses which
are fed to the stepping motor using a driver circuit. Three signals are generated from the digital control
signal at each sampling instant, namely, number of steps, speed of rotation, and direction of rotation. The
stepping-motor driver circuit converts this information into a single pulse train, which is fed to the stepping
motor. The valve characteristics between the number of steps of the stepping motor and the outflow from the
valve, are nonlinear.
The probe used for measurement of liquid level, consists of two concentric cylinders connected to a
bridge circuit, to provide an analog voltage. The liquid partially occupies the space between the cylinders,
with air in the remaining part. This device acts like two capacitors in parallel, one with dielectric constant of
air ( ~- 1) and the other with that of the liquid. Thus, the variation of the liquid level causes variation of the
electrical capacity, measured between the cylinders. The change in the capacitance causes a change in the
bridge output voltage which is fed to the PC through an amplifier circuit. The characteristics of the sensor
between the level and the voltage are approximately linear.

Step motor
controller card
Expansion
slots of PC

Driver Step
circuit motor
V1 V2

Pump Sump

A /D
conversion card Bridge and
amplifier circuit

Fig. 1.6 Liquid-level control system

In the liquid-level control system of Fig. 1.6, the command signal (which corresponds to the desired level
of the liquid in the cylinder) is fed through the keyboard; the actual level signal is received through the A/D
 Introduction 9
conversion card. The digital computer compares the two signals at each sampling instant, and generates a
control signal which is the sum of two components: one proportional to the error signal, and the other,
proportional to the integral of the error signal.

1.2 COMPUTER-BASED CONTROL: HISTORY AND TRENDS

Digital computers were first applied to the industrial process control in the late 1950s. The machines were
generally large-scale ‘main frames’ and were used in a so-called supervisory control mode; the individual
temperature, pressure, flow and the like, feedback loops were locally controlled by electronic or pneumatic
analog controllers. The main function of the computer was to gather information on how the overall process
was operating, feed this into a technical-economic model of the process (programmed into computer
memory), and then, periodically, send signals to the set-points of all the analog controllers, so that each
individual loop operated in such a way as to optimize the overall operation.
In 1962, a drastic departure from this approach was made by Imperial Chemical Industries in England—
a digital computer was installed, which measured 224 variables and manipulated 129 valves directly. The
name Direct Digital Control (DDC) was coined to emphasize that the computer controlled the process
directly. In DDC systems, analog controllers were no longer used. The central computer served as a single,
time-shared controller for all the individual feedback loops. Conventional control laws were still used for
each loop, but the digital versions of control laws for each loop resided in the software in the central compu-
ter. Though digital computers were very expensive, one expected DDC systems to have economic advantage
for processes with many (50 or more) loops. Unfortunately, this did not often materialize. As failures in the
central computer of a DDC system shut down the entire system, it was necessary to provide a ‘fail-safe’
back-up system, which usually turned out to be a complete system of individual loop analog controllers, thus
negating the expected hardware savings.
There was a substantial development of digital computer technology in the 1960s. By the early 1970s,
smaller, faster, more reliable, and cheaper computers became available. The term minicomputers was coined
for the new computers that emerged. DEC PDP11 is by far, the best-known example. There were, however,
many related machines from other vendors.
The minicomputer was still a fairly large system. Even as performance continued to increase and prices to
decrease, the price of a minicomputer main frame in 1975, was still about $10,000. Computer control was still
out of reach for a large number of control problems. However, with the development of microcomputer, the
price of a card computer, with the performance of a 1975 minicomputer, dropped to $500 in 1980. Another
consequence was that digital computing power in 1980 came in quanta as small as $50. This meant that
computer control could now be considered as an alternative, no matter how small the application [54-57].
Microcomputers have already made a great impact on the process control field. They are replacing analog
hardware even as single-loop controllers. Small DDC systems have been made using microcomputers.
Operator communication has vastly improved with the introduction of colour video-graphics displays.
The variety of commercially available industrial controllers ranges from single-loop controllers through
multiloop single computer systems to multiloop distributed computers. Although the range of equipment
available is large, there are a number of identifiable trends which are apparent.
Single-loop microprocessor-based controllers, though descendants of single-loop analog controllers, have
greater degree of flexibility. Control actions which are permitted, include on/off control, proportional action,
integral action, derivative action and the lag effect. Many controllers have self-tuning option. During the self-
tune sequence, the controller introduces a number of step commands, within the tolerances allowed by the
operator, in order to characterize the system response. From this response, values for proportional gain, reset
time, and rate time are developed. This feature of online tuning in industrial controllers is interesting, and
permits the concept of the computer automatically adjusting to changing process conditions [11-12].
 10 Digital Control and State Variable Methods
Multiloop single computer systems have variability in available interface and software design. Both the
single-loop and multiloop controllers may be used in stand-alone mode, or may be interfaced to a host
computer for distributed operation. The reducing costs and increasing power of computing systems, has
tended to make distributed computing systems for larger installations, far more cost effective than those built
around one large computer. However, the smaller installation may be best catered for by a single multiloop
controller, or even a few single-loop devices.
Control of large and complex processes using distributed computer control systems (DCCS), is facili-
tated by adopting a multilevel or hierarchical view point of control strategy. The multilevel approach sub-
divides the system into a hierarchy of simpler control design problems. On the lowest level of control (direct
process control level), the following tasks are handled: acquisition of process data, i.e., collection of instan-
taneous values of individual process variables, and status messages of plant control facilities (valves, pumps,
motors, etc.) needed for efficient direct digital control; processing of collected data; plant hardware moni-
toring, system check and diagnosis; closed-loop control and logic control functions, based on directives
from the next ‘higher’ level.
Supervisory level copes with the problems of determination of optimal plant work conditions, and genera-
tion of relevant instructions to be transferred to the next ‘lower’ level. Adaptive control, optimal control, plant
performance monitoring, plant coordination and failure detections are the functions performed at this level.
Production scheduling and control level is responsible for production dispatching, inventory control,
production supervision, production re-scheduling, production reporting, etc.
Plant(s) management level, the ‘highest’ hierarchical level of the plant automation system, is in charge of
the wide spectrum of engineering, economic, commercial, personnel, and other functions.
It is, of course, not to be expected that in all available distributed computer control systems, all four
hierarchical levels are already implemented. For automation of small-scale plants, any DCCS having at least
two hierarchical levels, can be used. One system level can be used as a direct process control level, and the
second one as a combined plant supervisory, and production scheduling and control level. Production
planning and other enterprise-level activities, can be managed by the separate mainframe computer or the
computer centre. For instance, in a LAN (Local Area Network)-based system structure, shown in Fig. 1.7a,
the ‘higher’ automation levels are implemented by simply attaching the additional ‘higher’ level computers
to the LAN of the system [89].

Process Computer Higher

Management Centre hierarchical
Computer levels

LAN

Supervisory and Lower

direct process control hierarchical
levels
Control Plant
signals inputs

PLANT

Fig. 1.7 a Hierarchical levels in Computer Integrated Process Systems (CIPS)

 Introduction 11
For complex process plant monitoring, SCADA (Supervisory Control And Data Acquisition) systems are
available. The basic functions carried out by a SCADA system are:
· Data acquisition and communication · Events and alarms reporting
· Data processing · Partial process control
The full process control functions are delegated to the special control units, connected to SCADA system,
and are capable of handling the emergency shut down situations.
The separation of SCADA and DCCS is slowly vanishing and the SCADA systems are being
brought within the field of DCCS; the hierarchical, distributed, flexible and extremely powerful Computer
Integrated Process Systems (CIPS), is now a technical reality.
The other main and early application area of digital methods was machine tool numerical control, which
developed at about the same time as computer control in process industries. Earlier, numerically controlled
(NC) machines used ‘hard-wired’ digital techniques. As the microcomputer price and performance im-
proved, it became feasible to replace the hard-wired functions with their software-implemented equivalents,
using a microcomputer as a built-in component of the machine tool. This approach has been called Compu-
terized Numerical Control (CNC) [20]. Industrial robots were developed simultaneously with CNC systems.
A quiet revolution is going on in the manufacturing world that is changing the look of the factory. Com-
puters are controlling and monitoring the manufacturing processes [21-22]. The high degree of automation
that, until recently, was reserved for mass production only, is also applied now to small batches. This
requires a change from hard automation in the production line, to a flexible manufacturing system (FMS),
which can be more readily rearranged to handle new market requirements.
Flexible manufacturing systems, combined with automatic assembly and product inspection on one hand,
and CAD/CAM systems on the other, are the basic components of the modern Computer Integrated Manu-
facturing System (CIMS). In a CIMS, the production flow, from the conceptual design to the finished prod-
uct, is entirely under computer control and management.
Figure 1.7b illustrates hierarchical structure of CIMS. The lowest level of this structure contains stand-
alone computer control systems of manufacturing processes and industrial robots. The computer control of
processes includes all types of CNC machine tools, welding, electrochemical machining, electrical discharge
machining, and a high-power laser, as well as the adaptive control of these processes.

Supervisory computer

Transfer Automatic
CAD/CAM FMS system inspection

Raw Material
Products
DNC Manufacturing Assembly
system cell station

Computer Control Industrial

of Processes Robots

Fig. 1.7 b Hierarchical levels in Computer Integrated Manufacturing Systems (CIMS)

 12 Digital Control and State Variable Methods
When a battery of NC or CNC machine tools is placed under the control of a single computer, the result
is a system known as direct numerical control (DNC).
The operation of several CNC machines and industrial robots, can be co-ordinated by systems called
manufacturing cells. The computer of the cell is interfaced with the computer of the robot and CNC
machines. It receives “completion of job” signals from the machines and issues instructions to the robot to
load and unload the machines, and change their tools. The software includes strategies permitting the
handling of machine breakdown, tool breakage, and other special situations.
The operation of many manufacturing cells can be coordinated by flexible manufacturing system (FMS).
The FMS accepts incoming workpieces and processes them under computer control, into finished parts.
The parts produced by the FMS must be assembled into the final product. They are routed on a transfer
system to assembly stations. In each station, a robot will assemble parts, either into a sub-assembly or (for
simple units), into the final product. The sub-assemblies will be further assembled by robots located in other
stations. The final product will be tested by an automatic inspection system.
The FMS uses CAD/CAM systems to integrate the design and manufacturing of parts. At the highest
hierarchical level, there will be a supervisory computer, which coordinates participation of computers in all
phases of a manufacturing enterprise: the design of the product, the planning of its manufacture, the auto-
matic production of parts, automatic assembly, automatic testing, and, of course, computer-controlled flow
of materials and parts through the plant.
In a LAN-based system, the ‘higher’ automation levels (production planning and other enterprise-level
activities), can be implemented by simply attaching the additional ‘higher’ level computers to the LAN of
the system.
One of the most ingenious devices ever devised to advance the field of industrial automation, is the
Programmable Logic Controller (PLC). The PLC, a microprocessor-based general purpose device, pro-
vides a ‘menu’ of basic operations that can be configured by programming to create logic control system for
any application [23–25]. So versatile are these devices, that they are employed in the automation of almost
every type of industry. CIPS and CIMS provide interfaces to PLCs for handling high-speed logic (and other)
control functions. Thousands of these devices go unrecognized in process plants and factory environments—
quietly monitoring security, manipulating valves, and controlling machines and automatic production lines.
Thus, we see that the recent appearance of powerful and inexpensive microcomputers, has made digital
control practical for a wide variety of applications. In fact, now every process is a candidate for digital
control. The flourishing of digital control is just beginning for most industries, and there is much to be
gained by exploiting the full potential of new technology. There is every indication that a high rate of growth
in the capability and application of digital computers, will continue far into the future.

1.3 CONTROL THEORY: HISTORY AND TRENDS

The development of control system analysis and design can be divided into three eras. In the first era,
we have the classical control theory, which deals with techniques developed during the 1940s and 1950s.
Classical control methods—Routh-Hurwitz, Root Locus, Nyquist, Bode, Nichols—have in common the use
of transfer functions in the complex frequency (Laplace variable s) domain, and the emphasis on the graphi-
cal techniques. Since computers were not available at that time, a great deal of emphasis was placed on
developing methods that were amenable to manual computation and graphics. A major limitation of the
classical control methods was the use of single-input, single-output (SISO) control configurations. Also, the
use of the transfer function and frequency domain limited one to linear time-invariant systems. Important
results of this era will be discussed in Part I of this book.
In the second era, we have modern control (which is not so modern any longer), which refers to state-
space based methods developed in the late 1950s and early 1960s. In modern control, system models are
 Introduction 13
directly written in the time domain. Analysis and design are also carried out in the time domain. It should be
noted that before Laplace transforms and transfer functions became popular in the 1920s, engineers were
studying systems in the time domain. Therefore, the resurgence of time-domain analysis was not unusual, but
it was triggered by the development of computers and advances in numerical analysis. As computers were
available, it was no longer necessary to develop analysis and design methods that were strictly manual.
Multivariable (multi-input, multi-output (MIMO)) control configurations could be analysed and designed.
An engineer could use computers to numerically solve or simulate large systems that were nonlinear and/or
time-varying. Important results of this era—Lyapunov stability criterion, pole-placement by state feedback,
state observers, optimal control—will be discussed in Part II of this book.
Modern control methods initially enjoyed a great deal of success in academic circles, but they did not
perform very well in many areas of application. Modern control provided a lot of insight into system struc-
ture and properties, but it masked other important feedback properties that could be studied and manipulated
using the classical control theory. A basic requirement in control engineering is to design control systems
that will work properly when the plant model is uncertain. This issue is tackled in the classical control theory
using gain and phase margins. Most modern control design methods, however, inherently require a precise
model of the plant. In the years since these methods were developed, there have been few significant
implementations and most of them have been in a single application area—the aerospace industry. The
classical control theory, on the other hand, is going strong. It provides an efficient framework for the design
of feedback controls in all areas of application. The classical design methods have been greatly enhanced by
the availability of low-cost computers for system analysis and simulation. The graphical tools of classical
design can now be more easily used with computer graphics for SISO as well as MIMO systems.
During the past three decades, the control theory has experienced a rapid expansion, as a result of the
challenges of the stringent requirements posed by modern systems such as: flight vehicles, weapon control
systems, robots, and chemical processes; and the availability of low-cost computing power. A body of
methods emerged during this third era of control-theory development, which tried to provide answers to the
problems of plant uncertainty. These techniques, commonly known as robust control, are a combination of
modern state-space and classical frequency-domain techniques. For a thorough understanding of these new
methods, we need to have adequate knowledge of state-space methods, in addition to the frequency-domain
methods. This has guided the preparation of this text.
Robust control system design has been dominated by linear control techniques, which rely on the key
assumption of availability of uncertainty model. When the required operation range is large, and a reliable
uncertainty model could not be developed, a linear controller is likely to perform very poorly. Nonlinear
controllers, on the other hand, may handle the nonlinearities in large range operation, directly. Also,
nonlinearities can be intentionally introduced into the controller part of a control system, so that the model
uncertainties can be tolerated. Advances in computer technology have made the implementation of nonlinear
control schemes—feedback linearization, variable structure sliding mode control, adaptive control,
gain-scheduling—a relatively simple task.
The third era of control-theory development has also given an alternative to model-based design meth-
ods: the knowledge-based control. In this approach, we look for a control solution that exhibits intelligent
behaviour, rather than using purely mathematical methods to keep the system under control.
Model-based control techniques have many advantages. When the underlying assumptions are satisfied,
many of these methods provide good stability, robustness to model uncertainties and disturbances, and speed of
response. However, there are many practical deficiencies of these ‘crisp’ (‘hard’ or ‘inflexible’) control algo-
rithms. It is generally difficult to accurately represent a complex process by a mathematical model. If the
process model has parameters whose values are partially known, ambiguous, or vague, crisp control algorithms
that are based on such incomplete information, will not usually give satisfactory results. The environment with
 14 Digital Control and State Variable Methods
which the process interacts, may not be completely predictable and it is normally not possible for a crisp
algorithm, to accurately respond to a condition that it did not anticipate, and that it could not ‘understand’.
Intelligent control is the name introduced to describe control systems in which control strategies are
based on AI techniques. In this control approach, which is an alternative to model-based control approach,
a behavioural (and not mathematical) description of the process is used, which is based on qualitative
expressions and experience of people working with the process. Actions can be performed either as a result
of evaluating rules (reasoning), or as unconscious actions based on presented process behaviour after a
learning phase. Intelligence comes in as the capability to reason about facts and rules, and to learn about
presented behaviour. It opens up the possibility of applying the experience gathered by operators and
process engineers. Uncertainty about the knowledge can be handled alongwith ignorance about the structure
of the system.
Fuzzy logic, and neural networks are very good methods to model real processes which cannot be described
mathematically. Fuzzy logic deals with linguistic and imprecise rules based on expert’s knowledge. Neural
networks are applied in the case where we do not have any rules but several data.
The main feature of fuzzy logic control is that a control engineering knowledge base (typically in terms of a
set of rules), created using expert’s knowledge of process behaviour, is available within the controller and the
control actions are generated by applying existing process conditions to the knowledge base, making use of an
inference mechanism. The knowledge base and the inference mechanism can handle noncrisp and incomplete
information, and the knowledge itself will improve and evolve through learning and past experience.
In the neural network based control, the goal of artificial neural network is to emulate the mechanism of
human brain function and reasoning, and to achieve the same intelligence level as the human brain in learn-
ing, abstraction, generalization and making decisions under uncertainty.
In the conventional design exercises, the system is modelled analytically by a set of differential equations,
and their solution tells the controller how to adjust the system’s control activities for each type of behaviour
required. In a typical intelligent control scheme, these adjustments are handled by an intelligent controller, a
logical model of thinking processes that a person might go through in the course of manipulating the system.
This shift in focus from the process to the person involved, changes the entire approach to automatic control
problems. It provides a new design paradigm such that a controller can be designed for complex, ill-defined
processes without knowing quantitative input-output relations, which are otherwise required by conven-
tional methods.
The ever-increasing demands of the complex control systems being built today, and planned for the
future, dictate the use of novel and more powerful methods in control. The potential for intelligent control
techniques in solving many of the problems involved is great, and this research area is evolving rapidly. The
emerging viewpoint is that model-based control techniques should be augmented with intelligent control
techniques in order to enhance the performance of the control systems. The developments in intelligent
control methods should be based on firm theoretical foundations (as is the case with model-based control
methods), but this is still at its early stages. Strong theoretical results guaranteeing control system properties
such as stability are still to come, although promising results reporting progress in special cases have been
reported recently. The potential of intelligent control systems clearly needs to be further explored and both
theory and applications need to be further developed. A brief account of nonlinear control schemes both the
conventional and the intelligent, will be given in Part III of this book.

1.4 AN OVERVIEW OF THE CLASSICAL APPROACH TO ANALOG CONTROLLER

DESIGN

The tools of classical linear control system design are the Laplace transform, stability testing, root locus, and
frequency response. Laplace transformation is used to convert system descriptions in terms of integro-
 Introduction 15
differential equations to equivalent algebraic relations involving rational functions. These are conveniently
manipulated in the form of transfer functions with block diagrams and signal flow graphs [155].
The block diagram of Fig. 1.8 represents the basic structure of feedback control systems. Not all systems
can be forced into this format, but it serves as a reference for discussion.
In Fig. 1.8, the variable y(t) is the controlled variable of the system. The desired value of the controlled
variable is yr(t), the command input. yr(t) and y(t) have the same units. The feedback elements with transfer
function H(s) are system components that act on the controlled variable y(t) to produce the feedback signal
b(t). H(s) typical represents the sensor action to convert the controlled variable y(t) to an electrical sensor
output signal b(t).

+ e
w
–
+ y
yr r + e u m+
A (s ) D (s) G A (s ) G P (s )
–
b
H (s )

Fig. 1.8 Generalised operational block diagram of a feedback system

The reference input elements with transfer function A(s) convert the command signal yr(t) into a form
compatible with the feedback signal b(t). The transformed command signal is the actual physical input to the
system. This actual signal input is defined as the reference input.
The comparison device (error detector) of the system compares the reference input r(t) with the feedback
signal b(t) and generates the actuating error signal e$ (t). The signals r(t), b(t), and e$ (t) have the same units. The
controller with transfer function D(s) acts on the actuating error signal to produce the control signal u(t).
The control signal u(t) has the knowledge about the desired control action. The power level of this signal
is relatively low. The actuator elements with transfer function GA(s), are the system components that act
on the control signal u(t) and develop enough torque, pressure, heat, etc. (manipulated variable m(t)), to
influence the controlled system. GP(s) is the transfer function of the controlled system.
The disturbance w(t) represents the undesired signals that tend to affect the controlled system. The
disturbance may be introduced into the system at more than one location.
The dashed-line portion of Fig. 1.8 shows the system error e(t) = yr – y(t). Note that the actuating error
signal e$ (t) and the system error e(t) are two different variables.
The basic feedback system block diagram of Fig. 1.8 is shown in abridged form in Fig. 1.9. The output
Y(s) is influenced by the control signal U(s) and the disturbance signal W(s) as per the following relation:
Y(s) = GP(s) GA(s) U(s) + GP(s) W(s) (1.1a)
= G(s) U(s) + N(s) W(s) (1.1b)
where G(s) is the transfer function from the control signal U(s) to the output Y(s), and N(s) is the transfer
function from the disturbance input W(s) to the output Y(s). Using Eqns (1.1), we can modify the block
diagram of Fig. 1.9 to the form shown in Fig. 1.10. Note that in the block diagram model of Fig. 1.10, the
plant includes the actuator elements.
 16 Digital Control and State Variable Methods
W (s)

R (s) + E (s) U (s) + Y (s)

Y4 ( s ) +
A (s) D (s) G A (s) GP ( s )
–
B (s)
H (s)

Fig. 1.9 A general linear feedback system

W (s) N (s )

Yr (s) R (s ) + E (s) U (s) + Y (s)

+
A (s) D (s) G (s)
–
B (s) Plant
H (s)

Fig. 1.10 Equivalent representation of the block diagram of Fig. 1.9

The actuating error signal

E$ (s) = R(s) – B(s) = A(s) Yr (s) – H(s) Y(s)

The control signal
U(s) = D(s) A(s) Yr (s) – D(s) H(s) Y(s) (1.2a)

= D(s) H(s)
LM A(s ) Y (s)
r Y (s)
OP (1.2b)
N H (s) Q
Using Eqns (1.2a) and (1.2b), we can simplify Fig. 1.10 to obtain the structure shown in Fig. 1.11.

W (s) N (s )

Yr (s) + U (s) + + Y (s)

A (s )
D (s) H (s) G (s)
H (s)
–

Fig. 1.11 Simplification of the block diagram of Fig. 1.10

W (s) N (s ) A further simplification of Fig. 1.11 is

possible if H = A. In this case, which is quite
common, we can model the system as the
Yr (s) + E (s) U (s) + + Y (s)
D (s) H (s) G (s) unity feedback system shown in Fig. 1.12,
and take advantage of the fact that now the
–
actuating signal is the system error e(t).
The block diagrams in Figs 1.10–1.12 are
Fig. 1.12 Unity feedback system very useful for the purpose of system design.
 Introduction 17
However, it should be clear that these block diagrams have lost the physical significance. For example, the
block in Fig. 1.11 with transfer function A(s)/H(s), does not refer to any physical portion of the original
system. Rather, it represents the result of manipulating Eqn. (1.2a) into the form given by Eqn. (1.2b).
Thus, the reader is advised to think in terms of the equations that the block diagrams represent, rather than
attach any special significance to the block diagrams themselves. The only role played by a block diagram is
that it is a convenient means of representing the various system equations, rather than writing them
out explicitly. Block diagram manipulation is nothing more than the manipulation of a set of algebraic
transform equations.
For the analysis of a feedback system, we
W (s) N (s ) require the transfer function between the
input—either reference or disturbance—and
R (s ) + E (s) U (s) + Y (s) the output. We can use block diagram ma-
+
D (s) G (s) nipulations to eliminate all the signals except
–
the input and the output. The reduced block
B (s) diagram leads to the desired result.
H (s)
Consider the block diagram of Fig. 1.13.
The feedback system has two inputs. We
Fig. 1.13 A typical feedback system with two inputs
shall use superposition to treat each input
separately.
R (s ) + E (s) U (s) Y (s ) When disturbance input is set to zero, the
D (s) G (s)
– single-input system of Fig. 1.14 results. The
transfer function between the input R(s) and
H (s) the output Y(s) is referred to as the reference
transfer function and will be denoted by
Fig. 1.14 Block diagram without disturbance input M(s). To solve for M(s), we write the pair of
transform equations
E (s) = R(s) – H(s) Y(s); Y(s) = G(s) U(s) = G(s) D(s) E$ (s)
$
and then eliminate E$ (s) to obtain
[1 + D(s) G(s) H(s)] Y(s) = D(s) G(s) R(s)
which leads to the desired result:
Y ( s) D( s)G ( s)
M(s) = = (1.3)
R( s) W ( s ) =0 1 + D( s)G ( s) H ( s)
Similarly, we obtain the disturbance transfer function Mw(s) by setting the reference input to zero in
Fig. 1.13 yielding Fig. 1.15, and then solving for Y(s)/W(s). From the revised block diagram,
E$ (s) = – H(s) Y(s); Y(s) = G(s) D(s) E$ (s) + N(s) W(s)
from which E$ (s) can be eliminated to give
Y ( s) N ( s)
Mw(s) = = (1.4)
W ( s) R ( s) = 0
1 + D( s)G ( s) H ( s)
The response to the simultaneous application of R(s) and W(s) is given by
Y(s) = M(s) R(s) + Mw(s) W(s) (1.5)
 18 Digital Control and State Variable Methods
Figure 1.16 shows the reduced block diagram model of the given feedback system.

W (s) N (s )
W (s)

E (s ) U (s) + + Y (s)
D (s) G (s) Mw ( s )

–1
R (s ) + + Y (s)
H (s) M (s)

Fig. 1.15 Block diagram without reference input Fig. 1.16 Reduced block diagram model
for system of Fig. 1.13

The transfer functions given by Eqns (1.3) and (1.4) are referred to as closed-loop transfer functions. The
denominator of these transfer functions has the term D(s)G(s)H(s) which is the multiplication of all the
transfer functions in the feedback loop. It may be viewed as the transfer function between the variables R(s)
and B(s) if the loop is broken at the summing point. D(s)G(s)H(s) may therefore be given the name open-
loop transfer function. The roots of denominator polynomial of D(s)G(s)H(s) are the open-loop poles, and
the roots of numerator polynomial of D(s)G(s)H(s) are the open-loop zeros.
The roots of the characteristic equation
1 + D(s)G(s)H(s) = 0 (1.6)
are the closed-loop poles of the system. These poles indicate whether or not the system is bounded-input
bounded-output (BIBO) stable, according to whether or not all the poles are in the left half of the complex
plane. Stability may be tested by the Routh stability criterion.
A root locus plot consists of a pole-zero plot of the open-loop transfer function of a feedback system,
upon which is superimposed the locus of the poles of the closed-loop transfer function as some parameter is
varied. Design of the controller (compensator) D(s) can be carried out using the root locus plot. One begins
with simple compensators, increasing their complexity until the performance requirements can be met. Prin-
cipal measures of transient performance are peak overshoot, settling time, and rise time. The compensator
poles, zeros, and multiplying constant are selected to give feedback system pole locations, that result in
acceptable transient response to step inputs. At the same time, the parameters are constrained so that the
resulting system has acceptable steady-state response to important inputs, such as steps and ramps.
Frequency response characterizations of systems have long been popular because of the ease and practi-
cality of steady-state sinusoidal response measurements. These methods also apply to systems in which
rational transfer function models are not adequate, such as those involving time delays. They do not require
explicit knowledge of system transfer function models; experimentally obtained open-loop sinusoidal
response data can directly be used for stability analysis and compensator design. A stability test, the Nyquist
criterion, is available. Principal measures of transient performance are gain margin, phase margin, and
bandwidth. The design of the compensator is conveniently carried out using the Bode plot and the Nichols
chart. One begins with simple compensators, increasing their complexity until the transient and steady-state
performance requirements are met.
There are two approaches to carrying out the digital controller (compensator) design. The first approach
uses the methods discussed above to design an analog compensator, and then transform it into a digital one.
The second approach, first transforms analog plants into digital plants, and then carries out the design using
digital techniques. The first approach performs discretization after design; the second approach performs
 Introduction 19
discretization before design. The classical approach to designing a digital compensator directly using an
equivalent digital plant for a given analog plant, parallels the classical approach to analog compensator
design. The concepts and tools of the classical digital design procedures will be given in Chapters 2–4. This
background will also be useful in understanding and applying the state-variable methods to follow.

1.5 SCOPE AND ORGANIZATION OF THE BOOK

This text is concerned with digital control and state variable methods for a special class of control systems,
namely time-invariant, lumped, and deterministic systems. We do not intend to solve all the problems that
can be posed under the defined category. Coverage of digital control theory and practice is modest. Various
concepts from interdisciplinary fields of computer science and computer engineering which relate to digital
control system development—number representation, logical descriptions of algorithmic processes, compu-
ter arithmetic operations, computer system hardware and software—are beyond the scope of this book. In
fact, a course on control engineering need not include these topics because specialized courses on computer-
system architecture are normally offered in undergraduate curricula of all engineering disciplines.
It is assumed that the reader has had an introductory course in control engineering concentrating on
the basic principles of feedback control and covering various classical analog methods of control system
design. The classical digital methods of design are developed in Part I of this book, paralleling and extend-
ing considerably the similar topics in analog control.
State variable analysis and design methods are usually not covered in an introductory course. It is
assumed that the reader is not exposed to the so-called modern control theory. Our approach in Part II of this
book is to first discuss state variable methods for continuous-time systems, and then give a compact presen-
tation of the methods for discrete-time systems, using the analogy with the continuous-time case.
This text also prepares a student for the study of advanced control methods. However, detailed coverage
of these methods is beyond the scope of the book; only a brief account is given in Part III.
There are eleven chapters in the book in addition to this introductory chapter. In each chapter, analysis/
design examples are interspersed to illustrate the concepts involved. At the end of each chapter, there are a
number of review examples that take the reader to a higher level of application; some of these examples also
serve the purpose of extending the text material. The same approach is followed in unsolved problems.
Answers to problems have been given to inspire confidence in the reader.
The examples we have considered in this book are generally low-order systems. Such a selection of
examples helps in conveying the fundamental concepts of feedback control without the distraction of large
amounts of computations, inherent in high-order systems. Many of the real-life design problems are more
complex than the ones discussed in this book. High-order systems are common and, in addition, several
parameters are to be varied during design stage to investigate their effect on the system performance. Com-
puter-Aided-Design (CAD) tools are extremely useful for complex control problems. Several software
packages with computer graphics are available commercially for CAD of control systems [151–154].
Let us now go through the organization of the chapters. Chapters 2–4 deal with digital control theory and
practice. The philosophy of presentation is that the new material should be closely related to material
already familiar, and yet, by the end, a direction to wider horizons should be indicated. The approach leads
us, for example, to relate the z-transform to the Laplace transform and to describe the implications of poles
and zeros in the z-plane to those known meanings attached to poles and zeros in the s-plane. Also, in devel-
oping the design methods we relate the digital control design methods to those of continuous-time systems.
Chapter 2 introduces the sampling theorem and the phenomenon of aliasing. Methods to generate
discrete-time models which approximate continuous-time dynamics, and stability analysis of these models
are also included in this chapter.
 20 Digital Control and State Variable Methods
Chapter 3 considers the hardware (analog and digital) of the control loop with emphasis on modelling.
Models of some of the widely used digital control systems are also included.
Chapter 4 establishes a toolkit of design-oriented techniques. It puts forward alternative design methods
based on root-locus and Bode plots. Design of digital controllers using z-plane synthesis is also included in
this chapter. References for the material in Chapters 2–4 are [11–12, 23–25, 30–31, 52, 80–98].
Chapters 5–8 deal with state variable methods in automatic control. Chapter 5 is on state variable analysis.
It exposes the problems of state variable representation, diagonalization, solution, controllability, and
observability. The relationship between the transfer function and state variable models is also given.
Although it is assumed that the reader has the necessary background on vector-matrix analysis, a reasonably
detailed account of vector-matrix analysis is provided in this chapter for convenient reference.
State variable analysis concepts, developed in continuous-time format in Chapter 5, are extended to
digital control systems in Chapter 6.
The techniques of achieving desired system characteristics by pole-placement using complete state
variable feedback are developed in Chapter 7. Also included is the method of using the system output to
form estimates of the states for use in state feedback. Results are given for both continuous-time and
discrete-time systems.
Lyapunov stability analysis is introduced in Chapter 8. In addition to stability analysis, Lyapunov func-
tions are useful in solving some optimization problems. We discuss in this chapter, the solution of linear
quadratic optimal control problem through Lyapunov synthesis. Results are given for both continuous-time
and discrete-time systems. References for the material in Chapters 5–8 are [27–28, 99–119, 122–124].
Describing function and phase-plane methods, which have demonstrated great utility in analysis of
nonlinear systems, have been paid considerable attention in Chapter 9. Also included is stability analysis of
nonlinear systems using Lyapunov functions. References for the material in Chapter 9 are [125-129].
The concepts of feedback linearization, model reference adaptive control, system identification and self-
tuning control, and variable structure control are briefly introduced in Chapter 10. Also included in this
chapter, is an introduction to an emerging nonlinear-control architecture—the reinforcement learning
control. For detailed study, refer [120–121, 125–136, 147–150].
A tutorial introduction to knowledge-based tools (neural networks, support vector machines, fuzzy logic,
genetic algorithms) for control system design is given in Chapters 11 and 12. Rigorous characterization of
theoretical properties of intelligent control methodology is not our aim in our tutorial presentation; rather,
we focus on the development of systematic engineering procedures, which will guide the design of controller
for a specific problem. For detailed study, refer [137–146].
Appendices A and B provide an introduction to the MATLAB environment for computer-aided control
system design.
 Signal Processing in Digital

2 Control

2.1 WHY USE DIGITAL CONTROL

Digital control systems offer many advantages over their analog counterparts. Of course, there are possible
problems also. Let us first look at the advantages of digital control over the corresponding analog control
before we talk of the price one has to pay for the digital option.

2.1.1 Advantages Offered by Digital Control

Flexibility An important advantage offered by digital control is in the flexibility of its modifying controller
characteristics, or of adapting the controller if plant dynamics change with operating conditions. The ability to
‘redesign’ the controller by changing software (rather than hardware) is an important feature of digital control
as against analog control.
Wide Selection of Control Algorithms Implementation of advanced control techniques was earlier
constrained by the limitations of analog controllers and the high costs of digital computers. However, with
the advent of inexpensive digital computers with virtually limitless computing power, the techniques of
modern control theory may now be put to practice. For example, in multivariable control systems with more
than one input and one output, modern techniques for optimizing system performance or reducing inter-
actions between feedback loops can now be implemented.
Integrated Control of Industrial Systems Feedback control is only one of the functions of the compu-
ter. In fact, most of the information transfer between the process and the computer exploits the logical
decision-making capability of the computer. Real-time applications of information processing and decision-
making, e.g., production planning, scheduling, optimization, operations control, etc., may now be integrated
with the traditional process control functions.
To enable the computer to meet the variety of demands imposed on it, its tasks are time-shared.
Future Generation Control Systems The study of emerging applications shows that artificial intelli-
gence (AI) will affect design and application of control systems, as profoundly as the impact of the micro-
processor has been in the last two decades. It is clear that future generation control systems will have a
significant AI component; the list of applications of computer-based control will continue to expand.
 22 Digital Control and State Variable Methods

2.1.2 Implementation Problems in Digital Control

The main problems associated with implementation of digital control, are related to the effects of sampling
and quantization.
Most processes that we are called upon to control, operate in continuous time. This implies, that we are
dealing largely with an analog environment. To this environment, we need to interface the digital computer,
through which we seek to influence the process.
The interface is accomplished by a system
Discrete-time signals of the form shown in Fig. 2.1. It is a cascade of
analog-to-digital (A/D) conversion system fol-
lowed by a computer which is, in turn, followed
A/D Computer D/A by a digital-to-analog (D/A) conversion system.
The A/D conversion process involves deriving
Continuous-time signals
samples of the analog signal at discrete instants
of time separated by sampling period T sec. The
Fig. 2.1 Discrete-time processing of continuous- D/A conversion process involves reconstructing
time signals continuous-time signals from the samples given
out by the digital computer.
Quantization Effects The conversion of signals from analog into digital form and vice versa is per-
formed by electronic devices (A/D and D/A converters) of finite resolution. A device of n-bit resolution
has 2n quantization levels. Here, the analog signal gets tied to these finite number of quantization levels in
the process of conversion to digital form. Therefore, by the sheer act of conversion, a valuable part of
information about the signal, is lost.
Furthermore, any computer employed as a real-time controller must perform all the necessary calcula-
tions with limited precision, thus introducing a truncation error after each arithmetic operation has been
performed. As the computational accuracy is normally much higher than the resolution of real converters, a
further truncation must take place before the computed data are converted into the analog form. The
repeated process of approximate conversion–computation–conversion may be costly, if not disastrous, in
terms of control system performance.
The process of quantization in signal conversion systems will be discussed shortly.
Sampling Effects The selection of a sampling period is a fundamental problem in digital control systems.
Later in this chapter, we will discuss the sampling theorem which states that the sampling period T should be
chosen such that
T < p /wm
where wm is the strict bandwidth of the signal being sampled. This condition ensures that there is no loss of
information due to sampling and the continuous-time signal can be completely recovered from its samples
using an ideal low-pass filter.
There are, however, two problems associated with the use of this theorem in practical control systems:
(i) Real signals are not band-limited and hence strict bandwidth limits are not defined.
(ii) The ideal low-pass filter, needed for the distortionless reconstruction of continuous-time signals from its
samples, is not physically realizable. Practical devices, such as the D/A converter, introduce distortions.
Thus, the process of sampling and reconstruction also affects the amount of information available to the
control computer, and degrades control system performance. For example, converting a given continuous-
time control system into a digital control system, without changing the system parameters, degrades the
system stability margin.
 Signal Processing in Digital Control 23
The ill-effects of sampling can be reduced, if not eliminated completely, by sampling at a very high rate.
However, excessively fast sampling (T ® 0) may result in numerical ill-conditioning in the implementation
of recursive control algorithms (described later in this chapter).
With the availability of low-cost, high-performance digital computers and interfacing hardware, the
implementation problems in digital control do not pose a serious threat to its usefulness. The advantages of
digital control outweigh its implementation problems for most of the applications.
This book attempts to provide a modest coverage of digital control theory and practice. In the present
chapter, we focus our attention on the digital computer and its interface to signal conversion systems
(Fig. 2.1). The goal is to formulate tools of analysis necessary to understand and guide the design of pro-
grams for a computer acting as a control logic component. Needless to say, digital computers can do many
things other than control dynamic systems; our purpose is to examine their characteristics when doing the
elementary control task.

2.2 CONFIGURATION OF THE BASIC DIGITAL CONTROL SCHEME

Figure 2.2 depicts a block diagram of a digital control system showing a configuration of the basic control
scheme. The basic elements of the system are shown by the blocks.
Controlled
Disturbance
output
Digital
set-point
Final
A/D Computer D/A control Plant
element

Clock

Anti-aliasing
S/H Sensor
filter

Fig. 2.2 Configuration of the basic digital control scheme

The analog feedback signal coming from the sensor is usually of low frequency. It may often include high
frequency ‘noise.’ Such noise signals are too fast for control system to correct; low-pass filtering is often
needed to allow good control performance. The anti-aliasing filter shown in Fig. 2.2 serves this purpose. In
digital systems, the phenomenon called aliasing introduces some new aspects to the noise problems. We will
study this phenomenon later in this chapter.
The analog signal, after anti-aliasing processing, is converted into digital form by the A/D conversion
system. The conversion system usually consists of an A/D converter preceded by a sample-and-hold (S/H)
device. The A/D converter converts a voltage (or current) amplitude at its input into a binary code represent-
ing a quantized amplitude value closest to the amplitude of the input. However, the conversion is not
instantaneous. Input signal variation, during the conversion time of the A/D converter, can lead to erroneous
results. For this reason, high performance A/D conversion systems include a S/H device, which keeps the
input to the A/D converter, constant during its conversion time.
 24 Digital Control and State Variable Methods
The digital computer processes the sequence of numbers by means of an algorithm and produces a new
sequence of numbers. Since data conversions and computations take time, there will always be a delay when
a control law is implemented using a digital computer. The delay, which is called the computational delay,
degrades the control system performance. It should be minimized by the proper choice of hardware and by
the proper design of software for the control algorithm. Floating-point operations take a considerably longer
time to perform (even when carried out by arithmetic co-processor) than the fixed-point ones. We therefore
try to execute fixed-point operations whenever possible. Alternative realization schemes for a control
algorithm will be given in the next chapter.
The D/A conversion system in Fig. 2.2 converts the sequence of numbers in numerical code into a
piecewise continuous-time signal. The output of the D/A converter is fed to the plant through the actuator
(final control element) to control its dynamics.
The basic control scheme of Fig. 2.2 assumes a uniform sampling operation, i.e., only one sampling rate
exists in the system and the sampling period is constant. The real-time clock in the computer, synchronizes
all the events of A/D conversion–computation–D/A conversion.
The control scheme of Fig. 2.2 shows a single feedback loop. In a control system having multiple loops,
the largest time constant involved in one loop may be quite different from that in other loops. Hence, it may
be advisable to sample slowly in a loop involving a large time constant, while in a loop involving only small
time constants, the sampling rate must be fast. Thus, a digital control system may have different sampling
periods in different feedback paths, i.e., it may have multiple-rate sampling. Although digital control
systems with multi-rate sampling are important in practical situations, we shall concentrate on single-rate
sampling. (The reader interested in multi-rate digital control systems may refer Kuo [87]).
The overall system in Fig. 2.2 is hybrid in nature; the signals are in a sampled form (discrete-time signals/
digital signals) in the computer and in continuous-time form in the plant. Such systems have traditionally
been called sampled-data control systems. We will use this term as a synonym to computer control systems/
digital control systems.
In the present chapter, we focus our attention on digital computer and its analog interfacing. For the time
being, we delink the digital computer from the plant. The link will be re-established in the next chapter.

2.3 PRINCIPLES OF SIGNAL CONVERSION

Figure 2.3a shows an analog signal y(t)—it is defined at the continuum of times, and its amplitudes assume
a continuous range of values. Such a signal cannot be stored in digital computers. The signal, therefore, must
be converted to a form that will be accepted by digital computers. One very common method to do this is
to record sample values of this signal at equally spaced instants. If we sample the signal every 10 msec,
for example, we obtain the discrete-time signal sketched in Fig. 2.3b. The sampling interval of 10 msec
corresponds to a sampling rate of 100 samples/sec. The choice of the sampling rate is an important one,
since it determines how accurately the discrete-time signal, can represent the original signal.
In a practical situation, the sampling rate is determined by the range of frequencies present in the original
signal. Detailed analysis of uniform sampling process, and the related problem of aliasing will appear later
in this chapter.
Notice that the time axis of the discrete-time signal in Fig. 2.3b, is labelled simply ‘sample number’ and
index k has been used to denote this number (k = 0, 1, 2, ...). Corresponding to different values of sample
number k, the discrete-time signal assumes the same continuous range of values assumed by the analog
signal y(t). We can represent the sample values by a sequence of numbers ys (refer Fig. 2.3b):
ys = {1.7, 2.4, 2.8, 1.4, 0.4, ...}
 Signal Processing in Digital Control 25
y (t) y ( t)

3 3

2 2

1 1

0 10 20 30 40 0 1 2 3 4
t (msec) Sample number k
y q (k ) (a) (b)

3
k Digital word
0 10
2
1 10
2 11
1
3 01
4 00
k
0 1 2 3 4
(c) (d)
Fig. 2.3 Sampling, quantization and coding of an analog signal

In general,
ys = {y(k)}, 0 ≤ k < ∞
where y(k) denotes the kth number in the sequence.
The sequence defined above is one-sided sequence; ys = 0 for k < 0. In digital control applications, we
normally encounter one-sided sequences.
Although, strictly speaking, y(k) denotes the kth number in the sequence, the notation given above is often
unnecessarily cumbersome, and it is convenient and unambiguous to refer to y(k) itself as a sequence.
Throughout our discussion on digital control, we will assume uniform sampling, i.e., sample values of the
analog signal are extracted at equally spaced sampling instants. If the physical time, corresponding to the
sampling interval is T seconds, then the kth sample y(k), gives the value of the discrete-time signal at t = kT
seconds. We may, therefore, use y(kT) to denote a sequence wherein the independent variable is the physical time.
The signal of Fig. 2.3b is defined at discrete instants of time. The sample values are, however, tied to a
continuous range of numbers. Such a signal, in principle, can be stored in an infinite-bit machine because a
finite-bit machine can store only a finite set of numbers.
A simplified hypothetical 2-bit machine can store four
numbers given alongside:
Binary number Decimal equivalent
The signal of Fig. 2.3b can be stored in such a machine if the
00 0 sample values are quantified to four quantization levels. Figure
01 1 2.3c shows a quantized discrete-time signal for our hypotheti-
10 2 cal machine. We have assumed that any value in the interval
[0.5, 1.5) is rounded to 1, and so forth. The signals for which
11 3
both time and amplitude are discrete, are called digital signals.
 26 Digital Control and State Variable Methods
After sampling and quantization, the final step required in converting an analog signal to a form accept-
able to digital computers is coding (or encoding). The encoder maps each quantized sample value into a
digital word. Figure 2.3d gives coded digital signal, corresponding to the analog signal of Fig. 2.3a for our
hypothetical 2-bit machine.
The device that performs the sampling, quantization, and coding is an A/D converter. Figure 2.4 is a
block diagram representation of the operations performed by an A/D converter.
Continuous-time
continuous-amplitude Digital
signal words

Sampler Quantizer Encoder

Discrete-time Discrete-time
continuous-amplitude discrete-amplitude
signal signal
Fig. 2.4 Operations performed by an A/D converter
It may be noted that the quantized discrete-time signal of Fig. 2.3c and the coded signal of Fig. 2.3d carry
exactly the same information. For the purpose of analytical study of digital systems, we will use the quantized
discrete-time form for digital signals.
The number of binary digits carried by a device is its word length, and this is obviously an important
characteristic related to the resolution of the device—the smallest change in the input signal that will pro-
duce a change in the output signal. The A/D converter that generates signals of Fig. 2.3 has two binary digits
and thus four quantization levels. Any change, therefore, in the input over the interval [0.5, 1.5) produces no
change in the output. With three binary digits, 23 quantization levels can be obtained, and the resolution of
the converter would improve.
The A/D converters in common use have word lengths of 8 to 16 bits. For an A/D converter with a word
length of 8 bits, an input signal can be resolved to one part in 28, or 1 in 256. If the input signal has a range
of 10 V, the resolution is 10/256, or approximately 0.04 V. Thus, the input signal must change by at least
0.04 V, in order to produce a change in the output.
With the availability of converters with resolution ranging from 8 to 16 bits, the quantization errors do not
pose a serious threat in the computer control of industrial processes. In our treatment of the subject, we
assume quantization errors to be zero. This is equivalent to assuming infinite-bit digital devices. Thus we
treat digital signals as if they are discrete-time signals with amplitudes assuming a continuous range of
values. In other words, we make no distinction between the words ‘discrete-time’ and ‘digital.’
A typical topology of a single-loop digital control system is shown in Fig. 2.2. It has been assumed that the
measuring transducer and the actuator (final control element) are analog devices, requiring respectively A/D
and D/A conversion at the computer input and output. The
Digital Discrete-time Analog
signal signal D/A conversion is a process of producing an analog signal
words
from a digital signal and is, in some sense, the reverse of
the sampling process discussed above.
Zero-order The D/A converter performs two functions: first,
Decoder
hold generation of output samples from the binary-form
Fig. 2.5 Operations performed by a D/A digital signals produced by the machine, and second, con-
converter version of these samples to analog form. Figure 2.5 is
 Signal Processing in Digital Control 27
a block diagram representation of the operations performed by a D/A converter. The decoder maps each
digital word into a sample value of the signal in discrete-time form. It is usually not possible to drive a
load, such as a motor, with these samples. In order to deliver sufficient energy, the sample amplitude
might have to be so large that it is infeasible to be generated. Also large-amplitude signals might saturate
the system being driven.
The solution to this problem is to smooth the output samples to produce a signal in analog form. The
simplest way of converting a sample sequence into a continuous-time signal is to hold the value of the
sample until the next one arrives. The net effect is to convert a sample to a pulse of duration T—the sample
period. This function of a D/A converter is referred to as a zero-order hold (ZOH) operation. The term zero-
order refers to the zero-order polynomial used to extrapolate between the sampling times (detailed discus-
sion will appear later in this chapter). Figure 2.6 shows a typical sample sequence produced by the decoder,
and the analog signal1 resulting from the zero-order hold operation.

y (k ) y h ( t)

3 3

2 2

1 1

k t
0 1 2 3 4 0 T 2T 3T 4T
(a) (b)
Fig. 2.6 (a) Sampled sequence (b) Analog output from ZOH

2.3.1 D/A Converter Circuits

Most D/A converters use the principle shown in the three-bit form in Fig. 2.7 to convert the HI/LO digital
signals at the computer output to a single analog voltage. The circuit of Fig. 2.7 is an ‘R–2R’ ladder; the
value of R typically ranges from 2.5 to 10K ohms.
Suppose a binary number b2 b1 b0 is given. The switch (actually, electronic gates) positions in Fig. 2.7
correspond to the digital word 100, i.e., b2 = 1 and b1 = b0 = 0. The circuit can be simplified to the equivalent
form shown in Fig. 2.8a. The currents in the resistor branches are easily calculated and are indicated in the
circuit (for the high gain amplifier, the voltage at point A is practically zero [155]). The output voltage is
i2 1
V0 = 3R = Vref
2 2
If b1 = 1 and b2 = b0 = 0, then the equivalent circuit is as shown in Fig. 2.8b. The output voltage is
i 1
V0 = 3R 1 = Vref
4 4

1
In the literature, including this book, the terms ‘continuous-time signal’ and ‘analog signal’ are frequently
interchanged.
 28 Digital Control and State Variable Methods

Full scale

7/8
(ratio to full scale)
6/8
Analog output

5/8
4/8

3/8

2/8
1/8

000 001 010 011 100 101 110 111

Digital code

3R

2R R R 2R
–
A
V0
2R 2R 2R +

b0 b1 b2

– V ref
Fig. 2.7 Three-bit D/A converter

Similarly, if b0 = 1 and b2 = b1 = 0, then the equivalent circuit is as shown in Fig. 2.8c. The output
voltage is
i 1
V0 = 3R 0 = Vref
8 8
In this way, we find that when the input data is b2b1b0 (where the bi’s are either 0 or 1), then the output
voltage is
V0 = (b22–1 + b12–2 + b02–3)VFS (2.1)
where VFS = Vref = full scale output voltage.
The circuit and the defining equation for an n-bit D/A converter easily follow from Fig. 2.7 and Eqn. (2.1),
respectively.
 Signal Processing in Digital Control 29
3R
1
i
2 2
2R 2R
–
1 1 A
i i
2 2 2 2 V0
i2
2R +

– V ref
(a)
3R
1
i
2R R 2R 4 1
–
1 1 1
i i i
2 1 2 1 4 1 V0
i1 +
2R 2R 1
i
4 1

– V ref
(b)
3R
1
i
2R R R 2R 8 0
–
1 1 1 1
i i i i
2 0 2 0 4 0 8 0 V0
2R i0 2R 2R +
1 1
i i
4 0 8 0

– Vref
(c)
Fig. 2.8 Equivalent circuits of the D/A converter shown in Fig. 2.7;
(a) b0 = b1 = 0, b2 = 1; (b) b0 = 0, b1 = 1, b2 = 0; (c) b0 = 1, b1 = b2 = 0.

2.3.2 A/D Converter Circuits

Most A/D converters use the principle of successive approximation. Figure 2.9 shows the organization of an
A/D converter that uses this method. Its principal components are a D/A converter, a comparator, a succes-
sive approximation register (SAR), a clock, and control and status logic.
On receiving the (start-of-conversion) SOC command, the SAR is cleared to 0’s and its most significant
bit is set to 1. This results in a V0 value that is one-half of the full scale (refer Eqn. (2.1)). The output of the
comparator is then tested to see whether VIN is greater than or less than V0. If VIN is greater, the most
significant bit is left on; otherwise it is turned off (complemented).
 30 Digital Control and State Variable Methods

Clock

Analog + Control and status SOC

input VIN logic
– EOC
Comparator
SAR

Digital
output

V0
D /A V FS

Fig. 2.9 Organization of a successive approximation A/D converter

111 In the next step, the next most significant bit of the
111
110 SAR is turned on. At this stage, V0 will become either
110 three-quarters or one-quarter of the full scale, depend-
101
101
ing on whether VIN was, respectively, greater than or
100 less than V0 in the first step. Again, the comparator is
100 tested and if VIN is greater than the new V0, the next
011
most significant bit is left on. Otherwise it is turned off.
011
010 The process is repeated for each remaining SAR bit.
010 When the process has been carried out for each bit, the
001 SAR contains the binary number that is proportional to
001
000 VIN, and the (end-of-conversion) EOC line indicates
Fig. 2.10 Code sequence for a three-bit that comparison has been completed and digital output
successive approximation is available for transmission. Figure 2.10 gives the code
sequence for a three bit successive approximation.
Typical conversion times of commercial A/D units range from 10 nsec to 200 msec. The input VIN
in Fig. 2.9 should remain constant during the conversion time of A/D converter. For this reason, a high
performance A/D conversion system includes a S/H device which keeps the input to the A/D converter,
constant during its conversion time. The S/H operation will be described in Section 2.10.

2.4 BASIC DISCRETE-TIME SIGNALS

There are a number of basic discrete-time signals which play an important role in the analysis of signals and
systems. These signals are direct counterparts of the basic continuous-time signals.2 As we shall see, many of
the characteristics of basic discrete-time signals are directly analogous to the properties of basic continuous-
time signals. There are, however, several important differences in discrete time, and we will point these out
as we examine the properties of these signals.
2
Chapter 2 of the companion book [155].
 Signal Processing in Digital Control 31
Unit Sample Sequence The unit sample sequence contains only one nonzero element and is defined by
(Fig. 2.11a)

d (k) =
RS1 for k = 0
(2.2a)
T0 otherwise
The delayed unit sample sequence, denoted by d (k – n), has its nonzero element at sample time n (Fig. 2.11b):

d (k – n) =
RS1 for k = n
(2.2b)
T0
otherwise
One of the important aspects of the unit sample sequence is that an arbitrary sequence can be represented
as a sum of scaled, delayed unit samples. For example, the sequence r (k) in Fig. 2.11c can be expressed as
r(k) = r(0)d (k) + r(1)d (k – 1) + r(2)d (k – 2) + L
¥
= å r(n)d (k – n) (2.3)
n = 0
r(0), r(1), K , are the sample values of the sequence r(k). This representation of a discrete-time signal is
found useful in the analysis of linear systems through the principle of superposition.

d (k ) d (k – n ) r (k )

1 1

k k k
0 1 2 3 0 1 2 n 0 1 2 3
(a) (b) (c)

m (k) m (k – n ) cos ( k) m (k)

1 1

k 3 5
k k
0 1 2 0 1 n 0 1
(d) (e)

(f)
Fig. 2.11 Basic discrete-time signals

As we will see, the unit sample sequence plays the same role for discrete-time signals and systems, that
the unit impulse function does for continuous-time signals and systems. For this reason, the unit sample
sequence is often referred to as the discrete-time impulse. It is important to note that a discrete-time impulse
does not suffer from the mathematical complexity that a continuous-time impulse suffers from. Its definition
is simple and precise.
 32 Digital Control and State Variable Methods
Unit Step Sequence The unit step sequence is defined as3 (Fig. 2.11d)

m(k) =
RS1 for k ³ 0
(2.4)
T0 otherwise
The delayed unit step sequence, denoted by m(k – n), has its first nonzero element at sample time n (Fig. 2.11e):

m(k – n) =
RS1 for k ³ n
(2.5)
T0 otherwise
An arbitrary discrete-time signal r(k) switched on to a system at k = 0 is represented as r(k)m(k).
Sinusoidal Sequence A one-sided sinusoidal sequence has the general form (Fig. 2.11f )
r(k) = A cos(W k + f) m(k) (2.6)
The quantity W is called the frequency of the discrete-time sinusoid and f is called the phase. Since k is a
dimensionless integer, the dimension of W must be radians (we may specify the units of W to be radians/
sample, and units of k to be samples).
The fact that k is always an integer in Eqn. (2.6) leads to some differences between the properties of
discrete-time and continuous-time sinusoidal signals. An important difference lies in the range of values the
frequency variable can take on. We know that for the continuous-time signal r(t) = A cos w t = real {Ae jw t},
w can take on values in the range (– ∞ , ∞ ). In contrast, for the discrete-time sinusoid r(k) = A cos Wk =
real {Ae jWk}, W can take on values in the range [– p, p].
To illustrate the property of discrete-time sinusoids, consider W = p + x, where x is a small number
compared with p. Since
e jWk = e j(p + x)k = e j(2p – p + x)k = e j(– p + x)k
a frequency of (p + x) results in a sinusoid of frequency (– p + x). Suppose now, that W is increased to 2p.
Since e j2pk = e j0, the observed frequency is 0. Thus, the observed frequency is always between – p and p, and
is obtained by adding (or subtracting) multiples of 2p to W until a number in that range is obtained.
The highest frequency that can be represented by a digital signal is therefore p radians/sample interval.
The implications of this property for sequences obtained by sampling sinusoids and other signals will be
discussed in Section 2.11.

2.5 TIME-DOMAIN MODELS FOR DISCRETE-TIME SYSTEMS

A discrete-time system is defined mathematically, as a transformation, or an operator, that maps an input

sequence r(k) into an output sequence y(k). Classes of discrete-time systems are defined by placing con-
straints on the transformation. As they are relatively easy to characterize mathematically, and as they can be
designed to perform useful signal processing functions, the class of linear time-invariant systems will be
studied in this book. In the control structure of Fig. 2.2, the digital computer transforms an input sequence
into a form which is in some sense more desirable. Therefore, the discrete-time systems we consider here
are, in fact, computer programs.
As we shall see, there is a similarity in the structure of models of continuous-time and discrete-time systems.
This has resulted in the development of similar methods of analysis. For example, the simulation diagrams of
discrete-time systems are similar to those for continuous-time systems, with only the dynamic element changed
3
In discrete-time system theory, the unit step sequence is generally denoted by u(k). In control theory, u(k) is used to
represent the control signal. In this book, m(k) has been used to represent the unit step sequence while u(k) denotes
the control signal.
 Signal Processing in Digital Control 33
from an integrator to a delayer. The convolution summation is similar to convolution integral, and the
z-transform method, tailored especially for linear discrete-time systems, bears many similarities to the Laplace
transform. There are differences also between the properties of discrete-time and continuous-time systems. In
this chapter, we are concerned with the analysis tools for discrete-time systems. Similarities with the tools for
continuous-time systems will be obvious. The differences will be pointed out specifically.
For linear time-invariant discrete-time systems, four different ways of mathematical representation will
be discussed. Time-domain models are described in this section, and a transform-domain model will be
given in Section 2.7.

2.5.1 State Variable Models

Consider a simple computer program expressed in MATLAB:
y(1) = 0
for i = 2, N (2.7)
y(i) = r(i – 1) + 0.1 * y(i – 1)
end
where r(i) is the ith sample of the input sequence, y(i) is the ith sample of the output sequence, and N is the
total length of the signal record. We must define the value y(1) in order to start signal processing. This
value is the initial condition of the signal processor. For the signal processor, represented by the computer
program (2.7), the initial condition has been taken as zero.
In the computer program (2.7), the initial condition is represented by y(1) and not by y(0) because
MATLAB does not allow arrays to be indexed starting with zero. For the analytical study of discrete-time
systems, starting a sequence y(k) with y(0) is more convenient.
Simulation Diagrams It is obvious that the computer program (2.7) is characterized by the three basic
operations:
(i) multiplication of a machine variable by a constant,
(ii) addition of several machine variables, and
(iii) storage of past values of machine variables.
These basic operations are diagrammatically
x(k ) a x (k ) x1 (k ) + x1(k) + x2(k) represented in Fig. 2.12. The unit delayer repre-
a sents a means for storing previous values of a
+ sequence. If the signal x1(k); k ≥ 0 is the input to
(a) Scalar the unit delayer, its output sequence x2(k) has the
x 2 (k)
sample values:
(b) Adder x2(0) = specified initial condition
x2(1) = x1(0)
x2(0) d (k)
x2(2) = x1(1)
+ M
x 1 (k) x 2 (k )
A specified initial condition is stored before the
+ commencement of the algorithm, in the appropri-
ate register (of the digital computer) containing
(c) Unit delayer x2(.). This can be diagrammatically represented by
adding a signal x2(0)d (k) to the output of the
Fig. 2.12 Basic building blocks of linear time-invariant delayer, where d (k) is the unit sample sequence
discrete-time systems defined by Eqn. (2.2a).
 34 Digital Control and State Variable Methods

The signal processing function performed by the

r (k ) + x (k + 1) x (k ) computer program (2.7) can be represented by a block
diagram shown in Fig. 2.13. Various blocks in this
+ figure represent the basic computing operations of a
digital computer. The unit delayer is the only dynamic
element involved. The signal processing configura-
+ 0.1 tion of Fig. 2.13, thus, represents a first-order
discrete-time system. The output x(k) of the dynamic
Fig. 2.13 A first-order linear discrete-time system
element gives the state of the system at any k. If the
signal r(k) is switched on to the system at k = 0 (r(k) = 0 for k < 0), the sample value x(0) of the output
sequence x(k), represents the initial state of the system. Since the initial state in the computer program (2.7)
is zero, a signal of the form x(0)d (k) does not appear in Fig. 2.13.
The defining equation for the computer program (2.7), obtained by forming an equation of the summing
junction in Fig. 2.13, is
x(k + 1) = 0.1 x(k) + r(k); x(0) = 0 (2.8)
The solution of this first-order linear difference equation for given input r(k) applied at k = 0, and given
initial state x(0), yields the state x(k); k > 0. Equation (2.8) is thus the state equation of the discrete-time
system of Fig. 2.13. Conversely, Fig. 2.13 is the simulation diagram for the mathematical model (2.8).
To solve an equation of the form (2.8) is an elementary matter. If k is incremented to take on values k = 0,
1, 2, ..., etc., the state x(k); k = 1, 2, ..., can easily be generated by an iterative procedure. The iterative
method, however, generates only a sequence of numbers and not a closed-form solution.

Example 2.1 In order to establish a feeling for the discrete-time systems, we study the signal processing
algorithm given by the difference equation
x(k + 1) = – a x(k) + r(k); x(0) = 0 (2.9)
where a is a real constant.
We shall obtain a closed-form solution of this equation by, what amounts to a brute force method
(z-transform method of solving linear difference equations will appear in Section 2.7). When solved repeti-
tively, Eqn. (2.9) yields
x(0) = 0; x(1) = r(0)
x(2) = – a r(0) + r(1); x(3) = (– a)2 r(0) – a r(1) + r(2)
M
The general term becomes (r(k) = 0 for k < 0),
x(k) = (– a)k – 1 r(0) + (– a)k – 2 r(1) + L + r(k – 1) (2.10)
Examining this equation, we note that the response x(k) is a linear combination of the input samples r(0),
r(1), ..., r(k – 1), and there appears to be a definite structure of the various weights.
The response of linear discrete-time systems to an impulse input d (k) (defined in Eqn. (2.2a)) will be of
special interest to us. Let us denote this response, called the impulse response, by g(k).
For the system described by Eqn. (2.9), the impulse response obtained from Eqn. (2.10) is given by

g(k) =
RS0 k -1
for k = 0
(2.11)
T(-a ) for k ³ 1
 Signal Processing in Digital Control 35
The question of whether or not the solution decays, is more closely related to the magnitude of a than to
its sign. In particular, for |a | > 1, g(k) grows with increasing k while it decays when |a | < 1. The nature of
time functions of the form (2.11) for different values of a will be examined in Section 2.9.

A discrete-time system is completely characterized by the output variables of independent dynamic

elements of the system. The outputs of independent dynamic elements, thus constitute a set of characterizing
variables of the system. The values of the characterizing variables at instant k describe the state of the
system at that instant. These variables are therefore the state variables of the system.
The discrete-time system shown in Fig. 2.14 has two dynamic elements; the outputs x1(k) and x2(k) of
these elements are therefore the state variables of the system. The following dynamical equations for the
state variables easily follow from Fig. 2.14:
x1(k + 1) = x2(k); x1(0) = x10 (2.12a)
x2(k + 1) = a1 x1(k) + a2 x2(k) + r(k); x2(0) = x 20
The solution of these equations for a given input r(k) applied at k = 0, and given initial state { x10 , x 20 },
yields the state {x1(k), x2(k)}, k > 0.

c2

x 2 (k )
+
r (k ) + + + x 1 (k) + y (k)
c1
+ + +
0 0
x 2 d (k ) x 1 d (k )

a2
+
+
a1

Fig. 2.14 A second-order discrete-time system

If y (k) shown in Fig. 2.14 is the desired output

p input q output
variables variables information, we have the following algebraic
relation to obtain y(k):
r1 y1 y(k) = c1 x1(k) + c2 x2(k) (2.12b)
r2 MIMO y2
discrete-time Equations (2.12a) are the state equations, and
Eqn. (2.12b) is the output equation of the
...

...

system
rp yq discrete-time system of Fig. 2.14.
In general, the state variable formulation may
...
be visualized in block diagram form as shown in
x1 x2 xn Fig. 2.15. We have depicted a multi-input, multi-
output (MIMO) system which has p inputs, q out-
n state variables puts, and n state variables; the different variables
Fig. 2.15 Structure of a general discrete-time system are represented by the input vector r(k), the out-
put vector y(k) and the state vector x(k), where
 36 Digital Control and State Variable Methods

LM r ( k ) OP 1 LM y (k ) OP LM x (k ) OP
1 1
r (k ) y ( k ) x (k)
r(k) = M
MM M PPP ; y(k) = MM M PP ; x (k) = MM M PP
Δ 2 Δ 2 Δ 2

MNr ( k )PQ p
MN y (k )PQ MN x (k )PQ
q n
Assuming that the input is switched on to the system at k = 0 (r(k) = 0 for k < 0), the initial state is given by
Δ
x(0) = x0, a specified n × 1 vector
The dimension of the state vector defines the order of the system. The dynamics of an nth-order linear
time-invariant system are described by equations of the form
x1(k + 1) = f11 x1(k) + f12 x2(k) + L + f1n xn(k) + g11 r1(k)
+ g12 r2(k) + L + g1p rp(k)
x2(k + 1) = f21 x1(k) + f22 x2(k) + L + f2n xn(k) + g21 r1(k)
+ g22 r2(k) + L + g2p rp(k)
M (2.13)
xn(k + 1) = fn1 x1(k) + fn2 x2(k) + L + fnn xn(k) + gn1 r1(k)
+ gn2 r2(k) + L + gnp rp(k)
where the coefficients fij and gij are constants.
In the vector-matrix form, Eqns (2.13) may be written as
Δ
x(k + 1) = Fx(k) + Gr(k); x(0) = x0 (2.14)
where
LM f 11 f 12 K f1nO LMg 11 g12 K g1 p OP
f f 22 K f PP g g22 K g2 p
F= M G= M P
21 2n 21
and
MM M M M P MM M M M P
Nf n1 f n2 K f Q
nn
P Ng n1 gn2 K g Q
np
P
are, respectively, n × n and n × p constant matrices. Equation (2.14) is called the state equation of the system.
The output variables at t = kT are linear combinations of the values of the state variables and input
variables at that time, i.e.,
y(k) = Cx(k) + Dr(k) (2.15)
where
LM
c11 c12 K c1n OP
d11 d12 K d1 p LM OP
c21 c22 K c2 n d 21 d 22 K d 2 p
C=
M
MM M M
and D = PP
M M M
MM PP
MMNc
q1 cq 2 K cqn
PPQ MMNd q1 d q 2 K d qp
PPQ
are, respectively, q × n and q × p constant matrices. Equation (2.15) is called the output equation of the system.
The state equation (2.14) and the output equation (2.15) together give the state variable model of the
MIMO system4:
Δ
x(k + 1) = Fx(k) + Gr(k); x(0) = x0 (2.16a)
y(k) = Cx(k) + Dr(k) (2.16b)

4
We have used lower case bold letters to represent vectors and upper case bold letters to represent matrices.
 Signal Processing in Digital Control 37
For single-input (p = 1) and single-output (q = 1) system, the state variable model takes the form
Δ
x(k + 1) = Fx(k) + gr(k); x(0) = x0 (2.17a)
y(k) = cx(k) + dr(k) (2.17b)
where g is n × 1 column vector, c is 1 × n row vector and d is a scalar:
g1 LM OP
g
g = M P ; c = [c
2
c2 L cn]
MM M PP 1

Ng Q n

Example 2.2 The discrete-time system of Fig. 2.16 has one dynamic element (unit delayer); it is there
fore, a first-order system. The state of the system at any k is described by x(k)—the output of the dynamic
element.
The equation of the input summing junction is
x(k + 1) = 0.95 x(k) + r(k); x(0) = 0 (2.18a)
This is the state equation of the first-order system.
The output y(k) is given by the following output equation:
y(k) = 0.0475 x(k) + 0.05 r(k) (2.18b)
Equations (2.18a) and (2.18b) together constitute the state variable model of the first-order system.

0.05

+
r (k) + x (k + 1) x (k) y (k)
0.0475
+
+

0.95

Fig. 2.16 A first-order linear discrete-time system

Let us study the response of the system of Fig. 2.16 to the unit step sequence

m(k) =
RS1 for k ³ 0
(2.19a)
T0 for k < 0
and the unit alternating sequence

r(k) =
RS(-1) k
for k ³ 0
(2.19b)
T0 for k < 0
We will first solve Eqn. (2.18a) for x(k) and then use Eqn. (2.18b) to obtain y(k).
The solution of Eqn. (2.18a) directly follows from Eqn. (2.10):
x(k) = (0.95)k – 1 r(0) + (0.95)k – 2 r(1) + L + r(k – 1)
k -1
= å (0.95)k – 1 – i r(i) (2.20)
i=0
 38 Digital Control and State Variable Methods

For the unit step input given by Eqns (2.19a), we have5

k -1 k -1 i
x(k) = å (0.95)k – 1 – i = (0.95)k – 1 å
FG 1 IJ
i=0 i=0
H 0.95K
LM1 - FG 1 IJ k OP
= (0.95)k – 1 MM H 0.951 K PP = 1 [1 – (0.95) ] k

MN 1 - 0.95 PQ 0.05
The output
0.0475
y1(k) = [1 – (0.95)k ] + 0.05; k ≥ 0
0.05
= 1 – (0.95)k + 1; k ≥ 0 (2.21)
Consider now the system excited by the unit alternating input given by Eqn. (2.19b). It follows from
Eqn. (2.20) that for this input, the state
k -1
1
x(k) = å (0.95)k – 1 – i (– 1) i = [(0.95)k – (– 1)k]
i=0 195
.

The output
y2(k) = 0.0475 x(k) + 0.05 (– 1)k
0.05
[(– 1)k + (0.95)k + 1]; k ≥ 0
= (2.22)
195
.
From Eqns (2.21) and (2.22), we observe that the steady-state values of y1(k) and y2(k) are
1
(– 1)k for large k
y1(k) = 1 for large k; y2(k) =
39
Thus, the discrete-time system of Fig. 2.16 readily transmits a unit step and rejects a unit alternating
input (reduces its magnitude by a factor of 39). Since the unit alternating signal is a rapidly fluctuating
sequence of numbers while the unit step can be viewed as a slowly fluctuating signal, the discrete-time
system of Fig. 2.16 represents a low-pass digital filter. In Example 2.11, we will study the frequency-domain
characteristics of this filter.

2.5.2 Difference Equation Models

Consider the single-input, single-output (SISO) system represented by the state model (2.17). The system
has two types of inputs; the external input r(k), and the initial state x(0) representing initial storage in the
appropriate registers (of the digital computer) containing xi(.).
If the dynamic evolution of the state x(k) is not required, i.e., we are interested only in the input-output
relation for k ≥ 0, a linear time-invariant discrete-time system composed of n dynamic elements can be

1 - ak + 1
k
5
å aj =
1- a
; a ¹1
j=0
 Signal Processing in Digital Control 39
analysed using a single nth-order difference equation as its model. General form of nth-order linear difference
equation relating output y(k) to input r(k) is given below:
y(k) + a1y(k – 1) + L + any(k – n) = b0r(k) + b1r(k – 1) + L + bmr(k – m)
The coefficients ai and bj are real constants; m and n are integers with m £ n.
We will consider the general linear difference equation in the following form:
y(k) + a1y(k – 1) + L + any(k – n)= b0r(k) + b1r(k – 1) + L + bnr(k – n) (2.23)
There is no loss of generality in this assumption; the results for m = n be used for the case of m < n by
setting appropriate bj coefficients to zero.
If the input is assumed to be switched on at k = 0 (r(k) = 0 for k < 0), then the difference equation model
(2.23) gives the output at instant ‘0’ in terms of the past values of the output; y(– 1), y(– 2), ..., y(– n), and the
present input r(0). Thus the initial conditions of the model (2.23) are {y(– 1), y(– 2), ..., y(– n)}.
Since the difference equation model (2.23) represents a time-invariant system, the choice of the initial
point on the time scale is simply a matter of convenience in analysis. Shifting the origin from k = 0 to k = n,
we get the equivalent difference equation model:
y(k + n) + a1 y(k + n – 1) + L + an y(k) = b0r(k + n) + b1r(k + n – 1) + L + b0r(k) (2.24)
Substituting k = 0 in Eqn. (2.24), we observe that the output at instant ‘n’ is expressed in tems of n values
of the past outputs: y(0), y(1), ..., y(n – 1), and in terms of inputs: r(0), r(1), ..., r(n). If k is incremented to
take on values k = 0, 1, 2, ..., etc., the y(k); k = n, n +1 , ..., can easily be generated by the iterative procedure.
Given { y(– 1), y(– 2), ..., y(– n)}, the initial conditions { y(0), y(1), ..., y(n – 1)}of the model (2.24) can be
determined by successively substituting k = – n, – n + 1, ..., – 2, – 1 in Eqn. (2.24).
In this book, we have not accommodated the classical methods of solution of linear difference equations
of the form (2.23) for given initial conditions and/or external inputs. Our approach is to transform the model
(2.23) to other forms which are more convenient for analysis and design of digital control systems. Our
emphasis is on the state variable models and transfer functions.
In Chapter 6, we will present methods of conversion of difference equation models of the form (2.23), to
state variable models. We will use state variable models to obtain the system response to given initial condi-
tions and external inputs, to construct digital computer simulation diagrams, and to design digital control
algorithms using modern methods of design.
Later in this chapter, we will present the z-transform technique for transforming difference equation
model (2.23) to transfer function form. We will use transfer function models to study input-output behaviour
of discrete-time systems; and to design digital control algorithms using classical methods of design.

2.5.3 Impulse Response Models

Consider the SISO system represented by the state model (2.17) or the difference equation model (2.23).
The system has two types of inputs: the external input r(k); k ≥ 0, and initial state x(0).
A system is said to be relaxed at k = 0 if the initial state x(0) = 0. In terms of the representation (2.23), a
system is relaxed if y(k) = 0 for k < 0.
We have earlier seen in Eqn. (2.3) that an arbitrary sequence r(k) can be represented as a sum of scaled,
delayed impulse sequences. It follows from this result, that a linear time-invariant, initially relaxed system
can be completely characterized by its impulse response. This can be easily established.
Let g(k) be the response of initially relaxed linear time-invariant discrete-time system to an impulse d(k).
Due to time-invariance property, the response to d (k – n) will be g(k – n). By linearity property, the
response to an input signal r (k) given by Eqn. (2.3) will be
 40 Digital Control and State Variable Methods
y(k) = r(0) g(k) + r(1) g(k – 1) + r(2) g(k – 2) + L
¥
= å r( j) g(k – j); k ≥ 0 (2.25)
j=0

As a consequence of Eqn. (2.25), a linear time-invariant system is completely characterized by its im-
pulse response g(k) in the sense that given g(k), it is possible to use Eqn. (2.25) to compute the output to any
input r(k). Equation (2.25) is commonly called the convolution sum.
It should be pointed out that the summation in Eqn. (2.25) is not really infinite in a practical situation,
since for causal systems, g(k – j) resulting from the input d (k – j) is zero for k < j. This is because a causal
system cannot respond until the input is applied. Thus, for a causal system with input r(k)m(k), Eqn. (2.25)
modifies to
k
y(k) = å r( j) g(k – j); k ≥ 0 (2.26)
j=0
Another important observation concerns the symmetry of the situation. If we let k – j = m in Eqn. (2.26),
we get
0
y(k) = å r(k – m) g(m)
m=k
Reversing the order of summation,
k
y(k) = å g(m) r(k – m) (2.27)
m=0
The symmetry shows that we may reverse the roles of r( . ) and g( . ) in the convolution formula.
We may remind the reader here, that whenever impulse response models are used to describe a system,
the system is always implicitly assumed to be linear, time-invariant, and initially relaxed.
We now transform r(k) and g(k) using the mapping
f (k) ® F(z)
where
¥
Δ
F(z) = å f ( k ) z – k ; z is a complex variable (2.28)
k=0

The application of this mapping to Eqn. (2.25) yields

Y(z) =
¥ ¥ L
å y ( k ) z – k = å M å r ( j) g ( k
¥ OP
– j) z – k
k=0 k =0 j=0MN PQ
Changing the order of summations, gives
¥ ¥
Y(z) = å r ( j)z – j å g(k – j ) z –( k – j )
j=0 k=0

Since g(k – j) = 0 for k < j, we can start the second summation at k = j. Then, defining the index m = k – j, we
can write
¥ ¥
Y(z) = å r ( j ) z – j å g (m) z – m
j=0 m=0

= R(z) G(z) (2.29)

 Signal Processing in Digital Control 41
where
¥ ¥
Δ Δ
R(z) = å r( k )z –k and G(z) = å g(k )z –k
k =0 k =0
We see that by applying the mapping (2.28), a convolution sum is transformed into an algebraic equation.
The mapping (2.28) is, in fact, the definition of z-transform.
The use of z-transform technique for the analysis of discrete-time systems runs parallel to that of Laplace
transform technique for continuous-time systems. The brief introduction to the theory of z-transform given
in this chapter provides the working tools adequate for the purposes of this text.

2.6 THE z-TRANSFORM

There are basically two ways to approach the z-transform. One way is to think in terms of systems that are
intrinsically discrete. Signal processing by a digital computer, as we have seen in the previous section, is an
example of such systems. In fact, intrinsically discrete-time systems arise in a number of ways. A model of
the growth of cancer is discrete, because the cancer cells divide at discrete points in time. A macroeconomic
model is usually discrete, because most economic data is usually reported monthly and quarterly. Represent-
ing the discrete instants of time by the integer variable k(k = 0, 1, 2, ...), we denote the output of a SISO
system by the sequence y(k); k ³ 0, and the input by the sequence r(k); k ³ 0.
The alternative approach to the z-transform is in terms of sampled-data systems. This is the approach we
will adopt because it best fits the problem we intend to solve, namely, the control of continuous-time systems
by a digital signal processor (refer Fig. 2.2). Sampling a continuous-time signal defines the discrete instants
of time. Interestingly, we will see that the z-transform (2.28) defined for analyzing systems that are intrinsi-
cally discrete, is equally useful for sampled-data systems.

2.6.1 The Ideal Sampler

Consider an analog signal xa(t); t ³ 0. By substituting t = kT; k = 0, 1, 2, ..., a sequence xa(kT ) is said to be
derived from xa(t) by periodic sampling, and T is called the fixed sampling period. The reciprocal of T is
called the sampling frequency or sampling rate. In a typical digital control scheme (refer Fig. 2.2), the
operation of deriving a sequence from a continuous-time signal is
T
x a (t) x(k) = xa(kT ) performed by an analog-to-digital (A/D) converter. A simple
ideal sampler representation of the sampling operation is shown
in Fig. 2.17.
Fig. 2.17 Ideal-sampler representation
The ideal sampler consists of a switch that closes and reopens
of the sampling operation
instantaneously. For our purposes, time will be in seconds,
although this is not always the case. In chemical processes, for instance, the unit of time could very well
be minutes.
Of course, no switch opens and closes instantaneously, but the modern A/D converter comes very close.
A/D converters with a conversion rate of 100 kHz are fairly inexpensive, and conversion rates in the mega-
hertz range are available. In control applications, the sampling rate is usually less than 100 Hz. If the conver-
sion rate is 100 kHz, the conversion is completed in less than one one-thousandth of the typical sample
period. Thus the conversion rate of A/D converter is close enough to instantaneous.
For intrinsically discrete-time systems, the sequence x(k) represents the values the variable x takes at
discrete instants k = 0, 1, 2, ... . When the sampling operation is involved, the sequence xa(kT ) represents the
values of continuous-time signal xa(t) derived at t = kT; k = 0, 1, 2, ...; and T is a fixed sampling period. Since
T remains fixed, there is no loss in information if variable x(k) is used to represent xa(kT); the advantage is in
terms of notational convenience. We will follow this notation.
 42 Digital Control and State Variable Methods
To establish a relationship of the sequence x(k) to the continuous-time function xa(t) from which this
sequence is derived, we take the following approach. We treat each sample of the sequence x(k) as an
impulse function of strength equal to the value of the sample (Impulse function Ad (t – t0) is an impulse of
strength A occurring at t = t0). The idea is to give a mathematical description to periodic samples of a
continuous-time function in such a way, that we can analyze the samples and the function simultaneously,
using the same tool (Laplace transform). The sequence x(k) can be viewed as a train of impulses represented
by the continuous-time function x*(t) :
x*(t) = x(0)d (t) + x(1)d (t – T ) + x(2)d (t – 2T ) + L
¥
= å x( k )d (t – kT ) (2.30)
k =0
Typical signals xa(t), x(k) and x*(t) are shown in Fig. 2.18. The sampler of Fig. 2.17 can thus be viewed
as an ‘impulse modulator’ with the carrier signal
¥
d T (t) = å d (t – kT ) (2.31)
k =0
and modulating signal xa(t). The modulation process is schematically represented in Fig. 2.19a, and the
impulse train d T (t) in Fig. 2.19b:
x*(t) = xa(t) dT (t) (2.32a)

Fig. 2.18 Conversion of a continuous-time signal to a sequence

d T (t)

x*(t) =
T
xa (t) x*(t) x a (t) x a ( t) dT ( t )
Impulse
Modulator
(a)
dT

0 T 2T 3T t

(b)
Fig. 2.19 (a) Representation of sampling as impulse modulator (b) Representation of impulse train
 Signal Processing in Digital Control 43
We will eliminate the impulse function by simply taking the Laplace transform of x*(t) to obtain (refer
Chapter 2 of the companion book [155] for definitions and properties of impulse function, and Laplace
transform)
¥
Δ
L [x*(t)] = X*(s) = z
0
xa (t )d T (t )e – st dt

¥ ¥
= å
k = 00
z xa (t )d (t – kT )e – st dt
¥
= å xa ( kT )e – skT (2.32b)
k =0
This expression for X *(s) represents a Laplace transform, but it is not a transform that is easy to use
because the complex variable s occurs in the exponent of the transcendental function e. By contrast, the
Laplace transforms that we have used previously in the companion book [155], have mostly been ratios of
polynomials in the Laplace variable s, with real coefficients. These latter transforms are easy to manipulate
and interpret.
Ultimately, we will be able to achieve these same ratios of polynomials in a new complex variable z by
transforming X *(s) to reach what we will call the z-plane.
We remind the reader here that X*(s) is the notation used for Laplace transform of impulse modulated
signal x*(t); the ‘star’ distinguishes it from X(s)—the conventional Laplace transform of the unsampled
continuous function xa(t). We have used the same complex plane (s-plane) for Laplace transform of “starred”
functions and conventional functions. This is the most compact approach used almost universally.

2.6.2 Definition of the z-Transform

The expression X*(s), given by (2.32b), contains the term e–Ts; T is the fixed sampling period. To transform
the irrational function X*(s) into a rational funciton, we use a transformation from the complex variable s to
another complex variable, say z. An obvious choice for this transformation is
z = eTs (2.33a)
–Ts
although z = e would be just as acceptable.
Solving for s in Eqn. (2.33a), we obtain
1
s = ln z (2.33b)
T
The relationship between s and z in Eqns (2.33) may be defined as the z-transformation. In these two
equations, z is a complex variable; its relation with real and imaginary parts of complex variable s is given by
(with s = s + jw ):
Im z = eT(s + jw ) = eTs e jwT = re jW (2.34)
For a fixed value of r, the locus of z is a circle in the complex z-plane.
z-plane Circle of radius unity in the complex z-plane will be of specific interest
1 to us. This circle is called a unit circle (Fig. 2.20).
When Eqns (2.33) are substituted in Eqn. (2.32b), we have
Re ¥
F
X* s =
1
ln z
I = X ( z) = å xa ( kT ) z – k (2.35)
H T K k =0

Thus the z-transformation given by Eqns (2.33) is same as defined

Fig. 2.20 Unit circle in z-plane earlier in Eqn. (2.28) for intrinsically discrete-time systems.
 44 Digital Control and State Variable Methods
Since T remains fixed, there is no loss of information if variable x (k) is used to represent xa (kT ). The
expression
¥
X (z) = å x (k)z–k
k =0
is often used as the definition of the z-transform of sequence x (k) (intrinsically discrete-time sequence or
Δ x (kT)), denoted symbolically as Z [x (k)]:
derived from continuous-time signal xa (t); x (k) = a
¥
Δ Z [x (k)] =
X (z) = å x (k)z–k (2.36)
k =0
The summation in Eqn. (2.36) does not converge for all functions; and when it does, it does so for
restricted values of z in the z-plane. Therefore, for each sequence for which the summation converges, there
is an associated convergence region in the complex z-plane (Examples of z-plane convergence regions will
shortly follow).
We now have a transform X(z) defined in terms of a complex variable z. However, our expression for X(z)
is an infinite sum in the complex variable z. What we would like to achieve is something of the form:
b0 z m + b1z m – 1 + L + bm
X(z) = ; m• £ n (2.37)
z n + a1z n – 1 + L + an
where all the ai and bj are real.
Under these conditions, we can write
m
K ∏ (z – β i )
i =1
X(z) = n (2.38)
∏ (z – α j )
j =1
where the b i and a j are either real or complex-conjugate pairs.
The z-transform zeros are the values of z for which the transform is zero. z = – b i are the zeros of X(z) in
Eqn. (2.38).
The z-transform poles are the values of z for which the transform is infinite. z = – a j are the poles of X(z)
in Eqn. (2.38).
Actually, we are not very far from our goal. We will now show that we can find closed-form expressions
for the z-transforms of all the functions that we will need to study the control of sampled-data systems.

2.6.3 z-Transforms of Functions Useful for Control System Analysis

In this sub-section, our goal is to find the z-transform of the functions we will subsequently need for analysis
of control systems.
Unit Sample Sequence The unit sample sequence contains only one non-zero element and is defined by

d (k) =
RS1 for k = 0
T0 otherwise
The z-transform of the elementary signal is
¥
Z [d (k)] = å d (k )z –k = z 0 = 1; | z| > 0 (2.39)
k =0
 Signal Processing in Digital Control 45
Unit Step Sequence The unit step sequence is defined as

m (k) =
RS1 for k ³ 0
T0 otherwise
The z-transform is
¥ ¥
Z [m (k)] = å m( k ) z – k = å z – k
k =0 k =0
Using the geometric series formula
¥
1
å xk =
1– x
; |x| < 1,
k =0
this becomes
1 z
Z [m (k)] = –1
= (2.40)
1– z z–1
Note that this equation holds only if the infinite sum converges, that is if |z–1| < 1 or |z| > 1. Thus, the
region of convergence is the area outside the unit circle in the z-plane.
Sampled Ramp Function Discrete form of y(t) = tm (t) is of the form

y(k) =
RSkT for k ³ 0

From the basic definition (2.36),

T0 otherwise

∞ ∞
dX ( z) d
dz
= ∑ dz x(k )z – k = ∑ (– k ) x(k )z – k –1
k =0 k =0
Consequently,
¥
dX ( z)
–z
dz
= å k x( k ) z – k
k =0
or
dX ( z)
Z [kT x(k)] = – Tz (2.41)
dz
For x(k) = m (k), we obtain

Z [kT m (k)] = Z [y(k)] = Y(z) = – Tz

d Fz I
H
dz z – 1 K
= –Tz
LM – z 2
+
1 OP
N (z – 1) z–1 Q
Tz
= ; |z| > 1 (2.42)
( z – 1) 2
Sampled Exponential Function Discrete form of x(t) = e–at is of the form

x(k) =
RSe – akT
for k ³ 0
T0 otherwise
 46 Digital Control and State Variable Methods
Then
¥ ¥
– akT – k
X(z) = å e z = å (e aT z) – k
k =0 k =0
¥
= å a – k ; a = e aT z
k =0
1 e aT z z
= –1 = aT– aT =
1–a e z–1 z – e
This equation holds only if the infinite sum converges, that is, if |e–aT z–1 |< 1 or |z| > |e–aT|. The result holds
for both real and complex a.
Sampled Sinusoids Discrete form of x(t) = A cos (wt + f) is of the form

x(k) =
RS
A cos(w kT + f ) for k ³ 0
0 T otherwise
The transform of sampled sinusoids can be found by expanding the sequence into complex exponential
components.

x(k) = A
FG e j (wkT + f )
+ e – j (wkT + f ) IJ
H 2 K
Then
¥ ¥
A A
X(z) =
2
å e jf e jwkT z – k + 2
å e – jf e jwkT z – k
k =0 k =0
jf – jf
A ze A ze
= +
2 z – e – jwT 2 z – e jwT

Az
LM z(e
+ e – jf ) e j (wT + f ) + e – j (wT + f )
–
jf
OP
=
2 N 2 Q
z2 – 2
e jwT
+ e – jwT
z+1
FG IJ
2 H K
Az z cos f – cos(wT + f )
= ; |z| > 1
z 2 – 2 z cos wT + 1
Given the z-transform of A cos(wkT + f ), we can obtain z-transform of Ae–akT cos(wkT + f) as follows.
¥
Z [e–akT x(k)] = å x ( k )e – akT z – k
k =0
∞
= ∑ x(k )(ze aT ) – k = X ( ze aT ) (2.43)
k =0
Noting that
Az[ z cos f – cos(wT + f )]
Z [A cos (wkT + f )] =
z 2 – 2 z cos wT + 1
we substitute zeaT for z to obtain the z-transform of Ae–akT cos(wkT + f ), as follows:
Aze aT [ ze aT cos f – cos(wT + f )]
Z [Ae–akT cos (wkT + f )] = ; |z| > e–aT (2.44)
z 2 e 2 aT – 2 ze aT cos wT + 1
 Signal Processing in Digital Control 47

Example 2.3 Let us find the z-transform of

1
X(s) =
s( s + 1)
Whenever a function in s is given, one approach for finding the corresponding z-transform is to convert X(s)
into x(t) and then find the z-transform of x(kT ) = x(k); T is the sampling interval.
The inverse Laplace transform of X(s) is
x(t) = 1–e–t; t ³ 0
Hence
Δ
x(kT) = x(k) = 1 –e–kT; k ³ 0
z z (1 – e – T ) z
X(z) = Z [x(k)] = – = ; |z| > 1
z – 1 z – e–T ( z – 1)( z – e – T )
We summarize the z-transforms we have derived up to this point, plus some additional transforms in
Table 2.1. The table lists commonly encountered functions x(t); t ³ 0 and z-transforms of sampled version of
these functions, given by x(kT ). We also include in this table, the Laplace transforms X(s) corresponding to
the selected x(t). We have seen in Example 2.3, that whenever a function in s is given, one approach for
finding the corresponding z-transform is to convert X(s) into x(t), and then find its z-transform. Another
approach is to expand X(s) into partial functions and use z-transform table to find the z-tranforms of the
expanded terms. Table 2.1 will be helpful for this second approach.
All the transforms listed in the table, can easily be derived from first principles. It may be noted that in
this transform table, regions of convergence have not been specified. In our applications of systems analysis,
which involve transformation from time-domain to z-domain and inverse transformation, the variable z acts
as a dummy operator. If transform pairs for sequences of interest to us are available, we are not concerned
with the region of convergence.

Table 2.1 Abbreviated table of z-transforms

X(s) x(t); t ³ 0 x(k); k = 0, 1, 2, K X(z)

– – d (k) 1
1 z
m (t) m (k)
s z –1
1 z
e– at e– akT
s+a z – e– aT
1 Tz
t kT
s2 ( z – 1) 2
1 Te – aT z
t e–at kTe–akT
( s + a )2 ( z – e – aT )2
a (1 – e– aT ) z
1 – e–at 1 – e–akT
s( s + a ) ( z – 1)( z – e – aT )

Contd.
 48 Digital Control and State Variable Methods
Table 2.1 Contd.

X(s) x(t); t ³ 0 x(k); k = 0, 1, 2, K. X(z)

w (sin wT ) z
sin wt sin wkT
s2 + w 2 z 2 – ( 2 cos wT ) z + 1
s z 2 – (cos wT ) z
cos wt cos wkT
s2 + w 2 z 2 – ( 2 cos wT ) z + 1

w (e – aT sin wT ) z
e–at sin wt e–akT sin w kT
( s + a )2 + w 2 z 2 – (2e – aT cos wT ) z + e –2 aT

s+ a z 2 – (e – aT cos wT ) z
e–at cos wt e–akT cos w kT
( s + a )2 + w 2 z – (2e– aT cos wT ) z + e –2 aT
2

2.6.4 Useful Operations with z-Transforms

Shifting Theorem: Advance (forward shift) z-transformation of difference equations written in terms
of advanced versions of the input and output variables (refer Eqn. (2.24)) requires the following results.
¥ ¥
Z [ y(k + 1)] = å y ( k + 1) z – k = z å y ( k + 1) z –( k + 1)
k =0 k =0
Letting k + 1 = m, yields
¥
Z [ y(k + 1)] = z å y ( m) z – m = z
LM å y(m) z
¥
–m
– y ( 0)
OP
k =0 Nk =0 Q
= zY(z) – zy(0) (2.45a)
Use of this result in a recursive manner leads to the following general result:
Z [y(k + n)] = zn Y(z) – zn y(0) –zn–1 y(1) – L – z2 y(n –2) – zy(n – 1) (2.45b)

Shifting Theorem: Delay (backward shift) z-transformation of difference equations written in terms of
delayed versions of input and output variables (refer Eqn. (2.23)) requires the following results.
¥
Z [ y(k – 1)] = å (k – 1) z – k
k =0
= y(– 1)z0 + y(0)z–1 + y(1)z–2 + L

= z–1
LM å y( k ) z OP + y (–1)
¥
–k

N
k =0 Q
= z–1 Y(z) + y(–1) (2.46a)
Use of this result in a recursive manner leads to the following general result:
Z [y(k – n)] = z–n Y(z) + z–(n–1) y(– 1) + z–(n–2) y(– 2) + L
+ z–1 y(– n + 1) + y(– n) (2.46b)
If y(k) = 0 for k < 0, we have
Z [y(k – n)] = z–n Y(z) (2.47)
 Signal Processing in Digital Control 49

Example 2.4 Let us find the z-transforms of unit-step functions that are delayed by 1 sampling period,
and n sampling periods respectively.
Using the shifting theorem given by Eqn. (2.47), we have
–1
Z [m(k – 1)] = z–1 Z [m (k)] = z–1
FG 1 IJ = z
H1 – z–1 K 1– z –1

Also
–n
Z [m(k – n)] = z–n Z [m (k)] = z–n
FG 1 IJ = z
H1 – z –1 K 1– z –1

Remember that multiplication of the z-transform X(z) by z has the effect of advancing the signal x(k) by
one sampling period, and that multiplication of the z-transform X(z) by z–1 has the effect of delaying the
signal x(k) by one sampling period. In control engineering and signal processing, X(z) is frequently
expressed as a ratio of polynomials in z–1 as follows (refer Table 2.1):
z 2 – (e – aT cos wT ) z
Z [e–akT cos w kT ] = (2.48a)
z – (2e – aT cos wT ) z + e –2aT
2

1 – (e – aT cos wT ) z –1
= (2.48b)
1 – (2e – aT cos wT ) z –1 + e –2aT z –2
z–1 is interpreted as the unit delay operator.
In finding the poles and zeros of X(z), it is convenient to express X(z) as a ratio of polynomials z, as is
done in Eqn. (2.48a). In this and the next chapter, X(z) will be expressed in terms of powers of z, as given by
Eqn. (2.48a), or in terms of powers of z–1, as given by Eqn. (2.48b), depending on the circumstances.

Example 2.5 Analogous to the operation of integration, we can define the summation operation
k
x(k) = å y(i ) (2.49)
i =0
In the course of deriving an expression for X(z) in terms of Y(z), we shall need the infinite series sum:
¥
1 z
å (az –1) k =
1 – az –1
=
z –a
(2.50)
k =0
–1
which converges, provided that |az | < 1, or |z| > a.
Successive values of x(k) are:
x(0) = y(0)
x(1) = y(0) + y(1)
M
x(k) = y(0) + y(1) + L + y(k)
Thus, X(z) is the infinite sum given below.
¥
X(z) = å x(k ) z –k = x(0) + z–1 x(1) + L + z–k x(k) + L
k =0

= y(0) + z–1 [y(0) + y(1)] + L + z–k [y(0) + L + y(k)] + L

 50 Digital Control and State Variable Methods

= y(0)[1 + z–1 + z–2 + L] + y(1)[z–1 + z–2 + L] + L

+ y(k)[z–k + z–k–1 + L] + L

=
F z I [y(0) + z y(1) + z –1 –2
y(2) + L]
H z – 1K
=
F z I LM å y(k )z OP
¥
–k
H z – 1K N Q
k =0
Therefore,
z
X(z) = Y(z) (2.51)
z–1

2.6.5 z-Transform Inversion

We will obtain the inverse z-transform in exactly the same way that we obtained the inverse Laplace trans-
form (Chapter 2 [155]), namely, by partial fraction expansion. The reason that the partial fraction expan-
sion method works is that we frequently encounter transforms that are rational functions, i.e., ratio of two
polynomials in z with real coefficients (refer Eqn. (2.37)). The fact that the coefficients are real is crucial,
because it guarantees that the roots of the numerator and denominator polynomials will be either real, or
complex-conjugate pairs. This, in turn, means that the individual terms in the partial fraction expansion of
the transform will be simple in form and we will be able to do the inverse transformation by inspection.
The transform pairs encountered using the partial fraction expansion technique will usually be those
found in Table 2.1. Those not found in the table can easily be derived by using basic properties of
z-transformation. The partial fraction expansion method for z-transforms is very straightforward, and
similar, in most respects, to the partial fraction expansion in Laplace transforms. We first illustrate the
method with some examples from which we can then abstract some general guidelines.

Example 2.6 We observe that the transforms of the elementary functions (see Table 2.1) contain a factor
of z in the numerator, e.g.,
z
Z [m(k)] =
z–1
where m (k) is a unit-step sequence.
To ensure that the partial fraction expansion will yield terms corresponding to those tabulated, it is cus-
tomary to first expand the function Y(z)/z, if Y(z) has one or more roots at the origin, and then multiply the
resulting expansion by z.
For instance, if Y(z) is given:
2 z 2 – 15. z z ( 2 z – 15 .)
Y(z) = 2
= ,
z – 15 . z + 0.5 ( z – 0.5)( z – 1)
we are justified in writing
Y (z) 2 z – 15
. A1 A2
= = +
z ( z – 0.5)( z – 1) z – 0.5 z – 1
Constants A1 and A2 can be evaluated by applying the conventional partial fraction expansion rules.

A1 = (z – 0.5)
LMY (z) OP = 1; A2 = (z – 1)
LMY (z) OP =1
NzQ z = 0.5 NzQ z =1
 Signal Processing in Digital Control 51
Y ( z) 1 1
= +
z z – 0.5 z – 1
or
z z
Y(z) = +
z – 0.5 z – 1
Using the transform pairs from Table 2.1,
y(k) = (0.5)kT + 1; k ³ 0

Example 2.7 When Y(z) does not have one or more zeros at the origin, we expand Y(z), instead of Y(z)|/z,
into partial fractions and utilize shifting theorem given by Eqn. (2.47) to obtain inverse z-transform.
In applying the shifting theorem, notice that if
Z –1[Y(z)] = y(k) then Z –1[z–1Y(z)] = y(k – 1)
Let us consider an example:
10
Y(z) =
( z – 1)( z – 0.2)
We first expand Y(z) into partial fractions:
12.5 12.5
Y(z) =–
z – 1 z – 0.2
Notice that the inverse z-transform of 1/(z–1) is not available in Table 2.1. However, using the shifting
theorem, we find that

Z –1
LM
1 OP
= Z –1 z
–1 zLM F = I OP RS
1; k = 1, 2, K
z–1N Q z –1 N H KQ T
0; k ≤ 0
Also,

Z –1
LM 1 OP
= Z –1 z
–1 LM
z F =
I OP RS
(0.2) k – 1 ; k = 1, 2, K
N
z – 0.2 Q N
z – 0.2 H 0 KQ T ;k £ 0
Therefore,
y(k) = Z –1[Y(z)] =
RS
12.5 1 – ( 0.2 ) k –1 ; k = 1, 2, K
0 T ;k ≤ 0
which can be written as
y(k) =12.5[1 – (0.2)k–1]m (k–1)

2.6.6 Final Value Theorem

The final value theorem is concerned with the evaluation of y(k) as k ® ¥ assuming, of course, that y(k) does
approach a limit. Using partial fraction expansion for inverting z-transforms, it is a simple matter to show
that y(k) approaches a limit as k ® ¥, if all the poles of Y(z) lie inside the unit circle ( |z| <1) in the complex
z-plane. The unit circle boundary is, however, excluded except for a single pole at z = 1. This is because
purely sinusoidal signals whose transforms will have poles on the unit circle, do not settle to a constant value
as k ® ¥. Multiple poles at z = 1 are also excluded because, as we have already seen in the table of
z-transform, these correspond to unbounded signals like ramps. A more compact way of phrasing these
conditions is to say that (z – 1)Y(z) must be analytic on the boundary, and outside the unit circle in the
complex z-plane. The final value theorem states that when this condition on (z – 1)Y(z) is satisfied, then
 52 Digital Control and State Variable Methods

lim y(k) = lim (z – 1)Y(z) (2.52)

k ®¥ z®1
The proof is as follows:
m
Z [y(k + 1) – y(k)] = lim å [ y( k + 1) – y ( k )]z – k
m® ¥ k =0

or
m
zY(z) – z y(0) – Y(z) = lim å [ y( k + 1) – y ( k )]z – k
m® ¥
k =0
Letting z ® 1 on both sides,
m
lim [(z – 1)Y(z)] = y(0) + lim lim å [ y(k + 1) – y ( k )]z – k
z®1 z®1 m® ¥
k =0
Interchanging the order of limits on the right-hand side, we have
m
lim [(z – 1)Y(z)] = y(0) + lim
z®1
å [ y ( k + 1) –
m® ¥ k
y ( k )] = y (¥)
=0

Example 2.8 Given

z z
X(z) =
– ;a > 0
z – 1 z – e – aT
By applying the final value theorem to the given X(z), we obtain
LM
lim x(k) = lim ( z – 1)
FG z – z IJ OP = lim LM1 – z – 1 OPz = 1
k ®¥ z ®1N Hz – 1 z – e – aT KQ N z – e Q
z ®1 – aT

It is noted that the given X(z) is actually the z-transform of

x(k) = 1 – e–akT
By substituting k = ¥ in this equation, we have

a
x(¥) = lim 1 – e – akT = 1
k ®¥
f
2.7 TRANSFER FUNCTION MODELS

The very first step in the analytical study of a system is to set up mathematical equations to describe the
system. Because of different analytical methods used, or because of different questions asked, we may often
set up different mathematical equations to describe the same system.
We have seen earlier in Section 2.5 (Eqn. (2.26)) that the input r(k) and output y(k) of any linear time-
invariant discrete-time system that is initially relaxed at k = 0, can be described by an equation of the form
k
y(k) = å r ( j) g(k – j ); k ³ 0 (2.53)
j=0

This is called the convolution sum. The function g(k) is defined for k ³ 0 and is called the impulse
response of the system.
 Signal Processing in Digital Control 53
The application of z-transform to Eqn. (2.53) gives us an extremely useful mathematical description, of a
linear time-invariant discrete-time system (refer Eqn. (2.29)).
Y(z) = Z [(y(k)] = G(z)R(z) (2.54)
where
Δ Δ
R(z) = Z [(r(k)] and G(z) = Z [(g(k)]
We see that by applying the z-transform, a convolution sum is transformed into an algebraic equation.
The function G(z) is called the transfer function of the discrete-time system.
The transfer function of a linear time-invariant discrete-time system is, by definition, the z-transform of
the impulse response of the system.
An alternative definition of transfer function follows from Eqn. (2.54).

Z [ y ( k )] Y (z)
G(z) = = System (2.55)
System
Z [r ( k )] initially relaxed R( z ) initially relaxed

Thus, the transfer function of a linear time-invariant discrete-

time system is the ratio of the z-transforms of its output and
input sequences, assuming that the system is initially relaxed.
R (z ) Y (z )
G (z ) Figure 2.21 gives the block diagram of a discrete-time
system in transform domain.
Let us use the definition given by Eqn. (2.55) to obtain trans-
Fig. 2.21 Block diagram of a discrete- fer function model of a discrete-time system, represented by a
time system difference equation of the form (2.23), relating its output y(k)
to the input r(k). We assume that the discrete-time system is
initially relaxed:
y(k) = 0 for k < 0
and is excited by an input
r(k); k ≥ 0
Taking z-transform of all the terms of Eqn. (2.23), under the assumption of zero initial conditions, we
obtain
Y(z) + a1 z–1 Y(z) + L + an z– n Y(z)= b0 R(z) + b1 z–1 R(z) + L + bn z– n R(z)
where
Δ Δ
Y(z) = Z [y(k)] and R(z) = Z [r(k)]
Solving for Y(z):

(b0 + b1 z -1 + L + bn z - n ) R( z)
Y(z) =
1 + a1 z -1 + L + a n z - n
Therefore, the transfer function G(z) of the discrete-time system represented by difference equation (2.23) is
-1 -
Y ( z ) b0 + b1 z + L + bn z n
G(z) = = (2.56)
R( z ) 1 + a1 z -1 + L + a n z - n
 54 Digital Control and State Variable Methods
The same result can be obtained by taking the z-transformation of the shifted difference equation (2.24).
We first consider the case with n = 2, and then present the general result.
y(k + 2) + a1y(k + 1) + a2y(k) = b0r(k + 2) + b1r(k + 1) + b2r(k) (2.57)
The z-transform of Eqn (2.57) gives (using the shifting theorem given by Eqns (2.45)):
[z2Y(z) – z2y(0) – zy(1)] + a1[zY(z) – zy(0)] + a2Y(z)
= b0[z2R(z) – z2r(0) – zr(1)] + b1[zR(z) – zr(0)] + b2R(z)
or
(z2 + a1z + a2) Y(z) = (b0z2 + b1z + b2) R(z) + z2 [y(0) – b0r(0)]
+ z[y(1) + a1y(0) – b0r(1) – b1r(0)] (2.58)
Since the system is initially at rest, and switched on at k = 0, we have y(k) = 0 for k < 0, and r(k) = 0 for
k < 0. To determine the initial conditions y(0) and y(1), we substitute k = – 2 and k = – 1, respectively, into
Eqn. (2.57).
y(0) + a1y(–1) + a2y(–2) = b0r(0) + b1r(–1) + b2r(–2)
which simplifies to
y(0) = b0r(0) (2.59a)
and
y(1) + a1y(0) + a2 y(–1) = b0r(1) + b1r(0) + b2r(–1)
or
y(1) = –a1y(0) + b0r(1) + b1r(0) (2.59b)
By substituting Eqns (2.59) into Eqn. (2.58), we get
(z2 + a1z + a2)Y(z) = (b0z2 + b1z + b2) R(z)
Therefore
Y ( z) b z 2 + b1z + b2 b + b1z –1 + b2 z –2
= 02
G(z) = = 0
R( z) z + a1 z + a 2 1 + a1z –1 + a 2 z –2
Therefore, we can express the general transfer function model (2.56) as
Y (z) b z n + b1z n – 1 + L + bn
G(z) = = 0n (2.60a)
R(z ) z + a1 z n – 1 + L + an
We will represent the numerator polynomial of G(z) by N(z), and the denominator polynomial by D(z):
N (z )
G(z) = (2.60b)
D(z )
where
N(z) = b0 zn + b1 zn –1 + L + bn; D(z) = zn + a1 zn –1 + L + an
The terminology used in connection with G(s)—the transfer function of continuous-time systems6—is
directly applicable in the case of G(z).
The highest power of the complex variable z in the denominator polynomial D(z) of the transfer function
G(z) determines the order of the transfer function model. The denominator polynomial D(z) is called the
characteristic polynomial.

6
Chapter 2 of reference [155].
 Signal Processing in Digital Control 55
The roots of the equation
D(z) = 0 (2.61a)
are called the poles of the transfer function G(z), and roots of the equation
N(z) = 0 (2.61b)
are called the zeros.
Equation (2.61a) is called the characteristic equation; the poles are the characteristic roots.
In Section 2.9, we will show that if the poles of the transfer function G(z) of a discrete-time system lie
inside the unit circle in the complex z-plane, the discrete-time system is stable.

2.7.1 Transfer Function of Unit Delayer

We now give a simple example of transfer function description of discrete-time systems. Figure 2.12
describes the basic operations characterizing a computer program. The unit delayer shown in this figure is a
dynamic system with input x1(k) and output x2(k); x2(0) represents the initial storage in the shift register.
We assume that the discrete-time system (unit delayer) is initially relaxed:
x2(0) = 0 (2.62a)
and is excited by an input sequence
x1(k); k ³ 0 (2.62b)
The following state variable model gives the output of the unit delayer at k = 0, 1, 2, ...
x2(k + 1) = x1(k) (2.63)
The z-transformation of Eqn. (2.63) yields
X2(z) = z –1 X1(z)
where
Δ Δ
X2(z) = Z [x2(k)]; X1(z) = Z [x1(k)]
Therefore, the transfer function of the unit delayer represented by Eqn. (2.63) is
X 2 (z)
= z –1 (2.64)
X 1 ( z)
The discrete-time system of Fig. 2.16 may be equivalently represented by Fig. 2.22a using the transfer
function description of the unit delayer. Use of the block-diagram analysis results in Fig. 2.22b, which gives

Y(z) =
LM 0.0475 OP
+ 0.05 R(z)
N z - 0.95 Q
Therefore, the transfer function G(z) of the discrete-time system of Fig. 2.22 is
Y (z) 0.05z
G(z) = = (2.65)
R( z ) z - 0.95

0.05

R (z) X (z ) + Y (z)
+ z–1 +
0.0475
+

0.95
(a)
 56 Digital Control and State Variable Methods

0.05

R (z) + Y (z)
1
0.0475
z – 0.95 +

(b)
Fig. 2.22 Equivalent representations of the discrete-time system of Fig. 2.16

2.7.2 Dynamic Response

A standard problem in control engineering is to find the response, y(k), of a system given the input, r(k), and
a model of the system. With z-transforms, we have a means for easily computing the response of linear time-
invariant systems to quite general inputs.
Given a general relaxed, linear discrete-time system and an input signal r(k), the procedure for determin-
ing the output y(k) is given by the following steps.
Step 1 Determine the transfer function G(z) by taking z-transform of equations of motion.
Step 2 Determine the z-transform of the input signal; R(z) = Z [r(k)].
Step 3 Determine the z-transform of the output; Y(z) = G(z)R(z).
Step 4 Break-up Y(z) by partial-fraction expansion.
Step 5 Invert Y(z) to get y(k): look-up the components of y(k) in a table of transform pairs and combine the
components to get the total solution in the desired form.

Example 2.9 A discrete-time system is described by the transfer function

Y ( z) 1 3 1
G(z) = = 2 ; a1 = – , a2 =
R( z) z + a1 z + a 2 4 8
Find the response y(k) to the input (i) r(k) = d (k), (ii) r(k) = m(k)
Solution The transfer function G(z) expressed as a ratio of polynomials in z–1:
z –2
G(z) =
3 –1 1 –2
1–z + z
4 8
Since z–1 is a unit delay operator, we can immediately write the corresponding difference equation:
3 1
y(k) – y(k – 1) + y(k – 2) = r(k – 2)
4 8
The difference equations can be solved by means of recursion procedure. The recursion procedure is
quite simple and convenient for digital computations.
In the following, we obtain the closed-form solutions.
(i) The z-transform of d(k) is (refer Table 2.1): Z [d (k)] = 1
Letting R(z) = 1, we obtain
1 1 4 4
Y(z) = G(z) = = = –
2 3
z – z+
1
z–
1F z–
1 IF
z –
1 Iz –
1
4 8 4H 2 KH 2 K 4
 Signal Processing in Digital Control 57
The impulse response g(k) is therefore (refer Table 2.1)

z –1
4zLM F I OP LMz –1 F 4z I OP
g(k) = Z –1
z–
1
MN GG JJ P –
Z –1
MN GG z – 1 JJ P
2 H KQ H 4KQ
k –1 k –1
=4
F 1I –4
F 1I ;k ³ 1
H 2K H 4K
The impulse input thus excites the system poles without creating any additional response term.
(ii) The z-transform of m(k) is (refer Table 2.1)
z
Z [m(k)] =
z–1
z
Letting R(z) = , we obtain
z–1
Y(z) =
1 z LM
=
zOP
F z–
1 IF
z–
1 z–1 IN z–
1
z–
Q F
1
( z – 1)
IF I
H 4 KH 2 K 4 2H KH K
16 8
z z
–8z
= 3 + + 3
1 1 z2
1 –31
z– z–
144 4 2443 2 Excitation poles
System poles
The inverse transform operation gives (refer Table 2.1)
L16 F 1 I – 8F 1I +
k k
8 k OP
y(k) = MM 3 H 4 K
(1)
144 42444
H 2 K3 2 23
1 PQ
MN Transient response Steady - state response

The transient response terms correspond to system poles excited by the input m(k). These terms
vanish as k ® ¥.
The second response term arises due to the excitation pole, and has the same nature as the input
itself except for a modification in magnitude caused by the system’s behaviour to the specified input.
Since the input exists as k ® ¥, the second response term does not vanish and is called the steady-state
response of the system.
The steady-state response can be quickly obtained without doing the complete inverse transform
operation by use of the final value theorem (Eqn. (2.52)):
lim y(k) = lim (z – 1)Y(z)
k ®¥ z®1

if (z – 1)Y(z) has no poles on the boundary and outside of the unit circle in the complex z-plane.

Example 2.10 Consider a discrete-time system described by the difference equation

1 1
y(k + 2) + y(k + 1) – y(k) = 3r(k + 1) – r(k)
4 8
The system is initially relaxed (y(k) = 0 for k < 0) and is excited by the input
r(k) = (–1)k m (k)
Obtain the transfer function model of the discrete-time system, and therefrom, find the output y(k); k ³ 0.
 58 Digital Control and State Variable Methods
Solution The given difference equation is first converted to the equivalent form:
1 1
y(k) + y(k – 1) – y(k – 2) = 3r(k – 1) – r(k – 2)
4 8
z-transformation of each term in this equation yields (using shifting theorem (2.47))
1 –1 1
Y(z) + z Y(z) – z–2 = 3z–1 R(z) – z–2R(z)
4 8
or
F1 + 1 z –1
–
1 –2
z
I
Y(z) = (3z–1 – z–2) R(z)
H 4 8 K
or
Fz 21
+
z–
1 I
Y(z) = (3z – 1) R(z)
H 4 8 K
The transfer function of the given discrete-time system is
Y ( z) 3z – 1
G(z) = =
R( z) 1
2 1
z + z –
4 8
For (refer Table 2.1; e–aT = –1)
z
R(z) = Z [(–1)k] = ,
z +1
z (3z – 1)
Y(z) = G(z)R(z) =
1 1 F
z 2 + z – ( z + 1)
I
4 8 H K
20 –4 –32
z z z
z (3z – 1) 3 15
= = + + 5
z+
F1
z–
1IF ( z + 1) z +
1 I z–
1 z+1
H2 4KH 2 K 4
Then

y(k) =
LM 20 F – 1I k
–
4 1 FI k
–
32 OP
(–1) k m (k)
N 3 H 2K 15 4HK 5 Q
2.8 FREQUENCY RESPONSE

Using the Laplace transform, we were able to show that if we applied the input r(t) = R0 cos wt to a linear
time-invariant system with transer function G(s), the steady-state output yss was of the form (refer [155])
yss = lim y(t) = R0|G( jw)| cos(w t + f)
t ®¥
where f = ÐG( jw)
A similar result can be obtained for discrete-time systems. Let G(z) be the z-transform of a discrete linear
time-invariant system, and let the input to this system be r(kT) = R0cos(w kT), with T the sampling period.
 Signal Processing in Digital Control 59
Then
R0 z ( z – cos wT ) R0 z ( z – cos wT )
Z [r(kT )] = =
2
z – 2z cos wT + 1 az – e faz – e f jwT – jwT

Suppose
m
k Õ ( z – pi )
i =1
G(z) = n
; |a j| < 1 (2.66)
Õ (z – a j )
j =1

For simplicity, we assume no repeated poles. Then

Y(z) = G(z)R(z)
can be expressed as

Az A* z n Bjz
Y(z) =
z – e jwT
+
z – e – jwT
+ åz–a
j =1 j

Now

a fa R0 ( z – cos wT )
A = z – e jwT
fa
z – e jwT z – e – jwT f G(z) z = e jwT

=
a
R0 e jωT – 0.5 e jωT + e – jωT R f a f
G e jωT = 0 G e jwT = |A|e jq a f
e jωT – e – jωT 2
*
The other residue A is the complex conjugate of A. Taking the inverse transform of Y(z), we obtain
n

Ae jw kT
+ A * – j w kT
e å B j (a j ) k
y(k) = 144424443 + 1 j =1
4 4244 3
Steady - state component
Transient component
If |aj| < 1 for j = 1, 2, K, n, then
n
k
lim
k ®¥
å B j ca j h =0
j =1
Thus,
Δ 2| A| e j (wkT +q ) + e – j (wkT +q )
yss = lim y(kT ) = Aejw kT + A*e–jw kT =
k ®¥ 2
= 2|A| cos(w kT + q )
a f
= R0 G e jwT cos(wkT + q ) (2.67)
We have obtained a result that is analogous to that for continous-time systems. For a sinusoidal input,
the steady-state output is also sinusoidal; scaled by the gain factor |G(e jwT)|, and shifted in phase by
q = ÐG(e jwT). An important difference is that in the continous-time case we have |G( jw)|, while in the
discrete-time case we have |G(e jwT)|. The implications of this difference are discussed later in this chapter.
The steady-state response is also given by (2.67) when the poles of G(z) in Eqn.(2.66) are repeated with
|aj| < 1. This can easily be verified.
 60 Digital Control and State Variable Methods

Example 2.11 The discrete-time system of Fig. 2.22 is described by the transfer function (refer Eqn.(2.65))
Y (z) 0.05 z
G(z) = =
R( z) z – 0.95
The given input sequence is
r(k) = R0 cos(W k) = Re {R0e jW k}
The output is then given by

y(k) = Re Z
RS -1 LMG(z) R z 0 OPUV = Re |RSZ LM 0.05 R z OP|UV
-1 0
2

T N z-e jW
QW |T N (z - 0.95) (z - e ) Q|W jW

R|
= Re S Z -1 LM 0.0475 R 0 FG z IJ + G(e ) R z OP U|V jW 0
|T N 0.95 - e jW H z - 0.95K ( z - e ) Q |W jW

where
0.05 e j W
G(e jW ) = G( z ) z = e jW =
e j W - 0.95
The inverse z-transform operation yields

y(k) = Re
LM 0.0475 R (0.95) + G(e ) R e OP
0 k jW
0
jWk
N10.444 jW
95 - e24443
1442443Q
Transient Steady-state
component component
This equation shows that as k increases, the transient component dies out. When this happens, the output
expression becomes
y( k ) = yss(k) = Re {G(e jW ) R0 e jWk}
k very large

Let G(e jW ) = |G(e jW )| e jf; f = Ð G(e jW )

Then
yss(k) = Re{R0 |G(e jW )|e j(Wk + f)} = R0 |G(e jW )| cos(Wk + f)
The steady-state response has the same form as the input (discrete sinusoidal), but is modified in amplitude
by |G(ejW)| and in phase by Ð G(ejW).
The graphs of |G(e jW)| and Ð G(e jW) as the frequency W is varied, are the frequency-response curves of
the given discrete-time system. The graphs are shown in Fig. 2.23. It is obvious from these curves, that the
given system is a low-pass digital filter.
Phase

1.2
0.2p 0.4p
0
Magnitude

0.8
– 30°
0.4
– 60°
0
0.2p 0.4p
– 90°
Fig. 2.23 Frequency-response curves of the discrete-time system of Fig. 2.22
 Signal Processing in Digital Control 61

2.9 STABILITY ON THE z-PLANE AND THE JURY STABILITY CRITERION

Stability is concerned with the qualitative analysis of the dynamic response of a system. This section
is devoted to the stability analysis of linear time-invariant discrete-time systems. Stability concepts and
definitions used in connection with continuous-time systems7 are directly applicable here.
A linear time-invariant discrete-time system described by the state variable model (refer Eqns (2.17)),
Δ
x(k + 1) = Fx(k) + gr(k); x(0) = x0
y(k) = cx(k) + d r(k)
has two sources of excitation:
(i) the initial state x0 representing initial internal energy storage; and (ii) the external input r(k).
The system is said to be in equilibrium state xe = 0, when both the initial internal energy storage and the
external input are zero.
In the stability study, we are generally concerned with the questions listed below.
(i) If the system with zero input (r(k) = 0; k ³ 0) is perturbed from its equilibrium state xe = 0 at k = 0, will
the state x(k) return to xe, remain ‘close’ to xe, or diverge from xe?
(ii) If the system is relaxed, will a bounded input r(k); k ³ 0, produce a bounded output y(k) for all k?
The first notion of stability is concerned with the ‘boundedness’ of the state of an unforced system in
response to arbitrary initial state, and is called zero-input stability. The second notion is concerned with the
boundedness of the output of a relaxed system in response to bounded input, and is called bounded-input,
bounded-output (BIBO) stability.

2.9.1 BIBO Stability

A relaxed system (zero initial conditions) is said to be BIBO stable if for every bounded input r(k); k ³ 0, the
output y(k) is bounded for all k.
For a linear time-invariant system to satisfy this condition, it is necessary and sufficient that
¥
å |g(k)| < ¥ (2.68)
k =0
where g(k) is the impulse response of the system.
To prove that condition (2.68) guarantees BIBO stability—i.e., sufficiency—we first establish an upper
bound on |y(k)|.
From Eqn. (2.25), we can express the output as a convolution sum:
¥
y(k) = å g(j) r(k – j)
j=0
If we consider the magnitude of the response y(k), it is easy to see that
¥
| y(k)| = å g ( j )r ( k - j)
j=0

which is surely less than the sum of the magnitudes as given by

¥
|y(k)| £ å |g( j)| |r(k – j)|
j=0

7
Chapter 5 of reference [155].
 62 Digital Control and State Variable Methods
Now take a bounded input, i.e.,
|r(k)| < M; 0 £ k < ¥
where M is an arbitrary but finite positive constant. With this input,
¥
|y(k)| £ M å |g( j)|
j=0

and if condition (2.68) holds true, |y(k)| is finite; hence the output is bounded and the system is BIBO stable.
The condition (2.68) is also necessary, for if we consider the bounded input
R| +1 if g ( j) > 0
r(k – j) = S 0 if g( j) = 0
|T-1 if g ( j) < 0
then the output at any fixed value of k is given by
¥ ¥
|y(k)| = å g( j )r ( k -
j = 0
j) = å | g( j )|
j=0

Thus, unless the condition given by (2.68) is true, the system is not BIBO stable.

The condition (2.68) for BIBO stability can be translated into a set of restrictions on the location of poles
of the transfer function G(z) in the z-plane. Consider the discrete-time system shown in Fig. 2.24. The block-
diagram analysis gives the following input-output relation for this system.
Y (z ) Y ( z) z
= G(z) = 2
R( z ) z + a1 z + a 2
R (z) +
z–1 z–1 z
+ or Y(z) = 2 R(z)
z + a1z + a 2
For the impulse input, R(z) = 1. Therefore,
– a1 response transform
+
+ z
Y(z) = 2
z + a1z + a 2
– a2 The impulse response of the system is given
by
Fig. 2.24 A simple discrete-time system
y(k) = g(k) = Z – 1
LM z OP
2

Assume that the poles of the response transform Y(z) are real and distinct:
Nz +az+a Q
1 2

z2 + a1 z + a2 = (z – a1) (z – a2)
Partial fraction expansion of Y(z)/z is then of the form
Y (z) A1 A2
= +
z z - a1 z - a2
where A1 and A2 are real constants.
A1z A2 z
This gives Y(z) = + ; y(k) = A1(a1)k + A2(a2) k
z - a1 z - a2
 Signal Processing in Digital Control 63
k k
The time functions (a1) and (a2) are the response functions contributed by the system poles at z = a1 and
z = a2, respectively. These time functions dictate the qualitative nature of the impulse response of the system.
A time function (a)k either grows or decays depending on |a | > 1 or |a | < 1, respectively. The growth or
decay is monotonic when a is positive and alternates in sign when a is negative. (a)k remains constant for
a = 1 and alternates in sign with constant amplitude for a = – 1 (Fig. 2.25).

z-plane

Unit circle

k
Fig. 2.25 The nature of response function (a ) for different values of a

Consider now the situation wherein the poles of response transform Y(z) are complex.
For the complex-conjugate pole pair at z = p = Roe jW and z = p* = Roe –jW of Y(z), the response y(k) is
obtained as follows.
Y (z) A A*
= +
z z − p z − p*
where A = |A| Ð f and A* is complex-conjugate of A.
The impulse response
y(k) = A(p)k + A* (p*)k = A(p)k + [A(p)k]*

= 2Re [A(p)k] = 2Re [|A| ejf R0k e jW k]

= 2|A| R0k Re[ej(Wk + f)] = 2|A| R0k cos(W k + f)

Therefore, the complex-conjugate pair of poles of the response transform Y(z) gives rise to a sinusoidal or
oscillatory response function R0k cos(Wk + f), whose envelope R0k can be constant, growing or decaying
depending on whether R0 = 1, R0 > 1, or R0 < 1, respectively (Fig. 2.26).
 64 Digital Control and State Variable Methods
For an nth-order linear discrete-time system, the response transform Y(z) has an nth-order characteristic
polynomial. Assume that Y(z) has a real pole at z = a of multiplicity m, and partial fraction expansion of Y(z)
is of the form
A1( m) z A1( m -1) z A12 z A11z
Y(z) = m
+ m -1
+L+ 2 + (2.69)
(z - a ) (z - a ) (z − α ) (z − α )
where A1(m), ..., A12, A11 are real constants.

z-plane

Unit circle

+
+

Fig. 2.26 The nature of response function (R0)k cos(Wk + f) for different values of R0

Response functions contributed by the real pole of multiplicity m can be evaluated as follows.
Consider the transform pair
z
« (a ) k
z -a
Application of the shifting theorem (Eqn. (2.47)) gives
1
« a k -1m( k - 1)
z-a
Using Eqn. (2.41),
−z
d F 1 I↔ kα k −1μ ( k − 1 or
z
« k a k -1 m (k - 1)
H
dz z − α K (z - a )2
From this pair, we may write (Eqn. (2.47)),
1
« ( k - 1)a k - 2 m ( k - 2)
( z - a )2
Performing differentiation operation once again, we have

-z
d LM1 OP
« k (k - 1) a k - 2 m (k - 2)
dz ( z - a ) 2
N Q
z 1
or 3
« k ( k - 1)a k - 2 m ( k - 2)
(z - a) 2!
 Signal Processing in Digital Control 65
In general
z k ! (a ) k - m + 1
« m ( k - m + 1) (2.70)
(z - a )m ( k - m + 1) ! (m - 1) !
It can easily be established using final value theorem (Eqn. (2.52)) that each response function in Eqn. (2.69)
equals zero as k ® ¥ if |a| < 1. However, the response functions in Eqn. (2.69) grow without bound for |a| ³ 1.
Similar conclusions can be derived in case of response functions corresponding to complex-conjugate pair
of poles ( R0 e ±W ) of multiplicity m. The limit of each response function as k ® ¥ equals zero if R0 < 1. The
case of R0 ³ 1 contributes growing response functions.

From the foregoing discussion it follows that the nature of the response terms contributed by the
system poles (i.e., the poles of the transfer function G(z)), gives the nature of the impulse response g(k)
(= Z –1[G(z)]) of the system. This therefore answers the question of BIBO stability through condition (2.68),
which says that for a system with transfer function G(z) to be BIBO stable, it is necessary and sufficient that
¥
å |g(k)| < ∞
k =0

N ( z)
The nature of response terms contributed by various types of poles of G(z) = , i.e., the roots of the
D( z)
characteristic equation D(z) = 0, has already been investigated. Observing the nature of response terms
carefully, leads us to the following general conclusions on BIBO stability.
(i) If all the roots of the characteristic equation lie inside the unit circle in the z-plane, then the impulse
¥
response is bounded and eventually decays to zero. Therefore å |g(k)| is finite and the system is
k =0
BIBO stable.
(ii) If any root of the characteristic equation lies outside the unit circle in the z-plane, g(k) grows without
¥
bound and å |g(k)| is infinite. The system is therefore unstable.
k =0
(iii) If the characteristic equation has repeated roots on the unit circle in the z-plane, g(k) grows without
¥
bound and å |g(k)| is infinite. The system is therefore unstable.
k =0

(iv) If one or more non-repeated roots of the characteristic equation are on the unit circle in the
¥
z-plane, then g(k) is bounded but å |g(k)| is infinite. The system is therefore unstable.
k =0
An exception to the definition of BIBO stability is brought out by the following observations. Consider a
system with transfer function
N (z)
G(z) =
( z - 1)( z - e jW )( z - e - jW )
The system has non-repeated poles on the unit circle in the z-plane. The response functions contributed by
the system poles at z = 1 and z = e ± jW are respectively (l)k and cos(Wk + f). The terms (l)k and cos(W k + f) are
¥
bounded, å |g(k)| is infinite and the system is unstable in the sense of our definition of BIBO stability.
k=0
 66 Digital Control and State Variable Methods
Careful examination of the input-output relation
N ( z)
Y(z) = G(z)R(z) = R(z)
( z - 1)( z - e jW )( z - e - jW )
shows that y(k) is bounded for all bounded r(k), unless the input has a pole matching one of the system poles
on the unit circle. For example, for a unit step input r(k) = m(k),
z zN ( z )
R(z) = and Y(z) =
z −1 ( z - 1) ( z - e jW )( z - e - jW )
2

The response y(k) is a linear combination of the terms cos(W k + f), (l)k, and k(l)k, and therefore y(k) ® ¥ as
k ® ¥. Such a system, which has bounded output for all bounded inputs, except for the inputs having poles
matching the system poles, may be treated as acceptable or non-acceptable. We will bring the situations
where the system has non-repeated poles on the unit circle under the class of marginally stable systems.

2.9.2 Zero-input Stability

This concept of stability is based on the dynamic evolution of the system state in response to arbitrary initial
state representing initial internal energy storage. State variable model (refer Eqn. (2.17))
x(k + 1) = Fx(k) (2.71)
is the most appropriate for the study of dynamic evolution of the state x(k) in response to the initial state x(0).
We may classify stability as follows:
(i) Unstable: There is at least one finite initial state x(0) such that x(k) grows thereafter without bound
as k ® ¥.
(ii) Asymptotically stable: For all possible initial states x(0), x(k) eventually decays to zero as k ® ¥.
(iii) Marginally stable: For all initial states x(0), x(k) remains thereafter within finite bounds for k > 0.
Taking z-transform on both the sides of Eqn. (2.71) yields
z X(z) – z x(0) = FX(z) where X(z) =Δ Z [x(k)]
Solving for X(z), we get
X(z) = (zI – F)–1 zx(0) = F(z) x(0)
where
( zI - F ) + z
F(z) = (zI – F)–1 z = (2.72a)
| zI - F |
The state vector x(k) can be obtained by inverse transforming X(z):
x(k) = Z –1 [F (z)] x(0) (2.72b)
Note that for an n × n matrix F, | zI – F| is an nth-order polynomial in z. Also, each element of the adjoint
matrix (zI – F)+ is a polynomial in z of order less than or equal to (n – 1). Therefore, each element of
F(z)/z is strictly proper rational function, and can be expanded in a partial fraction expansion. Using the
time-response analysis given earlier in this section, it is easy to establish that
lim x( k ) ® 0
k ®¥
if all the roots of the characteristic polynomial |zI – F|, lie strictly inside the unit circle of the complex plane.
In Chapter 6 we will see that under mildly restrictive conditions (namely the system must be both controlla-
ble and observable), the roots of the characteristic polynomial |zI – F| are same as the poles of the corre-
sponding transfer function, and asymptotic stability ensures BIBO stability and vice versa. This implies that
stability analysis can be carried out using the BIBO stability test (or only the asymptotic stability test).
 Signal Processing in Digital Control 67

We will use the following terminology and tests for stability analysis of linear time-invariant systems
described by the transfer function G(z) = N(z)/D(z), with the characteristic equation D(z) = 0.
(i) If all the roots of the characteristic equation lie inside the unit circle in the z-plane, the system is stable.
(ii) If any root of the characteristic equation lies outside the unit circle in the z-plane, or if there is a
repeated root on the unit circle, the system is unstable.
(iii) If condition (i) is satisfied except for the presence of one or more non-repeated roots on the unit circle
in the z-plane, the system is marginally stable.
It follows from the above discussion that stability can be established by determining the roots of the
characteristic equations. All the commercially available CAD packages ([151–154]) include root-solving
routines. However, there exist tests for determining the stability of a discrete-time system, without finding
the actual numerical values of the roots of the characteristic equation.
A well-known criterion to test the location of zeros of the polynomial
D(z) = a0 zn + a1 zn – 1 + L + an – 1 z + an
where a’s are real coefficients, is the Jury stability criterion. The proof of this criterion is quite involved and
is given in the literature (Jury and Blanchard [98]). The criterion gives the necessary and sufficient condi-
tions for the roots to lie inside the unit circle. In the following, we present the Jury stability criterion without
proof.

2.9.3 The Jury Stability Criterion

In applying the Jury stability criterion to a given characteristic equation D(z) = 0, we construct a table whose
elements are based on the coefficients of D(z).
Consider the general form of the characteristic polynomial D(z) (refer Eqn. (2.60b)):
D(z) = a0 zn + a1 zn –1 + L + ak zn – k + L + an – 1 z + an; a0 > 0 (2.73)
The criterion uses the Jury stability table given in Table 2.2.

Table 2.2 General form of the Jury stability table

Row z0 z1 z2 z3 L zk L zn – 2 z n –1 zn
1 an an – 1 an – 2 an – 3 L an – k L a2 a1 a0
2 a0 a1 a2 a3 L ak L an – 2 an – 1 an
3 bn – 1 bn – 2 bn – 3 bn – 4 L bn – k – 1 L b1 b0
4 b0 b1 b2 b3 L bk L bn – 2 bn – 1
5 cn – 2 cn – 3 cn – 4 cn – 5 L cn – k – 2 L c0
6 c0 c1 c2 c3 L ck L cn – 2
M
2n – 5 p3 p2 p1 p0
2n – 4 p0 p1 p2 p3
2n – 3 q2 q1 q0

The Jury stability table is formed using the following rules.

(i) The first two rows of the table consist of the coefficients of D(z), arranged in ascending order of power
of z in row 1, and in reverse order in row 2.
 68 Digital Control and State Variable Methods

(ii) All even-numbered rows are simply the reverse of the immediately preceding odd-numbered rows.
(iii) The elements for rows 3 through (2n – 3) are given by the following determinants:
an a n - 1- k
bk = ; k = 0, 1, 2, ..., n – 1
a0 ak +1

bn -1 bn - 2 - k
ck = ; k = 0, 1, 2, ..., n – 2 (2.74)
b0 bk + 1

M
p3 p2 - k
qk = ; k = 0, 1, 2
p0 pk +1
The procedure is continued until the (2n – 3)rd row is reached which will contain exactly three elements.
The necessary and sufficient conditions for polynomial D(z) to have no roots on and outside the unit circle
in the z-plane are:
D(1) > 0

D(– 1)
RS> 0 for n even
(2.75a)
T< 0 for n odd
| an | < | a0 |

| bn -1 | > | b0 | U|
| cn - 2 | > | c0 | |V
(n - 2) constraints (2.75b)
M ||
| q2 | > | q0 | W
The conditions on D(1), D(–1), and between a0 and an in (2.75a) form necessary conditions of stability
that are very simple to check without carrying out the Jury tabulation.
It should be noted that the test of stability given in (2.75) is valid only if the inequality conditions provide
conclusive results. Jury tabulation ends prematurely if, either the first and the last elements of a row are zero,
or, a complete row is zero. These cases are referred to as singular cases. These problems can be resolved by
expanding and contracting the unit circle infinitesimally, which is equivalent to moving the roots off the unit
circle. The transformation for this purpose is
z$ = (1 + e)z
where e is a very small real number.
This transformation can easily be applied since
(1 + e)n zn ≅ (1 + ne)zn
When e is a positive number, the radius of the unit circle is expanded to (1 + e), and when e is negative, the
radius of the unit circle is reduced to (1 – | e |). This is equivalent to moving the roots slightly. The difference
between the number of roots found inside (or outside) the unit circle when the circle is expanded and con-
tracted by e, is the number of roots on the circle [97].
 Signal Processing in Digital Control 69

Example 2.12 Consider the characteristic polynomial

D(z) = 2z4 + 7z3 + 10z2 + 4z + 1
Employing stability constraints (2.75a), we get
(i) D(1) = 2 + 7 + 10 + 4 + 1 = 24 > 0; satisfied (iii) |1| < |2| ; satisfied
(ii) D(– 1) = 2 – 7 + 10 – 4 + 1 = 2 > 0; satisfied
Next, we construct the Jury table:
Row z0 z1 z2 z3 z4
1 1 4 10 7 2
2 2 7 10 4 1
3 –3 – 10 – 10 –1
4 –1 – 10 – 10 –3
5 8 20 20

Employing stability constraints (2.75b), we get

(i) | –3| > | –1|; satisfied (ii) |8| > | 20 |; not satisfied
The system is therefore unstable.

Usefulness of the Jury stability test for the design of a digital control system from the stability point of
view, will be demonstrated in the next chapter.
The Jury criterion is of marginal use in designing a feedback system; it falls far short of the root locus
method discussed in Chapter 4. The root locus technique of factoring a polynomial is intrinsically geometric,
as opposed to the algebraic approach of the Jury criterion. The root locus method enables us to rapidly
sketch the locus of all solutions to the characteristic polynomial of the closed-loop transfer function. The
sketch is usually only qualitative, but even so, it offers great insight by showing how the locations of the
poles of closed-loop transfer function change as the gain is varied. As we will see in Chapter 4, the root locus
approach has been reduced to a set of “rules”. Applied in an orderly fashion, these rules quickly identify all
closed-loop pole locations.

2.10 SAMPLE-AND-HOLD SYSTEMS

The sampling operation—conversion of a continuous-time function to a sequence—has earlier been studied

in Section 2.6. We developed an impulse modulation model for the sampling operation (refer Fig. 2.19).
It is important to emphasize here that the impulse modulation model is a mathematical representation of
sampling; not a representation of any physical system designed to implement the sampling operation. We
have introduced this representation of the sampling operation because it leads to a simple derivation of a key
result on sampling (given in the next section) and because this approach allows us to obtain a transfer
function model of the hold operation.

2.10.1 The Hold Operation

It is the inverse of the sampling operation—conversion of a sequence to a continuous-time function. In
computer-controlled systems, it is necessary to convert the control actions calculated by the computer as a
sequence of numbers, to a continuous-time signal that can be applied to the process.
 70 Digital Control and State Variable Methods
The problem of hold operation may be posed as follows:
Given a sequence {y(0), y(1), ..., y(k), ...}, we have to construct ya(t), t ≥ 0.
A commonly used solution to the problem of hold operation, is polynomial extrapolation. Using a
Taylor’s series expansion about t = kT, we can express ya(t) as
y (kT )
&&
ya(t) = ya(kT) + y& a(kT) (t – kT) + a (t – kT)2 + L ; kT £ t < (k + 1)T (2.76)
2!
where
Δ dy ( t ) 1
y& a ( kT ) = a @ [y (kT) – ya((k – 1)T )]
dt t = kT T a
Δ d 2 y a (t ) 1 &
ya ( kT ) =
&& ≅ [ y a (kT) – y& a ((k – 1)T )]
dt 2 t = kT T
1
= [ ya(kT) – 2ya((k – 1)T ) + ya ((k – 2)T)]
T2
If only the first term in expansion (2.76) is used, the data hold is called a zero-order hold (ZOH). Here we
assume that the function ya(t) is approximately constant within the sampling interval, at a value equal to that
of the function at the preceding sampling instant. Therefore, for a given input sequence {y(k)}, the output of
ZOH is given by
ya(t) = y(k); kT £ t < (k + 1)T (2.77)
The first two terms in Eqn. (2.76) are used to realize the first-order hold. For a given input sequence {y(k)},
the output of the first-order hold is given by
t - kT
ya(t) = y(k) + [y(k) – y(k – 1)] (2.78)
T
It is obvious from Eqn. (2.76) that the higher the order of the derivative to be approximated, the larger will
be the number of delay pulses required. The time-delay adversely affects the stability of feedback control
systems. Furthermore, a high-order extrapolation requires complex circuitry and results in high costs of con-
struction. The ZOH is the simplest, and most commonly used, data hold device. The standard D/A converters
are often designed in such a way that the old value is held constant until a new conversion is ordered.

2.10.2 A Model of Sample-and-Hold Operation

In the digital control structure of Fig. 2.2, discrete-time processing of continuous-time signals is accom-
plished by the system depicted in Fig. 2.27. The system is a cascade of an A/D converter followed by a
discrete-time system (computer program), followed by a D/A converter. Note that the overall system is
equivalent to a continuous-time system, since it
xa (t) x(k ) Computer y ( k ) y a ( t ) transforms the continuous-time input signal xa(t)
A/D program D/A into the continuous-time signal ya(t). However,
the properties of the system are dependent on the
choice of the discrete-time system and the sam-
pling rate.
In the special case of discrete-time signal
Sampling
process processing with a unit-gain algorithm, and negli-
gible time delay (i.e., y(k) = x(k)), the combined
Fig. 2.27 Discrete-time processing action of the A/D converter, the computer, and the
of continuous-time signals D/A converter can be described as a system that
 Signal Processing in Digital Control 71
samples the analog signal and produces
S/H another analog signal that is constant over
x a ( t) ya (t) the sampling periods. Such a system is
called a sample-and-hold (S/H). Input-out-
put behaviour of a S/H system is described
diagrammatically in Fig. 2.28. In the follow-
ing, we develop an idealized model for S/H
systems.
t t S/H operations require modelling of two
Fig. 2.28 Input-output behaviour of a sample-and-
hold system processes:

(i) extracting the samples, and (ii) holding the result fixed for one period.
The impulse modulator effectively extracts the samples in the form of x(k)d (t – kT). The remaining
problem is to construct a linear time-invariant system which will convert this impulse into a pulse of height
x(k) and width T. The S/H may therefore be modelled by Fig. 2.29a, wherein the ZOH is a system whose
response to a unit-impulse d (t) is a unit-pulse gh0(t) of width T. The Laplace transform of the impulse
response gh0(t) is the transfer function of the hold operation, namely
¥ T
1 - e - sT
Gh0(s) = L –1[gh0(t)] = z
0
gh0(t) e–st dt = z
0
e -st dt =
s
(2.79)

Figure 2.29b is a block diagram representation of the transfer function model of the S/H operation.
T
xa(t) x*(t) y a( t )
ZOH

d ( t) gh0(t)

1 1

t 0 T t
(a)
T
X a( s ) 1 – e–sT Y a(s)
G h0(s) =
s
X *(s) = x (k) e–skT
k=0
(b)
Fig. 2.29 A model of a sample-and-hold operation

Practical Sample-and-Hold Circuit In a majority of practical digital operations, S/H functions are
performed by a single S/H device. It consists of a capacitor, an electronic switch, and operational amplifiers
(Fig. 2.30). Op amps are needed for isolation; the capacitor and switch cannot be connected directly to
analog circuitry because of the capacitor’s effect on the driving waveform.
 72 Digital Control and State Variable Methods

–
–
+ VOUT
VIN + Hold
capacitor
Control
logic
Sample/hold
pulse
Fig. 2.30 A sample-and-hold device

Since the voltage between the inverting and non-inverting inputs of an op amp is in microvolts, we can
approximate this voltage to zero. This implies that the voltage from the inverting input (– input) to ground in
Fig. 2.30 is approximately VIN; therefore the output of first op amp is approximately VIN.
When the switch is closed, the capacitor rapidly charges to VIN , and VOUT is equal to VIN approximately.
When the switch opens, the capacitor retains its charge; the output holds at a value of VIN.
If the input voltage changes rapidly while the switch is closed, the capacitor can follow this voltage
because the charging time-constant is very short. If the switch is suddenly opened, the capacitor voltage
represents a sample of the input voltage at the instant the switch was opened. The capacitor then holds this
sample until the switch is again closed and a new sample taken.
As an illustration of the application of a sampler/
ZOH circuit, consider the A/D conversion system of
Fig. 2.31. The two subsystems in this figure corre-
xa (t) S/H A/D x (k) spond to systems that are available as physical
device Converter devices. The A/D converter converts a voltage (or
current) amplitude at its input into a binary code
Fig. 2.31 Physical configuration for A/D representing a quantized amplitude value closest to
conversion the amplitude of the input. However, the conversion
is not instantaneous. Input signal variation during
the conversion time of A/D converter (typical conversion times of commercial A/D units range from
100 nsec to 200 m sec), can lead to erroneous results. For this reason, a high performance A/D conversion
system includes a S/H device as in Fig. 2.31.
Although a S/H is available commercially as one unit, it is advantageous to treat the sampling and holding
operations separately for analytical purposes, as has been done in the S/H model of Fig. 2.29b. This model
gives the defining equation of the sampling process and the transfer function of the ZOH. It may be empha-
sized here that X*(s) is not present in the physical system but appears in the mathematical model; the sampler
in Fig. 2.29 does not model a physical sampler and the block does not model a physical data hold. However,
the combination does accurately model a sampler/ZOH device.

2.11 SAMPLED SPECTRA AND ALIASING

We can get further insight into the process of sampling by relating the spectrum of the continuous-time
signal to that of discrete-time sequence that is obtained by sampling.
Let us define the continuous-time signal by xa(t). Its spectrum is then given by Xa(jw), where w is the
frequency in radians per second. The sequence x(k) with value x(k) = xa(kT) is derived from xa(t) by periodic
sampling. Spectrum of x(k) is given by X(ejW) where the frequency W has units of radians per sample interval.
 Signal Processing in Digital Control 73
st
The Laplace transform expresses an analog signal xa (t) as a continuous sum of exponemtials e ; s =
s + jw. The Fourier transform expresses xa(t) as a continuous sum of exponentrals ejwt. Similarly z-transform
expresses a sequence x(k) as a discrete sum of phasors z–k ; z = re jW. Fourier transform expresses x(k) as a
discrete sum of exponentials ejWk [31].
The Fourier transforms of xa(t) and x (k) are, respectively,
¥
Xa(jw) = z
-¥
xa(t) e –jw t dt (2.80)

¥
X(ejW) = å x(k) e –jWk (2.81)
k =-¥
We use the intermediate function x*(t)—the impulse modulated xa(t)—to establish a relation between Xa( jw)
and X(e jW ).
The Fourier transform of x*(t), denoted by X*(jw) is (refer Eqn. (2.30)) given by
¥

z LMMN å x(k) d(t - kT)OPPQ e

¥
∞
- jw t
X*(jw) = z
−∞
x * (t )e − jω t dtv =
-¥ k =0
dt

¥ ¥ ¥
= å z d (t - kT) x( k ) e- jw t dt = å x(k)e –jkwT (2.82)
k =-¥ -¥ k =-¥
(The summation over the interval – ¥ to ¥ is allowed, since x(k) = 0 for k < 0). We have arrived at our first
intermediate result. By comparing Eqn. (2.82) with Eqn. (2.81), we observe that
X(e jW )= X * ( jw ) W (2.83a)
w =
T
X(e jW ) is thus a frequency-scaled version of X*(jw) with the frequency scaling specified by
W = wT (2.83b)
*
We now determine X ( jw) in terms of the continuous-time spectrum Xa(jw). From Eqn. (2.30), we have
¥ ¥ ¥
x*(t) = å x(k) d (t – kT) = å xa(t) d (t – kT) = xa(t) å d (t – kT)
k =0 k =0 k =0
The summation over the interval – ¥ to ¥ is allowed since xa (t) = 0 for t < 0. Therefore,
¥
x*(t) = xa(t) å d (t – kT)
k =-¥
A non-sinusoidal periodic signal y(t) of period T0 can be expanded through the Fourier series [31]:
∞
y(t) = ∑ cn e jnω t ; 0
w0 = 2p /T0
n= –∞
T0 / 2
1
cn =
T0 z
- T0 / 2
y(t ) e – jnw 0 t dt

¥
Since å d (t – kT ) is a periodic function of period T, it can be expressed in terms of the following Fourier
k =-¥
series expansion.
 74 Digital Control and State Variable Methods
¥ ¥ j
2p n t

å d(t – kT) = å cn e T

k =-¥ n=-¥
where
1
T /2 é ¥ ù - j 2p n t
cn = ò ê å d (t - kT ) ú e T dt
T - T /2 ëê k = -¥ ûú
T /2 2p nt
1 -j 1 - j0 1
=
T z
- T /2
d (t )e T dt =
T
e =
T
for all n

Substituting this Fourier series expansion into the impulse modulation process, we get
¥
x*(t) = xa(t) å d (t – kT)
k =-¥

¥ 2p nt
1 ¥ 2p nt
1
å a
j j
= xa(t)
T
å e T =
T n=-¥
x (t) e T
n=-¥

The continuous-time spectrum of x*(t) is then equal to

¥ ¥ LM x (t) e
¥ 2p nt OPe
1 j
X*( jw) = z
-¥
x*(t)e–jw t dt =
T z MN å
-¥ n= -¥
a
T
PQ
- jw t
dt

Interchanging the order of summation and integration, we obtain

1 ¥ LM ¥ FG -j w -
2p n
t
IJ O
*
X ( jw) = å
T n=-¥ MN z x (t)e
H
a
T K dt P
PQ
-¥

¥
=
1 F 2p n I
å X GH jw - j T JK
a (2.84a)
T n=-¥

where Xa( jw) is the Fourier transform of xa(t).

We see from this equation that X*( jw) consists of periodically repeated copies of Xa(jw), scaled by 1/T.
The scaled copies of Xa( jw) are shifted by integer multiples of the sampling frequency
2p
ws = (2.84b)
T
and then superimposed to produce X*( jw).
Equation (2.84a) is our second intermediate result. Combining this result with that given by Eqn. (2.83a),
we obtain the following relations:
1 ¥ FG
2p k IJ
X*( jw) = å
T k = -¥
Xa jw - j
TH K (2.85a)

¥
X(e jW ) = X * j
F WI = 1 å Xa j
FG W
- j
2p k IJ (2.85b)
H TK T k =-¥
H T T K
 Signal Processing in Digital Control 75
2.11.1 Aliasing
While sampling a continuous-time signal xa(t) to produce the sequence x(k) with values x(k) = xa(kT), we
want to ensure that all the information in the original signal is retained in the samples. There will be no
information loss if we can exactly recover the continuous-time signal from the samples. To determine the
condition under which there is no information loss, let us consider xa(t) to be a bandlimited signal with
maximum frequency wm, i.e.,
Xa(jw) = 0 for |w | > w m (2.86)
as shown in Fig. 2.32a. Figure 2.32b shows a plot of X*(jw) under the condition
ws p
= > wm (2.87a)
2 T
Figure 2.32c shows the plot of X(e jW ), which is derived from Fig. 2.32b by simply scaling the frequency axis.

Fig. 2.32 Frequency-domain considerations in sampling and reconstruction

 76 Digital Control and State Variable Methods

X * ( jw) is seen to be a periodic function

G ( j w)
T with period 2p /T (X(e jW ) is a periodic func-
Xa ( j w) T X *( j w) X r ( j w) tion with period 2p ). The spectrum X*(jw)
for |w | ≤ p/T is identical to the continuous-
w
–p 0 p time spectrum Xa (jw) except for linear scal-
T T ing in amplitude (the spectrum X(e jW ) for
|W | ≤ p is identical to the continuous-time
Fig. 2.33 Ideal lowpass filter spectrum Xa( jw), except for linear scaling
in amplitude and frequency). The continu-
ous-time signal xa(t) can be recovered from its samples x(k) without any distortion by employing an ideal
lowpass filter (Fig. 2.33).
Figure 2.32d shows a plot of X*( jw) under the condition
ws p
= < wm (2.87b)
2 T
The plot of X(ejW) can easily be derived from Fig. 2.32d by scaling the frequency axis.
It is obvious from Fig. 2.32d that the shifted versions of Xa( jw) overlap; X*(jw) in the range | w | ≤ p /T
can be viewed as being found by superimposing onto this frequency range, the behaviour of the shifted
versions of Xa( jw).
Consider an arbitrary frequency point w1 in Fig. 2.32d which falls in the region of the overlap of shifted
versions of Xa( jw). The frequency spectrum at w = w1 is the sum of two components. One of these, the
larger one in the figure, has a value equal to Xa(jw1). The other component comes from the spectrum centred

at 2p /T, and has a value equal to X a j

FG FG
2p IJ IJ
- w 1 . Note that the high frequency
FG
2p IJ
- w 1 is ‘folded in’
HH
T KK HT K
about the folding frequency p/T; and appears as low frequency at w1. The frequency
FG 2p - w1
IJ which
HT K
shows up at w1 after sampling, is called in the trade as the ‘alias’ of w1. The superimposition of the high-
frequency behaviour onto the low frequency is known as frequency folding or aliasing. Under the condition
given by (2.87b), the form of X*( jw) in the frequency range |w | ≤ p/T is no longer similar to Xa(jw);
therefore the true spectral shape Xa( jw ) is no longer recoverable by lowpass filtering (refer Fig. 2.33). In
this case, the reconstructed signal xr(t) is related to the original signal xa(t) through a distortion introduced by
aliasing and therefore, there is loss of information due to sampling.

Example 2.13 We consider a simple example to illustrate the effects of aliasing.

Figure 2.34a shows a recording of the temperature in a thermal process. From this recording we observe that
there is an oscillation in temperature with a period of two minutes.
The sampled recording of the temperature obtained by measurement of temperature after every 1.8 min-
utes is shown in Fig. 2.34b. From the sampled recording, one might believe that there is an oscillation with
a period of 18 minutes. There seems to be loss of information because of the process of sampling.
The sampling frequency is ws = 2p /1.8 = p/0.9 rad/min, and the frequency of temperature oscillation is
w 0 = 2p /2 = p rad/min. Since w 0 is greater than ws/2, it does not lie in the passband of a lowpass filter with
a cut-off frequency ws/2. However, the frequency w 0 is ‘folded in’ at w s – w 0 = p /9 rad/min which lies in the
passband of the lowpass filter. The reconstructed signal has therefore a period of 18 minutes, which is the
period of the sampled recording.
 Signal Processing in Digital Control 77

Fig. 2.34 Continuous and sampled recording of temperature

2.11.2 Sampling Theorem

A corollary to the aliasing problem is the sampling theorem stated below.
Let xa(t) be a band-limited signal with Xa(jw) = 0 for |w | > wm. Then xa(t) is uniquely determined from its
samples x(k) = xa(kT) if the sampling frequency ws =
FG 2p IJ > 2w , i.e., the sampling frequency must be
H T K m

atleast twice the highest frequency present in the signal.

We will discuss the practical aspects of the choice of sampling frequency in Section 2.13.

2.12 RECONSTRUCTION OF ANALOG SIGNALS

Digital control systems usually require the transformation of discrete-time sequences into analog signals. In
such cases, we are faced with the converse problem from that of sampling xa(t) to obtain x(k). The relevant
question now becomes—how can xa(t) be recovered from its samples.
We begin by considering the unaliased spectrum of X*(jw) shown in Fig. 2.32b. Xa(jw) has the same form
-p p
as X*(jw) over £ w £ . Xa(jw) can be recovered from X*( jw) by a lowpass filter.
T T
Ideal Lowpass Filter Consider the ideal lowpass filter shown in Fig. 2.33. It is characterized by G(jw)
defined below:
LMT for
–p
£w £
p
G(jw) = T T (2.88)
MN 0 otherwise
Note that the ideal filter given by Eqn. (2.88) has a zero phase characteristic. This phase characteristic stems
from our requirement that any signal whose frequency components are totally within the passband of the
filter, be passed undistorted.
We will need the following basic mathemical background in this section [31].
 78 Digital Control and State Variable Methods
The Fourier transform pair:
∞
F [ y(t)] = Y( jw) =
Δ
z
–∞
y(t )e – jω t dt
∞
Δ 1
–1
F [ Y( jw)] = y(t) =
2π –∞
z Y ( jω )e jω t dω

Shifting Theorem:

F
LM yF t – T I OP = e – jwT/2
Y( jw)
N H 2KQ
The impulse response of the ideal lowpass filter is given by inverse Fourier transformation.
¥ p /T
1 1
g(t) =
2p z
-¥
G( jw) e jw t dw =
2p z
-p / T
Te jw t dw

T sin p t / T
= (e jp t / T - e - jp t / T ) = (2.89)
j 2p t pt/T
Figure 2.35 shows a plot of g(t) versus t. Notice that the response extends from t = – ∞ to t = ∞ . This
implies that there is a response for t < 0 to a unit impulse applied at t = 0 (i.e., the time response that begins
before an input is applied). This cannot be true in the physical world. Hence, such an ideal filter is physically
unrealizable.
g (t)

– 3T –T 0 T 3T t
Fig. 2.35 Impulse response of an ideal lowpass filter

Zero-order hold We consider polynomial holds as an approximation to the ideal lowpass filter. The ZOH
was considered in Section 2.10, and its transfer function was derived to be (Eqn. (2.79))
1 - e - sT
Gh0(s) =
s
Its frequency response is consequently given by
1 - e - jwT e - jwT / 2 (e jwT / 2 - e - jwT / 2 )
Gh0( jw) = =
jw jw
sin (wT / 2) - jwT / 2
= T e (2.90)
wT / 2
sin (wT / 2 )
Plot of versus w will be of the form shown in Fig. 2.35 with sign reversals at w =
wT / 2
2p 4p
, , L (i.e., w = ws, 2ws, K ). The sign reversals amount to a phase shift of –180º (it can be taken as
T T
+180º as well) at w = kws; k = 1, 2, K
 Signal Processing in Digital Control 79
Equation (2.90) can therefore be expressed as
Gh0( jw) = | Gh0( jw)| ∠ Gh0( jw)
where
sin(wT / 2)
|Gh0( jw)| = T (2.91a)
wT / 2
and
-wT FG IJ 2p k
∠ Gh0( jw) = - 180º at w = ; k = 1, 2, ... (2.91b)
2 H K T
A plot of magnitude and phase characteristics of ZOH is shown in Fig. 2.36. The ideal lowpass filter is
shown by dashed lines in Fig. 2.36a. The phase of the ideal filter, at all frequencies, is zero.
It is obvious that the hold device does not have the ideal filter characteristics.
(i) The ZOH begins to attenuate at frequencies considerably below ws/2.
(ii) The ZOH allows high frequencies to pass through, although they are attenuated.
(iii) The factor e– jwT/2 in Eqn. (2.90) corresponds to a delay of T/2 in the time domain. This follows from
the shifting theorem of Fourier transforms. Therefore the linear phase characteristic introduces a time
delay of T/2. When ZOH is used in a feedback system, the lag characteristic of the device degrades
the degree of system stability.

|G h0( j w)|

(a) 0.637 T

0
ws ws 2ws 3ws w
2
G h0( j w)
ws 2ws 3ws w
0

–p

– 2p

(b) – 3p

– 4p

– 5p

Fig. 2.36 ZOH device (a) Gain, and (b) Phase characteristics
 80 Digital Control and State Variable Methods
The higher-order holds, which are more sophisticated and which better approximate the ideal filter, are
more complex and have more time delay than the ZOH. As the additional time delay in feedback control
systems decreases the stability margin or even causes instability, higher-order holds are rarely justified in
terms of improved performance, and therefore the zero-order hold is widely used in practice.
Anti-aliasing Filter In practice, signals in control systems have frequency spectra consisting of low-
frequency components as well as high-frequency noise components. Recall that all signals with frequency
higher than ws/2 appear as signals of frequencies between 0 and ws/2 due to the aliasing effect. Therefore,
high-frequency noise will be folded in and will corrupt the low-frequency signal containing the desired
information.
To avoid aliasing, we must either choose the sampling frequency high enough (ws > 2wm, where wm is the
highest-frequency component present in the signal) or use an analog filter ahead of sampler (refer Fig. 2.2)
to reshape the frequency spectrum of the signal (so that the frequency spectrum for w > (1/2)ws is negli-
gible), before the signal is sampled. Sampling at very high frequencies introduces numerical errors. Anti-
aliasing filters are therefore, useful for digital control applications.
The synthesis of analog filters is now a very mature subject area. Extensive sets of tables exist, which
give, not only the frequency and phase response of many analog prototypes, but also the element values
necessary to realize those prototypes. Many of the design procedures for digital filters, have been developed
in ways that allow this wide body of analog filter knowledge, to be utilized effectively.

2.13 PRACTICAL ASPECTS OF THE CHOICE OF SAMPLING RATE

Every time a digital control algorithm is designed, a suitable sampling interval must be chosen. Choosing a
long sampling interval reduces both the computational load and the need for rapid A/D conversion, and
hence the hardware cost of the project.
However, as the sampling interval is increased, a number of potentially degrading effects start to become
significant. For a particular application, one or more of these degrading effects set the upper limit for the
sampling interval. The process dynamics, the type of algorithm, the control requirement and the characteris-
tics of input and noise signals, all interact to set the maximum usable value for T.
There is also a lower limit for the sampling interval. Digital hardware dictates the minimum usable value
for T.
We will discuss some of the factors which limit the choice of sampling interval. Some empirical rules for
the selection of sampling interval are also reported.

2.13.1 Limiting Factors for the Choice of Sampling Rate

Information Loss due to Sampling The sampling theorem states that a continuous-time signal whose
frequency spectrum is bounded by upper limit wm, can be completely reconstructed from its samples when
the sampling frequency is ws > 2wm. There are two problems associated with the use of the sampling theorem
in practical control systems.
(i) The frequency spectra of real signals do not possess strictly defined wm. There are almost always
frequency components outside the system bandwidth. Therefore, the selection of the sampling
frequency ws using the sampling theorem on the basis of system bandwidth (w b = w m ) is risky, as
frequency components outside w b will appear as low-frequency signals of frequencies between 0 and
ws /2 due to the aliasing effect, and lead to loss of information.
(ii) The ideal lowpass filter needed for perfect reconstruction of a continuous-time signal from its samples
is not physically realizable. Practical filters, such as the ZOH, introduce reconstruction errors
because of the limitations of their operation.
 Signal Processing in Digital Control 81
Figure 2.28 clearly indicates that the accuracy of the zero-order hold as an extrapolating device depends
greatly on the sampling frequency w s. The accuracy improves with increase in sampling frequency.
Information Loss due to Disturbances In practice, signals in control systems include low-frequency
components carrying useful information, as well as high-frequency noise components. The high-frequency
components appear as low-frequency signals (of frequencies between 0 and ws/2) due to the aliasing effect,
causing a loss of information.
To avoid aliasing, we use the analog filter ahead of sampler (refer Fig. 2.2) to reshape the frequency
spectrum of the signal, so that the frequency spectrum for w > (1/2)ws is negligible. The cut-off frequency
ws /2 of the anti-aliasing filter must be much higher than the system bandwidth, otherwise the anti-aliasing
filter becomes as significant as the system itself, in determining the sampled response.
Destabilizing Effects Due to the conversion times and the computation times, a digital algorithm con-
tains a dead-time that is absent from its analog counterpart. Dead-time has a marked destabilizing effect on
a closed-loop system due to the phase shift caused.
A practical approach of selecting the sampling interval is to determine the stability limit of the closed-
loop control system, as sampling interval T is increased. For control system applications, this approach is
more useful than the use of the sampling theorem for the selection of sampling interval. In the later chapters
of this book, we will use stability tests, root-locus techniques, and frequency-response plots to study the
effect of the sampling interval on closed-loop stability.
Algorithm-accuracy Effects A number of digital control algorithms are derived from analog algorithms
by a process of discretization. As we shall see in the next section, in the transformation of an algorithm, from
continuous-time to discrete-time form, errors arise and the character of the digital algorithm differs from
that of its analog counterpart. In general, these errors occurring during the discretization process, become
larger as the sampling interval increases.
This effect should rarely be allowed to dictate a shorter sampling interval, than would otherwise have
been needed. We will see in Chapter 4 that the direct digital design approach allows a longer sampling
interval without the introduction of unacceptable errors.
Word-length Effects As the sampling interval T becomes very short, a digital system does not tend to the
continuous-time case, because of the finite word-length. To visualize this effect, we can imagine that as a
signal is sampled more frequently, adjacent samples have more similar magnitudes. In order to realize the
beneficial effects of shorter sampling, longer word-lengths are needed to resolve the differences between
adjacent samples.
Excessively fast sampling (T → 0) may also result in numerical ill-conditioning in implementation of
recursive control algorithms (such as the PID control algorithm—discussed in the next section).

2.13.2 Empirical Rules for the Selection of Sampling Rate

Practical experience and simulation results have produced a number of useful approximate rules for the
specification of minimum sampling rates.
(i) The recommendations given in the adjacent table for the
Type of variable Sampling time most common process variables follow from the experience
(seconds) of process industries.
Flow 1–3 (ii) Fast-acting electromechanical systems require much shorter
Level 5–10 sampling intervals, perhaps down to a few milliseconds.
Pressure 1–5 (iii) A rule of thumb says that, a sampling period needs to be
Temperature 10–20 selected that is much shorter than any of the time constants
 82 Digital Control and State Variable Methods
in the continuous-time plant to be controlled digitally. The sampling interval, equal to one-tenth of the
smallest time-constant, or the inverse of the largest real pole (or real part of complex pole), has been
recommended.
(iv) For complex poles with the imaginary part wd, the frequency of transient oscillations, corresponding to
the poles, is wd. A convenient rule suggests sampling at the rate of 6 to 10 times per cycle. Thus, if the
largest imaginary part in the poles of the continuous-time plant is 1 rad/sec, which corresponds to
transient oscillations with a frequency of 1/6.28 cycles per second, T = 1 sec may be satisfactory.
(v) Rules of thumb based on the open-loop plant model, are risky under conditions where the high closed-
loop performance is forced from a plant with a low open-loop performance. The rational choice of the
sampling rate, should be based on an understanding of its influence on the closed-loop performance of
the control system. It seems reasonable that the highest frequency of interest, should be closely related
to the 3dB-bandwidth of the closed-loop system. The selection of sampling rates can then be based on
the bandwidth of the closed-loop system. Reasonable sampling rates are 10 to 30 times the bandwidth.
(vi) Another rule of thumb based on the closed-loop performance is to select sampling interval T equal to,
or less than, one-tenth of the desired settling time.

2.14 PRINCIPLES OF DISCRETIZATION

Most of the industrial processes that we are called upon to control are continuous-time processes. Math-
ematical models of continuous-time processes are usually based around differential equations or, equiva-
lently, around transfer functions in the operator s. A very extensive range of well-tried methods for control
system analysis and design are in the continuous-time form.
To move from the continuous-time form to the discrete-time form requires some mechanism for time
discretization (we shall refer to this mechanism simply as discretization). In this section, principles and
various methods of discretization will be presented. An understanding of various possible approaches helps
the formation of a good theoretical foundation for the analysis and design of digital control systems.
The main point is to be aware of the significant features of discretization and to have a rough quantitative
understanding of the errors that are likely to be introduced by various methods. We will shortly see that none
of the discretization methods preserves the characteristics of the continuous-time system exactly.
The specific problem of this section is: given a transfer function G(s), what discrete-time transfer function
will have approximately the same characteristics?
We present four methods for solution of this problem.
(i) Impulse-invariant discretization
(ii) Step-invariant discretization
(iii) Discretization based on finite-difference approximation of derivatives
(iv) Discretization based on bilinear transformation

2.14.1 Impulse Invariance

If we are given a continuous-time impulse response ga(t), we can consider transforming it to a discrete-time
system with impulse response g(k) consisting of equally spaced samples of ga(t) so that
g(k) = ga(t)| t = kT = ga(kT)
where T is a (positive) number to be chosen as part of the discretization procedure.
The transformation of ga(t) to g(k) can be viewed as impulse modulation (refer Fig. 2.19) giving impulse-
train representation g*(t) to the samples g(k):
¥
g*(t) = å g(k)d (t – kT ) (2.92)
k =0
 Signal Processing in Digital Control 83
From the discussion in Section 2.11, and specifically Eqns (2.85), it follows that
1 ¥ 2p k FG IJ
G*(jw) = å Ga jw - j T
T k =-¥ H K (2.93a)

1 ¥ F W 2 p kI
G(e jW ) =
T k =-¥
a å G GH j T - j T JK (2.93b)

w, in radians/second, is the physical frequency of the continuous-time function and W = wT, in radians, is the
observed frequency in its samples.
Thus, for a discrete-time system obtained from a continuous-time system through impulse invariance, the
discrete-time frequency response G(e jW ) is related to the continuous-time frequency response Ga( jw)
through replication of the continuous-time frequency response and linear scaling in amplitude and fre-
quency. If Ga( jw) is bandlimited and T is chosen so that aliasing is avoided, the discrete-time frequency
response is then identical to continuous-time frequency response, except for linear scaling in amplitude and
frequency.
Let us explore further the properties of impulse invariance. Applying the Laplace transform to
Eqn. (2.92), we obtain (refer Eqn. (2.32b))
¥
G*(s) = å g(k) e–skT (2.94a)
k =0
On the other hand, the z-transform of g (k) is, by definition,
¥
G(z) = å g(k) z – k (2.94b)
k =0
Comparing Eqns (2.94a) and (2.94b), it follows that
G( z) = G*(s) (2.95a)
z = e sT

Rewriting Eqn. (2.93a) in terms of the general transform variable s, gives a relationship between G*(s) and
Ga(s):
1 ∞ 2π k F I
G*(s) = ∑ Ga s − T
T k=−∞ H K (2.95b)

∞
1 F 2π k I
Therefore, G( z) =
T
∑ Ga s −
H T K (2.95c)
z=e sT
k=−∞

We note that impulse invariance corresponds to a transformation between G*(s) and G(z) represented by
the mapping
z = e sT= e (s ± jw)T = esT ∠ ± wT (2.96)
between the s-plane and the z-plane.
In the following, we investigate in more detail the mapping z = est. We begin by letting ga(t) = e–atm(t);
a > 0. The Laplace transform of this function is
1
Ga ( s) = (2.97a)
s+a
The starred transformation is (refer Eqn. (2.32b))
¥
e sT
G*(s) = å e - akT e - kTs = (2.97b)
k =0 e sT
- e - at
 84 Digital Control and State Variable Methods
Under the mapping z = est, G*(s) is mapped into the z-transform
z
G ( z) = (2.97c)
z - e - aT
The function Ga(s) has a pole at s = – a. The poles of G*(s) are at s = – a ± j2mp / T; m = 0, 1, 2, .... Thus
G*(s) has a countably infinite number of poles, one of which is the pole of Ga(s) at s = – a, as shown in
Fig. 2.37a. We see that the poles of G*(s) consist of a pole of Ga(s) and copies of this pole, repeated at
intervals of 2p/T. The same would be true for any other function whose Laplace transform exists. The poles
of the Laplace transform will lie in a strip of width 2p/T centered on the real axis of the s-plane. This strip
is called the primary strip. These poles are then repeated in complementary strips, above and below the
primary strip (refer Fig. 2.37a).
Im (s)

5p/T
Im (z)
4p /T
Unit
3p /T circle

2p /T

p /T
Primary
Re (s)
strip –a Re (z)
–p /T

– 2p /T

– 3p /T

– 4p /T

– 5p /T

(b)
(a)
Fig. 2.37 Mapping primary strip to the z-plane

Suppose we map the primary strip of the s-plane into the z-plane. We begin by mapping the points of a
vertical line s = s + jw, where s < 0 is fixed. Under the mapping z = est, a point on this line maps to
z = e (σ + jω ) T = eσT e jωT
The term es T is a real number that can be thought of as a scaling factor for the unit phasor e jwT .
-p p
If £ w £ , and s is fixed with s < 0, then the mapping is a circle with radius less than one. If s = 0,
T T
the line segment, maps onto the unit circle. For clarity, the circles in Fig. 2.37b have been divided into
dashed and solid portions; the solid portions correspond to mapping for 0 £ w £ p /T and the dashed
p
portions correspond to mapping for - £ w £ 0 .
T
 Signal Processing in Digital Control 85
The following points are worth noting at this juncture.
(i) The left half of the primary strip in the s-plane maps onto the interior of the unit circle, in the z-plane.
(ii) The imaginary axis between –jp /T and jp /T, associated with primary strip in the s-plane, maps onto
the unit circle in the z-plane.
(iii) The right half of the primary strip, in the s-plane, maps onto the region, exterior to the unit circle, in the
z-plane.
(iv) The same pattern holds for each of the complimentary strips.
The fourth point needs further discussion. We consider
g a ( t ) = cos(w t )
The corresponding Laplace transform is
s s
Ga ( s) = =
s2 + w 2 ( s - jw )( s + jw )
and the z-transform is
z ( z - cos wT )
G( z) = ( z - e jwT )( z - e - jwT )
The s-plane poles: s = jw and s = – jw, in the primary strip are mapped to the z-plane poles: z = e jw T and
z = e – jwT, respectively. However, these z-plane poles are also maps of s-plane poles: s = jw + 2p / T and
s = – jw + 2p / T; s = jw –2p /T and s = – jw – 2p / T; s = jw + 4p / T and s = –jw + 4p / T; s = jw – 4p /T and
s = – jw – 4p / T;..., in the complimentary strips. The z-plane poles: z = ejwT and z = e – jwT, cannot distinguish
the poles in the primary strip from the poles in the complimentary strips. Thus the largest frequency we can
distinguish is w = p/T, which is half of the sampling frequency 2p/T.
While sampling a continuous-time signal to produce the discrete-time sequence, we want to ensure that
all the information in the original signal is retained in the samples. There will be no information loss if we
can exactly recover the continous-time signal from the samples. To determine the condition under which
there is no information loss, let us consider the continous-time signal to be band-limited signal with maxi-
mum frequency wm. From Fig. 2.37, we observe that there is no information loss if
p
< wm
T
This, in fact, is the sampling theorem.
In summary, the use of impulse invariance corresponds to converting the continuous-time impulse
response to a discrete-time impulse response through sampling. To avoid aliasing, the procedure is restricted
to transforming band-limited frequency responses. Except for aliasing, the discrete-time frequency response
is a replication of the continuous-time frequency response, linearly scaled in amplitude and frequency.
Although useful for discretizing band-limited analog systems, the impulse-invariance method is unsuc-
cessful for discretizing transfer functions Ga(s) for which |Ga( jw)| does not approach zero for large w. In
these cases, an appropriate sampling rate cannot be found to prevent aliasing.
To overcome the problem of aliasing, we need a method in which the entire jw-axis in the s-plane, maps
uniquely onto the unit circle in the z-plane. This is accomplished by the bilinear transformation method,
described later in this section.
For a given analog system Ga(s), the impluse-invariant discrete-time system is obtained following the
procedure given below:
(i) Obtain the impulse response
ga(t) = L –1[Ga(s)]
 86 Digital Control and State Variable Methods
(ii) Select a suitable sampling interval and derive samples g(k) from ga(t),
g(k) = ga (t ) t = kT
(iii) Obtain z-transform of the sequence g(k)
G(z) = Z [g(k)]
The three steps given above can be represented by the following relationship:
G(z) = Z [ L –1[Ga(s)] t = kT ] (2.98a)
This z-transform operation is commonly indicated as
G(z) = Z [Ga(s)] (2.98b)
Single factor building blocks of the Laplace and z-transform pairs are given in Table 2.1. Expanding any
Ga(s) into partial fractions, G(z) can be found by use of this table.

Example 2.14 With the background on analog design methods, the reader will appreciate the value of
being able to correlate particular patterns in the s-plane with particular features of system behaviour. Some of
the useful s-plane patterns, which have been used in analog design, are the loci of points in the s-plane with (i)
constant damping ratio z, and (ii) constant undamped natural frequency wn. In this example, we translate these
patterns in the primary strip of the s-plane onto z-plane using the basic relation z = e sT, where T is some chosen
sampling period.
Consider a second-order system with transfer function
K
Ga(s) = 2
s + 2zw n s + w n2
where z = damping ratio, and wn = undamped natural frequency.
The characteristic root locations in the s-plane are
2
s1, s2 = – z wn ± jwn 1 - z = – zw n ± jw d
Figure 2.38a shows a locus of the characteristic roots, with z held constant and wn varying. Figure 2.38b
shows a locus with wn held constant and z varying. The loci in Figs 2.38a and 2.38b, correspond to an
underdamped second-order system.
Define the sampling frequency by
w s = 2p /T
Corresponding to each point s = s + jw in the primary strip of the s-plane, there is a point
z = exp [(s + jw)2p /ws]
in the z-plane.
(i) Mapping of constant damping ratio loci
A point on the constant-z line in the second quadrant (Fig. 2.38a), can be expressed as
2
s = s + jwd = – zwn + jwn 1 - z
zw n zw n z
Since cot q = = = ,
wd wn 1 - z2 1 - z2
the s-plane point may be described by the relation
s = – wd cotq + jwd
The z-plane relation becomes
z = exp [(– wd cot q + jw d) 2p /ws] = exp [((– 2p cotq)/ws) w d] exp [j(2p/ws) wd]
where wd varies from 0 to ws/2.
 Signal Processing in Digital Control 87
As per this equation, for a constant value of z (and hence cotq ), z is a function of wd only. The constant
damping ratio line in the s-plane maps into the z-plane as a logarithmic spiral (except for q = 0º and q = 90º).
The portion of z-line between wd = 0 and wd = ws /2 corresponds to one-half revolution of the logarithmic
spiral in the z-plane. Mapping of one representative constant-z line is shown in Fig. 2.38c.
jw jw
w
j p = j s
T 2 jwn
Constant- z
locus jwd jwd

q
– zwn s – zwn s
Constant-wn
locus
(a)

(b)
Im Im
Constant-z
wd = p / 2T locus (z = 0.6)

wd = 0

Re Re

1 1

wd = p / T
Constant-wn locus
(wn = 6p /10T)
(c) (d)
Fig. 2.38 Mapping of constant-z, and constant-wn loci from the s-plane to the z -plane
(ii) Mapping of constant undamped natural frequency loci
A point on the constant-wn locus in the second quadrant (Fig. 2.38b), can be expressed as
s = s + jwd = – zwn + jwd
It lies on the circle given by
s 2 + ω 2d = ω 2n
For points in the second quadrant,
s = – ω 2n − ω 2d
The locus of constant -wn in the z-plane is given by the relation

z= e
e− j
ω 2n −ω 2d + jω d T

where wd varies from 0 to ws /2.

Mapping of one representative constant-wn locus is shown in Fig. 2.38d.
 88 Digital Control and State Variable Methods

2.14.2 Step Invariance

The basis for impulse invariance is to choose an impulse response for the discrete-time system that is similar
to the impulse response of the analog system. The use of this procedure is often motivated not so much by a
desire to preserve the impulse-response shape, as by the knowledge that if the analog system is band-limited,
then the discrete-time frequency response will closely approximate the continuous-time frequency response.
In some design problems, a primary objective may be to control some aspect of the time response, such as
the step response. In such cases, a natural approach might be to discretize the continuous-time system by
waveform-invariance criteria. In this subsection, we consider the step-invariant discretization.
The step-invariant discrete-time system is obtained by placing a unit step on the input to the analog
system Ga(s), and a sampled unit step on the input to the discrete-time system. The transfer function G(z) of
the discrete-time system is adjusted, until the output of the discrete-time system represents samples of the
output of the analog system. The input to the analog system Ga(s) is m(t)—a unit step function. Since
L [m(t)] = 1/s, the output y(t) of the analog system is given by

y(t) = L –1
Ga ( s) RS UV
s T W
Output samples of the discrete-time system are defined to be

y(kT) = L –1 a
G ( s) RS UV
s T
t = kT W
The z-transform of this quantity yields the z-domain output of the discrete-time system. This gives
LM R G (s) U OP
Y(z) = Z L −1
MN ST s VW PQ
a
(2.99a)
t = kT
z
Since Z [m(k)] = , where m(k) is the unit-step sequence, the output y(k) of the discrete-time system G(z)
z −1
is given by
Y(z) = G(z) { z −z 1} (2.99b)

Comparing Eqn. (2.99b) with Eqn. (2.99a), we obtain

LM R| F G (s)I U|OP
G(z) = (1 – z –1) Z L
−1

MN S| H s K a
V|P (2.100a)
T t = kT WQ
G(z) = (1 – z–1
L F G ( s) IJ OP
) MZ G a
or
N H s KQ (2.100b)

Notice that Eqn. (2.100b) can be rewritten as follows:

G(z) = Z
LM1 - e - sT
Ga ( s)
OP (2.100c)
This can easily be established.
N s Q
Let L –1
LM G ( s) OP = g (t), and Z [g (kT)] = G (z)
a
N s Q 1 1 1

LM G ( s) O
s PQ
- sT a
Then L –1 e = g (t – T ), and Z [g (kT – T)] = z –1
G1(z)
N 1 1
 Signal Processing in Digital Control 89
Therefore,
Z
LM G ( s) - e
a - sT OP
Ga ( s)
= (1 – z–1) Z
LM
Ga ( s) OP
N s s Q s N Q
This establishes the equivalence of Eqns (2.100b) and (2.100c).
The right hand side of Eqn (2.100c) can be viewed as the z-transform of the analog system Ga (s),
preceded by zero-order hold (ZOH). Introducing a fictitious sampler and ZOH for analytical purposes, we
can use the model of Fig. 2.39 to derive a step-invariant equivalent of analog systems. For obvious reasons,
step-invariant equivalence is also referred to as ZOH equivalence. In the next chapter, we will use the ZOH
equivalence to obtain discrete-time equivalents of the plants of feedback control systems.
T
x(t) x*(t) y (t)
1 – e– sT Ga(s)
s

Fig. 2.39 Ga(s) preceded by a fictitious sample-and-hold device

Equivalent discrete-time systems obtained by the step-invariance method, may exhibit the frequency fold-
ing phenomena and may, therefore, present the same kind of aliasing errors as found in impulse-invariance
method. Notice, however, that the presence of 1/s term in Ga (s)/s causes high-frequency attenuation. Conse-
quently, the equivalent discrete-time system obtained by the step-invariance method, will exhibit smaller
aliasing errors than that obtained by the impulse-invariance method.
As for stability, the equivalent discrete-time system obtained by the step-invariance method is stable if
the original continuous-time system is a stable one (refer Review Example 6.2).

Example 2.15 Figure 2.40 shows the model of a plant driven by a D/A converter. In the following,
wederive the transfer function model relating y(kT) to r(kT).

r (kT ) 0.5 ( s + 4) y (t)

D/A G a(s) =
(s + 1)( s + 2)

Fig. 2.40 A plant driven by a D/A converter

The standard D/A converters are designed in such a way, that the old value of the input sample is held
constant until a new sample arrives. The system of Fig. 2.40 can therefore be viewed as an analog system
Ga(s), preceded by zero-order hold, and we can use ZOH equivalence to obtain the transfer function model
relating y(kT) to r(kT).
ZOH equivalent (step-invariant equivalent) of Ga (s) can be determined as follows. Since
1 0.5 (s + 4) 1 1.5 0.5
Ga ( s) = = - +
s s(s + 1) (s + 2) s s + 1 s + 2
we have (refer Table 2.1)

Z
LM1 G (s)OP = z - 1.5z +
0.5z

From Eqn. (2.100b),

Ns Q z - 1 z - e
a -T
z - e -2 T
 90 Digital Control and State Variable Methods

z -1
G(z) =
z LM
-
1.5z
+
0.5z OP = 1 − 1.5( z − 1) + 0.5( z − 1)
z N
z - 1 z - e -T z - e -2 T Q z−e −T
z−e −2 T

Let the sampling frequency be 20 rad/sec, so that

2p
T= = 0.31416 sec; e–T = 0.7304; e–2T = 0.5335
20
With these values, we get the following step-invariant equivalent of the given analog system:
0.17115z - 0.04535
G(z) = 2
z - 1.2639 z + 0.3897

2.14.3 Finite-Difference Approximation of Derivatives

Another approach to transforming a continuous-time system into a discrete-time one is to approximate de-
rivatives in a differential equation representation of the continuous-time system by finite differences. This is
a common procedure in digital simulations of analog systems, and is motivated by the intuitive notion that
the derivative of a continuous-time function, can be approximated by the difference between consecutive
samples of the signal to be differentiated. To illustrate the procedure, consider the first-order differential
equation
dy (t )
+ ay (t ) = r(t) (2.101)
dt
The backward-difference method consists of replacing r(t) by r(k), y(t) by y(k); and the first derivative
dy(t)/dt by the first backward difference:
dy (t ) y ( k ) - y ( k - 1)
= (2.102)
dt t = kT T
This yields the difference equation
y ( k ) - y ( k - 1)
+ ay(k) = r(k) (2.103)
T
If T is sufficiently small, we would expect the solution y(k) to yield a good approximation to the samples of
y(t).
To interpret the procedure in terms of a mapping of continuous-time function Ga(s) to a discrete-time
function G(z), we apply the Laplace transform to Eqn. (2.101) and z-transform to Eqn. (2.103), to obtain
Y ( s) 1
sY(s) + aY(s) = R(s) ; so that Ga(s)= =
R( s) s+a
F 1 - z -1I Y ( z) 1
GH T JKY(z) + a Y(z) = R(z) ; so that G(z)=
R( z )
=
FG1 − z −1 IJ
+a
Comparing Ga(s) with G(z), we see that
H T K
G(z) = Ga ( s)
s = (1- z -1 ) / T

1 - z -1 1
Therefore, s= ;z= (2.104)
T 1 - sT
is a mapping from the s-plane to the z-plane when the backward-difference method is used to discretize
Eqn. (2.101).
 Signal Processing in Digital Control 91
The stability region in the s-plane can be mapped by Eqn. (2.104) into the z-plane as follows. Noting that
the stable region in the s-plane is given by Re(s) < 0, the stability region in the z-plane under the mapping
(2.104), becomes
F 1 - z -1I FG IJ
z-1
Re GH T JK
= Re
H K
Tz
<0

Writing the complex variable z as a + jb, we may write the last inequality as

Re G
F a + jb - 1IJ < 0
H a + jb K
or
L (a + jb - 1) aa - jb f OP = Re LMa - a + b + jb OP = a - a + b
Re M
2 2 2 2
<0
N 2
a + b 2
Q N a +b Q a +b 2 2 2 2

which can be written as

(a – 1/2)2 + b 2 < (1/2)2
The stable region in the s-plane can thus be mapped into a circle with centre at a = 1/2, b = 0 and radius equal
to 1/2 as shown in Fig. 2.41a.
The backward-difference method is simple and will produce a stable discrete-time system for a stable
continuous-time system. Also, the entire s-plane imaginary axis is mapped once and only once onto the small
z-plane circle; the folding or aliasing problems do not occur. The penalty is a ‘warping’ of the equivalent s-
plane poles as shown in Fig. 2.41b. This situation is reflected in the relationship between the exact z-trans-
formation, and the backward-difference approximation.
Consider a pole in z-plane at z = ejaT. Inverse mapping of this pole to the s-plane, using the transformation
s = ln z/T, gives, s = ja (shown in Fig. 2.41b). Inverse mapping of the pole in the z-plane at z = ejaT to the s-
plane, using the backward-difference approximation s = (1– z –1)/T, gives s = jâ = (1 – e –jaT)/T (also shown
in Fig. 2.41b).
Thus a nonlinear relationship or ‘warping’:
jâ = (1 – e –jaT)/T (2.105)
exists between the two poles ja and jâ in the s-plane. Note that for small aT, using the first two terms of the
expansion of the exponential in Eqn. (2.105), yields
1
ja$ @ 1 - (1 - jaT )
T
@ ja
The ‘warping’ effect is thus negligible for relatively small aT (about 17° or less).
Let us now investigate the behaviour of the equivalent discrete-time system when the derivative dy(t)/dt
in Eqn. (2.101), is replaced by forward difference:
dy (t ) y ( k + 1) - y ( k )
=
dt t = kT T
This yields the following difference equation approximation for Eqn. (2.101):
y ( k + 1) - y ( k )
+ ay(k) = r(k) (2.106)
T
Applying Laplace transform to Eqn. (2.101) and z-transform to Eqn. (2.106), we obtain
Y ( s) 1
= Ga(s) = (2.107a)
R( s) s+ a
Y (z) 1
and = G(z) = (2.107b)
R( z ) z-1
+ a
T
 92 Digital Control and State Variable Methods
jw Im

Unit circle

Re
s

s-plane z-plane
(a)

jw s-plane
jws/2

ja
ja

s
(b)
Fig. 2.41 (a) Mapping of the s-plane to the z-plane using backward difference approximation of
the derivative (b) Warping effect

The right-hand sides of Eqns (2.107a) and (2.107 b) become identical if we let
z -1
s= (2.108)
T
We may consider Eqn. (2.108) to be the mapping from the s-plane to the z-plane, when the forward-
difference method is used to discretize Eqn. (2.101).
One serious problem with the forward-difference approximation method is regarding stability. The left
hand side of the s-plane is mapped into the region Re FH
z −1 IK
< 0 or Re (z) < 1. This mapping shows that the
T
poles of the left half of the s-plane, may be mapped outside the unit circle in z-plane. Hence the discrete-time
system obtained by this method may become unstable.
Rectangular rules for integration Consider the continuous-time system (2.101):
y& (t) = – ay(t) + r(t) (2.109a)
t t
or y(t) = y(0) – a z
0
y (t ) d t + z
0
r (t ) dt (2.109b)
 Signal Processing in Digital Control 93
In numerical analysis, the procedure known as the rectangular rule for integration proceeds by approxi-
mating the continuous-time function by continuous rectangles, as illustrated in Fig. 2.42, and then adding
their areas to compute the total integral. We thus approximate the area as given below:
(i) Forward rectangular rule for integration
kT

z
( k −1)T
y(t ) dt ≅ [y(k – 1)]T (2.110a)

(ii) Backward rectangular rule for integration

kT

z y ( t ) dt ≅ [y(k)]T
( k - 1) T
(2.110b)

With the forward rule for integration, the continuous-time system (2.109) is converted to the following
recursive algorithm:
y(k) = y(k – 1) – aTy(k – 1) + Tr (k – 1)
The z-transformation of this equation gives
Y(z) = z–1 Y(z) – aT z–1 Y(z) + T z–1 R(z)

y (t)
y (t)

0 T 2T 3T t 0 T 2T 3T t

(a) (b)
Fig. 2.42 (a) Forward rectangular rule, and (b) backward rectangular rule for integral approximation

Y ( z) 1
or =
R( z ) z-1
+a
T
Laplace transformation of Eqn. (2.109a) gives the transfer function of the continuous-time system:
Y ( s) 1
=
R( s) s+a
The forward rectangular rule for integration thus results in the s-plane to z-plane mapping:
z -1
s=
T
which is same as the one obtained by forward-difference approximation of derivatives (Eqn. (2.108)).
Similarly, it can easily be established that the backward rectangular rule for integration results in s-plane
to z-plane mapping, which is same as the one obtained by backward-difference approximation of derivatives
(Eqn. (2.104)).
 94 Digital Control and State Variable Methods

Example 2.16 The simplest formula for the PID or three-mode controller is the addition of the proportional,
integral, and derivative modes:
LM t OP
u(t) = Kc e( t ) +
MN
1
TI z
0
e( t ) dt + TD
de( t )
dt PQ
(2.111)

where
u = controller output signal TI = integral or reset time
e = error (controller input) signal TD = derivative or rate time
K = controller gain
For the digital realization of the PID controller, it is necessary to approximate each mode in Eqn. (2.111)
using the sampled values of e(t).
The proportional mode requires no approximation since it is a purely static part:
uP (k) = Kc e(k)
The integral mode may be approximated by the backward rectangular rule for integration. If S(k – 1)
approximates the area under the e(t) curve up to t = (k – 1)T, then the approximation to the area under the e(t)
curve up to t = kT is given by (refer Eqn. (2.110b))
S(k) = S(k – 1) + Te (k)
A digital realization of the integral mode of control is as follows:
K
uI (k) = c S(k)
TI
where S(k) = sum of the areas under the error curve = S(k – 1) + Te(k)
The derivative mode may be approximated by the backward-difference approximation:
de(t ) e( k ) - e( k - 1)
=
dt t = kT T
K T
Therefore, uD(k) = c D [e(k) – e(k – 1)]
T
Bringing all the three modes together, results in the following PID algorithm:
u(k) = uP(k) + uI(k) + uD(k)
LM
= Kc e ( k ) +
1 T
S(k ) + D [e(k ) − e(k − 1)]
OP (2.112a)
N TI T Q
where S(k) = S(k – 1) + Te(k) (2.112b)
We can directly use the s-plane to z-plane mapping given by Eqn. (2.104) to obtain the discrete equivalent
(2.112) of the PID controller (2.111).
The PID controller (2.111), expressed in terms of operator s, is given by the input-output relation
LM
U(s) = Kc 1 +
1 OP
+ TD s E(s) (2.113a)
N TI s Q
The mapping (2.104):
1 - z -1
s=
T
corresponds to backward-difference approximation of derivatives. This mapping transforms Eqn. (2.113a)
to the following system.
 Signal Processing in Digital Control 95
LM
U(z) = Kc 1 +
T FG1 T IJ
+ D (1 - z -1 ) E(z)
OP
N H
TI 1 - z -1 T K (2.113b)
Q
This is the input-output relation of the PID controller in terms of operator z. By the inverse transform
operation, we can express individual control modes by difference equations:
(i) uP(k) = Kc e(k)
K T
(ii) uI(k) – uI(k – 1) = c e(k)
TI
K T K
or uI(k) = uI(k – 1) + c e(k) = c S(k)
TI TI
where S(k) = S(k – 1) + T e(k)
K T
(iii) uD(k) = c D [e(k) – e(k – 1)]
T
Bringing all the three modes together results in the PID algorithm given by Eqns (2.112).

2.14.4 Bilinear Transformation

The technique, based on finite-difference approximation to differential equations, for deriving a discrete-
time system from an analog system, has the advantage that z-transform of the discrete-time system, is trivi-
ally derived from the Laplace transform of the analog system by an algebraic substitution. The disadvan-
tages of these mappings are that jw-axis in the s-plane, generally does not map into the unit circle in the z-
plane, and (for the case of forward-difference method) stable analog systems may not always map into stable
discrete-time systems.
A nonlinear one-to-one mapping from the s-plane to the z-plane which eliminates the disadvantages men-
tioned above and which preserves the desired algebraic form is the bilinear 8 transformation defined by

s=
2 z-1 FG IJ (2.114)
T z+1 H K
This transformation is invertible with the inverse mapping given by
1 + (T / 2) s
z= (2.115)
1 - (T / 2) s
The bilinear transformation also arises from a particular approximation method—the trapezoidal rule for
numerically integrating differential equations.
Let us consider a continuous-time system for which the describing equation is (Eqn. (2.101))
y& (t) = – ay(t) + r(t) (2.116a)

z z
t t
or y(t) = y(0) – a y(τ ) dτ + r (τ ) dτ (2.116b)
0 0
Laplace transformation of Eqn. (2.116a) gives the transfer function of the continuous-time system.
Y (s) 1
= Ga(s) =
R( s ) s+a
Applying bilinear transformation (Eqn. (2.114)) to this transfer function, we obtain
1
G(z) =
F
2 z -1 I
H
T z +1
+a
K
8
The transformation is called bilinear from consideration of its mathematical form.
 96 Digital Control and State Variable Methods

y(t) In numerical analysis, the procedure known as the trapezoidal

rule for integration proceeds by approximating the continuous-time
function by continuous trapezoids, as illustrated in Fig. 2.43, and
then adding their areas to compute the total integral. We thus ap-
proximate the area

z
kT
1
y(t )dt by [y(k) + y(k – 1)]T
2
( k -1) T
With this approximation, Eqn. (2.116b) can be converted to the
0 T 2T 3T t following recursive algorithm:
Fig. 2.43 Trapezoidal rule for
aT T
integral approximation y(k) = y(k – 1) – [y(k) + y(k – 1)] + [r(k) + r(k – 1)]
2 2
The z-transformation of this equation gives
aT T
Y(z) = z–1 Y(z) – [Y(z) + z–1 Y(z)] + [R(z) + z–1 R(z)]
2 2
T
(1 + z -1 )
Y(z) 2 1
or = =
R( z ) (1 - z ) +
- 1 aT -
(1 + z )1 2 z - 1 F I
2 T z +1
+a
H K
This result is identical to the one obtained from the transfer function of the continuous-time system by
bilinear transformation.
The nature of bilinear transformation is best understood from Fig. 2.44, which shows how the s-plane is
mapped onto the z-plane. As seen in the figure, the entire jw-axis in the s-plane, is mapped onto the unit
circle in the z-plane. The left half of s-plane is mapped inside the unit circle in the z-plane, and the right half
of s-plane is mapped outside the z-plane unit circle. These properties can easily be established. Consider, for
example, the left half of s-plane defined by Re (s) < 0. By means of Eqn. (2.114), this region of s-plane is
mapped into the z-plane region defined by

Re FH 2 z − 1IK < 0 or Re FH z − 1IK < 0

T z +1 z +1
Im
jw
Unit circle

Re
s

s-plane z-plane
Fig. 2.44 Mapping of the s-plane to the z-plane using bilinear transformation
 Signal Processing in Digital Control 97
By taking the complex variable z = a + jb, this inequality becomes

Re FH z − 1IK = Re FG α + jβ − 1IJ = Re LM (α − 1 + jβ ) (α + 1 − jβ ) OP
z +1 H α + jβ + 1K N (α + 1 + jβ ) (α + 1 − jβ ) Q
= Re M
L α − 1 + β + j 2 β OP < 0
2 2

N (α + 1) + β Q 2 2

which is equivalent to
a2 – 1 + b2 < 0 or a2 + b2 < 12
which corresponds to the inside of the unit circle in z-plane. The bilinear transformation thus produces a
stable discrete-time system for a stable continuous-time system.
Since the entire jw-axis of the s-plane is mapped once and only once onto the unit circle in the z-plane, the
aliasing errors inherent with impulse-invariant transformations are eliminated. However, there is again a
warping penalty.
Consider a pole in z-plane at z = ejaT, shown in Fig. 2.45c. Its inverse mapping in the s-plane using the
transformation s = ln z/T, gives s = ja (shown in Fig. 2.45a). Inverse mapping of the z-plane pole (Fig. 2.45b)
1 z -1
at z = ejaT, to the s-plane obtained using the bilinear transformation s = , is also shown in Fig. 2.45a:
T z +1
s=2z–1
Tz–1
Im (s) Approximate map

Im (z)
p /T

j 2 tan aT aT rad
T 2
Re(z)
ja
|s |= 0
|s |< 2/T
|s |>2/T
(b)
–s Re(s) Im(z)
= –2/T

aT rad

Re(z)

|s |= 0

|s |< 2/T
–p/T |s |>2/T

s = ln z/T
(a) (c) Exact map
Fig. 2.45 Mapping of vertical line in the s-plane to the z-plane
 98 Digital Control and State Variable Methods

2e jaT - 1 2 e jaT / 2 - e - jaT / 2

s = ja$ = =
Te jaT + 1 Te jaT / 2 + e - jaT / 2

2 aT
= j tan (2.117)
T 2
Equation (2.117) is a measure of the frequency distortion or warping caused by the bilinear transforma-
tion. Whereas the exact mapping gives the s-plane pole at s = ja, approximate mapping gives the pole at
2 aT aT 2p aT
s = j tan . This mapping will be correct only if << p , i.e., if w s = >> a . When < 17° ,
T 2 2 T 2
2 aT
or about 0.3 rad, then j tan @ a . This means that in frequency domain, the bilinear transformation is
T 2
good only for small values wT / 2 .
Some additional information about the nature of the bilinear mapping can be obtained, by considering the
2 2
mapping of s-plane poles s = – s + jw. Figure 2.45a shows the regions of poles with s > , and s < ;
2 T T
the vertical line through -s = - in the left half of the s-plane, maps to a closed curve inside the unit circle
T
in z-plane (Fig. 2.45b). The distortions introduced by bilinear mapping are clearly visible when we compare
this mapping with exact mapping z = esT shown in Fig. 2.45c.

Example 2.17 A method that has been frequently used by practising engineers to approximate a sampled-
data system by a continuous-time system, relies on the approximation of the sample-and-hold operation by
means of a pure time delay. Consider the sampled-data system of Fig. 2.46a. The sinusoidal steady-state
transfer function of the zero-order hold is (refer Eqn. (2.90))
sin (wT / 2) - jw T / 2
Gh0( jw) = T e
wT / 2

u(k) u*(t)
r (t) + e (k ) y (t)
D (z) Gh0(s) G (s)
T T
–

(a)

r (t) + y (t)
D (s ) e –sT/2 G (s )
–

(b)
Fig. 2.46 Discretization of analog design

The sinusoidal steady-state transfer function of the impulse-modulation model of the open-loop system is
given by (refer Eqn. (2.84a))
 Signal Processing in Digital Control 99
∞
Gh0 G*(jw) =
1
∑ FH
Gh 0 jω − j
2π nIK FH
G jω − j
2π n IK
T n= −∞
T T
Since in most control systems, G( jw) has low-pass filter characteristics, we can approximate the right hand
side of the equation given above just by the n = 0 term. At low frequencies, the magnitude of
sin(wT/2)/(w T/2) is approximately unity; therefore (refer Eqn. (2.90))
Gh0G*( jw) = G( jw) e–jwT/2
This means that the sample-and-hold operation can be approximated by a pure time delay of one-half the
sampling period T. Figure 2.46b shows the continuous-time approximation of the sampled-data system.
The approximating continuous-time system can be used for the design of the discrete-time system as
illustrated by the following example.
Consider that the transfer function of the controlled process of the system shown in Fig. 2.46a is
10
G(s) =
(1 + 0.5s) (1 + 0.1s) (1 + 0.05s)
We wish to design a digital controller for the process so that the closed-loop system acquires a damping ratio
of 0.4 without loss of steady-state accuracy. We select sampling time T = 0.04 sec for the proposed digital
controller. Our approach will be to first design an analog controller D(s) for the approximating continuous
time system in Fig. 2.46b (with G(s) = 10/[(1+ 0.5s)(1 + 0.1s)(1 + 0.05s)]), that meets the given design
specifications and then discretizing D(s) to obtain the digital controller D(z).
The plant model for the design of D(s) becomes
10 e −0.02s
GP(s) =
(1 + 0.5s) (1 + 0.1s) (1 + 0.05s)
Since the design is handled more conveniently in frequency domain in the presence of time delay, we can
translate damping ratio specification into equivalent phase margin specification and then proceed. Using the
standard frequency-domain design procedure9, we obtain the following analog compensator:
0.67s + 1
D(s) =
2s + 1
The bandwidth of the compensated system is wb = 10 rad/sec. Note that the sampling rate ws = 2p/T = 157 rad/
sec is factor of 16 faster than wb; therefore our selection of T = 0.04 sec is quite ‘safe’ (refer Section 2.13).
Using the bilinear transformation given by Eqn. (2.114), we obtain

0.67 LM 50(z − 1) OP + 1 34.5z − 32.5 U(z)

D(z) =
N z +1 Q = =
2L
50 ( z − 1) O 101z − 99 E( z )
NM z + 1 QP +1

which leads to
u(k) = 0.9802 u(k – 1) + 0.3416 e(k) – 0.3218 e(k – 1) where e(k) = r(k) – y(k)
This is the proposed algorithm for the digital control system of Fig. 2.46a. To make sure that with the
proposed design, the system will behave as expected, we must analyze the system response. Methods for
analysis of digital control systems are covered in the next chapter.

9
Chapters 8–10 of reference [155].
 100 Digital Control and State Variable Methods
In this section, we have presented several methods for obtaining discrete-time equivalents for continuous-
time systems. The response between sampling points is different for each discretization method used. Fur-
thermore, none of the equivalent discrete-time systems can have complete fidelity. The actual (continuous-
time) response between any two consecutive sampling points, is always different from the response between
the same two consecutive sampling points that is taking place in each equivalent discrete-time system, no
matter what method of discretization is used.
It is not possible to say which equivalent discrete-time system is best for any given analog system, since
the degree of distortions in transient response and frequency response characteristics, depends on the sam-
pling frequency, the highest frequency component involved in the system, transportation lag present in the
system, etc. It may be advisable for the designer to try a few alternate forms of the equivalent discrete-time
systems, for the given analog system.

Review Examples

Review Example 2.1 Consider a first-order discrete-time system described by the difference equation
y(k + 1) + a1 y(k) = b0 r(k + 1) + b1 r(k) (2.118)
The input is switched to the system at k = 0 (r (k) = 0 for k < 0); the initial state y(– 1) of the system is
specified. Obtain a simulation diagram for the system.
Solution State variable models of discrete-time systems can easily be translated into digital computer
simulation diagrams. Methods of conversion of difference equation models to state variable models, will be
presented in Chapter 6.
It can easily be verified that the following state variable model represents the given difference
equation (2.118):
x(k + 1) = – a1x(k) + r(k)
y(k) = (b1 – a1b0) x(k) + b0r(k) (2.119)
In terms of the specified initial condition y(–1) of the difference equation model (2.118), the initial state x(0)
of the state variable model (2.119) is given by
- a1
x(0) = y(–1)
b1 - a1b0
Note that if the first-order discrete-time system is relaxed before switching on the input r(k) at k = 0, the initial
condition y(–1) = 0 for the model (2.118), and equivalently the initial condition x(0) = 0 for the model (2.119).
Figure 2.47 shows a simulation diagram for the given discrete-time system.

b0
x(0) d (k)
+ +
r (k) + x (k) + y (k )
+ b1

+
– a1

Fig. 2.47 Simulation diagram for the state model

 Signal Processing in Digital Control 101

Review Example 2.2 Consider a discrete-time system

1 1
y(k + 2) + y(k + 1) – y(k) = 3r(k + 1) – r(k) (2.120)
4 8
with input
r(k) = (–1)k m(k)
and initial conditions
y(– 1) = 5, y(– 2) = – 6
Find the output y(k); k ≥ 0
Solution The difference equation (2.120) is first converted to the equivalent form
1 1
y(k) + 4 y(k – 1) – 8 y(k – 2) = 3r(k – 1) – r(k – 2) (2.121)
z-transformation of the linear difference equation (2.121) requires the following results:
¥
Z [y(k – 1)] = å y(k – 1)z–k
k= 0
¥
= y(–1) + z–1 å y(k)z– k = y(–1) + z–1 Y(z) (2.122a)
k= 0
Z [y(k – 2)] = y(–2) + z–1[y(–1) + z–1 Y(z)]
= y(–2) + z–1 y(–1) + z–2 Y(z) (2.122b)
z-transformation of each term in Eqn. (2.121) yields
Y(z) + 1
4 [z–1 Y(z) + y(–1)] – 1 –2 –1
8 [z Y(z) + z y(–1) + y(–2)]

= 3 [z–1 R(z) + r(–1)] – [z R(z) + z r(–1) + r(–2)]

–2 –1

Since r(–1) = r(–2) = 0, we have

–1 –2
d1 + 1
4 z -1 - 1
8 z -2 Y(z) = (3z – z )R(z) +
i 5
8 z–1 – 2
or dz 2
+ 1
4 z- 1
8 i Y(z) = (3z – 1) R(z) + 5
8 z – 2z2

3z - 1 -2 z 2 + 5
8 z
Therefore Y(z) = 2
R( z ) + 2
z + 14 z - 1
8 z + 1
4 z- 1
8
For (refer Example 2.10)

z
R(z) = Z [(–1)k] = ,
z +1

z(3z - 1) -2 z 2 + 58 z
Y(z) = +
dz 2
i
+ 14 z - 18 ( z + 1) z 2 + 14 z - 18

-2 z 3 + 138 z 2 - 83 z
=
b gb g
z + 12 z - 14 ( z + 1)
Expanding Y(z)/z into partial fractions,
Y(z) -2 z 2 + 138 z - 83 9 - 101 -32
5
= = 2 + +
z c hc
z + 12 z - 14 ( z + 1) z + h 1
2 z- 1
4 z +1
 102 Digital Control and State Variable Methods
Then (refer Table 2.1)

y(k) = ⎡ 92 − 12 ( ) − 101 ( 14 ) − 32 (−1) k ⎤ μ (k )

k k

⎣⎢ 5
⎦⎥

Review Example 2.3 Consider a second-order discrete-time system described by the difference equation

3 1 1
y(k + 2) – 2 y(k + 1) + 2 y(k) = r(k + 1) + 2 r(k)
The system is initially relaxed (y(k) = 0 for k < 0) and is excited by the input

r(k) =
RS0 ; k = 0
; k > 0 T1
Shifting the difference equation by two sampling intervals, we obtain
y(k) – 23 y(k – 1) + 1
2 y(k – 2) = r(k – 1) + 1
2 r(k – 2)
z-transformation of this equation gives
z -1 + 1
2 z -2 z+ 1
2
Y(z) = R(z) =
1- 3
2 z -1
+ 1
2 z -2
cz - h (z - 1) R(z)
1
2

1 k 1 k
The system modes are ch
2 and (l)k. The mode ch 2 decays as k → ∞ , and the mode (l)k is constant (i.e.,
it remains within finite bounds for all k).
The input r(k) = m(k – 1)

Therefore R(z) = z –1 LM z OP = 1
N z − 1Q z − 1
z + 12
For this input, Y(z) =
b z− 1
2 g
(z − 1)2
It is observed that excitation pole matches one of the system poles. Though the system modes, as well as
the input, do not grow with increasing k, the effect of the pole-matching is to give rise to a time function, in
forced response of the system that grows indefinitely as k → ∞ . This is evident from the inverse transform
of Y(z) (refer Table 2.1)
A1 A2 A
Y(z) = + + 3
1 ( z − 1)2 z − 1
z−
2
1
A1 = z − Y ( z )F =4 I
2 H z=
1 K
2
2
A2 = ( z − 1) Y ( z) z =1 = 3
d
A3 = [( z − 1)2 Y ( z )] =–4
dz z =1
Therefore
4 3 −4
Y(z) = + +
z− 1
2 ( z − 1)2 z −1
 Signal Processing in Digital Control 103

1 k -1
y(k) = 4ch 2 + 3( k - 1) (1) k -1 - 4(1) k - 1

1 k -1
= 4c h 2 + 3( k - 1) - 4; k ³ 1

= 4c h 1 k -1 + 3k - 7; k ³ 1
2

Review Example 2.4 Solve for y(k) the equation

y(k) = r(k) – r(k – 1) – y(k – 1), k ≥ 0

where r(k) =
RS1; k even ; y(–1) = r(–1) = 0
T0; k odd
Solution The z-transformation of the given equation yields
1 - z -1 z −1
Y(z) = R( z ) = R(z)
1+ z -1
z +1
For the given input,
2
1 1 z
R(z) = 1 + z–2 + z– 4 + L = = -2
= 2
1- x x = z -2 1- z z -1

Thus Y(z) = F z - 1I z2
= 2
z2
H z + 1K 2
z - 1 z + 2z + 1
We can expand Y(z) into a power series by dividing the numerator of Y(z) by its denominator:
1 - 2 z -1 + 3 z -2 - 4 z -3 + L
z 2 + 2z + 1 z 2
z2 + 2z + 1
– 2z – 1
– 2z – 4 – 2z–1
3 + 2z–1
3 + 6z–1 + 3z–2
– 4z–1 – 3z–2
M
Therefore Y(z) = 1 – 2z – 1 + 3z–2 – 4z–3 + L
and the values of y(k) are {1, – 2, 3, – 4, L }

Review Example 2.5 Through this simple example, we explain the phenomenon of aliasing.
Suppose the sampling rate is 10 Hz; ws = 20 p rad/sec, T = 0.1 sec. The primary strip in the s-plane
corresponding to this sampling rate is shown in Fig. 2.48a.
We try to sample 6 Hz sine wave (w0 = 12p). Note that the signal lies outside the primary strip. Consider the
mapping of the imaginary axis of s-plane to z-plane, as frequency increases from 0 to 6 Hz. The paths followed
as the frequency increases are shown in Fig. 2.48b.
 104 Digital Control and State Variable Methods

Note that at a frequency of 5 Hz, the two paths meet at z = – 1. The 6 Hz (w0 = 12 p) sine wave will appear
to be (10 Hz – 6 Hz) = 4 Hz sine wave. The high frequency w0 = 12 rad/sec is ‘folded in’ about the folding
frequency p/T = 10p; and appears as low frequency at (ws – w0) =
2p
- w 0 = 8p . F I
T H K
Im (s)
Im (z)

12p
f = 3 Hz f = 2 Hz
10p
8p
f = 4 Hz f = 1 Hz
f=
–6 (wT = 0.2p = 36°)
Hz
36°
f = 5 Hz f = –5 Hz
Re (z)
Re (s) f=
z
–4H –1
H
f =
f=– z
f=–

2 Hz
3 Hz

f = 6 Hz
–8p
–10p

(a) (b)
Fig. 2.48

The high frequency w0, which shows up at (ws – w0) after sampling, is called the ‘alias’ of the primary-
strip frequency (ws – w0). The superimposition of the high-frequency behaviour onto the low frequency is
known as frequency folding or aliasing.
Take a sine wave of 6 Hz frequency and extract the samples with T = 0.1 sec. Examine the sampled
recording carefully; it has a frequency of 4 Hz.
The phenomenon of aliasing has a clear meaning in time. Two continuous sinusoids of different frequen-
cies (6 Hz and 4 Hz in the example under consideration) appear at the same frequency when sampled. We
cannot, therefore, distinguish between them, based on their samples alone.
To avoid aliasing, the requirement is that the sampling frequency ws , must be atleast twice the highest
frequency wm present in the signal, i.e., ws > 2wm. This requirement is formally known as the sampling
theorem.

Review Example 2.6 Find the response of the system shown in Fig. 2.49 to a unit impulse input.
 Signal Processing in Digital Control 105

T=1
r(t) r*(t) 1 y (t)
1 – e–sT
s s(s + 1)

Fig. 2.49

Solution The discrete-time transfer function of the given system is obtained as follows (refer
Eqns (2.100)):
1 - e - sT 1
Gh0(s) = , Ga(s) =
s s(s + 1)
Y(z)
= Z [Gh0(s)Ga(s)] = (1 – z–1) Z
Ga ( s ) FH
IK
R( z ) s

= (1 – z–1 )ZM
L 1 OP = (1 – z ) Z L 1 − 1 + 1 O
–1
N s (s + 1) Q
2 MN s s s + 1QP 2

Using Table 2.1, we obtain

Y(z)
= (1 – z –1
L Tz - z + z OP
)M -T
R( z ) N (z - 1) z - 1 z - e Q
2

=
LM (ze -T
- z + Tz ) + (1 - e - T - Te - T ) OP
N ( z - 1)( z - e - T ) Q
For T = 1, we have
Y (z ) ze −1 + 1 − 2 e −1
=
R( z ) ( z − 1) ( z − e −1 )
0.3678z + 0.2642 0.3678z + 0.2642
= =
( z - 1)(z - 0.3679) z 2 - 1.3678z + 0.3679
For unit-impulse input, R(z) = 1.
0.3678z + 0.2642
Therefore Y(z) = 2
z - 1.3678z + 0.3679
We can expand Y(z) into a power series by dividing the numerator of Y(z) by its denominator:
0.3678 z -1 + 0.7675z -2 + 0.9145z -3 +L
2
z - 1.3678 z + 0.3678 0.3678 z + 0.2644
0.3678z - 0.5031 + 0.1353z -1
+ 0.7675 - 0.1353z -1
+ 0.7675 - 1.0497z -1 + 0.2823z -2
0.9145z -1 - 0.2823z -2
M
This calculation yields the response at the sampling instants, and can be carried on as far as needed. In
this case we have obtained y(kT) as follows:
y(0) = 0, y(T) = 0.3678, y(2T) = 0.7675, and y(3T) = 0.9145.
 106 Digital Control and State Variable Methods

Review Example 2.7 A PID controller is described by the following relation between input e(t) and
output u(t):
1 t LM de(t ) OP
TI 0
u(t) = Kc e(t ) +
e(t ) dt + TD
N dt z (2.123)
Q
Using the trapezoidal rule for integration and backward-difference approximation for the derivatives, obtain
the difference-equation model of the PID algorithm. Also obtain the transfer function U(z)/E(z).
Solution By the trapezoidal rule for integration, we obtain

z kT
e(t ) dt = T LM e(0) + e(T ) + e(T ) + e(2T ) + L + e((k - 1)T ) + e(kT ) OP
0 N 2 2 2 Q
L
= TM å
k
e((i - 1)T ) + e(iT ) O
PP
MN i =1 2 Q
By backward-difference approximation for the derivatives (refer Eqn. (2.102)), we get
de(t ) e( kT ) - e((k - 1)T )
=
dt t = kT T
A difference-equation model of the PID controller is, therefore, given by
|RS
u(k) = Kc e( k ) +
T
å
k
e(i - 1) + e(i ) TD
+ [e(k ) - e(k - 1)]
|UV (2.124)
|T TI i =1 2 T |W
Let us now obtain the transfer function model of
e (t)
the PID control algorithm given by Eqn. (2.124).
Define (refer Fig. 2.50)
e(i - 1) + e(i )
= f (i); f (0) = 0
2
k k
f (T ) f (2 T ) e(i - 1) + e(i )
Then å = å f(i)
f (0) i =1 2 i =1

Taking the z-transform of this equation (refer Eqn.

–T 0 T 2T 3T t (2.51)), we obtain
Fig. 2.50 Diagram depicting function f(kT) LM e(i - 1) + e(i) OP = Z LM f (i)OP
k k
Z
MN å 2 PQ MN å PQ
i =1 i =1
z
= F(z )
z −1
-1
Notice that LM e(i - 1) + e(i) OP = 1 + z E(z)
F(z) = Z
N 2 Q 2
Hence Z Må
L e(i - 1) + e(i) OP = 1 + z ( ) = z + 1
k -1

MN 2 i =1PQ 2(1 - z ) E z 2(z - 1) E(z) -1

The z-transform of Eqn. (2.124) becomes

L T 1 + z + T (1 − z )OP E(z)
U(z) = K M1 +
−1
D −1
c −1
N 2T 1 − z T Q I
 Signal Processing in Digital Control 107

= Kc 1 +
LM T F z + 1I + T F z − 1I OP E(z)
D
(2.125)
N 2TI H z − 1K T H z K Q
This equation gives the transfer function model of the PID control algorithm. Note that we can obtain the
discrete-time transfer function model (2.125) by expressing the PID controller (2.123) in terms of operator
s and then using the mapping (2.104) for the derivative term, and the mapping (2.114) for the integral term
of the controller.

Review Example 2.8 Derive the difference equation model for the numerical solution of the differential
equation
d 2 y(t ) dy(t ) dy
2
+ a1 + a2 y(t ) = r(t); y(0) = y10 , (0) = y 20 , 0 ≤ t ≤ tf (2.126)
dt dt dt
Use backward difference approximation for the derivatives.
Solution We divide the interval 0 ≤ t ≤ tf into N equal intervals of width equal to step-length T:
tf
= T; t = kT, k = 0, 1, 2, ..., N
N
By backward difference approximation,
dy(t ) Δ y(k ) - y(k - 1)
= y& ( k ) = (2.127)
dt t = kT T
d 2 y (t ) Δ y&(k ) - y&(k - 1) 1
= y&&( k ) = = 2 [y(k) – 2y(k – 1) + y(k – 2)] (2.128)
dt 2 t = kT
T T
From Eqn. (2.127), we have
y10 - y( -1)
y&(0) = y20 =
T
Substituting Eqns (2.127) and (2.128) into (2.126) at t = kT, we obtain
1 a
2
[y(k) – 2y(k – 1) + y(k – 2)] + 1 [y(k) – y(k – 1)] + a2 y(k) = r(k)
T T

or Fa +
a1 1
+ 2 I y(k) – F a 1
+
2 I y(k – 1) + 1 y(k – 2) = r(k);
H 2
T T K HT T2 K T2

y(0) = y10 , y(– 1) = y10 – Ty20 (2.129)

Incrementing k to take on values k = 1, 2, ..., N, we can easily obtain y(1), ..., y(N) from Eqn. (2.129) by the
iterative procedure.

Problems

2.1 Consider the signal processing algorithm shown in Fig. P2.1.

(a) Assign the state variables and obtain a state variable model for the system.
(b) Represent the algorithm of Fig. P2.1 by a signal flow graph and from there obtain the transfer function
model of the system using Mason’s gain formula.
108 Digital Control and State Variable Methods

0.368

+ y (k)
r (k ) + +
0.264
– +

1.368

0.368

Fig. P 2.1

2.2 Consider the signal processing algorithm shown in Fig. P2.2. Represent the algorithm by (a) difference
equation model, (b) a state variable model, and (c) a transfer function model.

– 3 z–1 1 z–1 1

R (z) Y (z)

– 5

– 3
Fig. P 2.2

2.3 Consider the discrete-time system shown in Fig. P2.3.

–1

+
R (z) + + Y (z)
z–1 2
+
1
–
2

Fig. P 2.3
 Signal Processing in Digital Control 109
(a) Obtain the difference equation model and from there the transfer function model of the system.
(b) Find the impulse response of the system.
(c) Find the response of the system to unit step input m(k).
2.4 A filter often used as part of computer-controlled algorithms is shown in Fig. P2.4 (b and a are real constants).
(a) Find the impulse response to an impulse of
R (z) + Y (z)
b strength A.
z–1
(b) Find the step response to a step of strength A.
+
(c) Find the ramp response to a ramp function with a
a slope A.
(d) Find the response to sinusoidal input A cos(W k).
Fig. P 2.4
2.5 Solve the following difference equations:
1 3
(a) y(k + 2) + 3y(k + 1) + 2y(k) = 0; y(–1) = - , y(–2) =
2 4
(b) 2y(k) – 2y(k – 1) + y(k – 2) = r(k)
y(k) = 0 for k < 0 and

r(k) =
RS1; k = 0, 1, 2, . . .
T0; k < 0
2.6 Consider the difference equation:
y(k + 2) – 1.3679 y(k + 1) + 0.3679 y(k) = 0.3679 r(k + 1) + 0.2642 r(k)
y(k) = 0 for k ≤ 0, and r(k) = 0 for k < 0,
r(0) = 1, r(1) = 0.2142, r(2) = – 0.2142 ; r(k) = 0, k = 3, 4, 5, ...
Determine the output y(k).
2.7 Solve the following difference equation using z-transforms:
y(k) – 3y(k – 1) + 2y(k – 2) = r(k)

where r(k) =
RS1 for k = 0, 1 ; y(–2) = y(–1) = 0
T 0 for k ³ 2
Will the final value theorem give the correct value of y(k) as k → ∞ ? Why?
2.8 For the transfer function models and inputs given below, find the response y(k) as a function of k:
Y (z) 2z - 3 Y (z) - 6z + 1
(a) G(z) = = (b) G(z) = =
R( z ) ( z - 0.5) ( z + 0.3) R( z ) c
z - 1
2 hc
+ j 14 z - 1
2 - j 1
4 h
r(k) =
RS1; k =1
r(k) =
RS0; k < 0
T0; k = 0, 2, 3, 4, K T1; k = 0, 1, 2, 3, . . .
2.9 For the transfer function models and inputs given below, find the response y(k) as a function of k:
Y (z) 1 Y (z) 1
(a) G(z) = = (b) G(z) = =
R( z ) ( z - 0.5) ( z + 0.3) R( z ) ( z - 0.5)2 ( z - 0.1)

r(k) =
RS1; k even
r(k) =
RS0; k < 0
T0; k odd T1; k = 0, 1, 2, 3, K
2.10 Determine y( ∞ ) for the following Y(z) function (a is a real constant):
K[ z 3 - 2 az 2 + ( a3 - a2 + a)z ]
Y(z) =
( z - 1) ( z - a) ( z - a2 )
110 Digital Control and State Variable Methods
Assuming stable response, determine what the value of K must be for y(¥) = 1.
k !(a )k - m + 1
2.11 Given: y(k) = m (k - m + 1)
( k - m + 1) ! ( m - 1) !
Prove that y(k) decays to zero as k ® ¥ if |a| < 1.
2.12 A system has the transfer function
Y (z) 1
G(z) = = .
R( z ) z 2 + 1
Show that when the input r(k) is a unit step function, the output y(k) is bounded; and when the input
r(k) = {1, 0, –1, 0, 1, 0, –1, ...},
the output y(k) is unbounded.
Explain why a bounded input produces a bounded output in the first case but an unbounded output in the
second case.
2.13 Using Jury stability criterion, check if all the roots of the following characteristic equations lie within the unit
circle:
(a) z3 – 1.3 z2 – 0.08z + 0.24 = 0
(b) z4 – 1.368z3 + 0.4z2 + 0.08z + 0.002 = 0
2.14 Using Jury stability criterion, find if all the poles of the following transfer function lie inside the unit circle on
the z-plane.
3z 4 + 2 z 3 - z 2 + 4 z + 5
G(z) =
z + 0.5z 3 - 0.2 z 2 + z + 0.4
4

2.15 Figure P2.15 shows the input-output description of a D/A converter. The converter is designed in such a way
that the old value of the input sample is held constant, until a new sample arrives. Treating each sample of the
sequence r(kT) as an impulse function of strength equal to the value of the sample, the system of Fig. P2.15
becomes a continuous-time system. Determine the transfer function model of the system.

r (kT) y (t )
D/A

Fig. P 2.15

2.16 (a) State and prove the sampling theorem.

10(s + 2)
(b) Given: E(s) =
s 2 ( s 2 + 2s + 2)
Based upon the sampling theorem, determine the maximum value of the sampling interval T that can be used
to enable us to reconstruct e(t) from its samples.
(c) Consider a system with sampling frequency 50 rad/sec. A noise signal cos 50t enters into the system. Show
that it can cause a dc component in the system output.
2.17 Draw the magnitude and phase curves of the zero-order hold, and compare these curves with those of the ideal
low-pass filter.
2.18 Consider a signal f(t), which has discrete values f(kT) at the sampling rate 1/T. If the signal f(t) is imagined to be
impulse sampled at the same rate, it becomes
¥
f * (t ) = å f (kT )d (t - kT )
k =0
(a) Prove that F ( z) = F*(s)
z = e sT
 Signal Processing in Digital Control 111

(b) Determine F ( z) in terms of F(s). Using this result, explain the relationship between the z-plane and
z = e sT
the s-plane.
2.19 Figure P2.19 shows two root paths in the s-plane:
(i) roots with the same time-constant t = 1/a;
(ii) roots with the same oscillation frequency w0.
Derive and sketch the corresponding root paths in the z-plane under the impulse-invariant transformation.
Sampling frequency is ws.

jw s /2 jw s /2

a w0

– jw s /2 – jw s /2

(i) (ii)
Fig. P 2.19

2.20 Figure P2.20 shows a discrete-time system. Determine the transfer function G(z) of this system assuming that
the samplers operate synchronously at intervals of T sec. Also find the unit-step response of the system.

T T
r(t) r*(t) 1 y (t ) y (k )
ZOH
(s + 1)

Fig. P 2.20

2.21 Figure P2.21 shows the model of a plant driven by a D/A converter. Derive the transfer function model relating
r(kT) and y(kT); T = 0.4 sec.

r(kT) 10 y (t )
D/A
s (s + 2)

Fig. P 2.21

2.22 Show that if y is the integral of a function r, then

(i) by the backward rectangular rule for integration,
Tz
Y(z) = R(z);
z-1
(ii) by the trapezoidal rule for integration
T z+1
Y(z) = R(z)
2 z-1
112 Digital Control and State Variable Methods
2.23 Consider the discretization method based on the backward-difference approximation of derivatives, as a map-
ping from the s-plane to the z-plane. Show that the mapping transforms the left-half of s-plane into a circle in
the z-plane. Is the size of the circle dependent on the choice of the sampling interval?
2.24 Prove that the bilinear transformation maps the left-half of the s-plane into the unit circle in the z-plane.
The transformation z = esT also maps the left-half of the s-plane into the unit circle in the z-plane. What is the
difference between the two maps?
2.25 A PID controller is described by the following relation between input e(t) and output u(t):
LM t
de(t ) O
u(t) = Kc e(t ) +
MN T
1
z e( t )dt + T
I 0 dt PQ
P D

Obtain the PID control algorithm by the discretization of the equation

L
u& (t) = K Me&(t ) +
c
1 O
e(t ) + T e&&(t ) P D
N T I Q
using the backward-difference approximation of the derivatives. Also find the transfer function U(z)/E(z).
2.26 For a plant 1.57/[s(s + 1)], we are required to design a digital controller so that the closed-loop system acquires
a damping ratio of 0.45 without loss of steady-state accuracy. The sampling period T = 1.57 sec. The following
design procedure may be followed.
(i) First we design the analog controller D(s) defined in Fig. P2.26. The transfer function Gh(s) has been
inserted in the analog control loop, to take into account the effect of the hold that must be included in the
25s + 1
equivalent digital control system. Verify that D(s) = meets the design requirements.
62.5s + 1
(ii) Discretize D(s) using bilinear transformation.

r(t) + 1.57 y (t)

D (s) G h(s) = e– sT /2
s(s + 1)
–

Fig. P 2.26

2.27 A PID controller is described by the following relation between input e(t) and output u(t):
LM
U(s) = Kc 1 +
1 OP
+ TD s E ( s)
N TI s Q
(a) Derive the PID algorithm using the s-plane to z-plane maps—bilinear transformation for integration and
backward-difference approximation for the derivatives.
(b) Convert the transfer function model of the PID controller obtained in step (a) into a difference equation
model.
2.28 Derive difference equation models for the numerical solution of the following differential equation using (a) the
backward rectangular rule for integration, and (b) the forward rectangular rule for integration:
y& (t) + ay(t) = r(t); y(0) = y0
2.29 Consider the second-order system
&&
y + a y& + by = 0; y(0) = a, y& (0) = b
Approximate this equation with a second-order difference equation for computer solution. Use backward-dif-
ference approximation for the derivatives.
 Models of Digital Control
3 Devices and Systems

3.1 INTRODUCTION

Now that we have developed the prerequisite signal processing techniques in Chapter 2, we can use them to
study closed-loop digital control systems. A typical topology of the type of systems to be considered in this
chapter, is shown in Fig. 3.1.
Command signal r (k)
Controlled
output

Controlled y (t )
A/D Computer D/A system

Clock

Analog
prefilter Sensor

Fig. 3.1 Configuration of the basic digital control scheme

Digital control systems with analog sensors include an analog prefilter between the sensor and the sampler
(A/D converter) as an anti-aliasing device. The prefilters are low-pass, and the simplest transfer function is
a
Hpf (s) =
s+ a
so that the noise above the prefilter breakpoint a, is attenuated. The design goal is to provide enough attenu-
ation at half the sample rate (ws /2) so that the noise above ws /2, when aliased into lower frequencies by the
sampler, will not be detrimental to the control-system performance.
Since the phase lag from the prefilter can significantly affect system stability, it is required that the con-
trol design be carried out with the analog prefilter included in the loop transfer function. An alternative
design procedure is to select the breakpoint and ws sufficiently higher than the system bandwidth, so that the
 114 Digital Control and State Variable Methods

phase lag from the prefilter does not significantly alter the system stability, and thus the prefilter design
problem can be divorced from the control-law design problem. Our treatment of the subject is based on
this alternative design procedure. We, therefore, ignore the prefilter design and focus on the basic control-
system design. The basic configuration for this design problem is shown in Fig. 3.2, where
· G(s) = transfer function of the controlled plant (continuous-time system);
· H(s) = transfer function of the analog sensor; and
· D(z) = transfer function of the digital control algorithm.

r (k)

b (k) u (k )
A/D D (z) D/A G (s ) y (t)
A C
Computer Plant

H (s )
Sensor
Fig. 3.2 Block diagram for basic control system design

The analog and digital parts of the system are connected through D/A and A/D converters. The computer,
with its internal clock, drives the D/A and A/D converters. It compares the command signal r(k) with the
feedback signal b(k) and generates the control signal u(k), to be sent to the final control elements of the
controlled plant. These signals are computed from the digital control algorithm D(z), stored in the memory
of the computer.
There are two different approaches for the design of digital algorithms.
(i) Discretization of Analog Design
The controller design is done in the s-domain using analog design methods.1 The resulting analog
control law is then converted to discrete-time form, using one of the approximation techniques given
in Section 2.14.
(ii) Direct Digital Design
In this approach, we first develop the discrete-time model of the analog part of the loop—from C to A
in Fig. 3.2—that includes the controlled plant. The controller design is then performed using discrete-
time analysis.
An actual design process is often a combination of the two methods. First iteration to a digital design can
be obtained using discretization of an analog design. Then the result is tuned up using direct digital analysis
and design.
The intent of this chapter is to provide basic tools for the analysis and design of a control system that is to
be implemented using a computer. Mathematical models of commonly used digital control devices and
systems are developed. Different ways to implement digital controllers (obtained by the discretization of
analog design (Section 2.14) or by direct digital design (Chapter 4), are also given in this chapter.

1
Chapters 7–10 of reference [155].
 Models of Digital Control Devices and Systems 115

3.2 z-DOMAIN DESCRIPTION OF SAMPLED CONTINUOUS-TIME PLANTS

Whenever a digital computer is used to control a continuous-time plant, there must be some type of interface
system that takes care of the communication between the discrete-time and the continuous-time systems. In
the system of Fig. 3.2, the interface function is performed by A/D and D/A converters.
T Simple models of the interface actions of A/D and
f (t) f (k )
D/A converters have been developed in Chapter 2. A
brief review is in order here.
(a)
Model of an A/D Converter A simple model of an
A/D converter is shown in Fig. 3.3a. A continuous-time
0 t 0 1 2 3 4 k function f (t), t ³ 0, is the input and the sequence of real
numbers f (k), k = 0, 1, 2, ..., is the output. The following
d T (t) relation holds between input and output:
f (k) = f (t = kT); T is the time interval between samples
(b)
(3.1)
f (t) Impulse f *(t) = f (t) d T (t)
The sequence f(k) can be treated as a train of impulses
modulator represented by continuous-time function f *(t):

Fig. 3.3 A model of an A/D converter f*(t) = f (0) d (t) + f (1) d (t – T) + f (2) d (t – 2T) + L
¥
= å f (k) d (t – kT ) (3.2)
k = 0

The sampler of Fig. 3.3a can thus be viewed as an ‘impulse modulator’ with the carrier signal
¥
d T (t) = å d (t – kT ) (3.3)
k = 0
and modulating signal f (t). A schematic representation of the modulation process is shown in Fig. 3.3b.
Model of a D/A Converter A simple model of a D/A converter is shown in Fig. 3.4a. A sequence of
numbers f (k), k = 0, 1, 2, ..., is the input, and the continuous-time function f + (t), t ³ 0 is the output. The
following relation holds between input and output:
f +(t) = f (k); kT £ t < (k + 1)T (3.4)
Each sample of the sequence f(k) may be
f (k) ZOH f +(t) treated as an impulse function of the form
f (k)d (t – kT). The zero-order hold (ZOH) of
(a)
Fig. 3.4a can thus be viewed as a linear time-
invariant system that converts the impulse
0 k 0 t f (k)d (t – kT) into a pulse of height f (k) and
width T. The D/A converter may, therefore,
be modelled by Fig. 3.4b, where the ZOH is a
f (k) f *(t) 1 – e – sT f +(t) system whose response to a unit impulse d (t),
(b) G h0(s) =
T s is a unit pulse gh0(t) of width T. The Laplace
transform of gh0(t) is the transfer function of
Fig. 3.4 A model of a D/A converter hold operation, namely (refer Eqn. (2.79))
 116 Digital Control and State Variable Methods

1 - e - sT
Gh0(s) = Z [gh0(t)] = (3.5)
s
Figure 3.5 illustrates a typical example of an interconnection of discrete-time and continuous-time sys-
tems. In order to analyse such a system, it is often convenient to represent the continuous-time system
together with the ZOH by an equivalent discrete-time system.
We assume that the continuous-time system of Fig. 3.5 is a linear system with the transfer function G(s).
A block diagram model of the equivalent discrete-time system is shown in Fig. 3.6a. As seen from this
figure, the impulse modulated signal u*(t) is applied to two s-domain transfer functions in tandem. Since the
two blocks with transfer functions Gh0(s) and G(s) are not separated by an impulse modulator, we can con-
sider them as a single block with transfer function [Gh0(s)G(s)], as shown in Fig. 3.6b. The continuous-time
system with transfer function [Gh0(s)G(s)] has input u*(t) and output y(t). The output signal y(t) is read off at
discrete synchronous sampling instants kT; k = 0, 1, ..., by means of a mathematical sampler T(M).

Fig. 3.5 Interconnection of discrete-time and continuous-time systems

u (k ) u* ( t ) y ( t) y (k )
(a) G h0(s) G (s)
T T (M )

u (k ) u* ( t ) y ( t) y (k )
(b) G h0(s)G (s)
T T (M )

Fig. 3.6 Block diagrams for the equivalent discrete-time system

We assume that g$ (t) is the impulse response of the continuous-time system Gh0(s)G(s):
g$ (t) = L –1[Gh0(s)G(s)] (3.6)
The input signal to the system is given by (refer Eqn. (3.2)),
¥
u*(t) = å u(kT)d (t – kT) (3.7)
k =0
This is a sequence of impulses with intensities given by u (kT). Since g$ (t) is the impulse response of the
system (response to the input d (t)), by super-position from Eqn. (3.7),
¥
y(t) = å u(jT) g$ (t – jT)
j = 0
At the sampling times t = kT, y(t) is given by
¥
y(kT) = å u( jT) g$ (kT – jT) (3.8)
j = 0
 Models of Digital Control Devices and Systems 117
We can recognize it at once as discrete-time convolution (refer Eqn. (2.25)). Taking the z-transform of
both sides of Eqn. (3.8), we obtain (refer Eqn. (2.29)),
Y(z) = U(z) G$ (z) (3.9a)
where $
G (z) = Z [ g$ (kT )]
–1
= Z [L {Gh0(s)G(s)}|t = kT] (3.9b)
The z-transforming operation of Eqn. (3.9b) is commonly indicated as
G$ (z) = Z [Gh0(s)G(s)] = Gh0G(z) (3.10)
It may be carefully noted that since the two blocks Gh0(s) and G(s) are not separated by an impulse
modulator,
G$ (z) ¹ Z [Gh0(s)] Z [G(s)] (3.11a)
¹ Gh0(z) G(z) (3.11b)
It follows from Eqns (3.9), that the block diagram of Fig. 3.6b becomes the z-domain block diagram of
Fig. 3.7. We could, of course, directly draw the z-domain block diagram of Fig. 3.7 from Fig. 3.6b, which
implies that
Y(s) = Gh0(s)G(s)U*(s) « Y(z) = Gh0G(z)U(z) (3.12a)
or Z [Gh0(s)G(s)U*(s)] = Z [Gh0(s)G(s)] Z [U*(s)]
= Gh0G(z)U(z) (3.12b)
We can use the following relation (refer Eqns (2.100 b) and (2.100 c)) for evaluation of Gh0G(z):
Gh0G(z) = Z [Gh0(s)G(s)]
U (z ) Y (z)
G h0 G (z) = [G h0(s) G (s)] LM
= Z (1 - e - sT )
OP
G( s)
N s Q
Fig. 3.7 The z-domain equivalent of Fig. 3.6 –1
= (1 – z ) Z
LM OP
G(s )
(3.13)
N Q s
Single factor building blocks of the Laplace and z-transform pairs are given in in Table 2.1. Expanding
G(s)/s into partial fractions, Z [G(s)/s] can be found by using this table.
Consider now the basic sampled-data feedback system, whose block diagram is depicted in Fig. 3.8a. In
terms of impulse modulation, this block diagram can be redrawn as in Fig. 3.8b.

t
–

Fig. 3.8 A sampled-data feedback system

 118 Digital Control and State Variable Methods
Using the relations previously established in this section, we have
U(z) = D(z) E(z) (3.14a)
Y(z) = Gh0G(z)U(z) (3.14b)
Since e(kT) = r (kT) – y(kT)
we have E(z) = R(z) – Y(z) (3.14c)
Combining Eqns (3.14a), (3.14b), and (3.14c) gives
Y (z) D ( z ) Gh 0 G ( z )
= (3.15)
R( z ) 1 + D ( z ) G h 0 G ( z )
Figure 3.9 gives the z-domain equivalent of Fig. 3.8.
Having become familiar with the technique, from
+ now onwards we may directly write z-domain relation-
– ships, without introducing impulse modulators in block
diagrams of sampled-data systems.
Consider the sampled-data feedback system of Fig. 3.10
Fig. 3.9 The z-domain equivalent of Fig. 3.8 where the sensor dynamics is represented by transfer
function H(s). The following equations easily follow:
E(z) = R(z) – B(z) (3.16a)
U(z) = D(z) E(z) (3.16b)
Y(z) = Gh0G(z) U(z) = Z [Gh0(s)G(s)] U(z) (3.16c)
B(z) = Gh0GH(z)U(z) = Z [Gh0(s)G(s)H(s)]U(z) (3.16d)

r (t) + e (t) e (k ) u (k ) y (t)

D (z ) ZOH G (s)
T
–

b (t)
H (s)

Fig. 3.10 A sampled-data feedback system

Equations (3.16a), (3.16b) and (3.16d) give

E( z ) 1
= (3.17)
R( z) 1 + D( z ) Gh 0 GH ( z)
Combining Eqns (3.16b), (3.16c) and (3.17), we get
Y (z ) D( z )Gh 0 G( z )
= (3.18)
R( z ) 1 + D( z )Gh 0 GH ( z )
Figure 3.11 illustrates a phenomenon that we have not yet encountered. When an input signal is acted upon
by a dynamic element before being sampled, it is impossible to obtain a transfer function for the system. The
system in Fig. 3.11 differs from that in Fig. 3.10, in that the analog error e(t) is first amplified before being
converted to digital form for the control computer. The amplifier’s dynamics are given by G1(s).
 Models of Digital Control Devices and Systems 119

r (t) + e (t) e 1 (t ) e 1 (k ) u (k ) u * ( t) y (t)

G 1 (s) D (z) G h0 (s) G (s)
– T T

b (t)
H (s)

Fig. 3.11 A sampled-data system

Consider first the subsystem shown in Fig. 3.12a. We can equivalently represent it as a block [G1(s)E(s)]
with input d (t), as in Fig. 3.12b. Now the input, and therefore, the output, does not change by imagining a
fictitious impulse modulator through which d (t) is applied to [G1(s)E(s)] as in Fig. 3.12c.
On application of Eqn. (3.12), we can write
E1(z) = Z [G1(s)E(s)] Z [d (k)] = Z [G1(s)E(s)] (3.19)

e (t) e 1 ( t) e 1 (k )
(a) G 1 (s )
T

d ( t) e 1 (t) e 1 (k )
(b) G 1 (s)E (s)
T

d (t) d * ( t) = d ( t) e 1 (t ) e 1 (k)
(c) G 1 (s)E (s )
T T

Fig. 3.12 Equivalent s -transfer function with impulse modulated input

Now, for the system of Fig. 3.11,

E(s) = R(s) – B(s) = R(s) – H(s)Y(s)
= R(s) – H(s) Gh0(s) G(s) U*(s) (3.20)
Therefore, from Eqns (3.19) and (3.20), we obtain
E1(z) = Z [G1(s)R(s)] – Z [G1(s)H(s)Gh0(s)G(s)]U(z)
= G1R(z) – G1Gh0GH(z) U(z) (3.21)
Since U(z) = D(z) E1(z),
we can write from Eqn. (3.21),
G1 R( z )
E1(z) =
1 + D( z )G1Gh 0 H ( z )
The output Y(z) = Gh0G(z) U(z) = D(z) Gh0G(z) E1(z)
D( z )Gh 0 G( z )G1 R( z )
= (3.22)
1 + D( z )G1Gh 0 GH ( z)
 120 Digital Control and State Variable Methods
Since it is impossible to form the ratio Y(z)/R(z) from Eqn. (3.22), we can not obtain a transfer function for
the system of Fig. 3.11, and we cannot analyse it further without specifying a functional form for the input r(t).
It is not permissible to create a transfer function for the system in Fig. 3.11 by inserting a fictitious
sampler at the input, because this would change the physics of the situation represented by the diagram (the
analog input would be replaced by the train of impulses). Fictitious samplers are permissible at only the
output, because they are simply a means of selecting the values of the output at the times of interest to us,
namely, the sample times.

Example 3.1 Consider the sampled-data system shown in Fig. 3.13a. From the block diagram, we obtain
(refer Eqn. (3.15))
Y (z) Gh 0 G( z )
= (3.23)
R( z ) 1 + Gh 0 G( z )
Figure 3.13b gives the z-domain equivalent of Fig. 3.13a. The forward path transfer function
Gh0G(z) = Z [Gh0(s)G(s)]

= (1 – z–1) Z
LM G(s) OP = (1 – z ) Z LM 1 OP
–1
2
NsQ N s (s + 1) Q
= (1 – z–1
L 1 1 1 OP = (1 – z ) LM Tz
)ZM - + 2
–1
-
z
+
z OP
N s s s + 1Q N (z - 1) 2
z - 1 z - e -T Q
z(T - 1 + e - T ) + (1 - e - T - Te -T )
=
( z - 1) ( z - e - T )

r ( t) + 1 y ( t)
G h0(s) G (s ) =
T = 1 sec s (s + 1)
–

(a)

R (z ) + Y (z)
G h0G (z)
–
(b)

Fig. 3.13 A closed-loop sampled-data system

When T = 1, we have
ze -1 + 1 - 2e -1 0.3679 z + 0.2642
Gh0G(z) = -1
= 2
( z - 1) ( z - e ) z - 1.3679 z + 0.3679
Substituting in Eqn. (3.23), we obtain
Y (z ) 0.3679 z + 0.2642
= 2
R( z ) z - z + 0.6321
 Models of Digital Control Devices and Systems 121
For a unit-step input,
z
R(z) =
z -1
and therefore
z(0.3679z + 0.2642) 0.3679z 2 + 0.2642 z
Y(z) = =
( z - 1) ( z 2 - z + 0.6321) z 3 - 2 z 2 + 1.6321z - 0.6321
By long-division process, we get
Y(z) = 0.3679 z–1 + z–2 + 1.3996 z–3 + 1.3996 z–4 + 1.1469 z–5 + 0.8944 z–6 + 0.8015 z–7+ L
Therefore, the sequence (k = 1, 2, ...)
y(kT) = {0.3679, 1, 1.3996, 1.3996, 1.1469, 0.8944, 0.8015, ...}
Note that the final value of y(kT) is (refer Eqn.
(2.52),
0.3679 0.2642
lim y(kT) = lim (z – 1)Y(z) = =1
k®¥ z®1 0.6321
The unit-step response is shown in Fig. 3.14. Also
shown in this figure is the unit-step response of the
continuous-time system (i.e., when T = 0). The
overshoot of the sampled system is 45%, in contrast
to 17% for the continuous-time system.
The performance of the digital system is, thus,
dependent on the sampling period T. Larger sam-
Fig. 3.14 The response of a second-order system: pling periods usually give rise to higher overshoots
(a) analog; (b) sampled in the step response, and may eventually cause insta-
bility if the sampling period is too large.

Example 3.2 Let us compare the stability properties of the system shown in Fig. 3.15, with and without a
sample-and-hold on the error signal.
Without sample-and-hold, the system in Fig. 3.15 has the transfer function
Y ( s) K
=
R( s) s 2 + 2 s + K
This system is stable for all values of K > 0.

r (t) + e (t) K y (t)

G h0(s)
T s (s + 2)
–

Fig. 3.15 A closed-loop sampled-data system

 122 Digital Control and State Variable Methods
For the system with sample-and-hold, the forward-path transfer function is given by

Gh0G(z) = (1 – z–1) Z
LM K OP
2
N s (s + 2) Q
= (1 – z ) Z M F
–1 L K 1 - 1 / 2 + 1 / 2 I OP
N 2 H s s s + 2K Q
2

=
K
(1 – z ) M
L Tz - (1 / 2 )z + (1 / 2 )z OP
–1
2 -2 T
2 N (z - 1) z - 1 z - e Q
K L 2T ( z - e ) - ( z - 1) (1 - e ) O
-2 T -2 T
= M PQ -2 T
2 N 2( z - 1) ( z - e )
The characteristic equation of the sampled-data system is
1 + Gh0G(z) = 0
–2T –2T
or 4(z – 1)(z – e ) + 2KT(z – e ) – K(z – 1) (1 – e–2T) = 0

or z2 + z
LM 1 K F T - 1 + 1 e I - 1 - e OP + e
-2 T -2 T -2 T
+
1
K
LM
1 1 -2T
- e
OP
- Te -2T = 0
N2 H 2 2 K Q 2 N
2 2 Q
Case 1: T = 0.4 sec
For this value of sampling period, the characteristic polynomial becomes
D(z) = z2 + (0.062K – 1.449)z + 0.449 + 0.048K
Applying the Jury stability test (refer Eqns (2.73)–(2.75)), we find that the system is stable if the follow-
ing conditions are satisfied:
D(1) = 1 + 0.062K – 1.449 + 0.449 + 0.048K > 0
D(–1) = 1 – 0.062K + 1.449 + 0.449 + 0.048K > 0
| 0.449 + 0.048K | < 1
These conditions are satisfied for 0 < K < 11.479.
Case II: T = 3 sec
For this value of sampling period, the characteristic polynomial becomes
D(z) = z2 + (1.2506K – 1.0025) z + 0.0025 + 0.2457K
The system is found to be stable for 0 < K < 1.995.
Thus, the system which is stable for all K > 0 when T = 0 (continuous-time system), becomes unstable for
K > 11.479 when T = 0.4 sec. When T is further increased to 3 sec, it becomes unstable for K > 1.995. It
means that increasing the sampling period (or decreasing the sampling rate), reduces the margin of stability.

3.3 z-DOMAIN DESCRIPTION OF SYSTEMS WITH DEAD-TIME

Figure 3.16 is the block diagram of a computer-controlled continuous-time system with dead-time. We
assume that the continuous-time system is described by transfer function of the form
Gp(s) = G(s) e -tD s (3.24)
where tD is the dead-time, and G(s) contains no dead-time.
 Models of Digital Control Devices and Systems 123

Digital u (k) y (t) y (k)

ZOH G (s)e–tDs
computer T (M )

Equivalent discrete-time system

Fig. 3.16 A sampled continuous-time system with dead-time

The equivalent discrete-time system, shown by dotted lines in Fig. 3.16, is described by the model
Y (z)
= Z [Gh0(s)Gp(s)] = Gh0Gp(z) (3.25a)
U(z)

= (1 – z–1) Z LM1 e - t Ds
G(s) OP (3.25b)
Ns Q
If N is the largest integer number of sampling periods in tD, we can write
tD = NT + DT; 0 £ D < 1 (3.26)
(Given the capabilities of modern microprocessors, we can adjust the sampling frequency slightly so that tD = NT)

Therefore, Gh0Gp(z) = (1 – z–1)z– N Z LM1 e - D Ts

G (s) OP
Ns Q
Let us take an example where
1
G(s) = (3.27)
s+a

For this example,

Y (z)
= Gh0 Gp(z) = (1 – z–1)z– N Z
e - D Ts LM OP
U (z) s( s + a ) N Q
=
1
(1 – z–1) z– N Z
LM
e - D Ts
-
e - D Ts OP (3.28)
a s N s+a Q
Now L –1
LM e OP = g (t) = m(t – DT); L LM e OP = g (t) = e
- D Ts
1
–1
- D Ts
2
–a(t – DT )
m(t – DT)
N s Q Ns + a Q
where m(t) is a unit-step function.
Therefore, g1(kT ) = m(kT – DT ); g2(kT ) = e–a(kT – DT) m(kT – DT )
¥
Z [g1(kT)] = å g1(kT )z– k = z–1 + z–2 + z–3 + L
k = 0

= z–1 (1 + z–1 + z–2 + L) = z–1 F 1 I= 1 (3.29)

H1 - z K z - 1
-1
 124 Digital Control and State Variable Methods
¥
Z [g2(kT )] = å g2(kT )z–k = e–a(T – DT) z–1 + e–a(2T – DT) z–2 + e–a(3T – DT) z–3 + L
k = 0

We introduce a parameter m, such that

m=1–D
– amT –1
Then Z [g2(kT)] = e z + e– amT e– aT z–2 + e–amT e–2aT z–3 + L
= e–amT z–1 [1 + e–aT z–1 + e–2aT z–2 + L]

= e–amT z–1 LM 1 OP
N1 - e
- aT -1
z Q
e - amT
= (3.30)
z - e - aT
Substituting the z-transform results given by Eqns (3.29) and (3.30) in Eqn. (3.28), we get
Y (z) 1
= Gh0Gp(z) = (1 – z–1) z–N
1
-
LM
e - amT OP
U(z) a z - 1 z - e - aT
N Q
1
a
d i
1 - e - amT z + e - amT - e - aT
= (3.31)
z N +1 ( z - e - aT )
Table 3.1 has been generated by applying the procedure outlined above, to commonly occurring functions.

Table 3.1 Table of transform pairs for systems with dead-time

Laplace transform z-transform

F(s)e–DTs; 0 £ D < 1 Z [F(s)e–DTs]; m = 1 – D

e -DTs 1
s z -1

e -DTs mT T
+
s2 z - 1 ( z - 1)2
2e -DTs
T2
LM m z2 2
+ (2m - 2m2 + 1)z + ( m - 1)2 OP
s3 N ( z - 1)3 Q
e - DTs e - amT
s+a z - e - aT
e - DTs 1 e - amTLM -
e- bmT OP
-
( s + a ) ( s + b) ( b - a) z - e aT
N z - e - bT Q
ae- DTs (1 - e- amT ) z + (e- amT - e - aT )
s( s + a ) ( z - 1) ( z - e - aT )
ae - DTs T amT - 1 e - amT
+ +
2
s (s + a) ( z - 1)2 a( z - 1) a( z - e - aT )
 Models of Digital Control Devices and Systems 125

Example 3.3 The scheme of Fig. 3.17 produces a steady-stream flow of fluid with controlled temperature
q. A stream of hot fluid is continuously mixed with a stream of cold fluid, in a mixing valve. The valve
characteristic is such that, the total flow rate Q (m3/sec) through it is maintained constant, but the inflow qi
(m3/sec) of hot fluid may be linearly varied by controlling valve stem position x. The valve stem position x,
thus controls the temperature qi (ºC) of the outflow from the mixing valve. Due to the distance between the
valve and the point of discharge into the tank, there is a time delay between the change in qi and the discharge
of the flow with the changed temperature, into the tank.

x
Q i + qi

Negligible heat loss in walls

Hot
Q, qi + qi

Flow control qid + qid

Q, q + q
Cold V

Fig. 3.17 Tank fluid temperature control

The differential equation governing the tank temperature is (assuming an initial equilibrium and taking all
variables as perturbations)
dq
Vrc = Qr c (qid – q) (3.32)
dt
where
q = tank fluid temperature, ºC
= temperature of the outflowing fluid from the tank;
c = specific heat of the fluid, Joules/(kg)(ºC);
V = volume of the fluid in the tank, m3;
r = fluid density, kg/m3;
Q = fluid flow rate, m3/sec; and
qid = temperature of the fluid entering the tank, ºC.
The temperature qid at the input to the tank at time t, however, is the mixing valve output temperature tD
seconds in the past, which may be expressed as
qid(t) = qi(t – tD) (3.33)
From Eqns (3.32)–(3.33), we obtain
q& (t) + aq (t) = aqi(t – tD)
where a = Q/V
 126 Digital Control and State Variable Methods

q (s ) ae -tD
s
Therefore, Gp(s) = = (3.34)
q i ( s) s+a
To form the discrete-time transfer function of Gp(s) preceded by a zero-order hold, we must compute
LMF 1 - e I ae OP
- sT -t Ds
Gh0Gp(z) = Z
MNGH s JK s + a PQ
–1
= (1 – z ) Z M
L a e OP -t Ds
(3.35)
N s( s + a ) Q
For the specific values of tD = 1.5, T = 1, a = 1, Eqn. (3.35) reduces to
–1
G G (z) = (1 – z ) Z M
h0 p
L 1 e e OP - s -0.5 s
N s(s + 1) Q
–1 –1
= (1 – z ) z Z M
L 1 e OP -0.5s
N s(s + 1) Q
Using transform pairs of Table 3.1, we obtain

Gh0Gp(z) = (1 – z–1) z–1

LM (1 - e )z + (e
-0.5 -0.5
- e -1 ) OP
-1
N (z - 1) (z - e ) Q
0.3935( z + 0.6066) q (z)
= = (3.36)
z 2 ( z - 0.3679) q i (z)
The relationship between x and qi is linear, as is seen below.
( Qi + qi) rcqH + [Q – ( Qi + qi)] rcqC = Qrc (q i + qi)
where qH and qC are constant temperatures of hot and cold streams respectively.
qi = Kv x
where Kv is the valve gain.
The perturbation equation is obtained as (neglecting second-order terms in perturbation variables),
Kv(qH – qC) x(t) = Qqi(t)
or x(t) = Kqi(t); K = Q/[Kv(qH – qC)]
Therefore,
q( z ) (0.3935/ K )(z + 0.6066)
=
X (z ) z 2 ( z - 0.3679)

3.4 IMPLEMENTATION OF DIGITAL CONTROLLERS

The application of conventional 8- and 16-bit microprocessors to control systems, is now well established.
Such processors have general purpose architectures which make them applicable for a wide range of tasks,
though none are remarkably efficient. In control applications, such devices may pose problems such as
inadequate speed, difficulties with numerical manipulation, and relatively high cost for the completed sys-
tem; the latter being due to both the programming effort and the cost of the peripheral hardware (memories,
I/O ports, timers/counters, A/D converters, D/A converters, PWM circuit, etc.).
 Models of Digital Control Devices and Systems 127
In applications requiring small amounts of program ROM, data RAM, and I/O ports, single-chip
microcontrollers are ideally suited. In these chips, the capabilities in terms of speed of computation, on-chip
resources, and software facilities are optimized for control applications. Should the on-chip features be
insufficient to meet control requirements, the microcontroller chips allow for easy expansion.
The Intel microcontroller family (MCS-48 group, MCS-51 group, MCS-96 group) includes 8- and 16-bit
processors with the following on-chip resources—ROM, RAM, I/O lines, timer/counter, A/D converter, and
PWM output. The Motorola microcontroller family (HC 05 group, HC 11 group, HC 16 group) also pro-
vides microcontroller chips with similar features.
In many application areas, processing requirements for digital control systems, such as execution time
and algorithm complexity, have increased dramatically. For example, in motor control, short sampling time
constraints can place exacting requirements on algorithm execution time. New airframe designs and ex-
tended aircraft performance envelopes, increase the complexity of flight control laws. Controller complexity
also increases with number of interacting loops (e.g., in robotics), or the number of sensors (e.g., in vision
systems). For a growing number of real-time control applications, conventional single-processor systems are
unable to satisfy the new demands for increased speed and greater complexity and flexibility.
The dramatic advances in VLSI technology leading to high transistor packing densities have enabled
computer architects to develop parallel-processing architectures consisting of multiple processors; thus real-
izing high-performance computing engines at relatively low cost. The control engineer can exploit a range of
architectures for a variery of functions.
Parallel-processing speeds up the execution time for a task. This is achieved by dividing the problem into
several sub-tasks, and allocating multiple processors to execute multiple sub-tasks simultaneously. Parallel
architectures differ from one another in respect of nature of interconnectivity between the processing ele-
ments and the processing power of each individual processing element.
The transputer is a family of single-chip computers which incorporates features to support parallel process-
ing. It is possible to use a network of transputers to reduce the execution time of a real-time control law.
Digital signal processors (DSPs) offer an alternative strategy for implementation of digital controllers.
DSPs use architectures and dedicated arithmetic circuits, that provide high resolution and high speed arith-
metic, making them ideally suited for use as controllers.
Many DSP chips, available commercially, can be applied to a wide range of control problems. The Texas
Instruments TMS 320 family provides several beneficial features through its architecture, speed, and
instruction set.
TMS 320 is designed to support both numeric-intensive operations, such as required in signal processing,
and also general purpose computation, as would be required in high speed control. It uses a modified archi-
tecture, which gives it speed and flexibility—the program and data memory are allotted separate sections on
the chip, permitting a full overlap of the instruction fetch and execution cycle. The processor also uses
hardware to implement functions which had previously been achieved using software. As a result, a multi-
plication takes only 200 nsec, i.e., one instruction cycle, to execute. Extra hardware has also been included
to implement shifting and some other functions. This gives the design engineer the type of power previously
unavailable on a single chip.
Implementation of a control algorithm on a computer consists of the following two steps:
(i) Block diagram realization of the transfer function (obtained by the discretization of analog controller
(Section 2.14), or by the direct digital design (Chapter 4) that represents the control algorithm.
(ii) Software design based on the block diagram realization.
In the following, we present several different structures of block diagram realizations of digital control-
lers using delay elements, adders, and multipliers. Different realizations are equivalent from the input-output
point of view if we assume that the calculations are done with infinite precision. With finite precision in the
 128 Digital Control and State Variable Methods

calculations, the choice of the realization is very important. A bad choice of the realization may give a
controller that is very sensitive to errors in the computations.
Assume that we want to realize the controller
U (z) b z n + b 1z n -1 + L + b n -1z + b n
D(z) = = 0n (3.37a)
E ( z) z + a 1 z n -1 + L + a n -1z + a n
where the ai’s and bi’s are real coefficients (some of them may be zero).
Transfer functions of all digital controllers can be rearranged in this form. For example, the transfer
function of PID controller, given by Eqn. (2.113b), can be rearranged as follows:

D(z) =
LM
U(z)
= Kc 1 +
T F1 I T
+ D (1 - z -1 )
OP
N
E( z ) H
TI 1 - z -1K T Q
KTF 1 c I + KT
=K + c D
(1 – z–1)
T H1 - z K T
c -1
I
2
b 0 z + b 1z + b 2
=
z 2 + a 1z + a 2
where
a1 = –1; a2 = 0

FG
b0 = Kc 1 +
T T IJ F2T KTI
+ D ; b1 = – Kc 1 + D ; b2 = c D
H TI T K H T TK
We shall now discuss different ways of realizing the transfer function (3.37a), or equivalently the transfer
function
U ( z ) b 0 + b 1z -1 + b 2 z -2 + L + b n -1z -( n -1) + b n z - n
D(z) = = (3.37b)
E( z ) 1 + a 1z -1 + a 2 z -2 + L + a n -1 z -( n -1) + a n z - n
The methods for realizing digital systems of the form (3.37) can be divided into two classes—recursive
and nonrecursive. The functional relation between the input sequence e(k) and the output sequence u(k) for
a recursive realization has the form
u(k) = f (u(k – 1), u(k – 2),..., e(k), e(k – 1), ...) (3.38)
For the linear time-invariant system of Eqn. (3.37b), the recursive realization has the form
u(k) = – a1u(k – 1) – a2u(k – 2) – L – anu(k – n) + b0e(k) + b1e(k – 1) + L + bne(k – n) (3.39)
The current output sample u(k) is a function of past outputs and present and past input samples. Due to the
recursive nature, the errors in previous outputs may accumulate.
The impulse response of the digital system defined by Eqn. (3.39), where we assume not all ai’s are zero,
has an infinite number of non-zero samples, although their magnitudes may become negligibly small as k
increases. This type of digital system is called an infinite impulse response (IIR) system.
The input-output relation for a nonrecursive realization is of the form
u(k) = f (e(k), e(k – 1), ...) (3.40a)
 Models of Digital Control Devices and Systems 129
For a linear time-invariant system, this relation takes the form
u(k) = b0e(k) + b1e(k – 1) + b2e(k – 2) + L + bN e(k – N) (3.40b)
The current output sample u(k) is a function only of the present and past values of the input.
The impulse response of the digital system defined by Eqn. (3.40b), is limited to a finite number of
samples defined over a finite range of time intervals, i.e., the impulse response sequence is finite. This type
of digital system is called a finite impulse response (FIR) system.
The digital controller given by Eqn. (3.37b) is obviously an FIR digital system when the coefficients ai
are all zero. When not all ai’s are zero, we can obtain FIR approximation of the digital system by dividing its
numerator by the denominator and truncating the series at z–N; N ³ n:
U(z)
= D(z) @ a0 + a1z–1 + a2z–2 + L + aN z–N; N ³ n (3.41)
E( z )
Notice that we may require a large value of N to obtain a good level of accuracy.
In the following, we discuss the most common types of recursive and nonrecursive realizations of digital
controllers of the form (3.37).

3.4.1 Recursive Realizations

The transfer function (3.37) represents an nth order system. Recursive realization of this transfer function
will require at least n unit delayers. Each unit delayer will represent a first-order dynamic system. Each of
the three recursive realization structures given below, uses the minimum number (n) of delay elements in
realizing the transfer function (3.37).
Direct Realization Let us multiply the numerator and denominator of the right-hand side of Eqn. (3.37b)
by a variable X(z). This operation gives
U ( z ) ( b 0 + b 1z -1 + b 2 z -2 + L + b n -1z - ( n -1) + b n z - n ) X ( z )
= (3.42)
E( z ) (1 + a 1z -1 + a 2 z -2 + L + a n -1z -( n -1) + a n z - n ) X ( z )
Equating the numerators on both sides of this equation gives
U(z) = (b0 + b1 z–1 + L + bn z– n) X(z) (3.43a)
The same operation on the denominator brings
E(z) = (1 + a1 z–1 + L + an z– n) X(z) (3.43b)
In order to construct a block diagram for realization. Eqn. (3.43b) must first be written in a cause-and-
effect relation. Solving for X (z) in Eqn. (3.43b) gives
X(z) = E(z) – a1 z–1 X(z) – L – an z–n X(z) (3.43c)
A block diagram portraying Eqns (3.43a) and (3.43c) is now drawn in Fig. 3.18 for n = 3. Notice that we
use only three delay elements. The coefficients ai and bi (which are real quantities) appear as multipliers. The
block diagram schemes where the coefficients ai and bi appear directly as multipliers are called direct structures.
Basically, there are three sources of error that affect the accuracy of a realization (Section 2.1):
(i) the error due to the quantization of the input signal into a finite number of discrete levels;
(ii) the error due to accumulation of round-off errors in the arithmetic operations in the digital system; and
(iii) the error due to quantization of the coefficients a i and b i of the transfer function. This error may
become large for higher-order transfer functions. That is, in a higher-order digital controller in direct
structure, small errors in the coefficients a i and b i cause large errors in the locations of the poles and
zeros of the controller (refer Review Example 3.3).
 130 Digital Control and State Variable Methods

b0

b1

b2

E (z) X (z) + U (z)

+ z–1 z–1 z–1 b3 +
–

a1

a2

a3

Fig. 3.18 Block diagram realization of the transfer function of Eqn. (3.37) with n = 3

These three errors arise because of the practical limitations of the number of bits that represent various
signal samples and coefficients. The third type of error listed above may be reduced by mathematically
decomposing a higher-order transfer function into a combination of lower-order transfer functions. In this
way, the system may be made less sensitive to coefficient inaccuracies.
For decomposing higher-order transfer functions in order to reduce the coefficient sensitivity problem,
the following two approaches are commonly used. It is desirable to analyse each of these structures for a
given transfer function, to see which one is better with respect to the number of arithmetic operations
required, the range of coefficients, and so forth.
Cascade Realization The sensitivity problem may be reduced by implementing the transfer function
D(z) as a cascade connection of first-order and/or second-order transfer functions. If D(z) can be written as
a product of transfer functions D1(z),... , Dm(z), i.e.,
D(z) = D1(z) D2(z) L Dm(z),
then a digital realization for D(z) may be obtained by a cascade connection of m component realizations for
D1(z), D2(z), ... , and Dm(z), as shown in Fig. 3.19.

E (z) U (z)
D 1 (z) D 2 (z) D m (z)

Fig. 3.19 Cascade decomposition of D(z)

In most cases, the Di(z); i = 1, 2, ..., m, are chosen to be either first-order or second-order functions. If the
poles and zeros of D(z) are known, then Di(z) can be obtained by grouping real poles and real zeros to
produce first-order functions, or by grouping a pair of complex-conjugate poles and a pair of complex-
conjugate zeros to produce a second-order function. It is, of course, possible to group two real poles with a
pair of complex-conjugate zeros and vice versa. The grouping is, in a sense, arbitrary. It is desirable to group
several different ways, to see which one is best with respect to the number of arithmetic operations required,
the range of coefficients, and so forth.
 Models of Digital Control Devices and Systems 131
In general, D(z) may be decomposed as follows:
p 1 + bi z -1 m 1 + ej z -1 + f j z -2
D(z) = P -1 P
i = 1 1 + ai z j= p + 1 1 + cj z -1 + dj z -2
The block diagram for
1 + bi z -1 U (z )
Di(z) = = i
1 + ai z -1 Ei ( z )
1 + ej z -1 + fj z -2 Uj ( z )
and that for Dj(z) = -1 -2 =
1 + cj z + dj z Ej ( z )
are shown in Figs 3.20a and 3.20b, respectively. The realization for the digital controller D(z) is a cascade
connection of p first-order systems of the type shown in Fig. 3.20a, and (m – p) second-order systems of the
type shown in Fig. 3.20b.

E i (z) + U i (z)
+ z–1 bi +
–
ai
(a)

ej

E j (z) + U j (z)
+ z–1 z–1 fj +
–
cj

dj
(b)
Fig. 3.20 Realizations of (a) first-order and (b) second-order functions

Parallel Realization Another approach to reduce the coefficient sensitivity problem is to expand the
transfer function D(z) into partial fractions. If D(z) is expanded so that
D(z) = A + D1(z) + D2(z) + L + Dr(z),
where A is simply a constant, then a digital realization for D(z) may be obtained by a parallel connection of
(r + 1) component realizations for A, D1(z), ... , Dr(z), as shown in Fig. 3.21. Due to the presence of the
constant term A, the first-order and second-order functions can be chosen in simpler forms:
q
b r ej + fj z -1
D(z) = A + å 1 + ai z -1 + å + cj z -1 + dj z -2
i =1 i j = q + 11
The block diagram for
bi Ui (z )
Dj(z) = -1 =
1 + ai z E( z )
 132 Digital Control and State Variable Methods

E (z) + U (z)
D 1 (z) +
+

Dr (z)

Fig. 3.21 Parallel decomposition of D(z)

and that for

ej + fj z -1 Uj ( z )
Dj(z) = -1 -2 =
1 + cj z + dj z E( z)
are shown in Figs 3.22a and 3.22b, respectively.

ei
fj
U i (z )
E (z) + U j (z)
bi + z–1 z–1
–
E (z)
+ z–1 cj
–
dj
ai
(a) (b)
Fig. 3.22 Realization of (a) first-order and (b) second-order functions

3.4.2 Nonrecursive Realizations

Nonrecursive structures for D(z) are similar to the recursive structures presented earlier in this section. In the
nonrecursive form, the direct and cascade structures are commonly used; the parallel structure is not used
since it requires more elements.
 Models of Digital Control Devices and Systems 133

Example 3.4 Consider the digital controller with transfer function model

U ( z ) 2 - 0.6 z -1
D(z) = =
E( z ) 1 + 0.5z -1
Recursive realization of D(z) yields the block diagram shown in Fig. 3.23a. By dividing the numerator of
D(z) by the denominator, we obtain
D(z) = 2 – 1.6z–1 + 0.8z–2 – 0.4z–3 + 0.2z–4 – 0.1z–5 + 0.05z–6 – 0.025z–7 +L
Truncating this series at z–5, we obtain the following FIR digital system:
U(z)
= 2 – 1.6 z–1 + 0.8 z–2 – 0.4 z–3 + 0.2 z–4 – 0.1 z–5
E( z )
Figure 3.23b gives a realization for this FIR system. Notice that we need a large number of delay elements
to obtain a good level of accuracy. An advantage of this realization is that, because of the lack of feedback,
the accumulation of errors in past outputs is avoided in the processing of the signal.

– +
+ –
–

– – – – –

– –

+ +

Fig. 3.23 (a) Recursive realization (b) nonrecursive realization of a first-order function

3.5 TUNABLE PID CONTROLLERS

The ultimate goal of control systems engineering is to build real physical systems to perform some specified
tasks. To accomplish this goal, design and physical implementation of a control strategy are required. The
 134 Digital Control and State Variable Methods
standard approach to design is this: A mathematical model is built making necessary assumptions about
various uncertain quantities on the dynamics of the system. If the objective is well defined in precise math-
ematical terms, then control strategies can be derived mathematically (e.g., by optimizing some criterion of
performance). This is the basis of all model-based control strategies. This approach is feasible when it is
possible to specify the objective and the model, mathematically. Many sophisticated methods based on
model-based control approach will appear in later chapters of the book.
For motion control applications (position, and speed control systems), identification of mathemati-
cal models of systems close enough to reality is usually possible. However, for process control applica-
tions (pressure, flow, liquid-level, temperature, and composition control systems), identification of
process dynamics precisely can be expensive even if meaningful identification is possible. This is
because industrial processes are relatively slow and complex. In process-control field, therefore, it is
not uncommon to follow adhoc approach for controller development, when high demands on control-
system performance are not made. In the adhoc approach, we select a certain type of controller based
on past experience with the process to be controlled, and then set controller parameters by experiment
once the controller is installed. The ‘experimental design’ of controller settings has come to be known
as controller tuning.
Many years of experience have shown that a PID controller is versatile enough to control a wide variety
of industrial processes. The common practice is to interface a PID controller (with adjustment features) to
the process and adjust the parameters of the controller online, by trial-and-error, to obtain acceptable per-
formance. A number of tuning methods have been introduced to obtain fast convergence to control solution.
These methods consist of the following two steps:
(i) experimental determination of the dynamic characteristic of the control loop; and
(ii) estimation of the controller tuning parameters that produce a desired response for the dynamic charac-
teristics determined in first step.
It may be noted that for tuning purposes, simple experiments are performed to estimate important dynamic
attributes of the process. The approximate models have proven to be quite useful for process control appli-
cations (For processes whose dynamics are precisely known, the use of trial-and-error tuning is not justified
since many model-based methods to the design of PID controllers are available which predict the controller
parameters fairly well at the design stage itself). The predicted parameter values based on approximate
models simply provide initial trial values for the online trial-and-error approach. These trial values may turn
out to be a poor guess. Fine tuning the controller parameters online is usually necessary to obtain acceptable
control performance.
Some of the tuning methods which have been successfully used in process industry, will be described here.

3.5.1 Analog PID Controllers

Approximately 75% of feedback controllers in the process industry are PI controllers; most of the balance
are PID controllers. Some applications require only P, or PD controllers, but these are few.
Proportional Controller The equation that describes the proportional controller is:
u(t) = Kce(t) (3.44a)
or U(s) = KcE(s) (3.44b)
where Kc is the controller gain, e is the error, and u is the perturbation in controller output signal from the
base value corresponding to the normal operating conditions; the base value on the controller is adjusted to
produce zero error under the conditions of no disturbance and/or set-point change.
 Models of Digital Control Devices and Systems 135
Some instrument manufacturers calibrate the controller gain as proportional band (PB). A 10% PB means
that a 10% change in the controller input causes a full-scale (100%) change in controller output. The conver-
sion relation is thus
100
Kc = (3.45)
PB
A proportional controller has only one adjustable or tuning parameter: Kc or PB.
A proportionally controlled process with no integration property will always exhibit error at steady-state
in the presence of disturbances and changes in set-point. The error, of course, can be made negligibly small
by increasing the gain of the proportional controller. However, as the gain is increased, the performance of
the closed-loop system becomes more oscillatory and takes longer to settle down after being disturbed.
Further, most process plants have a considerable amount of dead-time, which severely restricts the value of
the gain that can be used. In processes where the control within a band from the set-point is acceptable,
proportional control is sufficient. However, in processes which require perfect control at the set-point,
proportional controllers will not provide satisfactory performance [155].
Proportional-Integral Controller To remove the steady-state offset in the controlled variable of a
process, an extra amount of intelligence must be added to the proportional controller. This extra intelligence
is the integral or reset action, and consequently, the controller becomes a PI controller. The equation
describing a PI controller is as follows:
LM 1
t O
u(t) = Kc e( t ) +
MN T z e( t ) dt P
I 0 PQ
(3.46a)

or U(s) = Kc
LM1 + 1 OP E(s) (3.46b)
N T sQ I
where TI is the integral or reset time.
A PI controller has thus two adjustable or tuning parameters: Kc (or PB) and TI. The integral or reset
action in this controller removes the steady-state offset in the controlled variable. However, the integral
mode of control has a considerable destabilizing effect which, in most of the situations, can be compensated
by adjusting the gain Kc [155].
Some instrument manufacturers calibrate the integral mode parameter as the reset rate, which is simply
the reciprocal of the reset time.
Proportional-Integral-Derivative Controller Sometimes a mode faster than the proportional mode is
added to the PI controller. This new mode of control is the derivative action, also called the rate action, which
responds to the rate of change of error with time. This speeds up the controller action. The equation describing
the PID controller is as follows:
LM 1
t
de( t ) OP
MN T z
u(t) = Kc e(t ) +
e(t ) dt + T D (3.47a)
I 0 dt PQ
or
L 1 + T sOP E(s)
U(s) = K M1 +
c D (3.47b)
N Ts Q I
where TD is the derivative or rate time.
A PID controller has thus three adjustable or tuning parameters: Kc (or PB), TI, and TD. The derivative
action anticipates the error, initiates an early corrective action, and tends to increase the stability of the
system. It does not affect the steady-state error directly. A derivative control mode, in isolation, produces no
 136 Digital Control and State Variable Methods
corrective effort for any constant error, no matter how large, and would, therefore, allow uncontrolled steady-
state errors. Thus, we cannot consider derivative modes in isolation; they will always be considered as
augmenting some other mode [155].
The block diagram implementation of Eqn. (3.47b) is sketched in Fig. 3.24a. The alternative form,
Fig. 3.24b, is more commonly used, because it avoids taking the rate of change of the set-point input to the
controller, thus preventing the undesirable derivative ‘kick’ on set-point changes by the process operator.
Due to the noise-accentuating characteristics of derivative operation, the lowpass-filtered derivative
TD s/(aTD s + 1) is actually preferred in practice (Fig. 3.24c). The value of the filter parameter a is not
adjustable but is built into the design of the controller. It is usually of the order of 0.05 to 0.3.

1
TI s
+
r + e + u
Kc
– +
TD s
y

(a)

1
TI s
+
r + e + u
Kc
– –
TD s
y
(b)

1
TI s
+
r + e + u
Kc
– –
TD s
a TD s + 1
y
(c)
Fig. 3.24 Block diagram of PID controller

The controller of Fig. 3.24 is considered to be noninteracting in that its derivative and integral modes
operate independently of each other (although proportional gain affects all the three modes). Noninteraction
is provided by the parallel functioning of integral and derivative modes. By contrast, many controllers have
derivative and integral action applied serially to the controlled variable, resulting in interaction between
them. Many of the analog industrial controllers realize the following interacting PID control action.
 Models of Digital Control Devices and Systems 137

U(s) = K c¢
LM T ¢ s + 1 OP LM1 + 1 OP E(s)
D
(3.48)
NaT ¢ s + 1Q N T ¢s Q
D I
The first term in brackets is a derivative unit attached to the standard PI controller serially, to create the PID
controller (Fig. 3.25a). The derivative unit installed on the controlled-variable input to the controller avoids
the derivative kick (Fig. 3.25b).

1
T¢I s
+
r + e TD¢ s + 1 + u
K¢c
aTD¢ s + 1
–

y (a)

1
TI¢ s

r + e + + u
K¢c
–
b
¢ s +1
TD
a TD¢ s + 1

y (b)
Fig. 3.25 Alternative realization schemes for PID control action

Most commercially available tunable controllers use the noninteracting version of PID control. The
discussion that follows applies to noninteracting PID control.

3.5.2 Adjustment Features in Industrial Controllers

A great number of manufacturers are now making available in the market, process controllers (electronic,
and computer-based) with features that permit adjusting the set-point, transferring between manual and
automatic control modes, adjusting the output signal from the control-action unit (tuning the parameters Kc,
TI, and TD), and displaying the controlled variable, set-point, and control signal. Figure 3.26 shows the basic
structure of an industrial controller. The controller has been broken down into three main units:
(i) The set-point control unit
(ii) The PID control unit
(iii) The manual/automatic control unit.
The set-point control unit receives the measurement y of controlled variable of the process, together with
the set-point r of the control. A switch gives an option of choosing between local and remote (external) set-
point operation. If the set-point to the controller is to be set by the operating personnel, then the local option
rL is chosen. If the set-point to the controller is to be set by another control module, then remote (external)
option re is chosen. This is the case, for example, in cascade control where the drive of the controller in the
major loop constitutes the set-point of the minor-loop controller.
 138 Digital Control and State Variable Methods

+
–

Fig. 3.26 Basic structure of an industrial controller

The PID control unit receives the error signal e developed by the set-point control unit, and generates an
appropriate control signal uC. Adjustment features provided in the control unit, for generating appropriate
control signals, include tuning of the three parameters Kc, TI, and TD.
The manual/automatic control unit has a switch which determines the mode of control action. When the
switch is in the auto (A) position, the control signal uC calculated by PID control unit is sent to the process
(in such a case, the process is controlled in closed loop). When the switch is in the manual (M) position, the
PID control unit ‘freezes’ its output. The control signal uM can then be changed manually by the operating
personnel (the process is then controlled in open loop).
The basic structure of a process controller shown in Fig. 3.26 is common for electronic, and computer-
based controllers. These controllers are different in terms of realization of adjustment features.

3.5.3 Ziegler–Nichols Tuning Method Based on Ultimate Gain and Period

This pioneer method, also known as the closed-loop or on-line tuning method, was proposed by J.G. Ziegler
and N.B. Nichols around 1940. In this method, the parameters by which the dynamic characteristics of the
process are represented are the ultimate gain and period. These parameters are used in tuning the controller
for a specified response: the quarter-decay ratio (QDR) response.
Determination of Ultimate Gain and Period When the process is under closed-loop P control, the
gain of the P controller at which the loop oscillates with constant amplitude, has been defined as the ultimate
gain Kcu. Ultimate period Tu is the period of these sustained oscillations. The ultimate gain is, thus, a meas-
ure of difficulty in controlling a process; the higher the ultimate gain, the easier it is to control the process
loop. The ultimate period is, in turn, a measure of speed of response of the loop; the larger the period, the
slower the loop.
By its definition, it can be deduced that the ultimate gain is the gain at which the loop is at the threshold
of instability. At gains just below the ultimate, the loop signals will oscillate with decreasing amplitude, and
at gains above the ultimate, the amplitude of the oscillations will grow with time.
For experimental determination of Kcu and Tu, the controller is set in ‘auto’ mode and the following
procedure is followed (refer Fig. 3.26).
(i) Remove the integral mode by setting the integral time to its highest value. Alternatively, if the PID
controller allows for switching off the integral mode, switch it off.
(ii) Switch off the derivative mode, or set the derivative time to its lowest value, usually zero.
(iii) Increase the proportional gain in steps. After each increase, disturb the loop by introducing a small step
change in set-point and observe the response of the controlled variable, preferably on a trend recorder.
 Models of Digital Control Devices and Systems 139
The controlled variable should start oscillating as the gain is increased. When the amplitude of the oscil-
lations remains approximately constant, the ultimate controller gain has been reached. Record it as Kcu.
(iv) Measure the period of the oscillations from the trend recording. This parameter is Tu.
The procedure just outlined is simple and requires a minimum upset to the process, just enough to be able
to observe the oscillations. Nevertheless, the prospect of taking a process control loop to the verge of insta-
bility is not an attractive one from a process operation standpoint.
Tuning for Quarter-Decay Ratio Response Ziegler and Nichols proposed that the parameters Kcu and
Tu, characterizing a process, be used in tuning the controller for QDR response. The QDR response is
illustrated in Fig. 3.27 for a step change in disturbance, and for a step change in set-point. Its characteristic is
that each oscillation has an amplitude that is one-fourth of the previous oscillation.
Empirical relations [12] for calculating the QDR tuning parameters of P, PI and PID controllers, from the
ultimate gain Kcu and period Tu, are given in Table 3.2.
y (t)
A

1 A
4

(a) Disturbance input

y (t)
A
1 A
4

t
(b) Set-point change
Fig. 3.27 QDR response
PI and PID tuning parameters that produce quarter-decay response, are not unique. For each setting of the
integral and derivative times, there will usually be a setting of the controller gain that produces quarter-
decay response. The settings given in Table 3.2 are the figures based on experience; these settings have
produced fast response for most industrial loops.
Table 3.2 QDR tuning formulas based on ultimate gain and period

Controller Gain Integral time Derivative time

P Kc = 0.5 Kcu — —
PI Kc = 0.45 Kcu TI = Tu /1.2 —
PID Kc = 0.75 Kcu TI = Tu /1.6 TD = Tu /10
 140 Digital Control and State Variable Methods

3.5.4 Ziegler–Nichols Tuning Method Based on Process Reaction Curve

Although the tuning method based on ultimate gain and period is simple and fast, other methods of charac-
terizing the dynamic response of feedback control loops have been developed over the years. The need for
these alternative methods is based on the fact that, it is not always possible to determine the ultimate gain and
period of a loop; some loops would not exhibit sustained oscillations with a proportional controller. Also,
the ultimate gain and period do not give insight into which process or control system characteristics could be
modified to improve the feedback controller performance. A more fundamental method of characterizing
process dynamics is needed to guide such modifications. In the following, we present an open-loop method
for characterizing the dynamic response of the process in the loop.
Process Reaction Curve Process control is characterized by systems which are relatively slow and
complex and which, in many cases, include an element of pure time delay (dead-time). Even where a dead-
time element is not present, the complexity of the system which will typically contain several first-order sub-
systems, will often result in a process reaction curve (dynamic response to a step change in input), which has
the appearance of pure time delay.
Process reaction curve may be obtained by carry-
u (t) ing out the following step-test procedure.
With the controller on ‘manual’, i.e., the loop
opened (refer Fig. 3.26), a step change of magnitude
u
Du in the control signal u(t) is applied to the process.
The magnitude Du should be large enough for the con-
sequent change Dy(t) in the process output variable to
y ( t) be measurable, but not so large that the response will
be distorted by process nonlinearities. The process
yss output is recorded for a period from the introduction
of the step change in the input, until the process
reaches a new steady-state.
0
A typical process reaction curve is sketched in
Fig. 3.28 A process reaction curve Fig. 3.28. The most common model used to character-
ize the process reaction curve is the following:
Y ( s) Ke -t Ds
= G(s) = (3.49)
U ( s) ts + 1
where K = the process steady-state gain;
tD = the effective process dead-time; and
t = the effective process time-constant.
This is a first-order plus dead-time model. The model response for a step change in the input signal of
magnitude Du, is given by
Ke -t Ds Du LM 1 t OP
Y(s) = = KDu e -t D s - (3.50)
ts + 1 s N s ts + 1 Q
Inverting with the help of a transform table (Table 2.1), and applying the real translation theorem of Laplace
transforms L [ y (t – t0)m(t – t0)] = e–st0 Y (s); t0 > 0 [155], we get
t - t D )/ t
Dy(t) = KDu [1 - e - ( ] ; t > tD
(3.51)
=0 ; t £ tD
 Models of Digital Control Devices and Systems 141
The term Dy is the perturbation or change in the output from its initial value:
Dy(t) = y(t) – y(0)
Figure 3.29 shows the model response to a step change of magnitude Du in the input signal. Dyss is the steady-
state change in the process output (refer Eqn. (3.51)):
Dyss = lim Dy(t) = KDu
t®¥

y (t)

h yss = K u
0.632 yss

0 tD tD + t t

Fig. 3.29 Step-response of first-order plus dead-time model

At the point t = tD on the time axis, the process output variable leaves the initial steady-state with a maximum rate of
change (refer Eqn. (3.51)):
d
Dy(t ) FI
= KDu
1
=
Dyss
dt t =t D
HKt t
The time-constant t is the distance on the time axis between the point t = tD, and the point at which the
tangent to the model response curve, drawn at t = tD, crosses the new steady-state.
Note that the model response at t = tD + t is given by
Dy (tD + t) = KDu(1 – e–1) = 0.632 Dyss
The process reaction curve of Fig. 3.28 can be matched to the model response of Fig. 3.29 by the follow-
ing estimation procedure.
The model parameter K is given by
Change in process output at steady-state Dyss
K= = (3.52)
Step change in process input Du
The estimation of the model parameters tD and t can be done by, at least, three methods; each of which
results in different values.
Tangent Method This method makes use of the line that is tangent to the process reaction curve at the
point of maximum rate of change. The time-constant is then defined as the distance on the time axis, between
the point where the tangent crosses the initial steady-state of the output variable, and the point where it
crosses the new steady-state value. The dead-time is the distance on the time axis, between the occurrence of
the input step change, and the point where the tangent line crosses the initial steady-state. These estimates are
indicated in Fig. 3.30a.
Tangent-and-Point Method In this method, tD is determined in the same manner as in the earlier method,
but the value of t is the one that forces the model response to coincide with the actual response at t = tD + t.
Construction for this method is shown in Fig. 3.30b. The value of t obtained by this method is usually less
than that obtained by the earlier method, and the process reaction curve is usually closer to the response of the
model obtained by this method compared to the one obtained by the earlier method.
 142 Digital Control and State Variable Methods
y (t) Two-points Method The least precise step in
the determination of tD and t by the previous two
methods, is the drawing of the line tangent to the
process reaction curve at the point of maximum rate
of change. To eliminate this dependence on the tan-
0 t gent line, it is proposed that the values of tD and t
tD t be selected such that the model and the actual
(a) Tangent method
response coincide at two points in the region of
high rate of change. The two points recommended
y (t)
are (tD + 13 t ) and (tD + t ). To locate these points,
we make use of Eqn. (3.51):
yss
0.632 yss Dy(tD + 13 t ) = KDu[1 – e–1/3] = 0.283 Dyss
Dy(tD + t ) = KDu [1 – e–1] = 0.632 Dyss
0 t These two points are labelled t 1 and t 2 in
tD t
Fig. 3.30c. Knowing t1 and t2, we can obtain the
(b) Tangent-and-point method values of tD and t.
y (t) tD + t = t2; tD + 13 t = t1
which reduces to
3
h yss t = 2
(t2 – t1); tD = t2 – t (3.53)
0.632 h yss where t1 = time at which Dy(t) = 0.283 Dyss
0.283 yss
0 t1 t2 t t2 = time at which Dy(t) = 0.632 Dyss
(c) Two-points method
Tuning for QDR Response Besides the formulas
Fig. 3.30 Estimation of model parameters for QDR response tuning based on the ultimate gain
and period of the loop (refer Table 3.2), Ziegler and
Nichols also developed tuning formulas based on the parameters of a first-order model fit to the process reaction
curve. These formulas are given in Table 3.3 [12].
Ke - t D s
Table 3.3 QDR tuning formulas based on process reaction curve: process model G(s) =
ts + 1
Controller Gain Integral time Derivative time
P Kc = t /KtD — —
PI Kc = 0.9t /KtD TI = 3.33 tD —
PID Kc = 1.5t /KtD TI = 2.5tD TD = 0.4tD

Three major conclusions can be drawn from this table.

(i) The controller gain is inversely proportional to the process gain K which represents the product of gain
of all the elements in the loop other than the controller. It means that, if the gain of any of the elements
were to change because of recalibration, resizing, or nonlinearity, the response of the feedback loop
will change, unless the controller gain is readjusted.
(ii) The controller gain must be reduced when the ratio of the process dead-time to its time-constant,
increases. This means, that the difficulty in controlling the loop increases when the ratio of the process
dead-time to its time-constant, increases. This ratio, which can be used as a measure of difficulty in
controlling a process, will be called the normalized dead-time tND.
 Models of Digital Control Devices and Systems 143
Apparent dead-time t D
= Normalized dead-time tND (3.54)
Apparent time-constant t
tND can be estimated from the process reaction curve. Processes with small tND are easy to control and
processes with large tND are difficult to control. The parameter tND has been called the controllability
ratio in the literature. To avoid confusion with the standard terminology of modern control theory
(Chapter 5), the word normalized dead-time will be used here.
Notice that having a long dead-time parameter means that the loop is difficult to control only if the
time-constant is short. In other words, a loop with a dead-time of several minutes, would be just as
difficult to control as one with a dead-time of a few seconds—if the normalized dead-time for both the
loops is the same.
(iii) The speed of response of the controller, which is determined by integral and derivative times, must
match the speed of response of the process. The formulas in Table 3.3 match these response speeds by
relating the integral and derivative times of the controller to the process dead-time.
In using the formulas of Table 3.3, we must keep in mind that they were developed empirically for the
most common range of the normalized dead-time parameter, which is between 0.1 and 0.3, based on the fact
that most processes do not exhibit significant transportation lag (rather, the dead-time is the result of several
first-order lags in series).
As was pointed out in the earlier discussion on QDR tuning based on ultimate gain and period, the diffi-
culty of the QDR performance specification for PI and PID controllers is that there is an infinite set of values
of the controller parameters that can produce it; i.e., for each setting of the integral time on a PI controller,
and for each reset-derivative time combination on a PID controller, there is a setting of the gain that results
in QDR response. The settings given in Table 3.3 are the figures based on experience; these settings have
produced fast response for most industrial loops.

3.5.5 Digital PID Controllers

Most process industries today, use computers to carry out the basic feedback control calculations. The
formulas that are programmed to calculate the controller output are mostly the discrete versions of the
analog controllers presented earlier in this section. This practice allows the use of established experience
with analog controllers and, in principle, their well-known tuning rules which could be applied.
As there is no extra cost in programming all the three modes of control, most computer-based algorithms
contain all the three, and then use flags and logic to allow the process engineer to specify any signal mode or,
a combination of two or three modes. Most tunable commercially available controllers use the noninteracting
version of PID control (refer Eqn. (3.47b)). The discussion that follows applies to noninteracting PID control.
Position PID Algorithm The equation describing the idealized noninteracting PID controller is as
follows (refer Eqn. (3.47a)):
LM 1
t
de( t ) OP
u(t) = Kc e( t ) +
MN TI z
0
e( t ) dt + TD
dt PQ
(3.55)

with parameters
Kc = controller gain; TI = integral time; and TD = derivative time.
For small sample times T, this equation can be turned into a difference equation by discretization. Various
methods of discretization were presented in Section 2.14.
 144 Digital Control and State Variable Methods
Approximating the derivative mode by the backward-difference approximation and the integral mode by
backward integration rule, we obtain (refer Eqns (2.112))
LM
u(k) = Kc e(k ) +
1 T
S(k ) + D (e(k ) - e(k - 1))
OP (3.56)
N TI T Q
S(k) = S(k – 1) + Te(k)
where u(k) = the controller output at sample k;
S(k) = the sum of the errors; and
T = the sampling interval.
This is a nonrecursive algorithm. For the formation of the sum, all past errors e(×) have to be stored.
Equation (3.56) is known as the ‘absolute form’ or ‘position form’ of the PID algorithm. It suffers from one
particular disadvantage which is manifest when the process it is controlling, is switched from manual to auto-
matic control. The initial value of the control variable u will simply be (e(k – 1) = S(k – 1) = 0 in Eqn. (3.56))

u(0) = Kc 1 +
TLM T
+ D e(0)
OP
TIN T Q
Since the controller has no knowledge of the previous sample values, it is not likely that this output value
will coincide with that previously available under manual control. As a result, the transfer of control will
cause a ‘bump’, which may seriously disturb the plant operation. This can only be overcome by laboriously
aligning the manual and computer outputs or, by adding complexity to the controller so that it will automati-
cally ‘track’ the manual controller.
Practical implementation of the PID algorithm includes the following additional features.
(i) It is seldom desirable for the derivative mode of the controller to respond to set-point changes. This is
because the set-point changes cause large changes in the error that last for only one sample; when the
derivative mode acts on this error, undesirable pulses or ‘derivative kicks’ occur on the controller
output—right after the set-point is changed. These pulses, which last for one sampling interval, can be
avoided by having the derivative mode act on the controlled variable, rather than on the error.
(ii) A pure derivative term should not be implemented, because it will give a very large amplification of
the measurement noise. The gain of the derivative must thus be limited. This can be done by approxi-
mating the transfer function TD s as follows:
TD s
TD s @
a TD s + 1
where a is the filter parameter, whose value is not adjustable but is built into the design of the
controller. It is usually of the order of 0.05 to 0.3.
The PID controller, therefore, takes the form (refer Fig. 3.24c):
LM
U(s) = Kc E (s) +
1
E( s ) -
TsD O
Y ( s )P (3.57)
N Ts I aT s + 1
D Q
Discretization of this equation results in the following PID algorithm:
L
u(k) = K Me(k ) +
c
1 O
S(k ) + D(k )P (3.58)
N T I Q
S(k) = S(k – 1) + Te(k)
a TD TD
D(k) = D(k – 1) – [ y(k) – y(k – 1)]
aTD + T aTD + T
 Models of Digital Control Devices and Systems 145
Velocity PID Algorithm This is a recursive algorithm characterized by the calculation of the current
control variable u(k) based on the previous control variable u(k – 1) and correction terms. To derive the
recursive algorithm, we subtract from Eqn. (3.56)
LM
u(k – 1) = Kc e(k - 1) +
1 T
S( k - 1) + D (e(k - 1) - e(k - 2))
OP (3.59)
N TI T Q
This gives
LM
u(k) – u(k – 1) = Kc e(k ) - e(k - 1) +
T T
e(k ) + D [e( k ) - 2e(k - 1) + e( k - 2)]
OP (3.60)
N TI T Q
Now, only the current change in the control variable
Du(k) = u(k) – u(k – 1) (3.61)
is calculated. This algorithm is known as the ‘incremental form’ or ‘velocity form’ of the PID algorithm. The
distinction between the position and velocity algorithms is significant only for controllers with integral effect.
The velocity algorithm provides a simple solution to the requirement of bumpless transfer. The problem
of bumps arises mainly from the need for an ‘initial condition’ on the integral; and the solution adopted is to
externalize the integration as shown in Fig. 3.31. The external integration may take the form of an electronic
integrator but frequently the type of actuating element is changed, so that recursive algorithm is used with
actuators which, by their very nature, contain integral action. Stepper motor (refer Section 3.8) is one such
actuating element.

Incremental Auto
e (k )
PID
algorithm u (k) u (k )
1
s

Manual
Fig. 3.31 Control scheme of bumpless transfer
Practical implementation of this algorithm includes the features of avoiding derivative kicks and filtering
measurement noise. Using Eqn. (3.58) we obtain
LM
Du(k) = Kc e(k ) - e(k - 1) +
T
e( k ) -
T
D( k - 1) -
TD
( y(k ) - y(k - 1))
OP (3.62)
N TI a TD + T a TD + T Q
a TD TD
D(k) = D( k - 1) - [ y(k ) - y(k - 1)] (3.63)
a TD + T a TD + T
where y(k) = controlled variable; Du(k)= incremental control variable = u(k) – u(k – 1);
e(k) = error variable; Kc = controller gain;
TI = integral time; TD = derivative time; and T= sampling interval.
Tuning Rules for Digital Controllers Though tuning formulas that are specifically applicable to digital
control algorithms have been developed [12], the most popular and widely used tuning approach for digital
PID controllers is to apply rules of Tables 3.2–3.3 with a simple correction to account for the effect of
sampling. When a continuous-time signal is sampled at regular intervals of time, and is then reconstructed by
 146 Digital Control and State Variable Methods

holding the sampled values constant for each sampling interval, the reconstructed signal is effectively delayed
by approximately one half of the sampling interval, as shown in Fig. 3.32a (also refer Example 2.17). In the
digital control configuration of Fig. 3.32b, the D/A converter holds the output of the digital controller
constant between updates, thus adding one half the sampling time to the dead-time of the process components.
The correction for sampling is then, simply, to add one half the sampling time to the dead-time obtained from
the process reaction curve.
1
tCD = tD + 2 T (3.64)
where tCD is the corrected dead-time, tD is the dead-time of the process, and T is the sampling interval.
Continuous
signal
Reconstructed
signal

T
2

T 2T 3T 4T 5T 6T t

(a)

r (k) + e (k) Control u (k ) y (t)

algorithm D/A Process
–

A/D

(b)
Fig. 3.32 Time delay introduced by sampling and reconstruction

The tuning formulas given in Table 3.3 can directly be used for digital PID controllers with tD replaced by tCD.
Notice that the online tuning method, based on ultimate gain and period, inherently incorporates the effect of
sampling when the ultimate gain and period are determined with the digital controller included in the loop.
Tuning rules of Table 3.2 can, therefore, be applied to digital control algorithms without any correction.

3.6 DIGITAL TEMPERATURE CONTROL SYSTEM

This section describes the hardware features of the design of a microprocessor-based controller for tempera-
ture control in an air-flow system.
Figure 3.33 shows the air-flow system, provided with temperature measurement and having a heater grid
with controlled power input. Air, drawn through a variable orifice by a centrifugal blower, is driven past the
 Models of Digital Control Devices and Systems 147
heater grid and through a length of tubing, to the atmosphere again. The temperature sensing element con-
sists of a bead thermistor fitted to the end of a probe inserted into the air stream 30 cms from the heater. The
task is to implement a controller, in the position shown by dotted box, to provide temperature control of the
air stream. It is a practical process control problem in miniature, simulating the conditions found in furnaces,
boilers, air-conditioning systems, etc.
Adjustable orifice

Process Heated air

Heater Thermistor

Gate Bridge and

control Computer A/D amplifier
circuit circuit
AC
supply Reference
signal

Fig. 3.33 Temperature control in an air-flow system

The functions within the control loop can be broken down as follows:
(a) sampling of the temperature measurement signal at an appropriate rate;
(b) transfer of the measurement signal into the computer;
(c) comparison of the measured temperature with a stored desired temperature, to form an error signal;
(d) operation on the error signal by an appropriate algorithm, to form an output signal; and
(e) transfer of the output signal, through the interface, to the power control unit.

3.6.1 Digital Measurement/Control Schemes

Figure 3.34 gives hardware description of the temperature control system. Let us examine briefly the func-
tion of each block. The block labelled keyboard matrix, interfaced to the microcomputer through a program-
mable keyboard/display interface chip, enables the user to feed reference input to the temperature control
system. The LED display unit provides display of the actual temperature of the heating chamber.
The temperature range for the system under consideration is 20 to 60ºC. When a thermistor is used as
temperature sensor, it is necessary to convert the change in its resistance to an equivalent analog voltage.
This is accomplished with Wheatstone bridge; the thermistor exposed to the process air, forms one arm of
the bridge. The millivolt range of the bridge error voltage is amplified to the range required by A/D con-
verter. The output of the A/D converter is the digital measurement of the actual temperature of the process
air. This data is fed to the microcomputer through an input port. The microcomputer compares the actual
temperature with the desired temperature at each sampling instant, and generates an error signal. The error
signal is then processed as per the control algorithm (to be given later), resulting in a control signal in digital
form. The control signal is, in fact, the amount of power required to be applied to the plant, in order to reduce
the error between the desired temperature and the actual temperature. The power input to the plant may be
controlled with the help of triacs and firing circuit interface.
 148 Digital Control and State Variable Methods
Programmable
timer /counter
Triac control
pulses
Keyboard
matrix Micro-
Triacs and
computer
LED firing Heater
display circuit

Programmable Input port

keyboard/
display
A/D
interface Heating
chamber

Amplifier Wheatstone
bridge

Fig. 3.34 Block diagram of the temperature control system

A basic circuit using a triac (bidirectional thyristor) which controls the flow of alternating current through
the heater is shown in Fig. 3.35a. If the triac closes the circuit for tp seconds out of T seconds, the average
power applied to the plant over the sampling period T is
t
1 V2
p
V 2 tp
u=
T 0 R z
dt =
R T
:

V = rms value of the voltage applied to the heater,

R = resistance of the heater.
u
This gives tp = 2 T (3.65)
V /R
Depending on the control signal u (power required to be applied to the plant), tp is calculated in the
microcomputer. A number is latched in a down counter (in the programmable timer/counter chip interfaced
to the microcomputer) which is determined by the value of tp and the counter’s clock frequency. A pulse
of required width tp is thus available at each sampling instant from the programmable timer/counter chip.
This, in fact, is a pulse width modulated (PWM) wave whose time period is constant and width is varied in
accordance with the power required to be fed to the plant (Fig. 3.35b).
Heater

R
230 V
50 Hz
Triac control
pulses 0 tp T 2T Time
(a) (b)

Fig. 3.35 Control scheme for the thermal process

 Models of Digital Control Devices and Systems 149
The function of the triacs and firing circuit interface is thus, to process the PWM output of the microcom-
puter, such that the heater is ON when the PWM output is logic 1, and OFF when it is logic 0. Since the
heater is operated off 230 V ac at 50 Hz, the firing circuit should also provide adequate isolation between the
high voltage ac signals and the low voltage digital signals.

3.6.2 Control Algorithm

A model for the temperature control system under study is given by the block diagram of Fig. 3.36. A gain of
unity in the feedback path corresponds to the design of feedback circuit (temperature transducer + amplifier
+ A/D converter) which enables us to interpret the magnitude of the digital output of A/D converter directly
as temperature in ºC. The temperature command is given in terms of the digital number with magnitude equal
to the desired temperature in ºC. The error e (ºC) is processed by the control algorithm with transfer function
D(z). The computer generates a PWM wave whose time period is equal to the sampling interval, and width
is varied in accordance with the control signal u (watts). The PWM wave controls the power input to the
plant through the triacs and the firing circuit interface. Since the width of PWM remains constant over a
sampling interval, we can use S/H to model the input-output relation of the triacs and the firing circuit
interface.

r + e u y
D (z) G h0(s) G (s)
T
–
Controller Process
Triacs and firing circuit

Digital number °C
1

Feedback circuit
Fig. 3.36 A model for the temperature control system

To develop the digital controller D(z) for the process, we will follow the approach of controller tuning
(refer Section 3.5). A simple tuning procedure consists of the following steps:
(i) Obtain experimentally the dynamic characteristics of the process, either by open-loop or closed-loop tests.
(ii) Based on dynamic characteristics of a process, tuning rules have been developed by Ziegler and
Nichols (refer Tables 3.2–3.3). Use these rules to obtain initial settings of the controller parameters
Kc, TI, and TD of the PID controller
1
D(s) = Kc 1 +
LM
+ TD s
OP (3.66)
TI s N Q
(iii) Discretize the PID controller to obtain digital control algorithm for the temperature control process.
Thumb rules given in Section 2.13 may be followed for initial selection of sampling interval T.
In digital mode, the PID controller takes the form (refer Eqn. (2.125))
LM
D(z) = Kc 1 +
T F z + 1I + T F z - 1I OP = U(z)
D
(3.67)
N 2TI H z - 1K T H z K Q E(z)
(iv) Implement the digital PID controller. Figure 3.37 shows a realization scheme for the controller; the
proportional, integral, and derivative terms are implemented separately and summed up at the output.
(v) Fine tune Kc, TI, TD and T to obtain acceptable performance.
 150 Digital Control and State Variable Methods

E (z) + + U (z )
Kc T / (2 T I) + z– 1 + +
+ +

TD / T z– 1 –
+

Fig. 3.37 A realization scheme for the PID controller

An open-loop test was performed on the air-flow system (Fig. 3.33) to obtain its dynamic characteristics.
Input : heater power
Output : air temperature
The test was carried out with a dc input signal. A wattmeter, on the input side, measured the heater power,
and a voltmeter, on the output side, measured the output (in volts) of the bridge circuit, which is proportional
to the air temperature in ºC.
Figure 3.38 shows the response for a step input of 20 watts. This process reaction curve was obtained for
a specific orifice setting.

40

30

20
Volts

24.8
10

0 10 20 30 40
t (seconds)
Fig. 3.38 Process reaction curve of the air-flow system

Approximation of the process reaction curve by a first-order plus dead-time model is obtained as follows
(refer Fig. 3.29):
The change in the process output at steady-state is found to be Dyss = 24.8 volts. Therefore the process gain
24.8
K= = 1.24 volts/watt
20
The line that is tangent to the process reaction curve at the point of maximum rate of change gives tD =
0.3 sec. The time at which the response is 0.632Dyss is found to be 0.83 sec. Therefore t + tD = 0.83; which
gives t = 0.53 sec. (It may be noted that the response is oscillatory in nature; therefore, a second-order model
will give a better fit. However, for coarse tuning, we have approximated the response by a first-order plus
dead-time model).
 Models of Digital Control Devices and Systems 151
The process reaction curve of the air-flow system is thus represented by the model
Ke -tDs . e - 0.3s
124
G(s) = = (3.68)
ts + 1 0.53 s + 1
Taking a sampling interval T = 0.1 sec, we have (refer Eqn. (3.64))
1
tCD = tD + T = 0.35 sec
2
Using tuning formulas of Table 3.3, we obtain the following parameters for the PID controller.
Kc = 1.5t /(KtCD) = 1.832
TI = 2.5t CD = 0.875 (3.69)
TD = 0.4tCD = 0.14

3.7 DIGITAL POSITION CONTROL SYSTEM

This section describes hardware features of the design of a microprocessor based controller for a position
control system. The plant of our digital control system is an inertial load, driven by an armature-controlled
dc servomotor. The plant also includes a motor-drive circuit. The output of the drive circuit is fed to the
armature of the motor which controls the position of the motor shaft. In addition, it also controls the direc-
tion of rotation of the motor shaft.
Figure 3.39 gives hardware description of the position control system. Let us examine briefly the function
of each block.
Input port Output port

Digital
signal Micro-
Pre-
generator computer D/A amplifier

Commanded position
Input port Servo-
amplifier

Controlled position

Shaft Servomotor
encoder +
Load

Fig. 3.39 Block diagram of the digital positioning system

The block labelled digital signal generator, interfaced to the microcomputer through an input port,
enables the user to feed the desired position (set point) of the motor shaft. A keyboard matrix can be used for
entering numerical commands into the digital system.
The microcomputer compares the actual position of the motor shaft with the desired position at each
sampling instant, and generates an error signal. The error signal is then processed as per the control algo-
rithm (to be given later), resulting in a control signal in digital form. The digital control signal is converted
 152 Digital Control and State Variable Methods
to a bipolar (can be + ve or –ve) analog voltage in the D/A converter interfaced to the microcomputer. This
bipolar signal is processed in preamplifier and servoamplifier (power amplifier), enabling the motor to be
driven in one direction for positive voltage at preamplifier input, and in opposite direction for a negative
voltage.
With these units, the block diagram of Fig. 3.39 also shows a shaft encoder for digital measurement of
shaft position/speed. We now examine in detail the principle of operation of this digital device.

3.7.1 Digital Measurement of Shaft Position/Speed

The digital measurement of shaft position requires conversion from the analog quantity ‘shaft angle’ to a
binary number. One way of doing this would be to change shaft angle to a voltage using a potentiometer, and
then to convert it to a binary number through an electronic A/D converter. This is perfectly feasible, but is
not sensible because:
(i) high quality potentiometers of good accuracy are expensive and subject to wear; and
(ii) the double conversion is certain to introduce more errors than a single conversion would.
We can go straight from angle to number, using an optical angular absolute-position encoder. It consists
of a rotary disk made of a transparent material. The disk is divided into a number of equal angular sectors—
depending on the resolution required. Several tracks, which are transparent in certain sectors but opaque in
others, are laid out. Each track represents one digit of a binary number. Detectors on these tracks sense
whether the digit is a ‘1’ or a ‘0’. Figure 3.40 gives an example. Here, the disk is divided into eight 45º
sectors. To represent eight angles in binary code requires three digits (23 = 8), hence, there are three tracks.
Each track has a light source sending a beam on the disk and, on the opposite side, a photoelectric sensor
receiving this beam. Depending upon the angular sector momentarily facing the sensors, they transmit a bit
pattern representing the angular disk position. For example, if the bit pattern is 010, then Sector IV is facing
the sensors.
VIII I
100 000
Detectors

VII II
101 001

VI III
111 011

V IV
110 010
Fig. 3.40 Absolute encoder
 Models of Digital Control Devices and Systems 153
Figure 3.40 is an example of an ‘absolute encoder’, so called because for a given angle the digital output
must always be the same. Note that a cyclic (Gray) binary code is normally used on absolute encoders (in
cyclic codes, only one bit changes between adjacent numbers). If a natural binary-code pattern were used, a
transition from, say 001 to 010, would produce a race between the two right-hand bits. Depending on which
photosensor responded faster, the output would go briefly through 011 or 000. In either case, a momentary
false bit pattern would be sent. Cyclic codes avoid such races. A cyclic code can be converted into natural
binary code by either hardware or computer software.
Encoders similar to Fig. 3.40 have been widely used. But they have certain disadvantages.
(i) The resolution obtainable with these encoders is limited by the number of tracks on the encoder disk.
The alignment of upto ten detectors and the laying out of ten tracks is still quite difficult and thus
expensive.
(ii) The resulting digital measurement is in a cyclic code and must usually be converted to natural binary
before use.
(iii) The large number of tracks and detectors, inevitably increases the chance of mechanical and/or electri-
cal failure.
For these reasons, another form of encoder is commonly used today and is known as the incremental
encoder. The basis of an incremental encoder is a single track served by a single detector, and laid out in
equal segments of ‘0’ and ‘1’, as in Fig. 3.41. As the track moves relative to the detector, a pulse train is
generated, and can be fed to a counter to record how much motion has occurred. With regard to this scheme
of measurement, the following questions may be raised.
(i) How do we know which direction the motion was?
(ii) If we can record only the distance moved, how do we know where we were?
Detector

Track motion
relative to
Counter
detector

Fig. 3.41 Incremental encoder

The answer to the first question involves the addition of a second detector. Figure 3.42a shows two
detectors, spaced one half of a segment apart. As the track moves relative to the detectors (we assume at a
constant rate), the detector outputs vary with time, as shown in the waveforms of Fig. 3.42b. We can see that
the relative ‘phasing’ of the A and B signals depends upon the direction of motion, and so gives us a means
of detecting the direction.
For example, if signal B goes from ‘0’ to ‘1’ while signal A is at ‘1’, the motion is positive. For the same
direction, we see that B goes from ‘1’ to ‘0’ whilst A is at ‘0’. For negative motion, a similar but different pair
of statements can be made. By application of some fairly simple logic, it is possible to control a reversible
counter as is indicated in Fig. 3.43
This method of direction-sensing is referred to as quadrature encoding. The detectors are one half of a
segment apart, but reference to the waveforms of Fig. 3.42 shows that there are two segments to one cycle;
so the detectors are one-quarter of a cycle apart, and hence the name.
 154 Digital Control and State Variable Methods
Segment

– +
Track motion
A B relative to
(a) detector

A
0

1
Positive
motion B
0

1
Negative
motion B
0
Time proportional to steady-state motion
(b)
Fig. 3.42 Quadrature encoding

Encoder
A Up

Logic Reversible
counter
Down
B

Quadrature outputs
Fig. 3.43 Direction-detecting circuit for an incremental encoder

The solution to the second problem also requires an additional detector working on a datum track as
shown in Fig. 3.44. The datum resets the counter every time it goes by.
We have thus three detectors in an incremental encoder. But this is still a lot less than on an absolute
encoder.
In an analog system, speed is usually measured by a tachogenerator attached to the motor shaft. This is
because the time differentiation of analog position signal presents practical problems.
 Models of Digital Control Devices and Systems 155
Segment

A B Datum track

Datum detector
Encoder
A Up

Logic Reversible
counter
Down
B

Reset
Datum output

Fig. 3.44 Datum detector in an incremental encoder

In a digital system, however, it is relatively easy to carry out step-by-step calculation of the ‘slope’ of the
position/time curve. We have the position data in digital form from the shaft encoder, so the rest is fairly
straightforward.
A B Encoder Interface Implementation In a position
Preset control control system, the load is connected to the motor through
a gear train. The encoder may be connected to the load
shaft/motor shaft directly or through a pulley system.
D Reversible To know the shaft position, the number of pulses
Q U/D obtained at the output of detector A or detector B have
CLK counter
to be counted. To know the direction of rotation, the
Digital output relative phasing of the outputs of detectors A and B
has to be sensed. To implement the direction sensing,
Fig. 3.45 A simple encoder interface circuit a negative edge-triggered D-flipflop may be used
(Fig. 3.45). This flipflop has two inputs:
(i) clock input, derived from the output of detector A of the encoder; and
(ii) ‘D’ input, derived from the output of detector B of the encoder.
Every time the flipflop is triggered on the 1 ® 0 transition of waveform A, the output of the D flipflop is
either 1 or 0, depending on the direction of rotation of the shaft. The output of the D flipflop thus serves as
the control for the up/down input of the counter, reversing the direction of its count whenever the shaft
reverses its direction.
3.7.2 Control Algorithm
The mathematical model of the position control under study is given by the block diagram of Fig. 3.46. The
magnitude of the digital output of the shaft encoder can be interpreted directly as the position of the motor
 156 Digital Control and State Variable Methods
shaft in degrees, by proper design of the encoder interface. Similarly, the magnitude of the digital reference
input can be interpreted directly as reference input in degrees, by proper design of the keyboard matrix
interface. The error e (degrees) in position is processed by the control algorithm with transfer function D(z).
The control signal u (in volts) is applied to the preamplifier through the D/A converter. The plant
(preamplifier + servoamplifier + dc motor + load) is described by the transfer function
q ( s) 94
= G(s) = (3.70)
V ( s) s(0.3s + 1)

r + e u V q
D (z) G h0(s) G (s )
T
– Volts
Controller D/A Plant

Digital number Degrees

1
Shaft encoder
Fig. 3.46 Mathematical model for the position control system

To design the digital controller D(z) for this plant, we will follow the approach of discretization of analog
design (refer Section 2.14). The design requirements may be fixed as z = 0.7 and wn @ 10. The first step is to
find a proper analog controller D(s) that meets the specifications. The transfer function
( s + 3.33)
D(s) = Kc
s+a
cancels the plant pole at s = –3.33. The characteristic roots of
1 + D(s)G(s) = 0
give z = 0.7 and wn = 10 if we choose Kc = 0.32 and a = 14.
0.32(s + 3.33)
The controller D(s) = (3.71)
s + 14
gives the following steady-state behaviour:
Kv = lim sG(s)D(s) = 7.15
s®0
This may be considered satisfactory.
The discretized version of the controller D(s) is the proposed digital controller D(z) for the control loop
of Fig. 3.46. The D(z) will perform as per the specifications if the lagging effect of zero-order hold is
negligible. We take a small value for sampling interval T to satisfy this requirement. For a system with wn =
10 rad/sec, a very ‘safe’ sample rate would be—a factor of 20 faster than wn, yielding
ws = 10 ´ 20 = 200 rad/sec
2p
and T= @ 0.03 sec
ws
The dominant time constant of the plant is 0.3 sec. The sampling interval T is one-tenth of this value.
We use the bilinear transformation given by

s=
2 z-1 F I
T z +1H K
 Models of Digital Control Devices and Systems 157
to digitize D(s). This results in
22.4 z - 20.27
D(z) =
80.67z - 52.67
0.278 - 0.25z -1 U ( z )
= = (3.72a)
1 - 0.653z -1 E( z )
The control algorithm is, therefore, given by
u(k) = 0.653 u(k – 1) + 0.278 e(k) – 0.25 e(k – 1) (3.72b)
This completes the digital algorithm design.

3.8 STEPPING MOTORS AND THEIR CONTROL

The explosive growth of the computer industry in recent years has also meant an enormous growth for
stepping motors, because these motors provide the driving force in many computer peripheral devices. Step-
ping motors can be found, for example, driving the paper-feed mechanism in printers. These motors are also
used exclusively in floppy disk drives, where they provide precise positioning of magnetic head on the disks.
The X and Y coordinate pens in plotters, are driven by stepping motors.
The stepping motor can be found performing countless tasks outside the computer industry as well. The
most common application is probably in analog quartz watches where tiny stepping motors drive the hands.
These motors are also popular in numerical-control applications (positioning of the workpiece and/or the
tool in a machine according to previously specified numerical data).
A stepping motor is especially suited for applications mentioned above because, essentially, it is a device
that serves to convert input information in digital form to an output that is mechanical. It thereby provides a
natural interface with the digital computer. A stepping motor, plus its associated drive electronics, accepts a
pulse-train input and produces an increment of rotary displacement for each pulse. We can control average
speed by manipulating pulse rate, and motor position by controlling the total pulse count.
Two types of stepping motors are in common use—the permanent magnet (PM), and the variable
reluctance (VR). Attention here is directed first to the PM motor.

3.8.1 Permanent Magnet Motor

Constructional Features A PM stepping motor in its
A simplest form is shown in Fig. 3.47. The motor has a per-
1 manent magnet rotor that, in this example, has two poles,
though often many more poles are used. The stator is made
2 N N of soft iron with a number of pole pieces and associated
S windings. Only four windings (grouped into two sets of
N two windings each) are used in this example. These
windings must be excited sequentially in a certain order.
3 S S Although this is commonly done by solid-state switching
B 4 circuits, mechanical switches are shown in the figure since
their operation is easier to visualize.
Assume the switches to be in the positions shown.
Windings 1 and 3 are energized and, as a result, the pole
pieces have the polarities shown. The rotor is thus found
in the position shown with its S pole centred between the
Fig. 3.47 A PM stepping motor (four phase) two upper N pole pieces, and its N pole between the two
 158 Digital Control and State Variable Methods
lower S pole pieces. With the field maintained, if we try to twist the shaft away from its standstill (equilibri-
um) position, we feel a ‘magnetic spring’ restoring torque. However, a sufficiently large external torque can
overcome the magnetic spring.
With the rotor energized and in equilibrium position, the torque required from an external source to break
away the motor from this position is called the holding torque. The holding torque is a basic characteristic of
the stepping motors, and provides positional integrity under standstill conditions.
If we now imagine the position of switch A changed, then winding 2 is energized instead of winding 1. As
a result, the right upper pole piece becomes S instead of N, and the left lower one N, so that the rotor is forced
to rotate 90º counterclockwise. Changing switch B produces the next 90º step, etc. The rotor is thus forced to
realign itself continuously according to the prevalent magnetic fields. If it is desired to reverse the direction
of rotation, the order of changing the switch positions need only be reversed.
One characteristic feature of PM stepping motors is that they have a so-called residual or detent torque
when power to the stator windings is cut off. It is the result of the permanent-magnet flux of the PM motor
acting on residual flux on stator poles. The detent torque is naturally much lower than the holding torque,
produced when the stator is energized, but it does help in keeping the shaft from moving due to outside forces.
Many motors have more than four stator pole pieces—and possibly also more rotor poles—resulting in
smaller step angles. Typical step angles for PM motors range from 90º to as low as 1.8º. Stator pole windings
are connected in so-called phases, with all windings belonging to the same phase energized at the same time.
Typically, the phases can range from as low as two to as high as eight. The more phases the motor has, the
smoother is its output torque.
Sequence Logic and Drive Amplifier Figure 3.48 shows a simple power drive scheme; each time the
power transistors are switched as per the sequence given in the chart, the motor moves through a fixed angle,

4
Q1 Q2 Q3 Q4

Step Q1 Q2 Q3 Q4

1 ON OFF ON OFF
CCW rotation
CW rotation

2 ON OFF OFF ON
3 OFF ON OFF ON
4 OFF ON ON OFF

Fig. 3.48 A simple stepping motor drive scheme

 Models of Digital Control Devices and Systems 159
referred to as the step angle. The chart is circular in the sense that, the next entry after Step 4 is Step 1. To
rotate the motor in a clockwise direction, the chart is traversed from top to bottom, and to rotate the motor in
counterclockwise direction, the chart is traversed from bottom to top. Number of step movements/sec gives
the stepping rate—a parameter that gives a measure of the speed of operation of the stepping motor. The
stepping rate is controlled by changing the switching frequency of the transistors.
From the foregoing description of the method of operation of a stepping motor, we observe that the
stepping action of the motor is dependent on a specific switching sequence that serves to energize and de-
energize the stator windings. In addition to the sequence requirement, the windings must be provided with
sufficient current. These requirements are met by the stepping motor driver, whose block diagram is shown
in Fig. 3.49. The sequence-logic section of the motor driver accepts the pulse-train input, and also receives
a binary direction signal indicating the direction in which the motor is to step. It then produces an appropri-
ate switching sequence, so that each phase of the motor is energized at the proper time. The drive-amplifier
section consists of power transistors supplying sufficient current to drive the motor.
Pulse train

Sequence Drive Motor

logic amplifier (four phase)

Direction
CW/CCW logic input
Fig. 3.49 Stepping motor driver (four phase)

3.8.2 Variable Reluctance Motor

Figure 3.50 illustrates a typical variable reluctance (VR) motor. The rotor is made of magnetic material, but it
is not a permanent magnet, and it has a series of teeth (eight in this case) machined into it. As with the PM
stepping motor, the stator consists of a number of pole pieces with windings connected in phases; all windings
belonging to the same phase are energized at the same time. The stator in Fig. 3.50 is designed for 12 pole
pieces with 12 associated windings arranged in three phases (labelled 1, 2, and 3, respectively). The figure
shows a set of four windings for phase 1; the windings for the other two phases have been omitted for clarity.
The operating principle of the VR motor is straightforward. Let any phase of the windings be energized
with a dc signal. The magnetomotive force set up will position the rotor such that the teeth of the rotor
section in the neighbourhood of the excited phase of the stator, are aligned opposite the pole pieces associat-
ed with the excited phase. This is the position of minimum reluctance, and the motor is in a stable equilib-
rium. Figure 3.50 illustrates the rotor in the position it would assume when Phase 1 is energized. If we now
de-energize Phase 1 and energize Phase 2, the rotor rotates counterclockwise so that the four rotor teeth
nearest to the four pole pieces belonging to Phase 2, align themselves with these. The step angle of the motor
equals the difference in angular pitch between adjacent rotor teeth and adjacent pole pieces; in this case
45 – 30 = 15º. Due to this difference relationship, VR motors can be designed to operate with considerably
smaller step angles than PM motors. Other advantages of VR motors include faster dynamic response and
the ability to accept higher pulse rates.
Among the drawbacks—their output torque is lower than that of a PM motor of similar size, and they do
not provide any detent torque when not energized.
 160 Digital Control and State Variable Methods
1
2

Fig. 3.50 A VR stepping motor (three phase)

3.8.3 Torque-Speed Curves of a Stepping Motor

Torque versus speed curves of a stepping motor give the dynamic torque, produced by the stepping motor at
a given stepping rate, on excitation under rated conditions. The dynamic torque of a motor is the most
important data and it plays a major role in the selection of a motor for a specified application. In a load-
positioning application, for instance, the rotor would typically start from rest and accelerate the load to the
desired position. To provide this type of motion, the motor must develop sufficient torque to overcome
friction, and to accelerate the total inertia. In accelerating the inertia, the motor may be required to develop
a large amount of torque, particularly if the acceleration must be completed in a short time—so as to position
the load quickly. Inability of the motor to develop sufficient torque during motion may cause the motor to
stall, resulting in a loss of synchronization between the motor steps and phase excitation and, consequently,
resulting in incorrect positioning of the load.
A typical torque versus stepping rate characteristic graph is shown in Fig. 3.51, in which curve a gives
pull-in torque versus rotor steps/sec and curve b gives pull-out torque versus rotor steps/sec.
The pull-in-range (the area between axes and curve a) of the motor is the range of switching speeds at
which the motor can start and stop, without losing steps. For a frictional load requiring torque T1 to over-
come friction, the maximum pull-in-rate is S1 steps per sec. S2 is the maximum pull-in-rate at which the
unloaded motor can start and stop, without losing steps.
When the motor is running, the stepping rate can be increased above the maximum pull-in-rate, and when
this occurs, the motor is operating in the slew-range region (the area between horizontal axis, and curves a
and b). The slew-range gives the range of switching speeds within which the motor can run unidirectionally,
but cannot be started or reversed (at shaft torque T1, the motor cannot be started or reversed at step rate S3).
 Models of Digital Control Devices and Systems 161
When the motor is running in the slew-range, it can follow changes in the stepping rate without losing steps,
but only with a certain acceleration limit.
Holding torque

Maximum starting torque

b Pull-out torque
Torque

a Pull-in torque
Slew
c range
T1
Pull-in range
T2

S1 S2 S3 S4 S5
Stepping rate
Fig. 3.51 Torque versus stepping rate characteristics

For a frictional load requiring torque T1 to overcome friction, the maximum slewing rate at which the motor
can run is S4. S5 is the maximum slewing rate at which the unloaded motor can run without losing steps.
Curve c in Fig. 3.51 gives the pull-in torque with external inertia. It is obvious that if the external load
results in a pull-in torque curve c, the torque developed by the motor at step rate S1 is T2 < T1. The stepping
motors are more sensitive to the inertia of the load than they are to its friction.

3.8.4 Interfacing of Stepping Motors to Microprocessors

In motion control technology, the rise of the stepping motors has in fact begun with the availability of easy-
to-use integrated circuit chips to drive the stepping motors. These chips require, as inputs a pulse train at the
stepping frequency, a logic signal to specify CW and CCW rotation, and a logic signal for STOP/START
operation. An adjustable frequency pulse train is readily obtained from another integrated circuit chip—a
voltage-controlled oscillator.
The application of stepping motors has shot up with the availability of low-cost microprocessors. A
simplified form of microprocessor-based stepping motor drive is shown in Fig. 3.52. The system requires an
input port and an output port (this requirement is reduced to one port if a programmable I/O port is used).
Output port handles the binary pattern applied to the stepping motor (which is assumed to be a four-phase
motor). The excitation sequence is usually stored in a table of numbers. A pattern for four-phase motor is
shown in the chart of Fig. 3.52. The chart is circular in the sense that the next entry after Step 4 is Step 1. To
rotate the motor in a clockwise direction, the chart is traversed from top to bottom, and to rotate the motor in
counter-clockwise direction, the chart is traversed from bottom to top. By controlling the number of bit-
pattern changes, and the speed at which they change, it is possible to control the angle through which the
motor rotates and the speed of rotation. These controls can easily be realized through software.
The system operator has control over the direction of rotation of the motor by means of a DIRECTION
switch, which is interfaced to the CPU through the input port. The operator is also provided with a STOP
switch which is connected to an interrupt line of the CPU. The interrupt routine must stop the motor
by sending out logic ‘0’s on the data bus lines connected to the stepping motor windings through the
output port.
 162 Digital Control and State Variable Methods

Fig. 3.52 Microprocessor-based stepping motor drive

Figure 3.52 also shows a simple drive circuit for the stepping motor. Power transistors Q1-Q4 act as
switching elements.
When a power transistor is turned off, a high voltage builds up due to di/dt, which may damage the
transistor. This surge in voltage can be suppressed by connecting a diode in parallel with each winding in the
polarity shown in Fig. 3.52. Now, there will be a flow of circulating current after the transistor is turned off,
and the current will decay with time.

3.8.5 Comments
Stepping motors present a number of pronounced advantages, as compared to conventional electric motors:
(i) Since the stepping-motor shaft angle bears an exact relation to the number of input pulses, the motor
provides an accurate open-loop positioning system without the need for closing the loop with a posi-
tion encoder, comparator, and servoamplifier, as is done in conventional closed-loop systems.
(ii) If the stepping motor receives a continuous train of pulses at constant frequency, it rotates at a constant
speed provided neither the load torque nor the pulse frequency are excessive for the given motor. The
stepping motor can thus take the place of a velocity servo, again, without the need for a closed-loop
system. By changing pulse frequency, the motor speed can be controlled. Even low velocities can be
maintained accurately, which is difficult to do with conventional dc motors.
(iii) By driving several motors from the same frequency source, synchronized motions at different points in
a machine are easily obtained. Using standard frequency-divider chips, we can drive a motor at a
precise fraction of another motor’s speed, giving an electronic gear train.
(iv) If the motor stator is kept energized during standstill, the motor produces an appreciable holding
torque. Thus, the load position can be locked without the need for clutch-brake arrangements. The
motor can be stalled in this manner indefinitely, without adverse effects.
 Models of Digital Control Devices and Systems 163
There are, of course, also certain drawbacks.
(i) If the input pulse rate is too fast, or if the load is excessive, the motor will ‘miss’ steps, making the
speed and position inaccurate.
(ii) If the motor is at rest, an external disturbing torque greater than the motor’s holding torque, can twist
the motor shaft away from its commanded position by any number of steps.
(iii) With high load inertias, overshooting and oscillations can occur unless proper damping is applied, and
under certain conditions, the stepping motor may become unstable.
(iv) The stepping motors are only available in low or medium hp ratings, upto a couple of hp (in theory,
larger stepping motors could be built, but the real problem lies with the controller—how to get large
currents into and out of motor windings at a sufficiently high rate, inspite of winding inductance).
(v) Stepping motors are inherently low-speed devices, more suited for low-speed applications because
gearing is avoided. If high speeds are required, this of course becomes a drawback.
Since the cost and simplicity advantages of stepping-motor control systems erode when motion sensors
and feedback loops are added, much effort has gone into improving the performance of open-loop systems:
(i) As explained earlier in connection with Fig. 3.51, the permissible pulse rate for starting an inertia load
(i.e., the pull-in rate), is much lower than the permissible pulse rate once the motor has reached maxi-
mum speed (pull-out rate). A good controller brings the motor upto its maximum speed gradually, a
process called ramping2, in such a manner that no pulses are lost. Similarly, a good controller controls
deceleration when the motor is to be stopped.
(ii) Various schemes for improving damping to prevent overshooting and oscillations when the motor is to
be stopped, are available. Mechanical damping devices provide a simple solution, but these devices
reduce the available motor torque and also, mostly, require a motor with a double-ended shaft. There-
fore, electronic damping methods are usually preferred. A technique called back-phase damping con-
sists of switching the motor into the reverse direction using the last few pulses of a move.
(iii) The more sophisticated controllers are able to provide so-called microstepping. This technique per-
mits the motor shaft to be positioned at places other than the natural stable points of the motor. It is
accomplished by proportioning the current in two adjacent motor windings. Instead of operating the
winding in the on-off mode, the current in one winding is decreased slightly, but increased in the
adjacent winding.
(iv) Complex drive circuits that offer good current build-up without loss at high stepping rates, are used.
Although the advantages of stepping-motor drives in open-loop systems are most obvious, closed-loop
applications also exist. A closed-loop stepping motor drive can be analysed using classical techniques
employed for continuous-motion systems. For a detailed account of stepping motors, refer [52].

3.9 PROGRAMMABLE LOGIC CONTROLLERS

A great deal of what has been said in this book so far about control systems seems exotic: algorithms for
radar tracking, drives for rolling mills, filters for extracting information from noisy data, methods for
numerical control of machine tools, fluid-temperature control in process plants etc. Underlying most of these
are much more mundane tasks: turning equipment (pumps, conveyor belts, etc.) on and off; opening and
closing of valves (pneumatic, hydraulic); checking sensors to be certain they are working; sending alarms
when monitored signals go out of range; etc. Process control plants and manufacturing floors share this need
for simple, but important, tasks.

2
Refer [96] for detailed description of hardware and software.
 164 Digital Control and State Variable Methods
These so-called logic control functions can be implemented using one of the most ingenious devices ever
devised to advance the field of industrial automation. So versatile are these devices, that they are employed
in the automation of almost every type of industry. The device, of course, is the programmable controller,
and thousands of these devices go unrecognized in process plants and factory environments—quietly moni-
toring security, manipulating valves, and controlling machines and automatic production lines.
Industrial applications of logic control fall mainly into two types; those in which the control system is
entirely based on logic principles, and those that are mainly of a continuous feedback nature and use a
‘relatively small’ amount of logic in auxiliary functions such as start-up/shut-down, safety interlocks and
overrides, and mode switching. Programmable controllers, originally intended for ‘100%’ logic systems,
have, in recent years, added the capability of conventional feedback control; making them very popular—
since one controller can now handle in an integrated way all aspects of operation of a practical system, that
includes both types of control problems. General purpose digital computers could also handle such situa-
tions, but they are not as popular as the programmable controllers, for the following reasons:
In theory, general-purpose computers can be programmed to perform most of the functions of program-
mable controllers. However, these machines are not built to operate reliably under industrial conditions,
where they can be exposed to heat, humidity, corrosive atmosphere, mechanical shock and vibration, elec-
tromagnetic noise, unreliable ac power with dropping voltages, voltage spikes etc. A programmable control-
ler is a special-purpose computer especially designed for industrial environments. A general-purpose com-
puter is a complex machine, capable of executing several programs or tasks simultaneously, and in any
order. By contrast, a programmable controller typically executes its tiny program continuously hundreds of
millions of times before being interrupted to introduce a new program. General-purpose computers can be
interfaced with external equipment with special circuit cards. In programmable controllers by comparison,
the hardware interfaces for connecting the field devices are actually a part of the controller and are easily
connected. The software of the controllers is designed for easy use by plant technicians. A programmable
controller is thus a special-purpose device for industrial automation applications—requiring logic control
functions and simple PID control functions; it cannot compete with conventional computers when it comes
to complex control algorithms and/or fast feedback loops, requiring high program execution speeds.
Early devices were called ‘programmable logic controllers (PLCs)’, and were designed to accept on-off
(binary logic) voltage inputs from sensors, switches, relay contacts, etc., and produce on-off voltage outputs
to actuate motors, solenoids, control relays, lights, alarms, fans, heaters, and other electrical equipment. As
many of today’s ‘programmable controllers’ also accept analog data, perform simple arithmetic operations,
and even act as PID (proportional-integral-derivative) process controllers, the word ‘logic’ and the letter ‘L’
were dropped from the name long ago. This frequently causes confusion, since the letters ‘PC’ mean differ-
ent things to different people; the most common usage of these letters being for ‘Personal Computer’. To
avoid this confusion, there has been a tendency latety to restore the letter ‘L’ and revive the designation
‘PLC’. We will follow this practice.
Before the era of PLCs, hardwired relay control panels were, in fact, the major type of logic systems, and
this historical development explains why the most modern, microprocessor-based PLCs still are usually
programmed according to relay ladder diagrams. This feature has been responsible for much of the wide-
spread and rapid acceptance of PLCs; the computer was forced to learn the already familiar human language
rather than making the humans learn a new computer language. Originally cost-effective for only large-scale
systems, small versions of PLCs are now available.
A sequenced but brief presentation of building blocks of a PLC, ladder diagrams, and examples of
industrial automation, follows [23–25]. It is not appropriate here to teach the reader the internal details,
performance specifications and programming details for any particular manufacturer’s PLC. These aspects
are described in manufacturers’ literature.
 Models of Digital Control Devices and Systems 165
3.9.1 Logic Controls for Industrial Automation
A definition of logic controls, that adequately describes most applications, is that they are controls that work
with one-bit binary signals. That is, the system needs only to know that a signal is absent or present; its exact
size is not important. This definition excludes the field of digital computer control discussed so far in the
book. Conventional computer control also uses binary signals (though usually with many bits), the type of
application and the analysis methods are quite different for logic controls and conventional computer
controls, which is why we make the distinction.
Logic control systems can involve both combinational and sequential aspects. Combinational aspects are
implemented by a proper interconnection of basic logic elements such as AND, OR, NOT, so as to provide a
desired output or outputs, when a certain combination of inputs exists. Sequential effects use logic elements
together with memory elements (counters, timers, etc.), to ensure that a chain of events occurs in some desired
sequence. The present status of outputs depends, both, on the past and present status of inputs.
It is important to be able to distinguish between the nature of variables in a logic control system, and those
in a conventional feedback control system. To define the difference, we consider an example that employs
both the control schemes.
Figure 3.53 shows a tank with a valve that controls flow of liquid into the tank, and another valve that
controls flow out of the tank. A transducer is available to measure the level of the liquid in the tank. Also
shown is the block diagram of a feedback control system, whose objective is to maintain the level of the
liquid in the tank at some preset, or set-point value. We assume that the controller operates according to PID
mode of control, to regulate the level against variations induced from external influences. This is a continu-
ous variable control system because, both the level and the control valve setting can vary over a range to
achieve the desired regulation.
Input flow

Control valve

Controller Set-point

Solenoid
operated valve V 1

Photoelectric sensor
Empty Bottles Filled Bottles
Fixed
Rollers
LS
Limit Switch
Feed Motor Outfeed Motor Drive
Drive (always ON
M1 during process)
M2
Fig. 3.53 Composite conventional and logic control
 166 Digital Control and State Variable Methods

The liquid-level control system is a part of the continuous bottle filling process. Periodically, a bottle
comes into position under the outlet valve. The level must be maintained at the set-point while the outlet
valve is opened and the bottle filled. This requirement is necessary to assure a constant pressure head during
bottle-filling. Figure 3.53 shows a pictorial representation of process hardware for continuous bottle-filling
control. The objective is to fill bottles moving on a conveyor, from the constant-head tank. This is a typical
logic control problem. We are to implement a control program that will detect the position of a bottle under
the tank outlet via a mechanically actuated limit switch, stop the feed motor M1 to stop the feed conveyor,
open the solenoid-operated outlet valve V1, and then fill the bottle until the photo sensor detects the filled
position. After the bottle is filled, it will close the valve V1, and restart the conveyor to continue to the next
bottle. The start and stop push buttons (PB) will be included for the outfeed motor, and for the start of the
bottle-filling process. Once the start PB is pushed, the outfeed motor M2 will be ON until the stop PB is
pushed. The feed motor M1 is energized once the system starts (M2 ON), and is stopped when the limit
switch detects the correct position of the bottle.
The sensors used for the logic control problem have characteristics different from those used for the
regulator problem. For the regulator problem, the level sensor is an analog device producing analog signal as
its output. For the logic control problem, sensors used are binary sensors producing on-off (binary logic)
signals. For example, a limit-switch consists of mechanically actuated electrical contacts. The contacts open
or close when some object reaches a certain position (i.e., limit), and actuates the switch. Hence, limit-
switches are binary sensors. Photoelectric sensors consist, basically, of a source emitting a light beam and a
light-sensing detector receiving the beam. The object to be sensed interrupts the beam, thereby making its
presence known without physical contact between sensor and object. The filled-bottle state of the product
can thus be sensed by a binary photoelectric sensor.
The system of Fig. 3.53 involves solenoid and electric motors as motion actuators. Thus, when the logic
controller specifies that ‘output valve be opened’, it may mean moving a solenoid. This is not done by a
simple toggle switch. Instead, one would logically assume that a small switch may be used to energize a relay
with contact ratings that can handle the heavy load. Similarly, an on-off voltage signal from the logic
controller may actuate a thyristor circuit to run a motor.

3.9.2 Building Blocks of a PLC

The programmable logic controllers are basically computer-based; and therefore, their architecture is very
similar to computer architecture. The memory contains the operating system stored in fixed memory (ROM), and
the application programs stored in alterable memory (RAM). The Central Processing Unit (CPU) is a micro-
processor that coordinates the activities of the PLC system. Figure 3.54 shows basic building blocks of a PLC.

Power Supply

CPU
Signals from Input Signals to
Output
switches, Module solenoids,
Module
sensors, etc. Memory motors, etc.

Fig. 3.54 Basic building blocks of a PLC

Input devices such as push-buttons, sensors, and limit switches are connected to the input interface
circuit, called the input module. This section gathers information from the outside environment, and sends
 Models of Digital Control Devices and Systems 167
it on to the CPU. Output devices such as solenoids, motor controls, indicator lights and alarms are connected
to the output interface circuit, called the output module. This section is where the calculation results from the
CPU are output to the outside environment. With the control application program (stored within the PLC
memory) in execution, the PLC constantly monitors the state of the system through the field input devices;
and based on the program logic, it then determines the course of action to be carried out at the field output
devices. This process of sequentially reading the inputs, executing the program in memory, and updating the
outputs is known as scanning.
Input Module Intelligence of an automated system is greatly dependent on the ability of a PLC to read
in the signals from various types of input field devices. The most common class of input devices in an
automated system is the binary type. These devices provide input signals that are ON/OFF or OPEN/
CLOSED. To the input interface circuit, all binary input devices are essentially a switch that is either open or
closed, signalling a 1(ON) or 0(OFF). Some of the binary input field devices alongwith their symbolic repre-
sentation are listed in Fig. 3.55.

PB LS
LS LS LS
(a) (b) (c) (d) (e)
Fig. 3.55 (a) Push-button, (b) Position limit switch, (c) Level limit switch, (d) Temperature
limit switch, (e) Pressure limit switch
As mentioned earlier, a switch is a symbolic representation of the field input device, interfaced to the
input module of the PLC. The device may be a manually operated push button, mechanically actuated limit
switch (the contacts open/close when some object reaches a certain position and actuates the switch), prox-
imity switch (device based on inductive/capacitive/magnetic effect which, with appropriate electronics, can
sense the presence of an object without a physical contact with the object), photoelectric sensor, level sen-
sor, temperature sensor, shaft encoder, etc. The main purpose of the input module is to condition the various
signals, received from the field devices, to produce an output to be sensed by the CPU. The signal condition-
ing involves converting power-level signals from field devices to logic-level signals acceptable to the CPU,
and providing electrical isolation so that there is no electrical connection between the field device (power)
and the controller (logic). The coupling between the power and the logic sections is normally provided by an
optical coupler.
During our discussion on PLC programming, it will be helpful if we keep in mind the relationship be-
tween the interface signals (ON/OFF) and their mapping and addressing used in the program. When in
operation, if an input signal is energized (ON), the input interface circuit senses the field device’s supplied
voltage and converts it to a logic-level signal acceptable to the CPU, to indicate the status of the device. The
field status information provided to the standard input module is placed into the input table in memory
through PLC instructions. The I/O address assignment document of the PLC manufacturer identifies each
field device by an address. During scanning, the PLC reads the status of all field input devices, and places
this information at the corresponding address locations.
Output Module An automation system is incomplete without means for interface to the field output
devices. The output module provides connections between the CPU and output field devices. The output
module receives from the CPU logic-level signals (1 or 0).
The main purpose of the output interface circuit is to condition the signals received from the CPU, to
produce outputs to actuate the output field devices. The signal conditioning circuit consists, primarily, of the
 168 Digital Control and State Variable Methods
logic and power sections, coupled by an isolation circuit. The output interface can be thought of as a simple
switch through which power can be provided to control the output device.
During normal operation, the CPU sends to the output table, at predefined address locations, the output
status according to the logic program. If the status is 1, ON signal will be passed through the isolation circuit,
which, in turn, will switch the voltage to the field device through the power section of the module.
The power section of the output module may be transistor based, triac based, or simply, relay ‘contact
based’ circuit. The relay circuit output interface allows the output devices to be switched by NO (normally-
open) or NC (normally-closed) relay contact. When the processor sends the status (1 or 0) to the module
(through output table) during the output update, the state of the contact will change. If a 1 is sent to the
module from the processor, a normally open contact will close, and a normally closed contact will change to
an open position. If a 0 is sent, no change occurs to the normal state of the contacts. The contact output can
be used to switch either ac or dc loads; switching small currents at low voltages. High power contact outputs
are also available for switching of high currents.
Some of the output field devices, alongwith their symbolic representation, are given in Fig. 3.56.

SOL
PL AH
(a) (b) (c) (d)

H
NO NC
(e) (f) (g)
Fig. 3.56 (a) Motor, (b) Solenoid, (c) Pilot lamp, (d) Alarm horn, (e) Heater,
(f) Relay contact (Normally-Open), (g) Relay contact (Normally-Closed)

I/O Rack Enclosures Once we have the CPU programmed, we get information in and out of the PLC
through the use of input and output modules. The input module terminals receive signals from switches, and
other input information devices. The output module terminals provide output voltages to energize motors
and valves, operate indicating devices, and so on.
For small PLC systems, the input and output terminals may be included on the same frame as the CPU. In
large systems, the input and output modules are separate units; modules are placed in groups on racks, and
the racks are connected to the CPU via appropriate connector cables.
Generally speaking, there are three categories of rack enclosures—the master rack, the local rack, and
the remote rack. A master rack refers to the enclosure containing the CPU module. This rack may, or may
not, have slots available for the insertion of I/O modules. The larger the PLC system, the less likely that the
master rack will have I/O housing capability or space. A local rack is an enclosure which is placed in the
same location or area where the master rack is housed. If a master rack contains I/O, it can also be considered
a local rack. In general, a local rack contains a local I/O processor which receives and sends data to and from
the CPU.
As the name implies, remote racks are enclosures containing I/O modules located far away from the CPU.
A remote rack contains an I/O processor which communicates I/O information just like the local rack.
Timers/Counters Timers and counters play an important part in many industrial automation systems. The
timers are used to initiate events at defined intervals. The counters, on the other hand, are used to count the
occurrences of any defined event.
 Models of Digital Control Devices and Systems 169
Basically the operation of both the timer and the counter is same, as a timer operates like a counter. The
counter shown in Fig. 3.57a, counts down from Set Value when its execution condition (count input) goes
from OFF to ON. When the value reaches zero, the counter contact point is turned ON. It is reset with a reset
input. The Set Value is decided by the programmer, and stored in the internal register of the counter through
control program instructions. The count input signal may refer to any event which may occur randomly.
When a count input signal occurs at fixed frequency, i.e., after every fixed interval of time, the counter
performs as a timer. Now 10 pulses, i.e., counts, will mean an elapsed time of 5 seconds, if the signal is
occurring after a regular interval of 0.5 seconds. The timer, shown in Fig. 3.57b, is activated when its
execution condition goes ON and starts decreasing from the Set Value. When the value reaches zero, the
timer contact point is turned ON. It is reset to Set Value when the execution condition goes OFF.

Count input
Counter Start input Timer

Reset input
Set Value Set Value

(a) (b)
Fig. 3.57 (a) Counter, (b) Timer

Memory Map It is unlikely that two different PLCs will have identical memory maps, but a generalization
of memory organization is still valid in the light of the fact that all PLCs have similar storage requirements. In
general, all PLCs must have memory allocated for the four items described below.
Executive The executive software is a permanently stored collection of programs that are considered a part
of the system itself. These programs direct system activities such as execution of the control program, com-
munication with peripheral devices, and other housekeeping activities. The executive area of memory is not
accessible to the user.
Scratch Pad It is a temporary storage used by the CPU to store a relatively small amount of data for interim
calculations or control.
Data Table This area stores any data associated with the control program, such as timer/counter set values,
and any other stored constants or variables that are used by the control program. This section also retains the
status information of the system inputs once they have been read, and the system outputs once they have
been set by the control program.
User Program This area provides storage for any programmed instructions entered by the user. The
control program is stored in this area.
The Data Table and the User Program areas are accessible and are required by the user for control
application. The Executive and Scratch Pad areas together are normally referred to as ‘system memory’, and
Data Table and User Program areas together are labelled as ‘application memory’.
Data Table Memory Area The data table area of the PLC’s application memory is composed of several
sections described below.
Input Table The input table is an array of bits that stores the status of discrete inputs which are connected
to input interface circuit. The maximum number of bits in the input table is equal to the maximum number of
field inputs that can be connected to the PLC. For instance, a controller with 128 inputs would require an
input table of 128 bits. If the PLC system has 8 input modules, each with 16 terminal points, then the input
table in PLC memory (assuming 16 bit word length) will look like that in Fig. 3.58.
 170 Digital Control and State Variable Methods

Rack
Terminal 12

Input
Slot
15 13 10 08 06 04 02 00
0 0 0 1 000
Bit
address
002
Input module
with 16 terminals
placed in 004
master rack

007
Word
address
Fig. 3.58 An illustrative input table

Each terminal point on each of the input modules will have an address by which it is referenced. This
address will be a pointer to a bit in the input table. Thus, each connected input has a bit in the input table that
corresponds exactly to the terminal to which the input is connected. The address of the input device can be
interpreted as word location in the input table corresponding to the input module, and bit location in the
word corresponding to the terminal of the input module to which the device is connected.
Several factors determine the address of the word location of each module. The type of module, input or
output, determines the first number in the address from left to right (say, 0 for input, and 1 for output). The
next two address numbers are determined by the rack number and the slot location where the module is
placed. Figure 3.58 graphically illustrates a mapping of the input table, and the modules placed in rack 0
(master rack). Note that the numbers associated with address assignment depend on the PLC model used.
These addresses can be represented in octal, decimal, or hexadecimal. We have used decimal numbers.
The limit switch connected to the input interface (refer Fig. 3.58) has an address of 00012 for its corre-
sponding bit in the input table. This address comes from the word location 000 and the bit number 12; which
are related to the rack position where the module is installed, and the module’s terminal connected to the
field device, respectively. If the limit switch is ON (closed), the corresponding bit 00012 will be 1; if the
limit switch is OFF (open), its corresponding bit will be 0.
During PLC operation, the processor will read the status of each input in the input modules, and then
place this value (1 or 0) in the corresponding address in the input table. The input table is constantly chang-
ing—to reflect the changes in the field devices connected to the input modules. These changes in the input
table take place during the reading part of the processor scan.
Output Table The output table is an array of bits that controls the status of output devices, which are
connected to the output interface circuit. The maximum number of bits available in the output table is equal
to the maximum number of output field devices that can be interfaced to the PLC. For instance, a controller
with a maximum number of 128 outputs would require an output table of 128 bits.
Each connected output has a bit in the output table that corresponds exactly to the terminal to which the
output is connected. The bits in the output table are controlled (ON/OFF) by the processor, as it interprets
the control program logic. If a bit in the table is tuned ON (logic 1), then the connected output is switched
 Models of Digital Control Devices and Systems 171
ON. If a bit is cleared or turned OFF (logic 0), the output is switched OFF. Remember that the turning ON or
OFF, of the field devices occurs during the update of outputs after the end of the scan.
Storage Area This section of the data table may be subdivided in two parts consisting of a work bit storage
area and a word storage area. The purpose of this data table section is to store data that can change, whether
it is a bit or a word (16 bits). Work bits are internal outputs which are normally used to provide interlocking
logic. The internal outputs do not directly control output field devices. When the processor evaluates the
control program, and any of these outputs is energized (logic 1), then this internal output, in conjunction with
other internal and/or real signals from field devices, forms an interlocking logic sequence that drives an
output device or another internal output.
The outputs of timers and counters are used as internal outputs which are generated after a time interval
has expired, or a count has reached a set value.
Assume that the timer/counter table in storage area has 512 points. Address assignment for these points
depends on the PLC model used. We will use TIM/CNT000 to TIM/CNT512 as the addresses of these
points. The word storage area will store the set values of timers/counters.
In our application examples given in the next subsection, we shall use word addresses 000 to 007 for input
table, and addresses 100 to 107 for output table. The input devices will be labelled with numbers such as
00000,..., 00015, and output devices with numbers such as 10000, ..., 10015.
We shall use word addresses 010 to 017 for internal outputs. Examples of typical work bits (internal
outputs) are 01000, ..., 01015. TIM/CNT000 to TIM/CNT512 are the typical addresses of timer/counter
points.

3.9.3 Ladder Diagrams

Although specialized functions are useful in certain situations, most logic control systems may be imple-
mented with the three basic logic functions AND, OR, and NOT. These functions are used either singly or
in combinations, to form instructions that will determine if an output field device is to be switched ON or
OFF. The most widely used language for implementing these instructions are ladder diagrams. Ladder
diagrams are also called contact symbology, since its instructions, as we shall see, are relay-equivalent
contact symbols shown in Figs 3.56f and 3.56g.
An AND device may have any number of inputs and one output. To turn the output ON, all the inputs
must be ON. This function is most easily visualized in terms of switch arrangement of Fig. 3.59a, and timing
chart of Fig. 3.59b. The corresponding ladder diagram is given in Fig. 3.59c. Figure 3.59d gives the Boolean
algebra expression for the two-input AND, read as “A AND B equals C”.
The timing chart in Fig. 3.59b is simply a series of graphs, each representing a logic variable, in which the
horizontal axis is time and the vertical axis is logic state, that is, 0 or 1. The graphs are placed so that their
time-axis are synchronized; in this way, a vertical line at any point on the graph describes a point in time, and
all input and output variables can be evaluated at that point. The graph of the output variable is determined
by the structure of the logic system and, of course, the pattern of the input.
The input contacts in Fig. 3.59c are normally-open (NO) contacts (Do not confuse this symbol with the
familiar electrical symbol for capacitors). If the status of the input A is ‘1’, the contact A in ladder diagram
will close, and allow current to flow through the contact. If the status of the input A is ‘0’, the contact will
remain open, and not allow current to flow through the contact.
The ladder diagram of Fig. 3.59c can be thought of as a circuit having many inputs. A circuit is known as
the “rung” of the ladder. A complete PLC ladder diagram consists of several rungs; a rung controls an output
field device either through an output module or an internal output. The input to a rung can be logic com-
mands from input modules, or from output modules connected to field devices, or from internal outputs.
 172 Digital Control and State Variable Methods

Fig. 3.59 The AND function

Figure 3.60 gives similar details for logical OR operation and should be self-explanatory. The Boolean
expression is read as “A OR B equals C”.
Input
A
1
A
0
1
B
Output 0
C 1
Input C
B 0

1 3 5 7 9 Time
(a) Switch interpretation (b) Timing chart

B C
Output
Inputs A+B=C

(c) Ladder diagram (d) Boolean expression

Fig. 3.60 The OR function
 Models of Digital Control Devices and Systems 173
The contact in Fig. 3.61 in normally closed (NC) contact. If the status of the input A is ‘0’, the contact will
remain closed, thus allowing current to flow through the contact. If the status of the input A is ‘1’, the contact
will open and not allow current to flow through the contact. This symbol permits the use of logic NOT
operator. The Boolean expression is read as “NOT A equals B”; the overbar is used in general, for applying
NOT function.

Input 1
A A
0

Output 1
B
B 0

1 3 5 7 9 Time
(a) Switch interpretation (b) Timing chart

Normally closed (NC)

contact

B
A Output A=B
Input

(c) Ladder diagram (d) Boolean expression

Fig. 3.61 The NOT function

Consider the logic system

A× B = C
read as “A AND NOT B equals C”.
The ladder diagram and timing chart for this system are given in Fig. 3.62.

1
A
0
1
B
0
A B
1
C C
0

1 3 5 7 9 Time
(a) Ladder diagram (b) Timing chart
Fig. 3.62 AND NOT function
 174 Digital Control and State Variable Methods
Consider now the logic system
A+ B =C
read as “A OR NOT B equals C ”.
The ladder diagram and timing chart for this system are given in Fig. 3.63.

Fig. 3.63 OR NOT function

Example 3.5 Start/Stop Pushbutton System

Most large industrial machines are turned on and off by means of separate spring pushbuttons for start and
stop. This has safety implications in that the stop pushbutton can be given priority to shut down the machine in
an emergency, regardless of the status of the start pushbutton. The start/stop pushbutton system is a logic
control system with three variables, each of which can take on two, and only two values, or states. These
variables and their states are defined as follows:
Assume that 000 is the word address of the input module, and 100 is the word address of the output
module of a PLC. Each module is assumed to have 16 terminals: 00 to 15. The start pushbutton is connected
to terminal 00, and stop pushbutton is connected to terminal 01 of the input module 000; and the signal from
terminal 00 of the output module 100 controls the machine. The system variables may therefore be desig-
nated as 00000, 00001, and 10000.
The bit 00000 of the input table in PLC memory is 1 when the start pushbutton is pressed, and is 0 when
start pushbutton is released. The bit 00001 of the input table is 1 when the stop pushbutton is pressed, and
is 0 when the stop pushbutton is released. The bit 10000 of the output table is 1 when the machine is running,
and 0 when the machine is not running.
The logic system has three input variables and one output variable. There appears to be a contradiction,
but the statement is true. The variable 10000, representing the start of the machine, is both an input variable
and an output variable. This makes sense because the current state of the machine may affect the future state.
Figure 3.64a illustrates a simple situation in which pushbutton 00000 turns ON machine 10000. This of
course would not be satisfactory pushbutton switch because as soon as pushbutton is released, the machine
comes to OFF state. Figure 3.64b adds an OR condition that keeps the machine ON if it is already ON. This
is an improvement, but now there is a new problem; once turned ON, the output will never be turned OFF by
the logic system. We add another input switch in Fig. 3.64c. Note that 00001 contact is normally closed.
Input 00000 turns ON output 10000; input 10000 keeps output 10000 ON until input 00001 turns it OFF.
The timing chart of the logic system is shown in Fig. 3.64d.
 Models of Digital Control Devices and Systems 175

00000 10000
(a)

00000
10000

(b)

10000

00000
00001 10000
(c)

10000

1
00000
0
1
(d) 00001
0
1
10000
0

1 3 5 7 9
Fig. 3.64 Start/Stop pushbutton system

Example 3.6 Automatic Weigh Station

Consider the conveyer system of Fig. 3.65, in which an automatic weigh station activates a trap door, or
diverter, in the event an overweight item passes over the weigh station. The trap door opens immediately, and
remains open for 4 seconds to allow sufficient time for the item to drop through to the overweight track (For
the system to work properly, it is necessary for successive items on the conveyor to be separated by distances
of at least 5 seconds or so).

Trap door

Automatic
weigh
station

Overweight track
Fig. 3.65 Automatic weigh station on a conveyor system
 176 Digital Control and State Variable Methods

The variables of the logic system are defined as follows (refer Fig. 3.66a). 00000 represents the pressure
switch connected to terminal 00 of input module 000. It senses the overweight item on the automatic weigh
station. The bit 00000 in the input table latches 1 for the overweight item. 10000 represents a solenoid
connected to terminal 00 of the output module 100, which pushes the trap door open. When the bit 10000 in
the output table latches 1, the trap door is open and when the bit is 0, the trap door is closed.

Fig. 3.66

For a 4 sec delay, the set value 0040 is stored in word storage area of memory. Count down of this number
to 0000 will give an elapsed time of 4 seconds in our PLC system wherein we assume that the timer counts
time-based intervals of 0.1 sec (40 counts will mean an elapsed time of 4 sec).
The timer TIM000 is activated when its execution condition goes ON and starts decreasing from the set
value. When the value reaches 0000, the timer contact point is tuned ON. It is reset to set value when the
execution condition goes OFF. The timer contact point works as internal work bit (A work bit/internal output
is for use of program only; it does not turn ON/OFF external field devices).
It is obvious from the ladder diagram of Fig. 3.66a that once an overweight item is detected, the trap door
opens; it remains open for 4 sec, and thereafter it closes. Figure 3.66b shows the timing chart for the logic
system.

Example 3.7 Packaging Line Control

Figure 3.67 shows a pictorial representation of process hardware of a conveyor system used for automatic
packaging. The objective is to fill boxes moving on Box Conveyor from the Part Conveyor system.
Input devices : Box proximity sensor 00003
Part proximity sensor 00002
Stop pushbutton 00001
Start pushbutton 00000
Output devices : Part conveyor motor 10000
Box conveyor motor 10001
Pushbutton 00000 starts the packaging line control which stops when pushbutton 00001 is pressed. Let
us generate a work bit (internal output) 01000, which depends on the state of both the pushbuttons (refer
 Models of Digital Control Devices and Systems 177
Part conveyor

Part sensor

Box sensor

Box conveyor

Fig. 3.67 Packaging line control

Fig. 3.68). The work bit is 1 when packaging line control is ON, and it is 0 when packaging line control is
OFF. The work bit is useful where the same combination of input signals appears repeatedly in the ladder
diagram. We will shortly see that work bit 01000 is helpful in simplifying the ladder diagram.
The event sequences are as follows.
(i) Box proximity sensor in ‘0’ state; box conveyor will start, keeping part conveyor standstill.
(ii) Box proximity sensor in ‘1’ state; box conveyor will
stop, signalling the start of the part conveyor.
(iii) Part proximity sensor in ‘1’ state and box proximity sensor
in ‘1’ state; this state signals the execution of the counter.
A counter counts down how many times input is turned
ON. It counts down from set value, when its execution condi-
tion (count input) goes from OFF to ON. It decrements one
count every time the input signal goes from OFF to ON. When
the value reaches 0000, the counter contact point is turned
ON. It is reset with a reset input. The counter contact point
works as internal work bit.
The execution signal for our counter is generated by part
proximity sensor 00002 and reset signal is generated by box prox-
imity sensor 00003. For counting to occur, we need both the part
proximity sensor and the box proximity sensor in ‘1’ state.
(iv) Assume that set value 0010 of count has been loaded in
word storage area of memory. When the value reaches
0000, the work bit CNT010 takes ‘1’ state, which starts
the box conveyor.
(v) Box proximity switch gets deactivated and count stops.
Fig. 3.68 Ladder diagram of logic system The timing chart is shown in Fig. 3.69.
 178 Digital Control and State Variable Methods

1
00000
0
1
00001
0
1
01000
0
1
00002
0
1
00003
0
1
CNT010
0
1
10000
0
1
10001
0

Time
Fig. 3.69 Timing chart

Example 3.8 Automatic Bottle Filling Control

In this application (shown in Fig. 3.53), we are to implement
a control program that will detect the position of a bottle via
a limit switch, wait for 5 seconds, and then fill the bottle
until the photo sensor detects the filled position. After the
bottle is filled, it will wait for 7 seconds to continue to the
next bottle. The start and stop circuits will also be included
for the outfeed motor and for the start of the process. The
I/O assignment follows:
Start-process Pushbutton : 00000
Stop-process Pushbutton : 00001
Limit switch (position detect) : 00002
Photo sensor (level detect) : 00003
Feed-motor drive : 10000
Outfeed-motor drive : 10001
Solenoid control : 10002
The work bits may be assigned as follows :
Timer for 5 sec delay : TIM000
Timer for 7 sec delay : TIM001
Ladder diagram for automatic bottle filling controller is
shown in Fig. 3.70. Rung 1 provides a start/stop latch
for the system. The outfeed motor is always ON during Fig. 3.70 Ladder diagram of logic system
 Models of Digital Control Devices and Systems 179
process operation. Rung 2 drives the feed conveyor until a bottle is in position. Rung 3 introduces a time
delay of 5 sec. The work bit TIM000 turns ON the valve solenoid (Rung 4). Rung 5 introduces a time delay
of 7 sec after detecting bottle filled position. Rung 6 is necessary to detect that the bottle is full and 7 sec
waiting period is over, and to restart the conveyor to move the bottle out (01000 is a work bit).

3.9.4 PLC Programming

PLC programming methods vary from manufacturer to manufacturer, but the basic ladder diagram approach
appears to be the standard throughout the industry. A CRT connected to the CPU of the PLC through a
peripheral port, is perhaps the most common device used for programming the controller. A CRT is a self-
contained, video display unit with a keyboard and the necessary electronics to communicate with the CPU.
The graphic display on the CRT screen appears as a ladder diagram. This ladder diagram takes form while
the programmer builds it up using the keyboard. The keys themselves have symbols such as: ; ,
which are interpreted exactly as explained earlier in this section.
A limitation of CRT is that the device is not interchangeable from one manufacturer’s PLC family to
another. However, with the increasing number of products in the manufacturers’ product lines and user
standardization of products, these programming devices may be a good choice, especially if the user has
standardized with one brand of PLCs.
At the other end of the spectrum of PLC programming devices is a Programming Console for program-
ming small PLCs (upto 128 I/O). Physically, these devices resemble handheld calculators but have a larger
display and somewhat different keyboard. The Programming Console uses keys with two- or three-letter
abbreviations, to write programs that bear some semblance to computer coding. The display at the top of the
Console exhibits the PLC instruction located in the User Program memory area. As with CRTs, Program-
ming Consoles are designed so that they are compatible with controllers of the product family.
Common usage of personal computer (PC) in our daily lives has led to a new breed of PLC programming
devices. Due to the PC’s general purpose architecture and de facto standard operating system, PLC manu-
facturers provide the necessary software to implement the ladder diagram entry, editing and real-time moni-
toring of the PLC’s control program. PCs will soon be the programming device of choice, not so much
because of its PLC programming capabilities, but because these PCs may already be present at the location
where the user may be performing the programming.
The programming device is connected to the CPU through a peripheral port. After the CPU has been
programmed, the programming device is no longer required for CPU and process operation; it can be dis-
connected and removed. Therefore, we may need only one programming device for a number of operational
PLCs. The programming device may be moved about in the plant as needed.
Programming details for any manufacturer’s PLC are not included here. These aspects are described in
manufacturers’ literature.

3.9.5 Scope of Applications

Programmable logic controllers are available in many sizes, covering a wide spectrum of capability. On the
low end are ‘relay replacers’ with minimum I/O and memory capability. At the high end are large supervi-
sory controllers, which play an important role in distributed control systems—by performing a variety of
control and data acquisition functions. In between these two extremes are multifunctional controllers with
communication capability which allows integration with various peripherals, and expansion capability which
allows the product to grow, as the application requirements change.
PLCs with analog input modules and analog output modules, for driving analog valves and actuators
using the PID control algorithms, are being used in process industries.
 180 Digital Control and State Variable Methods
Large PLCs are used for complicated control tasks that require analog control, data acquisition, data
manipulation, numerical computations and reporting. The enhanced capabilities of these controllers allow
them to be used effectively in applications where LAN (local area network) may be required.
Some PLCs offer the ability to program in other languages beside the conventional ladder language. An
example is the BASIC programming language. Other manufacturers use what is called ‘Boolean Mnemonics’,
to program a controller. The Boolean language is really a method used to enter and explain the control logic
which follows Boolean algebra.

Review Examples

Review Example 3.1 We have so far used the z-transform technique to obtain system response at the
sampling instants only. In a large number of cases, this information is adequate because, if the sampling
theorem is satisfied, then the output will not vary too much between any two consecutive sampling instants.
In certain cases, however, we may need to find the response between consecutive sampling instants. Often,
for example, hidden oscillations, that may or may not decay with time, are present. We can compute ripple
(response between sampling instants) by introducing a fictitious delay of DT seconds at the output of
the system, where 0 £ D £ 1 and T is the sampling period. By varying D between 0 and 1, the output y(t) at
t = kT – DT (where k = 1, 2, ...) may be obtained.
In Example 3.1, unit-step response of the sampled-data system of Fig. 3.13a was obtained. The sampling
period T = 1 sec, and the output at the sampling instants is given by
y(T ) = 0.3679, y(2T ) = 1, y(3T ) = 1.3996, y(4T ) = 1.3996,
y(5T ) = 1.1469, y(6T ) = 0.8944, y(7T ) = 0.8015, ...
We now introduce at the output a fictitious delay of DT seconds with D = 0.5 as shown in Fig. 3.71. The
output y$ (kT) can be determined as follows:
Y$ (z) = Z [Gh0 (s) e–DTs] E(z)
E(z) = R(z) – Y(z)
Y(z) = Z [Gh0(s)G(s)] E(z)

r ( t) e* (t) y (t) y (t )

+ 1
G h0 (s) G (s ) = eE Ts
T s(s + 1)
–

Fig. 3.71 Closed-loop discrete-time system for Review Example 3.1

R( z)
Therefore E(z) =
1 + Z [Gh 0 ( s)G( s )]
Z [Gh 0 ( s) G(s )e - DTs ]
Y$ ( z ) = R(z) (3.73)
1 + Z [Gh 0 (s) G(s)]
 Models of Digital Control Devices and Systems 181
Referring to Example 3.1, we have
0.3679 z + 0.2642
Z [Gh0(s)G(s)] = 2
z - 1.3679 z + 0.3679
Referring to Table 3.1, we get

Z [Gh0(s)G(s)e–DTs] = (1 – z–1)
LM 1 2 -
0.5
+
0.6065 OP
N (z - 1) ( z - 1) ( z - 0.3679) Q
0.1065 z 2 + 0.4709 z + 0.0547
=
z 3 - 1.3679 z 2 + 0.3679 z
Referring to Eqn. (3.73) and noting that R(z) = z/(z – 1), we have
0.1065 z -1 + 0.4709 z -2 + 0.0547 z -3
Y$( z ) =
1 - 2 z -1 + 1.6321 z -2 - 0.6321 z -3
This equation can be expanded into an infinite series in z–1:
Y$ ( z ) = 0.1065 z–1 + 0.6839 z–2 + 1.2487 z–3 + 1.4485 z–4
+ 1.2913 z–5 + 1.0078 z–6 + 0.8236 z–7 + 0.8187 z–8 + L
Therefore
y$ (T ) = y(0.5T) = 0.1065; y$ (2T ) = y(1.5T) = 0.6839; y$ (3T ) = y(2.5T) = 1.2487;
y$ (4T ) = y(3.5T) =1.4485; y$ (5T ) = y(4.5T) = 1.2913; y$ (6T ) = y(5.5T) = 1.0078;
y$ (7T ) = y(6.5T) = 0.8236; y$ (8T ) = y(7.5T) = 0.8187; L
These values give the response at the midpoints between pairs of consecutive sampling points. Note that
by varying the value of D between 0 and 1, it is possible to find the response at any point between two
consecutive sampling points.

Review Example 3.2 Reconsider the sampled-data system of Example 3.2 (Fig. 3.15). The characteristic
polynomial of the system is
D(z) = 1 + Gh0G(z) = z2 – az + b
where a = 1 + e–2T – 0.5K (T + 0.5e–2T – 0.5)
b = e–2T + 0.5K (0.5 – 0.5e–2T – Te–2T)
Employing stability constraints (2.75a) of the Jury stability criterion, we get
(i) D(1) = 1 – a + b > 0
(ii) D(– 1) = 1 + a + b > 0
(iii) b < 1
Thus, for a stable system, 1+a+b>0 (3.74a)
1–b>0 (3.74b)
1–a+b>0 (3.74c)
Substituting for a and b into (3.74a) and solving for K, yields
4
K< -2 T
T - (1 - e )/(1 + e -2 T )
 182 Digital Control and State Variable Methods
From (3.74b), we get
2
K< 2T 2T
0.5 (Te )/(1 e )
and from (3.74c),
e–2T < 1
Table 3.4 indicates the values of K obtained for various values of T from (3.74a) and (3.74b). A sketch of
T versus the boundary value of K for a stable system is shown in Fig. 3.72. Note that lim K = ¥.
T® 0

Table 3.4 Stability requirements for Review Example 3.2

T Inequality Inequality Value of K

(3.74a) (3.74b) for stability
0.01 K < 3986844 K < 401.4 K < 401.4
0.1 K < 12042.7 K < 41.378 K < 41.378
1.0 K < 16.778 K < 5.8228 K < 5.8228
1.0 K<4 K<4 K<4

Unstable region

T
Fig. 3.72 T versus boundary value of K for stability

Review Example 3.3 Consider a digital control function

N ( z ) b 0 z n + b 1z n -1 + L + b n
D(z) = = n (3.75)
D( z) z + a 1 z n -1 + L + a n
For direct realization of this control function, the computer must store the parameters a1, a2, ..., an, b0,
b1, ..., bn (refer Fig. 3.18). If the machine uses fixed-point arithmetic, the parameter values will be rounded
off to the accuracy of the machine. Thus, a program designed to realize Eqn. (3.75) actually realizes3
( b + db )z n + (b 1 + db 1 )z n -1 + L + (b n + db n )
D$ (z) = 0 n 0
z + (a 1 + da 1 )z n -1 + L + (a n + da n )
To study the effects of this realization on the dynamic response, we consider the characteristic equation
and determine how a particular root changes when a particular parameter undergoes change.
D (z, a1, a2, ..., an) = zn + a1 zn – 1 + L + an = 0 (3.76)

3
In addition to parameter quantization error, accuracy of a realization is affected by the error due to quantization of
the input signal, and the error due to accumulation of round-off errors in arithmetic operations.
 Models of Digital Control Devices and Systems 183
is the characteristic equation with roots l1, l2, K, ln:
D (z, a1, a2, ..., an) = (z – l1) (z – l2) L (z – ln) (3.77)
We shall consider the effect of parameter aj on the root lk. By definition,
D(lk, aj) = 0
If aj is changed to aj + daj, then lk also changes and the new polynomial is
¶D ¶D
D(lk + dlk, aj + daj) = D(lk, aj) + dlk + daj + L = 0
¶z z = lk ¶a j
z = lk
Neglecting the higher-order terms, we obtain

dlk = –
FG ¶D / ¶a IJ j
daj (3.78)
H ¶D / ¶z K z = lk
From Eqn. (3.76),
¶D
= lnk- j
¶a j
z = lk
and from Eqn. (3.77),
¶D
= P (lk – li)
¶z z = lk i=k

Therefore Eqn. (3.78) gives

lnk- j
dlk = – da j
P (l k - l i )
i ¹ k
A measure of the sensitivity of the root lk to the parameter aj is:
¶l k
Sal kj =
¶a j /a j
- lnk- j a j
= (3.79)
P (l k - l i )
i ¹ k

Following observations are made from Eqn. (3.79).

(i) The numerator term in Eqn. (3.79) varies with j, the index number of the parameter whose variation is
under consideration. For a stable system, lk < 1 and, therefore, the numerator term in Eqn. (3.79) is
largest for j = n. Therefore, the most sensitive parameter in the characteristic equation (3.76), is an.
(ii) The denominator in Eqn. (3.79) is the product of vectors from the characteristic roots to lk. Thus, if all
the roots are in a cluster, the sensitivity is high.
In the cascade and parallel realizations the coefficients, mechanized in the algorithm are poles themselves;
these realizations are generally less sensitive than the direct realization.

Problems
3.1 Find Y(z)/R(z) for the sampled-data closed-loop system of Fig. P3.1.
184 Digital Control and State Variable Methods

r ( t) + e * (t) u * (t) y (t)

G h0 (s) G 1 (s ) G h0 (s) G 2 (s )
– T T

H (s )

Fig. P3.1

3.2 For the sampled-data feedback system with digital network in the feedback path as shown in Fig. P3.2, find
Y(z)/R(z).

r (t) r (k) + e * (t) y (t)

G h0 (s) G (s)
T – T

y (k)
H (z )
T
Fig. P3.2

3.3 Find Y(z) for the sampled-data closed-loop system of Fig. P3.3.

r (t) + u * (t) y (t)

G 1 (s ) G h0 (s) G 2 (s)
– T

H (s )

Fig. P3.3

3.4 Obtain the z-transform of the system output for the block diagram of Fig. P3.4.

H 2 (s)

r (t ) e (k ) u * (t ) + y ( t)
+ +
H 1 (s ) D (z ) G h0 (s) G p (s )
– T T

Fig. P3.4

3.5 Obtain the transfer function Y(z)/R(z) of the closed-loop control system shown in Fig. P3.5. Also obtain the
transfer function between X(z) and R(z).
r ( t) + + e (k) u * (t) x ( t) y (t)
D (z) G h0 (s) G 1 (s ) G 2 (s )
– – T T

Fig. P3.5
 Models of Digital Control Devices and Systems 185
3.6 Consider the block diagram of a digital control system shown in Fig. P3.6; r(t) stands for reference input and
w(t) for disturbance. Obtain the z-transform of the system output when r(t) = 0.
w (t)

+ y (t)
r ( t) + e (k) u * ( t) +
D (z) G h0 (s) G (s)
– T T

Fig. P3.6

3.7 Shown in Fig. P3.7 is the block diagram of the servo control system for one of the joints of a robot. With D(z)
= 1, find the transfer function model of the closed-loop system. Sampling period T = 0.25 sec.
Compensator Preamplifier Gears

qR (k ) + 1 1 qM 1 qL ( t)
D (z) D /A 20
– s+1 s 20

Motor

qL (k )
Shaft
encoder

Fig. P3.7

3.8 The plant of the speed control system shown in Fig. P3.8 consists of load, armature-controlled dc motor and a
power amplifier. Its transfer function is given by
w (s) 185
=
V( s ) 0.025 s + 1
Find the discrete-time transfer function w(z)/wr(z) for the closed-loop system. Sampling period T = 0.05 sec.

KF

+
w r (k ) + + v(t) w(t)
KP D/A Plant
–

w (k ) Shaft
encoder

Fig. P3.8
186 Digital Control and State Variable Methods
3.9 For the system shown in Fig. P3.9, the computer solves the difference equation u(k) = u(k – 1) + 0.5 e(k), where e(k)
is the filter input and u(k) is the filter output. If the sampling rate fs = 5 Hz, find Y(z)/R(z).

r (k ) + e(k ) u (k ) 1 y (t)
Digital
D/A
filter s (s + 1)
–

y (k )

A/D

Fig. P3.9

3.10 Consider the sampled-data system shown in Fig. P3.10. Find Y(z)/R(z) when (i) tD = 0.4 sec, (ii) tD = 1.4 sec.

Fig. P3.10

3.11 Figure P3.11 shows an electrical oven provided with temperature measurement by a thermocouple and having
a remotely controlled, continuously variable power input. The task is to design a microprocessor-based control
system to provide temperature control of the oven.

Temperature-
measuring device Microcontroller
Oven

Power control unit

Continuously-variable
electrical input Power supply
Fig. P3.11

The functions within the control loop can be broken down as follows:
(i) sampling of the output of thermocouple;
(ii) transfer of temperature signal into the computer;
(iii) comparison of the measured temperature with the stored desired temperature, to form an error signal;
(iv) operation on the error signal by an appropriate algorithm, to form a control signal; and
(v) processing of the control signal and its transfer through the interface to the power control unit.
Suggest suitable hardware to implement these control-loop functions. Make a sketch of the system showing
how the hardware is connected.
 Models of Digital Control Devices and Systems 187
3.12 (a) A unity feedback system has the open-loop transfer function
5
G(s) =
s(s + 1)( s + 2)
Using the Routh stability criterion, show that the closed-loop system is stable.
(b) A sampler and ZOH are now introduced in the forward path (Fig. P3.12). Show that the stable linear
continuous-time system becomes unstable upon the introduction of sampler and ZOH.

r (t ) + 5 y (t)
G h0(s)
T = 1 sec s(s + 1) ( s + 2)
–

Fig. P3.12

3.13 The characteristic equation of a linear digital system is

z3 – 0.1 z2 + 0.2Kz – 0.1K = 0
Using Jury stability criterion, determine the values of K > 0 for which the system is stable.
3.14 Compare the stability properties of the system shown in Fig. P3.14 with (i) T = 0.5, (ii) T = 1. Assume K > 0.
3.15 The block diagram of a digital control system is shown in Fig. P3.15. Apply the Jury stability criterion to
determine the range of values that K > 0 can have for a stable response. Also show graphically how these values
are affected by the sampling period T.

R (s) + 1 – e– Ts K Y (s)
T s (s + 3)
–

Fig. P3.14

R (s) + K Y (s)
G h0(s)
T s(s + 1)
–

Fig P3.15

3.16 Consider the system shown in Fig. P3.16. Using Jury stability criterion, find the range of K > 0 for which the
system is stable.
188 Digital Control and State Variable Methods

R (s) + 1 – e– Ts Ke–Ts Y (s )
T = 1 sec s s(s + 1)
–

Fig. P3.16
3.17 (a) A unity feedback system has the open-loop transfer function
Y (s ) 4500 K
G(s) = = ; K = 14.5
R( s) s(s + 361.2)
Find the response y(t) of the system to a unit-step input.
(b) A sampler and ZOH are now introduced in the forward path (Fig. P3.17). For a unit-step input, determine
the output y(k) for first five sampling instants when (i) T = 0.01 sec, and (ii) T = 0.001 sec. Compare the
result with that obtained earlier in part (a) above.

r (t) + y (t)
G h0(s) G (s)
T
–

Fig. P3.17
3.18 For the sampled-data system shown in Fig. P3.18, find the output y(k) for r(t) = unit step.

r (t) + 1 y (t)
ZOH G (s) =
T = 1 sec s+1
–

1
H (s ) =
s

Fig. P3.18
3.19 For the sampled-data system of Fig. P3.19, find the response y(kT); k = 0, 1, 2, ..., to a unit-step input r(t). Also,
obtain the output at the midpoints between pairs of consecutive sampling points.

r (t) + y ( t)
1
ZOH
T = 1 sec s+1
–

Fig. P3.19
3.20 Consider the digital controller defined by
U ( z) 4( z - 1) ( z 2 + 12
. z + 1)
D(z) = =
E (z) ( z + 0.1) ( z 2 - 0.3z + 0.8)
 Models of Digital Control Devices and Systems 189
Realize this digital controller in the cascade scheme and in parallel scheme. Use one first-order section and one
second-order section.
3.21 Consider the digital controller defined by
U (z) 10( z 2 z 1)
D(z) = = 2
E ( z) z ( z 0.5)( z 0.8)
Draw a parallel realization diagram.
3.22 Consider the temperature control system shown in Fig. P3.22a. A typical experimental curve, obtained by
opening the steam valve at t = 0 from fully closed position to a position that allows a flow Qm of 1 kg/min with
initial sensor temperature q of 0 ºC is shown in Fig. P3.22b.
(a) Approximate the process reaction curve by a first-order plus dead-time model using two-points method of
approximation.
(b) Calculate the QDR tuning parameters for a PID controller. The PID control is to be carried out with a
sampling period of 1 min on a computer control installation.
Dial
10 °C
20 °C
Environment
30 °C
40 °C
Steam
boiler Radiator
Valve
Room

(a) Steam exhaust

q, °C

qss = 30

0.632 qss

0.283 qss

0 25 65 t, min
(b)
Fig. P3.22
3.23 Consider the liquid-level control system shown in Fig. 1.6. The digital computer was programmed to act as
adjustable-gain proportional controller with a sampling period of T = 10 sec. The proportional gain was
increased in steps. After each increase, the loop was disturbed by introducing a small change in set-point, and
the response of the controlled variable (level in tank 2) was observed. The proportional gain of 4.75 resulted in
oscillatory behaviour, with amplitude of oscillations approximately constant. The period of oscillations
measured from the response is 800 sec.
The PC implements the digital PI control algorithm. Determine tuning parameters for the controller:
LM
Du(k) = u(k) – u(k – 1) = Kc e( k ) - e( k - 1) +
T
e(k )
OP
N TI Q
190 Digital Control and State Variable Methods
where
u(k) = output of controller at kth sampling instant
Du(k) = change in output of controller at kth sampling instant
e(k) = error at kth sampling instant
T = sampling time
TI = integral time
Kc = proportional gain
3.24 A traffic light controller is to be designed for a road, partly closed to traffic for urgent repair work (Fig. P3.24).
North traffic light will go GREEN for 30 sec with South traffic light giving RED signal. For the next 15 sec,
both the traffic lights will give RED signals. Thereafter South traffic light will go GREEN for 30 sec with North
traffic light giving RED signal. Both the traffic lights will give RED signal for the next 15 sec. Then this cycle
repeats.
Develop a PLC ladder diagram that accomplishes this objective.

Fig. P3.24

3.25 Consider the tank system of Fig. P3.25. Valve V1 opens on pressing a pushbutton PB1 and liquid begins to fill
the tank. At the same time, the stirring motor M starts operations. When the liquid level passes LL2 and reaches
LL1, the valve V1 closes and the stirring motor stops. When PB1 is pressed again, the valve V2 opens and starts
draining the liquid. When the liquid level drops below LL2, valve V2 closes. This cycle is repeated five times.
A Buzzer will go high after 5 repetitions. The buzzer will be silenced by pressing push button PB2. The process
will now be ready to take up another filling-stirring-draining operation under manual control. Develop a PLC
ladder diagram that accomplishes this objective.
3.26 A control circuit is to be developed to detect and count the number of products being carried on an assembly
line (Fig. 3.26). A sensor activates a counter as a product leaves the conveyor and enters the packaging section.
When the counter counts five products, the circuit energizes a solenoid. The solenoid remains energized for a
period of 2 seconds; the time being measured by a software timer. When the set-time has lapsed, the solenoid is
deenergized, causing it to retract; and the control circuit is ready for the next cycle.
Develop a suitable PLC ladder diagram.
 Models of Digital Control Devices and Systems 191

M
Liquid
V1
LL1

LL2 Solenoid

Sensor

Conveyor
V2

Fig. P3.25 Fig. P3.26

3.27 In the system of Fig. P3.27, a PLC is used to start and stop the motors of a segmented conveyor belt. This allows
only belt sections carrying an object to move. Motor M3 is kept ON during the operation. Position of a product
is first detected by proximity switch S3, which switches on the motor M2. Sensor S2 switches on the motor M1
upon detection of the product. When the product moves beyond the range of sensor S2, a timer is activated and
when the set-time of 20 sec has lapsed, motor M2 stops. Similarly, when the product moves beyond the range
of sensor S1, another timer is activated and when the set-time of 20 sec (for unloading the product) has lapsed,
motor M1 stops.
Develop a suitable ladder diagram for control.

S3
S2
S1

M3
M2
M1
Fig. P3.27

3.28 The system of Fig. P3.28 has the objective of drilling a hole in workpiece moved on a carriage. When the start
button PB1 is pushed and LS1 is ON (workpiece loaded), feed carriage motor runs in CW direction, moving the
carriage from left to right. When the work comes exactly under the drill, which is sensed by limit switch LS2,
the motor is cut-off and the work is ready for drilling operation. A timer with a set-time of 7 sec is activated.
When timer set-value has lapsed, the motor reverses, moving the carriage from right to left. When the work
piece reaches LS1 position, the motor stops.
The motor can be stopped by a stop pushbutton while in operation.
Develop a suitable ladder diagram for PLC control.
192 Digital Control and State Variable Methods

Drill

LS1 LS2

Clamp
Fig. P3.28
 Design of Digital Control
4 Algorithms

4.1 INTRODUCTION

During recent decades, the design procedures for analog control systems have been well formulated and a
large body of knowledge accumulated. The analog-design methodology, based on conventional techniques
of root locus and Bode plots or the tuning methods of Ziegler and Nichols, may be applied to designing
digital control systems. The procedure would be to first design the analog form of the controller, or compen-
sator, to meet a particular set of performance specifications. Having done this, the analog form can be
transformed to a discrete-time formulation. This approach is based on the fact that a digital system with a
high sampling rate approximates to an analog system. The justification for using digital control under these
circumstances must be that the practical limitations of the analog controller are overcome, the implementa-
tion cheaper, or that the supervisory control and communications more easily implemented.
However, the use of high sampling rates wastes computer power, and can lead to problems of arithmetic
precision, etc. One is, therefore, driven to find methods of design which take account of the sampling process.
The alternative approach is to design controllers directly in the discrete-time domain, based on the speci-
fications of closed-loop system response. The controlled plant is represented by a discrete-time model which
is a continuous-time system observed, analysed, and controlled at discrete intervals of time. This approach
provides a direct path to the design of digital controllers. The features of direct digital design are that sample
rates are generally lower than for discretized analog design, and the design is directly ‘performance based’.
Figure 4.1 shows the basic structure of a digital control system. The design problem generally evolves around
the choice of the control function D(z), in order to impart a satisfactory form to the closed-loop transfer
function. The choice is constrained by the form of the function Gh0G(z) representing the fixed process elements.
r (t) + y (t)
D (z) G h0 (s) G (s)
– T T
Control ZOH Process
algorithm

H (s)

Sensor
Fig. 4.1 Basic structure of a digital control system
 194 Digital Control and State Variable Methods
A wide variety of digital-design procedures is available; these fall into two categories:
(i) direct synthesis procedures; and
(ii) iterative design procedures.
The direct synthesis procedures assume that the control function D(z) is not restricted in any way by
hardware or software limitations, and can be allowed to take any form demanded by the nature of the fixed
process elements and the specifications of the required system performance. This design approach has found
wider applications in digital control systems—than has the equivalent technique used with the analog sys-
tems. In a digital control system, realization of the required D(z) may involve no more than programming a
special purpose software procedure. With analog systems, the limitation was in terms of the complications
involved in designing special purpose analog controllers.
The design obtained by a direct synthesis procedure will give perfect nominal performance. However,
the performance may be inadequate in the field because of the sensitivity of the design to plant disturbances and
modelling errors. It is important that a control system is robust in its behaviour with respect to the discrepancies
between the model and the real process, and uncertainties in disturbances acting on the process. Robustness
property of some of the standard control structures, such as a three-term (PID) control algorithm, has been very
well established. The design of such algorithms calls for an iterative design procedure where the choice of
control function D(z) is restricted to using a standard algorithm with variable parameters; the designer must
then examine the effect of the choice of controller parameters on the system performance, and make an appro-
priate final choice. The iterative design procedures for digital control systems are similar to the techniques
evolved for analog system design, using root locus and frequency response plots.
Figure 4.2 summarizes the basic routes to the design of digital controllers for continuous-time processes.

Continuous-time Process
Differential Experimentally obtained Differential
equations discrete data equations

Continuous-time
plant model Continuous-time
Identification of discrete-
model
time model

Continuous-time model Discrete-time model

of plant with sample
and hold Design Design Design Design

Mapping Mapping Root z-plane

Design
1 + wT/2 z = esT locus synthesis
z= and on
1 – wT/2
Continuous-time and Bode plots z-plane
controller Bode plots

Discrete-time
approximation

Digital controller

Fig. 4.2 Basic routes to the design of digital controllers

 Design of Digital Control Algorithms 195
The route : continuous-time modelling ® continuous-time control design ® discrete-time approximation of
the controller, was considered in Chapter 2 (refer Example 2.17). This chapter is devoted to the following route:
· Continuous-time modelling ® discrete-time approximation of the model ® discrete-time control
design.
Plant models can be obtained from the first principles of physics. The designer may, however, turn to
the other source of information about plant dynamics, which is the data taken from experiments directly
conducted to excite the plant, and to measure the response. The process of constructing models from experi-
mental data is called system identification. An introduction to system identification and adaptive control is
given later in Chapter 10.
One obvious, but fundamental, point is that control design always begins with a sufficiently accurate
mathematical model of the process to be controlled. For a typical industrial problem, the effort required for
obtaining a mathematical model of the process to be controlled, is often an order of magnitude greater
than the effort required for control design proper. Any control-design method that requires only a simple
process model, therefore, has a high appeal to those faced with real industrial control problems. Tunable PID
controllers which require only simple process models, have already been described in Chapter 3.
The design approaches discussed in this chapter, use z-transform as the background tool. In Chapters 7
and 8, we will solve the identical problems using state-space formulation.
The discussion in this chapter is based on the material that the student has already taken in a prerequisite,
introductory course on control systems. The presentation given in this chapter is not sufficient to learn
control system design for the first time; rather, it states, only concisely, the key concepts and relationships
from the continuous-time control for ready reference, as we move to the new concepts of discrete-time
control. For an indepth treatment of the principles of control system design, see the companion book [155].
In this chapter, the following class of feedback systems will be under consideration.
· The system can be represented by a unity-feedback structure shown in Fig. 4.3b.
· The open-loop transfer function G(s) in Fig. 4.3b has no poles in the right half s-plane.
· The feedback system of Fig. 4.3b is desired to be an underdamped system.

r (t) + y (t)
G h0 (s) G (s)
– T

H (s)

(a) A nonunity feedback discrete-time system

r (t) + e * (t) y (t)

G h0 (s) G (s)
– T

(b) A unity feedback discrete-time system

Fig. 4.3

In Section 1.4, we have described a generalized, operational block diagram of a feedback system
(Revisiting the section will be helpful). It was shown that the generalized, non-unity feedback block diagram
 196 Digital Control and State Variable Methods

can be converted to a unity-feedback structure when the sensor transfer function is equal to the transfer
function representing the reference input elements. This, in fact, is quite common; the two transfer functions
in many control system designs are assumed to have zero-order dynamics and are equal.
The majority of the systems we deal with, in practice, belong to the class under consideration in this
chapter. However, the control system design techniques presented in the chapter, are not limited to this class.
There are control design problems wherein the sensor transfer function H(s) (of first or higher-order), is
explicitly present in the feedback path (Fig. 4.3a); the open-loop tranfer function G(s) or G(s)H(s), has poles
in the right half s-plane; and/or the oscillations in feedback system cannot be tolerated, and the system is
required to be overdamped. A detailed treatment of such problems is given in the companion book [155]; the
principles given therein for continuous-time control, have direct extension to discrete-time control. Our
decision to leave these extensions to the reader, is prompted only by the desire to save space for other topics.

4.2 z-PLANE SPECIFICATIONS OF CONTROL SYSTEM DESIGN

The central concerns of controller design are for good relative stability and speed of response, good steady-
state accuracy, and sufficient robustness. Requirements on time response need to be expressed as constraints
on z-plane pole and zero locations, or on the shape of the frequency response in order to permit design in the
transform domain. In this section, we give an outline of specifications of controller design in the z-plane.
Our attention will be focused on the unity feedback systems1 of the from shown in Fig. 4.3b, with the
open-loop transfer function Gh0G(z) = Z [Gh0(s)G(s)], having no poles outside the unit circle in the
z-plane. Further, the feedback system of Fig. 4.3b is desired to be an underdamped system.
The nature of transient response of a linear control system is revealed by any of the standard test signals—
impulse, step, ramp, parabola—as this nature is dependent on system poles only and not on the type of the
input. It is, therefore, sufficient to specify the transient response to one of the standard test signals; a step is
generally used for this purpose. Steady-state response depends on both the system and the type of input.
From the steady-state viewpoint, the ‘easiest’ input is generally a step since it requires only maintaining the
output at a constant value, once the transient is over. A more difficult problem is the tracking of a ramp input.
Tracking of a parabola is even more difficult since a parabolic funciton is one degree faster than the ramp
funciton. In practice, we seldom find it necessery to use a signal faster than a parabolic function; character-
istics of actual signals which the control systems encounter, are adequately represented by step, ramp, and
parabolic functions.

4.2.1 Steady-State Accuracy

Steady-state accuracy refers to the requirement that after all transients become negligible, the error between
the reference input r and the controlled output y must be acceptably small. The specification on steady-state
tk
accuracy is often based on polynomial inputs of degree k: r(t) = m(t ) . If k = 0, the input is a step of unit
k!
amplitude; if k = 1, the input is a ramp with unit slope; and if k = 2, the input is a parabola with unit second
derivative. From the common problems of mechanical motion control, these inputs are called, respectively,
position, velocity, and acceleration inputs.

1
It is assumed that the reader is familiar with the design of unity and nonunity feedback continuous-time systems.
With this background, the results presented in this chapter for unity feedback discrete-time systems, can easily be
extended for the nonunity feedback case.
 Design of Digital Control Algorithms 197
For quantitative analysis, we consider the unity-feedback discrete-time system shown in Fig. 4.3b. The
steady-state error is the difference between the reference input r(k) and the controlled output y(k), when
steady-state is reached, i.e., steady-state error
* = lim e(k) = lim [r(k) – y(k)]
ess (4.1a)
k ®¥ k®¥
Using the final value theorem (Eqn. (2.52)),
* = lim [(z – 1)E(z)]
ess (4.1b)
z®1
provided that (z – 1) E(z) has no poles on the boundary and outside of the unit circle in the z-plane.
For the system shown in Fig. 4.3b, define

Gh0G(z) = (1– z–1) Z LM

G( s ) OP
s N Q
Y (z ) Gh 0 G( z )
Then we have =
R( z ) 1 + Gh 0 G ( z )
R( z )
and E(z) = R(z) – Y(z) = (4.2)
1 + Gh 0 G ( z )
By substituting Eqn. (4.2) into Eqn. (4.1b), we obtain
ess* = lim [(z – 1)E(z)] (4.3a)
z®1

= lim
LM(z - 1) R(z) OP (4.3b)
z®1 N 1 + G G( z ) Q h0
Thus, the steady-state error of a discrete-time system with unity feedback, depends on the reference input
signal R(z), and the forward-path transfer function Gh0G(z). By the nature of the limit in Eqns (4.3), we see
that the result of the limit can be zero, or can be a constant different from zero. Also, the limit may not exist,
in which case, the final-value theorem does not apply. However, it is easy to see from basic definition (4.1a)
that ess* = ¥ in this case anyway, because E(z) will have a pole at z = 1 that is of order higher than one.
Discrete-time systems, having a finite nonzero steady-state error when the reference input is a zero-order
polynomial input (a constant), are labelled ‘Type-0’. Similarly, a system that has finite nonzero steady-state
error to a first-order polynomial input (a ramp), is called a ‘Type-1’ system, and a system with finite nonzero
steady-state error to a second-order polynomial input (a parabola), is called a ‘Type-2’ system.
Let the reference input to the system of Fig. 4.3b be a step function of magnitude unity. The z-transform
of discrete form of r(t) = m (t) is (refer Eqn. (2.40))
z
R(z) = (4.4a)
z 1-
Substituting R(z) into Eqn. (4.3b), we have
* = lim 1 1
ess =
z ® 1 1 + Gh 0 G( z ) 1 + lim Gh 0 G( z )
z®1
In terms of the position error constant Kp, defined as
Kp = lim Gh0G(z) (4.4b)
z®1
the steady-state error to unit-step input becomes
* = 1
ess (4.4c)
1 + Kp
 198 Digital Control and State Variable Methods
For a ramp input r(t) = tm (t); the z-transform of its discrete form is (refer Eqn. (2.42))
Tz
R(z) = (4.5a)
( z - 1)2
Substituting into Eqn. (4.3b), we get
* = lim
T 1
ess =
z ® 1 ( z - 1)[1 + Gh 0 G( z )]
lim
z - 1 LM
Gh 0 G( z) OP
z ®1 T N Q
In terms of velocity error constant Kv, defined as
1
Kv = lim [(z – 1)Gh0G(z)] (4.5b)
T z®1
the steady-state error to unit-ramp input becomes
* =
1
ess (4.5c)
Kv
For a parabolic input r(t) = (t2/2) m (t); the z-transform of its discrete form is (from Eqns (2.41)–(2.42))
T 2 z( z + 1)
R(z) = (4.6a)
2( z - 1) 3
Substituting into Eqn. (4.3b), we get
* = lim
T2 1
ess =
z®1 ( z - 1)2 [1 + Gh 0 G( z )] LMF z - 1I 2
OP
lim Gh 0 G ( z)
z ®1
NH T K Q
In terms of acceleration error constant Ka, defined as
1
Ka = 2 lim [(z – 1)2 Gh0G(z)] (4.6b)
T z®1
the steady-state error to unit-parabolic input becomes
* =
1
ess (4.6c)
Ka
As said earlier, discrete-time systems can be classified on the basis of their steady-state response to
polynomial inputs. We can always express the forward-path transfer function Gh0G(z) as
K P( z - zi )
i
Gh0G(z) = ; pj ¹ 1, zi ¹ 1 (4.7)
( z - 1) N P( z - p j )
j
N
Gh0G(z) in Eqn. (4.7) involves the term (z – 1) in the denominator. As z ® 1, this term dominates in
determining the steady-state error. Digital control systems are, therefore, classified in accordance with the
number of poles at z = 1 in the forward-path transfer function, as described below.
Type-0 System If N = 0, the steady-state errors to various standard inputs, obtained from Eqns (4.1)–(4.7), are

1 K ( z – zi )
i
in response to unit-stepinput; Kp
1 Kp (z – p j )
* =
ess j z 1 (4.8a)
in response to unit- rampinput
in response to unit- parabolicinput
 Design of Digital Control Algorithms 199
Thus a system with N = 0, or no pole at z = 1 in Gh0G(z), has a finite non-zero position error, and infinite
velocity and acceleration errors at steady-state.
Type-1 System If N = 1, the steady-state errors to various standard inputs are
0 in response to unit-step input;
K
( z – zi )
* =
1 T i
ess in response to unit- rampinput; K v (4.8b)
Kv (z – p j )
j z 1
in response to unit- parabolic input
Thus a system with N = 1, or one pole at z = 1 in Gh0G(z), has zero position error, a finite non-zero velocity
error, and infinite acceleration error at steady-state.
Type-2 System If N = 2, the steady-state errors to various standard inputs are
0 in response to unit-step input
0 in response to unit- ramp input
* =
ess K (4.8c)
( z – zi )
1 T2 i
in response to unit- parabolic input; K a
Ka (z – p j )
j z 1

Thus a system with N = 2, or two poles at z = 1 in Gh0G(z), has zero position and velocity errors, and a finite
non-zero acceleration error at steady-state.
Steady-state errors for various inputs and systems are summarized in Table 4.1.

Table 4.1 Steady-state errors for various inputs and systems

Type of input Steady-state error

Type-0 system Type-1 system Type-2 system

1
Unit-step 0 0
1 + Kp
1
Unit-ramp ¥ 0
Kv
1
Unit-parabolic ¥ ¥
Ka
1 1
Kp = lim Gh 0G ( z ); Kv = lim [( z - 1) Gh0G( z )]; Ka = 2 lim [( z - 1) 2 Gh0G ( z )]
z ®1 T z®1 T z® 1

The development above indicates that, in general, increased system gain K, and/or addition of poles at
z = 1 to the open-loop transfer function Gh0G(z), tend to decrease steady-state errors. However, as will be
seen later in this chapter, both large system gain and the poles at z = 1 in the loop transfer function, have
destabilizing effects on the system. Thus, a control system design is usually a trade off between steady-state
accuracy and acceptable relative stability.
 200 Digital Control and State Variable Methods

Example 4.1 In the previous chapter, we have shown that sampling usually has a deterimental effect on
the transient response and the relative stability of a control system. It is natural to ask what the effect of
sampling on the steady-state error of a closed-loop system will be? In other words, if we start out with a
continuous-time system and then add S/H to form a digital control system, how would the steady-state errors of
the two systems compare, when subject to the same type of input?
Let us first consider the system of Fig. 4.3b without S/H. Assume that the process G(s) is represented by
Type-1 transfer function:
K (1 + t a s)(1 + t b s) L (1 + t m s)
G(s) =
s(1 + t 1 s )(1 + t 2 s ) L (1 + t n s)
having more poles than zeros.
The velocity error constant
Kv = lim sG(s) = K
s®0
The steady-state error of the system to unit-step input is zero, to unit-ramp input is 1/K, and to unit-parabolic
input is ¥.
We now consider the system of Fig. 4.3b with S/H:

Gh0G(z) = (1 – z–1) Z
LM K (1 + t s)(1 + t s) L (1 + t s) OP
a b m
2
N s (1 + t s)(1 + t s) L (1 + t s) Q
1 2 n

) Z LM + O
K K
= (1 – z–1 1
2
N s s + terms due to the non-zero poles PQ
= (1 – z–1)M
L KTz + K z + terms due to the non-zero poles OP
1
2
N (z - 1) z - 1 Q
It is important to note that the terms due to the non-zero poles do not contain the term (z – 1) in the
denominator. Thus, the velocity error constant is
1
Kv = lim [(z – 1)Gh0G(z)] = K
T z®1
The steady-state error of the discrete-time system to unit-step input is zero, to unit-ramp input is 1/K, and
to unit-parabolic input is ¥. Thus, for a Type-1 system, the system with S/H has exactly the same steady-
state error as the continuous-time system with the same process transfer function (this, in fact, is true for
Type-0 and Type-2 systems also).
Equations (4.5b) and (4.6b) may purport to show that the velocity error constant and the acceleration
error constant of a digital control system depend on the sampling period T. However, in the process of
evaluation, T gets cancelled, and the error depends only on the parameters of the process and the type
of inputs.

4.2.2 Transient Accuracy

Transient performance in time domain is defined in terms of parameters of the system response to a step in
command input. The most frequently used parameters are: rise time, peak time, peak overshoot, and setting
time. Figure 4.4 shows a typical unit-step response of a control system.
For underdamped systems, the rise time, tr, is normally defined as the time required for the step responce
to rise from 0 to 100% of its final value.
 Design of Digital Control Algorithms 201
y (t)

Allowable
tolerance
Mp
1.0

tr t
tp
ts

Fig. 4.4 Typical unit-step response of a digital control system

The peak overshoot, Mp, is the peak value of the response curve measured from unity. The time at which
peak occurs is referred to as the peak time, tp.
The time required for the response to damp out all transients, is called the settling time, ts. Theoretically,
the time taken to damp out all transients may be infinity. In pratice, however, the transient is assumed to be
over when the error is reduced below some acceptable value. Typically, the acceptable level is set at two or
five percent of the final value.
The use of root locus plots for the design of digital control systems necessitates the translation of time-
domain performance specifications into desired locations of closed-loop poles in the z-plane. However, the
use of frequency response plots necessitates the translation of time-domain specifications in terms of
frequency response features such as bandwidth, phase margin, gain margin, resonance peak, resonance
frequency, etc.
Specifications in Terms of Root Locations in z-plane Our approach is to first obtain the transient
response specifications in terms of characteristic roots in the s-plane, and then use the relation
z = esT (4.9)
to map the s-plane characteristic roots to the z-plane.
The transient response of Fig. 4.4 resembles the unit-step response of an underdamped second-order system
Y ( s) w 2n
= 2 (4.10)
R( s) s + 2zw n s + w n2
where
z = damping ratio, and wn = undamped natural frequency.
The transient response specifications in terms of rise time tr, peak time tp, peak overshoot Mp, and settling
time t s can be approximated to the parameters z and wn of the second-order system defined by Eqn. (4.10),
using the following correlations2:
p - cos -1 z
tr (0% to 100%) = (4.11)
wn 1 - z2

2
Chapter 6 of reference [155].
 202 Digital Control and State Variable Methods

p
tp = (4.12)
wn 1 - z2

Mp = exp ( - pz 1 - z2 ) (4.13)
4
ts (2% tolerance band) = (4.14)
zw n
Peak overshoot is used mainly for relative stability. Values in excess of about 40% may indicate that the
system is dangerously close to absolute instability. Many systems are designed for 5 to 25% overshoot. No
overshoot at all is sometimes desirable. However, this usually penalizes the speed of response needlessly.
The specification on speed of response in terms of tr, tp and/or ts, should be consistent as all these depend
on z and wn. The greater the magnitude of wn when z is constant, the more rapidly does the response
approach the desired steady-state value. The value of wn is limited by measurement noise considerations—
a system with large wn has large bandwidth and will, therefore, allow the high frequency noise signals to
affect its performance.
We need to now convert the specifications on z and wn into guidelines on the placement of poles and
zeros in the z-plane, in order to guide the design of digital controls. We do so through the mapping (4.9).
Figure 4.5 illustrates the translation of specifications on z and wn to the characteristic root locations in the
z-plane (referring to Section 2.14 will be helpful). The s-plane poles
s1,2 = – zwn ± jwn 1 - z 2 = – zwn ± jwd (4.15a)
for constant z, lie along a radial line in the s-plane (Fig. 4.5a). In the z-plane,
1- z 2
z1,2 = e -zw n T e ± jw n T = r e ± jq (4.15b)

jw
j3ws/2
Im
Constant-w n
locus
jws/2

jwd
Constant- z
locus r

s Re
q
Constant- w n
locus
– jws/2 Constant- z
locus
Unit
circle
– j3ws/2

(a) (b)
Fig. 4.5 Mapping of s-plane patterns on the z-plane
 Design of Digital Control Algorithms 203
-zw n T
The magnitude of z (i.e., the distance to the origin) is r = e and the angles with the positive real axis of
the z-plane, measured positive in the counterclockwise direction, are q = wnT 1 - z 2 . It should be observed
that the z-plane pole locations depend on the s-plane positions, as well as the sampling interval T.
As wn increases for a constant-z, the magnitude of z decreases and the phase angle increases; constant-z
locus is a logarithmic spiral in the z-plane (Fig. 4.5b). Increasing wn negatively, gives the mirror image.
In Fig. 4.5a, the s-plane has been divided into strips of width ws, where ws = 2p/T is the sampling
frequency. The primary strip extends from w = –ws/2 to + ws/2, and the complimentary strips extend from
– ws/2 to –3ws/2, ... , for negative frequencies, and from ws/2 to 3ws/2,..., for positive frequencies. We will
assume that the low-pass analog filtering characteristics of the continuous-time plant and the ZOH device,
attenuate the responses due to the poles in the complimentary strips; only the poles in the primary strip,
generally, need be considered.
Figure 4.5 illustrates the mapping of constant-z locus in the primary strip of the s-plane to the z-plane. As
the imaginary parts ± jwd = ± jwn 1 - z 2 of the s-plane poles move closer to the limit ± jws/2 of the
primary strip, the angles q = ± wdT = ± wnT 1 - z 2 of the z-plane poles approach the direction of the
negative real axis. The negative real axis in the z-plane, thus, corresponds to the boundaries of the primary
strip in the s-plane. Figure 4.5 also shows the mapping of a constant-wn locus, in the primary strip of the
s-plane, to the z-plane.
In the z-plane, the closed-loop poles must lie on the constant-z spiral to satisfy peak overshoot require-
ment, also the poles must lie on constant-wn curve to satisfy speed of response requirement. The intersection
of the two curves (Fig. 4.5b) provides the preferred pole locations, and the design aim is to make the root
locus pass through these locations.
If one chooses the following boundaries for the system response:
T = 1 sec
Peak overshoot £ 15% Þ z ³ 0.5
8
Settling time £ 25 sec Þ wn ³ ,
25
the acceptable boundaries for the closed-loop pole locations in z-plane are shown in Fig. 4.6.
In the chart of Fig. 4.6, the patterns are traced for various natural frequencies and damping ratios. Such a
chart is a useful aid in root-locus design technique. We will be using this chart in our design examples.
Dominant Poles Most control systems found in practice are of high order. The preferred locations of
closed-loop poles given by Fig. 4.5b, realize the specified transient performance only if the other closed-
loop poles and zeros of the system have negligible effect on the dynamics of the system, i.e., only if the
closed-loop poles corresponding to specified z and wn, are dominant.
In the following, we examine the relationship between the pole-zero patterns and the corresponding step-
responses of discrete-time systems. Our attention will be restricted to the step responses of the discrete-time
system with transfer function
Y (z) K ( z - z1 )( z - z2 ) K ( z - z1 )( z - z2 )
= = (4.16)
R( z ) ( z - p )( z - re )( z - re ) ( z - p)( z 2 - 2r cos q z + r 2 )
jq - jq

for a selected set of values of the parameters K, z1, z2, p, r and q.

 204 Digital Control and State Variable Methods

Fig. 4.6 Plot of an acceptable region for poles of a second-order system to satisfy dynamic
response specifications
Im z-plane
We assume that the roots of the equation
z = 0.5 z2 – 2r cosq z + r 2 = 0
are the preferred closed-loop poles corresponding to the
18° specified values of z and wn. Complex-conjugate pole
pairs corresponding to z = 0.5 with q = 18º, 45º and 72º,
Re will be considered in our study. The pole pair with q = 18º
is shown in Fig. 4.7.
To study the effect of zero location, we let z2 = p and
explore the effect of the (remaining) zero location z1 on
Unit circle the transient performance. We take the gain K to be such
Fig. 4.7 A complex-conjugate pole pair of that the steady-state output value equals the step size. For
the system (4.16) a unit-step input,
 Design of Digital Control Algorithms 205

Y(z) =
LM K ( z - z1 ) OP F z I (4.17)
Nz 2
- 2 r cos q z + r 2 Q H z - 1K
1 - 2 r cos q + r 2
with K=
(1 - z1 )
The major effect of the zero z1 on the step response y(k) is to change the peak overshoot, as may be seen
from the step responses plotted in Fig. 4.8a. Figure 4.8b shows plots of peak overshoot versus zero location
for three different cases of complex-conjugate pole pairs. The major feature of these plots is that the zero has
very little influence when on the negative real axis, but its influence is dramatic when it comes near +1.

500

Percent peak overshoot (log scale)

z1 = 0.9 300
200 (z = 0.5, q = 45° )

100
y (k )

(z = 0.5, q = 72° )
50
30
1.0
20
z1 = 0.8
10 (z = 0.5, q = 18° )

5
k – 1.0 – 0.5 0 0.5 1.0
Zero location
(a) (b)

50
30 (z = 0.5, q = 18° )
Rise time (log scale)
Number of samples

20
(z = 0.5, q = 45° )
10

5
3
2 (z = 0.5, q = 72° )
1
– 1.0 – 0.5 0 0.5 1.0
Pole location
(c)
Fig. 4.8 (a) Effect of an extra zero on a discrete-time second-order system, z = 0.5, q = 18º;
(b) effect of an extra zero on a discrete-time second-order system; and (c) effect of an
extra pole on a discrete-time second-order system

To study the influence of a third pole on a basically second-order response, we again consider the system
(4.16), but this time, we fix z1 = z2 = –1 and let p vary from near –1 to near +1. In this case, the major
 206 Digital Control and State Variable Methods

influence of the moving singularity is on the rise time of the step response. Figure 4.8c shows plots of rise
time versus extra pole location, for three different cases of complex-conjugate pole pairs. We see here that
the extra pole causes the rise time to get very much longer as the location of p moves toward z = +1, and
comes to dominate the response.
Our conclusions from these plots are that the addition of a pole, or a zero, to a given system has only a
small effect—if the added singularities are in the range 0 to –1. However, a zero moving toward z = +1
greatly increases the system overshoot. A pole placed toward z = +1 causes the response to slow down and
thus, primarily, affects the rise time—which is being progressively increased. The pole pair corresponding
to specified z and wn, is a dominant pole pair of the closed-loop system only if the influence of additional
poles and zeros is negligibly small on the dynamic response of the system.
Specifications in Terms of Frequency Response Features The translation of time-domain specifica-
tions into desired locations of pair of dominant closed-loop poles in z-plane, is useful if the design is to be
carried out by using root locus plots. The use of frequency response plots necessitates the translation of time-
domain performance specifications in terms of frequency response features.
All the frequency-domain methods of continuous-time systems can be extended for the analysis and
design of digital control systems. Consider the system shown in Fig. 4.3b. The closed-loop transfer function
of the sampled-data system is
Y (z) Gh 0 G ( z )
= (4.18)
R( z ) 1 + G h 0 G ( z )
Just as in the case of continuous-time systems, the absolute and relative stability conditions of the closed-
loop discrete-time system can be investigated by making the frequency response plots of Gh0G(z). The
frequency response plots of Gh0G(z) are obtained by setting z = e jwT, and then letting w vary from – ws /2 to
ws /2. This is equivalent to mapping the unit circle in the z-plane onto the Gh0G(ejwT)-plane. Since the unit
circle in the z-plane is symmetrical about the real axis, the frequency response plot of Gh0G(e jw T) will also be
symmetrical about the real axis, so that only the portion that corresponds to w = 0 to w = ws /2 needs to be
plotted.
A typical curve of (refer Eqn. (4.18))
Y jw T Gh 0 G(e jwT)
(e ) = , (4.19)
R 1 + Gh 0 G(e jwT )
the closed-loop frequency response, is shown in Fig. 4.9. The amplitude ratio and phase angle will approxi-
mate the ideal 1.0 Ð 0º for some range of ‘low’ frequencies, but will deviate for high frequencies. The height
Mr (resonance peak) of the peak is a relative stability criterion; the higher the peak, the poorer the relative
stability. Many systems are designed to exhibit a resonance peak in the range 1.2 to 1.4. The frequency w r
(resonance frequency) at which this peak occurs, is a speed of response criterion; the higher the w r, the faster
the system. For systems that exhibit no peak (sometimes the case), the bandwidth wb is used for speed
of response specifications. Bandwidth is the frequency at which amplitude ratio has dropped to 1/ 2 times
its zero-frequency value. It can, of course, be specified even if there is a peak.
Alternative measures of relative stability and speed of response are stability margins and crossover
frequencies. To define these measures, a discussion of the Nyquist stability criterion in the z-plane is
required. Given the extensive foundation for the Nyquist criterion for continuous-time systems, that we laid
in Chapter 8 of the companion book [155], it will not take us long to present the criterion for the discrete-
time case.
 Design of Digital Control Algorithms 207

1.0 0

Y Mr
(e jwT )

Decibels
R

0
wr w

Y
(e jwT ) 0°
R w

1.0 0

–3

Decibels
Y
(e jwT )
R 0.707

0 wb w
Fig. 4.9 Closed-loop frequency response criteria

4.2.3 Nyquist Stability Criterion in the z-Plane

The concepts involved in z-plane Nyquist stability criterion, are identical to those for s-plane criterion.
In the s-plane, the region of stability is infinite in extent, namely, the entire left half of the s-plane. In the
z-plane, this is not the case. The region of stability is the interior of the unit circle. This makes drawing the
locus of Gh0G(e jwT ), the open-loop frequency response on the polar plane, easier because the Nyquist con-
tour Gz in the z-plane is finite in extent, being simply the unit circle. We treat poles at z = 1 in the way we
treated poles at s = 0, by detouring around them on a contour of arbitrarily small radius.
Figure 4.10a shows a typical Nyquist contour along which we will evaluate Gh0G(z). Note that we detour
around the pole at z = 1, on a portion of a circle of radius e centered at z = 1. A typical Nyquist plot
Gh0G(e jwT ) is shown in Fig. 4.10b. We see from this figure, that the Nyquist plot is similar to those we obtain
for continuous-time functions with a single pole at s = 0, with the following exception. The plot does not
touch the origin in the z-plane. The reason is that we evaluate Gh0G(e jwT ) over a finite range of values of w,
namely, 0 £ w £ p /T, where T is the sampling interval.
We have labelled the segments of Gz in the same fashion as we did in the s-plane Nyquist analysis [155].
Segment C1 is the upper half of the unit circle, and segment C2 is the lower half of the unit circle. Segment C3
is the portion of a circle with radius e centered at z = 1. There is no segment corresponding to the s-plane
portion of a circle with infinite radius centered as s = 0, because the Nyquist contour Gz in the z-plane, unlike
its counterpart in the s-plane, is of finite extent.
 208 Digital Control and State Variable Methods

Im(z) Im(Gh0G)

C1 Gh0G(C2) Gh0G(C3)

C3
–1
Re(Gh0G)
Re(z)

C2 Gh0G(C1)

(a) Nyquist contour in z-plane

(b) Nyquist plot
Im(Gh0G)

1
GM
/T
Re(Gh0G)
–1

M Gh0G(ej T)

wg

0
(c) Relative stability margins
Fig. 4.10

Note that the locus Gh0G(C1) in Fig. 4.10b, is directly obtained from Gh0G(e jwT) for 0 £ w £ p /T, whereas
the locus Gh0G(C2) is the same information, with the phase reflected about 180°; Gh0G(C3) is inferred from
Fig. 4.10a based on pole-zero configuration.
In the case of open-loop transfer function Gh0G(z) with no poles outside the unit circle, the closed-loop
system of Fig. 4.3b is stable if
N = number of clockwise encirclements of the critical point –1 + j0 made by the Gh0G(e jwT ) locus of the
Nyquist plot
=0
Note that the necessary information to determine relative stability is contained in the portion Gh0G(C1) of
the Nyquist plot, which corresponds to the frequency response of the open-loop system Gh0G(z). This por-
tion of the Nyqusit plot of Fig. 4.10b has been redrawn in Fig. 4.10c. Gain and phase margins are defined so
as to provide a two-dimensional measure of how close the Nyquist plot is to encircling the –1 + j0 point,
and they are identical to the definitions developed for continuous-time systems. The Gain Margin (GM) is
the inverse of the amplitude of Gh0G(e jwT ) when its phase is 180°, and is a measure of how much gain of the
system can be increased before instability results. The Phase Margin (FM) is the difference between 180°
and the phase of Gh0G(e jwT ) when its amplitude is 1. It is a measure of how much additional phase lag, or
 Design of Digital Control Algorithms 209
time delay, can be tolerated before instability results, because the phase of a system is highly related to these
characteristics.
The Nyquist plot in Fig. 4.10c intersects the negative real axis at frequency wf. This frequency at which
the phase angle of Gh0G(e jwT) is 180°, has been designated as the phase crossover frequency. The gain
margin of the closed-loop system of Fig. 4.3b, is defined as the number
1
GM = jw T
Gh 0 G(e f )
For stable systems, GM is always a number greater than one.
A unit circle, centered at the origin, has been drawn in Fig. 4.10 c in order to identify the point at which
the Nyquist plot has unity magnitude. The frequency at this point has been designated wg, the gain crossover
frequency. The phase margin of the closed-loop system of Fig. 4.3b, is defined as
FM = 180° + ÐGh0G(e jw gT )
For stable systems, F M is always positive.
The GM and F M are both measures of relative stability. General numerical design goals for these mar-
gins cannot be given since systems that satisfy other specific performance criteria may exhibit a wide range
of these margins. It is possible, however, to give useful lower bounds—the gain margin should usually
exceed 2.5 and the phase margin exceed 30°.
For continuous-time systems, it is often pointed out that the phase margin is related to the damping ratio z
for a standard second-order system; the approximate relation being z = FM/100. The FM, from a z-plane
frequency response analysis, carries the same implications about the damping ratio of the closed-loop system.
The gain crossover frequency wg is related to the bandwidth of the system. The larger the wg, the wider
the bandwidth of the closed-loop system, and the faster its response.
The translation of time-domain specifications in terms of frequency response features, is carried out by
using the explicit correlations for second-order system (4.10). The following correlations are valid approxi-
mations for higher-order systems dominated by a pair of complex conjugate poles3.
1
Mr = ; z £ 0.707 (4.20)
2z 1 - z 2
wr = wn 1 - 2z 2 (4.21)
1
wb = wn 1 - 2z 2 + (2 - 4z 2 + 4z 4 ) 2 (4.22)

R| 1
U|
FM = tan–1 2z S| 1 + 4z 4 - 2z 2 2
V|
T W
@ 100z (4.23)

4.2.4 Disturbance Rejection

The effectiveness of a system in disturbance signal rejection is readily studied with the topology of
Fig. 4.11a. The response Y(z) to disturbance W(z), can be found from the closed-loop transfer function
Y (z) 1
= (4.24a)
W ( z) 1 + D( z )Gh 0 G( z )

3
Chapter 9 of reference [155].
 210 Digital Control and State Variable Methods
We now introduce the function
1
S(z) = (4.24b)
1 + D( z )Gh 0 G( z )
which we call the sensitivity function of the control system, for reasons to be explained later. To reduce the
effects of disturbances, it turns out that S(e jwT ) must be made small over the frequency band of disturbances.
If constant disturbances are to be suppressed, S(1) should be made small. If D(z)Gh0G(z) includes an inte-
grator (which means that D(z) or Gh0G(z) has a pole at z = 1), then the steady-state error due to constant
disturbance is zero. This may be seen as follows. Since for a constant disturbance of amplitude A, we have
Az
W(z) = ,
z -1
the steady-state value of the output is given by
yss = lim (z – 1)Y(z) = lim (z – 1)S(z)W(z) = lim AS(z)
z®1 z®1 z®1
which is equal to zero if D(z)Gh0G(z) has a pole at z = 1.
Note that the point where the disturbance enters the system is very important in adjusting the gain of
D(z)Gh0G(z). For example, consider the system shown in Fig. 4.11b. The closed-loop transfer function for
the disturbance is
Y (z) Gh 0 G( z )
=
W ( z) 1 + D( z )Gh 0 G( z )

W (z)
R (z ) = 0 + + +
D (z ) G h0G (z) Y (z)
–

(a)
W (z)

R (z ) = 0 + + +
D (z ) G h0G (z) Y (z)
–

(b)
R (z ) = 0 + Y (z)
D (z ) G h0G (z)
–
+

+
(c) Wn(z)
Fig. 4.11 Disturbance rejection

In this case, the steady-state error due to constant disturbance W(z) is not equal to zero when Gh0G(z) has
a pole at z = 1. This may be seen as follows.
Let Gh0G(z) = Q(z)/(z – 1)
 Design of Digital Control Algorithms 211
where Q(z) is a rational polynomial of z, such that Q(1) ¹ 0 and Q(1) ¹ ¥; and D(z) is a controller which
does not have pole at z = 1. Then
( z - 1)Gh 0 G( z ) AzQ( z ) A
yss = lim W(z) = lim =
z ® 1 1 + D( z )Gh 0 G( z ) z ® 1 z - 1 + D( z )Q( z ) D(1)
Thus the steady-state error is non-zero; the magnitude of the error can be reduced by increasing the
controller gain.
Figure 4.11c gives a block diagram of the situation where measurement noise Wn(z) enters the system
through feedback link. The closed-loop transfer function for this disturbance is
Y ( z) D ( z ) Gh 0 G ( z )
= (4.25)
Wn ( z) 1 + D ( z ) Gh 0 G ( z )
Thus, the measurement noise is transferred to the output whenever | D(z)Gh0G(z) | > 1. Hence, large gains
of D(z)Gh0G(z) will lead to large output errors due to measurement noise. This is in conflict with the distur-
bance-rejection property with respect to configurations of Figs 4.11a and 4.11b. To solve this problem, we
can generally examine the measuring instrument and modify the filtering, so that it satisfies the requirements
of a particular control problem.

4.2.5 Insensitivity and Robustness

Finally, in our design, we must take into account both the small and, often, the large differences between the
derived process model and the real process behaviour. The differences may appear due to modelling
approximations, and the process behaviour changes with time during operation. If, for simplicity, it is
assumed that the structure and order of the process model are chosen exactly, and they do not change with
time, then these differences are manifested as parameter errors.
Parameter changes with respect to nominal parameter vector q n are assumed. The closed-loop behaviour
for parameter vector
q = q n + Dq
is of interest. If the parameter changes are small, then sensitivity methods can be used. For controller design,
both good control performance (steady-state accuracy, transient accuracy, and disturbance rejection),
and small parameter sensitivity, are required. The resulting controllers are then referred to as insensitive
controllers. However, for large parameter changes, the sensitivity design is unsuitable. Instead, one has to
assume several process models with different parameter vectors q 1, q 2, ..., q M, and try to design a robust
controller which, for all process models, will maintain stability and certain control performance range.
For the design of insensitive controllers, the situation is very much like the disturbance-signal rejection.
The larger the gain of the feedback loop around the offending parameter, the lower the sensitivity of the
closed-loop transfer function to changes in that parameter.
Consider the digital control system of Fig. 4.11. The closed-loop input-output behaviour corresponding
to the nominal parameter vector, is described by
Y (z ) D( z )Gh 0 G(q n , z )
M(q n, z) = = (4.26)
R( z ) 1 + D( z )Gh 0 G(q n , z )
The process parameter vector now changes by an infinitesimal value Dq. For the control loop, it follows
that
¶ M (q , z ) D( z ) ¶Gh 0 G (q, z )
= 2
¶q q =qn [1 + D( z ) G h0 G ( q n , z )] ¶q q=q n
 212 Digital Control and State Variable Methods
T
For DGh0G(q n, z) =
F ¶G h 0 G( q, z) I Dq ,
GH ¶q q = qn
JK
T
DM (q n, z) =
D( z ) F
¶Gh 0 G(q, z ) I Dq
[1 + D( z)Gh 0 G(q n , z )] 2
¶q GHq=q n
JK
=
RS D(z)G G(q , z) UV RS
h0 n 1 UV RS 1 UV
{DGh0G(q n,z)} (4.27)
T1 + D( z )G G (
h0 q , z )nWT1 + D( z )G G( q , z ) W TG
h0 n h 0 G ( q n , z) W
From Eqns (4.26)–(4.27), it follows that
DM ( q n , z ) DGh 0 G ( q n , z )
= S(q n, z) (4.28a)
M (q n , z ) G h 0 G (q n , z )
with the sensitivity function S(q n, z) of the feedback control given as
1
SGMh 0G = S(q n, z) = (4.28b)
1 + D( z )Gh 0 G (q n , z)
This sensitivity function shows how relative changes of input/output behaviour of a closed loop, depend
on changes of the process transfer function. Small parameter-sensitivity of the closed-loop behaviour, can be
obtained by making S(q n, e jwT ) small in the significant frequency range.

4.2.6 The Case for High-Gain Feedback

Control system design with high-gain feedback results in:
(i) good steady-state tracking accuracy,
(ii) good disturbance-signal rejection, and
(iii) low sensitivity to process-parameter variations.
There are, however, factors limiting the gain:
(i) High gain may result in instability problems.
(ii) Input amplitudes limit the gain; excessively large magnitudes of control signals will drive the process
to saturation region of its operation, and the control system design, based on linear model of the plant,
will no longer give satisfactory performance.
(iii) Measurement noise limits the gain; with high gain feedback, measurement noise appears unattenuated
in the controlled output.
Therefore, in design, we are faced with trade-offs.

4.3 DIGITAL COMPENSATOR DESIGN USING FREQUENCY RESPONSE PLOTS

All the frequency response methods of continuous-time systems4, are directly applicable for the analysis and
design of digital control systems. For a system with closed-loop transfer function
Y (z ) Gh 0 G( z )
= (4.29)
R( z ) 1 + Gh 0 G( z )
the absolute and relative stability conditions can be investigated by making the frequency response plots of
Gh0G(z). The frequency response plots of Gh0G(z) can be obtained by setting
z = e jwT; T = sampling interval (4.30)
4
Chapters 8–10 of reference [155].
 Design of Digital Control Algorithms 213
and then letting the frequency w vary from –ws /2 to ws/2; ws = 2p/T. Computer assistance is normally
required to make the frequency response plots (refer Problem A.8 in Appendix A).
Since the frequency appears in the form z = e jwT, the discrete-time transfer functions are typically not
rational functions and the simplicity of Bode’s design technique is altogether lost in the z-plane. The
simplicity can be regained by transforming the discrete-time transfer function in the z-plane, to a different
plane (called w) by the bilinear transformation (refer Eqn. (2.115))
1 + wT / 2
z= (4.31a)
1 - wT / 2
By solving Eqn. (4.31a) for w, we obtain the inverse relationship
2 z -1
w= (4.31b)
T z +1
Through the z-transformation and the w-transformation, the primary strip of the left half of the s-plane
is first mapped into the inside of the unit circle in the z-plane, and then mapped into the entire left half of the
w-plane. The two mapping processes are depicted in Fig. 4.12. Notice that as s varies from 0 to jws/2 along
the jw-axis in the s-plane, z varies from 1 to –1 along the unit circle in the z-plane, and w varies from 0 to ¥
along the imaginary axis in the w-plane. The bilinear transformation (4.31) does not have any physical
significance in itself and, therefore, all w-plane quantities are fictitious quantities that correspond to the
physical quantities of either the s-plane or the z-plane. The correspondence between the real frequency w,
and the fictitious w-plane frequency, denoted as n, is obtained as follows:
From Eqn. (4.31b),
2 e jwT - 1 2 e jwT / 2 - e - jwT / 2 2 wT
jn = jwT
= jwT / 2 - jwT / 2
= j tan
T e +1 T e +e T 2
2 wT
or n= tan (4.32)
T 2

w-plane
s-plane z-plane
Im
jw Im
jws/2

0 s Re 0 Re
1

– jws/2

(a) (b) z = e sT
2 z–1
(c) w =
T z+1
Fig. 4.12 Diagrams showing mappings from s-plane to z-plane and from z-plane to w-plane
 214 Digital Control and State Variable Methods
Thus a nonlinear relationship or ‘warping’ exists between the two frequencies w and n. As w moves from 0
to ws/2, n moves from 0 to ¥ (Fig. 4.13).

Note that for relatively small

wT wT F
< 17º or about 0.3 rad , I
2 2 H K
n@
2 wT
@ w F I (4.33)
T 2 H K
and the ‘warping’ effect on the frequency response is negligible.
The distortion depicted in Fig. 4.13 may be taken into account in our design of digital compensation by
frequency ‘prewarping’. The idea of prewarping is simply to adjust the critical frequencies in our design; for
example, if the closed-loop bandwidth is specified as wb, then the corresponding bandwidth on the w-plane
2 F w T I
is nb = tan b . Our design based on the frequency response plots of Gh0G(jn) attempts to realize a
T H 2 K
closed-loop bandwidth equal to nb.

nT
2

0
ws ws w
4 2
Fig. 4.13 Relationship between the fictitious frequency n and actual frequency w

Example 4.2 Consider a process with transfer function

10
G(s) = (4.34)
s( 15 s + 1)
which, when preceded by a ZOH (T = 0.1 sec), has the discrete-time transfer function (refer Table 2.1)

Gh0G(z) = (1 – z–1) Z
LM 50 OP = 0.215(z + 0.85) (4.35)
2
N s (s + 5) Q (z - 1)(z - 0.61)
By use of the bilinear transformation
wT
1+
z= 2 = 1 + 0.05w
wT 1 - 0.05w
1-
2
 Design of Digital Control Algorithms 215
Gh0G(z) can be transformed into Gh0G(w) given below:
w F w IF I
Gh0G(w) =
10 1 -
20H 1+
246.67KH K (4.36)
w F I
w 1+
4.84 H K
Notice that the gain of Gh0G(w) is precisely the same as that of G(s)—it is 10 in both the cases. This will
always be true for a Gh0G(w) computed using the bilinear transformation given by Eqns (4.31). The gain of
10 in Eqn. (4.36) is the Kv of the uncompensated system (4.35).
We also note that in Eqn. (4.36), the denominator looks very much similar to that of G(s), and that the
denominators will be the same as T approaches zero. This would also have been true for any zeros of Gh0G(w)
that corresponded to zeros of G(s), but our example does not have any. Our example also shows the creation
of a right-half plane zero of Gh0G(w) at 2/T, and the creation of a fast left-half plane zero when compared to
the original G(s). The transfer function Gh0G(w) is thus a nonminimum phase function.

To summarize, the w-transformation maps the inside of the unit circle in the z-plane, into the left half of
the w-plane. The magnitude and phase of Gh0G( jn) are the magnitude and phase of Gh0G(z) as z takes on
values around the unit circle. Since Gh0G( jn) is a rational function of n, we can apply all the standard
straight line approximations to the log-magnitude and phase curves.

To obtain Gh0G(w) = Gh 0 G ( z ) 1 + wT / 2
z=
1 - wT / 2
the following ready-to-use formula may be used.
m
K P ( z + ai )
i =1
Gh0G(z) = n
(4.37a)
( z - 1) l P ( z + b j )
j =1
m FG
K P (1 + ai ) 1 -
w IJ l-m+n
m
P 1+
FG w IJ
Gh0G(w) =
i =1 H 2/T K i =1 H ( 2/T ) [(1 + ai )/(1 - ai )] K (4.37b)
n n F1 + I
j =1
l
P (1 + b j )T w l
P
j =1
GH ( 2/T )[(1 + wb )/(1 - b )]JK j j

The design of analog control systems usually falls into one of the following categories: (1) lead compen-
sation, (2) lag compensation, (3) lag-lead compensation. Other more complex schemes, of course, do exist,
but knowing the effects of these three basic types of compensation, gives a designer much insight into the
design problem. With reference to the design of digital control systems by Bode plots, the basic forms of
compensating network D(w) have also been classified as lead, lag, and lag-lead. In the following paragraphs,
we briefly review the fundamental frequency-domain features of these compensators.
Lead Compensation A simple lead compensator model in the w-plane is described by the transfer
function
1 + wt
D(w) = ; 0 < a < 1, t > 0 (4.38)
1 + a wt
The zero-frequency gain of the compensator is found by letting w = 0. Thus, in Eqn. (4.38), we are
assuming a unity zero-frequency gain for the compensator. Most of the designs require a compensator with
a nonunity zero-frequency gain to improve steady-state response, disturbance rejection, etc. A nonunity
 216 Digital Control and State Variable Methods

zero-frequency gain is obtained by multiplying the right side of Eqn. (4.38) by a constant equal to the value
of the desired zero-frequency gain. For the purpose of simplifying the design procedure, we normally add
the required increase in gain to the plant transfer function, and design the unity zero-frequency gain compen-
sator given by Eqn. (4.38), based on the new plant transfer function. Then the compensator is realized as the
transfer function of (4.38) multiplied by the required gain factor.
The Bode plot of the unity zero-frequency gain lead compensator is shown in Fig. 4.14. The maximum
phase lead fm of the compensator is given by the relation
1 - sin f m
a= (4.39)
1 + sin f m
and it occurs at the frequency

nm =
F 1I F 1 I
H t K H at K (4.40)

The magnitude of D( jn) at n = nm is 20 log (1/ a ) .

Slope = 20 dB / decade
dB

D( j ) 20 log 1

20 log 1

0
90°

45°

D( j ) m

0°
1/ m = 1/ 1/
log
Fig. 4.14 Bode plot of lead compensator

The phase lead is introduced in the vicinity of the gain crossover frequency of the uncompensated
system—in order to increase the system’s phase margin. Lead compensation increases the system gain at
higher frequencies, thereby increasing the system bandwidth and hence the speed of response. However, a
system with large bandwidth may be subjected to high-frequency noise problems.
Lag Compensation A simple lag compensator model in the w-plane is described by the transfer
function
1 + wt
D(w) = ; b > 1, t > 0 (4.41)
1 + b wt
 Design of Digital Control Algorithms 217
The Bode plot of this unity zero-frequency gain compensator is shown in Fig. 4.15.
0

dB
Slope = – 20 dB/decade 20 log b
D ( j n)

0°

D ( j n)
– 45 °

– 90 °
1/bt 1/t
log n
Fig. 4.15 Bode plot of lag compensator

Since the lag compensator reduces the system gain in the high frequency range, without reducing the gain
at low frequencies, the total system gain can be appreciably increased by a nonunity zero-frequency gain
obtained by multiplying the right side of Eqn. (4.41) by a constant. This is equivalent to increasing the gain
for the entire frequency range, and then attenuating the magnitude curve in the high frequency region. This
results in an appreciable increase in gain in the low frequency range of the lag-compensated system, thereby
improving steady-state accuracy.
In the design method using Bode plots, the attenuation property of lag compensator is utilized; the phase
lag characteristic is of no consequence. The attenuation provided by the lag compensator in the high fre-
quency range shifts the gain crossover frequency to a lower value, and gives the system sufficient phase
margin. So that a significant phase lag will not be contributed near the new gain crossover, the upper corner
frequency 1/t of D(w) is placed far below the new gain crossover.
With the reduction in system gain at high frequencies, the system bandwidth gets reduced and thus the
system has a slower speed of response. This may be an advantage if high frequency noise is a problem.
Lag-lead Compensation Equations (4.38) and (4.41) describe simple first-order compensators. In many
system design problems, however, the system specifications cannot be satisfied by a first-order compensator.
In these cases, higher-order compensators must be used. To illustrate this point, suppose that smaller steady-
state errors to ramp inputs are required for a type-2 system; this requires an increase in the low-frequency
gain of the system. If phase-lead compensation is employed, this increase in gain must be reflected at all
frequencies. It is then unlikely that one first-order section of phase-lead compensation can be designed to
give adequate phase margin. One solution to this problem would be to cascade two first-order lead compensa-
tors. However, if the noise in the control system is a problem, this solution may not be acceptable. A different
approach is to cascade a lag compensator with a lead compensator. This compensator is usually referred to
as a lag-lead compensator.
 218 Digital Control and State Variable Methods

Example 4.3 Consider the feedback control system shown in Fig. 4.16. The plant is described by the
transfer function
K
G(s) =
s( s + 5)
Design a digital control scheme for the system to meet the following specifications:
(i) the velocity error constant Kv ³ 10;
(ii) peak overshoot Mp to step input £ 25%, and
(iii) settling time ts (2% tolerance band) £ 2.5 sec.
e ( t) e (k) u (k) u * (t)

r (t) + y ( t)
D (z) G h0(s) G (s)
– T T

Compensator ZOH Plant

Fig. 4.16 A feedback control system with digital compensation

Solution The design parameters are the sampling interval T, the system gain K, and the parameters of the
unity zero-frequency gain compensator D(z).
Let us translate the transient accuracy requirements to frequency response measures. z = 0.4 corresponds
to a peak overshoot of about 25% (Eqn. (4.13)), and a phase margin of about 40º (Eqn. (4.23)). The require-
ment of ts @ 2.5 sec corresponds to wn = 4 rad/sec (Eqn. (4.14)) and closed-loop bandwidth wb @ 5.5 rad/sec
(Eqn. (4.22)). Taking the sampling frequency about 10 times the bandwidth, we choose the sampling interval
2p
T= @ 01
. sec
10w b
Our design approach is to first fix the system gain K to a value that results in the desired steady-state
accuracy. A unity zero-frequency gain compensator, that satisfies the transient accuracy requirements with-
out affecting the steady-state accuracy, is then introduced.
Since sampling does not affect the error constant of the system, we can relate K with Kv as follows, for the
system of Fig. 4.16 with D(z) = 1 (i.e., for uncompensated system):
K
Kv = lim sG(s) =
s®0 5
Thus K = 50 meets the requirements on steady-state accuracy.
For T = 0.1 and K = 50, we have

Gh0G(z) = Z
LM1 - e FG 50 IJ OP = 0.215(z + 0.85)
- Ts
(4.42)
N s H s(s + 5)K Q (z - 1)(z - 0.61)
10F1 -
wI F w I
G G(w) = G G ( z ) =
H 20 K H 1+
wT
246.67K
(4.43)
wF1 + I
h0 h0 1+
2 w
H 4.84 K
z=
1-
wT
2

10F1 -
jn I F jn I
Gh0 G( jn) = G G ( w) =
H 20 K H 1+
246 .67 K
(4.44)
jn F 1 + I
h0
j n
H 4.84 K
w = jn
 Design of Digital Control Algorithms 219
The Bode plot of Gh0G( jn) (i.e., the uncompensated system) is shown in Fig. 4.17. We find from this plot,
that the uncompensated system has gain crossover frequency nc1 = 6.6 rad/sec and phase margin FM1 @ 20º.
The magnitude versus phase angle curve of the uncompensated system is drawn in Fig. 4.18. The bandwidth5
of the system is read as
nb1 = 11
In terms of the real frequency, the bandwidth (Eqn. (4.32))
2
w b1 = tan -1 b1
F
n T I
= 10 rad/sec
T H 2 K
It is desired to raise the phase margin to 40º without altering Kv. Also the bandwidth should not increase.
Obviously, we should first try a lag compensator.

Fig. 4.17 Compensator design (Example 4.3)

From the Bode plot of uncompensated system, we observe that the phase margin of 40º is obtained if the
gain crossover frequency is reduced to 4 rad/sec. The high frequency gain –20 log b of the lag compensator
(Fig. 4.15) is utilized to reduce the gain crossover frequency. The upper corner frequency 1/t of the compen-
sator is placed one octave to one decade below the new gain crossover, so that the phase lag contribution of
the compensator, in the vicinity of the new gain crossover, is made sufficiently small. To nullify the small

5
The –3dB closed-loop gain contour of the Nichols chart has been used to determine bandwidth. The contour has
been constructed using the following table obtained from the Nichols chart.
Degrees –90 –100 – 120 –140 –160 –180 –200 – 220
dB 0 –1.5 – 4.18 – 6.13 –7.28 –7.66 –7.28 – 6.13
 220 Digital Control and State Variable Methods
16

G h0 G ( jn) 2
8 D ( jn)G h0G ( jn) 3
2

n
Magnitude (dB) 5
0
6 3

n
–8 5.5
11 Locus for
12 6
M = – 3 dB
– 16
10

– 200 – 180 – 160 – 140 – 120

Phase (deg)
Fig. 4.18 Compensator design (Example 4.3)

phase lag contribution which will still be present, the gain crossover frequency is reduced to a value nc2
where the phase angle of the uncompensated system is
f = –180º + FMs + e;
FMs is the specified phase margin and e is allowed a value 5º–15º.
The uncompensated system (Fig. 4.17) has a phase angle
f = –180º + FMs + e = –180º + 40º + 10º = –130º
at nc2 = 3 rad/sec. Placing the upper corner frequency of the compensator two octaves below nc2, we have
1 n 3
= c22 =
t (2 ) 4
To bring the magnitude curve down to 0 dB at nc2, the lag compensator must provide an attenuation of
9 dB (Fig. 4.17). Therefore
20 log b = 9 or b = 2.82
The lower corner frequency of the compensator is then fixed at
1
= 0.266
bt
The transfer function of the lag compensator is then
1 + tw 1 + 133
. w
D(w) = =
1 + bt w 1 + 3.76w
Phase lag introduced by the compensator at nc2 = tan–1(1.33 nc2) – tan–1 (3.76nc2) = 75.93º – 84.93º = –9º.
Therefore, the safety margin of e = 10º is justified.
The open-loop transfer function of the compensated system becomes
w F w IF w IF I
D(w)Gh0G(w) =
10 1 -
20 H1+
246.67 KH1+
0.75 KH K
w 1+
w
1+F w IF I
4.84 H 0.266 KH K
 Design of Digital Control Algorithms 221
The Bode plot of D(w)Gh0G(w) is shown in Fig. 4.17, from where the phase margin of the compensated
system is found to be 40º and the gain margin is 15 dB. The magnitude versus phase angle curve of the
compensated system is shown on Nichols chart in Fig. 4.18. The bandwidth of the compensated system is
FG
2
nb2 = 5.5 w b 2 =
n T
tan -1 b 2
F I
= 5.36 rad/sec
IJ
H
T 2 H K K
Therefore, the addition of the compensator has reduced the bandwidth from 10 rad/sec to 5.36 rad/sec.
However, the reduced value lies in the acceptable range.
2 z -1
Substituting w=
T z +1
in D(w), we obtain

F z - 0.928I = 0.362z - 0.336

H z - 0.974 K z - 0.974
D(z) = 0.362

L z - 0.928I OP = 1
Zero-frequency gain of D(z) = lim M0.362F
N H z - 0.974 K Q
z ®1

The digital controller D(z) has a pole-zero pair near z = 1. This creates a long tail of small amplitude in the
step response of the closed-loop system. This behaviour of the lag-compensated system will be explained
shortly, with the help of root locus plots.
To evaluate the true effectiveness of the design, we write the closed-loop transfer function of the compen-
sated system (Fig. 4.19) and therefrom obtain the response to step input. Computer assistance is usually
needed for this analysis.

D (z) = G h0G (z) =

R (z) + E (z) U (z) Y (z )
0.362 z – 0.336 0.215 ( z + 0.85)
– z – 0.974 (z – 1)( z – 0.61 )

Lag compensator Plant

Fig. 4.19 Compensator design (Example 4.3)

Comment We have obtained a digital control algorithm which meets the following objectives: Kv @ 10,
Mp @ 25%, ts @ 2.5 sec. We may attempt to improve upon this design to obtain Kv > 10, Mp < 25% and ts <
2.5 sec. However, the scope of such an exercise is limited, because the improvement in steady-state accuracy
will be at the cost of stability margins and vice versa. Also, the conflicting requirements of limiting the
magnitudes of control signals to avoid saturation problems, limiting the bandwidth to avoid high-frequency
noise problems, etc., have to be taken into consideration.

Example 4.4 Reconsider the feedback control system of Example 4.3 (Fig. 4.16). We now set the following
goal for our design
(i) Kv ³ 10
(ii) Phase margin @ 40º
(iii) Bandwidth @ 12 rad/sec.
 222 Digital Control and State Variable Methods
Sampling interval T = 0.1 sec corresponds to a sampling frequency which is about five times the closed-loop
bandwidth. A smaller value of T is more appropriate for the present design problem which requires higher
speed of response; we will however take T = 0.1 sec to compare our results with those of Example 4.3.
Following the initial design steps of Example 4.3, we find that K = 50 meets the requirement on steady-
state accuracy. For K = 50 and T = 0.1 sec, we have (refer Eqn. (4.44))
F jn IF jn I
Gh0G( jn) =
H
10 1 -
20 KH1+
246.67K
F jnI
H
jn 1 +
4.84K
The uncompensated system has a gain crossover frequency nc1 = 6.6 rad/sec, phase margin FM1 @ 20º
and bandwidth nb1 = 11(w b1 = 10 rad/sec). This follows from Figs 4.20 and 4.21.

Fig. 4.20 Compensator design (Example 4.4)

It is desired to raise the phase margin to 40º without altering Kv. The bandwidth should also increase.
Obviously, we should try a lead compensator for this situation.
The phase margin FM1 = 20º of the uncompensated system falls short of the specified phase margin FMs
= 40º by 20º. Additional phase margin can be provided by a lead compensator (Fig. 4.14), so placed that its
corner frequencies 1/t and 1/at are on either side of the gain crossover frequency nc1 = 6.6 rad/sec. The
compensator so placed will increase the system gain in the vicinity of nc1; this will cause the gain crossover
to shift to the right—to some unknown value nc2. The phase lead provided by the compensator at nc2 adds to
the phase margin of the system.
 Design of Digital Control Algorithms 223
16
2
G h0G ( jn)
3
8 D ( jn) G h0G ( jn) n
5 5
n
6
0
Magnitude (dB)

10

11 15
–8
12 22.5
25 Locus for
M = – 3 dB
– 16

– 200 – 180 – 160 – 140 – 120

Phase (deg)
Fig. 4.21 Compensator design (Example 4.4)

Phase margin of the uncompensated system at nc1 is FM1. At nc2, which is expected to be close to nc1, let us
assume the phase margin of the uncompensated system to be (FM1 – e) where e is allowed a value 5º – 15º. The
phase lead required at nc2 to bring the phase margin to the specified value FMs, is given by
fl = FMs – (FM1 – e) = FMs – FM1 + e
In our design, we will force the frequency nm of the compensator to coincide with nc2, so that maximum
phase lead fm of the compensator is added to the phase margin of the system. Thus we set
nc2 = nm
Therefore fm = fl
The a-parameter of the compensator can then be computed from (refer Eqn. (4.39))
1 - sin f m
a=
1 + sin f m
Since at nm, the compensator provides a dB-gain of 20 log (1 / a ) , the new crossover frequency nc2 = nm
can be determined as that frequency at which the uncompensated system has a dB-gain of –20 log (1/ a ) .
For the design problem under consideration,
fl = 40º – 20º + 15º = 35º
1 - sin 35º
Therefore a= = 0.271
1 + sin 35º
The magnitude contribution of the compensator at nm is 20 log (1/ 0.271) = 5.67dB.
From Bode plot of Fig. 4.20, we obtain
nc2 = 9.4 = nm
Therefore (refer Eqn. (4.40))
F 1I F 1 I = n
H t K H at K m = 9.4
 224 Digital Control and State Variable Methods

1 1 4.893
or = a ( vm ) = 4.893 and = = 18.055
t at 0.271
Since the compensator zero is very close to a pole of the plant, we may cancel the pole with the zero, i.e.,
we may choose
1 1
= 4.84; = 17.86
t at
The transfer function of the lead compensator becomes
1 + tw 1 + 0.21w
D(w) = =
1 + atw 1 + 0.056 w
Substituting
2 z -1
w=
T z +1
in D(w), we obtain
2.45( z - 0.616)
D(z) =
z - 0.057
The open-loop transfer function of the compensated system is
F I F1 + w I
w
D(w)Gh0G(w) =
H
10 1 -
K H 246.67K
20
wF1 +
w I
H 17.86 K
The Bode plot of D(w)Gh0G(w) is shown in Fig. 4.20, from where the phase margin of the compensated
system is found to be 38º, and gain margin is 7.5 dB. The magnitude versus phase angle curve of the compen-
sated system is shown in Fig. 4.21. The bandwidth of the compensated system is

nb2 = 22.5; wb2 =

2 F
n T
tan -1 b 2
I
= 16.9 rad/sec
T H 2 K
Thus, the addition of the lead compensator has increased the system bandwidth from 10 to 16.9 rad/sec. It
may lead to noise problems if the control system is burdened with high frequency noise.
A solution to noise problems involves the use of a lag compensator cascaded with lead compensator. The
lag compensation is employed to realize a part of the required phase margin, thus reducing the amount of
lead compensation required.

4.4 DIGITAL COMPENSATOR DESIGN USING ROOT LOCUS PLOTS

Design of compensation networks using the root locus plots is a well established procedure in analog control
systems. This is essentially a trial-and-error method where, by varying the controller parameters, the roots of
the characteristic equation are relocated to favourable locations. In the present section, we shall consider the
application of root locus method to the design of digital control systems.

4.4.1 The Root Locus on the z-plane

The characteristic equation of a discrete-time system can always be written in the form
1 + F(z) = 0 (4.45)
where F(z) is a rational function of z.
 Design of Digital Control Algorithms 225
From Eqn. (4.45), it is seen that the roots of the characteristic equation (i.e., the closed-loop poles of the
discrete-time system), occur only for those values of z where
F(z) = – 1 (4.46)
Since z is a complex variable, Eqn. (4.46) is converted into two conditions given below.
(i) Magnitude condition: |F(z)| = 1 (4.47a)
(ii) Angle condition: Ð F(z) = ± 180º (2q + 1 ); q = 0, 1, 2, ... (4.47b)
In essence, the construction of the z-plane root loci is to find the points that satisfy these conditions. If we
write F(z) in the standard pole-zero form:
K P( z - zi )
i
F(z) = ;K³0 (4.48a)
P( z - p j )
j
then the two conditions given in Eqns (4.47) become
K P| z - zi |
i
|F(z)| = =1 (4.48b)
P| z - p j |
j

and ÐF ( z) = å Ð z – zi – å Ð z – pj = ± 180º (2q + 1); q = 0, 1, 2, K (4.48c)

i j

Consequently, given the pole-zero configuration of F(z), the construction of the root loci on the z-plane
involves the following steps:
(i) A search for the points on the z-plane that satisfy the angle condition given by Eqn. (4.48c).
(ii) The value of K at a given point on a root locus is determined from the magnitude condition given by
Eqn. (4.48b).
The root locus method developed for continuous-time systems can be extended to discrete-time systems
without modifications, except that the stability boundary is changed from the jw axis in the s-plane, to the
unit circle in the z-plane. The reason the root locus method can be extended to discrete-time systems is that
the characteristic equation (4.45) for the discrete-time system, is of exactly the same form as the equation for
root locus analysis in the s-plane. However, the pole locations for closed-loop systems in the z-plane must be
interpreted differently from those in the s-plane.
We assume that the reader is already familiar with the s-plane root locus technique. We shall concentrate
on the interpretation of the root loci in the z-plane with reference to the system performance, rather than the
construction of root loci on the z-plane. Rules of construction of root loci are summarized in Table 4.2 for
ready reference.6
Table 4.2 Rules for construction of Root Locus Plot of 1 + F(z) = 0
m
K P ( z - zi )
i =1
F(z) = n
; K ³ 0, n ³ m; zi: m open-loop zeros; pj: n open-loop poles
P (z - p j )
j = 1
(i) The root locus plot consists of n root loci as K varies from 0 to ¥. The loci are symmetric with respect to the real axis.
(ii) As K increases from zero to infinity, each root locus originates from an open-loop pole with K = 0, and termi-
nates either on an open-loop zero or on infinity with K = ¥. The number of loci terminating on infinity equals
the number of open-loop poles minus zeros.

Contd.
6
Chapter 7 of reference [155].
 226 Digital Control and State Variable Methods
Table 4.2 Contd.

(iii) The (n – m) root loci which tend to infinity, do so along straight-line asymptotes radiating out from a single
point z = – s A on the real axis (called the centroid), where
å (real parts of open-loop poles) - å (real parts of open-loop zeros)
– sA =
n-m
These (n – m) asymptotes have angles
( 2 q + 1)180 º
fA = ; q = 0, 1, 2, ..., (n – m – 1)
n-m
(iv) A point on the real axis lies on the locus if the number of open-loop poles plus zeros on the real axis to the right
of this point, is odd. By use of this fact, the real axis can be divided into segments on-locus and not-on-locus;
the dividing points being the real open-loop poles and zeros.
(v) The intersections (if any) of root loci with the imaginary axis can be determined by use of the Routh criterion.
(vi) The angle of departure, fp of a root locus from a complex open-loop pole, is given by
fp = 180º + f
where f is the net angle contribution at this pole of all other open-loop poles ans zeros.
(vii) The angle of arrival, f z of a locus at a complex zero, is given by
f z = 180º – f
where f is the net angle contribution at this zero of all other open-loop poles and zeros.
(viii) Points at which multiple roots of the characteristic equation occur (breakaway points of root loci) are the
solutions of
n
P (z - p j )
dK j =1
= 0, where K = – m
dz P ( z - zi )
i =1

(ix) The gain K at any point z0 on a root locus, is given by

n
P | z0 - p j |
j =1
K= m
P | z0 - zi |
i = 1

[Product of phasor lengths (read to scale) from z0 to poles of F ( z )]

=
[ Product of phasor lengths (read to scale)from z0 to zeros of F ( z )]

Example 4.5 Consider a process with the transfer function

K
G(s) = (4.49a)
s(s + 2)
which, when preceded by a zero-order hold (T = 0.2 sec), has the discrete-time transfer function (refer
Table 2.1)
 Design of Digital Control Algorithms 227

Gh0G(z) = (1 – z – 1) Z
LM K OP
=
K ¢( z - b )
(4.49b)
Ns 2
( s + 2) Q ( z - a1 )( z - a 2 )
where K¢ = 0.01758K, b = – 0.876, a1 = 0.67, a2 = 1.
The root locus plot of
1 + Gh0G(z) = 0 (4.50)
can be constructed using the rules given in Table 4.2. Gh0G(z) has two poles at z = a1 and z = a2, and a zero
at z = b. From rule (iv), the parts of the real axis between a1 and a2, and between – ¥ and b constitute sections
of the loci. From rule (ii), the loci start from z = a1 and z = a2; one of the loci terminates at z = b, and the other
locus terminates at – ¥. From rule (viii), the breakaway points (there are two) may be obtained by solving
for the roots of
dK ¢ ( z - a1 ) ( z - a2 )
= 0, where K ¢ = –
dz ( z - b)
However, we can show that for this simple two-pole and one zero configuration, the complex-conjugate
section of the root locus plot is a circle. The breakaway points are easily obtained from this result, which is
proved as follows:
Let z = x + jy
Equation (4.49b) becomes
K ¢( x + jy - b) K ¢( x - b + jy )
Gh0G(z) = =
( x + jy - a1 )( x + jy - a2 ) ( x - a1 )( x - a 2 ) - y 2 + jy ( 2 x - a1 - a 2 )
On the root loci, z must satisfy Eqn. (4.50).
Therefore
y y (2 x - a1 - a2 )
Ð Gh0G(z) = tan–1 – tan–1 = (2q + 1) 180º
x-b ( x - a1 )( x - a2 ) - y 2
Taking the tangent of both sides of this equation, yields
y y (2 x - a1 - a2 )
-
x - b ( x - a1 ) ( x - a2 ) - y 2

1+
y LMy (2 x - a1 - a2 ) OP = 0
N
x - b ( x - a1 ) ( x - a2 ) - y 2 Q
1 2 x - a1 - a2
or - =0
x - b ( x - a1 ) ( x - a2 ) - y 2
Simplifying, we get
(x – b)2 + y2 = (b – a1)(b – a2) (4.51)
which is the equation of a circle with the centre at the open-loop zero z = b, and the radius equal to
[(b – a1) (b – a2)]1/2.
The root locus plot for the system given by Eqn. (4.49b), is constructed in Fig. 4.22. The limiting value
of K for stability may be found by graphical construction or by the Jury stability test. We illustrate the use of
graphical construction.
 228 Digital Control and State Variable Methods
Im

K = 22.18
P

¥ K K= K= 0 K= 0
– 0.876 0.67 Re

Unit circle

Fig. 4.22 Root locus plot for the system of Example 4.5

By rule (ix) of Table 4.2, the value of K¢ at point P where the root locus crosses the unit circle is given by
(Phasor length from P to pole at z = 1) (Phasor length from P to pol e z = 0.67)
K¢ =
(Phasor length from P to zero at z = – 0.876)
0.85 ´ 0.78
= = 0.39 = 0.01758K
1.7
0.39
Therefore K = = 22.18
0.01758
The relative stability of the system can be investigated by superimposing the constant-z loci on the system
root locus plot. This is shown in Fig. 4.23. Inspection of this figure shows that the root locus intersects the
z = 0.3 locus at point Q. The value of K¢ at point Q is determined to be 0.1; the gain
K = K¢/0.01758 = 5.7
The value of wn for K¢ = 0.1, may be obtained by superimposing constant-wn loci on the root locus plot
and locating the constant-wn locus which passes through the point Q. From Fig. 4.23, we observe that none
of the constant-wn loci on the standard chart passes through the point Q; we have to make a guess for the
wn value. We can, instead, construct a constant-wd locus passing through the point Q and from there obtain
wn more accurately.

s1, 2 = - zw n ± jw n 1 - z 2 = – zw n ± jw d

are mapped to z1, 2 = e -zw n T e ± jw d T = re ± jq

in the z-plane.
A constant-wd locus is thus a radial line passing through the origin at an angle q = wdT with the positive
real axis of the z-plane, measured positive in the counter clockwise direction.
 Design of Digital Control Algorithms 229

Fig 4.23 Relative stability analysis

The radial line passing through the point Q makes an angle q = 25º with the real axis (Fig. 4.23). This is
a constant-wd locus with wd given by
25 ´ p
wdT = rad
180
25p
Therefore w nT 1 - z 2 =
180
This gives wn = 2.29 rad/sec.
The value of K¢ at the breakaway point R, located at z = 0.824, is determined to be 0.01594. Therefore,
the gain K = 0.01594/0.01758 = 0.9067 results in critical damping (z = 1) with the two closed-loop poles at
z = 0.824.
 230 Digital Control and State Variable Methods

A pole in the s-plane at s = – a has a time constant of t = 1/a and an equivalent z-plane location of e–aT =
e–T/t. Thus, for the critically damped case,
e–0.2/t = 0.824
or t = 1.033 = time constant of the closed-loop poles.

In the frequency-response design procedure described in the previous section, we attempted to reshape
the open-loop frequency response to achieve certain stability margins, steady-state response characteristics
and so on. A different design technique is presented in this section—the root-locus procedure. In this proce-
dure, we add poles and zeros through a digital controller, so as to shift the roots of the characteristic
equation to more appropriate locations in the z-plane. Therefore, it is useful to investigate the effects of
various pole-zero configurations of the digital controller on the root locus plots.
Lead Compensation A simple lead compensator model in the w-plane is described by the transfer func-
tion (refer Eqn. (4.38))
1 + wt
D(w) = ; a < 1, t > 0
1 + a wt
The bilinear transformation
2 z -1
w=
T z +1
transforms D(w) into the following D(z).

D(z) =
1 + 2t /T LM z + (1 - 2t/T )/(1 + 2t /T ) OP
1 + 2at /T N z + (1 - 2at/T )/(1 + 2at / T ) Q
Since t and a are both positive numbers and since a < 1, the pole and zero of D(z) always lie on the real
axis inside the unit circle in the z-plane; the zero is always to the right of the pole. A typical pole-zero
configuration of a lead compensator
z - a1
D(z) = Kc1 (4.52)
z - a2
is shown in Fig. 4.24a.
For the purpose of simplifying the design pro-
cedure, we normally associate the gain Kc1 with
Im Im the plant transfer function, and design the lead
Unit circle Unit circle
compensator
z - a1
D(z) = (4.53a)
z - a2
based on the new plant transfer function. It
Re Re
may be noted that D(z) given by Eqn. (4.53a) is
not a unity-gain model; the dc gain of D(z) is
given by
(a) (b)
lim
FG z - a IJ = FG 1 - a IJ
1 1
(4.53b)
Fig. 4.24 Pole-zero configurations of compensators z®1 H z - a K H1 - a K
2 2
 Design of Digital Control Algorithms 231
To study the effect of a lead compensator on the root loci, we consider a unity feedback sampled-data
system with open-loop transfer function
K ( z + 0.368)
Gh0G(z) = ; T = 1 sec (4.54)
( z - 0.368)( z - 0.135)
The root locus plot of the uncompensated system is shown in Fig. 4.25a. The plot intersects the z = 0.5
locus7 at point P. The value of gain K at this point is determined to be 0.3823.
Constant-wd locus passing through point P is a radial line at an angle of 82º with the real axis (Fig. 4.25a).
Therefore
82p
wdT = wnT 1 - z 2 =
180
This gives wn = 1.65 rad/sec
Since Gh0G(z) given by Eqn. (4.54) is a Type-0 system, we will consider position error constant Kp to
study steady-state accuracy. For K = 0.3823,
0.3823 (1 + 0.368)
Kp = lim Gh0G(z) = = 0.957
z®1 (1 - 0.368)(1 - 0.135)
We now cancel the pole of Gh0G(z) at z = 0.135 by the zero of the lead compensator, and add a pole at
z = – 0.135, i.e., we select
z - 0135
.
D(z) =
z + 0135
.
Figure 4.25b shows the root locus plot of lead compensated system. The modified locus has moved to the
left, toward the more stable part of the plane. The intersection of the locus with the z = 0.5 line is at point Q.
The value of wn at this point is determined to be 2.2 rad/sec. The lead compensator has thus increased wn and
hence the speed of response of the system. The gain K at point Q is determined to be 0.433. The position
error constant of the lead compensated system is given by
0.433( z + 0.368)
Kp = lim D(z)Gh0G(z) = lim = 0.82
z®1 z®1 ( z - 0.368)( z + 0.135)

7
For a given z, the constant-z curve may be constructed using Eqn. (4.15b). The following table gives the real and
imaginary coordinates of points on some constant-z curves.

z = 0.3 { ReIm 0.932

0.164
0.735
0.424
0.360
0.623
0
0.610
– 0.259
0.448
– 0.380
0.220
– 0.373
0
z = 0.4 { ReIm 0.913
0.161
0.689
0.398
0.317
0.549
0
0.504
– 0.201
0.347
– 0.276
0.160
– 0.254
0
z = 0.5 { ReIm 0.891
0.157
0.640
0.370
0.273
0.473
0
0.404
– 0.149
0.259
– 0.191
0.110
– 0.163
0
z = 0.6 { ReIm 0.864
0.152
0.585
0.338
0.228
0.395
0
0.308
– 0.104
0.180
– 0.122
0.070
– 0.095
0
z = 0.7
{ ReIm 0.830
0.146
0.519
0.299
0.179
0.310
0
0.215
– 0.064
0.111
– 0.067
0.039
– 0.046
0
z = 0.8
{ ReIm 0.780
0.138
0.431
0.249
0.124
0.215
0
0.123
– 0.031
0.053
– 0.026
0.015
– 0.015
0
 232 Digital Control and State Variable Methods

Fig. 4.25 Root locus plot for (a) uncompensated; (b) lead compensated; and
(c) lag compensated system

The lead compensator has thus given satisfactory dynamic response, but the position error constant is too
low. We will shortly see how Kp can be increased by lag compensation.
The selection of the exact values of pole and zero of the lead compensator is done by experience and by
trial-and-error. In general, the zero is placed in the neighbourhood of the desired dominant closed-loop
poles, and the pole is located at a reasonable distance to the left of the zero location.
Lag Compensation A simple lag compensator model in the w-plane is described by the transfer function
(refer Eqn. (4.41))
1 + wt
D(w) = ; b > 1, t > 0
1 + b wt
The bilinear transformation
2 z -1
w=
T z +1
transforms D(w) into the following D(z).

D(z) =
1 + 2t /T LM z + (1 - 2t/T )/(1 + 2t/T ) OP
1 + 2 bt /T N z + (1 - 2bt /T )/(1 + 2bt / T ) Q
 Design of Digital Control Algorithms 233
Since t and b are both positive numbers and since b > 1, the pole and zero of D(z) always lie on
the real axis inside the unit circle; the pole is always to the right of the zero. A typical pole-zero configura-
tion of the lag compensator
z - b1
D(z) = K c2 (4.55)
z - b2
is shown in Fig. 4.24b. Note that both the pole and the zero have been shown close to z = 1. This, as we shall
see, gives better stability properties.
Again, we will associate the gain Kc2 with the plant transfer function and design the lag compensator
z - b1
D(z) = (4.56)
z - b2
based on the new plant transfer function. The dc gain of the lag compensator given by (4.56), is equal to
z - b1 1 - b1
lim = (4.57)
z®1 z - b2 1 - b2
To study the effect of lag compensator on the root loci, we reconsider the system described by Eqn. (4.54):
K ( z + 0.368)
Gh0G(z) = ; T = 1 sec
( z - 0.368)( z - 0135 . )
The root locus plot of the uncompensated system is shown in Fig. 4.25a. At point P, z = 0.5, wn = 1.65 and
K = 0.3823 (Kp = 0.957).
We now cancel the pole of Gh0G(z) at z = 0.368 by the zero of the lag compensator, and add a pole at z =
0.9, i.e., we select
z - 0.368
D(z) =
z - 0.9
Figure 4.25c shows the root locus plot of the lag compensated system. The intersection of the locus with
z = 0.5 line is at point R. The value of wn at this point is determined to be 1.2 rad/sec. The lag compensator
has thus reduced wn and hence the speed of response. The value of the gain K at point R is determined to be
0.478. The position error constant of the lag compensated system is
0.478( z + 0.368)
Kp = lim D(z)Gh0G(z) = lim = 7.56
z®1 z ® 1 ( z - 0135
. )( z - 0.9)
Thus we have been able to increase position error constant appreciably by lag compensation.
If both the pole and the zero of the lag compensator are moved close to z = 1, then the root locus plot of
the lag compensated system moves back towards its uncompensated shape. Consider the root locus plot of
the uncompensated system shown in Fig. 4.25a. The angle contributed at point P by additional pole-zero pair
close to z = 1 (called a dipole), will be negligibly small; therefore the point P will continue to lie on the lag
compensated root locus plot. However, the lag compensator
z - b1
D(z) =
z - b2
will raise the system Kp (refer Eqn. (4.57)), by a factor of (1 – b1)/(1 – b2).
The following examples illustrate typical digital control system design problems carried out in the
z-plane, using the root locus technique. As we shall see, the design of digital compensation using root locus
plots is essentially a trial-and-error method. The designer may rely on a digital computer to plot out a large
number of root loci by scanning through a wide range of possible values of the compensator parameters, and
select the best solution. However, one can still make proper and intelligent initial ‘guesses’ so that the
amount of trial-and-error effort is kept to a minimum.
 234 Digital Control and State Variable Methods

Example 4.6 Consider the feedback control system shown in Fig. 4.26. The plant is described by the
transfer function
K
G(s) =
s( s + 2)
Design a digital control scheme for the system to meet the following specifications;
(i) the velocity error constant Kv = 6;
(ii) peak overshoot Mp to step input £ 15%, and
(iii) settling time ts (2% tolerance band) £ 5 sec.

r e u y
+
D (z ) G h0(s) G (s )
– T T

Fig. 4.26 A feedback system with digital compensation

Solution The transient accuracy requirements correspond to z = 0.5 and wn = 1.6. We select T = 0.2 sec.
Note that sampling frequency ws = 2p/T is about 20 times the natural frequency; therefore, our choice of
sampling period is satisfactory.
The transfer function Gh0G(z) of the plant, preceded by a ZOH, can be obtained as follows:

Gh0G(z) = (1 – z–1) Z
LM K OP
2
N s ( s + 2) Q
0.01758K ( z + 0.876) K ¢( z + 0.876)
= = (4.58)
( z - 1)(z - 0.67) ( z - 1)( z - 0.67)
Im
The root locus plot of this system for
0 £ K¢ < ¥ was earlier constructed in Fig. 4.22.
Complex-conjugate sections of this plot are
Locus
shown in Fig. 4.27. The plot intersects the
for z = 0.5
z = 0.5 locus at point P. At this point wn = 1.7
rad/sec, K¢ = 0.0546.
P q = 17° Therefore, the transient accuracy require-
ments (z = 0.5, wn = 1.6) are almost satisfied by
gain adjustment only. Let us now examine the
– 0.876 0.67 Re
steady-state accuracy of the uncompensated
system (D(z) = 1) with K¢ = 0.0546.
The velocity error constant Kv of the sys-
tem is given by
1
Kv = lim (z – 1)Gh0G(z)
T z ®1
Root locus
Unit circle 5(0.0546)(1 + 0.876)
= = 1.55
Fig. 4.27 Root locus plot for system (4.58) (1 - 0.67)
 Design of Digital Control Algorithms 235
The specified value of Kv is 6. Therefore, an increase in Kv by a factor of 3.87 (= 6/1.55) is required.
The objective before us now is to introduce a D(z) that raises the system Kv by a factor of 3.87, without
appreciably affecting the transient performance of the uncompensated system, i.e., without appreciably
affecting the root locus plot in the vicinity of point P. This objective can be realized by a properly designed
lag compensator, as is seen below.
We add the compensator pole and zero as shown in Fig. 4.28. Since both the pole and the zero are very
close to z = 1, the scale in the vicinity of these points has been greatly expanded. The angle contributed by
the compensator pole at point P, is almost equal to the angle contributed by the compensator zero. There-
fore, the addition of dipole near z = 1 does not appreciably disturb the root locus plot in the vicinity of point
P. It only slightly reduces wn. The lag compensator
z - 0.96
D(z) =
z - 0.99
raises the system Kv by a factor of (1 – 0.96)/(1 – 0.99) = 4.

Fig. 4.28 Compensator design (Example 4.6)

Note that because of lag compensator, a third closed-loop pole has been added. This pole, as seen from
Fig. 4.28, is a real pole lying close to z = 1. This pole, fortunately, does not disturb the dominance of the
complex conjugate closed-loop poles. The reason is simple.
The closed-loop pole, close to z = 1, has a long time constant. However, there is a zero close to this
additional pole. The net effect is that the settling time will increase because of the third pole, but the ampli-
tude of the response term contributed by this pole will be very small. In system response, a long tail of small
amplitude will appear which may not appreciably degrade the performance of the system.

Example 4.7 Reconsider the feedback control system of Example 4.6 (Fig. 4.26). We now set the following
goal for our design.
(i) Kv ³ 2.5
(ii) z @ 0.5
(iii) ts (2% tolerance band) £ 2 sec
 236 Digital Control and State Variable Methods
The transient accuracy requirements correspond to z = 0.5 and wn = 4. For sampling interval T = 0.2 sec,
the sampling frequency is about eight times the natural frequency. A smaller value of T is more appropriate
for the present design problem—which requires higher speed of response. We will however take T = 0.2 sec
to compare our results with those of Example 4.6.
Following the initial design steps of Example 4.6, we find that
0.01758K ( z + 0.876) K ¢( z + 0.876)
Gh0G(z) = =
( z - 1)(z - 0.67) ( z - 1)( z - 0.67)
Complex conjugate sections of the root locus plot superimposed on z = 0.5 line are shown in Fig. 4.27.
The root locus plot intersects the constant-z locus at point P. At this point, wn = 1.7 rad/sec. The specified
value of wn is 4. Therefore, the transient accuracy requirements cannot be satisfied by only gain adjustment.
The natural frequency wn can be increased by lead compensation. To design a lead compensator, we
translate the transient performance specifications into a pair of dominant closed-loop poles, add open-loop
poles and zeros through D(z) to reshape the root locus plot, and force it to pass through the desired closed-
loop poles.
Point Q in Fig. 4.29 corresponds to the desired closed-loop pole in the upper half of z-plane. It is the point
of intersection of the z = 0.5 locus and the constant-wd locus, with wd given by
wd = wn 1 - z 2 = 3.464 rad/sec

For this value of wd, constant-wd locus is a radial line at an angle of wdT
F 180 I = 39.7º with the real axis.
HpK
Im

Locus
for z = 0.5
Q q = 39.7°

51.26°

– 0.876 0.254 0.67 Re

Unit circle

Fig. 4.29 Compensator design (Example 4.7)

If the point Q is to lie on the root locus plot of the compensated system, then the sum of the angles
contributed by open-loop poles and zeros of the plant, and the pole and zero of the compensator at the point
Q, must be equal to ± (2q + 1)180º; q = 0, 1, 2, K
 Design of Digital Control Algorithms 237
The sum of the angle contributions due to open-loop poles and zero of the plant at plant Q, is
17.10º – 138.52º – 109.84º = – 231.26º
Hence the compensator D(z) must provide +51.26º. The transfer function of the compensator may be
assumed to be
z - a1
D(z) =
z - a2
If we decide to cancel the pole at z = 0.67 by the zero of the compensator at z = a1, then the pole of the
compensator can be determined (from the condition that the compensator must provide + 51.26º) as a point
at z = 0.254 (a2 = 0.254). Thus, the transfer function of the compensator is obtained as
z - 0.67
D(z) =
z - 0.254
The open-loop transfer function now becomes
0.01758 K ( z + 0.876)( z - 0.67) 0.01758 K ( z + 0.876) K ¢ ( z + 0.876)
D(z)Gh0G(z) = = =
( z - 0.254)( z - 1)( z - 0.67) ( z - 0.254)( z - 1) ( z - 0.254)( z - 1)
The value of K¢ at point Q, obtained from Fig. 4.29 by graphical construction, is 0.2227. Therefore K = 12.67.
The velocity error constant of the compensated system is given by
1
Kv = lim [(z – 1)D(z)Gh0G(z)] = 2.8
T z®1
It meets the specification on steady-state accuracy.
If it is required to have a large Kv, then we may include a lag compensator. The lag-lead compensator can
satisfy the requirements of high steady-state accuracy and high speed of response.
From the viewpoint of microprocessor implementation of the lag, lead, and lag-lead compensators, the
lead compensators present the least coefficient quantization problems, because the locations of poles and
zeros are widely separated, and the numerical inaccuracies in realization of these compensators will result in
only small deviations in expected system behaviour. However, in the case of lag compensators and lag-lead
compensators, the lag section may result in considerable coefficient quantization problems, because the
locations of poles and zeros are usually close to each other (they are near the point z = 1). Numerical
problems associated with realization of compensator coefficients, may lead to significant deviations in
expected system behaviour.

4.5 z-PLANE SYNTHESIS

Much of the style of the transform domain techniques we have been discussing in this chapter, grew out of
the limitations of technology which was available for realization of the compensators with pneumatic com-
ponents, or electric networks and amplifiers. In the digital computer, such limitations on realization are, of
course, not relevant, and one can ignore these particular constraints. One design method which eliminates
these constraints begins from the very direct point of view that we are given a process (plus hold) transfer
function, Gh0G(z), that we want to construct a desired transfer function, M(z), between input r and output y
and that we have the computer transfer function, D(z), to do the job as per the feedback control structure of
Fig. 4.30.
 238 Digital Control and State Variable Methods

r + e u u + (t) y
D (z) G h0(s) G (s )
T T
–
Computer ZOH Process

Fig. 4.30 A feedback system with digital compensation

The closed-loop transfer function is given by the formula

D( z)Gh 0 G ( z )
M(z) = (4.59)
1 + D( z )Gh 0 G( z )
from which we get the design formula
1
D(z) =
M ( z) LM OP (4.60)
Gh 0 G( z ) 1 - M ( z ) N Q
As is seen from Eqn. (4.60), the controller transfer function consists of the inverse of the plant transfer
function and the additional term which depends on the system closed-loop transfer function. Thus, the
design procedure, outlined above, looks for a D(z) which will cancel the process effects and add whatever is
necessary to give the desired performance.
For prescribing the required closed-loop transfer function M(z), the following restrictions have to be
noted.
Realizability of Digital Controller Assume that a digital controller
v v -1
Qv ( z ) q0 z + q1 z + L + qv
D(z) = = m m -1
(4.61)
Pm ( z ) z + p1 z + L + pm
is cascaded with the process
Bm ( z) b0 z m + b1 z m -1 + L + bm
GhoG(z) = = n ;m£n (4.62)
An ( z) z + a1z n - 1 + L + a n
in the control loop given by Fig. 4.30.
For D(z) to be physically realizable, n £ m.
The closed-loop transfer function
D ( z ) Gh 0 G ( z ) Qv ( z) Bm ( z ) N v + m (z)
M(z) = = =
1 + D ( z ) Gh 0 G ( z ) Pm ( z) An ( z) + Qv ( z ) Bm ( z ) Dm + n ( z )
The order of the numerator polynomial of M(z) is n + m, and the order of the denominator polynomial of
M(z) is m + n. The pole excess8 of M(z) is therefore {(m – n) + (n – m)}.
This means that because of the condition of realizability of digital controller, the pole excess of the
closed-loop transfer function M(z) has to be greater than or equal to the pole excess of the process transfer
function Gh0G(z).
Cancellation of Poles and Zeros If the digital controller D(z) given by Eqn. (4.60) and the process
Gh0G(z) are in a closed loop, the poles and zeros of the process are cancelled by the zeros and poles of the
controller. The cancellation is perfect if the process model Gh0G(z) matches the process exactly. Since the

8
Pole excess of M(z) = {number of finite poles of M(z) – number of finite zeros of M(z)}.
 Design of Digital Control Algorithms 239
process models used for design practically never describe the process behaviour exactly, the corresponding
poles and zeros will not be cancelled exactly; the cancellation will be approximate. For poles and zeros of
Gh0G(z) which are sufficiently spread in the inner of the unit disc in the z-plane, the approximation in cancel-
lation leads to only small deviations of the assumed behaviour M(z) in general. However, one has to be
careful if Gh0G(z) has poles or zeros on or outside the unit circle. Imperfect cancellation may lead to weakly
damped or unstable behaviour. Therefore, the design of digital controllers, according to Eqn. (4.60), has to
be restricted to cancellation of poles and zeros of Gh0G(z) located inside the unit circle. This imposes certain
restrictions on the desired transfer function M(z) as is seen below.
Assume that Gh0G(z) involves an unstable (or critically stable) pole at z = a. Let us define
G1 ( z)
Gh0G(z) =
z -a
where G1(z) does not include a term that cancels with (z – a). Then the closed-loop transfer function becomes
G ( z)
D( z ) 1
z -a
M(z) = (4.63)
G ( z)
1 + D( z ) 1
z -a
Since we require that no zero of D(z) cancel the pole of Gh0G(z) at z = a, we must have
1 z-a
1 – M(z) = =
G1 ( z) z - a + D( z )G1 ( z )
1 + D( z )
z -a
that is, 1 – M(z) must have z = a as a zero. This argument applies equally if Gh0G(z) involves two or more
unstable (or critically stable) poles.
Also note from Eqn. (4.63) that if poles of D(z) do not cancel zeros of Gh0G(z), then the zeros of Gh0G(z)
become zeros of M(z).
Let us summarize what we have stated concerning cancellation of poles and zeros of Gh0G(z).
(i) Since the digital controller D(z) should not cancel unstable (or critically stable) poles of Gh0G(z), all
such poles of Gh0G(z) must be included in 1 – M(z) as zeros.
(ii) Zeros of Gh0G(z) that lie on or outside the unit circle should not be cancelled with poles of D(z); all
such zeros of Gh0G(z) must be included in M(z) as zeros.
The design procedure, thus, essentially involves the following three steps:
(I) The closed-loop transfer function M(z) of the final system is determined from the performance speci-
fications, and the fixed parts of the system, i.e., Gh0G(z).
(II) The transfer function D(z) of the digital controller is found using the design formula (4.60).
(III) The digital controller D(z) is synthesized.
Step (I) is certainly the most difficult one to satisfy. In order to pass step (I), a designer must fulfil the
following requirements.
(i) The digital controller D(z) must be physically realizable.
(ii) The poles and zeros of Gh0G(z) on or outside the unit circle should not be cancelled by D(z).
(iii) The system specifications on transient and steady-state accuracy should be satisfied.

Example 4.8 The plant of sampled-data system of Fig. 4.30 is described by the transfer function
1
G(s) = (4.64a)
s(10s + 1)
 240 Digital Control and State Variable Methods
The sampling period is 1 sec.
The problem is to design a digital controller D(z) to realize the following specifications:
(i) Kv ³ 1
(ii) z = 0.5, and
(iii) ts (2% tolerance band) £ 8 sec.
The selection of a suitable M(z) is described by the following steps.
(i) The z-transfer function of the plant is given by (refer Table 2.1)

Gh0G(z) = (1 – z–1) Z
LM 1 OP = 0.04837 (z + 0.9672) (4.64b)
2
N s (10s + 1) Q ( z - 1)( z - 0.9048)
Since Gh0G(z) has one more pole than zero, M(z) must have a pole excess of at least one.
(ii) Gh0G(z) has a pole at z = 1. This must be included in 1– M(z) as zero, i.e.,
1 – M(z) = (z – 1)F(z) (4.65)
where F(z) is a ratio of polynomials of appropriate dimensions.
(iii) The transient accuracy requirements are specified as z = 0.5, wn = 1(ts = 4/zwn). With a sampling
period T = 1 sec, this maps to a pair of dominant closed-loop poles in the z-plane with
-zw n T
| z1, 2 | = e = 0.6065
0.866 ´ 180
Ð z1, 2 = ± wnT 1 - z 2 = ± = ± 49.64º
314
.
This corresponds to
z1, 2 = 0.3928 ± j 0.4621
The closed-loop transfer function, M(z), should have dominant poles at the roots of the equation
D(z) = z2 – 0.7856 z + 0.3678 = 0 (4.66)
The steady-state accuracy requirements demand that steady-state error to unit-step input is zero, and
steady-state error to unit-ramp input is less than 1/Kv.
E(z) = R(z) – Y(z) = R(z)[1 – M(z)] = R(z) (z – 1)F(z)
*
ess = lim z( z - 1) F( z) = 0
unit step z®1
Thus, with the choice of M (z) given by Eqn. (4.65), the steady-state error to unit step input is always zero.
* Tz
ess = lim (z – 1) (z – 1)F(z) = T F(1) = 1/Kv
unit ramp z®1 ( z - 1) 2
For T = 1 and Kv = 1,
F(1) = 1 (4.67)
From Eqns (4.65) and (4.66), we observe that
z -a
F(z) =
z 2 - 0.7856z + 0.3678
meets the requirements on realizability of D(z), cancellation of poles and zeros of Gh0G(z), and transient
accuracy. The requirement on steady-state accuracy is also met if we choose a such that (refer Eqn. (4.67))
1-a
=1
1 - 0.7856 + 0.3678
 Design of Digital Control Algorithms 241
This gives a = 0.4178
z - 0.4178 ( z - 1)( z - 0.4178)
Therefore F(z) = 2
; 1 – M(z) = 2
z - 0.7856 z + 0.3678 z - 0.7856 z + 0.3678

0.6322 z - 0.05
M(z) = 2
(4.68)
z - 0.7856 z + 0.3678
Now, turning to the basic design formula (4.60), we compute

D(z) =
1 LMM ( z)
=
OP
( z - 1)(z - 0.9048) LM
0.6322 z - 0.05 OP
Gh 0 G( z) 1 - M ( z)
N (0.04837)( z + 0.9672) ( z - 1)( z - 0.4178)
Q N Q
(z - 0.9048)( z - 0.079)
= 13.07 (4.69)
( z + 0.9672 )(z - 0.4178)
A plot of the step response of the resulting design is provided in Fig. 4.31, which also shows the control
effort. The underdamped response settles within a two percent band of the desired value of unity in less
than 8 sec. We can see the oscillation of u(k)—associated with the pole of D(z) at z = –0.9672, which is
quite near the unit circle. Strong oscillations of u(k) are often considered unsatisfactory, even though the
process is being controlled as was intended. In the literature, poles near z = –1 are often referred to as
ringing poles.

Fig. 4.31 Step response (Example 4.8)

To avoid the ringing effect, we could include the zero of Gh0G(z) at z = – 0.9672 in M(z) as zero, so that
this zero of Gh0G(z) is not cancelled with pole of D(z). M(z) may have additional poles at z = 0, where the
transient is as short as possible. The result will be a simpler D(z) with a slightly more complicated M(z).
 242 Digital Control and State Variable Methods

Review Examples

Review Example 4.1 Consider the digital control system shown in Fig. 4.32. The transfer function of
the plant is G(s) = 1/[s(s + 1)]. Design a lead compensator D(z) in the w-plane such that the phase margin in
50º, the gain margin is at least 10 dB, and the velocity error constant Kv is 2. Assume that the sampling period
is 0.2 sec.

r + e u y
D (z) G h0(s) G (s )
T T
–
Compensator ZOH Plant

Fig. 4.32 A digital control configuration

Solution The digital controller is assumed to be of the form

U (z)
= KD1(z) = D(z)
E ( z)
To simplify the design procedure, we will associate the gain K of the controller with the plant model. The
design problem is therefore, to obtain compensator D1(z) for the plant
K
G(s) =
s( s + 1)
to meet the specifications on steady-state and transient performance.
We will fix the gain K to a value that realizes the given Kv. A unity dc gain compensator D1(z) will then be
introduced to meet the transient accuracy requirements without affecting the steady-state accuracy.
Kv = lim sG(s) = K
s®0
Therefore, K = 2 meets the requirement on steady-state accuracy.
For T = 0.2 and K = 2, we have (refer Table 2.1)

Gh0G(z) = (1 – z–1) Z
LM 2 OP = 0.03746 LM z + 0.9356 OP
2
N s (s + 1) Q N (z - 1)( z - 0.8187) Q
By use of the bilinear transformation
1 + wT/2 1 0.1w
z= =
1 - wT/2 1 0.1w
Gh0G(z) can be transformed to Gh0G(w) given below (refer Eqns (4.37)).
FG
2 1-
w IJ FG1 + w IJ
Gh0G(w) =
H 10 K H 300.6K
F w IJ
w G1 +
H 0.997 K
The Bode plot of Gh0G(jn) is shown in Fig. 4.33. The phase margin can be read from the Bode plot as 32º
and the gain margin as 14.2 dB.
 Design of Digital Control Algorithms 243
It is desired to raise the phase margin to 50º without altering Kv. Also the gain margin should be at least
10 dB. We now design a lead compensator
1 + wt
D1(w) = ; a < 1, t > 0
1 + a wt
to meet these objectives. We choose the zero of the compensator at 0.997 (This choice cancels a pole
of Gh0G(w)). Addition of this zero shifts the gain crossover frequency of the uncompensated system to
nc = 1.8. The phase margin of the uncompensated system at nc is FM1 = 22º. The phase lead required at
nc to bring the phase margin to the specified value FMs = 50º, is given by
fl = F Ms – FM1 + e = 50º – 22º + 3º = 31º

Fig. 4.33 Compensator design (Review Example 4.1)

By using Eqn. (4.39), we obtain

1 - sin 31º
a= = 0.3
1 + sin 31º
The phase lead of 31° is provided at the frequency (refer Eqn. (4.40))

nm =
1 1F I= 0.997 ´ 3.27 = 1.8
H K
t at
which is same as the gain crossover frequency.
Thus, the compensator transfer function is
w
1+
D1(w) = 0.997 (4.70)
w
1+
3.27
 244 Digital Control and State Variable Methods
The magnitude and phase angle curves for the compensated open-loop transfer function are shown by solid
curves in Fig. 4.33. From these curves, we see that the phase margin is 51º and the gain margin is 11.5 dB.
The compensator transfer function given by Eqn. (4.70) will now be transformed back to the z-plane by
the bilinear transformation
2 z-1 z -1
w= = 10
T z+1 z+1
This gives D1(z) = 2.718
FG
z - 0.8187 IJ
H
z - 0.5071 K
The system gain K was determined to be 2. Therefore, for the plant of the system of Fig. 4.32, the digital
controller is given by
U (z)
= D(z) = 2.718 K
FG
z - 0.8187 IJ
= 5.436
FG
z - 0.8187 IJ
E ( z) H
z - 0.5071 K H
z - 0.5071 K
Review Example 4.2 Consider the digital control configuration shown in Fig. 4.32. The transfer function

e -1.5 s
G(s) =
s +1
describes a process of temperature control via mixing (refer Example 3.3). In the following, we design a
digital compensator for the temperature control process; the sampling interval T is assumed to be 1 sec.
The transfer function Gh0G(z) derived in Example 3.3 is repeated below.
z + 0.6066
Gh0G(z) = 0.3935 2
z ( z - 0.3679 )
Since Gh0G(z) is a type-0 transfer function, the system will have a steady-state error to a constant com-
mand or disturbance. If we assume that such a behaviour in steady-state is unacceptable, we can correct the
problem by including integral control through the transfer function
Kz
D1(z) =
z -1
The effective plant transfer function is now
( z + 0.6066) K ¢( z + 0.6066)
D1(z)Gh0G(z) = 0.3935K =
z ( z - 1)( z - 0.3679) z ( z - 1)( z - 0.3679)
The unity-feedback root locus plot for this transfer function is sketched in Fig. 4.34. The point P on the
FG
root locus corresponds to z = 0.5, and wn = 0.423 q = w n T 1 - z 2 =
21p
.
IJ
H 180 K
The natural frequency wn has to be raised to improve the speed of response. We employ a lead compen-
sation which cancels the plant pole at z = 0.3679 and the plant zero at z = – 0.6066. The open-loop transfer
function of the lead-compensated system becomes

D2(z)D1(z)Gh0G(z) =
z - 0.3679 LM K ¢ ( z + 0.6066) OP=
K¢
z + 0.6066 z (Nz - 1 )( z - 0.3679 )Q z ( z - 1)
The root locus plot is sketched in Fig. 4.35. The point Q on the root locus corresponds to z = 0.5, wn = 0.826,
Kv = K¢ = 0.45.
Suppose we wish to raise Kv to 1. A lag compensator
z - 0.9
D3(z) =
z - 0.9545
 Design of Digital Control Algorithms 245
Im

Locus
for z = 0.5
P q = 21°

Re

Unit circle

Fig. 4.34 Root locus plot of mixing flow plant with integral control

will raise Kv by a factor of (1 – 0.9)/(1 – 0.9545). The lag pole-zero pair are very close to each other, and do
not change the root locus near the dominant roots significantly. However, the lag compensation does intro-
duce a small, but very slow transient, whose effect on dynamic response needs to be evaluated, especially in
terms of the response to disturbances.

Im

Locus q = 41°
for z = 0.5
Q

Re

Unit circle
Fig. 4.35 Root locus plot of mixing flow plant with integral control and lead compensation
246 Digital Control and State Variable Methods

Problems
4.1 For the system shown in Fig. P4.1, find
(i) position error constant, Kp,
(ii) velocity error constant, Kv, and
(iii) acceleration error constant, Ka.
Express the results in terms of K1, K2, J, and T.

r + e + 1 q 1 q
K1 G h0(s)
T Js s
– –

K2

Fig. P4.1

4.2 Consider the analog control system shown in Fig. P4.2a. Show that the phase margin of the system is about 45º.
We wish to replace the analog controller by a digital controller as shown in Fig. P4.2b. First, modify the
analog controller to take into account the effect of the hold that must be included in the equivalent digital
control system (the zero-order hold may be approximated by a pure time delay of one half the sampling period
T (Fig. P4.2c), and then a lag compensator D1(s) may be designed to realize the phase margin of 45º). Then, by
using the bilinear transformation, determine the equivalent digital controller.
Compare the velocity error constants of the original analog system of Fig. P4.2a, and the equivalent digital
system of Fig. P4.2b.

r + 1.57 y
D (s) = 1
– s(s + 1)

(a)

y
r + 1 – e– sT 1.57
D 1 (z )
T = 1.57 T s s(s + 1)
–

(b)

y
r + 1.57 e – sT/2
D 1 (s)
s(s + 1)
–

(c)
Fig. P4.2
 Design of Digital Control Algorithms 247
4.3 A unity feedback system is characterized by the open-loop transfer function
0.2385( z + 0.8760)
Gh0G(z) =
( z - 1)(z - 0.2644)
The sampling period T = 0.2 sec.
Determine steady-state errors for unit-step, unit-ramp, and unit-acceleration inputs.
4.4 Predict the nature of the transient response of a discrete-time system whose characteristic equation is given by
z2 – 1.9z + 0.9307 = 0
The sampling interval T = 0.02 sec.
4.5 The system of Fig. P4.5 contains a disturbance input W(s), in addition to the reference input R(s).
(a) Express Y(z) as a function of the two inputs.
(b) Suppose that D2(z) and D3(z) are chosen such that D3(z) = D2(z)Gh0G(z). Find Y(z) as a function of the two inputs.
(c) What is the advantage of the choice in part (b) if it is desired to minimize the response Y(z) to the distur-
bance W(s)?

D 2 (z) W (s)

+ + Y (s )
R (s ) + + +
D 3 (z) D 1 (z) G h0 (s) G (s)
T – T

Fig. P4.5

4.6 Consider the system of Fig. P4.6. The design specifications for the system require that
(i) the steady-state error to a unit-ramp reference input be less than 0.01, and
(ii) a constant disturbance w should not affect the steady-state value of the output.
Show that these objectives can be met if D(z) is a proportional-plus-integral compensator.
w

+
r + + 1 y
D (z) G h0(s)
– T T s+1

Fig. P4.6

4.7 Consider the feedback system shown in Fig. P4.7. The nominal values of the parameters K and t of the plant
G(s) are both equal to 1. Find an expression for the sensitivity S(z) of the closed-loop transfer function M(z),
with respect to incremental changes in open-loop transfer function Gh0G(z). Plot |S(ejwT)| for 0 £ w £ ws/2, where
ws is the sampling frequency. Determine the bandwidth of the system if it is designed to have |S(ejwT)| < 1.

r + K y
G h0(s) G (s ) =
T = 0.5 sec ts + 1
–

Fig. P4.7
248 Digital Control and State Variable Methods
4.8 A unity feedback digital control system has open-loop transfer function
0.368z + 0.264
Gh0G(z) = 2 ; T = 1 sec
z - 1.368z + 0.368
jwT
The function Gh0G(e ) may be used to obtain frequency response plots of the system. This function is,
however, irrational. Prove that the relation
2 nT
w = tan -1
T 2
approximates Gh0G(e jwT ) to a rational function Gh0G( jn).
For Gh0G( jn), construct the Bode plot, and the log-magnitude versus phase angle plot and obtain the gain
margin, the phase margin and the bandwidth nb. What is the corresponding value of wb? The –3 dB contour of
the Nichols chart may be constructed using the following table:

Phase, degrees 0 –30 –60 – 90 – 120 – 150 – 180 – 210

Magnitude, dB 7.66 6.8 4.18 0 – 4.18 – 6.8 – 7.66 – 6.8

4.9 Consider the control system of Fig. P4.9, where the plant transfer function
1
G(s) = , and T = 0.1 sec
s( s + 2)
(a) Increase the plant gain to the value that results in Kv = 5. Then find the phase margin.
(b) Design a lead compensator that results in 55º phase margin with Kv = 5.
(c) Design a lag compensator that results in 55º phase margin with Kv = 5.
(d) Obtain the bandwidth realized by the three designs corresponding to parts (a), (b) and (c). Comment on the
result.
(e) Is the selection of T = 0.1 sec justified from closed-loop bandwidth considerations?

r + e u y
D (z ) G h0 (s) G (s)
– T T
Compensator ZOH Plant

Fig. P4.9
4.10 Consider the control system of Fig. P4.9, where the plant transfer function is G(s) = 1/s2, and T = 0.1 sec.
Design a lead compensator such that the phase margin is 50º and the gain margin is at least 10 dB. Obtain the
velocity error constant Kv of the compensated system.
Can the design be achieved using a lag compensator? Justify your answer.
4.11 Consider the control system of Fig. P4.9, where the plant transfer function is
K
G(s) = , and T = 0.1 sec
s(s + 5)
The performance specifications are given as
(i) velocity error constant Kv ³ 10,
(ii) phase margin FM ³ 60º, and
(iii) bandwidth wb = 8 rad/sec
(a) Find the value of K that gives Kv = 10. Determine the phase margin and the bandwidth of the closed-loop
system.
(b) Show that if lead compensation is employed, the system bandwidth will increase beyond the specified
value, and if lag compensation is attempted, the bandwidth will decrease sufficiently so as to fall short of
the specified value.
 Design of Digital Control Algorithms 249
(c) Design a lag section of a lag-lead compensator to provide partial compensation for the phase margin. Add
a lead section to realize phase margin of 60º. Check the bandwidth of the lag-lead compensated system.
(d) Find the transfer function D(z) of the lag-lead compensator and suggest a realization scheme.
4.12 Shown in Fig. P4.12a is a closed-loop temperature control system. Controlled electric heaters maintain the
desired temperature of the liquid in the tank. The computer output controls electronic switches (triacs), to vary
the effective voltage supplied to the heaters, from 0 V to 230 V. The temperature is measured by a thermocouple
whose output is amplified to give a voltage in the range required by A/D converter. A simplified block diagram
of the system, showing perturbation dynamics, is given in Fig. P4.12b.
(a) Consider the analog control loop of Fig. P4.12c. Determine K that gives 2% steady-state error to a step
input.
(b) Let D(z) = K obtained in part (a). Is the sampled-data system of Fig. P4.12b stable for this value of D(z)?
(c) Design a lag compensator for the system of part (b), such that 2% steady-state error is realized, the phase
margin is greater than 40º and the gain margin is greater than 6 dB. Give the total transfer function D(z) of
the compensator.
(d) Can the design of part (c) be achieved using a lead compensator? Justify your answer.

+
–

+
–

Fig. P4.12
250 Digital Control and State Variable Methods
4.13 (a) Consider a unity-feedback system with open-loop transfer function
K ( z - z1 )
Gh0G(z) = ;0£K<¥
( z - p1 )(z - p2 )
The poles and zero of this second-order transfer function lie on the real axis; the poles are adjacent or
congruent, with the zero to their left. Prove that the complex-conjugate section of the root locus plot is a
circle with the centre at z = z1, and the radius equal to ( z1 - p1 ) ( z1 - p2 ) .
K ( z - 0.9048)
(b) Given Gh0G(z) =
( z - 1)2
Sketch the root locus plot for 0 £ K < ¥. Using the information in the root locus plot, determine the range
of values of K for which the closed-loop system is stable. Also determine the value of K for which the
system closed-loop poles are real and multiple.
4.14 A sampled-data feedback control system is shown in Fig. P4.14. The controlled process of the system is
described by the transfer function
K
G(s) = ;0£K<¥
s( s + 1)
The sampling period T = 1 sec.
(a) Sketch the root locus plot for the system on the z-plane and from there obtain the value of K that results in
marginal stability.
(b) Repeat part (a) for (i) T = 2 sec, (ii) T = 4 sec, and compare the stability properties of the system with
different values of sampling interval.

r + y
G h0(s) G (s )
– T

Fig. P4.14

4.15 The digital process of a unity feedback system is described by the transfer function
K ( z + 0.717)
Gh0G(z) = ; T = 1 sec
( z - 1)( z - 0.368)
Sketch the root locus plot for 0 £ K < ¥ and from there obtain the following information:
(a) The value of K that results in marginal stability. Also find the frequency of oscillations.
(b) The value of K that results in z = 1. What are the time constants of the closed-loop poles?
(c) The value of K that results in z = 0.5. Also find the natural frequency wn for this value of K. You may use
the following table to construct a constant-z locus on the z-plane corresponding to z = 0.5.
Re 0.891 0.64 0.389 0.169 0 –0.113 –0.174 –0.188 –0.163
Im 0.157 0.37 0.463 0.464 0.404 0.310 0.207 0.068 0
4.16 The characteristic equation of a feedback control system is
z2 + 0.2A z – 0.1 A = 0
Sketch the root loci for 0 £ A < ¥, and therefrom obtain the range of parameter A for which the system is stable.
4.17 The block diagram of a sampled-data system using a dc motor for speed control is shown in Fig. P4.17. The
encoder senses the motor speed, and the output of the encoder is compared with the speed command. Sketch the
root locus plot for 0 £ K < ¥.
 Design of Digital Control Algorithms 251
(a) For K = 1, find the time constant of the closed-loop pole.
(b) Find the value of K which results in a closed-loop pole whose time constant is less than or equal to one
fourth of the value found in part (a).
Use the parameter values:
Km = 1, tm = 1, T = 0.1 sec, P = 60 pulses/revolution.

wr + Km w
G h0(s) K
T tm s + 1
–
ZOH PWM Motor
amplifier
PT
2p

Encoder
Fig. P4.17

1
4.18 Consider the system shown in Fig. P4.9 with G(s) = and T = 0.2 sec.
s(s + 1)
(a) Design a lead compensator so that the dominant closed-loop poles of the system will have z = 0.5 and
wn = 4.5.
(b) Obtain the velocity error constant Kv of the lead compensated system.
(c) Add a lag compensator in cascade so that Kv is increased by a factor of 3. What is the effect of the lag
compensator on the transient response of the system?
(d) Obtain the transfer function D(z) of the lag-lead compensator, and suggest a realization scheme.
Use root locus method.
4.19 Consider the system shown in Fig. P4.9 with
1
G(s) = ; T = 1 sec
( s + 1)( s + 2)
Design a compensator D(z) that meets the following specifications on system performance:
(a) z = 0.5
(b) wn = 1.5, and
(c) Kp ³ 7.5
Use root locus method.
4.20 The block diagram of a digital control system is shown in Fig. P4.9. The controlled process is described by the
transfer function
K
G(s) = 2 ; T = 1 sec
s
which may represent a pure inertial load.
(a) The dominant closed-loop poles of the system are required to have z = 0.7, wn = 0.3 rad/sec. Mark the
desired dominant closed-loop pole locations in the z-plane. The root loci must pass through these points.
(b) Place the zero of the compensator D(z) below the dominant poles and find the location of pole of D(z), so
that the angle criterion at the dominant poles is satisfied. Find the value of K, so that the magnitude
criterion at the dominant poles is satisfied.
(c) Find the acceleration error constant, Ka.
(d) Your design will result in specified values of z and wn for the closed-loop system response, only if
the dominance condition is satisfied. Find the third pole of the closed-loop system and comment on the
effectiveness of your design.
252 Digital Control and State Variable Methods
4.21 The configuration of a commercial broadcast videotape positioning system is shown in Fig. P4.21. The rela-
tionship between the armature voltage (applied to drive motor) and tape speed at the recording and playback
heads, is approximated by the transfer function G(s). The delay term involved, accounts for the propagation of
speed changes along the tape, over the distance of physical separation of the tape drive mechanism and the
recording and playback heads. The tape position is sensed by a recorded signal on the tape itself.

G (s) =
qr + 40 e–s/120 1 q
D (z ) D /A
s + 40 s
–

A /D 1

Position sensor
Fig. P4.21

Design the digital controller that should result in zero steady-state error to any step change in the desired
tape position. The closed-loop poles of the system are required to lie within a circle of radius 0.56. Take the
sampling interval T = 1/120 sec.
4.22 Consider the sampled-data system shown in Fig. P4.22; the plant is known to have the transfer function
1
G(s) =
s(s + 1)
A sampling period of T = 0.1 sec is to be used.
(a) Design a digital controller to realize the following specifications:
(i) z = 0.8,
(ii) wn = 2p/10T, and
(iii) Kv ³ 5.
(b) Design a digital controller so that the response to unit-step input is
y(k) = 0, 0.5, 1, 1, K
Find the steady-state error to unit-ramp input.

r + e u y
D (z ) G h0(s) G (s )
– T T

Fig. P4.22

4.23 In the control configuration of Fig. P4.22, find the control algorithm D(z) so that the response to a unit-step
function will be y(t) = 1– e–t. The plant transfer function is
1
G(s) =
10 s + 1
Assume that the sampling interval T = 2 sec.
 Part II
STATE VARIABLE METHODS IN AUTOMATIC CONTROL:
CONTINUOUS-TIME AND SAMPLED-DATA SYSTEMS
In Part I of the book, we have developed some general procedures for the design of controllers. Our
discussion was basically centered around the generalized operational block diagram of a feedback
system, shown in Fig.1.8.
We have assumed in our presentation, that the dynamic behaviour of the plant can be represented (or
approximated with “sufficient” accuracy) by a linear time-invariant nth order system, which is described
by a strictly proper, minimal (controllable and observable) rational transfer function GP(s). We have also
assumed that any external disturbances that affect the plant, can be represented by a single, additive
signal w(t), with known dynamic properties (refer Fig.1.8). The dynamics in the feedback path (often
attributed to the sensor), was assumed to be characterized by the proper minimal transfer function H(s),
which produces a continuous measure of the potentially noisy output y(t).
We placed a restriction on the design of controllers: the controller can be represented by a linear time-
invariant system, whose single output (for the single-input single-output (SISO) systems) u(t) is produced
by the input r(t) – b(t) = ê(t). Therefore, its dynamic behaviour can be described by
U(s) = D(s)[R(s) – B(s)]
where D(s) is the proper minimal transfer function of the controller, whose degree defines the order of the
controller.
We have observed that in many cases involving the so-called classical control techniques, the transfer
function A(s) (corresponding to the reference-input elements (Fig.1.8)), is assumed to be equal to H(s).
This implies the more restrictive unity-feedback configuration depicted in Fig. 1.12. However, the choice
of A(s)H(s) (e.g., A(s) = 1) would imply non-unity-feedback structure; the design procedures for this
structure have been developed in our companion book [155]. In the vast majority of applications, the
unity-feedback configuration is preferred because the error (e(t) = r(t) – y(t) = yr(t) – y(t)) is explicitly
present—both to drive the controller, and to be zeroed via feedback.
In this part of the book, we intend to relax the restrictions we have so far imposed on the development
of general procedures for the design of controllers. We know that the output y(t) does not represent the
complete dynamic state of the plant at time t; it is the state vector x(t)=[x1(t), …, xn(t)]T which carries
complete knowledge on the dynamics at time t. In the output-feedback configurations of the form shown
in Fig. 1.8, only partial information on the dynamical state of the plant is fed back. We will relax this
restriction and allow the complete state x(t) to be fed back.
In the classical configuration of Fig. 1.8, the controller output u(t) is produced by one input:
[r(t) – b(t)]. We will relax this restriction also, and allow the controller u(t) to be a function of r(t), and
b(t) independently.
The control law will take the form
u(t) = kR r(t)–kx(t)
where kR (scalar) is the reference control gain, and k (1 ´ n vector) is the state feedback gain.
Design procedures for such a control law will require state-space formulation for the dynamic systems
in the feedback loop. We, therefore, begin our discussion with development of state variable models, and
their analysis (Chapters 5 and 6). This will be followed by development of design procedures (Chapters
7 and 8).
 Control System Analysis

5 Using State Variable

Methods

5.1 INTRODUCTION

In Part I of this book, we have seen that the root-locus method and the frequency-response methods are quite
powerful for the analysis and design of feedback control systems. The analysis and design are carried out
using transfer functions, together with a variety of graphical tools such as root-locus plots, Nyquist plots,
Bode plots, Nichols chart, etc. These techniques of the so called classical control theory have been greatly
enhanced by the availability, and low cost, of digital computers for system analysis and simulation. The
graphical tools can now be more easily used with computer graphics.
The classical design methods suffer from certain limitations, due to the fact that the transfer function
model is applicable only to linear time-invariant systems, and that, there too, it is generally restricted to
single-input, single-output (SISO) systems. This is because the classical design approach becomes highly
cumbersome for use in multi-input, multi-output (MIMO) systems. Another limitation of the transfer func-
tion technique is that it reveals only the system output for a given input and provides no information about
the internal behaviour of the system. There may be situations where the output of a system is stable and yet
some of the system elements may have a tendency to exceed their specified ratings. In addition to this, it may
sometimes be necessary, and advantageous, to provide a feedback proportional to the internal variables of a
system, rather than the output alone, for the purpose of stabilizing and improving the performance of a
system.
The limitations of classical methods, based on transfer function models, have led to the development of
state variable approach of analysis and design. It is a direct time-domain approach which provides a basis
for modern control theory. It is a powerful technique for the analysis and design of linear and nonlinear,
time-invariant or time-varying MIMO systems. The organization of the state variable approach is such that
it is easily amenable to solution through digital computers.
It will be incorrect to conclude from the foregoing discussion, that the state variable design methods can
completely replace the classical design methods. In fact the classical control theory, comprising a large body
of use-tested knowledge, is still going strong. State variable design methods prove their mettle in applica-
tions that are intractable by classical methods.
The state variable formulation contributes to the application areas of classical control theory in a differ-
ent way. To compute the response of G(s) to an input R(s), requires the expansion of {G(s)R(s)} into partial
fractions; which, in turn, requires computation of all the poles of {G(s)R(s)}, or all the roots of a polynomial.
 256 Digital Control and State Variable Methods
The roots of a polynomial are very sensitive to their coefficients (refer Review Example 3.3). Furthermore,
to develop a computer program to carry out partial fraction expansion is not simple. On the other hand, the
response of state variable equations is easy to program. Its computation does not require the computation of
roots or eigenvalues. Therefore, it is less sensitive to parameter variations. For these reasons, it is desirable
to compute the response of G(s) through state variable equations. State variable formulation is thus the most
efficient form of system representation—from the stand point of computer simulation. For this reason, many
Computer-Aided-Design (CAD) packages, handling both the classical and the modern tools of control
system design, use this notation. It is, therefore, helpful for the control engineer to be familiar with state
variable methods of system representation and analysis.
Part-II of this text presents an introduction to a range of topics which fall within the domain of state
variable analysis and design. Our approach is to build on, and complement, the classical methods of analysis
and design. State variable analysis and design methods use vector and matrix algebra and are, to some
extent, different from those based on transfer functions. For this reason, we have not integrated the state
variable approach with the frequency-domain approach based on transfer functions.
We have been mostly concerned with SISO systems in the text so far. In the remaining chapters also, our
emphasis will be on the control of SISO systems. However, many of the analysis and design methods based
on state variable concepts are applicable to both SISO and MIMO systems with almost equal convenience;
the only difference being the additional computational effort for MIMO systems, which is taken care of by
CAD packages. A specific reference to such results will be made at appropriate places in these chapters.

5.2 VECTORS AND MATRICES

This section is intended to be a concise summary of facts about vectors and matrices. Having them all at
hand will minimize the need to consult a book on matrix theory. It also serves to define the notation and
terminology which are, regrettably, not entirely standard.
No attempt has been made at proving every statement made in this section. The interested reader is urged
to consult a suitable book (for example [27, 28]) for details of proofs.

5.2.1 Matrices1
Basic definitions and algebraic operations associated with matrices are given below.
Matrix The matrix
LMa 11 a12 L a1m OP
L a2 m
A= M PP
a 21 a22
= [aij] (5.1)
MM M M M
PQ
Na n1 an 2 L anm
is a rectangular array of nm elements. It has n rows and m columns. aij denotes (i, j)th element, i.e., the
element located in ith row and jth column. A is said to be a rectangular matrix of order n ´ m.
When m = n, i.e., the number of columns is equal to that of rows, the matrix is said to be a square matrix
of order n.
A n ´ 1 matrix, i.e., a matrix having only one column is called a column matrix. A 1 ´ n matrix, i.e., a
matrix having only one row is called a row matrix.
Diagonal Matrix A diagonal matrix is a square matrix whose elements off the principal diagonal are all
zeros (aij = 0 for i ¹ j ). The following matrix is a diagonal matrix.
1
We will use upper case bold letters to represent matrices and lower case bold letters to represent vectors.
 Control System Analysis Using State Variable Methods 257

LMa 11 0 L 0 OP
0 a22 L 0
L = M
MM M M M P
P = diag [a 11 a12 L anm] (5.2)

N0 0 L a Q
nn
P
Unit (identity) Matrix A unit matrix I is a diagonal matrix whose diagonal elements are all equal to
unity (aii = 1, aij = 0 for i ¹ j).
LM1 0 L 0 OP
1 L 0
I = M
0
MM M M M
PP
N0 0 L 1
PQ
Whenever necessary, an n ´ n unit matrix will be denoted by In.
Null (zero) Matrix A null matrix 0 is a matrix whose elements are all equal to zero.
LM0 0 L 0 OP
0 L 0 P
0 =M
0
MM M M MP
N0 0 L 0Q
P
Whenever necessary, the dimensions of the null matrix will be indicated by two subscripts: 0nm.
Lower-triangular Matrix A lower-triangular matrix L has all its elements above the principal diagonal
equal to zero; lij = 0 if i < j for 1 £ i £ n and 1 £ j £ m.
LMl 11 0 L 0 OP
l22 L 0
L =M PP
l 21
MM M M M
PQ
Nl n1 ln 2 L lnm

Upper-triangular Matrix An upper-triangular matrix U has all its elements below the principal diagonal
equal to zero; uij = 0 if i > j for 1 £ i £ n and 1 £ j £ m.
LMu 11 u12 L u1m OP
0 u22 L u2 m
U= M
MM M M M
PP
N0 0 L unm
PQ
Matrix Transpose If the rows and columns of an n ´ m matrix A are interchanged, the resulting m ´ n
matrix, denoted as AT, is called the transpose of the matrix A. Namely, if A is given by Eqn. (5.1), then
LM a 11 a21 L an1 OP
L an 2 PP
A =M
T a 12 a22
MM M M M
PQ
Na 1m a2 m L anm
 258 Digital Control and State Variable Methods
Some properties of the matrix transpose are
(i) (AT)T = A (ii) (kA)T = kAT, where k is a scalar
(iii) (A + B)T = AT + BT (iv) (AB)T = BTAT
Symmetric Matrix If a square matrix A is equal to its transpose;
A = AT,
then the matrix A is called a symmetric matrix.
Skew-symmetric Matrix If a square matrix A is equal to the negative of its transpose;
A = –AT,
then the matrix A is called a skew-symmetric matrix.
Conjugate Matrix If the complex elements of a matrix A are replaced by their respective conjugates,
then the resulting matrix is called the conjugate of A.
Conjugate Transpose The conjugate transpose is the conjugate of the transpose of a matrix. Given a
matrix A, the conjugate transpose is denoted by A* , and is equal to conjugate of AT.
Hermitian Matrix If a square matrix A is equal to its conjugate transpose;
A = A* ,
then the matrix A is called a Hermitian matrix. For matrices whose elements are all real (real matrices),
symmetric and Hermitian mean the same thing.
Skew-Hermitian Matrix If a square matrix A is equal to the negative of its conjugate transpose;
A = – A* ,
then the matrix A is called a skew-Hermitian matrix. For real matrices, skew-symmetric and skew-Hermitian
mean the same thing.
Determinant of a Matrix Determinants are defined for square matrices only. The determinant of the
n ´ n matrix A, written as |A|, or det A, is a scalar-valued function of A. It is found through the use of minors
and cofactors.
The minor mij of the element aij is the determinant of a matrix of order (n – 1) ´ (n – 1), obtained from A
by removing the row and the column containing aij.
The cofactor cij of the element aij is defined by the equation
cij = (– 1)i + j mij
Determinants can be evaluated by the method of Laplace expansion. If A is an n ´ n matrix, any arbitrary
row k can be selected and |A| is then given by
n
|A| = å akj ckj
j =1
Similarly, Laplace expansion can be carried out with respect to any arbitrary column l, to obtain
n
|A| = å ail cil
i =1
Laplace expansion reduces the evaluation of an n ´ n determinant down to the evaluation of a string of
(n – 1) ´ (n – 1) determinants, namely, the cofactors.
Some properties of determinants are
(i) det AB = (det A)(det B) (ii) det AT = det A
n
(iii) det kA = k det A; A is n ´ n matrix and k is scalar
(iv) The determinant of any diagonal or triangular matrix is the product of its diagonal elements.
 Control System Analysis Using State Variable Methods 259
Singular Matrix A square matrix is called singular if the associated determinant is zero.
Nonsingular Matrix A square matrix is called nonsingular if the associated determinant is non-
zero.
Adjoint Matrix The adjoint matrix of a square matrix A is found by replacing each element aij of matrix
A, by its cofactor cij and then transposing.

LMc 11 c21 L cn1 OP

c c22 L cn 2
adj A = A+ =M
MM M
12
M M
= [cji] PP
Nc 1n c2 n L cnn
PQ
Note that
A(adj A) = (adj A)A = |A| I (5.3)
Matrix Inverse The inverse of a square matrix is written as A–1, and is defined by the relation
A–1A = AA–1 = I
From Eqn. (5.3) and the definition of the inverse matrix, we have
adj A
A–1 = (5.4)
A
Some properties of matrix inverse are
(i) (A–1)–1 = A (ii) (AT) –1 = (A– 1)T (iii) (AB)–1 = B–1A–1
1
(iv) det A–1 = (v) det P–1AP = det A
det A
(vi) Inverse of diagonal matrix given by Eqn. (5.2) is

LM1/ a
11 0 L 0 OP
L –1 =M
0 1/ a22 L 0
PP = diag
LM 1 1
L
1 OP
MM M M M
PQ Na
11 a22 a nn Q
N0 0 L 1/ ann

Rank of a Matrix The rank r(A) of a matrix A is the dimension of the largest array in A with a non-zero
determinant. Some properties of rank are
(i) r(AT ) = r (A)
(ii) The rank of a rectangular matrix cannot exceed the lesser of the number of rows or the number of
columns. A matrix whose rank is equal to the lesser of the number of rows and number of columns is
said to be of full rank.
r(A) £ min (n, m); A is n ´ m matrix
(iii) The rank of a product of two matrices cannot exceed the rank of the either:
r(AB) £ min [r(A), r(B)]
Trace of a Matrix The trace of a square matrix A is the sum of the elements on the principal diagonal.
tr A = å aii (5.5)
i
 260 Digital Control and State Variable Methods
Some properties of trace are
(i) tr AT = tr A (ii) tr (A + B) = tr A + tr B
(iii) tr AB = tr BA; tr AB ¹ (tr A)(tr B) (iv) tr P–1AP = tr A
Partitioned Matrix A matrix can be partitioned into submatrices or vectors. Broken lines are used to
show the partitioning when the elements of the submatrices are explicitly shown. For example
LMa 11 a12 a13 OP
A = Ma 21 a22 a23 PP
MNa 31 a32 a33 Q
The broken lines indicating the partitioning are sometimes omitted when the context makes it clear that
partitioned matrices are being considered. For example, the matrix A given above may be expressed as

A=
LMA 11 A12 OP
NA 21 A 22 Q
We will be frequently using the following forms of partitioning.
(i) Matrix A partitioned into its columns:
A = [ a1 a2 L am]
where
LM a OP
1i

a = M P = ith column in A
a2i
i
MM M PP
Na Q
ni

(ii) Matrix A partitioned into its rows:

LMa OP 1
a
A= M P
2
MM M PP
Na Q n
where
ai = [ai1 ai2 L aim ] = ith row in A
(iii) A block diagonal matrix is a square matrix that can be partitioned so that the non-zero elements are
contained only in square submatrices along the main diagonal,
LMA 1 0 L 0 OP
A2 L 0 PP
A= M
0
= diag [A1 A2 L Am ]
MM M M M
PQ
N0 0 L Am
For this case
(i) |A| = |A1| |A2| L |Am|
(ii) A– 1 = diag [A1– 1 A2– 1 L A–1m ], provided that A–1 exists.
 Control System Analysis Using State Variable Methods 261
5.2.2 Vectors
We will be mostly concerned with vectors and matrices that have real elements. We, therefore, restrict our
discussion to these cases only. An extension of the results to the situations where the vectors/matrices have
complex elements is quite straightforward.
Scalar (inner) Product of Vectors The scalar product of two n ´ 1 constant vectors x and y is defined as
< x, y > = xTy
LM y OP
1
y
x ]M P =x y
2
= [x1 x2 L + x2 y2 + L + xn yn
n
MM M PP 1 1

Ny Qn
= yTx
Vector Norm The concept of norm of a vector is a generalization of the idea of length. For the vector
LM x OP1
x
x= M P
2
MM M PP
Nx Q n
the Euclidean vector norm || x || is defined by
|| x || = (x12 + x22 + L + xn2)1/2 = (xTx)1/2 (5.6a)
In two or three dimensions, it is easy to see that this definition for the length of x satisfies the conditions
of Euclidean geometry. It is a generalization to n dimensions of the theorem of Pythagoras.
For any nonsingular matrix P, the vector
y = Px
has the Euclidean norm
|| y || = [(P x)T(P x)]1/2 = (xT PT P x)1/2
Letting Q = PT P, we write
|| y || = (xTQ x)1/2
or
|| x ||Q = (xTQ x)1/2 (5.6b)
We call || x ||Q the norm of x with respect to Q. It is, in fact, a generalization of the norm defined in (5.6a)
in that it is a measure of the size of x ‘weighted’ by the matrix Q.
Matrix Norm The norm of a matrix is a measure of the ‘size’ of the matrix (not its dimension). For
the matrix
LMa 11 a12 L a1n OP
a a22 L a2 n
A= M
21
MM M M M
PP
Nan1 an 2 L ann
PQ
 262 Digital Control and State Variable Methods
the Euclidean matrix norm || A || is defined by
L
|| A || = M
n
aij2
OP 1/ 2

å = [tr (ATA]1/2 (5.6c)

MN i, j = 1 PQ
We can also describe the size of A by
|| Ax ||
|| A || = max
x || x ||
i.e., the largest value of the ratio of the length || Ax || to the length || x ||.
Using the Euclidean norm for vectors, we obtain
1/ 2

|| A || = max
e x A Ax j
T T
= max
FG x A Ax IJ
T T 1/ 2

x
ex x j T 1/ 2 x H xx K T

The maximum value of the ratio in this expression can be determined in terms of the eigenvalues2 of the
matrix ATA. The real symmetric matrix ATA has all real and non-negative eigenvalues and the maximum
value of the ratio (xTATAx)/(xTx) is equal to the maximum eigenvalue of ATA (for proof, refer [107]).
Therefore,
|| A || = (Maximum eigenvalue of ATA)1/2 (5.6d)
3
This definition of the matrix norm is known as the spectral norm of A.
The square roots of the eigenvalues of ATA are called the singular values of A. The spectral norm of A is
equal to its largest singular value.
Singular values of a matrix are useful in numerical analysis. The ratio of the largest to the smallest singu-
lar values of A, called the condition number of A, is a measure of how close the matrix A comes to being
singular. The matrix A is therefore ‘ill-conditioned’ if its condition number is large.
Orthogonal Vectors Any two vectors which have a zero scalar product are said to be orthogonal vectors.
Two n ´ 1 vectors x and y are orthogonal if
xTy = 0
A set of vectors is said to be orthogonal if, and only if, every two vectors from the set are orthogonal:
xTy = 0 for all x ¹ y in the set.
Unit Vector A unit vector x$ is, by definition, a vector whose norm is unity; || x$ || = 1. Any non-zero vector
x can be normalized to form unit vector.
x
x$ =
|| x ||
Orthonormal Vectors A set of vectors is said to be orthonormal if, and only if, the set is orthogonal and
each vector in this orthogonal set is a unit vector.
Orthogonal Matrix Suppose that {x1, x2, K , xn} is an orthogonal set:
x iT xi = 1, for all i
and

2
The roots of the equation
|lI – A| = 0
are called the eigenvalues of matrix A. Detailed discussion will appear in Section 5.6.
3
Refer [105] for other valid vector and matrix norms.
 Control System Analysis Using State Variable Methods 263

x iT xj = 0 for all i and j with i ¹ j.

If we form the n ´ n matrix
P = [ x1 x2 L xn],
it follows from partitioned multiplication that
PTP = I
That is,
PT = P–1
Such a matrix P is called an orthogonal matrix.
Linearly Dependent Vectors Consider a set of m vectors {x1, x2, K , xm}, each of which has n compo-
nents. If there exists a set of m scalars ai , at least one of which is not zero, which satisfies
a1x1 + a2x2 + L + amxm = 0,
then the set of vectors {xi} is said to be linearly dependent.
Linearly Independent Vectors Any set of vectors {xi} which is not linearly dependent is said to be
linearly independent. That is, if
a1x1 + a2x2 + L + amxm = 0
implies that each ai = 0, then {xi} are linearly independent vectors.
Test for Linear Independence Consider the set of m vectors {xi}, each of which has n components, with
m ¹ –n. Assume that this set is linearly dependent so that
a1x1 + a2x2 + L + amxm = 0
with at least one non-zero ai.
Premultiplying both sides of this equation by x iT, gives a set of m simultaneous equations:
a1 x iT x1 + a2 x iT x2 + L + am x iT xm = 0; i = 1, 2, K , m
These equations can be written in the matrix form as
LM OP L
x 1T x 1 x 1T x 2 L x 1T x m a 1 OP 0LM OP
MM T T
PP MM
T
x 2 x1 x 2 x 2 L x 2 x m a 2 PP=
0
MM PP (5.7a)
MM M M
PP MMN
M M
PQ M
MN PQ
or
N Q
x Tm x 1 x Tm x 2 L x Tm x m a m 0

Ga = 0
If the m ´ m matrix G has a non-zero determinant, then G– 1 exists, and
a = G– 10 = 0 (5.7b)
This contradicts the assumption of at least one non-zero ai. The matrix G is called the Grammian matrix.
A necessary and sufficient condition for the set {xi} to be linearly dependent is that |G| = 0.
Linear Independence and Rank The column rank of a matrix A is equal to maximum number of linearly
independent columns in A.
The maximum number of linearly independent columns of a matrix is equal to the maximum number of
linearly independent rows. Therefore, the column rank of A = the row rank of A = r (A), which is, in turn,
equal to the order of the largest square array in A whose determinant does not vanish.
Vector Functions In Section 5.8, we will require a test for the linear independence of the rows of a matrix
whose elements are functions of time.
 264 Digital Control and State Variable Methods

Consider a matrix
LM f at ff 12 t
11 af
L f 1m t f1 t a f OP LM a f OP
F(t) = M M M M = M P MM P
MN f at f
n1 af
f n 2 t L f nm t fn t a f PQ N a f PQ
fi(t); i = 1, K , n are the n row vectors of matrix F; each vector has m components.
The scalar product of two 1 ´ m vector functions fi(t) and fj(t) on [t0, t1] is by definition,
t1
< fi , fj > = z
t0
fi(t)fjT(t) dt

The set of n row-vector functions {f1(t), K , fn(t)} are linearly dependent if there exists a set of n scalars
ai, at least one of which is not zero, which satisfies
a1f1(t) + a2f2(t) + L + anfn(t) = 01 ´ m
LM f (t) OP
11 LM f (t) OP
21 LM f (t) OP
n1 LM0OP
or a1 MM M PP + a 2 MM M PP + L + a n MM M PP = MM0M PP
N f (t ) Q
1m N f (t ) Q
2m N f (t ) Q
nm NQ
Equivalently, the n rows fi(t) are linearly dependent if
a T F(t) = 0 (5.8a)
for some
a1 LM OP
a= M ¹ 0 MM PP
an N Q
The Grammian matrix of functions fi(t), i = 1, K , n; where fi(t) is the ith row of the matrix F(t), is given
by (refer Eqns (5.7)),
t1
W(t0, t1) = z
t0
F(t)FT(t) dt (5.8b)

The functions f1(t), K , fn(t), which are the rows of the matrix F(t), are linearly dependent on [t0, t1] if, and
only if, the n ´ n constant matrix W(t0, t1) is singular.

5.2.3 Quadratic Forms and Definiteness of Matrices

An expression such as
n n
V(x1, x2, K , xn) = å å qij xi xj
i =1 j =1
involving terms of second degree in xi and xj, is known as the quadratic form of n variables. Such scalar-
valued functions are extensively used in stability analysis and modern control design.
In practice, one is usually concerned with quadratic forms V(x1, x2, K , xn) that assume only real values.
When xi, xj, and qij are all real, the value of V is real, and the quadratic form can be expressed in the vector-
matrix notation as
q11 q12 L q1n x1 LM OP LM OP
q21 q22 L q2 n x 2
V(x) = [x1 x2 L xn]
M M M M
MM PP MM PP
qn1 qn 2 L qnn x n MN PQ MN PQ
or V(x) = xTQx
 Control System Analysis Using State Variable Methods 265
Any real square matrix Q may be written as the sum of a symmetric matrix Qs and a skew-symmetric
matrix Qsk, as shown below.
Let
Q = Qs + Qsk
Taking transpose of both sides,
QT = Q Ts + Q Tsk = Qs – Qsk
Solving for Qs and Qsk, we obtain
Q + QT Q - QT
Qs = ; Qsk =
2 2
For a real matrix Q, the quadratic function V(x) is therefore given by
V(x) = xTQx = xT(Qs + Qsk)x = x T Q s x + 12 x T Qx - 12 x T Q Tx
Since xTQx = (xTQx)T = xTQTx, we have
V(x) = xTQsx
Thus, in quadratic function V(x), only the symmetric portion of Q is of importance. We shall, therefore,
tacitly assume that Q is symmetric.
It may be noted that real vector x and real matrix Q do not constitute necessary requirements for V(x) to
be real. V(x) can be real when Q and x are possibly complex; it can easily be established that for a Hermitian
matrix Q,
V(x) = x*Qx
has real values.
Our discussion will be restricted to real symmetric matrices Q.
If for all x ¹ 0,
(i) V(x) = xTQx ³ 0,
then V(x) is called a positive semidefinite function and Q is called a positive semidefinite matrix;
(ii) V(x) = xTQx > 0,
then V(x) is called a positive definite function and Q is called a positive definite matrix;
(iii) V(x) = xTQx £ 0,
then V(x) is called a negative semidefinite function and Q is called a negative semidefinite matrix;
(iv) V(x) = xTQx < 0,
then V(x) is called a negative definite function and Q is called a negative definite matrix.
Tests for definiteness (i) Eigenvalues of Q and the nature of quadratic form
A real symmetric matrix Q has all real eigenvalues, and the signs of the eigenvalues of Q determine the
nature of the quadratic form xTQx, as summarized in Table 5.1.
Table 5.1
Eigenvalues of Q Nature of quadratic form
xTQx
All li > 0 Positive definite
All li ³ 0 Positive semidefinite
All li < 0 Negative definite
All li £ 0 Negative semidefinite
Some li ³ 0, some lj £ 0 Indefinite
 266 Digital Control and State Variable Methods

(ii) Sylvester’s Criterion

The Sylvester’s criterion states that the necessary and sufficient conditions for
LM
q11 q12 L q1n OP L O
x1

V(x) = xTQx = [x1 x2 L xn]

q21 q22 L q2 n x 2
MM
M M M
PP MMM PPP
M
MNq q L q PQ MN x PQ
n1 n2 nn n
to be positive definite are that all the successive principal minors of Q be positive, i.e.,
q11 q12 q13
q11 q12
q11 > 0; > 0; q 21 q 22 q 23 > 0; L ; | Q | > 0 (5.9)
q 21 q 22
q 31 q 32 q33
The necessary and sufficient conditions for V(x) to be positive semidefinite are that Q is singular and all
the other principal minors of Q are non-negative.
V(x) is negative definite if [–V(x)] is positive definite. Similarly, V(x) is negative semidefinite if [–V(x)]
is positive semidefinite.

5.3 STATE VARIABLE REPRESENTATION

We will be mostly concerned with SISO system configurations of the type shown in the block diagram of
Fig. 5.1. The plant in the figure is a physical process, characterized by the state variables x1, x2, K , xn, the
output variable y and the input variable u.

r + u y
kR Plant
–

x1
k1

x2
k2

xn
kn

Fig. 5.1 A state-feedback control configuration

5.3.1 State Variable Concepts
The modelling process of linear systems involves setting up a chain of cause-effect relationships, beginning
from the input variable and ending at the output variable. This cause-effect chain includes a number of
 Control System Analysis Using State Variable Methods 267
internal variables. These variables are eliminated, both in the differential equation model and in the transfer
function model, to obtain the final relationship between the input and the output. Analysis of systems with
the input-output model will not give any information about the behaviour of the internal variables for differ-
ent operating conditions. For a better understanding of the system behaviour, its mathematical model should
include the internal variables also. The state variable techniques of system representation and analysis, make
the internal variables an integral part of the system model, and thus provide more complete information
about the system behaviour. In order to appreciate how the internal variables are included in the system
representation, let us examine the modelling process by means of a simple example.
Consider the network shown in Fig. 5.2a. The set of voltages and currents associated with all the branches
of the network at any time t, represents the status of the network at that time. Application of Kirchhoff’s
current law at nodes 1 and 2 of the network gives the following equations:
de1 de u - e1
+ 2 2 =
dt dt 2
de3 de2
2 = 2
dt dt
Application of Kirchhoff’s voltage law to the loop consisting of the three capacitors, yields
e1(t) – e2(t) = e3(t)
Displacement
y(t)
+ y – 1 + e2 – 2 Velocity v (t)
2 2F
K
+ + +
u 1F e1 2F e3
– – – Force
M
F(t)

Zero
friction B

(a) (b)
Fig. 5.2

All other voltage and current variables associated with the network are related to e1, e2, e3 and input u,
through linear algebraic equations. This means that their values (at all instants of time) can be obtained from
the knowledge of the network variables e1, e2, e3 and the input variable u, merely by linear combinations. In
other words, the reduced set {e1(t), e2(t), e3(t)} of network variables with the input variable u(t), completely
represents the status of the network at time t.
For the purpose of finding a mathematical model to represent a system, we will naturally choose a mini-
mal set of variables that describes the status of the system. Such a set would be obtained when none of the
selected variables is related to other variables and the input, through linear algebraic equations. A little
consideration shows that there is redundancy in the set{e1(t), e2(t), e3(t)} for the network of Fig. 5.2a; a set
of two variables, say{e1(t), e2(t)}, with the input u(t) represents the network completely at time t.
Manipulation of the network equations obtained earlier, gives
de1( t ) 1 1
=- 4 e1 t
( ) + 4 u(t )
dt
 268 Digital Control and State Variable Methods

de2( t ) 1 1
=- 8 e1 t
( ) + 8 u( t )
dt
This set of equations constitutes a mathematical model for the system. It is a set of two first-order differ-
ential equations. Its complete solution for any given u(t) applied at t = t0, will require a knowledge of the
value of the selected variables {e1, e2} at t = t0. To put it differently, we can say that if the values of {e1, e2}
at t = t0 are known, then the values of these variables at any time t > t0, in response to a given input u( t0 ,t ) , can
be obtained by the solution of the two first-order differential equations. A set of system variables having this
property, is called a set of state variables. The set of values of these variables at any time t is called the state
of the system at time t. The set of first-order differential equations relating the first derivative of the state
variables with the variables themselves and the input, is called a set of state equations. It is also to be noted
that the number of state variables needed to form a correct and complete model of the system is equal to the
order of the system.
An important point regarding the concept of state of a system is that the choice of state variables is not
unique. In the network of Fig. 5.2a, instead of voltages {e1, e2}, the voltages {e2, e3} or {e3, e1} may be taken
as state variables to define the state of the system. In fact, any set of variables x1(t) and x2(t), given by
LM OP L
x1 (t ) p11 p12 e1 (t )OP LM OP
N Q MN
x 2 (t )
=
p21 p22 e2 (t ) QN Q
where pij are constants such that the matrix

P=
LM
p11 p12 OP
N
p21 p22 Q
is nonsingular, is qualified to describe the state of the system of Fig. 5.2a because we can express the
capacitor voltages in terms of the selected variables x1(t) and x2(t). This brings out additional facts that there
are infinitely many choices of state variables for any given system, and that the selected state variables may
not correspond to physically measurable quantities of the system. Later in this chapter, we will see that all
the choices of state variables are not equally convenient. Usually, state variables are chosen so that they
correspond to physically measurable quantities, or lead to particularly simplified calculations.
For a particular goal of study of a given physical system, we may not be interested in the total information
about the system at a particular time. We may be interested only in a part of the total information. This is
called the output of the system, which can be obtained algebraically from the information of the system state
and the input. The output is, by definition, a physical attribute of the system and is measurable. For example,
in the electric network of Fig. 5.2a, the information of interest may be the voltage across the resistor. The
output
y(t) = – e1(t) + u(t)
As another example of the state of a system, consider the mechanical network shown in Fig. 5.2b. The
force F(t) is the input variable. Defining the displacement y(t) of the mass as the output variable, we obtain
the following input-output model for the system:
d 2 y( t ) dy(t )
M 2 + B + Ky(t) = F(t)
dt dt
An alternative form of the input-output model is the transfer function model:
Y (s ) 1
=
F ( s) Ms 2 + Bs + K
The set of forces, velocities, and displacements, associated with all the elements of the mechanical net-
work at any time t, represents the status of the network at that time. A little consideration shows that values
 Control System Analysis Using State Variable Methods 269
of all the system variables (at all instants of time) can be obtained from the knowledge of the system vari-
ables y(t) and v(t), and the input variable F(t), merely by linear combinations. The dynamics of y(t) and v(t)
are given by the following first-order differential equations:
dy (t )
= v(t)
dt
dv(t ) K B 1
=– y(t) – v(t) + F(t)
dt M M M
The variables {y(t), v(t)} are, therefore, the state variables of the system of Fig. 5.2b, and the two
first order differential equations given above, are the state equations of the system. Using standard symbols
for state variables and input variable, we can write the state equations as
x& 1 = x2
K B 1
x& 2 = – x1 – x2 + u
M M M
D D D
x1(t) = y(t); x2(t) = v(t); u(t) = F(t)
Defining y(t) as the output variable, the output equation becomes
y = x1
We can now appreciate the following definitions.
State The state of a dynamic system is the smallest set of variables (called state variables) such that
the knowledge of these variables at t = t0, together with the knowledge of the input for t ³ t0, completely
determines the behaviour of the system for any time t ³ t0.
State Vector If n state variables x1, x2, K , xn, are needed to completely describe the behaviour of a given
system, then these n state variables can be considered the n components of a vector x. Such a vector is called
a state vector.
State Space The n-dimensional space whose coordinate axes consist of the x1-axis, x2-axis, K , xn-axis, is
called a state space.
State Trajectory At any time t0, the state vector (and hence the state of the system) defines a point in the
state space. As time progresses and the system state changes, a set of points will be defined. This set of
points, the locus of the tip of the state vector as time progresses, is called the state trajectory of the system.
State space and state trajectory in two-dimensional cases are referred to as the phase plane and phase
trajectory, respectively.

5.3.2 State Variable Modelling

We know through our modelling experience that the application of physical laws to mechanical, electrical,
thermal, liquid-level, and other physical processes results in a set of first-order and second-order differential
equations.4 Linear time-invariant differential equations can be rearranged in the following form:

4
Chapter 2 of reference [155]
 270 Digital Control and State Variable Methods

x& 1(t) = a11 x1(t) + a12 x2(t) + L + a1n xn(t) +b1 u(t)
x& 2(t) = a21 x1(t) + a22 x2(t) + L + a2n xn(t) + b2 u(t) (5.10a)
M
x& n(t) = an1 x1(t) + an2 x2(t) + L + ann xn(t) + bnu(t)
where the coefficients aij and bi are constants. These n first-order differential equations are called state
equations of the system.
Integration of Eqn. (5.10a) gives
t
xi(t) = xi(t0) + z
t0
[ai1 x1(t) + L + ain xn(t) + bi u(t)]dt; i = 1, 2, K , n

Thus, the n state variables and hence the state of the system can be determined uniquely at any
t > t0, if each state variable is known at t = t0, and the control force u(t) is known throughout the interval t0 to t.
The output y(t) at any time t will be a function of x1(t), x2(t), K, xn(t). However, in some cases, the output
may also depend upon the instantaneous value of the input u(t). For linear systems, the output is a linear
combination of the state variables and the input:
y(t) = c1 x1(t) + c2 x2(t) + L + cn xn(t) + d u(t) (5.10b)
where ci and d are constants. The algebraic equation (5.10b) is called output equation of the system.
Since every real-world system has some nonlinearity, a mathematical model of the form (5.10) is an
approximation to reality. Many real-world nonlinearities involve a ‘smooth’ curvelinear relation between
independent and dependent variables. Nonlinear functions fi(×) and g(×) of the form
x& i(t) = fi (x1(t), x2(t), K , xn(t), u(t)); xi(t0) =D xi0 (5.11a)
y(t) = g(x1(t), x2(t), K , xn(t), u(t)) (5.11b)
may be linearized about a selected operating point using the multivariable form of the Taylor series:

f(x1, x2, x3, L) = f(x10, x20, K) +

LM ¶ f OP (x – x LM ¶ f OP (x – x +L
10) + 20 ) (5.11c)
MN¶ x1 x10 , x20, L PQ 1
MN¶ x
2 x10 , x20 , L PQ 2

One of the advantages of state variable formulation is that an extremely compact vector-matrix notation
can be used for the mathematical model. Using the laws of matrix algebra, it becomes much less cumber-
some to manipulate the equations.
In the vector-matrix notation, we may write Eqns (5.10) as
LM x& (t) OP LMa
1 11 a12 L a1n OP LM x (t) OP L b O LM x bt g OP LM x OP
1 1 1 0
0
1

MM x& (t) PP = MMaM

2 21 a22 L a2 n
M M
PP MM x (t) PP + MMMbM PPP u(t); MM x bMt g PP = MM x PP
2 2 2 0 D
0
2
(5.12a)
MN x& (Mt) PQ MNa PQ MN x (Mt) PQ MNb PQ MMN x bt g PPQ MM xM PP
n n1 an 2 L ann n n n 0 N Q 0
n

LM x (t)OP
1
x (t )
Lc ] M
MM M PPP + d u(t)
2
y(t) = [c1 c2 n (5.12b)

N x ( t )Q
n

In compact notation, Eqns (5.12) may be expressed as

x& (t) = Ax(t) + bu(t); x(t0) =D x0 : State equation (5.13a)
y(t) = cx(t) + du(t) : Output equation (5.13b)
 Control System Analysis Using State Variable Methods 271
where
x(t) = n ´ 1 state vector of nth-order dynamic system
u(t) = system input
y(t) = defined output
A = n ´ n matrix
b = n ´ 1 column matrix
c = 1 ´ n row matrix
d = scalar, representing direct coupling between input and output (direct coupling is rare in control
systems, i.e., usually d = 0)

Example 5.1 Two very usual applications of dc motors are in speed and position control systems.

Figure 5.3 gives the basic block diagram of a speed control system. A separately excited dc motor drives
the load. A dc tachogenerator is attached to the motor shaft; speed signal is fedback and the error signal is
used to control the armature voltage of the motor.
Reference signal corresponding Controlled output (speed)
to desired speed

er + u dc motor
Controller +
load
–

Error signal Armature voltage

Tachogenerator

Feedback signal corresponding

to actual speed
Fig. 5.3 Basic block diagram of a closed-loop speed control system
In the following, we derive the plant model for the speed control system. A separately excited dc motor
with armature voltage control, is shown in Fig. 5.4.
ij (const)

La Ra

ia

eb J, B
u

Fig. 5.4 Model of a separately excited dc motor

 272 Digital Control and State Variable Methods
The voltage loop equation is
dia(t )
u(t) = La + Ra ia(t) + eb(t) (5.14a)
dt
where
La = inductance of armature winding (henrys)
Ra = resistance of armature winding (ohms)
ia = armature current (amperes)
eb = back emf (volts)
u = applied armature voltage (volts)
The torque balance equation is
dw ( t )
TM(t) = J + Bw(t) (5.14b)
dt
where
TM = torque developed by the motor (newton-m)
J = equivalent moment of inertia of motor and load referred to motor shaft (kg-m2)
B = equivalent viscous friction coefficient of motor and load referred to motor shaft ((newton-m)/
(rad/sec))
w = angular velocity of motor shaft (rad/sec)
In servo applications, the dc motors are generally used in the linear range of the magnetization curve.
Therefore, the air gap flux f is proportional to the field current. For the armature controlled motor, the field
current if is held constant. Therefore, the torque TM developed by the motor, which is proportional to the
product of the armature current and the air gap flux, can be expressed as
TM(t) = KT ia(t) (5.14c)
where
KT = motor torque constant ((newton-m/amp))
The counter electromotive force eb, which is proportional to f and w, can be expressed as
eb(t) = Kb w(t) (5.14d)
where
Kb = back emf constant5 (volts/(rad/sec))
Equations (5.14) can be reorganized as
dia( t ) R K 1
= – a ia(t) – b w(t) + u(t) (5.15)
dt La La La
dw ( t ) KT B
= ia(t) – w(t)
dt J J
x1(t) = w (t), and x2(t) = ia(t) is the obvious choice for state variables. The output variable is y(t) = w(t).
The plant model of the speed control system, organized into the vector-matrix notation, is given below:

a fOP LM
B KT OP a f L O
LMx&1 t
=
-
J
MM J x1 t
PP LMN a fOPQ MM PP
+
0
1 u(t)
N a fQ
x& 2 t K
- b - a
R x2 t La N Q
LaMN La PQ
y(t) = x1(t)

5
In MKS untis, Kb = KT; Section 3.5 of reference [155].
 Control System Analysis Using State Variable Methods 273
Let us assign numerical values to the system parameters.
For the parameters6
Ra= 1 ohm, La = 0.1 H, J = 0.1 kg-m2, B = 0.1 (newton-m)/(rad/sec), Kb = KT = 0.1, (5.16)
the plant model becomes
x& (t) = Ax(t) + bu(t)
(5.17)
y(t) = cx(t)

where A=
LM-1 OP ; b = LM 0OP; c = [1
1
0]
N -1 -10Q N10Q
Example 5.2 Figure 5.5 gives the basic block diagram of a position control system. The controlled vari-
able is now the angular position q (t) of the motor shaft:

dq ( t )
= w (t) (5.18)
dt
Controlled output (position)
Reference signal corresponding
to desired position

er + dc motor
u q
Controller +
load
–
Error signal Armature voltage

Position sensor

Feedback signal corresponding

to actual position
Fig. 5.5 Basic block diagram of a closed-loop position control system
We make the following choice for state and output variables.
x1(t) = q (t), x2(t) = w (t), x3(t) = i a(t), y(t) = q (t)
For this choice, we obtain the following plant model from Eqns (5.15) and (5.18).

LMx&1 (t )
0
OP 1
B
LM 0
KT x1 t 0 OP L a f O L O
x& 2 (t ) =
MM
0 -
PP J MMJ x 2 t + 0 u(t)
1
PP MM a f PP MM PP
N
x&3 (t ) Q
0 -
Kb
La
-
Ra
MN
La
x3 t L a
PQ MN a f PQ MMN PPQ
y(t) = x1(t)

6
These parameters have been chosen for computational convenience
 274 Digital Control and State Variable Methods
For the system parameters given by (5.16), the plant model for position control system becomes
x& (t) = Ax(t) + bu(t)
(5.19)
y(t) = cx(t)
where
0LM 1 0 OP 0 LM OP
A = 0 -1
MM PP
1 ; b = 0 ; c = [1
MM PP 0 0]
N
0 -1 -10 10 Q N Q
In Examples 5.1 and 5.2 discussed above, the selected state variables are the physical quantities of the
systems which can be measured.
We will see in Chapter 7 that in a physical system, in addition to output, other state variables could be
utilized for the purpose of feedback. The implementation of design with state variable feedback becomes
straightforward if the state variables are available for feedback. The choice of physical variables of a system
as state variables therefore helps in the implementation of design. Another advantage of selecting physical
variables for state variable formulation is that the solution of state equation gives time variation of variables
which have direct relevance to the physical system.

5.3.3 Transformation of State Variables

It frequently happens that the state variables used in the original formulation of the dynamics of a system are
not as convenient as another set of state variables. Instead of having to reformulate the system dynamics, it
is possible to transform the set {A, b, c, d} of the original formulation (5.13), to a new set { A , b , c , d }. The
change of variables is represented by a linear transformation
x = Px (5.20a)
where x is a state vector in the new formulation, and x is the state vector in the original formulation. It is
assumed that the transformation matrix P is a nonsingular n ´ n matrix, so that we can always write
x = P–1x (5.20b)
We assume, moreover, that P is a constant matrix.
The original dynamics are expressed by
D
x& (t) = Ax(t) + bu(t); x(t0) = x0 (5.21a)
and the output by
y(t) = cx(t) + du(t) (5.21b)
Substitution of x as given by Eqn. (5.20a) into these equations, gives
P x& (t) = AP x (t) + bu(t)
y(t) = cP x (t) + du(t)
or
x& (t) = A x (t) + b u(t); x (t0) = P–1x(t0) (5.22a)
y(t) = c x (t) + d u(t) (5.22b)
with
A = P–1AP, b = P–1b, c = cP, d = d
In the next section, we will prove that both the linear systems (5.21) and (5.22) have identical output
responses for the same input. The linear system (5.22) is said to be equivalent to the linear system (5.21), and
P is called an equivalence or similarity transformation.
 Control System Analysis Using State Variable Methods 275
It is obvious that there exist an infinite number of equivalent systems since the transformation matrix P
can be arbitrarily chosen. Some transformations have been extensively used for the purposes of analysis
and design. Five of such special (canonical) transformations will be used in the present and the next two
chapters.

Example 5.3 Example 5.1 revisited

For the system of Fig. 5.4, we have taken angular velocity w(t) and armature current ia(t) as state variables:

x=
LM OP LM OP
x1
=
w
N Q NQ
x2 ia
We now define new state variables as
x 1 = w, x 2 = – w + ia
or
LM OP LM
x = 1 =
x x1
=
OP LM
1 0 x1OP LM OP
N Q N
x2 - x1 + x 2 Q N
-1 1 x 2QN Q
We can express velocity x1(t) and armature current x2(t) in terms of the variables x 1(t) and x 2(t):
x = Px (5.23)
with

P=
LM OP
1 0
N Q
1 1
Using Eqns (5.22) and (5.17), we obtain the following state variable model for the system of Fig. 5.4, in
terms of the transformed state vector x (t):
x& (t) = Ax (t) + b u(t)
(5.24)
y(t) = c x (t)
where

A = P–1A P =
LM 1 0OP LM-1 1OP LM1 0OP = LM 0 1 OP
N -1 1Q N-1 -10 Q N1 1Q N -11 -11 Q
b = P b= M
–1 L 1 0OP LM 0OP = LM 0OP
N -1 1Q N10Q N10Q
c = cP = [1 0] M
L1 0OP = [1 0]
N1 1Q
x 1(t0) = x1(t 0); x 2(t0) = – x1(t0) + x2(t0)
Equations (5.24) give an alternative state variable model of the system previously represented by Eqns
(5.17). x (t) and x(t) both qualify to be state vectors of the given system (the two vectors individually char-
acterize the system completely at time t), and the output y(t), as we shall see shortly, is uniquely determined
from either of the models (5.17) and (5.24). State variable model (5.24) is thus equivalent to the model
(5.17), and the matrix P given by Eqn. (5.23) is an equivalence or similarity transformation.
The state variable model given by Eqns (5.24) is in a canonical (special) form. In Chapter 7, we will use
this form of model for pole-placement design by state feedback.

5.3.4 State Diagrams

An important advantage of state variable formulation is that it is a straightforward method to obtain a
simulation diagram for the state equations. This is extremely useful if we wish to use computer simulation
 276 Digital Control and State Variable Methods
methods to study dynamic systems. In the following, we give an example of analog simulation diagram.
Examples of digital simulation will appear in Chapter 6.
For brevity, we consider a second-order system:
x& 1(t) = a11 x1(t) + a12 x2(t) + b1u(t)
x& 2(t) = a21 x1(t) + a22 x2(t) + b2u(t) (5.25)
y(t) = c1x1(t) + c2x2(t)
It is evident that if we knew x& 1 and x& 2, we could obtain x1 and x2 by simple integration. Hence x& 1 and x& 2
should be the inputs to two integrators. The corresponding integrator outputs are x1 and x2. This leaves only
the problem of obtaining x& 1 and x& 2 for use as inputs to the integrators. In fact, this is already specified by
state equations. The completed state diagram is shown in Fig. 5.6. This diagram is essentially an analog-
computer program for the given system.
x10

+
+ x1
b1 ò +
c1
+ +

a11

a21 +
u y

a12 +

a22

+
+ +
b2 ò c2
+ x2
+

x20
Fig. 5.6 State diagram for the system (5.25)

5.4 CONVERSION OF STATE VARIABLE MODELS TO TRANSFER FUNCTIONS

We shall derive the transfer function of a SISO system from the Laplace-transformed version of the state and
output equations. Refer Section 5.2 for the vector and matrix operations used in the derivation.
Consider the state variable model (Eqns (5.13)):
x& (t) = Ax(t) + bu(t); x(t0) =D x0 (5.26)
y(t) = cx(t) + du(t)
 Control System Analysis Using State Variable Methods 277
Taking the Laplace transform of Eqns (5.26), we obtain
sX(s) – x0 = AX(s) + bU(s)
Y(s) = cX(s) + dU(s)
where
D D D
X(s) = L [x(t)]; U(s) = L [u(t)]; Y(s) = L [y(t)]
Manipulation of these equations, gives
(sI – A)X(s) = x0 + bU(s); I is n ´ n identity matrix
or X(s) = (sI – A)–1x0 + (sI – A)–1bU(s) (5.27a)
Y(s) = c(sI – A)–1x0 + [c(sI – A)–1b + d]U(s) (5.27b)
Equations (5.27) are algebraic equations. If x0 and U(s) are known, X(s) and Y(s) can be computed from
these equations.
In the case of a zero initial state (i.e., x0 = 0), the input-output behaviour of the system (5.26) is deter-
mined entirely by the transfer function
Y( s )
= G(s) = c(sI – A)–1b + d (5.28)
U( s)
We can express the inverse of the matrix (sI – A) as
(s I − A) +
(sI – A)–1 = (5.29)
| s I − A|
where
|sI – A| = determinant of the matrix (sI – A)
(sI – A)+= adjoint of the matrix (sI – A)
Using Eqn. (5.29), the transfer function G(s) given by Eqn. (5.28) can be written as
c ( s I − A) + b
G(s) = +d (5.30)
| s I − A|
For a general nth-order matrix
LM
a11 a12 L a1n OP
a21 a22 L a2 n
A= MM
M M M
, PP
MNa
n1 an 2 L ann
PQ
the matrix (sI – A) has the following appearance:
LMs - a 11 - a12 L - a1n OP
-a s - a22 L - a2 n
(sI – A) = M PP
21
MM M M M
PQ
N- a
n1 - an 2 L s - ann
If we imagine calculating det(sI – A), we see that one of the terms will be the product of diagonal
elements of (sI – A):
(s – a11)(s – a22) L (s – ann) = sn + a¢1 sn – 1 + L + a¢n,
a polynomial of degree n with the leading coefficient of unity. There will be other terms coming from the off-
diagonal elements of (sI – A), but none will have a degree as high as n. Thus | sI – A| will be of the following
form:
| sI – A| = D(s) = sn + a 1 sn – 1 + L + a n (5.31)
where ai are constant scalars.
 278 Digital Control and State Variable Methods
This is known as the characteristic polynomial of the matrix A. It plays a vital role in the dynamic
behaviour of the system. The roots of this polynomial are called the characteristic roots or eigenvalues of
matrix A. These roots, as we shall see in Section 5.7, determine the essential features of the unforced
dynamic behaviour of the system (5.26).
The adjoint of an n ´ n matrix is itself an n ´ n matrix, whose elements are the cofactors of the original
matrix. Each cofactor is obtained by computing the determinant of the matrix that remains when a row and a
column of the original matrix are deleted. It thus follows that each element in (sI – A)+ is a polynomial in s
of maximum degree (n – 1). Adjoint of (sI – A) can therefore be expressed as
(sI – A)+ = Q1 sn – 1 + Q2 sn – 2 + L + Qn – 1 s + Qn (5.32)
where Qi are constant n ´ n matrices.
We can express transfer function G(s) given by Eqn. (5.30) in the following form:

c[ Q 1 s n - 1 + Q 2 s n - 2 + L + Q n - 1 s + Q n ] b
G(s) = +d (5.33)
sn + a1 sn -1 + L + a n -1 s + a n
G(s) is thus a rational function of s. When d = 0, the degree of numerator polynomial of G(s) is strictly
less than the degree of the denominator polynomial and, therefore, the resulting transfer function is a strictly
proper transfer function. When d ¹ 0, the degree of numerator polynomial of G(s) will be equal to the degree
of the denominator polynomial, giving a proper transfer function. Further,
d = lim [G(s)] (5.34)
s®¥
From Eqns (5.31) and (5.33) we observe that the characteristic polynomial of matrix A of the system
(5.26) is same as the denominator polynomial of the corresponding transfer function G(s). If there are no
cancellations between the numerator and denominator polynomials of G(s) in Eqn. (5.33), the eigenvalues
of matrix A are same as the poles of G(s). We will take up in Section 5.9, this aspect of the correspondence
between state variable models and transfer functions. It will be proved that for a completely controllable and
completely observable state variable model, the eigenvalues of matrix A are same as the poles of the corre-
sponding transfer function.

5.4.1 Invariance Property

It is recalled that the state variable model for a system is not unique, but depends on the choice of a set of
state variables. A transformation
x(t) = P x (t); P is a nonsingular matrix (5.35)
results in the following alternative state variable model (refer Eqns (5.22)) for the system (5.26):
x& (t) = A x (t) + b u(t); x (t0) = P–1x(t0) (5.36a)
y(t) = c x (t) + du(t) (5.36b)
–1 –1
where A = P AP, b = P b, c = cP
The definition of new set of internal state variables should, evidently, not affect the eigenvalues or input-
output behaviour. This may be verified by evaluating the characteristic polynomial and the transfer function
of the transformed system.
(i) | sI – A |= | sI – P–1AP| = |sP–1P – P– 1AP| = |P–1(sI – A)P| = |P–1| |sI – A| |P| = |sI – A| (5.37)
(ii) System output in response to input u(t) is given by the transfer function
G (s) = c (sI – A )–1 b + d = cP(sI – P–1AP)–1P–1b + d
 Control System Analysis Using State Variable Methods 279

= cP(sP–1P – P–1AP)–1 P–1b + d = cP[P–1(sI – A)P]–1P–1b + d

= cPP–1(sI– A)–1PP–1b + d = c(sI – A)–1b + d = G(s) (5.38)
(iii) System output in response to initial state x (t0) is given by (refer Eqn. (5.27b))
c (sI – A )–1 x (t0) = cP(sI – P–1AP)–1P–1x(t0) = c(sI – A)–1x(t0) (5.39)
The input-output behaviour of the system (5.26) is, thus, invariant under the transformation (5.35).

Example 5.4 Consider the position control system of Example 5.2. The plant model of the system is
reproduced below:

x& (t) = Ax(t) + bu(t)

(5.40)
y(t) = cx(t)
with
LM0 1 0 OP LM 0OP
-1
MM0
A= 0
P MM10PP
1 ; b = 0 ; c = [1
-10 QP
0 0]
N -1 N Q
The characteristic polynomial of matrix A is
s –1 0
|sI – A| = 0 s +1 -1 = s(s2 + 11s + 11)
0 1 s + 10
The transfer function
Y ( s) c ( sI - A ) + b
G(s) = =
U ( s) | sI - A|
LMs 2
+ 11 s + 11 s + 10 1 OP L 0 O
[1 0 0] M
MN
0 s(s + 10) s PP MM 0 PP
0 -s s(s + 1)Q MN10PQ
=
s(s 2 + 11 s + 11)
10
= (5.41)
s( s 2 + 11 s + 11)
Alternatively, we can draw the state diagram of the plant model in signal-flow graph form and from there,
obtain the transfer function using Mason’s gain formula. For the plant model (5.40), the state diagram is
shown in Fig. 5.7. Application of Mason’s gain formula7 yields
Y(s ) 10 s -3
= G(s) =
U(s ) 1 - ( -10 s - s -1 - s -2 ) + 10 s -2
-1

7
Section 3.4 of reference [155]
 280 Digital Control and State Variable Methods

10 10
= 3 2
= 2
s + 11 s + 11 s s( s + 11 s + 11)

u 10 s–1 x3 1 s–1 x2 1 s–1 x1 = y

–10 –1
–1

Fig. 5.7 State diagram for the system (5.40)

5.4.2 Resolvent Algorithm

The matrix
( sI - A ) +
F(s) = (sI – A)–1 = (5.42)
| sI - A|
is known in mathematical literature as the resolvent of A. Resolvent matrix F(s) can be expressed in the
following form (refer Eqns (5.31) and (5.32)):
–1
Q1s n - 1 + Q 2 s n - 2 + L + Q n - 1 s + Q n
F(s) = (sI – A) = (5.43)
sn + a1 sn - 1 + L + a n -1 s + a n
where Qi are constant (n ´ n) matrices and a j are constant scalars.
An interesting and useful relationship for the coefficient matrices Qi of the adjoint matrix, can be ob-
tained by multiplying both sides of Eqn. (5.43) by |sI – A|(sI – A). The result is
| sI – A| I = (sI – A)(Q1sn– 1 + Q2sn – 2 + L + Qn–1s + Qn)
or
snI + a1sn – 1I + L + anI = snQ1 + sn – 1(Q2 – AQ1) + L + s(Qn – AQn – 1) – AQn
Equating the coefficients of si on both the sides, gives
Q1 = I
Q2 = AQ1 + a1I
Q3 = AQ2 + a2I (5.44a)
M
Qn = AQn – 1 + an – 1I
0 = AQn + anI
We have thus determined that the leading coefficient of (sI – A)+ is the identity matrix, and that the
subsequent coefficients can be obtained recursively. The last equation in (5.44a) is redundant, but can be
used as a check when these recursion equations are used as the basis of a numerical algorithm.
An algorithm based on Eqns (5.44a) requires the coefficients ai (i = 1, 2, ..., n) of the characteristic
polynomial. Fortunately, the determination of these coefficients can be included in the algorithm, for it can
be shown that8
1
ai = – tr(AQi); i = 1, 2, ..., n (5.44b)
i
where tr(M), the trace of M, is the sum of all the diagonal elements of the matrix M.

8
The proof of relation (5.44b) is quite involved and will not be presented here. Refer [108].
 Control System Analysis Using State Variable Methods 281
The algorithm given by Eans (5.44), called the resolvent algorithm, is convenient for hand calculation
and also easy to implement on a digital computer.

Example 5.5 Here we again compute (sI – A)–1 which appeared in Example 5.4, but this time using the
resolvent algorithm (5.44).
Q1 = I, a1 = – tr(A) = 11
Q2 = A + a1I

LM11 1 0 OP 1
MM 0
= 0
PP
10 1 ; a2 = – 2 tr(AQ2) = 11
N -1 1 Q
Q3 = AQ2 + a2I

LM11 10 1OP
1
MM 0
= 0
PP
0 0 ; a3 = – 3 tr(AQ3) = 0
N 0 0 Q
As a numerical check, we see that the relation
0 = AQ3 + a 3I
is satisfied. Therefore,
Q1 s2 + Q 2 s + Q 3
(sI – A)–1 = F(s) =
s3 + a 1 s2 + a 2 s + a 3

1
LMs 2
+ 11 s + 11 s + 10 1 OP
=
s(s 2 + 11s + 11) MM 0 s( s + 10) s PP
N 0 -s s(s + 1) Q
Cayley-Hamilton Theorem Using resolvent algorithm, we develop here a fundamental property of the
characteristic equation. To this end, we write from Eqns (5.44a)
Q 2 = A + a 1I
Q3 = AQ2 + a2I = A2 + a1A + a2I
M
Qn = An – 1 + a1An – 2 + L + an – 1I
AQn = An + a1An – 1 + L + an – 1A = – anI
Therefore,
An + a1An – 1 + L + an – 1A + anI = 0 (5.45)
This is the well-known result known as the Cayley-Hamilton theorem. Note that this equation is same as
the characteristic equation
sn + a1sn – 1 + L + an – 1s + an = 0 (5.46)
i i
with the scalar s in the latter replaced by the matrix A ( i = 1, 2, K , n).
 282 Digital Control and State Variable Methods
Thus, another way of stating the Cayley-Hamilton theorem is as follows: Every matrix satisfies its own
characteristic equation.
Later we will use the resolvent algorithm and the Cayley-Hamilton theorem for evaluation of the state
transition matrix required for the solution of the state equations.

5.5 CONVERSION OF TRANSFER FUNCTIONS TO CANONICAL STATE VARIABLE

MODELS

In the last section, we studied the problem—finding the transfer function from the state variable model of a
system. The converse problem—finding a state variable model from the transfer function of a system, is the
subject of discussion in this section. This problem is quite important because of the following reasons:
(i) Quite often the system dynamics is determined experimentally using standard test signals like a step,
impulse, or sinusoidal signal. A transfer function is conveniently fitted to the experimental data in
some best possible manner.
There are, however, many design techniques developed exclusively for state variable models. In
order to apply these techniques, experimentally obtained transfer function descriptions must be real-
ized into state variable models.
(ii) Realization of transfer functions into state variable models is needed even if the control system design
is based on frequency-domain design methods. In these cases, the need arises for the purpose of
transient response simulation. Many algorithms and numerical integration computer programs de-
signed for solution of systems of first-order equations are available, but there is not much software for
the numerical inversion of Laplace transforms. Thus, if a reliable method is needed for calculating the
transient response of a system, one may be better off converting the transfer function of the system to
state variable description, and numerically integrating the resulting differential equations, rather than
attempting to compute the inverse Laplace transform by numerical methods.
We shall discuss here the problem of realization of transfer function into state variable models.
Note the use of the term ‘realization’. A state variable model that has a prescribed rational function
G(s) as its transfer function, is the realization of G(s). The term ‘realization’ is justified by the fact
that by using the state diagram corresponding to the state variable model, the system with the transfer
function G(s) can be built in the real world by an op amp circuit.9
The following three problems are involved in the realization of a given transfer function into state vari-
able models:
(i) Is it possible at all, to obtain state variable description from the given transfer function?
(ii) If yes, is the state variable description unique for a given transfer function?
(iii) How do we obtain the state variable description from the given transfer function?
The answer to the first problem has been given in the last section. A rational function G(s) is realizable by
a finite dimensional linear time-invariant state model if and only if G(s) is a proper rational function. A
proper rational function will have state model of the form:
x& (t) = Ax(t) + bu(t)
(5.47)
y(t) = cx(t) + du(t)
where A, b, c and d are constant matrices of appropriate dimensions. A strictly proper rational function will
have state model of the form
x& (t) = Ax(t) + bu(t)
(5.48)
y(t) = cx(t)
9
Section 2.13 of reference [155]
 Control System Analysis Using State Variable Methods 283
Let us now turn to the second problem. In the last section, we saw that there are innumerable systems that
have the same transfer function. Hence, the representation of a transfer function in state variable form is
obviously, not unique. However, all these representations will be equivalent.
In the remaining part of this section, we deal with the third problem. We shall develop three standard, or
‘canonical’ representations of transfer functions.
A linear time-invariant SISO system is described by transfer function of the form
b s m + b 1s m – 1 + L + b m
G(s) = 0 n ;m £ n
s + a 1 s n -1 + L + a n
where the coefficients ai and bi are real constant scalars. Note that there is no loss in generality to assume
the coefficient of sn to be unity.
In the following, we derive results for m = n; these results may be used for the case m < n by setting
appropriate bi coefficients equal to zero. Therefore, our problem is to obtain a state variable model corre-
sponding to the transfer function
b sn + b1 sn - 1 + L + b n
G(s) = 0 n (5.49)
s + a1 sn -1 + L + a n
5.5.1 First Companion Form
Our development starts with a transfer function of the form
Z (s) 1
= n (5.50)
U( s ) s + a1 sn - 1 + L + a n
which can be written as
(sn + a1sn – 1 + L + an) Z(s) = U(s)
The corresponding differential equation is
pnz(t) + a1pn – 1z(t) + L + anz(t) = u(t)
where
k
D d z t
pkz(t) =
af
dt k
Solving for highest derivative of z(t), we obtain
pnz(t) = – a1pn – 1z(t) – a2pn – 2 z(t) – L – anz(t) + u(t) (5.51)
Now consider a chain of n integrators as shown in Fig. 5.8. Suppose that the output of the last integrator
is z(t); then, the output of the just previous integrator is pz = dz/dt, and so forth. The output from the first
integrator is pn – 1z(t), and thus, the input to this integrator is pnz(t). This leaves only the problem of obtain-
ing pnz(t) for use as input to the first integrator. In fact, this is already specified by Eqn. (5.51). Realization
of this equation is shown in Fig. 5.8.

u + pnz pn – 1 z pn – 2z pz z
ò ò ò ò
–

1 2 n–1 n

+ + +
+ + +

Fig. 5.8 Realization of the system (5.51)

 284 Digital Control and State Variable Methods
Having developed a realization of the simple transfer function (5.50), we are now in a position to consider
the more general transfer function (5.49). We decompose this transfer function into two parts, as shown in
Fig. 5.9. The output Y(s) can be written as
Y(s)= (b0sn + b1sn – 1 + L + bn) Z(s) (5.52a)
where Z(s) is given by
Z( s ) 1
= n n-1
(5.52b)
U( s ) s + a1 s +L+ an

Fig. 5.9 Decomposition of the transfer function (5.49)

A realization of the transfer function (5.52b) has already been developed. Figure 5.8 shows this realiza-
tion. The output of the last integrator is z(t) and the inputs to the integrators in the chain—from the last to the
first— are the n successive derivatives of z(t).
Realization of the transfer function (5.52a) is now straightforward. The output
y(t) = b0 pnz(t) + b1pn – 1z(t) + L + bnz(t),
is the sum of the scaled versions of the inputs to the n integrators. Figure 5.10 shows complete realization of
the transfer function (5.49). All that remains to be done is to write the corresponding differential equations.
To get one state variable model of the system, we identify the output of each integrator in Fig. 5.10 with
a state variable starting at the right and proceeding to the left. The corresponding differential equations,
using this identification of state variables, are
x& 1 = x2
x& 2 = x3
M (5.53a)
x& n – 1 = xn
x& n = – an x1 – an – 1 x2 – L – a1 xn + u
+ + + + y

+ + + +

0 1 2 n–1 n

u + xn xn – 1 x2 x1 z

1 2 n–1 n

+ + +
+ + +

Fig. 5.10 Realization of the system (5.49)

 Control System Analysis Using State Variable Methods 285
The output equation is found by careful examination of the block diagram of Fig. 5.10. Note that there are
two paths from the output of each integrator to the system output—one path upward through the box labelled bi,
and a second path down through the box labelled ai and thence, through the box labelled b0. As a consequence,
y = (bn – anb0) x1 + (bn – 1 – an – 1b0) x2 + L + (b1 – a1b0) xn + b0u (5.53b)
The state and output equations (5.53), organized in vector-matrix form, are given below:
x& (t) = Ax(t) + bu(t)
(5.54)
y(t) = cx(t) + du(t)
with
LM 0 1 0 L 0 OP LM0OP
L
A = M
M 0M 0
M
1
M
0
M
PP ; b = MM0M PP
MM 0 0 0 L 1 P
P MM0PP
NM -an -a n - 1 -a n - 2 L -a QP 1 NM1QP
c = [bn – anb0, bn–1 – an–1 b0, ... , b1 – a1b0]; d = b0
If the direct path through b0 is absent (refer Fig. 5.10), then the scalar d is zero and the row matrix c
contains only the bi coefficients.
The matrix A in Eqns (5.54) has a very special structure: the coefficients of the denominator of the
transfer function preceded by minus signs, form a string along the bottom row of the matrix. The rest of the
matrix is zero except for the ‘superdiagonal’ terms which are all unity. In matrix theory, a matrix with this
structure is said to be in companion form. For this reason, we identify the realization (5.54) as companion-
form realization of the transfer function (5.49). We call this the first companion form; another companion
form follows.

5.5.2 Second Companion Form

In the first companion form, the coefficients of the denominator of the transfer function appear in one of the
rows of the A matrix. There is another companion form in which the coefficients appear in a column of the
A matrix. This can be obtained by writing Eqn. (5.49) as
(sn + a1sn –1 + L + an) Y(s) = (b0sn + b1sn – 1 + L + bn) U(s)
or sn [Y(s) – b0U(s)] + sn – 1 [a1Y(s) – b1U(s)] + L + [anY(s) – bnU(s)] = 0
On dividing by sn and solving for Y(s), we obtain
1 1
Y(s) = b0U(s) + [b1U(s) – a1Y(s)] + L + n [bnU(s) – anY(s)] (5.55)
s s
1
Note that 1/sn is the transfer function of a chain of n integrators. Realization of [bnU(s) – anY(s)]
sn
requires a chain of n integrators with input [bnu – an y] to the first integrator in the chain from left-to-right.
1
Realization of n -1 [bn – 1U(s) – an – 1Y(s)], requires a chain of (n–1) integrators, with input [bn – 1u – an – 1y]
s
to the second integrator in the chain, from left-to-right, and so forth. This immediately leads to the structure
shown in Fig. 5.11. The signal y is fedback to each of the integrators in the chain, and the signal u is fed
forward. Thus the signal [bnu – an y] passes through n integrators; the signal [bn – 1u – an – 1 y] passes through
(n – 1) integrators, and so forth—to complete the realization of Eqn. (5.55). The structure retains the ladder-
like shape of the first companion form, but the feedback paths are in different directions.
 286 Digital Control and State Variable Methods

n n–1 n–2 1 0

x1 x2 xn – 1 xn
+ + + + +
+ + + + y

– – – –

n n–1 n–2 1

Fig. 5.11 Realization of Eqn. (5.55)

We can now write differential equations for the realization given by Fig. 5.11. To get one state variable
model, we identify the output of each integrator in Fig. 5.11 with a state variable—starting at the left and
proceeding to the right. The corresponding differential equations are
x& n = xn – 1 – a1 (xn + b0u) + b1u
x& n – 1 = xn – 2 – a2 (xn + b0u) + b2u
M
x& 2 = x1 – an – 1 (xn + b0u) + bn – 1u
x& 1 = – an (xn + b0u) + bnu
and the output equation is
y = x n + b0 u
The state and output equations, organized in vector-matrix form, are given below:
x& (t) = Ax(t) + bu(t)
y(t) = cx(t) + du(t) (5.56)
with
LM0 0 L 0 -a n OP b n - an b 0LM OP
1 0 L 0 -a n - 1
MM
A= 0 1 L 0
PP b
-a n - 2 ; b = n - 1
- a n -1 b 0
MM PP
MM
M M M M
PP M
MN PQ
b 1 - a 1b 0
MN
0 0 L 1 -a 1 PQ
c = [0 0 L 0 1]; d = b0
Compare A, b, and c matrices of the second companion form with that of the first. We observe that A, b,
and c matrices of one companion form correspond to the transpose of the A, c, and b matrices, respectively,
of the other.
There are many benefits derived from the companion forms of state variable models. One obvious benefit
is that both the companion forms lend themselves easily to simple analog computer models. Both the
companion forms also play an important role in pole-placement design through state feedback. This will be
discussed in Chapter 7.
 Control System Analysis Using State Variable Methods 287
5.5.3 Jordan Canonical Form
In the two canonical forms (5.54) and (5.56), the coefficients of the denominator of the transfer function
appear in one of the rows or columns of matrix A. In another of the canonical forms, the poles of the transfer
function form a string along the main diagonal of the matrix. This canonical form follows directly from the
partial fraction expansion of the transfer function.
The general transfer function under consideration is (refer Eqn. (5.49))
-1
b 0 sn + b 1 sn + L + bn
G(s) = n n -1
s + a1 s + L + an
By long division, G(s) can be written as
-1 - 2
b 1¢ s n + b ¢2 s n + L + b ¢n
G(s) = b0 + n n -1
= b0 + G¢(s)
s + a1 s + L + an
The results are simplest when the poles of the transfer function are all distinct. The partial fraction
expansion of the transfer function, then has the form.
Y (s) r1 r2 rn
G(s) = = b0 + + +L+ (5.57)
U(s) s - l1 s - l2 s - ln
The coefficients ri (i = 1, 2, K, n) are the residues of the transfer function G¢(s) at the corresponding
poles at s = li (i = 1, 2, K, n). In the form of Eqn. (5.57), the transfer function consists of a direct path with
gain b0, and n first-order transfer functions in parallel. A block diagram representation of Eqn. (5.57) is shown
in Fig. 5.12. The gains, corresponding to the residues, have been placed at the outputs of the integrators.
This is quite arbitrary. They could have been located on the input side, or indeed, split between the input and
the output.

+
+ +
+ +

+ +
+ +

+
+

Fig. 5.12 Realization of G(s) in Eqn. (5.57)

 288 Digital Control and State Variable Methods
Identifying the outputs of the integrators with the state variables results in the following state and output
equations:
x& (t) = L x(t) + bu(t)
(5.58)
y(t) = cx(t) + du(t)
with
LMl 1 0 L 0 OP LM1OP
0 l2 L 0
PP ; b = M1P ; c = [r
L=M r2 L rn]; d = b0
MM M M M
P MMMPP 1

N0 0 L l Q
n N1Q
It is observed that for this canonical state variable model, the matrix L is a diagonal matrix with the poles
of G(s) as its diagonal elements. The unique decoupled nature of the canonical model is obvious from Eqns
(5.58); the n first-order differential equations are independent of each other:
x&i(t)= li xi(t) + u(t); i = 1, 2, K , n (5.59)
This decoupling feature, as we shall see later in this chapter, greatly helps in system analysis.
The block diagram representation of Fig. 5.12 can be turned into hardware only if all the poles at s = l1,
l2, ..., ln are real. If they are complex, the feedback gains and the gains corresponding to the residues, are
complex. In this case, the representation must be considered as being purely conceptual; valid for theoretical
studies, but not physically realizable. A realizable representation can be obtained by introducing an equiva-
lence transformation.
Suppose that s = s + jw , s = s – jw and s = l are the three poles of a transfer function. The residues at the
pair of complex conjugate poles must be themselves complex conjugates. Partial fraction expansion of the
transfer function, with a pair of complex conjugate poles and a real pole, has the form
p + jq p - jq r
G(s) = d + + +
s - (s + j w ) s - (s - j w ) s - l
A state variable model for this transfer function is given below (refer Eqns (5.58)):
x& = L x + bu
(5.60)
y = cx + du
with
LMs + j w 0 0 OP 1 LM OP
L= s - jw 0 ; b = 1 ; c = [p + jq
MM 00 l
PP MM PP p – jq r]
N 0 Q 1 NQ
Introducing an equivalence transformation
x = Px
with
1 /2LM - j1/ 2 0 OP
P = 1 /2 j1/ 2 0
MM 0 PP
N 0 1 Q
 Control System Analysis Using State Variable Methods 289
we obtain (refer Eqns (5.22))
x& (t) = A x (t) + b u(t)
(5.61)
y(t) = c x(t) + du(t)
where

–1
LM1 1 0OP LMs + jw 0 0 OP LM1/2 - j1/ 2 0 OP LM s w 0 OP
A =P LP = j - j 0 s - jw 0 = -w s
MM0 0 1PP MM 00 0 1/ 2
PM
l QP MN 0
j1/ 2
P M
1 QP MN 0
0
l QP
P
N QN 0 0 0

LM2OP
b = P –1b = 0 ; c = cP = [ p q r]
MM1PP
NQ
When the transfer function G(s) has repeated poles, the partial fraction expansion will not be as simple
as Eqn. (5.57). Assume that G(s) has m distinct poles at s = l1, l2, K, lm of multiplicity n1, n2, K, nm,
respectively; n = n1 + n2 + L + nm. That is, G(s) is of the form
-1 - 2
b 1¢ s n + b ¢2 s n + L + b n¢
G(s) = b 0 + n1
(5.62)
(s - l 1 ) ( s - l 2 ) L ( s - l m ) nm
n2

The partial fraction expansion of G(s) is of the form.

Y( s)
G(s) = b0 + H1(s) + L + Hm(s) = (5.63)
U( s)
where
ri1 ri 2 rini Yi ( s )
Hi(s) = ni
+ ni - 1
+L+ =
(s - l i ) (s - l i ) (s - l i ) U( s )
The first term in Hi(s) can be synthesized as a chain of ni identical, first-order systems, each having
transfer function 1/(s – li). The second term can be synthesized by a chain of (ni – 1) first-order systems, and
so forth. The entire Hi(s) can be synthesized by the system having the block diagram shown in Fig. 5.13.

+ +

+ +

+ + +
+ + +

Fig. 5.13 Realization of Hi(s) in Eqn. (5.63)

 290 Digital Control and State Variable Methods

We can now write differential equations for the realization of Hi(s), given by Fig. 5.13. To get one state
variable formulation, we identify the output of each integrator with a state variable—starting at the right and
proceeding to the left. The corresponding differential equations are
x&i1 = li xi1 + xi2
x& i2 = li xi2 + xi3 (5.64a)
M
x&ini = li xini + u
and the output is given by
yi = ri1 xi1 + ri2 xi2 + L + rini xini (5.64b)
If the state vector for the subsystem is defined by
xi = [xi1 xi2 L xini ]T
then Eqns (5.64) can be written in the standard form
x& i = L ixi + biu
(5.65)
yi = ci xi
where
LMl i 1 0 L 0 0OP L0O
MM 0 li 1 L 0 0P MM0PP
L = M
i MM M M M P
M ; b = M M P ; c = [r
P M0 P
i i i1 ri2 L rini ]
0 L li 1P MM PP
MM 00 0
0 0 L 0 l Q
P
P N1 Q
N i

Note that matrix L i has two diagonals—the principal diagonal has the corresponding characteristic root
(pole), and the superdiagonal has all 1’s. In matrix theory, a matrix having this structure is said to be in
Jordan form. For this reason, we identify the realization (5.65) as Jordan canonical form.
According to Eqn. (5.63), the overall trans-
˜0 fer function G(s) consists of a direct path with
gain b0 and m subsystems, each of which is
+ in the Jordan canonical form, as shown in
x1 = 1x1 + b1u y1 + y
Fig. 5.14. The state vector of the overall sys-
y1 = c1x1 +
u tem consists of the concatenation of the state
vectors of each of the Jordan blocks:
x2 = 2x2 + b2u y2 +
y2 = c2x2 +
LM x OP
1
x
x= M P
2
(5.66a)
xm = m xm + bm u ym
MM M PP
ym = cm xm Nx Q
m
Since there is no coupling between any of
Fig. 5.14 Subsystems of Jordan canonical form the subsystems, the L matrix of the overall
combined into overall system system is ‘block diagonal’:
 Control System Analysis Using State Variable Methods 291

LM L 1 0 L 0 OP
0 L2 L 0
L=M PP (5.66b)
MM M M M
PQ
N0 0 L Lm
where each of the submatrices L i is in the Jordan canonical form (5.65). The b and c matrices of the overall
system are the concatenations of the bi and ci matrices respectively of each of the subsystems:
LM b OP
1
b
b = M P ; c = [c
2
1 c 2 L c m ] ; d = b0 (5.66c)
MM M PP
Nb Q
m

The state variable model (5.58) derived for the case of distinct poles, is a special case of Jordan canonical
form (5.66) where each Jordan block is of 1 ´ 1 dimension.

Example 5.6 In the following, we obtain three different realizations for the transfer function

s + 3 Y ( s)
G(s) = =
s 3 + 9 s 2 + 24 s + 20 U( s)

First Companion Form Note that the given G(s) is a strictly proper fraction; the realization will therefore
be of the form (5.48), i.e., the parameter d in the realization {A, b, c, d} is zero.
The state variable formulation in the first companion form, can be written just by inspection of the given
transfer function. Referring to Eqns (5.54), we obtain

LM x& OP LM 0
1 1 OP LM x OP LM0OP
0 1

MM x&& PP = M 0
2 0
P 1
MM x PP + M0P u
2

N x Q MN-20
3 -24 -9QP N x Q MN1 PQ 3

LM x OP 1
y = [3 1 0] M x P 2
MN x PQ 3

Figure 5.15a shows the state diagram in signal flow graph form.

Second Companion Form Referring to Eqns (5.56), we obtain

LM x& OP LM0
1 0 -20 OP LM x OP LM3OP
1

MM x& PP = MM1
2 0 -24
PP MM xx PP + MM10PP u
2

N x& Q N0
3 1 -9 QN Q NQ3

y = x3
 292 Digital Control and State Variable Methods
Figure 5.15b shows the state diagram.
1

1 sE 1 sE 1 sE 1
u x3 x2 x1 3
(a)
y
E 9
E 24
E 20
1

sE 1 1 1
3 x1 1 sE x2 1 sE x3 1
(b) u y
E 9

E 24
E 20
2/9

sE 1 sE 1 x1 1/3
(c) 1 x2 1
u y

E 2
E 2 1 E 2/9
1 sE x3

E 5
Fig. 5.15 Three realizations of given G(s)

Jordan Canonical Form The given transfer function G(s) in the factored form:
s+3
G(s) =
( s + 2)2 (s + 5)
Using partial fraction expansion, we obtain
1/ 3 2/ 9 -2/ 9
G(s) = 2
+ +
( s + 2) s+2 s+5

A matrix of the state variable model in Jordan canonical form will be block-diagonal; consisting of two
Jordan blocks (refer Eqns (5.65)):

L1 =
LM-2 1 OP
; L 2 = [–5]
N0 -2 Q
The corresponding bi and ci vectors are (refer Eqns (5.65)):

b1 =
LM0OP ; c 1 = [ 13 2
9 ]; b2 = [1] ; c2 = [– 29 ]
N1Q
 Control System Analysis Using State Variable Methods 293
The state variable model of the given G(s) in Jordan canonical form is, therefore, given by (refer
Eqns (5.66))
LM x& OP LM-2
1 1 0 OP LM x OP LM 0 OP
1

MM x& PP = MM 00
2 -2 0
PP MM xx PP + MM 11 PP u
2

N x& Q N
3 0 -5 QN Q N Q
3

LM x OP
1
-2
y = [ 13 2
9 9 ] Mx P2
MN x PQ
3

Figure 5.15c shows the state diagram. We note that Jordan canonical state variables are not completely
decoupled. The decoupling is blockwise; state variables of one block are independent of state variables of all
other blocks. However, the state variables of one block, among themselves, are coupled; the coupling is
unique and simple.

5.6 EIGENVALUES AND EIGENVECTORS

The last section was concerned with the derivation of state variable models for a given transfer function. Out
of infinitely many realizations possible for a given transfer function, we have derived the following three
‘standard’ or canonical forms:
(i) First companion form
(ii) Second companion form
(iii) Jordan form
Consider now the situation where the system dynamics is already known in the form of a state variable
model. For example, state equations representing the dynamics of a physical system may be obtained by the
application of physical laws. However, state variables in such a formulation may not be as convenient as
some other canonical state variables. Transformation of an original state variable model to a canonical form
may, therefore, be helpful in solving analysis and design problems.
In this section, we deal with the problem of transformation of a given state variable model to Jordan
canonical form (transformation of a given model to other canonical forms will be taken up in Section 5.9,
and to companion forms in Chapter 7).
Given state variable model:
x& (t) = Ax(t) + bu(t)
(5.67)
y(t) = cx(t) + du(t)
where A, b, c and d are constant matrices of dimensions n ´ n, n ´ 1, 1 ´ n and 1 ´ 1 respectively.
The problem is to find an equivalence transformation
x = Px (5.68)
such that the equivalent model (refer Eqns (5.22))
x& (t) = P – 1AP x (t) + P –1bu(t)
(5.69)
y(t) = cP x(t) + du(t)
is in Jordan canonical form.
 294 Digital Control and State Variable Methods

5.6.1 Eigenvalues
For a general nth-order matrix
LMa 11 a12 L a1n OP
a a22 L a2 n
A=M PP
21
MM M M M
PQ
Na n1 an 2 L ann
the determinant
l - a11 - a12 L - a1n
- a21 l - a22 L - a2 n
|l I – A| =
M M M
- an1 - an 2 L l - ann

On expanding the determinant we find that |l I – A | , called the characteristic polynomial of the matrix A,
is a polynomial of degree n:
|l I – A| = D(l ) = l n + a 1l n – 1 + L + an – 1l + an
where ai are constant scalars.
The equation
D(l) = ln + a1ln – 1 + L + an – 1l +an = 0 (5.70)
is called the characteristic equation of the matrix A, and its n roots are called characteristic roots, or
characteristic values, or eigenvalues of the matrix A. When A represents the dynamic matrix of a linear
system, the eigenvalues determine the dynamic response of the system (the next section will establish this
fact), and also turn out to be the poles of the corresponding transfer function (refer Eqn. (5.31)).
Eigenvalues of a matrix A are invariant under equivalence transformation (refer Eqn. (5.37)), i.e.,
|lI – A| = |lI – P –1AP|
for any nonsingular matrix P.

5.6.2 Eigenvectors
Consider an n ´ n matrix A with eigenvalues {l1, l2, L, ln}. We start with the assumption of distinct
eigenvalues; later we will relax this assumption.
Case I: All Eigenvalues are Distinct State transformation to Jordan canonical form requires a trans-
formation matrix P such that

LM l1 0 L 0 OP
0 l2 L 0
P
P –1AP = L = M (5.71)
MM M M M P
N 0 0 L l Q
Pn

Let the transformation matrix P required to transform A to L , be of the form

P = [v1 v2 L vn]; (5.72a)
 Control System Analysis Using State Variable Methods 295

LMv OP 1i
v
v = M P = ith column of P
2i
(5.72b)
i
MM M PP
Nv Q ni

Equation (5.71) shows that

AP = P L
LM l 1 0 L 0 OP
0 l2 L 0
or A[v1 v2 L vn] = [v1 v2 L v ]M P
n
MM M M M P
P
N0 0 L l Q
n

By equating the ith columns, we obtain

Avi = livi

or ( l iI – A)vi = 0 (5.73)
This is a set of n homogeneous equations in n unknowns v1i, v2i, K, vni.
There are two questions of interest with regard to Eqn. (5.73):
(i) whether a solution to Eqn. (5.73) exists; and,
(ii) if the answer to the first question is yes, how many linearly independent solutions occur?
We consider an example to answer these questions. Refer Section 5.2 for the basic definitions from linear
algebra used in the sequel.

Example 5.7 The matrix

LM-4 1 0 OP
-3
A=
MM 00 1
-2
PP
N 0 Q
has the characteristic equation
l +4 -1 0
|l I – A| = 0 l +3 -1
0 0 l+2

= (l + 4)(l + 3)(l + 2) = 0
Therefore, the eigenvalues of A are l1 = – 2, l 2 = – 3 and l 3 = – 4.
Consider a set of homogeneous equations
(l1I – A)v1 = 0
LM2 -1 0 OP LMv11 OP LM0OP
-1
or
MM00 1
PP MMv
21 PP = MM0PP (5.74)
N 0 0 Q Nv
31 Q N0Q
 296 Digital Control and State Variable Methods
It is easy to check that rank of the matrix (l1I – A) is two, i.e.,
r (l1I – A) = 2
A highest-order array having a nonvanishing determinant, is

LM2 -1OP ,
N0 1Q
which is obtained from (l1I – A) by omitting the third row and the third column. Consequently, a set of
linearly independent equations is
2 v 11 – v 21 = 0
v 21 = v 31

or
LM2 -1OP LMv OP =
11 LM 0 OP
N0 1Q Nv Q 21 Nv Q31

-1
Therefore,
LMv OP =
11 LM2 -1OP LM 0 OP = LMv / 2OP
31

Nv Q
21 N0 1Q Nv Q Nv Q 31 31

There are three components in v1 and two equations governing them; therefore, one of the three compo-
nents can be arbitrarily chosen. For v 31 = 2, a solution to Eqn. (5.74) is
LM1OP
v1 =
MM22PP
NQ
A different choice for v 31 leads to a different solution to Eqn. (5.74). In fact, this set of equations has
infinite solutions as demonstrated below:
For v 31 = 2a (with a arbitrary), the solution
LM1OP
v1 = a 2
MM PP
2 NQ
Obviously, this solution is non-unique. However, all non-trivial solutions have a unique direction, and
they differ only in terms of a scalar multiplier. There is, thus, only one independent solution.
Corresponding to the eigenvalue l2 = – 3, a linearly independent solution to homogeneous equations
(l2I – A)v2 = 0
is given by
LM1OP
v2 =
MM10PP
NQ
And for l3 = – 4, the equations
(l 3I – A)v3 = 0
 Control System Analysis Using State Variable Methods 297
have a linearly independent solution
LM2OP
v3 = 0
MM PP
N0Q
In general, the number of equations that the vector vi in (5.73) has to obey, is equal to r(liI – A) where
r(M) denotes the rank of matrix M. There are n components in vi (n = number of columns of (liI – A));
therefore (n – r(liI – A)) components of vi can be arbitrarily chosen. Thus the number of linearly independ-
ent solutions of the homogeneous equation (5.73) = [n – r(liI – A)] = g (l iI – A), where g (M) denotes the
nullity of matrix M.
We have the following answers to the two questions raised earlier with regard to Eqn. (5.73):
(i) For Eqn. (5.73) to have a non-trivial solution, rank of (liI – A) must be less than n, or, equivalently,
det(liI – A) = 0. This condition is satisfied by virtue of the fact that li is an eigenvalue.
(ii) The number of linearly independent solutions to Eqn. (5.73), is equal to nullity of (l i I – A).
The nullity of matrix (liI – A) does not exceed the multiplicity of the eigenvalue li (refer Lancaster and
Tismenetsky [28] for proof of the result). Therefore, for distinct eigenvalue li, there is one, and only one,
linearly independent solution to Eqn. (5.73). This solution is called the eigenvector of A associated with the
eigenvalue li.
Theorem 5.1 Let v1, v2, K, vn be the eigenvectors associated with the distinct eigenvalues l1, l2, K, ln,
respectively, of matrix A. The vectors v1, v2, K, vn are linearly independent and the nonsingular matrix
P = [v1 v2 L vn]
transforms matrix A into Jordan canonical form.
Proof Let a1v1 + a2v2 + L + anvn = 0 (5.75)
If it can be shown that this implies that a1 = a2 = L = an = 0, then the set {vi} is linearly independent.
Define
Ti = liI – A
Note that Tivi = 0
and Tivj = (li – lj)vj if i ¹ j
Multiplying Eqn. (5.75) by T1 gives
a2(l1 – l2)v2 + a3(l1 – l3) v3 + L + an(l1 – ln)vn = 0
Multiplying this in turn by T2, T3, K, Tn – 1, gives
a3 (l1 – l3)(l2 – l3)v3 + L + an(l1 – ln)(l2 – ln)vn = 0
M
an – 1 (l1 – ln – 1)(l2 – ln – 1) L (ln – 2 – ln – 1)vn – 1
+ an (l1 – ln)(l2 – ln) L (ln – 2 – ln)vn = 0 (5.76)
an(l1 – ln)(l2 – ln) L (ln – 2 – ln)(ln – 1 – ln)vn = 0 (5.77)
 298 Digital Control and State Variable Methods

Since vn ¹ 0 and ln ¹ li for i ¹ n, Eqn. (5.77) requires that an = 0. This, plus Eqn. (5.76), requires that
an – 1 = 0
Continuing this reasoning shows that Eqn. (5.75) requires ai = 0 for i = 1, 2, K, n; so the eigenvectors vi
are linearly independent.
The matrix P, constructed by placing the eigenvectors (columns) together, is therefore a nonsingular
matrix.
As per Eqns (5.71)–(5.73), P –1AP = L .

Example 5.8 Consider the matrix

LM -4 1 0 OP
-3
A=
MM 00 1
-2
PP
N 0 Q
for which we found, in Example 5.7, the eigenvalues and eigenvectors to be
LM1OP LM1OP LM2OP
l 1 = – 2, v1 = 2 ;l 2= – 3, v2 = M1P ; l 3= – 4, v3 = M0P
MM2PP
NQ NM0PQ NM0PQ
The transformation matrix
LM
1 1 2 OP
P= 2 1 0
MM P
N2 0 0PQ
This gives

–1 1
LM0 0 2 OP LM -4 1 0 OP LM1 1 2 -2 OP LM 0 OP
0
-4 -3 -3 0 =L
P AP =
4 MM02 -2
4
PM 0
1 PQ MN 0
1
P M2
-2 PQ MN2
1 0 = 0
0PQ
P MM -4
PP
N 0 0 N 0 0 Q
which is the diagonal matrix (a special case of Jordan canonical form) with eigenvalues of A as its diagonal
elements. In fact, L could be written down directly without computing P –1AP.

Computation of Eigenvectors The eigenvectors vi which satisfy the equations

(liI – A)vi = 0 (5.78)
can be computed by solving the set of linear algebraic equations. The method of Gauss elimination is a
straightforward and powerful procedure for reducing systems of linear equations to a simple reduced form,
easily solved by substitution (refer Noble and Daniel [27]). High quality software is available commercially;
for example, the MATLAB system from the Math Works [152].
In the following, we give an analytical procedure of computing the eigenvectors. This procedure is quite
useful for hand calculations.
Using the property (refer Eqn. (5.3))
M adj M = |M|I
 Control System Analysis Using State Variable Methods 299
and letting M = (liI – A), yields
(liI – A) adj (liI – A) = |liI – A| I
Since |liI – A| is the characteristic polynomial and li is an eigenvalue, this equation becomes
(liI – A) adj (liI – A) = 0 (5.79)
A comparison of Eqn. (5.78) with (5.79) shows that vi is proportional to any non-zero column of
adj (liI – A).

Example 5.9 Consider the state variable model

x& = Ax + bu
y = cx

LM -9 1 0 OP LM2OP
with A = -26 0 1 ; b = 5 ; c = [1 2 – 1]
MM-24 0QP
P MM0PP
N 0 NQ
The characteristic equation
|lI – A| = 0

yields the roots l 1 = – 2, l 2 = – 3, and l 3 = – 4.

Ll + 9 -1 0 l2
OP LM l 1 OP
adj(lI – A) = adj M 26 l -1 = -26l - 24MM l2 + 9l l+9 PP
MM 24 0 l
PP
-24l -24 l2 + 9l + 26
N Q N Q
LM 4 -2 1 OP LM OP
1
For l1 = – 2, adj(l I – A) = 28 -14 7 ; v1 = 7
1 MM48 -24 12
PP MM PP
N 12 Q N Q
LM 9 -3 1O
P6P ; v = LM16OP
For l2 = – 3, adj(l I – A) = 54 -18
2 MM72 -24 8QP
2 MM8PP
N NQ
L16 -4 1O L1O
For l3 = – 4, adj(l I – A) = M80 -20 5P ; v = M5P
3 MM96 -24
P MM6PP
6PQ
3

N NQ
In each case, the columns of adj(liI – A) are linearly related. In practice, it is necessary to calculate only
one (non-zero) column of the adjoint matrix.
 300 Digital Control and State Variable Methods
The transformation matrix
LM 1 1 1 OP
P = [v1 v2 v] = 7 6 5
3 MM12 PP
N 8 6 Q
State transformation
x = Px
results in the following model (refer Eqns (5.22)):
x& = L x + b u
y = cx
with

–1 1
LM -4 2 -1 OP LM -9 1 0 OP LM 1 1 1 OP LM-2 0 0 OP
L = P AP = – -6 2 -26 -3 0
2 MM-1618 PM
-1QP MN-24
0 1
P0P MM127 6 5 = 0
P M
6QP MN 0 0 -4
PP
N 4 0 QN 8 Q
–1
LM -1OP
b= P b = -3 ; c = cP = [3 5 5]
MM 6PP
N Q
Case II: Some Eigenvalues are Multiple Roots of the Characteristic Equation For notational con-
venience, we assume that matrix A has an eigenvalue l1 of multiplicity n1, and all other eigenvalues l n1 + 1 ,
K, ln are distinct, i.e.,
|lI – A| = ( l - l 1 ) n1 ( l - l n1 + 1 ) L (l – ln)
Recall the result stated earlier: the nullity g of matrix (liI – A) does not exceed the multiplicity of li.
Therefore,
1 £ g (l1I – A)£ n1
g ( l n1 + 1 I – A) = 1
M
g (lnI – A) = 1
We know that the number of linearly independent eigenvectors associated with an eigenvalue li is equal
to the nullity g of the matrix (liI – A). Thus, when one or more eigenvalues is a repeated root of the
characteristic equation, a full set of n linearly independent eigenvectors may, or may not, exist.
It is convenient to consider three sub-classifications for Case II.
Case II1: Nullity of (l1I – A) = n1 In this case, the vector equation
(l1I – A)v = 0
has n1 linearly independent solutions, say, v1, v2, K, vn1. We have thus, a full set of n1 eigenvectors associ-
ated with multiple eigenvalue l1.
The remaining (n – n1) eigenvectors are obtained from the vector equations
(ljI – A)vj = 0, j = n1 + 1, K, n
Each of these vector equations has only one linearly independent solution.
 Control System Analysis Using State Variable Methods 301
The matrix
P = [v1 v 2 L vn 1 vn1+1 L vn]

LM l1 0 L 0 0 L 0 OP
MM 0 l1 L 0 0 L 0
M M M M L M P
P
gives P –1AP = L =
MM 0 0 L l1 0 L 0P
P
MM 0 0 L 0 l n1 +1 L 0P
P
MM M M M M M P
N 0 0 L 0 0 L l PQ
n

Case II 2: Nullity of (l1I – A) = 1 For this case, there is only one eigenvector associated with l1, regard-
less of multiplicity n1. This eigenvector is given by the linearly independent solution of the vector equation
(l1I – A)v = 0
The solution to this equation may be found as in Case I.
We have seen in Cases I and II1, that the transformation matrix P yields a diagonal matrix L if, and only
if, P has a set of n linearly independent eigenvectors. When nullity of the matrix (l1I – A) is one, n linearly
independent eigenvectors cannot be constructed and, therefore, the transformation to a diagonal matrix is
not possible.
The simplest form to which matrix A, having a multiple eigenvalue l1 of multiplicity n1 with g (l1I – A)
= 1 and all other distinct eigenvalues, can be reduced is the Jordan canonical form:
LM L 1 0 L 0 OP
0 L n1 L 0
L=M PP
+1
MM M M M
PQ
N0 0 L Ln
where the Jordan blocks L i are
LM l 1 1 0 L 0 OP
0 l1 1 L 0
L1 =M PP
MM M M M M
PQ
N0 0 0 L l1

L n1 +1 = [ln1 + 1]; L; L n = [ln]

The transformation matrix P is given by
P = [v1 v2 L vn 1 vn1 +1 L vn]
with v1 v2 L, vn1 determined as follows:
LMl 1 1 0 L 0 OP
A [v1 v2 L vn1] = [v1 v2 L vn1 ]M0 l1 1 L 0
MM M M M MP
P
N0 0 0 L l Q
P1
 302 Digital Control and State Variable Methods
or Av1 = l1v1
Av2 = v1 + l1v2
M
Avn1 = vn1 – 1 + l1vn1
Rearranging these equations, we obtain
(l1I – A)v1 = 0
(l1I – A)v2 = – v1
M
(l1I – A)vn1 = – vn1 – 1
It can easily be established that each of these vector equations gives one linearly independent solution,
and the solutions v1, v2, ..., vn 1 form a linearly independent set of vectors. We shall call the set of vectors {v1,
..., vn 1} the chain of generalized eigenvectors. Note that the vector v1 in the chain is, in fact, the eigenvector
associated with multiple eigenvalue l1.
Eigenvectors for the Jordan blocks L n 1 + 1, ..., L n are given by the solution of the vector equations
(ljI – A)vj = 0; j = n1 + 1, ..., n
The eigenvectors corresponding to distinct eigenvalues, and the chains of generalized eigenvectors corre-
sponding to multiple eigenvalues, form the transformation matrix P.
Case II 3: 1 < g (l 1I – A) < n1 For this case, there are g eigenvectors associated with l1. There will be one
Jordan block for each eigenvector; that is, l1 will have g blocks associated with it. This case is just a
combination of the Cases II1 and II2; there is only one ambiguity—the knowledge of n1 and g does not
directly give the information about the dimension of each of the Jordan blocks associated with l1.
Assume that l1 is a fourth-order root of the characteristic equation and g (l1I – A) = 2. The two
eigenvectors associated with l1 satisfy
(l1I – A)va = 0, (l1I – A)vb = 0
To form the transformation matrix, we require two generalized eigenvectors—but it is still uncertain
whether the generalized eigenvectors are both associated with va, or both with vb, or one with each. That is,
the two Jordan blocks could take one of the following forms:
LMl 1 1 0 OP
L1 = M0 l1 1 , L 2 = [l 1]
PP
MN 0 0 l1 Q
or L1 = M
Ll 1 1 OP , L = LMl
2
1 1 OP
N0 l Q
1 N0 l Q
1

The first pair corresponds to the equations

(l1I – A)v1 = 0
(l1I – A)v2 = – v1
(l1I – A)v3 = – v2
(l1I – A)v4 = 0
The second pair corresponds to the equations
(l1I – A)v1 = 0
(l1I – A)v2 = – v1
(l1I – A)v3 = 0
(l1I – A)v4 = – v3
 Control System Analysis Using State Variable Methods 303
Ambiguities such as this, can be resolved by the trial-and-error procedure.
An n-dimensional SISO system with m distinct eigenvalues l1, l2, ..., lm, of multiplicity n1, n2, ..., nm,
F m
I
GH å n JK , has the following Jordan canonical representation.
respectively n =
i =1
i

x& = L x + bu
y = cx + du
where L is a block diagonal matrix with Jordan blocks L 1, ..., L m corresponding to the eigenvalues
l1, ..., lm, respectively, on its principal diagonal; each Jordan block L i corresponding to the eigenvalue li is
again a block diagonal matrix with g (i) sub-blocks on its principal diagonal; g (i) being the number of
linearly independent eigenvectors associated with the eigenvalue li:
LML 1 0 L 0 OP
0 L2 L 0
L =M P
(n ´ n) MM M M M P
P
N0 0 L L Qm

LML 1i 0 L 0 OP
Li 0 L 2i L 0
=M PP ; i = 1, 2, K, m
( ni ´ ni ) MM M M M
PQ
N0 0 L L g ( i )i

LM l i 1 0 L 0 OP
0 li 1 L 0
L ki =M PP
; k = 1, 2, K, g (i)
MM M M M M
PQ
N0 0 0 L li
The topic of computation of eigenvectors and generalized eigenvectors for systems with multiple
eigenvalues is much too detailed and specialized for this book to treat (Refer Gopal [105] and Brogan
[106]). Over the years, experts have developed excellent general-purpose computer programs for the
efficient and accurate determination of eigenvectors and generalized eigenvectors [152-154].
In this book, the usefulness of the transformation of state variable models to Jordan canonical form will
be illustrated through system examples having distinct eigenvalues.

5.7 SOLUTION OF STATE EQUATIONS

In this section, we investigate the solution of the state equation

D
x& (t) = Ax(t) + bu(t); x(t0) = x0 (5.80)
where x is n ´ 1 state vector, u is a scalar input, A is n ´ n constant matrix, and b is n ´ 1 constant vector.

5.7.1 Matrix Exponential

Functions of square matrices arise in connection with the solution of vector differential equations. Of
immediate interest to us are matrix infinite series.
 304 Digital Control and State Variable Methods
Consider the infinite series in a scalar variable x:
¥
f(x) = a0 + a1x + a2 x2 + L = å a i xi (5.81a)
i= 0
with the radius of convergence r.
We can define infinite series in a matrix variable A, as
¥
f(A) = a0I + a1A + a2A2 + L = å aiAi (5.81b)
i= 0
An important relation between the scalar power series (5.81a) and the matrix power series (5.81b) is that
if the absolute values of eigenvalues of A are smaller than r, then the matrix power series (5.81b) converges
(for proof, refer Lefschetz [33]).
Consider, in particular, the scalar power series
¥
1 2 1 k 1
f (x) = 1 + x + x +L+ x + L = å xi (5.82a)
2! k! i= 0 i!
It is well-known that this power series converges on to the exponential ex for all finite x, so that
f (x) = ex (5.82b)
It follows from this result that the matrix power series
¥
1 2 1 k 1 i
f (A) = I + A +
2!
A +L+
k!
A +L= å i!
A
i= 0
converges for all A. By analogy with the power series in Eqns (5.82) for the ordinary exponential function,
we adopt the following nomenclature:
If A is an n ´ n matrix, the matrix exponential of A is
¥
D 1 2 1 k 1 i
eA = I + A + A +L+ A +L= å A
2! k! i= 0 i!
The following matrix exponential will appear in the solution of state equations:
¥
1 22 1 kk 1
eAt = I + At + A t +L+ A t + L = å Aiti (5.83)
2! k! i = 0 i!
It converges for all A and all finite t.
In the following, we examine some of the properties of the matrix exponential.
(i) eA0 = I (5.84)
This is easily verified by setting t = 0 in Eqn. (5.83)
(ii) eA(t + t ) = eAteAt = eAt eAt (5.85)
This is easily verified by multiplying out the first few terms for eAt and eAt .
(iii) (eAt)–1 = e– At (5.86)
Setting t = – t in Eqn. (5.85), we obtain
eAte– At = eA0 = I
At – At
Thus the inverse of e is e .
Since the inverse of eAt always exists, the matrix exponential is nonsingular for all finite values of t.
d At
(iv) e = AeAt = eAtA (5.87)
dt
Term-by-term differentiation of Eqn. (5.83), gives
 Control System Analysis Using State Variable Methods 305
d At 1 1
e = A + A2t + A3t2 + L + Ak tk – 1 + L
dt 2! ( k - 1) !
1 22 1
= A[I + At + A t +L+ Ak – 1tk – 1 + L ] = AeAt
2! ( k - 1) !
1 1
= [I + At + A2t2 + L + Ak – 1tk – 1 + L ]A = eAtA
2! ( k - 1) !

5.7.2 Solution of Homogeneous State Equation

The simplest form of the general differential equation (5.80) is the homogeneous, i.e., unforced equation
D
x& (t) = Ax(t); x(t0) = x0 (5.88)
We assume a solution x(t) of the form
x(t) = eAtk (5.89)
At
where e is the matrix exponential function defined in Eqn. (5.83), and k is a suitably chosen constant vector.
The assumed solution is, in fact, the true solution since it satisfies the differential equation ( 5.88) as is
seen below.
d At d At
x& (t) = [e k] = [e ]k
dt dt
Using property (5.87) of the matrix exponential, we obtain
x& (t) = AeAt k = Ax(t)
To evaluate the constant vector k in terms of the known initial state x(t0), we substitute t = t0 in Eqn. (5.89):
x(t0) = eAt0 k
Using property (5.86) of the matrix exponential, we obtain
k = (eAt0) –1x(t0) = e– At0x(t0)
Thus, the general solution to Eqn. (5.88) for the state x(t) at time t, given the state x(t0) at time t0, is
x(t) = eAt e–At0 x(t0) = eA(t – t0) x(t 0) (5.90a)
We have used the property (5.85) of the matrix exponential to express the solution in this form.
If the initial time t0 = 0, i.e., the initial state x0 is known at t = 0, we have from Eqn. (5.90a):
x(t) = eAtx(0) (5.90b)
D
From Eqn. (5.90b), it is observed that the initial state x(0) = x0 at t = 0 is driven to a state x(t) at time t.
This transition in state is carried out by the matrix exponential eAt. Due to this property, eAt is known as the
state transition matrix, and is denoted by f (t).
Properties of State Transition Matrix Properties of the matrix exponential, given earlier in Eqns
(5.84)–(5.87), are restated below in terms of state transition matrix f (t).
d
(i) f (t) = A f (t); f (0) = I
dt
(ii) f (t2 – t1) f (t 1 – t0) = f (t2 – t0) for any t0, t1, t2
This property of the state transition matrix is important since it implies that a state transition process
can be divided into a number of sequential transitions. The transition from t0 to t2:
x(t2) = f (t2 – t0)x(t0);
 306 Digital Control and State Variable Methods
is equal to the transition from t0 to t1 and then from t1 to t2:
x(t1) = f (t1 – t0)x(t0)
x(t2) = f (t2 – t1)x(t1)
(iii) f –1(t) = f (–t)
(iv) f (t) is a nonsingular matrix for all finite t.
Evaluation of State Transition Matrix The state transition matrix f (t) = eAt of an n ´ n matrix A, is
given by the infinite series (5.83). The series converges for all A and all finite t. Hence, eAt can be evaluated
within prescribed accuracy by truncating the series at, say, i = N. An algorithm for evaluation of matrix series
is given in Section 6.3.
In the following, we discuss the commonly used methods for evaluating eAt in closed form.
Evaluation Using Inverse Laplace Transforms Taking the Laplace transform on both sides of Eqn. (5.88),
yields
sX(s) – x0 = AX(s)
D D
where X(s) = L [x(t)]; x0 = x(0)
Solving for X(s), we get
X(s) = (sI – A)– 1x0
The state vector x(t) can be obtained by inverse transforming X(s):
x(t) = L – 1[(sI – A)– 1]x0
Comparing this equation with Eqn. (5.90b), we get
eAt = f (t) = L – 1[(sI – A)– 1] (5.91)
–1
The matrix (sI – A) = F(s) is known in mathematical literature as the resolvent of A. The entries of the
resolvent matrix F(s) are rational functions of s. Resolvent matrix F(s) can be expressed in the following
form (refer Eqn. (5.43)):
Q (s) Q1 s n - 1 + Q 2 s n - 2 + L + Q n - 1 s + Q n
F(s) = = (5.92a)
D (s ) sn + a 1 sn - 1 + L + a n - 1 s + a n
where Qi are constant (n ´ n) matrices and aj are constant scalars. The coefficients of the scalar polyno-
mial D(s) and the matrix polynomial Q(s) may be determined sequentially by resolvent algorithm (conven-
ient for digital computer) given in Eqns (5.44).
The inverse transform
L –1[Q(s)/D(s)] = eAt (5.92b)
can be expressed as a power series in t.

Example 5.10 Consider the system

LM0 0 -2 OP 0 LM OP
x& = 0 1 0 x; x(0) = 1
MM 1 PP MM PP
N 0 3 Q 0 NQ
By direct computation, we have
-1
LM s 0 2 OP ( s I - A) +
(sI – A)– 1 = 0 s -1 0 =
MM-1 s-3
PP | s I - A|
N 0 Q
 Control System Analysis Using State Variable Methods 307

2 +
LM(s - 1) (s - 3) 0 -2( s - 1) OP
|sI – A| = (s – 1) (s – 2); (sI – A) (s - 1) (s - 2)
=
MM (s 0- 1) 0
s(s - 1) PQ
P
N 0

LM (s - 3) 0
-2 O
MM (s - 1) (s - 2) ( s - 1) (s - 2) PP
1 PP
eAt = L – 1 [(sI – A)– 1] = L –1
MM 0 (s - 1)
0
PP
MN (s - 1)1(s - 2) 0
s
( s - 1) (s - 2) Q

LM 2 e - e
t 2t
0 2 et - 2 e2t OP
=M 0 et 0 PP
MN - e + e
t 2t
0 2 e - et
2t
Q
Consequently, the free response of the system is

At
LM 0 OP t
x(t) = e x(0) = e
MM 0 PP
N Q
Note that x(t) could be more easily computed by taking the inverse Laplace transform of
X(s) = [(sI – A)– 1x(0)].

Evaluation Using Similarity Transformation Suppose that A is an n ´ n nondiagonal matrix with distinct
eigenvalues l1, l2, ..., ln. We define the diagonal matrix L as

LM l 1 0 0 L 0 OP
0 l2 0 L 0
L= M PP
MM M M M M
PQ
N0 0 0 L ln

A and L are similar matrices; there exists a nonsingular transformation matrix P such that (refer
Eqns (5.22))
L = P –1AP

Now
1 22 1 –1 2 2
P –1eAtP = P –1[I + At + A t + L ] P = I + P –1APt + P A Pt + L
2! 2!
1 –1 1 22
L t + L = eL
t
= I + P –1APt + P APP –1APt 2 + L = I + L t +
2! 2!
 308 Digital Control and State Variable Methods
Lt Lt
Thus the matrices eAt and e are similar. Since L is diagonal, e is given by
LMe l 1t
0 0 L 0 OP
l 2t
0 e 0 L 0
eLt = M PP
MM M M M M
PP
MN 0 0 0 L e l nt Q
The matrix exponential eAt of matrix A with distinct eigenvalues l1, l2, ..., ln may, therefore, be evalu-
ated using the following relation:
LMe l1 t
0 L 0 OP
l2 t
0 e L 0
=P M PP P
Lt
eAt = P e P –1 –1
(5.93)
MM M M M
PP
MN 0 0 L eln t Q
where P is a transformation matrix that transforms A into the diagonal form.
(For the general case wherein matrix A has multiple eigenvalues, refer [105]. Also refer Review Example 5.3
given at the end of this chapter).

Example 5.11 Consider the system

x& =
LM 0 1OP x; x(0) = LM0OP
N-2 -3Q N1Q
The characteristic equation for this system is
l -1
|lI – A| = =0
2 (l + 3)
or (l + 1)(l + 2) = 0
Therefore, the eigenvalues of system matrix A are
l1 = – 1, l2 = – 2
Eigenvectors v1 and v2 corresponding to the eigenvalues l1 and l2, respectively, can be determined from
the adjoint matrix (lI – A)+ (refer Eqn. (5.79)).

(lI – A)+ =
LM(l + 3) 1 OP
N -2 l Q
For l = l = – 1,
1

(l I – A) = M
1
L 2 1OP ; v = LM 1OP
+
1
N-2 -1Q N-1Q
For l = l = – 2,
2

(l I – A) = M
2
L 1 1OP ; v = LM 1OP
+
2
N -2 -2 Q N -2 Q
The transformation matrix P that transforms A into diagonal form, is

P= M
L 1 1OP
N-1 -2Q
 Control System Analysis Using State Variable Methods 309
The matrix exponential

eAt = P
LMe-t
0 OP P = L 1 1 O LMe
–1
-t
0 OP L 2 1O
N0 e -2 t
Q MN-1 -2PQ N0 e -2 t Q MN-1 -1PQ
=
LM 2e -t
- e -2 t e - t - e -2 t OP
-t -2 t -t -2 t
N -2 e + 2e -e + 2e Q
Consequently, the free response of the system is

x(t) = eAtx(0) =
LM e -t
- e -2 t OP
-t -2 t
N- e + 2e Q
Evaluation Using Cayley-Hamilton Technique The state transition matrix may be evaluated using a
technique based on the Cayley-Hamilton theorem. To begin, with, we restate the theorem proved earlier in
Section 5.4 (refer Eqns (5.45)–(5.46)).
Every square matrix A satisfies its own characteristic equation.
Thus if we have, for an n ´ n matrix A, the characteristic equation
D(l) = |lI – A| = ln + a1ln – 1 + L + a n – 1 l + an = 0,
then, according to this theorem
D(A) = An + a1 An– 1 + L + an–1 A + an I = 0
where I is an identity matrix and 0 is a null matrix.
This theorem provides a simple procedure for evaluating the function of a matrix. In the study of linear
systems, we are mostly concerned with functions which can be represented as a series of the powers of a
matrix. Consider the matrix polynomial
f (A) = a0I + a1A + a2A2 + L + anAn + an + 1An + 1 + L (5.94a)
This matrix polynomial, which is of degree higher than the order of A, can be computed by consideration
of the scalar polynomial
f (l) = a0 + a1l + a2l2 + L + anln + an + 1ln + 1 + L (5.94b)
Dividing f (l) by the characteristic polynomial D(l), we get
f(l ) g(l )
= q(l) + (5.95a)
D(l ) D(l )
where g(l) is the remainder polynomial of the following form:
g(l) = b0 + b1l + L + bn – 1ln – 1 (5.95b)
Equation (5.95a) may be written as
f (l) = q(l)D(l) + g(l) (5.96)
Assume that the n ´ n matrix A has n distinct eigenvalues l1, l2, ..., ln;
D(li) = 0; i = 1, 2, ..., n
If we evaluate f (l) in Eqn. (5.96) at the eigenvalues l1, l2, ..., ln, we have
f (li) = g(li), i = 1, 2, ..., n (5.97)
 310 Digital Control and State Variable Methods
The coefficients b0, b1, ... , bn–1 in Eqn. (5.95b) can be computed by solving the set of n simultaneous
equations obtained by successively substituting l1, l2, ... , ln in Eqn. (5.97).
Substituting A for l in Eqn. (5.96), we get
f (A) = q(A)D(A) + g(A)
Since D(A) is identically zero, it follows that
f(A) = g(A) = b0I + b1A + L + bn – 1 An – 1
If A possesses an eigenvalue lk of multiplicity nk, then only one independent equation can be obtained by
substituting lk into Eqn. (5.97). The remaining (nk –1) linear equations, which must be obtained in order to
solve for bi’s, can be found by differentiating both sides of Eqn. (5.97). Since

LM d j
D( l )
OP = 0 ; j = 0, 1, ..., (nk – 1),
j
N dl Q l = lk

it follows that

LM dj
f (l )
OP =
LM dj
g( l )
OP ; j = 0, 1, ... , (nk – 1)
j j
N dl Q l = lk N dl Q l = lk

The formal procedure of evaluation of the matrix polynomial f(A) is given below.
D
(i) Compute D(l) = |lI – A|
(ii) Find the roots of D(l) = 0, say
D(l) = (l – l1)n1 (l – l2)n2 L (l – lm)nm (5.98a)
where n1 + n2 + L + nm = n. In other words, D(l) has root li with multiplicity ni. If li is a complex
number, then its complex conjugate is also a root of D(l).
(iii) Form a polynomial g(l ) of degree (n – 1); i.e.,
g(l) = b0 + b1l + L + bn – 1ln – 1 (5.98b)
where the unknown parameters b0, b1, ..., bn – 1 are to be solved in Step (v).
(iv) Form the following n equations:

LM d j
O
f (l ) P =
LM d j
g( l )
OP ;
j = 0, 1, K , (ni – 1)
(5.98c)
j i = 1, 2, K , m
N dl j
Q l = li N dl Q l = li

(v) Solve for the n unknown parameters b0, b1, ... , bn – 1 from the n equations in Step (iv).
Then
f(A) = g(A) = b0I + b1A + L + bn – 1An – 1 (5.98d)

Example 5.12 Find f (A) = A10 for

A=
LM 0 1OP
N-2 -3Q
Solution The characteristic polynomial is
l -1
D(l) = |lI – A| = = (l + 1)(l + 2)
2 l +3
 Control System Analysis Using State Variable Methods 311
The roots of D(l) = 0 are l1 = – 1, l2 = – 2.
Since A is of second order, the polynomial g(l) will be of the following form:
g(l) = b0 + b1l
The coefficients b0 and b1 are evaluated from equations
f(l1) = (l1)10 = g(l1) = b0 + b1l1
f(l2) = (l2)10 = g(l2) = b0 + b1l2
The result is
b0 = – 1022, b1 = – 1023
Therefore

f(A) = A10 = b0I + b1A =

LM-1022 -1023OP
N 2046 2047Q

The Cayley-Hamilton technique allows us to solve the problem of evaluation of eAt, where A is a constant
n ´ n matrix. Since the matrix power series
A2 t 2 An t n
eAt = I + At + +L+ +L
2! n!
converges for all A and for all finite t, the matrix polynomial f(A) = eAt can be expressed as a polynomial
g(A) of degree (n – 1). This is illustrated below with the help of an example.

Example 5.13 Consider the system

x& = Ax
with
LM0 0 -2 OP LM0OP
A= 0 1 0 ; x(0) = 1
MM 1 3QP
P MM0PP
N
In the following, we evaluate the function
0 NQ
f(A) = eAt
using the Cayley-Hamilton technique.
The characteristic polynomial of matrix A is
l 0 2
D(l) = |lI – A| = 0 (l - 1) 0 = (l – 1)2 (l – 2)
-1 0 (l - 3)
The characteristic equation D(l) = 0 has a second-order root at l1 = 1 and a simple root at l2 = 2.
Since A is of third order, the polynomial g (l) will be of the form
g(l) = b0 + b1l + b2l2
The coefficients b0, b1, and b2 are evaluated using the following relations:
f(l1) = g(l1)
d d
f(l ) = g( l )
dl l = l1 dl l = l1

f(l2) = g(l2)
 312 Digital Control and State Variable Methods
These relations yield the following set of simultaneous equations.
et = b0 + b1 + b2
tet = b1 +2b2
e2t = b0 + 2b1 + 4b2
Solving these equations, we obtain
b0 = – 2tet + e2t
b1 = 3tet + 2et – 2e2t, and
b2 = e2t – et – tet
Hence, we have
eAt= g(A) = b0I + b1A + b2A2
= (– 2tet + e2t)I + (3tet + 2et – 2e2t)A + (e2t – et – tet)A2
LM2e t
- e2 t 0 2e t - 2e 2 t OP
=M 0 et 0 PP
MN-e t
+ e2 t 0 2e - e t
2t
Q
Consequently, the free response (u(t) = 0) of the system is

At
LM 0 OP t
x(t) = e x(0) = e
MM 0 PP
N Q
This result is identical to the one obtained earlier in Example 5.10.

5.7.3 Solution of Non-homogeneous State Equation

When an input u(t) is present, the complete solution x(t) is obtained from the non-homogeneous
equation (5.80).
By writing Eqn. (5.80) as
x& (t) – Ax(t) = bu(t)
and premultiplying both sides of this equation by e– At, we obtain
e– At [ x& (t) – Ax(t)] = e– At bu(t) (5.99)
By applying the rule for the derivative of the product of two matrices, we can write (refer Eqn. (5.87))
d – At d d – At
[e x(t)] = e–At (x(t)) + (e )x(t) = e–At x& (t) – e– At Ax(t)
dt dt dt
= e– At [ x& (t) – Ax(t)]
Use of this equality in Eqn. (5.99), gives
d – At
[e x(t)] = e– Atbu(t)
dt
Integrating both sides with respect to t between the limits 0 and t, we get
t t
e–Atx(t)
0
= z
0
e - At bu (t)dt
 Control System Analysis Using State Variable Methods 313
t

or e–Atx(t) – x(0) = z
0
e - At bu(t) dt

Now, premultiplying both sides by eAt, we have

t
At
x(t) = e x(0) + z0
eA(t – t)bu(t) dt (5.100)

If the initial state is known at t = t0, rather than t = 0, Eqn. (5.100) becomes
t
x(t) = e
A t ( –t0 )
x(t0) + z
t0
eA(t – t)bu(t)dt (5.101)

Equation (5.101) can also be written as

t
x(t) = f (t – t0) x(t0) + z
t0
f (t – t ) bu(t) dt (5.102)

where
f (t) = eAt
Equation (5.102) is the solution of Eqn. (5.80). This equation is called the state transition equation. It
describes the change of state relative to the initial conditions x(t0) and the input u(t).

Example 5.14 For the speed control system of Fig. 5.3, the following plant model was derived in Example 5.1
(refer Eqns. (5.17)):
x& = Ax + bu
y = cx
with
A=
LM-1 OP ; b = LM 0OP ; c = [ 1 0 ]
1
N-1 -10Q N10Q
State variables x1 and x2 are the physical variables of the system:
x1(t) = w (t), angular velocity of the motor shaft
x2(t) = ia(t), armature current
The output
y(t) = x1(t) = w (t)
In the following, we evaluate the response of this system to a unit step input, under zero initial conditions.

(sI – A)–1 =
LMs + 1 -1 OP = 1
-1
LMs + 10 1 OP
N 1 s + 10Q s + 11s + 11 N -1 s + 1Q 2

LM s + 10 1 OP
(s + a )(s + a ) (s + a )( s + a )
= M
MM (s + a -)(1s + a ) (s + as )(+s1+ a ) PPP ; a = 1.1125, a
1 2 1 2
1 2 = 9.8875

N 1 2 Q 1 2
 314 Digital Control and State Variable Methods
eAt = L –1[(sI – A)–1]

=
LM1.0128e- a1 t
- 0.0128e - a2 t 0.114e - a1 t - 0 .114e - a2 t OP
- a1 t - a2 t - a1 t - a2 t
N -0.114e + 0 .114e -0 .0128e + 1.0128e Q
u(t) = 1; t ³ 0; x(0) = 0
Therefore,
t t LM . de
114 - a1 ( t - t )
- e - a2 (t - t ) i OP
x(t) = z
0
eA(t – t )bdt = z
0
MN114
. d - 0.1123e - a1 ( t - t )
+ 8.8842e - a2 ( t - t )
iPQ
dt

=
LM 0.9094 - 1.0247 e - a1 t
+ 0.1153 e - a2 t OP
- a1 t
N- 0.0132 + 0.1151e - 0.1019 e - a2 t Q
The output
y(t) = w(t) = 0.9094 – 1.0247 e– 1.1125t + 0.1153e– 9.8875t; t ³ 0

5.8 CONCEPTS OF CONTROLLABILITY AND OBSERVABILITY

Controllability and observability are properties which describe structural features of a dynamic system.
These properties play an important role in modern control system design theory; the conditions on control-
lability and observability often govern the control solution.
To illustrate the motivation of investigating controllability and observability properties, we consider the
problem of the stabilization of an inverted pendulum on a motor-driven cart.

Example 5.15 Figure 5.16 shows an inverted pendulum with its pivot mounted on a cart. The cart is driven
by an electric motor. The motor drives a pair of wheels of the cart; the whole cart and the pendulum become the
‘load’ on the motor. The motor at time t exerts a torque T(t) on the wheels. The linear force applied to the cart
is u(t); T(t) = Ru(t), where R is the radius of the wheels.
The pendulum is obviously unstable. It can, however, be kept
upright by applying a proper control force u(t). This somewhat
artificial system example represents a dynamic model of a space
booster on take off—the booster is balanced on top of the rocket
engine thrust vector.
dm From inspection of Fig. 5.16, we construct the differential
r

equations describing the dynamics of the inverted pendulum and

the cart. The horizontal displacement of the pivot on the cart
with respect to the fixed nonrotating frame, is z(t), while the
rotational angle of the pendulum is q (t). The parameters of the
system are as follows.
M = the mass of the cart
L = the length of the pendulum = 2l
m = the mass of the pendulum
J = the moment of inertia of the pendulum with respect to
Fig. 5.16 Inverted pendulum system centre of gravity (CG)
 Control System Analysis Using State Variable Methods 315
3 l
l l
LM r OP FG 2l IJ = ρA(2l)FG l IJ
3 2
J= z
-l
z
r 2 dm = r 2 ( rAdr ) = rA
-l N3Q -l
= ρA
H3K H 3K
ml 2
=
3
where A = area of cross-section, and r = density
The horizontal and vertical positions of the CG of the pendulum are given by (z + l sinq ) and (l cosq ),
respectively.
The forces exerted on the pendulum are—the force mg on the centre of gravity, a horizontal reaction
force H and a vertical reaction force V (Fig. 5.17a). H is the horizontal reaction force that the cart exerts on
the pendulum, whereas –H is the force exerted by the pendulum on the cart. Similar convention applies to
forces V and –V.

H
CG

mg
z
V u
0 Pivot H
(a) (b)
Fig. 5.17

Taking moments around CG of the pendulum, we get

d 2 q( t )
J = V(t) l sin q(t) – H(t) l cos q(t) (5.103a)
dt 2
Summing up all forces on the pendulum in vertical and horizontal directions, we obtain
d2
m (l cosq (t)) = V(t) – mg (5.103b)
dt 2
d2
m 2 (z(t) + l sinq (t)) = H(t) (5.103c)
dt
Summing up all the forces on the cart in the horizontal direction (Fig. 5.17b), we get
d 2 z( t )
M = u( t ) - H ( t ) - Fc (5.103d)
dt 2
where
Fc = Bc sign( z&) (5.103e)
is the model of the frictional force of the cart wheels on the track; Bc is the cart friction coefficient.
Performing the required differentiation in Eqns (5.103b) and (5.103c), we get
ml(-q& 2 cos q - q&& cos q ) = V - mg (5.104a)
dz + l( -q& sin q + q&& cos q ) = H
m && 2
i (5.104b)
Substituting (5.103d) into (5.104b), gives
mz&& + mlq&& cos q - mlq& 2 sin q + Fc = u - Mz&& (5.104c)
 316 Digital Control and State Variable Methods
Now, substituting Eqns (5.104a) and (5.103d) into (5.103a), yields
&2 &&
Jq&& = mg - mlq cos q - mlq sin q l sin q + Fc l cos q - (u - Mz&&)l cos q (5.104d)
We next substitute (u - M &&)
z from (5.104c) into (5.104d) and perform manipulations to get
Jq&& = mgl sin q - ml q&& - mzl
2
&& cos q (5.104e)
1
Let a=
m+M

Then, we can represent (5.104e) as

z = – malq&& cos q + malq& 2 sin q - aFc + au
&& (5.104f)

We substitute (5.104f ) into (5.104e), to obtain

mgl sin q - ( m 2 l 2 aq& 2 sin 2q ) / 2 + ( mal cos q ) Fc - ( mal cos q ) u
q&& = (5.104g)
J - m 2 l 2 a cos2 q + ml 2

We next substitute q&& from (5.104e) into (5.104f) to get

-(m 2 l 2 ag sin 2q )/ 2 + (malq& 2 sin q + a(u - Fc ))( J + ml 2 )
&&
z = (5.104h)
J + ml 2 - m 2 l 2 a cos 2 q
1 2
Since J = ml , Eqns (5.104g) and (5.104h) reduce to the following nonlinear set of equations.
3
g sin θ – (mlaθ& 2 sin 2θ )/ 2 + a cos θ ( Fc – u)
q&& = (5.105a)
4l / 3 – mla cos 2θ
-( mag sin 2q )/ 2 + ( aq sin q ) 4 ml / 3 + (u - Fc )4 a / 3
&&
z= (5.105b)
4 / 3 - ma cos 2 q
Suppose that the system parameters are:
M = 1 kg; m = 0.15 kg; and l = 0.5 m.
Recall that g = 9.81 m/sec2.
In our problem, since the objective is to keep the pendulum upright, it seems reasonable to assume that q& (t )
and q(t ) will remain close to zero. In view of this, we can set with sufficient accuracy sin q ; q; cos q ; 1.
Also, the second-order deviations q ´ q ; 0; q& ´ q& ; 0 . We further assume, for simplified analysis, that F = 0.
c
With these assumptions, we have from Eqns (5.105)
q&& (t) = 16.3106 q (t) – 1.4458 u(t)
&&z (t) = – 1.0637 q (t) + 0.9639 u(t)
Choosing the states x1 = q, x2 = q& , x3 = z, and x4 = z& , we obtain the following state model for the inverted
pendulum on moving cart.
x& = Ax + bu (5.106)
with
LM 0 1 0 0 OP LM
0 OP
16.3106 0 0 0 -1.4458
A= MM 0 0 0 1
PP
;b= MM
0
PP
MN
-1.0637 0 0 0
PQ MN
0.9639
PQ
 Control System Analysis Using State Variable Methods 317
The plant (5.106) is said to be completely controllable if every state x(t0) can be affected or controlled to
reach a desired state in finite time, by some unconstrained control u(t). Shortly, we will see that the plant
(5.106) satisfies this condition and, therefore, a solution exists to the following control problem:
Move the cart from one location to another without causing the pendulum to fall.
The solution to this control problem is not unique. We normally look for a feedback control scheme so
that the destabilizing effects of disturbance forces (due to wind, for example) are filtered out. Figure 5.18a
shows a state-feedback control scheme for stabilizing the inverted pendulum. The closed-loop system is
formed by feeding back the state variables through a real constant matrix k;
u(t) = – kx(t)
The closed-loop system is thus described by
x& (t) = (A – bk)x(t)
The design objective in this case is to find the feedback matrix k such that the closed-loop system is
stable. The existence of a solution to this design problem is directly based on the controllability property of
the plant (5.106). This will be established in Chapter 7.
Implementation of the state-feedback control solution requires access to all the state variables of the plant
model. In many control situations of interest, it is possible to install sensors to measure all the state variables.
This may not be possible or practical in some cases. For example, if the plant model includes non-physical
state variables, measurement of these variables using physical sensors is not possible. Accuracy require-
ments or cost considerations may prohibit the use of sensors for some physical variables also.
The input and the output of a system are always physical quantities, and are normally easily accessible to
measurement. We, therefore, need a subsystem that performs the estimation of state variables based on the
information received from the input u(t) and the output y(t). This subsystem is called an observer whose
design is based on observability property of the controlled system.
The plant (5.106) is said to be completely observable if all the state variables in x(t) can be observed from
the measurements of the output y(t) = q (t) and the input u(t). Shortly, we will see that the plant (5.106) does
not satisfy this condition and, therefore, a solution to the observer-design problem does not exist when the
inputs to the observer subsystem are u(t) and q (t).
Cart position z(t) is easily accessible to measurement and, as we shall see, the observability condition is
satisfied with this choice of input information to the observer subsystem. Figure 5.18b shows the block
diagram of the closed-loop system with an observer that estimates the state vector from measurements of u(t)
and z(t). The observed or estimated state vector, designated as x$ , is then used to generate the control u
through the feedback matrix k.

u x = Ax + bu z
Plant
z = cx
u x
x = Ax + b u x
Observer

E k E k

(a) (b)
Fig. 5.18 Control system with state feedback
 318 Digital Control and State Variable Methods

A study of controllability and observability properties, presented in this section, provides a basis for the
state-feedback design problems discussed in Chapter 7. Further, these properties establish the conditions for
complete equivalence between the state variable and transfer function representations.

5.8.1 Definitions of Controllability and Observability

In this section, we study the controllability and observability of linear time-invariant systems described by
state variable model of the following form:
x& (t) = Ax(t) + bu(t) (5.107a)
y(t) = cx(t) + du(t) (5.107b)
where A, b, c and d are respectively n ´ n, n ´ 1, 1 ´ n and 1 ´ 1 matrices, x(t) is n ´ 1 state vector, y(t) and
u(t) are respectively output and input variables.
Controllability For the linear system given by Eqns (5.107), if there exists an input u[0, t1] which transfers
D
the initial state x(0) = x0 to the state x1 in a finite time t1, the state x0 is said to be controllable. If all initial
states are controllable, the system is said to be completely controllable, or simply controllable. Otherwise,
the system is said to be uncontrollable.
From Eqn. (5.100), the solution of Eqn. (5.107a) is
t
x(t) = eAtx0 + z
0
eA(t – t ) bu(t ) dt

To study the controllability property, we may assume, without loss of generality, that x1 º 0. Therefore,
if the system (5.107) is controllable, there exists an input u[0, t1] such that
t1

– x0 = z
0
e–At bu(t ) dt (5.108)

From this equation, we observe that complete controllability of a system depends on A and b, and is
independent of output matrix c. The controllability of the system (5.107) is frequently referred to as the
controllability of the pair {A, b}.
It may be noted that according to the definition of controllability, there is no constraint imposed on the
input or on the trajectory that the state should follow. Further, the system is said to be uncontrollable
although it may be ‘controllable in part’.
From the definition of controllability, we observe that by complete controllability of a plant we mean that
we can make the plant do whatever we please. Perhaps this definition is too restrictive in the sense that we
are asking too much of the plant. But if we are able to show that system equations satisfy this definition,
certainly there can be no intrinsic limitation on the design of the control system for the plant. However, if the
system turns out to be uncontrollable, it does not necessarily mean that the plant can never be operated in a
satisfactory manner. Provided that a control system will maintain the important variables in an acceptable
region, the fact that the plant is not completely controllable, is immaterial.
Another important point which the reader must bear in mind, is that almost all physical systems are
nonlinear in nature to a certain extent, and a linear model is obtained after making certain approximations.
Small perturbations of the elements of A and b may cause an uncontrollable system to become controllable.
It may also be possible to increase the number of control variables and make the plant completely controlla-
ble (controllability of multi-input systems will be discussed in Section 5.10).
 Control System Analysis Using State Variable Methods 319
A common source of uncontrollable state variable models arises when redundant state variables are
defined. No one would intentionally use more state variables than the minimum number needed to character-
ize the behaviour of the dynamic system. In a complex system with unfamiliar physics, one may be tempted
to write down differential equations for everything in sight and, in doing so, may write down more equations
than are necessary. This will invariably result in an uncontrollable model for the system.
Observability For the linear system given by Eqns (5.107), if the knowledge of the output y and the input
D
u over a finite interval of time [0, t1] suffices to determine the state x(0) = x0, the state x0 is said to be
observable. If all initial states are observable, the system is said to be completely observable, or simply
observable otherwise, the system is said to be unobservable.
The output of the system (5.107) is given by
t

z
y(t) = c eAt x0 + c eA(t – t )bu(t )dt + du(t)
0
The output and the input can be measured and used, so that the following signal h (t) can be obtained from
u and y.
t

T
z
h(t) =D y(t) – c eA(t – t )b u(t )dt – d u(t) = c eAtx0
0
(5.109)

Premultiplying by eA t cT and integrating from 0 to t1, gives

RS e
t1
U t1

Tz z
AT t T T
c ce At
dt Vx 0
= e A t c T h( t ) dt (5.110)
0 W 0
When the signal h(t) is available over a time interval [0, t1], and the system (5.107) is observable, then the
initial state x0 can be uniquely determined from Eqn. (5.110).
From Eqn. (5.110) we see that complete observability of a system depends on A and c, and is independent
of b. The observability of the system (5.107) is frequently referred to as the observability of the pair {A, c}.
Note that the system is said to be unobservable, although it may be ‘observable in part’. Plants that are not
completely observable can often be made observable by making more measurements (observability of multi-
output systems will be discussed in Section 5.10). Alternately, one may examine feedback control schemes
which do not require complete state feedback.

5.8.2 Controllability Tests

It is difficult to guess whether a system is controllable or not from the defining equation (5.108). Some
simple mathematical tests which answer the question of controllability, have been developed. The following
theorem gives two controllability tests.
Theorem 5.2 The necessary and sufficient condition for the system (5.107) to be completely controllable is
given by any one of the following:
t1

I. W(0, t1) = z0
e–Atb bT e-A
Tt
dt (5.111)

is nonsingular.
 320 Digital Control and State Variable Methods
II. The n ´ n controllability matrix
D
U = [b Ab A2b L An – 1b] (5.112)
has rank equal to n, i.e., r(U) = n.
Since Test II can be computed without integration, it allows the controllability of a system to be easily
checked.
Proof of Test I
Sufficiency: If W(0, t1) given in Eqn. (5.111) is nonsingular, the input
Tt
u(t) = – bT e-A W–1(0, t1)x0 (5.113)
can be applied to the system. This input satisfies the condition given in Eqn. (5.108):
t1 t1

z
0
z T
e - At bu(t ) dt = - e - At bb T e - A t W -1 (0, t1 )x 0 dt
0

R| t1
U|
=– S|
T z
0
T
e - At b b T e-A tdt W –1(0, t1)x0 = – x0
V|
W
Necessity: Assume that the system is controllable, though W(0, t1) is singular for any t1. Then, as per the
results given in Eqns (5.8), the n rows of e– Atb are linearly dependent, i.e., there exists a nonzero n ´ 1 vector
a such that
aTe– At b = 0 (5.114)
From the assumption of controllability, there exists an input u satisfying Eqn. (5.108); therefore from
Eqns (5.108) and (5.114),
t1

– a Tx0 = z
0
a Te– Atb u(t) dt = 0 (5.115)

holds for any initial state x0. By choosing x0 = a, Eqn. (5.115) gives (refer Eqn. (5.6a)),
a T a = [|| a ||]2 = 0
This is true only for a = 0, which contradicts the nonzero property of a . Therefore, the nonsingularity of
W(0, t1) is proved.
Proof of Test II
Sufficiency: It is first assumed that though r (U) = n, the system is not controllable, and by showing that
this is a contradiction, the controllability of the system is proved.
By the above assumption,
r (U) = n and W(0, t1) is singular.
Therefore Eqn. (5.114) holds, i.e.,
a Te– At b = 0; t ³ 0, a ¹ 0
Derivatives of the above equation at t = 0, yield (refer Eqn. (5.87)),
a TAk b = 0; k = 0, 1, ..., (n – 1)
which is equivalent to
a T [b Ab ××× An – 1b] = aT U = 0
 Control System Analysis Using State Variable Methods 321
Therefore, n rows of controllability matrix U are linearly dependent (refer Eqn. (5.8a)). This contradicts
the assumption that r(U) = n; hence the system is completely controllable.
Necessity: It is assumed that the system is completely controllable but r (U) < n. From this assumption,
there exists nonzero vector a satisfying
aTU = 0
or
a TAk b = 0; k = 0, 1, ..., (n – 1) (5.116a)
– At
Also from the Cayley-Hamilton theorem, e can be expressed as a linear combination of I, A, ..., An–1:
e– At = b0I + b1A + L + bn – 1 An–1 (5.116b)
From Eqns (5.116a) and (5.116b), we obtain
a Te– Atb = 0, t ³ 0, a ¹ 0
and therefore (refer Eqns (5.8)),
t1

z
0
Tt
aTe– Atb bT e-A a dt = a TW(0, t1) a = 0

Since the system is completely controllable, W(0, t1) should be nonsingular from Test I; this contradicts
the assumption that a is nonzero. Therefore, r(U) = n.

Example 5.16 Recall the inverted pendulum of Example 5.15, shown in Fig. 5.16, in which the object is
to apply a force u(t) so that the pendulum remains balanced in the vertical position. We found the linearized
equations governing the system to be:
x& = Ax + bu
where x = [q q& z z& ]T
LM 0 1 0 0 OP LM 0 OP
16.3106 0 0 0 -1.4458
A= M
MM 0 0 0 1P
P ;b= M
M 0
PP
N-1.0637 0 0 0Q
P MN 0.9639PQ
z(t) = horizontal displacement of the pivot on the cart
q (t) = rotational angle of the pendulum.
To check the controllability of this system, we compute the controllability matrix U:
LM 0 -1.4458 0 -23.5816 OP
-1.4458 0 - 23.5816 0
U = [b Ab A b A b] = M
2 3
MM 0 0.9639 0 1.5379
PP
N 0.9639 0 1.5379 0
PQ
Since |U| = 420.4851, U has full rank, and by Theorem 5.2, the system is completely controllable. Thus if
the angle q departs from equilibrium by a small amount, a control always exists which will drive it back to
zero.10 Moreover, a control also exists which will drive both q and z, as well as their derivatives, to zero.

10
This justifies the assumption that q(t ) @ 0 , provided we choose an appropriate control strategy.
 322 Digital Control and State Variable Methods

It may be noted that Eqn. (5.113) suggests a control law to prove the sufficiency of the controllability test.
It does not necessarily give an acceptable solution to the control problem. The open-loop control given by
Eqn. (5.113) is normally, not acceptable. In Chapter 7, we will derive a state-feedback control law for the
inverted pendulum. As we shall see, for such a control to exist, complete controllability of the plant is a
necessary requirement.

Example 5.17 Consider the electrical network shown in Fig. 5.19. Differential equations governing the
dynamics of this network, can be obtained by various standard methods. By use of nodal analysis, for example,
we get
de e - e2 e - e0
C1 1 + 1 + 1 =0
dt R3 R1
de2 e - e1 e - e0
C2 + 2 + 2 =0
+ R1 R2 dt R3 R2
e0 The appropriate state variables for the network are the capacitor
R3
voltages e1 and e2. Thus, the state equations of the network are
e1 e2
C1 C2 x& = Ax + be0
where x = [e1 e2]T

LM- F 1 + 1 I 1 1 OP LM
1 OP
Fig. 5.19 GH R R JK C
A= M P
R3 C1 RC
MM
1
1
3 1
FG 1 IJ PP
1 1
MM
; b= 11 1 PP
- +
MN R C 3 2 H R2 K PQ
R3 C2 MN
R2 C2 PQ
The controllability matrix of the system is

LM 1 -
1
+
1 FG
1
-
1 IJ OP
U = [b Ab] = M
RC
1 1 ( R1C1 ) 2
H
R3 C1 R2 C2 R1C1 KP
MM 1 -
1
+
1 FG
1
-
1 IJ PP
MN R C
2 2 ( R2 C2 ) 2
H
R3 C2 R1C1 R2 C2 K PQ
We see that under the condition
R1C1 = R2C2
r (U) = 1 and the system becomes ‘uncontrollable’. This condition is the one required to balance the bridge,
and in this case, the voltage across the terminals of R3 cannot be influenced by the input e0.

5.8.3 Observability Tests

The following theorem gives two observability tests.
Theorem 5.3 The necessary and sufficient condition for the system (5.107) to be completely observable, is
given by any one of the following:
 Control System Analysis Using State Variable Methods 323
t1
I. z
M(0, t1) = e A tcTceAtdt
0
T
(5.117)

is nonsingular.
II. The n ´ n observability matrix
LM c OP
cA
V= M
D
MM M PP (5.118)

NcA n- 1 PQ
has rank equal to n, i.e., r(V) = n.
Proof Using the definning equation (5.110), this theorem can be proved in a manner similar to Theorem 5.2.

Example 5.18 We now return to the inverted pendulum of Example 5.16. Assuming that the only output
variable to be measured is q (t), the position of the pendulum, then the linearized equations governing the
system are
x& = Ax + bu
y = cx
LM 0 1 0 0 OP LM 0 OP
-1. 4458
where A= MM160.3106 0
0
0
0
0
1P
P ;b= M
M 0
PP
MN-1.0637 0 0 0Q
P MN 0.9639PQ
c= [1 0 0 0]
The observability matrix
LM c OP LM 1 0 0 0 OP
V= MMcAcA PP = MM160.3106
2
1
0
0
0
0
0P
P
MNcA PQ MN 0
3
16.3106 0 0Q
P
|V| = 0, and therefore, by Theorem 5.3, the system is not completely observable.
Consider now, the displacement z(t) of the cart as the output variable. Then
c = [0 0 1 0]
and the observability matrix
LM 0 0 1 0 OP
V= MM-01.0637 0
0
0
0
1
0
PP
MN 0 -1.0637 0 0
PQ
|V| = 1.1315 ¹ 0; the system is therefore completely observable. The values of z& (t), q (t) and q& (t) can all
be determined by observing z(t) over an arbitrary time interval. Observer design for the inverted-pendulum
system is given in Chapter 7.
 324 Digital Control and State Variable Methods

5.8.4 Invariance Property

It is recalled that the state variable model for a system is not unique, but depends on the choice of a set of
state variables. A transformation
x(t) = P x (t); P is a nonsingular constant matrix,
results in the following alternative state variable model (refer Eqns (5.22)) for the system (5.107):
x& (t) = A x (t) + b u(t); x (t0) = P –1x(t0)
y(t) = c x (t) + du(t)
where
A = P –1AP, b = P –1b, c = cP
The definition of a new set of internal state variables should, evidently, not affect the controllability and
observability properties. This may be verified by evaluating the controllability and observability matrices of
the transformed system.
I. U = [b A b L ( A ) n - 1 b ] (5.119a)
b = P–1b
A b = P–1APP– 1b = P–1Ab
M
( A ) b = A ( A b ) = P–1APP–1Ab = P–1A2b
2

( A ) n - 1 b = P–1An–1b
Therefore,
U = [P–1b P–1Ab L P–1An –1b] = P–1U
where U = [b Ab L An – 1b] (5.119b)
Since P–1 is nonsingular,
r ( U ) = r (U) (5.119c)
II. A similar relationship can be shown for the observability matrices.

5.8.5 Controllability and Observability of State Variable Model in Jordan Canonical

Form
If the system equations are known in Jordan canonical form, then one need not resort to controllability and
observability tests given by Theorems 5.2 and 5.3. These properties can be determined almost by inspection
of the system equations, as will be shown below.
Consider a SISO system with distinct eigenvalues l1, l2, ..., ln. The Jordan canonical state model of this
system is of the form
x& = L x + bu
y = cx + du (5.120)
LMl 1 0 L 0 OPb1 LM OP
0 l2 L 0 b2
with L= M
MM M M M
;b=PPM
MM PP
; c = [c1 c2 L cn]

N0 0 L ln
PQ
bn
MN PQ
Theorem 5.4 The system (5.120) is completely controllable if, and only if, none of the elements of the
column matrix b is zero, and (5.120) is completely observable if, and only if, none of the elements of the row
matrix c is zero.
 Control System Analysis Using State Variable Methods 325
Proof The controllability matrix

U = [b L b L L n – 1b]
LM b1 b1l 1 L b1ln1 - 1 OP
b2 l n2 - 1
=M PP
b
2 b2 l 2 L
MM M M M
PP
MNb
n bn l n L bn l nn - 1 Q
1 l1 L ln1 - 1
1 l2 L ln2 - 1
|U| = b1 ´ b2 ´ L ´ bn ¹ 0 if bi ¹ 0, i = 1, 2, ..., n.
M M M
1 ln L lnn - 1
This proves the first part of the theorem. The second part can be proved in a similar manner.11

5.9 EQUIVALENCE BETWEEN TRANSFER FUNCTION AND STATE VARIABLE

REPRESENTATIONS

In frequency-domain analysis, it is tacitly assumed that the dynamic properties of a system are completely
determined by the transfer function of the system. That this is not always the case is illustrated by the
following examples.

Example 5.19 Consider the system

x& = Ax + bu
y = cx (5.121)

with A=
LM-2 1
;b=
OP
LM OP
1
; c = [0 1]
N1 -2 NQ
1Q
The controllability matrix
L1 -1OP
U = [b Ab] = M
N1 -1Q
Since r(U) = 1, the second-order system (5.121) is not completely controllable. The eigenvalues of
matrix A are the roots of the characteristic equation
s + 2 -1
|sI – A| = =0
-1 s + 2
The eigenvalues are obtained as – 1, – 3. The modes of the transient response are therefore e– t and e– 3t.
The transfer function of the system (5.121) is calculated as

G(s) = c(sI – A)–1b = [0 1]

LMs + 2 -1 OP LM1OP
-1

N -1 s+2 Q N1Q
11
Refer Gopal [105] for controllability and observability tests using Jordan canonical representation of systems with
multiple eigenvalues
 326 Digital Control and State Variable Methods

LM s + 2 1 OP
= [0 1] M
(s + 1)(s + 3)
MM (s + 1)(1 s + 3)
(s + 1)(s + 3)
s+2 PP LMN11OPQ = s +1 1
N (s + 1)(s + 3) PQ
We find that because of pole-zero cancellation, both the eigenvalues of matrix A do not appear as poles in
G(s). The dynamic mode e–3t of the system (5.121), does not show up in input-output characterization given by
the transfer function G(s). Note that the system under consideration is not a completely controllable system.

Example 5.20 Consider the system

x& = Ax + bu
y = cx (5.122)

with A=
LM -2 OP LM OP
1
;b=
1
; c = [1 –1]
N Q NQ
1 -2 0
The observability matrix
L c O L 1 -1 OP
V=M P =M
NcAQ N-3 3 Q
Since r(V) = 1, the second-order system (5.122) is not completely observable.
The eigenvalues of matrix A are – 1, – 3. The transfer function of the system (5.122) is calculated as
G(s) = c(sI – A)– 1b
LM s + 2 1 OP
= [1 –1] ( s + 1) ( s + 3) ( s + 1) ( s + 3)
MM 1 s + 2
1
0
=
1
s + 3
PP LMN OPQ
MN
( s + 1) ( s + 3) ( s + 1) ( s + 3) PQ
The dynamic mode e–t of the system (5.122), does not show up in the input-output characterization given by
the transfer function G(s). Note that the system under consideration is not a completely observable system.

In the following, we give two specific state transformations to reveal the underlying structure imposed
upon a system by its controllability and observability properties (for proof, refer [105]). These results are
then used to establish equivalence between transfer function and state variable representations.
Theorem 5.5 Consider an nth-order system
x& = Ax + bu (5.123a)
y = cx
Assume that
r(U) = r[b Ab L An – 1b] = m < n
Consider the equivalence transformation
x = P x = [ P1 P2] x (5.123b)
where P1 is composed of m linearly independent columns of U, and (n – m) columns of P2 are chosen arbitrar-
ily so that matrix P is nonsingular.
The equivalence transformation (5.123b) transforms the system (5.123a) to the following form:
LM x& OP = LMA
1 c A12 OP L x O + LMb OP u
1 c

Nx& Q N 0
2 A 22 Q MNx PQ N 0 Q
2

= A x + bu (5.123c)
 Control System Analysis Using State Variable Methods 327

y = [c1 c2 ]
LM x OP = c x
1

Nx Q
2
where the m-dimensional subsystem
x& 1 = A c x1 + bc u + A12 x2
is controllable from u (the additional driving term A12 x 2 has no effect on controllability); the (n – m) dimen-
sional subsystem
x& 2 = A 22 x 2
is not affected by the input and is, therefore, entirely uncontrollable.
This theorem shows that any system which is not completely controllable, can be decomposed into
controllable and uncontrollable subsystems shown in Fig. 5.20. The state model (5.123c) is said to be in
controllability canonical form.

u x1
x1= A cx1 + b cu + A 12 x2 c1

y
+

x2
x2= A 22 x2 c2

Fig. 5.20 The controllability canonical form of a state variable model

In Section 5.4, it was shown that the characteristic equations and transfer functions of equivalent systems
are identical. Thus, the set of eigenvalues of matrix A of system (5.123a) is same as the set of eigenvalues of
matrix A of system (5.123c), which is a union of the subsets of eigenvalues of matrices Ac and A 22 . Also the
transfer function of system (5.123a) must be the same as that of (5.123c). The transfer function of (5.123a)
is calculated from Eqn. (5.123c) as 12
-1
G(s) = c1 c2
LMsI - A - A OP LMb OP
c 12 c

N 0 sI - A Q N0Q 22

= c1 c2
LM(s I - A ) (s I - A ) A (s I - A
c
–1
c
–1
12 22 )
–1
OP LMb OP
c

N 0 (s I - A ) 22
–1
Q N0Q
= c1 ( s I - A c ) –1 bc
Therefore, the input-output relationship for the system is dependent only on the controllable part of
the system. We will refer to the eigenvalues of Ac as controllable poles and the eigenvalues of A 22 as
uncontrollable poles.
Only the controllable poles appear in the transfer function model; the uncontrollable poles are cancelled
by the zeros.

12
LMA
1 A2 OP LMB 1 B2 OP = LM I 0OP
N0 A3 Q NB 3 B4 Q N0 I Q
gives
LM
B1 B 2
=
A1-1 OP LM - A1-1 A 2 A 3-1 OP
N
B3 B4 0 Q MN A 3-1 PQ
 328 Digital Control and State Variable Methods

Theorem 5.6 Consider the nth order system

x& = Ax + bu
y = cx (5.124a)
Assume that
LM OP
c
cA
r (V) = r M
MM M PPP = l < n
n -1
NcA Q
Consider the equivalence transformation

x = Qx = M P x
LQ O 1
(5.124b)
NQ Q 2
where Q1 is composed of l linearly independent rows of V, (n – l) rows of Q2 are chosen arbitrarily so that
matrix Q is nonsingular.
The equivalence transformation (5.124b) transforms the system (5.124a) to the following form:
LM x& OP = LM A
1 0 0OP L x O + LM b OP u = A x + b u
1 1

Nx& Q NA
2 21 A Q MNx PQ
22 2 Nb Q
2
(5.124c)
y = [ c0
Lx O 1
0]M P = c x
Nx Q 2
where the l-dimensional subsystem
x& 1 = A 0 x1 + b1 u
y = c 0 x1
is observable from y, and the (n – l)-dimensional subsystem
x& 2 = A 22 x2 + b2 u + A 21 x1
has no effect upon the output y, and is therefore entirely unobservable, i.e., nothing about x 2 can be inferred
from output measurement.
This theorem shows that any system which is not completely observable, can be decomposed into the
observable and unobservable subsystems shown in Fig. 5.21. The state model (5.124c) is said to be in
observability canonical form.

u x1 y
x1= A 0x1 + b 1u c0

x2= A 22 x2 + b2 u + A 21 x1

Fig. 5.21 The observability canonical form of a state variable model

Since systems (5.124a) and (5.124c) are equivalent, the set of eigenvalues of matrix A of system (5.124a)
is same as the set of eigenvalues of matrix A of system (5.124c), which is a union of the subsets of
eigenvalues of matrices A 0 and A 22 . The transfer function of the system (5.124a) may be calculated from
(5.124c) as follows:
 Control System Analysis Using State Variable Methods 329

G(s) = [ c 0
Ls I - A
0]M 0 0 OP LM b OP -1
1

N - A s I - A Q Nb Q
21 22 2

= [c0
L
0]M
( sI - A ) 0
-1
0 OP LM b OP
1
-1 -1 -1
N ( s I - A )
22 A ( sI - A )
21 0 ( sI - A 22 ) Q Nb Q
2

= c0 ( s I - A 0 ) - 1 b1 (5.125)

which shows that the unobservable part of the system does not affect the input-output relationship. We will
refer to the eigenvalues of A0 as observable poles and the eigenvalues of A 22 as unobservable poles.
We now examine the use of state variable and transfer function models of a system to study its dynamic
properties.
We know that a system is asymptotically stable if all the eigenvalues of the characteristic matrix A of its
state variable model, are in the left-half of complex plane. Also, we know that a system is (bounded-input
bounded-output) BIBO stable if all the poles of its transfer function model are in the left-half of complex
plane. Since, in general, the poles of the transfer function model of a system are a subset of the eigenvalues
of the characteristic matrix A of the system, asymptotic stability always implies BIBO stability.
The reverse, however, may not always be true because the eigenvalues of the uncontrollable and/or
unobservable part of the system are hidden from the BIBO stability analysis. These may lead to instability of
a BIBO stable system. When a state variable model is both controllable and observable, all the eigenvalues
of characteristic matrix A appear as poles in the corresponding transfer function. Therefore BIBO stability
implies asymptotic stability only for completely controllable and completely observable system.
To conclude, we may say that the transfer function model of a system represents its complete dynamics
only if the system is both controllable and observable.

5.10 MULTIVARIABLE SYSTEMS

Many of the analysis results developed in earlier sections of this chapter for SISO systems have obvious
extensions for MIMO systems.
Consider a general MIMO system shown in the block diagram of Fig. 5.22. The input variables are
represented by u1, u2, ..., up, and the output variables by y1, y2, ..., yq. The state, as in the case of SISO
systems, is represented by variables x1, x2, ..., xn.
The state variable model for a MIMO system takes the following form:
0
x& (t ) = Ax(t) + Bu(t); x(t0) =D x (5.126a)
u1 y1
y(t) = Cx(t) + Du(t) (5.126b)
u2 MIMO y2
system where
up yq LMa 11 a12 L a1n OP
b11 b12 L b1 p LM OP
a a22 L a2 n b21 b22 L b2 p
A= M
MM M
21
M M
;B=
M
PP
M M
MM PP
x1 x2 xn
Na n1 an 2 L ann
PQ
bn1 bn 2 L bnp
MN PQ
Fig. 5.22 A general MIMO system
 330 Digital Control and State Variable Methods

LMc 11 c12 L c1n OP LMd 11 d12 L d1 p OP

c c22 L c2 n d d 22 L d2 p
C= M
MM M
21
M M P
P ,D= M
MM M
21
M M
PP
Nc q1 cq 2 L c Q
qn
P Nd q1 dq 2 L d qp
PQ
A, B, C and D are respectively n ´ n, n ´ p, q ´ n, and q ´ p constant matrices; x is n ´ 1 state vector, u is
p ´ 1 input vector, and y is q ´ 1 output vector.
The solution of the state equation (5.126a) is given by (refer Eqn. (5.100))
t
x(t) = eAt x(0) + z
0
eA(t – t) Bu(t ) dt (5.127a)

The output

LM t OP
y(t) = C e At x ( 0 ) +
MN z
0
e A( t - t ) Bu(t ) dt + Du(t)
PQ
(5.127b)

In the transform domain, the input-output behaviour of the system (5.126) is determined entirely by the
matrix (refer Eqn. (5.28))
G(s) = C(sI – A)–1 B + D (5.128a)
This matrix is called the transfer function matrix of system (5.126), and it has the property that the input
U(s) and output Y(s) of Eqns (5.126) are related by
Y( s ) = G( s) U( s) (5.128b)
( q ´1) ( q ´ p ) ( p ´1)

whenever x0 = 0.
In an expanded form, Eqn. (5.128b) can be written as

LMY (s) OP LM G (s)

1 11 G12 ( s) L G1 p ( s) OP LMU (s) OP1
G22 ( s) L G2 p ( s )
MMY M(s)PP = MM G M(s)
2 21
M M
P U (s)
PP MM M PPP
M 2

MNY (s)PQ MN G (s)

q q1 Gq 2 ( s) L G ( s ) Q NU ( s )Q
qp p

The (i, j)th element Gij(s) of G(s) is the transfer function relating the ith output to the jth input.

Example 5.21 The scheme of Fig. 5.23 describes a simple concentration control process. Two concentrated
solutions of some chemical with constant concentrations C1 and C2 are fed with flow rates Q1(t) = Q1 + q1(t),
and Q2(t) = Q2 + q2(t), respectively, and are continuously mixed in the tank. The outflow from the mixing
tank is at a rate Q(t) = Q + q(t) with concentration C(t) = C + c(t). Let it be assumed that stirring causes
perfect mixing so that the concentration of the solution in the tank is uniform throughout, and equals that of
the outflow. We shall also assume that the density remains constant.
Let V(t) = V + v(t) be the volume of the fluid in the tank.
 Control System Analysis Using State Variable Methods 331

Flow Concentration Reference inputs

Controller

Q 1, C 1 Q 2, C 2

Feed 1 Feed 2

V
H
C
Q, C Flow
sensor
Outgoing stream

Concentration
analyser

Fig. 5.23 A concentration control process

The mass balance equations are

d
[ V + v(t)] = Q1 + q1(t) + Q2 + q2(t) – Q – q(t) (5.129a)
dt
d
[{ C + c(t)}{ V + v(t)}] = C1[ Q1 + q1(t)] + C2[ Q2 + q2(t)] – [ C + c(t)][ Q + q(t)] (5.129b)
dt
The flow Q(t) is characterized by the turbulent flow relation
V (t )
Q(t) = k H (t ) = k (5.130)
A
where H(t) = H + h(t) is the head of the liquid in the tank, A is the cross-sectional area of the tank and k is a
constant.
The steady-state operation is described by the equations (obtained from Eqns (5.129) and (5.130))
0 = Q1 + Q2 - Q
0 = C1Q1 + C2 Q2 - C Q

V
Q = k
A
For small perturbations about the steady-state, Eqn. (5.130) can be linearized using Eqn. (5.11c):
k ¶ V (t )
Q(t) – Q = (V(t) – V )
A ¶ V (t ) V= V
 332 Digital Control and State Variable Methods

k V Q
or q(t) = v(t) = v(t)
2V A 2V
From the foregoing equations, we obtain the following relations describing perturbations about
steady-state:
1Q
v& (t) = q1(t) + q2(t) – v(t) (5.131a)
2 V
1 CQ
C v& (t ) + Vc&( t ) = C1 q1(t) + C2 q2(t) – v(t) – Q c(t) (5.131b)
2 V
(Second-order terms in perturbation variables have been neglected)
The hold-up time of the tank is
V
t=
Q
Let us define
x1(t) = v(t), x2(t) = c(t), u1(t) = q1(t), u2(t) = q2(t), y1(t) = q(t), and y2(t) = c(t)
In terms of these variables, we get the following state model from Eqns (5.131):
LM- 1 0 OP L 1 1 OP
x& (t) = M t
2 x(t) + M C − C C2 − C u(t) (5.132a)
1P 1
MN 0 t PQ MN V
- V
PQ
L 1 0OP x(t)
y(t) = M 2t (5.132b)
MN 0 1 PQ
For the parameters
Q1 = 10 litres/sec, Q2 = 20 litres/sec, C1 = 9 g-moles/litre, C2= 18 g-moles/litre, V = 1500 litres,
the state variable model becomes
x& (t) = Ax(t) + Bu(t) (5.133a)
y(t) = Cx(t) (5.133b)
with

A=
LM-0.01 0 OP
;B=
1 1 LM ;C=
0.01 0 OP LM OP
N 0 -0.02 Q-0.004 0.002
N 0 1 Q N Q
In the transform domain, the input-output behaviour of the system is given by
Y(s) = G(s) U(s)
where
G(s) = C(sI – A)–1B
For A, B, and C given by Eqns (5.133), we have

(sI – A) =
LMs + 0.01 0 OP
N 0 s + 0.02 Q
 Control System Analysis Using State Variable Methods 333

L L 1
0.01 0O M s + 0.01 0 OP L 1 1 OP
–1
G(s) = C(sI – A) B =M
N 0 1PQ MMN 0 1 PP MN-0.004 0.002 Q
s + 0.02 Q

LM 0.01 0.01 OP
= M s + 0.01 s + 0.01 P (5.134)
MN s-+0.004 0.002
P
0.02 s + 0.02 Q

Controllability Test The necessary and sufficient condition for the system (5.126) to be completely
controllable, is that the n ´ np matrix
D
U = [B AB A2B L An – 1B] (5.135)
has rank equal to n, i.e., r (U) = n.

Observability Test The necessary and sufficient condition for the system (5.126) to be completely
observable, is that the nq ´ n matrix

LM C OP
CA
V= M
D
MM M PP (5.136)

NCA n -1 PQ
has rank equal to n, i.e., r (V) = n.

Controllability and Observability of State Variable Model in Jordan Canonical Form The con-
trollability and observability properties can be determined by the inspection of the system equations in
Jordan canonical form. A MIMO system with distinct eigenvalues l1, l2, ..., ln has the following Jordan
canonical state model.
x& = L x + Bu (5.137a)
y = Cx + Du (5.137b)
with

LMl 1 0 L 0 b11 b12 L b1 p

OP LM OP
0 l2 L 0 b21 b22 L b2 p
c11 c12 L c1n LM OP
L=M
MM M ;B=
M
PP M M
MM
;C= M M M PP MM PP
M M
N0 0 L ln
PQ
bn1 bn 2 L bnp MNcq1 cq 2 L cqn PQ N Q
The system (5.137) is completely controllable if, and only if, none of the rows of B matrix is a zero
row, and (5.137) is completely observable if, and only if, none of the columns of C matrix is a zero
column.
 334 Digital Control and State Variable Methods

We have been using Jordan canonical structure only for systems with distinct eigenvalues. Refer [105] for
controllability and observability tests using Jordan canonical representation of systems with multiple
eigenvalues.

Example 5.22 Consider the mixing-tank system discussed in Example 5.21. Suppose the feeds Q1 and Q2
have equal concentrations, i.e., C1 = C2 = C0 (Fig. 5.23). Then the steady-state concentration C in the tank
is also C0, and from Eqn. (5.132a) we have
LM- 1 0 OP L1 1OP u(t)
x& (t) = M 2t P x(t) + M
1 N0 0 Q
MN 0 - P
tQ
This state variable model is in Jordan canonical form. Since one row of the B matrix is a zero row, the
system is not completely controllable. As is obvious from the Jordan canonical model, the input u(t) affects
only the state variable x1(t), the incremental volume. The variable x2(t), the incremental concentration, has
no connection with the input u(t).
If C1 ¹ C2, the system is completely controllable.

Review Examples

Review Example 5.1 A feedback system has a closed-loop transfer function

Y ( s) 10(s + 4)
=
R( s) s( s + 1)( s + 3)
Construct three different state models for this system:
(a) One where the system matrix A is a diagonal matrix.
(b) One where A is in first companion form.
(c) One where A is in second companion form.
Solution
(a) The given transfer function can be expressed as
Y (s) 10(s + 4) 40/3 -15 5/3
= = + +
R( s ) s(s + 1)(s + 3) s s +1 s +3
Therefore,
40/3 -15 5/3
Y(s) = R(s) + R(s) + R(s)
s s +1 s+3
40 / 3 40
Let X1(s) = R(s); this gives x&1 = r
s 3
-15
X2(s) = R(s); this gives x& 2 + x2 = – 15r
s +1
 Control System Analysis Using State Variable Methods 335
5/ 3 5
X3(s) = R(s); this gives x& 3 + 3x3 = r
s+3 3
In terms of x1, x2 and x3, the output y(t) is given by
y(t) = x1(t) + x2(t) + x3(t)
A state variable formulation, for the given transfer function, is defined by the following matrices:
LM
0 0 0 40 / 3 OP LM OP
L = 0 -1 0 ; b = -15 ; c = [1 1 1]; d = 0
MM
0 0 -3
PP MM PP
N 5/ 3 Q N Q
Note that the coefficient matrix A is diagonal, and the state model is in Jordan canonical form.
We now construct two state models for the given transfer function in companion form. To do this, we
express the transfer function as
Y ( s) 10(s + 4) 10 s + 40 b s 3 + b 1s 2 + b 2 s + b 3
= = 3 = 03 ;
2
R( s) s( s + 1)( s + 3) s + 4 s + 3s s + a 1s 2 + a 2 s + a 3
b0 = b1 = 0, b2 = 10, b3 = 40, a1 = 4, a2 = 3, a3 = 0
(b) With reference to Eqns (5.54), we obtain the following state model in the first companion form:
LM0 1 0 OP LM0OP
A= 0 0 1 ; b = 0 ; c = [40 10 0]; d = 0
MM
0 -3 - 4
PP MM PP
N 1 Q NQ
(c) With reference to Eqns (5.56), the state model in second companion form becomes
LM0 0 0 OP LM40OP
A = 1 0 - 3 ; b = 10 , c = [0 0 1]; d = 0
0 1 -4
MM PP MM PP
N 0 Q N Q
Review Example 5.2 A linear time-invariant system is characterized by the homogeneous state equation
LM x& OP = LM0
1 1OP LM x OP1

N x& Q N0
2 -2Q N x Q 2

(a) Compute the solution of the homogeneous equation assuming the initial state vector

x(0) =
LM1OP
N0Q
Employ both the Laplace transform method and the canonical transformation method.
(b) Consider now that the system has a forcing function and is represented by the following non-
homogeneous state equation:
LM x& OP = LM0
1 1 OP LM x OP + LM0OP u
1

N x& Q N0
2 -2 Q N x Q N1Q 2

where u is a unit-step input.

 336 Digital Control and State Variable Methods
Compute the solution of this equation assuming initial conditions of part (a).
Solution
(a) Since

(sI – A) =
LM s -1 OP
N0 s + 2Q
we obtain

LM1 1 OP
–1
= Ms s( s + 2 )
(sI – A)
MN0 1 PP
s+2 Q
Hence

eAt = L –1 [(sI – A)–1] =

LM1 1
2 (1 - e -2 t ) OP
-2 t
N0 e Q
To obtain the state transition matrix eAt by the canonical transformation method, we compute the
eigenvalues and eigenvectors of matrix A. The roots of the characteristic equation
|lI – A| = 0
are l1 = 0, and l2 = – 2. These are the eigenvalues of matrix A. Eigenvectors corresponding to the
distinct eigenvalues l i, may be obtained from the non-zero columns of adj(l iI – A).
For the given A matrix

adj(lil – A) =
LMl + 2 1 OP i

N 0 lQ i

For l1 = 0, adj(l I – A) = M
1
L2 1OP
N0 0 Q
The eigenvector v1 corresponding to the eigenvalue l1 is therefore, given by

v1 =
LM1OP
N0Q
For l2 = – 2, adj (l2I – A)=
LM0 1 OP
N0 -2 Q
The eigenvector v2 corresponding to the eigenvalue l2 is given by

v2 =
LM 1OP
N-2Q
The transformation matrix

P=
LM1 1OP
N0 -2 Q
 Control System Analysis Using State Variable Methods 337
gives

P–1AP = L =
LM0 OP
0
N0 -2Q
The state transition matrix (refer Eqn. (5.93))

eAt = PeL t P–1 =

LM1 1 OP LMe 0
0 OP LM1 1
2 OP = LM1 1
2 (1 - e -2 t ) OP
-2 -2 t 1 -2 t
N0 Q N0 e Q N0 - 2 Q N0 e Q
x(t) = eAt
L1O
x(0) = M P
N0Q
t
(b) x(t) = eAtx(0) + z
0
eA(t – t )bu(t)dt

Now
LM t
OP
t
bu(t )dt = MM
1
2 z [1 - e -2 ( t - t ) ]dt
PP = LM- 1
+ 12 t + 14 e -2 t OP
z eA(t – t ) 0
t
PP MN
4
1
2 (1 - e
-2 t
) PQ
0 MM
N z
0
e -2( t - t ) dt
Q
Therefore,
1 1 –2t
x1(t) = - 14 + 2 t+ 4e

x2(t) = 1
2 (1 – e–2t)

Review Example 5.3 Given

LMl 1 1 0 L 0 OP
MM 0M l1 1 L 0 PP
L = M M M
n´ n MM 0 0 0 L 1
PP
MN 0 0 0 L l1 PQ
Compute eLt using the Cayley-Hamilton technique.
Solution Equations (5.98) outline the procedure of evaluation of matrix exponential using the Cayley-
Hamilton technique.
The matrix L has n eigenvalues at l = l1. To evaluate f ( L ) = e Lt , we define (refer Eqn. (5.98b)) the
polynomial g(l) as
g(l) = b0 + b1l + L + bn – 1ln – 1
This polynomial may be rearranged as
g(l) = b0 + b1(l – l1) + L + bn – 1(l – l1)n – 1
The coefficients b0, b1, K, bn – 1 are given by the following equations (refer Eqns (5.98c)):
f(l1) = g(l1)
338 Digital Control and State Variable Methods

d d
f (l ) = g(l )
dl l = l1 dl l = l1

M
n -1
d d n -1
f (l ) = g(l )
dln -1 l = l1
dln -1 l = l1

Solving, we get

b0 = el1t
t l 1t
b1 = e
1!
t 2 l 1t
b2 = e
2!
M
t n -1
bn – 1 = e l 1t
( n - 1) !
Therefore,
eL t = b0I + b1( L – l1I) + L + bn –1(L – l1I)n – 1

LM0 1 0 0 L 0OP
0 0 1 0 L 0
(L – l I) = M 1
MM M M M M M
PP
N0 0 0 0 L 0
PQ
LM0 0 1 0 L 0O
0 0 0 1 L 0P
( L – l I)( L – l I) = M
1 1
P
MM M M M M MP
P
N0 0 0 0 L 0Q

M
LMb 0 b1 b2 L bn -1 OP
0 b0 b1 L bn - 2
e Lt =M PP
MM M M M M
PQ
N0 0 0 L b0

LMe l 1t
te l 1t t 2 e l 1t/ 2 ! L t n -1e l 1t/( n - 1) ! OP
l 1t l 1t
0 e te L t n - 2 e l 1t/(n - 2 ) !
=M PP
MM M M M M
PP
l 1t
MN 0 0 0 L e Q
 Control System Analysis Using State Variable Methods 339

Review Example 5.4 The motion of a satellite in the equatorial (r, q) plane is given by [122] the state
equation
LM x& OP LM 0
1
2
1 0 0 OP LM x OP LM0
1 0 OP
MM xx&& PP = MM3w0
2 0
0
0 2w
0 1
PP MM xx PP + MM10
2 0
0
PP LMuu OP = Ax + Bu
1

3
MN x& PQ MN 0
4 -2w 0 0
3
PQ MN x PQ MN0
4 1
PQ N Q
2

where w is the angular frequency of the satellite in circular, equatorial orbit; x1(t) and x3(t) are, respectively,
the deviations in position variables r(t) and q(t) of the satellite; and x2(t) and x4(t) are, respectively, the
deviations in velocity variables r& (t) and q& (t). The inputs u1(t) and u2(t) are the thrusts ur and uq in the radial
and tangential directions, respectively, applied by small rocket engines or gas jets (u = 0 when x = 0).
(a) Prove that the system is completely controllable.
(b) Suppose that the tangential thruster becomes inoperable. Determine the controllability of the system
with the radial thruster alone.
(c) Suppose that the radial thruster becomes inoperable. Determine the controllability of the system with
the tangential thruster alone.
(d) Prove that the system is completely observable from radial (x1 = r) and tangential (x3 = q ) position
measurements.
(e) Suppose that the tangential measuring device becomes inoperable. Determine the observability of the
system from radial position measurement alone.
(f) Suppose that the radial measurements are lost. Determine the observability of the system from tangen-
tial position measurement alone.
Solution
(a) The controllability matrix
U = [B AB A2B A3B]
Consider the matrix:
LM0 0 1 OP
0
1 0 0 2w
U1 = [B AB] = M
MM0 0 0 1
PP
N0 1 -2w 0Q
P
|U1| = – 1
Therefore r(U1) = r(U) = 4; the system is completely controllable.
(b) With u2 = 0, the B matrix becomes
LM0OP
1
b=M P
MM0PP
N0Q
The controllability matrix
340 Digital Control and State Variable Methods

LM0 1 0 -w 2 OP
2
1 0 -w 0
U = [b Ab A2b A b] = M
3 PP
MM0 0 -2w 0
PPQ
MN0 -2w 0 2w 3
1 0 -w 2
|U| = – 0 -2w 0 = –[– 2w (2w3 – 2w3)] = 0
-2w 0 2w 3
Therefore, r (U) < 4, and the system is not completely controllable with u1 alone.
(c) With u1 = 0, the B matrix becomes
0 LM OP
0
b=M P
MM0PP
N1Q
The controllability matrix
LM0 0 2w 0 OP
0 2w 0 - 2w 3
U=M
MM0 1 0 - 4w 2
PP
N1 0 - 4w 2 0
PQ
0 2w - 2w 3
|U| = 2w 0 1 - 4w 2 = –12w 4 ¹ 0
1 0 0
Therefore, r(U) = 4, and the system is completely controllable with u2 alone.
(d) The observability matrix
C LM OP
CA
V=M
MMCA PPP
2

3
NCA Q
Taking radial and tangential position measurements as the outputs, we have
y1 = x1; y2 = x3

or y = Cx =
LM1 0 0 0
x
OP
N0 0 1 0 Q
Consider the matrix
LM1 0 0 0 OP
LC O 0
V =M P =M
1
0 1 0
PP
NCAQ MM0 1 0 0
PQ
N0 0 0 1
|V1| ¹ 0
 Control System Analysis Using State Variable Methods 341
Therefore, r(V1) = r(V) = 4, and the system is completely observable.
(e) With x3 = 0, the C matrix becomes
c = [1 0 0 0]
The observability matrix
LM c OP LM 1 0 0 0 OP
cA 0 1 0 0
V= M 2
P =M
MMcA PP MM 3w 2
0 0 2w
PP
3
NcA Q N 0 -w 2 0 0
PQ
|V| = 0
Therefore, r(V) < 4, and the system is not completely observable from y1 = x1 alone.
(f) With x1 = 0, the C matrix becomes
c = [0 0 1 0]
The observability matrix
LM 0 0 1 0 OP
0 0 0 1
V=M
MM 0 - 2w 0 0
PP
N - 6w 3
0 0 - 4w 2
PQ
|V| = – 12w4 ¹ 0
Therefore, r (V) = 4, and the system is completely observable from y2 = x3 alone.

Problems

5.1 Figure P5.1 shows a control scheme for controlling the azimuth angle of a rotating antenna. The plant consists of an
armature-controlled dc motor with dc generator used as an amplifier. The parameters of the plant are given below:
Motor torque constant, KT = 1.2 N-m/amp
Motor back emf constant, Kb = 1.2 V/(rad/sec)
Generator gain constant, Kg = 100 V/amp
Motor to load gear ratio, n = (q& L / q& M ) = 1/2

Rg Lg Ra La

qM
eg eb
qL

J, B
Lf
Rf
u
+ –
Fig. P5.1
342 Digital Control and State Variable Methods

Rf = 21 W, Lf = 5H, Rg = 9 W, Lg = 0.06 H, Ra = 10 W, La = 0.04 H,

J = 1.6 N-m/(rad/sec2), B = 0.04 N-m/(rad/sec), motor inertia and friction are negligible.
Taking physically meaningful and measurable variables as state variables, derive a state model for the system.
5.2 Figure P5.2 shows a position control system with state variable feedback. The plant consists of a field-
controlled dc motor with a dc amplifier. The parameters of the plant are given below:
Amplifier gain, KA = 50 volt/volt
Motor field resistance, Rf = 99 W
Motor field inductance, Lf = 20 H
Motor torque constant, KT = 10 N-m/amp
Moment of inertia of load, J = 0.5 N-m/(rad/sec2)
Coefficient of viscous friction of load, B = 0.5 N-m/(rad/sec)
Motor inertia and friction are negligible.
Taking x1 = q, x2 = q& , and x3 = if as the state variables, u = ef as the input, and y = q as the output, derive a
state variable model for the plant.

x1 = q
Potentiometer

Ia
–
u = ef Rf
r (t) + KA
–
–
Lf J, B

x 3 = if

R=1 W

x2 = q
Tachogenerator

Fig. P5.2

5.3 Figure P5.3 shows the block diagram of a motor-driven, single-link robot manipulator with position and veloc-
ity feedback. The drive motor is an armature-controlled dc motor; ea is armature voltage, ia is armature current,
qM is the motor shaft position and q& M is motor shaft velocity. qL is the position of the robot arm.
Taking qM, q& M and ia as state variables, derive a state model for the feedback system.
5.4 Figure P5.4 shows the block diagram of a speed control system with state variable feedback. The drive motor is
an armature-controlled dc motor with armature resistance Ra, armature inductance La, motor torque constant
KT, inertia referred to motor shaft J, viscous friction coefficient referred to motor shaft B, back emf constant
Kb, and tachogenerator constant Kt. The applied armature voltage is controlled by a three-phase full-converter.
We have assumed a linear relationship between the control voltage ec and the armature voltage ea, er is the
reference voltage corresponding to the desired speed.
Taking x1 = w (speed) and x2 = ia (armature current) as the state variables, u = er as the input, and y = w as the
output, derive a state variable model for the feedback system.
 Control System Analysis Using State Variable Methods 343

k2

0.5

– – qL
ea 1 ia 1 1 qM 1
38
+ 2s + 21 2s + 1 s 20
+

k1 qM

–
+

qR
Fig. P5.3
ia
k2

–
er + + ec + 1 1 w
k1 Kc KT
sL a + R a Js + B
– –
ea Kb

Kt

Fig. P5.4

5.5 Consider the system

x& =
LM -3 1OP x + LM0OP u
N-2 0Q N1Q
y = [1 0] x
A similarity transformation is defined by

x = Px =
LM 2 -1OP x
N-1 1Q
(a) Express the state model in terms of the states x (t).
(b) Draw state diagrams in signal-flow graph form for the state models in x(t) and x (t).
(c) Show by Mason’s gain formula that the transfer functions for the two state diagrams in (b) are equal.
5.6 Consider a double-integrator plant described by the differential equation
d 2q( t )
= u(t)
dt 2
344 Digital Control and State Variable Methods

(a) Develop a state equation for this system with u as the input, and q and q& as the state variables x1 and x2,
respectively.
(b) A similarity transformation is defined as

x = Px =
1 0
x
LM OP
1 1 N Q
Express the state equation in terms of the states x (t).
(c) Show that the eigenvalues of the system matrices of the two state equations in (a) and (b), are equal.
5.7 A system is described by the state equation
LM
0 1 0 OP
0 LM OP
&x = 0 0 1 x + 0 u; x(0) = x 0
MM PP MM PP
-1 0 -3
N 1Q NQ
Using the Laplace transform technique, transform the state equation into a set of linear algebraic equations in
the form
X(s) = G(s)x0 + H(s)U(s)
5.8 Give a block diagram for the programming of the system of Problem 5.7 on an analog computer.
5.9 The state diagram of a linear system is shown in Fig. P5.9. Assign the state variables, and write the dynamic
equations of the system.
1

1 s–1 1 s–1 s–1 2 1 1

U Y
–1 –2

–1
–1
Fig. P5.9

5.10 Construct a state model for the system of Fig. P5.10.

3
y1 (0)
+ – +
u1 + + + +
3
y1
– +

2
4
–
+ + +
u2 3 y2
– +

y2(0)

4
Fig. P5.10
 Control System Analysis Using State Variable Methods 345
5.11 Derive transfer functions corresponding to the following state models:

(a) x& =
LM 0 1OP x + LM1OP u; y = [1 0] x (b) x& =
LM-3 1OP x + LM0OP u; y = [1 0] x
N-2 -3Q N0Q N-2 0Q N1Q
5.12 Derive the transfer function matrix corresponding to the following state model, using resolvent algorithm.
LM 2 -1 0OP LM-1 0 OP
x& =
MM-11 01 21PP x + MM 01 0 u
2 QP
P
N Q N
y=M
L1 1 0OP x
N1 0 1Q
5.13 Figure P5.13 shows the block diagram of a control system with state variable feedback and integral control. The
plant model is
LM x& OP = LM-3 2OP LM x OP + LM1OP u
1 1
N x& Q N 4 -5Q N x Q N0Q
2 2

y = [0 1] x
(a) Derive a state model of the feedback system.
(b) Derive the transfer function Y(s)/R(s).

r + x3 + u y
(× )dt – 3.5 Plant
– – –
x1 x2
2

1.5

Fig. P5.13

5.14 Construct state models for the systems of Fig. P5.14a and Fig. P5.14b, taking outputs of simple lag blocks as
state variables.

1
+ y
u + 1 + s+1
– s+1
u + 1 + + y
1 s
–
s+2
1
(a)
s+2

(b)
Fig. P5.14
346 Digital Control and State Variable Methods
5.15 Derive a state model for the two-input, two-output feedback control system shown in Fig. P5.15. Take outputs
of simple lags as state variables.

r1 + – u1 1 y1
+
K1
s+1
+

5
s+5

0.4
s + 0.5

r2 + + y2
u2 4 +
K2
– s+2

Fig. P5.15
5.16 Construct state models for the following transfer functions. Obtain different canonical form for each system.
s+3 5 s 3 + 8s 2 + 17s + 8
(i) (ii) (iii)
s 2 + 3s + 2 ( s + 1)2 ( s + 2) ( s + 1) ( s + 2) ( s + 3)
Give block diagrams for the analog computer simulation of these transfer functions.
5.17 Construct state models for the following differential equations. Obtain a different canonical form for each system.
(i) &&&
y + 3&&
y + 2 y& = u& + u (ii) &&&
y + 6 y&& + 11y& + 6y = u (iii) &&&
y + 6 y&& + 11y& + 6y = &&&
u + 8u&& + 17u& + 8u
5.18 Derive two state models for the system with transfer function
Y (s ) 50(1 + s / 5)
=
U ( s) s(1 + s / 2) (1 + s / 50)
(a) One for which the system matrix is a companion matrix.
(b) One for which the system matrix is diagonal.
5.19 (a) Obtain state variable model in Jordan canonical form for the system with transfer function
Y (s) 2s2 + 6s + 5
=
U ( s ) ( s + 1)2 ( s + 2)
(b) Find the response y(t) to a unit-step input using the state variable model in (a).
(c) Give a block diagram for analog computer simulation of the transfer function.
5.20 Find the eigenvalues and eigenvectors for the following matrices.

LM OP
1 1 LM
-3 2 OP 0 1 LM
0 OP
(i) (ii) (iii) 3 0 2
0 2N Q -1 0N Q -12 -7 -6
MM PP
N Q
5.21 (a) If l1, l2, L, ln are distinct eigenvalues of
0 LM
1 0 L 0 OP
0 0 1 L 0
A = M
MM
M M M
PP
0 0
MM 0 L 1
PP
MN
-a n -a n -1 -a n - 2 L -a 1 PQ
 Control System Analysis Using State Variable Methods 347
prove that the matrix
LM 1 1 L 1 OP
L
Ml
P =M l
1
2
l2
l22 L
ln
l2n
PP
1
MM M M M
PP
MNl n -1
1 ln2-1 L lnn-1 PQ
transforms A into Jordan canonical form.
(b) Using the result in (a), find the eigenvalues and eigenvectors of the following matrix:
LM 0 1 OP
0
A =
MM-240 -260 1
-9 PQ
P
N
5.22 Consider the matrix
LM 0 1 OP
0
A = 0 0 1
MM-2 - 4 - 3 QP
P
N
(a) Suggest a transformation matrix P such that L = P–1AP is in Jordan canonical form.
(b) Matrix L in (a) has complex elements. Real arithmetic is often preferable, and can be achieved by further
transformation. Suggest a transformation matrix Q-such that Q–1LQ has all real elements.
5.23 Given the system

x& =
-4 3 LM
x = Ax
OP
-6 5 N Q
Determine eigenvalues and eigenvectors of matrix A, and use these results to find the state transition matrix.
5..24 Using Laplace transform method, find the matrix exponential eAt for

(a) A =
LM
0 -3 OP (b) A =
LM
0 1 OP
1 -4
N Q -3 - 4
N Q
5.25 Using the Cayley-Hamilton technique, find eAt for

(a) A =
LM
0 1 OP (b) A =
LM
0 2 OP
-6 - 5
N Q -2 - 4
N Q
5.26 Given the system

x& =
-2 1LM x+
1
u
OP LM OP
1 -2N 1 Q NQ
(a) Obtain a state diagram in signal-flow graph form.
(b) From the signal-flow graph, determine the state equation in the form
X(s) = G(s)x(0) + H(s)U(s)
(c) Using inverse Laplace transformation, obtain the
(i) zero-input response to initial condition
0 0
x(0) = [ x1 x 2 ]T;
(ii) zero-state response to unit-step input.
5.27 A linear time-invariant system is described by the following state model:
LM
0 1 0 0 OP LM OP
x& = 0 0 1 x+ 0 u
MM
-6 -11 -6
PP MM PP
N 2 Q NQ
y = [1 0 0] x
348 Digital Control and State Variable Methods
Diagonalize the coefficient matrix of the state model using a similarity transformation, and from there obtain
the explicit solutions for the state vector and output when the control force u is a unit-step function and the
initial state vector is
x(0) = [0 0 2]T
5.28 Consider the system

x& =
LM
0 1
x+
0 OP
u; x(0) =
1 LM OP LM OP
-2 -3N 1 Q 1 NQ NQ
y = [1 0] x
(a) Determine the stability of the system.
(b) Find the output response of the system to unit-step input.
5.29 Find the response of the system

x& =
LM
0 1
x+
OP LM OP
2 1
u; x(0) =
0 LM OP
-2 -3
N Q N Q
0 1 0 NQ
y=M
L1 0O
N1 1PQ x
to the following input:

u(t) = M
Lu (t) OP = LM m (t ) OP ; m (t) is unit-step function.
1
-3 t
Nu (t)Q Ne m (t )Q
2
5.30 Figure P5.30 shows the block diagram of a control system with state variable feedback and feedforward control.
The plant model is

x& =
-3 2 LM x+
1
u
OP LM OP
4 -5 N 0 Q NQ
y = [0 1]x
(a) Derive a state model for the feedback system.
(b) Find the output y(t) of the feedback system to a unit-step input r(t); the initial state is assumed to be zero.

y = x2
r + u
7 Plant
–
x1
3
+

+
1.5

Fig. P5.30

5.31 Consider the state equation

x& =
LM 0 1OP x
N-1 -2Q
Find a set of states x1(1) and x2(1) such that x1(2) = 2.
5.32 Consider the system
LM 0 1 0 OP
x& = 0 2 x
MM-123 -7 -6
PP
N Q
 Control System Analysis Using State Variable Methods 349
(a) Find the modes of the system.
(b) Find the initial condition vector x(0) which will only excite the mode corresponding to the eigenvalue with
the most negative real part.
5.33 Consider the system

x& (t) =
LM OP
0 1
x(t)
N Q
2 1
y(t) = [1 2] x(t)
(a) Show that the system modes are e–t and e2t.
(b) Find a set of initial conditions such that the mode e2t is suppressed in y(t).
5.34 The following facts are known about the linear system
x& (t) = Ax(t).

If x(0) =
LM OP
1 LM
, then x(t) =
e -2 t OP
-2
N Q N-2e -2 t Q
If
L 1O L
x(0) = M P , then x(t) = M
e -t OP
-t
-
N Q
1 Ne- Q
Find eAt and hence A.
5.35 Show that the pair {A, c} is completely observable for all values of a i ’s.
LM0 0 L 0 -an OP
0 L 0 - a n -1
M1
A = M0 1 L 0 - a n-2
PP
MM M M M M
PP
MN0 0 L 1 - a1 PQ
c = [0 0 L 0 1]
5.36 Show that the pair {A, b} is completely controllable for all values of ai’s.
LM
0 1 0 L 0 OP LM0 OP
0 0 1 L 0
A = M
MM M M M ;b=
PP MM0M PP
0
MM 0 0 L 1
PP MM0 PP
MN
-a n -a n -1 -a n - 2 L -a 1 PQ MN1 PQ
5.37 Given the system
-1 0 LM 0 OP
0 1 LM 0 1
OP LM 2 OP
x& = 0 -2 0 x + 2 0 u; y =
MM PP MM PP x
0 0 -3 N 0 1 Q N
0 1
Q N 0 Q
What can we say about controllability and observability—without making any further calculations?
5.38 Determine the controllability and observability properties of the following systems:

(i) A =
-2 1 LM ;b=
OP LM OP
1
; c = [1 –1]
1 -2N Q NQ
0

(ii) A= M
L -1 0 OP ; b = LM2OP ; c = [0 1]
N 0 -2Q N5Q
LM-1 0 0 OP LM1 0OP L1 1 2 OP
(iii) A = M 0 -2 0 ;B= 1 2 ;C= M
MN 0 0 -3 PPQ MMN2 1PPQ N3 1 5 Q
350 Digital Control and State Variable Methods

LM0 1 0OP 0 LM OP
(iv) A = M0 0 1 ; b = 0 ; c = [10
PP 0 0]
MN0 -2 -3
MM PP
Q1 NQ
LM0 0 0O L 40 O
(v) A= 1 0 -3P ; b = M10 P ; c = [0 0 1]
MM0 P MM 0 PP
-4 PQ
N 1 N Q
1
5.39 The following models realize the transfer function G(s) = .
s +1
(i) A =
LM-2 1 OP L O
;b=
1
; c = [0 1]
N1 -2 Q MN PQ
1

(ii) A= M
L-1 0OP LM OP
;b=
1
; c = [1 0]
N0 -3 Q NQ 1
L-2 0O L0 O
(iii) A= M P ; b = M P ; c = [0 1]
N0 -1Q N1 Q
Investigate the controllability and observability properties of these models.
Find a state variable model, for the given transfer function, which is both controllable and observable.
5.40 Consider the systems

(i) A =
LM0 -2OP ; b = LM1OP ; c = [0 1]
N1 -3Q N1Q
LM 0 1 0OP LM0OP
(ii) A= M 0 0 1 ; b = 0 ; c = [4 5 1]
MN-6 -11 -6PPQ MMN1PPQ
Determine the transfer function in each case. What can we say about controllability and observability
properties—without making any further calculations?
5.41 Consider the system
1 1LM 0 0OP LM OP
x& = 0 -2 MM 1 x + 1 u; y = [1 0 0] x
PP
0 0 -1
MM PP
N -2 Q N Q
(a) Find the eigenvalues of A and from there determine the stability of the system.
(b) Find the transfer function model and from there determine the stability of the system.
(c) Are the two results the same? If not, why?
5.42 Given a transfer function
10 Y (s )
G(s) = =
s( s + 1) U ( s)
Construct three different state models for this system:
(a) One which is both controllable and observable.
(b) One which is controllable but not observable.
(c) One which is observable but not controllable.
5.43 Prove that the transfer function
G(s) = Y(s)/U(s)
of the system
x& (t) = Ax(t) + bu(t)
y(t) = cx(t) + du(t)
is invariant under state transformation x(t) = P x (t); P is a constant nonsingular matrix.
 State Variable Analysis of

6 Digital Control Systems

6.1 INTRODUCTION

In the previous chapter of this book, we treated in considerable detail, the analysis of linear continuous-time
systems using state variable methods. In this chapter, we give a condensed review of the same methods for
linear discrete-time systems. Since the theory of linear discrete-time systems—very closely—parallels the
theory of linear continuous-time systems, many of the results are similar. For this reason, the comments in
this chapter are brief, except in those cases where the results for discrete-time systems deviate markedly
from the continuous-time situation. For the same reason, many proofs are omitted.
We will be mostly concerned with single-input, single-output (SISO) system configurations of the
type shown in the block diagram of Fig. 6.1. The plant in the figure, is a physical process characterized by
continuous-time input and output variables. A digital computer is used to control the continuous-time plant.
The interface system that takes care of the communication between the digital computer and the continuous-
time plant consists of analog-to-digital (A/D) converter and digital-to-analog (D/A) converter. In order to
analyse such a system, it is often convenient to represent the continuous-time plant, together with the D/A
converter and the A/D converter, by an equivalent discrete-time system.

Digital set-point Controlled

Digital output
D/A Plant
computer

Sensor

A/D

Fig. 6.1 Basic structure of digital control systems

The discrete-time systems we will come across can, therefore, be classified into two types.
(i) Inherently discrete-time systems (digital processors), where it makes sense to consider the system at
discrete instants of time only, and what happens in between is irrelevant.
 352 Digital Control and State Variable Methods
(ii) Discrete-time systems that result from considering continuous-time systems at discrete instants of
time only.

6.2 STATE DESCRIPTIONS OF DIGITAL PROCESSORS

A discrete-time system is a transformation, or operator, that maps a given input sequence u(k) into an output
sequence y(k). Classes of discrete-time systems are defined by placing constraints on the transformation. As
they are relatively easy to characterize mathematically, and as they can be designed to perform useful signal
processing functions, the class of linear time-invariant systems will be studied here.
In the control structure of Fig. 6.1, the digital computer transforms an input sequence into a form which is,
in some sense, more desirable. Therefore, the discrete-time systems we consider here, are in fact computer
programs. Needless to say, digital computers can do many things other than control dynamic systems; it is
our purpose to examine their characteristics when doing this elementary control task.
State variable model of a SISO discrete-time system consists of a set of first-order difference equations
relating state variables x1(k), x2(k), ..., xn(k) of the discrete-time system to the input u(k); the output y(k) is
algebraically related to the state variables and the input. Assuming that the input is switched on to the system
at k = 0 (u(k) = 0 for k < 0), then the initial state is given by
D x0; a specified n × 1 vector
x(0) =
The dynamics of a linear time-invariant system is described by equations of the form:
D 0
x(k + 1) = Fx(k) + gu(k); x(0) = x (6.1a)
y(k) = cx(k) + du(k) (6.1b)
LM x (k) OP
1
x (k )
x(k) = M
MM M PPP = n × 1 state vector of nth-order system
2
where

N x (k )Q
n

u(k) = system input

y(k) = defined output
LM f f L f OP
11 12 1n
f f L f
F=M
21
MM M M
22 P = n × n constant matrix
2n
M P
N f f L f PQ
n1 n2 nn

LM g OP
1
g
g = M P = n × 1 constant column matrix
2
MM M PP
nNg Q
c = [c1 c2 L cn] = 1 × n constant row matrix
d = scalar, representing direct coupling between input and output
Equation (6.1a) is called the state equation of the system, Eqn. (6.1b) is called the output equation; the two
equations together give the state variable model of the system.
 State Variable Analysis of Digital Control Systems 353
6.2.1 Conversion of State Variable Models to Transfer Functions
In the study of linear time-invariant discrete-time equations, we may also apply the z-transform techniques.
Taking the z-transform of Eqns (6.1), we obtain:
zX(z) – zx0 = FX(z) + gU(z)
Y(z) = cX(z) + dU(z)
where D Z [x(k)]; U(z) D Z [u(k)]; Y(z) D Z [y(k)]
X(z) = = =
Manipulation of these equations, gives
(zI – F) X(z) = zx0 + gU(z); I is n × n identity matrix
or X(z) = (zI – F)–1 zx0 + (zI – F)–1 gU(z) (6.2a)
–1 0 –1
Y(z) = c(zI – F) zx + [c(zI – F) g + d] U(z) (6.2b)
0
Equations (6.2) are algebraic equations. If x and U(z) are known, X(z) can be computed from these equa-
tions.
In the case of zero initial state (i.e., x0 = 0), the input-output behaviour of the system (6.1) is determined
entirely by the transfer function
Y (z)
= G(z) = c(zI – F)–1 g + d (6.3a)
U(z)
( zI - F ) + g
=c +d (6.3b)
| zI - F |
where (zI – F)+ = adjoint of the matrix (zI – F)
|zI – F | = determinant of the matrix (zI – F)
|lI – F | is the characteristic polynomial of matrix F. The roots of this polynomial are the characteristic
roots or eigenvalues of matrix F.
From Eqn. (6.3b), we observe that the characteristic polynomial of matrix F of the system (6.1), is same
as the denominator polynomial of the corresponding transfer function G(z). If there are no cancellations
between the numerator and denominator polynomials of G(z) in Eqn. (6.3b), the eigenvalues of matrix F are
same as the poles of G(z).
In a later section, we shall see that for a completely controllable and observable state variable model, the
eigenvalues of matrix F are same as the poles of the corresponding transfer function.

6.2.2 Conversion of Transfer Functions to Canonical State Variable Models

In Chapters 2– 4, we have seen that transform-domain design techniques yield digital control algorithms in
the form of transfer functions of the form
b 0 z n + b 1z n - 1 + L + b n - 1z + b n
D(z) = (6.4)
z n + a 1z n - 1 + L + a n - 1z + a n
where the coefficients ai and bi are real constant scalars. Equation (6.4) represents an nth-order digital
controller. Several different structures for realization of this controller—using delay elements, adders, and
multipliers—were presented in Section 3.4. Each of these realizations is a dynamic system with n first-order
dynamic elements—the unit delayers. We know that output of a first-order dynamic element represents the
state of that element. Therefore, each realization of Eqn. (6.4) is, in fact, a state diagram; by labelling the
unit-delayer outputs as state variables, we can obtain the state variable model.
 354 Digital Control and State Variable Methods

In what follows, we shall use two of the structures presented in Section 3.4 for obtaining canonical state
variable models corresponding to the general transfer function
n n -1
Y ( z) b 0 z + b 1z + L + b n - 1z + b n
G(z) = = n n -1 (6.5)
U(z) z + a 1z + L + a n - 1z + a n
Revisiting Section 3.4 at this stage will be helpful in our discussion.
First Companion Form A direct realization structure for the system described by Eqn. (6.5) is shown in
Fig. 6.2. Notice that n delay elements have been used in this realization. The coefficients a1, a2, ..., an appear
as feedback elements, and the coefficients b0, b1, ..., bn appear as feedforward elements. To get one state
variable model, we identify the output of each unit delayer with a state variable—starting at the right and
proceeding to the left. The corresponding difference equations are
x1(k + 1) = x2(k)
x2(k + 1) = x3(k)
M (6.6a)
xn – 1(k + 1) = xn(k)
xn(k + 1) = – an x1(k) – an – 1 x2(k) – L – a1 xn(k) + u(k)

+ + +

+ + +

+ +
+ +

Fig. 6.2 A direct realization structure for system given by Eqn. (6.5)

Careful examination of Fig. 6.2 reveals that there are two paths from the output of each unit delayer to the
system output: one path upward through the box labelled bi, and a second path down through the box
labelled ai and thence through the box labelled b0. As a consequence
y(k) = (bn – an b0) x1(k) + (bn–1 – an–1 b0) x2(k) + L + (b1 – a1 b0) xn(k) + b0 u(k) (6.6b)
The state and output equations (6.6), organized in vector-matrix form, are given below.
x(k + 1) = Fx(k) + gu(k)
(6.7)
y(k) = cx(k) + du(k)
 State Variable Analysis of Digital Control Systems 355

LM 0 1 0 L OP L0O
0

M0 0 1 L 0
P MM0PP
with F=M M M M M P ; g = MMP
MM 0 0 0 L 1 P
P MM0PP
NM-a n -a n -1 -a n - 2 L -a QP 1 NM1QP
c = [bn – anb0, bn–1 – an –1 b0, K, b1 – a1b0]; d = b0
The matrix F in Eqns (6.7) has a very special structure—the coefficients of the denominator of the trans-
fer function preceded by minus signs form a string along the bottom row of the matrix. The rest of the matrix
is zero except for the ‘superdiagonal’ terms which are all unity. A matrix with this structure is said to be in
companion form. We call the state variable model (6.7) the first companion form1 state model for the trans-
fer function (6.5); another companion form follows.
Second Companion Form In the first companion form, the coefficients of the denominator of the transfer
function appear in one of the rows of the F matrix. There is another companion form in which the coefficients
appear in a column of the F matrix. This can be obtained from another direct realization structure shown in
Fig. 6.3. We identify the output of each unit delayer with a state variable—starting at the left and proceeding
to the right. The corresponding difference equations are
xn(k + 1) = xn–1(k) – a1(xn(k) + b0u(k)) + b1u(k)
xn–1(k + 1) = xn–2(k) – a2(xn(k) + b0u(k)) + b2u(k)
M
x2(k + 1) = x1(k) – an–1(xn(k) + b0u(k)) + bn–1u(k)
x1(k + 1) = – an(xn(k) + b0u(k)) + bnu(k)
y(k) = xn(k) + b0u(k)
u (k)

n n–1 1 0

+ + + +
+ + + y (k )

– – –
x1 (k ) x n – 1 (k )
n n–1 1 x n (k )

Fig. 6.3 An alternative direct realization structure for the system given by Eqn. (6.5)
The state and output equations, organized in vector-matrix form, are given below.
x(k + 1) = Fx(k) + gu(k)
(6.8)
y(k) = cx(k) + du(k)
1
The pair (F, g) of Eqns (6.7) is completely controllable for all values of ai’s (Refer Problem 5.36).
 356 Digital Control and State Variable Methods

LM0 0 L 0 -a n OP LM
b n - a nb 0 OP
0 L 0 -a n -1
with
M1
F = M0 1 L 0 -a n - 2 ; g =
PP
b n -1 - a n -1 b 0
MM PP
MM M M M M
PP M
MN PQ
b 1 - a 1b 0
NM0 0 L 1 -a 1 PQ
c = [0 0 L 0 1]; d = b0
Comparing the F, g and c matrices of the second companion form2 with that of the first, we observe that
F, g, and c matrices of one companion form correspond to the transpose of F, c, and g matrices, respectively,
of the other.
Both the companion forms of state variable models play an important role in pole-placement design
through state feedback. This will be discussed in Chapter 7.
Jordan Canonical Form In the two canonical forms (6.7) and (6.8), the coefficients of the denominator of
the transfer function appear in one of the rows or columns of matrix F. In another of the canonical forms, the
poles of the transfer function form a string along the main diagonal of the matrix. The canonical form follows
directly from the parallel realization structure of transfer function. We shall first discuss the case where all
poles are distinct. Then we shall consider the case where multiple poles are involved.
Case I: The transfer function involves distinct poles only
Assume that z = li (i = 1, 2, ..., n) are the distinct poles of the given transfer function (6.5). Partial-fraction
expansion of the transfer function gives
Y (z) b z n + b 1 z n - 1 + L + b n -1 z + b n
= G(z) = 0n
U(z) z + a 1 z n -1 + L + a n -1 z + a n
b 1¢ z n -1 + b 2¢ z n - 2 + L + b ¢n
= b0 +
z n + a 1 z n -1 + L + a n
b 1¢ z n -1 + b ¢2 z n - 2 + L + b ¢n
= b0 + = b0 + G¢(z)
( z - l 1 )( z - l 2 ) L ( z - l n )
r1 r2 rn
= b0 + + +L+ (6.9)
z - l1 z - l2 z - ln
The coefficients ri (i = 1, 2, ..., n) are the residues of the transfer function G¢(z) at the corresponding poles
at z = li (i = 1, 2, ..., n). A parallel realization structure of the transfer function (6.9) is shown in Fig. 6.4.
Identifying the outputs of the delayers with the state variables results in the following state and output
equations:
x(k + 1) = L x(k) + gu(k)
(6.10)
y(k) = cx(k) + du(k)
LMl1 0 L 0 OP 1LM OP
0 l2 L 0 1
with L= MMM M M
PP
;g=
M
MM PP
MN
0 0 L ln
PQ 1
MN PQ
c = [r1 r2 L rn]; d = b0

2
The pair {F, c} of Eqns (6.8) is completely observable for all values of ai’s (Refer Problem 5.35).
 State Variable Analysis of Digital Control Systems 357
It is observed that for this canonical state variable model, the matrix L is a diagonal matrix with the poles of
G(z) as its diagonal elements.

+
u (k ) + x1(k) + y (k )
r1
+ +

+ xn(k)
rn
+

Fig. 6.4 A parallel structure realization for the system given by Eqn. (6.9)

Case II: The transfer function involves multiple poles

When the transfer function G(z) involves multiple poles, the partial fraction expansion will not be as
simple as (6.9). In the discussion that follows, we assume that G(z) involves a multiple pole of order m at
z = l1, and that all other poles are distinct. Performing the partial-fraction expansion for this case, we get
Y (z) b z n + b 1 z n -1 + L + b n -1 z + b n
= G(z) = 0 n
U(z) z + a 1 z n -1 + L + a n -1 z + a n
b 1¢ z n -1 + b 2¢ z n - 2 + L + b n¢
= b0 +
z n + a 1 z n -1 + L + a n
b 1¢ z n -1 + b ¢2 z n - 2 + L + b ¢n
= b0 +
( z - l 1 ) m ( z - l m +1 ) L ( z - l n )
= b0 + H1(z) + Hm +1(z) + L + Hn(z) (6.11a)
rm + 1 rn
where Hm +1(z) = , K , Hn (z) = , (6.11b)
z - l m +1 z - ln
r11 r12 r1m
and H1(z) = m
+ m -1
+L+ (6.11c)
(z - l 1 ) (z - l 1 ) (z - l 1 )
A realization of H1(z) is shown in Fig. 6.5. Other terms of Eqn. (6.11a) may be realized as per Fig. 6.4.
Identifying the outputs of the delayers with the state variables results in the following state and output
equations:
x(k + 1) = L x(k) + gu(k)
(6.12)
y(k) = cx(k) + du(k)
 358 Digital Control and State Variable Methods

+ + y1 (k )

+ +

r1m r12 r11

xm (k ) x 2 (k )
u (k ) + + + x 1 (k)

+ + +

1 1 1

Fig. 6.5 A realization of H1(z) given by Eqn. (6.11c)

with m ´ m Jordan block

LMl 1 1 0 L 0 0 L 0 OP L 0 OP
MM 0M l1 1 L 0 0 L 0
P MM 0
PP
M M M M M P MM M
L=M0
M 0 0 L l1 0 L
P
0 P; g = M 1 PP
MM 0 0 0 L 0 l m +1 L 0P
P MM 1 PP
MM M M M M M M P MM M PP
N0 0 0 L 0 0 L l PQ
n N 1 Q
c = [r11 r12 L r1m rm +1 L rn]; d = b0
Note that the L matrix in Eqns (6.12) is block diagonal:

LML 1 0 L 0 OP
0 L m +1 L 0
L=M PP
MM M M M
PQ
N0 0 L Ln

LMl 1 1 0 L 0O
0 l1 1 L 0 P m ´ m sub- matrix corresponding
with L =M P=
1
MM M M M M P to eigenvalue l of multiplicity m
P 1

N0 0 0 L l Q
1

L m + 1 = lm + 1 = 1´ 1 sub-matrix corresponding to simple eigenvalue lm+1

M
L n = ln = 1´ 1 sub-matrix corresponding to simple eigenvalue ln
The matrix L 1 has two diagonals: the principal diagonal has the corresponding pole l1 and the super
diagonal has all 1’s. This structure is said to be in Jordan form; for this reason the model (6.12) is identified
as Jordan canonical form state model.
The state variable model (6.10) derived for the case of distinct poles, is a special case of Jordan canonical
form wherein each Jordan block is of 1 ´ 1 dimension.
 State Variable Analysis of Digital Control Systems 359

6.3 STATE DESCRIPTION OF SAMPLED CONTINUOUS-TIME PLANTS

Systems that consist of an interconnection of a discrete-time system and a continuous-time system are
frequently encountered. An example of particular interest occurs when a digital computer is used to control
a continuous-time plant. Whenever such interconnections exist, there must be some type of interface system
that takes care of the communication between the discrete-time and continuous-time systems. In the system
of Fig. 6.1, the interface function is performed by D/A and A/D converters.
Simple models of the interface actions of D/A and A/D converters have been developed in Chapter 2. A
brief review is in order here.
A simple model of A/D converter is shown in Fig. 6.6. A continuous-time function f(t), t ³ 0, is the input,
and the sequence of real numbers f(k), k = 0, 1, 2, ..., is the output; the following relation holds between input
and output:
f (k) = f (t = kT ); T is the time interval between samples (6.13a)

Fig. 6.6 A model of an A/D converter

A simple model of D/A converter is shown in Fig. 6.7. A sequence of numbers f(k), k = 0, 1, 2, ..., is the
input, and the continuous-time function f +(t), t ³ 0, is the output; the following relation holds between input
and output:
f +(t) = f(k); kT £ t < (k + 1)T (6.13b)

f (k) f + ( t)
Zero-order hold

0 1 2 3 k t
Fig. 6.7 A model of a D/A converter

Figure 6.8 illustrates a typical example of an interconnection of discrete-time and continuous-time systems.
In order to analyse such a system, it is often convenient to represent the continuous-time system together
with the zero-order hold (ZOH); and the sampler by an equivalent discrete-time system.
We assume that the continuous-time system of Fig. 6.8 is a linear system with state variable model
x& (t) = Ax(t) + bu+(t) (6.14a)
y(t) = cx(t) + du+(t) (6.14b)
where x(t) is n ´ 1 state vector, u+(t) is scalar input, y(t) is scalar output; A, b, c, and d are, respectively, n ´ n,
n ´ 1, 1 ´ n, and 1 ´ 1 real constant matrices.
 360 Digital Control and State Variable Methods

u (k) u + (t) y(t) y(k )

Discrete- Continuous- Discrete-

time ZOH time Sampler time
system system system

Equivalent discrete-time system

Fig. 6.8 Interconnection of discrete-time and continuous-time systems

The solution of Eqn. (6.14a) with t0 as initial time is

t

z
x(t) = e A (t -t 0 ) x(t 0 ) + eA(t – t ) bu+(t) dt
t0
(6.15)

Since we use a ZOH (refer Eqn. (6.13b)),

u+ (t) = u(kT); kT £ t < (k + 1)T; k = 0, 1, 2, ...
Then from Eqn. (6.15) we can write,
LM e
t OP u(kT); kT £ t < (k + 1)T
MN z
A(t - t )
x(t) = eA(t – kT) x(kT) + b dt (6.16)
kT PQ
In response to the input u(kT), the state settles to the value x((k + 1)T) prior to the application of the input
u((k + 1)T), where
LM ( k + 1)T OP
x((k + 1)T) = eAT x(kT) +
MN kT
z A[( k +1)T - t ]
e b dt u(kT)
PQ
= Fx(kT) + gu(kT) (6.17)
Letting s = (t – kT) in Eqn. (6.17), we have
T

z
g = eA(T – s ) bds
0
With q = T – s, we get
T

z
g = eAq bdq
0
If we are interested in the value of x(t) (or y(t)) between sampling instants, we first solve for x(kT) for any
k using Eqn. (6.17), and then use Eqn. (6.16) to determine x(t) for kT £ t < (k + 1)T.
Since we have a sampler in configuration of Fig. 6.8 (refer Eqn. (6.13a)), we have from Eqn. (6.14b),
y(kT) = cx(kT) + du(kT)
State description of the equivalent discrete-time system of Fig. 6.8 is, therefore, of the form:
x(k + 1) = Fx(k) + gu(k) (6.18a)
y(k) = cx(k) + du(k) (6.18b)
where F = eAT (6.18c)
T

z
g = eAq bdq
0
(6.18d)
 State Variable Analysis of Digital Control Systems 361
AT AT
Machine Computation of e There are several methods available for computing e . Some of these
methods have been discussed in the earlier chapter. Standard computer programs based on these methods are
available.
In the following, we present an alternative technique of computing eAT. The virtues of this technique are
its simplicity and the ease of programming.
The infinite series expansion for F = eAT is
1 2 2 1 3 3
F = eAT = I + AT + AT + A T +L
2! 3!
¥
AiT i
= å i ! ; A0 = I (6.19)
i = 0
For a finite T, this series is uniformly convergent (Section 5.7). It is, therefore, possible to evaluate F
within prescribed accuracy. If the series is truncated at i = N, then we may write the finite series sum as
N
Ai T i
F = å (6.20)
i = 0 i!

which represents the infinite series approximation. The larger the N, the better is the approximation. We
evaluate F by a series in the form
FG
F = I + AT I +
AT RS
I+
AT
I+L+
AT
I+
LM
AT
L F I OP UV IJ (6.21)
H 2 T 3 N -1 N
N H K QWK
which has better numerical properties than the direct series of powers. Starting with the innermost factor,
this nested product expansion lends itself easily to digital programming. The empirical relation giving the
number of terms, N, is
N = min {3 || AT || + 6, 100} (6.22)
where || AT || is a norm of the matrix AT. There are several different forms of matrix norms commonly used.
Any one of them may be used in Eqn. (6.22). Two forms of matrix norms are defined in Section 5.2.
The relation (6.22) assumes that no more than 100 terms are included. The series eAT will be accurate to,
at least, six significant figures.
The integral in Eqn. (6.18d) can be evaluated term by term, to give
LM F I + Aq + 1 A q
T
I OP ¥
AiT i + 1
MNz H
2 2
g= +L dq b =
K å b (6.23)
0
2! PQ i = 0 (i + 1) !
¥
AiT i
= å (i + 1) !
Tb
i = 0

F AT A 2T 2 I T b = (e AT
– I) A–1b
= I+ GH 2!
+
3!
+L JK (6.24)

The transition from Eqn. (6.23) to (6.24) is possible only for a nonsingular matrix A. For a singular A, we
may evaluate g from Eqn. (6.23) by the approximation technique described above. Since the series expan-
sion for g converges faster than that for F, it suffices to determine N for F from Eqn. (6.22) and apply the
same value for g.
 362 Digital Control and State Variable Methods

Example 6.1 Figure 6.9 shows the block diagram of a digital positioning system. Defining the state
variables as
x1(t) = q (t), x2(t) = q& (t),
the state variable model of the plant becomes
x& (t) = Ax(t) + bu+(t)
y(t) = cx(t) (6.25)

with A=
0 1 LM
;b=
0 OP
; c = [1 0]
LM OP
0 -5 N 1 Q NQ
e (k) u (k ) u +(t)

r ( t) + k1z 2 + k 2z + k3 1 (t)
G h0(s)
T z (z – 1) s (s + 5)
–
= 0.1 sec ZOH
Digital Processor Plant

Fig. 6.9 A digital positioning system

We apply here the Cayley-Hamilton technique to evaluate state transition matrix eAt.
Eigenvalues of matrix A are given by

| lI – A | =
l -1 LM
=0
OP
0 l +5 N Q
Therefore l1 = 0, l2 = – 5
Since A is of second order, the polynomial g(l) will be of the form (refer Eqns (5.98)),
g(l) = b0 + b1l
The coefficients b0 and b1 are evaluated from the following equations:
1 = b0
e–5t = b0 – 5b1
The result is b0 = 1
b1 = 15 (1 – e–5t )

Hence eAt = b0I + b1A =

LM1
(1 - e -5t ) 1
5
OP
e -5 t N0 Q
The equivalent discrete-time plant with input u(k) and output q (k) (refer Fig. 6.9) is described by the equations
x(k + 1) = Fx(k) + gu(k)
(6.26)
y(k) = cx(k)

where F = eAT =
LM
1 15 (1 - e -5T ) OP
0 e -5T N Q
LM T
OP
T
bdq = MM
z 1
5 (1 - e -5q )dq
PP = LM 1
(T − 15 + 15 e −5T ) OP
g = eAq z 0
T
PP MN
5
1
5 (1 − e
−5 T
) PQ
0 MM
N z
0
e -5q dq
Q
 State Variable Analysis of Digital Control Systems 363
For T = 0.1 sec,

F=
LM1 0.0787OP ; g = LM0.0043OP
N0 0.6065Q N0.0787Q
Consider now the digital processor. The input-output model of the processor is
U ( z ) k1 z 2 + k2 z + k3
=
E( z ) z( z - 1)
Direct digital realization of the processor is shown in Fig. 6.10. Taking outputs of unit delayers as state
variables, we get the following state description for the processor dynamics (refer Eqns (6.8)):

LM x (k + 1)OP = LM0 0OP LM x (k )OP + LM

3 3 k3 OP
e(k) (6.27)
N x (k + 1)Q N1 1Q N x (k )Q Nk
4 4 2 + k1 Q
u(k) = x4(k) + k1e(k)

k1

k2

k3 z–1 1 z–1 1
e u
x3 x4

1
Fig. 6.10 Realization of digital processor of positioning system shown in Fig. 6.9

The processor input is derived from the reference input and the position feedback (Fig. 6.9):
e(k) = r(k) – x1(k) (6.28)
From Eqns (6.26)–(6.28), we get the following state variable model for the feedback system of Fig. 6.9.

LM x (k + 1)OP LM1-0.0043k
1 1 0.0787 0 0.0043 OP LM x (k) OP LM0.0043k OP
1 1

MM xx ((kk ++ 11))PP = MM -0.0787

2 k 1 0.6065 0 0.0787
PP MM xx ((kk))PP + MM0.0787
2 k
PP r(k)
1

MN x (k + 1)PQ MN -(k-k+ k )
3

4 2
3

1
0
0
0
1
0
1
PQ 3 k3
MN x (k )PQ MN k + k PQ
4 2 1
(6.29)
LM x (k) OP
1
x (k )
0] M
MM x (k )PPP
2
y(k) = [1 0 0
3

N x (k )Q
4
 364 Digital Control and State Variable Methods

6.4 STATE DESCRIPTION OF SYSTEMS WITH DEAD-TIME

Consider a state equation of a single-input system which includes delay in control action:
x& (t) = Ax(t) + bu+(t – tD) (6.30)
+
where x is n ´ 1 state vector, u is scalar input, tD is the dead-time, and A and b are respectively n ´ n and
n ´ 1 real constant matrices.
The solution of Eqn. (6.30) with t0 as initial time is
t
x(t) = e
A(t -t 0 )
z
x(t0) + e A( t - t ) bu + (t - t D )dt
t0

If we let t0= kT and t = kT + T, we obtain

kT + T
x(kT + T) = eAT x(kT) + z
kT
eA(kT + T – t ) bu+(t – tD) dt

With s = kT + T – t, we get
T

z
x(kT + T) = eAT x(kT) + eAs bu+(kT + T – tD – s ) ds
0
(6.31)

If N is the largest integer number of sampling periods in tD, we can write

tD = NT + DT; 0 £ D < 1 (6.32a)
Substituting in Eqn. (6.31), we get
T

z
x(kT + T) = eAT x(kT) + eAs bu+(kT + T – NT – DT – s ) ds
0
We introduce a parameter m such that
m=1–D (6.32b)
Then
T

z
x(kT + T ) = eAT x(kT ) + eAs bu+(kT – NT + mT – s )ds
0
(6.33)

Since we use a ZOH, u+ is piecewise constant. The nature of the integral in Eqn. (6.33), with respect to
variable s, becomes clear from the sketch of the piecewise constant input u+ over a segment of time axis near
t = kT – NT (Fig. 6.11). The integral runs for s from 0 to T—which corresponds to t from kT – NT + mT
backward to kT – NT – T + mT. Over this period, the control first takes on the value u(kT – NT ) and then the
value u(kT – NT – T ). Therefore we can break the integral in Eqn. (6.33) into two parts as follows:
LM e
mT OP LM e
T OP
MN z MN z
As As
x(kT + T) = eAT x(kT) + bds u(kT – NT) + bds u(kT – NT – T)
0 PQ mT PQ
= Fx(kT) + g1u(kT – NT – T) + g2u(kT – NT) (6.34a)

where F = eAT (6.34b)

 State Variable Analysis of Digital Control Systems 365
T
g1 = z
mT
eAs bds (6.34c)

mT
g2 = z0
eAs bds (6.34d)

u + (t ) s = mT
s=T s=0

u(kT – NT )

u(kT – NT – T )

t
kT – NT – T kT – NT + T
kT – NT
kT – NT – T + mT kT – NT + mT
Fig. 6.11
Setting q = s – mT in Eqn. (6.34c), we get
DT DT
g1 = z
0
eA(mT + q) bdq = eAmT z
0
eAq bdq (6.34e)

The matrices/vectors F, g1 and g2 can be evaluated by series truncation method discussed in the earlier
section.
Equation (6.34a) can be expressed in the standard state variable format. To do this, we consider first the
case of N = 0. For this case, Eqn. (6.34a) becomes
x(k + 1) = Fx(k) + g1u(k – 1) + g2u(k)
We must eliminate u(k – 1) from the right-hand side, which we do by defining a new state
xn +1(k) = u(k – 1)
The augmented state equation is given by
LM x(k + 1) OP LM
=
F g1 OP LM x(k) OP + LMg OP u(k)
2
(6.35)
Nx n +1 ( k + 1)Q N0 0 Q N x (k )Q N 1 Q
n +1

For N > 0, Eqn. (6.34a) can be expressed as

x(k + 1) = Fx(k) + g1u(k – N – 1) + g2u(k – N)
Let us introduce (N + 1) new states, defined below:
xn + 1(k) = u(k – N – 1)
 366 Digital Control and State Variable Methods
xn + 2(k) = u(k – N)
M
xn + N + 1(k) = u(k – 1)
The augmented state equation now becomes
LM x(k + 1) OP LMF g1 g2 0 L 0 OP LM x(k ) OP LM0OP
MM xx ((kk ++ 11)) PP MM 00
n +1 0
0
1
0
0 L 0
1 L 0
PP MM xx ((kk)) PP MM00PP
n +1

n+2 n+2
MM M PP = MM M M M M MP M
P M M PP + MM M PP u(k) (6.36)

MM x (k + 1) PP MM 0
n+ N 0 0 0 L 1P M x
P0Q MN x ((kk))PPQ MMN10PPQ
n +N

N x (k + 1)Q N 0
n + N +1 0 0 0 L n + N +1

Example 6.2 In the following, we reconsider the tank fluid temperature control system discussed in
Example 3.3 (see Fig. 3.17). The differential equation governing the tank fluid temperature was found to be
x&1 (t) = – x1(t) + u(t – 1.5) (6.37)
where
x1(t) = q(t) = tank fluid temperature
u(t) = qi(t) = temperature of the incoming fluid (control temperature)
tD = 1.5 sec
Assume that the system is sampled with period T = 1 sec. From Eqn. (6.32), we have
N = 1, D = 0.5, m = 0.5
Equations (6.34b), (6.34d), and (6.34e) give
F = e–1 = 0.3679
0 .5
g2 = z
0
e–s ds = 1 – e–0.5 = 0.3935

0 .5
g1 = e–0.5 z
0
e–q dq = e–0.5 – e–1 = 0.2387

The discrete-time model of the tank fluid temperature control system becomes (refer Eqn. (6.34a))
x1(k + 1) = 0.3679 x1(k) + 0.2387 u(k – 2) + 0.3935 u(k – 1) (6.38)
Let us introduce two new states, defined below.
x2(k) = u(k – 2)
x3(k) = u(k – 1)
The augmented state equation becomes
x(k + 1) = Fx(k) + gu(k)
(6.39)
y(k) = cx(k)
LM0.3679 0.2387 0.3935 OP 0 LM OP
with F= 0 0 1 ;g= 0
MM 0 PP MM PP
N 0 0 Q1 NQ
c = [1 0 0]
 State Variable Analysis of Digital Control Systems 367
From Eqns (6.39), the transfer function model is given as follows:
Y (z)
G(z) = = c(zI – F)–1 g
U(z)
LMz -0.3679 -0.2387 -0.3935 OP -1
LM0OP
-1 MM0PP
= [1 0 0]
MM 00 z
PP
N 0 z Q N1Q
z2 LM 0.2387z 0.2387 + 0.3935z OP L0O
= 2
1
z ( z - 0.3679)
[1 0 0] 0 MM z( z - 0.3679) z - 0.3679 PP MM0PP
0 N 0 z( z - 0.3679) Q MN1PQ
0.2387 + 0.3935z 0.3935(z + 0.6066)
= = (6.40)
z 2 ( z - 0.3679) z 2 ( z - 0.3679)
Note that the same result was obtained earlier in Example 3.3.

6.5 SOLUTION OF STATE DIFFERENCE EQUATIONS

6.5.1 Solution by Recursion

In this section, we investigate the solution of the state equation
x(k + 1) = Fx(k) + gu(k); x(0) = D x0 (6.41)
where x is n ´ 1 state vector, u is a scalar input, F is n ´ n real constant matrix, and g is n ´ 1 real constant
vector.
In general, discrete-time equations are easier to solve than differential equations because the former can
be solved easily by means of a recursion procedure. The recursion procedure is quite simple and convenient
for digital computations.
The solution of Eqn. (6.41) for any positive integer k may be obtained directly by recursion as follows.
From x(0) and u(0), x(1) can be calculated:
x(1) = Fx(0) + gu(0) (6.42a)
Then using x(1) and u(1):
x(2) = Fx(1) + gu(1) (6.42b)
From x(2) and u(2):
x(3) = Fx(2) + gu(2) (6.42c)
M
From x(k – 1) and u(k – 1):
x(k) = Fx(k – 1) + gu(k – 1) (6.42d)

6.5.2 Closed-Form Solution

In the following, we obtain the closed-form solution of state equation (6.41).
From Eqns (6.42a)–(6.42b), we obtain
x(2) = F[Fx(0) + gu(0)] + gu(1)
= F2x(0) + Fgu(0) + gu(1) (6.43)
From Eqns (6.43) and (6.42c), we get
x(3) = F[F2x(0) + Fgu(0) + gu(1)] + gu(2)
= F3x(0) + F2 gu(0) + Fgu(1) + gu(2)
 368 Digital Control and State Variable Methods
By repeating this procedure, we obtain
x(k) = Fkx(0) + Fk – 1 gu(0) + Fk – 2 gu(1) + L + F0 gu(k – 1); F0 = I
k -1
= Fkx(0) + å Fk–1– i gu(i) (6.44)
i = 0
Clearly x(k) consists of two parts, one representing the contribution of the initial state x(0), and the other
the contribution of the input u(i); i = 0, 1, 2, ..., (k – 1).
State Transition Matrix Notice that it is possible to write the solution of the homogeneous state
equations
D x0
x(k + 1) = Fx(k); x(0) = (6.45a)
as x(k) = Fkx(0) (6.45b)
From Eqn. (6.45b) it is observed that the initial state x(0) at k = 0 is driven to the state x(k) at the sampling
instant k. This transition in state is carried out by the matrix Fk. Due to this property, Fk is known as the state
transition matrix, and is denoted by f (k):
f (k) = Fk; f (0) = I (Identity matrix) (6.46)
In the following, we discuss commonly used methods for evaluating state transition matrix in closed form.
Evaluation Using Inverse z-Transforms Taking the z-transform on both sides of Eqn. (6.45a), yields
zX(z) – zx(0) = FX(z)

where D Z [x(k)]
X(z) =
Solving for X(z), we get
X(z) = (zI – F)–1zx(0)
The state vector x(k) can be obtained by inverse transforming X(z):
x(k) = Z –1[(zI – F)–1z] x(0)
Comparing this equation with Eqn. (6.45b), we get
Fk = f (k) = Z –1[(zI – F)–1z] (6.47)

Example 6.3 Consider the matrix F =

LM 0 1OP
N-0.16 -1Q
-1
For this F, (zI – F)–1 =
LM z -1 OP
N0.16 z + 1Q
LM z + 1 1 OP
( z + 0.2)( z + 0.8) ( z + 0.2)( z + 0.8)
=M PP
MM (z + 0.-20)(.16z + 0.8) z
PQ
N ( z + 0.2)( z + 0.8)

LM 4 / 3 + -1/ 3 5/3
+
-5 / 3 OP
z + 0.2 z + 0.8 z + 0.2 z + 0.8
=M
MN z-0+.80/.32 + z0+.8/03.8 -1/ 3 4/3 PP
+
z + 0.2 z + 0.8 Q
 State Variable Analysis of Digital Control Systems 369
k –1 –1
Therefore f (k) = F = Z [(zI – F) z]
LM 4
3
z
- 13
z 5
3
z
- 53
z OP
–1 z + 0.2 z + 0.8 z + 0.2 z + 0.8
=Z MM -0.8 z
+ 3 0.8 z -1 z
+ 3 4 z PP
N 3
z + 0.2 z + 0.8 3
z + 0.2 z + 0.8 Q
=
LM 4
3 ( -0.2 )
k
- 1
3 ( -0.8)
k 5
3 ( -0.2 )
k
- 5
3 ( -0.8)
k
OP
MN -0.8
3 ( -0.2)
k
+ 0.8
3 ( -0.8)
k -1
3 ( -0.2 )
k
+ 4
3 ( -0.8)
k
PQ
Evaluation Using Similarity Transformation Suppose that F is an n ´ n non-diagonal matrix with distinct
eigenvalues l1, l2, ..., ln. We define the diagonal matrix
0 L 0 LMl 1 OP
l2 L 0 0
L=M PP
M M MM M PQ
0 L ln N0
F and L are similar matrices; there exists a nonsingular matrix P such that (refer Eqns (5.22))
L = P–1 FP
Now P F P = P–1 [FF L F]P
–1 k

= P–1 [(P L P–1) (P L P–1) L (P L P–1)]P = L k

Thus the matrices F and L are similar. Since L is diagonal, L k is given by
k k

LMl k
1 0 L 0 OP
lk2
=M
0 L 0P
Lk
MM M M M P
P
0 L kn MN 0 l PQ
The state transition matrix Fk of matrix F with distinct eigenvalues l1, l2, ..., ln may, therefore, be evaluated
using the following relation:
LMl k
1 0 L 0 OP
l k2
=P M
0 L 0P
Fk = P L k P–1 P –1
(6.48)
MM M M M P
P
0 L kn MN 0 l PQ
where P is a transformation matrix that transforms F into the diagonal form (For the general case where
matrix F has multiple eigenvalues, refer [105]; also refer Review Example 6.5 given at the end of this
chapter).

Example 6.4 Consider the matrix

F=
LM 0 1OP
N-2 -3Q
The characteristic equation of the system is
|lI – F| = l2 + 3l + 2 = 0
which yields l1 = – 1 and l2 = – 2 as the eigenvalues of F.
 370 Digital Control and State Variable Methods
Since the matrix F is in companion form, the eigenvectors3, and hence the transformation matrix, can
easily be obtained (refer Problem 5.21).

P=
1 1
=
LMOP LM OP
1 1
l1 l 2 N Q N Q
-1 -2
gives the diagonalized matrix

L = P–1 FP =
-1 0 LM OP
0 -2 N Q
From Eqn. (6.48), we may write

Fk
L(-1) 0 OP P
=P M
k
–1
k
N 0 (-2) Q
L 1 1OP LM(-1) 0 OP LM 2 1OP
=M
k

N-1 -2Q N 0 (-2) Q N-1 -1Q k

L 2(-1) - (-2)
=M
k
( -1) - ( -2 ) O
k k k

N-2(-1) + 2(-2) -(-1) + 2(-2) PQ

k k k k

Evaluation Using Cayley-Hamilton Technique The Cayley-Hamilton technique has already been explained
in the earlier chapter. We illustrate the use of this technique for evaluation of Fk by an example.

Example 6.5 Consider the matrix

F=
LM 0 1OP
N-1 -2Q
Let us evaluate f(F) = Fk.
Matrix F has two eigenvalues at l1 = l2 = –1.
Since F is of second order, the polynomial g(l) will be of the form (refer Eqns (5.98)):
g(l) = b0 + b1l
The coefficients b0 and b1 are evaluated from the following equations:
f (–1) = (–1)k = b0 – b1
d d
f (l ) = k(–1)k –1 = g (l ) = b1
dl l = -1 dl l = -1

The result is b0 = (1 – k) (–1)k

b1 = – k(–1)k
Hence f(F) = Fk = b0I + b1F

= (1 – k) (–1)k
LM1 0OP – k(–1) LM 0 1OP k
N 0 1Q N-1 -2Q
= (–1)k
LM(1 - k) -k OP
N k (1 + k)Q
3
Refer Section 5.6 for methods of determination of eigenvectors for a given general matrix F.
 State Variable Analysis of Digital Control Systems 371
State Transition Equation The solution of the non-homogeneous state difference equation (6.41) is
given by Eqn. (6.44). In terms of the state transition matrix f (k), Eqn. (6.44) can be written in the form
k -1
x(k) = f (k) x(0) + å f (k – 1 – i) gu(i) (6.49)
i = 0
This equation is called the state transition equation; it describes the change of state relative to the initial
conditions x(0) and the input u(k).

Example 6.6 Consider the system

LM x (k + 1) OP = LM 0 1OP LM x (k) OP + LM0OP (– 1)

1 1 k
N x (k + 1)Q N-2 -3Q N x (k)Q N1Q
2 2

x1(0) = 1 = x2(0)
y(k) = x1(k)
Find y(k) for k ³ 1.
Solution For the given state equation, we have

F=
LM 0 1OP
N-2 -3Q
For this F, f (k) = Fk was evaluated in Example 6.4:

f (k) = Fk =
LM 2(-1) k
- ( -2 ) k ( -1) k - ( -2 ) k OP
k k k k
N-2(-1) + 2 ( -2 ) -( -1) + 2 ( -2 ) Q
k -1
The state x(k) = Fk x(0) + å Fk – 1 – i gu(i)
i = 0

With
L0O L1O
g = M P , x(0) = M P , and u(k)= (–1)k,
N1Q N1Q
we get
k -1
y(k) = x1(k) = 3(–1)k – 2(– 2)k + å [(–1)k – 1 – i – (–2)k – 1 – i] (–1)i
i = 0
k -1
1 i
= 3(–1)k – 2(–2)k + k(– 1)k – 1 – (–2)k – 1 å
i = 0
ch2

Since4
1 k
k -1
å 1 i
ch =
1- ch
2
=–2[ 1 k
ch - 1] , we have
2 1 - 12 2
i = 0

y(k) = 3(–1)k – 2(–2)k – k(– 1)k + (–2)k [1 - 1 k

di]
2
= 3(–1)k – 2(–2)k – k(–1)k + (– 2)k – (–1) = (2 – k) (– 1)k – (–2)k
k

k
1 - ak +1
4
å aj =
1- a
; a ¹ 1.
j = 0
 372 Digital Control and State Variable Methods

6.6 CONTROLLABILITY AND OBSERVABILITY

In this section, we study the controllability and observability properties of linear time-invariant systems
described by state variable model of the following form:
x(k + 1) = Fx(k) + gu(k) (6.50a)
y(k) = cx(k) + du(k) (6.50b)
where F, g, c and d are respectively n ´ n, n ´ 1, 1 ´ n, and 1 ´ 1 matrices. x is n ´ 1 state vector, and y and
u are, respectively, output and input variables.

6.6.1 Controllability
For the linear system given by Eqns (6.50), if there exists an input u(k); k Î [0, N – 1] with N a finite positive
integer, which transfers the initial state x(0) =D x0 to the state x1 at k = N, the state x0 is said to be controllable.

If all initial states are controllable, the system is said to be completely controllable or simply controllable.
Otherwise, the system is said to be uncontrollable.
The following theorem gives a simple controllability test.
Theorem 6.1 The necessary and sufficient condition for the system (6.50) to be completely controllable is
that the n ´ n controllability matrix,
D [g
U = Fg F2g L Fn – 1 g] (6.51)
has rank equal to n, i.e., r (U) = n.
Proof Solution of Eqn. (6.50a) is
k -1
x(k) = Fkx(0) + å Fk –1– i g u(i)
i =0
D x 0 and x(n) D x1, we obtain
Letting x(0) = =
x1 – Fnx0 = Fn – 1g u(0) + Fn – 2g u(1) + L + g u(n – 1)
LM u(n - 1) OP
u(n - 2)
or x1 – Fnx0 = [g Fg L Fn – 1 g] M
MM M PPP (6.52)

N u( 0 ) Q
Since g is an n ´ 1 matrix, we find that each of the matrices g, Fg, ..., Fn – 1 g is an n ´ 1 matrix. Therefore
U = [g Fg L Fn – 1 g]
is an n ´ n matrix. If the rank of U is n, then for arbitrary states x0 and x1, there exists a sequence of
unconstrained control signals u(0), u(1), ..., u(n – 1) that satisfies Eqn. (6.52). Hence, the condition that the
rank of the controllability matrix is n gives a sufficient condition for complete controllability.
To prove that the condition r (U) = n is also a necessary condition for complete controllability, we
assume that
r[g Fg L F n –1 g] < n
The matrix U is therefore singular and for arbitrary x0 and x1, a solution {u(0), u(1), ..., u(n – 1)} satisfy-
ing Eqn. (6.52) does not exist.
Let us attempt a solution of the form {u(0), u(1), ..., u(N – 1)}; N > n. This will amount to adding
columns Fng, Fn+1g, ..., FN–1 g in the U matrix. But by Cayley-Hamilton theorem, f(F) = Fj; j ³ n is a linear
combination of F n–1, L, F 1, F0 (refer Eqn. (5.98d)) and therefore columns F ng, F n+1g, ..., FN–1g add no
 State Variable Analysis of Digital Control Systems 373
new rank. Thus, if a state cannot be transferred to some other state in n sampling intervals, no matter
how long the input sequence {u(0), u(1), ..., u(N – 1)}; N > n is, it still cannot be achieved. Consequently,
we find that the rank condition given by Eqn. (6.51) is necessary and sufficient condition for complete
controllability.

6.6.2 Observability
For the linear system given by Eqns (6.50), if the knowledge of the input u(k); k Î [0, N – 1] and the output
D
y(k); k Î [0, N – 1] with N a finite positive integer, suffices to determine the state x(0) = x0, the state x0 is
said to be observable. If all initial conditions are observable, the system is said to be completely observable,
or simply observable. Otherwise, the system is said to be unobservable.
The following theorem gives a simple observability test.
Theorem 6.2 The necessary and sufficient condition for the system (6.50) to be completely observable is
that the n × n observability matrix
LM c OP
cF
M
V = M cF 2
PP (6.53)
MM M PP
MNcF
n -1 PQ
has rank equal to n, i.e., r (V) = n.
Proof The solution of Eqns (6.50) is

LM cF
k -1
k -1 - i
OP
y(k) = cFkx(0) +
MN å
i = 0
g u(i ) + du(k)
PQ
This gives y(0) = cx(0) + du(0)
y(1) = cFx(0) + cg u(0) + du(1)
M
y(n – 1) = cFn – 1x(0) + cFn – 2g u(0) + cFn – 3g u(1) + L + cg u(n – 2) + du(n – 1)
From these equations, we may write

LM y(0) - du(0) OP LM c OP
MM y(1) - cg u(0) - du(1)
= PP MM cFcF
2
PP x 0
= V × x0
MN y(n - 1) - cF n-2
M
PQ MM M PP n×n n ×1

g u(0) - cF n - 3 g u(1) - L - cg u(n - 2) - du(n - 1)

MNcF
n -1 PQ
(6.54)
If the rank of V is n, then there exists a unique solution x0 of Eqn. (6.54). Hence, the condition that the
rank of the observability matrix is n, gives a sufficient condition for complete observability.
It can easily be proved (refer proof of Theorem 6.1) that the condition r (V) = n is also a necessary
condition for complete observability.
 374 Digital Control and State Variable Methods

6.6.3 Controllability and Observability of State Variable Model in Jordan Canonical

Form
The following result for discrete-time systems easily follows from the corresponding result for continuous-
time systems, given in the earlier chapter.
Consider a SISO system with distinct eigenvalues5 l1, l2, ..., ln.
The Jordan canonical state model of the system is of the form
x(k + 1) = L x(k) + gu(k)
(6.55)
y(k) = cx(k) + du(k)
LMl 1 0 L 0 g1OP LM OP
0 l2 L 0 g2
with L=M ;g= PP MM PP
; c = [c1 c2 L cn]
MM M M M M
PQ MN PQ
N0 0 L ln gn
The system (6.55) is completely controllable if, and only if, none of the elements of the column matrix g,
is zero. The system (6.55) is completely observable if, and only if, none of the elements of the row matrix c,
is zero.

6.6.4 Equivalence Between Transfer Function and State Variable Representations

The following result for discrete-time systems easily follows from the corresponding result for continuous-
time systems, given in the earlier chapter.
The general state variable model of nth-order linear time-invariant discrete-time system is given by
Eqns (6.50):
D x0
x(k + 1) = Fx(k) + gu(k); x(0) =
(6.56)
y(k) = cx(k) + du(k)
The corresponding transfer function model is
Y ( z ) c( zI − F ) + g + d | zI − F|
= (6.57)
U(z) | zI − F |
The uncontrollable and unobservable modes of the state variable model (6.56) do not show up in the
corresponding transfer function representation (6.57); the poles of the transfer function are therefore a sub-
set of the eigenvalues of matrix F, and the asymptotic stability of the system always implies bounded-input,
bounded-output (BIBO) stability. The reverse, however, may not be true because the eigenvalues of uncon-
trollable and/or unobservable parts of the system are hidden from the BIBO stability analysis. When the state
variable model (6.56) is both controllable and observable, all the eigenvalues of F appear as poles in the
transfer function (6.57), and therefore BIBO stability implies asymptotic stability only for controllable and
observable systems.
Conclusion The transfer function model of a system represents its complete dynamics only if the system
is both controllable and observable.

5
Refer Gopal [105] for the case of multiple eigenvalues.
 State Variable Analysis of Digital Control Systems 375
6.6.5 Loss of Controllability and Observability due to Sampling
Sampling of a continuous-time system gives a discrete-time system with system matrices that depend on the
sampling period. How will that influence the controllability and observability of the sampled system? To get
a controllable sampled system, it is necessary that the continuous-time system also be controllable, because
the allowable control signals for the sampled system—piecewise constant signals—are a subset of allowable
control signals for the continuous-time system. However, it may happen that the controllability is lost for
some sampling periods.
The conditions for unobservability are more restricted in the continuous-time case because the output has to
be zero over a time interval, while the sampled system output has to be zero only at the sampling instants. This
means that the continuous output may oscillate between the sampling times and be zero at the sampling instants.
This condition is sometimes called hidden oscillations. The sampled system can thus be unobservable—even if
the corresponding continuous-time system is observable.
The harmonic oscillator can be used to illustrate the preceding discussion. The transfer function model of
the oscillator system is
Y ( s) w2
= 2 (6.58)
U ( s) s + w2
From this model, we have
y + w2y = w2u
&&
1
Define y& x1 = y; x2 =
w
This gives the following state variable representation of the oscillator system:
LM x& OP = LM 0 w OP LM x OP + LM 0 OP u
1 1

N x& Q N-w 0 Q N x Q Nw Q
2 2
(6.59)
Lx O
y = [1 0] M P
1

Nx Q 2
The discrete-time state variable representation of the system is obtained as follows. Noting that

A=
LM 0 w OP , b = LM 0 OP
N-w 0 Q Nw Q
we have

F = eAT = L –1 [(sI – A)–1]| t = T = L –1

FL s -w OP IJ-1

GH MNw s Q K
t=T

LM s w OP
=L –1
MM s
2
+ w2 s + w2
2
PP =
LM cos wT sin wT OP
-w s N- sin wT cos wT Q
Ns 2
+ w2 s + w2
2
Q t =T
and
LM e
T OP LM F cos wq
T
sin wq IJ dq OP LM 0 OP = LM 1 - cos w T OP
MNz MN z GH - sin wq
Aq
g= dq b =
0 PQ 0
cos wq K PQ N w Q N sin wT Q
 376 Digital Control and State Variable Methods
Hence, the discrete-time state variable representation of the oscillator system becomes
LM
x1 ( k + 1)OP LM
=
cos w T sin w T x1 (k ) OP LM+
OP LM
1 - cos w T
u(k)
OP
N
x 2 (k + 1) Q N QN
- sin w T cos w T x 2 ( k ) Q N
sin w T Q (6.60)

y(k) = [1
L x (k) OP
0] M
1

N x ( k )Q
2
The determinants of the controllability and observability matrices are
| U| = [g Fg] = – 2 sinwT (1 – coswT )

| V| =
LM c OP = sinwT
NcFQ
Both controllability and observability are lost for wT = np, n = 1, 2, ... (i.e., when the sampling interval is
half the period of oscillation of the harmonic oscillator, or an integer multiple of that period), although the
corresponding continuous-time system given by Eqns (6.59), is both controllable and observable.
Loss of controllability and/or observability due to sampling occurs only when the continuous-time system
has oscillatory modes and the sampling interval is half the period of oscillation of an oscillatory mode, or an
integer multiple of that period. This implies that controllability and observability properties of a continuous-
time system, are preserved after introduction of sampling if, and only if, for every eigenvalue of the
characteristic equation, the relation
Re li = Re lj
2np (6.61)
implies Im (li – lj) ¹
T
where T is the sampling period and n = ± 1, ± 2, ... .
We know that controllability and/or observability is lost when the transfer function corresponding to a
state model has common poles and zeros. The poles and zeros are functions of sampling interval. This
implies that if the choice of sampling interval does not satisfy the condition given by (6.61), pole-zero
cancellation will occur in passing from the continuous-time to the discrete-time case; the pole-zero cancella-
tion will not take place if the continuous-time system does not contain complex poles.
It is very unlikely that the sampling interval chosen for a plant control system, would be precisely the one
resulting in loss of controllability and/or observability. In fact the rules of thumb, for the choice of sampling
interval given in Section 2.13, imply a sampling interval of about one-tenth of the period of oscillation of an
oscillatory mode, and not just half.

6.7 MULTIVARIABLE SYSTEMS

The state variable model of the multi-input, multi-output (MIMO) system takes the following form (refer
Eqns (2.14)–(2.15)):
x(k + 1) = Fx(k) + Gu(k); x(0) =D x0 (6.62a)
y(k) = Cx(k) + Du(k) (6.62b)
F, G, C, and D are, respectively, n ´ n, n ´ p, q ´ n and q ´ p constant matrices, x is n ´ 1 state vector,
u is p ´ 1 input vector, and y is q ´ 1 output vector.
Many of the analysis results developed in earlier sections of this chapter for SISO systems, have obvious
extensions for the system (6.62).
The solution of the state equation (6.62a) is given by (refer Eqn. (6.44))
 State Variable Analysis of Digital Control Systems 377
k -1
x(k) = Fkx(0) + å F k -1- i Gu(i ) (6.63a)
i = 0
The output
LM k -1 O
k
y(k) = C F x(0) + å F k -1- i Gu(i) P + Du(k) (6.63b)
MN i = 0 PQ
In the transform domain, the input-output behaviour of the system (6.62) is determined entirely by the
transfer function matrix (refer Eqns (6.3))
G (z) = C(zI – F)–1 G + D (6.64a)
The output Y( z ) = G( z) U( z ) (6.64b)
q × 1 q × p p × 1

The necessary and sufficient condition for the system (6.62) to be completely controllable is that the
n ´ np matrix
D [G FG
U= F2G L Fn – 1 G] (6.65)
has rank equal to n.
The necessary and sufficient condition for the system (6.62) to be completely observable is that the nq ´ n
matrix
C LM OP
CF
V= M D
MM M PP (6.66)

N CF n -1 PQ
has rank equal to n.
A MIMO system with distinct eigenvalues6 l1, l2, ..., ln has the following Jordan canonical state model.
x(k + 1) = L x(k) + Gu(k)
(6.67)
y(k) = Cx(k) + Du(k)
with
LMl 1 0 L 0 OP LMg 11 g12 L g1 p OP Lc 11 c12 L c1n OP
L=M
0
MM M
l2 L 0
P ;G= M
g
MM M
21 g22 L g2 p
M P
P ; C = MM M M M PP
M M P M
N0 0 L l Q
n
P Ng n1 gn 2 L g Q
np
P MNc q1 cq 2 L cqn Q
The system (6.67) is completely controllable if, and only if, none of the rows of G matrix is a zero row,
and (6.67) is completely observable if, and only if, none of the columns of C matrix is a zero column.

Example 6.7 The scheme of Fig. 5.23 (refer Example 5.21) describes a simple concentration control
process. Mathematical model of the plant, given by Eqns (5.133), is reproduced below:
x& = Ax + Bu
(6.68)
y = Cx
6
Refer Gopal [105] for the case of multiple eigenvalues.
 378 Digital Control and State Variable Methods

with A=
LM- 0.01 0
;B=
OP 1 1 LM
;C=
0.01 0 OP LM OP
N 0 - 0.02 Q
- 0.004 - 0.002 N 0 1 Q N Q
The state, input, and output variables are deviations from steady-state values:
x1 = incremental volume of fluid in the tank (litres)
x2 = incremental outgoing concentration (g-moles/litre)
u1 = incremental feed 1 (litres/sec)
u2 = incremental feed 2 (litres/sec)
y1 = incremental outflow (litres/sec)
y2 = incremental outgoing concentration (g-moles/litre)
Matrix A in Eqns (6.68) is in diagonal form; none of the rows of B matrix is a zero row, and none of the
columns of C matrix is a zero column. The state model (6.68) is, therefore, completely controllable and
observable.
With initial values of x1 and x2 equal to zero at t = 0, a step of 2 litres/sec in feed 1 results in
LM et OP
MNz
A( t - t )
y(t) = C Bu(t ) dt
0 PQ
with
LM e
eAt =
- 0.01t
0 OP
- 0 .02 t
N0 e Q
and
L2O
u(t ) = M P
N0Q
Solving for y(t), we get
LM 2e t
OP
y(t) = C MM t
z
0
-0 .01( t - t )
dt
PP = C LM (1 - e 2
0.01
- 0.01t
) OP
- 0.02 t
MM- z 0.008e -0.02( t - t )
dt P N- 0.4 (1 - e )Q
N0 PQ
Therefore y1(t) = 2(1 – e–0.01t ) (6.69a)
y2(t) = – 0.4(1 – e–0.02t ) (6.69b)
Suppose that the plant (6.68) forms part of a process commanded by a process control computer. As
a result, the valve settings change at discrete instants only and remain constant in between. Assuming
that these instants are separated by time period T = 5 sec, we derive the discrete-time description of the
plant.
x(k + 1) = Fx(k) + Gu(k) (6.70a)
y(k) = Cx(k) (6.70b)

F = eA T =
LMe - 0.01T
0 OP L
0.9512 0 OP
Q MN
- 0.02 T
= (6.70c)
N0 e 0 0.9048 Q
 State Variable Analysis of Digital Control Systems 379

LM e T T
OP
T
Bdq = MM
z - 0 .01q
dq z e - 0.01q dq
PP = LM 4.88 4.88 OP
G= z eAq 0

MM- 0.004z e
T
0
T
N- 0.019 0.0095 Q (6.70d)
0

N 0
- 0.02q
dq 0.002 e z
0
- 0.02q
dq P
PQ
Matrix F in Eqns (6.70) is in diagonal form; none of the rows of G matrix is a zero row, and none of the
columns of C matrix is a zero column. The state model (6.70) is, therefore, completely controllable and
observable.
With initial values of x1 and x2 equal to zero at k = 0, a step of 2 litres/sec in feed 1 results in

LM Fk -1
k - 1- i
OP
y(k) = C
MN å
t = 0
Gu(i )
PQ
with Fk =
LM(0.9512) k
0 OP and u(i) =
LM2OP
N 0 (0.9048) k Q N0Q
Solving for y(k), we get
k -1
y1(k) = 0.01x1(k) = 0.01 å 9.76(0.9512)k – 1 – i
i = 0

Since (refer footnote 4)

k
1-
FG 1 IJ
k -1
å FG 1 IJ i
=
H 0.9512 K =
0.9512
[1 – (0.9512)– k ]
i = 0
H 0.9512 K 1-
1 - 0.0488
0.9512

we have y1(k) = 2[1 – (0.9512)k] (6.71a)

k -1
y2(k) = x2(k) = – 0.038 å (0.9048)k – 1 – i = – 0.4 [1 – (0.9048)k] (6.71b)
i = 0

Comparison of y1(k) and y2(k) with y1(t) and y2(t), shows that the two sets of responses match exactly at
the sampling instants.

Review Examples

Review Example 6.1 Give three different canonical state variable models corresponding to the transfer
function

4 z 3 - 12 z 2 + 13z - 7
G(z) =
( z - 1) 2 ( z - 2)
 380 Digital Control and State Variable Methods
Solution The given transfer function is

Y (z ) 4 z 3 - 12z 2 + 13z - 7 b 0 z 3 + b 1 z 2 + b 2 z + b 3
G(z) = = = 3
U ( z) z 3 - 4 z 2 + 5z - 2 z + a 1z 2 + a 2 z + a 3
The controllable canonical state model (first companion form) follows directly from Eqns (6.5) and (6.7):

LM x (k + 1) OP LM0 1
1 0 OP LM x (k) OP LM0OP
1

MM x (k + 1)PP = M0 0
2 P4P MM xx ((kk))PP + MM01PP u(k)
1 2

N x (k + 1)Q MN2 -5
3 QN Q NQ
3

L x (k) O
1
y(k) = [1 – 7 4] MM x (k )PP + 4u(k)
2
MN x (k )PQ
3

The observable canonical state model (second companion form) follows directly from Eqns (6.5) and (6.8):

LM x (k + 1) OP LM0
1 0 2 OP LM x (k) OP LM 1 OP
1

MM x (k + 1)PP = MM1
2 0 -5 P M x (k)P + M-7 P u(k)
2

N x (k + 1)Q N0
3 1 4 PQ MN x ( k )PQ MN 4 PQ
3

LM x (k) OP
1
y(k) = [0 0 1] M x (k )P + 4u(k)
2
MN x (k )PQ
3

The state variable model in Jordan canonical form follows from Eqns (6.11) and (6.12):

Y (z) 4 z 3 - 12 z 2 + 13z - 7 4z 2 - 7z + 1
= G(z) = = 4 +
U (z) z 3 - 4 z 2 + 5z - 2 ( z - 1) 2 ( z - 2)
2 1 3
=4+ 2
+ +
( z - 1) ( z - 1) ( z - 2)

LM x$ (k + 1) OP LM1
1 1 0 OP LM x$ (k) OP LM0OP
1

MM x$$ (k + 1)PP = MM0

2 1 0
PP MM xx$$ ((kk))PP + MM11PP u(k)
2

N x (k + 1)Q N0
3 0 2 QN Q NQ
3

LM x$ (k) OP
1
y(k) = [2 1 3] M x$ ( k )P + 4u(k)
2

NM x$ (k)QP
3

Review Example 6.2 Prove that a discrete-time system obtained by zero-order-hold sampling of an
asymptotically stable continuous-time system is also asymptotically stable.
 State Variable Analysis of Digital Control Systems 381
Solution Consider an asymptotically stable continuous-time system
x& (t) = Ax(t) (6.72)
We assume for simplicity, that the eigenvalues l1, l2, ..., ln of matrix A are all distinct. Let P be a
transformation matrix such that

LMl 1 0 L 0OP
0 l2 L 0
P–1 AP = L = M P
MM M M M P
P
N0 0 L l Q n

This gives (refer Eqn. (5.93))

Lt
P–1eAtP = e
LMe l 1t
0 L 0 OP
l 2t
L
=M
0 e 0 P
MM M M M P
P
l nt
MN 0 0 L e PQ

The zero-order-hold sampling of the continuous-time system (6.72) results in a discrete-time system
x(k + 1) = Fx(k)

where F = eAT; T = sampling interval

The characteristic polynomial of the system is
|zI – F| = |zI – eAT | = |P–1 | |zI – eAT | |P|

= |zP–1P – P–1eAT P | = |zI – e L T |

l 1T l2T l nT
= (z – e ) (z – e ) L (z – e )
lT
Notice that the eigenvalues of F are given by zi = e i ; i = 1, 2, ..., n. We see the equivalence of Reli < 0
and | zi | < 1. Thus, the discrete-time system, obtained by zero-order-hold sampling of an asymptotically
stable continuous-time system, is also asymptotically stable.
The proof for the case where matrix A has multiple eigenvalues follows on identical lines.

Review Example 6.3 Consider a unity feedback system with the plant

x& = Ax + bu
y = cx

where A=
LM0 1 OP
;b=
0
; c = [1
LM OP 0]
N0 -2 Q K N Q
(a) Find the range of values of K for which the closed-loop system is stable.
 382 Digital Control and State Variable Methods
(b) Introduce now a sampler and zero-order hold in the forward path of the closed-loop system. Show
that sampling has a destabilizing effect on the stability of the closed-loop system. To establish this
result, you may find the range of values of K for which the closed-loop digital system is stable when
(i) T = 0.4 sec, and (ii) T = 3 sec; and then compare with that obtained in (a) above.
Solution Consider the feedback system of Fig. 6.12a. Substituting
u = r – y = r – x1
in the plant model, we get the following state variable description of the closed-loop system:
LM x& OP = LM 0
1 x1
+
01
r
OP LM OP LM OP
2 N x& Q N- K
-2 x 2 K QN Q N Q
The characteristic equation of the closed-loop system is
l2 + 2l + K = 0
The closed-loop system is stable for all values of K > 0.
Figure 6.12b shows a block diagram of the closed-loop digital system. The discrete-time description of
the plant is obtained as follows:
x(k + 1) = Fx(k) + g u(k)
y(k) = cx(k)
T
where F = eAT; and g = eAq b dq z
0

r + u x y = x1
x = Ax + b u c
–

Plant

(a)

r + e u y = x1
ZOH Plant
T
–

(b)
Fig. 6.12

FLs LM 1 1 OP
Now e At
=L –1
[( sI - A ) ] = L -1 –1
GH MN0
-1OP IJ = L
-1
–1
MM s s( s + 2 )
PP
s + 2Q K 1
N0 s+2 Q
=
LM1 1
2
(1 - e -2 t ) OP
-2 t
N0 e Q
 State Variable Analysis of Digital Control Systems 383

Therefore F = eAT =
LM1 1
2
(1 - e -2 T ) OP
-2 T
N0 e Q
T LM K (1 - e -2q
)OP LMT - + 1 1 -2 T
e OP
g= z
0
MN 2 Ke - 2q PQ dq =
1
2 K
N 1- e
2 2
- 2T
Q
The state variable description of the closed-loop digital system, becomes

x(k + 1) =
LM1 - 1
2
K (T - 21 + 12 e -2 T ) 1
2
(1 - e -2 T ) OP x(k) + 1
K
LMT - + 1
2
1 - 2T
2
e OP r(k)
MN - 1
2
K (1 - e -2 T ) e -2 T
PQ 2
N 1-e - 2T
Q
The characteristic equation is given by
l2 + [ - (1 + e - 2 T ) + 1
2 K (T - 12 + 12 e - 2 T )] l + e –2T + 1
2 K( 12 – 1
2 e–2T – Te–2T ) = 0
Case I: T = 0.4 sec
For this value of sampling period, the characteristic polynomial becomes
D(l) = l2 + (0.062K – 1.449)l + 0.449 + 0.048K
Applying the Jury stability test (refer Eqns (2.73)–(2.75)), we find that the system is stable if the following
conditions are satisfied:
D(1) = 1 + 0.062K – 1.449 + 0.449 + 0.048K > 0
D(– 1) = 1 – 0.062K + 1.449 + 0.449 + 0.048K > 0
| 0.449 + 0.048K | < 1
These conditions are satisfied for 0 < K < 11.479.
Case II: T = 3 sec
For this value of the sampling period, the characteristic polynomial becomes
D(l) = l2 + (1.2506K – 1.0025)l + 0.0025 + 0.2457K
The system is found to be stable for 0 < K < 1.995.
Thus, the system which is stable for all K > 0 when T = 0 (continuous-time system), becomes unstable for
K > 11.479 when T = 0.4 sec. When T is increased to 3 sec, it becomes unstable for K > 1.995. It means that
increasing the sampling period (or decreasing the sampling rate) reduces the margin of stability.

Review Example 6.4 A closed-loop computer control system is shown in Fig. 6.13. The digital controller
is described by the difference equation
e2(k + 1) + ae2(k) = be1(k)
The state variable model of the plant is given below:
x& = Ax + bu
y = cx

with A=
LM0 1OP ; b = LM0OP ; c = [1 0]
N0 - 1Q N1Q
 384 Digital Control and State Variable Methods
Obtain discrete-time state description for the closed-loop system.

r + e 1 (t) e 1 (k) Digital e 2 (k) u y = x1

compensator ZOH Plant
– T = 1 sec

Fig. 6.13

Solution Given A=
LM0 1OP
N0 - 1Q
LM 1 1 OP L OP
1 - e- t
eAt = L –1[(s I – A) –1] = L –1 MM s s(s +1 1) PP = MN10 e- t Q
N0 s +1 Q
The discretized state equation of the plant is
x(k + 1) = Fx(k) + gu(k) (6.73a)

where F = eA T =
LM1 1 - e- T OP
N0 e- T Q
LM (1 - e T OP
Mz
-q
) dq
T

z
g = eAq b dq = M 0
T
PP = LMT - 1 + e OP -T
-T
(6.73b)
0 MM z e -q
dq PP N 1 - e Q
N 0 Q
For T = 1 sec, we have

F=
LM1 0.632OP ; g = LM0.368OP (6.73c)
N0 0.368Q N0.632Q
Consider now, the feedback system of Fig. 6.13 with the plant described by the equation (refer Eqns 6.73)):
LM x ( k + 1) OP = LM 1
1 0.632 OP L x (k ) O + LM0.368OP e (k)
1

N x (k + 1) Q N 0
2 0.368 Q MN x ( k )PQ N0.632Q
2
2 (6.74)

e2(k) may be taken as the third state variable x3(k) whose dynamics are given by
x3(k + 1) = – a x3(k) + b e1(k) = – a x3(k) + b (r(k) – x1(k)) = – b x1(k) – a x3(k) + b r(k) (6.75)
From Eqns (6.74)–(6.75), we get the following state variable model for the closed-loop digital system of
Fig. 6.13:
x1 (k + 1)LM OP
1 0.632 0.368 x1 (k ) LM 0 OP LM OP LM OP
x 2 (k + 1) = 0 0.368 0.632 x 2 ( k ) + 0 r(k)
MM PP PP
MM PP MM MM PP
x 3 (k + 1) N -b 0 Q N
- a x3 ( k ) b QN Q NQ
 State Variable Analysis of Digital Control Systems 385

LM x (k) OP
1
y(k) = [1 0 0] M x (k )P
2
MN x (k )PQ
3

Review Example 6.5 Given

LMl 1 1 0 L 0 OP
l1 1 L 0
L
MM 0M M M M
PP
=
n ´ n MM 0 0 0 L 1
PP
MN 0 0 0 L l1 PQ
Compute L k using the Cayley-Hamilton technique.
Solution Equations (5.98) outline the procedure of evaluation of functions of a matrix using the
Cayley-Hamilton technique.
The matrix L has n eigenvalues at l = l1. To evaluate f( L ) = L k, we define (refer Eqn. (5.98b)) the
polynomial g(l) as
g(l) = b0 + b1 l + L + b n –1 l n – 1
This polynomial may be rearranged as
g(l) = b0 + b1(l – l1) + L + b n – 1 (l – l1)n – 1
The coefficients b0, b1, ..., bn – 1 are given by the following equations (refer Eqns (5.98c)):
f(l1) = g(l1)

d d
f (l ) = g(l )
dl l = l1 dl l = l1

M
n -1
d dn-1
f (l ) = g (l )
dln - 1 l = l1
dln - 1 l = l1
Solving, we get
b0 = l k1

b1 = k lk1 -1
1!
b2 = k ( k - 1) lk1 - 2
2!
M
k ( k - 1)( k - 2) L ( k - n + 2) k - n + 1
bn – 1 = l1
(n - 1) !
 386 Digital Control and State Variable Methods

=
k (k - 1)(k - 2) L (k - n + 2)(k - n + 1)( k - n) L 1 1
lk1 - n + 1
LM OP
(k - n + 1)(k - n) L 1 (n - 1) ! N Q
k!
= lk - n + 1
( k - n + 1) !(n - 1) ! 1
Therefore (refer Review Example 5.3),
L k = b0I + b1( L – l1I) + L + bn – 1 ( L – l1I)n –1

LMl k k k -1
l1
k ( k - 1) k - 2
l1 L
k ! l1
k- n +1
OP
1
MM 1! 2! (k - n + 1) !(n - 1) ! PP
k k -1
0 lk1 l1 L ·
=M 1! PP
MM 0 0 lk1 L ·
PP
MM M M M M
PQ
N0 0 0 L lk1

Problems
6.1 A system is described by the state equation

LM
-3 1 0 -3OP LM OP
x(k + 1) = - 4 0 1 x(k) + -7 u(k); x(0) = x0
MM PP MM PP
N
-1 0 0 0 Q N Q
Using the z-transform technique, transform the state equation into a set of linear algebraic equations in the form
X(z) = G(z)x0 + H(z)U(z)
6.2 Give a block diagram for digital realization of the state equation of Problem 6.1.
6.3 Obtain the transfer function description for the following system:

LM x (k + 1) OP = LM2
1
-5 OP LM x (k) OP + L1O u(k); y(k) = 2x (k)
1

N x (k + 1)Q N
2
1
2
-1 Q N x (k)Q MN0PQ
2
1

6.4 A second-order multivariable system is described by the following equations:

LM x (k + 1) OP = LM2
1 -5 OP LM x (k ) OP + L1
1 -2 OP LMuu ((kk))OP
0 1

N x (k + 1)Q N
2
1
2
- 1Q N x ( k )Q MN0
2 1 Q MMNu (k)PPQ
3 2
3

LM y (k) OP = LM2 0OP LM x (k ) OP + LM0

1 1 4 0O M
L u (k) OP
1
u (k )
-2 PQ M
MNu (k )PPQ
2
N y (k )Q N1 -1Q N x (k)Q N0
2 2 0
3

Convert the state variable model into a transfer function matrix.

6.5 The state diagram of a linear system is shown in Fig. P6.5. Assign the state variables and write the dynamic
equations of the system.
 State Variable Analysis of Digital Control Systems 387

z–1

1/2 5
1

1/4
–1 z–1 6
U (z) Y (z)
–1/2
–1 3 2
2 –7
z–1

1/3

Fig. P6.5

6.6 Set up a state variable model for the system of Fig. P6.6.

+ + y1
u1 + + +
3 2
+

12 2

+
7
+

u2 + – + y2
– +

Fig. P6.6
388 Digital Control and State Variable Methods
6.7 Obtain the companion form realizations for the following transfer functions. Obtain different companion form
for each system.
Y (z) 3z 2 - z - 3
(i) = 2 1
R( z ) z + 3 z - 23

Y (z) -2z 3 + 2z 2 - z + 2
(ii) =
R( z ) z 3 + z 2 - z - 34
6.8 Obtain the Jordan canonical form realizations for the following transfer functions.
Y (z) z 3 + 8z 2 + 17z + 8
(i) =
R( z ) ( z + 1) ( z + 2) ( z + 3)

Y (z) 3z 3 - 4 z + 6
(ii) =
R( z ) ( z - 13 )3
6.9 Find state variable models for the following difference equations. Obtain different canonical form for each
system.
(i) y(k + 3) + 5 y(k + 2) + 7 y(k + 1) + 3 y(k) = 0
(ii) y(k + 2) + 3 y(k + 1) + 2 y(k) = 5 r(k + 1) + 3 r(k)
(iii) y(k + 3) + 5 y(k + 2) + 7 y(k + 1) + 3 y(k) = r(k + 1) + 2 r(k)
6.10 Given

F=
LM 0 1OP
N- 3 4Q
Determine f(k) = Fk using
(a) the z-transform technique;
(b) similarity transformation; and
(c) Cayley-Hamilton technique.
6.11 Consider the system

x(k + 1) = Fx(k) + gu(k); x(0) =

LM 1OP
N-1Q
y(k) = cx(k)

with F=
LM 0 1OP ; g = LM1OP ; c = [1 0]
N- 0.16 - 1Q N1Q
Find the closed-form solution for y(k) when u(k) is unit-step sequence.
6.12 Consider the system
x(k + 1) = Fx(k) + gu(k)
y(k) = Cx(k) + du(k)

LM - 1OP ; g = L3O ; x(0) = L-5O

3
2
N1 - 1Q MN2PQ MN 1PQ
with F=

C=M
L-3 4OP ; d = LM- 2OP ; u(k) = c h , k ³ 0
1 k
2
N-1 1Q N 0Q
Find the response y(k), k ³ 0.
 State Variable Analysis of Digital Control Systems 389
6.13 Consider the system
x(k + 1) = Fx(k)
LM-1 1 0 OP
with F= -1 0
MM 00 PP
N 0 -2 Q
(a) Find the modes of the free response.
(b) Find x(k) for
x(0) = [0 1 1]T
6.14 Consider the continuous-time system
Y (s ) 1
Ga(s) = =
R( s) s( s + 2)
Insert sample-and-hold devices and determine the vector difference state model for digital simulation of the
continuous-time system when the computation interval is T = 1 sec. Use the following methods to obtain the
simulation model.
(a) Obtain G(z) by taking the z transform of Ga(s) when it is preceded by a sampler-and-hold; convert G(z)
into a vector difference state model.
(b) Obtain a continuous-time state model for the given Ga(s); insert sample-and-hold and discretize the model.
6.15 Consider a continuous-time system

x& (t) =
LM-2 2OP x(t) + LM-1OP u(t)
N 1 - 3Q N 5Q
y(t) = [2 – 4] x(t) + 6 u(t)
Insert sample-and-hold devices and determine the vector difference state model for digital simulation of the
continuous-time system when the computation interval T = 0.2.
6.16 The plant of a single-input, single-output digital control system is shown in the block diagram of Fig. P6.16,
where u(t) is the control input and w(t) is a unit-step load disturbance. Obtain the state difference equations of
the plant. Sampling period T = 0.1 second.

w (t)

+
u ( t) + 1 1 y (t)
10 s
10 s + 1

Fig. P6.16

6.17 The mathematical model of the plant of a two-input, two-output temperature control system is given below:
x& = Ax + Bu
y = Cx

A=
LM- 0.1 0OP ;B=
100 LM0
;C=
1 0 OP LM OP
N 0.1 Q
- 0.1 N
0 100 0 1 Q N Q
For the computer control of this system, obtain the discrete-time model of the plant. Sampling period T =
3 seconds.
6.18 Consider the closed-loop control system shown in Fig. P6.18.
(a) Obtain the z-transform of the feedforward transfer function.
(b) Obtain the closed-loop transfer function, and convert it into a state variable model for digital simulation.
390 Digital Control and State Variable Methods

R (s ) + 1 – e– Ts 1 Y (s)
T = 1 sec s s(s + 1)
–

Fig. P6.18

6.19 The mathematical model of the plant of a control system is given below:
Y (s) e -0. 4 s
= Ga(s) =
U(s) s +1
For digital simulation of the plant, obtain a vector difference state model with T = 1 sec as the sampling period.
Use the following methods to obtain the plant model:
(a) Sample Ga(s) with a zero-order hold and convert the resulting discrete-time transfer function into a state
model.
(b) Convert the given Ga(s) into a state model and sample this model with a zero-order hold.
6.20 Determine zero-order hold sampling of the process
x& (t) = – x(t) + u(t – 2.5)
with sampling interval T = 1.
6.21 Convert the transfer function
Y (s ) e - st D
= Ga(s) = ; 0 £ tD < T
U ( s) s2
into a state model and sample this model with a zero-order hold; T is the sampling interval.
6.22 The plant of a unity-feedback continuous-time control system is described by the equations

x& =
LM0 1OP
x+
0 LM OP
u
N0 -2 Q 2 NQ
y = x1
(a) Show that the continuous-time closed-loop system is stable.
(b) A sampler and zero-order hold are now introduced in the forward loop. Show that the stable linear
continuous-time system becomes unstable upon the introduction of a sampler and a zero-order hold with
sampling period T = 3 sec.
6.23 The block diagram of a sampled-data system is shown in Fig. P6.23.
(a) Obtain a discrete-time state model for the system.
(b) Obtain the equation for intersample response of the system.

r + u 1 1 1 y
ZOH s+2 s+1 s
– T

Plant

Fig. P6.23

6.24 The block diagram of a sampled-data system is shown in Fig. P6.24. Obtain the discrete-time state model of the
system.

Given A=
LM 0 1OP ; b = LM0OP ; c = [1 0]
N- 2 - 3Q N1Q
 State Variable Analysis of Digital Control Systems 391

r + u x y
ZOH x = Ax + bu c
– T = 1 sec

Plant

Fig. P6.24

6.25 A closed-loop computer control system is shown in Fig. P6.25. The digital compensator is described by the
difference equation
e2(k + 1) + 2e2(k) = e1(k)
The state model of the plant is, as given in Problem 6.24. Obtain the discrete-time state model for the system.

r + e 1 (k) e 2 (k) u y
Digital
ZOH Plant
compensator
– T = 1 sec

Fig. P6.25

6.26 Consider the closed-loop analog control system shown in Fig. P6.26. For computer control of the process,
transform the controller transfer function into a difference equation using backward-difference approximation
of the derivative.
Sample the process model with a zero-order hold and obtain the state variable model of the closed-loop com-
puter-controlled system. Take T = 0.1 sec as sampling interval.

R (s) + E (s) U (s ) 10 Y (s)

9 + 4.1 s
s (10 s + 1)
–
Controller Process

Fig. P6.26

6.27 Investigate the controllability and observability of the following systems:

(a) x(k + 1) =
LM1 -2OP x(k) + LM 1 -1OP u(k)
N1 -1Q N0 0 Q
y(k) = M
L1 0OP x(k)
N0 1 Q
(b) x(k + 1) = M
L-1 1OP x(k) + LM0OP u(k)
N 0 -1Q N1 Q
y(k) = [1 1] x(k)
392 Digital Control and State Variable Methods
6.28 Consider the following continuous-time control system:
LM x& (t ) OP = LM 0 1OP LM x (t) OP + LM0OP u(t)
1 1
N x& (t )Q N-1 0Q N x (t)Q N1Q
2 2

y(t) = x1(t)
Show that the system is completely controllable and completely observable. The control signal u+(t) is now
generated by processing the signal u(t), through a sampler and a zero-order hold. Study the controllability and
observability properties of the system under this condition. Determine the values of the sampling period for
which the discretized system may exhibit hidden oscillations.
6.29 For the digital system shown in Fig. P6.29, determine what values of T must be avoided so that the system will
be assured of complete controllability and observability.

r 1 y
ZOH +
T (s + 0.02) 2 + 2 s+1

Fig. P6.29

6.30 Consider the state variable model

x(k + 1) = Fx(k) + g r(k)
y(k) = cx(k)
LM 0 1 OP; g = L0O; c = [- 1
1]
N- Q MN1PQ
with F= 1 3 2
8 4
(a) Find the eigenvalues of matrix F.
(b) Find the transfer function G(z) = Y(z)/R(z) and determine the poles of the transfer function.
(c) Comment upon the controllability and observability properties of the given system without making any
further calculations.
 Pole-Placement Design and
7 State Observers

7.1 INTRODUCTION

The design techniques presented in the preceding chapters are based on either frequency response or the root
locus. These transfer function-based methods have been referred to as classical control design. The goal of
this chapter is to solve the identical problems using different techniques based on state variable formulation.
The use of the state-space approach has often been referred to as modern control design. However, since the
state-space method of description for differential equations is over 100 years old, and was introduced in the
control design in the late 1950s, it seems somewhat misleading to refer to it as ‘modern’. We prefer to refer
to the two approaches to design as state variable methods and transform methods.
The transform methods of design are powerful methods of practical design. Most control systems are
designed using variations of these methods. An important property of these methods is robustness. The
resultant closed-loop system characteristics tend to be insensitive to small inaccuracies in the system model.
This property is very important because of the difficulty in finding an accurate linear model of a physical
system and also, because many systems have significant nonlinear operations.
The state variable methods appear to be much more dependent on having an accurate system model for
the design process. An advantage of these methods is that the system representation provides a complete
(internal) description of the system, including possible internal oscillations or instabilities that might be
hidden by inappropriate cancellations in the transfer function (input/output) description. The power of state
variable techniques is especially apparent when we design controllers for systems with more than one con-
trol input or sensed output. However, in this chapter, we will illustrate the state variable design methods
using single-input, single-output (SISO) systems. Methods for multi-input, multi-output (MIMO) design are
discussed in Chapter 8.
In this chapter, we present a design method known as pole placement or pole assignment. This method is
similar to the root-locus design in that, the closed-loop poles may be placed in desired locations. However,
pole-placement design allows all closed-loop poles to be placed in desirable locations, whereas the root-
locus design procedure allows only the two dominant poles to be placed. There is a cost associated with
placing all closed-loop poles, however, because placing all closed-loop poles requires measurement and
feedback of all the state variables of the system.
In many applications, all the state variables cannot be measured because of cost considerations, or
because of the lack of suitable transducers. In these cases, those state variables that cannot be measured
 394 Digital Control and State Variable Methods
must be estimated from the ones that are measured. Fortunately, we can separate the design into two phases.
During the first phase, we design the system as though all states of the system will be measured. The second
phase is concerned with the design of the state estimator. In this chapter, we consider both phases of the
design process, and the effects that the state estimator has on closed-loop system operation.
Figure 7.1 shows how the state-feedback control law and the state estimator fit together, and how the
combination takes the place of, what we have been previously referring to as, dynamic compensation. We
will see in this chapter that the estimator-based dynamic compensators are very similar to the classical
compensators of Chapter 4, inspite of the fact that they are arrived at by entirely different means.

r + u y
kR Plant Sensor
–

Control law
x
Estimator
k

State vector
Constant estimate
gain matrix

Compensation
Fig. 7.1 Schematic diagram of a state-feedback control system

7.2 STABILITY IMPROVEMENT BY STATE FEEDBACK

An important aspect of the feedback system design is the stability of the control system. Whatever we want
to achieve with the control system, its stability must be assured. Sometimes the main goal of a feedback
design is actually to stabilize a system if it is initially unstable, or to improve its stability if transient phenom-
ena do not die out sufficiently fast.
The purpose of this section is to investigate how the stability properties of linear systems can be improved
by state feedback.
Consider the single-input linear time-invariant system with nth-order state differential equation
x& (t) = Ax(t) + bu(t) (7.1)
If we suppose that all the n state variables x1, x2, ..., xn can be accurately measured at all times, it is
possible to implement a linear control law of the form
u(t) = – k1x1(t) – k2x2(t) – L – kn xn(t) = – kx(t) (7.2)
where k = [k1 k2 L kn]
is a constant state-feedback gain matrix. With this state-feedback control law, the closed-loop system is
described by the state differential equation
x& (t) = (A – bk) x(t) (7.3)
and the characteristic equation of the closed-loop system is
| sI – (A – bk) | = 0 (7.4)
 Pole-Placement Design and State Observers 395
When evaluated, this yields an nth-order polynomial in s
containing the n gains k1, k2, ..., kn. The control-law design
u
Plant then consists of picking the gains so that the roots of Eqn.
– (7.4) are in desirable locations. In the next section, we find
that under a mildly restrictive condition (namely, the system
(7.1) must be completely controllable), all the eigenvalues of
(A – bk) can be arbitrarily located in the complex plane by
+ x1
k1 choosing k suitably (with the restriction that complex
+ eigenvalues occur in complex-conjugate pairs). If all the
eigenvalues of (A – bk) are placed in the left-half plane, the
+ x 2
closed-loop system is, of course, asymptotically stable; x(t)
k2 will decay to zero irrespective of the value of x(0)—the initial
+ perturbation in the state. The system state is thus maintained
at zero value inspite of disturbances that act upon the system.
xn Systems with this property are called regulator systems. The
kn
origin of state space is the equilibrium state of the system.
Control configuration for a state regulator is shown in
Fig. 7.2 Control configuration for a Fig. 7.2. In this structure, there is no command input (r = 0).
state regulator Control systems in which the output must follow the command
signals (called servo systems) will be considered later.
Selection of desirable locations for the closed-loop poles requires some iteration by the designer. Some
of the issues in their selection will be discussed later in this chapter. For now, we will assume that the desired
locations are known, say,
s = l1, l2, K, ln
Then the desired characteristic equation is
(s – l1)(s – l2) L (s – ln) = 0 (7.5)
The required elements of k are obtained by matching
coefficients in Eqns (7.4) and (7.5), thus forcing the system
characteristic equation to be identical with the desired equation.
An example should clarify this pole placement idea.

Example 7.1 Consider the problem of designing an attitude

control system for a rigid satellite. Satellites usually require attitude
d
control so that antennas, sensors, and solar panels are properly
oriented. For example, antennas are usually pointed towards a par-
Gas jet ticular location on the earth, while solar panels need to be oriented
F towards the sun for maximum power generation. To gain an insight
into the full three-axis attitude-control system, we often consider
one axis at a time. Figure 7.3 depicts this case. The angle q that
describes the satellite orientation, must be measured with respect to
an ‘inertial’ reference, that is, a reference that has no angular
Inertial reference acceleration. The control signal comes from the reaction jets that
Fig. 7.3 Satellite control schematic produce a torque T(t) (= Fd) about the mass centre.
 396 Digital Control and State Variable Methods
The satellite is assumed to be in frictionless environment. If T(t) is the system input and q (t) is the system
output, we have
d 2q (t )
T(t) = J
dt 2
where J is the moment of inertia of the satellite. Normalizing, we define
u = T(t)/J
and obtain
q(s ) 1
q&& = u or =
U ( s) s 2
This is a reasonably accurate model of a rigid satellite in a frictionless environment, and is useful in
examples because of its simplicity.
Choosing x1 = q and x2 = q& as state variables, we obtain the following state equation for the system.

x& = Ax + bu =
LM OP
0 1
x+
0
u
LM OP (7.6)
0 0N Q 1 NQ
To stabilize the system, the input signal is chosen to be of the form
u(t) = – k1x1(t) – k2x2(t) = – kx(t)
The state equation for the closed-loop system (Fig. 7.4), then becomes

x& = (A – bk)x =
FG L
0 1 OP LM OP
0
[k1 k2 ] x =
IJ
0 LM
1
x
OP
H MN
0 0
-
Q NQ
1 K
- k1 - k 2
N Q
Satellite The characteristic equation of the closed-loop
system is
|sI – (A – bk)| = s2 + k2s + k1 = 0 (7.7)
u 1 x2 1 x1
s s Suppose that the design specifications for this
– – system require z = 0.707 with a settling time of 1 sec
FG 4 IJ
= 1 or zw n = 4 . The closed-loop pole
H zw n K
k2 locations needed are at s = – 4 ± j4. The desired
characteristic equation is
k1 (s + 4 + j4) (s + 4 – j4) = s2 + 8s + 32 (7.8)
Equating the coefficients with like powers of s in
Fig. 7.4 Simulation diagram for the satellite Eqns (7.7) and (7.8) yields
control system k1 = 32, k2 = 8

The calculation of the gains using the technique illustrated in this example, becomes rather tedious when the
order of the system is larger than three. There are, however, ‘canonical’ forms of the state variable equations
where the algebra for finding the gains is especially simple. One such canonical form—useful in control-law
design—is the controllable canonical form. Consider a system represented by the transfer function
Y ( s) b s n -1 + b 2 s n - 2 + L + b n
= 1 n
U ( s) s + a 1 s n -1 + L + a n
A companion-form realization of this transfer function is given below (refer Eqns (5.54)):
x& = Ax + bu (7.9)
y = cx
 Pole-Placement Design and State Observers 397
where
LM 0 1 0 L 0 OP L0O
M
A=M M
0 0 1 L 0
PP ; b = MM0M PP
MM 0 M M M
P MM PP
0 0 L 1 P MM0PP
MN-a n -a n -1 -a n - 2 L -a PQ 1 N1 Q
c = [bn bn – 1 L b2 b1 ]
The matrix A in Eqns (7.9) has a very special structure: the coefficients of the denominator of the transfer
function preceded by minus signs, form a string along the bottom row of the matrix. The rest of the matrix is
zero except for the superdiagonal terms, which are all unity. It can easily be proved that the pair (A,b) is
completely controllable for all values of a i’s. For this reason, the companion-form realization given by
Eqns (7.9) is referred to as the controllable canonical form.
One of the advantages of the controllable canonical form is that the controller gains can be obtained from
it, just by inspection. The closed-loop system matrix
LM 0 1 0 L 0 OP
L
MM 0M 0
M
1
M
0
M
PP
A – bk =
MM 0 0 0 L 1
PP
MN- a - k
n 1 - a n - 1 - k2 - a n - 2 - k3 L - a - k PQ
1 n

has the characteristic equation

sn + (a1 + kn)sn– 1 + L + (an – 2 + k3)s2 + (an–1 + k2)s + an + k1 = 0
and the controller gains can be found by comparing the coefficients of this characteristic equation with
Eqn. (7.5).
We now have the basis for a design procedure. Given an arbitrary state variable model and a desired
characteristic polynomial, we transform the model to controllable canonical form and solve for the control-
ler gains, by inspection. Since these gains are for the state in the controllable canonical form, we must
transform the gains back to the original state. We will develop this pole-placement design procedure in the
subsequent sections.

7.3 NECESSARY AND SUFFICIENT CONDITIONS FOR ARBITRARY POLE-PLACEMENT

Consider the linear time-invariant system (7.1) with state feedback control law (7.2); the resulting closed-
loop system is given by Eqn. (7.3). In the following, we shall prove that a necessary and sufficient condition
for arbitrary placement of closed-loop eigenvalues in the complex plane (with the restriction that complex
eigenvalues occur in complex-conjugate pairs), is that the system (7.1) is completely controllable. We shall
first prove the sufficient condition, i.e., if the system (7.1) is completely controllable, all the eigenvalues of
(A – bk) in Eqn. (7.3) can be arbitrarily placed.
In proving the sufficient condition on arbitrary pole-placement, it is convenient to transform the state
equation (7.1) into the controllable canonical form (7.9). Let us assume that such a transformation exists and
is given by
x = Px (7.10)
 398 Digital Control and State Variable Methods

LM p 11 p12 L p1n p1 OP LM OP
p p22 L p2 n p2
=M
MM M
21
M M
x=
M
x PP MM PP
Np n1 pn 2 L pnn pn
PQ MN PQ
pi = [pi1 pi2 L pin]; i = 1, 2, ..., n
Under the transformation (7.10), system (7.1) is transformed to the following controllable canonical model:
x& = A x + b u (7.11)
where
LM 0 1 0 L 0OP LM0OP
0 0 1 L 0
A = PAP–1
M
=M M
PP b = Pb = M0M P
MM 0 M M M
PP MM PP
0 0 1 M 0P
MN- a n - a n -1 - a n-2 L - a PQ 1
MN1PQ
n n–1
| sI – A | = s + a1 s + L + an – 1 s + an = |sI – A|
(Characteristic polynomial is invariant under equivalence transformation)
The first equation in the set (7.10) is given by
x1 = p11 x1 + p12 x2 + L + p1n xn = p1x
Taking the derivative on both sides of this equation, we get
x&1 = p1 x& = p1Ax + p1bu
But x&1 (= x 2 ) is a function of x only as per the canonical model (7.11).
Therefore
p1 b = 0 and x 2 = p1Ax
Taking derivative on both sides once again, we get
p1Ab = 0 and x 3 = p1A2x
Continuing the process, we obtain
p1 An – 2b = 0 and x n = p1An – 1x
Taking derivative once again, we obtain
p1 An – 1b = 1
Thus
LM p 1 OP
pA
x = Px = M
MM M
1 PP x
Np A 1
n -1 PQ
where p1 must satisfy the conditions
p1b = p1Ab = L = p1An – 2b = 0, p1An – 1b = 1
 Pole-Placement Design and State Observers 399
From Eqn. (7.11), we have
LM0OP LM p b 1 OP
M0P Mp Ab
Pb = M M P = M M
1 PP
MM0PP MMp A 1
n-2
bP
P
NM1QP NM p A 1
n -1
b QP
or
p1[b Ab L An –2b An – 1b] = [0 0 L 0 1]
This gives
p1 = [0 0 L 0 1]U – 1
where
U = [b Ab L An –1b]
is the controllability matrix, which is nonsingular because of the assumption of controllability of the
system (7.1).
Therefore, the controllable state model (7.1) can be transformed to the canonical form (7.11) by the
transformation
x = Px (7.12)
LMp1 OP
pA
where P=M
1
MMM PP ; p 1 = [0 0L0 1]U –1

Np A
1
n -1 PQ
Under the equivalence transformation (7.12), the state-feedback control law (7.2) becomes
u = – kx = – k x (7.13)
where k = kP –1 = [ k1 k2 L k n ]
With this control law, system (7.11) becomes
x& = ( A - b k ) x

LM 0 1 0 L 0 OP
0
L
MM 0M 0
M
1
M L
0
M
0
M
PP
=M
MM 0 0 0 0 1
PP x (7.14)

MN-a M- k M M M M PP
n 1 -a n -1 - k2 -a n - 2 - k3 L -a 2 - k n -1 -a - k Q
1 n

|sI – ( A − b k ) | = sn + (a1 + k n )sn – 1 + (a2 + kn-1 )sn –2 + L + (an – 1 + k 2 ) s + (an + k1 ) (7.15)

Since the coefficients ki are arbitrarily chosen real numbers, the coefficients of the characteristic polyno-
mial of (A – bk) can be given any desired values. Hence, the closed-loop poles can be placed at any desired
 400 Digital Control and State Variable Methods
locations in the complex plane (subject to conjugate pairing: coefficients of a characteristic polynomial will
be real only if the complex poles are present in conjugate pairs).
Assume that the desired characteristic polynomial of (A – bk), and hence ( A − b k ) , is
sn + a1 sn –1 + L + an
From Eqn. (7.15), it is obvious that this requirement is met if k is chosen as
k = [an – an an –1 – an –1 L a1 – a1]
Transforming the feedback controller (7.13) to the original coordinates, we obtain
k = k P = [an – an an –1 – an –1 L a1 – a1]P (7.16)
This proves that if (7.1) is controllable, the closed-loop poles can be arbitrarily assigned (sufficient condi-
tion).
We now derive the necessary condition by proving that if the system (7.1) is not completely controllable,
then there are eigenvalues of (A – bk) that cannot be controlled by state-feedback.
It was shown in Section 5.9 that an uncontrollable system can be transformed into controllability canoni-
cal form (Eqn. (5.123c))
LM x& OP = LMA
1 c A 12 OP L x O + LMb OP u = A x + bu
1 c

Nx& Q N 0
2 A 22 Q MNx PQ N 0 Q
2

where the pair ( A c , bc ) is completely controllable.

The set of eigenvalues of A is the union of the sets of eigenvalues of Ac and A 22 . In view of the form
of b , it is obvious that the matrix A 22 is not affected by the introduction of any state-feedback of the form
u = – k x . Therefore, the eigenvalues of A 22 cannot be controlled. This proves the necessary condition.

7.4 STATE REGULATOR DESIGN

Consider the nth-order, single-input linear time-invariant system

x& (t) = Ax(t) + bu(t) (7.17)
with state-feedback control law
u(t) = – kx(t) (7.18)
The resulting closed-loop system is
x& (t) = (A – bk) x(t) (7.19)
The eigenvalues of (A – bk) can be arbitrarily placed in the complex plane (with the restriction that complex
eigenvalues occur in complex-conjugate pairs) by choosing k suitably if, and only if, the system (7.17) is
completely controllable.
This important result on pole placement was proved in the previous section. The following design steps
for pole placement, emerge from the proof.
Step 1: From the characteristic polynomial of matrix A:
|sI – A| = sn + a1 sn –1 + L + an –1 s + an (7.20)
determine the values of a1, a2, ..., an –1, an.
Step 2: Determine the transformation matrix P that transforms the system (7.17) into controllable
canonical form:
 Pole-Placement Design and State Observers 401

LM p 1 OP
pA
PP ; p = [0 0L0 1]U -1
P=M
1 1
(7.21)
MM M PQ U = [b Ab L A n - 1b ]
n -1
Np A1

Step 3: Using the desired eigenvalues (desired closed-loop poles) l1, l2, ..., ln, write the desired character-
istic polynomial:
(s – l 1)(s – l 2) L (s – l n) = sn + a1 sn –1 + L + an – 1 s + an (7.22)
and determine the values of a1, a2, L, an–1, an.
Step 4: The required state-feedback gain matrix is determined from the following equation:
k = [an – an an –1 – an –1 L a1 – a1] P (7.23)
There are other approaches also, for the determination of the state-feedback gain matrix k. In what follows,
we shall present a well-known formula, known as the Ackermann’s formula, which is convenient for compu-
ter solution.
From Eqns (7.23) and (7.21), we get
LM(0 0 L 0 1) U -1 OP
-1
(0 0 L 0 1) U A
k = [an – an an –1 – an–1 L a1 –a ] M PP
MM
1
M
PP
MN(0 0 L 0 1) U -1 A n - 1 Q
= [0 0L0 1]U –1 [(a1 – a1)An –1 + (a2 – a2)An –2 + L + (an – an)I] (7.24)
The characteristic polynomial of matrix A is (Eqn. (7.20))
|sI – A | = sn + a1s n–1 + a2sn – 2 + L + an –1s + an
Since the Cayley-Hamilton theorem states that a matrix satisfies its own characteristic equation, we have
An + a1An –1 + a2An – 2 + L + an – 1A + anI = 0
Therefore An = – a1An –1 – a2An – 2 – L – an – 1A – anI (7.25)
From Eqns (7.24) and (7.25), we get
k = [0 0L0 1]U–1 f (A) (7.26a)
where
f (A) = An + a1An –1 + a2An –2 + L + an –1A + anI (7.26b)
n –1
U = [b Ab L A b] (7.26c)
Equations (7.26) describe the Ackermann’s formula for the determination of the state-feedback gain matrix k.

Example 7.2 Recall the inverted pendulum of Example 5.15, shown in Fig. 5.16, in which the object is to
apply a force u(t) so that the pendulum remains balanced in the vertical position. We found the linearized
equations governing the system to be:
x& = Ax + bu
where x = [q q& z z& ]T
 402 Digital Control and State Variable Methods

LM 0 1 0 0 OP LM 0 OP
16.3106 0 0 0 -1.4458
A=M
MM 0 0 0 1
PP
;b=
0
MM PP
N-1.0637 0 0 0Q
P MN 0.9639 PQ
z(t) = horizontal displacement of the pivot on the cart
q (t) = rotational angle of the pendulum
It is easy to verify that the characteristic polynomial of matrix A is
| sI – A | = s4 – 16.3106s2
Since there are poles at 0, 0, 4.039, and – 4.039, the system is quite unstable, as one would expect from
physical reasoning.
Suppose we require a feedback control of the form
u(t) = – kx = – k1x1 – k2x2 – k3x3 – k4x4,
such that the closed-loop system has the stable pole configuration given by multiple poles at –1. We
verified in Example 5.16 that the system under consideration is a controllable system; therefore, such a feed-
back gain matrix k does exist. We will determine the required k by using the design equations (7.17)–(7.23).
The controllability matrix
0 LM -14458
. 0 -235816
. OP
-14458
. 0 -235816
. 0
U = [b Ab A2b A b] = M
3 PP
MM 0 0.9639 0 15379
.
PQ
N 0.9639 0 15379
. 0
LM 0 0.0750 0 11500
. OP
0.0750 0 11500
. 0 P
U =M –1
MM 0 -0.0470 0 -0.0705P
N -0.0470 0 -0.0705 0
PQ
Therefore
p1 = [– 0.0470 0 – 0.0705 0]
LM p OP LM-0.0470
1 0 -0.0705 0 OP
pA P M 0 -0.0470 0 -0.0705 PP
P=M
1
MMp A PP = MM-0.6917
1
2
0 0 0
PQ
3
Np A Q N 0
1 -0.6917 0 0
|sI – A| = s4 + a1 s3 + a2 s2 + a3 s + a4 = s4 + 0s3 – 16.3106s2 + 0 s + 0
|sI – (A – bk) | = s4 + a1 s3 + a2 s2 + a3 s + a4 = (s + 1)4 = s4 + 4s3 + 6s2 + 4s + 1
k = [a4 – a4 a3 – a3 a2 – a2 a1 – a1]P
= [1 4 22.3106 4]P = [–15.4785 –2.9547 –0.0705 –0.2820]
This feedback control law yields a stable closed-loop system so that the entire state vector, when disturbed
from the zero state, returns asymptotically to this state. This means that not only is the pendulum balanced
(q ® 0), but that the cart returns to its origin as well (z ® 0).
 Pole-Placement Design and State Observers 403

Example 7.3 Let us use Ackermann’s formula to the state-regulator design problem of Example 7.1. The
plant model is given by (Eqn. (7.6))

x& = Ax + bu =
LM0 1OP x + LM0OP u
N0 0 Q N1Q
The desired characteristic polynomial is (Eqn. (7.8))
s2 + a1s + a2 = s2 + 8s + 32
To use Ackermann’s formula (7.26) to calculate the gain matrix k, we first evaluate U–1 and f(A):

U = [b
LM0 1OP; U = LM0
Ab] = –1
1 OP
N1 0 Q N1 0 Q
f (A) = A2 +a A+a I= M
1
L0 1OP LM0
2
1O
P0Q + 8 LMN0 0OPQ + 32 LMN0 1OPQ = LMN 0 32OPQ
0 1 1 0 32 8
N0 0 Q N0
Now using Eqn. (7.26a), we obtain

k = [0 1] U–1 f (A) = [0 1]
LM0 1OP LM32 8 OP = [32 8]
N1 0Q N 0 32Q
The solution is seen to be the same as that obtained in Example 7.1.

Comments 1. Through the pole-placement design procedure described in the present section, it is always
possible to stabilize a completely controllable system by state feedback, or to improve its stability by assign-
ing the closed-loop poles to locations in the left-half complex plane. The design procedure, however, gives
no guidance as to where, in the left-half plane, the closed-loop poles should be located.
It appears that we can choose the magnitude of the real part of the closed-loop poles to be arbitrarily
large, making the system response arbitrarily fast. However, to increase the rate at which the plant responds,
the input signal to the plant must become larger, requiring large values of gains. As the magnitudes of the
signals in a system increase, the likelihood of the system entering nonlinear regions of operation, increases.
For very large signals, this nonlinear operation will occur for almost every physical system. Hence, the linear
model that is used in design no longer accurately models the physical system.
Thus, the selection of desired closed-loop poles requires a proper balance of bandwidth, overshoot,
sensitivity, control effort, and other design requirements. If the system is of second-order, then the system
dynamics (response characteristics) can be precisely correlated to the locations of the desired closed-loop
poles. For higher-order systems, the location of the closed-loop poles and the response characteristics are
not easily correlated. Hence, in determining the state-feedback gain matrix k for a given system, it is desir-
able to examine, by computer simulations, the response characteristics for several different matrices k (based
on several different characteristic equations), and choose the one that gives the best overall performance.
2. For the case of single-input systems, the gain matrix k, which places the closed-loop poles at the
desired locations, is unique.
If the dynamic system under consideration
x& = Ax + Bu
has more than one input, that is, B has more than one column, then the gain matrix K in the control law
u = – Kx
 404 Digital Control and State Variable Methods
has more than one row. Since each row of K furnishes n gains (n is the order of the system) that can be
adjusted, it is clear that in a controllable system there will be more gains available—than are needed—to
place all of the closed-loop poles. This is a benefit: the designer has more flexibility in the design; it is
possible to specify all the closed-loop poles and still be able to satisfy other requirements. How should these
other requirements be specified? The answer to this question may well depend on the circumstances of the
particular application. A number of results using the design freedom in multi-input systems to improve
robustness of the control system, have appeared in the literature. We will not be able to accommodate these
results in this book.
The non-uniqueness in the design of state-feedback control law for multi-input systems, is removed by
optimal control theory which is discussed in Chapter 8.

7.5 DESIGN OF STATE OBSERVERS

The pole-placement design procedure introduced in the preceding sections results in control law of the form
u(t) = – kx(t) (7.27)
which requires the ability to directly measure the entire state vector x(t). Full state-feedback control for
many second-order systems, requires feedback of position and rate variables which can easily be measured.
However, for most of the higher-order systems, full state measurements are not practical. Thus, either a new
approach that directly accounts for the non-availability of the entire state vector (Chapter 8) is to be devised,
or a suitable approximation of the state vector must be determined. The latter approach is much simpler in
many situations.
The purpose of this section is to demonstrate the estimation of all the state variables of a system, from the
measurements that can be made on the system. If the estimate of the state vector is denoted by x$ , it would be
nice if the true state in the control law given by Eqn. (7.27), could be replaced by its estimate:
u(t) = – kx$ (t) (7.28)
This indeed is possible, as we shall see in the next section.
A device (or a computer program) that estimates the state variables is called a state observer, or simply an
observer. If the state observer estimates all the state variables of the system, regardless of whether some state
variables are available for direct measurement, it is called a full-order state observer. However, if accurate
measurements of certain states are possible, we may estimate only the remaining states, and the accurately
measured signals are then used directly for feedback. The resulting observer is called a reduced-order state
observer.

7.5.1 Full-Order State Observer

Consider a process described by the state equation
x& (t) = Ax(t) + bu(t) (7.29a)
where A and b are, respectively, n × n and n × 1 real constant matrices. The measurement y(t) is related to the
state by the equation
y(t) = cx(t) (7.29b)
where c is 1 ´ n real constant matrix. Without loss of generality, the direct transmission part has been
assumed to be zero.
One method of estimating all the state variables that we may consider, is to construct a model of the plant
dynamics:
x&$ (t) = A x$ (t) + bu(t) (7.30)
 Pole-Placement Design and State Observers 405
Plant where x$ is the estimate of the actual state x. We
u x y know A, b and u(t), and hence this estimator is
x = Ax + b u c satisfactory if we can obtain the correct initial
condition x(0) and set x$ (0) equal to it. Figure 7.5
depicts this ‘open-loop’ estimator. However, it is
Model precisely the lack of information on x(0) that
x y requires the construction of an estimator. If x$ (0)
x = Ax + b u c
¹ x(0), the estimated state x$ (t) obtained from the
open-loop scheme of Fig. 7.5 would have a con-
Fig. 7.5 Open-loop estimator tinually growing error or an error that goes to zero
too slowly to be of any use. Furthermore, small
errors in our knowledge of the system (A, b), and the disturbances that enter the system, but not the model,
would also cause the estimate to slowly diverge from the true state.
In order to speed up the estimation process and provide a useful state estimate, we feedback the differ-
ence between the measured and the estimated outputs—and correct the model continuously with this error
signal. This scheme, commonly known as ‘Luenberger state observer’, is shown in Fig. 7.6, and the equation
for it is
x&$ (t) = A x$ (t) + bu(t) + m(y(t) – y$ (t)) (7.31)
where m is an n ´ 1 real constant gain matrix.
The state error vector
~
x (t) = x(t) – x$ (t) (7.32)
Differentiating both sides, we get
x& (t) = x& (t) – x&$ (t)
~

Plant
u x y
x = Ax + bu c

+ +
+ x x y –
b n
c
+ n-parallel
integrators

Observer

Fig. 7.6 Luenberger state observer

 406 Digital Control and State Variable Methods

Substituting for x& (t) and x&$ (t) from Eqns (7.29) and (7.31) respectively, we get
~
x& (t) = Ax(t) + bu(t) – Ax$ (t) – bu(t) – mc(x(t) – x$ (t))
= (A – mc) ~
x (t) (7.33)
The characteristic equation of the error is given by
| sI – (A – mc) | = 0 (7.34a)
~
If m can (we hope) be chosen so that (A – mc) has stable and reasonably fast roots, x (t) will decay to zero
irrespective of ~
x (0). This means that x$ (t) will converge to x(t) regardless of the value of x$ (0), and further-
more, the dynamics of the error can be chosen to be faster than the open-loop dynamics. Note that Eqn.
(7.33) is independent of applied control. This is a consequence of assuming A, b and c to be identical in the
plant and the observer. Therefore, the estimation error ~ x converges to zero and remains there, independent
of any known forcing function u(t) on the plant and its effect on the state x(t). If we do not have a very
accurate model of the plant (A, b, c), the dynamics of the error are no longer governed by Eqn. (7.33).
However, m can typically be chosen so that the error system is stable and the error is acceptably small, even
with small modelling errors and disturbance inputs.
The selection of m can be approached in exactly the same fashion as the selection of k in the control law
design. If we specify the desired location of the observer-error roots as
s = l1, l2, K, ln,
the desired observer characteristic equation is
(s – l1) (s – l2) L (s – ln) = 0 (7.34b)
and one can solve for m by comparing coefficients in Eqns (7.34a) and (7.34b). However, as we shall see
shortly, this can be done only if the system (7.29) is completely observable.
The calculation of the gains using this simple technique becomes rather tedious when the order of the
system is larger than three. As in the controller design, there is an observable canonical form for which the
observer design equations are particularly simple. Consider a system represented by the transfer function
Y ( s) b s n -1 + b 2 s n - 2 + L + b n
= 1 n
U ( s) s + a 1 s n -1 + L + a n
A companion-form realization of this transfer function is given below (refer Eqns (5.56)):
x& = Ax + bu
(7.35)
y = cx
where
LM0 0 L 0 -a n OP LM b n OP
0 L 0 -a n -1
M 1
PP MMbb n -1 PP
A = M0 1 L 0 -a n - 2 ; b = 0L0
MM M M M M
PP MM Mn-2
PP ; c = [0 1]

MN0 0 L 1 -a 1 PQ MN b 1
PQ
It can easily be proved that the pair (A,c) is completely observable for all values of ai’s. For this reason,
the companion form realization given by Eqn. (7.35) is referred to as observable canonical form.
 Pole-Placement Design and State Observers 407
One of the advantages of the observable canonical form is that the observer gains m can be obtained from
it, just by inspection. The observer-error matrix is
LM 0 0 L 0 -a n - m1 OP
1 0 L 0 -a n -1 - m2
M
(A – mc) = M 0 1 L 0 -a n - 2 - m3
PP
MM M M M M
PP
MN 0 0 L 1 -a 1 - m n PQ
which has the characteristic equation
sn + (a1 + mn) sn –1 + L + (an – 2 + m3)s2 + (an –1 + m2) s + an + m1 = 0
and the observer gains can be found by comparing the coefficients of this equation with Eqn. (7.34b).
A procedure for observer design, therefore, consists of transforming the given state variable model to
observable canonical form, solving for the observer gains, and transforming the gains back to the original
state.
We can, however, directly use the equations of the control-law design for computing the observer gain
matrix m, if we examine the resemblance between the estimation and control problems. In fact, the two
problems are mathematically equivalent. This property is called duality. The design of a full-order observer
requires the determination of the gain matrix m such that (A – mc) has desired eigenvalues li; i = 1, 2, ..., n.
This is mathematically equivalent to designing a full state-feedback controller for the ‘transposed auxiliary
system’,
z& (t) = ATz(t) + cTh(t) (7.36a)
with feedback
h(t) = – mTz (t) (7.36b)
so that the closed-loop auxiliary system
z& (t) = (AT – cTmT )z(t) (7.37)
has eigenvalues li; i = 1, 2, ..., n.
Since
det W = det WT,
one obtains
det [sI – (AT – cTmT)] = det [sI – (A – mc)]
i.e., the eigenvalues of (AT – cTmT) are same as the eigenvalues of (A – mc).
By comparing the characteristic equation of the closed-loop system (7.19) and that of the auxiliary system
(7.37), we obtain the duality relations given in Table 7.1 between the control and estimation problems. The
Ackermann’s control-design formula given by Eqns (7.26) becomes the observer-design formula if the
substitutions of Table 7.1 are made.
A necessary and sufficient condition for
Table 7.1 Duality between control and estimation determination of the observer gain matrix m for
the desired eigenvalues of (A – mc), is that the
Control Estimation
auxiliary system (7.36) be completely controlla-
A AT ble. The controllability condition for this system
b cT is that the rank of
k mT
[cT ATcT L (AT)n –1 cT]
 408 Digital Control and State Variable Methods

is n. This is the condition for complete observability of the original system defined by Eqns (7.29). This
means that a necessary and sufficient condition for estimation of the state of the system, defined by Eqns
(7.29), is that the system be completely observable.
Again by duality, we can say that for the case of single-output systems, the gain matrix m, which places
the observer poles at desired locations, is unique. In the multi-output case, the same pole configuration can
be achieved by various feedback gain matrices. This non-uniqueness is removed by optimal control theory
which is discussed in Chapter 8.

Example 7.4 We will consider the satellite control system of Example 7.3. The state equation of the plant is
x& = Ax + bu

with A =
LM0 1OP ; b = LM0OP
N0 0Q N1Q
x1 = q, the orientation of the satellite; x2 = q&
We assume that the orientation q can be accurately measured from the antenna signal; therefore
y = cx(t)
with c = [1 0]
Let us design a state observer for the system. We choose the observer to be critically damped with a

settling time of 0.4 sec

FG 4 IJ
= 0.4; zw n = 10 . To satisfy these specifications, the observer poles will be
H zw n K
placed at s = – 10, – 10.
The transposed auxiliary system is given by
z& = AT z + cTh; h = – mT z
The desired characteristic equation of the closed-loop auxiliary system is
s2 + a1s + a2 = (s + 10)(s + 10) = s2 + 20s + 100
To apply Ackermann’s formula given by Eqns (7.26), we compute

U–1 = [cT ATcT]–1 =

LM1 0OP
N0 1 Q
f(AT) = (AT)2 + a1AT + a2I

=
LM0 0OP LM0 0OP + 20 LM0 0OP + 100 LM1 0OP = LM100 0 OP
N1 0 Q N1 0 Q N1 0 Q N0 1Q N 20 100 Q
The observer gain matrix is given by the equation

mT = [0 1]
LM100 0 OP
= [20 100]
N 20 100 Q
Therefore m=
LM 20OP
N100Q
 Pole-Placement Design and State Observers 409

Example 7.5 Consider once again the inverted-pendulum system of Example 7.2. Suppose that the only
output available for measurement is z(t), the position of the cart. The linearized equations governing this
system are
x& = Ax + bu; y = cx

LM 0 1 0 0 OP
0 LM OP
16.3106 0 0 0 -1.4458
where A=M
MM 0 0 0 1
;b= PP
0
MM
; c = [0 PP 0 1 0]

N -1.0637 0 0 0Q
P MN 0.9639PQ
We verified in Example 5.18 that this system is completely observable. In the following, we design a
full-order observer for this system. We choose the observer pole locations as – 2, – 2 ± j1, – 3. The corre-
sponding characteristic equation is
s4 + 9s3 + 31s2 + 49s + 30 = 0
The transposed auxiliary system is given by

z& (t) = AT z (t) + cTh(t); h(t) = – mT z (t)

We will determine the gain matrix m using the design equations (7.17)–(7.23).
The controllability matrix

LM0 0 -1.0637 0 OP
0 0 0 -1.0637
U = [cT ATcT (AT)2cT (A ) c ] = M
T 3 T
MM1 0 0 0
PP
N0 1 0 0
PQ
LM 0 0 1 0O
0 0 0 1P
U–1 =M
MM-0.9401 0 0 0P
P
N0 -0.9401 0 0Q
P
Therefore
p1 = [0 – 0.9401 0 0]

LMp 1 OP L 0 -0.9401 0 0 OP
p ( A ) P M - 0.9401
T
0 0 0
P=M PP
MMp ( A ) PP = MMM 0
1
T 2
1 - 15.3333 0 1
1
T 3
Np ( A ) Q N-15.3333 0 1 0
PQ
| sI – AT | = s4 + a1s3 + a2s2 + a3s + a4 = s4 + 0s3 – 16.3106s2 + 0s + 0

| sI – (AT – cTmT)| = s4 + a1s3 + a2s2 + a3s + a4 = s4 + 9s3 + 31s2 + 49s + 30

 410 Digital Control and State Variable Methods

mT = [a4 – a4 a3 – a3 a2 – a2 a1 – a1] P
= [30 49 47.3106 9] P = [–184.0641 –753.6317 9 47.3106]

LM-184.0641OP
-753.6317
Therefore m=M
MM 9 PPP
N 47.3106 Q
With this m, the observer

x&$ = (A – mc) x$ + bu + my
will process the cart position y(t) = z(t) and input u(t), to continuously provide an estimate x$ (t) of the state
vector x(t); and any errors in the estimate will decay at least as fast as e–2t.

7.5.2 Reduced-Order State Observer

The observer developed in the previous sub-section reconstructs the entire state vector. However,
the measurements usually available are some of the states of the plant. For example, for the satellite
considered in the previous sub-section, the measurement is orientation of the satellite, which is x 1(t). The
measurement of a state, in general, will be more accurate than any estimate of the state based on the
measurement. Hence, it is not logical in most cases to estimate states that we are measuring. One possible
exception is the case in which a measurement is very noisy. The state observer for this case may furnish
some beneficial noise filtering.
Since we will not usually want to estimate any state that we are measuring, we prefer to design an observer
that estimates only those states that are not measured. This type of observer is called a reduced-order state
observer. We develop design equations for such an observer in this sub-section. We consider only the case
of one measurement. It is assumed that the state variables are always chosen such that the state measured is
x1(t); we can do this without loss of generality. The output equation then is given by
y(t) = x1(t) = cx(t)
where c = [1 0 0 L 0]
To derive the reduced-order observer, we partition the state vector into two parts: one part is x1 which is
directly measured and the other part is xe, representing the state variables that need to be estimated:

x(t) =
LM x (t ) OP
1

Nx (t )Qe

If we partition the system matrices accordingly, the complete description of the system is given by
LM x& OP = LMa
1 11 OP LM x OP + LM b OP u
a 1e 1 1
(7.38a)
Nx& Q Na
e e1 A ee
Q Nx Q Nb Q
e e

y = [1
Lx O
0] M P 1
(7.38b)
Nx Qe
 Pole-Placement Design and State Observers 411
The dynamics of the unmeasured state variables are given by
x& e = Aee xe + a e1 x1 + b e u (7.39)
14243
known input
where the two rightmost terms are known and can be considered as an input into the xe dynamics. Since x1 = y,
measured dynamics are given by the scalar equation
x&1 = y& = a11 y + a1e xe + b1u (7.40)
If we collect the known terms of Eqn. (7.40) on one side, we get
y& - a y - b1u = a1exe (7.41)
144112443
known measurement
Note that Eqns (7.39) and (7.41) have the same relationship to the state xe that the original equations (7.38)
had to the entire state x. Following this line of reasoning, we can establish the following substitutions in the
original observer-design equations, to obtain an (reduced-order) observer of xe:
x ¬ xe
A ¬ Aee
bu ¬ ae1 y + beu (7.42)
y ¬ y& – a11 y – b1u
c ¬ a1e
Making these substitutions into the equation for full-order observer (Eqn. (7.31)), we obtain the equation of
the reduced-order observer:
x&$ e = Aee x$ e + a1e14
y2+ b e u + m ( y& - a11 y - b1u – a1e x$ e)
43 1442443
(7.43)
input measurement
If we define the estimation error as
~
x e = xe – x$ e (7.44)
the dynamics of error are given by subtracting Eqn. (7.43) from Eqn. (7.39):
x& = (A – ma ) ~
~
e x
ee 1e e (7.45)
Its characteristic equation is given by
| sI – (Aee – ma1e) | = 0 (7.46)
We design the dynamics of this observer by selecting m so that Eqn. (7.46) matches a desired reduced-
order characteristic equation. To carry out the design using state regulator results, we form a ‘transposed
auxiliary system’
z& (t) = A Tee z (t) + a1e
T
h(t)
(7.47)
h(t) = – mT z (t)
Use of Ackermann’s formula given by Eqns (7.26) for this auxiliary system gives the gains m of the
reduced-order observer. We should point out that the conditions for the existence of the reduced-order
observer are the same as for the full-order observer—namely observability of the pair (A, c).
Let us now look at the implementational aspects of the reduced-order observer given by Eqn. (7.43). This
equation can be rewritten as
x&$ = (A – ma ) x$ + (a – ma )y + (b – mb )u + m y&
e ee 1e e e1 11 e 1 (7.48)
 412 Digital Control and State Variable Methods
The fact that the reduced-order observer requires the derivative of y(t) as an input, appears to present a
practical difficulty. It is known that differentiation amplifies noise, so if y is noisy the use of y& is unaccept-
able. To get around this difficulty, we define the new state as
x ¢e =D x$ e – my (7.49a)
Then, in terms of this new state, the implementation of the reduced-order observer is given by
. ee – ma1e) x$ e + (ae1 – ma11)y + (be – mb1)u
x& e¢ = (A (7.49b)
and y& no longer appears directly. A block-diagram representation of the reduced-order observer is shown in
Fig. 7.7.
y

ae1 – ma11
m

+ + + + x
u xe xe e
be – mb1
n–1
+
n – 1 parallel
integrators

A ee – ma1e

Fig. 7.7 Reduced-order observer structure

Example 7.6 In Example 7.4, a second-order observer for the satellite control system was designed with
the observer poles at s = – 10, – 10. We now design a reduced-order (first-order) observer for the system
with observer pole at s = – 10.
The plant equations are
x&1
=
0 1 x1LM OP LM OP LM OP LM OP
+
0
u
x&2 0 0 x2 N Q N QN Q NQ
1
Lx O
1
y = [1 0] M P
Nx Q
2
The partitioned matrices are

LMa 11 a1e OP LM OP LM OP LM OP
=
0 1 b
= 1 =
0
Na e1 Aee Q N Q N Q NQ
0 0 be 1
From Eqn. (7.46), we find the characteristic equation in terms of m:
s – (0 – m) = 0
We compare it with the desired equation
s + 10 = 0
 Pole-Placement Design and State Observers 413
which yields,
m = 10
The observer equations are (refer Eqns. (7.49))
x& 2¢ = – 10 x$ 2 + u
x$ 2 = x 2¢ + 10y
This completes the design of the reduced-order observer which estimates the angular velocity of the satellite
from the measurement of the angular position.

7.6 COMPENSATOR DESIGN BY THE SEPARATION PRINCIPLE

In Sections 7.2–7.4 we studied the design of control laws for systems in which the state variables are all
accessible for measurement. We promised to overcome the difficulty of not being able to measure all the
state variables by the use of an observer to estimate those state variables that cannot be measured. Then in
Section 7.5, we studied the design of observers for systems with known inputs, but not when the state esti-
mate is used for the purpose of control. We are now ready to combine the state-feedback control law with the
observer to obtain a compensator for linear systems in which not all the state variables can be measured.
Consider the completely controllable and completely observable system defined by the equations
x& = Ax + bu
y = cx (7.50)
Suppose we have designed a state-feedback control law
u = – kx (7.51)
using the methods of Section 7.4. And also suppose we have designed a full-order observer
x$& = A x$ + bu + m(y – c x$ ) (7.52)
using the methods of Section 7.5.
For the state-feedback control based on the observed state x$ ,
u = – k x$ (7.53)
The control system based on combining the state-feedback control law and state observer, has the con-
figuration shown in Fig. 7.8. Note that the number of state variables in the compensator is equal to the order
of the embedded observer and hence is equal to the order of the plant. Thus, the order of the overall closed-
loop system, when a full-order observer is used in the compensator, is 2n for a plant of order n. We are
interested in the dynamic behaviour of the 2nth-order system comprising the plant and the compensator.
With the control law (7.53) used, the plant dynamics become
x& = Ax – bk x$ = (A – bk)x + bk(x – x$ ) (7.54)
Plant Sensor
u x y
x = Ax + bu c

x x = Ax + bu
–k
+ m (y – cx )
Control law Observer

Compensator
Fig. 7.8 Combined state-feedback control and state estimation
 414 Digital Control and State Variable Methods

The difference between the actual state x and observed state x$ , has been defined as the error ~
x:
~
x = x – x$
Substitution of the error vector into Eqn. (7.54) gives
x& = (A – bk)x + bk ~
x (7.55)
Note that the observer error was given by Eqn. (7.33), repeated here:
x& = (A – mc) ~
~ x (7.56)
Combining Eqns (7.55) and (7.56) we obtain
LMx& OP = LMA - bk OP LMxOP
bk
(7.57)
N~x& Q N 0 A - mcQ N~ xQ
Equation (7.57) describes the dynamics of the 2n-dimensional system of Fig. 7.8. The characteristic equation
for the system is
| sI – (A – bk) | | sI – (A – mc) | = 0
In other words, the poles of the combined system consist of the union of control and observer roots. This
means that the design of the control law and the observer can be carried out independently. Yet, when they
are used together the roots are unchanged. This is a special case of the separation principle which holds
in much more general contexts and allows for the separate design of control law and estimator in certain
stochastic cases.
To compare the state-variable method of design with the transform methods discussed in earlier chapters,
we obtain the transfer function model of the compensator used in the control system of Fig. 7.8. The state
variable model for this compensator is obtained by including the feedback law u = – k x$ (since it is part of the
compensator) in the observer equation (7.52).
x$& = (A – bk – mc) x$ + my
(7.58)
u = – k x$
The formula for conversion of state variable model to the transfer function model is given by Eqn. (5.28).
Applying this result to the model given by Eqn. (7.58), we obtain
U (s )
= D(s) = k(sI – A + bk + mc)–1m (7.59)
- Y ( s)
Figure 7.9 shows the block diagram representation of the system with observer-based controller.

R (s) = 0 U (s ) Y (s)
D (s) Plant
–

Fig. 7.9 Block diagram representation of a system with observer-based controller

Note that the poles of D(s) in Eqn. (7.59) were neither specified nor used during the state-variable design
process. It may even happen that D(s) has one or more poles in the right-half plane; the compensator, in other
words, could turn out to be unstable. But the closed-loop system, if so designed, would be stable. There is,
however, one problem if the compensator is unstable. The open-loop poles of the system are the poles of the
plant and also the poles of the compensator. If the latter are in the right-half plane, then the closed-loop poles
may be in the right-half plane when the loop gain becomes too small. Robustness considerations put certain
restrictions on the use of unstable compensators to stabilize a system.
 Pole-Placement Design and State Observers 415

Example 7.7 In this example, we study the closed-loop system obtained by implementing the state-feed-
back control law of Example 7.3 and state-observer design of Examples 7.4 and 7.6, for the attitude control
of a satellite. The plant model is given by
x& = Ax + bu; y = cx
with

A=
LM OP
0 1
;b=
0LM OP
; c = [1 0]
N Q
0 0 1 NQ
In Example 7.3, the gain matrix required to place the closed-loop poles at s = – 4 ± j4 was calculated to be
k = [32 8]
If both the state variables are available for feedback, the control law becomes
u = – kx = – [32 8]x
resulting in the closed-loop system

x& = (A – bk)x =
LM 0 1OP x
N-32 -8Q
Figure 7.10a shows the response of the system to an initial condition x(0) = [1 0]T. Assume now that the
state-feedback control law is implemented using a full-order observer. In Example 7.4, the observer gain
matrix was calculated to be

m=
20 LM OP
100 N Q
1

(a)

(b)
0.5
(c)

0.8 1.0
0
0.2 0.4 0.6

t (sec)

– 0.5

Fig. 7.10 Initial condition response; (a) without observer (b) with full-order observer, and
(c) with reduced-order observer
 416 Digital Control and State Variable Methods
The state variable model of the compensator, obtained by cascading the state-feedback control law and the
state observer, is obtained as (refer Eqns (7.58))

x&$ = (A – bk – mc) x$ + my =
-20 1 LM
x$ +
20
y
OP LM OP
-132 -8 N 100 Q N Q
u = – k x$ = – [32 8] x$
The compensator transfer function is (refer Eqn. (7.59))
U ( s) 1440 s + 3200
D(s) = = k(sI – A + bk + mc)–1m = 2
-Y ( s ) s + 28s + 292
The state variable model of the closed-loop system can be constructed as follows:
x&1 = x2
x& 2 = u = – 32 x$1 – 8 x$ 2

x&$1 = – 20 x$1 + x$ 2 + 20y = – 20 x$1 + x$ 2 + 20x1

x&$ 2 = – 132 x$1 – 8 x$ 2 + 100x1
Therefore
LM x& OP LM 0
1 1 0 0 OP LM x OP
1

MM xx&&$ PP = MM 200
2 0
0
-32 -8
-20 1
PP MM xx$ PP
2

1 1

2
MN x&$ PQ MN100
0 -132 2 -8Q
P MN x$ PQ
Figure 7.10b shows the response to an initial condition
[1 0 0 0]T
Consider now the implementation of state-feedback control law using reduced-order observer. In
Example 7.6, the following model was obtained to estimate the state x2 (state x1 is directly measured and
fedback, and is not estimated using an observer):
x$ 2 = x 2¢ + 10y
x& 2¢ = –10 x$ 2 + u
The control law is given by
u = – 32x1 – 8 x$ 2
From these equations, the following transfer function model of the compensator is obtained:
U ( s) 112(s + 2.86)
=
- Y ( s) s + 18
The reduced-order compensator is precisely the lead network; this is a pleasant discovery, as it shows that
the classical and state variable methods can result in exactly the same type of compensation.
The state variable model of the closed-loop system with the reduced-order compensator is derived below.
x&1 = x2
x& 2 = u = – 32x1 – 8 x$ 2 = – 32x1 – 8( x 2¢ + 10x1) = – 112x1 – 8 x 2¢
x& 2¢ = – 10 x$ 2 + u = – 10 x$ 2 – 32x1 – 8 x$ 2
= – 18( x 2¢ + 10x1) – 32x1 = – 18 x 2¢ – 212x1
 Pole-Placement Design and State Observers 417
Therefore
LM x& OP LM 0
1 1 0 OP LM x OP
1

MM x&& PP = MM-112
2 0 -8
P Mx P2

N x ¢ Q N-212
2 0 -18PQ MN x ¢ PQ
2

Figure 7.10c shows the response to an initial condition

[1 0 0]T

Comments 1. Underlying the separation principle is a critical assumption, namely, that the observer in-
cludes an exact dynamic model of the plant—the process under control. This assumption is almost never
valid in reality. In practical systems, the precise dynamic model is rarely known. Even that which is known
about the real process dynamics, is often too complicated to include in the observer. Thus, the observer
must, in practice, be configured to use only an approximate model of the plant. This encounter with the real
world does not vitiate the separation principle, but means that the effect of an inaccurate plant model must be
considered. If the design achieved through use of the separation principle is robust, it will be able to tolerate
uncertainty of the plant dynamics. Doyle and Stein [124] have proposed a ‘design adjustment procedure’ to
improve robustness with observers.
2. One of the considerations in the design of a gain matrix k in the state-feedback control law, is that the
resulting control signal u must not be too large; the use of large control effort increases the likelihood of the
system entering nonlinear regions of operation. Since the function of the observer is only to process data,
there is no limitation on the size of the gain matrix m for its realization. (Nowadays, it is all but certain that
the entire compensator would be realized by a digital computer. With floating-point numerics, a digital
computer would be capable of handling variables of any reasonable dynamic range). Though the realization
of observer may impose no limitation on the observer dynamics, it may, nevertheless, be desirable to limit
the observer speed of response (bandwidth). Remember that real sensors are noisy, and much of the noise
occurs at relatively high frequencies. By limiting the bandwidth of the observer, we can attenuate and
smoothen the noise contribution to the compensator output—which is the control signal.
3. The desired closed-loop poles, to be generated by state-feedback are chosen to satisfy the performance
requirements. The poles of the observer are usually chosen so that the observer response is much faster than
the system response. A rule of thumb is to choose an observer response at least two to five times faster than
the system response. This is to ensure a faster decay of estimation errors compared with the desired dynam-
ics, thus causing the closed-loop poles generated by state-feedback to dominate the total response. If the
sensor noise is large enough to be a major concern, one may decide to choose the observer poles to be slower
than two times the system poles, which would yield a system with lower bandwidth and more noise-smooth-
ing. However the total system response in this case will be strongly influenced by the observer poles. Doyle
and Stein [124] have shown that the commonly suggested approach of ‘speeding-up’ observer dynamics will
not work in all cases. They have suggested that procedures which drive some observer poles toward stable
plant zeros and the rest toward infinity achieve the desired objective.
4. A final comment concerns the reduced-order observer. Due to the presence of a direct transmission
term (see Fig. 7.7), the reduced-order observer has much higher bandwidth from sensor to control, com-
pared with the full-order observer. Therefore, if sensor noise is a significant factor, the reduced-order
observer is less attractive, since the potential savings in complexity is more than offset by the increased
sensitivity to noise.
 418 Digital Control and State Variable Methods

7.7 SERVO DESIGN: INTRODUCTION OF THE REFERENCE INPUT BY FEEDFORWARD

CONTROL

In the state regulator design studied in Section 7.4, the characteristic equation of the closed-loop system is
chosen so as to give satisfactory transients to disturbances. However, no mention is made of a reference
input or of design considerations to yield good transient response with respect to command changes. In
general, these considerations should be taken into account in the design of a control system. This can be
done by proper introduction of the reference input into the system equations.
Consider the completely controllable SISO linear time-invariant system with nth-order state variable model
x& (t) = Ax(t) + bu(t)
(7.60)
y(t) = cx(t)
We assume that all the n state variables can be accurately measured at all times. Implementation of appropri-
ately designed control law of the form
u(t) = – kx(t)
results in a state regulator system: any perturbation in the system state will asymptotically decay to the
equilibrium state x = 0.
Let us now assume that for the system given by Eqns (7.60), the desired steady-state value of the control-
led variable y(t) is a constant reference input r. For this servo system, the desired equilibrium state xs is a
constant point in state space and is governed by the equations
cxs = r (7.61)
We can formulate this command-following problem as a ‘shifted regulator problem’, by shifting the origin
of the state space to the equilibrium point xs. Formulation of the shifted regulator problem is as follows.
Let us be the needed input to maintain x(t) at the equilibrium point xs, i.e. (refer Eqns (7.60)),
0 = Axs + bus (7.62)
Assuming for the present that a us exists that satisfies Eqns (7.61)–(7.62), we define shifted input,
shifted state, and shifted controlled variable as
u~ (t) = u(t) – us
~
x (t) = x(t) – xs (7.63)
~
y (t) = y(t) – r
The shifted variables satisfy the equations
x& = A ~
~ x + b u~ (7.64)
~y = c~
x
This system possesses a time-invariant asymptotically stable control law
u~ = – k ~
x (7.65)
The application of this control law ensures that
~x ® 0 (x(t) ® xs, y(t) ® r)
In terms of the original state variables, total control effort
u(t) = – kx(t) + us + kxs (7.66)
Manipulation of Eqn. (7.62) gives
(A – bk)xs + b(us + kxs) = 0 or xs = – (A – bk)–1b(us + kxs)
or cxs = r = – c(A – bk)–1b(us + kxs)
 Pole-Placement Design and State Observers 419
w This equation has a unique solution for (us + kxs):
(us + kxs) = Nr
r + u y where N is a scalar feedforward gain, given by
N Plant
– (N)–1 = – c(A – bk)–1b (7.67)
The control law (7.66), therefore, takes the form
x
u(t) = – kx(t) + Nr (7.68)
k The block diagram of Fig. 7.11 shows the configu-
ration of feedback control system with feedforward
Fig. 7.11 Control configuration of a servo system compensation for non-zero equilibrium state.

Example 7.8 The system considered in this example is the attitude control system for a rigid satellite. The
plant equations are (refer Example 7.1)
x& = Ax + bu; y = cx
where

A=
LM0 1OP ; b = LM0OP ; c = [1 0]
N0 0Q N1Q
x1(t) = position q (t); x2(t) = velocity q& (t)
The reference input r = qr is a step function. The desired steady-state is
xs = [qr 0]T,
which is a non-null state.
As the plant has integrating property, the steady-state value us of the input must be zero (otherwise the
output cannot stay constant). For this case, the shifted regulator problem may be formulated as follows:
~
x =x –q; ~ x =x
1 1 r 2 2
Shifted state variables satisfy the equation
~
x& = A~x + bu
The state-feedback control
u = – k~
x
results in dynamics of ~
x given by
x& = (A – bk) ~
~ x
In Example 7.1, we found that the eigenvalues of (A – bk) are placed at the desired locations – 4 ± j4 when
k = [k1 k2] = [32 8]
The control law expressed in terms of the original state variables is given as
u = – k1 ~
x1 – k2 ~
x 2 = – k1x1 – k2x2 + k1qr = – kx + k1qr
~
As t approaches infinity, x ® 0 (x ® [q 0]T ), and u ® 0.
r
Figure 7.12 shows a configuration for attitude control of the satellite.
In fact, control configuration of the form shown in Fig. 7.12 may be used for any SISO plant with integrating
property.
 420 Digital Control and State Variable Methods

r + + u 1 x2 1 y = x1 =
k1
s s
– –

k2

Fig. 7.12 Attitude control of a satellite

7.8 STATE FEEDBACK WITH INTEGRAL CONTROL

Control configuration of Fig. 7.12 produces a generalization of proportional and derivative feedback but it
does not include integral control unless special steps are taken in the design process. One way to introduce
integral control is to augment the state vector with the desired integral. More specifically, for the system
(7.60), we can feedback the state x as well as the integral of the error in output by augmenting the plant state
x with the extra ‘integral state’ z, defined by the equation
t

z
z(t) = (y(t) – r)dt
0
(7.69a)

where r is the constant reference input of the system. Since z(t) satisfies the differential equation
z& (t) = y(t) – r = cx(t) – r (7.69b)
it is easily included by augmenting the original system (7.60) as follows:

LMx& OP = LMA 0OP LMxOP + LMbOP u + LM 0OP r (7.70)

N z&Q N c 0Q N z Q N0Q N-1Q
Since r is constant, in the steady-state x& = 0, z& = 0, provided that the system is stable. This means that the
steady-state solutions xs, zs and us must satisfy the equation
LM 0OP r = – LMA 0OP Lx O – LMbOP u
s

N-1Q N c 0Q MN z PQ N0Q s
s

Substituting this for the last term in Eqn. (7.70) gives

LMx& OP = LMA 0OP LMx - x OP + LMbOP (u – u )

s
s (7.71)
N z&Q N c 0Q N z - z Q N0Q s

Now define new state variables as follows, representing the deviations from the steady-state:

~
x =
LM
x - xs ~ OP
; u = u – us (7.72a)
z - zs
N Q
In terms of these variables, Eqn. (7.71) becomes
~ ~ + b u~
x& = Ax (7.72b)
 Pole-Placement Design and State Observers 421

A=
LMA 0OP , b = LMbOP
N c 0Q N0Q
The significance of this result is that, by defining the deviations from steady-state as state and control
variables, the design problem has been reformulated to be the standard regulator problem, with ~ x = 0 as the
desired state. We assume that an asymptotically stable solution to this problem exists, and is given by
u~ = – k ~
x
Partitioning k appropriately and using Eqns (7.72a) yields
k = [kp ki]

u – us = – [kp ki]
LMx - x OP = – k (x – x ) – k (z – z )
s
p s i s
Nz - z Q s
The steady-state terms must balance, therefore
t

z
u = – kp x – ki z = – kp x – ki (y(t) – r)dt
0
(7.73)

The control, thus, consists of proportional state feedback and integral control of output error. At steady-
state, ~
x& = 0; therefore
lim z& (t)® 0 or lim y(t) ® r
t ®¥ t®¥
Thus, by integrating action, the output y is driven to the no-offset condition.
This will be true even in the presence of constant disturbances acting on the plant. Block diagram of
Fig. 7.13 shows the configuration of feedback control system with proportional state feedback and integral
control of output error.
w

r + + u y
ki Plant
– –
x
kp

Fig. 7.13 State feedback with integral control

Example 7.9 Suppose the system is given by

Y ( s) 1
=
U ( s) s + 3
with a constant reference command signal. We wish to have integral control with closed-loop poles at wn = 5
and z = 0.5, which is equivalent to asking for a desired characteristic equation
s2 + 5s + 25 = 0
The plant model is
x& = – 3x + u; y = x
 422 Digital Control and State Variable Methods
Augmenting the plant state x with the integral state z—defined by the equation
t

z
z(t) = (y(t) – r)dt
0
we obtain
LM x& OP = LM-3 0OP LMxOP + LM1OP u + LM 0OP r
N z& Q N 1 0Q N z Q N0Q N-1Q
In terms of state and control variables representing deviations from the steady-state:

~
x =
LM
x - xs ~ OP
; u = u – us
z - zs
N Q
the state equation becomes
~ L-3 0OP ~x + LM1OP u~
x& = M
N 1 0 Q N0Q
We can find k from
F L-3 0O + L1O kIJ = s + 5s + 25
det G sI - M 2
H N 1 0PQ MN0PQ K
or s2 + (3 + k1)s + k2 = s2 + 5s + 25
Therefore
k = [2 25] = [kp ki]
The control
t

z
u = – kp x – ki z = – 2x – 25 (y(t) – r)dt
0
The control configuration is shown in Fig. 7.14, along with a disturbance input w. This system will behave
according to the desired closed-loop roots (wn = 5, z = 0.5) and will exhibit the characteristics of integral
control: zero steady-state error to a step r and zero steady-state error to a constant disturbance w.
w
+
r + 1 + 1 y
25
s s+3
– –

Fig. 7.14 Integral control example

7.9 DIGITAL CONTROL SYSTEMS WITH STATE FEEDBACK

This section covers the key results on the pole-placement design, and state observers for discrete-time
systems. Our discussion will be brief because of the strong analogy between the discrete-time and continu-
ous-time cases. Consider the discretized model of the given plant:
 Pole-Placement Design and State Observers 423
x(k + 1) = Fx(k) + gu(k)
y(k) = cx(k) (7.74)
where x is the n ´ 1 state vector, u is the scalar input, y is the scalar output; F, g, and c are, respectively,
n ´ n, n ´ 1 and 1 ´ n real constant matrices; and k = 0, 1, 2, ...
We will carry out the design of digital control system for the plant (7.74) in two steps. One step assumes
that we have all the elements of the state vector at our disposal for feedback purposes. The next step is to
design a state observer which estimates the entire state vector, when provided with the measurements of the
system indicated by the output equation in (7.74).
The final step will consist of combining the control law and the observer, where the control law calcula-
tions are based on the estimated state variables rather than the actual state.

7.9.1 State Regulator Design

Consider the nth-order, single-input, linear time-invariant system (7.74) with state-feedback control law
u(k) = – kx(k) (7.75)
where
k = [k1 k2 L kn]
The resulting closed-loop system is
x(k + 1) = (F – gk)x(k) (7.76)
If all the eigenvalues of (F – gk) are placed inside the unit circle in the complex plane, the state x(k) will
decay to the equilibrium state x = 0 irrespective of the value of x(0)—the initial perturbation in the state.
A necessary and sufficient condition for arbitrary placement of closed-loop eigenvalues (with the restric-
tion that complex eigenvalues occur in conjugate pairs), is that the system (7.74) is completely controllable.
The characteristic equation of the closed-loop system is
|zI – (F – gk)| = 0 (7.77a)
Assuming that the desired characteristic equation is
(z – l1)(z – l2) L (z – ln) = zn + a1zn –1 + L + an –1z + an = 0 (7.77b)
the required elements of k are obtained by matching coefficients in Eqns (7.77a) and (7.77b).
The calculation of the gains using this method becomes rather tedious when the order of the system is
greater than three. The algebra for finding the gains becomes especially simple when the state variable
equations are in controllable canonical form. A design procedure based on the use of controllable canonical
state variable model, is given below (refer Eqns (7.20)–(7.23)):
Step 1: From the characteristic polynomial of matrix F:
| zI – F | = zn + a1zn –1 + L + an –1 z + an (7.78)
determine the values of a1, a2, ..., an.
Step 2: Determine the transformation matrix P that transforms the system (7.74) into controllable canoni-
cal form:
LM p1 OP
pF PP ; p = [0 0 L 0 1] U −1
P=M
1 1
(7.79)
MM M PQ U = [g Fg L F n −1 g]
n-1
Np F1
Step 3: Using the desired eigenvalues (desired closed-loop poles) l1, l2, ..., ln, write the desired character-
istic polynomial:
(z – l1)(z – l2) L (z – ln) = zn + a1 zn –1 + L + an –1 z + an, (7.80)
and determine the values of a1, a2, L, an –1, an.
 424 Digital Control and State Variable Methods
Step 4: The required state-feedback gain matrix is determined from the following equation:
k = [an – an an –1 – an–1 L a1 – a1]P (7.81)
The Ackermann’s formula given below is more convenient for computer solution (refer Eqns (7.26)).
k = [0 0 L 0 1 ]U–1 f (F) (7.82)
n n–1
where f (F) = F + a1F + L + an – 1 F + anI
U = [g Fg L Fn –1g]

Example 7.10 Consider the problem of attitude control of a rigid satellite. A state variable model of the
plant is (refer Eqn. (7.6))

x& = Ax + bu =
LM0 1OP x + LM0OP u
N0 0 Q N1Q
where x1 = q is the attitude angle and u is the system input.
The discrete-time description of the plant (assuming that the input u is applied through a zero-order hold
(ZOH)) is given below (refer Section 6.3):
x(k + 1) = Fx(k) + gu(k) (7.83)
T
LM1 T OP ; g = e LM T /2 OP
2
F = eAT =
N0 1 Q z
At
where bdt =
0 NT Q
The characteristic equation of the open-loop system is
z-1 -T
| zI – F | = = (z – 1)2 = 0
0 z-1
With the control law
u(k) = – kx(k) = – [k1 k2] x(k)
the closed-loop system becomes
x(k + 1) = (F – gk)x(k)
The characteristic equation of the closed-loop system is
|zI – (F – gk)| = z2 + (Tk2 + (T2/2)k1 – 2) z + (T2/2)k1 – Tk2 + 1 = 0 (7.84a)
We assume that T = 0.1 sec, and the desired characteristic roots of the closed-loop system are z1.2 =
0.875 Ð ± 17.9º.
Note that these roots correspond to z = 0.5, and wn = 3.6 (refer Eqns (4.15)):
-zw n T ± jw n T 1 - z 2
z1, 2 = e e
The desired characteristic equation is then (approximately)
z2 – 1.6z + 0.70 = 0 (7.84b)
Matching coefficients in Eqns (7.84a) and (7.84b), we obtain
k1 = 10, k2 = 3.5

7.9.2 Design of State Observers

The control law designed in the last sub-section assumed that all states were available for feedback. Since,
typically, not all states are measured, the purpose of this sub-section is to show how to determine algorithms
 Pole-Placement Design and State Observers 425
which will reconstruct all the states, given measurements of a portion of them. If the state is x, then the
estimate is x$ and the idea is to let u = – k x$ ; replacing the true states by their estimates in the control law.
Prediction Observer An estimation scheme employing a full-order observer is shown in Fig. 7.15, and
the equation for it is
x$ (k + 1) = F x$ (k) + gu(k) + m(y(k) – c x$ (k)) (7.85)
where m is an n ´ 1 real constant gain matrix. We will call this a prediction observer because the estimate
x$ (k + 1) is one sampling period ahead of the measurement y(k).

u (k) x(k) y (k )
x (k + 1) = Fx (k) + gu( k) c

+ +
+ x(k + 1) x(k) y (k) –
g c
+
n-parallel
unit delayers

m
Observer
Fig. 7.15 Prediction observer
A difference equation describing the behaviour of the error is obtained by subtracting Eqn. (7.85) from
Eqn. (7.74):
~
x (k + 1) = (F – mc) ~ x (k) (7.86)
where
~
x = x – x$
The characteristic equation of the error is given by
| zI – (F – mc)| = 0 (7.87a)
Assuming that the desired characteristic equation is
(z – l1)(z – l2) L (z – ln) = 0, (7.87b)
the required elements of m are obtained by matching coefficients in Eqns (7.87a) and (7.87b). A necessary
and sufficient condition for the arbitrary assignment of eigenvalues of (F – mc), is that the system (7.74) is
completely observable.
The problem of designing a full-order observer is mathematically equivalent to designing a full state-
feedback controller for the ‘transposed auxiliary system’
z (k + 1) = FTz(k) + cTh(k) (7.88a)
with feedback
h(k) = – mTz(k) (7.88b)
so that the closed-loop auxiliary system
z (k + 1) = (FT – cTmT)z(k) (7.88c)
has eigenvalues li; i = 1, 2, ..., n.
 426 Digital Control and State Variable Methods
This duality principle may be used to design full-order state observers by Ackermann’s formula (7.82), or
by design procedure given in Eqns (7.78)–(7.81).
Current Observer The prediction observer given by Eqn. (7.85) arrives at the state estimate x$ (k) after
receiving measurements up through y(k – 1). Hence the control u(k) = – k x$ (k) does not utilize the information
on the current output y(k). For higher-order systems controlled with a slow computer, or any time the sample
rates are fast compared to the computation time, this delay between making a measurement and using it in
control law may be a blessing. In many systems, however, the computation time required to evaluate Eqn.
(7.85) is quite short—compared to the sample period—and the control based on prediction observer may not
be as accurate as it could be.
An alternative formulation of the state observer is to use y(k) to obtain the state estimate x$ (k). This can be
done by separating the estimation process into two steps. In the first step we determine x (k + 1), an approxi-
mation of x(k + 1) based on x$ (k) and u(k), using the model of the plant. In the second step, we use y(k + 1)
to improve x (k + 1). The improved x (k + 1) is x$ (k + 1). The state observer based on this formulation is
called the current observer. The current observer equations are given by
x (k + 1) = F x$ (k) + gu(k) (7.89a)
x$ (k + 1) = x (k + 1) + m[y(k + 1) – c x (k + 1)] (7.89b)
In practice, the current observer cannot be implemented exactly because it is impossible to sample, perform
calculations, and output with absolutely no time elapse. However, the errors introduced due to computa-
tional delays will be negligible if the computation time is quite short—compared to the sample period.
The error equation for the current observer is similar to the error equation for the prediction observer,
given in (7.86). The current-estimate error equation is obtained by subtracting Eqns (7.89) from (7.74).
~
x (k + 1) = x(k + 1) – x$ (k + 1)
= Fx(k) + gu(k) – F x$ (k) – gu(k) – mc[x(k + 1) – x (k + 1)]
= F~x (k) – mcF ~x (k) = (F – mcF) ~x (k) (7.90)
Therefore, the gain matrix m is obtained exactly as before, except that c is replaced by cF.
Reduced-Order Observer The observers discussed so far, are designed to reconstruct the entire state
vector, given measurements of some of the states. To pursue an observer for only the unmeasured states, we
partition the state vector into two parts: one part is x1 which is directly measured, and the other part is xe,
representing the state variables that need to be estimated. If we partition the system matrices accordingly, the
complete description of the system (7.74) is given by
LM
x1 ( k + 1)OP LMf
= 11 1e
f OP LM
x1 ( k ) OP LM OP
g
+ 1 u(k) (7.91a)
x e ( k + 1)
N Q N f e1 Fee x e ( k )
QN ge
Q N Q
y(k) = [1 0]
LM x ( k ) OP
1
(7.91b)
Nx ( k )Q
e
The portion describing the dynamics of unmeasured states is
xe(k + 1) = Fee xe(k) + f e1 x1 ( k ) + g e u( k ) (7.92)
144 42444 3
known input
The measured dynamics are given by the scalar equation
y ( k + 1) - f y ( k ) - g1u( k ) = f1exe(k) (7.93)
14444411 2444443
known measurement
Equations (7.92) and (7.93) have the same relationship to the state xe that the original equation (7.74) had to
the entire state x. Following this reasoning, we arrive at the desired observer by making the following sub-
stitutions into the observer equations:
 Pole-Placement Design and State Observers 427
x ¬ xe
F ¬ Fee
gu(k) ¬ fe1 y(k) + geu(k) (7.94)
y(k) ¬ y(k + 1) – f11 y(k) – g1u(k)
c ¬ f1e
Thus, the reduced-order observer equations are
x$ e (k + 1)= Fee x$ e (k) + f e1 y ( k ) + g e u ( k ) + m ( y(k + 1) - f11 y(k) - g1u(k ) - f1ex$ e (k )) (7.95)
144 42444 3 14444444 4244444444 3
input measurement
Subtracting Eqn. (7.95) from (7.92) yields the error equation
~
x e (k + 1) = (Fee – mf1e) ~
x e (k) (7.96)
where
~
x e = xe – x$ e
The characteristic equation is given by
|zI – (Fee – m f1e) | = 0 (7.97)
We design the dynamics of this observer by selecting m so that Eqn. (7.97) matches a desired reduced-order
characteristic equation. The design may be carried out directly or by using duality principle.

7.9.3 Compensator Design by the Separation Principle

If we implement the state-feedback control law using an estimated state vector, the control system can be
completed. A schematic of such a scheme, using a prediction observer1, is shown in Fig. 7.16. Note that by the
separation principle, the control law and the state observer can be designed separately, and yet used together.
The portion within the dotted line in Fig. 7.16 corresponds to dynamic compensation. The state variable
model of the compensator is obtained by including the state-feedback control (since it is a part of the com-
pensator) in the observer equations, yielding
x$ (k + 1) = (F – gk – mc) x$ (k) + my(k)
(7.98)
u(k) = – k x$ (k)
Plant Sensor
u x y
x (k + 1) = Fx (k) + gu ( k) c

x x (k + 1) = Fx (k) + gu( k)
–k
+ m [y(k) – cx (k)]
Control law Observer

Compensator
Fig. 7.16 Combined state-feedback control and state estimation

1
We will design the compensator only for the prediction observer case. The other observers give very similar results.
 428 Digital Control and State Variable Methods
The formula for conversion of a discrete-time state variable model to the transfer function model is given by
Eqn. (6.3). Applying this result to the model (7.98), we obtain
U(z)
= D(z) = k(zI – F + gk + mc)–1m (7.99)
-Y ( z )

Example 7.11 As an example of complete design, we will add a state observer to the satellite attitude
control considered in Example 7.10. The system equations of motion are (refer Eqn. (7.83))

x(k + 1) = Fx(k) + gu(k) =

1 T LM x(k) +
OP T 2/2 LMu(k)
OP
0 1 N Q T N Q
We assume that the position state x1 is measured and the velocity state x2 is to be estimated; the measurement
equation is therefore
y(k) = cx(k) = [1 0] x(k)
We will design a first-order observer for the state x2(k).
The partitioned matrices are
LM
f 11 f 1e
=
OP LM
1 T g
; 1 =
OP L O LM
T 2/2 OP
N
f e1 Fee Q N
0 1 ge Q MN PQ N
T Q
From Eqn. (7.97), we find the characteristic equation in terms of m:
z – (1 – mT) = 0
For the observer to be about four times faster than the control, we place the observer pole at
z = 0.5 (@ (0.835)4); therefore 1 – mT = 0.5
For: T = 0.1 sec, m = 5. The observer equation is (refer Eqn. (7.95))
T2
x$ 2 (k + 1) = x$ 2 (k) + Tu(k) + m(y(k + 1) – y(k) – u(k) – T x$ 2 (k))
2
= 0.5 x$ 2 (k) + 5(y(k + 1) – y(k)) + 0.075u(k)
Substituting for u(k) from the control law (refer Example 7.10)
u(k) = – 10y(k) – 3.5 x$ 2 (k), (7.100a)
we obtain
x$ 2 (k + 1) = 0.2375 x$ 2 (k) + 5y(k + 1) – 5.75y(k) (7.100b)
The two difference equations (7.100a) and (7.100b) complete the design and can be used to control the plant
to the desired specifications.
To relate the observer-based state-feedback design to a classical design, one needs to compute the
z-transform of Eqns (7.100a) and (7.100b), obtaining
U ( z) 27.5( z - 0.818)
=
-Y ( z ) z - 0.2375
The compensation looks very much like the classical lead compensation that would be used for 1/s2 plant.

7.9.4 Servo Design

Let us assume that for the system given by Eqns (7.74), the desired steady-state value for the controlled
variable y(k) is a constant reference input r. For this servo system, the desired equilibrium state xs is a
constant point in state space, and is governed by the equation
cxs = r (7.101a)
 Pole-Placement Design and State Observers 429
We formulate this command-following problem as a ‘shifted regulator problem’ by shifting the origin of
the state space to the equilibrium point xs. Let us be the needed input to maintain x(k) at the equilibrium point
xs, i.e. (refer Eqns (7.74)),
xs = Fxs + gus (7.101b)
Assuming for the present, that a us exists that satisfies Eqns (7.101a)–(7.101b), we define shifted input,
shifted state, and shifted controlled variable as
u~ (k) = u(k) – us
~
x (k) = x(k) – xs (7.102)
~
y (k) = y(k) – r
The shifted variables satisfy the equations
~
x (k + 1) = F ~x (k) + g u~ (k)
~ ~ (7.103)
y (k) = c x (k)
This system possesses a time-invariant asymptotically stable control law (assuming {F, g} is controllable)
u~ = – k ~x
The application of this control law ensures that
~
x (k) ® 0 (x(k) ® xs; y(k) ® r)
In terms of the original state variables, total control effort
u(k) = – kx(k) + us + kxs (7.104)
Manipulation of Eqn. (7.101b) gives
(F – gk – I)xs + g(us + kxs) = 0
or xs = – (F – gk – I)–1g(us + kxs)
or cxs = r = – c(F – gk – I)–1g(us + kxs)
This equation has a unique solution for (us + kxs):
(us + kxs) = Nr
where N is a scalar feedforward gain, given by
(N)–1 = – c(F – gk – I)–1g (7.105)
The control law (7.104), therefore, takes the form
u(k) = – kx(k) + Nr (7.106)

7.9.5 State Feedback with Integral Control

In the following, we study a control scheme for the system (7.74) where we feedback the state x as well as the
integral of the error in the output.
One way to introduce an integrator is to augment the plant state vector x with the ‘integral state’ v that
integrates the difference between the output y(k) and the constant reference input r. The ‘integral state’ v is
defined by
v(k) = v(k – 1) + y(k) – r (7.107a)
This equation can be rewritten as follows:
v(k + 1) = v(k) + y(k + 1) – r = v(k) + c[Fx(k) + gu(k)] – r
= cFx(k) + v(k) + cgu(k) – r (7.107b)
From Eqns (7.74) and (7.107b), we obtain
LMx(k + 1)OP = LM F 0OP LMx(k )OP + LM g OP u(k) + L 0O r (7.108)
Nv(k + 1)Q NcF 1Q Nv(k )Q NcgQ MN-1PQ
 430 Digital Control and State Variable Methods
Since r is constant, in the steady-state x(k + 1) = x(k) and v(k + 1) = v(k) provided that the system is stable.
This means that the steady-state solutions xs, vs and us must satisfy the equation
LM OP LM OP LM
0 x
r= s –
F 0 xs OP L O LM OP
–
g
u
-1 N Q N Q N
vs cF 1 v s Q MN PQ N Q
cg s
Substituting this for the last term in Eqn. (7.108) gives
~
x (k + 1) = F x~ (k) + g u~ (k) (7.109)
where
~
x =
LMx - xs OP
; u~ = u – us
Nv - vs Q
F =M
LF 0O
P1Q ; g = LMNcgOPQ
g
NcF
The significance of this result is that by defining the deviations from steady-state as state and control
variables, the design problem has been reformulated to be the standard regulator problem, with ~ x = 0 as the
desired state. We assume that an asymptotically stable solution to this problem exists and is given by
u~ (k) = – k ~
x (k)
Partitioning k appropriately and using Eqn. (7.109) yields
k = [kp ki]

u – us = – [kp ki]
LM
x - xs OP
= – kp(x – xs) – ki(v – vs)
N
v - vs Q
The steady-state terms must balance, therefore
u(k) = – kp x(k) – kiv(k) (7.110)
~ ~
At steady-state, x (k + 1) – x (k) = 0; therefore
v(k + 1) – v(k) = 0 = y(k) – r, i.e., y(k) ® r
The block diagram of Fig. 7.17 shows the control configuration.
w

r + + – v (k ) + u (k) y (k)
ki Plant
– + –

x(k)
kp
– v(k – 1)

Fig. 7.17 State feedback with integral control

Example 7.12 Consider the problem of digital control of a plant described by the transfer function
1
G(s) =
s+3
 Pole-Placement Design and State Observers 431
Discretization of the plant model gives

Gh0G(z) =
Y (z)
=Z
LMF 1 - e I F 1 I OP- sT

U(z) MNGH s JK H s + 3K PQ
= (1 – z–1 )ZM
L 1 OP = 1 FG 1 - e IJ -3T

-3T
N s( s + 3) Q 3 H z - e K
For a sampling interval T = 0.1 sec,
0.0864
Gh0G(z) =
z - 0.741
The difference equation model of the plant is
y(k + 1) = 0.741y(k) + 0.0864u(k)
The plant has a constant reference command signal. We wish to design a PI control algorithm that results in
system response characteristics: z = 0.5, wn = 5. This is equivalent to asking for the closed-loop poles at
2
z1,2 = e -zw n Te ± jw n T 1- z = 0.7788 Ð ± 24.82º = 0.7068 ± j0.3269
The desired characteristic equation is therefore
(z – 0.7068 – j0.3269)(z – 0.7068 + j0.3269) = z2 – 1.4136z + 0.6065 = z2 + a1z + a2 = 0
Augmenting the plant state y(k) with the ‘integral state’ v(k) defined by
v(k) = v(k – 1) + y(k) – r,
we obtain
LM
y ( k + 1)
=
OP LM
OP LM OP LM OP LM OP
0.741 0 y ( k )
+
0.0864
u(k) +
0
r
N
v( k + 1) Q NQN Q N Q N Q
0.741 1 v( k ) 0.0864 -1
In terms of state variables representing deviations from the steady-state:
~
x =M
L y - y OP , s

Nv - v Q s
the state equation becomes
x (k + 1) = F ~
~ x (k) + g u~ (k)
where

F =
LM0.741 0OP ; g = LM0.0864OP
N0.741 1Q N0.0864Q
By Ackermann’s formula (7.82),
k = [0 1]U–1 f ( F )
where

f ( F ) = F 2 + a1 F + a2 I =
LM 0108
. 0 OP
N0.2425 01929
. Q
-1
U –1
= [g F g] -1 L0.0864
=M
0.064 OP =
1 LM 0.15 -0.064 OP
N0.0864 0.15 Q 7.43 ´ 10 - 3 N-0.0864 0.0864 Q
This gives
k = [1.564 2.243]
 432 Digital Control and State Variable Methods
The control algorithm is given by
u(k) = – 1.564y(k) – 2.243v(k)

7.10 DEADBEAT CONTROL BY STATE FEEDBACK AND DEADBEAT OBSERVERS

A completely controllable and observable SISO system of order n is considered.

x(k + 1) = Fx(k) + gu(k)
(7.111)
y(k) = cx(k)
With the state-feedback control law
u(k) = – kx(k) (7.112a)
the closed-loop system becomes
x(k + 1) = (F – gk)x(k) (7.112b)
with the characteristic equation
| zI – (F – gk)| = 0 (7.112c)
The control-law design consists of picking the gains k so that Eqn. (7.112c) matches the desired character-
istic equation
zn + a1zn – 1 + L + an–1z + an = 0
A case of special interest occurs when a1 = a2 = L = an –1 = an = 0, that is, desired characteristic equation is
zn = 0 (7.113)
By the Cayley-Hamilton theorem (a matrix satisfies its own characteristic equation),
(F – gk)n = 0
This result implies that the force-free response of closed-loop system (7.112b),
x(k) = (F – gk)kx(0) = 0 for k ³ n
In other words, any initial state x(0) is driven to the equilibrium state x = 0 in (at most) n steps. The
feedback control law that assigns all the closed-loop poles to origin is, therefore, a deadbeat control law.
A state observer defined by the equation
x$ (k + 1) = F x$ (k) + gu(k) + m[y(k) – c x$ (k)] (7.114)
$
gives an estimate x (k) of the state x(k); the observer design procedure consists of picking the gains m so that
the error system
~
x (k + 1) = (F – mc) ~ x (k) (7.115)
has the desired characteristic equation. A case of special interest occurs when all the observer poles (i.e.,
eigenvalues of (F – mc)) are zero. In analogy with the deadbeat control law, we refer to observers with this
property as deadbeat observers.
Comments The concept of deadbeat performance is unique to discrete-time systems. By deadbeat con-
trol, any nonzero error vector will be driven to zero in (at most) n sampling periods if the magnitude of the
scalar control u(k) is unbounded. The settling time depends on the sampling period T. If T is chosen very
small, the settling time will also be very small, which implies that the control signal must have an extremely
large magnitude. The designer must choose the sampling period for which an extremely large control
magnitude is not required in normal operation of the system. Thus, in deadbeat control, the sampling period
is the only design parameter.

Example 7.13 The system considered in this example is the attitude control system for a rigid satellite.
The plant equations are (refer Example 7.10)
x(k + 1) = Fx(k) + gu(k)
 Pole-Placement Design and State Observers 433
where

F=
LM1 T OP ; g = LM T /2 OP 2

N0 1 Q N T Q
x1(k) = position state q ; x2(k) = velocity state w
The reference input r = qr, a step function. The desired steady-state
xs = [qr 0]T
which is a non-null state.
As the plant has integrating property, the steady-state value us of the input must be zero (otherwise the
output cannot stay constant). For this case, the shifted regulator problem may be formulated as follows:
~
x =x –q; ~ x =x
1 1 r 2 2
Shifted state variables satisfy the equations
~
x (k + 1) = F ~
x (k) + gu(k)
The state-feedback control
u(k) = – k ~
x (k)
results in the dynamics of ~
x given by
~
x (k + 1) = (F – gk) ~
x (k)
We now determine the gain matrix k such that the response to an arbitrary initial condition is deadbeat. The
desired characteristic equation is
z2 = 0
Using Ackermann’s formula (7.82), we obtain
k = [0 1]U–1 f (F)
where

L1 2TOP ; U LM- 1 2
3 OP
f (F) = F 2
=M –1
= [g Fg] –1
=M T 2T
N0 1Q
MN T1 -
1 PP
2
2T Q
This gives

k= LM 1 3 OP
NT2
2T Q
For T = 0.1 sec, k = [100 15]
The control law expressed in terms of original state variables is given as
u(k) = – k1 ~
x1 (k) – k2 ~
x 2 (k) = – 100(x1(k) – qr) – 15x2(k)

Example 7.14 Reconsider the problem of attitude control of a satellite. For implementation of the design
of the previous example, we require the states x1(k) and x2(k) to be measurable. Assuming that the output y(k)
 434 Digital Control and State Variable Methods
= x1(k) is the only state variable that can be measured, we design a state observer for the system. It is desired
that the error vector exhibits deadbeat response. The measurement equation is
y(k) = cx(k) = [1 0]x(k)
The prediction observer for the system is given as
x$ (k + 1) = F x$ (k) + gu(k) + m(y(k) – c x$ (k))
The gains m may be calculated by solving the state regulator design problem for the ‘transposed auxiliary
system’
z(k + 1) = FTz(k) + cTh(k)
h(k) = – mTz(k)
The desired characteristic equation is
z2 = 0
Using Ackermann’s formula, we obtain
mT = [0 1]U–1 f (FT)
where

f (FT) = (FT)2 =
LM 1 0OP ; U –1
= [cT FTcT]–1 =
LM1 OP
-1/ T
N2T 1Q N0 1/ TQ
This gives
mT = [2 1/T]

For T = 0.1 sec, m=

LM 2 OP
N10Q
Review Examples

Review Example 7.1 DC motors are widely used in speed-control drives. In most applications, the
armature voltage of the motor is controlled in a closed-loop feedback system. Figure 7.18a shows a plant
model of a speed control system.
The state variables of the plant can be chosen as the motor shaft velocity w (t), and the armature current
ia(t). If both the state variables are used in feedback, then two voltages proportional, respectively, to these
two state variables must be generated. The generation of the voltage proportional to w (t) can be achieved by
use of a tachogenerator. A voltage proportional to ia(t) can be generated by inserting a sampling resistor Rs
in the armature circuit, as shown in Fig. 7.18a. It may, however, be noted that if Rs is very small, the voltage
across Rs may consist largely of noise; and if Rs is large, the voltage is more accurate, but a considerable
amount of power is wasted in Rs and the efficiency of the system is reduced.
In modern speed-control drives, thyristor rectifier is used as a power amplifier [5]. The thyristor rectifier
is supplied by an external single-phase or three-phase ac power, and it amplifies its input voltage u, to
produce an output voltage ea, which is supplied to the armature of the dc motor. The state-feedback control,
requiring the feedback of both the motor-shaft velocity and the armature current can, in fact, be effectively
used to provide current-limiting protective feature to prevent damage to the thyristors.
 Pole-Placement Design and State Observers 435
ac supply

La Ra
Thyristor
rectifier
ia
u ea Motor

Tacho

TL
,J
,B
Rs

(a)
x 2 = ia TL

–
u + 1 + 1
Kr KT
La s + R a + R s Js + B
–

Kb

Rs
x1 =
Kt

(b)
Fig. 7.18 Plant model of a speed-control system
The voltage u is fed to the driver of the thyristor rectifier. The driver produces time-gate pulses that
control the conduction of the thyristors in the rectifier module. The rectified output voltage ea depends on
the firing angle of the pulses relative to the ac supply waveform. A linear relationship between the input
voltage u and the output voltage ea can be obtained when a proper firing control scheme is used. The time
constants associated with the rectifier are negligibly small. Neglecting the dynamics of the rectifier, we get
ea(t) = Kr u(t)
where Kr is the gain of the rectifier.
Figure 7.18b shows the functional block diagram of the plant with
B = viscous-friction coefficient of motor and load,
J = moment of inertia of motor and load,
KT = motor torque constant,
Kb = motor back-emf constant,
TL = constant load torque,
La = armature inductance, and
Ra + Rs = armature resistance.
 436 Digital Control and State Variable Methods
As seen from Fig. 7.18b, the plant is a type-0 system. A control law of the form
u(t) = – kx + Nr
can shape the dynamics of the state variables x1(t) = w (t) and x2(t) = i a(t) with zero steady-state error in
w (t) to constant reference input r. The closed-loop system will, however, be a type-0 system resulting in
steady-state errors to constant disturbances. We assume that steady-state performance specifications require
a type-1 system. Hence we employ state feedback with integral control. A block diagram of the control
configuration is shown in Fig. 7.19.

wr er + k1 k3 + u
Kt + Plant
Kt sK t
– –
ia

k2 /R s Rs

Kt

Fig. 7.19 Control configuration for a speed-control system

The state equations of the plant are

J x&1 + Bx1 = KT x2 – TL
&
La x 2 + (Ra + Rs)x2 = Kr u – Kb x1
or x& = Ax + bu + g TL
where
LM - B KT
0
OP 1 LM OP LM OP
-
A = M KJ ( R
J
+ R ) ; b = Kr ; g =
PP J
b
MMN- L - a s
La 0 MN PQ MN PQ
a La PQ
Let the parameter values be such that these matrices become

A=
-0.5LM 10 OP ;b=
0
;g =
-10 LM OP LM OP
N
-0.1 -10 Q 100 0 N Q N Q
We define an additional state variable x3 as
t
x3 = z
0
(w – r) dt,

i.e., x& 3 = w – r = x1 – r
Augmenting this state variable with the plant equations, we obtain
x& = A x + b u + Gw
where
x = [x1 x2 x3]T; w = [TL r]T
LM-0.5 10 0 OP 0 -10 0LM OP LM OP
A = M -0.1 -10 0 ; b = 100 ; G =
PP 0 0
MM PP MM PP
MN 1 0 0 Q 0 0 -1 N Q N Q
 Pole-Placement Design and State Observers 437
The controllability matrix
U = [b A b A 2b]
LM 0 1,000 -10,500 OP
= 100 -1,000 9,900
MM 0 1,000QP
P
N 0
The determinant of U is nonzero. The pair ( A, b ) is, therefore, completely controllable and the conditions
for pole placement by state feedback and integral control, are satisfied.
The characteristic polynomial of the closed-loop system is given by
LM s + 0.5 -10 0 OP
| sI - ( A - b k )| = 0.1 + 100k s + 10 + 100 k2 100k3
MM -1 1 PP
N 0 s Q
= s3 + (10.5 + 100k2) s2 + (6 + 50k2 + 1,000k1)s + 1,000k3 (7.116a)
Let the desired characteristic polynomial be
s3 + 87.5 s2 + 5,374.5 s + 124,969 = (s + 35.4)(s + 26.05 + j53.4)(s + 26.05 – j53.4) (7.116b)
The quadratic term has a natural frequency wn = 59.39 rad/sec, and a damping ratio z = 0.44.
Matching the corresponding coefficients of Eqns (7.116a) and (7.116b), we obtain
k1 = 5.33, k2 = 0.77, k3 = 124.97
With these values of the feedback gains, the state variable model of the closed-loop system becomes
LM x& OP LM -0.5
1 10 OP LM x OP LM-10
0 1 0 OP LT O
L
MM x&& PP = M-5331.
2 -87 -12497 M x P +
P 2 M 0 0 M P
P NrQ
N x Q MN 1
3 0 0PQ MN x PQ MN 0
3 -1PQ
At steady-state, x& = 0 and, therefore, the motor velocity x1 = w (t) will approach the constant reference set
point r as t approaches infinity, independent of the disturbance torque TL.

Review Example 7.2 One of the most common uses of feedback control is to position an inertia load
using an electric motor. The inertia load may consist of a very large, massive object such as a radar antenna
or a small object such as a precision instrument. Armature-controlled dc motors are used in many applica-
tions for positioning the load.
We consider here a motor-driven inertia system described by the following equations (refer Eqns (5.14)).
dq(t )
u(t) = Ra ia(t) + Kb w (t) = Ra ia(t) + Kb
dt
dw (t ) d 2q (t )
KT ia(t) = J =J
dt dt 2
where
u = applied armature voltage,
Ra = armature resistance,
ia = armature current,
q = angular position of the motor shaft,
 438 Digital Control and State Variable Methods
w = angular velocity of the motor shaft,
Kb = back emf constant,
KT = motor torque constant, and
J = moment of inertia referred to the motor shaft.
Taking x1 = q, and x2 = q& = w as the state variables, we obtain the following state variable equations for the
system.
x&1 = x2
KT Kb K
x& 2 = – x2 + T u = – a x 2 + b u
JRa JRa
Assume that the physical parameters of the motor and the load yield a = 1, b = 1. Then
x& = Ax + bu
where

A=
LM0 1OP ; b = LM0OP
N0 -1Q N1Q
The discrete-time description of this system, with sampling period T = 0.1 sec, is given by the following
equations (refer Section 6.3).
x(k + 1) = Fx(k) + gu(k) (7.117)
where
T
F = eAT =
LM1 0.0952 OP z
; g = eAt bdt =
0.00484 LM OP
N0 0.905 Q 0
0.0952 N Q
In this model, x1(k) is the shaft position and x2(k) is the shaft velocity. We assume that x1(k) and x2(k) can
easily be measured using shaft encoders.
We choose the control configuration of Fig. 7.20 for digital positioning of the load; qr is a constant
reference command. In terms of the error variables
~
x (k) = x (k) – q ; ~x (k) = x (k) (7.118)
1 1 r 2 2
the control signal
u(k) = – k1 ~ x 2 (k) = – k~
x1 (k) – k2 ~ x (k) (7.119)
where the gain matrix
k = [k1 k2]

r + + u (k )
k1 ZOH Plant
– –
T = 0.1 sec
x2 (k )
k2
T = 0.1 sec
x 1 (k )

Fig. 7.20 Control configuration for a digital positioning system

 Pole-Placement Design and State Observers 439
The dynamics of the error-vector ~x (k) are given by the equations
~
x1 (k + 1) = x1(k + 1) – qr = x1(k) + 0.0952 x2(k) + 0.00484 u(k) – qr
=~x (k) + 0.0952 ~
1 x (k) + 0.00484 u(k)
2
~
x 2 (k + 1) = 0.905 ~ x 2 (k) + 0.0952 u(k)
or ~
x (k + 1) = F ~x (k) + gu(k)
where F and g are given by Eqn. (7.117).
Substituting for u(k) from Eqn. (7.119), we obtain the following closed-loop model of the error dynamics:
~
x (k + 1) = (F – gk) ~ x (k)

=
LM1 - 0.00484k 1 OP
0.0952 - 0.00484 k2 ~
x (k) (7.120)
N - 0.0952k 1 0.905 - 0.0952 k2Q
The characteristic equation is
| zI – (F – gk) | = z2 + (0.00484k1 + 0.0952k2 – 1.905)z + 0.00468k1 – 0.0952k2 + 0.905 = 0 (7.121a)
We choose the desired characteristic-equation zero locations, to be
z1, 2 = 0.888 ± j0.173 = 0.905 Ð ± 11.04º
Note that this corresponds to z = 0.46 and wn = 2.17 (refer Eqns (4.15)):
-zw n T ± jw n T 1 - z 2
z1, 2 = e e
The desired characteristic equation is given by
(z – 0.888 – j0.173)(z – 0.888 + j0.173) = z2 – 1.776z + 0.819 = 0 (7.121b)
Equating coefficients in Eqns (7.121a) and (7.1121b) yields the equations
0.00484k1 + 0.0952k2 = – 1.776 + 1.905
0.00468k1 – 0.0952k2 = 0.819 – 0.905
These equations are linear in k1 and k2 and upon solving yield
k1 = 4.52, k2 = 1.12
The control law is therefore given by
u(k) = – k1 ~
x1 (k) – k2 ~
x 2 (k) = – 4.52(x1(k) – qr) – 1.12x2(k)
The implementation of this control law, requires the feedback of the states x1(k) and x2(k). If we measure
x1(k) using a shaft encoder and estimate x2(k) using a state observer, the control configuration will take the
form shown in Fig. 7.21.
The state-feedback control has been designed for z = 0.46, wn = 2.17; zwn @ 1 sec. The reduced-order
observer for estimating velocity x2(k) from measurements of position x1(k) is a first-order system; we choose
the time constant of this system to be 0.5 sec. Hence, the desired pole location3 in the observer design
problem is
z = e–T/t = e– 0.1/0.5 = 0.819
The observer characteristic equation is then
z – 0.819 = 0 (7.122)

3
The pole at s = – 1/t is mapped to z = e–T/t; T = sampling interval.
 440 Digital Control and State Variable Methods

+ + u (k)
r
k1 ZOH Plant
– –
T = 0.1 sec

x 2 (k ) x 1 (k )
k2 Observer

Fig. 7.21 Observer-based digital positioning system

From the plant state equation (7.117), and Eqns (7.91), the partitioned matrices are seen to be
f11= 1, f1e = 0.0952, fe1 = 0, Fee = 0.905, g1 = 0.00484, ge = 0.0952
The observer equation is (refer Eqn. (7.95))
x$ 2 (k + 1) = 0.905 x$ 2 (k) + 0.0952u(k) + m(q (k + 1) – q (k) – 0.00484u(k) – 0.0952 x$ 2 (k))
= (0.905 – m(0.0952)) x$ 2 (k) + mq (k + 1) – mq (k) + (0.0952 – m(0.00484))u(k) (7.123)
The characteristic equation is given by
z – (0.905 – 0.0952m) = 0
Comparing the coefficients with those of Eqn. (7.122), we obtain
m = 0.903
Substituting in Eqn. (7.123), we get
x$ 2 (k + 1) = 0.819 x$ 2 (k) + 0.903q (k + 1) – 0.903q(k) + 0.0908u(k)
The control system is implemented as follows. A measurement q (k) is made at t = kT. The observer state is
calculated from
x$ 2 (k) = 0.819 x$ 2 (k – 1) + 0.903q(k) – 0.903q(k – 1) + 0.0908u(k – 1)
Then the control input is calculated, using
u(k) = – 4.52(q(k) – qr) – 1.12 x$ 2 (k)

Problems
7.1 Consider an nth-order single-input, single-output system
x& = Ax + bu; y = cx
and assume that we are using feedback of the form
u = – kx + r
where r is the reference input signal.
Show that the zeros of the system are invariant under state-feedback.
7.2 A regulator system has the plant
LM 0 1 0OP LM0OP
x& = 1 x + 0 u; y = [1 0 0] x
MM-60 0
-6 PQ
P MM1PP
N -11 NQ
 Pole-Placement Design and State Observers 441
(a) Design a state-feedback controller which will place the closed-loop poles at – 2 ± j3.464, – 5. Give a block
diagram of the control configuration.
(b) Design a full-order state observer; the observer-error poles are required to be located at – 2 ± j3.464, – 5.
Give all the relevant observer equations and a block diagram description of the observer structure.
(c) The state variable x 1(which is equal to y) is directly measurable and need not be observed. Design
a reduced-order state observer for the plant; the observer-error poles are required to be located at
– 2 ± j3.464. Give all the relevant observer equations.
7.3 A regulator system has the plant
x& = Ax + bu; y = cx
with
LM0 0 -6 OP LM1 OP
A= 1 0 -11 ; b = 0 ; c = [0 0 1]
MM0 -6 QP
P MM0 PP
N 1 NQ
(a) Compute k so that the control law u = – kx, places the closed-loop poles at – 2 ± j3.464, – 5. Give the state
variable model of the closed-loop system.
(b) For the estimation of the state vector x, we use an observer defined by
x$& = (A – mc) x$ + bu + my
Compute m so that the eigenvalues of (A – mc) are located at – 2 ± j3.464, – 5.
(c) The state variable x3 (which is equal to y) is directly measurable and need not be observed. Design a
reduced-order observer for the plant; the observer-error poles are required to be located at – 2 ± j3.464.
Give all the relevant observer equations.
7.4 Consider the system
x& = Ax + Bu; y = cx + du
where

A=
LM-2 -1OP ; B = LM1 0OP ; c = [0 1]; d = [2 0]
N 1 0Q N1 1Q
Design a full-order state observer so that the estimation error will decay in less than 4 seconds.
7.5 Consider the system

x& =
LM1 0OP x + LM1OP u; y = [2 – 1] x
N0 0Q N1Q
Design a reduced-order state observer that makes the estimation error to decay at least as fast as e–10t.
7.6 Consider the system with the transfer function
Y (s ) 9
=
U ( s) s 2 - 9
(a) Find (A, b, c) for this system in observable canonical form.
(b) Compute k so that the control law u = – kx places the closed-loop poles at – 3 ± j3.
(c) Design a full-order observer such that the observer-error poles are located at – 6 ± j6. Give all the relevant
observer equations.
(d) Suppose the system has a zero such that
Y ( s) 9(s + 1)
= 2
U(s) s -9
Prove that if u = – kx + r, there is a feedback matrix k such that the system is unobservable.
442 Digital Control and State Variable Methods
7.7 The equation of motion of an undamped oscillator with frequency w 0 is
y + w 20 y = u
&&
(a) Write the equations of motion in the state variable form with x1 = y and x2 = y& as the state variables.
(b) Find k1 and k2 such that u = – k1x1 – k2x2 gives closed-loop characteristic roots with wn = 2w 0 and z = 1.
(c) Design a second-order observer that estimates x1 and x2, given measurements of x1. Pick the characteristic
roots of the state-error equation with wn = 10w0 and z = 1. Give a block diagram of the observer-based
state-feedback control system.
(d) Design a first-order observer that estimates x2, given measurements of x1. The characteristic root of
the state-error equation is required to be located at –10w0. Give a block diagram of the observer-based
state-feedback control system.
7.8 A regulator system has the plant

x& =
LM 0 1OP x + LM0OP u; y = [1 0] x
N20.6 0Q N1Q
(a) Design a control law u = – kx so that the closed-loop system has eigenvalues at –1.8 ± j2.4.
(b) Design a full-order state observer to estimate the state vector. The observer matrix is required to have
eigenvalues at – 8, – 8.
(c) Find the transfer function of the compensator obtained by combining (a) and (b).
(d) Find the state variable model of the complete observer-based state-feedback control system.
7.9 A regulator system has the double integrator plant
Y (s) 1
=
U(s) s 2
(a) Taking x1 = y and x2 = y& as state variables, obtain the state variable model of the plant.
(b) Compute k such that u = – kx gives closed-loop characteristic roots with wn = 1, z = 2 / 2 .
(c) Design a full-order observer that estimates x1 and x2, given measurements of x1. Pick the characteristic
roots of the state-error equation with wn = 5, z = 0.5.
(d) Find the transfer function of the compensator obtained by combining (b) and (c).
(e) Design a reduced-order observer that estimates x2 given measurements of x1; place the single observer pole
at s = – 5.
(f) Find the transfer function of the compensator obtained by combining (b) and (e).
7.10 A servo system has the type-1 plant described by the equation
x& = Ax + bu; y = cx
where
LM0 1 OP LM0OP
0
A= 0 -1 1 ; b = 0 ; c = [1 0 0]
MM0 -2PQ
P MM1PP
N 0 NQ
(a) If u = – kx + Nr, compute k and N so that the closed-loop poles are located at –1 ± j1, – 2; and y(¥) = r, a
constant reference input.
(b) For the estimation of the state vector x, we use a full-order observer
x&$ = (A – mc) x$ + bu + my
Compute m so that observer-error poles are located at – 2 ± j2, – 4.
(c) Replace the control law in (a) by u = – k x$ + Nr, and give a block diagram of the observer-based servo system.
7.11 A plant is described by the equation

x& =
LM-1 0OP x + LM1OP u; y = [1 3] x
N 0 -2Q N1Q
 Pole-Placement Design and State Observers 443
Add to the plant equations an integrator z& = y – r (r is a constant reference input) and select gains k, ki so that if
u = – kx – ki z, the closed-loop poles are at – 2, –1 ± j 3 . Give a block diagram of the control configuration.
7.12 Figure P7.12 shows the block diagram of a position control system employing a dc motor in armature control
mode; q (rad) is the motor shaft position, q& (rad/sec) is the motor shaft velocity, ia (amps) is the armature
current, and KP (volts/rad) is the sensitivity of the potentiometer. Find k1, k2 and k3 so that the dominant poles of
the closed-loop system are characterized by z = 0.5, wn = 2, non-dominant pole is at s = – 10; and the steady-state
error to constant reference input is zero.

KP
u x3 = ia x2 = x1 =

+ k1 + + 1 1 1
KP 0.1 s + 1 s+1 s
– – – –

0.1
k3

k2

KP

Fig. P7.12

7.13 A dc motor in armature control mode has been used in speed control system of Fig. P7.13 employing state-
feedback with integral control; w (rad/sec) is the motor shaft velocity, ia (amps) is the armature current and Kt
(volts/(rad/sec)) is the tachogenerator constant. Find k1, k2 and k3 so that the closed-loop poles of the system are
placed at –1 ± j 3 , –10; and the steady-state error to constant reference input is zero.

Kt
u x2 = i a x1 =

+ k1 k3 + + 1 1
+
Kt sK t 0.1 s + 1 s+1
– – –

0.1
k2

Kt

Fig. P7.13
444 Digital Control and State Variable Methods
7.14 The control law u = – kx – k1qr for position control system of Fig. P7.12 is to be replaced by u = – k x$ + k1qr
where x$ is the estimate of the state vector x given by the observer system
x&$ = (A – mc) x$ + bu + mq
Find the gain matrix m which places the eigenvalues of (A – mc) at – 3 ± j 3 , – 10. Give a block diagram of the
observer-based position control system.
7.15 Consider the position control system of Fig. P7.15 employing a dc motor in armature control mode with state
variables defined on the diagram. Full state-feedback is employed, with position feedback being obtained from
a potentiometer, rate feedback from a tachogenerator and current feedback from a voltage sample across a
resistance in the armature circuit. KA is the amplifier gain. Find the adjustable parameters KA, k2, and k3 so that
the closed-loop poles of the system are placed at – 3 ± j3, – 20.

x1 = K P
k1 = 1 Pot

–
r La Ra
KP +
KA
–
– ia

J, B

x3 = 0.1 ia
k3

0.1

x2 = K t
k2 Tacho

Fig. P7.15

Given:
Potentiometer sensitivity, KP = 1 volt/rad
Tachogenerator constant, Kt = 1 volt/(rad/sec)
Armature inductance, La = 0.005 H
Armature resistance, Ra = 0.9 W
Moment of inertia of motor and load, J = 0.02 N-m/(rad/sec2)
Viscous-friction coefficient of motor and load, B=0
Back emf constant, Kb = 1 volt/(rad/sec)
Motor torque constant, KT = 1 N-m/amp
7.16 Consider the position control system of Fig. P7.16 employing a dc motor in the field control mode, with state
variables defined on the diagram. Full state-feedback is employed with position feedback being obtained from a
potentiometer, rate feedback from a tachogenerator and current feedback from a voltage sample across a resistor
connected in the field circuit. KA is the amplifier gain.
 Pole-Placement Design and State Observers 445
Find the adjustable parameters KA, k2, and k3 so that the closed-loop system has dominant poles characterized by
z = 0.5, wn = 2, and the third pole at s = – 10.
Given:
Potentiometer sensitivity, KP = 1 volt/rad
Tachogenerator constant, Kt = 1 volt/(rad/sec)
Field inductance, Lf = 20 H
Field resistance, Rf = 99 W
Moment of inertia of motor and load, J = 0.5 N-m/(rad/sec2)
Viscous-friction coefficient of motor and load, B = 0.5 N-m/(rad/sec)
Motor torque constant, KT = 10 N-m/amp

x1 = K P
k1 = 1 Pot

–
r
KP Rf
+ Ia
KA
–
–

Lf
J, B

x 3 = if
k3

x2 = K t
k2 Tacho

Fig. P7.16

7.17 Figure P7.17 shows control configuration of a type-1 servo system. Both the state variables x1 and x2, are
assumed to be measurable. It is desired to regulate the output y to a constant value r = 5. Find the values of k1,
k2 and N so that
(i) y(¥) = r = 5
(ii) The closed-loop characteristic equation is
s2 + a1s + a2 = 0
7.18 A speed control system, employing a dc motor in the armature control mode, is described by the following state
equations:
dw (t ) B K 1
= – w (t ) + T ia (t ) - TL
dt J J J
dia ( t ) K R 1
= – b w (t ) - ia (t ) + u(t )
dt L L L
446 Digital Control and State Variable Methods

r + u y = x1
N
s(s + )
–

x2
k2 s

k1

Fig. P7.17

where
ia(t) = armature current, amps
u(t) = armature applied voltage, volts
w (t) = motor velocity, rad/sec
B = viscous-friction coefficient of motor and load = 0
J = moment of inertia of motor and load = 0.02 N-m/(rad/sec2)
KT = motor torque constant = 1 N-m/amp
Kb = motor back emf constant = 1 volt/(rad/sec)
TL = constant disturbance torque (magnitude not known)
L = armature inductance = 0.005 H
R = armature resistance = 1 W
The design problem is to find the control u(t) such that
di (t ) dw(t )
(i) lim a = 0 and lim = 0, and (ii) lim w (t) = constant set-point r.
t ®¥ dt t ®¥ dt t ®¥
Show that the control law of the form
t

z
u(t) = – k1w (t) – k2ia(t) – k3 (w (t) – r)dt
0
can meet these objectives. Find k1, k2, and k3 so that the closed-loop poles are placed at –10 ± j10, –300. Suggest
a suitable scheme for implementation of the control law.
7.19 Figure P7.19 shows a process consisting of two interconnected tanks. h1 and h2 represent deviations in tank
levels from their steady-state values H1 and H2 respectively; q represents deviation in the flow rate from its
steady-state value Q . The flow rate q is controlled by signal u via valve and actuator. A disturbance flow rate w
enters the first tank via a returns line from elsewhere in the process. The differential equations for levels in the
tanks are given by
h& = – 3h + 2h + u + w
1 1 2
h&2 = 4h1 – 5h 2
(a) Compute the gains k1 and k2 so that the control law u = – k1h1(t) – k2h2(t) places the closed-loop poles at
– 4, – 7.
(b) Show that the steady-state error in the output y(t) = h2(t), in response to constant disturbance input w, is
non-zero.
(c) Add to the plant equations, an integrator z& (t) = y(t) and select gains k1, k2 and k3 so that the control law
u = – k1h1(t) – k2h2(t) – k3z(t) places the closed-loop poles at –1, – 2, – 7. Find the steady-state value of the
output in response to constant disturbance w. Give a block diagram depicting the control configuration.
 Pole-Placement Design and State Observers 447

u
Actuator

Q+q

H 1 + h1 H 2 + h2
w

Fig. P7.19

7.20 The plant of a servo system is described by the equations

x& = Ax + bu + bw; y = cx
where

A=
LM
-3 2 OP
; b=
1 LM OP
; c = [0 1]
N
4 -5 Q 0 NQ
w is a disturbance input to the system.
A control law of the form u = – kx + Nr is proposed; r is a constant reference input.
(a) Compute k so that the eigenvalues of (A – bk) are – 4, – 7.
(b) Choose N so that the system has zero steady-state error to reference input, i.e., y(¥) = r
(c) Show that the steady-state error to a constant disturbance input w, is non-zero for the above choice of N.
(d) Add to the plant equation, an integrator equation (z(t) being the state of the integrator):
z& (t) = y(t) – r
and select gains k1, k2 and k3 so that the control law u = – k1x1(t) – k2 x2(t) – k3 z(t) places the eigenvalues
of closed-loop system matrix at – 1, – 2, – 7.
(e) Draw a block diagram of the control scheme employing integral control and show that the steady-state error
to constant disturbance input, is zero.
7.21 Consider a plant consisting of a dc motor, the shaft of which has the angular velocity w (t) and which is driven by
an input voltage u(t). The describing equation is
w& (t) = – 0.5 w (t) + 100 u(t) = Aw(t) + bu(t)
It is desired to regulate the angular velocity at the desired value w 0 = r.
(a) Use control law of the form u = – Kw (t) + Nr. Choose K that results in closed-loop pole with time constant
0.1 sec. Choose N that guarantees zero steady-state error, i.e., w(¥) = r.
(b) Show that, if A changes to A + d A subject to (A + dA – bK ) being stable, then the above choice of N will no
longer make w (¥) = r. Therefore, the system is not robust under changes in system parameters.
(c) The system can be made robust by augmenting it with an integrator
z& = w – r
where z is the state of the integrator. To see this, first use a feedback of the form u = – K1w(t) – K2z(t)
and select K1 and K2 so that the characteristic polynomial of the closed-loop system becomes D(s) =
s2 + 11s + 50. Show that the resulting system will have w (¥) = r no matter how the matrix A changes so
long as the closed-loop system remains asymptotically stable.
448 Digital Control and State Variable Methods
7.22 A discrete-time regulator system has the plant
LM 0 1 OP
0 LM0 OP
x(k + 1) = 0 0 1 x(k) + 0 u(k)
MM-4 -1QP
P MM1 PP
N -2 NQ
1
Design a state-feedback controller which will place the closed-loop poles at – 2 ± j 12 , 0. Give a block diagram
of the control configuration.
7.23 Consider a plant defined by the following state variable model:
x(k + 1) = Fx(k) + Gu(k); y(k) = cx(k) + du(k)
where

LM 1
2
1 0 OP LM 1 4 OP
F = M -1 0 1 ;G= 0 ; c = [1 0 0]; d = [0 4]
MN 0 0 PQ
P MM-30 2 PQ
P
0 N
Design a prediction observer for the estimation of the state vector x; the observer-error poles are required to lie at
– 12 ± j 14 , 0. Give all the relevant observer equations and a block diagram description of the observer structure.
7.24 Consider the system defined by
LM 0 1 0 OP L0 O
x(k + 1) = x(k) + M0 P u(k)
MM-0.50 0 1
. PQ
P MM1 PP
N -0.2 11 NQ
Determine the state-feedback gain matrix k such that when the control signal is given by u(k) = – kx(k), the
closed-loop system will exhibit the deadbeat response to any initial state x(0). Give the state variable model of
the closed-loop system.
7.25 Consider the system

x(k + 1) =
LM 0 1OP x(k) + LM0OP u(k); y(k) = [1 1] x(k)
N-0.16 -1Q N1 Q
Design a current observer for the system; the response to the initial observer error is required to be deadbeat.
Give all the relevant observer equations.
7.26 Consider the plant defined in Problem 7.24. Assuming that only y(k) = x 2(k) is measurable, design a reduced-
order observer such that the response to the observer error is deadbeat. Give all the relevant observer
equations.
7.27 A discrete-time regulator system has the plant

x(k + 1)
LM 2 -1OP x(k) + LM4OP u(k); y(k) = [1 1] x(k) + 7u(k)
N-1 1Q N 3Q
(a) Design a state-feedback control algorithm u(k) = – kx(k) which places the closed-loop characteristic roots
at ± j 12 .
(b) Design a prediction observer for deadbeat response. Give the relevant observer equations.
(c) Combining (a) and (b), give a block diagram of the control configuration. Also obtain state variable model
of the observer-based state-feedback control system.
 Pole-Placement Design and State Observers 449
7.28 A regulator system has the plant with transfer function
Y (z) z -2
=
U ( z ) (1 + 0.8z -1 ) (1 + 0.2 z -1 )
(a) Find (F, g, c) for the plant in controllable canonical form.
(b) Find k1 and k2 such that u(k) = – k1 x1(k) – k2 x2(k) gives closed-loop characteristic roots at 0.6 ± j0.4.
(c) Design a first-order observer that estimates x2, given measurements of x1; the response to initial observer
error is required to be deadbeat.
(d) Give a z-domain block diagram of the closed-loop system.
7.29 Consider the system
x(k + 1) = Fx(k) + gu(k); y(k) = cx(k)
where

F=
LM 0.16 OP
2.16
; g=
-1
;
LM OP c = [1 1]
N-0.16 -116
. Q 1 N Q
(a) Design a state-feedback control algorithm which gives closed-loop characteristic roots at 0.6 ± j 0.4.
(b) Design a reduced-order observer for deadbeat response.
(c) Find the transfer function of the compensator obtained by combining (a) and (b). Give a block diagram of
the closed-loop system showing the compensator in the control loop.
7.30 A double integrator plant is to be controlled by a digital computer employing state-feedback. Figure P7.30
shows a model of the control scheme. Both the state variables x1 and x2 are assumed to be measurable.
(a) Obtain the discrete-time state variable model of the plant.
(b) Compute k1 and k2 so that the response y(t) of the closed-loop system has the parameters: z = 0.5, wn = 4.
(c) Assume now that only x1 is measurable. Design a prediction observer to estimate the state vector x; the
estimation error is required to decay in a deadbeat manner.
(d) Find the transfer function of the compensator obtained by combining (b) and (c).

Plant

x2 x1 = y
u (k ) 1 – e– sT 1 1
– – s s s

T = 0.1 sec

k2
T = 0.1 sec

k1

Fig. P7.30

7.31 Figure P7.31 shows the block diagram of a digital positioning system. The plant is a dc motor driving inertia
load. Both the position q, and velocity q& , are measurable.
(a) Obtain matrices (F, g, c) of the discrete-time state variable model of the plant.
(b) Compute k1 and k2 so that the closed-loop system positions the load in a deadbeat manner in response to
any change in step command qr.
450 Digital Control and State Variable Methods
(c) Assume now that the position q is measured by a shaft encoder and a second-order state observer is used to
estimate the state vector x from plant input u and measurements of q. Design a deadbeat observer. Give a
block diagram of the observer-based digital positioning system.
(d) Design a first-order deadbeat observer to estimate velocity w from measurements of position q.

Plant

u (k) x2 = x1 =
r + + 1 – e– sT 1 1
k1
– – s s+1 s

x 2 (k )
k2
T = 0.1 sec

x 1 (k )
T = 0.1 sec
Fig. P7.31

7.32 A continuous-time plant described by the equation

y& = – y + u + w
is to be controlled by a digital computer; y is the output, u is the input, and w is the disturbance signal. Sampling
interval T = 1 sec.
(a) Obtain a discrete-time state variable model of the plant.
(b) Compute K and N so that the control law
u(k) = – Ky(k) + Nr
results in a response y(t) with time constant 0.5 sec, and y(¥) = r (r is a constant reference input).
(c) Show that the steady-state error to a constant disturbance input w is non-zero for the above choice of the
control scheme.
(d) Add to the plant equation, an integrator equation (v(k) being the integral state)
v(k) = v(k – 1) + y(k) – r
and select gains K1 and K2 so that the control law
u = – K1y(k) – K2v(k)
results in a response y(t) with parameters: z = 0.5, wn = 4.
(e) Give a block diagram depicting the control configuration employing integral control and show that the
steady-state error to constant disturbance w, is zero.
 Linear Quadratic Optimal
Control Through Lyapunov
8 Synthesis

8.1 INTRODUCTION

It should be obvious by now that stability plays a major role in control systems design. We have earlier
introduced, in Chapters 2 and 5, the concept of stability based on the dynamic evolution of the system state
in response to arbitrary initial state, representing initial energy storage. State variable model
x& (t) = Ax(t); x(t = 0) = D x0 (8.1)
0
is most appropriate to study dynamic evolution of the state x(t), in response to the initial state x with zero
external input. At the origin of the state space, x& (t) = 0 for all t; the origin is, thus, the equilibrium point of
the system and xe = 0 is the equilibrium state. This system is marginally stable if, for all possible initial
states, x0, x(t) remains thereafter within finite bounds for t > 0. This is true if none of the eigenvalues of A are
in the right half of the complex plane, and eigenvalues on the imaginary axis, if any, are simple (A multiple
eigenvalue on the imaginary axis would have a response that grows in time and could not be stable). Further-
more, the system is asymptotically stable if for all possible initial states x0, x(t) eventually decays to zero as
t approaches infinity. This is true if all the eigenvalues of A are inside the left half of the complex plane.
This concept of stability, in fact, follows from Lyapunov stability theory. A.M. Lyapunov considered the
stability of general nonlinear systems described by state equation of the form
x& (t) = f(x (t)); x (0) =D x0 (8.2)
If f (x) = Ax, then we have the linear case considered throughout the book. We assume that the equation
has been written so that x = 0 is an equilibrium point, which is to say that f (0) = 0, i.e., the system will
continue to be in equilibrium state xe = 0 for all time. This equilibrium point is said to be stable in the sense
of Lyapunov, if we are able to select a bound on initial condition x0, that will result in state trajectories x(t),
that remain within a chosen finite limit. The system is asymptotically stable at x = 0, if it is stable in the sense
of Lyapunov and, in addition, the state x(t) approaches zero as time t approaches infinity. The term ‘margin-
ally stable’ used in earlier chapters of the book, in fact, is an adaptation of the term “stable in the sense of
Lyapunov”, for our specific needs in linear system analysis.
No new results are obtained by the use of Lyapunov’s method for the stability analysis of linear time-
invariant systems. Simple and powerful methods discussed in earlier chapters are adequate for such systems.
However, Lyapunov functions supply certain performance indices and synthesis data of linear time-invariant
 452 Digital Control and State Variable Methods

systems. Chapter 10 will demonstrate the use of Lyapunov functions in variable structure sliding mode
control, and model reference adaptive control. In this chapter, we introduce the concept of Lyapunov stabil-
ity and the role it plays in optimal control design.
The Lyapunov’s method of stability analysis, in principle , is the most general method for determination
of stability of nonlinear systems. The major drawback which seriously limits its use in practice, is the diffi-
culty often associated with the construction of the Lyapunov function required by the method. Guidelines for
construction of Lyapunov functions for nonlinear systems will be given in Chapter 9. For linear time-
invariant systems of main concern in this chapter, the quadratic function (refer Section 5.2) is adequate for
demonstrating Lyapunov stability. The concept of Lyapunov stability, and working knowledge required for
synthesis of linear time-invariant systems, will be provided here, in this chapter, before we start the discus-
sion on optimal control. Detailed account of Lyapunov stability will be given later in Chapter 9.

8.2 THE CONCEPT OF LYAPUNOV STABILITY

8.2.1 Lyapunov Stability Definitions

We shall confine our attention to systems described by state equation of the form
x& (t) = f(x(t)); f(0) = 0; x(0) =D x0 (8.3)
Note that the origin of the state space has been taken as the equilibrium state of the system, i.e.,
xe = 0
The system described by Eqn. (8.3) is stable in the sense of Lyapunov, if we are able to select a bound
on initial conditions x0, that will result in state trajectories that remain within a chosen finite limit. More
formally, the system described by Eqn. (8.3) is stable in the sense of Lyapunov at x = 0 if, for every real
number e > 0, there exists a real number d > 0 such that ||x(0)|| < d results in ||x(t)|| < e for all t ³ 0; ||x|| = a
norm of vector x (refer Section 5.2). The system is asymptotically stable at x = 0 if it is stable in the sense of
Lyapunov and, in addition, the state x(t) approaches zero as time t approaches infinity. Responses that are
stable in the sense of Lyapunov, and asymptotically stable, are shown in Fig. 8.1.
||x(t)|| < e ||x(0)|| <
||x(0)|| <
||x(t)|| < e
x(t) x(t)

0 0
x(0) x(0)

(a) (b)
Fig. 8.1 Stability definitions

To prove stability results for systems described by equations of the form (8.3), Lyapunov introduced a
function that has many of the properties of energy. The basic Lyapunov stability result is given below.
 Linear Quadratic Optimal Control Through Lyapunov Synthesis 453
8.2.2 Lyapunov Stability Theorems
For the system (8.3), sufficient conditions of stability are as follows.
Theorem 8.1 Suppose that there exists a scalar function V(x) which satisfies the following properties:
(i) V (x) > 0; x ¹ 0
(ii) V (0) = 0 (8.4)
(iii) V (x) is continuous and has continuous partial derivatives with respect to all components of x.
.
(iv) V (x) £ 0 along trajectories of Eqn. (8.3).
We call V (x) having these properties, a Lyapunov function for the system. Properties (i) and (ii) mean
that, like energy, V (x) > 0 if any state is different from zero, but V (x) = 0 when the state is zero. Property
(iii) ensures that V (x) is a smooth function and, generally, has the shape of a bowl near the equilibrium.
A visual analysis may be obtained by considering the
V = k3 z = V(x) 0 0 0
surface
(x1 , x2, V(x ))
V(x1, x2 ) = 1
2
p1 x12 + 1
2
p2 x 22 ; p1 > 0, p2 > 0
This is a paraboloid (a solid generated by rotation of
parabola about its axis of symmetry) surface as shown in
Fig. 8.2. The value V (x1, x2) = ki (a constant) is represented
by the intersection of V (x1, x2) surface and the plane z = ki.
V = k2 This intersection results in a closed curve, an oval. If one
d a fi
plots a trajectory from the point x10 , x 20 , V x 0 , the trajec-
tory crosses the ovals V(x1, x2) = ki for successively
V = k1 smaller values of V(x1, x2), and moves towards the point
corresponding to V(x1, x2) = 0, which is the equilibrium
point. Figure 8.2 shows a typical trajectory.
x1 Property (iv) guarantees that any trajectory moves in a
way, so as never to climb higher on the bowl, than where
x2 it. started out. If property (iv) was made stronger so that
V (x) < 0 for x ¹ 0, then the trajectory must be drawn to
Fig. 8.2 Constant-V curves the origin (The trajectory in Fig. 8.2, in fact, corresponds
.
to this case, i.e., V (x) < 0).
The Lyapunov stability theorem states that, given the system of equations x& = f (x) with f (0) = 0, if there
exists a Lyapunov function for this equation, then the origin is stable in the sense of Lyapunov; in addition,
.
if V (x) < 0, x ¹ 0, then the stability is asymptotic.
This theorem on asymptotic stability and stability in the sense of Lyapunov applies in a local sense if the
region ||x(0)|| < d is small (refer Fig. 8.1); the theorem applies in the global sense when the region includes
the entire state space. The value of Lyapunov function for global stability becomes infinite with infinite
deviation (i.e., V(x) ® ¥ as ||x|| ® ¥).
The determination of stability through Lyapunov analysis centres around the choice of a Lyapunov
function V (x). Unfortunately, there is no universal method for selecting the Lyapunov function which is
unique for a given nonlinear system. Several techniques have been devised for the systematic construction of
Lyapunov functions; each is applicable to a particular class of systems. If a Lyapunov function cannot be
found, it in no way implies that the system is unstable. It only means that our attempt in trying to establish the
stability of an equilibrium state has failed. Therefore, faced with specific systems, one has to use experience,
intuition, and physical insights to search for an appropriate Lyapunov function. An elegant and powerful
 454 Digital Control and State Variable Methods
Lyapunov analysis may be possible for complex systems if engineering insight and physical properties are
properly exploited. Inspite of these limitations, Lyapunov’s method is the most powerful technique available
today for the stability analysis of nonlinear systems.
For linear time-invariant systems of main concern in this chapter, the quadratic function (refer Section
5.2) is adequate for demonstrating Lyapunov stability. Consider the function
V(x) = xT Px (8.5)
where P is a symmetric positive definite matrix. The quadratic function (8.5) satisfies properties (i), (ii) and
(iii) of a Lyapunov function. We need to examine property (iv), the derivative condition, to study the stabil-
ity properties of the system under consideration.
Discrete-Time Systems: In the following, we extend the Lyapunov stability theorem to discrete-time systems:
x (k + 1) = f (x (k)); f (0) = 0 (8.6)
Our discussion will be brief because of the strong analogy between the discrete-time and continuous-
time cases.
Theorem 8.2 Suppose that there exists a Lyapunov function V(x(k)) which satisfies the following properties:
(i) V (x) > 0; x ¹ 0
(ii) V (0) = 0
(8.7)
(iii) V (x) is a smooth function; it is continuous for all x.
(iv) DV (x (k)) = [V(x (k + 1)) – V(x(k))] £ 0 along trajectories of Eqn. (8.6).
The Lyapunov stability theorem states that, given the system of equations, x (k + 1) = f (x (k)); f (0) = 0,
if there exists a Lyapunov function for this equation, then the origin is stable in the sense of Lyapunov; in
addition, if DV (x (k)) < 0 for x ¹ 0, then the stability is asymptotic.

8.3 LYAPUNOV FUNCTIONS FOR LINEAR SYSTEMS

As has been said earlier, the Lyapunov theorems give only sufficient conditions on the stability of the
equilibrium state of a nonlinear system and, furthermore, there is no unique way of constructing a Lyapunov
function. For a linear system, a Lyapunov function can always be constructed and both the necessary and
sufficient conditions on stability established.

8.3.1 Stability of Continuous-Time Linear Systems

Consider a linear system described by the state equation
x& = Ax (8.8)
where A is n ´ n real constant matrix.
Theorem 8.3 The linear system (8.8) is globally asymptotically stable at the origin if, and only if, for any
given symmetric positive definite matrix Q, there exists a symmetric positive definite matrix P, that satisfies
the matrix equation
ATP + PA = – Q (8.9)
Proof Let us first prove the sufficiency of the result. Assume that a symmetric positive definite matrix P
(refer Section 5.2) exists, which is the unique solution of Eqn. (8.9). Consider the scalar function
V(x) = xTPx
Note that
V(x) > 0 for x ¹ 0 and V(0) = 0
The time derivative of V(x) is
.
V (x) = x& TPx + xTP x&
 Linear Quadratic Optimal Control Through Lyapunov Synthesis 455
Using Eqns (8.8) and (8.9), we get
.
V (x) = xTATPx + xTPAx
= xT(ATP + PA)x = – xTQx
.
Since Q is positive definite, V (x) is negative definite. Norm of x may be defined as (Eqn. (5.6b))
|| x || = (xTPx)1/2
Then
V(x) = || x || 2
V(x) ® ¥ as || x || ® ¥
Therefore, the system is globally asymptotically stable at the origin.
To prove the necessity of the result, the reader is advised to refer [105] where the proof has been
developed in two parts:
(i) If (8.8) is asymptotically stable, then for any Q there exists a matrix P satisfying (8.9).
(ii) If Q is positive definite, then P is also positive definite.
Comments
(i) The implication of Theorem 8.3 is that if A is asymptotically stable and Q is positive definite, then the
solution P of Eqn. (8.9) must be positive definite. Note that it does not say that if A is asymptotically
stable and P is positive definite, then Q computed from Eqn. (8.9) is positive definite. For an arbitrary
P, Q may be positive definite (semidefinite) or negative definite (semidefinite).
(ii) Since matrix P is known to be symmetric, there are only n(n + 1)/2 independent equations in (8.9)
rather than n2.
(iii) In very simple cases, Eqn. (8.9), called the Lyapunov equation, can be solved analytically,
but usually numerical solution is required. A number of computer programs for this purpose are
available [152–154].
(iv) Since Theorem 8.3 holds for any positive definite symmetric matrix Q, the matrix Q in Eqn. (8.9) is
often chosen to be a unit matrix.
.
(v) If V (x) = – xTQx does not vanish identically along any trajectory, then Q may be chosen to be positive
semidefinite.
.
A necessary and sufficient condition that V (x) does not vanish identically along any trajectory (meaning
.
that V (x) = 0 only at x = 0), is that
LM H OP
HA
rMM PP
= n; Q = HTH (8.10)
M
MNHA n-1
PQ
where r (.) stands for rank of a matrix.
.
This can be proved as follows. Since V (x) can be written as
.
V (x) = – xTQx = – xTHTHx,
.
V (x) = 0 means that Hx = 0
Differentiating with respect to t, gives
H x& = HAx = 0
Differentiating once again, we get
HA x& = HA2x = 0
 456 Digital Control and State Variable Methods
Repeating the differentiation process and combining the equations, we obtain
H LM OP
MMHAM
PP x = 0
MNHA n -1 PQ
A necessary and sufficient condition for x = 0 to be the only solution of this equation is given by (8.10).

Example 8.1 Let us determine the stability of the system described by the following equation:
x& = Ax
with

A=
LM-1 -2OP
N 1 -4 Q
We will first solve Eqn. (8.9) for P for an arbitrary choice of real symmetric positive definite matrix Q. We
may choose Q = I, the identity matrix. Equation (8.9) then becomes
ATP + PA = – I

or
LM-1 OP LM
1 p11 p12 OP LM
+ 11
p p12 -1 -2OP LM =
-1 0 OP LM OP (8.11)
N QN
-2 -4 p12 p22 Q N p12 p22 QN
1 -4 0 -1 Q N Q
Note that we have taken p12 = p21. This is because the solution matrix P is known to be a positive definite real
symmetric matrix for a stable system.
From Eqn. (8.11), we get
– 2 p11 + 2 p12 = – 1
– 2 p11 – 5 p12 + p22 = 0
– 4 p12 – 8 p22 = – 1
Solving for pij’s, we obtain

P=
LM p
11 p12 23
= 607
OP LM - 7
60
OP
Np12 p22 - 60 Q MN 11
60 PQ
Using Sylvester’s test (Section 5.2), we find that P is positive definite. Therefore, the system under consid-
eration is globally asymptotically stable at the origin.
In order to illustrate the arbitrariness in the choice of Q, consider

Q=
0 0LM OP (8.12)
0 1 N Q
This is a positive semidefinite matrix. This choice of Q is permissible since it satisfies the condition (8.10),
as is seen below.

Q=
LM OP LM OP
0 0
=
0
[0 1] = HTH
N Q NQ
0 1 1
L H OP = r LM0 1 OP = 2
rM
NHAQ N1 – 4Q
It can easily be verified that with the choice of Q given by Eqn. (8.12), we derive the same conclusion about
the stability of the system as obtained earlier with Q = I.
 Linear Quadratic Optimal Control Through Lyapunov Synthesis 457
8.3.2 Stability of Discrete-Time Linear Systems
Consider a linear system described by the state equation
x(k + 1) = Fx(k) (8.13)
where F is n ´ n real constant matrix.
Theorem 8.4 The linear system (8.13) is globally asymptotically stable at the origin if, and only if, for any
given symmetric positive definite matrix Q, there exists a symmetric positive definite matrix P, that satisfies
the matrix equation
FT PF – P = – Q (8.14)
Proof Let us first prove the sufficiency of the result. Assume that a symmetric positive definite matrix P
exists, which is the unique solution of Eqn. (8.14). Consider the scalar function
V(x) = xTPx
Note that
V(x) > 0 for x ¹ 0 and V(0) = 0
The difference
DV(x) = V(x(k + 1)) – V(x(k))
= xT(k + 1)Px(k + 1) – xT(k)Px(k)
Using Eqns (8.13)–(8.14), we get

DV(x) = xT(k)FTPFx(k) – xT(k)Px(k)

= xT(k)[FTPF – P]x(k) = – xT(k)Qx(k)
Since Q is positive definite, DV(x) is negative definite. Further V(x) ® ¥ as || x || ® ¥. Therefore, the system
is globally asymptotically stable at the origin.
The proof of necessity is analogous to that of continuous-time case (refer [105]).
Comments (i) In very simple cases, Eqn. (8.14), called the discrete Lyapunov equation, can be solved
analytically, but usually a numerical solution is required. A number of computer pro-
grams for this purpose are available [152–154].
(ii) If DV(x(k)) = – xT(k)Qx(k) does not vanish identically along any trajectory, then Q may be
chosen to be positive semidefinite.
A necessary and sufficient condition that DV(x(k)) does not vanish identically along any trajectory (mean-
ing that DV(x(k)) = 0 only at x = 0), is that
LMH OP
HF
rM
MM M PP = n; Q = H H
T
(8.15)

NH F n -1 PQ
where r (.) stands for rank of a matrix.

Example 8.2 Let us determine the stability of the system described by the following equation:
x(k + 1) = Fx(k)
 458 Digital Control and State Variable Methods
with

F=
LM-1 -2OP
N 1 - 4Q
We will first solve Eqn. (8.14) for P for an arbitrary choice of real symmetric positive definite matrix Q. We
may choose Q = I, the identity matrix. Equation (8.14) then becomes
FTPF – P = – I

or
LM -1 1OP LM p
11 p12 OP LM-1 -2OP - LM p 11 p12 OP = LM-1 0OP
N-2 -4Q N p 12 p22 Q N 1 -4Q N p 12 p22 Q N 0 -1Q
or – 2p12 + p22 = – 1
2p11 + p12 – 4p22 = 0
4p11 + 16p12 + 15p22 = – 1
Solving for pij’s, we obtain

P=
LM p
11 p12OP LM
=
- 43
60
11
30
OP
Np12 p22 Q N 11
30 - 4
15 Q
Using Sylvester’s test (Section 5.2) we find that P is negative definite. Therefore, the system under consid-
eration is unstable.

8.4 PARAMETER OPTIMIZATION AND OPTIMAL CONTROL PROBLEMS

In previous chapters, we encountered various methods for designing feedback control laws, ranging from
root-locus and Bode-plot techniques to pole-placement by state feedback and state estimation. In each case,
the designer was left with decisions regarding the locations of closed-loop poles. We have given a fairly
complete treatment of these design techniques for linear time-invariant single-variable systems.
Here in this chapter, a somewhat different approach to design is taken. The performance of the system is
measured with a single scalar quantity—the performance index. A configuration of the controller is selected
and free parameters of the controller that optimize (minimize or maximize as the case may be) the perform-
ance index are determined. In most industrial control problems, the nature of the performance index is such
that the design process requires its minimization.
The design approach based on parameter optimization consists of the following steps:
(i) Compute the performance index J as a function of the free parameters k1, k2, K, kn, of the system with
fixed configuration:
J = J(k1, k2, K, kn) (8.16)
(ii) Determine the solution set ki of the equations
¶J
= 0; i = 1, 2, K, n (8.17)
¶k i
Equations (8.17) give the necessary conditions for J to be minimum. From the solution set of these
equations, find the subset that satisfies the sufficient conditions, which require that the Hessian matrix
given below is positive definite.
 Linear Quadratic Optimal Control Through Lyapunov Synthesis 459

LM ¶ J 2
¶ 2J
L
¶ 2J OP
2
MM ¶k 2
1
¶k1¶k 2 ¶k1¶k n PP
¶ J ¶ 2J ¶ 2J
H = M ¶k ¶k
MM M 2 1 ¶k 22
L
¶k 2 ¶k n
PP (8.18)

MM ¶ J
2
M
2
M
2
PP
¶ J ¶ J
MN ¶k ¶k L
¶k n2
PP
n 1 ¶k n ¶k 2 Q
¶2J ¶ 2J
Since = ,
¶ki ¶k j ¶k j ¶k i
the matrix H is always symmetric.
(iii) If there are two or more sets of ki satisfying the necessary as well as sufficient conditions of minimiza-
tion of J, then compute the corresponding J for each set. The set that gives the smallest J is the
optimum set.
Selection of an appropriate performance index is as much a part of the design process, as the minimization of
the index. We know that the performance of a control system can be adequately specified in terms of settling
time, peak overshoot, and steady-state error. The performance index could then be chosen as
J=D K (settling time) + K (peak overshoot) + K (steady-state error)
1 2 3
where the Ki are weighing factors.
Although the criterion seems reasonable, it is not trackable analytically. A compromise must be made
between specifying a performance index which includes all the desired system characteristics, and a per-
formance index which can be minimized with a reasonable amount of computation.
In the following, we present several performance indices which include the desired system characteristics
and, in addition, have good mathematical trackability. These indices often involve integrating some function
of system error over some time interval when the system is subjected to a standard command or disturbance
such as step. A common example is the integral of absolute error (IAE) defined by
¥
D
IAE = z
0
|e(t)| dt

If the index is to be computed numerically, the infinite upper limit can be replaced by the limit tf , where tf is
large enough so that e(t) is negligible for t > tf . This index is not unreasonable since both the fast but highly
oscillatory systems and the sluggish systems will give large IAE value (see Fig. 8.3). Minimization of IAE by
adjusting system parameters will provide acceptable relative stability and speed of response. Also, a finite
value of IAE implies that the steady-state error is zero.
Another similar index is the integral of time multiplied by absolute error (ITAE), which exhibits the
additional useful features that the initial large error (unavoidable for a step input) is not heavily weighted,
whereas errors that persist are more heavily weighted.
¥
D
ITAE = z
0
t | e(t)| dt

The integral of square error (ISE) and integral of time multiplied by square error (ITSE) indices are analo-
gous to IAE and ITAE criteria, except that the square of the error is employed for three reasons: (i) in some
applications, the squared error represents the system’s power consumption, (ii) squaring the error weighs
large errors more heavily than small errors, and (iii) the squared error is much easier to handle analytically.
 460 Digital Control and State Variable Methods
¥
D
ISE = z
0
e2(t) dt

¥
ITSE = z
D te2(t) dt

y
r Gain Gain Gain
too high optimum too low

IAE

t
Fig. 8.3 The IAE optimal response criterion

The system whose design minimizes (or maximizes) the selected performance index with no constraints
on controller configuration is, by definition, optimal.
The difference between parameter optimization and optimal control problems is that no constraint on
controllers is imposed on the latter. In optimal design, the designer is permitted to use controllers of any
degree and any configuration, whereas in parameter optimization the configuration and the type of control-
lers are predetermined. Since there is no constraint imposed on controllers, optimal design results in a better
system, i.e., lower value of the performance index.
However, because of considerations other than minimization of the performance index, one may not build
an optimal control system. For example, optimal solutions to the problem of control of a linear time-invari-
ant plant may result in a nonlinear and/or time-varying system. Hardware realization of such an optimal
control law may be quite difficult and expensive. Also, in many control problems, the optimal solution gives
an open-loop control system which is successful only in the absence of meaningful disturbances. In practical
systems, then, it may be more sensible to seek suboptimal control laws: we select a feedback control con-
figuration and the type of controller, based on considerations of cost, availability of components, etc., and
then determine the best possible values of the free parameters of the controller that minimize the given
performance index. Modifications in control configuration and the type of controller are made until a satis-
factory system is obtained—which has performance characteristics close to the optimal control system we
have worked out in theory.
There exists an important class of optimal control problems for which quite general results have been
obtained. It involves control of linear systems with the objective of minimizing the integral of a quadratic
performance index. An important feature of this class of problems is that optimal control is possible by
feedback controllers. For linear time-invariant plants, the optimal control results in a linear time-invariant
closed-loop system. The implementation of optimal control is, therefore, simple and less expensive. Many
problems of industrial control belong to this class of problems—linear quadratic optimal control problems.
 Linear Quadratic Optimal Control Through Lyapunov Synthesis 461
As we shall see later in this chapter, the linear quadratic optimal control laws have some computational
advantage, and a number of useful properties. The task of the designer shifts to the one of specifying various
parameters in the performance index.
In the previous chapters, we have been mostly concerned with the design of single-variable systems.
Extensions of the root-locus method and the Bode/Nyquist-plot design to multivariable cases have been
reported in the literature. However, the design of multivariable systems using these techniques is much more
complicated than the single-variable cases. Design of multivariable systems through pole-placement can
also be carried out; the computations required are however highly complicated.
The optimal control theory provides a simple and powerful tool for designing multivariable systems.
Indeed, the equations and computations required in the design of optimal single-variable systems and those
in the design of optimal multivariable systems are almost identical. We will use, therefore, in this chapter,
the multi-input, multi-output (MIMO) state variable model in the formulation of optimal control problem.
The objective set for the rest of this chapter is the presentation of simple, and analytically solvable,
optimal control and parameter optimization problems. This will provide insight into optimal and suboptimal
structures and algorithms that may be applied in practical cases. For detailed study, specialized books on
optimal control [109–124] should be consulted. A moderate treatment of the subject is also available in
reference [105].
Commercially available software [152–154] may be used for solving complex optimal/suboptimal control
problems.

8.5 QUADRATIC PERFORMANCE INDEX

A commonly used performance criterion is the integral square error (ISE):

¥

z
J = [y(t) – yr]2 dt
0
(8.19a)

z
= e2(t) dt
0
(8.19b)

where yr is the command or set-point value of the output, y(t) is the actual output, e(t) = y(t) – yr is the error
of the system.
This criterion, which has good mathematical trackability properties, is acceptable in practice, as a meas-
ure of system performance. The criterion penalizes positive and negative errors equally. It penalizes heavily
on large errors; hence a small J usually results in a system with small overshoot. Since the integration is
carried out over [0, ¥), a small J limits the effect of small error lasting for long time and, thus, results in a
small settling time. Also, a finite J implies that the steady-state error is zero.
The optimal design obtained by minimizing the performance index given by Eqns (8.19) may be
unsatisfactory, because it may lead to excessively large magnitudes of control signals. A more realistic
solution to the problem is reached, if the performance index is modified to account for physical constraints
like saturation in physical devices. Therefore, a more realistic performance index is of the form:
¥

z
J = e2(t) dt
0
(8.20a)

subject to the following constraint on control signal u(t),

max |u(t)| £ M (8.20b)
for some constant M. The constant M is determined by the linear range of the plant.
 462 Digital Control and State Variable Methods
Although the criterion expressed in (8.20) can be used in the design, it is not convenient to work with. In
a number of problems, u2(t) is a measure of the instantaneous rate of expenditure of energy. To minimize
energy expenditure, we minimize
¥

z0
u2(t) dt (8.21)

We would, very much, like to replace the performance criterion given by (8.20) by the following quadratic
performance index:
¥
J= z
0
[e2(t) + u2(t)] dt

To allow greater generality, we can insert a real positive constant l to obtain

¥
J= z
0
[e2(t) + l u2(t)] dt (8.22)

By adjusting the weighting factor l, we can weigh the relative importance of the system error and the
expenditure of energy. By increasing l, i.e., by giving sufficient weight to control effort, the amplitude of the
control signal, which minimizes the overall performance index, may be kept within practical bounds,
although at the expense of the increased system error. Note that as l ® 0, the performance index reduces to
the integral square error criterion. In this case, the magnitude of u(t) will be very large and the constraint
given by (8.20b) may be violated. If l ® ¥, the performance index reduces to the one given by Eqn. (8.21),
and the optimal system that minimizes this J is one with u = 0. From these two extreme cases, we conclude
that if l is properly chosen, then the constraint of Eqn. (8.20b) will be satisfied.

Example 8.3 For the system of Fig. 8.4, let us compute the value of K that minimizes ISE for the unit-
step input.
For the system under consideration,
E ( s) s
r + 1 y =
e K u R( s) s + K
s
– For unit-step input,
1
Controller Plant E(s) =
s+ K
Therefore, e(t) = e–Kt
Fig. 8.4 Control system for parameter
¥
optimization
0
z
1
ISE = e2(t) dt =
2K
Obviously, the minimum value of ISE is obtained as K ® ¥. This is an impractical solution, resulting in
excessive strain on the physical components of the system.
Sound engineering judgement tells us that we must include the ‘cost’ of the control effort in our perform-
ance index. The quadratic performance index
¥
J= z
0
[e2(t) + u2(t)] dt

may serve the objective.

From Fig. 8.4, u(t) = K e(t) = K e–Kt
 Linear Quadratic Optimal Control Through Lyapunov Synthesis 463
Therefore,
1 K
J= +
2K 2
The minimum value of J is obtained when
¶J 1 1
=– + = 0 or K = 1
¶K 2K 2 2
Note that
¶2 J 1
= >0
¶K 2 K3
The minimum value of J is 1.
This solution, which weighs error and control effort equally, seems to be acceptable.
The following performance index assigns larger weight to error minimization:
¥

z
J = [e2(t) + lu2(t)] dt; l = 0.5
0
For the system under consideration,
J 1 K
J= +l
2K 2
2 ¶J
= 0 gives
=1 ¶K

= 0.5 K = 2 , Jmin = 0.707

1
When l is greater than unity, it means that more
=0 importance is given to the constraint on amplitude of
u(t) compared to the performance of the system. A suit-
0 0.5 1 1.5 2 K able value of l is chosen so that relative importance of
the system performance is contrasted with the impor-
Fig. 8.5 Performance index versus gain for tance of the limit on control effort. Figure 8.5 gives a
the system shown in Fig. 8.4 plot of the performance index versus K for various
values of l.

Example 8.4 Consider the liquid-level system shown in Fig. 8.6. h represents the deviation of liquid head
from the steady-state value H .
The pump controls the liquid head h by supplying liquid at a rate ( Qi + qi) m3/sec to the tank. We shall
assume that the flow rate qi is proportional to the error in liquid level (desired level – actual level). Under
these assumptions, the system equations are [155]:
dh rgh
(i) A = qi –
dt R
where A = area of cross-section of the tank;
D incremental change in pressure across
R = total resistance offered by the tank outlet and pipe (R =
the restriction/incremental change in flow through the restriction);
r = density of the liquid, and
g = acceleration due to gravity.
 464 Digital Control and State Variable Methods
Desired
level Actual level
Controller

Measurement
system

H +h
Q i +qi

Pump R
Fig. 8.6 A liquid-level system

(ii) qi = Ke
where e = error in liquid level and K = gain constant
R
Let A = 1, and = 1. Then
rg
H ( s) 1
=
Qi ( s) s + 1
The block diagram representation is given in Fig. 8.7. The output y(t) = h(t) is the deviation in liquid head
from steady-state value. Therefore, the output y(t) is itself the error which is to be minimized. Let us pose the
problem of computing the value of K that minimizes the ISE for the initial condition y(0) = 1.
y (0)

+ y(t) = h(t)
e qi = u + +
K
– –

Process

Fig. 8.7 Block diagram representation of the system shown in Fig. 8.6

From Fig. 8.7, we get

Y(s) =
s FG IJ
y (0)
=
1
s +1+ K s H Ks +1+ K
Therefore,
y(t) = e–(1 + K)t
¥
1
ISE = z
0
y2(t) dt =
2 (1 + K )
Obviously, the minimum value of ISE is obtained as K ® ¥.
 Linear Quadratic Optimal Control Through Lyapunov Synthesis 465
This is an impractical solution, resulting in excessive strain on the physical components of the system.
Increasing the gain means, in effect, increasing the pump size.
Now, consider the problem of minimization of
¥

z
J = [y2(t) + u2(t)] dt
0
From Fig. 8.7, we have
u(t) = – Ky(t)
Therefore,
u(t) = – Ke–(1 + K)t
1 K2
J= +
2 (1 + K ) 2 (1 + K )
¶J b g
= 0 gives K = 2 -1
¶K
Note that
¶ 2J 2
= >0
¶K 2 (1 + K ) 3
The minimum value of J is d 2 -1 . i
8.5.1 State Regulator Problem
The performance index given by Eqn. (8.22), is a translation of the requirement of regulation of the system
output, with constraints on amplitude of the input applied to the plant. We now extend the proposed perform-
ance index for the control problem where all the state variables of the system are to be regulated. We use
multivariable formulation of the plant model.
Consider the control problem where the objective is to maintain the system state given by the n ´ 1 state vector
x(t), near the desired state xd (which, in many cases, is the equilibrium point of the system) for all time.
Relative to the desired state xd, (x(t) – xd) can be viewed as the instantaneous system error. If we trans-
form the system coordinates such that the desired state becomes the origin of the state space, then the new
state x(t) is itself the error.
One measure of the magnitude of the state vector x(t) (or of its distance from the origin) is the norm || x(t) ||
defined by
||x(t)||2 = xT(t)x(t)
Therefore,
¥ ¥

z z
J = [xT(t)x(t)] dt = [ x12 (t) + x 22 (t) + L + x n2 (t)] dt
0 0
is a reasonable measure of the system transient response.
In practical systems, the control of all the states of the system is not equally important. To be more
general,
¥

z
J = [xT(t)Qx(t)]dt
0
(8.23)

with Q as n × n real, symmetric, positive definite (or positive semidefinite) constant matrix, can be used as
a performance measure. The simplest form of Q one can use is the diagonal matrix:
 466 Digital Control and State Variable Methods

LMq 1 0 L 0 OP
0 q2 L 0
Q= M P
MM M M MP
P
N0 0 L q Q
n
The ith entry of Q represents the weight the designer places on the constraint on the state variable xi(t). The
larger the value of qi relative to the other values of q, the more control effort is spent to regular xi(t).
The design obtained by minimizing the performance index of the form (8.23) may be unsatisfactory in
practice. A more realistic solution is obtained if the performance index is modified by adding a penalty term
for physical constraints on the p × 1 control vector u(t). One of the ways of accomplishing this is to introduce
the following quadratic control term in the performance index:
¥

z
J = [uT(t)Ru(t)] dt
0
(8.24)

where R is p ´ p real, symmetric, positive definite,1 constant matrix.

By giving sufficient weight to control terms, the amplitudes of control signals which minimize overall
performance index may be kept within practical bounds, although at the expense of increased error in x(t).
For the state regulator problem, a useful performance measure is therefore2
¥
J= 1
2 z
0
[xT(t)Qx(t) + uT(t)Ru(t)] dt (8.25)

8.5.2 Output Regulator Problem

In the state regulator problem, we are concerned with maintaining the n × 1 state vector x(t) near the origin
of the state space for all time. In the output regulator problem, on the other hand, we are concerned with
maintaining the q ´ 1 output vector y(t) near origin for all time. A useful performance measure for the output
regulator problem is
¥
J= 1
2 z
0
[yT(t)Qy(t) + uT(t)Ru(t)] dt (8.26a)

where Q is a q ´ q positive definite (or positive semidefinite) real, symmetric constant matrix, and R is a p ´ p
positive definite, real, symmetric, constant matrix.
Substituting y = Cx in Eqn. (8.26a), we get
¥
J= 1
2 z
0
(xTCTQCx + uTRu) dt (8.26b)

Comparing Eqn. (8.26b) with Eqn. (8.25) we observe that the two indices are identical in form; Q in
Eqn. (8.25) is replaced by CTQC in Eqn. (8.26b). If we assume that the plant is completely observable, then
C cannot be zero; CTQC will be positive definite (or positive semidefinite) whenever Q is positive definite
(or positive semidefinite).
Thus, the solution to the output regulator problem directly follows from that of the state regulator problem.

1
As we shall see in Section 8.7, positive definiteness of R is a necessary condition for the existence of the optimal
solution to the control problem.
2
Note that multiplication by 1/2 does not affect the minimization problem. The constant helps us in mathematical
manipulations as we shall see later in Section 8.7.
 Linear Quadratic Optimal Control Through Lyapunov Synthesis 467

8.6 CONTROL CONFIGURATIONS

8.6.1 State Regulator

Consider the plant represented by linear state equations of the form
D x0
x& (t) = Ax(t) + Bu(t); x(0) = (8.27a)
y(t) = Cx(t) (8.27b)
where x(t) is the n ´ 1 state vector, u(t) is the p ´ 1 input vector, y(t) is the q ´ 1 output vector; A, B and C
are, respectively, n ´ n, n ´ p and q ´ n real constant matrices. We will assume that the null state x = 0 is the
desired state; x(t) is thus system-error vector at time t.
We shall be interested in selecting the controls u(t) which quickly move the system state x(t) to the null
state x = 0 for any initial perturbation x0. The control problem is, thus, to determine u(t) which minimizes
performance index of the form
¥
J= 1
2 z
0
[xT(t)Qx(t) + uT(t)Ru(t)] dt (8.28)

where Q is a positive definite (or positive semidefinite), real, symmetric constant matrix, and R is a positive
definite, real, symmetric, constant matrix.
An important feature (proved later in Section 8.7) of this class of problems, is that optimal control is
possible by feedback control law of the form
u(t) = – Kx(t) (8.29a)
where K is a p ´ n constant matrix, or
LM u OP LM k
1 11 k12 L k1n OP LM x OP
1
L
MMuM PP = – MM kM
2 21 k 22
M
k2n P M
M P MMP
x P
2
(8.29b)
MMNu PPQ MMNk
p p1 k p2 L pn
PM P
k PQ N x Qn

If the unknown elements of the matrix K are determined so as to minimize the performance index given
by Eqn. (8.28), then the control law given by Eqns (8.29) is optimal. The configuration of the optimal
closed-loop control system is represented by the block diagram of Fig. 8.8. As we shall see later in this
chapter, controllability of the plant (8.27) and positive definiteness of matrix Q in the performance index
(8.28), are sufficient conditions for the existence of
Disturbance w asymptotically stable (x(t) ® 0 as t ® ¥) optimal
solution to the control problem.
It may be noted that the optimal solution obtained
0 + Input u Output y
by minimizing the performance index (8.28) may not
Plant
be the best solution in all circumstances. For exam-
–
ple, all the elements of the matrix K may not be free;
State x some gains are fixed by the physical constraints of
K the system and are, therefore, relatively inflexible.
Similarly, if all the states x(t) are not accessible for
Fig. 8.8 Control configuration of a state feedback, one has to go for a state observer whose
regulator complexity is comparable to that of the system itself.
 468 Digital Control and State Variable Methods

It is natural to seek a procedure that relies on the use of feedback from the accessible state variables only,
constraining the gain elements of matrix K corresponding to the inaccessible state variables, to have zero
value (Section 8.9). Thus, whether one chooses an optimal or suboptimal solution depends on many factors
in addition to the performance required out of the system.

8.6.2 State Observer

Implementation of the optimal control law given by Eqns (8.29) requires the ability to directly measure the
entire state vector x(t). For many systems, full state measurements are not practical. In Section 7.5, we found
that the state vector of an observable linear system can be estimated using a state observer which operates on
input and output measurements. We assumed that all inputs can be specified exactly and all outputs can be
measured with unlimited precision. The dynamic behaviour of the observer was assumed to be specified in
terms of its characteristic equation.
Here, we are concerned with the optimal design of the state observer for the multivariable system given
by Eqns (8.27).
We postulate the existence of an observer of the form
x$& (t) = A x$ (t) + Bu(t) + M[y(t) – C x$ (t)] (8.30)
where x$ is the estimate of state x and M is an n × q real constant gain matrix. The observer structure is shown
in Fig. 8.9, which is of the same form as that considered in Section 7.5. The estimation error is given by
~x (t) = x(t) – x$ (t) (8.31a)
From Eqns (8.27) and (8.30), we have
x& (t) = (A – MC) ~
~ x (t) (8.31b)

Plant
u x y
x = Ax + Bu C

+ +
+ x x y –
B n
C
+
n-parallel
integrators

M
Observer

Fig. 8.9 State-observer structure

To design gain matrix M, we may use the duality between control and estimation problems, developed in
Section 7.5. (refer Table 7.1). As per the duality principle, the problem of determination of gain matrix
 Linear Quadratic Optimal Control Through Lyapunov Synthesis 469
M, for the optimal state observer, is mathematically equivalent to designing optimal state regulator for the
‘transposed auxiliary system’ (refer Eqns (7.36))
z& (t) = AT z (t) + CTh (t) (8.32a)
The design problem is to determine n × q gain matrix M such that
h(t) = – MT z (t) (8.32b)
minimizes a quadratic performance index of the form
¥
J= 1
2 z
0
( z TQ0 z + h TR0h) dt (8.33)

where Q0 is a positive definite (or positive semidefinite), real, symmetric constant matrix, and R0 is a
positive definite, real, symmetric constant matrix.
The solution to this problem exists if the auxiliary system (8.32) is completely controllable. This condi-
tion is met, if the original system (8.27) is completely observable.
The separation principle (refer Section 7.6) allows for the separate designs of state-feedback control law
and state observer; the control law and the observer are then combined as per the configuration of Fig. 8.10.
The weighting matrices Q0 and R0, for the observer design, can be assumed to be equal to the weighting
matrices Q and R, respectively, of the control-law design. Generally, however, one would design a faster
Plant Sensor observer in comparison with the regulator, i.e., for Q0 =
u x y Q, the elements of R0 are chosen smaller than those of R.
x = Ax + Bu C
The above solution, to the state estimation problem
based on duality between the control and estimation,
can be formalized by using “Optimal Filtering Theory”.
The formal development of the result extends beyond
the scope of this text [105]. We may, however, use the
x term “optimal filter” for the state observer designed by
–K x = Ax + Bu + M (y – Cx )
the procedure given here in the text.
Control law Observer An optimal filter whose weighting motion Q0 and R0
Fig. 8.10 Combined state-feedback are determined by the “spectral properties” of the exog-
control and state estimation enous noise signals is termed a Kalman filter [105].

8.6.3 Servo Systems

The control configuration of Fig. 8.8, implicitly assumes that the null state x = 0 is the desired equilibrium
state of the system. It is a state regulator with zero command input.
In servo systems, where the output y(t) is required to track a constant command input, the equilibrium
state is a constant point (other than the origin) in state space. This servo problem can be formulated as a
‘shifted regulator problem’, by shifting the origin of the state space to the equilibrium point. Formulation of
the shifted regulator problem for single-input systems was given in Section 7.7. Extension of the formulation
to the multi-input case is straightforward.

8.6.4 State-Feedback with Integral Control

In a state-feedback control system (which is a generalization of proportional plus derivative feedback), it is
usually required that the system have one or more integrators within the closed loop. This will lead to zero
steady-state error when the command input and disturbance have constant steady-state values. Unless the
plant to be controlled has integrating property, it is generally necessary to add one or more integrators within
the loop.
 470 Digital Control and State Variable Methods
For the system (8.27), we can feedback the state x as well as the integral of the error in output by augment-
ing the plant state x with the extra ‘integral state’. For single-input systems, the problem of state-feedback
with integral control was formulated as a state regulator problem in Section 7.8. This was done by augment-
ing the plant state with ‘integral state’, and shifting the origin of the state space to the equilibrium point.
Multivariable generalization of state-feedback with integral control is straightforward.

8.7 OPTIMAL STATE REGULATOR

The parameter optimization problem, as explained earlier, refers to the problem of obtaining the optimum
values of free parameters in a predetermined control configuration. In optimal control design, the designer
is permitted to use controllers of any degree and any configuration. Optimal solutions to the problem of
control of a linear time-invariant plant, may result in a nonlinear and/or time-varying system [105]. Also, in
many control problems, the optimal solution gives an open-loop control system [105].
For an important class of control problems, which involves control of linear time-invariant plants with the
objective of minimizing the integral of a quadratic performance index, the optimal control is possible by
state-feedback control that results in a linear time-invariant closed-loop system. Many problems of industrial
control belong to this class of problems—linear quadratic optimal control problems.
To prove this important result, we consider the optimal control problem for a linear multivariable com-
pletely controllable plant
x& = Ax + Bu (8.34)
where x is the n ´ 1 state vector, u a p ´ 1 input vector; A and B are, respectively, n ´ n and n ´ p real
constant matrices, and the null state x = 0 is the desired steady-state.
The objective is to find the optimal control law that minimizes the following performance index, subject
D x0 :
to the initial conditions x(0) =
¥
J = 1
2 z
0
(xT Qx + u T Ru)dt (8.35)

where Q is n ´ n positive definite, real, symmetric, constant matrix, and R is p ´ p positive definite, real,
symmetric, constant matrix.
Since the (A, B) pair is completely controllable3, there exists a state-feedback control law
u = – Kx (8.36)
where K is p ´ n real, constant, unconstrained gain matrix, that results in an asymptotically stable closed-
loop system (refer Section 7.3)
x& (t) = (Ax – BKx) = (A – BK)x (8.37)
This implies that there is a Lyaponov function V = 12 xT Px for the closed-loop system (8.37); that is, for
some positive definite matrix P, the time derivative dV/dt evaluated on the trajectories of the closed-loop
system is negative definite. We now state and prove a condition for u = – Kx (t) to be optimal [35].

3
The controllability of the (A, B) pair is not a necessary condition for the existence of the optimal solution. If the
(A, B) pair is not completely controllable, we can transform the plant model to controllability canonical from given
in Eqn. (5.123c). It decomposes the model into two parts: the controllable part and the uncontrollable part. If the
uncontrollable part is stable, then the model is said to be stabilizable. Stabilizability of the (A, B) pair is a necessary
condition for the existence of optimal solution.
 Linear Quadratic Optimal Control Through Lyapunov Synthesis 471
Theorem 8.5 If the state-feedback controller u = – Kx (t) is such that it minimizes the function

f (u ) =
dV 1 T
dt
(
+ 2 x Qx + uT Ru , ) (8.38)

and the minimum value of f (u) = 0 for some V = 12 xT Px, then the controller is optimal.
Proof We can represent (8.38) as (denoting minimizing u by u*)
dV T
+ 12 xT Qx + 12 u* Ru* = 0
dt u =u*
Hence
dV T
= − 12 xT Qx − 12 u* Ru*
dt u =u*
Integrating both sides with respect to time from 0 to ¥, we obtain
V ( x ( ∞ )) − V ( x ( 0 )) = − 12 ∫
∞
0 (x Qx + u
T *T
Ru* dt)
Because, by assumption, the closed-loop system is asymptotically stable, we have x(¥) = 0; therefore,
V ( x ( 0 )) = 12 xT ( 0 ) Px ( 0 ) = 1 ∞
∫
2 0 (x Qx + u
T *T
Ru* dt)
Thus, if a linear stabilizing controller satisfies the hypothesis of the theorem, then the value of the per-
formance index (8.35) for such a controller is

( )
J u* = 12 xT ( 0 ) Px ( 0 )
Since u minimizes the function in (8.38) and the minimum value is zero, for any u different from u*, the
*

value of the function will be greater than/equal to zero.

dV
(
+ 1 xT Qx + uˆ T Ruˆ ≥ 0
dt u=uˆ 2
)
or
dV
dt u=uˆ
(
≥ − 12 xT Qx + uˆ T Ruˆ )
Integrating both sides with respect to time from 0 to ¥, yields
V ( x (0 )) ≤ 1
∞
∫
2 0 ( x Qx + uˆ Ruˆ ) dt
T T

implying that
( )
J u* ≤ J (uˆ )
for any uˆ ≠ u . Therefore, the controller u is optimal.
* *

It follows from the above theorem that the synthesis of optimal control law involves finding an appropri-
ate Lyapunov function, or equivalently, the matrix P. The matrix P is found by minimizing
dV 1 T
f (u ) = + x Qx + 12 uT Ru (8.39)
dt 2
We first apply to (8.39) the necessary conditions for unconstrained minimization.
∂ ⎛ dV 1 T ⎞
⎜ + 2 x Qx + 12 uT Ru ⎟ =0
∂u ⎝ dt ⎠ u =u*
 472 Digital Control and State Variable Methods
Differentiating the above yields
∂ ⎛ dV 1 T
⎜
∂u ⎝ dt
⎞ ∂ 1 T
+ 2 x Qx + 12 uT Ru ⎟ =
⎠ ∂u
2 (( )
x& Px + xT Px& + 12 xT Qx + 12 uT Ru )
∂ T
=
∂u
(
x Px& + 12 xT Qx + 12 uT Ru )
∂ T
=
∂u
(
x PAx + xT PBu + 12 xT Qx + 12 uT Ru )
= BT PT x + Ru = 0 = BT Px + Ru
Hence, a candidate for an optimal control law has the form
u* = −R −1BT Px = −Kx (8.40)
where K = R − 1B T P
∂2 ∂
Note that
∂u 2
f (u ) =
∂u
(BT PT x + Ru ) = RT = R, a positive definite matrix.
Thus, the second-order sufficiency condition, for u* to minimize (8.39), is satisfied.
We now turn our attention to finding an appropriate P. The optimal closed-loop system has the form
x& = ( A − BR −1BT P ) x ; x ( 0 ) @ x 0
Our optimal controller satisfies the equation
dV T
+ 12 xT Qx + 12 u* Ru* = 0
dt u=u*
that is,
T
xT PAx + xT PBu* + 12 xT Qx + 12 u* Ru* = 0
We substitute u* given by (8.40) into the above equation, and represent it as
1
2 ( )
xT AT P + PA x − xT PBR −1BT Px + 12 xT Qx + 12 xT PBR −1BT Px = 0
Factoring out x yields
x T ( A T P + PA + Q – PBR –1B T P )x = 0
1
2
The above equation should hold for any x. For this to be true, we have to have
(A
P + PA + Q − PBR −1BT P = 0
T
) (8.41)
The above equation is referred to as algebraic Riccati equation. In conclusion, the synthesis of the opti-
mal linear state-feedback controller, minimizing the performance index
J= 1 ∞
∫
2 0 (x Qx + u Ru ) dt
T T

subject to
x& = Ax + Bu; x ( 0 ) @ x0
requires solving the matrix Riccati equation given by (8.41).
Controllability of (A, B) pair and positive definiteness of Q are sufficient conditions for the existence of
asymptotically stable optimal solution to the control problem. This implies that there is a Lyapunov function
V = 12 xT Px for the closed-loop system (8.37); that is, for positive definite matrix P, the time derivative
evaluated on the trajectories of the closed-loop system,
 Linear Quadratic Optimal Control Through Lyapunov Synthesis 473
dV .
= V = − 12 xT Qx − 12 uT Ru
dt
= − 12 xT (Q + K T RK )x (8.42)
is always negative definite.
We now study the effect of choosing a positive semidefinite Q in the performance index J. If Q is positive
semidefinite and in addition the following rank condition is satisfied,
LM H OP
HA
rM
MM M PP = n; Q = H H, T
(8.43)

NHAn -1 PQ
.
then V (x) < 0 for all x ¹ 0.
.
We prove this result by contradiction: the rank condition is satisfied but V (x) = 0 for some x ¹ 0. Substitut-
ing Q = HTH in Eqn. (8.42), we obtain (refer Eqns (5.6))
.
V (x) = – 1
2 (xTHTHx + xTKTRKx) = – 1
2 [|| Hx ||2 + || Kx || 2R ]
Therefore, Hx = 0 and Kx = 0 should be simultaneously satisfied. Kx = 0 reduces the closed-loop system
(8.37) to the open-loop system
x& = Ax
From the condition
Hx = 0
we obtain
H x& = HAx = 0
Continuing the process of taking derivative, we get
HA2x = 0
M
HAn – 1x = 0
LM H OP
or MM HA PP x = Vx = 0
M
MNHA n -1 PQ
Since r[V] = n, Vx = 0 only when x = 0. This proves the result.
Let us see an alternative interpretation of the rank condition (8.43). The rank condition implies that the system
x& = Ax + Bu
with the ‘auxiliary output’
z (t) = Hx(t)
is completely observable. Since the performance index
¥ ¥
J= 1
2 z
0
(xTQx + uTRu)dt = 1
2 z
0
(xTHTHx + uTRu) dt

¥
= 1
2 z
0
( z T z + uTRu) dt,
 474 Digital Control and State Variable Methods

the observability of the pair (A, H) implies that all the modes of the state trajectories are reflected in the
performance index. A finite value of J, therefore, ensures that unstable modes (if any) have been stabilized4
by the control u = – Kx.
The observability condition is always satisfied when the matrix Q is positive definite.
The design steps may now be stated as follows:
(i) Solve the matrix Riccati equation (8.41) for the positive definite matrix P.
(ii) Substitute this matrix P into Eqn. (8.40); the resulting equation gives optimal control law.
This is a basic and well-known result in the theory of optimal control. Once the designer has
specified Q and R, representing his/her assessment of the relative importance of various terms in the per-
formance index, the solution of Eqn. (8.41) specifies the optimal control law (8.40). This yields the optimal
closed-loop system. If the resulting transient response is unsatisfactory, the designer may alter the weighting
matrices Q and R, and try again.
Comments
(i) The matrix R has been assumed to be positive definite. This is a necessary condition for the existence
of the optimal solution to the control problem, as seen from Eqn. (8.40).
(ii) We have assumed that the plant (8.34) is completely controllable, and the matrix Q in performance
index J, given by Eqn. (8.35), is positive definite. These are sufficient conditions for the existence of
asymptotically stable optimal solution to the control problem. The requirement on matrix Q, may be
relaxed to a positive semidefinite matrix with the pair (A,H) completely observable, where Q = HTH.
(iii) It is important to be able to find out whether the sought-after solution exists or not, before we start
working out the solution. This is possible only if necessary conditions for the existence of asymptoti-
cally stable optimal solution are established. A discussion on this subject entails not only controlla-
bility and observability, but also the concepts of stabilizability and detectability. Basic ideas about
these concepts have been given in footnotes of this chapter; a detailed discussion is beyond the scope
of this book.
(iv) Equation (8.41) is a set of n2 nonlinear algebraic equations. Since P is a symmetric matrix, we need to
solve only
n(n + 1) equations.
2
(v) The solution of Eqn. (8.41) is not unique. Of the several possible solutions, the desired answer is
obtained by enforcing the requirement that P be positive definite. The positive definite solution of
Eqn. (8.41) is unique.
(vi) In very simple cases, the Riccati equation can be solved analytically, but usually a numerical solution
is required. A number of computer programs for the purpose are available [152–154]. Appendix A
provides some MATLAB support.

4
Observability canonical form for a state model which is not completely observable, is given in Eqn. (5.124c). It
decomposes the model into two parts: the observable part and the unobservable part. If the unobservable part is
stable, then the model is said to be detectable.
In the optimization problem under consideration, the observability of the pair (A, H) is not a necessary condition
for the existence of a stable solution. If the pair (A, H) is detectable, then the modes of state trajectories, not reflected
in J, are stable and a finite value of J will ensure asymptotic stability of the closed-loop system.
 Linear Quadratic Optimal Control Through Lyapunov Synthesis 475
(vii) Note that the optimal state regulator requires that all the parameters of matrix K in the control law
(8.36), are free parameters. However, all the parameters of matrix K may not be available for adjust-
ments. The gain elements of matrix K, corresponding to inaccessible state variables, may be con-
strained to zero value (otherwise, a state observer will be required). Also, some gains may be fixed by
physical constraints of the system. This leads to the parameter optimization problem: optimization of
free parameters in the matrix K. The difference between parameter optimization and optimal control
problems is that no constraint on controllers is imposed on the latter. A solution to parameter optimi-
zation (suboptimal control) problem will be given in Section 8.9.

Example 8.5 Consider the problem of attitude control of a rigid satellite which was discussed in Example 7.1.
An attitude control system for the satellite that utilizes rate feedback is shown in Fig. 8.11; q (t) is the actual
attitude, qr(t) is the reference attitude which is a step function, and u(t) is the torque developed by the
thrusters.

+ + u 1 x2 1 y = x1 =
r
k1
s s
– –

k2

Fig. 8.11 Attitude control of a satellite

The state variable model of the system is

x& = Ax + Bu (8.44)
y = Cx
with

A=
LM0 1OP ; B = LM0OP ; C = [1 0]
N0 0 Q N1Q
The problem is to obtain the optimal control law
u = – k1(x1 – qr) – k2x2; x1 = q, x2 = q&
that minimizes the performance index
¥

z
J = [(qr – q) 2 + u2] dt
0
(8.45)

In terms of the shifted state variables

~
x1 = x1 – qr; ~
x 2 = x2
the state equation becomes
~
x& = A ~
x + Bu (8.46)
where A and B are given by Eqns (8.44).
Now, the problem is to find optimal values of the parameters k1 and k2, such that the control law
u = – k1 ~
x1 – k2 ~
x2
minimizes the performance index
¥
J = (~z
x12 + u2)dt (8.47)
0
 476 Digital Control and State Variable Methods
The Q and R matrices are

Q=
LM2 0OP ; R = 2
N 0 0Q
Note that Q is positive semidefinite matrix.
L 2O
T
Q=H H= M P [ 2 0]
N0Q
The pair (A,H) is completely observable. Also, the pair (A,B) is completely controllable. Therefore,
sufficient conditions for the existence of asymptotically stable optimal solution are satisfied.
The matrix Riccati equation is
ATP + PA – PBR–1BTP + Q = 0

or
LM0 0OP L p p O + L p
11 12 11 p12 OP LM0 1OP
N1 0Q MN p p PQ MN p
12 22 12 p22 Q N 0 0Q
–M
L p p OP LM0OP [ ] [0
11 12 1
1]
LM p 11 p12
+
OP LM OP LM OP
2 0
=
0 0
2
N p p Q N1Q
12 22 Np 12 p22 Q N Q N Q
0 0 0 0
Upon simplification, we get
2
- p12
+2=0
2
p p
p11 – 12 22 = 0
2
2
- p22
+ 2p12 = 0
2
Solving these three simultaneous equations for p11, p12, and p22, requiring P to be positive definite, we obtain

P=
LM2 2 2 OP
N2 2 2Q
The optimal control law is given by

u = – R–1BTP ~
x (t) = – [ 12 ] [0 1]
LM2 2 2 OP L x~ O
1

N2 2
M x PQ
2 Q N~ 2

=– ~x1 (t) – 2 ~
x 2 (t) = – (x1 – qr) – 2 x2
It can easily be verified that the closed-loop system is asymptotically stable.

Example 8.6 Consider the liquid-level system of Fig. 8.6. In Example 8.4, we designed an optimal regulator
for this process by direct parameter optimization. In the following, we use the Riccati equation for designing
the optimal regulator. The state equation of the process is
dy
=–y+u (8.48)
dt
where
y = deviation of the liquid head from the steady-state; and
u = rate of liquid inflow.
 Linear Quadratic Optimal Control Through Lyapunov Synthesis 477
The performance index
¥

z
J = (y2 + u2) dt
0
For this design problem,
A = –1, B = 1, Q = 2, R = 2
The Riccati equation is
ATP + PA – PBR–1BTP + Q = 0
P2
or –P–P– +2=0
2
Solving for P, requiring it to be positive definite, we get
P=2 b 2 -1 g
The optimal control law is
u = – R–1BTPy(t) = – b g
2 - 1 y(t)
Substituting in Eqn. (8.48), we get the following equation for the closed-loop system:
dy
=– 2y
dt
Obviously, y(t) ® 0 for any initial displacement y(0).
Assume now that a constant disturbance due to the pump enters the system as shown in Fig. 8.12. This
type-0 regulator system cannot reject the disturbance; there will be a steady-state offset in the liquid head y.
Let us introduce integral control to eliminate this offset.
Defining the integral state z by
Disturbance w
z& (t) = y(t),
we get the following augmented system:
+
u + y LM y& OP = L-1 0O LM y OP + LM1OP u
Process (8.49)
– N z& Q MN 1 0PQ N z Q N0Q
Now the design problem is to obtain the control law
( 2 – 1) u = – k1y(t) – k2z(t)
that minimizes
Fig. 8.12 Optimal control of the liquid-level ¥

process shown in Fig. 8.6 z

J = (y2 + u2)dt
0
The Q and R matrices are

Q=
LM2 0OP , R = 2
N0 0Q
The state equation (8.49) is completely controllable, satisfying one of the sufficient conditions for the exist-
ence of the optimal solution.
The matrix Q is positive semidefinite;

Q = HTH =
LM 2 OP [ 2 0]
N0Q
 478 Digital Control and State Variable Methods

The pair
FG L-1 0O, 2 0
IJ is not completely observable. Therefore, the other sufficient condition for the
H MN 1 0PQ K
existence of the asymptotically stable optimal solution is not satisfied. It can easily be verified that a positive
definite solution to the matrix Riccati equation does not exist in this case; the chosen matrix Q cannot give a
closed-loop stable optimal system.
We now modify the performance index to the following:
¥

z
J = (y2 + z2 + u2) dt
0
The Q and R matrices are

Q=
LM2 0OP , R = 2
N0 2Q
Since Q is positive definite matrix, the asymptotically stable optimal solution exists.
The matrix Riccati equation is
ATP + PA – PBR–1BTP + Q = 0

or
LM-1 1OP LM p 11 OP + LM p
p12 11 p12OP LM-1 0OP
N 0 0Q N p 12 p Q Np 22 12 p Q N 1 0Q
22

Lp 11 p O L 1O
12 1 L p p OP + LM2 0OP = LM0 0OP
11 12
– M M P [ ][1 0] M
Np 12 p PQ N0Q
22
2
N p p Q N 0 2 Q N0 0 Q
12 22
From this equation, we obtain the following three simultaneous equations:
2
– 2p11 + 2p12 – 21 p11 +2=0
p p
– p12 + p22 – 11 12 = 0
2
2
- p12
+2=0
2
Solving for p11, p12 and p22, requiring P to be positive definite, we obtain

P=
2 2 LM OP
2 4 N Q
The gain matrix

K = [k1 k2] = R–1BTP = [ 21 ] [1 0]

2 2
= [1 1]
LM OP
2 4 N Q
Therefore,
¥

+ u
u = – y(t) – z(t) = – y(t) – y(t)dt
y
z
0
Process The block diagram of Fig. 8.13 shows the configura-
– –
tion of the optimal control system employing state-
feedback and integral control. It is a type-1 regulator
system.
Since at steady-state
Fig. 8.13 Optimal control system employing
z& (t) ® 0,
state-feedback and integral control
 Linear Quadratic Optimal Control Through Lyapunov Synthesis 479
therefore,
y(t) ® 0,
and there will be no steady-state offset in the liquid head y, even in the presence of constant disturbances
acting on the plant.

8.8 OPTIMAL DIGITAL CONTROL SYSTEMS

This section covers the key results on the design of optimal controllers for discrete-time systems. Our
discussion will be brief because of the strong analogy between the discrete-time and continuous-time cases.
Consider the discretized model of the given plant:
x(k + 1) = Fx(k) + Gu(k); x(0) = D x0
(8.50)
y(k) = Cx(k)
where x is the n ´ 1 state vector, u is the p ´ 1 input vector, y is the q ´ 1 output vector; F, G, and C are,
respectively, n ´ n, n ´ p, and q ´ n real constant matrices; and k = 0, 1, 2, K We will assume that the null
state x = 0 is the desired steady-state; x(k) is, thus, the system-error vector at t = kT , where T is the sampling
interval. The state variable model (8.50) is assumed to be completely controllable and observable.
We shall be interested in selecting the controls u(k); k = 0, 1, K, which minimize a performance index of
the form
¥
J= 1
2 å [xT(k)Qx(k) + uT(k)Ru(k)] (8.51)
k = 0
where Q is an n ´ n positive definite, real, symmetric, constant matrix (or a positive semidefinite, real,
symmetric, constant matrix with the restriction that the pair (F,H) is observable, where HTH = Q), and R is
a p ´ p positive definite, real, symmetric, constant matrix. This criterion is the discrete analog of that given
by Eqn. (8.25); a summation replaces integration.
An important feature (proved later in this section) of this class of problems is that optimal control is
possible by feedback control law of the form (refer Fig. 8.14)
u(k) = – Kx(k) (8.52)
where K is a p ´ n constant matrix. If the unknown elements of matrix K are determined so as to minimize
the performance index given by Eqn. (8.51), the control law given by Eqn. (8.52) is optimal.
The control problem stated above, as we know, is a
state regulator design problem. The equations developed
Plant
below for the state regulator design, can also be used for
u(k) servo design by an appropriate transformation of the state
x(k + 1) = Fx (k) + Gu (k)
variables (refer Section 7.9 for details). State regulator
– design equations can also be used for state-feedback con-
u(k) trol schemes with integral control. This is done by the
K augmentation of the plant state with integral state and
appropriate transformation of the state variables (refer
Fig. 8.14 Control configuration of the state Section 7.9 for details).
regulator for discrete-time systems In the following, we develop state regulator design
equations through Lyapunov synthesis.
With the linear feedback control law (8.52), the closed-loop system is described by
x(k + 1) = (F – GK)x(k) (8.53)
 480 Digital Control and State Variable Methods
We will assume that a matrix K exists such that (F – GK) is a stable matrix. The controllability of the
model (8.50) is sufficient to ensure this. This implies that there exists a Lyapunov function V (x ( k )) =
1
2
xT (k) Px (k) for the closed-loop system (8.53). Therefore, the first forward difference, DV(x (k)) =
V (x (k + 1)) – V (x (k)), evaluated on the trajectories of the closed-loop system, is negative definite. We now
state and prove a condition for u ( k ) = −Kx ( k ) to be optimal [35].
Theorem 8.6 If the state feedback controller u ( k ) = −Kx ( k ) is such that it minimizes the function
f (u) = ΔV ( x ( k )) + 12 xT ( k ) Qx ( k ) + 12 u T ( k ) Ru ( k ) (8.54)

and the minimum value of f (u) = 0 for some V = 12 xT ( k ) Px ( k ) , then the controller is optimal.
Proof We can represent (8.54) as (denoting minimizing u by u*)

ΔV ( x ( k ))
*T
+ 12 xT ( k ) Qx ( k ) + 12 u (k ) Ru* (k ) = 0
u =u*
Hence
V ( x ( k + 1)) − V ( x ( k ))
*T
= − 12 xT ( k ) Qx ( k ) − 12 u (k ) Ru* ( k )
u=u*
We sum both sides from k = 0 to ¥ to get

( )
∞
V ( x ( ∞ )) − V ( x (0 )) = − 12 ∑ xT ( k ) Qx ( k ) + u*
T
(k ) Ru* (k )
k =0
Because, by assumption, the closed-loop system is stable, we have x(¥) = 0. Hence

∑ ( xT (k ) Qx (k ) + u* (k ) Ru* (k ))
∞
V ( x ( 0 )) = 12 xT ( 0 ) Px ( 0 ) = 1 T
2
k =0
Thus, if a linear stabilizing controller satisfies the hypothesis of the theorem, then the value of the per-
formance index (8.51) resulting from applying the controller is

( )
J u* = 12 xT ( 0 ) Px ( 0 )
Since u minimizes the function in (8.54) and the minimum value is zero, for any u different from u*, the
*

value of the function will be greater than/equal to zero.

ΔV ( x ( k )) + 12 xT ( k ) Qx ( k ) + 12 uˆ T ( k ) Ruˆ ( k ) ≥ 0
u=uˆ
or ,
ΔV ( x ( k )) ≥ − 12 xT ( k ) Qx ( k ) − 12 uˆ T ( k ) Ruˆ ( k )
u =uˆ
Summing the above from k = 0 to ¥, yields
∞
V ( x ( 0 )) ≤ 1
2 ∑ ( xT (k ) Qx (k ) + 12 uˆ T ( k ) Ruˆ (k ));
k =0
that is,
( )
J u* ≤ J ( uˆ )
for any uˆ ≠ u . Therefore, the controller u is optimal.
* *

Finding an optimal controller involves finding an appropriate quadratic Lyapunov function V ( x ( k )) =

1
2
xT (k )Px ( k ) , which is used to construct the optimal controller. We first find u* that minimizes the function
 Linear Quadratic Optimal Control Through Lyapunov Synthesis 481

f (u ( k )) = ΔV ( x ( k )) + 12 xT ( k ) Qx ( k ) + 12 uT (k )Ru ( k )
= 12 xT ( k + 1) Px ( k + 1) − 12 xT ( k ) Px ( k ) + 12 xT ( k ) Qx ( k ) + 12 uT (k )Ru ( k )

(Fx (k ) + Gu (k )) P (Fx (k ) + Gu (k ))
T
= 1
2

− 12 xT ( k ) Px ( k ) + 12 xT ( k ) Qx (k ) + 12 uT (k )Ru ( k )
(8.55)
⎛ ∂ T ∂f ∂g ⎞
Necessary condition for unconstrained minimization is ⎜
⎝ ∂x
f g =
∂x
g + f⎟ :
∂x ⎠
( )
∂f (u (k )) ∂ ( Fx ( k ) + Gu ( k ))
= 1 ⎡⎣ P (Fx ( k ) + Gu ( k ))⎤⎦
∂u ( k ) 2
∂u ( k )

+ 1
(
∂ P (Fx ( k ) + Gu ( k )) ) ⎡Fx
(k ) + Gu (k )⎤⎦ + 1
(
∂ uT ( k ) Ru ( k ) )
2
∂u ( k ) ⎣ 2
∂u ( k )
= 1 T
2
G P (Fx (k ) + Gu (k )) + (Fx (k ) + Gu (k )) + Ru (k )
1 T
2
G P
=G T
P (Fx ( k ) + Gu ( k )) + Ru ( k )
= G T PFx ( k ) + (R + G T PG )u ( k ) = 0
The matrix R + GTPG is positive definite, and therefore it is invertible. Hence
( )
−1
u* ( k ) = − R + G T PG G T PFx ( k ) = −Kx ( k ) (8.56)

K = ( R + G T PG ) G T PF
−1
where
We next check if u* satisfies the second-order sufficient condition for a relative minimizer of the function
(8.55).

∂ 2 f (u ( k ))
=
⎣ (
∂ ⎡G T PFx ( k ) + R + G T PG u ( k )⎤
⎦ )
∂u 2
(k ) ∂u (k )
= R + GT PG; a positive-definite matrix
that is, u* satisfies the second-order sufficient condition for a relative minimizer of the function (8.55).
The optimal controller (8.56) can be constructed if we have found an appropriate positive definite matrix
P. Our next task is to devise a method that would allow us to compute the desired matrix P. For this, we, first,
find the equation describing the closed-loop system driven by the optimal controller (8.56):

(
x ( k + 1) = F − GS −1 GT PF x ( k ) ) (8.57)
where S = R + GTPG
Our controller satisfies the hypothesis of Theorem (8.6), that is
T
1
2
xT ( k + 1) Px ( k + 1) − 12 xT ( k ) Px ( k ) + 12 xT ( k ) Qx ( k ) + 12 u* (k )Ru* (k ) = 0
or,
1 xT
2
(k )[F − GS −1G T PF ]T P[F − GS −1G T PF ]x (k ) − 12 xT (k ) Px(k ) + 12 xT (k )Qx(k )
+ 12 xT ( k )[ FT PGS −1 RS −1GT PF]x( k ) = 0
 482 Digital Control and State Variable Methods
or,
1 T
2
x (k )FT PFx( k ) − 12 xT ( k )FT PGS −1G T PFx( k ) − 12 xT (k )F T PGS −1G T PFx(k )
+ 12 xT (k )FT PGS −1GT PGS−1GT PFx(k ) − 12 xT (k ) Px(k ) + 12 xT (k )Qx(k )
+ 12 xT (k )FT PGS −1RS −1GT PFx(k ) = 0
or,
1
2
xT ( k )FT PFx ( k ) − 12 xT ( k ) Px ( k ) + 12 xT ( k )Qx ( k ) − xT ( k )FT PGS −1GT PFx ( k )
+ 12 xT ( k ) FT PGS −1 ( R + GT PG) S−1GT PFx( k ) = 0
or,
1
2
xT ( k )FT PFx ( k ) − 12 xT ( k ) Px ( k ) + 12 xT ( k )Qx ( k ) − xT ( k )FT PGS −1GT PFx ( k )
+ 12 xT ( k ) FT PGS −1GT PFx( k ) = 0
or,

2
[x (k )[FT PF − P + Q − FT PGS −1G T PF ]x(k )]
1 T
=0
The above equation should hold for any x. For this to be true we have to have

( )
−1
FT PF − P + Q − FT PG R + G T PG G T PF = 0 (8.58)
This equation is called the discrete algebraic Riccati equation.
The discrete matrix Riccati equation given in (8.58), is one of the many equivalent forms which satisfy the
optimal regulator design. The analytical solution of the discrete Riccati equation is possible only for very
simple cases. A number of computer programs for the solution of the discrete Riccati equation are available
[152–154]. Appendix A provides some MATLAB support.

Example 8.7 Consider the problem of digital control of a plant described by the transfer function
1
G(s) =
s+1
Discretization of the plant model gives

Gh0G(z) =
Y (z)
=Z
LMFG 1 - e IJ F 1 I OP = (1 – z
- sT
–1
)Z
LM 1 OP = 1 - e -T

U ( z) NH s K H s + 1K Q N s(s + 1) Q z - e -T

For a sampling interval T = 1 sec,

0.632
Gh0G(z) =
z - 0.368
The difference equation model of the plant is
y(k + 1) = 0.368 y(k) + 0.632 u(k) (8.59)
The design specifications are given below:
(i) Minimize
¥
J= 1
2 å [ ~y 2(k) + u~ 2(k)]
k = 0
where ~ y and u~ are, respectively, the deviations of the output and the control signal from their steady-
state values.
(ii) For a constant yr , y(¥) = yr , i.e., there is zero steady-state error.
 Linear Quadratic Optimal Control Through Lyapunov Synthesis 483

y(k ) For this design problem, we select the

yr + u (k) 0.632
N feedback plus feedforward control scheme
z – 0.368 shown in Fig. 8.15. The feedback gain K is
–
obtained from the solution of the shifted
regulator problem, as is seen below.
K Let ys and us be the steady-state values of
the output and the control signal, respectively.
Fig. 8.15 Feedback plus feedforward control scheme Equation (8.59) at steady-state becomes
ys = 0.368 ys + 0.632 us
The state equation (8.59) may equivalently be expressed as
~
y (k + 1) = 0.368 ~ y (k) + 0.632 u~ (k)
where
~
y = y – ys; u~ = u – us
In terms of this equivalent formulation, the optimal control problem is to obtain
u~ (k) = – K ~
y (k)
so that
¥
J= 1
2 å [ ~y 2(k) + u~ 2(k)]
k = 0
is minimized.
For this shifted regulator problem,
F = 0.368, G = 0.632, Q = 1, R = 1
The Riccati equation (8.58) gives
P = Q + FTPF – FTPG(R + GTPG)–1GTPF
= 1 + (0.368)2P – (0.368)P(0.632)[1 + (0.632)2P]–1(0.632)P(0.368)
0.054 P 2
= 1 + 0.135P –
1 + 0.4 P
Solving for P, requiring it to be positive definite, we obtain,
P = 1.11
Feedback gain (refer Eqn. (8.56))
K = (R + GTPG)–1GTPF = 0.18
The feedforward gain (refer Eqn. (7.105)) N, is given by
1
= – C(F – GK – I )–1G = – [0.368 – 0.632 (0.18) – 1]–1 (0.632)
N
0.746
or N= = 1.18
0.632
The optimal control sequence (refer Eqn. (7.106))
u(k) = – Ky(k) + Nyr = – 0.18 y(k) + 1.18 yrh
Substituting in Eqn. (8.59), we obtain
y(k + 1) = 0.254 y(k) + 0.746 yr
At steady-state,
0.746
y(¥) = ys = yr = yr
0.746
 484 Digital Control and State Variable Methods

8.9 CONSTRAINED STATE FEEDBACK CONTROL

In classical control theory, as we have seen in Chapters 1–4, output feedback controller is the common
control structure. These controllers are designed in the context of an input-output transfer function model.
When we move to a state variable model for the system, we have time-domain information about the internal
structure of the system, available in the form of a state vector. To investigate a link between the state infor-
mation and output feedback, we set up an output feedback regulator system.
The system equations are
x& = A x + B u ; x(0) = D x0
( n ×1) ( p ×1)

y = Cx (8.60)
( q ×1)
and the output feedback control law is
u (t ) = − K 0 y (t ) (8.61)
If we now progress this analysis for the closed-loop system, we substitute for the control law in system
equations. This gives
x& = ( A − BK 0 C) x (8.62)
The design problem can be stated through the closed-loop characteristic polynomial D(s) that specifies
the poles of the closed-loop system (eigenvalues of the matrix (A – BK0C)):
Δ( s) = s n + α1s n−1 + L + α n
The problem is to choose the available controller gain matrix K0 so that the specified characteristic
polynomial D(s) equals the characteristic polynomial of the matrix (A – BK0C):
s n + α1s n −1 + L + α n = sI − ( A − BK 0 C ) (8.63)
The output feedback in a state variable framework does not necessarily have sufficient degrees of free-
dom to satisfy this design requirement. The output feedback gain K0 will have p ´ q parameters to tune n
coefficients of the closed-loop characteristic polynomial. In most real systems, the order of the system n will
be very much greater than the number of measurements q and/or control p. The important issue here is that
the output vector is only a partial view of the state vector.
With full state feedback, a closed-form solution to the pole-placement problem exists under mildly
restrictive condition on controllability of the system. In case of single-input systems, the solution is unique
(given earlier in Chapter 7). In case of multi-input systems, the solution is not unique. In fact, there is lot of
freedom available in the choice of state-feedback gain matrix; this freedom is utilized to serve other objec-
tives on system performance [105].
With partial state feedback (output feedback), a closed-form solution to the pole-placement problem may
not exist. The designer often solves the problem numerically; tuning the output feedback gain matrix by trial
and error to obtain approximate pole-placement solution and checking the acceptability of the approxima-
tion by simulation.
The output feedback law is restricted in design achievements, while the state feedback law is able to give
total control over system dynamics. In fact, as we have seen, the design flexibility of the state feedback law
is supported by deep technical results to guarantee the design properties. However, this design flexibility of
state-feedback is achieved because it has been assumed that we can access each state variable. In practice,
this means that we must have a more complicated system where we include an observer which provides
us with the state information. Since an observer incorporates model of the process, dependence on very
 Linear Quadratic Optimal Control Through Lyapunov Synthesis 485
accurate representation of the process being controlled for accurate state information, is obvious. We know
that industrial process models are not usually so well known or accurate. Therefore, inclusion of an observer
is bound to deteriorate the robustness properties of the feedback system.
Therefore, inspite of excellent design flexibility in state feedback, we are, many a times, forced to look at
the alternatives, not so excellent in terms of design flexibility but not dependent on inclusion of an observer.
Constrained state feedback is an interesting alternative. Here, we set the gains of the state-feedback gain
matrix corresponding to unmeasurable states, to zero and try to exploit the rest of the gains to achieve the
desired properties of the closed-loop system. This, in fact, may also be viewed as an output feedback prob-
lem where the state x passes through an appropriately selected output matrix C to give the output variables
in the vector y.
The constrained state feedback control law is not supported by deep technical results and does not guar-
antee the design properties; nonetheless, it yields robust feedback control systems for many industrial con-
trol problems. Existence of a control law that stabilizes the feedback system is a pre-requisite of the design
algorithm. Unfortunately, general conclusions for existence of a stabilizing control law with constrained
state feedback cannot be laid down; therefore one resorts to numerical methods to establish the existence.
In the following, we outline a procedure for the design of constrained state feedback control law that
minimizes a quadratic performance index. By constrained state feedback we mean that all the parameters of
the matrix K are not available for adjustments. Some of them may have zero values (output feedback). The
procedure described below is equally applicable to situations wherein some of the parameters of K have
fixed values.
Let us consider the system (8.60). It is desired to minimize the following performance index:
¥
J= 1
2 z
0
(xTQx + uTRu) dt (8.64)

where Q is n ´ n positive definite, real, symmetric, constant matrix, and R is p ´ p positive definite, real,
symmetric, constant matrix.
We shall obtain a direct relationship between Lyapunov functions and quadratic performance measures,
and solve the constrained parameter-optimization problem using this relationship. We select the feedback
control configuration described by the control law
u = – K0C x = – Kx (8.65)
where K is p × n matrix which involves adjustable parameters. With this control law, the closed-loop system
becomes
x& = Ax – BKx = (A – BK)x (8.66)
All the parameters of the matrix K are not available for adjustments. Some of them have fixed values
(zero values). We will assume that a matrix K satisfying the imposed constraints on its parameters exists
such that (A – BK) is a stable matrix.
The optimization problem is to determine the values of free parameters of the matrix K so as to minimize
the performance index given by Eqn. (8.64).
Substituting the control vector u from Eqn. (8.65) in the performance index J of Eqn. (8.64), we have
¥
J= 1
2 z
0
(xTQx + xTKTRKx)dt

¥
= 1
2 z
0
xT(Q + KTRK)x dt (8.67)
 486 Digital Control and State Variable Methods
Let us assume a Lyapunov function
¥
V(x(t)) = 1
2 z
0
xT(Q + KTRK)x dt

Note that the value of the performance index for system trajectory starting at x(0) is V(x(0)).
The time derivative of the Lyapunov function is
¥

V& (x) = 1
2
T
x ( Q + K RK ) x
T

t
T T
= 1
2 x (¥) [Q + K RK]x(¥) – 1
2 xT(t) [Q + KTRK] x(t)
Assuming that the matrix (A – BK) is stable, we have from Eqn. (8.66),
x(¥) ® 0
Therefore,
V& (x) = –
xT(Q + KTRK)x
1
2 (8.68)
&
Since V (x) is quadratic in x and the plant equation is linear, let us assume that V(x) is also given by the
quadratic form:
V(x) = 1
2 xTPx (8.69)
where P is a positive definite real, symmetric, constant matrix. Therefore,
V& (x) = 1 ( x& T Px + x T Px& )
2
Substituting for x& from Eqn. (8.66), we get
V& (x) = 1 xT[(A – BK)T P + P(A – BK)]x
2
Comparison of this result with Eqn. (8.68) gives
1
2 xT[(A – BK)TP + P(A – BK)]x = – 1
2 xT(Q + KTRK)x
Since the above equality holds for arbitrary x(t), we have
(A – BK)T P + P(A – BK) + KTRK + Q = 0 (8.70)
This equation is of the form of Lyapunov equation defined in Section 8.3. In Eqn. (8.70) we have n2
n (n + 1)
nonlinear algebraic equations. However, since n ´ n matrix P is symmetric, we need to solve only
2
equations for the elements pij of the matrix P. The solution will give pij as functions of the feedback matrix K.
As pointed out earlier, V(x(0)) is the value of the performance index for the system trajectory starting at
x(0). From Eqn. (8.69), we get,
J= 1
2 xT(0) Px(0) (8.71)
A suboptimal control law may be obtained by minimizing J with respect to the available elements kij of K,
i.e., by setting
∂[x T (0)Px(0)]
=0 (8.72)
∂kij
If for the suboptimal solution thus obtained, the matrix (A – BK) is stable, then the minimization of J as
per the procedure described above gives the correct result. From Eqn. (8.68) we observe that for a positive
definite Q, V& (x) < 0 for all x ¹ 0 (note that KTRK is non-negative definite). Also Eqn. (8.69) shows that
 Linear Quadratic Optimal Control Through Lyapunov Synthesis 487
V(x) > 0 for all x ¹ 0 if P is positive definite. Therefore minimization of J with respect to kij (Eqn. (8.72))
will lead to a stable closed-loop system if the optimal kij result in a positive definite matrix P.
One would like to examine the existence of a solution to the optimization problem before actually starting
the optimization procedure. For the problem under consideration, existence of K that minimizes J is ensured
if, and only if, there exists a K satisfying the imposed constraints on the parameters, such that (A – BK) is
asymptotically stable. The question of existence of K has, as yet, no straightforward answer. We resort to
numerical trial-and-error to find a stabilizing matrix K (Such a matrix is required by numerical algorithms
for optimization of J [105]). Failure to do so does not mean that a suboptimal solution does not exist.
Also note that the solution is dependent on initial condition (Eqn. (8.72)). If a system is to operate satisfac-
torily for a range of initial disturbances, it may not be clear which is the most suitable for optimization.
The dependence on initial conditions can be avoided by averaging the performance obtained for a linearly
independent set of initial conditions. This is equivalent to assuming the initial state x (0) to be a random
variable, uniformly distributed on the surface of an n-dimensional sphere, i.e.,
E{x(0)xT (0)} = I
where E{.} denotes the expected value.
We define a new performance index
Jˆ = E {J } = E {
1 T
2
x }
(0)Px(0)

= 12 E{trace (Px(0)xT (0))}

= 12 trace (PE{x(0) xT (0)})
= 12 trace P (8.73)
Reference [105] describes a numerical algorithm for the minimization of Ĵ . When feedback matrix K is
unconstrained, the resulting value of J is optimal; J(optimal) < J (suboptimal).
As we have seen earlier in this chapter, the optimal solution is independent of initial conditions. It is
computationally convenient to use Riccati equation (8.41) for obtaining optimal control law (8.40).
Consider now the discretized model of the given plant:
x(k + 1) = Fx(k) + Gu(k); x(0) =D x0
(8.74)
y(k) = Cx(k)
where x is the n ´ 1 state vector, u is the p ´ 1 input vector, y is the q ´ 1 output vector; F, G, and C are,
respectively, n ´ n, n ´ p, and q ´ n real constant matrices; and k = 0, 1, 2, . . .
We shall be interested in selecting the controls u(k); k = 0, 1, . . ., which minimize a performance index of
the form
¥
J= 1
2 å [xT(k)Qx(k) + uT(k)Ru(k)] (8.75)
k=0
where Q is an n ´ n positive definite, real, symmetric, constant matrix, and R is a p ´ p positive definite, real,
symmetric, constant matrix.
The constrained state-feedback control law is
u(k) = – K0C x (k) = – Kx(k) (8.76)
where K is a p × n constant matrix.
 488 Digital Control and State Variable Methods
With the linear feedback control law (8.76), the closed-loop system is described by
x(k + 1) = (F – GK)x(k) (8.77)
We will assume that a matrix K exists such that (F – GK) is a stable matrix.
Substituting for the control vector u(k) from Eqn. (8.76) in the performance index J given by Eqn. (8.75),
we get
¥
J= 1
2 å xT(k) [Q + KTRK]x(k) (8.78)
k=0
Let us assume a Lyapunov function
¥
V(x(k)) = 12 å xT(i) [Q + KTRK]x(i) (8.79)
i=k
Note that the value of the performance index for system trajectory starting at x(0) is V(x(0)). The difference
V(x(k + 1)) – V(x(k)) = DV(x(k)) = – 1
2 xT(k)[Q + KTRK]x(k) (8.80)
(Note that x(¥) has been taken as zero under the assumption of asymptotic stability of the closed-loop
system).
Since DV(x(k)) is quardratic in x(k) and the plant equation is linear, let us assume that V(x(k)) is also
given by the quadratic form:
V(x(k)) = 1
2 xT(k) Px(k) (8.81)
where P is a positive definite, real, symmetric, constant matrix.
Therefore
DV(x(k)) = 1
2 xT(k + 1)Px(k + 1) – 1
2 xT(k)Px(k)
Substituting for x(k + 1) from Eqn. (8.77), we get

DV(x(k)) = 1
2 xT(k) [(F – GK)TP(F – GK) – P]x(k)
Comparing this result with Eqn. (8.80), we obtain
(F – GK)TP(F – GK) – P + KTRK + Q = 0 (8.82)
This equation is of the form of Lyapunov equation defined in Section 8.3.
Since V(x(0)) is the value of the performance index, we have
J= 1
2 xT(0)Px(0) (8.83)
Assuming x(0) to be a random variable uniformly distributed on the surface of n-dimensional sphere, the
problem reduces to minimization of
J$ = 1
2
trace P (8.84)
Reference [105] describes a numerical algorithm for this minimization problem.

Review Examples

Review Example 8.1 Using the Lyapunov equation, determine the stability range for the gain K of
the system shown in Fig. 8.16.
 Linear Quadratic Optimal Control Through Lyapunov Synthesis 489

r + K x3 1 x2 1 x1
s+1 s+2 s
–

Fig. 8.16
Solution The state equation of the system is
x&1 0 LM
1 0 OP LM OP LM x OP LM 0 OP
1
x&2 = 0 -2
x&3 -K
1
MM
0 -1
PP MM PP MMxx PP + MMK0 PP r
2

N Q N QN Q N Q 3
For the investigation of asymptotic stability, we consider the system
x& = Ax
with
0 1 0 LM OP
A= -2
MM- K0 1
-1PQ
P
Clearly, the equilibrium state is the origin.
N 0

Let us choose a Lyapunov function

V(x) = xTPx
where P is to be determined from the Lyapunov equation
ATP + PA = – Q
The matrix Q could be chosen as identity matrix. However, we make the following choice for Q:
LM0 0 0 OP
Q= 0 0 0
MM PP
0 0 1 N Q
This is a positive semidefinite matrix which satisfies the condition (8.10) as is seen below.
0 0 0 0LM OP LM OP
Q = 0 0 0 = 0 [0 0 1] = HTH
MM PP MM PP
0 0 1 1N Q NQ
H LM 0 0 1 OP LM OP
r HA = r - K 0 -1 = 3
2
MM
K -K
PP MM PP
HA N 1 Q N Q
With this choice of Q, as we shall see, manipulation of the Lyapunov equation for its analytical solution
becomes easier.
Now let us solve the Lyapunov equation
ATP + PA = – Q
LM0 0 -K OP LM p 11 p12 p13 p11 p12 OP LM p13 OP LM 0 1 0OP LM
0 0 0 OP
-2 p22 p23 + p12 p22 -2
or
MM01 0
PP MM pp
12 PP MM p23 PP M 0 1 = 0 0 0
PP MM PP
N 1 -1 QN 13 p23 p33 Q Np 13 p23 p33 Q MN- K 0 -1 Q 0 0 -1
N Q
 490 Digital Control and State Variable Methods
Solving this equation for pij’s, we obtain
LM K + 12K2
6K
0
OP
MM 12 6−K2 K 12 − 2 K
3K K
PP
P = M 12 − 2 K 12 − 2 K 12 − 2 K P
MM K 6
PP
MN 0 12 − 2 K 12 − 2 K PQ
For P to be positive definite, it is necessary and sufficient that
(12 – 2K) > 0 and K > 0 or 0 < K < 6
Thus for 0 < K < 6, the system is asymtotically stable.

Review Example 8.2 Consider the system described by the equations

x1(k + 1) = 2x1(k) + 0.5x2(k) – 5
x2(k + 1) = 0.8x2(k) + 2
Investigate the stability of the equilibrium state using Lyapunov equation.

Solution The equilibrium state xe =

LM x OP can be determined from the equations
e
1
e
Nx Q2

x1e = 2x1e + 0.5x2e – 5

x2e = 0.8x2e + 2
Solving, we get
LM x OP = L 0 O
e
1

N x Q MN10PQ
e
2
Define
~
x1(k) = x1(k) – x1e
~
x2(k) = x2(k) – x2e
In terms of the shifted variables, the system equations become
x~1( k + 1)
LM 2 0.5 x~1( k )
OP LM OP LM OP
=
x~ ( k + 1)
N 2 0 0.8 x~ ( k )
Q N QN 2 Q
or ~x (k +1) = F ~x (k) (8.85)
Clearly ~
x = 0 is the equilibrium state of this autonomous system.
Let us choose a Lyapunov function
V( ~
x)= ~ x TP ~
x
where P is to be determined from the Lyapunov equation
FTPF – P = – I

or
LM 2 0 OP LM p
11 p12 OP LM2 0.5OP LM
p
- 11
p12 OP = LM-1 0 OP
N 0.5 0.8 Q Np 12 p22 Q N0 0.8 p12
Q N p22 Q N 0 -1Q
 Linear Quadratic Optimal Control Through Lyapunov Synthesis 491
Solving for pij’s, we get

P=
LM- 1
3
5
9
OP
5 1225
MN 9 324 PQ
By applying the Sylvester’s test for positive definiteness, we find that the matrix P is not positive definite.
Therefore the origin of the system (8.85) is not asymptotically stable.
In terms of the original state variables, we can say that the equilibrium state
xe = [0 10]T of the given system is not asymptotically stable.

Review Example 8.3 Referring to the block diagram of Fig. 8.17, consider that G(s) = 100/s2 and R(s)
= 1/s. Determine the optimal values of parameters k1 and k2 such that
¥

z
J = [e2(t) + 0.25u2(t)]dt
0
is minimized.
R (s ) + E (s) + U (s) Y (s)
k1 G (s)
– –

sk 2

Fig. 8.17
Solution From Fig. 8.17, we obtain
y (t) = 100u(t); y(0) = y& (0) = 0
&&
u(t) = k1[r – y(t) – k2 y& (t)]
In terms of the state variables
~
x1 (t) = y(t) – r
~
x 2 (t) = y& (t),
the state variable model of the system becomes
~
x& =
LM OP
0 1 ~
x +
0 LM OP
u; ~
x (0) =
-1 LM OP
0 0 N Q 100 N Q 0 N Q
u = – k x~ – k k x~ = – K x ~ (8.86)
1 1 1 2 2
where
K = [k1 k1 k2]
The optimization problem is to find K such that
¥
J= z (~
x12 + 0.25u2)dt
0
is minimized.
Note that

A=
LM0 1OP ; B = LM 0 OP ; Q = L2 0O ; R = 0.5
N0 0Q N100Q MN0 0PQ
 492 Digital Control and State Variable Methods
The matrix Riccati equation is
ATP + PA – PBR–1BTP + Q = 0
or
LM0 0OP LM p p OP + LM p
11 12 OP LM0 1OP 11 p12
N1 0 Q N p p Q N p
12 22 Q N0 0 Q 12 p22

–
LM p
11 p 12 OP LM 0 OP [ ] [0 100] LM p
1
p O L2 0 O L0 0 O
+ =
11 12

Np12 p 22 Q N100Q 0.5

Np p PQ MN0 0 PQ MN0 0 PQ
12 22
Solving for p11, p12, and p22 , requiring P to be positive definite, we obtain

P= M
L2 ´ 10 10 OP -1 -2

-2 -3
N 10 10 Q
The feedback gain matrix
K = R–1BTP

= 1
[0 100]
LM2 ´ 10 -1
10 - 2 OP = [2 0.2]
0.5 -2 -3
N 10 10 Q
From Eqn. (8.86), we obtain
[k1 k1k2] = [2 0.2] or k1 = 2, k2 = 0.1

Review Example 8.4 Figure 8.18 shows the optimal control configuration of a position servo system.
Both the state variables—angular position q and angular velocity q& —are assumed to be measurable.

r + + u 20 x2 = 1 x1 =
k1
s+2 s
– –

k2

Fig. 8.18 A position servo system

It is desired to regulate the angular position to a unit-step function qr. Find the optimum values of the
gains k1 and k2 that minimize
¥

z
J = [(x1 – qr)2 + u2 ]dt
0

Solution The state variable description of the system, obtained from Fig. 8.18, is given by

x& =
0 1
x+
0
u
LM OP LM OP
0 -2 20 N Q N Q
y = x1
In terms of the shifted state variables
~
x1 = x1 – qr
~
x 2 = x2,
 Linear Quadratic Optimal Control Through Lyapunov Synthesis 493
the state variable model becomes
~
x& = A ~
x + Bu (8.87)
with
1
,B= A=
0 LM 0 OP LM OP
-2 20 N0 Q N Q
The design problem is to determine optimal control
u = – k1 ~
x1 – k2 ~
x2 (8.88)
that minimizes
¥
J = (~ z
x12 + u2)dt
0
For this J,
LM2 0OP ; R = 2
Q=
N0 0 Q
The matrix Q is positive semidefinite;
L 2O
T
Q = H H = M P [ 2 0]
N0Q
The pair (A,H) is completely observable. Also, the pair (A,B) is completely controllable. Therefore, the
sufficient conditions for the existence of asymptotically stable optimal closed-loop system are satisfied.
The matrix Riccati equation is
ATP + PA – PBR–1BTP + Q = 0

or
LM0 0 OP LM p
OP + LM p p OP LM0 1 OP
11 p12 11 12

N1 -2 Q N p p Q N0 -2Q
Q Np 12 p22 12 22

L p p OP LM 0 OP [ ] [0 20] LM p p OP + LM2 0OP = LM0 0OP

–M 11 12 1 11 12
2
N p p Q N20Q12 22 N p p Q N0 0Q N0 0Q 12 22

LM 0 p - 2 p O L 200 p
11 12200 p p O L2 0O L0 0O2
12 12 22

N p - 2 p 2 p - 4 p Q N200 p p 200 p PQ MN0 0PQ MN0 0PQ

or P –M + = 2
11 12 12 22 12 22 22
2
or 2 – 200 p12 =0
2
2p12 – 4p22 – 200 p22 =0
p11 – 2p12 – 200p12 p22 = 0
The positive definite solution of the Riccati equation is

P=
0.664 0.1 LM OP
0.1 0.0232 N Q
The optimal gain matrix
K = R–1BTP = [1 0.232]
The minimum value of J for an initial condition ~ x (0) = [–1 0]T, is
p
J = 21 ~
x T(0)P ~
x (0) = 11 = 0.332
2
It can easily be verified that the optimal closed-loop system is stable.
 494 Digital Control and State Variable Methods

Review Example 8.5 Figure 8.19a illustrates a typical sampled-data system. The transfer functions G(s)
and Gh0(s) of the controlled plant and the hold circuit, respectively, are known. The data-processing unit
D(z) which operates on the sampled error signal e(k) is to be designed.
Assuming the processing unit D(z) to be simply an amplifier of gain K, let us find K so that the sum square
error
¥
J= å [e 2 ( k ) + 0.75u 2 ( k )]
k = 0
is minimized.
Solution From Fig. 8.19a, we have
1 - e - sT
Gh0(s)G(s) =
s2
Therefore

Gh0G(z) = (1 – z–1) Z
LM 1 OP = 1
2
Ns Q z - 1
Figure 8.19b shows an equivalent block diagram of the sampled-data system.
r = unit step T = 1 sec
+ e (t ) 1 – e–sT 1 y ( t)
D (z) Gh0 = G= s
s
–

(a)

r + e(k ) u (k ) y (k )
1
K
z–1
–

(b)
Fig. 8.19 A sampled-data system

From this figure, we obtain the following state variable model:

y(k + 1) = y(k) + u(k)
(8.89)
u(k) = – K [y(k) – r]
In terms of the shifted state variable,
~
x (k) = y(k) – r,
the state equation becomes
~ ~
x (k + 1) = x (k) + u(k) (8.90)
The problem is to obtain optimal control sequence
u(k) = – K ~
x (k)
 Linear Quadratic Optimal Control Through Lyapunov Synthesis 495
that minimizes the performance index
¥
J= å [ ~x 2(k) + 0.75u2(k)]
k = 0

For this problem,

F = 1, G = 1, Q = 2, R = 1.5
The Riccati equation is (refer Eqn. (8.58))
P = Q + FTPF – FTPG(R + GTPG)–1GTPF
P2
=2+P–
1.5 + P
Solving for P, requiring it to be positive definite, we get
P=3
The optimal control (refer Eqn. (8.56))
u(k) = – K ~
x (k)
where
P 2
K = (R + GTPG)–1GTPF = = .
1 .5 + P 3

Problems
8.1 Consider the linear system

x& =
LM 0 1OP x
N-1 -2Q
Using Lyapunov analysis, determine the stability of the equilibrium state.
8.2 Using Lyapunov analysis, determine the stability of the equilibrium state of the system
x& = Ax

with A=
LM 0 1OP
N-1 1Q
8.3 Consider the system described by the equations
x&1 = x2
x& 2 = – x1 – x2 + 2
Investigate the stability of the equilibrium state. Use Lyapunov analysis.
8.4 A linear system is described by the state equation
x& = Ax
where

A=
LM- 4K 4K OP
N 2K - 6K Q
Using Lyapunov analysis, find restrictions on the parameter K to guarantee the stability of the system.
496 Digital Control and State Variable Methods
8.5 Consider the system of Fig. P8.5. Find the restrictions on the parameter K to guarantee system stability. Use
Lyapunov’s analysis.

K x1 1 x2 1 x3
s+1 s+1 s+1
–

Fig. P8.5

8.6 Consider the linear system

x(k + 1) =
LM 0.5 1OP x(k)
N-1 -1Q
Using Lyapunov analysis, determine the stability of the equilibrium state.
8.7 Using Lyapunov analysis, determine the stability of the equilibrium state of the system
x(k + 1) = Fx(k)
with

F=
LM 0 0. 5 OP
N-0.5 -1 Q
8.8 Consider the system shown in Fig. P8.8. Determine the optimal feedback gain matrix K, such that the following
performance index is minimized:
¥
LM2 0OP
J= 1
2 z
0
(xTQx + 2u2)dt; Q =
N0 2 Q
u 1 x2 1 x1
s s

–K

Fig. P8.8

8.9 The matrix Q in Problem 8.8 is replaced by the following positive semidefinite matrix:

Q=
LM2 0OP
N0 0 Q
Show that sufficient conditions for the existence of the asymptotically stable optimal control solution are
satisfied. Find the optimal feedback matrix K.
8.10 Test whether sufficient conditions for the existence of the asymptotically stable optimal control solution for the
plant

x& =
0 0 LM
x+
1
u
OP LM OP
0 1 N 1 Q NQ
 Linear Quadratic Optimal Control Through Lyapunov Synthesis 497
with the performance index
¥

z
J = ( x12 + u2) dt
0

are satisfied. Find the optimal closed-loop system, if it exists, and determine its stability.
8.11 Consider the plant

x& =
LM
-1 0
x+
1OP LM OP
u
N
1 0 0 Q NQ
with the performance index
¥

z
J = ( x12 + u2) dt
0

Test whether an asymptotically stable optimal solution exists for this control problem.
8.12 Consider the system described by the state model:

x& =
LM0 1 OP x + LM 0OP u
N0 -2 Q N20Q
y = [1 0] x
Find the optimal control law that minimizes
¥
J= 1
2 z
0
[(y(t) – 1)2 + u2] dt

8.13 Determine the optimal control law for the system

LM0
x& =
1 OP LM OP
x+
0
u
N0 0 Q NQ
1

L1 0O
y=M x
N0 2PQ
such that the following performance index is minimized
¥
J= z
0
( y12 + y22 + u2)dt

8.14 Consider the plant

x& = Ax + Bu
y = Cx

A=
LM0 1OP , B = LM0OP , C = [1 0]
N0 -1Q N1Q
with the performance index
¥
J= z
0
( x12 + x22 + u2) dt

Choose a control law that minimizes J. Design a state observer for implementation of the control law; both the
poles of the state observer are required to lie at s = – 3.
8.15 Figure P8.15 shows the optimal control configuration of a position servo system. Both the state variables—
angular position q and angular velocity q& —are assumed to be measurable.
498 Digital Control and State Variable Methods
It is desired to regulate the angular position to a constant value qr = 5. Find optimum values of the gains k1 and
k2 that minimize
¥
J= z
0
[(x1 – qr)2 + 1
2
u2] dt

What is the minimum value of J?

r + + u 1 x2 = 1 x1 =
k1 s+5 s
– –

k2

Fig. P8.15

8.16 Consider now that for the position servo system of Problem 8.15, the performance index is
¥

z
J = [(x1 – qr)2 + r u 2]dt
0
1 1 1
For r = 10
, 100
, and 1000
, find the optimal control law that minimizes the given J. Determine closed-loop poles
for various values of r and comment on your result.
8.17 In the control scheme of Fig. P8.17, the control law of the form u = – Ky + Nyr has been used; yr is the constant
command input.
w

+ y
yr + u + 1
N
s+1
–

Fig. P8.17

(a) Find K such that

¥
J = (~
y 2 + u~ 2) dt
z
0

is minimized; ~ y and u~ are, respectively, the deviations of the output and the control signal from their
steady-state values.
(b) Choose N so that the system has zero steady-state error, i.e., y(¥) = yr.
(c) Show that the steady-state error to a constant disturbance input w, is non-zero for the above choice of N.
(d) Add to the plant equation, an integrator equation (z(t) being the state of the integrator),
z& (t) = y(t) – yr
 Linear Quadratic Optimal Control Through Lyapunov Synthesis 499
and select gains K and K1 so that if u = – Ky – K1z, the performance index
¥
J = (~
z z 2 + u~ 2)dt
y2 + ~
0
is minimized.
(e) Draw a block diagram of the control scheme employing integral control and show that the steady-state error
to constant disturbance w, is zero.
8.18 Consider a plant consisting of a dc motor, the shaft of which has the angular velocity w (t), and which is driven
by the input voltage u(t). The describing equation is
w& (t) = – 0.5w (t) + 100u(t) = Aw(t) + Bu(t)
It is desired to regulate the angular velocity at the desired value w0 = r.
(a) Use the control law of the form u(t) = – Kw (t) + Nr.
¥
~ and u~ are, respectively, the deviations of the output
~ 2 + 100 u~ 2 ) dt; w
Choose K that minimizes J = ( w z
0
and the control signal, from their steady-state values.
Choose N that guarantees zero steady-state error, i.e.,
w (¥) = w 0 = r.
(b) Show that if A changes to A + dA, subject to (A + dA – BK) being stable, then the above choice of N will no
longer make w (¥) = r. Therefore, the feed forward-feedback control system is not robust under changes in
system parameters.
(c) The system can be made robust by augmenting it with an integrator:
z& = w – r
where z is the state of the integrator. To see this, first use the feedback of the form u = – Kw (t) – K1z(t) and
select K and K1 so that
¥
~
z z 2 + 100 u~ 2) dt
J = (w 2 + ~
0
is minimized. Show that the resulting system will have w (¥) = r, no matter how the matrix A changes, so
long as the closed-loop system remains asymptotically stable.
8.19 Consider the system
x(k + 1) = 0.368 x(k) + 0.632 u(k)
Using the discrete matrix Riccati equation, find the control sequence
u(k) = – Kx(k)
that minimizes the performance index
∞
J= ∑ [ x 2 (k ) + u2 (k )]
k = 0
8.20 Consider the sampled-date system shown in Fig. P8.20.
(a) Find K so that
∞
J= 1
2 ∑ [~y 2 (k ) + u~2 (k )]
k = 0

is minimized; ~y and u~ are, respectively, the deviations of the output and the control signal, from their
steady-state values.
(b) Find the steady-state value of the output.
(c) To eliminate steady-state error, introduce a feedforward controller. The control scheme now becomes u(k)
= – Ky(k) + Nr. Find the value of N so that y(¥) = r.
500 Digital Control and State Variable Methods
r = unit step T = 0.5 sec
y
+ u 1 – e– sT 1
K Gh0 = G=
s s+1
–

Fig. P8.20

8.21 A plant is described by the state equation

x(k + 1) = 0.5x(k) + 2u(k) = Fx(k) + Gu(k)
(a) Find K such that if u(k) = – Kx(k) + Nr, the performance index
¥

J= 1
2
å [ ~x 2(k) + u~ 2(k)]
k=0
is minimized; r is a constant reference input, and ~ x and u~ are, respectively, the deviations in state and
control signal, from their steady-state values.
(b) Find N so that x(¥) = r, i.e., there is no steady-state error.
(c) Show that the property of zero steady-state error is not robust with respect to changes in F.
(d) In order to obtain robust steady-state accuracy with respect to changes in F, we may use integral control in
addition to state feedback. Describe through block diagram, the structure of such a control scheme.
 Part III
NONLINEAR CONTROL SYSTEMS: CONVENTIONAL
AND INTELLIGENT

In this part of the book, we will explore tools and techniques for attacking control problems that contain
significant nonlinearities.
Nonlinear control system design has been dominated by linear control techniques, which rely on the
key assumption of a small range of operation for the linear model to be valid. This tradition has produced
many reliable and effective control systems. However, the demand for nonlinear control methodologies
has recently been increasing for several reasons.
First, modern technology, such as applied in high-performance aircraft and high-speed high-accuracy
robots, demands control systems with much more stringent design specifications, which are able to han-
dle nonlinearities of the controlled systems more accurately. When the required operation range is large,
a linear controller is likely to perform very poorly or to be unstable, because the nonlinearities in the
system cannot be properly compensated for. Nonlinear controllers, on the other hand, may directly handle
the nonlinearities in large range operation. Also, in control systems there are many nonlinearities whose
discontinuous nature does not allow linear approximation.
Second, controlled systems must be able to reject disturbances and uncertainties confronted in real-
world applications. In designing linear controllers, it is usually necessary to assume that the parameters of
the system model are reasonably well known. However, many control problems involve uncertainties in
the model parameters. This may be due to a slow time variation of the parameters (e.g., of ambient air
pressure during an aircraft flight), or to an abrupt change in parameters (e.g., in the inertial parameters of
a robot when a new object is grasped). A linear controller, based on inaccurate values of the model
parameters, may exhibit significant performance degradation or even instability. Nonlinearities can be
intentionally introduced into the controller part of a control system, so that model uncertainties can be
tolerated.
Third, advances in computer technology have made the implementation of nonlinear controllers a
relatively simple task. The challenge for control design is to fully utilize this technology to achieve the
best control system performance possible.
Thus, the subject of nonlinear control is an important area of automatic control. Learning basic tech-
niques of nonlinear control analysis and design can significantly enhance the ability of control engineers
to deal with practical control problems, effectively. It also provides a sharper understanding of the real
world, which is inherently nonlinear.

Nonlinear Systems Analysis

No universal technique works for the analysis of all nonlinear control systems. In linear control, one can
analyze a system in the time-domain or in the frequency-domain. However, for nonlinear control systems,
none of these standard approaches can be used, since direct solutions of nonlinear differential equations
are generally difficult, and frequency-domain transformations do not apply.
While the analysis of nonlinear control systems is difficult, serious efforts have been made to develop
appropriate theoretical tools for it. Many methods of nonlinear control system analysis have been
proposed. Let us briefly describe some of these methods before discussing their details in the following
chapters.
502 Digital Control and State Variable Methods

Phase Plane Analysis Phase plane analysis, discussed in Chapter 9, is a method of studying second-
order nonlinear systems. Its basic idea is to solve a second-order differential equation and graphically
display the result as a family of system motion trajectories on a two-dimensional plane, called the phase
plane, which allow us to visually observe the motion patterns of the system. While phase-plane analysis
has a number of important advantages, it has the fundamental disadvantage of being applicable only to
systems which can be well approximated by a second- order dynamics. Because of its graphical nature, it
is frequently used to provide intuitive insights about nonlinear effects.
Lyapunov Theory In using the Lyapunov theory to analyze the stability of a nonlinear system, the
idea is to construct a scalar energy-like function (a Lyapunov function) for the system, and to see whether
it decreases. The power of this method comes from its generality; it is applicable to all kinds of control
systems. Conversely, the limitation of the method lies in the fact that it is often difficult to find a Lyapunov
function for a given system.
Although Lyapunov’s method is originally a method of stability analysis, it can be used for synthesis
problems. One important application is the design of nonlinear controllers. The idea is to somehow for-
mulate a scalar positive definite function of the system states, and then choose a control law to make this
function decrease. A nonlinear control system thus designed, will be guaranteed to be stable. Such a
design approach has been used to solve many complex design problems, e.g., in adaptive control and in
sliding mode control (discussed in Chapter 10). The basic concepts of Lyapunov theory have earlier been
presented in Chapter 8. We will extend our presentation on Lyapunov theory in Chapter 9, wherein guide-
lines for construction of Lyapunov functions for nonlinear systems will be given.
Describing Functions The describing function method, discussed in Chapter 9, is an approximate
technique for studying nonlinear systems. The basic idea of the method is to approximate the nonlinear
components in nonlinear control systems by linear “equivalents”, and then use frequency domain tech-
niques to analyze the resulting systems. Unlike the phase plane method, it is not restricted to second-order
systems. Rather, the accuracy of describing function analysis improves with an increase in the order of the
system. Unlike Lyapunov method, whose applicability to a specific system hinges on the success of a
trial-and-error search for a Lyapunov function, its application is straightforward for a specific class of
nonlinear systems.

Nonlinear Control Design

As in the analysis of nonlinear control systems, there is no general method for designing nonlinear con-
trollers. What we have is a rich collection of alternative and complementary techniques, each of them best
applicable to particular classes of nonlinear control problems.
Trial-and-Error Based on the analysis methods, one can use trial-and-error to synthesize controllers.
The idea is to use the analysis tools to guide the search for a controller, which can then be justified by
analysis and simulations. The phase plane method, the describing function method, and Lyapunov analy-
sis can all be used for this purpose. Experience and intuition are critical in this process. However, for
complex systems, trial-and-error often fails.
Feedback Linearization Feedback linearization discussed in Chapter 10, can be used as a nonlinear
design methodology. The basic idea is to first transform a nonlinear system into a linear system using
feedback, and then use the well-known and powerful linear design techniques to complete the control
design. The approach has been used to solve a number of practical nonlinear control problems. It applies
to important classes of nonlinear systems.
 Nonlinear Systems Analysis 503

Variable structure sliding mode control In pure model-based nonlinear control (such as the basic
feedback linearization control approach), the control law is designed based on a nominal model of the
physical system. How the control system will behave in the presence of model uncertainties is not clear at
the design stage. In robust nonlinear control (e.g., variable structure sliding mode control), on the other
hand, the controller is designed based on the consideration of both the nominal model and some charac-
terization of the model uncertainties. Sliding mode control techniques, discussed in Chapter 10, have
proven very effective in a variety of practical control problems.
Adaptive Control Adaptive control is an approach to dealing with uncertain systems or time-varying
systems. Although the term “adaptive” can have broad meanings, current adaptive control designs apply
mainly to systems with known dynamic structure but unknown constant or slowly-varying parameters.
Adaptive controllers, whether developed for linear systems or for nonlinear systems, are inherently
nonlinear.
Systematic theories exist for the adaptive control of linear systems. Frequently used adaptive control
structures are discussed in Chapter 10.
Gain-scheduling Gain scheduling is an attempt to apply the well developed linear control methodol-
ogy to the control of nonlinear systems. The idea of gain-scheduling is to select a number of operating
points which cover the range of system operation. Then, at each of these points, the designer makes a
linear time-invariant approximation to the plant dynamics, and designs a linear controller for each
linearized plant. Between operating points, the parameters of the compensators are then interpolated, or
scheduled; thus resulting in a global compensator.
Intelligent Control In order to handle the complexities of nonlinearities and accomodate the demand
for high-performance control systems, intelligent control takes advantage of the computational struc-
tures—fuzzy systems and neutral networks—which are inherently nonlinear; a very important property,
particularly if the underlying physical mechanisms for the systems are highly nonlinear. Whereas classi-
cal control is rooted in the theory of differential equations, intelligent control is largely rule-based be-
cause the dependencies involved in its deployment are much too complex to permit an analytical repre-
sentation. To deal with such dependencies, the mathematics of fuzzy systems and neutral networks inte-
grates the experience and knowledge gained in the operation of a similar plant, into control algorithm.
The power of fuzzy systems lies in their ability (i) to quantify linguistic inputs, and (ii) to quickly give a
working approximation of complex, and often unknown, system input-output rules. The power of neural
networks is in their ability to learn from data. There is a natural synergy between neural networks and
fuzzy systems that makes their hybridization a powerful tool for intelligent control and other applications.
Intelligent control is one of the most serious candidates for the future control of the large class of
nonlinear, partially known, and time-varying systems.
With no agreed upon scientific definition of intelligence, and due to space limitations, we will not
venture into the discussion of what intelligence is. Rather, we will confine our brief exposition in Chap-
ters 11 and 12 to intelligent machines—neural networks, support vector machines, fuzzy inteference
systems, genetic algorithms—in the context of applications in control.
 9 Nonlinear Systems Analysis

9.1 INTRODUCTION

Because nonlinear systems can have much richer and complex behaviours than linear systems, their analysis
is much more difficult. Mathematically, this is reflected in two aspects. Firstly, nonlinear equations, unlike
linear ones, cannot, in general, be solved analytically, and therefore, a complete understanding of the behav-
iour of a nonlinear system is very difficult. Secondly, powerful mathematical tools like Laplace and Fourier
transforms do not apply to nonlinear systems. As a result, there are no systematic tools for predicting
the behaviour of nonlinear systems. Instead, there is a rich inventory of powerful analysis tools, each best
applicable to particular classes of nonlinear control problems [125–129].
Perhaps the single most valuable asset to the field of engineering is the simulation tool—constructing a
model of the proposed or actual system and using a numerical solution of the model to reveal the behaviour
of the system. Simulation is the only general method of analysis applicable to finding solutions of linear and
nonlinear differential and difference equations. Of course, simulation finds specific solutions; that is, solu-
tions to the equations with specific inputs, initial conditions, and parametric conditions. It is for this reason
that simulation is not a substitute for other forms of analysis. Important properties such as stability and
conditional stability are not proven with simulations. When the complexity of a system precludes the use of
any analytical approach to establish proof of stability, simulations will be the only way to obtain necessary
information for design purposes. A partial list of the simulation programs available today is contained in
references [151–154].
This chapter also does not provide a magic solution to the analysis problem. In fact, no universal analyti-
cal technique exists that can cater to our demand on analysis of the effects of nonlinearities. Our focus in this
chapter, is only on some important categories of nonlinear systems for which significant analysis (and
design) can be done.
Describing Function Analysis For the so-called separable systems, which comprise a linear part
defined by its transfer function, and a nonlinear part defined by a time-independent relationship between its
input and output variables, the describing function method is most practically useful for analysis. It is
an approximate method but experience with real systems and computer simulation results shows adequate
 506 Digital Control and State Variable Methods

accuracy in many cases. Basically, the method is an approximate extension of frequency response methods
(including Nyquist stability criterion) to nonlinear systems.
In terms of mathematical properties, nonlinearities may be categorized as continuous and discontinuous.
Because discontinuous nonlinearities cannot be locally approximated by linear functions, they are also called
“hard” nonlinearities. Hard nonlinearities (such as saturation, backlash, or coulomb friction) are commonly
found in control systems, both in small range and large range operations. Whether a system in small range
operation should be regarded as nonlinear or linear depends on the magnitude of the hard nonlinearities and
on the extent of their effects on the system performance.
The continuous or so-called “soft” nonlinearities are present in every control system, though not visible
because these are not separable. Throughout the book, we have neglected these nonlinearities in our deriva-
tions of transfer function and state variable models. For example, we have assumed linear restoring force of
a spring, a constant damping coefficient independent of the position of the mass, etc. In practice, none of
these assumptions is true for a large range operation. Also, there are situations, not covered in this book,
wherein the linearity assumption gives too small range of operation to be useful; linear design methods
cannot be applied for such systems. If the controlled systems are not too complex and the performance
requirements are not too stringent, the linearity assumptions made in this book, give satisfactory results in
practice.
Describing function analysis is applicable to separable hard nonlinearities. For this category of nonlinear
systems, as we shall see later in this chapter, the predictions of describing function analysis usually are a
good approximation to actual behaviour when the linear part of the system provides a sufficiently strong
filtering effect. Filtering characteristics of the linear part of a system improve as the order of the system goes
up. The ‘lowpass filtering’ requirement is never completely satisfied; for this reason, the describing function
method is mainly used for stability analysis and is not directly applied to the optimization of system design.
Phase Plane Analysis Another practically useful method for nonlinear system analysis is the phase-
plane method. While phase-plane analysis does not suffer from any approximations and hence can be used
for stability analysis as well as optimization of system design, its main limitation is that it is applicable to
systems which can be well approximated by second-order dynamics. Its basic idea is to solve second-order
differential equation and graphically display the result as a family of system motion trajectories on a two-
dimensional plane, called the phase-plane, which allows us to visually observe the motion patterns of the
system. The method is equally applicable to both hard and soft nonlinearities.
Lyapunov Stability Analysis The most fundamental analysis tool is the concept of a Lyapunov func-
tion and its use in nonlinear stability analysis. The power of the method comes from its generality; it is
applicable to all kinds of control systems; systems with hard or soft nonlinearities, and of second-order or
higher-order. The limitation of the method lies in the fact that it is often difficult to find a Lyapunov function
for a given system.
The aim of this chapter is to introduce the two classical, yet practically important tools—the describing
function method and the phase-plane method—for a class of nonlinear systems. The two methods are com-
plementary to a large extent, each being available for the study of the systems which are most likely to be
beyond the scope of the other. The phase plane analysis applies primarily to systems described by second-
order differential equations. Systems of order higher than the second are likely to be well filtered and
tractable by the describing function method.
The use of Lyapunov functions for stability analysis of nonlinear systems will also be given in this
chapter.
 Nonlinear Systems Analysis 507

9.2 SOME COMMON NONLINEAR SYSTEM BEHAVIOURS

As a minimum, it is important to be aware of the main characteristics of nonlinear behaviour, only to permit
recognition if these are encountered experimentally or in system simulations.
The previous chapters have been predominantly concerned with the study of linear time-invariant control
systems. We have observed that these systems have quite simple properties, such as
· a linear system x& = Ax , with x being the vector of states and A being the system matrix, has a unique
equilibrium point (if A is nonsingular; normally true for feedback system matrices);
· the equilibrium point is stable if all eigenvalues of A have negative real parts, regardless of initial
conditions;
· the transient response is composed of the natural modes of the system, and the general solution can be
solved analytically;
· in the presence of the external input u(t), the system response has a number of interesting properties:
(i) it satisfies the principle of superposition, (ii) the asymptotic stability of the system implies bounded-
input, bounded-output stability, and (iii) a sinusoidal input leads to a sinusoidal output of the same
frequency.
The behaviour of nonlinear systems, however, is much more complex. Due to the lack of superposition
property, nonlinear systems respond to external inputs and initial conditions quite differently from linear
systems. Some common nonlinear system properties are as follows [126]:
(i) Nonlinear systems frequently have more than one equilibrium point. For a linear system, stability is
seen by noting that for any initial condition, the motion of a stable system always converges to the
equilibrium point. However, a nonlinear system may converge to an equilibrium point starting with
one set of the initial conditions, and may go to infinity starting with another set of initial conditions.
This means that the stability of nonlinear systems may depend on initial conditions. In the presence of
a bounded external input, unlike linear systems, stability of nonlinear systems may also be dependent
on the input value.
(ii) Nonlinear systems can display oscillations of fixed amplitude and fixed period without external excitation.
These oscillations are called limit cycles. Consider the well-known Van der Pol’s differential equation
My&& + B ( y 2 − 1) y& + Ky = 0; M > 0, B > 0, K > 0 (9.1)
which describes physical situations in many nonlinear systems. It can be regarded as describing a
mass-spring-damper system with a position-dependent damping coefficient B(y2–1). For large values
of y, the damping coefficient is positive and the damper removes energy from the system. This implies
y(t) that the system motion has a convergent tendency. How-
ever, for small values of y, the damping coefficient is
negative and the damper adds energy into the system.
This suggests that the system motion has a divergent ten-
dency. Therefore, because the nonlinear damping varies
with y, the system motion can neither grow unboundedly
t nor decay to zero. Instead, it displays a sustained oscil-
0
lation independent of initial conditions, as illustrated
in Fig. 9.1.
Of course, sustained oscillations can also be found in
linear systems, e.g., in the case of marginally stable
linear systems. However, the oscillation of a marginally
Fig. 9.1 Responses of the Van der Pol oscillator stable linear system has its amplitude determined by its
 508 Digital Control and State Variable Methods

initial conditions, and such a system is very sensitive to changes in system parameters (a slight change
in parameters is capable of leading either to stable convergence or to instability). In nonlinear systems,
on the other hand, the amplitude of sustained oscillations is independent of the initial conditions, and
limit cycles are not easily affected by parameter changes.
(iii) A nonlinear system with a periodic input may exhibit a periodic output whose frequency is either a
subharmonic or a harmonic of the input frequency. For example, an input of frequency of 10 Hz may
result in an output of 5 Hz for the subharmonic case or 30 Hz for the harmonic case.
(iv) A nonlinear system can display jump resonance, a form of hysteresis, in its frequency response.
Consider a mass-spring-damper system
My&& + By& + K1 y + K 2 y 3 = F cos ω t ; M > 0, B > 0, K1 > 0, K2 > 0 (9.2)
Note that the restoring force of the spring is assumed
|Y( jw)| to be nonlinear. If in an experiment, the frequency w is
B
C varied and the input amplitude F is held constant, fre-
quency-response curve of the form shown in Fig. 9.2,
may be obtained. As the frequency w is increased, the
F response y follows the curve through the points A, B
and C. At point C, a small change in frequency results
in a discontinuous jump to point D. The response then
D follows the curve to point E upon further increase in
A frequency. As the frequency is decreased from point E,
E
the response follows the curve through points D and F.
w At point F, a small change in frequency results in a
discontinuous jump to point B. The response follows
the curve to point A for further decrease in frequency.
Fig. 9.2 Frequency response of a system with
Observe from this description that the response never
jump resonance
actually follows the segment CF. This portion of the
curve represents a condition of unstable equilibrium.

9.3 COMMON NONLINEARITIES IN CONTROL SYSTEMS

In this section, we take a closer look at the nonlinearities found in control systems. Consider the typical
block diagram of closed-loop system shown in Fig. 9.3. It is composed of four parts: a plant to be controlled,
sensors for measurements, actuators for control action, and a control law usually implemented on a compu-
ter. Nonlinearities may occur in any part of the system.

r(t) + y(t)
Controller Actuator Plant
–

Sensor

Fig. 9.3 General diagram of a control system

 Nonlinear Systems Analysis 509
Saturation Saturation is probably the most commonly encountered nonlinearity in control systems. It is
often associated with amplifiers and actuators. In transistor amplifiers, the output varies linearly with the
input, only for small amplitude limits. When the input amplitude gets out of the linear range of the amplifier,
the output changes very little and stays close to its maximum value. Figure 9.4a shows a linear-segmented
approximation of saturation nonlinearity.

Output Output

Input Input

(a) Saturation nonlinearity (b) Deadzone nonlinearity

C B

H –H A
O
H
D
E

(c) Backlash nonlinearity

Force Output Output

Input Input
Velocity

(d) Coulomb friction nonlinearity (e) On-Off (two position/three position) control nonlinearities
Fig. 9.4

Most actuators display saturation characteristics. For example, the output torque of a servo motor
cannot increase infinitely, and tends to saturate due to the properties of the magnetic material. Similarly,
valve-controlled hydraulic actuators are saturated by the maximum flow rate.
Deadzone A deadzone nonlinearity may occur in sensors, amplifiers and actuators. In a dc motor, we
assume that any voltage applied to the armature windings will cause the armature to rotate if the field current
is maintained constant. In reality, due to static friction at the motor shaft, rotation will occur only if the
torque provided by the motor is sufficiently large. This corresponds to a so-called deadzone for small
 510 Digital Control and State Variable Methods

voltage signals. Similar deadzone phenomena occur in valve-controlled pneumatic and hydraulic actuators.
Figure 9.4b shows linear-segmented approximation of deadzone nonlinearity.
Backlash A backlash nonlinearity commonly occurs in mechanical components of control systems. In
gear trains, small gaps exist between a pair of mating gears (refer Fig. 9.4c). As a result, when the driving
gear rotates a smaller angle than the gap H, the driven gear does not move at all, which corresponds to the
deadzone (OA segment in Fig. 9.4c); after contact has been established between the two gears, the driven
gear follows the rotation of the driving gear in a linear fashion (AB segment). When the driving gear rotates
in the reverse direction, by a distance of 2H, the driven gear again does not move, corresponding to the
segment BC in Fig. 9.4c. After the contact between the two gears is re-established, the driven gear linearly
follows the rotation of the driving gear in the reverse direction (CD segment). Therefore, if the driving gear
is in periodic motion, the driven gear will move in the fashion represented by the closed path EBCD.
A critical feature of backlash, a form of hysteresis, is its multi-valued nature. Corresponding to each
input, two output values are possible. Which one of the two occurs depends on the history of the input.
Coulomb Friction In any system where there is a relative motion between contacting surfaces, there are
several types of friction: all of them nonlinear—except the viscous components. Coulomb friction is, in
essence, a drag (reaction) force which opposes motion, but is essentially constant in magnitude, regardless
of velocity (Fig. 9.4d). The common example is an electric motor, in which we find Coulomb friction drag
due to the rubbing contact between the brushes and the commutator.
On-Off Nonlinearity In this book we have primarily covered three modes of control:
(i) proportional control;
(ii) integral control; and
(iii) derivative control.
Another important mode of feedback control is the on-off control. This class of controllers have only two
fixed states rather than a continuous output. In its wider application, the states of an on-off controller may
not, however, be simply on and off but could represent any two values of a control variable. Oscillatory
behaviour is a typical response characteristic of a system under two-position control, also called bang-bang
control. The oscillatory behaviour may be avoided using a three-position control (on-off controller with a
deadzone). Figure 9.4e shows the characteristics of on-off controllers.
The on-off mode of control results in a variable structure system whose structure changes in accordance
with the current value of its state. A variable structure system can be viewed as a system composed of independ-
ent structures, together with a switching logic between each of the structures. With appropriate switching logic,
a variable structure system can exploit the desirable properties of each of the structures the system is composed
of. Even more, a variable structure system may have a property that is not a property of any of its structures. The
variable structure sliding mode control law is usually implemented on a computer. The reader will be exposed
to simple variable structure systems in this chapter; details to follow later in Chapter 10.
We may classify the nonlinearities as inherent and intentional. Inherent nonlinearities naturally come
with the system’s hardware (saturation, deadzone, backlash, Coulomb friction). Usually such nonlinearities
have undesirable effects, and control systems have to properly compensate for them. Intentional
nonlinearities, on the other hand, are artificially introduced by the designer. Nonlinear control laws, such
as bang-bang optimal control laws and adaptive control laws (refer Chapter 10), are typical examples of
intentional nonlinearities.
 Nonlinear Systems Analysis 511

9.4 DESCRIBING FUNCTION FUNDAMENTALS

Of all the analytical methods developed over the years for nonlinear systems, the describing function method
is generally agreed upon as being the most practically useful. It is an approximate method, but experience
with real systems and computer simulation results, shows adequate accuracy in many cases. The method
predicts whether limit cycle oscillations will exist or not, and gives numerical estimates of oscillation fre-
quency and amplitude when limit cycles are predicted. Basically, the method is an approximate extension of
frequency-response methods (including Nyquist stability criterion) to nonlinear systems.
To discuss the basic concept underlying the describing function analysis, let us consider the block dia-
gram of a nonlinear system shown in Fig. 9.5, where the blocks G1(s) and G2(s) represent the linear elements,
while the block N represents the nonlinear element.

r=0 + x y
G1(s) N G2(s)
–

Fig. 9.5 A nonlinear system

The describing function method provides a “linear approximation” to the nonlinear element based on the
assumption that the input to the nonlinear element is a sinusoid of known, constant amplitude. The funda-
mental harmonic of the element’s output is compared with the input sinusoid, to determine the steady-state
amplitude and phase relation. This relation is the describing function for the nonlinear element. The method
can, thus, be viewed as “harmonic linearization” of a nonlinear element.
The describing function method is based on the Fourier series. A review of the Fourier series will be in
order here.

9.4.1 Fourier Series

We begin with the definition of a periodic signal. A signal y(t) is said to be periodic with the period T if
y(t + T ) = y(t) for every value of t. The smallest positive value of T for which y(t + T ) = y(t), is called
fundamental period of y(t). We denote the fundamental period as T0. Obviously, 2T0 is also a period of y(t),
and so is any integer multiple of T0.
A periodic signal y(t) may be represented by the series [31]:
¥
a0
y(t) =
2
+ å [an cos nw0t + bn sin nw0t] (9.3a)
n =1
¥
a0
=
2
+ å Yn sin (nw 0t + f n) (9.3b)
n =1
T0
2
where an =
T0 z
0
y(t) cosnw0t dt; n = 0, 1, 2, ... (9.3c)

T0
2
bn =
T0 z
0
y(t) sin nw0t dt; n = 1, 2, ... (9.3d)
 512 Digital Control and State Variable Methods

Yn = a n2 + bn2 (9.3e)

fn = tan–1
FG a IJ
n
(9.3f )
Hb K
n
In Eqn. (9.3b), the term for n = 1 is called fundamental or first-harmonic, and always has the same frequency
as the repetition rate of the original periodic waveform; whereas n = 2, 3, ..., give second, third, and so forth
harmonic frequencies as integer multiples of the fundamental frequency.
Introducing a change of variable, y = w 0 t, we obtain the following alternative equations for the coeffi-
cients of Fourier series:
2p
1
an =
p
z
0
y(t) cosnw0t d(w0t); n = 0, 1, 2, ... (9.4a)

2p
1
bn =
p
z
0
y(t) sin nw0t d(w0t); n = 1, 2, ... (9.4b)

Certain simplifications are possible when y(t) has a symmetry of one type or another.
(i) Even symmetry: y(t) = y(– t) results in
bn = 0; n = 1, 2, ... (9.4c)
(ii) Odd symmetry: y(t) = –y(–t) results in
an = 0; n = 0, 1, 2, ... (9.4d)
(iii) Odd half-wave symmetry: y(t ± T0/2) = – y(t) results in
an = bn = 0; n = 0, 2, 4, ... (9.4e)

9.4.2 Describing Function for the Nonlinear Element

Let us assume that input x to the nonlinearity in Fig. 9.5 is sinusoidal, i.e.,
x = X sinw t
With such an input, the output y of the nonlinear element will, in general, be a nonsinusoidal periodic
function which may be expressed in terms of Fourier series as follows (refer Eqns (9.3)–(9.4)):
y = Y0 + A1cos w t + B1sin w t + A2 cos 2w t + B2 sin 2w t + L
The nonlinear characteristics listed in the previous section, are all odd-symmetrical/odd half-wave sym-
metrical; the mean value Y0 for all such cases is zero and therefore the output
y = A1cos w t + B1sin w t + A2 cos 2w t + B2 sin 2w t + L
In the absence of an external input (i.e., r = 0 in Fig. 9.5), the output y of the nonlinear element N is
fedback to its input, through the linear elements G2(s) and G1(s) in tandem. If G2(s)G1(s) has lowpass char-
acteristics (this is usually the case in control systems), it can be assumed, to a good degree of approximation,
that all the higher harmonics of y are filtered out in the process, and the input x to the nonlinear element N is
mainly contributed by the fundamental component (first harmonic) of y, i.e., x remains sinusoidal. Under
such conditions, the second and higher harmonics of y can be thrown away for the purpose of analysis, and
the fundamental component of y, i.e.,
y1 = A1 cos w t + B1 sin w t
need only be considered.
The above procedure heuristically linearizes the nonlinearity since, for a sinusoidal input, only a sinusoi-
dal output of the same frequency is now assumed to be produced. This type of linearization, called the first-
harmonic approximation, is valid for large signals as well, so long as the filtering condition is satisfied.
 Nonlinear Systems Analysis 513
We can write y1(t) in the form
y1(t) = A1 sin (w t + 90°) + B1 sin w t = Y1 sin (w t + f1) (9.5a)
where, by using phasors,
2 2
Y1Ðf1 = B1 + jA1 = B1 + A1 Ð tan–1 (A1/B1) (9.5b)
The coefficients A1 and B1 of the Fourier series are given by (refer Eqns 9.3)
2p
1
A1 =
p z0
y cos w t d(w t) (9.5c)

2p
1
B1 =
p z
0
y sinw t d(w t) (9.5d)

As we shall see shortly, the amplitude Y1 and the phase shift f1 are both functions of X, but independent
of w. We may combine the amplitude ratio and the phase shift in a complex equivalent gain N(X), such that
Y (X) B + jA1
N(X) = 1 Ðf 1( X ) = 1 (9.6)
X X
Under first-harmonic approximation, the nonlinear element is completely characterized by the function
N(X); this function is usually referred to as the describing function of the nonlinearity.
The describing function differs from a linear system transfer function, in that its numerical value will vary
with input amplitude X. Also, it does not depend on frequency w (there are, however, a few situations in which
the describing function for the nonlinearity is a function of both, the input amplitude X, and the frequency w
(refer [128–129]). When embedded in an otherwise linear system (Fig. 9.6), the describing function can be
combined with the ‘ordinary’ sinusoidal transfer function of the rest of the system, to obtain the complete open-
loop function; however, we will get a different open-loop function for every different amplitude X. We can
check all of these open-loop functions for closed-loop stability, using Nyquist stability criterion.

+
–

Fig. 9.6 Nonlinear system with nonlinearity replaced by describing function

It is important to remind ourselves here that the simplicity in analysis of nonlinear systems using describ-
ing functions, has been achieved at the cost of certain limitations; the foremost being the assumption that in
traversing the path through the linear parts of the system from nonlinearity output back to nonlinearity input,
the higher harmonics will have been effectively lowpass filtered, relative to the first harmonic. When the
linear part of the system does indeed provide a sufficiently strong filtering effect, then the predictions of
describing function analysis, usually, are a good approximation to actual behaviour. Filtering characteristics
of the linear part of the system improve as the order of the system goes up.
The ‘lowpass filtering’ requirement is never completely satisfied; for this reason, the describing function
method is mainly used for stability analysis and is not directly applied to the optimization of system design.
Usually, the describing function analysis will correctly predict the existence and characteristics of limit
cycles. However, false indications cannot be ruled out; therefore, the results must be verified by simulation.
Simulation, in fact, is an almost indispensable tool for analysis and design of nonlinear systems; describing
function and other analytical methods, provide the background for intelligent planning of the simulations.
 514 Digital Control and State Variable Methods
We will limit our discussion to separable nonlinear systems with reference input r = 0, and with symmetri-
cal nonlinearities (listed in Section 9.3) in the loop. Refer [128–129] for situations wherein dissymmetrical
nonlinearities are present, and/or the reference input is non-zero.

9.5 DESCRIBING FUNCTIONS OF COMMON NONLINEARITIES

Before coming to the stability study by the describing function method, it is worthwhile to derive the
describing functions of some common nonlinearities. Our first example is an on–off controller with a
deadzone as in Fig. 9.7. If X is less than deadzone D, then the controller produces no output; the first har-
monic component of the Fourier series is, of course, zero, and the describing function is also zero. If X > D,
the controller produces the ‘square wave’ output y. One cycle of this periodic function of period 2p is
described as follows:
0 ; 0 £ wt < a
M ; a £ wt < (p – a)
0 ; (p – a) £ wt < (p + a) (9.7)
y=
–M ; (p + a) £ wt < (2p - a)
0 ; (2p - a) £ wt £ 2p

where X sina = D; or a = sin–1(D/X).

y y

+ 2 –
M

–
x 0 t

–M
2 + 3 –
–
x
X

–
+

2 –
2 +

3 –

t
Fig. 9.7 Fourier-series analysis of an on–off controller with deadzone
 Nonlinear Systems Analysis 515
This periodic function has odd symmetry:
y(w t) = – y(–w t)
Therefore, the fundamental component of y is given by (refer Eqn. (9.4d)
y1 = B1sinw t
2p
1
where
p 0
B1 = z
y sinw t d(w t)

Due to the symmetry of y (refer Fig. 9.7), the coefficient B1 can be calculated as follows:
p p
2 2
4 4M 4M
B1 =
p z
0
y sinw t d(w t) =
p z
a
sin w t d (w t ) =
p
cos a (9.8)

Since A1 (the Fourier series cosine coefficient) is zero, the first harmonic component of y is exactly in
phase with X sinwt, and the describing function N(X) is given by (refer Eqns (9.6)–(9.8))
0 ;X<D
N(X) = 2
4M
1 -
F DI ;X³D (9.9)
pX H XK
For a given controller, M and D are fixed and the describing function is a function of input amplitude X,
which is graphed in Fig. 9.8a, together with peak location and value, found by standard calculus
maximization procedure. Note that for a given X, N(X) is just a pure real positive number, and thus, plays the
role of a steady-state gain in a block diagram of the form shown in Fig. 9.6. However, this gain term is
unusual in that it changes when X changes.

N(X)

2M

0 2 X
(a)

X Im
X = 2
Increasing
X
1
A

X B Re
X
Increasing
1
–1 = –
-locus N 2M
N(X)
(b)
Fig. 9.8 Describing function of an on–off controller with deadzone
 516 Digital Control and State Variable Methods
A describing function N(X) may be equivalently represented by a plot of
1 1
- = - Ð ( - 1/ N ( X )) (9.10)
N( X) N( X )
as a function of X on the polar plane. We will use this form of representation in the next section for stability
analysis.
Rearrangement of Eqn. (9.9) gives
1 pD ( X / D) 2
- = - (9.11)
N( X) 4 M ( X / D )2 - 1
Figure 9.8b gives the representation on the polar plane of the describing function, for an on–off controller
with deadzone. It may be noted that though the points A and B lie at the same place on the negative real axis,
they belong to different values of X/D.
We choose as another example the backlash, since its behaviour brings out certain features not encoun-
tered in our earlier example. The characteristics of backlash nonlinearity, and its response to sinusoidal
input, are shown in Fig. 9.9. The output y is again a periodic function of period 2p ; one cycle of this function
is described as follows:
x – H ; 0 £ w t < p /2
X – H ; p /2 £ w t < (p – b )
y= x + H ; (p – b ) £ w t < 3p /2 (9.12)
– X + H ; 3p /2 £ w t < (2p – b )
x – H ; (2p – b ) £ w t £ 2p

F
where X sinb = X – 2 H; or b = sin–1 1 -
2H
.I
H X K

Fig. 9.9 Fourier-series analysis of backlash

 Nonlinear Systems Analysis 517
The periodic function does not possess odd symmetry:
y(w t) ¹ – y(–w t),
but possesses odd half-wave symmetry:
y(w t ± p) = – y(w t)
Therefore, the fundamental component of y is given by (refer Eqn. (9.4e))
y1 = A1cosw t + B1 sinw t
2p
1
where A1 =
p z
0
y cosw t d(w t)

2p
1
B1 =
p z
0
y sinw t d(w t)

Due to the symmetry of y, only the positive half wave need be considered (Fig. 9.9):

LM p
2 (p - b )
2
A1 =
p
MMz ( X sin w t - H) cos w t d(w t) + z
p
( X - H ) cos w t d (w t )
0
N 2
p O
+ z ( X sin w t + H ) cos w t d (w t ) P
(p - b ) PQ
p p
2 2 (p - b )
2X 2H 2( X - H )
=
p z 0
sinq cosq dq –
p z
0
cosq dq +
p p
z cosq dq

p p
2X 2H
+
p z
(p - b )
sinq cosq dq +
p z
(p - b )
cosq dq

3X 2( X - 2 H ) X
=- + sin b + cos 2 b
2p p 2p

3X 2X X X
=- + sin 2 b + cos 2 b = - cos 2 b (9.13a)
2p p 2p p
L p
(p - b )
2 M 2

p M z z
B1 = M a X sin w t - H f sin w t d(w t) + p
( X - H ) sin w t d (w t )
0
N 2

p O
+ z ( X sin w t + H ) sin w t d (w t ) P
(p - b )
PQ
 518 Digital Control and State Variable Methods
p p
2 2 (p - b )
2X 2H 2( X - H )
=
p z0
sin 2q dq -
p z
0
sin q dq +
p z
p
sin q dq

2
p p
2X 2H
+
p z sin 2q d q +
(p - b )
p z sin q dq
(p - b )

=
LM
X p
+ b +
OP
2( X - 2 H )
cos b -
X
sin 2 b
N
p 2 Q p 2p
X Lp O 2 X sin b cos b - X sin 2 b = X LMp + b + 1 sin 2b OP
= + bP + (9.13b)
p MN 2 Q p 2p p N2 2 Q
It is clear that the fundamental component of y will have a phase shift with respect to X sinwt (a feature not
present in our earlier example). The describing function N(X) is given by (refer Eqns (9.6), (9.12), (9.13))

N(X) =
1
(B1 + jA1) =
1 p LM
1
+ b + sin 2 b - j cos2 b
OP (9.14)
X p 2 2N Q
b = sin–1 1 -
FG 2H IJ
H X K
Note that N(X) is a function of the non-dimensional ratio H/X; we can thus tabulate or plot a single graph
of N(X) that will be usable for any numerical value of H (see Table 9.1, and Fig. 9.10).

Table 9.1 Describing function for backlash

H/X |–1/N(X)| Ð(–1/N(X)) Im
0.000 1.000 180.0 – 1.0
0.050 1.017 183.5 Re
0.125 1.066 188.5
0.200 1.134 193.4
0.300 1.259 199.7 H
=0
0.400 1.435 206.0 X
0.500 1.687 212.5
–1
0.600 2.072 219.3
N(X)
0.700 2.720 226.7
0.800 4.024 235.1 H
1
0.850 5.330 239.9 X
0.900 7.946 245.6
0.925 10.560 248.9
0.950 15.800 252.8
0.975 31.500 257.9 Fig. 9.10 Describing function for backlash

We have so far given illustrative derivations of describing functions for on–off controller with deadzone,
and backlash. By similar procedures, the describing functions of other common nonlinearities can be
derived; some of these are tabulated in Table 9.2.
 Nonlinear Systems Analysis 519
Table 9.2 Describing functions of common nonlinearities

Nonlinearity Describing function (input = X sinw t)

M
4M
1. N(X) =
pX
–M

–
R| 0 ; X < D
2
2. N ( X) = S| 4 M 1-
F DI ; X ³ D
T pX H XK
–M

Slope = K

R| K ;X <S
–S
N( X ) = S 2K M
L -1 S S
+ 1-
F S I OP ; X ³ S
2
3. S
|T p MNsin X X H X K PQ

R| 0 ; X <D
– L D O 2
1- F I P ; X ³ D
4. N ( X) = S 2 K Mp -1 D D
|T p MN 2 - sin -
X X H X K PQ
Slope = K

5. –H N(X) =
LM
1 p 1
+ b + sin 2 b - j cos2 b
OP
H p 2N 2 Q
b = sin -1
1-
F2H I
Slope = 1
H X K

9.6 STABILITY ANALYSIS BY THE DESCRIBING FUNCTION METHOD

Consider the linear system of Fig. 9.11a. Application of Nyquist stability criterion1 to this system involves
the following steps.
1
Chapter 8 of reference [155].
520 Digital Control and State Variable Methods
(i) Define the Nyquist contour in the s-plane that encloses the entire right-hand side (unstable region) of
the s-plane (Fig. 9.11b).
(ii) Sketch the Nyquist plot, which is the locus of KG(s)H(s), when s takes on values along the Nyquist
contour (Fig. 9.11c).

R (s ) + Y (s )
KG (s)
–

H (s )

(a)

j Im
j

0 –1 Re
Re j
R

–j KG ( j )H ( j )

(b) (c)
Im Im

1 Re 1 Re
K K

w
G ( j )H ( j )
G ( jw)H ( jw)
(d) (e)
Fig. 9.11 Application of the Nyquist stability criterion

(iii) The characteristic equation of the system is

1 + KG(s)H(s) = 0
or
KG(s)H(s) = –1 (9.15)
The stability of the closed-loop system is determined by investigating the behaviour of the Nyquist
plot of KG(s)H(s) with respect to the critical point (–1 + j0) in the KG(s)H(s)-plane.
 Nonlinear Systems Analysis 521
For the predominant case of systems wherein open-loop transfer function KG(s)H(s) has no poles in
the right-half of s-plane, the Nyquist stability criterion is stated below:
If the Nyquist plot of the open-loop transfer function KG(s)H(s) corresponding to the Nyquist contour
in the s-plane, does not encircle the critical point (–1 + j0), the closed-loop system is stable.
(iv) The characteristic equation (9.15) may be rearranged as follows:
G(s)H(s) = –1/K (9.16)
For the linear system with open-loop transfer function KG(s)H(s), we can count the number of en-
circlements of (–1/K + j0) point if the Nyquist plot of G(s)H(s) is constructed (Fig. 9.11d).
(v) When the Nyquist plot of G(s)H(s) passes through (–1/K + j0) point, the number of encirclements is
indeterminate. This corresponds to the condition where 1 + KG(s)H(s) has zeros on the imaginary axis
(i.e., the closed-loop system has poles on the imaginary axis). The gain corresponding to this situation,
will yield oscillatory behaviour (we have assumed that the zeros are non-repeated).
(vi) The most commonly occurring situation in control system design is that the system becomes unstable
if the gain increases past a certain critical value. Stability condition for such systems, becomes
| G( jw)H( jw) | < 1/K at Ð G( jw)H( j w) = –180º
The stability may, therefore, be examined from polar plot (plot of G( jw) H( jw) on polar plane with w
varying from 0 to ¥) only (Fig. 9.11e).
Consider now a nonlinear system of Fig. 9.12 N(X) is the describing function of the nonlinear element and
G(s) is the transfer function of the linear part of the system. G(s) is assumed to have no poles in the right half
s-plane.
The validity of the block diagram shown in Fig. 9.12 is based on the assumption that the input to the
nonlinearity is a pure sinusoid x = X sinw t. This necessarily requires that r is zero, since non-zero values of
the system input, usually result in the nonlinearity input signal containing components in addition to the
assumed sine wave. So, the describing function approach is applicable when the input r is zero and the
system is excited by some initial conditions. For different values of the initial conditions, a signal of the form
x = X sinw t will be generated at the input of an odd-symmetrical/odd half-wave symmetrical nonlinearity,
with X varying from 0 to ¥. This is true only if the linear part of the system possesses the required lowpass
characteristics. For situations where the nonlinearity input signal contains components in addition to X sinw t
(such as r not being zero), the method of dual-input describing functions may be useful (refer [128–129]).
For a given X, N(X) in Fig. 9.12 is just a real/complex number; the condition (9.16) therefore, becomes
G(s) = – 1/N(X) (9.17)
This modified condition differs from the
r=0 + x condition (9.16), in the fact that the critical
N(X) G(s)
– point (–1/K + j0), now becomes the critical
locus – 1/N(X) as a function of X. The stability
analysis can be carried out by examining the
Fig. 9.12 A nonlinear system relative position of the following plots on polar
plane.
(i) Plot of G( jw) with w varying from 0 to ¥, called the polar plot of G( jw) (note that the Nyquist plot is
the plot of G( jw) with w varying from – ¥ to + ¥).
(ii) Plot of –1/N(X) with X varying from 0 to ¥.
When the critical points of – 1/N(X) lie to the left of the polar plot of G( jw) (or are not encircled by the
Nyquist plot of G( jw)), the closed-loop system is stable; any disturbances which appear in the system will
tend to die out. Conversely, if any part of the –1/N(X) locus lies to the right of the polar plot of G( jw) (or is
enclosed by the Nyquist plot of G( jw)), it implies that any disturbances which are characterized by the
 522 Digital Control and State Variable Methods

values of X corresponding to the enclosed critical points, will provide unstable operations. The intersection
of G( jw) and – 1/N(X) loci, corresponds to the possibility of a periodic oscillation (limit cycle) character-
ized by the value of X on the – 1/N(X) locus, and the value of w on the G( jw) locus.
Figure 9.13a shows a G( jw) plot superimposed on a – 1/N(X) locus. The values of X, for which the
–1/N(X) locus lies in the region to the right of an observer traversing the polar plot of G( jw) in the direction
of increasing w, correspond to unstable conditions. Similarly, the values of X, for which the –1/N(X) locus
lies in the region to the left of an observer traversing the polar plot of G( jw) in the direction of increasing w,
correspond to the stable conditions. The locus of – 1/N(X) and the polar plot of G( jw) intersect at the
point A(w = w2, X = X2), which corresponds to the condition of limit cycle. The system is unstable for
X < X2 and is stable for X > X2. The stability of the limit cycle can be judged by the perturbation technique
described below.
Suppose that the system is originally operating at A under the state of a limit cycle. Assume that a slight
perturbation is given to the system, so that the input to the nonlinear element increases to X3, i.e., the oper-
ating point is shifted to B. Since B is in the range of stable operation, the amplitude of the input to the
nonlinear element progressively decreases, and hence the operating point moves back towards A. Similarly,
a perturbation which decreases the amplitude of input to the nonlinearity, shifts the operating point to C
which lies in the range of unstable operation. The input amplitude now progressively increases and the
operating point again returns to A. Therefore, the system has a stable limit cycle at A.

Fig. 9.13 Prediction and stability of limit cycles

Figure 9.13b shows the case of an unstable limit cycle. For systems having G( jw) plots and –1/N(X) loci
as shown in Figs 9.14a and 9.14b, there are two limit cycles; one stable and the other unstable.
Describing function method usually gives sufficiently accurate information about stability and limit
cycles. This analysis is invariably followed by a simulation study.
The transfer function G(s) in Fig. 9.12, when converted to state variable formulation, takes the form
x& (t) = Ax(t) + bu(t); x(0) =D x0
y(t) = cx(t)
where
x(t) = n ´ 1 state vector for nth order G(s)
u(t) = input to G(s)
y(t) = output of G(s)
 Nonlinear Systems Analysis 523
A = n ´ n matrix
b = n ´ 1 column matrix
c = 1 ´ n row matrix

Im G( j )-plot Im
G( j )-plot

Re Re

Stable
Unstable
Unstable
Stable
X
–1
–1 -locus X
-locus N(X)
N(X)
(a) (b)
Fig. 9.14 Prediction and stability of limit cycles

In Appendix A we use MATLAB Software SIMULINK to obtain response of nonlinear systems of the
form given in Fig. 9.12 with zero reference input and initial state x0.

Example 9.1 Let us investigate the stability of a with system on–off controller, shown in Fig. 9.15. Using
the describing function of an on–off nonlinearity given in Table 9.2, we have
1 pE
- = - (9.18)
N ( E) 4

r=0 1 y
+ e u K
G( s) =
(s + 1)(0.1s + 1)2
– –1

Fig. 9.15 A system with on–off controller

where E is the maximum amplitude of the sinusoidal signal e. Figure 9.16 shows the locus of – 1/N(E) as a
function of E, and the plot of G( jw) for K = 5. Equation (9.17) is satisfied at A since the two graphs
intersect at this point.
The point of intersection on the G( jw) plot gives a numerical value w1 for the frequency of the limit
cycle; whereas, the same point on the – 1/N(E) locus gives us the predicted amplitude E1 of the oscillation.
As an observer traverses the G( jw) plot in the direction of increasing w, the portion O-A of the –1/N(E)
locus lies to its right and the portion A-C lies to its left. Using the arguments presented previously, we can
conclude that the limit cycle is a stable one.
 524 Digital Control and State Variable Methods

Fig. 9.16 Stability analysis of the system in Fig. 9.15

Since – 1/N(E) is a negative real number, it is clear that intersection occurs at – 180° phase angle. The
frequency w1 that gives ÐG( jw1) = – 180° is 10.95 rad/sec. Furthermore, at point A
1
|G( jw1)| = -
N ( E1 )
At w1 = 10.95, |G( jw1)| = 0.206 and, therefore, (refer Eqn. (9.18)).
1 p E1
- = = 0.206
N ( E1 ) 4
This gives E1 = 0.262.
The describing function analysis, thus predicts a limit cycle (sustained oscillation)
y(t) = – e(t) = – 0.262 sin10.95t
Im For K > 5, the intersection point shifts to B (Fig. 9.16)
resulting in a limit cycle of amplitude E2 > E1 and
frequency w2 = w1. It should be observed that the system
E has a limit cycle for all positive values of gain K.
5
To gain some further insight into on–off control
Re
behaviour and describing function analysis, let us
modify the system of Fig. 9.15 by letting the linear por-
–1 tion be of second-order with
N(E)
5
G(s) =
( s + 1)(0.1 s + 1)
Figure 9.17 shows the plot of G( jw) superimposed
G( j ) on the locus of – 1/N(E). The intersection of the two
Fig. 9.17 Describing function analysis of a graphs is now impossible, since the phase angle of
system with second–order plant neither G( jw) nor – 1/N(E), can be more lagging than
and on–off controller –180°. Describing function analysis thus seems to
 Nonlinear Systems Analysis 525
predict no limit cycling, whereas, the fact that the control signal u must be either + 1.0 or –1.0 dictates that
the system oscillate. One possible interpretation to this analysis would be that the second-order linear system
provides less of the lowpass filtering assumed in the describing function method, than did the third-order
system and thus the approximation has become inaccurate, to the point of predicting no limit cycle when
actually one occurs. Another interpretation would be that the curves actually do ‘intersect’ at the origin,
predicting a limit cycle of infinite frequency and infinitesimal amplitude. This latter interpretation, even
though it predicts a physically impossible result, agrees with the rigorous mathematical solution of the
differential equations: for some non-zero initial value of y, we find that y(t) oscillates about zero, with ever-
decreasing amplitude and ever-increasing frequency. We will examine this solution on the phase plane in a
later section.
Let us now modify the system of Fig. 9.15 by giving the controller a deadzone D, as shown in Fig. 9.18.
The –1/N(E) locus for this type of controller is given by Fig. 9.8b. Plots of G( jw) for different values of K,
superimposed on – 1/N(E) locus, are shown in Fig. 9.19. From this figure, we observe that for K = K1, the
G( jw) plot crosses the negative real axis at a point to the right of – p D/2, such that no intersection takes
place between the graphs of G( jw) and –1/N(E), and therefore, no limit cycle results. With such a gain, the
–1/N(E) locus lies entirely to the left of the G( jw) plot; the system is, therefore, stable, i.e., it has effectively
positive damping.

1
e u K y
(s + 1)(0.1s + 1)2
–

Fig. 9.18 A system controlled by on–off Fig. 9.19 Stability analysis of the system of Fig. 9.18
controller with deadzone

If the gain K is now increased to a value K2, such that the G( jw) plot intersects the –1/N(E) locus at the
point A (i.e., on the negative real axis at – p D/2), then there exists a limit cycle. Now, suppose that the
system is operating at the point A. Any increase in the amplitude of E takes the operating point to the left so
that it is not enclosed by the G( jw) plot, which means that the system has positive damping. This reduces E
till the operating point comes back to A. Any decrease in the amplitude of E again takes the operating point
to the left of the G( jw) plot, i.e., the system has positive damping which further reduces E, finally driving the
system to rest. Since random disturbances are always present in any system, the system under discussion
cannot remain at A. Therefore, the limit cycle represented by A is unstable.
When the gain K is further increased to K3, the graphs of G( jw) and –1/N(E) intersect at two points B and
C. By arguments similar to those advanced earlier, it can be shown that the point B represents an unstable
limit cycle and C represents a stable limit cycle. It may be noted that though the points B and C lie at the same
place on the negative real axis, they belong to different values of E/D.
 526 Digital Control and State Variable Methods
It is also clear that limit cycling is predicted only for deadzone D smaller than the value given by
pD
= |G( jw1)|
2
where w1 is the frequency at which the plot G( jw) intersects the negative real axis. A deadzone in on–off
controllers appears to be a desirable feature to avoid limit cycling. However, as we shall see later in this
chapter, a large value of D would cause the steady-state performance of the system to deteriorate.

Example 9.2 Figure 9.20a shows a block diagram for a servo system consisting of an amplifier, a motor,
a gear train, and a load (gear 2 shown in the diagram includes the load element). It is assumed that the inertia
of the gears and load element is negligible compared with that of the motor, and backlash exists between
gear 1 and gear 2. The gear ratio between gear 1 and gear 2 is unity.

e x Gear 1
Amplifier Motor y
e 5 –1
– G(s) =
s(s + 1) 1
–
y
Gear 2
Slope = 1

(a) (b)
Fig. 9.20 A servo system with backlash in gears

The transfer function of the amplifier-motor combination, is given

by 5/s(s + 1) and the backlash amplitude is given as unity (H = 1).
From the problem statement, the block diagram of the system may
be redrawn as shown in Fig. 9.20b. Let us investigate the stability of
this system. The –1/N(X) locus for the backlash nonlinearity is given
by Fig. 9.10 (Table 9.1). Plot of G( jw) superimposed on –1/N(X)
locus is shown in Fig. 9.21. As seen from this figure, there are two
intersections of the two loci. Applying the stability test for the limit
cycle reveals that point A corresponds to a stable limit cycle and point
B corresponds to an unstable limit cycle. The stable limit cycle has a
frequency of 1.6 rad/sec and an amplitude of 2 (the unstable limit
cycle cannot physically occur). To avoid limit-cycle behaviour, the
gain of the amplifier must be decreased sufficiently, so that the entire
G( jw) plot lies to the left of –1/N(X) locus.
Note that checking for an intersection must be done graphically/
numerically, since no analytical solution for limit-cycle amplitude or
frequency is possible. A computer program that tabulates G( jw)
and –1/N(X) is useful in searching for intersections and is not difficult
to write. Once the general region of an intersection is found, we can
Fig. 9.21 Stability analysis of the use smaller increments of H/X and w to pinpoint the intersection, as
system of Fig. 9.20 accurately as we wish.
 Nonlinear Systems Analysis 527

9.7 CONCEPTS OF PHASE PLANE ANALYSIS

The free motion of any second-order nonlinear system can always be described by an equation of the form
y + g ( y, y& ) y& + h( y , y& ) y = 0
&& (9.19)
The state of the system, at any moment, can be represented by a point of coordinates ( y , y& ) in a system of
rectangular coordinates. Such a coordinate plane is called a ‘phase plane’.
In terms of the state variables
x1 = y, x2 = y& , (9.20a)
Second-order system (9.19) is equivalent to the following canonical set of state equations:
dx
x&1 = 1 = x2
dt
(9.20b)
dx2
x& 2 = = − g ( x1 , x2 ) x2 − h( x1 , x2 ) x1
dt
By division, we obtain a first-order differential equation relating the variables x1 and x2:
dx2 g ( x1 , x2 ) x2 + h( x1 , x2 ) x1
=− (9.21)
dx1 x2
Thus, we have eliminated the independent variable t from the set of first-order differential equations given
by (9.20b). In Eqn. (9.21), we consider x1 and x2 as independent and dependent variables, respectively.
For a given set of initial conditions {x1(0),x2(0)}, the solution to Eqn. (9.21) may be represented by a
single curve in the phase plane, for which the coordinates are x1 and x2. The curve traced out by the state
point {x1(t), x2(t)}, as time t is varied from 0 to ¥, is called the phase trajectory, and the family of all possible
curves for different initial conditions is called the phase portrait. Normally, a finite number of trajectories,
defined in a finite region, is considered a portrait.
One may obviously raise the question that when time solutions x1(t) and x2(t), as time t is varied from 0 to
¥, may be obtained by direct integration of Eqns (9.20b) analytically or numerically, where is the necessity
of drawing phase portraits? In fact, as we shall see, the phase portraits provide a powerful qualitative aid for
investigating system behaviour and the design of system parameters, to achieve a desired response. Further-
more, the existence of limit cycles is sharply brought into focus by the phase portrait.
Figure 9.22a shows the output response, and the corresponding phase trajectory, for a linear second-order
servo system described by the differential equation
y + 2z y& + y = 0; y(0) = y0, y& (0) = 0, 0 < z < 1
&&
In terms of the state variables x1 = y and x2 = y& , the system model is given by the equations
x& 1 = x2; x& 2 = – 2z x2 – x1; x1(0) = y0, x2(0) = 0
The origin of the phase plane (x1 = 0, x2 = 0) is the equilibrium point of the system since, at this point, the
derivatives x& 1 and x& 2 are zero (the system continues to lie at the equilibrium point unless otherwise dis-
turbed). The nature of the transient can be readily inferred from the phase trajectory of Fig. 9.22; starting
from the point P, i.e., with initial deviation but no initial velocity, the system returns to rest, i.e., to the origin,
with damped oscillatory behaviour.
Consider now the well-known Van der Pol’s differential equation (refer Eqn. (9.1))
y – m(1 – y2) y& + y = 0
&&
 528 Digital Control and State Variable Methods
which describes physical situations in many nonlinear systems. It terms of the state variables x1 = y and x2 =
y& , we obtain
x& 1 = x2; x& 2 = m(1 – x12)x2 – x1
y

x2 = y
y0
t2

t3
t2 t=0
0
t1 t3 t4 t t4 P x1 = y

t1

Fig. 9.22 A second-order linear system on the phase plane

Origin of the phase plane is the equilibrium point of the system. Figure 9.23 shows phase portraits for
(i) m > 0, (ii) m < 0. In the case of m > 0, we observe that for large values of x1(0), the system response is
damped and the amplitude of x1(t) = y(t) decreases till the system state enters the limit cycle, as shown by the
outer trajectory. On the other hand, if initially x1(0) is small, the damping is negative, hence the amplitude of
x1(t) = y(t) increases till the system state enters the limit cycle, as shown by the inner trajectory. The limit
cycle is a stable one, since the paths in its neighbourhood converge toward the limit cycle. Figure 9.23 shows
an unstable limit cycle for m < 0.
The phase plane for second-order systems is indeed a special case of phase space, or state space, defined
for nth-order systems. Much work has been done to extend this approach of analysis to third-order systems.
Though a phase trajectory for a third-order system can be graphically visualized through its projections on
two planes, say (x1, x2) and (x2, x3) planes, this complexity causes the technique to lose its major power of
quick graphical visualization of the total system response. The phase trajectories are, therefore, generally
restricted to second-order systems only.

x2 = y x2 = y

Stable
limit
cycle

x1 = y x1 = y

Unstable
limit
cycle

(i) >0 (ii) <0

Fig. 9.23 A second-order nonlinear system on the phase plane
 Nonlinear Systems Analysis 529
For time-invariant systems, the entire phase plane is covered with trajectories with one, and only one,
curve passing through each point of the plane, except for certain critical points through which, either
infinite number or none of the trajectories pass. Such points (called singular points) are discussed later
in Section 9.9.
If the parameters of a system vary with time, or if a time-varying driving function is imposed, two or more
trajectories may pass through a single point in a phase plane. In such cases, the phase portrait becomes
complex and more difficult to work with and interpret. Therefore, the use of phase-plane analysis is
restricted to second-order systems with constant parameters and constant or zero input. However, it may be
mentioned that investigators have made fruitful use of the phase-plane method in investigating second-order
time-invariant systems under simple time-varying inputs, such as ramp. Some simple time-varying systems
have also been analysed by this method. Our discussion will be limited to second-order time-invariant
systems with constant or zero input.
From the above discussion, we observe that the phase-plane analysis applies primarily to systems
described by second-order differential equations. In the case of feedback control systems, systems of order
higher than the second, are likely to be well filtered and tractable by the describing-function method
discussed earlier in this chapter. The two methods of the phase plane and of the describing function are
therefore complementary to a large extent, each being available for the study of the systems which are most
likely to be beyond the scope of the other.

9.8 CONSTRUCTION OF PHASE PORTRAITS

Today, phase portraits are routinely computer-generated. However, of course (as, for example, in the case of
root locus for linear systems), it is still practically useful to learn how to roughly sketch phase portraits or
quickly verify the plausibility of computer outputs.
For some special nonlinear systems, particularly piecewise linear systems (whose phase portraits can be
constructed by piecing together the phase portraits of the related linear systems), phase portraits can be
constructed analytically. Analytical methods are useful for systems modelled by differential equations that
can be easily solved. If the system of differential equations cannot be solved analytically, we can use graphi-
cal methods. A number of graphical methods for constructing phase-plane trajectories are now available; we
will describe in this section, the method of isoclines.

9.8.1 Analytical Method

Most nonlinear systems cannot be easily solved by analytical techniques. However for piecewise linear
systems, an important class of nonlinear systems, this method can be conveniently used, as the following
examples show.

Example 9.3 In this example, we consider a model of a satellite shown in Fig. 7.3. We assume that the
satellite is rigid and is in a frictionless environment. It can rotate about the reference axis as a result of torque
T applied to the satellite about its mass centre by firing the thrusters (T = Fd ). The system input is the applied
torque T and the system output is the attitude angle q. The satellite’s moment of inertia is J. The input-outout
model of the system is
d 2θ
J =T (9.22a)
dt 2
 530 Digital Control and State Variable Methods
We assume that when the thrusters fire, the thrust is constant; that is, T = A, a constant greater than or less
than zero. In terms of output variable y (= q ), we obtain the equation
y =A
J && (9.22b)
In terms of the state variables x1 = y and x2 = y& , the state equations become
A
x& 1 = x& 2; x& 2 = (9.22c)
J
Elimination of t by division, yields the equation of the trajectories:
dx2 A
= (9.23)
dx1 Jx 2
or J x2 dx2 = A dx1
This equation is easily integrated; the general solution is
Jx 22 ( t )
x1(t) = +C (9.24a)
2A
where C is a constant of integration and is determined by initial conditions, i.e.,
Jx 2 (0)
C = x1(0) – 2 (9.24b)
2A
For an initial state point (x1(0), x2(0)), the trajectory is a parabola passing through the point x1 = C on the
x1-axis where C is defined by Eqn. (9.24b).
A family of parabolas in the (x1, x2) plane is shown in Fig. 9.24a for A > 0. As time t increases, each
trajectory is described in the clockwise direction, as indicated by the arrows. The direction of the phase
trajectories is dictated by the relationship x&1 = x2; x1 increases with time in the upper half of the phase plane
and the state point, therefore, moves from left to right (®); in the lower half of the phase plane, x1 decreases
with time and the state point must, therefore, move from right to left (¬).
The time interval between two points of a trajectory is given by Dt = Dx1/x2av. The trajectories may be
provided with a time scale by means of this equation. This operation is, however, often unnecessary since the
phase portrait is mainly used to display the general features of the system transients.
The phase portrait for A < 0 is shown in Fig. 9.24b. In the special case of A = 0 (no driving torque), the
integration of the trajectory equation (9.23) gives x2(t) = x2(0). The trajectories are, therefore, straight lines
parallel to x1-axis.
x2 = y
x2av x2

x1 x1 = y
x1
t

C<0 C=0 C>0

(b) A < 0
(a) A > 0
Fig. 9.24 Phase potraits for system (9.22)
 Nonlinear Systems Analysis 531

Example 9.4 Consider now the equation

Jθ&& + Bθ& = T (9.25a)
corresponding to a torque T driving a load comprising inertia J and viscous friction B. For a constant torque,
the equation may be expressed as
τ &&
y + y& = A (9.25b)
where y = q is the system output and A represents normalized torque; a constant greater than or less than zero.
The equivalent system is
x& 1 = x2; t x& 2 = A – x2 (9.25c)
Let us take a new variable z such that
A – x2 = z; dx2 = – dz
Eliminating the time variable by division, we obtain
1 dx1 A − z A
= − =1−
τ dz z z
This first-order equation is readily integrated.
1
x = z – A ln z + C
τ 1
1
or x1(t) = A – x2(t) – A ln(A – x2(t)) + C (9.26a)
τ
where the constant of integration C is determined by the initial conditions, i.e.,
1
C= x1(0) – A + x2(0) + A ln(A – x2(0)) (9.26b)
τ
Therefore, the trajectory equation becomes
1
(x1 – x1(0)) = – (x2 – x2(0)) – A ln
FG
A - x2 IJ (9.26c)
t H
A - x 2 ( 0) K
The phase portrait is shown in Fig. 9.25a for A > 0.
For the case of initial state point at the origin (x1(0) = x2(0) = 0), Eqn. (9.26c) reads
1
x1 = – x2 – A ln
F A - x2 I (9.26d)
t H A K
The phase trajectory described by this equation is shown in Fig. 9.25a as the curve G0. It is seen that the
trajectory is asymptotic to the line x2 = A, which is the final velocity.
For an initial state point (x1(0), x2(0)), the trajectory will have the same shape as the curve G0, except that
it is shifted horizontally—so that it passes through the point (x1(0), x2(0)). This is obvious from Eqn. (9.26c)
which can be written as
1
(x1 – K) = – x2 – A ln
FA - x2 I
t H A K
where K = x1(0) + t x2(0) + t A ln
F A - x ( 0) I
2
H A K
 532 Digital Control and State Variable Methods
x2 = y
A x2

x1 = y

x1

K<0 K=0 K>0 A

(a) A > 0 (b) A < 0

Fig. 9.25 Phase portraits for system (9.25)

For an initial state point (x1(0), x2(0)), the trajectory is G0, shifted horizontally by K units. The phase
portrait for A < 0 is shown in Fig. 9.25b. In the special case of A = 0, the phase portrait consists of a family
of straight lines of slope –1/t.

9.8.2 The Method of Isoclines

Consider a time-invariant second-order system described by equations of the form (refer Eqns (9.20b))
x&1 = x2
(9.27)
x& 2 = f ( x1 , x2 )
The equation of the trajectories is
dx2 f ( x1 , x 2 )
= (9.28)
dx1 x2
*
At a point ( x1* , x*2 ) in the phase plane, the slope m of the tangent to the trajectory can be determined
from
f ( x1* , x2* )
= m* (9.29)
x2*
An isocline is defined to be the locus of the points corresponding to a given constant slope m of the
trajectories, on the phase plane. All trajectories passing through the points on the curve
f (x1, x2 ) = mx2 (9.30)
will have the same tangent slope m at the points on the curve; the curve, thus, represents an isocline corre-
sponding to trajectories of slope m. All trajectories crossing this isocline will have tangent slope m at the
points on the isocline.
The idea of the method of isoclines is to construct several isoclines and a field of local tangents m. Then,
the trajectory passing through any given point in the phase plane, is obtained by drawing a continuous curve
following the directions of the field.
 Nonlinear Systems Analysis 533
Consider the Van der Pol equation (refer Eqn. (9.1))

y + μ ( y 2 − 1) y& + y = 0
&& (9.31)
With x1 = y and and x2 = y& , the equation of the trajectories becomes
dx2 − μ ( x12 − 1) x2 − x1
=
dx1 x2
Therefore, the points on the curve

− μ ( x12 − 1) x2 − x1
=m
x2
all have the same slope m. The isocline equation becomes
x1
x2 =
( μ − μ x12 ) − m

By taking m of different values, different isoclines can be obtained. Short line segments are drawn on
the isoclines to generate a field of tangent directions. A trajectory starting at any point can be constructed
by drawing short lines from one isocline to another at average slope corresponding to the two adjoining
isoclines, as shown in Fig. 9.26.
Of course, the construction is much simpler if isoclines are straight lines.

x2

Initial point
3 = tan–1m

2
m=0
m = –1
–3 –2 1 2 3
x1
Trajectory

Isocline
m=1
Field of
tangent –2
directions
–3

m = –1 m=0

Fig. 9.26 Phase portrait of the Van der Pol equation

 534 Digital Control and State Variable Methods

Example 9.5 The satellite in Example 9.3, is now placed in a feedback configuration in order to maintain
the attitude q at 0°. This feedback control system, called an attitude control system, is shown in Fig. 9.27.
When q is other than 0°, the appropriate thruster will fire to force q towards 0°. When q = x1 is greater than
0°, u (torque T) = –U, and the trajectories of Fig. 9.24b (corresponding to A < 0) apply. When q = x1 is less
than 0°, u (torque T ) = U, and the trajectories of Fig. 9.24a (corresponding to A > 0) apply. Note that the
switching of u occurs at x1 = 0. Thus the line x1 = 0 (the x2-axis) is called the switching line. From this
discussion we see that Fig. 9.28a illustrates a typical trajectory for the system corresponding to the initial
condition (x10, x20). The system response is thus a periodic motion. Figure 9.28b shows many closed curves for
different initial conditions.
Satellite

U
u(t) 1 1 =y
s x2 s x1
– –U

Fig. 9.27 A satellite attitude control system

x2 x2

Each curve
(x01 , x02 ) parabolic

x1 x1

Switching
line

(a) (b)
Fig. 9.28 Typical trajectories for the system of Fig. 9.27

By controlling the switching line in the phase plane, we can control the performance of the attitude
control system. This simple control strategy leads to a robust nonlinear control structure: the variable
structure sliding mode control. The details will follow later in this chapter.
In the following, we obtain the phase portrait of the closed-loop system of Fig. 9.27 using the method of
isoclines. The purpose here is to illustrate the method of isoclines.
The state equations are
x&1 = x2
x&2 = −U sgn x1
 Nonlinear Systems Analysis 535

⎡ 1; x1 > 0
where sgn x1 = ⎢
⎣ −1; x1 < 0
dx2 −U sgn x1
Then m= =
dx1 x2
Suppose that U is normalized to a value of unity for convenience. Then
1
x2 = − sgn x1
m
For x1 > 0, sgn x1 = 1, and the isocline equation is
1
x2 = − ; x1 > 0
m
For x1 < 0, sgn x1 = –1, and the isocline equation is
1
x2 =
; x1 < 0
m
Given in Fig. 9.29, is the phase plane showing the isoclines and a typical phase trajectory. Note the
parabolic shape, as was determined analytically earlier in this example.

x2
m=1 m=– 1
2 2
2
(x10 , x20 )
m=1 m=–1
1

m= m=
0
x1
m=–1 –1 m=1

1 1
m =– –2 m=
2 2

Trajectory
Isocline
Fig. 9.29 The isoclines and a typical trajectory for the system of Fig. 9.27

9.9 SYSTEM ANALYSIS ON THE PHASE PLANE

In the phase plane analysis of nonlinear systems, two points should be kept in mind:
· Phase plane analysis of nonlinear systems is related to that of linear systems because the local
behaviour of a nonlinear system can be approximated by a linear system behaviour.
· Yet, nonlinear systems can display much more complicated patterns on the phase plane, such as multiple
equilibrium points, and limit cycles.
 536 Digital Control and State Variable Methods
Consider a time-invariant second-order system described by Eqns of the form (refer Eqns (9.27))
x&1 = x2 ; x&2 = f ( x1 , x2 ) (9.32)
Elimination of independent variable t gives the equation of the trajectories of phase plane (refer Eqn. (9.28)):
dx 2 f ( x1 , x 2 ) (9.33)
=
dx1 x2
In this equation, x1and x2 are independent and dependent variables, respectively. Integration of the
equation, analytically, graphically or numerically, for various initial conditions, yields a family of phase
trajectories which displays the general features of the system transients.

9.9.1 Singular Points

Every point (x1, x2) of the phase plane has associated with it, the slope of the trajectory which passes through
that point. The slope m at the point (x1, x2) is given by the equation
f ( x1 , x2 )
m = dx2 =
dx1 x2
With the function f (x1, x2) assumed to be single valued, there is usually a definite value for this slope at
any given point in phase plane. This implies that the phase trajectories will not intersect. The only exceptions
are the singular points at which the trajectory slope is indeterminate:
dx2 0 f ( x1 , x2 )
= = (9.34a)
dx1 0 x2
Many trajectories may intersect at such points. This indeterminacy of the slope accounts for the adjective
‘singular’.
Singular points are very important features on the phase plane. Examination of the singular points can
reveal a great deal of information about the properties of a system. In fact, the stability of linear systems is
uniquely characterized by the nature of their singular points. For nonlinear systems, besides singular points,
there may be more complex features such as limit cycles.
We need to know the following:
(i) Where will the singular points be and how many will be there?
(ii) What is the behaviour of trajectories (i.e., the system) in the vicinity of a singular point?
The first question is answered by our definition of the singular point. There will be singular points at all
the points of the phase plane, for which the slope of the trajectory is undefined. These points are given by the
solution of the equations
x2 = 0; f (x1, x2) = 0 (9.34b)
Singular points of the nonlinear system (9.32), thus, lie on the x1-axis of the phase plane.
Since at singular points on the phase plane, x&1 = x& 2 = 0 , these points, in fact, correspond to the equilib-
rium states of the nonlinear system. We know a nonlinear system often has multiple equilibrium states.
To determine the behaviour of the trajectories in the vicinity of a singular point (equilibrium state of the
nonlinear system), we first linearize the nonlinear equations at the singular point, and then determine the
nature of phase trajectories around the singular point by linear system analysis. If the singular point of
interest is not at the origin, by defining the difference between the original state and the singular point as a
new set of state variables, one can always shift the singular point to the origin. Therefore, without loss of
generality, we can simply consider Eqns (9.32) with a singular point at 0. Using Taylor series expansion,
Eqns (9.32) can be rewritten as
x& = x2
x& 2 = ax1 + bx2 + g2 (x1, x2)
where g 2 (g ) contains higher-order terms.
 Nonlinear Systems Analysis 537
In the vicinity of the origin, the higher-order terms can be neglected and, therefore, the nonlinear system
trajectories essentially satisfy the linearized equations
x& 1 = x2
x& 2 = ax1 + bx2
Transforming these equations into a scalar second-order equation, we get
x1 = ax1 + bx&1
&&
Therefore, we will simply consider the second-order linear system described by
y + 2ζωn y& + ωn2 y = 0
&& (9.35a)
The characteristic roots of this equation are assumed to be l1 and l2:
y + 2ζω n y& + ω n2 y = (s − λ1 ) (s − λ2 ) = 0
&& (9.35b)
The corresponding canonical state model is
x& 1 = x2; x& 2 = –2zwn x2 – w2n x1 (9.35c)
and the differential equation of the trajectories is
dx 2 - 2zw n x 2 - w 2n x1
= (9.35d)
dx1 x2
By inspection of this equation, it is easily seen that at x1 = x2 = 0, the slope dx2 /dx1 is indeterminate:
dx 2 0
=
dx1 0
In the following, we discuss the behaviour of the trajectories in the vicinity of this point with undefined
slope (the singular point).
According to the values of l1 and l2, one is led to distinguish between the six types of singular points
shown in Fig. 9.30. Let us examine each of these cases in detail.
Stable System with Complex Roots
l1 = – a + jb , l2 = – a – jb ; a > 0, b > 0
The response y(t) = C1e–a t sin(b t + C2) (9.36)
x2 = y x2 = y x2 = y

x1 = y
x1 = y x1 = y

(a) Stable focus (b) Unstable focus (c) Centre

x2 = y x2 = y x2 = y

x1 = y
x1 = y x1 = y

x2 = 2 x1 x2 = 1x1 x2 = 2 x1
x2 = 1x1
(d) Stable node (e) Unstable node (f) Saddle point
Fig. 9.30 Phase portraits for system (9.35)
 538 Digital Control and State Variable Methods
where the constants C1 and C2 are determined by the initial conditions.
Using Eqn. (9.36), we can construct a phase portrait on the (x1, x2)-plane with x1 = y and x2 = y& . A typical
phase trajectory is shown in Fig. 9.30a which is a logarithmic spiral into the singular point. This type of
singular point is called a stable focus.
Unstable System with Complex Roots
l1 = a + jb , l2 = a – jb ; a > 0, b > 0
The response y(t) = C1ea t sin(b t + C2) (9.37)
The transient is an exponentially increasing sinusoid; the phase trajectory on the (x1 = y, x2 = y& )-plane is
a logarithmic spiral expanding out of the singular point (Fig. 9.30b). This type of singular point is called an
unstable focus.
Marginally Stable System with Complex Roots
l1 = jb , l2 = – jb; b > 0
The response y(t) = C1 sin(b t + C2) (9.38)
The phase trajectories are closed curves (elliptical), concentric with the singular point (Fig. 9.30c). This
type of singular point is called a centre, or a vortex.
Stable System with Real Roots Assume that l1 and l2 are two real, distinct roots in the left half of the
s-plane; l1 is the root with the smaller modulus. The response
y(t) = C1 e l 1t + C2 e l 2t (9.39a)
It is an overdamped system. The phase portrait in the vicinity of the singular point on the (x1 = y, x2 = y& )-
plane is shown in Fig. 9.30d. Such a singular point is called a stable node.
The phase portrait has two straightline trajectories, defined by the equations
x2(t) = l1 x1(t); x2(t) = l2 x1(t) (9.39b)
It can easily be verified that these trajectories satisfy the differential equation of the given system.
The transient term e l 2t decays faster than the term e l 1t . Therefore, as t increases indefinitely, x1 ®
C1 e l 1t ® 0, and x2 ® l1C1 e l 1t ® 0, so that all the trajectories are tangential at the origin to the straightline
trajectory x2(t) = l1x1(t). The other straightline trajectory, x2(t) = l2 x1(t), is described only if the initial
conditions are such that x2(0) = l2 x1(0).
For stable systems with repeated real roots, the two straightline trajectories coalesce into a single trajec-
tory, again with the slope determined by the root value.
Unstable System with Positive Real Roots Assume that l1 and l2 are two real distinct roots in the
right half of the s-plane; l1 is the smaller root. The phase portrait in the vicinity of the singular point on the
(x1 = y, x2 = y& )-plane is shown in Fig. 9.30e. Such a singular point is called an unstable node.
All trajectories emerge from the singular point and go to infinity. The trajectories are tangential at the origin
to the straightline trajectory, x2(t) = l1x1(t).
Unstable System with One Negative Real Root and One Positive Real Root The phase portrait in
the vicinity of the singular point on the (x1 = y, x2 = y& )-plane is shown in Fig. 9.30f. Such a singular point is
called a saddle.
There are two straightline trajectories with slopes defined by the root values. The straightline due to the
negative root, provides a trajectory that enters the singular point, while the straightline trajectory due to the
positive root, leaves the singular point. All other trajectories approach the singular point adjacent to the
incoming straightline, then curve away and leave the vicinity of the singular point, eventually approaching
the second straightline asymptotically.
 Nonlinear Systems Analysis 539

Example 9.6 Consider the nonlinear system shown in Fig. 9.31. The nonlinear element is an on–off
controller with deadzone whose characteristics are shown in the figure.

1
r = const + e –1 u 1 y
1 s(s + 1)
– –1

Fig. 9.31 A system controlled by on–off controller with deadzone

The differential equation governing the dynamics of the system is given by

&&
y + y& = u; or e + e& = – f (e)
&& (9.40)
where e = r – y; r is constant, and
R|
+1 ; e > 1
f(e) = S0; -1< e <1
|T
-1 ; e < - 1
Choosing the state variables x1 = e and x2 = e& , we obtain the following first-order equations:
x& 1 = x2; x& 2 = – x2 – f(x1)
These equations are same as Eqns (9.25c) with t = 1 and A = – f (x1).
The phase plane may be divided into three regions:
(i) Region I (defined by x1 > 1): The trajectories in this region are given by the equation (refer Eqn. (9.26c):
t = 1, A = – 1)

x1 – x1(0) = – (x2 – x2(0)) + ln

FG 1 + x IJ 2
(9.41a)
H 1 + x (0) K
2
The trajectories are asymptotic to the ordinate –1.
(ii) Region II (defined by –1 < x1 < 1): The trajectories in this region are given by the equation (refer
Eqn. (9.26c): t = 1, A = 0)
x1 – x1(0) = – (x2 – x2(0)) (9.41b)
The trajectories are straightlines of slope –1.
(iii) Region III (defined by x1 < –1): The trajectories in this region are given by the equation (refer
Eqn. (9.26c): t = 1, A = 1)

x1 – x1(0) = – (x2 – x2(0)) – ln

FG 1 - x IJ 2
(9.41c)
H 1 - x (0) K
2
The trajectories are asymptotic to the ordinate + 1.
For a step input r = 3 and zero initial conditions, the initial point of the phase trajectory is located at P in
Fig. 9.32. The figure also shows a phase trajectory, constructed using Eqns (9.41).
It is important to note that a small deadzone region is not always undesirable in on–off controllers. Let us
investigate the behaviour of the system of Fig. 9.31 using on–off with (no deadzone) as a controller. For such
a controller, the width of region II (corresponding to deadzone) in the phase plane, reduces to zero. The
phase trajectory of such a system with r = 3 is shown in Fig. 9.33 : e(t) oscillates about the origin, with ever-
decreasing amplitude and ever-increasing frequency.
 540 Digital Control and State Variable Methods

Comparison of Figs 9.32 and 9.33 reveals that deadzone in on–off controller characteristic helps to
reduce system oscillations, thereby reducing settling time. However, the on–off controller with deadzone
drives the system to a point within the deadzone width. A large deadzone would of course cause the steady-
state performance of the system to deteriorate.
x2

Region III Region II Region I

1 x2
1
–1 1 2 3
P x1
1 2 3
–1
P x2

Deadzone –1

Fig. 9.32 A typical trajectory for the system Fig. 9.33 Phase trajectory for the system in
in Fig. 9.31 Fig. 9.31 when the deadzone is absent

Example 9.7 Let us investigate the performance of a second-order position control system with Coulomb
friction. Figure 9.34 is a model for a motor position servo with Coulomb friction on the motor shaft. The
dynamics of the system is described by the following differential equation:
K e – Tc sgn( y& ) = J &&
y + B y&
where Tc is the Coulomb frictional torque.
For constant input r, y& = – e& and &&
y = – e&& . Therefore
J &&
e + B e& + Tc sgn( e& ) + Ke = 0

J T K
or e&& + e& + c sgn( e&) + e =0 (9.42)
B B B
Letting J/B = t, we get
Tc K
t e&& + e& + sgn( e&) + e =0 (9.43)
B B
In terms of state variables
x1 = e; x2 = e&
we get the following description of the system:
K T
x& 1 = x2; t x& 2 = - x1 - x 2 - c sgn( x2 ) (9.44)
B B
 Nonlinear Systems Analysis 541

r = const + e + 1 y 1 y
K
Js + B s
– –

Tc

Fig. 9.34 A motor position servo with Coulomb friction on the motor shaft

The singular points are given by the solution of the equations (refer Eqns (9.34b))
K T
0 = x2; 0 = - x1 - x 2 - c sgn( x2 )
B B
The solution gives
T
x1 = – c sgn(x2)
K
Thus, there are two singular points. Their location can be interpreted physically—they are at a value of
e = x1, such that Ke = Tc , i.e., the drive torque is exactly equal to the Coulomb-friction torque. We note that
both the singular points are on the x1-axis (x2 º 0), and that the singular point given by x1 = Tc /K is related to
the lower-half phase plane (x2 negative), and the singular point given by x1 = – Tc /K is related to the upper-
half phase plane (x2 positive).
Let us now investigate the stability of the singular points. For e& > 0, Eqn. (9.43) may be expressed as
d2 FG e + T IJ + d FG e + T IJ + K FG e + T IJ = 0
c c c
t (9.45)
dt 2 H K K dt H K K B H K K
This is a linear second-order system with the singular point at (– Tc /K, 0) on the (e, e& )-plane. The character-
istic equation of this system is given by
1 K
l2 + l + =0
t tB
Let us assume the following parameter values for the system under consideration:
(K/B) = 5, t = 4 (9.46)
With these parameters, the roots of the characteristic equation are complex-conjugate with negative real
parts; the singular point is, therefore, a stable focus (refer Fig. 9.30a).
Let us now investigate the system behaviour when large inputs are applied. Phase trajectories may be
obtained by solving the following second-order differential equations for given initial state points (refer
Eqns (9.35b)–(9.36)).
Region I (defined by x2 > 0) :
T
z + z& + 5 z = 0; z = x1 + c ; x 2 = x&1
4 &&
K
Region II (defined by x2 < 0) :
T
z + z& + 5 z = 0; z = x1 – c ; x 2 = x&1
4 &&
K
 542 Digital Control and State Variable Methods
Figure 9.35 shows a few phase trajectories. It is observed that for small as well as large inputs, the
resulting trajectories terminate on a line along the x1-axis from – Tc /K to +Tc /K, i.e., the line joining the
singular points. Therefore, the system with Coulomb friction is stable; however, there is a possibility of large
steady-state error.
x2

Tc /K
x1
– Tc /K

Fig. 9.35 Phase portrait for the system in Fig. 9.34

9.9.2 Limit Cycles

In the phase portrait of the nonlinear Van der Pol equation, shown in Fig. 9.26 , one observes that there is a
closed-curve in the phase portrait. Trajectories inside the curve—and those outside the curve—all tend to
this curve, while a motion started on this curve will stay on it forever. This curve is an instance of the
so-called “limit cycle” phenomenon. Limit cycles are unique features of nonlinear systems.
On the phase plane, a limit cycle is defined as an isolated closed curve. The trajectory has to be both
closed, indicating the periodic nature of motion, and isolated, indicating the limiting nature of the cycle
(with neighboring trajectories converging to or diverging from it). Thus, while there are many closed curves
in the satellite system in Example 9.5, these are not limit cycles because they are not isolated.
A limit cycle is stable if all trajectories in the vicinity of the limit cycle converge to it as t ® ¥ (Fig. 9.23a).
A limit cycle is unstable if all trajectories in the vicinity of the limit cycle diverge from it as t ® ¥ (Fig. 9.23b)

9.10 SIMPLE VARIABLE STRUCTURE SYSTEMS

The purpose of this section is to informally introduce the reader to variable structure sliding mode control
systems. Formal introduction to sliding mode control will appear later in Section 10.5.
A variable structure system is a dynamical system, whose structure changes in accordance with the cur-
rent value of its state. A variable structure system can be viewed as a system composed of independent
structures, together with a switching logic between each of the structures. With appropriate switching logic,
a variable structure system can exploit the desirable properties of each of the structures the system is com-
posed of. Even more, a variable structure system may have a property that is not a property of any of its
structures. We illustrate the above ideas with two numerical examples.
 Nonlinear Systems Analysis 543

Example 9.8 We consider a double integrator model

e = −u
&& (9.47)
having two structures corresponding to u = – 1 and u = + 1 (refer Fig. 9.36).

1
r = const e Switching (e, e) u y
+ 1
function
s2
– –1

Fig. 9.36 A simple variable structure system

Choosing x1 = e and x2 = e& as state variables, we have

x&1 = x2 (9.48)
x& 2 = −u
The trajectories corresponding to the structure u = – 1are given by (refer Eqns (9.24))
x1 (t ) = 12 x22 (t ) + x1 (0) − 12 x22 (0) (9.49a)
and the trajectories corresponding to the structure u = + 1are given by
x1 (t ) = − 12 x22 (t ) + x1 (0) + 12 x22 (0) (9.49b)
The phase-plane portraits of the two structures are shown in Figs 9.37a and 9.37b; the individual struc-
tures are families of parabolas. Neither of the structures is asymptotically stable; each structure is unstable.
However, by choosing a suitable switching logic between the two structures, we can make the resulting
variable structure system, asymptotically stable.
x2
x2

A
x2
u = –1 u = +1
>0
=0
<0
<0 O x1
O x1
=0
x1
>0

B
x22(0)
x22(0) = x1(0) +
= x1(0) – 2
2
(b) u = + 1 (c)
(a) u = – 1
Fig. 9.37 Phase trjectories for the system of Fig. 9.36
 544 Digital Control and State Variable Methods
Suppose the structure of the system is changed at any time the system’s trajectory crosses the vertical axis
of the state plane, that is,
⎧⎪ +1 if x1 > 0
u=⎨ (9.50)
⎪⎩ −1 if x1 < 0
A phase portrait of the system (9.48) with the switching logic specified by (9.50) is shown in Fig. 9.37c; the
system always enters into a limit cycle (In fact, we are familiar with the switching function given by (9.50); it is
on–off switching.
A x2 To achieve asymptotic stability, we redesign
P1 the switching logic. We note that one trajectory of
P3 each family in Figs 9.37a and 9.37b goes through
the origin. Segments A-O and B-O of these two
trajectories terminating at the origin form the
curve shown by the thick line in Fig. 9.38.
The following switching logic seems to give an
O x1 optimal control performance.
(i) If the initial state point lies at P1 on the seg-
ment A-O (Fig. 9.38), the state point (x1(t),
x2(t)) is driven to the origin along a segment
P4 of a parabola corresponding to u = + 1.
B (ii) If the initial point lies at P2 on the segment
P2
B-O (Fig. 9.38), the state point (x1(t), x2(t))
Fig. 9.38 Optimal switching curve is driven to the origin along a segment of a
parabola corresponding to u = – 1.
(iii) If the initial state point lies above or below the curve A-O-B, then only one switching is required to drive
the state point to the origin. Consider the initial state point at P3, which is above the curve A-O-B
(Fig. 9.38). The state point (x1(t), x2(t)) follows a parabola corresponding to u = + 1 till it reaches the
segment B-O. This is followed by switching of the control to –1, and driving of the state point to the
origin along B-O with u = – 1.
(iv) Consider the initial point at P4, which is below the curve A-O-B (Fig. 9.38). The state point (x1 (t),
x2 (t)) follows a parabola corresponding to u = – 1 till it reaches the segment A-O. This is followed by
switching of the control to + 1, and driving of the state point to the origin along A-O with u = + 1.
The double integrator model with the switching strategy described above is in fact an on–off closed-loop
control system shown in Fig. 9.36, in which the controller is actuated by a signal which is a function σ ( e, e& ) of
the error and its first derivative. The differential equation describing the dynamics of the system is given by
y = u or &&
&& e = −u (9.51a)
where e = r – y; r is constant, x1 = e and x2 = e& are state variables and
⎧ +1; σ ( x1 , x2 ) > 0
u=⎨ (9.51b)
⎩ −1; σ ( x1 , x2 ) < 0
It is also clear from Fig. 9.38 that, for all initial conditions, the state point is driven to the origin along the
shortest-time path with no oscillations (the output reaches the final value in minimum time with no ripples,
and stays there; this type of response is commonly called a deadbeat response). Such bang-bang control
systems provide optimal control (minimum-time control) [105].
The equation of optimal switching curve A-0-B can be obtained from Eqns (9.49) by setting (refer
Fig. 9.37)
 Nonlinear Systems Analysis 545

x22 (0) x 2 (0)

x1 (0) − = x1 (0) + 2 =0
2 2
This gives x1(t ) = − 12 x2 (t ) x2 (t ) (9.52a)
Let us define the switching function σ (e, e&) = σ ( x1 , x2 ) as
σ ( x1 , x2 ) = x1 (t ) + 12 x2 (t ) x2 (t ) (9.52b)
s (x1, x2) > 0 implies that the state point (x1, x2) lies above the curve A-O-B. s (x1, x2) = 0 and x2 > 0
implies that the state point (x1, x2) lies on the segment A-O. s (x1, x2) = 0 and x2 < 0 implies that the state
point (x1, x2) lies on the segment B-O. s (x1, x2) < 0 implies that the state point (x1, x2) lies below the segment
A-O-B.
In terms of the optimal switching function s (x1, x2), the control law becomes
⎧ +1 when σ ( x1 , x2 ) > 0
⎪
⎪ −1 when σ ( x1 , x2 ) = 0 and x2 (t ) < 0
u (t ) = ⎨ (9.53)
⎪ −1 when σ ( x1 , x2 ) < 0
⎪ +1 when σ x , x = 0 and x t > 0
⎩ ( 1 2) 2( )

The optimal switching may be realized by a computer. It accepts the state point (x1, x2) and computes the
switching function given by Eqn. (9.52b). It then manipulates the on–off controller to produce the optimal
control components according to Eqn. (9.53).

Example 9.9 In this example, we discuss a suboptimal method of switching the on–off controller in

Fig. 9.36. The advantage of the method described below, is that implementation of the switching function is
simple. The cost paid is in terms of increase in settling time compared to the optimal solution.
Consider the following suboptimal switching function:
σ ( e, e&) = e + K D e&
The system equations now become
e = −u;
&& u = sgn(e + K D e&)
In the state variable form ( x1 = e, x2 = e&) ,
x&1 = x2 ; x&2 = −sgn(x1 + K D x2 )
The phase plane is divided into two regions by the switching line
x1 + K D x2 = 0 (9.54)
The trajectory equation for the region defined by x1 + KD x2 > 0, is (refer Eqn. (9.49a))
x1 (t ) = 12 x22 (t ) + x1 (0) − 12 x22 (0 )
and the trajectory equation for the region defined by x1 + KD x2 > 0, is (refer Eqn. (9.49b))
x1 (t ) = − 12 x22 (t ) + x1 (0) + 12 x22 (0)
In each half of the phase plane separated by the switching line (9.54), the system trajectories would
be parabolas. Assume that the system under consideration starts with initial conditions corresponding to
point A in Fig. 9.39. The on–off controller switches when the representative point reaches B. By geometry of
the situation, we see that the trajectory resulting from the reversal of the drive at point B, will bring the
 546 Digital Control and State Variable Methods
1 representative point on a parabola passing
Slope = – K
D x2 much closer to the origin. This will continue
until the trajectory intersects the switching
line at a point closer to the origin than the
points A1 and A2 which are points of intersec-
A1
tion of the switching line with parabola pass-
C ing through the origin. In Fig. 9.39, point C
corresponds to this situation. Here, an instant
A x1 after the on–off controller is switched, the
system trajectory will recross the switching
line and the on–off controller must switch
A2
back. The on–off controller will, thus, chatter
while the system stays on the switching line.
B In a second-order system, the chattering
Region II Region I
frequency will be infinite and amplitude will
x1+ KD x2 < 0 x1 + KD x2 > 0 be zero; the representative point, thus, slides
u = –1 u= +1 along the switching line. It can easily be
Fig. 9.39 Suboptimal switching curve verified that with KD = 0 (on–off controller
switching on the vertical axis), the system always enters into a limit cycle. The switching process given by
the switching line (9.54) has, thus, converted the oscillating system into an asymptotically stable one; though
the goal has been achieved in a suboptimal way.

9.11 LYAPUNOV STABILITY DEFINITIONS

The general state equation for a nonlinear system can be expressed as

x& = f(x(t), u(t), t); x(t0) =D x0 (9.55)
where x is the n ´ 1 state vector, u is the p ´ 1 input vector, and
LM f (. ) OP
1
f (. ) P
f (.) = M
2
MM M PP
N f (. ) Q
n

is the n ´ 1 function vector.

Suppose that all the states of the system (9.55) settle to constant values (not necessarily zero values) for
a constant input vector u(t) = uc. The system is then said to be in an equilibrium state corresponding to the
input uc. The state trajectories converge to a point in state space, called the equilibrium point. At this point,
no states vary with time. Thus, we have the following definition of equilibrium point (equilibrium state).
If for any constant input vector u(t) = uc, there exists a point x(t) = xe = constant in state space, such that
at this point x& (t) = 0 for all t, then this point is called an equilibrium point of the system corresponding to the
input uc. Applying this definition to the system (9.55), any equilibrium point must satisfy
f(xe, uc, t) = 0 for all t (9.56)
The number of solutions depends entirely upon the nature of f (.) and no general statement is possible.
 Nonlinear Systems Analysis 547

Example 9.10 Consider the nonlinear system described by the state equations:
x& 1 = x2
x& 2 = – x1 – x21 – x2
The equilibrium states of this system are given by the solutions of the following set of equations (refer
Eqn. (9.56)):
x&1e = x2e = 0
x&2e = – x1e – (x1e)2 – x2e = 0
e
From the first equation, x2 is equal to zero. From the second equation,
(x1e)2 + x1e = x1e(x1e + 1) = 0
which has the solutions x1e = 0 and x1e = – 1. Thus, there are two equilibrium states, given by

xe1 =
LM OP
0
, xe2 =
LM OP
-1
0NQ N Q
0

In the stability analysis of a system, we are usually concerned with the following two notions of stability:
(i) when a relaxed system (x(t0) = 0) is excited by a bounded input, the output must be bounded; and
(ii) in an unforced system (u = 0) with arbitrary initial conditions, the system state must tend towards the
equilibrium point in state space.
We have seen earlier in Chapter 5 that the two notions of stability defined above are essentially equivalent
for linear time-invariant systems.
Unfortunately in nonlinear systems, there is no definite correspondence between the two notions. Most of
the important results obtained thus far concern the stability of nonlinear autonomous2 systems:
x& (t) = f(x(t)); x(t0) =D x0 (9.57)
in the sense of second notion above.
It may be noted that even for this class of systems, the concept of stability is not clear cut. The linear
autonomous systems have only one equilibrium state (the origin of the state space), and their behaviour
about the equilibrium state completely determines the qualitative behaviour in the entire state space. In
nonlinear systems, on the other hand, system behaviour for small deviations about the equilibrium point may
be different from that for large deviations. Therefore, local stability does not imply stability in the overall
state space, and the two concepts should be considered separately.
Secondly, the set of nonlinear equations (refer Eqns (9.56)–(9.57)),
f(xe) = 0 (9.58)
may result in a number of solutions (equilibrium points). Due to the possible existence of multiple equilib-
rium states, the system trajectories may move away from one equilibrium state to the other as time
progresses. Thus, it appears that in the case of nonlinear systems, it is simpler to speak of system stability
relative to the equilibrium state rather than using the general term ‘stability of a system’.
We shall confine our attention to nonlinear autonomous systems described by state equation of the form
D x0
x& (t) = f(x(t)); f(0) = 0; x(0) = (9.59)
Note that the origin of the state space has been taken as the equilibrium state of the system. There is no
loss in generality in this assumption, since any non-zero equilibrium state can be shifted to the origin by
appropriate transformation. Further, we have taken t0 = 0 in Eqn. (9.59), which is a convenient choice for
time-invariant systems.

2
An unforced (i.e., u = 0) and time-invariant system is called an autonomous system.
 548 Digital Control and State Variable Methods

For nonlinear autonomous systems, local stability may be investigated through linearization in the neigh-
bourhood of the equilibrium point. The validity of determining the stability of the unperturbed solution near
the equilibrium points from the linearized equations was developed independently by Poincaré and Lyapunov
in 1892. Lyapunov designated this as the first method. This stability determination is applicable only in a small
region near the equilibrium point and results in stability in-the-small.
The region of validity of local stability is generally not known. In some cases, the region may be too small
to be of any use practically; while in others the region may be much larger than the one assumed by the
designer—giving rise to systems that are too conservatively designed. We, therefore, need information about
the domain of stability. We present, in this chapter, the ‘second method of Lyapunov’ (also called the ‘direct
method of Lyapunov’) which is used to determine stability in-the-large.
The concept of stability formulated by Russian mathematician A.M. Lyapunov is concerned with the
following question:
If a system with zero input is perturbed from the equilibrium point xe at t = 0, will the state x(t) return to
xe, remain ‘close’ to xe, or diverge from xe?
Lyapunov stability analysis is thus concerned with the boundedness of the free (unforced) response of a
system. The free response of a system is said to be stable in the sense of Lyapunov at the equilibrium point
xe if, for every initial state x(t0) which is sufficiently close to xe, x(t) remains near xe for all t. It is asymptoti-
cally stable at xe if x(t), in fact, approaches xe as t ® ¥.
In the following, we give mathematically precise definitions of different types of stability with respect to
the system described by Eqn. (9.59).
Stability in the Sense of Lyapunov The system described by Eqn. (9.59) is stable in the sense of
Lyapunov at the origin if, for every real number e > 0, there exists a real number d(e) > 0 such that ||x(0)|| <
d results in ||x(t) || < e for all t ³ 0.
This definition uses the concept of the vector norm. The Euclidean norm for a vector with n components
x1, x2, . . ., xn is (refer Eqn. (5.6a))
|| x|| = (x12 + x22 + L + xn2)1/2
||x|| £ R defines a hyper-spherical region S(R) of radius R, surrounding the equilibrium point xe = 0. In terms
of the Euclidean norm, the above definition of stability implies that for any S(e) that we may designate, the
designer must produce S(d) so that the system state, initially in S(d), will never leave S(e). This is illustrated
in Fig. 9.40a.

S(e ) S(d )
S(d )

x(t) S(e )

0 0
x(0) x(0)

(a) (b)
Fig. 9.40 Stability definitions
 Nonlinear Systems Analysis 549
Note that this definition of stability permits the existence of continuous oscillation about the equilibrium
point. The state-space trajectory for such an oscillation is a closed path. The amplitude and frequency of the
oscillation may influence whether it represents acceptable performance.

Example 9.11 Consider a linear oscillator described by the differential equation

y (t) + w2 y(t) = 0
&&
where w is the frequency of oscillations.
Define the state variables as
x1(t) = y(t), x2(t) = y& (t)
This gives the state equations
x&1(t) = x2(t)
x&2(t) = – w2 x1(t)
x2
From these equations, we obtain the following equation
for state trajectory:
dx2 x
= - w2 1 or x22 + w2 x12 = c2; c = constant
dx1 x2
x1 Several state trajectories for various values of c, cor-
responding to various initial conditions of x1 and x2, are
shown in Fig. 9.41. For a specified value of e, we can
find a closed state trajectory whose maximum distance
from the origin is e. We then select a value of d which is
Fig. 9.41 less than the minimum distance from that curve to the
origin. The d (e) so chosen, will satisfy the conditions
that guarantee stability in the sense of Lyapunov.

Asymptotic Stability The system (9.59) is asymptotically stable at the origin if

(a) it is stable in the sense of Lyapunov, i.e., for each S(e) there is a region S(d ) such that trajectories,
starting within S(d ), do not leave S(e) as t ® ¥, and
(b) each trajectory starting within S(d ) converges to the origin as t ® ¥ (Fig. 9.40b).
Local and Global Stability The definitions of asymptotic stability and stability in the sense of Lyapunov
apply in a local sense (stability in-the-small) if the region S(d ) is small. When the region S(d ) includes the
entire state space, the definitions of asymptotic stability and stability in the sense of Lyapunov are said to
apply in a global sense (stability in-the-large).

9.12 LYAPUNOV STABILITY THEOREMS

The Lyapunov stability analysis is based upon the concept of energy, and the relation of stored energy with
system stability. We first give an example to motivate the discussion.
 550 Digital Control and State Variable Methods

x1
Consider the spring-mass-damper system of Fig. 9.42. The govern-
ing equation of the system is
K x&&1 + B x&1 + Kx1 = 0
A corresponding state variable model is
M=1
x&1 = x2 (9.60)
B x&2 = – Kx1 – Bx2
At any instant, the total energy V in the system consists of the kinetic
energy of the moving mass, and the potential energy stored in the spring.
Fig. 9.42 A spring-mass-damper V(x1, x2) = 1 x22 + 1 Kx12 (9.61)
2 2
system
Thus V(x) > 0 when x ¹ 0
V(0) = 0
This means that the total energy is positive unless the system is at rest at the equilibrium point xe = 0, where
the energy is zero.
The rate of change of energy is given by
. d ¶ V dx1 ¶ V dx 2
V (x1, x2) = V( x1 , x2 ) = + = - Bx 22 (9.62)
dt ¶ x1 dt ¶ x 2 dt
Case I: Positive damping (B > 0)
Let (x10, x20) be an arbitrary initial state of the system of Fig. 9.42. The solution of the differential equations
(9.60) corresponding to this initial state, gives the state trajectory x(t) for t > 0. Since the linear system (9.60)
is stable under the condition of positive damping, x(t) ® 0 as t ® ¥.
Let us study the relation of stored energy with system stability. The initial energy in the system is
V(x01, x20) = 1
2
(x20)2 + 1
2
K(x10)2
As per Eqn. (9.62), the rate of change of energy is negative and, therefore, system energy V(x1, x2) continu-
ally decreases along the trajectory x(t), t > 0. There is only one exception; when the representative point x(t)
of the trajectory reaches x2 = 0 points in the state plane, the rate of change of energy becomes zero. However,
as seen from Eqns (9.60), x& 2 = – Kx1 at the points where x2 = 0. The representative point x(t), therefore,
cannot stay at the points in the state plane where x2 = 0 (except at the origin). It immediately moves to the
points at which the rate of change of energy is negative and the system energy, therefore, continually
decreases from its initial value V(x01, x02) along the trajectory x(t), t > 0, till it reaches a value V = 0 at the
equlibrium point xe = 0.
A visual analogy may be obtained by considering the surface
V(x1, x2) = 1
2
x22 + 1
2
Kx21 (9.63)
This is paraboloid (a solid generated by rotation of parabola about its axis of symmetry) surface as shown
in Fig. 9.43. The value V(x1, x2) = ki (a constant) is represented by the intersection of the surface V(x1, x2) and
the plane z = ki. The projection of this intersection on the (x1, x2)-plane is a closed curve, an oval, around the
origin. There is a family of such closed curves in the (x1, x2)-plane for different values of ki. The closed curve
corresponding to V(x1, x2) = k1, lies entirely inside the closed curve corresponding to V(x1, x2) = k2 if k1 < k2.
The value V(x1, x2) = 0 is the point at the origin. It is the innermost curve of the family of closed curves,
representing different levels on the paraboloid for V(x1, x2) = ki.
If one plots a state-plane trajectory starting from the point (x01, x02), the representative point x(t) crosses the
ovals for successively smaller values of V(x1, x2), and moves towards the point corresponding to V(x1, x2) =
0, which is the equilibrium point. Figure 9.43 shows a typical trajectory.
 Nonlinear Systems Analysis 551
V = k3 z = V(x)
V = k3
k1 < k2 < k3
V = k2
t1 < t2 < t3
V = k1

V = k2

t3 x1
V = k1
t2 (x10, x20)
x1 t1

x2
x2
(a) (b)
Fig. 9.43 Constant-V curves

Note also that V(x) given by Eqn. (9.63) is radially unbounded 3, i.e., V(x) ® ¥ as ||x|| ® ¥. The ovals on
the (x1, x2)-plane extend over the entire state plane and, therefore, for any initial state x0 in the entire state
plane, the system energy continually decreases from the value V(x0) to zero.
Case II: Zero damping (B = 0)
Under the condition of zero damping, Eqns (9.60) become
x& = Ax

with A=
LM 0 1OP
N- K 0Q
The eigenvalues of A lie on the imaginary axis in the complex plane; the system response is, therefore,
oscillatory in nature.
.
From Eqn. (9.62) we observe that when B = 0, the rate of change of energy V (x1, x2) vanishes identically
along any trajectory; the system energy V(x1, x2) = V(x10, x20) for all t ³ 0. The representative point x(t) cannot
cross the V-contours in Fig. 9.43; it simply moves along one of these contours.
In the example given above, it was easy to associate the energy function V with the given system. How-
ever, in general, there is no obvious way of associating an energy function with a given set of equations
describing a system. In fact, there is nothing sacred or unique about the total energy of the system which
allows us to determine system stability in the way described above. Other scalar functions of the system state
can also answer the question of stability. This idea was introduced and formalized by the mathematician
A.M. Lyapunov. The scalar function is now known as the Lyapunov function and the method of investigating
stability using Lyapunov’s function is known as the Lyapunov’s direct method.
In Section 5.2, we introduced the concept of sign definiteness of scalar functions. Let us examine here the
scalar function V(x1, x2, . . ., xn) =
D V(x) for which V(0) = 0 and the function is continuous in a certain region
surrounding the origin in state space. Due to the manner in which these V-functions are used later, we define
the sign definiteness with respect to a region around the origin represented as ||x || £ K (a positive constant)
where ||x|| is the norm of x.
3
Use of the norm definition given by Eqn. (5.6b) immediately proves this result.
 552 Digital Control and State Variable Methods
Positive Definiteness of Scalar Functions A scalar function V(x) is said to be positive definite in the
region ||x|| £ K (which includes the origin of the state space), if V(x) > 0 at all points of the region except at
the origin, where it is zero.
Negative Definiteness of Scalar Functions A scalar function V(x) is said to be negative definite if
[ – V(x)] is positive definite.
Positive Semidefiniteness of Scalar Functions A scalar function V(x) is said to be positive semidefinite
in the region ||x || < K, if its value is positive at all points of the region except at finite number of points,
including origin, where it is zero.
Negative Semidefiniteness of Scalar Functions A scalar function V(x) is said to be negative semidefinite
if [ – V(x)] is positive semidefinite.
Indefiniteness of Scalar Functions A scalar function V(x) is said to be indefinite in the region || x || < K,
if it assumes both positive and negative values, within this region.
Examples For all x in the state plane,
(i) V(x) = x21 + x22 is positive definite,
(ii) V(x) = (x1 + x2)2 is positive semidefinite,
(iii) V(x) = – x21 – (x1 + x2)2 is negative definite,
(iv) V(x) = x1x2 + x22 is indefinite.
An important class of scalar functions is a quadratic form:
V(x) = xTPx
where P is a real, symmetric constant matrix. In this form, the definiteness of V is usually attributed to P. We
speak of the positive (negative) definite and the positive (negative) semidefinite P depending upon the
definiteness of V(x) = xTPx.
Tests for checking definiteness of a matrix were described in Section 5.2. We consider an example here:
LM
10 1 -2 OP
P= 4 -1
MM-21 PP
N
-1 1 Q
As per Sylvester’s test, the necessary and sufficient condition for P to be positive definite is that all the
successive principal minors of P be positive.
Applying Sylvester’s test to the given P, we obtain
10 > 0
10 1
>0
1 4
10 1 -2
1 4 -1 > 0
-2 -1 1
Since all the successive principal minors of the matrix P are positive, V(x) = xTPx is a positive definite fun
ction.
In the following, we state, without proof, the basic Lyapunov stability results. For proof refer [105].
Theorem 9.1 For the autonomous system (9.59), sufficient conditions of stability are as follows:
 Nonlinear Systems Analysis 553
Suppose that there exists a scalar function V(x) which, for some real number e > 0, satisfies the following
properties for all x in the region ||x || £ e :
(i) V(x) > 0; x ¹ 0
(i.e., V(x) is positive definite function)
(ii) V(0) = 0
(iii) V(x) has continuous partial derivatives with respect to all components of x.
Then the equilibrium state xe = 0 of the system (9.59) is
. .
(iva) asymptotically stable if V (x) < 0, x ¹ 0, i.e., V (x) is a negative definite function;
.
(ivb) asymptotically stable in-the-large if V (x) < 0, x ¹ 0, and in addition V(x) ® ¥ as ||x|| ® ¥.

Example 9.12 Consider a nonlinear system described by the equations

x& 1 = x2 – x1(x 21 + x22)
(9.64)
x& 2 = – x1 – x2(x 21 + x22)
Clearly, the origin is the only equilibrium state.
Let us choose the following positive definite scalar function as a possible Lyapunov function:
V(x) = x21 + x22 (9.65)
Time derivative of V(x) along any trajectory, is given by
. dV ( x1 , x 2 ) ¶ V dx1 ¶ V dx 2
V (x) = = +
dt ¶x1 dt ¶ x 2 dt
= 2x1 x& 1 + 2x2 x& 2 = – 2(x21 + x22)2 (9.66)
which is negative definite. This shows that V(x) is continually decreasing along any trajectory; hence V(x) is
a Lyapunov function. By Theorem 9.1, the equilibrium state (at the origin) of the system (9.64) is asymptoti-
cally stable.
Further, V(x) ® ¥ as || x|| ® ¥, i.e., V(x) becomes infinite with infinite deviation from the equilibrium
state. Therefore, as per condition (ivb) of Theorem 9.1, the equilibrium state of the system (9.64) is asymp-
totically stable in-the-large.

Although Theorem 9.1 is a basic theorem of Lyapunov stability analysis, it is somewhat restrictive
. .
because V (x) must be negative definite. This requirement can be relaxed to V (x) £ 0 (a negative semidefinite
.
V (x)) under proper conditions. This relaxed requirement is sufficient if. it can be shown that no trajectory can
stay forever at the points or on the line other than the origin, at which V (x) = 0. This is the case for the system
of Fig. 9.42 as described at the beginning of this section.
.
If, however, there exists a positive definite function V(x) such that V (x) is identically zero along a trajec-
tory, the system will remain in that trajectory and will not approach the origin. The equilibrium state at the
origin, in this case, is said to be stable in the sense of Lyapunov.
Theorem 9.2 For the autonomous system (9.59), sufficient conditions of stability are as follows.
Suppose that there exists a scalar function V(x) which, for some real number e > 0, satisfies the following
properties for all x in the region ||x || £ e :
(i ) V ( x) > 0; x ¹ 0 UV (i.e, V (x) is positive definite function)
(ii ) V (0) = 0 W
(iii) V(x) has continuous partial derivatives with respect to all components of x.
Then the equilibrium state xe = 0 of the system (9.59) is
 554 Digital Control and State Variable Methods
. . .
(iva) asymptotically stable if V (x) < 0, x ¹ 0, i.e., V (x) is a negative definite function; or if V (x) £ 0 (i.e.,
.
V (x) is negative semidefinite)
. and no trajectory can stay forever at the points or on the line other than
the origin, at which V (x) = 0;
(ivb) asymptotically stable in-the-large if conditions (iva) are satisfied, and in addition V(x) ® ¥ as ||x|| ® ¥;
.
(ivc) stable in the sense of Lyapunov if V (x) is identically zero along a trajectory.

Example 9.13 Consider the linear feedback system shown in Fig. 9.44 with r(t) = 0. We know that the
closed-loop system will exhibit sustained oscillations.
The differential equation for the error signal is
r=0 + e K y e&& + a 2e = Ky = – Ke
G(s) =
s2 + a2 Taking e and e& as state variables x1 and x2 respec-
–
tively, we obtain the following state equations:
x& 1 = x2 (9.67)
Fig. 9.44 Linear feedback system
x& 2 = – (K + a 2)x1
Let us choose the following scalar positive definite function as a possible Lyapunov function:
V(x) = x21 + x22 (9.68)
.
Then V (x) becomes
.
V (x) = 2x1 x& 1 + 2x2 x& 2 = 2[1 – (K + a 2)]x1x2
.
V (x) is indefinite. This implies that V(x), given by Eqn. (9.68), is not a Lyapunov function and stability can-
not be determined by its use (the system is known to be stable in the sense of Lyapunov as per the stability
definition given in Section 9.11).
We now test
V(x) = p1x21 + p2 x22; p1 > 0, p2 > 0
for Lyapunov properties. Conditions (i)–(iii) of Theorem 9.2 are obviously satisfied.
.
V (x) = 2p1x1x2 – 2p2(K + a 2)x1x2
2
.
If we set p1 = p2(K + a ), V (x) = 0 and, as per Theorem 9.2, the equilibrium state of the system (9.67) is
stable in the sense of Lyapunov.

Example 9.14 Reconsider the system of Fig. 9.44 with

K
G(s) =
s(s + a )
If the reference variable r(t) = 0, then the differential equation for the actuating error will be
e&& + a e& + Ke = 0
Taking e and e& as state variables x1 and x2 respectively, we obtain the following state equations:
x& 1 = x2
(9.69)
x& 2 = – K x1 – a x2
A candidate for a Lyapunov function is
V(x) = p1x21 + p2 x22; p1 > 0, p2 > 0,
which is a positive definite function.
 Nonlinear Systems Analysis 555
Its derivative is
.
V (x) = 2(p1 x1 x& 1 + p2 x2 x& 2) = 2(p1 – p2 K)x1x2 – 2 p2a x22
If we take p1 = K p2 with K > 0, a > 0, we obtain
.
V (x) = – 2a p2 x22
which is negative semidefinite.
. .
The condition V (x) = 0 exists along the x1-axis where x2 = 0. A way of showing that V (x), being negative
semidefinite, is sufficient for asymptotic stability is to show that x1-axis is not a trajectory of the system
differential equation (9.69). The first equation yields x& 1 = 0 or x1 = c. The x1-axis can be a trajectory only if
x2 = 0 and x& 2 = 0. Since on x1-axis, x& 2 = – Kc ¹ 0, x1-axis is not a trajectory, and the equilibrium state at the
origin of the system (9.69) is asymptotically stable.
Further, since V(x) ® ¥ as || x || ® ¥, the equilibrium state is asymptotically stable in-the-large.
This result, obtained by Lyapunov’s direct method, is readily recognized as being correct either from the
Routh stability criterion or from the root locus.

Example 9.15 Consider the system described by the state equations

x& 1 = x2
x& 2 = – x1 – x2
Let us choose,
V(x) = x21 + x22
which is a positive definite function; V(x) ® ¥ as ||x|| ® ¥.
This gives
.
V (x) = 2 x1 x& 1 + 2 x2 x& 2 = – 2x22
which is negative semidefinite. As per the procedure described in the earlier example, it can be established
.
that V (x) vanishes identically only at the origin. Hence, by Theorem 9.2, the equilibrium state at the origin
is asymptotically stable in-the-large.
To show that a different choice of a Lyapunov function yields the same stability information, let us
choose the following positive definite function as another possible Lyapunov function:
V(x) = 1
2 [(x1 + x2)2 + 2 x21 + x22]
.
Then V (x) becomes
.
V (x) = (x1 + x2)( x& 1 + x& 2) + 2x1 x& 1 + x2 x& 2
= (x1 + x2)(x2 – x1 – x2) + 2x1x2 + x2 (– x1 – x2) = – (x21 + x22)
which is negative definite. Since V(x) ® ¥ as || x || ® ¥, by Theorem 9.2, the equilibrium state at the origin
is asymptotically stable in-the-large.

Instability It may be noted that instability in a nonlinear system can be established by direct recourse to
the instability theorem of the direct method. The basic instability theorem is presented below.
Theorem 9.3 For the autonomous system (9.59), sufficient conditions for instability are as follows:
Suppose that there exists a scalar function W(x) which, for some real number e > 0, satisfies the following
properties for all x in the region || x || £ e :
 556 Digital Control and State Variable Methods

(i) W(x) > 0; x ¹ 0

(ii) W(0) = 0
(iii) W(x) has continuous partial derivatives with respect to all components of x.
Then the equilibrium state xe = 0 of the system (9.59) is unstable if W& (x) > 0, x ¹ 0, i.e., W& (x) is a positive
definite function.
Note that it requires as much ingenuity to devise a suitable W function, as to devise a Lyapunov function
V. In the stability analysis of nonlinear systems, it is valuable to establish conditions for which the system is
unstable. Then the regions of asymptotic stability need not be sought for such conditions, and the analyst is
saved from this fruitless effort.

9.13 LYAPUNOV FUNCTIONS FOR NONLINEAR SYSTEMS

The determination of stability through Lyapunov’s direct method centres around the choice of a positive
definite function V(x), called the Lyapunov function. Unfortunately, there is no universal method for select-
ing the Lyapunov function which is unique for a given nonlinear system. Some Lyapunov functions may
provide better answers than others. Several techniques have been devised for the systematic construction of
Lyapunov functions; each is applicable to a particular class of systems.
In addition, if a Lyapunov function cannot be found, it in no way implies that the system is unstable
(stability theorems presented in the earlier section, merely provide sufficient conditions for stability). It only
means that our attempt in trying to establish the stability of an equilibrium state of the system has failed.
Also, if a certain Lyapunov function provides stability for a specified parameter region, this does not neces-
sarily mean that leaving that region will result in system instability. Another choice of Lyapunov function
may lead to a larger stability region.
Further, for a given V-function, there is no general method which will allow us to ascertain whether it is
positive definite. However, if V(x) is in quadratic form in xi’s, we can use simple tests given in Section 5.2 to
ascertain definiteness of the function.
Inspite of all these limitations, Lyapunov’s direct method is the most powerful technique available today
for the stability analysis of nonlinear systems. Many mathematical methods for constructing Lyaunov func-
tions are available in the literature. Two of these methods—“Krasovskiis Method”, and the “Variable Gradi-
ent Method”—are simple and systematic ways of constructing Lyapunov functions. Howerver, the math-
ematical approach of these methods, though effective for simple systems, is often of little use for complex
dynamic equations.
Therefore, faced with specific systems, one has to use experience, intuition, and physical insights to
search for an approptiate Lyapunov function. An elegant and powerful Lyapunov analysis may be possible
for complex systems if engineering insight and physical properties are properly exploited [126].

Example 9.16 Consider a nonlinear system governed by the equations

x& 1 = – x1 + 2x21 x2
x& 2 = – x2
Note that x = 0 is the equilibrium point.
A candidate for a Lyapunov function is
V = p11x21 + p22 x22 ; p11 > 0, p22 > 0
which is a positive definite function.
 Nonlinear Systems Analysis 557
Then
dV
= 2p11x1 x& 1 + 2p22 x2 x& 2
dt
= 2p11x1(– x1 + 2x21x2) + 2p22x2(–x2)
= – 2p11x21(1 – 2x1x2) – 2p22 x22
dV/dt is negative definite if
1 – 2x1x2 > 0 (9.70)
Therefore, for asymptotic stability we require that the condition (9.70) is satisfied. The region of state space
where this condition is not satisfied is possibly the region of instability. Let us concentrate on the region of
state space where this condition is satisfied. The limiting condition for such a region is
1 – 2x1x2 = 0
The dividing lines lie in the first and the third quadrants and are rectangular hyperbolas as shown in
Fig. 9.45. In the second and the fourth quadrants, the inequality is satisfied for all values of x1 and x2.
Figure 9.45 shows the regions of stability and possible instability. Since the choice of the Lyapunov function
is not unique, it may be possible to choose another Lyapunov function for the system under consideration
which yields a larger region of stability.
x2
2
Possibly unstable
Stable quadrant
1 – 2x1x2 = 0
1

Stable

0 1 2 x1
Stable

Stable quadrant
1 – 2x1x2 = 0

Possibly unstable

Fig. 9.45 Stability regions for the nonlinear system of Example 9.16

9.13.1 The Krasovskii Method

In the following, we describe the Krasovskii Method of constructing Lyapunov functions for nonlinear
systems [105].
Consider the nonlinear autonomous system
x& = f(x); f(0) = 0 (9.71)
f = [ f1 f2 L fn ]T; x = [ x1 x2 L xn ]T
We assume that f(x) has continuous first partial derivatives.
We define a Lyapunov function as
V(x) = fT(x)P f(x) (9.72)
where P = a symmetric positive definite matrix.
 558 Digital Control and State Variable Methods
Now,
.
V (x) = &f T(x)P f(x) + fT(x)P &f (x) (9.73a)

where &f (x) = ¶ f (x ) dx = J(x)f(x);

¶ x dt
LM
¶ f1 ¶ f1
L
¶ f1 OP
¶ x1 ¶ x 2 ¶ xn
MM
¶ f2 ¶ f2 ¶ f2
PP
L
MM
J(x) = ¶ x1 ¶ x 2 ¶ xn PP (9.73b)
M M M
MM
¶ fn ¶ f n ¶ fn PP
L
MN
¶ x1 ¶ x 2 ¶ xn PQ
is the Jacobian matrix of f(x).
Substituting &f (x) in Eqn. (9.73a), we have
.
V (x) = fT(x)[JT(x)P + PJ(x)]f(x) (9.74a)
Let Q = – [JT(x)P + PJ(x)] (9.74b)
.
Since V(x) is positive definite, for the system to be asymptotically stable at the origin, V (x) should be
negative definite, or equivalently, Q should be positive definite. If, in addition, V(x) ® ¥ as || x || ® ¥, the
system is asymptotically stable in-the-large.

Example 9.17 As an illustration of the Krasovskii method, consider the nonlinear system shown in
Fig. 9.46, where the nonlinear element is described as
u = g(e) = e3

r=0 + e u K y
g(.)
s(s + 1)
–

Fig. 9.46 A nonlinear system

The system is described by the differential equation
e&& + e& = – Ke3; K > 0
Defining x1 = e and x2 = e& , we get the following state equations:
x& 1 = f1(x) = x2
(9.75)
x& 2 = f2(x) = – x2 – Kx31
The equilibrium point lies at the origin.
LM ¶ f1 ¶ f1 OP
Now J(x) = MM¶ x1 ¶ x 2
¶ f2 ¶ f 2
=
0
-3Kx12
PP LMN 1
-1
OP
Q
MN
¶ x1 ¶ x 2 PQ
Let P=M
Lp 11 p12 OP
Np 12 p22 Q
 Nonlinear Systems Analysis 559
For P to be positive definite,
p11 > 0 (9.76a)
2
p11 p22 – p12 > 0 (9.76b)
The matrix
Q = – [JT(x)P + PJ(x)]
R|SL0 -3Kx12 OP L p 11 p12OP LM
p11 p12OP LM 0 1 OPU|V
=–
|TMN1 -1 Q MN p12 p22
+
Q N
p12 p22 Q N-3Kx 2
1 -1 Q|W
L
=–M
-6 p12 K x12 p11 - p12 - 3p K x O
22
PQ
2
1

Np 11 - p12 - 3 p22 K x12 2( p12 - p ) 22

For the system (9.75) to be asymptotically stable at the origin, Q should be positive definite, i.e.,
6p12 Kx21 > 0 (9.76c)
– 12p12 Kx21(p12 – p22) – (p11 – p12 – 3p22 K x21)2 >0
or 12p12 K x21(p22 – p12) > (p11 – p12 – 3p22Kx21)2 (9.76d)
x2 Choose p12 > 0. Inequality (9.76c) then yields the condition
x21 > 0, which is always met.
Choose, p11 = p12, and p22 = b p12 with b > 1. Inequalities
(9.76a) and (9.76b) are satisfied, and inequality (9.76d)
x1
gives the condition

12(b – 1) > 9b 2Kx12 or

4 1
x21 <
1 F I
–
1 1
3K 3K b
- 2
b
GH JK
3K
It can easily be shown that the largest value of x1 occurs
when b = 2. Therefore,
Fig. 9.47 Stability region of the nonlinear 1 1 1
system of Fig. 9.46 x21 < or – < x1 <
3K 3K 3K
This region of asymptotic stability is illustrated in Fig. 9.47.

9.13.2 The Variable Gradient Method

In searching for a Lyapunov function, we can approach the problem in a backward manner. We begin with an
. .
assumed form for the derivative V (x), and go back to choose the parameters of V (x) so as to make V (x)
negative definite. This is a useful idea in searching for a Lyapunov function. A procedure that exploits this
idea is known as the variable gradient method.
To describe the procedure, let V (x) be a scalar function of x, and the vector function
⎡ ∂V ⎤
⎢ ∂x ⎥
⎢ 1 ⎥ ⎡ g1 (x) ⎤
⎢ ∂V ⎥ ⎢
∂V (x) ⎢ g (x) ⎥
g(x) = gradient of V(x) = = ⎢ ∂x2 ⎥⎥ = ⎢ 2 ⎥ (9.77)
∂x ⎢ M ⎥
⎢ M ⎥ ⎢ ⎥
⎢ ⎥ ⎢⎣ g n (x) ⎥⎦
⎢ ∂V ⎥
⎢⎣ ∂xn ⎥⎦
 560 Digital Control and State Variable Methods
.
The derivative V (x) along the trajectories of Eqn. (9.71) is given by
. ∂V ∂V ∂V
V (x) = x&1 + x&2 + L + x&n
∂x1 ∂x2 ∂xn
= (g(x))T x& = (g(x))T f(x) (9.78)
The idea now is to try to choose a vector function g(x), such that it would be a gradient of a positive definite
.
scalar function V(x) and, at the same time, V (x) would be negative definite. However, for a vector function
g(x) to be gradient of a scalar V(x), the Jacobian matrix of g(x) (refer Eqn. (9.73b))
⎡ ∂g1 ∂g1 ∂g1 ⎤
⎢ ∂x L
∂x2 ∂xn ⎥
∂g(x) ⎢ ⎥
1
=⎢ M M M ⎥
∂x ⎢ ⎥
⎢ ∂gn ∂gn L ∂gn ⎥
⎣⎢ ∂x1 ∂x2 ∂xn ⎦⎥
must be symmetric, i.e.,
∂ gi ∂ g j
= ∀i , j = 1, 2,..., n (9.79)
∂xj ∂xi
This is so because the Hessian matrix (refer Eqn.(8.18))
⎡ ∂ 2V ∂ 2V ∂ 2V ⎤
⎢ L ⎥
⎢ ∂x1
2 ∂x1 ∂x2 ∂x1 ∂xn ⎥
∂ ⎛ ∂V ⎞ ∂ V
2
( x)
⎜ ⎟= =⎢ M M M ⎥
∂x ⎝ ∂x ⎠ ∂x2 ⎢ ⎥
⎢ ∂ 2V ∂ V2
∂ 2V ⎥
⎢ L ⎥
⎣⎢ ∂xn ∂x1 ∂xn ∂x2 ∂xn ∂xn ⎦⎥

⎛ ∂ 2V ∂ 2V ⎞
is always symmetric ⎜ = ⎟.
⎜ ∂xi ∂xj ∂xj ∂xi ⎟
⎝ ⎠
Under the constraints (9.79), we start by choosing g(x) such that (g(x))T f (x) is negative definite. The func-
tion V(x) is then computed from the integral
x x n
V (x) = ∫ gT ( y ) d y = ∫ ∑ gi ( y1 , y2 ,..., yn ) dyi (9.80a)
0 0 i =1
The integration is taken over any path joining the origin to x (The line integral of a gradient vector is
independent of the path [125]). Usually, this is done along the axes; that is
x1 x2
V ( x) = ∫ g1 ( y1 , 0, 0,..., 0 ) dy1 + ∫ g2 ( x1 , y2 , 0,..., 0 ) dy2
0 0 (9.80b)
xn
+L + ∫ g n ( x1 , x2 ,..., xn−1 , yn ) dyn
0
By leaving some parameters of g(x) undetermined, one would try to choose them to ensure that V(x) is
positive definite.
 Nonlinear Systems Analysis 561

Example 9.18 Let us use the variable gradient method to find a Lyapunov function for the nonlinear
system
x&1 = – x1
(9.81)
x&2 = – x2 + x1 x22
We assume the following form for the gradient of the undetermined Lyapunov function.
⎡ g (x) ⎤ ⎡ a x + a x ⎤
g(x) = ⎢ 1 ⎥ = ⎢ 11 1 12 2 ⎥ ; aij may be functions of x (9.82)
⎣ g 2 (x)⎦ ⎣ a21 x1 + a22 x2 ⎦
The function has to satisfy the constraints (9.79):
∂g1 ∂g2 ∂a ∂a
= , i.e., a12 + x2 12 = a21 + x1 21
∂x2 ∂x1 ∂x2 ∂x1
If the coefficients are chosen to be
a11 = a22 = 1; a12 = a21 = 0
then
g1(x) = x1 and g2(x) = x2
and
.
V (x) = (g(x))T f(x)
= − x12 − x22 (1 − x1 x2 ) (9.83)
.
Thus, if (1–x1x2) > 0, then V is negative definite. The function V(x) can be computed as
x1 x2 2 2
x1 + x2
V (x ) = ∫ y1dy1 + ∫ y2 dy2 = (9.84)
0 0
2
This is a positive definite function and, therefore, the asymptotic stability of the origin in the region
1 > x1x2 is guaranteed.
Note that (9.84) is not the only Lyapunov function obtainable by the variable gradient method. A different
choice of aij’s may lead to another Lyapunov function for the system.

Review Examples

Review Example 9.1 Figure 9.48 shows the input-output waveforms of a saturating element or a limiter.
For small input signals (X < S), the output is proportional to the input. However, if the input amplitude is
sufficiently large to cause saturation (X > S), the output is a clipped sine wave. One cycle of the output,
which is a periodic function of period 2p, is described as follows:

K x; 0 £ w t < a
KS; a £ w t < (p – a)
y= K x; (p – a) £ w t < (p + a)
– KS; (p + a) £ w t < (2p – a)
K x; (2p – a) £ w t £ 2p
where a = sin–1(S/X)
 562 Digital Control and State Variable Methods
y y
Slope = K
+ 2
KS
–S
0
S x t
– KS
– 2 +

x
X

–
+

2 –
2 +

3 –

t
Fig. 9.48 Fourier-series analysis of a saturating element

This periodic function has odd symmetry:

y(w t) = – y(– w t)
Therefore, the fundamental component of y is given by (refer Eqn. (9.4d))
y1 = B1sinw t
2p
where B1 =
1
p z
0
y sinw t d(wt)

Due to symmetry of y (refer Fig. 9.48), the coefficient B1 can be calculated as follows:
p
2
LMa
OP p
2
4 4K
B1 =
p z0
y sin q dq =
p
MMz X sin q dq
0
+ z S sin q dq P
2

PPQ a
MN
4K L X O
= M (a - sin a cos a ) + S cos a P
p N2 Q
B1 2K ML S S F S I 2S 1 - FG S IJ OP
2 2
= sin -1
- 1- G J +
X p M
N X X H XK X H X K PQ
Therefore
 Nonlinear Systems Analysis 563

R| 2 K L S S FG S IJ 2 OP
N(X) = S p M
Msin -1
X
+
X
1-
H XK PQ; X ³ S (9.85)
||T K N ; X < S
The describing functions given by Eqn. (9.85) and nonlinearity 4 in Table 9.2, have a common term of
the form
2 LM 1 1 1 FG IJ 2 OP
Nc(z) = sin - 1 + 1- (9.86)
p MN z z z H K PQ
In terms of Nc(z), the describing function (9.85) may be expressed as

N(X) = KNc
X FG IJ (9.87)
S H K
The function Nc(z) is listed in Table 9.3.

Table 9.3 Values of Nc(z) given by Eqn. (9.86)

z Nc (z) z Nc (z) z Nc (z)

1.0 1.000 6.0 0.211 11.0 0.116
1.5 0.781 6.5 0.195 11.5 0.111
2.0 0.609 7.0 0.181 12.0 0.106
2.5 0.495 7.5 0.169 12.5 0.102
3.0 0.416 8.0 0.159 13.0 0.0978
3.5 0.359 8.5 0.149 14.0 0.0909
4.0 0.315 9.0 0.141 15.0 0.0848
4.5 0.281 9.5 0.134 19.0 0.0670
5.0 0.253 10.0 0.127 25.0 0.0509
5.5 0.230 10.5 0.121 30.0 0.0424
50.0 0.0255
100.0 0.0127

Review Example 9.2 Consider the nonlinear system of Fig. 9.49a, with a saturating amplifier having
gain K in its linear region. Determine the largest value of gain K for the system to stay stable. What would be
the frequency, amplitude and nature of the limit cycle for a gain K = 3?

r=0 + e 1 y
S=1 s(1 + 2s)(1 + s)
–

(a)
 564 Digital Control and State Variable Methods

Im

E increasing
(w1, E1)
A –1 Re

–1
N(X)

KG( jw )
(b)
Fig. 9.49 Nonlinear system with saturating amplifier

Solution It is convenient to regard the amplifier to have unit gain and the gain K to be attached to the
linear part. From Eqn. (9.87), we obtain, for S = 1 and K = 1, N(E) = Nc (E); the function Nc(E) is listed in
Table 9.3.
The locus of – 1/N(E) thus starts from (– 1 + j0) and travels along the negative real axis for increasing E,
as shown in Fig. 9.49b. Now, for the equation
KG ( jw) = –1/N(X)
to be satisfied, G( jw) must have an angle of – 180°:
ÐG( jw) = – 90° – tan– 1 2w – tan– 1w = –180°
This gives
2w + w
= tan 90° = ¥ or w = 1/ 2 rad/sec.
1 - 2w 2
The largest value of K for stability is obtained when KG( jw) passes through (–1 + j0), i.e.,
K
| KG( jw )|w =1/ =1 or
2
FG 1 IJ d 3 iFG 3 IJ = 1 or K = 3/2
H 2K H 2K
For K = 3, KG( jw) plot intersects – 1/N(X) locus, resulting in a limit cycle at (w1, E1) where w1 = 1/ 2 ,
while E1 is obtained from the relation
| –1/N(E1)| = |3G( jw1)| = 2 or |N(E1)| = 0.5
From Table 9.3, we obtain
E1 @ 2.5
Applying the stability test for the limit cycle reveals that point A in Fig. 9.49b corresponds to a stable limit
cycle.
 Nonlinear Systems Analysis 565

Review Example 9.3 Consider the servomechanism of Fig. 9.50, having a deadzone nonlinearity in
the feedback loop. With x1 = y and x2 = y& , we get
x&1 = x2
⎧ − x2 ; −1 < x1 < 1
⎪
x&2 = ⎨ − x2 − 2( x1 − 1); x1 > 1
⎪ − x − 2( x + 1); x < −1
⎩ 2 1 1

Slope = 2

Plant

e –1 u 1 y
1 s(s + 1)
–

Fig. 9.50 A system with deadzone nonlinearity

The phase plane may be divided into three regions:

Region I (defined by –1 < x1 < 1) The isocline equation is
−x
m = 2 = −1
x2
Since m is the slope of phase trajectories, all trajectories in Region I have a slope of –1. Typical trajectories
are shown in Fig. 9.51.
m = –1 x2 m = –1 m = –3
m=0 m = –2
m=1

m= 1 m=
–1 x1

m = –3 m = –2 m=1
m=0

Fig. 9.51 Isoclines and typical trajectories for the system of Fig. 9.50
 566 Digital Control and State Variable Methods
Region II (defined by x1 > 1) The isocline equation is
− x − 2 ( x1 − 1)
m= 2
x2
or
−2 ( x1 − 1)
x2 =
m +1
The isoclines are straightlines intersecting the x1-axis at x1 = 1, with slope equal to –2/(m + 1). Some of these
isoclines are shown in Fig. 9.51.
Region III (defined by x1 < –1) The isocline equation is
−2 ( x1 + 1)
x2 =
m +1
These isoclines are straightlines intersecting the x1-axis at x1 = –1, with slope equal to –2/(m + 1). Some of
these isoclines are shown in Fig. 9.51.
Some typical phase trajectories are also shown in Fig. 9.51. Note that for the given system, we have an
equilibrium zone, –1 £ x1 £ 1, with x2 = 0. The system comes to rest at any point within this zone.

Review Example 9.4 Consider the nonlinear system

x&1 = – x1 – x 22
x& 2 = – x2
Investigate the stability of the equilibrium points.
Solution The given system is
x& = f(x)
or x&1 = f1(x) = – x1 – x22
x&2 = f2(x) = – x2
Clearly, x = 0 is the only equilibrium point.
In the following, we apply Krasovskii method to determine sufficient conditions for asymptotic stability,
in the vicinity of the equilibrium point.
A candidate for a Lyapunov function is
V(x) = fT(x)Pf(x)
Selecting P = I may lead to a successful determination of the conditions for asymptotic stability in the
vicinity of the equilibrium point.
With this choice of Lyapunov function, we have (refer Eqns (9.74))
V& (x) = fT(x)[JT(x) + J(x)]f(x)
where
LM¶ f 1 ¶ f1 OP
J(x) = M
¶x
MM¶f
1
2
¶ x2
¶f 2
PP = LMN -10 -2 x 2
-1
OP
Q
N¶ x1 ¶ x2 PQ
 Nonlinear Systems Analysis 567
x2 The matrix
1
Q = – [JT(x) + J(x)] =
LM 2 OP
2 x2
N2 x 2 2 Q
Using Sylvester’s criterion (Section 5.2), we find that the matrix Q is
0 x1
positive definite if
4 – 4x22 > 0 or | x2 | < 1
–1 The shaded region in Fig. 9.52 is the region of asymptotic stability.
Fig. 9.52 It is, however, not the largest region. Another choice of Lyapunov
function for the system under consideration, may lead to a larger region of asymptotic stability in the vicinity
of the equilibrium point.

Problems

9.1 For a sinusoidal input x = X sin w t, find the output waveforms for each of the nonlinearities listed in Table 9.2.
By Fourier-series analysis of the output waveforms, derive the describing function for each entry of the table.
9.2 Consider the system shown in Fig. P9.2. Using the describing-function analysis, show that a stable limit cycle
exists for all values of K > 0. Find the amplitude and frequency of the limit cycle when K = 4, and plot y(t) versus t.

M
e K y

– s(1 + s)2

Fig. P9.2
9.3 Consider the system shown in Fig. P9.3. Use the describing function technique to investigate the possibility of
limit cycles in this system. If a stable limit cycle is predicted, determine its amplitude and frequency.

1 y
e 5
0.1 s(0.1s + 1)2
–

Fig. P9.3
9.4 Using the describing-function analysis, prove that no limit cycle exists in the system shown in Fig. P9.4. Find
the range of values of the deadzone of the on–off controller for which limit cycling will be predicted.

1
e 5 y
0.2 (s + 1)(0.1s + 1)2
–

Fig. P9.4
568 Digital Control and State Variable Methods
9.5 Consider the system shown in Fig. P9.5. Using the describing-function technique, show that a stable limit cycle
cannot exist in this system for any K > 0.

e K y

–
1 s(1 + s)2
Slope = 1

Fig. P9.5
9.6 Consider the system shown in Fig. P9.6. Using the describing-function analysis, investigate the possibility of a
limit cycle in the system. If a limit cycle is predicted, determine its amplitude and frequency, and investigate its
stability.

1
e 10 y
1 s(s + 1)(0.5s + 1)
–

Fig. P9.6
9.7 An instrument servo system used for positioning a load, may be adequately represented by the block diagram
in Fig. P9.7a. The backlash characteristic is shown in Fig. P9.7b. Show that the system is stable for K = 1. If the
value of K is now raised to 2, show that limit cycles exist. Investigate the stability of these limit cycles. Deter-
mine the amplitude and frequency of the stable limit cycle.
Given:
H/X 0 0.1 0.2 0.5 0.7 0.8 0.9 0.95 1.0

|N(X)| 1 0.954 0.882 0.593 0.367 0.249 0.126 0.063 0

ÐN(X) 0 – 6.7° – 13.4° – 32.5° – 46.6° – 55.1° – 65.6° – 72.8° – 90°

Slope = 1 y

e K x y x
N(X)
– s(s + 1)(0.5s + 1) H=1

(a) (b)
Fig. P9.7
9.8 Determine the kind of singularity for each of the following differential equations. Also locate the singular points
on the phase plane.
(a) y&& + 3 y& + 2 y = 0 (b) y&& + 5 y& + 6 y = 6 (c) y&& - 8 y& + 17 y = 34
 Nonlinear Systems Analysis 569
9.9 Consider the position control system shown in Fig. P9.9. The feedback action is provided by synchros, which
generate the actuating signal e = sin(qR – q). Prove that this nonlinear system has multiple singular points. Linear-
ize the system around the singular points (0, 0) and (p, 0), and identify the kind of singularity at each point.

R =0 + e K
f (.)
s(s + a)
–

Synchros

Fig. P9.9

9.10 A linear second-order servo is described by the equation

y + 2zwn y& + w2n y = w2n
&&
where wn = 1, y(0) = 2.0, y& (0) = 0
Determine the singular points when (i) z = 0, (ii) z = 0.15. Construct the phase trajectory in each case.
9.11 Consider the block diagram of a system, shown in Fig. P9.11.
(a) Derive state variable model with x1 = e and x2 = e& .
(b) Write equations of the isoclines on the x1 versus x2 plane.
(c) Given: x1(0) = 1, x2(0) = 0, obtain a trajectory on the x1 versus x2 plane. Show singular points (if any) and
some isoclines.

0.1
r = const + 0.1 7 y
e
s(s + 1)
–

Fig. P9.11

9.12 Consider the block diagram of a system, shown in Fig. P9.12.

(a) Derive state variable model with x1 = e and x2 = e& .
(b) Write equations of the isoclines on the x1 versus x2 plane.
(c) Given: x1(0) = 1, x2(0) = 0, obtain a trajectory on the x1 versus x2 plane. Show singular points (if any) and
some isoclines.

Slope
=1 y
r = const + e 7
s(s + 1)
– 0.1

Fig. P9.12
570 Digital Control and State Variable Methods

9.13 Consider the block diagram of a system, shown in Fig. P9.13.

(a) Derive state variable model with x1 = e and x2 = e& .
(b) Write equations of the isoclines on the x1 versus x2 plane.
(c) Given: x1(0) = 1, x2(0) = 0, obtain a trajectory on the x1 versus x2 plane. Show singular points (if any) and
some isoclines.

0.1
r = const + e 7 y
s(s + 1)
–
– 0.1

Fig. P9.13

9.14 Consider the block diagram of a system, shown in Fig. P9.14.

(a) Derive state variable model with x1 = e and x2 = e& .
(b) Write equations of the isoclines on the x1 versus x2 plane.
(c) Given: x1(0) = 1, x2(0) = 0, obtain a trajectory on the x1 versus x2 plane. Show singular points (if any) and
some isoclines.

0.1
r = const + e 7 y
0.1 s(s + 1)
–

Fig. P9.14

9.15 Consider the block diagram of a system, shown in Fig. P9.15.

(a) Derive state variable model with x1 = e and x2 = e& .
(b) Write equations of the isoclines on the x1 versus x2 plane.
(c) Given: x1(0) = 1, x2(0) = 0, obtain a trajectory on the x1 versus x2 plane. Show singular points (if any) and
some isoclines.

r = const + e + 1 y 1 y
– – s s

0.1 sgn y

Fig. P9.15
 Nonlinear Systems Analysis 571
9.16 The position control system shown in Fig. P9.16 has Coulomb friction Tc sgn(q& ) at the output shaft. Prove that
the phase trajectories on (e, e& /wn)-plane are semicircles, with the centre on horizontal axis at + Tc /K for e& < 0
and –Tc /K for e& > 0.
Tc

q R = const + e v
KA T = K1v J
–
Motor

Fig. P9.16
Examine the phase trajectory corresponding to qR = unit step, q& (0) = q (0) = 0; and find the value of the steady-
state error. What is the largest possible steady-state error which the system in Fig. P9.16 can possess?
Given:
wn = K / J = 1.2 rad/sec; Tc /K = 0.3 rad
where K = KAK1.
9.17 Consider the nonlinear system with deadzone shown in Fig. P9.17. Using the method of isoclines, sketch some
typical phase trajectories with and without deadzone, and comment upon the effect of deadzone on transient and
steady-state behaviour of the system.
9.18 Consider the system shown in Fig. P9.18 in which the nonlinear element is a power amplifier with gain equal to
1.0, which saturates for error magnitudes greater than 0.4. Given the initial condition: e(0) = 1.6, e& (0) = 0, plot
phase trajectories with and without saturation, and comment upon the effect of saturation on the transient
behaviour of the system. Use the method of isoclines for construction of phase trajectories.

0.4
e u 1 y e u 1 y
1 s(s + 1) 0.4 s(s + 1)
– –
Slope = 2

Fig. P9.17 Fig. P9.18

2
9.19 (a) A plant with model G(s) = 1/s is placed in a feedback configuration as in Fig. P9.19a. Construct a trajec-
tory on the (e, e& ) plane with r = 2 and y(0) = y& (0) = 0. Show that the system response is a limit cycle. What
is the switching line of this variable structure system?
(b) To the feedback control system of Fig. P9.19a, we add a derivative feedback with gain KD as in Fig. P9.19b.
Show that the limit cycle gets eliminated by the introduction of derivative-control term. What is the switch-
ing line of this variable structure system?
(c) Show that if KD is large, the trajectory may slide along the switching line towards the origin.

1 y
r = const + e u 1
– s2

(a)
572 Digital Control and State Variable Methods

1
r = const + e + u 1 y
– – s2

sKD

(b)
Fig. P9.19

9.20 A position control system comprises of a dc servomotor, potentiometer error detector, an on–off controller, and
a tachogenerator coupled to the motor shaft.
The following equations describe the system:
Reaction torque = q&& + 0.5q&
Drive torque = 2 sgn(e + 0.5 e& ); e = qR – q
(a) Make a sketch of the system showing how the hardware is connected.
(b) Construct a phase trajectory on (e, e& )-plane with e(0) = 2 and e& (0) = 0, and comment upon the transient and
steady-state behaviour of the system.
(c) What is the switching line of the variable structure system?
9.21 (a) Describe the system of Fig. P9.21a on the (e, e& )-plane, and show that with the on–off controller switching
on the vertical axis of the phase plane, the system oscillates with increasing frequency and decreasing
amplitude. Obtain a phase trajectory with (e(0) = 1.4, e& (0) = 0) as the initial state point.
(b) Introduce now a deadzone of ± 0.2 in the on–off controller characteristic. Obtain a phase trajectory for the
modified system with (e (0) = 1.4, e& (0) = 0) as the initial state point and comment upon the effect of
deadzone.

d i
(c) The on–off controller with deadzone is now controlled by the signal e + 13 e& , combining proportional and
derivative control (Fig. P9.21b). Draw the switching line on the phase plane and construct a phase trajec-
tory with (e(0) = 1.4, e& (0) = 0) as the initial state point. What is the effect of the derivative-control action?

1
e u 1 y
s(s + 1)
–

(a)

1 y
e 1 u 1
1+ s
3 0.2 s(s + 1)
–

(b)
Fig. P9.21
 Nonlinear Systems Analysis 573
9.22 Consider the second-order system
x& 1 = x2 ; x& 2 = – u
It is desired to transfer the system to the origin in minimum time from an arbitrary initial state. Use the bang-
bang control strategy with |u| = 1. Derive an expression for the optimum switching curve. Construct a phase
portrait showing a few typical minimum-time trajectories.
1
9.23 A plant with model G(s) = is placed in a feedback configuration as shown in Fig. P9.23. It is desired to
s ( s + 1)
transfer the system from any initial state to the origin in minimum time. Derive an expression for optimum switch-
ing curve and construct a phase portrait on the (e, e& )-plane showing a few typical minimum-time trajectories.

1
e u 1 y

– s(s + 1)

Fig. P9.23
9.24 Consider the nonlinear system described by the equations
x&1 = x2
x& 2 = – (1 – |x1|)x2 – x1
Find the region in the state plane for which the equilibrium state of the system is asymptotically stable.
9.25 Check the stability of the equilibrium state of the system described by
x&1 = x2
x& 2 = – x1 – x21 x2
9.26 Consider a nonlinear system described by the equations
x&1 = – 3x1 + x2
x& 2 = x1 – x2 – x32
Using the Krasovskii method for constructing the Lyapunov function with P as identity matrix, investigate the
stability of the equilibrium state.
Find a region of asymptotic stability using Krasovskii method.
9.27 Check the stability of the system described by
x&1 = – x1+ 2x21x2
x& 2 = – x2
by use of the variable gradient method.
 Nonlinear Control
10 Structures

10.1 INTRODUCTION

In the previous chapter, we covered tools and techniques for the analysis of control systems containing
nonlinearities. In the present chapter, we will discuss the deliberate introduction of nonlinearities into the
controller for performance improvement over that of a simpler linear controller. Although there are numer-
ous techniques, several of the most common are illustrated with examples, to give the reader the feel of the
general approach to designing nonlinear controllers.
If the system is only mildly nonlinear, the simplest approach might be to ignore the nonlinearity in
designing the controller (i.e., to omit the nonlinearity in the design model), but to include its effect in evalu-
ating the system performance. The inherent robustness of the feedback control law designed for the approxi-
mating nonlinear system is relied upon to carry it over to the nonlinear system.
When a system is significantly nonlinear, it is traditionally dealt with by linearization (refer Eqns (5.11))
about a selected operating point using Taylor series. We design a linear controller based on first-order
approximation. If the controller works effectively, the perturbations in actual state about the equilibrium
state will be small; if the perturbations are small, the neglected higher-order terms will be small and can be
regarded as a disturbance. Since the controller is designed to counteract the effects of disturbances, the
presence of higher-order terms should cause no problems. This reasoning cannot be justified rigorously, but,
nevertheless, it usually works. Needless to say, it may not always work; so it is necessary to test the design
that emerges, for stability and performance—analytically, by Lyapunov’s stability analysis for example,
and/or by simulation.
In many systems, the nonlinearity inherent in the plant, is so dominant that the linearization approach
described above can hardly meet the stringent requirements on systems performance. This reality, inevita-
bly, promotes the endeavour to develop control approaches that will more or less incorporate the nonlinear
dynamics into the design process. One such approach is feedback linearization. Unlike the first-order
approximation approach, wherein the higher-order terms of the plant are ignored, this approach utilizes the
feedback to render the given system, a linear input-output dynamics. On the basis of the linear system thus
obtained, linear control techniques can be applied to address design issues.
The roughness of the linearization approach based on first-order approximation, can be viewed from two
perspectives. First, it neglects all higher-order terms. Second, the linear terms depend on the equilibrium
 Nonlinear Control Structures 575
point. These two uncertainties may explain why this linearization approach is incapable of dealing with the
situation where the system operates over wide dynamic regions. Although the feedback linearization may
overcome the first drawback, its applicability is limited, mainly because it rarely leads to a design that
guarantees the system performance over the whole dynamic regime. This is because feedback linearization
is often performed locally—around a specific equilibrium point. The resulting controller is of local nature.
Another remedy to linearization based on first-order approximation, is to design several control laws corre-
sponding to several operating points that cover the whole dynamics of the system. Then these linear control-
lers are pieced together to obtain a nonlinear control law. This approach is often called gain scheduling.
Though this approach does not account for the higher-order terms, it does accommodate the variation of the
first-order terms with respect to the equilibrium points.
Adaptive control theory provides an effective tool for the design of uncertain systems. Unlike fixed-
parameter controllers (e.g., H¥ theory based robust controller), adaptive controllers adapt (adjust) their
behaviour on-line to the changing properties of the controlled processes.
If a fixed-parameter automatic control system is used, the plant-parameter variations directly affect the
capability of the design to meet the performance specifications under all operating conditions. If an adaptive
controller is used, the plant-parameter variations are accounted for at the price of increased complexity of
the controller. Adaptive control is certainly more complex than fixed-parameter control, and carries with it
more complex failure mechanisms. In addition, adaptive control is both time-varying and nonlinear,
increasing the difficulty of stability and performance analysis. It is this tradeoff, of complexity versus per-
formance, that must be examined carefully in choosing the control structure.
The main distinctive feature of variable structure systems (VSS), setting them apart as an independent
class of control systems, is that changes can occur in the structure of the system during the transient process.
The structure of a VSS is changed intentionally in accordance with some law of structural change; the times
at which these changes occur (and the type of structure formed) are determined not by a fixed program, but
in accordance with the current value of the states of the system.
The basic idea of feedback linearization in Section 10.2 is the algebraic transformation of the dynamics of
a nonlinear system to that of a linear system, on which linear control designs can in turn be applied. Sections
10.3–10.5 then study how to reduce, or practically eliminate, the effects of model uncertainties on the
stability and performance of feedback controllers using so-called adaptive and variable structure systems.
Section 10.6 introduces an emerging nonlinear control structure—the reinforcement learning control.
The chapter concentrates on nonlinear systems represented in continuous-time form. Even though most
control systems are implemented digitally, nonlinear physical systems are continuous in nature and are hard
to discretize meaningfully, while digital control systems may be treated as continuous-time systems in analy-
sis and design if high sampling rates are used. Thus, we perform nonlinear system analysis and controller
design in continuous-time form. However, of course, the control law is generally implemented digitally.
The objective set for this chapter is to make the reader aware of the nonlinear control structures com-
monly used for dealing with practical control problems in industry. The chapter is not intended to train the
reader on designing nonlinear control systems. For a comprehensive treatment of the subject, refer Slotine
and Li[126].

10.2 FEEDBACK LINEARIZATION

In this section, we describe basic concepts of feedback linearization intuitively with the help of an example.
Mathematical tools from differential geometry, which are useful to generalize these concepts to a broad class
of nonlinear systems, are not presented here for want of space; Slotine and Li [126], and Zak [35] are good
references for this subject.
 576 Digital Control and State Variable Methods

Y In its simplest form, feedback linearization amounts to

(X2, Y2) m2
cancelling the nonlinearities in a nonlinear system, so that the
closed-loop dynamics is in a linear form. This very simple idea
is demonstrated in the following example of control design for
l2 a two-link robot.
A two-link planar robot-arm manipulator, used extensively
2
(X1, Y1) for simulations in the literature, is shown in Fig. 10.1. This arm
is simple enough to simulate, yet has all the nonlinear effects
m1
l1 common to general robot manipulators.
1 To determine the arm dynamics, we assume that the link
0 X masses m1 and m2 are concentrated at the ends of the links of
Fig. 10.1 A two-link robot lengths l 1 and l 2, respectively. We define the angle of the
first link q1 with respect to an inertial frame, as depicted in
Fig. 10.1. The angle of the second link q2 is defined with respect to the orientation of the first link. Torques
t1 and t2 are applied by the actuators to control the angles q1 and q2, respectively.
Let us derive the dynamics of the 2-link arm from first principles using Lagrange’s equation of
motion [38]:
d ∂L ∂L
− = ô ; q = [q1 q2]T; t = [t1 t2]T (10.1)
dt ∂è& ∂è
with the Lagrangian L defined in terms of the kinetic energy K and potential energy P as

L = K (è, è& ) − P (è ) (10.2)

For link 1, we have the positions and velocities:
X 1 = l1 cos θ1; Y1 = l1 sin θ1
X& 1 = − l1 θ&1 sin θ1 ; Y&1 = l1 θ&1 cosθ1
v 2 = X& 2 + Y& 2 = l 2θ& 2
1 1 1 1 1
The kinetic and potential energies, for link 1, are
K1 = 12 m1v12 = 12 m1l12θ&12
P1 = m1 gY1 = m1 gl1 sin θ1
For link 2, we have the positions and velocities:
X 2 = l1 cos θ1 + l2 cos(θ1 + θ 2 )
Y2 = l1 sin θ1 + l2 sin (θ1 + θ 2 )
X& 2 = −l1θ&1 sin θ1 − l2 (θ&1 + θ&2 )sin(θ1 + θ 2 )
Y& = l θ& cosθ + l (θ& + θ& ) cos(θ + θ )
2 1 1 1 2 1 2 1 2

v22 = X& 22 + Y&22 = l12θ&12 + l22 (θ&1 + θ&2 )2 + 2l1l 2 (θ&12 + θ&1θ&2 ) cos θ 2
Therefore, kinetic energy for link 2 is
K 2 = 12 m2 v22
= 12 m2 l12θ&12 + 12 m2 l22 (θ&1 + θ&2 )2 + m2 l1l2 (θ&12 + θ&1θ&2 ) cosθ 2
 Nonlinear Control Structures 577
The potential energy for link 2 is
P2 = m2 gY2 = m2 g (l1 sin θ1 + l2 sin (θ1 + θ 2 ))
The Legrangian for the entire arm is
L = K − P = K1 + K 2 − P1 − P2
= 12 ( m1 + m2 )l12θ&12 + 12 m2 l22 (θ&1 + θ&2 ) 2 + m2 l1 l2 (θ&12 + θ&1θ&2 ) cosθ 2

−(m1 + m2 ) gl1 sin θ1 − m2 gl2 sin (θ1 + θ 2 )

Equation (10.1) is a vector equation comprised of two scalar equations. The individual terms needed to
write down these two equations are
∂L
= ( m1 + m2 )l12θ&1 + m2 l22 (θ&1 + θ&2 ) + m2 l1 l2 (2θ&1 + θ&2 ) cos θ 2
∂θ&
1

d ∂L
= ( m1 + m2 )l12θ&&1 + m2 l22 (θ&&1 + θ&&2 ) + m2 l1 l2 (2θ&&1 + θ&&2 ) cos θ 2 − m2 l1 l2 (2θ&1 θ&2 + θ&22 ) sin θ 2
dt ∂θ&1

∂L
= −( m1 + m2 ) gl1 cos θ1 − m2 gl2 cos(θ1 + θ 2 )
∂θ1
∂L
= m2 l22 (θ&1 + θ&2 ) + m2 l1 l2θ&1 cosθ 2
∂θ&
2

d ∂L
= m2 l22 (θ&&1 + θ&&2 ) + m2 l1 l2θ&&1 cosθ 2 − m2 l1 l2θ&1 θ&2 sin θ 2
dt ∂θ&2

∂L
= − m2 l1 l2 (θ&12 + θ&1 θ&2 ) sin θ 2 − m2 gl2 cos (θ1 + θ 2 )
∂θ 2
According to Lagrange’s equation (10.1), the arm dynamics are given by the two coupled nonlinear
differential equations

τ1 = [(m1 + m2 )l12 + m2 l22 + 2m2 l1 l2 cos θ 2 ]θ&&1 + [m2 l22 + m2 l1 l2 cos θ 2 ]θ&&2 − m2 l1 l2 (2θ&1 θ&2 + θ&22 ) sin θ 2
+ (m1 + m2 ) gl1 cosθ1 + m2 gl2 cos(θ1 + θ 2 )
τ 2 = [m2 l22 + m2 l1 l2 cos θ 2 ]θ&&1 + m2 l22θ&&2 + m2 l1 l2θ&12 sin θ 2 + m2 gl2 cos(θ1 + θ 2 )
Writing the arm dynamics in vector form, yields
⎡ (m1 + m2 )l12 + m2 l22 + 2m2 l1 l2 cos θ 2 m2 l22 + m2 l1 l2 cosθ 2 ⎤ ⎡θ&&1 ⎤ ⎡ − m2 l1 l2 (2θ&1 θ&2 + θ&22 )sin θ 2 ⎤
⎢ ⎥ ⎢ && ⎥ + ⎢ ⎥
⎣⎢ m2 l22 + m2 l1 l2 cos θ 2 m2 l22 ⎦⎥ ⎣θ 2 ⎦ ⎣⎢ m2 l1 l2θ&12 sin θ 2 ⎦⎥
⎡(m + m2 ) gl1 cosθ1 + m2 gl2 cos(θ1 + θ 2 ) ⎤ ⎡τ 1 ⎤ (10.3)
+⎢ 1 ⎥=⎢ ⎥
⎢⎣ m2 gl2 cos(θ1 + θ 2 ) ⎥⎦ ⎣τ 2 ⎦

One may write the dynamics of the two links arm completely as
M (è)&&
è + V(è, è& ) + G(è) = ô (10.4)
 578 Digital Control and State Variable Methods
where the symmetric inertia matrix
⎡α + β + 2η cos θ 2 β + η cosθ 2 ⎤
M (è ) = ⎢ ⎥,
⎢⎣ β + η cosθ 2 β ⎥⎦
and nonlinear terms
N(è, è& ) = V (è, è& ) + G(è)
⎡ −η (2θ&1 θ&2 + θ&22 ) sin θ 2 ⎤ ⎡α e cos θ1 + η e1 cos(θ1 + θ 2 ) ⎤
are given by V (è, è& ) = ⎢ ⎥ ; G ( è) = ⎢ 1 ⎥
⎣⎢ ηθ&12 sin θ 2 ⎦⎥ ⎣ ηe1 cos(θ1 + θ 2 ) ⎦

α = (m1 + m2 )l12 ; β = m2l22 ; η = m2 l1 l2 ; e1 = g / l1

This is a special form of state model called “Brunovsky canonical form”. Many systems, like the robot
arm, are naturally in the Brunovsky form. Moreover, it is often possible to transform general state models to
Brunovsky form. This is accomplished by finding a suitable state-space transformation followed by an input
transformation [126].
Defining the state vector as
⎡ x11 ⎤ ⎡ θ1 ⎤
⎢ ⎥ ⎢ ⎥
⎡ x1 ⎤ x12 ⎡è ⎤ θ 2
x = ⎢ ⎥ = ⎢ ⎥ = ⎢&⎥ = ⎢ & ⎥ (10.5)
⎣ x 2 ⎦ ⎢ x21 ⎥ ⎣ è ⎦ ⎢ θ1 ⎥
⎢ ⎥ ⎢& ⎥
⎢⎣ x22 ⎥⎦ ⎢⎣θ 2 ⎥⎦
we get the following state equations:
x& = è& = x
1 2 (10.6a)
x& 2 = &&
è = −M−1 (è)[V (è, è& ) + G(è)] + M−1 (è) ô
= −M −1 (x1 )[ V( x1 , x2 ) + G (x1 )] + M −1 (x1 )ô
= f (x) + g(x) ô (10.6b)
−1 −1
where f (x) = −M (x1 )[V (x1 , x2 ) + G (x1 )]; g(x) = M (x1 )
The control law
ô = g −1 (x)[−f (x) + u] (10.7)
linearizes the system (10.6) to yield
x& 1 = x 2
(10.8)
x& 2 = u
or
⎡ x&11 ⎤ ⎡0 0 1 0 ⎤ ⎡ x11 ⎤ ⎡0 0⎤
⎢ ⎥ ⎢
⎢ x&12 ⎥ = ⎢0 0 0 1 ⎥⎥ ⎢⎢ x12 ⎥⎥ ⎢⎢0 0 ⎥⎥ ⎡ u1 ⎤
+ (10.9)
⎢ x& ⎥ ⎢0 0 0 0 ⎥ ⎢ x21 ⎥ ⎢1 0 ⎥ ⎢⎣u2 ⎥⎦
⎢ 21 ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥
⎣⎢ x&22 ⎦⎥ ⎣⎢0 0 0 0 ⎦⎥ ⎣⎢ x22 ⎦⎥ ⎣⎢0 1 ⎦⎥
which may be completely expressed as
⎡ x& 1 ⎤ ⎡0 I ⎤ ⎡ x1 ⎤ ⎡0⎤
⎢ x& ⎥ = ⎢0 0⎥ ⎢ x ⎥ + ⎢ I ⎥ u (10.10)
⎣ 2⎦ ⎣ ⎦⎣ 2⎦ ⎣ ⎦
 Nonlinear Control Structures 579
A two-step design procedure follows.
Step 1 Use linear system design techniques to select a feedback control u(t) using the linear model (10.10).
Step 2 Compute the required arm torques from (10.7):
ô = g −1 (x)[−f (x) + u ] (10.11)
This is a nonlinear feedback control law that guarantees desired behaviour. It relies on computing the
torque t using Eqn. (10.7), that makes the nonlinear dynamics (10.6) equivalent to the linear dynamics
(10.10); this is termed feedback linearization.
Let us consider a specific design problem: tracking the desired motion trajectory q d (t).
Define the tracking error as
e(t) = q d (t) – q (t) (10.12)
Therefore
e& (t ) = è& (t ) − è& (t ); &&
d e(t ) = è&& (t ) − è&&(t )
d (10.13)
Defining x% 1 = e , and x% 2 = e& (10.14)
we can write robot-arm dynamics (10.10) in the form
x&% 1 = x% 2 (10.15a)
x&% = && 2è − &&
è d

= &&
è d + M −1 (è)[ V(è, è& ) + G(è)] − M −1 (è) ô
= &&
è d + M −1 (x1 )[ V(x1 , x2 ) + G(x1 )] − M −1 (x1 ) ô
= &&
è − f (x) − g(x)ô
d (10.15b)
The control law
&& ]
ô = g −1 (x)[−f (x) − u + è (10.16)
d
linearizes the system (10.15) to yield
⎡ x%& 1 ⎤ ⎡0 I ⎤ ⎡ x% 1 ⎤ ⎡0⎤
⎢& ⎥ = ⎢ ⎥ ⎢ ⎥ + ⎢ ⎥u (10.17)
⎣ x% 2 ⎦ ⎣0 0⎦ ⎣ x% 2 ⎦ ⎣ I ⎦
The following design procedure may be used.
Use linear quadratic regulator design to select a feedback control u(t), that stabilizes the tracking error
system (10.17). The performance measure
∞
J = ∫ (x% T Qx% + uT Ru )dt (10.18)
0

with Q = diag {1, 0.1, 1, 0.1}; R = diag {0.005, 0.005}

penalizes the joint angles q 1 and q 2 much more strongly than the control inputs, to achieve high speed and
accuracy. The feedback control law is of the form
u = −K1x% 1 − K 2 x% 2
(10.19)
= − [ K11 K12 ] x% 1 − [ K 21 K22 ] x% 2
The computed-torque controller than becomes (refer (10.16))
&& ]
ô = g −1 (x)[−f (x) + K1x% 1 + K 2x% 2 + è (10.20)
d
The computed-torque controller is shown in Fig. 10.2, which has a multiloop structure; with a nonlinear
inner feedback linearization loop and an outer tracking loop. Simulation of this controller for the two-link
robot arm (m1 = 1, m2 = 1, l1 = 1, l2 = 1, g = 9.8; qd1(t) = sin(p t), qd2 (t) = cos (p t)) was done using MATLAB
(refer Problems A.18 and A. 22 in Appendix A). Figures 10.3 show the tracking performance.
580 Digital Control and State Variable Methods

f (x)

[x~1 ~x2]T d

+ – 2-Link
+ u +
[–K1 –K2] g–1(x) Robot
–
– Arm
Controller
xd = [ d d]
T x=[ ]T
Fig. 10.2
1.5
Desired
Actual
1

0.5
q1

–0.5

–1
0 1 2 3 4 5 6 7 8
t
(a)
1
Desired
Actual
0.5

0
q2

–0.5

–1

–1.5
0 1 2 3 4 5 6 7 8
t
(b)
Fig. 10.3 Desired and actual trajectories
 Nonlinear Control Structures 581
Unlike the linearization approach which ignores all higher-order terms of the plant, the feedback
linearization approach utilizes the feedback, to render the given system a linear input-output dynamics.
Then, on the basis of the linear system thus obtained, linear control techniques can be applied to address
design issues. Finally, the resulting nonlinear control law is implemented to achieve the desired dynamical
behaviour.
The main drawback of this technique is that it relies on exact cancellation of nonlinear terms to get linear
input-output behaviour (This is equivalent to cancellation of the nonlinearity with its inverse). Consequently,
if there are errors or uncertainty in the model of the nonlinear terms, the cancellation is no longer exact.
Therefore, the applicability of such model-based approaches to feedback control of actual systems is quite
limited, because they rely on the exact mathematical models of system nonlinearities.
If the functions f (x) and g–1 (x) are not exactly known in the control scheme of Fig. 10.2, we may explore
the option of constructing their estimates by two neural networks. We shall study this option of intelligent
control in Chapter 11.

10.3 MODEL REFERENCE ADAPTIVE CONTROL

When we use a model of the plant as the basis of a control system design, we are tacitly assuming that this
model is a reasonable representation of the plant. Although the design model almost always differs from the
true plant in some details, we are confident that these details are not important enough to invalidate the
design.
There are many applications, however, for which a design model cannot be developed with a reasonable
degree of confidence. Moreover, most dynamic processes change with time. Parameters may vary because
of normal wear, aging, breakdown, and changes in the environment in which the process operates. The
feedback mechanism provides some degree of immunity to discrepancies between the physical plant and the
model that is used for the design of the control system. But sometimes, this is not enough. A control system
designed on the basis of a nominal design model may not behave as well as expected, because the design
model does not adequately represent the process in its operating environment.
How can we deal with processes that are prone to large changes, or for which adequate design models are
not available? One approach is brute force, i.e., high loop-gain: as the loop-gain becomes infinite, the output
of the process tracks the input with vanishing error. Brute force rarely works, however, for well-known
reasons—dynamic instability, control saturation, and susceptibility to noise and other extraneous inputs.
In robust control design methods, model uncertainties are captured in a family of perturbed plant models,
where each member of the family may represent the nominal plant, but which member does so, remains
unknown. A robust controller satisfies the design requirements in connection with all the members of the
family. Robust control design techniques are sophisticated and make it possible, for the control system
design, to tolerate substantial variations in the model. But the price of achieving immunity to model uncer-
tainties may be a sacrifice in performance. Moreover, robust control design techniques are not applicable to
processes for which no (uncertainty) design model is available.
The adaptive control theory provides another approach to the design of uncertain systems. Unlike the
fixed-parameter controller, adaptive controllers adjust their behaviour on-line, in real-time, to the changing
properties of the controlled processes. For example, in some control tasks, such as those in robot manipula-
tions, the systems to be controlled have parameter uncertainty at the beginning of the control operation.
Unless this uncertainty is gradually reduced on-line by an adaptation or estimation mechanism, it may cause
inaccuracy or instability for the control systems. In many other tasks, such as those in power systems, the
system dynamics may have well-known dynamics at the beginning, but experience unpredictable parameter
variations as the control operation goes on. Without continuous “redesign” of the controller, the initially
 582 Digital Control and State Variable Methods

appropriate controller design may not be able to control the changing plant well. Generally, the basic objec-
tive of adaptive control is to maintain consistent performance of a system in the presence of uncertainty
or unknown variation in plant parameters. Since such parameter uncertainty, or variation occurs in many
practical problems, adaptive control is useful in many industrial contexts. These include:
Robot Manipulation Robots have to manipulate loads of various sizes, weights, and mass distributions.
It is very restrictive to assume that the inertial parameters of the loads are well known before a robot picks
them up and moves them away. If controllers with constant gains are used, and the load parameters are not
accurately known, robot motion can be either inaccurate or unstable. Adaptive control, on the other hand,
allows robots to move loads of unknown parameters with high speed and high accuracy.
Ship Steering On long courses, ships are usually put under automatic steering. However, the dynamic
characteristics of a ship strongly depend on many uncertain parameters, such as water depth, ship loading,
and wind and wave conditions. Adaptive control can be used to achieve good control performance under
varying operating conditions.
Aircraft Control The dynamic behaviour of an aircraft depends on its altitude, speed, and configuration.
The ratio of variations of some parameters can lie between 10 to 50 in a given flight. Adaptive control can be
used to achieve consistent aircraft performance over a large flight envelope.
Process Control Models for metallurgical and chemical processes are usually complex and also hard to
obtain. The parameters characterizing the processes vary from batch to batch. Furthermore, the working condi-
tions are usually time-varying (e.g., reactor characteristics vary during the reactor’s life, the raw materials
entering the process are never exactly the same, atmospheric and climatic conditions also tend to change). In
fact, process control is one of the most important, application areas of adaptive control.
Adaptive control has also been applied to other areas, such as power systems.
The concept of controlling a process that is not well understood, or one in which the parameters are
subject to wide variations, has a history that predates the beginning of modern control theory. The early
theory was empirical, and was developed before digital computer techniques could be used for extensive
performance simulations. Prototype testing was one of the few techniques available for testing adaptive
control. At least one early experiment had disastrous consequences. As the more mathematically rigorous
areas of control theory were developed starting in the 1960s, interest in adaptive control faded for a time,
only to be reawakened in the late 1970s with the discovery of mathematically rigorous proofs of the conver-
gence of some popular adaptive control algorithms. This interest continues unabated [130–136].
Many, apparently different, approaches to adaptive control have been proposed in the literature. Two
schemes in particular have attracted much interest: “Self-Tuning Regulator” (STR), and “Model Reference
Adaptive Control” (MRAC). These two approaches actually turn out to be special cases of a more general
design philosophy.
In the following, we describe model reference adaptive control (MRAC); the discussion on self-tuning
regulator will appear in the next section.
Generally, a model-reference adaptive control system can be schematically represented by Fig. 10.4. It is
composed of four parts: a plant/process containing unknown parameters, a reference model for compactly
specifying the desired output of the control system, a feedback control law containing adjustable param-
eters, and an adaptation mechanism for updating the adjustable parameters.
The plant is assumed to have a known structure, although the parameters are unknown. For linear plants,
this means that the number of poles and the number of zeros are assumed to be known, but that the locations
of these poles and zeros are not.
 Nonlinear Control Structures 583
ym
Model

Controller parameters Adjustment

mechanism

Command
signal y
Controller Process
Control
signal

Fig. 10.4 Block diagram of a model-reference adaptive control system

A reference model is used to specify the ideal response of the adaptive control system to the external
command. Intuitively, it provides the ideal plant response, which the adaptation mechanism should seek in
adjusting the parameters. The choice of the reference model is part of the adaptive control system design.
This choice should reflect the performance specifications in the control tasks, such as rise time, setting time,
overshoot or frequency domain characteristics.
The controller is usually parameterized by a number of adjustable parameters (implying that one may
obtain a family of controllers by assigning various values to the adjustable parameters). The controller
should have perfect tracking capacity in order to allow the possibility of tracking convergence. That is, when
the plant parameters are exactly known, the corresponding controller parameters should make the plant
output identical to that of the reference model. When the plant parameters are not known, the adaptation
mechanism will adjust the controller parameters, so that perfect tracking is asymptotically achieved. If the
control law is linear in terms of the adjustable parameters, it is said to be linearly parameterized. Existing
adaptive control designs normally require linear parameterization of the controller—in order to obtain adap-
tation mechanisms with guaranteed stability and tracking convergence.
The adaptation mechanism is used to adjust the parameters in the control law. In MRAC systems, the
adaptation law searches for parameters such that the response of the plant under adaptive control, becomes the
same as that of the reference model, i.e., the objective of the adaptation is to make the tracking error converge
to zero. Clearly, the main difference from conventional control, lies in the existence of this mechanism. The
main issue in adaptation design is to synthesize an adaptation mechanism which will guarantee that the control
system remains stable and the tracking error converges to zero—even if the parameters are varied. Many
formalisms in nonlinear control can be used to this end, such as Lyapunov theory, hyperstability theory, and
passivity theory[126]. In this section, we shall use Lyapunov theory.
Thus, the desired performance in an MRAC (Fig. 10.4) is given in terms of a reference model, which,
in turn, gives the desired response to the command signal. The inner control loop is an ordinary feedback
loop composed of the plant and the controller; the parameters of the controller are adjusted by the outer loop
in such a way that the error between the plant and model outputs becomes small. The key problem is to
determine the adjustment mechanism so that a stable system, which brings the error to zero, is obtained.
From the block diagram of Fig 10.4, one may jump to the false conclusion that MRAC has an answer to
all control problems with uncertain plants. Before using such a scheme, important theoretical problems such
as stability and convergence have to be considered. Since adaptive control schemes are both time-varying
and nonlinear, stability and performance analysis becomes very difficult. Many advances have been made in
proving stability under certain (sometimes limited) conditions. However, not much ground has been gained
in proving performance bounds.
 584 Digital Control and State Variable Methods

10.3.1 Lyapunov Stability of Non-Autonomous Systems

In Chapters 8 and 9, we studied Lyapunov analysis of autonomous systems. An adaptive control system is a
non-autonomous system because the parameters involved in its dynamic equations vary with time. Although
many of the ideas in Chapters 8 and 9 can be similarly applied to the non-autonomous case, the conditions
required in the treatment of non-autonomous systems are more restrictive. Scalar functions with explicit
time-dependence, V(x, t), are required while in autonomous system analysis, time-invariant functions V(x)
suffice. Asymptotic stability analysis of non-autonomous systems is generally harder than that of autono-
mous systems since it is usually very difficult to find Lyapunov functions with a negative definite derivative.
.
When V (x, t) is only negative semidefinite, Lyapunov theorems on asymptotic stability are not applicable.
Lyapunov Theorem for Non-autonomous Systems [126] If, in a certain neighbourhood of the equilib-
rium point 0, there exists a scalar function V(x, t ) with continuous partial derivatives such that
· V(x, t ) is positive definite, and
.
· V (x, t) is negative semidefinite,
then the equilibrium point 0, is stable in the sense of Lyapunov.
Similar to the case of autonomous systems, if in a certain neighbourhood of the equilibrium point, V(x, t)
.
is positive definite and V (x, t ), its derivative along the system trajectories, in negative semidefinite, then
V (x, t ) is called a Lyapunov function for the non-autonomous system.
The theorem stated above establishes stability in the sense of Lyapunov, and not asymptotic stability of
the origin. An important and simple result which partially remedies this situation is Barbalat’s lemma.
Barbalat’s lemma is a purely mathematical result concerning the asymptotic properties of functions and their
derivatives. When properly used for non-autonomous systems, it may lead to satisfactory solution of many
asymptotic stability problems.
Barbalat’s Lemma Before describing Barbalat’s lemma itself, let us clarify a few points concerning the
asymptotic properties of functions and their derivatives. Given a differentiable function f (t), the following
facts are important to keep in mind [126]:
· If f is lower bounded (for some l > 0, the region defined by f (t) < l is bounded) and decreasing
( f £ 0), then it converges to a limit. This is a standard result in calculus.
· The fact that f (t) converges as t ® ¥, does not imply that f& ( t ) → 0 . For example, while the function,
f (t) = e–t sin(e2t ) ® 0, its derivative f& ( t ) is unbounded.
Given that a function tends towards a finite limit, what additional requirement can guarantee that its
derivative actually converges to zero? Barbalat’s lemma indicates that the derivative itself should have some
smoothness properties; it should be uniformly continuous.
A function f& ( t ) is uniformly continuous if one can always find a dR for a given R > 0 such that for any ti
and t Î[0, dR ], we have
f& ( ti + τ ) − f& ( ti ) < R
Uniform continuity of a function is often difficult to assert from the above definition. A more convenient
approach is to examine the function’s derivative. A very simple sufficient condition for a differentiable
function to be uniformly continuous, is that its derivative must be bounded. Thus, if the function f (t) is twice
differentiable, then its derivative f& ( t ) is uniformly continuous if its second derivative &&
f ( t ) is bounded. This
can easily be seen from the finite difference theorem:
For all ti and all ti + t, there exists t¢ (between ti and ti + t ), such that
f& (t + t ) - f& (t ) = &&
i i f (t ¢ )[(t + t ) - t ]
i i
 Nonlinear Control Structures 585

If R1 > 0 is an upper bound on the function &&

f , then
f& ( ti + τ ) − f& ( ti ) ≤ R1τ
For t Î[0, dR] and dR = R/R1, we have
f& ( ti + τ ) − f& ( ti ) < R
which is the definition of uniform continuity.
We can now state the Barbalat’s lemma (for proof, refer [35, 126]):
If a differentiable function f (t) has a finite limit, and if f& ( t ) is uniformly continuous, then f& ( t ) → 0 as
t ® ¥.
To apply Barbalat’s lemma to the analysis of non-autonomous systems, one typically uses the following
immediate corollary:
If a scalar function V (x, t ) satisfies the following conditions:
· V (x, t) is lower bounded
.
· V. (x, t ) is negative semidefinite
· . V (x, t ) is uniformly continuous in time
then V (x, t) ® 0 as t ® ¥
10.3.2 Application to a First-Order Control System
We illustrate the design methodology of MRAC through a simple example. Consider a system with the plant
model of the form
y& p = − a p y p + b p u; y p (0) @ y 0p (10.21)
where u is the control variable, and yp is the measured state (output); ap and bp are unknown coefficients.
Assume that it is desired to obtain a closed-loop system described by
y&m = − am ym + bm r ym (0) @ ym0 (10.22)
am and bm are known coefficients of the reference model.
When the parameters of the plant are known, the following control law gives perfect model following:
u (t) = br (t) – ayp (t) (10.23)
with the parameters
b am − a p
b = m ,a = (10.24)
bp bp
An MRAC which can find the appropriate gains a(t) and b(t) when parameters ap and bp are not known,
may be obtained as follows (Fig. 10.5).
ym
Reference model –
ym = – amym + bmr e

u yp
Controller Plant

r Adjustment
mechanism
Fig. 10.5 A model reference adaptive system
 586 Digital Control and State Variable Methods
Introduce the error variable
e(t) = yp (t) – ym (t) (10.25a)
The rate of change of the error is given by
de(t )
= [ − a p y p (t ) + b p u (t )] − [ − am ym (t ) + bm r (t )]
dt
= −am e(t ) + [am − a p − b p a(t )] y p (t ) + [b pb(t ) − bm ]r (t ) (10.25b)
Notice that the error goes to zero if the parameters a(t) and b(t) are equal to the ones given by (10.24). We
will now attempt to construct a parameter adjustment mechanism that will drive the parameters a(t) and b(t)
to appropriate values, such that the resulting control law (10.23) forces the plant output yp(t) to follow the
model output ym(t). For this purpose, we introduce the Lyapunov function,
⎡ 2⎤
( )
V(e,a,b) = 1 ⎢ e2 (t ) + 1 bp a(t) + a p − am + 1 bp b(t) − bm ⎥ ( )
2
2
⎣⎢ bp γ bp γ ⎦⎥
where g > 0.
This function is zero when e(t) is zero and the controller parameters a(t) and b(t) are equal to the optimal
values given by (10.24). The derivatives of V is
dV de(t ) 1 da(t ) 1 db(t )
= e (t ) + [b p a(t ) + a p − am ] + [bp b(t ) − bm ]
dt dt γ dt γ dt
1 ⎡ da(t ) ⎤ 1 ⎡ db(t ) ⎤
= −am e 2 (t ) + [b p a(t ) + a p − am ] ⎢ − γ y p (t )e (t )⎥ + [bp b(t ) − bm ] ⎢ + γ r ( t )e (t ) ⎥
γ ⎣ dt ⎦ γ ⎣ dt ⎦
If the parameters are updated as
db(t )
= −γ r (t )e(t )
dt
(10.26)
da (t )
= γ y p (t )e (t ),
dt
we get
dV
= − a m e 2 (t )
dt
Thus, the adaptive control system is stable in the sense of Lyapunov, i.e., the signals e, a and b are bounded.
To ensure that tracking error goes to zero, we compute second time derivative of Lyapunov function:
d 2V de(t )
= −2 am e(t )
dt 2 dt
From (10.25b), it follows that
d 2V
= −2am e(t )[− am e(t ) + (am − a p − bp a(t )) y p (t ) + (b p b(t ) − bm )r (t )]
dt 2
= f (e, a, b, yp, r)
..
Since all the parameters are bounded, and yp(t) = e(t) + ym(t) is bounded, V is also bounded, which, in
.
turn, implies that V is uniformly continuous. Therefore, the asymptotic convergence of the tracking error e(t)
is guaranteed by Barbalat’s lemma.
Figure 10.6 shows the simulation (refer Problem A.19 in Appendix A) of MRAC system with ap = 1,
bp = 0.5, am = 2 and bm = 2. The input signal is a square wave with amplitude 1. The adaptation gain g = 2.
The closed-loop system is close to the desired behavior, after only a few transients. The convergence rate
depends critically, on the choice of the parameter g.
 Nonlinear Control Structures 587
1.5
Reference model output
Plant output
1

0.5

–0.5

–1

–1.5
0 10 20 30 40 50 60 70 80 90 100
t
Fig 10.6 Simulation of the MRAC system of Fig. 10.5

Plots in Fig. 10.6 were generated by simulating the following sets of equations.
(i) y&m (t ) = −2 ym (t ) + 2r (t ); ym (0) = 0
This gives ym(t).
(ii) y& p (t ) = − y p (t ) + 0.5u(t ); y p (0) = 0
u (t ) = b(t )r (t ) − a (t ) y p (t )
db(t )
= −2r (t )e (t ); b (0) = 0.5
dt
da (t )
= 2 y p (t )e(t ); a (0) = 1
dt
e(t ) = y p (t ) − ym (t )
From this set of equations, we obtain u(t) and yp(t).

10.4 SYSTEM IDENTIFICATION AND GENERALIZED PREDICTIVE CONTROL

IN SELF-TUNING MODE

10.4.1 System Identification

The types of models that are needed for the design methods presented in this book, can be grouped into two
categories: transfer function model and state variable model. If we have a transfer function description, we
can obtain an equivalent state variable description and vice versa. These equivalent models are described by
certain parameters—the elements of F, g, c matrices of the state model
x( k + 1) = Fx( k ) + gu ( k )
y ( k ) = cx ( k )
x(k) : n ´ n state vector
 588 Digital Control and State Variable Methods
u(k) : scalar input
y(k) : scalar output
or the parameters ai and bj of the transfer function
m −1
Y ( z ) β1 z + β 2 z + L + β m +1
m
G( z) = = −
U ( z) z + α1 z + L + α n
n n 1

The category of such models gives us the parameteric description of the plant. The other category of
models such as frequency-response curves (Bode plots, polar plots, etc.), time-response curves, etc., gives
non-parametric description of the plant.
Plant models can be obtained from the first principles of physics. In many cases, however, it is not
possible to make a complete model only from physical knowledge. In these circumstances, the designer may
turn to the other source of information about plant dynamics—the data taken from experiments directly
conducted to excite the plant, and to measure the response. The process of constructing models from experi-
mental data is called system identification. In this section, we restrict our attention to identification of
discrete parametric models, which includes the following steps:
(i) Experimental planning
(ii) Selection of model structure
(iii) Parameter estimation
Experimental Planning The choice of experimental conditions for parameter estimation is of consider-
able importance. It is clear that the best experimental conditions are those that account for the final applica-
tion of the model. This may occur naturally in some cases; e.g., in adaptive control (discussed later in this
section), the model is adjusted under normal operating conditions. Many ‘classic’ methods depend strongly
on having specific input, e.g., sinusoid or impulse. There could be advantages in contriving such an artificial
experiment if it subjects the system to a rich and informative set of conditions, in the shortest possible time.
A requirement on the input signal is that it should sufficiently excite all the modes of the process.
One broad distinction in identification methods is between on-line and off-line experimentation. The
on-line methods give estimates recursively, as the measurements are obtained, and are the only alternative if
the identification is going to be used in an adaptive controller.
Selection of Model Structure The model structures are derived from prior knowledge of the plant. In
some cases the only a priori knowledge is that the plant can be described as a linear system in a particular
range. It is, then, natural to use general representations of linear systems.
Consider a SISO dynamic system with input {u(t)} and output {y(t)}. The sampled values of these signals
can be related through the linear difference equation
y ( k + n) + α1 y ( k + n − 1) + L + α n y ( k ) = β1u ( k + m) + β 2 u ( k + m − 1) + L + β m+1u ( k ); n ≥ m (10.27)
ai and bj are constant (unknown) parameters.
The number of parameters to be identified, depends on the order of the selected model, i.e., n in Eqn.
(10.27). The calculations can be arranged so that it is possible to make a recursion in the number of parameters
in the model. The methods of model-order selection are usually developed for the off-line solution.
The unknown process is not completely a black box. Some information about its dynamic behaviour is
known from basic principles and/or plant experience. Therefore, some estimate of the model’s order, and
some initial values for the unknown parameters, will be available. The more we know about the process, the
more effective the postulated model will be. Consequently, we should use all available information for its
development. The order of the postulated model is a very important factor. Remember, that complex models
of high order will not necessarily produce better controller designs and may burden the computational effort,
without tangible results.
 Nonlinear Control Structures 589
Equation (10.27) may be expressed as
y (k ) + α1 y (k − 1) + L + α n y (k − n) = β1u (k + m − n) + β 2 u (k + m − 1 − n) + L + β m +1u (k − n)
or
y (k ) + α1 y (k − 1) + L + α n y (k − n) = β1u (k − d ) + β 2 u (k − 1 − d ) + L + β n −d +1u (k − n) (10.28)
d = n – m ³ 0 is the relative degree or control delay.
We shall use operator notation for conveniently writing linear difference equations. Let z–1 be the back-
ward shift (or delay) operator:
z −1 y (k ) = y (k − 1) (10.29)
Then Eqn.(10.28) can be written as
A( z −1 ) y (k ) = B ( z −1 )u (k )
where A(z–1) and B(z–1) are polynomials in the delay operator:
A( z −1 ) = 1 + α1 z –1 + L + α n z – n
B ( z −1 ) = z − d ( β1 + β2 z −1 + L + βn −d +1 z −( n− d ) )
We shall present the parameter-estimation algorithms for the case of d = 1 without any loss of generality;
the results for any value of d easily follow.
For d = 1, we get the input-output model structure
A( z −1 ) y (k ) = B ( z −1 )u (k ) (10.30)
where
A( z −1 ) = 1 + α1 z –1 + L + α n z –1
B ( z −1 ) = β1 z −1 + β 2 z −2 + L + β n z − n
In the presence of the disturbance, model (10.30) takes the form
A( z −1 ) y (k ) = B ( z −1 )u (k ) + ε (k ) (10.31)
where e (k) is some disturbance of unspecified character.
The model (10.31) describes the dynamic relationship between the input and output signals, expressed in
terms of the parameter vector
θ = [α1 ... α n β1... β n ]T (10.32)
Introducing the vector of lagged input-output data,
φ( k ) = [ − y ( k − 1) − L − y ( k − n) u ( k − 1) L u ( k − n )]; (10.33)
Eqn. (10.31) can be rewritten as
y (k ) = φ(k )θ + ε (k ) (10.34)
A model structure should be selected (i) that has a minimal set of parameters and is yet equivalent to
the assumed plant description; (ii) whose parameters are uniquely determined by the observed data; and
(iii) which will make subsequent control design simple.
Parameter Estimation The dynamic relationship between the input and output of a scalar system is
given by the model (10.34). Ignoring random effects e (k) on data collection, we have
y ( k ) = φ ( k )θ (10.35)
where f (k) is given by Eqn. (10.33) and q is given by Eqn. (10.32).
Using the observations
{u (0), u (1),..., u ( N ), y (0), y (1),..., y ( N )}
we wish to compute the values of ai and bj in parameter vector q, which will fit the observed data.
 590 Digital Control and State Variable Methods
Thus, solving the parameter-estimation problem requires techniques for selecting a parameter estimate,
which best represents the given data. For this, we require some idea of the goodness of the fit of a proposed
value of q to the true q°. Because, by the very nature of the problem, q° is unknown, it is unrealistic to define
a direct parameter error between q and q°. We must define the error in a way that can be computed from
{u(k)} and {y(k)}.
Let e(k,q) be the equation error comprising the extent to which the equations of motion (10.35) fail to be
true for a specific value of q when used with the specific actual data:
e ( k , θ ) = y ( k ) − φ ( k )è (10.36)
A simple criterion representing the goodness of fit, of a proposed value of q, is given by
N
J (θ ) = ∑ e 2 ( k , θ ) (10.37)
k =1
The method called the Least Squares Method, based on minimizing the sum of the squares of the error,
is a very simple and effective method of parameter estimation.
Since y(k) depends on past data upto n earlier periods, the first error we can form is e(n,q ); the subsequent
errors being e(n + 1, q), ..., e(N, q):
e( n, è) = y ( n ) − φ( n )è
e(n + 1, è) = y (n + 1) − φ(n + 1)è
M
e ( N , è ) = y ( N ) − φ ( N )è
In vector-matrix notation,
e( N , è ) = y ( N ) − Ö ( N )è (10.38)
T
where e( N , è) = [e(n, è) e(n + 1, è) L e( N , è )]
Ö( N ) = [φT (n) φT (n + 1) L φT( N )]T
y( N ) = [ y (n) y (n + 1)L y ( N )]T
Note that e is (N – n + 1) ´ 1 vector, y is (N – n + 1) ´ 1 vector, F is (N – n + 1) ´ 2n matrix and q is
2n ´ 1 vector.
The principle of least squares says that the parameters should be selected in such a way that the perform-
ance measure
N
J (è ) = ∑ e2 (k , è ) = eT ( N , è)e( N , è ) (10.39)
k =n
is minimized
The performance measure J(q) can be written as
J (è ) = [y ( N ) − Ö ( N )è]T [y ( N ) − Ö ( N )è ]
= yT ( N ) y( N ) − èT ÖT ( N ) y ( N ) − y T ( N )Ö( N )è + èT ÖT ( N )Ö( N )è
= yT ( N )y ( N ) − èT ÖT ( N )y ( N ) − yT ( N )Ö( N )è + èT ÖT ( N )Ö ( N )è
+ yT ( N )Ö( N )(ÖT ( N )Ö( N )) −1 ÖT ( N ) y ( N ) −yT ( N )Ö( N )(ÖT ( N )Ö( N ))−1 ÖT ( N )y ( N )
(Note that we have simply added and subtracted the same terms under the assumption that [F T (N) F (N)] is
invertible).
Hence
J (è ) = y T ( N )[1 − (Ö ( N )(ÖT ( N )Ö ( N ))−1ÖT ( N )]y ( N )
+(è − (ÖT ( N )Ö( N )) −1 ÖT ( N )y ( N ))T ÖT ( N )Ö( N ) × (è − (ÖT ( N )Ö( N ))−1 ÖT ( N )y ( N )
 Nonlinear Control Structures 591
The first term in this equation is independent of q, so we cannot reduce J via this term. Hence, to get the
smallest value of J, we choose q so that the second term is zero. Denoting the value of q that achieves the
minimization of J by è̂ , we notice that
èˆ = [ÖT ( N )Ö( N )]−1 ÖT ( N )y ( N ) (10.40a)
= P( N ) Ö ( N ) y ( N )
T
(10.40b)
−1
where P( N ) = [Ö ( N )Ö( N ) ]
T

The least squares calculation for è̂ given by (10.40) is a ‘batch’ calculation since one has a batch of data
from which the matrix F, and vector y, are composed according to (10.38). In many cases, the observations
are obtained sequentially. If the least squares problem has been solved for N observations, it seems to be a
waste of computational resources to start from scratch when a new observation is obtained. Hence, it is
desirable to arrange the computations in such a way that the results obtained for N observations can be used
in order to get the estimates for (N + 1) observations. The algorithm for calculating the least-squares estimate
recursively is discussed below.
Let èˆ ( N ) denote the least-squares estimate based on N measurements. Then from (10.40)
èˆ ( N ) = [ÖT ( N )Ö ( N )]−1ÖT ( N ) y ( N )
It is assumed that the matrix [ ÖT ( N )Ö( N )] is nonsingular for all N. When an additional measurement is
obtained, a row is added to the matrix F and an element is added to the vector y. Hence,
⎡Ö ( N ) ⎤ ⎡ y( N ) ⎤
Ö ( N + 1) = ⎢ ⎥ ; y ( N + 1) = ⎢ ⎥
⎣ φ( N + 1) ⎦ ⎣ y ( N + 1) ⎦

The estimate èˆ ( N + 1) based on N + 1 measurements, can then be written as

èˆ ( N + 1) = [ÖT ( N + 1)Ö ( N + 1)]−1 ÖT ( N + 1)y ( N + 1)
= [ÖT ( N )Ö( N ) + φT ( N + 1)φ( N + 1)]−1[ÖT ( N )y ( N ) + φT ( N + 1) y ( N + 1)] (10.41)
−1
P( N + 1) = [Ö (N + 1 ) Ö( N + 1) ]
T

Then from (10.41), we obtain

P( N + 1) = [ P −1 ( N ) + φT ( N + 1) φ ( N + 1)]−1 (10.42)
We now need the inverse of a sum of two matrices. We will use the well-known matrix inversion lemma1
for this purpose.

1
Matrix inversion lemma is proved below.
−1
[A + BCD]⎨⎧ A −1 − A –1B ⎣⎡C−1 + DA −1B ⎦⎤ ⎫
DA −1 ⎬
⎩ ⎭
−1 −1
= I + BCDA −1 − B ⎡ C−1 + DA −1B ⎤ DA −1 − BCDA −1B ⎡C −1 + DA −1B ⎤ DA −1
⎣ ⎦ ⎣ ⎦
−1
= I + BCDA −1 − BC ⎡C −1 − DA −1B ⎤ ⎡C−1 − DA −1B ⎤ DA −1
⎣ ⎦⎣ ⎦
= I + BCDA −1 − BCDA −1
=I
 592 Digital Control and State Variable Methods
Let A, C and C–1 + DA–1 B be nonsingular square matrices; then
[ A + BCD]−1 = A−1 − A −1B[C−1 + DA−1B]−1 DA −1 (10.43)
To apply (10.43) to (10.42), we make the associations
A = P −1 ( N )
B = φΤ ( N + 1)
C =1
D = φ( N + 1)
Now, the following result can easily be established.
P( N + 1) = P( N ) − P( N )φT( N + 1)[1 + φ( N + 1)P( N ) × φT ( N + 1)]−1 φ( N + 1)P( N )
Substituting the expression for P (N + 1) into (10.41), we obtain
èˆ ( N + 1) = {P( N ) − P( N )φT ( N + 1)[1 + φ( N + 1)P( N )φT ( N + 1)]−1 ×
φ( N + 1)P ( N )}[ÖT ( N ) y( N ) + φT ( N + 1) y ( N + 1)]
= P ( N )ÖT ( N )y ( N ) + P( N )φT ( N + 1) y ( N + 1) − P( N )φT ( N + 1) ×
[1 + φ( N + 1) P( N )φT( N + 1)]−1 φ( N + 1) P( N )ÖT ( N )y ( N ) − P( N ) ×
φT ( N + 1)[1 + φ( N + 1)P ( N )φT ( N + 1)]−1 φ( N + 1)P ( N ) × φT ( N + 1) y ( N + 1)
−1
= èˆ ( N ) + P( N )φ ( N + 1)[1 + φ( N + 1)P( N )φ ( N + 1)] ×
T T

[1 + φ( N + 1) P( N )φT( N + 1)] y ( N + 1) − P( N )φT ( N + 1) ×

[1 + φ( N + 1)P ( N )φT( N + 1)]−1 φ( N + 1)èˆ ( N ) − P( N ) ×
φT ( N + 1)[1 + φ( N + 1)P ( N )φT ( N + 1)]−1 φ( N + 1)P ( N ) φ ( N + 1) y ( N + 1)
T

This gives
èˆ ( N + 1) = èˆ ( N ) + K ( N )[ y ( N + 1) − φ( N + 1)èˆ ( N )] (10.44a)
T T −1
K ( N ) = P ( N )φ ( N + 1)[1 + φ( N + 1)P ( N )φ ( N + 1)] (10.44b)
P( N + 1) = [1 − K ( N ) φ( N + 1)]P( N ) (10.44c)
Equations (10.44) give the recursive least squares algorithm. Notice that for a single-output system, no
matrix inversion is required.
The Eqns (10.44) have a strong intuitive appeal. The estimate q$ (N + 1) is obtained by adding a correction to
the previous estimate q$ (N). The correction is proportional to y ( N + 1) − φ( N + 1)èˆ ( N ) where the term
φ( N + 1)èˆ ( N ) is the expected output at the time N + 1, based on the previous data f(N + 1) and the previous
estimate q$ (N). Thus, the next estimate of q is given by the old estimate corrected by a term, linear in error,
between the observed output y(N + 1) and the predicted output f(N + 1) q$ (N). The components of the vector
K(N) are weighting factors that show how the correction and the previous estimate should be combined.
Replacing N by recursive parameter k in Eqns (10.44), we rewrite the recursive least squares (RLS)
algorithm as
èˆ (k + 1) = èˆ (k ) + K (k )[ y (k + 1) − φ(k + 1)èˆ (k )] (10.45a)
K (k ) = P(k )φT(k + 1)[1 + φ(k + 1) P(k )φT(k + 1)]−1 (10.45b)
P(k + 1) = [1 − K (k )φ(k + 1)]P(k ) (10.45c)
 Nonlinear Control Structures 593

Any recursive algorithm requires some initial value to be started up. In (10.44), we require q$ (N) and P(N)
(equivalently, in (10.45) we require q$ (k) and P(k)). We may collect a batch of N > 2n data values, and solve
the batch formula once for P(N) and q$ (N).
However, it is more common to start the recursion at k = 0 with P(0) = a I and q$ (0) = 0, where a is some
large constant. You may pick P(0) = a I, but choose q$ (0) to be the best guess that you have, at what the
parameter values are.
We have presented the least squares method ignoring random effects on data collection, i.e., e(k) in
Eqn. (10.34) has been neglected. If e(k) is white noise, the least squares estimate given by (10.45) converges to
the desired value. However, if e(k) is coloured noise, the least squares estimation usually gives a biased (wrong
mean value) estimate. This can be overcome by using various extensions of the least squares estimation.
We have seen that parameter estimation can be done either on-line or off-line. Off-line estimation may be
preferable if the parameters are constant, and there is sufficient time for estimation before control. However,
for parameters which vary(even though slowly) during operation, on-line parameter estimation is necessary
to keep track of the parameter values. Since problems in the adaptive control context usually involve slowly
time-varying parameters, on-line estimation methods are, thus, more relevant.
The main purpose of the on-line estimators is to provide parameter estimates for self-tuning control.

10.4.2 Self-Tuning Regulator

If the plant is imperfectly known, perhaps because of random time-varying parameters or because of the
effects of environmental changes on the plant’s dynamic characteristics, then the initial plant model and the
resulting control design will not be sufficient to obtain an acceptable performance for all time. It then be-
comes necessary to carry out plant-identification and control-design procedures continuously, or at intervals
of time, depending on how fast the plant parameters change. This “self-design” property of the system, to
compensate for unpredictable changes in the plant, is the aspect of performance that is usually considered in
defining an adaptive control system.
The identification of the dynamic characteristics of the plant should be accomplished without affecting the
normal operation of the system. To identify the plant model, we must impose a control signal on the plant and
analyze the system response. Identification may be made from normal operating data of the plant or by use of
test (additional) signals, such as sinusoidal ones of small amplitude. The plant should be in normal operation
during the test; the test signals imposed should not unduly disturb normal outputs. Furthermore, inputs and
system noise should not confuse the test. Normal inputs are ideal as test signals since no difficulties with
undesired outputs, or confusing inputs, will arise. However, identification with normal inputs is only possible
when they have adequate signal characteristics (bandwidth, amplitude, and so on) for proper identification.
Once the plant has been identified, a decision must be made about how the adjustable parameters (con-
troller characteristics) should be varied to maintain acceptable performance. The control signals are then
modified according to the plant identification and control decision. The three functions:
(i) plant identification,
(ii) control design based on the identification results, and
(iii) actuation based on the control design
can easily by implemented using a digital computer. Figure 10.7 shows a block diagram representation of the
adaptive control scheme. The system obtained is called a self-tuning regulator (STR) because it has facilities
for tuning its own parameters. The regulator can be thought of as being composed of two loops:
(i) The inner loop is the conventional feedback control loop consisting of the plant and the regulator.
(ii) The parameters of the regulator are adjusted on-line by the outer loop, which is composed of the
recursive-parameter estimator and design calculations.
 594 Digital Control and State Variable Methods
Process parameters
Design

Controller Estimation
parameters

Command
signal
Output
Controller Process
Control
signal

Fig. 10.7 Block diagram of a self-tuning regulator

A self-tuning regulator, therefore, consists of a recursive parameter estimator (plant identifier) coupled
with a control-design procedure, such that the currently estimated parameter values are used to provide
feedback-controller coefficients. At each sampling, an updated parameter estimate is generated and a con-
troller is designed, assuming that the current parameter estimate is actually the true value. The approach of
using the estimates as if they were the true parameters for the purpose of design, is called certainty equiva-
lence adaptive control.
From the block diagram of Fig. 10.7, one may jump to the false conclusion that such regulators can be
switched on and used blindly without any a priori considerations; the only requirement being a recursive
parameter estimation scheme and a design procedure. We have, no doubt, an array of parameter-estimation
schemes and an array of controller-design methods for plants with known parameters. However, all the
possible combinations may not have a self-tuning property, which requires that the performance of the regu-
lator coincides with the performance that would be obtained, if the system parameters were known exactly.
Before using a combination, important theoretical problems, such as stability and convergence, have to be
tackled. There are cases wherein self-tuning regulators have been used profitably, though some of their
properties have not been fully understood theoretically; on the other hand, bad choices have been disastrous
in some other cases.
So far, only a small number of available combinations have been explored from the stability, conver-
gence, and performance points of view. It is a striking fact, uncovered by Astrom and Wittenmark [132], that
in some circumstances a combination of simple least-squares estimation and minimum-variance control has
a self-tuning property. The same is true for some classes of pole-shifting regulators. Computer-based
controllers incorporating these concepts are now commercially available.

10.4.3 Generalized Predictive Control

Clarke, Mothadi and Tuffs [120,121] proposed an alternative—Generalized Predictive Control (GPC), to
pole placement and minimum variance designs used in self-tuning regulators. The argument for introducing
GPC in a self-tuning context was that it was based on a more flexible criterion than minimum variance
controllers without requiring an excessive amount of computations. Although it originated in an adaptive
control context, GPC has many attractive features which definitely makes it worthwhile considering even for
non-adaptive control structures.
The idea behind generalized predictive control is to minimize, at each iteration, a criterion of the following
type.
N2 Nu
J = ∑ [ y$( k + j | k ) – r (k + j )] + ρ ∑ [Δu(k + j – 1)]2
2
(10.46)
j = N1 j =1
 Nonlinear Control Structures 595

a f
where y$ k + j | k is an optimal system output prediction sequence performed with data known upto time k,
Du(k + j – 1) is a future control increment obtained from minimization of the cost function J, N1 and N2 are
the minimum and maximum prediction horizons, Nu is the control horizon and r is a weighting factor penal-
izing changes in controls. The horizons (N1, N2, Nu) and the weighting factor (r) are design parameters. The
reference trajectory r(k + j ) can be a known constant, or known future variations.
A prime characteristic that distinguishes predictive control from other controllers, is the idea of receding
horizon; at each sample, the control signal is determined to achieve a desired behavior in the following N2
time steps. At each sample instant k,
(i) the future set-point sequence r (k + j ) is calculated;
(ii) the prediction model is used to generate a set of predicted outputs y$ (k + j | k);
(iii) the cost function J (Eqn.(10.46)) of future errors and controls is minimized; and
(iv) the first element u(k) of the sequence is asserted and the appropriate data vectors shifted so that the
calculation can be repeated at the next sample instant.
Note that the effective control law is stationary (optimization problem can be solved beforehand and the
solution transformed into a fixed (non-adaptive) controller). However, in the self-tuned case, new estimates
of the plant parameters require the optimization problem be solved at each sample, on-line.
If the dead-time of the process is exactly known, there is no point in setting N1 to be less than dead-time
since there would then be superfluous calculations in that the corresponding outputs cannot be affected by
the first action u(k). If dead-time is not known or is variable, then N1 can be set to 1 with no loss of stability.
A large value of N2 (of the order of 10) is generally recommended, corresponding more closely to the rise
time of the plant. Nu is an important design parameter, which is smaller than prediction horizon, N2. The
principle is that only the first Nu future control inputs are determined; from that point on the control input is
assumed constant. For simple plants, a value of Nu of 1 gives generally acceptable control. A larger value of
Nu is desirable for complex systems. In the derivation that follows, N1 is set to 1 and N2 to N: the “output
horizon”.
The Predictor When considering regulation about a particular operating point, even a nonlinear plant
generally admits a locally-linearized model (refer Eqns (10.30)–(10.32)):
A( z -1 ) y( k ) = B( z -1 )u( k - 1) + e ( k ) (10.47a)
–1
where A and B are polynomials in the backward shift operator z :
A( z -1 ) = 1 + a 1 z -1 + L + a n z - n (10.47b)
-1 -1 -n
B( z ) = b 1 + b 2 z + L + b n z (10.47c)
e (k) = some disturbance of unspecified character.
The leading elements b 1, b 2, ..., of the polynomial B are set to zero to account for the dead-time of the
plant; and the trailing elements bn, bn–1,…, are set to zero if the degree of polynomial B is less than n.
Principal disturbances encountered in industrial applications are accommodated by modelling e (k) as a
white noise sequence independent of past control inputs. To obtain a controller with integral action, it is
further assumed that the term e (k) is modelled as integrated white noise:
x (k )
e (k ) = (10.48)
1 - z -1
when x( k ) is an uncorrelated random sequence. Combining with (10.47a), we obtain
A( z -1 ) y(k ) = B( z -1 )u(k - 1) + x ( k )/ D (10.49)
–1
where D is the differencing operator (1 – z ).
 596 Digital Control and State Variable Methods
Considering the time instant k + j, model (10.47) equivalently reads
A( z −1 )Δ y( k + j ) = B( z −1 )Δu(k + j − 1) + ξ(k + j ) (10.49a)
To systematically derive a j-step ahead predictor of output, the model is reorganized by introduction of
the following identity (Diophantine equation).
1 = DA( z -1 )E j ( z -1 ) + z - j Fj ( z -1 ) (10.50a)
where E j and Fj are polynomials uniquely defined, given the polynomial A and the prediction interval j;
deg ( E j ) = j - 1 and deg ( Fj ) = deg ( A) = n :
( ) j( ) j ( ) j j – 1)
E j = e0 + e1 z –1 + L + e j –1 z –( (10.50b)
Fj = f 0( J ) + f1( j ) z –1
+L + f n( j ) z – n (10.50c)
Multiplying Ej to both sides of (10.49) and using the identity (10.50a) gives
(1 - z - j Fj ) y( k + j ) = E j BDu( k + j - 1) + E j x ( k + j )
or y(k + j ) = E j BD u(k + j - 1) + Fj y(k ) + E j x (k + j ) (10.51)
Given that the sequence of future control inputs (i.e., u(k + i ) for i > 1) is known, and measured output
data upto time k is available, the optimal predictor for y(k + j ) is the expectation conditioned on the informa-
tion gathered upto time k (since Ej is of degree j – 1, the noise components are all in the future):
y$(k + j | k ) = G j Du(k + j – 1) + Fj y( k ) (10.52a)
where G j = Ej B is a polynomial of order n + j–1
= g0( j ) + g1( j ) z –1 + L + gn( j+) j –1 z – (n + j –1) (10.52b)
By multiplying B/AD to both sides of identity (10.50a), we obtain
B B
Gj = – z – j Fj
AD AD
B B
= – z – j f0( j ) + f1( j ) z –1 + L + fn( j ) z – n (10.52c)
AD AD
B B( z –1 )
The polynomial = represents the z-transform of the response y(k) of the process to
AD A( z –1 )(1 – z –1 )
unit step input.
Step response = g0 + g1z–1 + L + gj–1z–( j–1) + gj z– j + L (10.52d)
It is obvious that the first j terms in Gj are same as the j coefficients of the step response of the process.
This gives us one way of computing Gj for the prediction equation (10.52a). Both Gj and Fj may be com-
puted by recursion of the Diophantine equation (10.50a), as is explained below.
~
With A defined as DA, we have from (10.50a)
~
1 = E j A + z – j Fj (10.53a)
~
Since A is monic, the solution to
~
1 = E1 A + z –1 F1
is obviously
~
E1 = 1; F1 = z[1 – A]
 Nonlinear Control Structures 597
Assume now that the solution to (10.53a) for some j exists, and consider the equation for j + 1:
~
1 = E j +1 A + z –( j +1) Fj +1 (10.53b)
Subtracting the two gives
~
0 = A[ E j + 1 – E j ] + z – j [ z –1 Fj +1 – Fj ] (10.54a)
Since deg (Ej + 1 – Ej ) = deg (Ej + 1 ) = j, it turns out to be good idea to define
E j +1 – E j = E j + e (j j +1) z –1 (10.54b)
where e (j j +1) –j
specifies the coefficient to z in the polynomial Ej + 1. Using this, (10.54a) can be rewritten as
~ ~
0 = AE j + z – j [ z –1 Fj + 1 – Fj + Ae (j j + 1) ] (10.54c)
~
By again exploiting that A is monic, it is evident that E j = 0, leading to
E j + 1 = E j + e (j j + 1) z – j
Consequently,
~
z –1 Fj + 1 – Fj + Ae (j j +1) = 0
or
~
Fj +1 = z[ Fj – Ae (j j + 1) ]
Comparing the coefficients, we obtain
e (j j +1) = f 0( j )
~
fi( j +1) = fi(+j1) – a ( j)
i +1 f0 , i = 0, 1, K , n
The following equations provide a summary of the recursions of Diophantine equation:
E j +1 = E j + f0( j ) z – j (10.55a)
fi ( j +1)
= fi(+j1) ~ f ( j ) ; i = 0, 1, K , n
–a (10.55b)
i +1 0
with
j)
f n(+1 =0
~
E1 = 1, and F1 = z (1 – A) (10.55c)
The only unknown quantities in the prediction equation (10.52a) are now the future control inputs. In
order to derive the control law, it is necessary to separate these from the part of the expression containing
known (= past) data.
y$ ( k + 1) = ( G1 – g0 ) Δu( k ) + F1 y( k ) + g0 Δu( k ) (10.56a)
y$ ( k + j ) = z j – 1 [G j – G j ]Du( k ) + Fj y( k ) + G Du( k + j – 1); 1 < j £ N (10.56b)
where G j = g0 + g1 z –1 + L + g j – 1 z –( j – 1) (10.56c)

Deriving the Control Law First the predictions derived above are expressed in the following vector
notation.
y$ = G u ~+j (10.57)
where
y$ = [ y$ ( k + 1) y$ ( k + 2 ) L y$ ( k + N )]T
~ = [Δu(k )
u Δu( k + 1) L Δu(k + N – 1)]T u
j = [j ( k + 1) j ( k + 2 ) L j ( k + N )]
 598 Digital Control and State Variable Methods
with j (k + j ) = z j–1 [G j – G j ]Du(k ) + Fj y(k )
G is a matrix of dimension N ´ Nu:
LM g 0 0 L 0 OP
L 0
MM gM 1 g0
M L M
PP
G = M PP (10.58)
MM · · L g0

MMg M M M PP
NN –1 gN – 2 L gN – Nu PQ
With the reference trajectory expressed in the form of a vector
r = [r(k + 1) r(k + 2) L r(k + N)]T,
the expression of the cost function of (10.46) can be written as
J = (G ~
u + j – r )T (G ~ ~Tu
u + j – r ) + ru ~ (10.59)
The sequence of future controls is determined by setting the derivative of the cost function to zero:
¶J
~ = 2G T (G ~
u + j – r ) + 2 r~
u
¶u
T ~ T ~ = 0
= 2 G G u + 2 G ( j – r ) + 2 ru
or
~
u = [G T G + rI]–1 G T (r – j ) (10.60)
The matrix involved in the inversion is of the much reduced dimension Nu ´ Nu. In particular, if Nu = 1 (as
is usefully chosen for a ‘simple’ plant), this reduces to a scalar computation.

10.4.4 Application to a First-Order Control System

Consider a first-order system with a plant model of the form
y (k + 1) = f y (k ) + gu (k ); y (0)Δ y 0 (10.61)
where u is the control variable and y is the measured state (output); f and g are unknown coefficients.
An especially simple adaptive controller results by combining the least squares method of parameter
estimation with the generalized predictive controller. The least squares parameter-estimation algorithm re-
quires relatively small computational effort and has a reliable convergence, but is applicable only for small
noise-signal ratios. Several applications have shown that the combination of least squares parameter estima-
tion with generalized predictive control gives good results.
Let us assume that for the system given by Eqn. (10.61), the desired steady-state value for the controlled
variable y(k) is a constant reference input r. We select the generalized predictive control parameters: P = 0.1,
N = 4, Nu = 1.
If the system parameters were known, the feedback controller should take the form (10.60). Since the
parameters are assumed to be unknown, the least squares error estimates will be used in place of the true
values of f ° and g° of the parameters f and g. The parameter estimates fˆ and ĝ are derived from the input-
output measurements.
To simulate the system (refer Problem B.20 in Appendix B), the data values were obtained from
Eqn. (10.61) assuming the true parameters
f ° = 1.1052; g° = 0.0526
and sampling interval T = 0.1 sec.
 Nonlinear Control Structures 599
30
Plant output
25 Command signal

20

15

10

–5

–10

–15
0 50 100 150 200 250
Sampling instants
Fig. 10.8 Simulation results

⎡0⎤
With the initial estimate èˆ (0) = ⎢ ⎥ , P(0) = a I with large value of a , we use Equations (10.45) to
⎣0⎦
generate the new parameter estimate, and implement the generalized predictive control law. The plot of
Fig. 10.8 was generated using this procedure. The input signal is a square wave with amplitude 10. The
closed-loop system is close to the desired behaviour after a few transients.

10.4.5 Relations between MRAC and STR

As described above, MRAC control and STR arise from different perspectives; with the parameters in
MRAC systems being updated so as to minimize the tracking errors between the plant output and reference
model output, and the parameters in STR systems being updated so as to minimize the data-fitting error in
input-output measurements. However, there are strong relations between the two design methodologies.
Comparing Figs 10.4 and 10.7, we note that the two kinds of systems have both an inner loop for control and
an outer loop for parameters estimation. From a theoretical point of view, it can actually be shown that
MRAC and STR systems can be put under a unified framework.
The two methods can be quite different in terms of analysis and implementation. Compared with MRAC
systems, STR systems are more flexible because of the possibility of coupling various controllers with
various estimators (i.e., the separation of control and estimation). However, the stability and convergence of
self-tuning regulators are generally quite difficult to guarantee, often requiring the signals in the system to be
sufficiently rich so that the estimated parameters converge to the true parameters. If the signals are not very
rich (for example, if the reference signal is zero or a constant), the estimated parameters may not be close to
the true parameters, and the stability and convergence of the resulting control system may not be guaranteed.
In this situation, one must either introduce perturbation signals in the input, or somehow modify the control
law. In MRAC systems, however, the stability and tracking error convergence are usually guaranteed—
regardless of the richness of the signals.
 600 Digital Control and State Variable Methods

10.5 SLIDING MODE CONTROL

As discussed earlier, modelling inaccuracies can have strong adverse effects on nonlinear control systems.
Therefore, any practical design must address them explicitly. A major approach to dealing with model
uncertainly is adaptive control, which we have discussed earlier in this chapter. Another major approach is
the variable structure sliding mode control, which is the subject of this section.
For the class of problems to which it applies, sliding mode controller
design provides a systematic approach to the problem of maintaining
stability and consistent performance in the face of modelling impreci-
sions. Sliding mode control has been successfully applied to robot
manipulators, underwater vehicles, automotive engines, high-perform-
ance electric motors, and power systems.
We have, informally, already introduced the reader to variable struc-
ture sliding mode control systems in Section 9.10 (revisiting this section
will be helpful). In the following, a formal introduction is presented. For a
detailed account of the subject, refer Slotine and Li [126], and Zak [35].
Consider a simple pendulum of mass M, suspended by a string of
Mg
length l having negligible mass. Let q be the angular displacement as
(Gravity-restoring
force Mgl sin )
shown in Fig. 10.9. t represents the torque applied at the point of
suspension, which will be considered to be the control input to the
Fig. 10.9 Schematic of a system. Ignoring the effects of friction, the system can be represented
pendulum mathematically as
Jθ&&(t ) + Mgl sin θ (t ) = τ (t )
where g is the acceleration due to gravity, and J = Ml2 is the moment of inertia.
By appropriate scaling, the essential dynamics of the system are captured by
y (t ) = −α sin y (t ) + u (t )
&& (10.62)
where a is a positive scalar
Ignoring the nonlinear sine term, we get the following linear approximation of the pendulum equation:
y ( t ) = u (t )
&& (10.63a)
Choosing x1 = y and x2 = y& as state variables, we have
x&1 (t ) = x2 (t );
(10.63b)
x&2 (t ) = u ( t )
The linear control methodologies (pole placement, optimal control) attempt to minimize, in some sense,
the transfer functions relating the disturbances to the outputs of interest. Here we explore discontinuous con-
trol (variable structure control, refer Section 9.10) methodology for disturbance rejection. The system with two
control structures, corresponding to u = + 1 and u = – 1, is considered. The variable structure law governing the
dynamics is given by
⎧ −1 if σ ( x1 , x2 ) > 0
u (t ) = ⎨ (10.64a)
⎩ +1 if σ ( x1 , x2 ) < 0
where the switching function is defined by
σ ( x1 , x2 ) = λ x1 + x2 (10.64b)
 Nonlinear Control Structures 601
l is a positive design scalar. The reason for the use of the term “switching function” is clear, since the
function in Eqns (10.64) is used to decide which control structure is in use at any point (x1, x2) in the phase
plane. The expression in Eqn. (10.64a) is usually written more concisely as
u (t ) = − sgn(σ (t )) (10.64c)
where sgn (×) is the sign function. This function exhibits the property that
σ sgn(σ ) = σ
Figure 10.10a shows typical trajectories (parabolas) for u = ± k, and a typical switching line. Close to the
origin, on either side of the switching line, the trajectories point towards the line; an instant after the control
structure changes, the system trajectory will recross the switching line and the control structure must switch
back. Intuitively, high-frequency switching between the two control structures will take place as the system
trajectories repeatedly cross the line. This high frequency motion is described as chattering. If infinite-
frequency switching were possible, the motion would be trapped or constrained to remain on the line. The
motion when confined to the switching line satisfies the differential equation obtained from rearranging
s (x1, x2) = 0, namely
x2 = – l x1 (10.65a)
or y (t ) = − λ y (t )
& (10.65b)
This represents a first-order decay and the trajectories will ‘slide’ along the switching line to the origin.
Such a dynamical behaviour is described as sliding mode and the switching line is termed the sliding
surface. During sliding motion, the system behaves as a reduced-order system which is apparently independ-
ent of the control. The choice of the sliding surface, represented in our example by the parameter l, governs
the performance response whilst the control law itself, is designed to guarantee that trajectories are driven to
the ‘region’ of the sliding surface where the sliding motion takes place. To achieve this objective, the control
action is required to satisfy certain conditions, called reachability conditions.
x2

u=k
u = –k

x1

<0
x
x
=0
x >0

>0 =0

<0
Fig. 10.10a An illustration of the reachability conditions

To develop the reachability conditions and the region of sliding motion, we consider the system (10.63b)
with u given by (10.64), i.e., u = ± 1 corresponding to the two control structures. Figure 10.10b shows typical
 602 Digital Control and State Variable Methods
trajectories of the control system with these control structures, and a typical switching line. Assume that the
system under consideration starts with initial conditions corresponding to point A in Fig.10.10b. The control
switches when the representative point reaches B. By geometry of the situation, we see that the trajectory,
resulting from the reversal of the control at point B, will bring the representative point on a parabola much
closer to the origin. This will continue until the trajectory intersects the switching line at a point closer to the
origin than the points A1 and A2 which are points of intersection of the switching line s (x1, x2), with parabo-
las passing through the origin: x1 = – 12 x 2 | x 2 | . The coordinates of the points A1 and A2 are obtained as
F -2 2
,
Iand
F2 -2
,
I
, respectively. The region where the sliding motion takes place, is a part of the
H l2 l K H
l2 l K
switching line between the points A1 and A2 as is seen below.
σ ( t ) σ& ( t ) = σ ( λx&1 + x&2 )
= σ ( λx 2 + u)
= σ (λx2 − sgn(σ ))

= λ σ x2 − σ
Since
ls x 2 £ l s x 2
we have
s (t )s& (t ) £ l s x 2 - s
£ s (l x 2 - 1)

x2

A1

A x1

A2
u = –1
u=1
B
(x1, x2) = 0

Fig. 10.10b Typical trajectories of the closed-loop system (10.53)–(10.54)

For values of x2 satisfying the inequality

λ x2 < 1 , (10.66a)
we have
σ ( t )σ& ( t ) < 0;
(10.66b)
T
⎛ ∂σ ⎞ dx
σ& (t ) = ⎜ ⎟ = (∇σ )T x&
⎝ ∂x ⎠ dt
 Nonlinear Control Structures 603
Inequality (10.66b) is equivalently expressed by the two conditions:
lim s& = lim+ (Ñs )T x& < 0
s ® 0+ s ®0
(10.67)
and lim– s& = lim– (Ñs ) T x& > 0
s ®0 s ®0
These conditions, called reachability conditions, ensure that when l |x2 | < 1, the system trajectories on
either side of the line s (x1, x2) = 0 point towards the line. This is illustrated in Fig.10.10a.
The control action, rather than prescribing the dynamic performance, ensures, instead, that the
reachability conditions are satisfied. The choice of the sliding surface governs the system performance. It
should be noted that the control action required to satisfy reachability conditions is discontinuous in nature.
The double-integrator system of Eqns (10.63) is a linear approximation of the pendulum dynamics given
in Eqn. (10.62). An alternative interpretation is that the nonlinear term a sin y(t) is a disturbance or uncer-
tainly in the nominal double-integrator system. The key result is that, in finite time, the phase portrait inter-
cepts the sliding surface and is forced to remain there. The significance of this is that, once sliding is estab-
lished, the double-integrator system and the pendulum behave in an identical fashion, namely,
y& (t ) = −λ y (t )
The effect of disturbance or uncertainty in the nominal double-integrator system has been completely
rejected. As such, the closed-loop system is robust, i.e., it is insensitive to mismatches between the model
used for control law design, and the plant on which it will be implemented. The control action applied to the
plant does not utilize any knowledge of the uncertainty.
The concepts of sliding mode control, developed through an example, are summarized below.

10.5.1 Sliding Mode Control Algorithm

The problem is to regulate a dynamic system subject to parameter uncertainties and nonlinearities. A con-
troller is sought to force the system to reach, and subsequently remain on, a predefined surface (called the
sliding surface) within the state space. The dynamical behaviour of the system, when confined to the surface,
is called the sliding motion. The advantages of obtaining such a motion are two fold: firstly, there is a
reduction in order; and, secondly, the sliding motion is insensitive to parameter variations. The latter prop-
erty of invariance towards uncertainty makes the methodology an attractive one for designing robust control
for uncertain systems.
The design approach comprises two components.
· The design of a sliding surface in the state space, so that the reduced-order sliding motion satisfies the
specifications imposed on the design.
· The synthesis of a control law, discontinuous about the sliding surface, such that the trajectories of the
closed-loop motion are directed towards the surface.
The closed-loop dynamical behaviour obtained for using a variable structure control law, comprises two
distinct types of motion. The initial phase, often referred to as the reaching phase, occurs whilst the states
are being driven towards the sliding surface. This motion is, in general, affected by the disturbances present.
Only when the states reach the surface, and the sliding motion takes place, does the system become insensi-
tive to uncertainty.
A sliding mode will exist if, in the vicinity of the sliding surface, the state velocity vectors are directed
towards the surface. In such a case, the sliding surface attracts trajectories when they are in its vicinity; and,
once a trajectory intersects the sliding surface, it will stay on it thereafter.
A hyper surface
S : σ ( x1 , x2 ,K , xn ) = σ (x ) = 0 (10.68a)
 604 Digital Control and State Variable Methods
is attractive if
(i) any trajectory starting on the surface remains there, and
(ii) any trajectory starting outside the surface tends to it at least asymptotically
The following conditions (called reachability conditions) ensure that the motion of the state trajectory
x(t) of the single-input dynamical system
x& = f (x, u, t) (10.68b)
on either side of the sliding surface s (x) = 0, is toward the surface.
lim σ&+ < 0; and lim σ&– > 0
σ →0 σ →0
in some domain W of the state space
The two conditions may be combined to give
σ σ& < 0
in the neighbourhood of the sliding surface, i.e.,
dσ
lim σ < 0 (10.68c)
σ →0 dt
In the sliding mode, the trajectory remains on the hyper surface S for all times after hitting S, and so in the
sliding mode we require
T
d σ ⎛ ∂σ ⎞
=⎜ ⎟ x& = 0
dt ⎝ ∂x ⎠ (10.69)
σ (x) = 0
In general, if the reachability conditions are satisfied globally, i.e., W is the entire state space, then, since
1 d
2 s 2 = s s& < 0 ,
dt
it follows that
V(s ) = 1
2 s2 (10.70)
is a Lyapunov function for s (t).
Extension of these concepts to the multi-input situations is straight forward.
We illustrate these concepts through the design of a sliding mode controller for a two-link robot .

10.5.2 Sliding Mode Controller for a Two-Link Robot

The plant model, derived earlier in Section 10.2, is given by Eqns (10.6):
x&1 = θ& = x2
(10.71)
x& 2 = &&
θ = f (x) + g(x)τ
where
⎡ x11 ⎤ ⎡θ1 ⎤
⎢ ⎥ ⎢ ⎥
⎡x ⎤ x ⎡θ ⎤ θ 2 ⎡τ ⎤
x = ⎢ 1 ⎥ = ⎢ 12 ⎥ = ⎢ & ⎥ = ⎢ & ⎥ ; τ = ⎢ 1 ⎥
⎣ 2⎦
x ⎢ x21 ⎥ θ
⎣ ⎦ ⎢ 1⎥⎢θ ⎥ ⎣τ 2 ⎦
⎢ ⎥
⎢⎣ x22 ⎥⎦ ⎢⎣θ&2 ⎥⎦
q1 and q2 are the angles of the two links, defined in Fig. 10.1, and t1 and t2 are the torques applied by the
actuators to control the angles q1 and q2 , respectively.
The design of a variable structure sliding mode controller consists of two phases:
· Sliding (switching) surface design so as to achieve the desired system behaviour, when restricted to
the surface.
· Selecting feedback gains of the controller, so that the closed-loop system is stable to the sliding surface.
 Nonlinear Control Structures 605
Let us consider a specific design problem for the two-link robot under study: tracking the desired motion
trajectory qd (t).
Define the tracking error as
e (t ) = θ d (t ) − θ (t ) (10.72a)
Therefore & &
e& (t ) = θ (t ) − θ(t ); && && &&
e ( t ) = θ (t ) − θ ( t ) (10.72b)
d d
Defining x% 1 = e and x% 2 = e& , we can write robot dynamics (10.71) in the form (refer Eqns (10.15))
x&% 1 = x% 2
(10.73)
x&% = &&
2 θ − f (x) − g( x)τ
d
Here, we use linear sliding surface (although we can use nonlinear sliding surface as well) defined by the
equation
σ ( x% ) = λ x% 1 + I x% 2 = 0 (10.74a)
σ
⎡ 1 ⎤ ⎡ λ1 0 ⎤ ⎡ x11 ⎤ ⎡ 1 0 ⎤ ⎡ x21 ⎤ ⎡ 0 ⎤
( %
x) % %
or ⎢ = + = (10.74b)
% ⎦⎥ ⎣⎢ 0 λ2 ⎦⎥ ⎣⎢ x%12 ⎦⎥ ⎢⎣ 0 1 ⎥⎦ ⎣⎢ x%22 ⎦⎥ ⎢⎣ 0 ⎥⎦
⎣σ 2 ( x)
Note that
σ 1 (x% ) = λ1 x%11 + x%21 = λ1 (θ1d − θ1 ) + (θ&1d − θ&1 )
(10.75)
σ 2 (x% ) = λ2 x%12 + x%22 = λ2 (θ 2d − θ 2 ) + (θ&2d − θ&2 )
We have assumed the coefficient of x% 21 to be unity, without loss of generality. If this were not the case, we
could divide both sides of σ 1 (x% ) = 0 by the coefficient of x%21 , to ensure that coefficient of x%21 is 1. The same
argument applies to the selection of unity coefficient of x%22 .
We now combine equations of the plant and that of the sliding surface (Eqns (10.73) and (10.74)).
~
x 2 = –l ~
x1
Therefore
~
x1 = l ~
x1 (10.76)
The above equation describes the system dynamics in sliding (observe the order-reduction of system
dynamics in sliding). The response of the system in sliding is completely specified by an appropriate choice
of the parameters l1 and l2 of the switching surface. While in sliding, the system is not affected by model
uncertainties.
After designing a sliding surface, we construct a feedback controller. The controller objective is to
drive the plant state to the sliding surface, and maintain it on the surface for all subsequent time. We use a
generalized Lyapunov approach in constructing the controller. Specifically, we use a distance measure,
V = 12 σT σ = 12 (σ12 + σ 22 ) , from the sliding surface s as a Lyapunov function candidate. Then, we select the
controller so that the time derivative of the chosen Lyapunov function candidate, evaluated on the solution
of the controlled system, is negative-definite with respect to the switching surface; thus ensuring the motion
of the state trajectory to the surface, as it is illustrated in Fig. 10.10a. Our goal in to final t so that
d 1 T
σ σ = σ T σ& < 0 (10.77)
dt 2
σT σ& = σT ⎡⎣λ x%& 1 + I x&% 2 ⎤⎦ = σT [λ (x&1d − x&1 ) + (x& 2 d − f (x) − g(x)τ) ]
We consider the controller structure of the form
⎡ ⎧k sgn(σ 1 ) ⎫⎤
τ = g −1 (x) ⎢−f (x) + x& 2 d + λ (x& 1d − x& 1 ) + ⎨ 1 ⎬⎥ (10.78)
⎣ ⎩k2 sgn(σ 2 ) ⎭⎦
 606 Digital Control and State Variable Methods

where k1 > 0 and k2 > 0 are the gains to be determined so that the condition σT σ& < 0 is satisfied. To determine
these gains, we substitute t, given by (10.78), into the expression σ σ& .
T

⎡ k1 sgn(σ 1 ) ⎤
σT σ& = − [σ 1 σ 2 ] ⎢ ⎥
⎣ k 2 sgn(σ 2 ) ⎦
= −σ 1k1 sgn(σ 1 ) − σ 2 k2 sgn(σ 2 ) = − k1 σ 1 − k2 σ 2 < 0

1.5
Desired
Actual
1

0.5

–0
1

–0.5

–1

–1.5
0 1 2 3 4 5 6 7 8
t
(a)
1.5
Desired
Actual
1

0.5

–0
2

–0.5

–1

–1.5
0 1 2 3 4 5 6 7 8
t
(b)
Fig. 10.11 Desired and actual trajectories
 Nonlinear Control Structures 607
Thus, the sliding surface σ(x% ) = 0 is asymptotically attractive. The larger the values of gains, the faster
the trajectory converges to the sliding surface. Note that tolerance of sliding mode control to model
imprecision and disturbances, is high; satisfying asymptotic stability requirement, despite the presence of
uncertainties, ensures asymptotic tracking.
Simulation of this controller for the two-link robot arm (m1 = 1, m2 = 1, l1 = 1, l2 = 1, g = 9.8; èd1 (t ) =
sin(π t ) , èd 2 (t ) = cos(π t )) was done using MATLAB (refer Problem A.21 in Appendix A). Figures 10.11
show the tracking performance.

10.6 REINFORCEMENT LEARNING CONTROL

Complex dynamic systems have become integral and critical components of modern society. The unprec-
edented rate at which computers, networks, and other technologies are being developed, ensures that our
dependence on such systems will continue to increase. Examples of such systems include computer and
communication networks, transportation networks, banking and finance systems, electric power grid, and
systems for national defense. While advances in science and technology have enabled us to design and build
complex systems, comprehensive understanding how to control and optimize them is clearly lacking. The
mere existence of complex systems does not necessarily mean that they are operating under the most desir-
able conditions with enough robustness to withstand the limits of disturbances that inevitably arise. This was
made clear, for example, by the major power outage across dozens of cities in the Eastern United States and
Canada in August of 2003.
Many problem-solving architectures have been developed over the past thirty years—Feedback
Linearization, Model-Reference Adaptive Control, Self-Tuning Control, Sliding Mode Control, etc. These
architectures fall well short of the requirements of modern complex systems. While extensions and modifi-
cations to these architectures continue to be popular, other approaches are being considered as well. This
section presents a recently developed new control architecture that is based on reinforcement learning. Sev-
eral recent developments in reinforcement learning have substantially increased its viability as a general
approach to intelligent control.
Reinforcement-learning approach is based on the common sense idea that if an action is followed by a
satisfactory state of affairs, then the tendency to produce that action is strengthened, i.e., reinforced. Other-
wise, the tendency of the system to produce that action is weakened. In the following, we describe the
framework for reinforcement learning (RL) solutions to optimal control problems. To do so, we consider
building a reinforcement learning controller for stabilization of an inverted pendulum (Fig. 5.16). The con-
troller, or agent, has a set of sensors to observe the state of its environment (the dynamic system: inverted
pendulum mounted on a cart). For example, a controller may have sensors to measure angular position q and
velocity q& of the pendulum, and horizontal position z and velocity z& of the cart; and actions such as “push
left”, “push right”, by applying force of u newtons to the cart. Its task is to learn control strategy, or policy,
for choosing actions that achieve its goals.
It is generally assumed that the sets of possible states S and available actions A are both finite; it greatly
simplifies the theory. As we shall see later in this section, the reinforcement-learning theory backed by the
use of neural networks, makes learning tractable on the realistic control problems with continuous state
spaces (infinitely large set S).
Assume that our stabilization problem demands that the pendulum must be kept within ± 12º from vertical
and the cart must be kept within ± 2.4 m from the centre of the track. We define the following finite sets of
possible states S and actions A.
 608 Digital Control and State Variable Methods

State® 1 2 3 4 5 6
Pend. angle (deg) <–6 – 6 to – 1 – 1 to 0 0 to 1 1 to 6 >6
Pend. velocity < 50 – 50 to 50 > 50
Cart position (m) < – 0.8 – 0.8 to 0.8 > 0.8
Cart velocity < – 0.5 – 0.5 to 0.5 > 0.5

Actions® 1 2 3 4 5 6 7
Apply force of – 10 –6 –2 0 2 6 10
u newtons

Therefore, there are 6 ´ 3 ´ 3 ´ 3 = 162 states: s1, s2,…, s162. The action set size is seven: a1, a2, ..., a7.
Reinforcement learning is concerned with how such agents can learn successful control polices by experi-
menting in their environment. We assume that the goals of the agent can be defined by a cost function (perform-
ance measure) that assigns a numerical value—an immediate cost—to each distinct action the agent may take
from each distinct state. For example, the goal of stabilizing the inverted pendulum can be captured by assign-
ing a negative number (e.g., –1) to the state-action transitions that immediately result in failure state (12º <
pendulum angle < –12º), and a cost of zero to every other state-action transition. This cost function may be built
into the controller, or known only to an external critic who provides the cost value for each action performed by
the controller. The task of the controller is to perform sequence of actions, observe their consequences, and
learn a control policy. The control policy we desire is one that from any initial state, chooses actions that
minimize the cumulative cost—the cost accumulated over time by the agent.
This general setting for reinforcement learning is summarized in Fig. 10.12. Each time the agent performs
an action at in some state st, it receives a real-valued cost (reinforcement signal) rt that indicates the imme-
diate value of this state-action transition. This produces a sequence of states si, actions ai, and immediate
costs ri. The agent’s task is to learn a control policy p : S®A, that minimizes the cumulative cost.

10.6.1 Q-Learning
We now define one quite general formulation of the problem of learning sequential control strategies. This
formulation is based on Markov decision processes, and follows the problem illustrated in Fig. 10.12.
In a Markov Decision Process (MDP), the agent can perceive a set S of distinct states of its environment
and has a set A of actions that it can perform. At each discrete time step t, the agent senses the current state
st, chooses a current action at, and performs it. The environment responds by giving the agent a cost rt =
r(st, at) and by producing the succeeding state st + 1 = f (st, at). Hence the functions f and r are part of the
environment and are not necessarily known to the agent. In an MDP, the functions f (st, at) and r (st, at)
depend only on the current state and action, and not on earlier states and actions. State-space description of
the discrete-time dynamic system (environment) in Fig. 10.12 obeys this condition. In general, f and r may
be non-deterministic (stochastic) function, but we consider only the determinate case.
The task of the agent is to learn a policy p : S ®A, for selecting its next action at based on the current
observed state st; that is p (st ) = at. How shall we specify precisely which policy p we would like the agent to
learn? One obvious approach is to require the policy that produces the lowest possible cumulative cost for
the controller over time (greedy policy). To state this requirement more precisely, we define the cumulative
cost V p (st) achieved by following a policy p from an arbitrary initial state st as follows:
¥
V p (st ) = rt + g rt +1 + g 2 rt + 2 + L = å g i rt + i (10.79)
i=0
 Nonlinear Control Structures 609
where the sequence of costs rt + i is generated by beginning at state st and by repeatedly using the policy p to
select actions (i.e., at = p (st ), at + 1 = p (st + 1), ...). Here 0 £ g < 1 is a constant (called the discount factor)
that determines the relative value of delayed versus immediate costs. In particular, costs received i time steps
into the future are discounted exponentially by a factor of g i. Note that if we set g = 0, only the immediate
cost is considered. If we set g closer to 1, future costs are given greater emphasis relative to the immediate
cost. The meaning of g < 1 is that future costs matter to us less than the same costs paid at this present time.

Action at Environment (Discrete-time st + 1

Deterministic Dynamic system)
st + 1 = f (st, at); rt = r(st, at)

Reinforcement
rt

State st
Agent (Controller) at = p (st)

Fig. 10.12 A reinforcement learning framework for control problems

We speak of quantity Vp(s) defined by Eqn.(10.79) as the value of the state s; it is the discounted cumu-
lative cost incurred by the policy p from initial state s.
We are now in a position to state precisely the agent’s learning task. We require that the agent learn a
policy p that minimizes Vp(s) for all states s. We will call such a policy an optimal policy and denote it by p*.

p * º arg min V p (s)"s (10.80)

a
The “arg min” notation denotes the value of a at which Vp(s) is minimized
*
To simplify notation, we will refer to the value function V π (s) of such an optimal policy as V * (s ) . V *(s)—
the optimal value for a state s, gives the minimum discounted cumulative cost that the agent has to incur
starting form state s; that is, the discounted cumulative cost incurred by following the optimal policy begin-
ning at state s.
How can an agent learn an optimal policy p* from an arbitrary environment? It is difficult to learn
the function p*: S ® A directly from the information available to the learner; a sequence of immediate costs
r (si, ai) for i = 0, 1, 2, ... As we shall we, given this kind of information, it is easier to learn a numerical
evaluation function defined over states and actions; then implement the optimal policy in terms of this
evaluation function.
What evaluation function should the agent attempt to learn? One obvious choice is V*. The agent should
prefer state s1 our state s2 whenever V*(s1) < V*(s2), because the cumulative future cost will be lesser from s1.
Of course, the agent's policy must choose among actions, not among states. However, it can use V* in certain
settings to choose among actions as well, as follows. An optimal action in state s is the action a that minimizes
the sum of the immediate cost r(s, a) plus the value V* of the immediate successor state, discounted by g.
π * (s ) = arg min[r (s, a) + γ V * ( f (s, a))] (10.81)
a

Recall that f (s, a) denotes the state s¢ resulting from applying action a to state s. Thus, the agent can
acquire the optimal policy by learning V* for each state on-line, provided it has perfect knowledge of the
successor function f. This prior knowledge is not always available (“curse of modelling”).
 610 Digital Control and State Variable Methods
One of the important breakthroughs in reinforcement learning is the method known as Q-learning. In this
method, the real system underlying the decision problem, and not the system model, plays the role of the
successor function. To appreciate the idea, let us rewrite Eqn.(10.81) as
Q(s, a) = r(s, a) + g V*( f (s, a)) (10.82a)
p * (s ) = arg min Q (s, a) (10.82b)
a
The key is to learn the action-value function Q directly from the observed transitions. This can be accom-
plished through iterative approximation. To see how, notice the close relationship between Q and V*:
V*(s) = min Q(s, a′ ) (10.83)
a′
which allows rewriting Eqn.(10.82a) as
Q(s, a) = r(s, a) + γ min Q( f (s, a), a′ ) (10.84)
a′
This recursive definition of Q provides the basis for the algorithm that iteratively approximates Q. To
describe the algorithm, we will use the symbol Q$ to refer to learner’s estimate of the actual Q function.
Assume that the learner represents the estimate Q$ by a large table with a separate entry for each state-action
pair. The table entry for the pair (s, a) stores the value for Q$ (s, a)—the learner’s current estimate of the
actual but unknown value Q(s, a). The table can be initially filled with random values. The agent repeatedly
observes its current state, chooses some action a, executes this action; then observes the resulting cost r =
r(s, a) and the new state s¢ = f (s, a). It then updates the table entry for Q$ (s, a), following each such transition,
according to the rule:
Q$ (s, a) ← r + γ min Q$ ( s′, a′ ) (10.85)
a′

Note that rule uses the agent’s current Q$ values for the new state s¢ to refine its estimate of Q$ (s, a). This
rule is motivated by Eqn.(10.84), although the rule (10.85) concerns the agent’s approximation Q$ , whereas
Eqn.(10.84) applies to the actual Q function. While Eqn.(10.84) requires the knowledge of successor func-
tion f (s, a), the agent does not need to know this function to apply the rule (10.85). Instead, it executes the
action in its environment and then observes the resulting new state s¢ and cost r.
Will the algorithm (10.85) converge toward a Q$ equal to the true Q function? The answer is yes, under
certain conditions. We assume the agent selects actions in such a fashion that it visits every possible state-
action pair infinitely often. By this we mean that if action a is a legal action from state s, then over time the
agent must execute action a from state s repeatedly with nonzero frequency as the length of its action
sequence approaches infinity. The key idea underlying the proof of convergence is that the table entry
Q$ (s, a) with the largest error must have its error reduced by a factor of g whenever it is updated [149]. The
reason is that its new value depends only in part on error-prone Q$ estimates, with the remainder depending
on error-free observed immediate cost r.
Notice that algorithm (10.85) does not specify how actions are chosen by the agent. One obvious strategy
would be for the agent in state s to select the action a that minimizes Q$ (s, a), thereby exploiting the current
approximation Q$ by following a greedy policy. However, with this strategy, the agent runs the risk that it will
overcommit to actions that are found during early stages to have high Q$ values, while failing to explore other
actions that can have even higher values. In fact, the convergence condition requires that each state-action
transition occur infinitely often. This will clearly not occur if the agent always follows the greedy policy.
The Q-learning agent must therefore follow the policy of exploration and exploitation: exploration
ensures that all admissible state-action pairs are visited enough to satisfy the Q-learning convergence condi-
tion, and exploitation seeks to minimize the cost by following a greedy policy. An agent therefore must make
a trade off between its immediate good and its long-term well being. The “wacky” approach acts randomly,
 Nonlinear Control Structures 611
in the hope that it will eventually explore the entire environment, and the ‘greedy’ approach acts to minimize
the cost using current estimates. We need an approach somewhat between wackiness and greediness.
Many exploration schemes have been used in RL literature. The simplest one is to behave greedily most
of the time, but every once a while, say with small probability e, instead select an action at random, uni-
formly, independently of the action-value estimates. We call methods using this near-greedy action selection
rule e-greedy methods. An advantage of these methods is that in the limit as the number of iterations tends to
¥, every action will be sampled an infinite number of times, and thus ensuring convergence. This of course
implies that the probability of selecting the optimal action converges to (1– e), i.e., to near certainty.
What we need is a strategy that expands on Q-learning by providing a compromise between two conflict-
ing objectives:
· Exploration, which ensures that all admissible state-action pairs are explored often enough to satisfy
the Q-learning convergence condition.
· Exploitation, which seeks to minimize the cost by following a greedy policy.
The lookup table representation of Q$ -values is directly used to extract optimal action in each state (refer
Fig. 10.13). But very few decision or control problems in the real world fit exactly into this strategy for
solution; the number of possible states in the real world is often much too large to accommodate the compu-
tational and storage requirements.

State s¢ s¢
Action selector Look-up table
Plant agent (controller) ^ ( s , a)
Reinforcement r Q

Action a¢
^ s¢ ) a¢ Î A
Q(

Fig. 10.13 General controller structure

Early attempts in reinforcement learning were mostly based on a very simple approximation to the envi-
ronment. The true environment, however complex, was approximated as a finite-state system manageable by
a lookup-table representation of Q$ functions. Typical MDPs have a very large number of discrete states and
actions. Computational obstacles arise from the large number of possible state/action vectors, and the size of
the lookup table grows exponentially with the number of possible states; the situation pointing to the “curse
of dimensionality”.
The core of reinforcement learning is to use implicit representations of functions learned by agents
(instead of explicit representation in tabular form)—a form that allows one to calculate the output for any
input but that is usually much more compact than tabular form. We can exploit the universal approximation
property of neural networks and fuzzy inference systems to achieve this objective. In fact, any supervised
learning system with universal approximation property can be used [148].

10.6.2 Neural Q-Learning

Perhaps the most constraining assumption in our treatment of Q learning up to this point is that the function
Q$ (s, a) is represented as an explicit lookup table, with a distinct table entry for every distinct state-action
pair. Thus the algorithm we discussed makes no attempt to estimate the Q value for unseen state-action pairs
by generalizing from those that have been seen. This learning requirement is reflected in the convergence
 612 Digital Control and State Variable Methods
condition which assumes that every possible state-action pair is visited (infinitely often!). This is clearly an
unrealistic assumption in large or infinite spaces. As a result, more practical systems often combine function
approximation methods discussed later in Chapters 11 and 12, with the Q learning training rule described
earlier in this section.
It is easy to incorporate function approximation algorithms into Q learning algorithm. For example, we
may substitute a neural network for the look-up table and use each Q$ (s, a) update as a training example. We
could encode the state s as network input and train the network to output Q$ for each action a Î A. The
training rule given by Eqn.(10.85) based on lookup table, may be modified when neural network is substi-
tuted for the table. The modified rule given below needs background on neural networks (Chapter 11) for
proper appreciation.
Consider a Q learning agent in an MDP with bounded rewards r (s, a)" admissible state-action pairs
(s, a). The Q learning agent uses the training rule of Eqn.(10.85), initializes Q$ (s, a) to arbitrary finite values,
and uses discount factor g such that 0 £ g < 1.
Let Q$ n (s, a) denote the agent's hypothesis Q$ (s, a) for all admissible state-action pairs, following the nth
update. The Q-learning algorithm backs up the Q$ value for only a single state-action pair at each update
(time step of control), where the state-action pair consists of the observed current state and the action actu-
ally executed. For the current state-action pair (s, a), the Q$ n (s, a) value is updated on iteration n + 1 as per
the following rule:
Q$ (s, a) = Q$ (s, a) + h [r + g min Q$ (s¢ , a) – Q$ (s, a)]
n +1 n n n n (10.86)
a ÎA
where s¢ denotes the succeeding state and hn is the learning-rate parameter in iteration n for the current state-
action pair.
The update Eqn. (10.86) applies to the current state-action pair. For other admissible state-action pairs,
Q-values remain unchanged.
Treating the expression inside the square brackets of Eqn.(10.86) as the error signal involved in up-dating
the current value of Q$ n (s, a), we may identify the target (desired) value of Q$ at time step n as
Q$ target (s, a) = r + g min Q$ (s¢, a)
n n (10.87)
a ÎA
Figure 10.14 illustrates the neural Q-learning algorithm. Q$ n (s, a, w) denotes the approximation to Q$ n (s, a),
computed by means of a neural network. The current state s is input to the neural network with parameter vector
w, producing the output Q$ n (s, a, w) " a Î A. At each iteration of the algorithm, the weight vector w of
the neural network is changed slightly in a way that brings the output Q$ n (s, a, w) closer to the target value
Q$ ntarget (s, a) for the current (s, a) pair.

Fig. 10.14 Neural Q-learning

 Nonlinear Control Structures 613
Despite the success of these algorithms, for many tasks reinforcement learning fails to converge once a
generalizing function approximator is introduced. To see the difficulty, consider a neural network substitut-
ing an explicit table to represent Q$ . Note that if the learner updates the network to better fit the training Q$
value for a particular transition (s¢, a¢ ), the altered network weights may also change the Q$ estimates for
arbitrary other state-action pairs; in the process may increase the error in Q$ estimates for these other state-
action pairs. Neural Q learning has thus potential of divergence. Fortunately, the algorithm does converge
for large number of applications.

10.6.3 Temporal Difference Learning

If a non-equilibrium state is visited at time t, then the estimate Q$ (st) is updated based on what happens after
that visit. One way of updating would be to wait until equilibrium state is reached, and use the complete cost:
rt + g rt + 1 + g 2 rt + 2 + L
of the episode, assuming termination of the task once equilibrium state is reached (The control tasks are
usually continuing tasks, and an episode will, theoretically, consist of infinite number of steps).
The Q learning described earlier in the section, waits only until the next time step, and makes update
based on the observed cost rt, and an estimate of the remaining cost till the equilibrium state. What is the
space of methods lying between waiting till the equilibrium state or waiting only one step? One kind of
intermediate method would wait for intermediate number of steps: more than one, but less than all of them
until equilibrium state is reached. We thus have a range of methods based on the backing of different number
of samples of experience. These n-sample backup algorithms form a general class of Temporal Difference
(TD) algorithms. In this sense, Q learning is a special case of general class of TD learning.
To explore this issue further, recall that our Q learning algorithm (one-sample back up TD algorithm)
calculates a training value for Q$ (st, at) in terms of the observed cost for one step and the values for Q$ (st + 1,
a) where st + 1 is the result of applying action at to the state st. Let Q(1) (st, at) denote the training value
calculated by this one-step look ahead:
Q$ (1) (s , a ) ¬ Q$ (1) (s , a ) + h[r + g min Q$ (s , a) – Q$ (1) (s , a )]
t t t t t t +1 (10.88a)t t
a
Because this update rule uses the difference in Q values of successive states, it is a temporal difference or
TD equation. Notice that the update only involves the actual successor, where the actual equilibrium condi-
tions involve all possible next states.
The two-sample back up TD algorithm calculates a training value for Q$ (st, at) based on the observed cost
for two time steps.
Q$ ( 2 ) (s , a ) ¬ Q$ ( 2 ) (s , a ) + h[r + g r + g 2 min Q$ ( s , a) – Q$ ( 2 ) (s , a )]
t t t t t t +1 t +2(10.88b) t t
a
The TD equation for the general n-sample back up TD algorithm can, accordingly, be written.
Backups can be done not just toward any n-step costs, but toward any average of the n-step costs. For
example, a backup can be done toward a cost that is half of two-step cost and half of a four-step cost:
Q$ ave = 12 Q$ ( 2 ) + 12 Q$ ( 4 ) Any set of costs can be averaged in this way, as long as the wrights on the compo-
nent costs are positive and sum to 1.
A general method for blending these alternative estimates uses a constant 0 £ l £ 1 to combine the
estimates obtained from various look ahead in the following fashion:
Q$ l (s , a ) ¬ (1 – l )[Q (1) (s , a ) + l Q( 2 ) (s , a ) + l2 Q( 3) (s , a ) + L ]
t t t t t t t (10.89)
t

An equivalent recursive definition for Q$ l is

Q$ λ (st , at ) = rt + γ [(1 – λ ) min Q$ (st , at )] + λQλ ( st +1 , at +1 )] (10.90)
a
 614 Digital Control and State Variable Methods

Note that if l = 0, we have our original training estimate Q$ (1) , which considers only one-step discrepan-
cies in the Q$ estimates. As l is increased, the algorithm places increasing emphasis on discrepancies based
on more distant lookaheads. At the extreme value l = 1, only the observed rt + i values are considered, with
no contribution from the current Q$ estimate.
This is only a theoretical view of the TD algorithms; it is not directly implementable because it is acausal
which requires using at each step knowledge of what will happen many steps later. In a practical (causal)
view of TD algorithms, called TD(l) algorithms, Q$ -values are backed up based on information gained over
sequences of state transitions. One way to implement this kind of extension is to use the ‘eligibility trace’
idea to back up the Q$ -values of all the state-action pairs experienced in the past, with the magnitudes of the
backups decreasing to zero with increasing time in the past.
The eligibility trace provides a measure of how recently each state-action pair was selected by the
agent. Actions that are chosen often in frequently-encountered states will maintain an eligibility trace that
does not decay as fast as one that is seldom chosen/encountered. We can update the eligibility trace value
online as follows:
When a state-action pair is first visited, it initiates an additional memory variable, a state-action specific
eligibility trace et (s, a). On each step, the eligibility traces (nonnegative scalars) for all state-action pairs
decay by g l , and the eligibility trace of one pair visited on the step is incremented by 1:

et (s, a) =
RSγλe
t – 1 ( s, a ) + 1 if s = st and a = at
(10.91)
Tγλe
t – 1 ( s, a ) otherwise
where g is the discount rate and l (0 £ l £ 1) is the parameter we refer to as the trace-decay parameter. This
type of eligibility trace is called an accumulating trace because it accumulates each time the state-action pair
is visited; then fades away gradually when the pair is not visited, as illustrated in Fig. 10.15.
Accumulating eligibility trace

Time of visits to a state

Fig. 10.15

At each step, we look at the current TD error and assign it backward to each prior state-action pair
according to its eligibility trace at that time.
Q$ t + 1 ( s, a ) = Q$ t ( s, a ) + h t d t et ( s, a ) "s, a (10.92a)
where
d t = (rt + g [min Q$ t (s¢, a)] – Q$ t (s, a)) (10.92b)
a
s¢ is the successor state and ht is the learning-rate parameter at time step t for the state-action pair (s, a).
At any time, the traces record which state-action pairs have recently been visited, where “recently” is
defined in terms of l. When l = 0, then by (10.91), all traces are zero at t except for the trace corresponding
to st; backing up reduces to the one described earlier in Eqn.(10.86) for Q-learning. The learning methods
 Nonlinear Control Structures 615
with l = 0 are accordingly called TD(0)/Q(0). TD(0)/Q(0) is the case in which only one state preceding the
current state is changed by the TD error.
In general TD(l)/Q(l) methods, the parameter l characterizes how far into the past the methods look in
determining the backup. For values of l = 0, only one preceding state is changed. For larger values of l, but
still l < 1, more of the preceding states are changed but each more temporally distant state is changed less
because the eligibility trace is smaller. We say that the earlier states are given less credit for the TD error. If
l = 1, then the credit given to earlier states falls only by g per step.
It is computationally more expensive to execute general TD(l )/Q(l ), though it often converges faster for
large l.
The objective set for this section was to introduce the reader to the emerging nonlinear control structure.
For details refer [147–150].

Problems
10.1 The system
x&1 = x1x2 + x3
x&2 = −2 x2 + x1u
x&3 = sin x1 + 2 x1x2 + u
y = x1
can be transformed to Brunovsky form by differentiating y(t) repeatedly, and substituting state derivatives from
the given system equations, until the control input u(t) appears:
y& = x&1 = x1x2 + x3
y = x1x&2 + x&1 x2 + x&3
&&
= ⎡sin x1 + x2 x3 + x1x22 ⎤ + ⎡1 + x12 ⎤ u
⎣ ⎦ ⎣ ⎦
≡ f ( x) + g ( x)u; x = [ x1 x2 x3 ]T
Defining variables as z1 ≡ y, z2 ≡ y& , we obtain
z&1 = z2
z&2 = f (x ) + g (x )u
This may be converted to a linear system by redefinition of the input as
v(t ) ≡ f ( x) + g (x )u (t )
so that
1
u (t ) ≡ (− f ( x) + v(t ))
g (x )
for then one obtains
z&1 = z2 ; z&2 = v
which is equivalent to
y=v
&&
With a PD tracking control
v = &&
yd + K D e& + K Pe
where tracking error is defined as
e(t ) ≡ yd (t ) − y(t ); yd (t ) is the desired trajectory;
616 Digital Control and State Variable Methods
the closed-loop system becomes
e + K D e& + K P e = 0
&&
The complete controller implied by this feedback linearization technique, is given by
1
u (t ) = (− f ( x) + &&yd + K De& + K P e)
g (x )
(a) Draw the structure of the feedback linearization controller showing PD outer loop and nonlinear inner loop.
(b) Select gains KP and KD so that the closed-loop system has a natural frequency of wn = 10 rad/sec, and a
damping ratio of z = 0.707.
(c) Suppose that it is desired for the plant output y(t) to follow the trajectory yd = sin (2pt). Simulate the system
and plot actual output y(t), desired output yd (t), and the tracking error e(t); given x(0) = [1 1 1]T.
10.2 One useful method for specifying system performance is by means of a model that will produce the desired
output for a given input. The model need not be actual hardware. It can only be a mathematical model simulated
on a computer. In a model-reference control system, the outputs of the model and that of the plant are compared
and the difference is used to generate the control signals.
Consider a nonlinear, time-varying plant described by
⎡ x&1 ⎤ ⎡ 0 1 ⎤ ⎡ x1 ⎤ ⎡ 0⎤
x& = ⎢ ⎥ = ⎢ ⎥ ⎢ ⎥ + ⎢ ⎥ u = f ( x, u , t )
⎢⎣ x&2 ⎥⎦ ⎣⎢ −b − a (t ) x2 ⎦⎥ ⎢⎣ x2 ⎥⎦ ⎣1 ⎦
where a(t) is time-varying and b is a positive constant.
Assume the reference model equation to be
⎡ x&m1 ⎤ ⎡ 0 1 ⎤ ⎡ xm1 ⎤ ⎡ 0 ⎤
x& m = ⎢ ⎥=⎢ 2 ⎥⎢ ⎥ + ⎢ ⎥v
⎢⎣ x&m2 ⎥⎦ ⎢⎣ −ωn −2ζωn ⎥⎦ ⎢⎣ xm2 ⎥⎦ ⎢⎣ωn2 ⎥⎦

= A mx m + b mv
The error vector
e = xm − x
Define a function
V (e) = eT Pe
where P is positive-definite, real, symmetric matrix.
.
Then V (e) is defined as
.
V (e) = eT ( ATm P + PAm )e + 2M
where
M = eT P [ Am x – f ( x, u, t ) + bm v]
The assumed V(e) function is a Lyapunov function if
T
(i) A m P + PA m = − Q is a negative-definite matrix; and
(ii) the control u can be chosen to make the scalar quatity M nonpositive.
Choosing the matrix Q to be
⎡q 0 ⎤
Q = ⎢ 11 ⎥ = positive definite,
⎣ 0 q22 ⎦
we obtain
.
(
V (e) = − q11e12 + q22e22 + 2M )
 Nonlinear Control Structures 617
where
⎡p p12 ⎤ ⎧⎪ ⎡ 0 1 ⎤ ⎡ x1 ⎤ ⎡ 0 1 ⎤ ⎡ x1 ⎤ ⎡ 0 ⎤ ⎡ 0 ⎤ ⎫⎪
M = [e1 e2 ] ⎢ 11 ⎥ ⎨⎢ 2 ⎥⎢ ⎥−⎢ ⎥⎢ ⎥− +⎢ ⎥⎬
⎣ p12 p22 ⎦ ⎪ ⎢⎣ −ω n −2ζω n ⎥⎦ ⎢ x ⎥ ⎣ −b −2ζωn ⎦ ⎢ x ⎥ ⎣⎢u ⎦⎥ ⎢⎣ω n2 v⎥⎦ ⎪
⎩ ⎣ 2⎦ ⎣ 2⎦ ⎭
= (e1 p12 + e2 p22 ) ⎡−(ω n − b) x1 − 2ζω n x2 + a(t ) x2 + ω n v − u ⎤
2 2 2
⎣ ⎦
If we choose u so that
u(t ) = −(ω n2 − b) x1 − 2ζω n x2 + ω n2 v + am x22 sign ( e1 p12 + e2 p22 ); am = max | a(t ) |
then
M = (e1 p12 + e2 p22 )[ a(t ) − amsign(e1 p12 + e2 p22 )] x22
= nonpositive
(a) Draw a block diagram representing the structure of the model-reference adaptive control system.
(b) For the parameters: a(t)= 0.2 sin t, b = 8, z = 0.7, wn = 4; simulate the closed-loop system and plot
x (t ) , xm (t ) , and e (t ).
10.3 Consider the adaptive control design problem for a plant, approximately represented by a first-order differential
equation
y& = – a p y + b p u
where y(t) is the plant output, u(t) is its input, and ap and bp are constant plant parameters (unknown to the
adaptive controller). The desired performance of the control system is specified by a first-order reference model
y&m = −am ym + bm r
where am and bm are known constant parameters, and r(t) is a bounded external reference signal. Using Lyapunov
synthesis approach, formulate a control law, and an adaptation law, such that the resulting model-following
error y(t) – ym(t), asymptotically converges to zero.
Simulate the MRAC system with ap = 1, bp =2, am = 3 and bm = 3, adaptation gain g = 1.5; initial values of both
parameters of the controller are chosen to be 0, indicating no a priori knowledge, and the initial conditions of
the plant and the model are both zero. Use two different reference signals in the simulation: r (t) = 2, and r (t) =
2 sin (3t).
10.4 The following data were collected from a cell concentration sensor, measuring absorbance in a biochemical
stream. The input u is the flow rate deviation (in dimensionless units) and the sensor output y is given in volts.
The flow rate (input) is piecewise constant between sampling instants. The process is not at steady state initially;
so y can change even though u = 0. Fit a first-order model
y ( k ) = a1 y (k − 1) + b1u( k − 1)
to the data using the least-squares approach. Plot the model response and the actual data.
Time(sec) u y
0 0 3.000
1 3 2.456
2 2 5.274
3 1 6.493
4 0 6.404
5 0 5.243
6 0 4.293
7 0 3.514
8 0 2.877
9 0 2.356
10 0 1.929
618 Digital Control and State Variable Methods
10.5 Step test data have been obtained for the off-gas CO2 concentration response—obtained by changing the feed
rate to a bioreactor. At k = 0, a unit step change in input u occurs, but the output change at the first sample
(k = 1) is not observed until the next sampling instant. The data is given in the table below.
Estimate the model parameters in the second-order difference equation
y ( k ) = a1 y ( k − 1) + a2 y (k − 2) + b1u ( k − 1) + b2u (k − 2)
from the input-output data using the least-squares approach. Plot the model response and the actual data.

k 0 1 2 3 4 5 6 7 8 9 10
y(k) 0 0.058 0.217 0.360 0.488 0.6 0.692 0.772 0.833 0.888 0.925

10.6 The following data were collected for a process:

Time(sec) Input u Output y
0 1 4.0000
1 1 –2.0000
2 0 –1.0000
3 1 8.5000
4 1 –9.7500
5 1 1.6250
6 1 21.0625
7 0 –30.8438
8 1 7.1406
9 0 51.9766
10 1 –89.2461

Fit a second-order model

y ( k ) + a1 y ( k − 1) + a2 y ( k − 2) = b1u( k − 1) + b2u (k − 2)
to the data using the least-squares approach. Plot the model response and the actual data.
Simulate a self-tuner based on least squares estimation of the parameters of the model, and the pole-placement
design given in Chapter 7.The system is required to track a constant input of amplitude 10; the dynamics speci-
fied as z = 0.5, wn = 1.
10.7 Consider a nonlinear system described by the equation
x + a(t ) x& 2 cos3x = u
&&
where a(t) in unknown but satisfies
1 ≤ a (t ) ≤ 2
With the nominal value of a(t) = 1.5, we have
x = f + u; f = −1.5 x& 2 cos3 x
&&
In order to have the system track x(t) = xd (t) = sin (p t /2), we define a sliding surface
σ = x&% + λ x% ; x% = x − x d
We then have
x − &&xd + λ x%& = f + u − &&
σ& = && xd + λ x%&
Selecting
xd − λ x%& − k sgn(σ ); k > 0,
u = − f + &&
we get
σ& = −k sgn(σ )
 Nonlinear Control Structures 619

We consider the function V = 12 σ 2 . Note that V is positive-definite with respect to the sliding surface (s = 0).
The time derivative of V, evaluated on the trajectories of the closed-loop system, is
.
V = σσ&
= −σ k sgn(σ ) = − k | σ | < 0
Thus, the sliding surface is asymptotically attractive, and the system, restricted to the sliding surface, can be
made asymptotically stable by an appropriate choice of the parameter l of the sliding surface.
Simulate the system, and plot tracking error and control law with
λ = 2, k = 0.1, a(t ) =| sin t | +1
 Neural Networks
11
11.1 INTRODUCTION

The goal of mathematical modelling is to provide a set of equations that purports to describe interrelations
between the system quantities, as well as the relations between these quantities and external inputs. We can
use different types of equations and their combinations like algebraic, differential (ordinary/partial), differ-
ence, integral, or functional equations. A mathematical model can be viewed as “a mathematical representa-
tion of the significant relevant aspects of a physical system (significance and relevance being in relation to
an application where the model is to be used)”.
When, devising algebraic, differential, difference equations (or any other model from first principles) is
not feasible, one seeks other avenues to obtain analytical models. Such models are devised by solving two
cardinal problems in modern science and engineering:
· Learning from experimental data (examples, samples, measurements, patterns, or observations) by
neural network.
· Embedding existing structured human knowledge (experience, expertise, heuristics) into workable
mathematics by fuzzy logic models.
Whenever there is an analytical model using a reasonable number of equations that can solve the given
problem in a reasonable time, at a reasonable cost, and with reasonable accuracy, there is no need to resort
to learning from experimental data or embedding structured human knowledge into fuzzy logic models.
Today, however, the two approaches are vital tools when at least one of these criteria is not fulfilled. There
are many such instances in contemporary science and engineering.
Neural networks and fuzzy logic models are two of the most important constituents of soft computing.
Soft computing is not a closed and clearly defined discipline at present. It includes an emerging and, more or
less, established family of problem-solving methods that attempt to mimic the intelligence found in nature.
In addition to neural networks and fuzzy logic models, there are, for example, support vector machines,
rough sets, genetic or evolutionary algorithms—and other methods in soft computing toolkit.
We have said that soft computing should mimic the intelligence found in nature. What is the character of
nature’s intelligence? Is it precise, quantitative, rigorous and computational? Just looking at human beings, the
most intelligent species, the answer is negative. We are very bad at calculations or at any kind of computing. A
negligible percentage of human beings can multiply two three-digit numbers in their heads. The basic function
 Neural Networks 621
of human intelligence is to ensure survival in nature, not to perform precise calculations. The human brain can
process millions of visual, acoustic, olfactory (concerned with smelling), tactile (the sense of touch), and motor
data, and it shows astonishing abilities to learn from experience, generalize from learned rules, recognize
patterns and make decisions. We want to transfer some of the human mental faculties of learning, generalizing,
memorizing, and predicting into our models, algorithms, smart machines, and intelligent artificial systems in
order to enable them to survive in highly technological environment, that is, to solve given tasks based on
previous experience with reasonable accuracy, at reasonable cost and in a reasonable amount of time.
The basic premises of soft computing are:
· The real world is pervasively imprecise and uncertain.
· Precision and certainty carry a cost.
The guiding principle of soft computing, which follows from these premises is:
· Exploit tolerance for imprecision, uncertainty and partial truth to achieve tractability, robustness and
low solution costs.
This guiding principle of soft computing differs strongly from that of classical hard computing which
requires precision, certainty, and rigour. Many contemporary problems do not lend themselves to precise
solutions within the framework of classical hard computing, for instance, recognition problems (handwrit-
ing, speech, objects, and images), computer graphics, mobile robot coordination, and data compression. To
be able to deal with such problems, there is often no choice but to accept solutions that are suboptimal and
inexact. In addition, even when precise solutions can be obtained, their cost is generally much higher than
that of solutions that are imprecise and yet yield results within the range of acceptability.

11.1.1 Learning from Experimental Data

Attempting to incorporate humanlike abilities into software solutions is not an easy task. Only recently, after
an attempt to analyze an ocean of data obtained from various sensors, it became clear how complex are the
problems our senses routinely solve, and how difficult it is to replicate, in software, even the simplest aspects
of human information processing. How, for example, one can make machines “see”, where “see” means to
recognize different objects and classify them into different classes. For smart machines to recognize or to
make decisions, they must be trained first on a set of training examples. Each new smart machine (software)
should be able to learn the problem in its areas of operation.
A machine learning problem setting is as follows: there is some unknown nonlinear dependency (map-
ping, function) y = f (x) between some high-dimensional input vector x and scalar y (or vector output y).
There is no information about the underlying joint probability functions. The only information available is a
training data set {x(p) , y(p) ; p = 1, 2, ..., P}, where P stands for the number of training data pairs.
Artificial neural networks are composed of many computing units popularly (but perhaps misleadingly)
called neurons. The strength of the connection, or link, between input and neurons or between two neurons
is called the weight. The number of neurons, their link structure and the corresponding values of the weights
w1, w2, ..., are the subjects of learning procedure in neural networks. Depending upon the problem, the
weights have different physical meanings, and sometimes it is hard to find any physical meaning at all.
Neural network learning is thus a “black box” design situation ( Fig. 11.1a) in which the process is entirely
unknown but there are known examples {x(p), y(p); p = 1, 2, ..., P}; measurements, records, observations,
samples, or data pairs.

11.1.2 Embedding Existing Structured Human Knowledge

Another machine learning problem setting is to emulate human thought processes that are imprecise,
deliberate, uncertain and usually expressed in linguistic terms. In addition, human ways of reasoning are
approximate, non-quantitative, linguistic, and dispositional (usually qualified).
 622 Digital Control and State Variable Methods

x1
x2 maps to
y
..
.
xn
Input Inference
x y

Controlling the distance between two cars

w1 w7
Rule 1: IF the distance is small AND
x1 w2 w8 w13 speed is medium, THEN force on brake
w9 y should be large.
w3
w4 w10 w14
Rule 2: IF the distance is medium AND
x2 w5 w11 speed is large, THEN force on brake
w6 w12
should be medium.

(a) (b)
x y

(c)
Fig. 11.1 Neural networks and fuzzy logic modelling as examples of ‘black box’ ‘white box’,
and ‘grey box’ modelling approaches

Why is it that way? It is a consequence of the fact that the world we live in is not a binary world. There are
many states between old and young, good and bad, ill and healthy, sad and happy, cold and hot, no and yes,
short and tall, black and white, and so on. Changes between these different extremes are gradual and have a
great deal of ambiguity. Language serves as a way of expressing imprecise knowledge, and the tolerance for
imprecision about the vague environment we live in. Most human knowledge is fuzzy and expressed in terms
that are usually without quantitative meaning. Fuzzy logic is a tool for transforming such linguistically
expressed knowledge into workable algorithm called a fuzzy logic model. In its newest incarnation, fuzzy logic
is called “computing with words”.
The point of departure in fuzzy logic is the existence of a human solution to the problem. If there is no
human solution, there will be no knowledge to model and, consequently, no sense in applying fuzzy logic.
However, the existence of human solution is not sufficient. One must be able to articulate to structure the
human solution in the language of fuzzy rules. These are typically IF-THEN rules. Almost all structured
human knowledge can be expressed in the form of IF-THEN rules. The fuzzy logic modelling is thus a
“white box” design situation in which the solution to the problem is known, that is, structured human knowl-
edge (experience, expertise, heuristics) about the process exists (Fig. 11.1b).

11.1.3 Integrating Learning with Knowledge

Neural networks and fuzzy logic models are modelling tools. They perform in the same way after the learn-
ing stage of neural networks or the embedding of human knowledge about some specific task in fuzzy-logic
structure, is finished. Whether the more appropriate tool for solving a given problem is a neural network or
a fuzzy logic model, depends upon the availability of previous knowledge about the system to be modelled
 Neural Networks 623
and the amount of measured data. The less previous knowledge exists, the more likely it is that a neural
network approach will be used to attempt a solution. The more knowledge available, the more suitable the
problem will be for fuzzy logic modelling.
Through integration of the techniques of fuzzy logic models and neural networks, we can reap the
benefits of both the fuzzy logic models and the neural networks. One such integrated system, a neuro-fuzzy
system, transforms the burden of designing fuzzy logic systems to the training and learning of neural
networks. That is, the neural networks provide learning abilities to the fuzzy logic systems. Neuro-fuzzy
systems are functionally fuzzy logic models; they only utilize learning ability of neural networks to realize
the key components of the fuzzy logic model. Integrated systems may also be formed by incorporating fuzzy
logic into the neural network models. In such an integration, called a fuzzy-neural network, the numerical
parameters (such as input-output data, weights, etc.) of a neural network are fuzzified. Fuzzy-neural
networks are fuzzified neural networks, and thus are functionally neural networks.
Instances involving some knowledge and some data correspond to “grey box” design situation (Fig.11.1c)
covered by the paradigm of neuro-fuzzy and fuzzy-neural models.
This chapter is devoted to learning from experimental data by neural networks. A brief introduction to
support vector machines will also be given in this chapter. Embedding existing structured human knowledge
into fuzzy logic models and neuro-fuzzy models, will be the subject of discussion in the next chapter. A brief
presentation of genetic algorithm will also be given in the next chapter. Neural networks and fuzzy logic
models are modelling tools, whereas genetic algorithm is an optimization algorithm that can be applied for
parameter adjustment during the learning (training, adaptation) phase of neural or fuzzy models.

11.1.4 Neural Networks

Historically, research in artificial neural networks was inspired by the desire to produce artificial systems
capable of sophisticated ‘intelligent’ processing similar to the human brain. The science of artificial neural
networks made its first significant appearance in 1943 when Warren McCulloch and Walter Pitts published
their study in this field. They suggested a simple neuron model (known today as MP artificial neural model)
and implemented it as an electrical circuit. In 1949, Donald Hebb highlighted the connection between psy-
chology and physiology, pointing out that a neural pathway is reinforced each time it is used. Hebb’s learn-
ing rule, as it is sometimes known, is still used and quoted today. Improvements in hardware and software in
the 1950s ushered in the age of computer simulation. It became possible to test theories about nervous
system functions. Research expanded; neural network terminology came into its own.
The perceptron is the earliest of the neural network paradigms. Frank Rosenblatt built this learning ma-
chine device in hardware in 1958 and caused quite a stir.
The perceptron has been a fundamental building block for more powerful models, such as the ADALINE
(ADAptive LINear Elements) and MEDALINE (Multiple ADALINEs in parallel), developed by Bernard
Widrow and Marcian Hoff in 1959. Their learning rule, sometimes known as Widrow-Hoff rule, was simple
yet elegant.
Affected by the predominately rosy outlook of the time, some people exaggerated the potential of neural
networks. Biological comparisons were blown out of proportion. In 1969, in the midst of many outrageous
claims, Marvin Minsky and Seymour Papert published ‘Perceptrons’, an influential book condemning
Rosenblatt’s perceptron. The limitations of the perceptron were significant; the charge was that it could not
solve any ‘interesting’problems. It brought to a halt, much of the activity in neural network research.
Nevertheless, a few dedicated scientists such as Teuvo Kohonen and Stephen Grossberg, continued their
efforts. In 1982, John Hopfield introduced a recurrent-type neural network that was based on the interaction
of neurons through a feedback mechanism. His approach was based on Hebb’s learning rule. The back-
propagation learning rule arrived on the neural-network scene at approximately the same time from several
 624 Digital Control and State Variable Methods

independent sources (Werbos; Parker; Rumelhart, Hinton and Williams). Essentially, a refinement of the
Widrow-Hoff learning rule, the backpropagation learning rule provided a systematic means for training
multilayer networks, thereby overcoming the limitations presented by Minsky. Minsky’s appraisal has
proven excessively pessimistic; networks now routinely solve many of the problems that he posed in his
book.
Research in the 1980s triggered the present boom in the scientific community. New and better models are
being proposed, and the limitations of some of the ‘old’ models are being chipped away. A number of today’s
technological problems are in areas where neural-network technology has demonstrated potential: speech
processing, image processing and pattern recognition, time-series prediction, real-time control and others.
As the research on neural networks evolved, more and more types of networks are being introduced,
while still less emphasis is placed on the connection to the biological neural network. In fact, the neural
networks that are most popular today have very little resemblance to the brain, and one might argue that it
would be more fair to regard them simply as a discipline under statistics.
The application of artificial neural networks in closed-loop control, has recently been rigorously studied.
One property of these networks, central to most control applications, is that of function approximation. Such
networks can generate input/output maps which can approximate any continuous function with the required
degree of accuracy. This emerging technology has given us control design techniques that do not depend on
parametrized mathematical models. Neural networks are used to estimate the unknown nonlinear functions;
the controller formulation uses these estimated results.
When neural networks are used for control of systems, it is important that results and claims are based on
firm analytical foundations. This is especially important when these control systems are to be used in areas
where the cost of failure is very high, for example when human life is threatened, as in aircrafts, nuclear
plants, etc. It is also true that without a good theoretical framework, it is unlikely that the research in the
discipline will progress very far, as intuitive invention and tricks cannot be counted on to provide good
solutions to controlling complex systems under a high degree of uncertainty. Strong theoretical results guar-
anteeing control system properties such as stability are still to come, although promising results have been
reported recently of progress in special cases. The potential of neural networks in control systems clearly
needs to be further explored and both, theory and applications, need to be further developed.
The rest of the chapter gives a gentle introduction to the application of neural networks in control sys-
tems. A single chapter can in no way do justice to the multitude of interesting neural network results, that
have appeared in the literature. Not only would space be required, but in the time required to detail current
results, new results would certainly arise. Instead of trying to cover a large spectrum of such a vast field, we
will focus on what is generally regarded as the core of the subject. This chapter is meant to be a stepping-
stone that could lead interested readers on to other books for additional information on the current status,
and future trends of the subject.

11.2 NEURON MODELS

A discussion of anthropomorphism to introduce neural network technology may be worthwhile—as it helps

explain the terminology of neural networks. However, anthropomorphism can lead to misunderstanding
when the metaphor is carried too far. We give here a brief description of how the brain works; a lot of details
of the complex electrical and chemical processes that go on in the brain, have been ignored. A pragmatic
justification for such a simplification is that by starting with a simple model of the brain, scientists have been
able to achieve very useful results.
 Neural Networks 625
11.2.1 Biological Neuron
To the extent a human brain is understood today, it seems to operate as follows: bundles of neurons, or nerve
fibres, form nerve structures. There are many different types of neurons in the nerve structure, each having a
particular shape, size and length depending upon its function and utility in the nervous system. While each
type of neuron has its own unique features needed for specific purposes, all neurons have two important
structural components in common. These may be seen in the typical biological neuron shown in Fig. 11.2. At
one end of the neuron are a multitude of tiny, filament-like appendages called dendrites, which come to-
gether to form larger branches and trunks where they attach to soma, the body of the nerve cell. At the other
end of the neuron is a single filament leading out of the soma, called an axon, which has extensive branching
on its far end. These two structures have special electrophysiological properties which are basic to the
function of neurons as information processors, as we shall see next.

Synapse

Cell body (Soma)

Axon

Nucleus
Dendrites
Synaptic
terminals

Fig. 11.2 A typical biological neuron

Neurons are connected to each other via their axons and dendrites. Signals are sent through the axon of one
neuron to the dendrites of other neurons. Hence dendrites may be represented as the inputs to the neuron, and
the axon as its output. Note that each neuron has many inputs through its multiple dendrites, whereas it has only
one output through its single axon. The axon of each neuron forms connections with the dendrites of many other
neurons, with each branch of the axon meeting exactly one dendrite of another cell at what is called a synapse.
Actually, the axon terminals do not quite touch the dendrites of the other neurons, but are separated by a very
small distance of between 50 and 200 angstroms. This separation is called the synaptic gap.
A conventional computer is typically a single processor acting on explicitly programmed instructions.
Programmers break tasks into tiny components, to be performed in sequence rapidly. On the other hand, the
brain is composed of ten billion or so neurons. Each nerve cell can interact directly with up to 200,000 other
neurons (though 1000 to 10,000 is typical). In place of explicit rules that are used by a conventional compu-
ter, it is the pattern of connections between the neurons, in the human brain, that seems to embody the
‘knowledge’ required for carrying out various information-processing tasks. In human brain, there is no
equivalent of a CPU that is in overall control of the actions of all the neurons.
The brain is organized into different regions, each responsible for different functions. The largest parts of
the brain are the cerebral hemispheres, which occupy most of the interior of the skull. They are layered
structures; the most complex being the outer layer, known as the cerebral cortex, where the nerve cells are
extremely densely packed to allow greater interconnectivity. Interaction with the environment is through the
visual, auditory and motion control (muscles and glands) parts of the cortex.
 626 Digital Control and State Variable Methods
In essence, neurons are tiny electrophysiological information-processing units which communicate with
each other through electrical signals. The synaptic activity produces a voltage pulse on the dendrite which is
then conducted into the soma. Each dendrite may have many synapses acting on it, allowing massive
interconnectivity to be achieved. In the soma, the dendrite potentials are added. Note that neurons are able to
perform more complex functions than simple addition on the inputs they receive, but considering a simple
summation is a reasonable approximation.
When the soma potential rises above a critical threshold, the axon will fire an electrical signal. This
sudden burst of electrical energy along the axon is called axon potential and has the form of an electrical
impulse or spike that lasts about 1 msec. The magnitude of the axon potential is constant and is not related to
the electrical stimulus (soma potential). However, neurons typically respond to a stimulus by firing not just
one but a barrage of successive axon potentials. What varies is the frequency of axonal activity. Neurons can
fire between 0 to 1500 times per second. Thus, information is encoded in the nerve signals as the instantane-
ous frequency of axon potentials and the mean frequency of the signal.
A synapse couples the axon with the dendrite of another cell. The synapse releases chemicals called
neurotransmitters, when its potential is raised sufficiently by the axon potential. It may take the arrival of
more than one spike before the synapse is triggered. The neurotransmitters that are released by the synapse
diffuse across the gap and chemically activate gates on the dendrites, which, when open, allow charged ions
to flow. It is this flow of ions that alters the dendritic potential and provides voltage pulse on the dendrite,
which is then conducted into the neighbouring neuron body. At the synaptic junction, the number of gates
that open on the dendrite depends upon the number of neurotransmitters released. It also appears that some
synapses excite the dendrites they affect, whilst others serve to inhibit it. This corresponds to altering the
local potential of the dendrite in a positive or negative direction.
Synaptic junctions alter the effectiveness with which the signal is transmitted; some synapses are good
junctions and pass a large signal across, whilst others are very poor, and allow very little through.
Essentially, each neuron receives signals from a large number of other neurons. These are the inputs to the
neuron which are ‘weighted’. That is, some signals are stronger than others. Some signals excite (are posi-
tive), and others inhibit (are negative). The effects of all weighted inputs are summed. If the sum is equal to
or greater than the threshold for the neuron, the neuron fires (gives output). This is an “all-or-nothing”
situation. Because the neuron either fires or doesn’t fire, the rate of firing, not the amplitude, conveys the
magnitude of information.
The ease of transmission of signals is altered by activity in the nervous system. The neural pathway
between two neurons is susceptible to fatigue, oxygen deficiency, and agents like anaesthetics. These events
create a resistance to the passage of impulses. Other events may increase the rate of firing. This ability to
adjust signals is a mechanism for learning.
After carrying a pulse, an axon fibre is in a state of complete non-excitability for a certain time called the
refractory period. For this time interval, the nerve does not conduct any signals, regardless of the intensity of
excitation. Thus, we may divide the time scale into consecutive intervals, each equal to the length of the
refractory period. This will enable a discrete-time description of the neurons’ performance in terms of their
states at discrete-time instances.

11.2.2 Artificial Neuron

Artificial neurons bear only a modest resemblance to real things. They model approximately three of the
processes that biological neurons perform (there are at least 150 processes performed by neurons in the
human brain).
An artificial neuron
(i) evaluates the input signals, determining the strength of each one;
 Neural Networks 627
(ii) calculates a total for the combined input signals and compares that total to some threshold level; and
(iii) determines what the output should be.
Inputs and Outputs Just as there are many inputs (stimulation levels) to a biological neuron, there should
be many input signals to our artificial neuron (AN). All of them should come to our AN simultaneously. In
response, a biological neuron either ‘fires’ or ‘doesn’t fire’ depending upon some threshold level. Our AN will
be allowed a single output signal, just as is present in a biological neuron: many inputs, one output (Fig. 11.3).
Weighting Factors Each input will be given a relative weighting, which will affect the impact of that
input (Fig. 11.4). This is something like varying synaptic strengths of the biological neurons—some inputs
are more important than others in the way they combine to produce an impulse. Weights are adaptive coeffi-
cients within the network, that determine the intensity of the input signal. In fact, this adaptability of connec-
tion strength is precisely what provides neural networks with their ability to learn and store information, and,
consequently, is an essential element of all neuron models.
Inputs Connection
x1 weights

w1
Inputs Total input
x2
w2 S wixi

Output
wn

xn
Fig. 11.3 Many inputs, one output model of a neuron Fig. 11.4 A neuron with weighted inputs
Excitatory and inhibitory inputs are represented simply by positive or negative connection weights,
respectively. Positive inputs promote the firing of the neuron, while negative inputs tend to keep the neuron
from firing.
Mathematically, we could look at the inputs and the weights on the inputs as vectors.
The input vector
LM
x1 OP
x2MM PP
x= (11.1a)
M
xn
MN PQ
and the connection weight vector
w = [w1 w2 ... wn] (11.1b)
The total input signal is the product of these vectors. The result is a scalar
n
∑ wi xi = wx (11.1c)
i =1

Activation Functions Although most neuron models sum their input signals in basically the same manner,
as described above, they are not all identical in terms of how they produce an output response from this input.
Artificial neurons use an activation function, often called a transfer function, to compute their activation as a
function of total input stimulus. Several different functions may be used as activation functions, and, in fact, the
most distinguishing feature between existing neuron models is precisely which transfer function they employ.
 628 Digital Control and State Variable Methods
We will, shortly, take a closer look at the activation functions. We first build a neuron model, assuming
that the transfer function has a threshold behaviour, which is, in fact, the type of response exhibited
by biological neurons: when the total stimulus exceeds a certain threshold value q, a constant output is
produced, while no output is generated for input levels below the threshold. Figure 11.5a shows this neuron
model. In this diagram, the neuron has been represented in such a way that the correspondence of each
element with its biological counterpart may be easily seen.

Fig. 11.5 A neuron model

Equivalently, the threshold value can be subtracted from the weighted sum and the resulting value com-
pared to zero; if the result is positive, then output a 1, else output a 0. This is shown in Fig. 11.5b; note that
the shape of the function is the same but now the jump occurs at zero. The threshold effectively adds an
offset to the weighted sum.
 Neural Networks 629
An alternative way of achieving the same effect is to take the threshold out of the body of the model
neuron, and connect it to an extra input value that is fixed to be ‘on’ all the time. In this case, rather than
subtracting the threshold value from the weighted sum, the extra input of +1 is multiplied by a weight and
added in a manner similar to other inputs—this is known as biasing the neuron. Figure 11.5c shows a neuron
model with a bias term. Note that we have taken constant input ‘1’ with an adaptive weight ‘b’ in our model.
The first formal definition of a synthetic neuron model, based on the highly simplified considerations of
the biological neuron, was formulated by McCulloch and Pitts (1943). The two-port model (inputs-activa-
tion value-output mapping) of Fig. 11.5 is essentially the MP neuron model. It is important to look at the
features of this unit—which is an important and popular neural network building block.
It is a simple enough unit, thresholding a weighted sum of its inputs to get an output. It specifically does
not take any account of the complex patterns and timings of the actual nervous activity in real neural
systems, nor does it have any of the complicated features found in the body of biological neurons. This
ensures its status as a model, and not a copy of a real neuron.
The MP artificial neuron model involves two important processes:
(i) Forming net activation by combining inputs. The input values are amalgamated by a weighted additive
process to achieve the neuron activation value a (refer Fig. 11.5c).
(ii) Mapping this activation value a into the neuron output y$ . This mapping from activation to output may
be characterized by an ‘activation’ or ‘squashing’ function.
For the activation functions that implement input-to-output compression or squashing, the range of the func-
tion is less than that of the domain. There is some physical basis for this desirable characteristic. Recall that
in a biological neuron, there is a limited range of output (spiking frequencies). In the MP model, where DC
levels replace frequencies, the squashing function serves to limit the output range. The squashing function
shown in Fig. 11.6a limits the output values to {0, 1}, while that in Fig. 11.6b limits the output value to
{–1, 1}. The activation function of Fig. 11.6a is called unipolar, while that in Fig. 11.6b is called bipolar
(both positive and negative responses of neurons are produced).
y y

1 1

0 a a

–1

(a) Unipolar squashing function (b) Bipolar squashing function

Fig. 11.6

11.2.3 Mathematical Model

From the above discussion, it is evident that the artificial neuron is really nothing more than a simple math-
ematical equation, for calculating an output value from a set of input values. From now onwards, we will be
more on a mathematical footing; the reference to biological similarities will be reduced. Therefore, names like
a processing element, a unit, a node, a cell, etc., may be used for the neuron. A neuron model (a processing
element/a unit/a node/a cell of our neural network), will be represented as follows:
The input vector
x = [x1 x2 ... xn]T;
 630 Digital Control and State Variable Methods
the connection weight vector
w = [w1 w2 ... wn];
the unity-input weight b (bias term), and the output y$ of the neuron are related by the following equation:
F wx n I
y$ = s (wx + b) = s GH ∑i =1
i i +b JK (11.2)

where s (×) is the activation function (transfer function) of the neuron.

The weights are always adaptive. We can simplify our diagram as in Fig. 11.7a; adaptation need not be
specifically shown in the diagram.

x0 = 1
x1
w1 w0 = b

x2 w2 x1 w1
a y
s( ) w2 a y
x2 s( )
wn
wn
xn b
1 (a) xn (b)
Fig. 11.7 Mathematical model of a neuron (perceptron)

The bias term may be absorbed in the input vector itself as shown in Fig. 11.7b.
y$ = s (a)
F w x I ; w = b, x = 1
n
=s GH ∑ JK
i = 0
i i 0 0 (11.3a)

F n I
= s G ∑ w x + w J = s (wx + w ) (11.3b)
i i 0 0
H i = 1 K
In the literature, this model of an artificial neuron is also referred to as a perceptron (the name was given
by Rosenblatt in 1958).
The expressions for the neuron output y$ are referred to as the cell recall mechanism. They describe how
the output is reconstructed from the input signals and the values of the cell parameters.
The artificial neural systems under investigation and experimentation today, employ a variety of activa-
tion functions that have more diversified features than the one presented in Fig. 11.6. Below, we introduce
the main activation functions that will be used later in this chapter.
The MP neuron model shown in Fig. 11.5 used the hard-limiting activation function. When artificial
neurons are cascaded together in layers (discussed in the next section), it is more common to use a soft-
limiting activation function. Figure 11.8a shows a possible bipolar soft-limiting semilinear activation
function. This function is, more or less, the ON-OFF type, as before, but has a sloping region in the middle.
With this smooth thresholding function, the value of the output will be practically 1 if the weighted sum
exceeds the threshold by a huge margin and, conversely, it will be practically –1 if the weighted sum is much
less than the threshold value. However, if the threshold and the weighted sum are almost the same, the output
from the neuron will have a value somewhere between the two extremes. This means that the output from the
neuron can be related to its inputs in a more useful and informative way. Figure 11.8b shows a unipolar soft-
limiting function.
 Neural Networks 631
s (a) s (a)

1 1

a 0 a

–1
(a) (b)
Fig. 11.8 Self-limiting activation functions

For many training algorithms (discussed in later sections), the derivative of the activation function is
needed; therefore, the activation function selected must be differentiable. The logistic or sigmoid function,
which satisfies this requirement, is the most commonly used soft-limiting activation function. The sigmoid
function (Fig. 11.9a):
1
s (a) = (11.4)
1 + e - la
is continuous and varies monotonically from 0 to 1 as a varies from – ¥ to ¥. The gain of the sigmoid, l,
determines the steepness of the transition region. Note that as the gain approaches infinity, the sigmoid
approaches a hard-limiting nonlinearity. One of the advantages of the sigmoid is that it is differentiable. This
property had a significant impact historically, because it made it possible to derive a gradient search learning
algorithm for networks with multiple layers (discussed in later sections).
(a)
(a) =1 1
=1
1

a
= 0.2

–1
0 a
(a) Sigmoid functions (b) Hyperbolic tangent function
Fig. 11.9

The sigmoid function is unipolar. A bipolar function with similar characteristics is a hyperbolic tangent
(Fig. 11.9b):
1 − e − λa
s (a) = = tanh 12 λa c h (11.5)
1 + e − λa
The biological basis of these activation functions can easily be established. It is known that neurons
located in different parts of the nervous system have different characteristics. The neurons of the ocular
motor system have a sigmoid characteristic, while those located in the visual area have a Gaussian character-
istic. As we said earlier, anthropomorphism can lead to misunderstanding when the metaphor is carried too
far. It is now a well-known result in neural network theory that a 2-layer neural network is capable of solving
 632 Digital Control and State Variable Methods
any classification problem. It has also been shown that a 2-layer network is capable of solving any nonlinear
function approximation problem [138, 141]. This result does not require the use of sigmoid nonlinearity. The
proof assumes only that nonlinearity is a continuous, smooth, monotonically increasing function that is bounded
above and below. Thus, numerous alternatives to sigmoid could be used, without a biological justification. In
addition, the above result does not require that the nonlinearity be present in the second (output) layer. It is
quite common to use linear output nodes since this tends to make learning easier. In other words
s (a) = l a; l > 0 (11.6)
is used as an activation function in the output layer. Note that this function does not “squash” (compress) the
range of output.
Our focus in this chapter will be on two-layer perceptron networks with the first (hidden) layer having
log-sigmoid
s (a) = 1 (11.7a)
1 + e-a
-a
or tan-sigmoid s (a) = 1 - e (11.7b)
1 + e -a
activation function, and the second (output) layer having linear activation function
s (a) = a (11.8)
The log-sigmoid function has historically been a very popular choice, but since it is related to the tan-
sigmoid by the simple transformation
s log-sigmoid = (s tan-sigmoid + 1)/2 (11.9)
it makes no difference which of these is used.
We have so far described two classical neuron models:
· perceptron—a neuron with sigmoidal activation function (sigmoidal function is a softer version of the
original perceptron’s hard limiting or threshold activation function); and
· linear neuron—a neuron with linear activation function.
Another neuron model which has gained considerable attention is the “Radial Basis Function” (RBF)
model, which uses radially symmetric activation function f (×). We add RBF neuron in our toolkit for build-
ing neural networks.
A radial basis function is typically parameterized by two parameters: the centre which defines its posi-
tion, and a spread parameter that determines its shape. The spread parameter is equal to the standard devia-
tion s in case of a one-dimensional Gaussian function (do not confuse the standard deviation parameter s
with the sigmoidal activation function s (×)). In the case of a multivariate input vector x, the parameters that
define the shape of the hyper-Gaussian function are elements of a covariance matrix S. With the selection of
the same spread parameter s for all components of the input vector, the covariance matrix S = diag(s 2).
The input vector
x = [x1 x2 ... xn]
and the output f (×) of an RBF (Gaussian) neuron are related
x1 by the following equation.
⎛ x−c 2 ⎞
φ ( x, c, σ) = exp ⎜ − ⎟ (11.10)
x2 f (x,c,s) ⎜ 2 σ 2 ⎟
⎝ ⎠
where c is the centre and s is the spread parameter of the
Gaussian function.
xn Unlike sigmoidal neuron, there are no connection weights
Fig. 11.10 Gaussian function in RBF between the input terminals and the RBF unit (refer Fig. 11.10);
neuron model the centre c and the spread parameter s represent the weights.
 Neural Networks 633

11.3 NETWORK ARCHITECTURES

In the biological brain, a huge number of neurons are interconnected to form the network and perform
advanced intelligent activities. The artificial neural network is built by neuron models. Many different types
of artificial neural networks have been proposed, just as there are many theories on how biological neural
processing works. We may classify the organization of the neural networks largely into two types: a
feedforward net and a recurrent net. The feedforward net has a hierarchical structure that consists of several
layers, without interconnection between neurons in each layer, and signals flow from input to output layer in
one direction. In the recurrent net, multiple neurons in a layer are interconnected to organize the network. In
the following, we give typical characteristics of the feedforward net and the recurrent net, respectively.

11.3.1 Feedforward Networks

A feedforward network consists of a set of input terminals which feed the input patterns to a layer or
subgroup of neurons. The layer of neurons makes independent computations on data that it receives, and
passes the results to another layer. The next layer may, in turn, make its independent computations and pass
on the results to yet another layer. Finally, a subgroup of one or more neurons determines the output from the
network. This last layer of the network is the output layer. The layers that are placed between the input
terminals and the output layer are called hidden layers.
Some authors refer to the input terminals as the input layer of the network. We do not use that convention
since we wish to avoid ambiguity. Note that each neuron in a network makes its computation based on the
weighted sum of its inputs. There is one exception to this rule: the role of the ‘input layer’ is somewhat
different as units in this layer are used only to hold input data, and to distribute the data to units in the next
layer. Thus, the ‘input layer’ units perform no function—other than serving as a buffer, fanning out the
inputs to the next layer. These units do not perform any computation on the input data, and their weights,
strictly speaking, do not exist.
The network outputs are generated from the output layer units. The output layer makes the network
information available to the outside world. The hidden layers are internal to the network and have no direct
contact with the external environment. There may be from zero to several hidden layers. The network is said
to be fully connected if every output from a single node is channeled to every node in the next layer.
The number of input and output nodes needed for a network will depend on the nature of the data
presented to the network, and the type of the output desired from it, respectively. The number of neurons to
use in a hidden layer, and the number of hidden layers required for processing a task, is less obvious. Further
comments on this question will appear in a later section.
A Layer of Neurons A one-layer network with n inputs and q neurons is shown in Fig. 11.11. In the
network, each input xi; i = 1, 2, ..., n is connected to the j th neuron input through the weight wji; j = 1, 2, ...,
q. The jth neuron has a summer that gathers its weighted inputs to form its own scalar output
n
å wji xi + wj0; j = 1, 2, ..., q
i =1
Finally, the jth neuron outputs y$ j through its activation function s (×):
F wx n I
y$ j = s GH ∑i =1
ji i + wj 0 ; j = 1, 2, ..., q
JK (11.11a)
= s (wj x + wj0); j = 1, 2, ..., q (11.11b)
where weight vector wj is defined as
wj = [wj1 wj2 ...wjn] (11.11c)
 634 Digital Control and State Variable Methods
Note that it is common for the number of inputs to be different from the number of neurons (i.e., n ¹ q).
A layer is not constrained to have the number of its inputs equal to the number of its neurons.
In vector-matrix notation, the layer shown in Fig. 11.11 has q ´ 1 output vector
LM y$ OP1
y$
y$ = M P ,
2
(11.12a)
MM M PP
MN y$ PQ
q

y1
x1
S s (× )

y2
x2
S s (× )

y3
x3 S s (× )

yq
xn
S s (× )

Inputs Neuron layer

x1 1
S wji xi + wj0
wj1 wj0 i

x2 wj2 yj

wjn
S s (× )

xn

Inputs jth neuron

Fig. 11.11 A one-layer network

 Neural Networks 635
q ´ n weight matrix
LM w11 w12 L w1n w1 OP LM OP
w w22 L w2 n w2
W= M PP MM PP
21
= (11.12b)
MM M M M w3
PPQ MMN PPQ
MNw
q1 wq 2 L wqn wq
and q ´ 1 bias vector
LM w10 OP
w
=M PP
20
w0 (11.12c)
MM M PPQ
MNw
q0

Note that the row indices on the elements of matrix W indicate the destination neuron for the weight, and
the column indices indicate which source is the input for that weight. Thus, the indices in wji say that the
signal from the ith input is connected to the jth neuron.
The activation vector is
w 1x + w10
LM OP
w x+w
Wx + w = M PP
2 20
0
MM M PPQ
MNw x + wq q0
The outputs are
y$1 = s (w1x + w10)
y$ 2 = s (w2x + w20)
M
y$ q = s (wqx + wq0)
Introducing the nonlinear matrix operator G, the mapping of the
input space x to output space y$ , implemented by the network, can be
x G (Wx + w0) y^
expressed as (Fig. 11.12)
y$ = G(Wx + w0) (11.13a)
Fig. 11.12 Input–Output map of where LMσ (⋅) 0 L 0 OP
a one-layer network 0 σ (⋅) L 0
G(×) = M D P (11.13b)
MM M M L M P
P
N0 0 L σ (⋅) Q
Note that the nonlinear activation functions s (×) on the diagonal of the matrix operator G, operate
componentwise on the activation vector of each neuron.
The input and output vectors x and y$ are often called input and output patterns, respectively. The
mapping of the input pattern to an output pattern as given by (11.13), is of the feedforward and instantaneous
type since it involves no time delay between input x and the output y$ .
Neural networks normally have atleast two layers of neurons, with the first layer neurons having
nonlinear and differentiable activation functions. Such networks, as we will see shortly, can approximate any
continuous function. In real life, we are faced with nonlinear problems, and multilayer neural network
structures have the capability of providing solutions to these problems.
 636 Digital Control and State Variable Methods
If the relationship between the input and output signals is linear, or can be treated as such, a single layer
neural network having linear neurons is the best solution. “Adaptive Linear Element” (Adaline) is the name
given to a neuron with linear activation function and a learning rule for adapting the weights. Single-layer
adaline networks have a capacity for a wide range of applications, whenever the problem at hand can be
treated as linear.
Multilayer Perceptrons A two-layer NN, depicted in Fig. 11.13, has n inputs and two layers of neurons,
with the first layer having m neurons that feed into the second layer having q neurons. The first layer is
known as the hidden layer, with m the number of hidden-layer neurons; the second layer is known as the
output layer, with q the number of output-layer neurons. It is common for different layers to have different
numbers of neurons. Note that the outputs of the hidden layer are inputs to the following layer (output layer);
and the network is fully connected. Neural networks with multiple layers are called multilayer perceptrons
(MLP); their computing power is significantly enhanced over the one-layer NN.
1 1

z1 y1
x1 ()

1 1

z2 y2
x2 ()

1 1

zm yq
xn ()

Fig. 11.13 A two-layer network

All continuous functions (exhibiting certain smoothness) can be approximated to any desired accuracy
with a network of one hidden layer of sigmoidal hidden units, and a layer of linear output units [141]. Does
it mean that there is no need to use more than one hidden layer and/or mix different types of activation
functions? This is not quite true. It may be that the accuracy can be improved using a more sophisticated
network architecture. In particular, when the complexity of the mapping to be learned is high, it is likely that
the performance can be improved. However, since implementation and training of the network become more
complicated, it is customary to apply only a single hidden layer of similar activation functions, and an output
layer of linear units. Our focus is on two-layer feedforward neural networks with hyperbolic tangent hidden
units and linear output units. This is probably the most commonly used network architecture, as it works
quite well in many practical applications.
Defining the hidden-layer outputs zl allows one to write
F wx
n I
zl = s GH ∑
i =1
li i + wl 0 ; l = 1, 2, L, m
JK (11.14)

= s (wlx + wl0)
 Neural Networks 637
where
wl =D [wl1 wl2 ... wln]
In vector-matrix notation, the hidden layer in Fig. 11.13 has m ´ 1 output vector
LM OP
z1
z2
z= MM PP
, (11.15a)
M
zm
MN PQ
m × n weight matrix
LM
w11 w12 L w1n OP
w21 w22 L w2 n
W= MM PP (11.15b)
M M M
MN
wm1 wm2 L wmn
PQ
and m × 1 bias vector
w10 LM OP
w
= M
MM M PPP
20
w0 (11.15c)

Nw Q
m0
The output
z = G(Wx + w0) (11.16a)
LMσ (⋅) 0 L 0 OP
0 σ (⋅) L 0
G(×) = M
MM M M M M PPP
D
where (11.16b)

N 0 0 L σ (⋅)Q
Defining the second-layer weight matrix as
LMv v L v OP
11 12 1m
v v L v P
V= M
21 22 2m
(11.17a)
MM M M M M PP
MNv v L v PQ
q1 q2 qm
and bias vector as
LMv OP 10
v
v = M P,
20
(11.17b)
0
MM M PP
MNv PQ
q0
one may write the NN output as

j
F m
I
y$ = G å v z + v J ; j = 1, 2, ..., q
jl l j0 (11.18)
H l =1 K
= vj z + vj0
where vj =D [vj1 vj 2 ... vjm]
 638 Digital Control and State Variable Methods
The output vector
x V(G (Wx + w 0 ) + v 0 ) y LM y$ OP1
y$
y$ = M P
2
(11.19a)
Fig. 11.14 Input–output map of
a two-layer network
MM M PP
MN y$ PQ
q
is given by the expression
y$ = Vz + v0
= V(G(Wx + w0) + v0) (11.19b)
Figure 11.14 shows the input–output map.
RBF Networks Radian Basis Function (RBF) networks have gained considerable attention as an alterna-
tive to multilayer perceptrons, with the hidden layers of neurons having sigmoidal activation functions. Both
the multilayer perceptrons and the RBF networks are the basic constituents of the feedforward neural
network. They are structurally equivalent.
The architecture of RBF networks is presented in Fig. 11.15. The network consists of n inputs x1, x2, ... xn;
and a hidden layer of m nonlinear processing units (refer Eqn. (11.10)):
⎛ x−c 2 ⎞
φ i ( x, ci ,s i) = exp ⎜ − i
⎟ ; i = 1, 2, ..., m (11.20a)
⎜ 2 s 2 ⎟
⎝ i ⎠
The output of the network is computed as a weighted sum of the outputs of the RBF units:
m
yˆ j = ∑ w jiφi ( x, c i , σ i ) ; j = 1, 2, ..., q (11.20b)
i =1
where wji is the connection weight between the RBF unit i and the j th component of the output vector.
It follows from equations (11.20) that the parameters (ci, si, wji ) govern the mapping properties of
RBF neural network. It has been shown [141] that RBF network can fit any arbitrary function with just one
hidden layer.

w11 y1
x1 f1(.) w12 S
w1m

w21
w22 y2
x2 f1(.)
w2m S
. . .
. . .
. . wq1 .
wq2 yq
xn fm(.) wqm
S

Fig. 11.15 The RBF network architecture

 Neural Networks 639
It is commonly held that a fundamental difference between multilayer perceptron and RBF network is that
the feedforward multilayer perceptron networks are representatives of global approximation schemes,
whereas neural networks with RBFs (typically with Gaussian functions) are representative of local approxi-
mation schemes. The adjectives global and local are connected with the region of input space of the network
for which the output is non-zero (Here, non-zero means a computationally relevant, not very small output).
For Gaussians, non-zero output is a small region around the centres, and for sigmoidal logistic functions,
non-zero output is for a large input space.

11.3.2 Recurrent Networks

The feedforward networks (Figs 11.11–11.15) implement fixed-weight mappings from the input space to the
output space. Because the networks have fixed weights, the state of any neuron is solely determined by the
input to the unit, and not the initial and past states of the neurons. This independence of initial and past states
of the network neurons limits the use of such networks because no dynamics are involved. The maps imple-
mented by the feedforward networks of the type shown in Figs 11.11–11.15, are static maps.
To allow initial and past state involvement along with serial processing, recurrent neural networks utilize
feedback. Recurrent neural networks are also characterized by use of nonlinear processing units; thus, such
networks are nonlinear dynamic systems (Networks of the form shown in Figs 11.11–11.15 are nonlinear
static systems).
The architectural layout of a recurrent network takes many different forms. We may have feedback from
the output neurons of a feedforward network to the input terminals. Yet another possible form is feedback
from the hidden neurons of the network to the input terminals. When the feedforward network has two or
more hidden layers, the possible forms of feedback expand even further. Recurrent networks have a rich
repertoire of architectural layouts.
It should be noted that many real-world problems, which one might think would require recurrent
architectures for their solution, turn out to be solvable with feedforward architectures as well. A multilayer
feedforward network, which realizes a static map can represent the input/output behaviour of a dynamic
system. For this to be possible, one must provide the neural network with information about the history of the
system—typically, delayed inputs and outputs. How much history is needed, depends on the desired accu-
racy. There is a trade-off between accuracy and computational complexity of training, since the number of
inputs used, affects the number of weights in the neural network—and subsequently, the training time (Sec-
tion 11.7 will give more details). One sometimes starts with as many delayed signals as the order of the
system, and then modifies the network accordingly. It also appears that using a two hidden-layer network—
instead of one hidden layer—has certain computational advantages. The number of neurons in the hidden
layer(s) is typically chosen based on empirical criteria, and one may iterate over a number of networks to
determine a neural network that has a reasonable number of neurons and accomplishes the desired degree of
approximation.
From numerous practical applications published over the past decade, there seems to be substantial
evidence that multilayer feedforward networks possess an impressive ability to perform reasonably well in
most cases of practical interest. Lately, there have also been some theoretical results that attempt to explain
the reasons for the success [138].
Our focus is on two-layer feedforward neural networks with hyperbolic tangent hidden units and linear
output units. This is probably the most commonly used network architecture as it works quite well in many
practical applications.
 640 Digital Control and State Variable Methods

11.4 LEARNING IN NEURAL NETWORKS

We are accustomed to solving computational problems by breaking them down into steps and solving each
small step. Neural networks don’t work by developing an algorithm for each individual processing element.
The knowledge is in the state of the whole network, not any one piece of it. Consequently, we have to think
in overall terms of inputs, transformations, and outputs of the network.
Neural networks deal only with numeric input data. Therefore, we must convert or encode information
from the external environment to numeric data form. Additionally, it is often necessary to scale data. Inhibi-
tory inputs are just as important as excitatory inputs. The input scheme should adequately allow for both the
types (allow positive and negative weights). A provision is also usually made for constant-source input to
serve as an offset or bias term for the transfer or activation function.
The numeric output data of a neural network will, likewise, require decoding and scaling to make it
compatible with the external environment.
Important characteristics of the network depend on:
(i) the transfer or activation functions of the processing elements;
(ii) the structure of the network (number of neurons, layers and interconnections); and
(iii) the learning mechanism of the network.
We have seen, in Section 11.2, that several different functions may be used as activation functions, and in
fact, one of the distinguishing features between existing neural network models is precisely which transfer
function they employ. Another important distinguishing feature is the network structure. Two major classes
of networks, feedforward and recurrent networks, were introduced in the last section. In this section, we
present basic learning features of neural networks.

11.4.1 The Basic Learning Mechanism

Each processing element (neuron) in a neural network has a number of inputs (xi), each of which must store
a connection weight (wji). The element sums up the weighted input (wji xi) and computes one, and only one,
activation signal (aj). The output signal is a function (s (×)) of the weighted sum. Figure 11.16 summarizes
how a processing element works.
The function s (×) remains fixed for the life of the processing element. It is generally decided upon as part of
the design, and it cannot be changed dynamically. In other words, the transfer function currently cannot be
adjusted or modified during the operation or running of the network.
However, the weights (wji) are variables. They can be dynamically adjusted to produce a given output
(yj). This dynamic modification of the variable weights is the very essence of learning. At the level of a
single processing element, this self-adjustment is a very simple thing. When many processing elements do it
collectively, we say it resembles ‘intelligence’. The meaningful information is in the modified weights. The
ability of an entire neural network to adapt itself (change the wji values) to achieve a given output (yj), is its
uniqueness (of course, uniqueness for a nonliving entity).
x1 In traditional programming, the programmer
wj1 will be given the inputs, some type of processing
requirements (what to do with the inputs), and the
x2 wj2 yj desired output. The programmer’s job is to apply
aj
the necessary step-by-step instructions to develop
the required relationship between the input and
wjn
the output.
xn In contrast, neural networks do not require any
Fig. 11.16 A processing element instructions, rules or processing requirements
 Neural Networks 641
about how to process the input data. In fact, neural networks determine the relationship between input and
output by looking at examples of many input-output pairs.
Pairs of inputs and outputs are applied to the neural network. These pairs of data are used to teach or train
the network, and as such are referred to as the training set. Knowing what output is expected from each
input, the network automatically adjusts or adapts the strengths of the connections between processing
elements. The method used for the adjusting process is called the learning rule.
The training set needs to be fairly large; it must also contain all the necessary information if the network
is to ‘learn’ the features and relationships that are important. Also, note that the approach of teaching the
network one thing at a time does not work. All the weights set so meticulously for one fact, could be drasti-
cally altered in learning the next fact. The network has to learn everything together, finding the best weight
settings for the total set of facts.
After training, the network is ready for use. Depending on the task to be done, the network may have its
currently learned weights ‘frozen’—thus disabling its learning laws. This network will give reasonable
outputs for the data it has not seen before. For some applications, such as real-time process control, the
network is made to learn continuously, i.e., it must continue to adjust to changing conditions.
In the scheme of learning described above, weight adjustment is correctional—adjusting network output
to desired output for a given input pattern. A number of other variations of learning are in use. For example,
weights could be modified to cluster similar input patterns into distinct classes. This is the property of
self-organizing networks; they ‘learn’ without being given the correct answer for an input pattern.
Let us look at some of the characteristics of neural-network technology which set it apart from conven-
tional computing.
Inherent Parallelism Processing sequence in neural networks is parallel and simultaneous. All the
processing elements in one layer operate in concert. Computation is distributed over more than one process-
ing element and is done simultaneously.
Although digital computers have to simulate this parallelism, true neural network hardware will really
perform the operations in parallel. Very fast decisions, made in real time, will be possible.
Storing Knowledge Knowledge within a neural network is not stored in specific memory locations, as it
is in conventional computing. Knowledge is distributed throughout the system; it is the dynamic response to
the inputs and the network architecture. Because knowledge is distributed, the system uses many connec-
tions to retrieve solutions to particular problems.
Not only is the memory in a neural network distributed, it may also be associative, like in biological
neural systems. For example, if we hum a few bars of a tune and it is one we have heard before, we may be
able to ‘name that tune’. This capability of looking at partial patterns and guessing the whole, is an important
characteristic of the neural networks because most of the data in this world is inexact.
In a conventional digital computer, each fact is stored in a unique location. A fact is retrieved by provid-
ing the address, even though there is no particular relationship between the fact and the address. Neural
networks don’t work this way at all. The associative memories of neural networks are content-addressable;
the address may be a subpart of the memory content.
Fault Tolerance Neural networks are extremely fault tolerant and degrade gracefully. They can learn
from and make decisions based on incomplete data. Because the knowledge is distributed throughout the
system rather than residing in a single memory location, a percentage of the nodes can be inoperative without
significantly changing the overall system behaviour. Resistance to hardware failure is, therefore, much
greater in a neural network than in conventional computers, where hardware failure is usually catastrophic.
 642 Digital Control and State Variable Methods

Generalization Generalization is the ability of the network to respond to input that it has not seen before.
The input may be partial or incomplete. Generalization takes the ability of the neural network to learn and
self-adjust a step further. The system can ‘hypothesize’ a response.
As we have seen in the preceding paragraphs that learning in neural networks refers to the process of acquir-
ing a desired behaviour by changing the connection weights. The main rules to produce these weight changes
may be classified into three important categories, each of which borrow concepts from behavioural theories
of learning in biological systems.

11.4.2 Supervised Learning

These rules compute the necessary change in the connection weights by presenting the network given input
pattern, comparing the obtained response with a desired response known a priori and then changing the
weights in the direction of decreasing error. More clearly, in the supervised learning mode, a neural network
is supplied with a sequence of examples ( x (1) , y (1) ), ( x ( 2 ) , y ( 2 ) ), ..., ( x ( p ) , y ( p ) ), ..., of desired input-output
pairs. When each input x(p) is fed into the neural network, the corresponding desired output y( p) is also
supplied to the neural network. As shown in Fig. 11.17a, the difference between the actual neural network
(NN) output y$ ( p ) and the desired output y(p) is measured in the error-signal generator, which then produces
error signals for the NN to correct its weights in such a way that the actual output will move closer to the
desired output.

q (Wx C w 0 )

Policy
(Actor NN)
q (Wa x C w a0 )
Internal (heuristic)
reinforcement

Value Function
state (Critic NN) action
q (Wc x C wc0)
Cost (External (primary)
reinforcement)

Environment

(b) A reinforcement learning framework

 Neural Networks 643

q (Wx C w 0 )

Fig. 11.17
With supervised learning, it is necessary to ‘train’ the neural network before it becomes operational.
Training consists of, as we have discussed earlier, presenting to the network sets of input patterns and
desired responses. During the training phase, connection weights are adapted to learn the desired input-
output behaviour. During the operation phase, the network works with fixed weights to produce outputs in
response to new patterns.
Least Squares Algorithm As an illustration of error-correction learning, consider the simple case of a
single neuron with linear activation function. Figure 11.18 is a schematic diagram of such a network. An
input signal x = [x1, x2, ..., xn]T, comprising features and augmented by a constant input component (bias), is
applied to the neuron; weighted and summed to give an output signal ŷ :
n
yˆ = ∑ wi xi + w0
i =1
= wx + w0 (11.21a)
where w = [w1 w2 ... wn ]
Defining (n + 1) ´ 1 vector
x = [1 x1 x2 ... xn ]
T

and 1 ´ (n + 1) vector
w = [ w0 w1 w2 ... wn ],
we can express Eqn. (11.21a) as
ŷ = w x (11.21b)
1
w0

x1 w1

y – + y

w2
x2 e
.
. wn
.

xn
Fig. 11.18 Learning scheme for a linear neuron
(Dashed arrows indicate that weights adaptation depends on the output error)
 644 Digital Control and State Variable Methods
The learning task is to find the weights of the neuron (estimate the parameters of the proposed linear
model (11.21)) using a finite number of measurements, observations, or patterns. The learning environment,
thus, comprises a training set of measured data (patterns):
{x( p), y( p); p = 1, 2, ..., P}
consisting of an input vector x and output or system response y, and the corresponding learning rule for the
adaptation of the weights (In what follows, learning algorithm is given for the case of one neuron only, and
the desired output is a scalar variable. The extension of the algorithm for y, a vector, is straightforward). The
choice of a performance criterion, or the measure of goodness of the estimation, depends primarily on the
data, and on the desired simplicity of the learning algorithm. In the neural network field, the most widely
used performance criterion (cost function) is the sum of error squares:
P P
( ) ( )
2 2
E = 12 ∑ e( p ) = 12 ∑ y ( p ) − yˆ ( p ) (11.22)
p =1 p =1
(The constant ½ is used for computational convenience only. It gets cancelled out by the differentiation
required in the error minimization process).
It is obvious that network equation (11.21) is exactly a linear model with (n + 1) linear parameters. So we
can employ the least-squares methods, discussed in Chapter 10, to minimize the error in the sense of least
squares.
A matrix of input vectors x( p); p = 1, 2, ..., P (called the data matrix X) and vector y of the desired outputs
( p)
y ; p = 1, 2, ..., P, are introduced as follows:
⎛ 1 1 K 1 ⎞
⎜ (1) ⎟
⎜ x1 x1(2) L x1( P )
⎟
X = ⎜⎜ x2(1) x2(2) L
⎟ = ⎡ x (1) x (2) ...x ( P) ⎤
x2( P )
⎟ ⎣⎢ ⎦⎥
⎜ M M M ⎟
⎜ ⎟
⎜ x (1) xn(2) L xn( P ) ⎟⎠
⎝ n
(11.23a)
T
y = ⎡ y (1) y (2) ... y ( P ) ⎤
⎢⎣ ⎥⎦
The weights w are required to satisfy the following equations (refer Eqn.(11.21b)):
y (1) = w x (1)
y (2) = w x (2)
M
y( P) = w x ( P)
Therefore
[ y(1) y(2) .... y( P)] = w [ x (1) x(2) ... x ( P )]
yT = w X

or y = XT wT
P ×1 P × ( n + 1) ( n + 1) ×1

In the least squares sense, the best or optimal w that minimizes E results from the equation (refer
Eqns (10.38–10.40))
wT = ( XXT )−1 Xy = [ w0 w1 w2 ...wn ]T (11.23b)
 Neural Networks 645
From a computational point of view, the calculation of optimal weights requires the pseudoinverse of the
P ´ (n + 1) matrix X.
An alternative solution to this type of problem is the “Recursive Least Squares” (RLS) algorithm (refer
Eqns (10.45)). For the learning problem in hand, the steps given in Table 11.1 implement the algorithm.

Table 11.1 Recursive Least Squares Algorithm

Given is a set of P measured data points that are used for training:
{x( p), y( p); p = 1, 2, . . . , P}
consisting of the input pattern vector x and the desired response y;
x = [1 x0 x1 x2 K xn ]
T

The weight vector

w = [ w0 w1 w2 ... wn ]
is to be constructed.
Perform the following training steps for p = 1, 2, …, P.
Step 1 Set the iteration index k = 0. Initialize the weight vector w ( 0 ) = 0 and the matrix P ( 0 ) = α I ( n +1) ,
where a should be very large number, say of the order of 108 to 1015.
Step 2 { }
Apply the next (p = 1 for the first one) training pair x ( p ) , y ( p ) to the linear neuron.

Step 3 Calculate the error for the applied data pair: e( k + 1) = y (k + 1) − xT ( k + 1)w T (k ) .
Step 4 Calculate the vector K (k ).
K (k ) = P (k ) x (k + 1)[1 + x T(k + 1)P (k ) x (k + 1)]−1
Step 5 Calculate the updated weight vector.
wT (k + 1) = wT (k ) + K (k )e(k + 1)
Step 6 Find the matrix P (k + 1)
P (k + 1) = P (k ) − K (k ) xT (k + 1)P (k )
Step 7 Stop the adaptation of the weights if the error is smaller than the predefined value. Otherwise set
k ® k + 1 and go back to step 2.

11.4.3 Reinforcement Learning

The reinforcement learning problem is meant to be a straightforward framing of the problem of learning
from interaction to achieve a goal. The learner and decision-maker is called the agent. The thing it interacts
with, comprising everything outside the agent, is called the environment. These interact continually; the
agent selecting actions and the environment responding to those actions and presenting new situations to the
agent. Reinforcement learning is learning what to do—how to map situations to actions.
Beyond the agent and the environment, one can identify three sub-elements in the framework of rein-
forcement learning: a policy, a cost function, and a value function. A policy defines the learning agent’s way
of behaving at a given time. Roughly speaking, a policy is a mapping from perceived states of the environ-
ment to actions to be taken when in those states. A cost function defines the goal in a reinforcement learning
problem. Roughly speaking, it maps perceived states (or state-action pairs) of the environment to a single
number, a cost, indicating the intrinsic desirability of the state. Whereas a cost function indicates what is
good in an immediate sense, a value function specifies what is good in the long run. Roughly speaking, the
value of a state is the total amount of cost an agent can expect to incur over the future, starting from that state.
While costs determine the immediate, intrinsic desirability of the environment states, values indicate the
 646 Digital Control and State Variable Methods
long-term desirability of states, after taking into account the states that are likely to follow, and the costs
attributed to those states.
The agent and environment interact at each of the sequence of discrete time steps. At each time step,
the agent receives some representation of the environment’s state, and on that basis selects an action. One
time step later, in part as a consequence of its action, the agent incurs a numerical cost and finds itself in a
new state.
Action choices are made on the basis of value judgments. We seek actions that bring about states of
lowest values, not lowest costs, because these actions obtain for us the lowest amount of cost over the long
run. Unfortunately, it is much harder to determine values than it is to determine costs. Costs are basically
directly given by the environment, but values must be estimated and re-estimated from the sequence of
observations an agent makes over its entire lifetime. Infact, the most important component of almost all
reinforcement learning algorithms, is a method for efficiently estimating values.
Figure 11.17b shows a reinforcement learning framework; this framework is abstract and very flexible,
allowing it to be applied to many different problems in many different ways. (The Q-learning algorithm
defined in Section 10.6 can also be framed in this framework). The policy structure is known as the actor,
because it is used to select actions; and the estimated value function is known as the critic, because it
criticizes the actions made by the actor. The critic must learn about, and critique whatever policy is currently
being followed by the actor. The critique, a scalar signal, is the sole output of the critic and drives all
learning in both actor and critic networks.
The reinforcement learning is similar in some aspects to supervised learning, and dissimilar in some other
aspects. The target function to be learned is a control policy p that outputs an appropriate action, given the
current states s. In learning the target function p be supervised learning, each training example would be a
pair of the form (s, p (s)). In reinforcement learning, however, training information is not available in this
form. Instead the trainer provides only a sequence of immediate cost values as the agent executes its
sequence of actions. The agent, therefore, faces the problem of determining which of the actions in its
sequence are to be credited with reducing the eventual costs. The agent influences the distribution of train-
ing examples by the action sequence it chooses. Thus in supervised learning scheme, the agent is trained by
an expert who shows it examples of optimal action sequences, while in reinforcement learning the agent
must train itself by performing actions of its own choice. The only training information available to the
learner is the sequence of immediate costs.
Reinforcement learning is recommended when the knowledge needed for supervised learning is not
available.

11.4.4 Unsupervised Learning

In unsupervised learning, there is no teacher to provide any feedback information (see Fig. 11.17c). There is
no feedback from the environment to say what the outputs should be, or whether they are correct. The
network must discover for itself, patterns, features, regularities, correlations, or categories in the input data,
and code for them in the output. While discovering these features, the network undergoes changes in
its parameters; this process is called self-organizing. A typical example is making an unsupervised classifi-
cation of objects without providing information about the actual classes. Proper clusters are formed by
discovering the similarities and dissimilarities among the objects.
In general, self-organized learning involves the frequent modification of the network’s synaptic weights,
in response to a set of input patterns. The weight modifications are carried out in accordance with a set of
learning rules. After repeated application of these patterns to the network, a configuration emerges that is of
some significance. Basically, globel order emerges from numerous, originally random, local interactions.
This global order can ultimately lead to some form of congruous behaviour.
 Neural Networks 647
A special class of self-organizing neural networks is based on competitive learning. In competitive learn-
ing networks, the output neurons compete among themselves to be activated. Only a single output neuron is
active at a time. The output neuron that wins the competition is called the winner-takes-all neuron.
Self-Organizing Map The self-organizing map (SOM) developed by Kohonen is an unsupervised, com-
petitive learning, clustering network. The SOM is an artificial system that emulates certain mappings that
occur in brain. Our brain is dominated by the cerebral cortex, a very complex structure of billions of neurons
and hundreds of billions of synapses. The cortex is neither uniform nor homogeneous. It includes areas,
identified by the thickness of their layers and the types of neurons within them, that are responsible for
different human activities (motor, visual, auditory, sensory, etc.), and thus are associated with different
sensory inputs. We can say that each sensory input is mapped into a corresponding area of the cerebral
cortex; in other words, the cortex is a self-organizing computational map in the human brain.
Kohonen formulated the principle of topographic map formation. This principle states that the spatial
location of an output neuron in the topographic map corresponds to a particular feature of the input pattern.
The basic idea of SOM is that inputs (from a primary event space) are received by a simple network of
adaptive elements; the signal representations are mapped (automatically) onto a set of outputs, in such a
manner that the responses attain the same topological order as that of the primary events.
Figure 11.19 shows a conventional feature-mapping architecture. The inputs to the network can be
written in the vector form as
x = [x1 x2 ... xn]T
and the synaptic weight vector of the neuron i in the two-dimensional array is given by
w i = [ wi1 wi 2 ... win ] ; i = 1, 2,…, m
T

where m is the total number of output neurons in the array. The best match of the input vector x with
the synaptic weight vector wi is determined from the Euclidean distance between the input and the weight
vector. The output unit with the smallest Euclidean distance, i.e.,
min || x – w i ||
∀i
in the winning neuron q.
The next step in the Kohonen’s algorithm is to update the synaptic weight vector associated with the
winning neuron and the neurons within a defined neighborhood Nq of the winning neuron. The learning rule
is given by
⎧⎪ wi (k ) + η (x(k ) – wi (k )) if i ∈ Nq
wi (k + 1) = ⎨ (11.24)
⎪⎩ wi (k ) if i ∉ Nq
where h is the learning rate parameter, and k is the iteration index.
The winner’s neighbourhood may include neurons within one, two or even three positions on either side.
Generally, training a SOM begins with the winner’s neighbourhood of a fairly large size. Then, as training
proceeds, the neighbourhood size gradually decreases.
Initialization of the network weights can be carried out by either randomly initializing them, or selecting
a set of weights that reflect some a priori knowledge about the input data, that is, information regarding the
possible distribution of the output clusters. Stopping conditions can be, for example, based on total number
of specified iterations or based on monitoring the weight changes.
Given is a set of P patterns {x( p); p = 1, 2, ... , P} to be divided into m clusters. On each iteration, a pattern
( p)
x is presented to the network. Repeated presentations (that is, large number of iterations) of the input
patterns to the network are normally required before the map begins to “unfold”.
 648 Digital Control and State Variable Methods

neuron m

Nq

w1n wm1
wm2 wmn
w12
w11

x1 x2 . . . xn
Fig.11.19 Conventional feature-mapping architecture of SOM

A competitive learning network performs an on-line clustering process on the input patterns. A cluster
centre’s position is specified by the weight vector connected to the corresponding output unit. The update
formula in Eqn.(11.24) implements a sequential scheme for finding the cluster centres of a data set. When an
input x( p) is presented to the network, the weight vector closest to x( p) rotates towards it. Consequently,
weight vectors move toward those areas where most inputs appear, and, eventually, the weight vectors
become the cluster centres for the data set. When the process is complete, the input data are divided into
disjoint clusters such that similarities between individuals in the cluster are larger than dissimilarities in
different clusters. Here the dissimilarity measure of the Euclidean distance is used as a metric of similarity.
Obviously, other metrics can be used, and different selections lead to different clustering results.
Training the RBF Network As an illustration, consider the problem of designing and training an RBF
network shown in Fig. 11.15. It follows from Eqns (11.20) that there are two sets of parameters governing
the mapping properties of the RBF network: the weights wji in the output layer, and the parameters (ci, si) of
the radial basis functions.
Several schemes have been proposed to learn the parameters ci and si of the RBF units. One scheme is to
learn only the centres ci and therefrom determine the spread parameters si. The basic idea is to ensure
suitable overlapping of the basis functions. A rule of thumb is to take si equal to, or a multiple of, the average
distance to the several nearest neighbors of ci. The centres ci are learned by means of data clustering based
on competitive learning.
Once the centres and the spread parameters are chosen, the weights wji in the output layer of the network
in Fig. 11.15, can be determined as follows.
Output neuron j in Fig. 11.15 is driven by a signal vector f(×) produced by the layer of RBF neurons, which
are themselves driven by input vector (stimulus) x applied to the input terminals. For fixed RBF centres and
spread parameters, the output yˆ j of the network can be computed as (refer to Eqn. (11.20b))
m
yˆ j = ∑ w jiφi ( x ) ; j = 1, 2, …, q
i =1
Supervised learning may be visualized as learning an environment, with the help of a teacher having
(1) (1) (2) (2) ( p) ( p)
knowledge of the environment in the form of input-output examples ( x , y j ), ( x , y j ),K , ( x , y j ), K
 Neural Networks 649
For fixed RBF centres and spread parameters, the knowledge may be translated (refer Eqns (11.20)) to the
(1) (2) ( p)
form (φ , y j ), (φ , y j ),..., (φ , y j ),... . Suppose now the neuron j in output layer is exposed to the
(1) (2) ( p)
( p)
vector f . By such virtue of the built-in knowledge, the teacher is able to provide the neural network with
a desired response y (j p ) from f ( p). The network parameters are adjusted under the combined influence of the
training vector f ( p), and the error signal ej = y j − yˆ j ; which is the difference between the desired
( p) ( p) (p

response and the actual response of the network. The least squares estimation given by Eqns (11.23) or
recursive least squares algorithm described in Table 11.1, may be used for learning the weights wji.

11.5 TRAINING THE MULTILAYER NEURAL NETWORK—BACKPROPAGATION

TUNING

Gradient descent is a basic technique that plays an important role in learning algorithm for neural networks.
Application of this technique to a single-layer network will be discussed as a prelude to backpropagation
learning algorithm for multilayer feedforward neural networks.
Consider the one-layer NN shown in Fig. 11.11 and described by the recall equation (refer Eqns (11.11))
F wx
n I ; j = 1, 2, ..., q
y$j = s GH ∑
i =1
ji i + wj 0 JK (11.25a)

or in matrix form (refer Eqns (11.13))

y$ = G(Wx + w0) (11.25b)
Since NN training is usually performed using digital computers, a convenient form of weight update
equation is expressed in terms of discrete iterations. We denote the iteration index by k. One should not think
of k as time index as the iteration index is not necessarily the same as the time index. Let wji(k) be the NN
weights on iteration k so that
F w (k ) x
n I
y$ j (k) = s GH ∑
i =1
ji i JK
+ wj 0 ( k ) ; j = 1, 2, ..., q (11.26)

In this equation, xi are the components of a prescribed constant input vector x, that stays the same during
the training of the NN. A general class of weight update algorithms is given by the recursive update equation
¶E ( k )
wji(k + 1) = wji(k) – h (11.27)
¶wji ( k )
where E(k) is a cost function that is selected depending on the application. In this algorithm, the weights wji
are updated at each iteration number k in such a manner that the prescribed cost function decreases. This is
¶E ( k )
accomplished by going “downhill” against the gradient . The positive step-size parameter h is taken
¶wji ( k )
as less than 1 and is called the learning rate. To see that the gradient descent algorithm decreases the cost
function, note that
Dwji(k) º wji(k + 1) – wji(k) (11.28a)
and, to first order
DE(k) º E(k + 1) – E(k)
» å ¶E( k ) Dwji(k)
j , i ¶wji ( k )
F ¶E(k) I
» –h å G
2

H ¶w (k) JK
j, i ji
(11.28b)
 650 Digital Control and State Variable Methods
Let a prescribed vector x be input to the NN and desired target output, associated with x, be y. Then at
iteration index k, jth component of the output error is
ej(k) = yj – y$ j (k) (11.29)
where yj is the desired output and y$ j (k) is the actual output with input x. We define the cost function as least-
squares NN output error:
q q
2
E(k) = 1
2 å e2j (k) = 12 å d yj - y$j ( k ) i (11.30)
j =1 j = 1
Note that the components xi of the input x and the desired NN output components yj are not functions of
the iteration index k.
To derive the gradient descent algorithm with the least-squares output-error cost, the gradients with
respect to the weights and biases are computed using the product rule and the chain rule as
¶E ( k ) ¶y$j (k ) F w (k) x
n I
¶wji ( k )
= –ej(k)
¶wji( k )
= – ej (k)s ¢ GH å
i =1
ji i JK
+ wj 0 (k ) xi (11.31a)

¶E ( k ) F w (k ) x
n I
¶wj 0 ( k )
= – ej(k)s ¢ GH å
i =1
ji i + w (k ) J
j0
K
(11.31b)

where Eqns (11.30) and (11.26) were used. The notation s¢ (×) denotes the derivative of the activation
function evaluated at the argument. The argument is the activation value aj :
n
aj =
å wji( k ) xi + wj 0 ( k )
i = 1

s¢(aj) = ds
daj
For a linear activation function, s (aj) = aj and s¢(aj) = 1. The gradient descent algorithm becomes
wji (k + 1) = wji (k) + hej (k) xi (11.32a)
wj0 (k + 1) = wj0 (k) + hej (k) (11.32b)
For a sigmoidal activation function
1
s (aj) = -a
1+ e j
1 -a
s ¢(aj) = –
-aj 2
-e j e j
1+e e j
1 F e I = s (a )[1– s (a )]
-aj
=
1+ e
- aj GH 1 + e JK - aj j j (11.33)

The gradient descent algorithm becomes

wji (k + 1) = wji (k) + hej (k) y$ j (1 – y$ j ) xi (11.34a)
wj0 (k + 1) = wj0 (k) + hej (k) y$ j (1 – y$ j ) (11.34b)
In terms of matrices, algorithm (11.32) may be expressed as
W(k + 1) = W(k) + he(k)xT (11.35a)
where e(k) = y – y$ (k)
The bias vector w0 is updated according to
w0(k + 1) = w0(k) + he(k) (11.35b)
Similar expressions may be written down for the algorithm (11.34).
 Neural Networks 651
We have just discussed NN weight training procedure when one input-vector/desired output-vector pair
(x, y) is given. In practical situations, there will be multiple input vectors prescribed by the user, each with an
associated desired output vector. Suppose there are presecibed P desired input/output pairs ( x (1) , y (1) ) ,
( x ( 2 ) , y ( 2 ) ) , ..., ( x( P ) , y ( P ) ) for the NN. In such situations, the NN must be trained to associate each input
vector with its prescribed output vector. There are many strategies for training the net in this scenario; at the
two extremes are incremental updating and batch updating. For this discussion, we shall use matrix updates.
Define for p = 1, 2, ..., P the quantities
y$ ( p )(k) = G (W(k)x(p)+ w0)
e(p)(k) = y(p) – y$ ( p ) (k) (11.36)
(p) 1 ( p) T (p)
E (k) = (e ( k )) e (k)2
In incremental updating, the vectors (x(p), y(p)) are sequentially presented to the NN. At each presentation,
one step of the training algorithm is performed, so that (refer Eqns (11.35))
W(k + 1) = W(k) + he(p)(k)(x(p))T; p = 1, 2, ..., P (11.37a)
(p)
w0(k + 1) = w0(k) + he (k); p = 1, 2, ..., P (11.37b)
which updates both the weights and biases. An epoch is defined as one complete run through all the P associ-
e j
ated pairs. When one epoch has been completed, the pair x (1) , y (1) is presented again and another run through
all the P pairs is performed. It is hoped that after many epochs, the output error will be small enough.
In batch updating, all P pairs are presented to the NN (one at a time) and a cumulative error is computed,
after all have been presented. At the end of this procedure, the NN weights and biases are updated once. The
result is
P
W(k + 1) = W(k) + h ∑ e(p)(k)(x(p) )T (11.38a)
p = 1
P
∑e
(p)
w0(k + 1) = w0(k) + h (k) (11.38b)
p =1
In batch updating, the iteration index corresponds to the number of times the set of P patterns is presented
and the cumulative error computed. That is, k corresponds to the epoch number.

11.5.1 Backpropagation Algorithm

We shall derive the backpropagation algorithm for the two-layer NN shown in Fig. 11.13. We shall conserve
simplicity of notation by dispensing with the iteration index k. Let there be a prescribed input vector x and an
associated desired output vector y for the network. Define the least-squares NN output error as
q
E= 1
2 å e2j
j =1

ej = yj – y$ j
where y$ j is evaluated using the equations
m
y$ j = å vjlzl; z0 º 1
l = 0

F w x I; x
n
zl = s GH å JK
i = 0
li i 0 º 1

with the components xi of the input vector.

 652 Digital Control and State Variable Methods
We first consider the output layer of the network. Weights and biases in the layer are updated according to
¶E
vjl = vjl – h
¶vjl
With linear activation functions in the output layer, the update rule becomes (refer Eqns 11.32))
vjl = vjl + hej zl
We now consider the hidden layer of the network. Unlike the output nodes, the desired output of the
hidden nodes is unknown. For a given input/output pair, (x, y), the backpropagation algorithm performs two
phases of data flow. First, the input pattern x is propagated from the input terminals to the output layer and,
as a result of the forward flow of data, it produces an actual output y$ . Then the error signals, resulting from
the difference between y$ and y, are backpropagated from the output layer to the previous layers, to update
their weights. Error backpropagation may be computed by expanding the error derivative using the chain
rule as follows:
zl = s (al)
n
al = å wli xi; x0 º 1
i = 0
¶E
wli = wli – h
¶ w li
¶E ¶E ¶al LM q
¶E ¶y$j ¶zl OP ¶a l

MN å
= =
¶wli ¶al ¶wli j =1 ¶y$ j ¶zl ¶al PQ ¶w
li
¶E
= – ej
¶y$ j
¶y$ j
= vjl
¶zl
zl = s¢ (a ) = z (1 – z )
l l l
¶al
¶al = x
i
¶w i
Therefore
¶E LM b
= –xi zl 1 - zl g å v e PPO
q
jl j
¶ w li MN j =1 Q
and the update rule becomes
q
wli = wli + h xi zl(1 – zl) å vjl ej
j =1
The backpropagation algorithm consists of repeating the following iterative procedure until the NN
output error has become sufficiently small. Incremental or batch processing may be used.
Forward Recursion to Compute NN Output Present input vector x to the NN, and compute the NN
output using
F w x I ; l = 1, 2, ..., m
n
zl = s GH å JK
i = 0
li i (11.39a)

F m I
y$ = G å v z J ; j = 1, 2, ... , q (11.39b)
j jl l
H l = 0 K
with x0 = 1 and z0 = 1, where y is the desired output vector.
 Neural Networks 653
Backward Recursion for Backpropagated Errors
ej = yj – y$ j ; j = 1, 2, ..., q (11.39c)
q
dl = zl(1 – zl) å vjl ej; l = 1, 2, ..., m (11.39d)
j =1
Computation of the NN Weights and Bias Updates
njl = njl + hzl ej; j = 1, 2, ..., q; l = 0, 1, ..., m (11.39e)
wli = wli + hxi dl; l = 1, 2,..., m; i = 0, 1, ..., n (11.39f)

11.5.2 Improvements on Gradient Descent

There are many sorts of training algorithms for NN; the basic type we have discussed in the previous subsec-
tion is the Backpropagation Training Algorithm. Though the backpropagation algorithm enjoys great
success, one must remember that it is a gradient-based technique, so that the usual caveats associated with
step sizes, local minima and so on, must be kept in mind while using it.
The NN weights and biases are typically
Error
Case 1 initialized to small random (positive and negative)
Case 2
(Bad initial (Learning rate values. A typical error surface graph in 1-D is
condition) too large) shown in Fig. 11.20, which shows a local minimum
and a global minimum. If the weight is initialized as
shown in Case 1, there is a possibility that the gra-
dient descent might find the local minimum. Sev-
eral authors have determined better techniques to
Local initialize the weights than the random selection,
minimum particularly for the multilayer NN. Among these are
Global
minimum Nguyan and Widrow, whose techniques are used,
for instance, in MATLAB. Such improved initiali-
zation techniques can also significantly speed up
convergence of the weights to their final values.
Weight
An improved version of gradient descent is
Fig. 11.20 Typical 1-D NN error surface
given by Momentum Gradient Algorithm. Momen-
tum allows a network to respond, not only to the local gradient, but also to recent trends in error surface. The
learning rule with the inclusion of a momentum term can be written as (refer Eqn. (11.27))
¶E ( k )
Dwji(k) = –h + a Dwji(k – 1); 0 £ | a | £ 1 (11.40)
¶wji ( k )
Without momentum, a network may get stuck in a shallow local minimum; adding mementum can help the
NN “ride through” local minima. (Case 1 in Fig. 11.20 may not get stuck in local minimum while learning
with momentum). In the MATLAB Neural Network Toolbox are some examples showing that learning with
momentum can significantly speed up and improve the performance of backpropagation.
Only small learning constants h guarantee a true gradient descent. The price of this guarantee is an
increased total number of learning steps that need to be made to reach a satisfactory solution. It is desirable
to monitor the progress of learning so that h can be increased at appropriate stages of training to speed up the
minimum seeking.
When broad minima yield small gradient values, then a larger value of h will result in a more rapid
convergence. However, for problems with steep and narrow minima, if the learning rate h is too large, then
the NN can overshoot the minimum cost value, jumping back and forth over the minimum, and failing
to converge, as shown in Fig. 11.20, Case 2. Adapting the learning rates can significantly speed up the
convergence of the weights.
 654 Digital Control and State Variable Methods
All the refinements: selecting better initial conditions, using learning with momentum, and using an
adaptive learning rate, are available in the MATLAB NN Toolbox.
In practice, the gradient method is quite slow. Other methods are available which converge much faster. In
most applications, it is therefore difficult to justify using the gradient method. Nevertheless, the method has
gained a remarkable popularity in the neural network community. The primary properties in favour of the
method are the simplicity at which it is implemented, and the modest requirement of data storage. In most
situations, the drawback associated with slow convergence motivates the use of more sophisticated methods.
The category of fast algorithms uses standard numerical optimization techniques. Three types of numeri-
cal optimization techniques for neural network training have been incorporated in MATLAB: Conjugate
gradient; quasi-Newton; and Levenberg-Marquardt.
The reader is advised to refer the literature [137–143] for details on improvements suggested above.

11.6 FUNCTION APPROXIMATION WITH NEURAL NETWORKS

Of fundamental importance in NN closed-loop control applications is the universal function approximation

property of NNs having at least two layers (one-layer NNs do not generally have a universal approximation
capability).
The basic universal approximation result says [141] that any smooth function f(x) can be approximated
arbitrarily closely on a compact set using a two-layer NN with appropriate weights. This result has been
shown using sigmoid activations, RBF activations, and others. Specifically, let f(x) be a smooth function;
x = [x1 x2 ... xn]T, f(×) = [f1(×) f2(×) ... fq(×)]T, S be a compact set in n-dimensional state space and eN be a
positive number. There exists a two-layer NN (Eqn. (11.19b)) such that
f(x) = V(G(Wx + w0) + v0) + e (11.41)
with ||e || < eN for all x Î S, for some (sufficiently large) number m of hidden-layer neurons. The value e
(generally a function of x), is called the NN function approximation error, and it decreases as the hidden-
layer size m increases. We say that on the compact set S, f(x) is ‘within eN of the NN functional range’.
Approximation results have also been shown for smooth functions with a finite number of discontinuities.
Note that in this result, the activation functions are not needed on the NN output layer (i.e., the output
layer activation functions are linear). It also happens that the bias terms nj0 on the output layers are not
needed, though the hidden layer bias terms wl0 are required.
Note further that, though the result says ‘there exists an NN that approximates f(x)’, it does not show how
to determine the required number of units in the hidden layer. The issue of finding the required number of
units in the hidden layer such that an NN does indeed approximate a given function f(x) closely enough, is
not an easy one (If the function approximation is to be carried out in the context of a dynamic closed-loop
feedback control scheme, the issue is thornier and is discussed in subsequent sections). This issue has been
addressed in the literature [138, 141], and a significant result has been derived about the approximation
capabilities of two-layer networks when the function to be approximated exhibits a certain smoothness.
Unfortunately, the result is difficult to apply for selecting the number of hidden units. The guidelines to
select the appropriate number of hidden neurons are rather empirical at the moment. To avoid large number
of neurons and the corresponding inhibitively large training times, the smaller number of hidden layer neu-
rons are often used in the first trial. One increases accuracy by adding more hidden neurons. We will learn in
the next section that excessively large number of hidden units may lead to poor generalization, a key feature
of the performance of NN. Note that the number of inputs and outputs in the neural network are determined,
respectively, by the number of the data presented to it, and the type of the output desired from it.
Because of the above-mentioned results, one might think that there is no need for using more than
one hidden layer, and/or mixing different types of activation functions. This is not quite true: it may be that
 Neural Networks 655
accuracy can be improved using a more sophisticated network architecture. In particular, when the complexity
of the mapping to be learned is high (e.g., functions with discontinuities), it is likely that the performance can be
improved. Experimental evidence tends to show that using a two hidden-layer network for continuous func-
tions has sometimes advantages over a one hidden-layer network, as the former requires shorter training times.
As an illustration of the NN function approximation in the context of closed-loop control, revisit the
problem of control of two-link robot manipulator discussed earlier in Section 10.2. Refer to [140] for the
detailed account of learning the nonlinear functions f (x) and g–1(x) (Fig. 10.2) using neural networks.
In control applications and signal processing, learning machines are used to predict output signals y =
f (x) based on the observed data. In the case of character recognition tasks, learning machines are used to
recognize particular alphabetic, numeric, or symbolic characters based on the data obtained by scanning a
piece of paper. Two different types of outcomes: continuous variables in control applications and signal
processing, and categorical variables (class labels) in pattern recognition, are involved. The function ap-
proximation problem is a regression problem if the outcome is continuous variables; and it is a classification
problem if the outcome is categorical variables. Because of their utmost importance in control applications,
the book deals only with the regression models.

11.7 SYSTEM IDENTIFICATION WITH NEURAL NETWORKS

The main goal of the present chapter is to describe approaches to neural-network-based control that are
found to be practically applicable to a reasonably wide class of unknown nonlinear systems. Systems
identification is an integral part of such a control system design and, consequently, it calls for considerable
attention as well. The system identification is necessary to establish a model based on which the controller
can be designed, and it is useful for tuning and simulation before applying the controller to the real system.
In this section, attention is drawn to identification of neural network models for nonlinear dynamic systems
from a series of measurements on the systems.
We give here a generic working procedure for system identification with neural networks. One should be
aware, of course, that such a procedure must always be squeezed a little here and there to conform to the
application under consideration.
The multilayer feedforward network is straightforward to em-
Experiment ploy for the discrete-time modeling of dynamic systems for which
there is a nonlinear relationship between the system’s input and
output. Let k count the multiple sampling periods so that y(k)
specifies the present output while y(k – 1) signifies the output
Select
model structure observed at the previous sampling instant etc. It is assumed that
the output of the dynamic system at discrete-time instances can be
described as a function of number of past inputs and outputs:
y(k) = f (y(k – 1), ..., y(k – n), u(k – 1), ..., u(k – m)) (11.42)
Train model
A multilayer network can be used for approximating f (×) if the
inputs to the network are chosen as the n past outputs and m past
inputs of the dynamic system.
Validate model When attempting to identify a model of a dynamic system, it is
Not accepted a common practice to follow the procedure depicted in Fig. 11.21.

Accepted 11.7.1 Experiment

Fig. 11.21 System identification The primary purpose of an experiment is to produce a set of
procedure examples of how the dynamic system to be identified responds to
 656 Digital Control and State Variable Methods
various control inputs (These examples can later be used to train neural network to model the system). The
experiment is particularly important in relation to nonlinear modelling; one must be extremely careful to
collect a set of data that describes how the system behaves over its entire range of operation. The following
issues must be considered in relation to acquisition of data (For detailed information, see [131]).
Sampling Frequency The sampling frequency should be chosen in accordance with the desired dynamics of
the closed-loop system consisting of controller and the system. A high sampling frequency permits a rapid
reference tracking and a smoother control signal, but the problems with numerical ill-conditioning will become
more pronounced. Consequently, the sampling frequency should be selected as a sensible compromise.
Input Signals While for identification of linear systems, it is sufficient to apply a signal containing a
finite number of frequencies, a nonlinear system demands, roughly speaking, that all combinations of fre-
quencies and amplitudes in the system’s operating range are represented in the signal. As a consequence, the
necessary size of the data set increases dramatically with the number of inputs and outputs. Unfortunately,
there is no obvious remedy to this curse of dimensionality.
Before an input signal is selected, it is important to identify the operating range of the system. Special
care must be taken not to excite dynamics that one does not intend to incorporate in the model (e.g.,
mechanical resonances).
Processing the Data Intelligent processing of the data is often much more important than trying a large
number of different model structures and training schemes. Many different types of processing can be con-
sidered for extracting the most valuable information from the measured data, and to make it suitable for
neural-network modelling. Some suggestions are given in the following paragraphs.
Filtering is widely used for removing from the measured signals, noise, periodic disturbances, offsets,
and the effects of “uninteresting” dynamics. When high-frequency noise/disturbances cause problems, it is
recommended to remove them by using an analog presampling filter to avoid an aliasing phenomenon.
Offset, drift, and low-frequency disturbances can be removed by filtering the data after the sampling.
Sometimes, a large number of input-output pairs from a small regime of entire operating range, dominates
the data set. When training on such a data set, it is likely that the model obtained will be very accurate in the
regime that was over-represented at the expense of poor performance outside the regime. A little “surgery”
on the data set might be necessary here to eliminate redundant information. Apart from obtaining a more
equal weighting of the information, a reduction of the data set size also has the benefit that training times will
be reduced.
It is also recommended to remove outliers from the data set (or, alternatively, insert interpolated values of
the output signal). Outliers will often have a fatal impact on the training model.
Before training, it is often useful to scale all the signals so that they always fall within a specified range,
say [–1, 1]. Another approach for scaling is to normalize the mean and standard deviation of the training set,
e.g., to zero mean and unity standard deviation. The signals are likely to be measured in different physical
units, and without scaling there is a tendency that the signal of largest magnitude will be too dominating.
Moreover, scaling makes the training algorithm numerically robust and leads to faster convergence.

11.7.2 Model Structure Selection

The model structure selection is basically concerned with the following two issues:
· Selecting an internal network architecture
· Selecting the inputs to the network
An often-used approach is to let the internal architecture be feedforward multilayer network. Probably the
most commonly used network architecture is a two-layer feedforward network with hyperbolic tangent
hidden units and linear output units. This architecture works quite well in many practical applications. In our
 Neural Networks 657
presentation, we use this architecture. However, the reader is referred to more fundamental textbooks/
research papers for a treatment of other types of neural networks in the control loop.
The input structure we use here consists of a number of past inputs and outputs (Refer Fig. 11.22):

y$ ( k | q ) =
M
F N
I
ånls GH å wlif i (k ) + wl0 JK + n0 (11.43a)
l= 1 i =1
where y$ is the predicted value of the output y at sampling instant t = kT (T = sampling interval), q = {nl wli}
is the vector containing the adjustable parameters in the neural network (weights), f is the regression vector
which contains past outputs and past inputs (regressors’s dependency on the weights is ignored):
f (k) = [y(k – 1) L y(k – n) u(k – 1) L u(k – m)]T (11.43b)
= [f1(k) f2(k) L fN (k)]T

f1(k) = y(k – 1) z1

y (k – 2)

n0
y (k – n ) z2 y (k )
u (k – 1)

u (k – 2)

zM
f N (k ) = u (k – m )

Fig. 11.22 Input structure

Often, it is of little importance that the network architecture has selected vector q too small or too large.
However, a wrong choice of lag space, i.e., the number of delayed signals used as regressors, may have a
disastrous impact on some control applications. Too small obviously implies that essential dynamics will not
be modeled, but too large can also be a problem. From the theory of linear systems, it is known that too large
a lag space may manifest itself as common factors in the identified transfer function. An equivalent behav-
iour must be expected in the nonlinear case. Although it is not always a problem, common factors (corre-
sponding to hidden modes) may lead to difficulties in some of the controller designs.
It is necessary to determine both, a sufficiently large lag space and an adequate number of hidden units.
While it is difficult to apply physical insight towards the determination of number of hidden units, it can
often guide the proper lag space. If the lag space is properly determined, the model structure selection
problem is substantially reduced. If one has no idea regarding the lag space, it is sometimes possible to
determine it empirically.

11.7.3 Training
Assume now, that a data set has been acquired and that some model structure has been selected. According
to the identification procedure in Fig. 11.21, the next step is to apply the data set to pick “the best” model
among the candidates contained in the model structure. This is the training stage. The training can be
 658 Digital Control and State Variable Methods

computationally intensive, but it is generally one of the easiest stages in the identification. It is not very
difficult to implement a training algorithm in a computer, but one might as well resort to one of the many
available software packages, e.g., MATLAB.
The training procedure can be rephrased in more formal terms. Given a set of data
ZP = {[u(k), y(k)], k = 1, ..., P} (11.44a)
and a set of candidate models
y$ (k |q) = g[f(k),q] (11.44b)
the purpose of the training is to determine a mapping from the data set to the set of candidate models
ZP ® q$ (11.44c)
so that a model is obtained which provides predictions that are in some sense close to the true outputs of the
system. The most commonly used measure of closeness for this type of problems is specified in terms of a
mean square error criterion
1 P
J(q, ZP) = å [y(k) – y$ (k | q)]2
2 P k =1
(11.45)

The most appealing feature of mean square error criterion is the simplicity with which a weight update
rule can be derived. The principle of the gradient (descent) iterative search method, is that at each iteration,
the weights are modified along the opposite direction of the gradient. That is, the search direction is selected
¶J
as – .
¶q
¶J
q(i + 1) = q (i) – h(i) ( i ) (11.46)
¶q
When applying the gradient method to the training of multilayer feedforward networks, it is useful to order
the computations in a fashion that utilizes the particular structure of the network. The method, called
the backpropagation algorithm, was discussed in Section 11.5. Batch method of the backpropagation
algorithm refers to the fact that each iteration on the parameter vector requires an evaluation of the entire
data set, ZP.
It is sometimes useful to identify a system online, simultaneously, with the acquirement of measurements.
Adaptive control is an example of such an application. In this case, a model must be identified and a control
system designed online, because the dynamics of the system to be controlled vary with time. Obviously
batch methods are unsuitable in such applications as the amount of computation, required in each iteration,
might exceed the time available within one sampling interval. Moreover, old data will be obsolete when the
system to be identified is time-dependent.
In a recursive algorithm, one input-output pair from the training set, [f(k), y(k)], is evaluated at a time
and used for updating the weights. In the neural network community, this is frequently referred to as incre-
mental or online backpropagation (refer to Section 11.5).
Improving Generalization The NN modelling error may be separated into two contributions:
(i) The bias error The portions of the error that are due to insufficient model structure (number of
hidden units, lag space), and an insufficient sample time.
(ii) The variance error The portion of the error that is due to the fact that the function implemented by
the network on a specific data set deviates from the average function.
The bias error is, in practice, inevitable; there will always be a certain error even with large data sets.
However, typically, one will find that the bias error decreases as more weights are added, since the network
 Neural Networks 659
will be able to describe the system more accurately. The reason for not just selecting a gigantic network
architecture, is that one must expect the variance error to work in the opposite direction: as more weights are
estimated on the same data set, the variance on the estimated weights will increase. This quandary is often
referred to as the bias/variance dilemma [138]. If a network has more hidden units, the variance error
dominates. To describe this situation, it is common to use the expression overfitting. Overfitting means that
the network not only models the features of the system, but, to an undesired extent, also the noise in the
training set. The bias error on the training set is driven to a very small value with large number of hidden
units, but when new data are presented to the network, the error is large. The network has memorized the
training examples, but it has not learnt to generalize to new situations. Likewise, the expression underfitting
is used when the bias error dominates.
One method for improving network generalization, is to use a network which is just large enough to
provide an adequate fit. However, it is difficult to know beforehand, how large a network should be selected
for a specific application. There are two other methods for improving generalization: regularization and
early stopping.
Regularization This means, to augment the mean square error criterion with a regularization (or complex-
ity) term. The most commonly used augmentation is the so-called simple weight decay term:
1 P 1
J= å
2 P k =1
[y(k) – y$ (k | q)]2 +
2P
a q Tq (11.47)
where a denotes the weight decay.
Early Stopping An effect similar to regularization can be accomplished by stopping the training session,
before the minimum of the criterion has been reached. Error reduces in the beginning; after some iterations
it reaches a minimum, and then starts growing. Loosely speaking, the network initially captures the essential
features of the system, but after a while it adapts mostly to the noise.

11.7.4 Validation
In the validation stage, the trained model is evaluated to clarify if it represents the underlying system
adequately. Ideally, the validation should be performed in accordance with the intended use of the model. As
it turns out, this is often rather difficult. For instance, if the intention is to use the model for designing a
control system, the validation ought to imply that a controller was designed and its performance tested in
practice. For most applications, this level of ambition is somewhat high, and it is common to apply a series
of simple “standard” tests instead of concentrating on investigating particular properties of the model.
Although this is less than ideal, it is good as a preliminary validation to quickly exclude really poor models.
Most of the tests require a set of data that was not used during training. Such a data set is commonly
known as test or validation set. It is desirable that the test set satisfies the same demands as the training set,
regarding representation of the entire operating range.
A very important part of the validation is to simply inspect the plot, comparing observed outputs to
predictions. Unless the signal-to-noise ratio is very poor, it can show the extent of overfitting as well as
possible systematic errors.
If the sampling frequency is high, compared to the dynamics of the system, a visual inspection of the
predictions will not reveal possible problems. Some scalar quantities (correlation functions) to measure the
accuracy of the predictions, have been suggested. Reliable estimates of the average generalization
error are also useful for validation purposes, but their primary application is for model structure selection.
The estimates are good for rapidly comparing different model structures to decide which one is likely to be
the best.
 660 Digital Control and State Variable Methods

11.8 CONTROL WITH NEURAL NETWORKS

Neural-network-based control constitutes a very large research field, and it is difficult to give a clear over-
view of the entire field. Here, in this section, an attempt has been made to outline a feasible path through the
“jungle” of neural network solutions. A completely automatic procedure for control system design is not
realistic; the emphasis is on the guidelines for working solutions.
It is believed that one of the most important lessons to be learnt from the numerous automatic control
applications developed over the past half century, is that simple solutions actually solve most problems quite
well. Regardless of the fact that all systems, to some extent, exhibit a nonlinear behaviour, it turns out that
they can often be controlled satisfactorily with simple linear controllers. When neural networks are intro-
duced as a tool for improving the performance of control systems for a general class of unknown nonlinear
systems, it should be done in the same spirit. A consequence of this philosophy is that our focus is on simple
control structures that yield good performance in practice.

11.8.1 Inverse Model of the System used as Controller

When neural networks were originally proposed for controlling unknown nonlinear systems, one of the first
methods being reported was on training a network to act as the inverse of the system, and use this as a
controller. This, in fact, amounts to linearization (the input-output transfer function unity) of the nonlinear
system by properly compensating for the nonlinearity involved (Refer Section 10.2). Explained in brief, the
basic principle is as follows:
Assume that the system to be controlled can be described by
y(k) = f1[y(k – 1), ..., y(k – n), u(k – 1),..., u(k – m)] (11.48)
Perform an experiment on the system to collect a set of data, that describes how the system behaves over
its entire range of operation:
{[u(k), y(k)], k = 1, ..., P} (11.49)
Using identification procedures described in the earlier section, we can infer a neural network model of
the system using this data set.
An inverse model of the system can be inferred from the data set
{[y(k), u(k)], k = 1, ..., P} (11.50)
The output of the inverse model is u(k):
u(k) = f2[(y(k + 1), y(k), ..., y(k – n + 1), u(k – 1),..., u(k – m + 1)] (11.51)
The inverse model can be used as controller for the system. Let the “desired” closed-loop system behave as
Y (z )
= M(z) = z–1; y (k + 1) = r(k) (11.52)
R( z )
Substitute in Eqn. (11.51), the output y(k + 1) by the desired output—the reference, r(k). If the network
represents the exact inverse, the control input produced by it will drive the system output at time k + 1 to r(k).
The principle is illustrated in Fig. 11.23a.
The most straightforward way of training a network as the inverse of a system, is to approach the problem
as a system-identification problem analogous to the one considered in the previous section—an experiment
is performed, a network architecture is selected, and the network is trained off-line. The difference from
system identification lies in the choice of regressors and network output. They are now selected as shown in
a functional relation (11.51). The network is then trained to minimize the criterion
1 P
J= å [u(k) – u$ (k | q)]2
2P k = 1
(11.53)

We will call this procedure, the general training procedure for an inverse model.
 Neural Networks 661

z–1

z–2
NN u (k)
Inverse System
Model y (k + 1)
r (k )

z–1

(a)
Desired output
yd = r NN Inverse u^ Output y
Plant
Model

(b)
Fig. 11.23 Inverse model of the system as a controller

This is basically an off-line procedure. After this training phase, the structure for an on-line operation
looks like the one shown in Fig. 11.23b, that is, the neural network representing the inverse of the plant
precedes the plant. The trained neural network should be able to take the desired output value yd = r as its
input, and produce appropriate û as an input to the plant.
The practical relevance of using an inverse model of the system as a controller is limited, due to a number
of serious inconveniences. The control scheme will typically result in a poor robustness with a high sensitiv-
ity to noise and high-frequency disturbances (corresponding to unity forward-path transfer function in the
linear case). In addition, one will often encounter a very active control signal, which may adversely affect the
system/actuators. If the system is linear, this occurs when its zeros are situated close to the unit circle. In the
nonlinear case, there is no unique set of zeros, but, of course, a similar phenomenon exists.
If the inverse model is unstable (corresponding to zeros of the system outside the unit circle in the linear
case), one must anticipate that the closed-loop system becomes unstable. Unfortunately, this situation occurs
quite frequently in practice. Discretization of linear continuous-time models under quite common circum-
stances, can result in zeros outside the unit circle—regardless that the continuous-time model has no zeros,
or all zeros are in the left half of the plane. In fact, for a model with a pole excess of at least two, one or more
zeros in the discretized model will converge to the unit circle, or even outside, as the sampling frequency is
increased. It must be expected that a similar behaviour can also be found in discrete models of nonlinear
systems.
Another problem with the design arises when the system to be controlled is not one-to-one, since then a
unique inverse model does not exist. If this non-uniqueness is not reflected in the training set, one can, in
 662 Digital Control and State Variable Methods
principle, yield a particular inverse which might be adequate for controlling the system. Most often, how-
ever, one will end up with a useless, incorrect inverse model.
Despite these drawbacks, in a number of domains (stable systems, and one-to-one mapping plants), this
general training architecture is a viable alternative.

11.8.2 Feedforward-Feedback Control

Many of the problems mentioned in the previous subsection can be taken care of by employing a control
structure of the form shown in Fig. 11.24. The feedforward control is used for improving the reference
tracking, while feedback is used for stabilizing the system and for suppressing disturbances.

NN Inverse
Model

uff

+ y
+ PID + u
r System
Controller ufb
–

Fig. 11.24 Feedforward-feedback control structure

An inverse model is trained as discussed earlier (refer Eqn. (11.51)):

u(k) = f [y(k + 1), y(k), ..., y(k – n + 1), u(k – 1), ..., u(k – m + 1)] (11.54)
The feedforward component of the control input is then composed by substituting all system outputs by
corresponding reference values:
uff (k) = f [r(k + 1), ..., r(k – n + 1), uff (k – 1), ..., uff (k – m + 1)] (11.55)
If the complete reference trajectory is known in advance, implementation of the scheme is particularly easy.
It is then possible to compute the contribution from the feedforward controller beforehand, and store the entire
sequence of control inputs {uff} for use in the computer program implementing the control system.
Although a neural network feedforward can be useful for optimizing many control systems, one must be
careful not to use it uncritically. An inaccurate feedforward control may actually harm, rather than enhance,
performance.

11.8.3 Model Reference Adaptive System

In the context of training inverse models, which are to be used as controllers, the trained inverse model,
somehow, ought to be validated in terms of performance of the final closed-loop system. This points out a
serious weakness associated with the general training procedure for an inverse model: the criterion (11.53)
expresses the objective to minimize the discrepancy between the network output and a sequence of “true”
control inputs. This is not really a relevant objective. In practice, it is not possible to achieve zero generali-
zation error and consequently, the trained network will have certain inaccuracies. Although these are reaso-
nably small in terms of the network output being close to the ideal control signal, there may be large devia-
tions between the reference and the output of the system when the network is applied as controller for the
system. The weakness lies in the fact that the training procedure is not goal directed. The goal is that, in
some sense, the system output should follow the reference signal closely. It would be more desirable to
minimize a criterion of the following type:
 Neural Networks 663

1 P
2
J= å [r(k) – y(k)] (11.55a)
2P k =1
which clearly is goal directed. Unfortunately, the minimization of this criterion is not easily carried out off-
line, considering that the system output, y(k), depends on the output of the inverse model, u(k – 1).
Inspired by the recursive training algorithms, the network might alternatively be trained to minimize
Jk = Jk – 1 + [r(k) – y(k)]2 (11.56)
This is an on-line approach and, therefore, the scheme constitutes an adaptive controller.
Assuming that Jk – 1 has already been minimized, the weights at time k are adjusted according to
2
q$ (k) = q$ (k – 1) – h de ( k ) (11.57a)
dq
where e(k) = r(k) – y(k) (11.57b)
2
and de (k ) = – dy( k ) e(k) (11.57c)
dq dq
By application of the chain rule, the gradient dy(k ) can be calculated:
dq
dy(k ) = ¶y(k ) du(k - 1) (11.58a)
dq ¶u( k - 1) dq
¶y(k )
Jacobians of the system, , are required. These are generally unknown since the system is unknown.
¶u( k - 1)
To overcome this problem, a forward model of the system is identified to provide estimates of the Jacobians:
¶y(k ) ¶y$(k )
» (11.58b)
¶u( k - 1) ¶u( k - 1)
The forward model is obtained by the system identification procedure described in the earlier section.
Fortunately, inaccuracies in the forward model need not have a harmful impact on the training. The
Jacobian is a scalar factor and, in the simplified algorithm (11.57), will only change the step-size of the
algorithm. Thus, as long as the Jacobians have the correct sign, the algorithm will converge if the step-size
parameter is sufficiently small. We will call this procedure the specialized training procedure for the
inverse model.
The deadbeat character, appearing when inverse models are used directly as controllers, will often result
in an unnecessarily fast response to reference changes. An active control signal may even harm the system or
the actuators. Consequently, it might be desirable to train the network to achieve some prescribed low-pass
behaviour of the closed-loop system. Say, have the closed-loop system following the model:
Bm ( z -1 )
ym(k) = r(k) (11.59)
Am ( z -1 )
The polynomials Am and Bm are selected arbitrarily, by the designer.
The control design is, in this case, related to “Model Reference Adaptive System” (MRAS); a popular
type of adaptive controller (discussed earlier in Section 10.3).
Since this specialized training is an on-line approach, the combination of having many weights to adjust
and having only the slow convergence of a gradient method, will often be disastrous. Before the weights are
properly adjusted, the system may have been driven outside the operating range with possibly serious conse-
quences. Often general training can be used to provide a decent initialization of the network so that the
specialized training is only used for “fine tuning” of the controller.
 664 Digital Control and State Variable Methods
The simplified specialized training is quite easily implemented with the backpropagation algorithm (refer
Fig. 11.25). This algorithm is applied on the inverse model NN2:
u(k – 1) = f [ y(k + 1), y(k), ..., y(k – n + 1), u(k – 2),..., u(k – m)]
by assuming the following “virtual” error eu(k) on the output of the controller:
de 2 ( k ) dy ( k ) ¶y$ ( k ) du( k – 1) du( k – 1) ∂u( k –1)
= – e( k ) = – e( k ) = – eu ( k ) ; – eu (k )
dq dq ¶u( k – 1) dq dq ∂θ
∂y$( k )
where eu(k) = e(k) (11.60)
∂u ( k − 1)
∂yˆ( k ) NN1(u ( k – 1) + ε ) – NN1(u ( k – 1))
;
∂u ( k – 1) ε
A better estimate of the derivative is obtained as follows:
For a multilayer feedforward network (NN 1) with one hidden layer of sigmoid units and a linear output
(Fig. 11.22),
M F N I +n
y$ (k) = åvls G å wlif i (k ) + wl0 J 0 (11.61a)
l =1 H i =1 K
f(k) = [y(k – 1), ..., y(k – n), u(k – 1), ..., u(k – m)] (11.61b)

Reference ym
Model

+
e
–
Model

NN 1
Forward y^
eu Model

r
NN 2
Inverse System
u y
Model

Fig. 11.25 Specialized training

The derivative of the output with respect to the regressor fi(k), is given by
M
¶y$(k )
= å vl wlis ¢( a ) (11.62a)
¶f i (k ) l = 1
 Neural Networks 665
M
= ∑ vl wliσ (a ) [1 – s (a)]
l =1
N
where a= å wlif i (k ) + wl0 (11.62b)
i =1

We can obtain ∂y$ ( k )/ ∂u( k – 1) from this relation.

11.9 SUPPORT VECTOR MACHINES

Support vector machine (SVM) theory provides the most principled approach to the design of neural networks.
Statistical learning theory provides a sound mathematical basis for the formulation of support vector machines.
SVM theory applies to pattern classification and nonlinear regression, using any one of the following network
architectures: RBF networks, MLPs with a single hidden layer, and Polynomial machines. For each of these
feedforward networks, we may use the support vector learning algorithm to implement the learning process
using a given set of training data; automatically determining the required number of hidden units.
Our interest in control problems is more on nonlinear regression. To explain how a support vector
machine works for regression problems, it is perhaps easiest to start with the case of linearly separable
patterns that could arise in the context of binary pattern classification. In this context, the main idea of a
support vector machine is to construct a hyperplane as the decision surface in such a way that the margin of
separation between Class 1 and Class 2 examples is maximized. We will then take up the more difficult case
of linearly nonseparable patterns; moving closer to real-life situations.
With the material on how to find the optimal hypersurface for linearly nonseparable patterns at hand, we
will formally describe the construction of a support vector machine for real-life (nonlinear) pattern recogni-
tion task. As we shall see shortly, basically the idea of a support vector machine hinges on two mathematical
operations:
(i) Nonlinear mapping of input patterns into high-dimensional feature space.
(ii) Construction of optimal hyperplane for linearly separating the features discovered in step (i).
The final stage of our presentation will be to extend these results for application to nonlinear regression
problems.

11.9.1 Hard-Margin Linear SVM

Our presentation on SVM begins with the easiest classification problem: binary classification of linearly sepa-
rable data (separating functions will be hyperplanes). The presentation will gradually increase in complexity.
Let the set of training (data) examples D be
D = {(x1, y1), (x2, y2), ..., (xp, yp)} (11.63)
where xi = [ xi1 xi 2 K xin ] is an n-dimensional input vector (pattern with n-features) for the ith exam-
T

ple in a real-valued space X ⊆ ℜn ; yi is its class label (output value), and yi ∈ {+1, −1}. + 1 denotes Class 1
and – 1 denotes Class 2.
To build a classifier, SVM finds a linear function of the form
f (x) = wT x + b (11.64)
so that the input vector xi is assigned to Class 1 if f (xi) ³ 0, and to Class 2 if f (xi) < 0 i.e.,
⎧⎪ +1 if wT xi + b ≥ 0
yi = ⎨ (11.65)
⎪⎩ −1 if wT xi + b < 0
 666 Digital Control and State Variable Methods

Hence f (x) is a real-valued function f : X ⊆ ℜn → ℜ.

w = [w1 w2 K wn ] ∈ ℜn is called the weight vector and b ∈ ℜ is called the bias.
T

In essence, SVM finds a hyperplane

wT x + b = 0 (11.66)
that separates Class 1 and Class 2 training examples. This hyperplane is called the decision boundary or
decision surface. Geometrically, the hyperplane (11.66) divides the input space into two half spaces: one
half for Class 1 examples and the other half for Class 2 examples. Note that hyperplane (11.66) is a line in a
2-dimensional space and a plane in a 3-dimensional space.
For linearly separable data, there are many hyperplanes (lines in 2-dimensional feature space; Fig. 11.26)
that can perform separation. How can one find the best one? The SVM framework provides good answer to
this question. Among all the hyperplanes that minimize the training error, find the one with the largest
margin—the gap between the data points of the two classes. This is an intuitively acceptable approach:
select the decision boundary that is far away from both the classes (Fig. 11.27). Large-margin separation is
expected to yield good classification on previously unseen data, i.e., good generalization.

x2 Separating line x2
(decision boundary)
x2
Class 2 Class 2

Class 2

Separating
line

Class 1 Class 1
Class 1 x1 x1
x1 (a) Large margin separation (b) Small margin separation
Fig. 11.26 Possible decision boundaries Fig. 11.27

From linear algebra, we know that in wTx + b = 0, w defines a direction perpendicular to the hyperplane.
w is called the normal vector (or simply normal) of the hyperplane. Without changing the normal vector w,
varying b moves the hyperplane parallel to itself. Note also that wTx + b = 0 has an inherent degree of
freedom. We can rescale the hyperplane to kwTx + kb = 0 for k ∈ ℜ+ (positive real number), without chang-
ing the hyperplane.
Since SVM maximizes the margin between Class 1 and Class 2 data points, let us find the margin. The linear
function f (x) = wTx + b gives an algebraic measure of the distance from x to the hyperplane wTx + b = 0. We
can express x as
w
x = xN + r (11.67)
|| w ||
where xN is the normal projection of x onto the hyperplane and r is the desired algebraic distance (Fig. 11.28).
Since by definition, f (x N ) = wT x N + b = 0, it follows that
 Neural Networks 667

⎛ w ⎞
f (x) = wT x + b = wT ⎜ x N + r +b
⎝ || w || ⎟⎠
wT w (|| w ||) 2
=r =r = r || w ||
|| w || || w ||
f (x)
or r= (11.68)
|| w ||

( )
Now consider a Class 1 data point x(1) , +1 that is closest to the hyperplane wTx + b = 0 (Fig. 11.28). The
(1)
distance d of this data point from the hyperplane is
f ( x (1) ) w T x (1) + b
d (1) = = (11.69a)
|| w || || w ||
Similarly
f ( x (2) ) w T x (2) + b
d (2) =
= (11.69b)
|| w || || w ||
(2)
where (x , –1) is a Class 2 data point closest to the hyperplane wTx + b = 0.

x2
M

x Class 2
r
xN
x(2)
( 2)
) d
(1
w d (2):wTx +b=–1
x(1)
w Tx + b = 0
(1):wTx +b=+1
Class 1
x1
Fig. 11.28 Geometric interpretation of algebraic distances of points
to a hyperplane for two-dimensional case

We define two parallel hyperplanes H(1) and H(2) that pass through x(1) and x(2), respectively. H(1) and
(2)
H are also parallel to the hyperplane wT x + b = 0. We can rescale w and b to obtain (this rescaling, as we
shall see later, simplifies the quest for significant patterns, called support vectors)
H(1) : wT x(1) + b = +1
(11.70)
H(2) : wT x(2) + b = −1
such that
wT x i + b ≥ 1 if yi = +1
(11.71a)
w xi + b ≤ −1
T
if yi = −1
 668 Digital Control and State Variable Methods
or equivalently

(
yi wT xi + b ≥ 1 ) (11.71b)
which indicates that no training data fall between hyperplanes H(1) and H(2). The distance between the two
hyperplanes is the margin M. In the light of rescaling given by (11.70),
1 −1
d (1) = ; d (2) = (11.72)
|| w || || w ||
where the ‘–’ sign indicates that x(2) lies on the side of the hyperplane wTx + b = 0 opposite to that where x(1)
lies. From Fig. 11.28, it follows that
2
M = (11.73)
|| w ||
Equation (11.73) states that maximizing the margin of separation between classes is equivalent to mini-
mizing the Euclidean norm of the weight vector w.
Since SVM looks for the separating hyperplane that minimizes the Euclidean norm of the weight vector,
this gives us an optimization problem. A full description of the solution method requires a significant amount
of optimization theory, which is beyond the scope of this book. We will only use relevant results from
optimization theory, without giving formal definitions, theorems or proofs (see [29] for details).
Our interest here is in the following nonlinear optimization problem with inequality constraints:
minimize f ( x)
subject to gi ( x) ≥ 0; i = 1,K , m (11.74)

where x = [ x1 x2 K xn ] , and the functions f and gi are continuously differentiable.

T

The optimality conditions are expressed in terms of the Lagrangian function

m
L ( x, λ ) = f ( x ) − ∑ λi gi ( x) (11.75)
i =1

where λ = [λ1 K λm ] is a vector of Lagrange multipliers.

T

An optimal solution to the problem (11.74) must satisfy the following necessary conditions, called
Karush-Kuhn-Tucker (KKT) conditions:
∂L ( x, λ )
(i) = 0; j = 1,K , n
∂x j
(ii) gi ( x) ≥ 0; i = 1,K , m (11.76)
(iii) λi ≥ 0; i = 1,K , m
(iv) λi gi (x) = 0; i = 1,K , m
m
In view of condition (iii), the vector of Lagrange multipliers belongs to the set {λ ∈ ℜ , λ ≥ 0}. Also
note that condition (ii) is the original set of constraints.
Our interest, as we will see shortly, is in convex functions f and linear functions gi. For this class of
0 0
optimization problems, when there exist vectors x0 and l0 such that the point (x , λ ) satisfies the KKT
conditions (11.76), then x0 gives the global minimum of the function f (x), with the constraint given in
(11.74).
 Neural Networks 669
Let
L* (x) = max L(x ,λ ), and L* (λ ) = min L(x ,λ )
n
m
λ ∈ℜ x∈ℜ

It is clear from these equations that for any x ∈ℜ and λ ∈ ℜ ,

n m

L* (λ) ≤ L(x , λ ) ≤ L* (x)

and thus in particular
L* (λ ) ≤ L* ( x )
This holds for any x ∈ℜ and λ ∈ℜ ; so it holds for the l that maximizes the left hand side, and the x
n m

that minimizes the right-hand side. Thus

max min L( x ,λ ) ≤ minn maxm L( x , λ )
λ ∈ℜm x∈ℜn x∈ℜ λ∈ℜ

The two problems, min-max and max-min, are said to be dual to each other. We refer to the min-max
*
problem as the primal problem. The objective to be minimized, L (x) is referred to as the primal function.
The max-min problem is referred to as the dual problem, and L* (λ) as the dual function. The optimal primal
and dual function values are equal when f is a convex function and gi are linear functions. The concept of
duality is widely used in the optimization literature. The aim is to provide an alternative formulation of the
problem which is more convenient to solve computationally and/or has some theoretical significance. In the
context of SVM, the dual problem is not only easy to solve computationally, but also crucial for using kernel
functions to deal with nonlinear decision boundaries. This will be clear later in this section.
The nonlinear optimization problem defined in (11.74) can be represented as min-max problem, as is
seen below:
For the Lagrangian (11.75), we have
⎡ m
⎤
L* (x) = max ⎢ f (x) − ∑ λi gi ( x )⎥
λ∈ℜ m ⎣ i =1 ⎦
Since gi(x) ³ 0 for all i, li = 0 (i = 1,…, m) would maximize the Lagrangian. Thus
L* (x) = f (x)
Therefore, our original constrained problem (11.74) becomes the min-max primal problem:
minimize L* (x)
n
x∈ℜ
subject to gi ( x) ≥ 0; i = 1,K , m
The concept of duality gives the following formulation for max-min dual problem:
maximize L* (λ )
m
ë∈ℜ ,λ ≥ 0
More explicitly, this nonlinear optimization problem with dual variables l, can be written in the form:
⎡ m ⎤
maximize minn ⎢ f (x) − ∑ λi gi (x) ⎥ (11.77)
x∈ℜ ⎢
λ ≥0
⎣ i =1 ⎦⎥
Let us now state the learning problem in SVM.
Given a set of linearly separable training examples,
D = {( x1 , y1 ), ( x 2 , y2 ),K , ( x P , yP )},
 670 Digital Control and State Variable Methods
learning is to solve the following constrained minimization problem:
minimize f (w) = 12 wT w
(11.78)
subject to yi (wT xi + b) ≥ 1; i = 1, K, P
This formulation is called the hard-margin SVM. Solving this problem will produce the solutions for w
and b, which in turn, give us the maximal margin hyperplane wT x + b = 0 with the margin 2 / || w || .
The objective function is quadratic and convex in parameters w, and the constraints are linear in param-
eters w and b. The dual formulation of this constrained optimization problem is obtained as follows.
First we construct the Lagrangian:
P
L ( w , b, λ ) = 12 w T w − ∑ λi ⎡⎣ yi (wT xi + b) − 1⎤⎦ (11.79)
i =1
The KKT conditions are:

(i) ∂L = 0; which gives w = ∑ λ y x

P
i i i
∂w i =1

∂L P

∂b
= 0; which gives ∑ λi yi = 0
i =1

T
(ii) yi (w xi + b) − 1 ≥ 0; i = 1,K , P (11.80)
(iii) λi ≥ 0; i = 1,K , P

(iv) λi [ yi (wT xi + b) − 1] = 0; i = 1,K, P

From condition (i) of KKT conditions (11.80), we observe that the solution vector has an expansion in
terms of training examples. Note that although the solution w is unique (due to the strict convexity of the
function f (w)), the dual variables li need not be. There is a dual variable li for each training data point.
Condition (iv) of KKT conditions (11.80) shows that for data points not on the margin hyperplanes (i.e.,
H(1) and H(2)), l i = 0:
yi (wT xi + b) − 1 > 0 ⇒ λi = 0
For data points on the margin hyperplanes, li ³ 0:
yi (wT xi + b) − 1 = 0 ⇒ λi ≥ 0
However, the data points on the margin hyperplanes with li = 0 do not contribute to the solution w, as is seen
from condition (i) of KKT conditions (11.80). The data points on the margin hyperplanes with associated dual
variables li > 0 are called support vectors, which give the name to the algorithm, support vector machines.
To postulate the dual problem, we first expand Eqn. (11.79), term by term, as follows.
P P P
L( w , b, λ) = 12 w T w − ∑ λi yi w T x i − b ∑ λi yi + ∑ λi (11.81)
i =1 i =1 i =1
Transforming from the primal to its corresponding dual can be done by setting to zero the partial deriva-
tives of the Lagrangian (11.81) with respect to the primal variables (i.e., w and b), and substituting the
resulting relations back into the Lagrangian. This is simply to substitute condition (i) of KKT conditions
(11.80) into the Lagrangian (11.81) to eliminate the primal variables; which gives us the dual objective
function.
 Neural Networks 671
The third term on the right-hand side of Eqn. (11.81) is zero by virtue of condition (i) of KKT conditions
(11.80). Furthermore, from this condition we have
P P P
wT w = ∑ λi yi wT xi = ∑ ∑ λi λ j yi y j xiT x j
i =1 i =1 j =1

Accordingly, minimization of function L in Eqn. (11.81) with respect to primal variables w and b, gives
us the following dual objective function:
P P P
L* (λ) = ∑ λi − 12 ∑∑ λi λ j yi y j xTi x j (11.82)
i =1 i =1 j =1

We may now state the dual optimization problem.

Given a set of linearly separable training examples {(xi , yi )}iP=1 , find the dual variables {λi }iP=1 , that
maximize the objective function (11.82) subject to the constraints
P
· ∑ λi yi =0
(11.83)
i =1

· λi ≥ 0; i = 1,K , P
This formulation is dual formulation of the hard-margin SVM.
Having solved the dual problem numerically (using MATLAB’s quadprog function, for example), the
resulting optimum li values are then used to compute w and b. w is computed using condition (i) of KKT
conditions (11.80):
P
w = ∑ λi yi xi (11.84a)
i =1
and b is computed using condition (iv) of KKT conditions (11.80). For support vectors {xs, ys}, this condi-
tion becomes li > 0, and

ys ( wT xs + b) = 1
Instead of depending on one support vector to compute b, in practice all support vectors are used to
compute b, and then their average is taken on the final value for b. This is because the values of li are
computed numerically and can have numerical errors.
N SV
1 ⎡1 ⎤
b=
N SV
∑⎢y T
− w x s ⎥; N SV = total number of support vectors (11.84b)
s =1 ⎣ s ⎦

11.9.2 Soft-Margin Linear SVM

The linear separable case is the ideal situation. In practice, however, the training data is almost always noisy,
i.e., containing errors due to various reasons. For example, some examples may be labelled incorrectly.
Furthermore, practical problems may have some degree of randomness. Even for two identical input vectors,
their labels may be different.
For SVM to be useful, it must allow noise in the training data. However, with noisy data, the linear SVM
algorithm presented earlier, will not find a solution because the constraints cannot be satisfied. For example,
in Fig. 11.29, there is a Class 2 point (circle) in the Class 1 region, and a Class 1 point (square) in the
 672 Digital Control and State Variable Methods

Class 2 region. However, inspite of the couple of mistakes, the decision boundary seems to be good. But the
hard margin classifier presented previously cannot be used, because all the constraints.
yi (wT xi + b) ≥ 1; i = 1,K , P
cannot be satisfied.
x2 wTx + b = 0

Class 2
w
zj |
| xl
||w
b ||
xj ||w
zl |
|
||w
Class 1
x1
Fig. 11.29 Soft decision boundary: xj and xl are error data points

So the constraints have to be modified to permit mistakes. To allow errors in data, we can relax the
margin constraints by introducing slack variables, ζ i (≥ 0), as follows:
w T xi + b ≥ 1 − ζ i for yi = +1
w xi + b ≤ −1 + ζ i
T
for yi = −1
Thus, we have the new constraints
yi ( w T xi + b) ≥ 1 − ζ i ; i = 1,K , P
(11.85)
ζi ≥ 0
The geometric interpretation is shown in Fig. 11.29.
We also need to penalize the errors in the objective function. A natural way is to assign an extra cost for
errors, to change the objective function to
⎛ P ⎞
wT w + C ⎜ ∑ ζ i ⎟ ; C ≥ 0
1
2
⎝ i =1 ⎠
where C is a user specified penalty parameter. This parameter is a trade-off parameter between margin and
mistakes.
The new optimization problem becomes:
P
minimize 1
2
wT w + C ∑ζ i
i =1

subject to yi (w xi + b) ≥ 1 − ζ i ; i = 1,K, P
T
(11.86)
ζ i ≥ 0; i = 1,K , P
This formulation is called the soft-margin SVM.
Proceeding in the manner similar to that described earlier for separable case, we may formulate the dual
problem for nonseparable patterns as follows.
 Neural Networks 673
The Lagrangian
P P P
L( w , b, ζ, λ , μ) = 12 w T w + C ∑ ζ i − ∑ λi ⎡⎣ yi ( w T x i + b ) − 1 + ζ i ⎤⎦ − ∑ μ iζ i (11.87)
i =1 i =1 i =1

where l i, mi ³ 0 are the dual variables.

The KKT conditions for optimality are:
∂L P
(i) = w − ∑ λi yi xi = 0
∂w i =1

∂L P
= −∑ λi yi = 0
∂b i =1
∂L
= C − λi − μi = 0; i = 1,K, P
∂ζ i

(ii) yi (w xi + b) − 1 + ζ i ≥ 0; i = 1,K , P
T

ζ i ≥ 0; i = 1,K, P (11.88)
(iii) λi ≥ 0; i = 1,K,P
μi ≥ 0; i = 1,K , P
(iv) λi ( yi (w xi + b) − 1 + ζ i ) = 0; i = 1,K, P
T

μiζ i = 0; i = 1,K, P
We substitute the relations in condition (i) of KKT conditions (11.88) into the Lagrangian (11.87) to
obtain dual objective function. From the relation C – li – mi = 0, we can deduce that li = C because mi ³ 0.
Thus the dual formulation of the soft-margin SVM is:
P P P
maximize L* (λ ) = ∑ λi − 12 ∑∑ λi λ j yi y j xTi x j
i =1 i =1 j =1
P
subject to ∑ λi yi = 0 (11.89)
i =1
0 ≤ λi ≤ C; i = 1,K, P
Interestingly, zi and mi are not in the dual objective function; the objective function is identical to that for
the separable case. The only difference is the constraint l i £ C (inferred from C – li – mi = 0 and mi ³ 0).
The dual problem (11.89) can also be solved numerically, and the resulting li values are then used to
compute w and b. The weight vector w is computed using Eqn. (11.84a).
The bias parameter b is computed using condition (iv) of KKT conditions (11.88):
λi ( yi (wT xi + b) − 1 + ζ i ) = 0 (11.90a)
μiζ i = 0 (11.90b)
Since we do not have values for zi, we have to get around it. li can have values in the interval 0 £ li £ C.
We will separate it into three cases.
Case 1: li = 0
We know that C – li – mi = 0. With li = 0, we get mi = C. Since mizi = 0 (Eqn. (11.90b)), this implies that zi
= 0; which means that the corresponding ith pattern is correctly classified without any error (as it would have
 674 Digital Control and State Variable Methods

been with hard-margin SVM). Such patterns may lie on margin hyperplanes or outside the margin. However,
they don’t contribute to the optimum value of w, as is seen from Eqn. (11.84a).
Case 2: 0 < li < C
We know that C – li – mi = 0. Therefore mi = C – li, which means mi > 0. Since mizi = 0 (Eqn. (11.90b)), this
implies that zi = 0. Again the corresponding ith pattern is correctly classified. Also from Eqn. (11.90a), we
see that for zi = 0 and 0 < λi < C , yi (wT xi + b) = 1; so the corresponding patterns are support vectors.
Case 3 : li = C
It can easily be seen that zi ¹ 0 in this case. But zi ³ 0 is a constraint of the problem. So zi > 0; which means
that the corresponding pattern is mis-classified or lies inside the margin.
We can use support vectors, as in Eqn. (11.84b), to compute the value of b.
The following points need attention of the reader.
· One of the most important properties of SVM is that the solution is sparse in li. Most training data
points are outside the margin area and their li’s in the solution are 0. The data points on the margin
having li = 0, also do not contribute to solution. Only those data points that are on the margin hyper-
planes with 0 < li < C (support vectors) and inside the margin (errors; li = C) contribute to solution.
Without this sparsity property, SVM would not be practical for large data sets.
· The final decision boundary is
wT x + b = 0
Substituting for w and b from Eqns (11.84), we obtain
T N SV
⎛ P ⎞ P P
1 ⎡1 P
⎤
⎜ ∑ λi yi x i ⎟ x + b = ∑∑ λi yi xTi x j + N ∑ ⎢ − ∑ λs y s x s x s ⎥ = 0
T
(11.91)
⎝ i =1 ⎠ i =1 j =1 SV s =1 ⎣ y s s =1 ⎦
We notice that w and b do not need to be explicitly computed. As we will shortly see, this is crucial for
using kernel functions to handle nonlinear decision boundaries.
· Finally, we still have the problem of determining the parameter C. The value of C is usually chosen by
trying a range of values on the training set to build multiple classifiers and then testing them on valida-
tion set before selecting the one that gives the best classification result on the validation set.

11.9.3 Nonlinear SVM

The SVM formulations discussed so far, require that Class 1 and Class 2 examples can be linearly repre-
sented, i.e., the decision boundary must be a hyperplane. However, for many real-life data sets, the decision
boundaries are nonlinear. To deal with nonlinearly separable data, the same formulation and solution tech-
niques as far the linear case are still used. We only transform the input data from its original space into
another space (usually, a much higher dimensional space) so that a linear decision boundary can separate
Class 1 and Class 2 examples in the transformed space, which is called the feature space. The original data
space is called the input space.
Through a nonlinear mapping f, the original data set {(x1 , y1 ),..., (x P , yP )} becomes:
{φ( x1 , y1 ),..., φ( x P , yP )} (11.92)
Figure 11.30 illustrates the process. In the input space, the training examples cannot be linearly sepa-
rated; in the feature space, they can be separated linearly.
 Neural Networks 675
With the transformation, the optimization problem in (11.86) becomes:
P
minimize 1
2
wT w + C ∑ ζ i
i =1

subject to yi (wT φ(xi ) + b) ≥ 1 − ζ i ; i = 1, K, P (11.93)

ζ i ≥ 0; i = 1,K, P
The corresponding dual is
P P P
minimize L* ( λ ) = ∑ λi − 12 ∑∑ λi λ j yi y j (fT (xi ) f (x j))
i =1 i =1 j =1
P
subject to ∑ λi yi = 0 (11.94)
i =1
0 ≤ λi ≤ C; i = 1, K, P
The potential problem with this approach is that it may suffer from the curse of dimensionality. The
number of dimensions in the feature space can be huge with some useful transformations, even with reason-
able number of attributes in the input space. Fortunately, explicit transformations can be avoided if we notice
that for the dual problem (11.94), the construction of the decision boundary only requires the evaluation of
fT( x i ) f( x j ) in the feature space. With reference to (11.91), we have the following decision boundary in
feature space:
P P
∑∑ λi y j φT(xi ) φ(x j ) + b = 0 (11.95)
i =1 j =1

Input space Feature space

x® (x)

Fig. 11.30 Transformation from the input space to feature space

Thus, if we have to compute φ (x i ) φ ( x j ) in the feature space using the input vectors xi and xj directly,
T

then we would not need to know the feature vector f (x) or even the mapping f itself. In SVM, this is done
through the use of kernel functions, denoted by K:
K ( x i , x j ) = φT( x i ) φ( x j ) (11.96)
 676 Digital Control and State Variable Methods
Commonly used kernels include :
Polynomial of degree d: K (x i , x j ) = (xTi x j + 1) d
⎛ 1 ⎞ (11.97)
Gaussian RBF: K ( xi , x j ) = exp ⎜ − 2 || xi − x j ||2 ⎟
⎝ 2σ ⎠
We replace φ ( x i ) φ( x j ) in (11.94) and (11.95) with kernel. We would never need to explicitly know what
T

f is.
However, how do we know that a kernel function is indeed a dot product in some feature space? This
question is answered by a theorem, called the Mercer’s theorem, which we will not discuss here. The kernels
in (11.97) satisfy this theorem. See [138,141] for details.

11.9.4 Function Approximation using SVM

Suppose we are given training data
{(x1 , y1 ),K , (x P , yP )}; x ∈ ℜn , y ∈ ℜ
where xi ∈ℜn are the input patterns, as in classification problems; and yi ∈ ℜ now has continuous values.
Our goal is to find a function f (x) that has at most e deviation (where e is a prescribed parameter) from the
actually obtained targets yi for all the training data, and at the same time, is as flat as possible. In other words,
we do not care about errors as long as they are less than e, but will not accept any deviation larger than this.
For pedagogical reasons, we begin by describing the case of linear functions f, taking the form
f (x) = wT x + b; w ∈ ℜn , b ∈ℜ (11.98)
Flatness in the case of (11.98) means that one seeks small w. One way to ensure this is to minimize the
Euclidean norm, i.e. || w ||2, This additional requirement on performance (in addition to the constraint on
maximum allowable error in estimate of yi) improves generalization.
Formally we can write this problem as a constrained optimization problem:
1 T
minimize 2
w w
subject to yi − wT xi − b ≤ ε ; i = 1,K, P
w T xi + b − yi ≤ ε ; i = 1, K, P (11.99)
The tacit assumption in (11.99) is that a function f given by (11.98) actually exists that approximates all
pairs (xi, yi) with e precision, or in other words, that the constrained optimization problem is feasible. It
should be noted that the optimization problem cannot accommodate data points with errors larger than e;
constraints cannot be satisfied for such data points. For SVM to be useful, it must allow noise in the training
data. Analogously to the “soft margin” classifier described earlier, one can introduce slack variables zi, zi* to
cope with otherwise infeasible constraints of the optimization problem (11.99). Hence we arrive at the
following formulation:
P
minimize 1 T
2
w w + C ∑ (ζ i + ζ i* )
i =1

subject to yi − wT xi − b ≤ ε + ζ i ; i = 1,K, P (11.100)

w xi + b − yi ≤ ε
T
+ ζ i* ; i = 1,K, P
ζ i , ζ i* ≥ 0; i = 1,..., P
 Neural Networks 677
The constant C > 0 determines the trade-off between the flatness of f given by (11.98), and the amount by
which deviations larger than e are tolerated.
The formulation (11.100) corresponds to dealing with a so called e-insensitive loss (error) function,
described below:
⎧0 if | yi − yˆi | ≤ ε ; yˆi @ f (xi )
yi − yˆi ε =D ⎨ (11.101)
⎩| yi − yˆi | −ε otherwise
This loss function defines an e-insensitive tube (Fig. 11.31); the loss (error) is equal to zero for training
data points inside the tube (| yi − yˆ i | ≤ ε ), the loss is zi for data ‘above’ the tube ( yi − yˆi − ε = ζ i ) and zi* for
data ‘below’ the tube ( yˆi − yi − ε = ζ i* ) . Only the data points outside the tube contribute to the loss (error),
with deviations penalized in a linear fashion.
|yi – y^i|e
x +e
zi
0
–e
zi
z j*
x
–e +e |yi – y^i|
e + zi

Fig. 11.31 The soft margin loss setting

As with procedures applied to SVM classifiers, the constrained optimization problem (11.100) is solved
by forming the Lagrangian:
L(w, b, ζ , ζ* , λ, λ* , μ, μ* )
P P
( )
= 12 wT w + C ∑ ζ i + ζ i* − ∑ λi (ε + ζ i − yi + wT xi + b)
i =1 i =1
P P
( )
− ∑ λi* ε + ζ i* + yi − wT xi − b − ∑ μiζ i + μi*ζ i*
i =1 i =1
( ) (11.102)

where w, b, z l and zi* are the primal variables, and li, li*, mi, mi* ³ 0 are the dual variables.
The KKT conditions are:

(i) ∂L = w − ∑ λi − λi* xi = 0
P

∂w i =1
( )
∂L P *
(
= ∑ λi − λi = 0
∂b i =1
)
∂L
= C − λi − μi = 0; i = 1,K , P
∂ζ i
∂L
= C − λi* − μi* = 0; i = 1,K , P
∂ζ i
*
 678 Digital Control and State Variable Methods

(ii) ε + ζ i − yi + w xi + b ≥ 0; i = 1,K, P
T
(11.103)
ε + ζ i* + yi − w xi − b ≥ 0; i = 1,K, P
T

ζ i , ζ i* ≥ 0; i = 1,..., P
(iii) λi , λi* , μi , μi* ≥ 0; i = 1,K , P
( )
(iv) λi ε + ζ i − yi + wT xi + b = 0; i = 1,K , P

λ (ε + ζ
*
i
*
i + yi − wT x − b ) = 0; i = 1,..., P
i

μiζ i = 0; i = 1,K , P
μi*ζ i* = 0; i = 1,K , P
Substituting the relations in condition (i) of KKT conditions (11.103) yields the dual objective function.
The procedure is parallel to what has been followed earlier. The resulting dual optimization problem is:
P P P P
maximize L* (λ, λ* ) = −ε ∑ (λi + λi* ) + ∑ (λi − λi* )yi − 12 ∑∑ (λi − λi* )(λ j − λ *j )xTi x j
i =1 i =1 i =1 j =1
P
subject to ∑ (λi − λi* ) = 0 (11.104)
i =1

λi , λi* ∈ [0, C ]
From condition (i) of KKT conditions (11.103), we have
P
(
w = ∑ λi − λi* xi
i =1
) (11.105)

Thus the weight vector w is completely described as a linear combination of the training patterns xi. One
of the most important properties of SVM is that the solution is sparse in li, li*. For | yˆi − yi | < ε , the second
factor in the following KKT conditions:
λi (ε + ζ i − yi + wT xi + b) = λi (ε + ζ i − yi + yˆ i ) = 0
(11.106)
λi* (ε + ζ i* + yi − wT xi − b) = λi* (ε + ζ i* + yi − yˆ i ) = 0
is nonzero; hence λi , λi* has to be zero. This equivalently means that all the data points inside the e-insensitive
tube (a large number of training examples belong to this category) have corresponding λi , λi* equal to zero.
Further, from (11.106) it follows that only for | y$i – yi | ³ e , the dual variables λi , λi* may be nonzero. Since
there can never be a set of dual variables λi , λi* which are both simultaneously nonzero, as this would require
slacks in both directions (‘above’ the tube and ‘below’ the tube), we have λi × λi* = 0.
From KKT conditions (11.103), it follows that
(C − λ )ζ = 0 i i
(11.107)

(C =0 − λi* )ζ i*
Thus the only samples (xi, yi) with corresponding λi , λi* = C lie outside the e-insensitive tube around f.
For λi , λi* ∈ (0, C ), we have z i, z i*= 0 and moreover the second factor in (11.106) has to vanish. Hence b can
be computed as follows:
b = yi − wT xi − ε for λi ∈ ( 0, C )
(11.108)
b = yi − w xi + ε
T
for λi* ∈ (0, C )
 Neural Networks 679

All data points with λi , λi* ∈ (0, C ) are used to compute b, and then their average is taken as the final value
for b.
The examples that come with nonvanishing dual variables li, li* are called Support Vectors.
The next step is to make SVM algorithm nonlinear. This would be achieved by simply preprocessing the
training patterns xi by a map f into some feature space, and then applying the standard SVM algorithm.
All pattern-recognition/function approximation (classification/regression) problems when solved using
SVM algorithms presented in this section, are basically quadratic optimization problems. Attempting
MATLAB functions for SVM algorithms discussed in this section, will be a rich learning experience for the
reader.

Review Examples

Review Example 11.1 A high performance drive system consists of a motor and a controller integrated
to perform a precise mechanical maneuver. This requires the shaft speed, and/or position of the motor to clearly
follow a specified trajectory, regardless of unknown load variations and other parameter uncertainties.
A backpropagation neural network can be trained to emulate the unknown nonlinear plant dynamics by
presenting a suitable set of input/output patterns generated by the plant. Once system dynamics have been
identified using a neural network, many conventional control techniques can be applied to achieve the
desired objective of trajectory tracking.
In this example, we study a neural-network-based identification and control strategy for trajectory control
of a dc motor.
DC Motor Model Although it is not mandatory to obtain a motor model if a neural network (NN) is used in
the motor-control system, it may be worth doing so, from the analytical perspective, in order to establish the
foundation of the NN structure. We will use input/output patterns generated by simulation of this model for
training of NN (In a real life situation, experimentally generated input/output patterns will be used for training).
The dc motor dynamics are given by the following equations (refer Fig. 11.32):

Fig. 11.32 Armature-controlled dc motor with load

 680 Digital Control and State Variable Methods

dia
va(t) = Ra ia(t) + La + eb(t) (11.109)
dt
eb(t) = Kbw (t) (11.110)
TM (t) = KT ia(t) (11.111)
dw(t )
=J + Bw (t) + TL(t) + TF (11.112)
dt
where
va(t) = applied armature voltage (volts);
eb(t) = back emf (volts);
ia(t) = armature current (amps);
Ra = armature winding resistance (ohms);
La = armature winding inductance (henrys);
w(t) = angular velocity of the motor rotor (rad/sec);
TM (t) = torque developed by the motor (Newton-m);
KT = torque constant (Newton-m/amp);
Kb = back emf constant (volts/(rad/sec));
J = moment of inertia of the motor rotor with attached mechanical load (kg-m2);
B = viscous-friction coefficient of the motor rotor with attached mechanical load ((Newton-m)/
(rad/sec));
TL(t) = disturbance load torque (Newton-m); and
TF = frictional torque (Newton-m).
The load torque TL(t) can be expressed as
TL(t) = y (w) (11.113)
where the function y (×) depends on the nature of the load.
For most propeller driven or fan type loads, the function y (×) takes the following form:
TL(t) = mw2(t)[sgnw(t)] (11.114)
where m is a constant.
DC motor drive system can be expressed as single-input, single-output system by combining Eqns
(11.109)–(11.110):
d 2w(t ) dw( t )
LaJ + (Ra J + LaB) + (RaB + KbKT)w (t)
dt 2 dt
dT ( t )
+ La L + Ra[TL(t) + TF] + KT va(t) = 0 (11.115)
dt
The discrete-time model is derived by replacing all continuous differentials with finite differences.

LaJ LMw (k + 1) - 2w (k ) + w (k - 1) OP + (R J + L B) LMw (k + 1) - w (k ) OP

N T 2
Q a
N a
T Q
T ( k ) - T ( k - 1) O
+ (R B + K K )w (k) + L LM L L
PQ + R T (k) + R T + K v (k) = 0 (11.116)
a b T a
N T
a L a F T a

TL(k) = m w2(k)[sgnw (k)] (11.117)

2
TL(k – 1) = m w (k – 1)[sgnw (k)] (11.118)
T = sampling period
D
w (k) = w (t = kT ); k = 0, 1, 2, ...
 Neural Networks 681
Manipulation of Eqns (11.116)–(11.118) yields
w (k + 1)= K1w (k) + K2w (k – 1) + K3[sgnw (k)]w2(k) + K4[sgnw (k)]w 2(k – 1) + K5va(k) + K6 (11.119)
2
2 La J + T ( Ra J + La B) - T ( Ra B + Kb KT )
where K1 =
La J + T ( Ra J + La B)
La J
K2 = –
La J + T ( Ra J + La B)
T (mLa + mRa T )
K3 = –
La J + T ( Ra J + La B)
(11.120)
TmLa
K4 =
La J + T ( Ra J + La B)
KT T 2
K5 =
La J + T ( Ra J + La B)
TF Ra T 2
K6 = –
La J + T ( Ra J + La B)
The following parameter values are associated with the dc motor.
J = 0.068 kg-m2
B = 0.03475 N-m/(rad/sec)
Ra = 7.56 W
La = 0.055 H
KT = 3.475 N-m/amp (11.121)
Kb = 3.475 volts/(rad/sec)
m = 0.0039 N-m/(rad/sec)2
TF = 0.212 N-m
T = 40 msec = 0.04 sec
With these motor parameters, the constants K1, K2, K3, K4, K5 and K6 become
K1 = 0.34366
K2 = –0.1534069
K3 = – 2.286928 × 10–3 (11.122)
K4 = 3.5193358 × 10–4
K5 = 0.2280595
K6 = – 0.105184
Identification of Inverse Dynamics Equation (11.119) can be manipulated to obtain the inverse dynamic
model of the drive system as
va(k) = f [w (k + 1), w (k), w (k – 1)] (11.123)
The right hand side of Eqn. (11.123) is a nonlinear function of the speed w and is given by
f(w(k + 1), w (k), w (k – 1))
1
= [w(k + 1) – K1w (k) – K2w (k – 1) – K3{sgnw (k)}w2(k) – K4{sgnw (k)}w2(k – 1) – K6] (11.124)
K5
which is assumed to be unknown (It is assumed that the only available qualitative a priori knowledge about
the plant is a rough estimate of the order of the plant). A neural network is trained to emulate the unknown
 682 Digital Control and State Variable Methods

function f (×). The values w (k + 1), w (k) and w (k – 1), which are the independent variables of f (×), are
selected as the inputs to the NN. The corresponding target f (w (k + 1), w (k), w (k – 1)) is given by Eqn.
(11.124). This quantity is also equal to the armature voltage va(k), as seen from Eqn. (11.123). Randomly
generated input patterns of [w (k + 1), w (k), w (k – 1)] and the corresponding target va(k), are used for off-
line training. The training data is generated within the constrained operating space. In conforming with the
mechanical and electrical hardware limitations of the motor, and with a hypothetical operating scenario in
mind, the following constrained operating space is defined:
– 30 < w (k) < 30 rad/sec
|w (k – 1) – w (k)| < 1.0 rad/sec (11.125)
|va(k)| < 100 volts
The estimated motor armature voltage given by the NN identifier is
v$ a (k – 1) = N(w (k), w (k – 1), w (k – 2)) (11.126)
Trajectory Control of DC Motor Using Trained NN The objective of the control system is to drive the
motor so that its speed w (k) follows a reference (prespecified) trajectory w r(k). A controller topology is
presented in Fig. 11.33. The NN trained to emulate inverse dynamics of the dc motor, is used to estimate the
motor armature voltage v$ a (k), which enables accurate trajectory control of the shaft speed w (k). Refer
Appendix B for realization of the controller.

(k + 1)
va (k) (k + 1)
DC
Motor
z–1 NN
Inverse
Model Plant

z–2

Controller

Fig. 11.33 A structure for NN-based speed control

Review Example 11.2 In this example, we study a neural-network–based identification and control
strategy for temperature control of a water bath.
The plant consists of a laboratory 7-litre water bath as depicted in Fig. 11.34. A personal computer reads
the temperature of the water bath through a link consisting of a diode-based temperature sensor module
(SM) and an 8-bit A/D converter. The plant input, produced by the computer, is limited between 0 and 5
volts, and controls the duty cycle for a 1.3 kW heater via a pulse-width-modulation (PWM) scheme.
The temperature of water in a stirred tank is described by the equation
dy(t ) Y0 - y(t )
C = + h(t) (11.127)
dt R
where y(t) is the temperature of water in the tank (ºC), h(t) is the heat flowing into the tank through the base
heater (watts), Y0 is the temperature of the surroundings (assumed constant, for simplicity), C denotes the tank
thermal capacity (Joules/ºC), and R is the thermal resistance between tank borders and surroundings. Assuming
R and C as essentially constant, we can obtain discrete-time description of the thermal system as follows:
 Neural Networks 683
D y(t) – Y
x(t) = 0
1 1 (11.128)
x& ( t ) = – x(t) + h(t) = –a x(t) + b h(t)
RC C

Fig. 11.34 Water bath control system

The discrete-time state equation (sampling period = T ):

x(k + 1) = F x(k) + g h(k) (11.129)
β
where F = e–aT; g = b z
0
T –at
e dt =
α
[1 – e–aT]

We modify this model to include a saturating nonlinearity, so that the water temperature cannot exceed some
limitation. The nonlinear plant model then becomes (obtained from real plant by experimentation)
g
y(k + 1) = F y(k) + u(k) + (1 – F)Y0 (11.130)
1 + exp [0.5 y( k ) – 40]
a = 1.00151 × 10–4
b = 8.67973 × 10–3 (11.131)
Y0 = 25ºC
T = 30 sec
u = input to the PWM, limited between 0 and 5 volts.
With these parameters, the simulated system is equivalent to a SISO temperature control system of a water
bath, that exhibits linear behaviour upto about 70ºC and then becomes nonlinear and saturates at about 80ºC.
The task is to learn how to control the plant described in Eqn. (11.130), in order to follow a specified
reference yr(k), minimizing some norm of error e(k) = yr(k) – y(k) through time. It is assumed that the model
in Eqn. (11.130) is unknown; the only available qualitative a priori knowledge about the plant is a rough
estimate of the order of the plant.
A neural network is trained to emulate the inverse dynamics of the plant. Assume that at instant k + 1, the
current output y(k + 1), the P – 1 previous values of y, and P previous values of u are all stored in memory.
Then the P pairs (xT(k – i), u(k – i)); i = 0, 1, ..., P – 1, xT(k) = [y(k + 1), y(k)], can be used as patterns for
training the NN at time k + 1. A train of pulses is applied to the plant and the corresponding input/output
pairs are recorded. The NN is then trained with reasonably large sets of data, chosen from the experimentally
 684 Digital Control and State Variable Methods

obtained data bank, in order to span a considerable region of the control space (We will use input/output
patterns generated by simulation of the plant model for training the NN).
A controller topology is presented in Fig. 11.35. It is assumed that the complete reference trajectory yr(k)
is known in advance. The feedforward component of the control input is then composed by substituting all
system outputs by corresponding reference values. Refer Appendix B for realization of the controller.

NN uff (k)
Inverse
model
z–1
Dynamic
feed forward

+
yr (k + 1) + + u (k ) y(k + 1)
Water
PID
ufb (k) bath
–
Controller Plant

Fig. 11.35 A structure for NN-based temperature control

Problems
11.1 It is believed that the output y of a plant is linearly related to the input u; that is,
y$ = w1u + w2
(a) What are the values of w1 and w2 if the following measurements are obtained:
u = 2, y = 5, u = –2, y = 1.
(b) One more measurement is taken: u = 5, y = 7. Find a least-squares estimate of w1 and w2 using all the three
measurements.
(c) Find the unique minimal sum of error squares in this linear fit to the three points.
11.2 Consider the network in Fig. P11.2. An input signal x comprising features and augmented by a constant input
component (bias) is applied to the network with linear activation function. The network gives the output ŷ .
(a) Organize the weights as row vectors:
w j = ⎡⎣ w j1 w j 2 K w jn w j0 ⎤⎦ ; j =1, 2, …., q

and write the equations (model) that this network represents.

(b) The learning environment comprises a training set of P data pairs {x( p), y( p); p = 1, 2, . . . , P} consisting
of the input vector x and output vector y.
Prove that the gradient descent learning rule for the network is
wj (k + 1) = wj (k) + h ej (k) x
where k is the iteration index, h is the learning rate parameter, and e j = y j − yˆ j
 Neural Networks 685

w11 y^1
w12
x1 w1n
w10

x2
y^2
.
.
.
.
.
xn .

y^q

1
Fig. P11.2
11.3 Consider the RBF network shown in Fig. 11.15. There are two sets of parameters governing the mapping
properties of this network: the weights w ji ; i = 1,2,..., m; j = 1, 2,..., q , in the output layer and the centre ci of the
radial basis functions. The simplest form of RBF network training is with fixed centres. In particular, they are
commonly chosen, in a random manner, as a subset of the input data set. A sufficient number of centres
randomly selected from the input data set, would distribute according to the probability density function of the
training data, thus providing an adequate sampling of the input space. Because the centres are fixed, the map-
ping performed by the hidden layer is fixed as well. Derive gradient descent training algorithm to determine the
appropriate settings of the weights in the network output layer so that the performance of the network mapping
is optimized.
11.4 It is desired to design a one-layer NN with one input x and one output y$ that associates input x(1) = – 3 with the
target output y(1) = 0.4, and input x(2) = 2 with the target output y(2) = 0.8. Determine the parameters w and w0 of
the network
y$ = s (wx + w0)
with unipolar sigmoidal (log-sigmoid) activation function, that minimize the error

E = 12 LMe y
N
(1)
- y$ (1)
2
j + ey (2)
- y$( 2 ) j OQP
2

11.5 Streamline the notation in Chapter 11 for a 3-layer NN. For instance, define Wh1 as weights of hidden layer 1
with m nodes; Wh2 as weights of hidden layer 2 with p nodes; and V as weights of output layer with q nodes.
Input variables : xi; i = 1, ..., n
Outputs of hidden layer 1 : zl; l = 1, ..., m
Outputs of hidden layer 2 : tr; r = 1, ..., p
Outputs of output layer : y$ j; j = 1, ..., q
Desired outputs : yj
Learning constant : h
Derive the backpropagation algorithm for the 3-layer network, assuming the output layer has linear activation
and the two hidden layers have unipolar sigmoidal activations.
11.6 Consider a four-input single-node perceptron with a bipolar sigmoidal function (tan-sigmoid)
2
s (a) = -1
1 + e-a
where ‘a’ is the activation value for the node.
686 Digital Control and State Variable Methods
(a) Derive the weight update rule for {wi} for all i. The learning rate h = 0.1. Input variables: xi; i = 1, 2, 3, 4.
Desired output is y.
(b) Use the rule in part (a) to update the perceptron weights incrementally for one epoch. The set of input and
desired output patterns is as follows:
x(1) = [1 –2 0 –1]T, y(1) = –1
x(2) = [0 1.5 –0.5 –1]T, y(2) = –1
x(3) = [–1 1 0.5 –1]T, y(3) = 1
The initial weight vector is chosen as
w 0 = [1 –1 0 0.5]
The perceptron does not possess bias term.
(c) Use the training data and initial weights given in part (b) and update the perceptron weights for one epoch
in batch mode.
11.7 We are given the two-layer backpropagation network shown in Fig. P11.7.
(a) Derive the weight update rules for {nl} and {wli} for all i and l. Assume that activation function for all the
nodes is a unipolar sigmoid function
1
s (a) =
1 + e-a
where ‘a’ represents the activation value for the node. The learning constant h = 0.1. The desired output is y.
1
w10
1
w11 z1
x1

w12
y
1
w21 w20

x2
w22 z2

Fig. P11.7

(b) Use the equations derived in part (a) to update the weights in the network for one step with input vector x
= [1 0]T, desired output y = 1, and the initial weights:
w10 = 1, w11 = 3, w12 = 4, w20 = –6, w21 = 6, w22 = 5
n0 = –3.92, n1 = 2, and n2 = 4
(c) As a check, compute the error with the same input for initial weights and updated weights and verify that
the error has decreased.
11.8 We are given the two-layer backpropagation network in Fig. P11.8.
Derive the weight update rules for {nl} and {wl} for all l. Assume that activation function for all the nodes is a
bipolar sigmoid function
2
s (a) = -1
1 + e-a
where ‘a’ is the activation value for the node. The learning constant is h = 0.4. The desired output is y.
 Neural Networks 687
1
w10

w1 z1

n1
1
1 n0
w20
w2 z2 n2 y
x

1
wm0 nm
wm zm

Fig. P11.8
 Fuzzy Logic Models
12
12.1 INTRODUCTION

In a man–machine system, there arises the problem of processing information with the “vagueness” that is
characteristic of man. We consider here a real-life situation in process control.
The basic structure of a feedback control
yr + e u y system is shown in Fig. 12.1. G represents the
D G system to be controlled (plant or process). The
– purpose of the controller D is to guarantee a
Controller Plant desired response of the output y. The process of
keeping the output y close to the set-point (refer-
Fig. 12.1 Basic structure of a feedback control system ence input) yr, despite the presence of distur-
bances, fluctuations of the system parameters,
and noisy measurements, is called regulation. The law governing corrective action of the controller is called
the control algorithm. The output of the controller is the control action u.
The general form of the control law (implemented using a digital computer) is
u(k) = f(e(k), e(k – 1), ..., e(k – m), u(k – 1), ..., u(k – m)) (12.1)
providing a control action that describes the relationship between the input and the output of the controller.
In Eqn. (12.1), e = yr – y represents the error between the desired set-point yr and the output of the controlled
system; parameter m defines the order of the controller; and f (×) is, in general, a nonlinear function. k is an
index representing sampling instant; T is the sampling interval used for digital implementation (Fig. 12.2).
To distinguish control law (12.1) from the control schemes based on fuzzy logic/neural networks, we shall
call this, conventional control law.
e(t) e (k ) u (k)
yr(t) + y ( t)
Zero-order
Controller Plant
– T Hold

Fig. 12.2 Basic structure of a digital control system

 Fuzzy Logic Models 689
A common feature of conventional control is that the control algorithm is analytically described by equa-
tions—algebraic, difference, differential, and so on. In general, the synthesis of such control algorithms
requires a formalized analytical description of the controlled system by a mathematical model. The concept
of analyticity is one of the main paradigms of conventional control theory. We will also refer to conventional
control as model-based control.
When the underlying assumptions are satisfied, many of the model-based control techniques provide
good stability, robustness to model uncertainties and disturbances, and speed of response. However, there
are many practical deficiencies of these control algorithms. It is, generally, difficult to accurately represent
a complex process by a mathematical model. If the process model has parameters whose values are partially
known, ambiguous or vague, the control algorithms that are based on such incomplete information will not,
usually, give satisfactory results. The environment with which the process interacts may not be completely
predictable, and it is normally not possible for a model-based algorithm to accurately respond to a condition
that it did not anticipate. Skilled human operators are, however, controlling complex plants quite success-
fully on the basis of their experience, without having quantitative models.
Regulatory control objectives, typical of many industrial applications, are
· to remove any significant errors in process output y(t) by appropriate adjustment of the controller
output u(k),
· to prevent process output from exceeding some user-specified constraint yc, i.e., for all t, y(t) should be
less than or equal to yc, and
· to produce smooth control action near the set-point, i.e., minor fluctuations in the process output are
not passed further to the controller.
A conventional PI controller uses an analytical expression of the following form to compute the control
action:
LM 1 e(t ) dt OP
T z
u(t) = K c¢ e(t ) + (12.2)
where K c¢
N
is the controller gain, and T is integral or reset time.
Q I
I
When this expression is differentiated, we obtain
u& (t) = Kc¢ e& (t) + Kc¢ e(t)
TI
The discrete-time version of this equation may be written as
u(k ) - u(k - 1)
= K c¢ LM
e( k ) - e( k - 1) OP
K ¢ e (k)
+ c
T N T Q
TI
or Du(k) = Kcv(k) + KI e(k) (12.3)
where
Du(k) = incremental change in control variable = u(k) – u(k – 1);
e(k) = error variable = yr – y(k);
e( k ) - e( k - 1)
v(k) = time rate of change of error1 =
T

1
A PD controller in position form is
u(k) = Kce(k) + KDv(k)
We see that PD controller in position form is structurally related to PI controller in incremental form.
 690 Digital Control and State Variable Methods
The control objectives, listed earlier, would require variable gains when the process output is in different
regions around the set-point. Figure 12.3 illustrates the type of control action desired; Du should be “near
zero” in the set-point region, very large in the constraint region, and normal in between.

yc

yr

u
Set-point Normal Constraint
region region region
Smooth u u umax

Time
Fig. 12.3 Type of control action desired in different regions around the set-point

A simple PI controller is inherently incapable of achieving all of the above control objectives, and has to be
implemented with additional (nonlinear) control laws for set-point and constraint regions, making the control
scheme a complex adaptive control scheme which would allow the desired gain modification when required.
On the other hand, an experienced process op-
yr Rule-base
u(k) y (t ) erator can easily meet all the three control objec-
of the Plant tives. An expert operator employs, consciously or
Expert Operator
subconsciously, a set of IF-THEN rules to control
a process (Fig. 12.4). He estimates the error e(k)
and time rate of change of error v(k) at a specific
Fig. 12.4 A man–machine control system time instant, and based on this information he
changes the control by Du(k).
A typical production rule of the rule-base in Fig. 12.4 is of the form:
IF (process state) THEN (control action) (12.4)
instead of an analytical expression defining the control variable as a function of process state. The “process
state” part of the rule is called the rule premise (or antecedent), and contains a description of the process
state at the kth sampling instant. This description is done in terms of particular values of error e(k), velocity
(time rate of change of error) v(k), and the constraint. The “control action”, part of the rule is called the
conclusion (or consequent), and contains a description of the control variable which should be produced
given the particular process state in the rule antecedent. This description is in terms of the value of the
change-in-control, Du(k).
Negative values of e(k) mean that the current process output y(k) has a value above the set-point yr, since
e(k) = yr – y(k) < 0. The magnitude of a negative value describes the magnitude of the difference yr – y. On
 Fuzzy Logic Models 691
the other hand, positive values of e(k) express the knowledge that the current value of the process output y(k)
is below the set-point yr. The magnitute of such a positive value is the magnitude of the difference yr – y.
Negative values of v(k) mean that the current process output y(k) has increased compared with its
previous value y(k – 1), since v(k) = – (y(k) – y(k – 1))/T < 0. The magnitude of such a negative value describes
the magnitude of this increase. Positive values of v(k) express the knowledge that y(k) has decreased its
value when compared to y(k – 1). The magnitude of such a value is the magnitude of the decrease.
Positive values of Du(k) mean that the value of the control u(k – 1), has to be increased to obtain the value
of the control for the current sampling time k. A value with a negative sign means a decrease in the value of
u(k – 1). The magnitude of such a value is the magnitude of increase/decrease in the value u(k – 1).
The possible combinations of positive/negative values of e(k) and v(k) are as follows:
· positive e, negative v
· negative e, positive v
· negative e, negative v
· positive e, positive v
The combination (positive e(k), negative v(k)) implies that y < yr, since e(k) = yr – y(k) > 0; and y& > 0,
since v(k) = – (y(k) – y(k – 1))/T < 0. This means that the current process output y(k) is below the set-point
and the controller is driving the system upward, as shown by point D in Fig. 12.5. Thus, the current process
output is approaching the set-point from below. The combination (negative e(k), positive v(k)) implies that
y > yr, and y& < 0. This means that the current process output is above the set-point and the controller is
driving the system downward, as shown by point B in Fig. 12.5. Thus the current process output is approach-
ing the set-point from above. The combination (negative e(k), negative v(k)) implies that y > yr and y& > 0.
This means that the current process output y(k) is above the set-point and the controller is driving the system
upward, as shown by point C in Fig. 12.5. Thus the process output is moving further away from the set-point
and approaching overshoot. The combination (positive e(k), positive v(k)) implies that y < yr and y& < 0. This
means that the current process output is below the set-point and the controller is driving the system down-
ward, as shown by point A in Fig. 12.5. Thus the process output is moving further away from the set-point
and approaching undershoot.

y (k )

1.5
B

C
yr
A
D

0.5

0 5 10 15 20 25 30 35 40 k
Fig. 12.5 Process output deviations from the set-point
 692 Digital Control and State Variable Methods
In a man-machine control system of the type shown in Fig. 12.4, experience-based knowledge of the
process operator and/or control engineer is instrumental in changing the control by Du(k), for a given
estimate of error e(k) and time rate of change of error v(k).
(i) If both e(k) and v(k) (positive or negative) are small (or zero), it means that the current value of the
process output variable y(k) has deviated from the set-point but is still close to it. The amount of
change Du(k) in the previous control u(k – 1) should also be small (or zero) in magnitude, which is
intended to correct small deviations from the set-point.
(ii) Consider the situation when e(k) has large negative value (which implies that y(k) is significantly
above the set-point). If v(k) is positive at the same time, this means that y is moving towards the set-
point. The amount of change Du to be introduced is intended to either speed up or slow down the
approach to the set-point. For example, if y(k) is much above the set-point (e(k) has a large negative
value) and it is moving towards the set-point with a small step (v(k) has small positive value), then the
magnitude of this step has to be significantly increased (Du(k) ® large negative value).
(iii) e(k) has either a small value (positive, negative, zero) or a large positive value, which implies that y(k)
is either close to the set-point or significantly below it. If v(k) is positive at the same time, this means
that y is moving away from the set-point. Then, a positive change Du(k) in the previous control u(k – 1)
is required to reverse this trend and make y start moving towards it, instead of moving away from the
set-point.
(iv) Consider a situation when e(k) has large positive value (which implies that y(k) is significantly below
the set-point) and v(k) is negative (which implies that y is moving towards the set-point). The amount
of change Du to be introduced is intended to either speed up, or slow down, the approach to the set-
point. For example, if y(k) is much below the set-point (e(k) has large positive value), and it is moving
towards the set-point with somewhat large step (v(k) has large negative value), then the magnitude of
this step need not be changed (Du(k) ® 0), or only slightly enlarged (Du(k) ® small positive value).
(v) e(k) has either a small value (positive, negative, zero) or a large negative value, and this implies that y(k)
is either close to the set-point or significantly above it. If v(k) is negative at the same time, y is moving
away from the set-point. Thus, a negative change Du(k) in the previous control u(k – 1) is required to
reverse this trend and make y start moving towards it instead of moving away from the set-point.
The variables e, v and Du are described as consisting of a finite number of verbally expressed values
which these variables can take. Values are expressed as tuples of the form {value sign, value magnitude}.
The ‘value sign’ component of such a tuple takes on either one of the two values: positive or negative. The
‘value magnitude’ component can take on any number of magnitudes, e.g., {zero, small, medium, big}, or
{zero, small, big}, or {zero, very small, small, medium, big, very big}, etc.
The tuples of values may therefore look like: Negative Big (NB), Negative Medium (NM), Negative
Small (NS), Zero (ZO), Positive Small (PS), Positive Medium (PM), Positive Big (PB) or an enhanced set/
subset of these values.
We consider here a simple rule-based controller which employs only three values of the variables e, v,
and Du: Negative (N), Near Zero (NZ), Positive (P), for e and v; and Negative (N), Zero (Z), Positive (P) for
Du. A typical production rule of the rule-base in Fig. 12.4 is
IF e(k) is Positive and v(k) is Positive THEN Du(k) is Positive (12.5)
Let us see now what such a rule actually means. A positive e(k) implies that y(k) is below the set-point. If
v(k) is positive at the same time, it means that y(k) is moving away from the set-point. Thus, a positive
change Du(k) in the previous control u(k – 1) is required to reverse this trend.
Consider another rule:
IF e(k) is Positive and v(k) is Negative THEN Du(k) is Zero (12.6)
 Fuzzy Logic Models 693
This rule says that if y(k) is below the set-point, and is moving
towards the set-point, then, no change in control is required.
We will present the rule-base in table format, shown in Fig. 12.6.
The cell defined by the intersection of the third row and third column
Fig. 12.6 The rule-base for represents the rule given in (12.5), and the cell defined by the inter-
system of Fig. 12.4 section of the third row and first column represents the rule given
in (12.6).
The rule-base shown in Fig. 12.6 is designed to remove any significant errors in process output by
appropriate adjustment of the controller output. Note that the rule
IF e(k) is Near Zero and v(k) is Near Zero THEN Du(k) is Zero (12.7)
ensures smooth action near the set-point, i.e., minor fluctuations in the process output are not passed further
to the controller.
The rule-base of Fig. 12.6 is thus effective for control action in the set-point region and the normal region
in Fig. 12.3. However, we require additional rules for the constraint region. The following three rules pre-
scribe a control action when the error is in the constraint region, approaching it or leaving it.
(i) IF e(k) is in constraint region THEN value of Du(k) is drastic change.
This rule specifies the magnitude of additional DU(k) to be added to the one already determined by the
rules of Fig. 12.6 when e(k) is in the constraint region.
(ii) IF e(k) enters constraint region, THEN start summing up the values of Du(k) determined by constraint
rule 1.
(iii) IF e(k) leaves constraint region, THEN subtract the total value of Du(k) determined by constraint rule 2.
The man-machine control system of Fig. 12.4 has the capability of representing and manipulating data
that is not precise, but rather fuzzy. The error variable is ‘near zero’, change in control is ‘drastic’, etc.,—are
the type of linguistic information which the expert controller is required to handle. But what is a ‘drastic
change’ in control? The property ‘drastic’ is inherently vague, meaning that the set of signals it is applied to,
has no sharp boundaries between ‘drastic’ and ‘not drastic’. The fuzziness of a property lies in the lack of
well-defined boundaries of the set of objects to which the property applies.
Problems featuring uncertainty and ambiguity have been successfully addressed subconsciously by hu-
mans. Humans can adapt to unfamiliar situations and they are able to gather information in an efficient
manner, and discard irrelevant details. The information gathered need not be complete and precise and could
be general, qualitative and vague because humans can reason, infer and deduce new information and knowl-
edge. They can learn, perceive and improve their skills through experience.
How can humans reason about complex systems, when the complete description of such a system often
requires more detailed data than a human could ever hope to recognize simultaneously, and assimilate with
understanding? The answer is that humans have the capacity to reason approximately. In reasoning about a
complex system, humans reason approximately about its behaviour, thereby maintaining only a generic
understanding about the problem.
The seminal work by Dr. Lotfi Zadeh (1965) on system analysis based on the theory of fuzzy sets, has
provided a mathematical strength to capture the uncertainties associated with human cognitive processes,
such as thinking and reasoning. The conventional approaches to knowledge representation, lack the means
for representing the meaning of fuzzy concepts. As a consequence, the approaches based on classical logic
and probability theory, do not provide an appropriate conceptual framework for dealing with the representa-
tion of commonsense knowledge, since such knowledge is by its nature, both, lexically imprecise and
 694 Digital Control and State Variable Methods

non-categorical. The development of fuzzy logic was motivated, in large measure, by the need for a concep-
tual framework which can address the issue of uncertainty and lexical imprecision. Fuzzy logic provides an
inference morphology, that enables approximate human reasoning capabilities to be applied to knowledge-
based systems.
Since the publication of Zadeh’s seminal work “Fuzzy Sets” in 1965, the subject has been the focus of
many independent research investigations by mathematicians, scientists and engineers from around the
world. Fuzzy logic has rapidly become one of the most successful of technologies today, for developing
sophisticated control systems. With its aid, complex requirements may be implemented in amazingly simple,
easily maintained and inexpensive controllers. Of course, fuzzy logic is not the best approach for every
control problem. As designers look at its power and expressiveness, they must decide where to apply it.
The criteria, in order of relevance, as to when and where to apply fuzzy logic are as follows:
· Human (structured) knowledge is available.
· A mathematical model is unknown or impossible.
· The process is substantially nonlinear.
· There is lack of precise sensor information.
· It is applied at the higher levels of hierarchical control systems.
· It is applied in generic decision-making problems
Possible difficulties in applying fuzzy logic, arise from the following:
· Knowledge is subjective.
· For high-dimensional inputs, the increase in the required number of rules is exponential.
· Knowledge must be structured, but experts bounce between a few extreme poles: they have trouble
structuring the knowledge; they are too aware of their “expertise”; they tend to hide knowledge; and
there may be some other subjective factors working against the whole process of human knowledge
transfer.
Note that the basic premise of fuzzy logic is that a human solution is good. When applied, for example, in
control systems, this premise means that a human being is a good controller. Today, after several thousands
successful applications, there is more or less convergence on trustworthiness of this premise.
The word ‘fuzzy’ may sound to mean intrinsically imprecise, but there is nothing ‘fuzzy’ about fuzzy
logic. It is firmly based on multivalued logic theory and does not violate any well-proven laws of logic. Also
fuzzy logic systems can produce answers to any required degree of accuracy. This means that these models
can be very precise if needed (There is a trade-off between precision and cost). However, they are aimed at
handling imprecise and approximate concepts that cannot be processed by any other known modelling tool.
In this sense, fuzzy logic models are invaluable supplements to classical hard computing techniques. For
example in a hierarchical control system, classical control at the lowest level, supplemented by fuzzy logic
control at higher levels provides good hybrid solution in many situations.
Our focus in this chapter is on the essential ideas and tools necessary for the construction of the fuzzy
knowledge-based models, that have been successful in the development of intelligent controllers. Fuzzy
control and modelling use only a small portion of the fuzzy mathematics that is available; this portion is also
mathematically quite simple and conceptually, easy to understand. This chapter begins with an introduction
to some essential concepts, terminology, notations and arithmetic of fuzzy sets and fuzzy logic. We include
only a minimum, though adequate, amount of fuzzy mathematics necessary for understanding fuzzy control
and modelling. To facilitate easy reading, this background material is presented in a rather informal manner,
with simple and clear notation as well as explanation. Whenever possible, excessively rigorous mathematics
is avoided. This material is intended to serve as an introductory foundation for the reader to understand not
only the fuzzy controllers presented later in this chapter but also others in the literature. We point to refer-
ences [137, 138, 142–145] as further sources on fuzzy set theory.
 Fuzzy Logic Models 695

12.2 FUZZY QUANTIFICATION OF KNOWLEDGE

Up to this point we have only quantified, in an abstract way, the knowledge that the human expert has about
how to control the plant. Next, we will show how to use fuzzy logic to fully quantify the meaning of linguistic
descriptions so that we may automate in the fuzzy controller, the control rules specified by the expert.

12.2.1 What is Fuzzy Logic?

Knowledge is structured information and knowledge acquisition is done through learning and experience,
which are forms of high-level processing of information. Knowledge representation and processing are the
keys to any intelligent system. In logic, knowledge is represented by propositions and is processed through
reasoning, by the application of various laws of logic, including an appropriate rule of inference.
Fuzzy logic focusses on linguistic variables in natural language, and aims to provide foundations for
approximate reasoning with imprecise propositions.
In classical logic, a proposition is either TRUE, denoted by 1, or FALSE, denoted by 0. Consider the
following proposition p:
“Team member is female”
Let X be a collection of 10 people: x1, x2, K, x10, who form a project team. The entire object of discussion is
X = {x1, x2, K, x10}
In general, the entire object of discussion is called a “universe of discourse”, and each constituent member x
is called an “element” of the universe (the fact that x is an element of X, is written as x Î X).
If x1, x2, x3 and x4 are female members in the project team, then the proposition p on the universe of
discourse X is equally well represented by the crisp (non-fuzzy) set A defined below:
A = {x1, x2, x3, x4}
The fact that A is a subset of X is denoted as A Ì X.
The proposition can also be expressed by a mapping mA from X into the binary space {0, 1}.
mA : X ® {0, 1}
such that

mA =
RS0 ; x = x , x , x , x , x , x
5 6 7 8 9 10

T1 ; x = x , x , x , x
1 2 3 4
That is to say, the value mA(x) = 1 when the element x satisfies the attributes of set A; 0 when it does not.
mA is called the characteristic function of A.
Next, supposing that, within X, only x1 and x2 are below age 20; we may call them “minors”. Then
B = {x1, x2}
consists of minor team members. In this case

mB (x) =
RS1 ; x = x , x
1 2

T 0 ; otherwise
B is obviously a subset of A; we write B Ì A.
We have considered the “set of females A”, and the “set of minors B” in X. Is it also possible to consider
a “set of young females C”? If, for convenience, we consider the attribute “young” to be same as “minor”,
then C = B; but, in this case, we have created a sharp boundary, under which x2 who is 19 is still young
(mC (x2) = 1), but x3 who just turned 20 today is no longer young (mC (x3) = 0). In just one day, the value
changed from yes (1) to no (0), and x3 is now an old maid.
 696 Digital Control and State Variable Methods
However, is it not possible that a young woman becomes an old maid over a period of 10 to 15 years, so
that we ought to be patient with her? Prof. Zadeh admitted values such as 0.8 and 0.9 that are intermediate
between 0 and 1, thus creating the concept of a “fuzzy set”. Whereas a crisp set is defined by the charac-
teristic function that can assume only the two values {0, 1}, a fuzzy set is defined by a “membership
function” that can assume an infinite number of values; any real number in the closed interval [0, 1].
With this definition, the concept of “young women” in X can be expressed flexibly in terms of member-
ship function (Fuzzy sets are denoted in this book by a set symbol with a tilde understrike).
μ C : X ® [0, 1]
~
such that
R|1; x = x1 , x 2

μ =S
C
| 0.9; x = x3
~
|| 0.2; x = x4
T 0; otherwise
The significance of such terms as “patient” and “ flexibly” in the above description may be explained as
follows. For example, we have taken μ C (x3) = 0.9, but suppose that x3 objects that “you are being unfair; I
~
really ought to be a 1 but if you insist we can compromise on 0.95”. There is a good amount of subjectivity
in the choice of membership values. A great deal of research is being done on the question of assignment of
membership values. However, even with this restriction, it has become possible to deal with many problems
that could not be handled with only crisp sets.
Since [0, 1] incorporates {0, 1}, the concept of fuzzy set can be considered as an extended concept, which
incorporates the concept of crisp set. For example, the crisp set B of ‘minors’ can be regarded as a fuzzy set
B with the membership function:

m B (x) =
RS
1; x = x1 , x 2
~
T
0; otherwise

Example 12.1 One of the most commonly used examples of a fuzzy set is the set of tall people. In this case,
the universe of discourse is potential heights (the real line), say, from 3 feet to 9 feet. If the set of tall people is
given the well-defined boundary of a crisp set, we might say all people taller than, 6 feet are officially consid-
ered tall. The characteristic function of the set A = {tall men} then, is

mA(x) =
RS
1 for 6 ≤ x
0 for 3 ≤ x < 6
T
Such a condition is expressed by a Venn diagram shown in Fig. 12.7a, and a characteristic function shown in
Fig. 12.8a.

Fig. 12.7
 Fuzzy Logic Models 697

Fig. 12.8

For our example of universe X of heights of people, the crisp set A of all people with x ³ 6 has a sharp
boundary: individual, ‘a’ corresponding to x = 6 is a member of the crisp set A, and individual ‘b’ corre-
sponding to x = 5.9 is unambiguously not a member of set A. Is it not an absurd statement for the situation
under consideration? A 0.1" reduction in the height of a person has changed mA from 1 to 0, and the person
is no more tall.
It may make sense to consider the crisp set of all real numbers greater than 6 because the numbers belong
to an abstract plane, but when we want to talk about real people, it is unreasonable to call one person short
and another one tall, when they differ in height by the width of a hair. But if this kind of distinction is
unworkable, than what is the right way to define the set of all people? Much as with our example of “set of
young females”, the word ‘tall’ would correspond to a curve that defines the degree to which any person is
tall. Figure 12.8b shows a possible membership function of this fuzzy set ~ A; the curve defines the transition
from not tall to tall. Two people with membership values 0.9 and 0.3 are tall to some degree, but one
significantly less than the other.
Note that there is inherent subjectivity in fuzzy set description. Figure 12.9 shows a smoothly varying
curve (S-shaped) for transition from not tall to tall. Compared to Fig. 12.8b, the membership values are
lower for heights close to 3¢ and are higher for heights close to 6¢. This looks more reasonable; however, the
price paid is in terms of a more complex function, which is more difficult to handle.
A

0 x
3 6 9
Fig. 12.9 A smoothly varying membership function for fuzzy set A
~
 698 Digital Control and State Variable Methods

Figure 12.7b shows the representation of a fuzzy set by a Venn diagram. In the central (unshaded) region
of the fuzzy set, μ ~A (x) = 1. Outside the boundary region of fuzzy set, μ ~A (x) = 0. On the boundary region,
μ A (x) assumes an intermediate value in the interval (0, 1). Presumably, the membership value of an x in
~
fuzzy set, ~A , approaches a value of 1 as it moves closer to the central (unshaded) region; it approaches a
value of 0 as it moves closer to leaving the boundary region of A .

Thus far we have discussed the representation of knowledge in logic. We have seen that the concept of
fuzzy sets makes it possible to describe vague information (knowledge). But description alone will not lead
to the development of any useful products. Indeed, a good deal of time passed after fuzzy sets were first
proposed, until they were applied at the industrial level. However, eventually it became possible to apply
them in the form of “fuzzy inference”, and fuzzy logic theory has now become legitimized as one component
of applied high technology.
In fuzzy logic theory, nothing is done at random or haphazardly. Information containing a certain amount
of vagueness is expressed as faithfully as possible, without the distortion produced by forcing it into a
“crisp” mould, and it is then processed by applying an appropriate rule of inference.
‘Approximate reasoning’ is the best known form of fuzzy logic processing and covers a variety of infer-
ence rules.
Fuzziness is often confused with probability. The fundamental difference between them is that fuzziness
deals with deterministic plausibility, while probability concerns the likelihood of non-deterministic
(stochastic) events. Fuzziness is one aspect of uncertainty. It is the ambiguity (vagueness) found in the
definition of a concept, or the meaning of a term. However, the uncertainty of probability generally relates to
the occurrence of phenomena, not the vagueness of phenomena. For example “There is a 50–50 chance that
he will be there” has the uncertainty of randomness. “He is a young man”, has the uncertainty in definition of
“young man”. Thus, fuzziness describes the ambiguity of an event, whereas randomness describes the uncer-
tainty in the occurrence of an event.
We can now give a formal definition to fuzzy sets.

12.2.2 Fuzzy Sets

A universe of discourse, X, is a collection of objects all having the same characteristics. The individual
elements in the universe X will be denoted as x.
A universe of discourse and a membership function that spans the universe, completely define a fuzzy set.
A in X has the member-
Consider a universe of discourse X with x representing its generic element. A fuzzy set ~
ship function μ ~A (x) which maps the elements of the universe onto numerical values in the interval [0, 1]:
μ A (x) : X ® [0, 1] (12.8a)
~
Every element x in X has a membership function m A (x) Î [0, 1]. ~
A is then defined by the set of ordered pairs:
~

~A = {( x, μ A ( x)) | x ∈ X , μ A ( x) ∈ [0,1]}
~ ~
(12.8b)
A membership value of zero implies that the corresponding element is definitely not an element of the
fuzzy set ~
A . A membership function of unity means that the corresponding element is definitely an element
of fuzzy set ~A . A grade of membership greater than zero, and less than unity, corresponds to a noncrisp (or
fuzzy) membership of the fuzzy set A . Classical sets can be considered as special case of fuzzy sets with all
membership grades equal to unity.
A fuzzy set A is formally given by its membership function μ ~A (x). We will identify any fuzzy set with its
membership function, and use these two terms interchangeably.
 Fuzzy Logic Models 699
Membership functions characterize the fuzziness in a fuzzy set. However, the shape of the membership
functions, used to describe the fuzziness, has very few restrictions indeed. It might be claimed that the rules
used to describe fuzziness are also fuzzy. Just as there are an infinite number of ways to characterize
fuzziness, there are an infinite number of ways to graphically depict the membership functions that describe
fuzziness. Although the selection of membership functions is subjective, it cannot be arbitrary; it should be
plausible.
To avoid unjustified complications, μ ~A (x) is usually constructed without a high degree of precision. It is
advantageous to deal with membership functions involving a small number of parameters. Indeed, one of
the key issues in the theory and practice of fuzzy sets is how to define the proper membership functions.
Primary approaches include (1) asking the control expert to define them; (2) using data from the system to be
controlled, to generate them; and (3) making them in a trial-and-error manner. In more than 25 years of
practice, it has been found that the third approach, though adhoc, works effectively and efficiently in many
real-world applications.
Numerous applications in control have shown that only four types of membership functions are needed
in most circumstances: trapezoidal, triangular (a special case of trapezoidal), Gaussian, and bell-shaped.
Figure 12.10 shows an example of each type. Among the four, the first two are more widely used. All these
fuzzy sets are continuous, normal and convex.
A fuzzy set is said to be continuous if its membership function is continuous.
A fuzzy set is said to be normal if its height is one (The largest membership value of a fuzzy set is called
the height of the fuzzy set).
The convexity property of fuzzy sets is viewed as a generalization of the classical concept of crisp sets.
Consider the universe X to be a set of real numbers ℜ. A subset A of ℜ is said to be convex if, and only if, for
all x1, x2 Î A and for every real number l satisfying 0 £ l £ 1, we have
lx1 + (1 – l)x2 Î A (12.9)
(x) (x)

1 1

0 x 0 x
(a) (b)
(x) (x)
1 1

0 x 0 x
(c) (d)
Fig. 12.10 Examples of fuzzy sets; (a) trapezoidal, (b) triangular (c) Gaussian, and (d) bell-shaped

It can easily be established that any set defined by a single interval of real numbers is convex; any set
defined by more than one interval, that does not contain some points between the intervals, is not convex.
 700 Digital Control and State Variable Methods
An alpha-cut of a fuzzy set A is a crisp set Aa that contains all the elements of the universal set X that have
~
a membership grade in A greater than or equal to a (refer Fig. 12.11). The convexity property of fuzzy sets
~
is viewed as a generalization of the classical concept of convexity of crisp sets. In order to make the gener-
alized convexity consistent with the classical definition of convexity, it is required that a-cuts of a convex
fuzzy set be convex for all a Î (0, 1] in the classical sense (0-cut is excluded here since it is always equal to
ℜ in this sense and thus includes – ¥ to + ¥). Figure 12.11a shows a fuzzy set that is convex. Two of the
a-cuts shown in this figure are clearly convex in the classical sense, and it is easy to see that any other a-cuts
for a > 0 are convex as well. Figure 12.11b illustrates a fuzzy set that is not convex. The lack of convexity
of this fuzzy set can be demonstrated by identifying some of its a-cuts (a > 0) that are not convex.

(x) (x)
~A
~A
1 1

x
A 2
x
A 1 A—
(a) (b)
Fig. 12.11 (a) Convex fuzzy set (b) Nonconvex fuzzy set
A is the crisp set of all x Î X such that μ ~A (x) > 0. That is
The support of a fuzzy set ~
supp( A ) = {x ∈ X | μ A ( x) > 0} (12.10)
~ ~
The element x Î X at which μ ~A (x) = 0.5, is called the crosspoint.
A whose support is a single point in X with μ ~A (x) = 1, is referred to as a fuzzy singleton.
A fuzzy set ~

Example 12.2 Consider the fuzzy set described by membership function depicted in Fig. 12.12, where
the universe of discourse is
X = [32ºF, 104ºF]

x
32°F 64° 70° 74° 78° 104°
Fig. 12.12 A fuzzy set: linguistic ‘warm’
 Fuzzy Logic Models 701
This fuzzy set ~
A is linguistic ‘warm’, with membership function
R| 0 ; x < 64º
( x - 64º ) / 6 ; 64º £ x < 70 º
μA (x) = S
| 1 ; 70 º < x £ 74º
~
|| (78º - x)/ 4 ; 74 < x £ 78º
|T 0 ; x > 78º
The support of A is the crisp set
~
kx | 64º < x < 78ºp
Example 12.3 Consider a natural language form expression:
“Speed sensor output is very large”
The formal, symbolic translation of this natural language expression, in terms of linguistic variables,
proceeds as follows:
(i) An abbreviation ‘Speed’ may be chosen to denote the physical variable “Speed sensor output”.
(ii) An abbreviation ‘XFast’ (i.e., extra fast) may be chosen to denote the particular value “very large” of speed.
(iii) The above natural language expression is rewritten as “Speed is XFast”.
Such an expression is an atomic fuzzy proposition. The ‘meaning’ of the atomic proposition is
then defined by a fuzzy set XFast , or a membership function m XFast (x), defined on the physical domain
~ ~
X = [0 mph, 100 mph] of the physical variable ‘Speed’.
Many atomic propositions may be associated with a linguistic variable, e.g.,
“Speed is Fast”
“Speed is Moderate”
“Speed is Slow”
“Speed is XSlow”
Thus, the set of linguistic values that the linguistic variable ‘Speed’ may take is
{XFast, Fast, Moderate, Slow, XSlow}
These linguistic values are called terms of the linguistic variable. Each term is defined by an appropriate
membership function.

It is usual in approximate reasoning to have the following frame associated with the notion of a linguistic
variable:

~A, L~A , X , μLA~ (12.11)

where ~A denotes the symbolic name of a linguistic variable, e.g., speed, temperature, level, error, change-
A can take, i.e., L~A is the term set of ~
of-error, etc. L A is the set of linguistic values that ~
~ A . X is the actual
physical domain over which linguistic variable ~ A takes its quantitative (crisp) values, and m LA is a member-
~
ship function which gives a meaning to the linguistic value in terms of the quantitative elements of X.

Example 12.4 Consider speed, interpreted as a linguistic variable with X = [0mph, 100mph]; i.e., x =
‘speed’. Its term set could be
{Slow, Moderate, Fast}
 702 Digital Control and State Variable Methods

Slow = the fuzzy set for “a speed below about 40 miles per hour (mph)”, with membership function μSlow
~ ~
Moderate = the fuzzy set for “a speed close to 55 mph”, with membership function μModerate .
~ ~
Fast = the fuzzy set for “a speed above about 70 mph”, with membership function μ Fast .
~ ~
The frame of speed is

Speed
~
, LSpeed
~
, X , μLSpeed
~

where
L Speed = {Slow ,
~ Moderate
~
, Fast
~
}
~
X = [0, 100] mph
mSlow , mModerate , mFast are given in Fig. 12.13.
~ ~ ~
The frame of speed helps us to decide the degree
to which an atomic proposition associated with
Slow Moderate Fast ‘speed’ is satisfied, given a specific physical value
1
of speed. For example, for crisp input
Speed = 50 mph,
mSlow (50) = 1/3
0.5 ~
mModerate (50) = 2/3
~
m Fast (50) = 0
~
0 Therefore, the proposition “Speed is Slow” is satis-
40 55 70 x fied to a degree of 1/3, the proposition “Speed is
Fig. 12.13 Terms of linguistic variable ‘speed’ Moderate” is satisfied to a degree of 2/3, and the
proposition “Speed is Fast” is not satisfied.

An extension of ordinary fuzzy sets is to allow the membership values to be a fuzzy set—instead of a
crisply defined degree. A fuzzy set whose membership function is itself a fuzzy set, is called a type-2 fuzzy
set [143]. A type-1 fuzzy set is an ordinary fuzzy set. We will limit our discussion to type-1 fuzzy sets. The
reference to a fuzzy set in this chapter, implies a type-1 fuzzy set.

12.2.3 Fuzzy Operations

There are a variety of fuzzy set theories which differ from one another by the set operations (complement,
intersection, union) they employ. The fuzzy complement, intersection and union are not unique operations,
contrary to their crisp counterparts; different functions may be appropriate to represent these operations in
different contexts. That is, not only membership functions of fuzzy sets, but also operations on fuzzy sets, are
context-dependent. The capability to determine appropriate membership functions, and meaningful fuzzy op-
erations in the context of each particular application, is crucial for making fuzzy set theory practically useful.
The intersection and union operations on fuzzy sets are often referred to as triangular norms (t-norms),
and triangular conorms (t-conorms; also called s-norms), respectively. The reader is advised to refer [143]
for the axioms which t-norms, t-conorms, and the complements of fuzzy sets are required to satisfy.
In the following, we define standard fuzzy operations, which are generalizations of the corresponding
crisp set operations.
Consider the fuzzy sets ~ A and B in the universe X.
~
~A = {( x, m ~A ( x )) | x Î X; m ~A ( x) Î [0, 1]} (12.12)
B = {( x, m B ( x )) | x Î X; m B ( x ) Î [0, 1]}
~ (12.13)
~ ~
 Fuzzy Logic Models 703
A and ~
The operations with ~ B are introduced via operations on their membership functions mA(x) and
~
μ B (x) correspondingly.
~
Complement A with respect to the universal set X , is defined
The standard complement, A , of fuzzy set ~
~
for all x Î X by the equation
μ A ( x) =D 1 − μ A ( x) ∀x ∈ X (12.14)
~ ~
Intersection The standard operation, A ∩ B is defined for all x Î X by the equation
~ ~
μ A ∩ B ( x) =D min [μ A ( x), μ B ( x)] º μ A ( x) Ù μB ( x) "x Î X (12.15)
~ ~ ~ ~ ~ ~
where Ù indicates the min operation.
Union The standard union, A ∪ B , is defined for all x Î X by the equation
D max [μ ( x), μ ( x)] º μ ( x) Ú μ ( x) "x Î X
μ A ∪ B ( x) = (12.16)
B
~ ~ ~A B
~ ~A ~
where Ú indicates the max operation.

12.2.4 Fuzzy Relations

Consider two universes (crisp sets) X and Y. The Cartesian product (or cross product) of two sets X and Y (in
this order) is the set of all ordered pairs, such that, the first element in each pair is a member of X, and the
second element is a member of Y. Formally,
X ´ Y = {(x, y); x Î X, y Î Y } (12.17)
where X ´ Y denotes the Cartesian product.
A fuzzy relation on X ´ Y, denoted by R
~ , or R
~ (X, Y ) is defined as the set
~ = {(( x, y), μR ( x, y)) |( x, y) ∈ X × Y , μR ( x, y) ∈ [0, 1]}
R
~ ~
(12.18)
where μR ( x, y ) is a function in two variables, called membership function of the fuzzy relation. It gives the
~
degree of membership of the ordered pair (x, y) in R ~ , associating with each pair (x, y) in X ´ Y, a real number
in the interval [0, 1]. The degree of membership indicates the degree to which x is in relation with y. It is
clear that a fuzzy relation is basically a fuzzy set.

Example 12.5 Consider an example of fuzzy sets: the set of people with normal weight. In this case, the
universe of discourse appears to be all potential weights (the real line). However, the knowledge representa-
tion in terms of this universe, is not useful. The normal weight of a person is a function of his/her height.
Weight. kg
Body Mass Index (BMI) =
(Height, m) 2
Normal BMI for males is 20–25, and for females is 19–24. Values between 25 to 27 in men and 24 to 27 in
women indicate overweight; and those over 27 indicate obesity. Of course, values below 20 for man and
below 19 for women indicate underweight.
The universe of discourse for this fuzzy set is more appropriately the Cartesian product of two universal
sets: X, the set of all potential heights, and Y, the set of all potential weights. The Cartesian product space
X ´ Y is a universal set which is a set of ordered pairs (x, y), for each x Î X and each y Î Y.
A subset of the Cartesian product X ´ Y, satisfying the knowledge attribute “normal weight” is a set of
(height, weight) pairs. This is called a relation R~ . The membership value for each element of R ~ depends on
 704 Digital Control and State Variable Methods
BMI. For men, a BMI of 27 and more could be given a membership value of 0, and a BMI of less than 18 could
also be given a membership value of 0; and membership value between 0 and 1 for BMI between 18 and 27.

Example 12.6 Because fuzzy relations, in general, are fuzzy sets, we can define the Cartesian product to
be a relation between two or more fuzzy sets. Let ~ A be a fuzzy set on universe X, and ~ B be a fuzzy set on
universe Y; then the Cartesian product between fuzzy sets ~A and ~ R , which is
B will result in a fuzzy relation ~
contained within the full Cartesian product space, or
A× B
~ Ì X ´Y
~ ~ =R (12.19a)
where the fuzzy relation ~ R has membership function
μR ( x, y) = μ A × B (x, y) = min [ μ A ( x), μ B ( y)] " x Î X, " y Î Y (12.19b)
~ ~ ~ ~ ~
Note that the min combination applies here because each element (x, y), in the Cartesian product, is
formed by taking both elements x, y together, not just the one or the other.
As an example of the Cartesian product of fuzzy sets, we consider premise quantification. Atomic fuzzy
propositions do not, usually, make a knowledge base in real-life situations. Many propositions connected by
logical connectives may be needed. A set of such compound propositions, connected by IF-THEN rules,
makes a knowledge base.
Consider two propositions defined by
p =D x is A
D y is B
~
q =
~
where A and B are the fuzzy sets:
~ ~
⎧
( )
~A = ⎨ x, μ~A ( x) x ∈ X ⎬
⎩
⎫
⎭
⎧
( )
~ = ⎨⎩ y, μ~B ( y) y ∈Y ⎬⎭
B
⎫

The meaning of the linguistic terms “x is ~ A ”, and “y is B ~” is quantified via the membership functions μ~A ( x)
and mB ( y ), respectively. Now, we seek to quantify the linguistic premise “x is A and y is ~ B ” of the rule:
~ ~
A B (x, y)
IF x is A and y is ~ B THEN z is C (12.20a)
~ ~
The main item to focus on is, how to quantify the
B logical and operation that combines the meaning of
two linguistic terms. As said earlier, there are actually
several ways (t-norms) to define this quantification.
In the following, we use min operation:
y μ premise ( x , y ) = min [ μ A ( x), μ B ( y)] "x Î X , "y Î Y
A ~ ~
(12.20b)
Does this quantification make sense? Notice
A B
that this way of quantifying the and operation in the
premise, indicates that you can be no more certain
about the conjunction of two statements, than you are
about the individual terms that make them up.
The conjunction operator (logical connective
and), implemented as Cartesian product, is described
in Fig. 12.14.
x
Fig. 12.14 The conjunction operator implemented μ premise ( x , y ) = μ ~A × ~B (x, y) = min [ μ ~A ( x), μ B~ ( y )]
as Cartesian product "x Î X , "y Î Y (12.20c)
 Fuzzy Logic Models 705

Example 12.7 We consider here quantification of “implication” operator via fuzzy logic. Consider the
implication statement
IF pressure is high THEN volume is small
The membership function of the fuzzy set ~
A = “big pressure”,
R|
1 ; x ³ 5
μ A ( x) = 1 - (5 - x )/4 ; 1 £ x £ 5
S|
~
T
0 ; otherwise
is shown in Fig. 12.15a. The membership function of the fuzzy set B = “small volume”,
~
R| 1 ; y £1
μ ( y ) = S1 - ( y - 1) /4 ; 1 £ y £ 5
B
~
|T 0 ; otherwise
is shown in Fig. 12.15b.

A B

1 1

0 0
1 5 x 1 5 y
(a) “x is big pressure” (b) “y is small volume”
Fig. 12.15

If p is a proposition of the form “x is A ” where A is a fuzzy set on the universe X, e.g., “big pressure”, and
~ ~
q is a proposition of the form “y is ~
B ” where ~ B is a fuzzy set on the universe Y, e.g., “small volume”, then one
encounters the following problem:
How does one define the membership function of the fuzzy implication ~ A®~ B?
There are different important classes of fuzzy implication operators based on t-norm and t-conorm. In
many practical applications, one uses Mamdani’s implication operator to model causal relationship
between fuzzy variables:
μ A→ B ( x, y) = min [ μ A ( x), μ B ( y)] "x Î X , "y Î Y (12.21)
~ ~ ~ ~
The fuzzy implication A ® B is a fuzzy relation in the Cartesian product space X ´Y.
~ ~
Note that Mamdani’s implication operator gives a relation which is symmetric with respect to ~ A and ~B.
This is not intuitively satisfying, because “implication” is not a commutative operation. In practice, how-
ever, the method provides good, robust results. The justification for the use of the min operator to represent
the implication, is that we can be no more certain about our consequent than our premise.
 706 Digital Control and State Variable Methods

12.3 FUZZY INFERENCE

Problems featuring uncertainty and ambiguity have been successfully addressed subconsciously by humans.
Humans can adapt to unfamiliar situations and they are able to gather information in an efficient manner and
discard irrelevant details. The information gathered need not be complete and precise, and could be general,
qualitative and vague, because humans can reason, infer and deduce new information and knowledge. They
can learn, perceive and improve their skills through experience.
How can humans reason about complex systems, when the complete description of such a system often
requires more detailed data than a human could ever hope to recognize simultaneously, and assimilate with
understanding? The answer is that humans have the capacity to reason approximately. In reasoning about a
complex system, humans reason approximately about its behaviour, thereby maintaining only a generic
understanding about the problem.
The fuzzy set theory has provided a mathematical strength to capture the uncertainties associated with
human congnitive processes, such as thinking and reasoning. Fuzzy logic provides an inference morphology
that enables approximate human reasoning capabilities to be applied to knowledge-based systems.
Fuzzy conditional, or fuzzy IF-THEN production rules are symbolically expressed as
N
l q
IF (premise i ) THEN (consequent i ) i = 1
Here N is the number of rules.
Two major types of fuzzy rules exist: Mamdani fuzzy rules, and Sugeno fuzzy rules.

12.3.1 Mamdani Fuzzy Rules

In Mamdani fuzzy rules, both the premises and the consequents are fuzzy propositions (atomic/compound).
Consider first the case of a rule with atomic propositions. For example:
“IF x is A THEN y is B ” (12.22a)
~ ~
If we let X be the premise universe of discourse, and Y the consequent universe of discourse, then the
relation between the premise ~ A and consequent ~ B can be described using fuzzy sets on the Cartesian product
space X ´ Y. Using Mamdani’s implication rule,
~=~
R A®~ B
μ R ( x, y) = μ A→ B ( x, y )
~ ~ ~
= min [ μ A ( x), μ B ( y )] "x Î X , "y Î Y (12.22b)
~ ~
When the rule premise or rule consequent are compound fuzzy propositions, then the membership
function, corresponding to each such compound proposition, is first determined. The above operation is
applied to represent IF-THEN relation. Quite often, in control applications, we come across logical connec-
tive and (conjunction operation on atomic propositions), which, as we have seen in Example 12.6, may be
implemented by Cartesian product.
The rules of inference in fuzzy logic govern the deduction of final conclusion from IF-THEN rules for
known inputs (Fig. 12.16). Consider the statements:
rule : IF x is ~A THEN y is ~
B
input : x is A ¢ (12.23)
~
inference : B¢
y is ~
Here the propositions “x is ~ A ¢ ”, “y is ~
A ”, “x is ~ B ¢” are characterized by fuzzy sets ~
B ” and “y is ~ A¢, ~
A, ~ B,
and B ¢, respectively.
~
 Fuzzy Logic Models 707

Input IF-THEN Inference

~A = {( x, μ~A ( x )) | x ∈ X ; μ~A ∈ [0, 1]}
Relation {( x, μ A′ ( x)) | x ∈ X ; μ A′ ∈ [0, 1]}
A R B ~A ¢ = ~ ~
(12.24)
B= {( y, μ B ( y)) | y ∈ Y ; μ B ∈ [0, 1]}
~ ~ ~
Fig. 12.16 Inference mechanism
B ¢ = {( y, μ ~B ′ ( y )) | y ∈ Y ; μ ~B ′ ∈ [0, 1]}
~
Fuzzy sets ~ A ¢ are close but not equal, and same is valid for the sets ~
A and ~ B and ~ B ¢.
Inference mechanism is based on matching of two fuzzy sets A ¢ and R , and determining membership
~ ~
function of ~B ¢ according to the result. Note that X denotes the space in which the input ~ A ¢ is defined, and it
is subspace of the space X ´ Y in which the rule-base relation R ~ is defined. It is, therefore, not possible to
take the intersection of ~A ¢ and R~ ; an operation required for matching the two sets, to incorporate the knowl-
edge of the membership functions of both the input and the rule base. But when ~ A ¢ is extended to X ´ Y, this
is possible.
Cylindrical extension of ~ A ¢ (a fuzzy set defined on X ) on X ´ Y is the set of all tuples (x, y) Î X ´ Y, with
membership degree equal to μ A¢ ( x), i.e.,
~
μce( A′) ( x, y ) = μ A′ ( x) for every y Î Y (12.25)
~ ~
Now, the intersection operation, to incorporate the knowledge of membership functions of input and rule
base, is possible. It is given by
ce(~A′) ∩ R
~
In terms of membership functions, this operation may be expressed as
μ ce( A′) ( x, y ) Ù μ R ( x, y) = min[ μce( ~A′) (x, y), μ R (x, y)] ∀x ∈ X , ∀y ∈ Y
~ ~ ~
μ R ( x, y) = μ A→ B ( x, y ) = min[ μ ~A (x), μ B~ (y)]
~ ~ ~
μ ce( A′) ( x, y ) = μ A′ ( x)
~ ~
Therefore
μ S (x, y) = mce( A¢) (x, y) Ù μ R (x, y) = min( μ A¢ (x, y), min( μ A (x), μ B (y))) (12.26)
~ ~ ~ ~ ~ ~
By projecting this matched fuzzy set (defined on X ´ Y) over the inference subspace Y, we can determine
the membership function m B ¢ (y) of the fuzzy set B ¢ (defined on Y).
~ ~
Projection of μ~S (x, y) (a fuzzy set defined on X ´ Y ) on Y, is a set of all y Î Y with membership grades
equal to max{μS ( x, y)}; max means maximum with respect to x while y is considered fixed, i.e.,
x ~ x
μ proj ( S ) (y) = max{μ S ( x, y )} (12.27)
~ x ~
Projection on Y means that yi is assigned the highest membership degree from the tuples (x1, yi), (x2, yi),
(x3, yi), ..., where x1, x2, x2, ... Î X and yi Î Y. The rationale for using the max operation on the membership
functions of S should be clear in view of the fact that we have a many-to-one mapping.
~
The combination of fuzzy sets with the aid of cylindrical extension and projection, is called composition.
It is denoted by ° .
If A ¢ is a fuzzy set defined on X and R is a fuzzy relation defined on X ´ Y, then the composition of A ¢
~ ~ ~
and R resulting in a fuzzy set B ¢ defined on Y is given by
~ ~
A′ ° R
B¢ = ~ = proj (ce ( A′) ∩ R ) on Y (12.28)
~ ~ ~ ~
 708 Digital Control and State Variable Methods

Note that, in general, intersection is given by a t-norm, and projection by a t-conorm, resulting in many
definitions of composition operator. In our applications, we will mostly use min operator for t-norm and
max operator for t-conorm. Therefore, we have the following compositional rule of inference:
μ B′ (y) = max{min(μ A′ ( x), min(μ A ( x), μB ( y )))} (12.29)
~ x ~ ~ ~
This inference rule, based on max-min composition, uses Mamdani’s rule for implication operator.
A ¢ is fuzzy singleton, i.e.,
In control applications, as we shall see later, the fuzzy set ~

μ A′ (x) = RS1 for x = x 0 Î X

(12.30)
~ for all other x Î X
T0
This results in a simple inference procedure, as is seen below.
min( μ A ( x ), μ B ( y )) for x = x0 , ∀y ∈ Y
μ B′ (y) = ⎧⎪ ~ ~ (12.31)
~ ⎨
⎪⎩ 0 for all other x, ∀y ∈Y
Graphical representation of the procedure is shown in Fig. 12.17.

A (x)

B ( y)

B ( y)
A (x0)

y
x0 x
Fig. 12.17 Inference procedure for singleton fuzzy system

When the rule (12.20a) has a compound proposition in premise part, connected by logical connectives, then
m A in (12.31) is replaced by μ premise . For rules with AND’ed premise, one might use min or product t-norm
~
for calculating μ premise (we have used min in Eqn. (12.20b)); and for rules with OR’ed premise, we may use
max t-conorm for the calculation of μ premise. Of course, other t-norms and t-conorms are also premissible.
Singleton fuzzy system is most widely used because of its simplicity and lower computational require-
ments. However, this kind of fuzzy system may not always be adequate, especially in cases where noise is
present in the data. Nonsingleton fuzzy system becomes necessary to account for uncertainty in the data.

12.3.2 Sugeno Fuzzy Rules

Unlike Mamdani fuzzy rules, Sugeno rules are functions of input variables on the rule consequent. A typical
rule, with two input variables and one output variable, is of the form:
IF x1 is A and x2 is B THEN y = f (x1, x2) (12.32a)
~ ~
where f (×) is a real function.
In theory, f (×) can be any real function, linear or nonlinear. It seems to be appealing to use nonlinear
functions; rules are more general and can potentially be more powerful. Unfortunately, the idea is impracti-
cal; properly choosing or determining the mathematical formalism of nonlinear functions for every fuzzy
 Fuzzy Logic Models 709
rule in the rule base, is extremely difficult, if not impossible. For this reason, linear functions have been
employed exclusively in theoretical research, and practical development, of Sugeno fuzzy models. For a
system with two input variables and one output variable, ith rule in the rule base is of the form:
IF x1 is A (i) and x2 is ~
B (i) THEN y(i) = ai,0 + ai,1 x1 + ai,2 x2 (12.32b)
~
where the ai, j are real numbers.
We can view the Sugeno fuzzy system as a nonlinear interpolator between the linear mappings that are
defined by the functions in the consequents of the rules. It is important to note that a Sugeno fuzzy system
may have any linear mapping as its output function which contributes to its generality. One mapping that has
proven to be particularly useful, is to have a linear dynamic system as the output function so that the ith rule
(12.32b) takes the form:
IF x1 is A (i) and x2 is B (i) THEN x& (t) = Ai x(t) + biu(t); i = 1, 2, ..., R (12.32c)
~ ~
(i) (i) th
where A and B are the fuzzy sets of the i rule, and Ai and bi are state and input matrices (of appropriate
~ ~
dimensions) of the local description of the linear dynamic system. Such a fuzzy system can be thought of as
a nonlinear interpolator between R linear systems. The premise membership functions for each rule quantify
whether the linear system in the consequent is valid for a specific region on the state space. As the state
evolves, different rules turn on, indicating that other combinations of linear models should be used. Overall,
we find that the Sugeno fuzzy system provides a very intuitive representation of a nonlinear system as a
nonlinear interpolation between R linear models [145].
We will limit our discussion to the more widely used controllers—the Mamdani type singleton fuzzy
logic systems.

12.4 DESIGNING A FUZZY LOGIC CONTROLLER (MAMDANI ARCHITECTURE)

Figure 12.18 shows the basic configuration of a fuzzy logic controller (FLC), which comprises four princi-
pal components: a rule base, a decision-making logic, an input fuzzification interface, and an output defuzzi-
fication interface. The rule base holds a set of IF-THEN rules, that quantify the knowledge that human
experts have amassed about solving particular problems. It acts as a resource to the decision-making logic,
which makes successive decisions about which rules are most relevant to the current situation, and applies
the actions indicated by these rules. The input fuzzifier takes the crisp numeric inputs and, as its name
implies, converts them into the fuzzy form needed by the decision-making logic. At the output, the
defuzzification interface combines the conclusions reached by the decision-making logic and converts them
into crisp numeric values as control actions.
We will illustrate the FLC methodology, step by step, on a water-heating system.
Consider a simple water-heating system shown in Fig. 12.19. The water heater has a knob (HeatKnob) to
control the steam for circulation through the radiator. The higher the setting of the HeatKnob, the hotter the
water gets, with the value of ‘0’ indicating the steam supply is turned off, and the value of ‘10’ indicating the
maximum possible steam supply. There is a sensor (TempSense) in the outlet pipe to tell us the temperature
of the outflowing water, which varies from 0ºC to 125ºC. Another sensor (LevelSense) tells us the level of
the water in the tank, which varies from 0 (= empty) to 10 (= full). We assume that there is an automatic flow
control that determines how much cold water flows into the tank from the main water supply; whenever the
level of the tank gets below 4, the flow control turns ON, and turns OFF when the level of the water gets
above 9.5.
710 Digital Control and State Variable Methods
Control commands

F D
U Decision- E
making Control Process
Z F
Logic actions state/output
Z U
I Z
F Z
I I Process
C F
A I
T C
I A
O T
N Rule I
Base O
N

Fuzzy logic controller

Measurements
Sensors

Fig. 12.18 A simple fuzzy logic control system block diagram

10
HeatKnob
0 Water inlet

Steam 10
boiler
LevelSense

Radiator
0

0°C 125°C
TempSense
Steam
exhaust
Water outlet

Fig. 12.19 Water heating system

 Fuzzy Logic Models 711
Figure 12.20 shows a FLC diagram for the water-heating system.
The design objective can be stated as:
Keep the water temperature as close to 80ºC as possible, inspite of changes in the hot water flowing out
of the tank, and the cold water flowing into the tank.

Fig. 12.20 Fuzzy control of a water heater

Step One: Define Inputs and Outputs for the FLC Three fuzzy variables characterize the behaviour
of the water-heating system.
Input Variables: TempSense and LevelSense
Output Variable: HeatKnob
For x = outlet water temperature (linguistic variable TempSense), the universe of discourse is
X = [0ºC,125ºC]
For y = level of water in the tank (linguistic variable LevelSense), the universe of discourse is
Y = [0,10]
For z = HeatKnob setting (linguistic variable HeatKnob), the universe of discourse is
Z = [0,10]

Step Two: Define Frames for Fuzzy Variables The frame of TempSense is

TempSense, LTempSense, X , μLTempSense

~ ~ ~

where LTempSense
~ is the set of linguistic values that TempSense can take. We may use the following fuzzy
subsets to describe the linguistic values:
XSmall (XS); Small (S); Medium (M); Large (L); XLarge (XL)
i.e.,
LTempSense = {XSmall, Small, Medium, Large, XLarge}
~
 712 Digital Control and State Variable Methods
The frame of LevelSense is

LevelSense , LLevelSense, Y , μ LLevelSense

~ ~ ~

LLevelSense = {XSmall, Small, Medium, Large, XLarge}

In our system, we have just one output which is the HeatKnob. We take the following frame for this
linguistic variable.

HeatKnob, LHeatKnob, Z , μ LHeatKnob

~ ~ ~
LHeatKnob = {VeryLittle, ALittle, AGoodAmount, ALot, AWholeLot}
~
Step Three: Assign Membership Values to Fuzzy Variables Since the membership function essen-
tially embodies all fuzziness for a particular fuzzy set, its description is the essence of a fuzzy property or
operation. Because of the importance of the “shape” of the membership function, a great deal of attention
has been focussed on development of these functions. Many ways to develop membership functions, i.e.,
to assign membership values to fuzzy variables, have been reported in the literature—methods based on
Inference, Neural Networks, Genetic Algorithms, Inductive Reasoning, etc. The assignment process can be
intuitive, or it can be based on some algorithmic or logical operations. We shall rely on intuition in our
application examples.
The input variables TempSense and LevelSense,
as well as the output variable HeatKnob, are
Table 12.1 Fuzzy variable ranges for TempSense
restricted to positive values. In Table 12.1 and
Fig. 12.21, we show a possible assignment for
Crisp Input Range Fuzzy Variable
ranges and triangular membership functions for
0–20 XSmall TempSense. Similarly, we assign ranges and fuzzy
10–35 Small membership functions for LevelSense in Table 12.2
30–75 Medium and Fig. 12.22; and HeatKnob in Table 12.3 and
60–95 Large Fig. 12.23. The optimization of these assignments is
85–125 XLarge often done through trial and error for achieving
optimum performance of FLC.

Table 12.2 Fuzzy variable ranges for LevelSense Table 12.3 Fuzzy variable ranges for HeatKnob

Crisp Input Range Fuzzy Variable Crisp Input Range Fuzzy Variable
0–2 XSmall 0–2 VeryLittle
1.5–4 Small 1.5–4 ALittle
3–7 Medium 3–7 AGoodAmount
6–8.5 Large 6–8.5 ALot
7.5–10 XLarge 7.5–10 AWholeLot

The following guidelines were kept in mind while determining range of fuzzy variables as related to the
crisp inputs.
(i) Symmetrically distribute the fuzzified values across the universe of discourse.
(ii) Use an odd number of fuzzy sets for each variable so that some set is assured to be in the middle. The
use of 5 to 7 sets is fairly typical.
(iii) Overlap adjacent sets (by 15% to 25%, typically).
 Fuzzy Logic Models 713

Fig. 12.21 Fuzzy membership functions for TempSense

Fig. 12.22 Fuzzy membership functions for LevelSense

Fig. 12.23 Fuzzy membership functions for the output HeatKnob

Step Four: Create a Rule Base Now that we have the inputs and the outputs in terms of fuzzy variables,
we need only specify what actions to take, under what conditions; i.e., we need to construct a set of rules that
describe the operation of the FLC. These rules usually take the form of IF-THEN rules, and can be obtained
from a human expert (heuristics).
 714 Digital Control and State Variable Methods

The rule-base matrix for our example is given in Table 12.4. Our heuristic guidelines, in determining this
matrix, are the following:
(i) When the temperature is low, the HeatKnob should be set higher than when the temperature is high.
(ii) When the volume of water is low, the HeatKnob does not need to be as high as when the volume of
water is high.

Table 12.4 Decision table

TempSense ® XS S M L XL
LevelSense
B
XS AGoodAmount ALittle VeryLittle
S ALot AGoodAmount VeryLittle VeryLittle
M AWholeLot ALot AGoodAmount VeryLittle
L AWholeLot ALot ALot ALittle
XL AWholeLot ALot ALot AGoodAmount

In FLCs we do not need to specify all the cells in the matrix. No entry signifies that no action is taken. For
example, in the column for TempSense = XLarge, no action is required since the temperature is already at or
above the target temperature.
Let us examine a couple of typical entries in the table: For LevelSense = Small, and TempSense = XSmall,
the output is HeatKnob = ALot. Now for the same temperature, as the water level rises, the setting on
HeatKnob should also rise—to compensate for the added volume of water. We can see that for LevelSense =
Large and TempSense = XSmall, the output HeatKnob = AWholeLot.
We can translate the table entries into IF-THEN rules. We give here a couple of rules.
· IF TempSense is Small and LevelSense is Small THEN set HeatKnob to ALot.
· IF TempSense is XSmall and LevelSense is Large THEN set HeatKnob to AWholeLot.
Step Five: Choose Scaling Gains for the Variables Using standard ideas from control engineering,
we have introduced gains Gi1 and Gi2 with the input variables, as shown in Fig. 12.20, and at the same time
we also put a gain Go between FLC and the plant. Change in the scaling gains, at the input and output of
FLC, can have a significant impact on the performance of the resulting control system, and hence they are
often convenient parameters for tuning.
First, let us consider the effect of input scaling gain Gi1. Note that we can actually achieve the same effect
as scaling via Gi1, by simply changing the labelling of the temperature axis for the membership function of
the input variable TempSense. The case when Gi1 = 1 corresponds to our original choice of the membership
functions in Fig. 12.21. The choice of Gi1 = 0.5 as the scaling gain for the FLC with these membership
functions, is equivalent to having the membership functions shown in Fig. 12.24 with Gi1 = 1. Thus, the
choice of a scaling gain Gi1 results in scaling the horizontal axis of the membership functions by 1/Gi1
(multiplication of each number on the horizontal axis of Fig. 12.21 by 1/0.5 produces Fig. 12.24; member-
ship functions are uniformly “spread out” by a factor of 1/0.5). Similar statements can be made about Gi2
(Fig. 12.25).
 Fuzzy Logic Models 715

Fig. 12.24 Scaled membership functions for TempSense

Fig. 12.25 Scaled membership functions for LevelSense

Figure 12.23 shows our choice of output membership functions with Go = 1. There is a proportional effect
between the scaling gain Go and the output membership functions as shown in Fig. 12.26 for Go = 2.
If, for the process under consideration, the effective universes of discourse for all inputs and output are
common, say, [0, 1], then we may say that the FLC is normalized. Clearly, scaling gains can be used to
normalize the given FLC. Denormalization of the output of such a FLC will yield the required control action.

Fig. 12.26 The effect of scaling gain Go on the spacing of the output membership functions
 716 Digital Control and State Variable Methods
It is important to realize that the scaling gains are not the only parameters that can be tuned to improve the
performance of the fuzzy control system. Membership function shapes, positioning, and number and type of
rules are often the other parameters to tune.
We set Gi1 = Gi2 = Go = 1 for our design problem.
Step Six: Fuzzify Inputs to the FLC Fuzzy sets are used to quantify information in the rule base, and
the inference mechanism operates on fuzzy sets and produces fuzzy sets. Inputs to the FLC are the measured
output variables of the controlled process, which are crisp variables. And input to the controlled process
(control action) is required to be crisp. Therefore, we must specify how the fuzzy system will convert the
numeric (crisp) inputs to the FLC into fuzzy sets (a process called “fuzzification”). Also we must specify
how the fuzzy system will convert the fuzzy sets produced by inference mechanism into numeric (crisp)
FLC output (a process called “defuzzification”), which is the input to the controlled process.
Fuzzification can be defined as a mapping from an observed input space to fuzzy sets in a specified
universe of discourse. A natural and simple fuzzification approach is to convert a crisp measurement into a
fuzzy singleton within the specified universe of discourse. This approach is based on the assumption that the
observed data is crisp, and not corrupted by random noise.
To understand fuzzification, we consider an example. Assume that at a particular point in time,
LevelSense = 6.5 and TempSense = 65ºC. These are the crisp inputs directly from the sensors. Figures 12.21
and 12.22 show the membership functions for the input variables and indicate with vertical lines the meas-
ured values of LevelSense and TempSense. These vertical lines are, in fact, graphical representation of the
two fuzzy singletons obtained by the fuzzification process.
Step Seven: Determine which Rules Fire We see that with singleton fuzzification, combining the
fuzzy sets that were created by the fuzzification process to represent the inputs with the premise membership
functions for the rules, is particularly simple. It simply reduces to computing the membership values of the
input fuzzy sets for the given inputs.
From Fig. 12.21 we find that for input TempSense = 65ºC, μM (65) = 0.45, μ L (65) = 0.28, and all other
~ ~
membership functions are off (i.e., their values are zero). Therefore, the proposition “TempSense is
Medium” is satisfied to a degree of 0.45, the proposition “TempSense is Large” is satisfied to a degree of
0.28; all other atomic propositions associated with TempSense are not satisfied.
From Fig. 12.22 we find that for input LevelSense = 6.5, μM (6.5) = 0.25, μ L (6.5) = 0.38; all other
~ ~
membership functions are off.
We next form membership values of premises of all the rules. From the induced decision table (Table 12.5),
we observe that the rules that have the premise terms:

Table 12.5 Induced Decision table

TempSense® μ XS = 0 μS = 0 μ M = 0.45 μ L = 0.28 μ XL = 0

~ ~ ~ ~ ~
LevelSense
B
μ XS = 0 0 0 0 0 0
~
μS = 0 0 0 0 0 0
~
μ M = 0.25 0 0 AGoodAmount VeryLittle 0
~
μ L = 0.38 0 0 ALot ALittle 0
~
μ XL = 0 0 0 0 0 0
~
 Fuzzy Logic Models 717
(i) TempSense is Medium and LevelSense is Medium
(ii) TempSense is Large and LevelSense is Medium
(iii) TempSense is Medium and LevelSense is Large
(iv) TempSense is Large and LevelSense is Large
have m premise > 0. For all other rules, m premise = 0.
Determining applicability of each rule is called “firing”. We say that, a rule fires at time t if its premise
membership value at time t is greater than zero. The inference mechanism seeks to determine which rules
fire, to find out which rules are relevant to the current situation. The inference mechanism combines the
recommendations of all the rules, to come up with a single conclusion.
For crisp input TempSense = 65ºC, and LevelSense = 6.5, four rules fire. m premise for the four rules (refer
Table 12.5), which amounts to firing strength in each case, can be calculated as follows.
(i) μTempSense × LevelSense = min(0.45,0.25) = 0.25
~ ~
(ii) μTempSense × LevelSense = min(0.28,0.25) = 0.25
~ ~
(iii) μTempSense × LevelSense = min(0.45,0.38) = 0.38
~ ~
(iv) μTempSense × LevelSense = min(0.28,0.38) = 0.28
~ ~
Step Eight: Infer the Output Recommended by Each Rule From the induced decision table (Table 12.5),
we observe that only four cells contain nonzero terms. Let us call these cells active. The active cells corre-
spond to the following rules.
(i) TempSense is Medium and LevelSense is Medium : p1
Set HeatKnob to AGoodAmount : q1
IF p1 THEN q1
μ premise(1) = 0.25
μ inference(1) is obtained by “chopping off” the top of μ AGoodAmount function of the output variable
~
HeatKnob, as shown in Fig. 12.27a.
(ii) TempSense is Large and LevelSense is Medium : p2
Set HeatKnob to VeryLittle : q2
IF p2 THEN q2
μ premise( 2) = 0.25
μ inference(2) is shown in Fig. 12.27b.
(iii) TempSense is Medium and LevelSense is Large : p3
Set HeatKnob to ALot : q3
IF p3 THEN q3
μ premise( 3) = 0.38
μ inference( 3) is shown in Fig. 12.27c.
(iv) TempSense is Large and LevelSense is Large : p4
Set HeatKnob to ALittle : q4
IF p4 THEN q4
μ premise( 4) = 0.28
μ inference(4) is shown in Fig. 12.27d.
 718 Digital Control and State Variable Methods

Fig. 12.27 Inference for each rule

The reader should note that for different crisp measurements TempSense and LevelSense, there will be
different values of μ premise and, hence, different μ inference functions will be obtained.
Step Nine: Aggregate the Fuzzy Outputs Recommended by Each Rule In the previous step, we
noticed that the input to the inference process is the set of rules that fire; its output is the set of fuzzy sets that
represent the inference reached by all the rules that fire. We now combine all the recommendations of all the
rules to determine the control action. This is done by aggregating the inferred fuzzy sets. Aggregated fuzzy
set, obtained by drawing all the inferred fuzzy sets on one axis, is shown in Fig. 12.28. This fuzzy set
represents the desired control action.

agg (z)
z* = 4.66
1

z
1 2 3 4 5 6 7 8 9 10

Fig. 12.28 Aggregated fuzzy set

Step Ten: Defuzzify the Aggregated Fuzzy Set to form Crisp Output from the FLC Defuzzification
is a mapping from a space of fuzzy control actions defined by fuzzy sets on an output universe of discourse, into
nonfuzzy (crisp) control actions. This process is necessary because crisp control action is required to actuate
the control.
There are many approaches to defuzzification. We will consider here the “Centre of Area” (COA)
method, which is known to yield superior results.
 Fuzzy Logic Models 719
We may discretize the universe Z into q equal (or almost equal) subintervals by the points z1, z2, ..., zq – 1.
The crisp value z*, according to this method is
q −1
∑ zk μagg ( zk )
z* = k =1 (12.33)
q −1
∑ μagg ( zk )
k =1
From Fig. 12.28, we obtain
S zk magg = 1 × 0.25 + 1.5 × 0.25 + 2 × 0.28 + 3 × 0.28 + 4 × 0.25
+ 5 × 0.25 + 6 × 0.25 + 7 × 0.38 + 8 × 0.38 = 11.475
S magg = 0.25 + 0.25 + 0.28 + 0.28 + 0.25 + 0.25 + 0.25 + 0.38 + 0.38 = 2.57
Therefore
11475
.
z* = = 4.46
2.57
The physical interpretation of z* is that, if the area is cut of a thin piece of metal or wood, the centre of the
area will be the centre of gravity.
In fact, there is hardly any need of discretization of the universe for situations like the one shown in
Fig. 12.28; we can split up geometry into pieces and place a straight edge (centroid) through the figure to
have it perfectly balanced with equal area of the figure on either side. Analytical expression for z* is

z* = ∫z μagg ( z ) zdz (12.34)

∫z μagg ( z )dz
This completes the design for the simple example we chose.

12.5 DATA BASED FUZZY MODELING (SUGENO ARCHITECTURE)

The Mamdani architecture is widely used for capturing expert knowledge. It allows us to describe the exper-
tise in more intuitive, more human-like manner. On the other hand, the Sugeno architecture is by far the most
popular candidate for data-based fuzzy modelling.
Basically, a fuzzy model is a “Fuzzy Inference System (FIS)” composed of four principal components: a
fuzzification interface, a knowledge base, a decision-making logic, and a defuzzification interface. Figure 12.29
shows the basic configuration of FIS for data-based modelling.

Knowledge Base

Data Base Rule Base

Non-fuzzy Fuzzification Defuzzification Non-fuzzy

Input Output

Decision-making
Fuzzy logic Fuzzy
Fig. 12.29 Basic configuration of a fuzzy inference system
 720 Digital Control and State Variable Methods

We consider here, a single-output FIS in the n-dimensional input space. Let us assume that the following
P input-output pairs are given as training data for constructing FIS model:
{x( p) , y( p ) p = 1, 2, K, P} (12.35)
T
where x = ⎡⎣ x1 x2 L xn ⎤⎦ is the input vector of pth input-output pair and y( p) is the corresponding
( p) ( p) ( p) ( p)

output.
The fuzzification interface performs a mapping that converts crisp values of input variables into fuzzy
singletons. A singleton is a fuzzy set with a membership function that is unity at a single particular point on
the universe of discourse (the numerical-data value), and zero everywhere else. Basically, a fuzzy singleton
is a precise value and hence no fuzziness is introduced by fuzzification in this case. This strategy, however,
has been widely used in fuzzy modelling applications because it is easily implemented.
There are two factors that determine a database (i) a fuzzy partition of an input space, and (ii) member-
ship functions of antecedent fuzzy sets. Assume that the domain interval of the ith input variable xi, is equally
divided into Ki fuzzy sets labeled A i1, A i2, ..., AiKi , for i = 1, 2, ..., n. Then the n-dimensional input space is
~ ~ ~
divided into K1 × K 2 × L × K n fuzzy partition spaces:
( A1 j1 , A2 j2 , K , Anjn ); j1 = 1, 2, ..., K1; ...; jn = 1, ..., Kn (12.36)
~ ~ ~
Though any type of membership functions (e.g., triangle-shaped, trapezoid-shaped, bell shaped, etc.) can
be used for fuzzy sets, we employ the symmetric triangle-shaped fuzzy sets, Ai ji, with the following member-
~
ship functions:
2 xi − c(i , ji )
m A i j ( xi ) @ μ i j ( xi ) = 1 − ; ji = 1, 2,K, Ki (12.37)
~ i
w( i, ji )
i ji(xi)
c(i , ji ) is the centre of the membership function, where the member-
1
ship grade is equal to 1, and w( i , ji ) denotes the width of the mem-
bership function (Fig.12.30).
By means of the input-output data, the range ⎡⎣ ximin , ximax ⎤⎦ of the
ith input variable is determined, where
c(i, ji) xi
w(i, ji) ximin = min xi( p ) , ximax = max xi( p ) (12.38a)
p ={1,KP} p ={1,KP}
Fig. 12.30 Parameters of a member- The centre position of each membership function with respect to
ship function the ith variable is determined by
c(i , ji ) = ximin + ( ji − 1) ⎡ ( ximax − ximin ) /( K i − 1) ⎤ ; c(i ,1) = ximin ; c(i ,K i ) = ximax (12.38b)
⎣ ⎦
To achieve sufficient overlap from one linguistic label to another, we take
w(i , ji ) = 2(c(i , ji +1) − c(i , ji ) ) (12.38c)
Figure 12.31 shows an example where the domain interval of x1 is divided into K1 = 5 fuzzy sets.
The rule base consists of a set of fuzzy IF-THEN rules in the form “IF a set of conditions are satisfied
THEN a set of consequences can be inferred”. Different types of consequent parts have been used in fuzzy
rules; the two commonly used fuzzy models are based on Mamdani’s approach and Sugeno’s approach. We
restrict our discussion to Sugeno architecture: the domain interval of y is represented by R linear functions,
giving rise to R fuzzy rules. All the rules corresponding to the possible combinations of the inputs are
implemented. The total number of rules R for an n-input system is : K1 ´ K2 ´ L ´ Kn.
 Fuzzy Logic Models 721

ij1(x1)
1 ~A11 ~A12 ~A13 ~A14 ~A15

0
ximin ximax
Fig. 12.31 Fuzzy partition of an input space and membership functions of fuzzy sets

The format of fuzzy rules is,

Rule r: IF x1 is A1 j and L and xn is Anjn THEN
~ 1 ~
yˆ (r)
= a0( r ) + a1( r ) x1 + K + an( r ) xn ; r = 1, 2, K , R (12.39)
The consequent part is a linear function of the input variable xi; a0, a1, ..., an are the (n + 1) parameters that
determine the real consequent value. The aim of the linear function is to describe the local linear behaviour
of the system. Each rule r gives rise to a local linear model. The selected R rules are required to approximate
the function that theoretically underlines the system behaviour most consistently, with the given sample of
input–output data (12.35) (When y$ is a constant in (12.39), we get a Sugeno model in which the consequent
of a rule is specified by a singleton).
The decision-making logic employs fuzzy IF-THEN rules from the rule base to infer the output by a fuzzy
reasoning method. The contribution of each local linear model (i.e., each rule) in the estimated output of the
FIS is dictated by the firing strength of the rule. We use product strategy to assign firing strength m (r) to each
rule r = 1, 2, …, R.

Given an input vector, x( p ) = ⎡ x1( p ) , x2( p ) ,K , xnp ⎤ , the degree of compatibility of x( p) to the rth fuzzy
⎣ ⎦
IF-THEN rule is the firing strength m (r) of the rule, and is given by (note that we have used product t-norm
operator on the premise part of the rule)
r p p p p
μ ( ) ( x ( ) ) = μ1 j1 ( x1( ) ) × μ 2 j2 ( x2( ) ) × L × μ njn ( xn( ) )
p (12.40)
= ∏ μ i ji ( xi( ) )
( i , ji )∈I r

where Ir is the set of all ~Ai ji associated with the premise part of rule r.
The main idea of the Sugeno architecture is that in each input fuzzy region A1 j1 × A2 j2 × L × Anjn of the
~ ~ ~
input domain, a local linear system is formed. The membership function μ ( r ) (x( p) ) of each region is a map
indicating the degree of the output of the associated linear system to the region. A simple defuzzification
procedure is to take the output of the system as the “fuzzy” combination of the outputs of local systems in all
regions:
R
∑ (a0(r ) + a1(r ) x1 + L + an(r ) xn ) μ (r )
y$ = r =1 (12.41a)
R
∑ μ (r )
r =1
 722 Digital Control and State Variable Methods
R
= ∑ (a0(r ) + a1(r ) x1 + L + an(r ) xn ) μ (r ) (12.41b)
r =1

μ (r )
where μ (r) = R
(12.41c)
∑μ (r)

r =1
is the normalized firing strength of rule r; a ratio of firing strength of rule r, to the sum of the firing strengths
of all the rules.
Note that the output of the fuzzy model can be determined only if the parameters in rule consequents are
known. However, it is often difficult, or even impossible, to specify a rule consequent in a polynomial form.
Fortunately, it is not necessary to have any prior knowledge of rule consequent parameters for the Sugeno
fuzzy modelling approach to deal with a problem. These parameters can be determined using least- squares
estimation method as follows.
Given the values of the membership parameters and a training set of P input-output patterns {x( p ) , y ( p ) ;
p = 1, 2,K, P}, we can form P linear equations in terms of the consequent parameters.
y ( p ) = μ (1) ( x( p ) ) ⎡⎣ a0(1) + a1(1) x1( p ) + L + an(1) xn( p ) ⎤⎦ + μ (2) ( x ( p ) ) ⎡⎣ a0(2) + a1(2) x1( p ) + L + an(2) xn( p ) ⎤⎦ + L

+ μ ( R ) (x( p ) ) ⎡ a0( R ) + a1( R) x1( p) + L + an( R ) xn( p ) ⎤ ; p = 1, 2,L , P (12.42)

⎣ ⎦
where μ (r ) ( x( p ) ) is the normalized firing strength of rule r, fired by the input pattern x( p).
In terms of vectors
T
x ( p ) = ⎡⎣1 x1( p ) x2( p ) L xn( p ) ⎤⎦

θ( r ) = ⎡⎣ a0( r ) a1( r ) L an( r ) ⎤⎦

Θ = ⎡⎣a0(1) a1(1) L an(1) a0(2) L an(2) L a0( R ) L an( R ) ⎤⎦ (12.43)

we can write the P linear equations as follows.
y (1) = μ (1) (x(1) ) ⎡⎣θ(1) x (1) ⎤⎦ + μ (2) (x(1) ) ⎡⎣θ(2) x (1) ⎤⎦ + L + μ ( R ) ( x (1) ) ⎡ θ ( R ) x (1) ⎤
⎣ ⎦
y (2) = μ (1) (x(2) ) ⎡⎣θ(1) x(2) ⎤⎦ + μ (2) (x(2) ) ⎡⎣θ(2) x (2) ⎤⎦ + L + μ ( R ) (x(2) ) ⎡θ( R) x (2) ⎤
⎣ ⎦
M
y ( P ) = μ (1) ( x ( P ) ) ⎡⎣ θ(1) x ( P ) ⎤⎦ + μ (2) ( x ( P ) ) ⎡⎣ θ(2) x ( P ) ⎤⎦ + L + μ ( R ) ( x ( P ) ) ⎡⎣ θ ( R ) x ( P ) ⎤⎦ (12.44)
These P equations can be rearranged into a single vector-matrix equation:
⎡ T T T ⎤
⎡ (1) ⎤ ⎢ ⎡ x (1) ⎤ μ (1) (x(1) ) ⎡ x (1) ⎤ μ (2) (x(1) )L ⎡ x (1) ⎤ μ ( R ) (x(1) ) ⎥ ⎡[θ( r ) ]T ⎤
⎢y ⎥ ⎢ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎥ ⎢ (r ) T ⎥
⎢ (2) ⎥ ⎢ ⎡ (2) ⎤T (1) (2) ⎢[θ ] ⎥
⎡ x (2) ⎤ μ ( R ) (x(2) ) ⎥ ⎢
T T
⎡ (2) ⎤ (2) (2)
⎢ y ⎥ = ⎢ ⎣ x ⎦ μ ( x ) ⎣ x ⎦ μ ( x )L ⎣ ⎦ ⎥ M ⎥
(12.45a)
⎢M ⎥ ⎢ ⎥⎢ ⎥
⎢ ⎥ ⎢ M ⎥ ⎢M ⎥
⎢ y ⎥ ⎢ ⎡ ( P ) ⎤T (1) ( p ) ⎡ ( P ) ⎤T (2) ( P )
(P) T ( R ) ( P) ⎥ ⎢ ⎥
⎣⎢ ⎦⎥ ⎢ ⎣ x ⎦ μ (x ) ⎣ x ⎦ μ (x )L ⎡ x ⎤ μ (x ) ⎥ ⎢[θ( r ) ]T ⎥
( P )
⎣ ⎦ ⎣ ⎦
⎢⎣ ⎥⎦
or y = XT ΘT (12.45b)
 Fuzzy Logic Models 723
In the Sugeno fuzzy model given above, we have used most intuitive approach of implementing all possi-
ble combinations of the given fuzzy sets as rules. In fact, if data is not uniformly distributed, some rules may
never be fired. This and other drawbacks are handled by many variants of the basic ANFIS model, described
in the next section.

12.6 SYSTEM IDENTIFICATION AND CONTROL WITH NEURO-FUZZY SYSTEMS

Fuzzy logic and neural networks are natural complementary tools in building intelligent systems. While
neural networks are computational structures that perform well when dealing with raw data, fuzzy logic
deals with reasoning, using linguistic information acquired from domain experts. However, fuzzy systems
lack the ability to learn and cannot adjust themselves to a new environment. On the other hand, although
neural networks can learn, they are opaque to the user. The merger of a neural network with a fuzzy system
into one integrated system, therefore, offers a promising approach to building intelligent systems. Integrated
systems can combine the parallel computation and learning abilities of neural networks, with the human-like
knowledge representation and explanation abilities of fuzzy systems. As a result, neural networks become
more transparent, while fuzzy systems become capable of learning.
The structure of a neuro-fuzzy system is similar to a multilayer neural network. In general, a neuro- fuzzy
system has input terminals, output layer, and hidden layers that represent membership functions and fuzzy rules.
Roger Jang [142] proposed an integrated system that is functionally equivalent to a Sugeno fuzzy infer-
ence model. He called it an Adaptive Neuro-Fuzzy Inference System or ANFIS. Similar network structures
have also been proposed for Mamdani fuzzy inference model [137]. However, the Sugeno model is by for
the most popular candidate for data-based fuzzy modelling. Our brief presentation of the subject is, there-
fore, focused on ANFIS based on Sugeno fuzzy model.
12.6.1 ANFIS Architecture
Figure 12.32 shows the ANFIS architecture. For simplicity, we assume that the ANFIS has two inputs, x1 and
x2, and one output y. Each input is represented by two fuzzy sets, and the output by a first-order polynomial.
The ANFIS implements four rules:
Rule 1: IF x1 is ~A11 and x2 is ~A21 THEN y(1) = a(1) (1)
0 + a1 x1 + a 2 x2
(1)

(2) (2)
A22 THEN y = a 0 + a1 x1 + a2( 2) x2
( 2 )
Rule 2: IF x1 is ~A12 and x2 is ~
(12.46)
Rule 3: IF x1 is ~A12 and x2 is ~A21 THEN y(3) = a(3) ( 3)
0 + a1 x1 + a 2 x2
( 3)

(4) (4) ( 4 ) ( 4 )
Rule 4: IF x1 is ~A11 and x2 is ~A22 THEN y = a 0 + a1 x1 + a2 x2
where ~A11 and ~A12 are fuzzy sets on the universe of discourse, of input variable x1, A21 and A2 are fuzzy sets
~ ~
on the universe of discourse of input variable x2; a0(r ) , a1( r ) and a2(r ) is a set of parameters specified for rule r.
Let us now discuss the purpose of each layer in ANFIS of Fig. 12.32.
x1 x2
Layer 1 Layer 2 Layer 3 Layer 4 Layer 5

~A11 N 1
x1
y^
~A12 N 2

~A21 N 3
x2

~A22 N 4

Fig. 12.32 An Adaptive Neuro-Fuzzy Inference System (ANFIS)

 724 Digital Control and State Variable Methods
Layer 1 The inputs to the nodes in the first layer are the input fuzzy sets of the ANFIS. Since these fuzzy
sets are fuzzy singletons, numerical inputs are directly transmitted to the first-layer nodes.
Nodes in this layer represent the membership functions associated with each linguistic term of input
variables. Every node here is an adaptive node. Links in this layer are fully connected between input termi-
nals and their corresponding membership function nodes. Membership functions can be any appropriate
parameterized function; we use Gaussian function.
⎡ ⎛ x −c 2⎤
⎢ i (i , ji ) ⎞ ⎥
Ai ji ( xi ) @ μ i j ( xi ) = exp − ⎜ ⎟ (12.47)
~ i ⎢ ⎜ w(i , j ) ⎟ ⎥
⎣ ⎝ i ⎠ ⎦
The nodes are labelled ~Ai ji; i = 1, 2; ji = 1, 2. Total number of nodes in this layer is, therefore, four. c(i , ji )
is the centre (mean) and w( i , ji ) is the width (variance), respectively, of the membership function correspond-
ing to the node ~Ai ji; xi is the input and i j is the output of the node. The adjusted weights in layer 1 are
c(i , ji )’s and w( i , ji ) ’s. As the values of these parameters change, the Gaussian function varies accordingly; thus
exhibiting various forms of membership functions of fuzzy set ~Ai ji. Parameters in this layer are referred to as
premise parameters.
Layer 2 Every node in this layer is a fixed node labelled P, whose output is the product of all the incoming
signals. Each node output represents firing strength of a rule. In fact, other t-norm operators could also be
used as node functions.
Each node, representing a single Sugeno fuzzy rule, has the output
r
μ ( ) ( x) @ ∏ μiji ( xi ) (12.48)
( i , ji ) ∈ I r

where Ir is the set of all ~Ai ji associated with the premise part of rule r.
Layer 3 Every node in this layer is a fixed node labelled N. The rth node calculates the ratio of the rth
rule’s firing strength, to the sum of all rules’ firing strengths:
μ(r)
μ (r) = R
= Normalized firing strength of rule r (12.49)
∑μ (r)

r =1

Layer 4 Every node is this layer is an adaptive node, is connected to the respective normalization node in
the previous layer, and also receives inputs x1 and x2. It calculates the weighted consequent value of a given
rule as
yˆ ( r ) = μ ( r ) ⎡⎣ a0 + a1 x1 + a2 x2 ⎤⎦
(r ) (r ) (r )
(12.50)

where μ (r ) is the normalized firing strength from layer 3, and a0( r ) , a1( r ) and a2( r ) are the parameters of this
node. Parameters in this layer are referred to as consequent parameters.
Each node in layer 4 is a local linear model of the Sugeno fuzzy system; integration of outputs of all local
linear models yields global output.
Layer 5 The single node in this layer is a fixed mode labelled ∑ , which computes the overall output as the
summation of all incoming signals:
R
yˆ = ∑ ( a0( r ) + a1( r ) x1 + a2( r ) ) μ ( r ) ; R = 4 (12.51)
r =1
 Fuzzy Logic Models 725
12.6.2 How does an ANFIS learn?
An ANFIS uses a hybrid learning algorithm that combines the least squares estimator and the gradient
descent method. First, initial activation functions are assigned to each membership neuron. The function
centres of the neurons connected to input xi, are set so that the domain of xi is divided equally, and the widths
are set to allow sufficient overlapping of the respective functions.
In an ANFIS training algorithm, each epoch is composed of a forward pass and a backward pass. In the
forward pass, a training set of input patterns (input vector x) is presented to the ANFIS, neurons outputs are
calculated on the layer-by-layer basis, and the rules consequent parameters are identified by the least squares
estimator. In the Sugeno fuzzy inference, an output y$ is a linear function. Thus, given the values of the
membership parameters and a training set of P input-output patterns, we can form P linear equations in terms
of the consequent parameters (refer Eqns (12.45)). Least- squares solution of these equations yields the
consequent parameters.
As soon as the rule consequent parameters are established, we can compute actual network output, y$ , and
determine the error
e = y − yˆ (12.52)
In the backward pass, the backpropagation algorithm in applied. The error signals are propagated back,
and the premise parameters are updated according to the chain rule.
The goal is to minimize the error function
E = 12 ( y − yˆ )2 (12.53)
The error at layer 5:
∂E
= ( y$ – y ) (12.54)
∂y$
Back propagating to layer 3 via layer 4 (refer Eqn. (12.51)),
∂E ∂E ∂yˆ ∂E ⎡ ( r )
= = (a + a1( r ) x1 + a2( r ) x2 )⎤
∂μ (r )
∂yˆ ∂μ ( r ) ∂yˆ ⎣ 0 ⎦ (12.55)

Back propagating to layer 2 (refer Eqn. (12.49)),

∂E ∂E ∂μ ( r ) ∂E ⎡ μ ( r ) (1− μ ( r ) ) ⎤
= = ⎢ ⎥ (12.56)
∂μ ( r ) ∂μ ( r ) ∂μ ( r ) ∂μ ( r ) ⎣ μ (r) ⎦
The error at layer 1:
Ir is the set of all ~Ai ji associated with the premise part of rule r. Reverse pass: I (i , ji ) is the set of all rule
nodes in layer 2 connected to (i, ji )th node (corresponding to ~Ai ji) of layer 1.
Back propagating error to layer 1 (refer Eqn. (12.48)),
r
∂E ∂E ∂μ ( )
= ∑
∂μi ji r∈I( i, ji ) ∂μ ( r ) ∂μi ji
(12.57a)
∂E ⎡ μ (r) ⎤
= ∑ ⎢ ⎥
⎢⎣ μi ji ⎥⎦
r
r∈I( i , j ) ∂μ ( )
i

From Eqn. (12.47), we obtain

∂μi ji
= 2μi ji ( xi − c(i , ji ) ) / w(2i , ji ) (12.57b)
∂c(i , ji )
 726 Digital Control and State Variable Methods

∂μi ji
= 2μi ji ( xi − c(i , ji ) ) 2 / w(3i , ji ) (12.57c)
∂w(i , ji )
Denoting the iteration index by k (refer Eqn. (11.39)),
∂E ( k )
c(i , j ) ( k + 1) = c(i , j ) ( k ) − η
i i
∂c(i , j ) ( k ) (12.58a)
i

∂E ( k )
w(i , j ) ( k + 1) = w(i , j ) ( k ) − η
i i
∂w(i , j ) ( k ) (12.58b)
i

where h is the learning rate.

For given input-output pairs (x( p), y( p); p = 1, 2, ..., P), the batch-updating algorithm back propagates the
cumulative error resulting from the difference between y ( p) ; p = 1,K , P, and yˆ ( p ) ; p = 1,K , P, from output
layer to the previous layers to update weights of the network.
In this section, we have described a method that can be used to construct identifiers of dynamical systems
that, in turn, could be employed to construct neuro-fuzzy control systems. The idea behind the method is to
apply the backpropagation algorithm to a fuzzy logic system.
Neuro-fuzzy control refers to the design methods for fuzzy logic controllers that employ neural network
techniques. The design methods for neuro-fuzzy control are derived directly from methods for neural
control. Thus, if we replace the NN blocks in Figs 11.23–11.25 with ANFIS blocks, then we end up with
neuro-fuzzy control systems.

12.7 GENETIC ALGORITHMS

A genetic algorithm (GA) uses the principles of evolution, natural selection and genetics from natural
biological systems in a computer algorithm to simulate evolution. Essentially, the genetic algorithm is an
optimization technique that performs a parallel, stochastic, but directed search to evolve the fittest popula-
tion. In this section, we will introduce the genetic algorithm and explain how it can be used for the design and
tuning of fuzzy systems.

12.7.1 Introduction
In the 1950s and 1960s, several computer scientists independently studied evolutionary systems, with the
idea that evolution could be used as an optimization tool for engineering problems. The idea, in all these
systems, was to evolve a population of candidate solutions to a given problem, using operators inspired by
natural genetic variation and natural selection.
Why use evolution as an inspiration for solving computational problems? To evolutionary computation
researchers, the mechanisms of evolution are well suited for some of the most pressing computational prob-
lems in many fields. Many computational problems require searching through a huge number of possibilities
for solutions. An example is the problem of searching for proteins, in which an algorithm is sought that will
search, among the vast number of possible amino acid sequences, for a protein with specified properties.
Such search problems can often benefit from an effective use of parallelism, in which many different
possibilities are explored simultaneously, in an efficient way. For example, in searching for proteins with
specified properties, rather than evaluate one amino acid sequence at a time, it would be much faster to
evaluate many, simultaneously. What is needed is, both computational parallelism and an intelligent strat-
egy, for choosing the next set of sequences to evaluate.
 Fuzzy Logic Models 727
Biological evolution is an appealing source of inspiration for addressing such problems. Evolution is, in
effect, a method of searching among an enormous number of possibilities for “solutions”. In biology, the
enormous set of possibilities is the set of possible genetic sequences, and the desired “solutions” are highly
fit organisms—organisms well able to survive and reproduce in their environments. Of course, the fitness of
a biological organism depends on many factors—for example, how well it can weather the physical charac-
teristics of the environment, and how well it can compete with, or cooperate with, the other organisms
around it. The fitness criteria continually changes as creatures evolve; so evolution is searching a constantly
changing set of possibilities. Searching for solutions in the face of changing conditions is precisely what is
required for adaptive computer programs. Furthermore, evolution is a massively parallel search method:
rather than work on one species at a time, evolution tests and changes millions of species in parallel. Finally,
the “rules” of evolution are remarkably simple: species evolve by means of random variation (via mutation,
recombination and other parameters), followed by natural selection in which the fittest tend to survive and
reproduce, thus propagating their genetic material to future generations. Yet, these simple rules are thought
to be responsible, in large part, for the extraordinary variety and complexity we see in the biosphere.
Knowledge of biological terminology, though not necessary, may help better appreciation of genetic
algorithms. All living organisms consist of cells, and each cell contains the same set of one or more chromo-
somes—strings of DNA (deoxyribonucleic acid). A chromosome can be conceptually divided into genes—
functional blocks of DNA, each of which encodes a particular protein. Very roughly, one can think of a gene
as encoding a trait, such as eye colour. The different possible “settings” for a trait (e.g., blue, brown, hazel)
are called alleles. Each gene is located at a particular locus (position) on the chromosome.
Many organisms have multiple chromosomes in each cell. The complete collection of genetic material (all
chromosomes taken together) is called the organism’s genome. The term genotype refers to the particular set of
genes contained in a genome. The genotype gives rise, under foetus and later development, to the organism’s
phenotype—its physical and mental characteristics, such as eye colour, height, brain size and intelligence.
Organisms whose chromosomes are arrayed in pairs are called diploid; organisms whose chromosomes
are unpaired are called haploid. In nature, most sexually reproducing species are diploid, including human
beings. In diploid sexual reproduction, recombination (or crossover) occurs: in each parent, genes are ex-
changed between each pair of chromosomes to form a gemete (a single chromosome), and then gemetes from
the two parents pair up to create a full set of diploid chromosomes. In haploid sexual reproduction, genes are
exchanged between the two parents’ single-strand chromosomes. Offspring are subject to mutation, in which
single nucleotides (elementary bits of DNA) are changed from parent to offspring; mutation may cause the
chromosomes of children to be different from those of their biological parents. The fitness of an organism is
typically defined as the probability that the organism will live to reproduce (viability) or, as a function of the
number of offspring the organism has (fertility).
The basic idea of a genetic algorithm is very simple. The term chromosome typically refers to a candidate
solution to a problem, typically stored as strings of binary digits (1s and 0s) in the computer’s memory. The
‘genes’ are short blocks of adjacent bits that encode a particular element of the candidate solution (e.g., in
the context of multi-parameter function optimization, the bits encoding a particular parameter might be
considered to be a gene). An ‘allele’ in a bit string, is either 0 or 1. Crossover typically consists of exchang-
ing genetic material between two single-chromosome haploid parents. Mutation consists of flipping the bit
at a randomly-chosen locus.
Most applications of genetic algorithms employ haploid individuals, particularly, single-chromosome
individuals. The genotype of an individual, in a genetic algorithm using bit strings, is simply the configura-
tion of bits in that individual’s chromosome. Genetic algorithms with more complex representation,
including diploid and multiple chromosomes, and higher cardinality alphabets, have also been developed.
However, the binary case is both the simplest and the most general.
 728 Digital Control and State Variable Methods

12.7.2 How are Genetic Algorithms Different from Traditional Methods?

The current literature identifies three main types of search methods: calculus-based, enumerative and
random. Calculus-based methods have been studied extensively. These subdivide into two main classes:
indirect and direct. Indirect methods seek local extrema by solving the usually nonlinear set of equations,
resulting from setting the gradient of the objective function equal to zero. Given a smooth, unconstrained
function, finding a possible peak starts by restricting search to those points with slopes of zero in all direc-
tions. On the other hand, direct (search) methods seek local optima by hopping on the function and moving
in a direction related to the local gradient. This is simply the notion of hill climbing: to find the local best,
climb the function in the steepest permissible direction.
Both the calculus-based methods are local in scope: the optima they seek are the best in a neighbourhood
of the current point. Clearly, starting the search procedures in the neighbourhood of the lower peak will
cause us to miss the main event (the higher peak). Furthermore, once the lower peak is reached, further
improvement must be sought through random restart or other trickery. Another problem with calculus-based
methods is that, they depend upon the existence of derivatives (well-defined slope values). Even if we allow
numerical approximation of derivatives, this is a severe shortcoming. The real world of search is fraught
with discontinuities and vast multi-modal (i.e., consisting of many ‘hills’) noisy search spaces; methods
depending upon restrictive requirements of continuity and derivative existence, are unsuitable for all, but a
very limited, problem domain.
Enumerative schemes have been considered in many shapes and sizes. The idea is fairly straightforward:
within a finite search space, the search algorithm starts looking at objective function values at every point in
the space, one at a time. Although the simplicity of the type of algorithm is attractive, and enumeration is a
very human kind of search, such schemes have applications wherein the number of possibilities is small.
Even the highly touted enumerative scheme, dynamic programming, breaks down on problems of moderate
size and complexity.
Random walks and random schemes that search and save the best, in the long run, can be expected to do
no better than enumerative schemes. We must be careful to separate the strictly random search methods from
randomized techniques. The genetic algorithm is an example of a search procedure that uses random choice
as a tool, to guide a highly exploitative search through a coding of parameter space. Using random choice as
a tool in a directed search process seems strange at first, but nature contains many examples.
The traditional schemes have been used successfully in many applications; however, as more complex
problems are attacked, other methods will be necessary. We shall soon see how genetic algorithms help
attack complex problems [146].
The GA literature describes a large number of successful applications, but there are also many cases in
which GAs perform poorly. Given a potential application, how do we know if a GA is a good method to use?
There is no rigorous answer, though many researchers share the intuitions that if the space to be searched is
large, is known not to be perfectly smooth and unimodal, or is not well understood; or if the fitness function
is noisy; and if the task does not require a global optimum to be found—i.e., if quickly finding a sufficiently
good solution is enough—a GA will have a good chance of being competitive or surpassing other methods.
If the space is not large, it can be searched exhaustively by enumerative search methods, and one can be sure
that the best possible solution has been found, whereas a GA might give only a ‘good’ solution. If the space
is smooth and unimodal, a gradient ascent algorithm will be much more efficient than a GA. If the space is
well understood, search methods using domain-specific heuristics can often be designed to outperform any
general-purpose method such as a GA. If the fitness function is noisy, a one-candidate-solution-at-a-time
search method such as simple hill climbing might be irrecoverably led astray by the noise; but GAs, since
they work by accumulating fitness statistics over many generations, are thought to outperform robustly in the
presence of small amounts of noise.
 Fuzzy Logic Models 729
These intuitions, of course, do not rigorously predict when a GA will be an effective search procedure,
competitive with other procedures. It would be useful to have a mathematical characterization of how
the genetic algorithm works, that is, predictive. Research on this aspect of genetic algorithms has not yet
produced definite answers.

12.7.3 Basics of Genetic Algorithms

Encoding Simple genetic algorithms require the natural parameter set of the problem to be coded as a
finite-length string of binary bits 0 and 1. For example, given a set of two-dimensional data ((x, y) data
points), we want to fit a linear curve (straight line) through the data. To get a linear fit, we encode the
parameter set for a line y = q1x + q2, by creating independent bit strings for the two unknown constants q1 and
q2 (parameter set describing the line) and then joining them (concatenating the strings). A bit string is a
combination of 0s and 1s, which represents the value of a number in binary form. An n-bit string can accom-
modate all integers upto the value 2n – 1.
For problems that are solved by the genetic algorithm, it is usually known that the parameters, that are
manipulated by the algorithm, will lie in a certain fixed range, say {qmin, qmax}. A bit string may then be
mapped to the value of a parameter, say qi, by the mapping
b
qi = qmini + L (qmaxi – qmin i) (12.59)
2 −1
where “b” is the number in decimal form that is being represented in binary form (e.g., 152 may be repre-
sented in binary form as 10011000), L is the length of the bit string (i.e., the number of bits in each string),
and qmax and qmin are user-specified constants, which depend on the problem in hand.
The length of the bit strings is based on the handling ca-
Table 12.6 Data set through which
pacity of the computer being used, i.e., how long a string
a line fit is required
(strings of each parameter are concatenated to make one
Data number x y long string representing the whole parameter set) the com-
puter can manipulate at an optimum speed.
1 1.0 1.0 Let us consider the data set in Table 12.6. For performing
2 2.0 2.0 a line (y = q1x + q2) fit, as mentioned earlier, we encode the
3 4.0 4.0 parameter set (q1, q2) in the form of binary strings. We take
4 6.0 6.0 the string length to be 12 bits. The first six bits encode the
parameter q1, and the next six bits encode the parameter q2.
The strings (000000, 000000) and (111111, 111111), represent the points (qmin1, qmin2) and (qmax1, qmax2),
respectively, in the parameter space for the parameter set (q1, q2). Decoding of (000000) and (111111) to
decimal form gives 0 and 63, respectively. However, problem specification may impose different values of
minimum and maximum for qi. We assume that the minimum value to which we would expect q1 or q2 to go
would be –2, and the maximum would be 5. Therefore
qmini = –2 and qmaxi = 5
Consider a string (a concatenation of two substrings)
000111 010100 (12.60)
representing a point in the parameter space for the set (q1, q2). The decimal value of the substring (000111)
is 7 and that of (010100) is 20. This, however, does not give the value of the parameter set (q1, q2) corre-
sponding to the string in (12.60). The mapping (12.59) gives the value:
b 7
q1 = qmin1 + L (qmax1 – qmin1) = – 2 + 6 (5 – (– 2)) = – 1.22
2 -1 2 -1
 730 Digital Control and State Variable Methods

b 20
q2 = qmin2 + (qmax2 – qmin2) = – 2 + 6 (5 – (– 2)) = 0.22
2L - 1 2 -1
Fitness Function A fitness function takes a chromosome (binary string) as an input, and returns a number
that is a measure of the chromosome’s performance on the problem to be solved. Fitness function plays the
same role in GAs as the environment plays in natural evolution. The interaction of an individual with its
environment, provides a measure of fitness to reproduce. Similarly, the interaction of a chromosome with a
fitness function, provides a measure of fitness that the GA uses when carrying out reproduction. Genetic
algorithm is a maximization routine; the fitness function must be a non-negative figure of merit.
It is often necessary to map the underlying natural objective function to a fitness function form through
one or more mappings. If the optimization problem is to minimize cost function J (q ), where q denotes the
parameter set, then the following cost-to-fitness transformation may be used:
1
J(q ) = (12.61)
J (q) + ε
where e is a small positive number. Maximization of J can be achieved by minimization of J ; so the desired
effect is achieved.
Another way to define the fitness function is to let
J(q(k)) = – J (q(k)) + max J (q ( k ))
q( k )
m
(12.62) r
The minus sign in front of the J (q(k)) term turns the minimization problem into a maximization problem
m r
and max J (q ( k )) term is needed to shift the function up, so that J (q(k)) is always positive; k is the
q( k )
iteration index.
A fitness function can be any nonlinear, non-differentiable, discontinuous, positive function because the
algorithm only needs a fitness value assigned to each string.
For the problem in hand (fit a line through a given data set), let us choose a fitness function. Using
decoded values of q1 and q2 of a chromosome, and the four data values of x given in the data table, calculate
y$i = q1x + q2; i = 1, 2, 3, 4
$
These computed values of yi are compared with the correct values yi, given in the data table, and square
of errors in estimating the y’s is calculated for each string. The summation of the square of errors is sub-
tracted from a large number (400 in this problem) to convert the problem into a maximization problem:
∑ b y$i g 2
J(q) = 400 – – yi ; q = [q1 q2] (12.63)
i
The fitness value of the string (12.60) is calculated as follows:
q1 = –1.22, q2 = 0.22
For x = 1.0, y$1 = q1x + q2 = –1.00
For x = 2.0, y$2 = – 2.22
For x = 4.0, y$3 = – 4.66
For x = 6.0, y$4 = – 7.10
4
∑ b y$i g 2
J(q) = 400 – − yi = 131.586
i =1
Initialization of Population The basic element processed by a GA is the string formed by concatenating
substrings, each of which is a binary coding of a parameter of the search space. If there are N decision
variables in an optimization problem, and each decision variable is encoded as an n-digit binary number,
 Fuzzy Logic Models 731
then a chromosome is a string of n ´ N binary digits. We start with a randomly selected initial population of
such chromosomes; each chromosome in the population represents a point in the search space, and hence, a
possible solution to the problem. Each string is then decoded to obtain its fitness value, which determines the
probability of the chromosome being acted on by genetic operators. The population then evolves, and a new
generation is created through the application of genetic operators (The total number of strings included in a
population, is kept unchanged throughout generations, for computational economy and efficiency). The new
generation is expected to perform better than the previous generation (better fitness values). The new set of
strings is again decoded and evaluated, and another generation is created using the basic genetic operators.
This process is continued until convergence is achieved within a population.
Let q j(k) be a single parameter in chromosome j of generation k. Chromosome j is composed of N of
these parameters:
q j(k) = [q 1j (k), q 2j (k), K, q Nj (k)] (12.64)
The population of chromosomes, in generation k:
n
P(k) = q j( k ) | j = 1, 2, K, S s (12.65)
where S represents the number of chromosomes in the population. We want to pick S to be big enough, so
that the population elements can cover the search space. However, we do not want S to be too big, since this
increases the number of computations we have to perform.
For the problem in hand, Table 12.7 gives an initial population of 4 strings, the corresponding decoded
values of q 1 and q 2, and the fitness value for each string.

Table 12.7 Initial population

String number String q1 q2 J

1 000111010100 –1.22 0.22 131.586

2 010010001100 0.00 –0.67 323.784
3 010101101010 0.33 2.67 392.41
4 100100001001 2.00 –1.00 365.00
SJ 1212.8
Av.J 303.2
Max. J 392.41

Evolution occurs as we go from generation k to the next generation k + 1. Genetic operations of selection,
crossover and mutation are used to produce a new generation.
Selection Basically, according to Darwin, the most qualified (fittest) creatures survive to mate. Fitness is
determined by a creature’s ability to survive predators, pestilence, and other obstacles to adulthood and
subsequent reproduction. In our unabashedly artificial setting, we quantify “most qualified” via a chromo-
some’s fitness J(q j(k)). The fitness function is the final arbiter of the string-creature’s life or death. Selecting
strings according to their fitness values means that the strings with a higher value have a higher probability
of contributing one or more offspring in the next generation.
Selection is a process in which good-fit strings in the population are selected to form a mating pool, which
we denote by
M(k) = {m j(k) | j = 1, 2, ..., S} (12.66)
 732 Digital Control and State Variable Methods

The mating pool is the set of chromosomes that are selected for mating. A chromosome is selected for
mating pool according to the probability proportional to its fitness value. The probability for selecting the ith
string is

pi =
e
J q i( k ) j (12.67)
S
∑ J eq ( k ) j
j

j =1

For the initial population of four strings in Table 12.7, the probability for selecting each string is calcu-
lated as follows:
131586
. 131586
.
p1 = = = 0.108
131586
. + 323.784 + 392.41 + 365 1212.8
323.784 392.41 365.00
p2 = = 0.267; p3 = = 0.324; p4 = = 0.301
1212.8 1212.8 1212.8
To clarify the meaning of the formula and, hence, the selection strategy, Goldberg [146] uses the analogy
of spinning a unit-circumference roulette wheel; the wheel is cut like a pie into S regions where the ith region
is associated with the ith element of P(k). Each pie-shaped region has a portion of the circumference that is
given by pi in Eqn. (12.67).
The roulette wheel for the problem in hand is shown in Fig. 12.33.
37.5 String 1 has solution probability of 0.108. As a result, string 1 is
given 10.8% slice of the roulette wheel. Similarly, string 2 is given
26.7%
26.7% slice, string 3 is given 32.4% slice and string 4 is given 30.1%
10.8 2 of the roulette wheel.
30.1% You spin the wheel, and if the pointer points at region i when the
10.8%
1 4 wheel stops, then you place q i into the mating pool M(k). You spin
0 the wheel S times, so that S strings end up in the mating pool.
100 32.4% Clearly, the strings which are more fit will end up with more copies
3 in the mating pool; hence, chromosomes with larger-than-average
fitness, will embody a greater portion of the next generation. At the
67.6 same time, due to the probabilistic nature of the selection process,
Fig. 12.33 Roulette Wheel it is possible that some relatively unfit strings may end up in the
mating pool.
For the problem in hand, the four spins might choose strings 3, 3, 4 and 2 as parents (String 1 also may be
selected in the process of roulette wheel spin; it is just the luck of the draw. If the roulette wheel were spun
many times, the average results would be closer to the expected values).
Reproduction Phase; Crossover We think of crossover as mating in biological terms, which, at the
fundamental biological level, involves the process of combining chromosomes. The crossover operation
operates on the mating pool M(k). First, specify the “crossover probability” pc (usually chosen to be near
one, since, when mating occurs in biological systems, genetic material is swapped between the parents).
The procedure for crossover consists of the following steps:
(i) Randomly pair off the strings in the mating pool M(k) (i.e., form pairs to mate by the flip of a coin). If
there are an odd number of strings in M(k), then, for instance, simply take the last string and pair it off
with another string which has already been paired off.
 Fuzzy Logic Models 733
j i
(ii) Consider chromosome pair (q , q ) that was formed in step 1. Generate a random number r Î [0, 1].
(a) If r < pc, then crossover q j and q i. To crossover these chromosomes, select at random a “cross
site” and exchange all bits to the right of the cross site of one string, with those of the other. This
process is pictured in Fig. 12.34. In this example, the cross site is position four on the string, and
hence we swap the last eight bits between the two strings. Clearly, the cross site is a random
number between one and the number of bits in the string, minus one.
(b) If r > pc, then the crossover will not take place; hence, we do not modify the strings.
(iii) Repeat step 2 for each pair of strings that is in M(k).
Before crossover After crossover
0101 01 101010
? 0101 10 001100
0100 10 001100 0100 01 101010
A
Cross site
A
Cross sitè

Fig. 12.34 Crossover operation example

For the problem in hand, Table 12.8 shows the power of crossover. The first column shows the four
strings selected for mating pool. We randomly pair off the strings. Suppose that random choice of mates has
selected the first string in the mating pool, to be mated with the fourth. With a cross site 4, the two strings
cross and yield two new strings as shown in Table 12.8. The remaining two strings in the mating pool are
crossed at site 9; the resulting strings are given in the table.

Table 12.8 Selection and crossover processes

String Mating pool Couples After crossover q1 q2 J
number
3 010101 101010 0101 | 01 101010 010110 001100 0.44 – 0.67 370.574
3 010101 101010 0100 | 10 001100 010001 101010 – 0.11 2.67 378.311
4 100100 001001 010101 101 | 010 010101 101001 0.33 2.56 392.794
2 010010 001100 100100 001 | 001 100100 001010 2.00 –0.89 362.972
SJ 1504.7
Av.J 376.163
Max. J 392.794

In nature, an offspring inherits genes from both the parents. The crossover process creates children strings
from the parent strings. The children strings thus produced, may or may not, have combination of good
substrings from parents strings, but we don’t worry about this too much, because if good strings are not
created by crossover, they will not survive too long because of the selection operator. If good strings are
created by crossover, there will be more copies of it in the next mating pool generated by the selection
operator.
Besides the fact that crossover helps to model the mating part of the evolution process, why should the
genetic algorithm perform crossover? Basically, the crossover operation perturbs the parameters near good
positions to try to find better solutions to the optimization problem. It tends to help perform a localized
 734 Digital Control and State Variable Methods

search around the more fit strings (since, on average, the strings in the generation k mating pool are more fit
than the ones in the generation k population).
Reproduction Phase; Mutation Selection ac-
cording to fitness, combined with crossover, gives
Before mutation After mutation genetic algorithms the bulk of their processing
100100 001010 100100 101010 power. Mutation plays a secondary role in the
A A operation of GAs. Mutation is needed because,
Mutation site Mutation site occasionally, chromosomes may lose some poten-
tially useful genetic material. In artificial genetic
Fig. 12.35 Mutation operation example systems, mutation is realized by inverting a ran-
domly chosen bit in a string. This is illustrated in
Fig. 12.35.
Besides the fact that this helps to model mutation in a biological system, why should the genetic algorithm
perform mutation? Basically, it provides random excursions into new parts of the search space. It is possible
that we will get lucky and mutate to a good solution. It is the mechanism that tries to make sure that we do not
get stuck at a local maxima, and that we seek to explore other areas of the search space to help find a global
maximum for J(q). Usually, the mutation probability pm is chosen to be quite small (e.g., less than 0.01)
since this will help guarantee that all the strings in the mating pool are not mutated so that any search
progress that was made is lost (i.e., we keep it relatively low to avoid degradation to exhaustive search via a
random walk in the search space).
After mutation, we get a modified mating pool M(k). To form the next generation for the population, we let
P(k + 1) = M(k) (12.68)
where this M(k) is the one that was formed by selection and modified by crossover and mutation. Then the
above steps repeat, successive generations are produced, and we thereby model evolution (of course, it is a
very crude model).
Terminal Conditions While the biological evolutionary process continues, perhaps indefinitely, we
would like to terminate our artificial one and find the following:
· The population string—say, q *(k)—that best maximizes the fitness function. Notice that, to determine
this, we also need to know the generation number k where the most fit string existed (it is not necessar-
ily in the last generation). A computer code, implementing the genetic algorithm, keeps track of the
highest J value, and the generation number and string that achieved this value of J.
· The value of the fitness function J(q *(k)).
There is then the question of how to terminate the genetic algorithm. There are many ways to terminate a
genetic algorithm, many of them similar to termination conditions used for conventional optimization algo-
rithms. To introduce a few of these, let e > 0 be a small number and n1 > 0 and n2 > 0 be integers. Consider
the following options for terminating the GA:
· Stop the algorithm after generating the generation P(n1)—that is, after n1 generations.
· Stop the algorithm after at least n2 generations have occurred and, at least n1 steps have occurred when
the maximum (or average) value of J for all population members has increased by no more than e.
· Stop the algorithm once J takes on a value above some fixed value.
 Fuzzy Logic Models 735
The above possibilities are easy to implement on a computer but, sometimes, you may want to watch the
parameters evolve and decide yourself when to stop the algorithm.
Working Parameters A set of parameters is predefined to guide the genetic algorithm, such as:
· the length of each decision variable encoded as a binary string,
· the number of chromosomes to be generated and operated in each generation, i.e., population size,
· the crossover probability pc,
· the mutation probability pm ,
· and the stopping criterion.

12.7.4 GA for Fuzzy System Design and Tuning

The genetic algorithm can be used in the computer-aided design of control systems, since it can artificially
evolve an appropriate controller that meets the performance specifications to the greatest extent possible. To
do this, the genetic algorithm maintains a population of strings, each representing a different controller (bits
on the strings characterize parameters of the controller), and it uses a fitness function that characterizes the
closed-loop specifications. Suppose, for instance, that the closed-loop specifications indicate that we want,
*
for a step input, a (stable) response with a settling time of t s and a percent overshoot of M *p . We need to
define the fitness function so that it measures how close each string in the population at time k (i.e., each
controller candidate) is meeting these specifications. Suppose that we let ts and Mp denote the settling time
and overshoot, respectively, for a given string (we compute these for a string in the population by performing
a simulation of the closed-loop system with the candidate controller and a model of the plant). Given these
values, we let (for each string and every time step k)
*
J = w1(ts – t s )2 + w2(Mp – M *p )2
where wi > 0; i = 1, 2, are weighting factors. The function J characterizes how well the candidate controller
meets the closed-loop specifications, where if J = 0 it meets the specifications perfectly. The weighting
factors can be used to prioritize the importance of meeting the various specifications (e.g., a high value of w2
relative to w1 indicates that the percent overshoot specification is more important than the settling time
specification).
To minimize J with the genetic algorithm, we can choose the fitness function
1
J=
J +ε
where e > 0 is a small positive number.
This completes the description of how to use a genetic algorithm for computer-aided control system
design. Note that the above approach depends in no way on whether the controller that is evolved is a
conventional controller (e.g., a PID controller) or a fuzzy system or a neural network. For instance, you
could use a fuzzy system for the controller and let the genetic algorithm tune the appropriate parameters
(e.g., scaling factors).

Example 12.8 Consider the problem of maximizing the function

J(q ) = q 2 (12.69)
where q is permitted to vary between 0 and 31.
To use a GA, we must first code the decision variables of our problem as some finite length string. For
this problem, we will code the variable q simply as a binary unsigned integer of length 5. With a 5-bit
unsigned integer, we can obtain numbers between 0 (00000) and 31 (11111). The fitness function is simply
defined as the function J(q ).
 736 Digital Control and State Variable Methods
To start off, we select an initial population at random. We select a population of size 4. Table 12.9 gives
the selected initial population, decoded q values, and the fitness function values J(q). As an illustration of
the calculations done, let’s take a look at the third string of the initial population, string 01000. Decoding this
string gives q = 8, and the fitness J(q) = 64. Other q and J(q) values are obtained similarly.
The mating pool of the next generation may be selected by spinning a roulette wheel. Alternatively, the
roulette-wheel technique may be implemented using a computer algorithm:
(i) Sum the fitness of all the population members, and call this result the total fitness S J.
(ii) Generate r, a random number between 0 and total fitness.
(iii) Return the first population member whose fitness, added to the fitness of the preceding population
members (running total), is greater than or equal to r.
We generate numbers randomly from the interval [0, 1170] (refer Table 12.9). For each number, we
choose the first chromosome for which the running total of fitness is greater than, or equal to, the random
number. Four randomly generated numbers are 233, 9, 508, 967; string 1 and string 4 give one copy to the
mating pool, string 2 gives two copies, and string 3 gives no copies.
With the above active pool of strings looking for mates, simple crossover proceeds in two steps: (1)
strings are mated randomly, (2) mated-strings couples crossover. We take the crossover probability pc = 1.
Looking at Table 12.10, we find that, random choice of mates has selected the second string in the mating
pool to be mated with the first. With a crossing site of 4, the two strings 01101, and 11000 cross and yield
two new strings, 01100 and 11001. The remaining two strings in the mating pool are crossed at site 2; the
resulting strings are given in Table 12.10.

Table 12.9 Selection process

String number Initial q J(q ) Running Total
population
1 01101 13 169 169
2 11000 24 576 745
3 01000 8 64 809
4 10011 19 361 1170
S J 1170
Av. J 293
Max. J 576

Table 12.10 Crossover process

Mating pool New Population q J(q)

0110 1 01100 12 144

1100 0 11001 25 625

11 000 11011 27 729

10 011 10000 16 256

SJ 1754
Av. J 439
Max. J 729
 Fuzzy Logic Models 737
The last operator, mutation, is performed on a bit-by-bit basis. We assume that the probability of muta-
tion in this test is 0.001. With 20 transferred bit positions, we should expect 20 ´ 0.001 = 0.02 bits to
undergo mutation during a given generation. Simulation of this process indicates that no bits undergo muta-
tion for this probability value. As a result, no bit positions are changed from 0 to 1, or vice versa, during this
generation.
Following selection, crossover and mutation, the new population is ready to be tested. To do this, we
simply decode the new strings created by the simple GA, and calculate the fitness function values from the q
values thus decoded. The results are shown in Table 12.10. While drawing concrete conclusions from a
single trial of a stochastic process is, at best, a risky business, we start to see how GAs combine high-
performance notions to achieve better performance. Both the maximal and average performance have
improved in the new population. The population average fitness has improved from 293 to 439 in one
generation. The maximum fitness has increased from 576 to 729 during that same period.

Review Examples

Review Example 12.1 We consider here the simplest fuzzy PI control scheme for a servo motor with
the control model (refer Fig. 12.36a)
Y ( s) 1
= G(s) = (12.70)
U ( s) s (s + 3.6)
The objective of the fuzzy controller is to control angular position y(t) of the servo motor to achieve a
given set-point yr, within desired accuracy.
The discretized model for the plant (refer Chapter 3) is

Y (z) 0.0237z -1 + 0.0175z -2

Gh0G(z) = =
U ( z ) 1 - 1.407z -1 + 0.407z -2
y(k) = 1.407y(k – 1) – 0.407y(k – 2) + 0.0237u(k – 1) + 0.0175u(k – 2) (12.71)
The proposed fuzzy controller (refer Fig. 12.36b) has two input variables:
e(k) = error between the set-point and actual position of the shaft,
v(k) = rate of change of error;
and one output variable:
Du(k) = incremental voltage signal to the driver circuit of the motor.
Universe of discourse for e(k) = {– Le, Le}
Universe of discourse for v(k) = {– Lv, Lv}
Universe of discourse for Du(k) = {– HDu, HDu}
Clockwise and counterclockwise rotations are defined as positive and negative, respectively.
The two input variables are quantized to two fuzzy subsets: Positive ( P) , Negative ( N )
~ ; and one output
~
variable is quantized to three fuzzy subsets: Positive ( P ) , Zero ( Z ) , Negative ( N
~
). Triangular membership
~ ~
functions are used.
 738 Digital Control and State Variable Methods
e(k) u(k) u(t)

yr + Fuzzy y(t)
PI ZOH Plant
– T = 0.25 sec Controller

(a)

v(k) v* (k)

– 1
GV u* (k) u(k)
T F Inference
+
U Engine
Z
z–1 e* (k) Z GU
I
e(k) F Rule
Y Base
GE

(b)
Fig. 12.36 A structure for fuzzy controller

The scaling factors GE (gain for error variable), and GV (gain for velocity variable) describe input
normalization:
e*(k) = GE × e(k); GE = L / Le
v*(k) = GV × v(k); GV = L /Lv
where e*, v* Î {– L, L}
The output Du* of the fuzzy controller is denormalized to Du, by the relation
Du(k) = GU¢ ´ Du*(k); GU¢ = HDu /H
where Du* Î {– H, H}
Without loss of generality, we take L = H = 1 (refer Fig. 12.37).
The fuzzy PI controller uses the following four fuzzy rules:
IF e*(k) is P and v*(k) is P THEN Du*(k) is P
~ ~ ~
IF e*(k) is P and v*(k) is N THEN Du*(k) is Z
~ ~
IF e*(k) is N and v*(k) is P THEN Du*(k) is Z
~ ~
IF e*(k) is N and v*(k) is N THEN Du*(k) is N ~
The initial value of the system output and the initial velocity are set to zero, as is the initial output of the
fuzzy PI controller.
 Fuzzy Logic Models 739

1
N P
Z

N 1 P
0.5

–1 0 1 e*, v* –1 0 1 u*

Fig. 12.37 Membership functions for input and output variables of the fuzzy controller

The scaling factors GE, GV and GU¢ of the fuzzy controller may be tuned by trial and error. Refer Appen-
dix B for realization of the controller.

Review Example 12.2 Although applied in many complex industrial processes, fuzzy logic-based expert
systems experience a deficiency in knowledge acquisition, and rely, to a great extent, on empirical and heuris-
tic knowledge which, in many cases, cannot be elicited objectively. Fuzziness describes event ambiguity. It
measures the degree to which an event occurs, not whether it occurs. Fuzzy controller design involves the
determination of the linguistic state space, definition of membership grades of each linguistic term, and the
derivation of the control rules. The information on the above aspects can be gathered by interviewing process
operators, process knowledge experts, and other sources of domain knowledge and theory.
The choice of a FLC depends more on the intuition of the designer, and its effectiveness depends on the
following parameters:
(i) Selection of rule set.
(ii) Number, shape and size of the membership functions of the input and output variables.
(iii) Value of the normalizing factors for the input variables to the FLC.
(iv) Value of the denormalizing factors for the output variables of the FLC.
Genetic Algorithm (GA) has a capability to guide in poorly understood, irregular spaces. In the
following, we illustrate the use of GA in designing a FLC for the thermal system described in Review
Example 11.2. We design FLC by tuning only the normalizing and denormalizing factors.
The proposed fuzzy controller has two input variables (Refer Figs 12.36):
e(k) – error between set-point and actual temperature of the tank,
v(k) – rate of change of error,
and one output variable:
Du(k) – incremental heat input to the tank.
The universe of discourse for all the three variables may be taken as [–1, 1]. Proposed membership
functions are shown in Fig. 12.38.
 740 Digital Control and State Variable Methods
The selected rules are

Rate of
change of error® N NZ P
Error
B
N N N Z
NZ N Z P
P Z P P

The initial value of the system output is Y0. The initial velocity, and the initial output of the Fuzzy PI
controller are set to zero.
The scaling factors GE, GV and GU¢ of the fuzzy controller may be tuned using genetic algorithm. Refer
Appendix B for realization of the controller.

1
N NZ P

e*, v*
–1 –0.1 0.1 1

1
N Z P

u*
–1 0 1
Fig. 12.38 Membership functions for the input and output variables of the fuzzy controller

Problems
12.1 (a) In the following, we suggest a membership function for fuzzy description of the set “real numbers close to 2”:
A μ
~ = {x, ~A ( x)}
 Fuzzy Logic Models 741
where
R| 0 2
; x <1
m (x) = S− x + 4x − 3 ; 1 ≤ x ≤ 3
~A
|T 0 ; x > 3
Sketch the membership function (arc of a parabola) and determine its supporting interval, and a-cut interval for
a = 0.5.
(b) Sketch the piecewise-quadratic membership function
R|
2( x - 1)2 ; 1 £ x < 3 /2
2
m B(x) =
|S
1 - 2( x - 2) ; 3 / 2 £ x < 5 /2
~ 2( x - 3)2
|| ; 5/2 £ x £ 3
T 0 ; otherwise
and show that it also represents “real number close to 2”. Determine its support, and a-cut for a = 0.5.
12.2 (a) The well known Gaussian distribution in probability is defined by
2

1 - F
1 x- m I
f (x) = e 2 s
H K ;–¥<x<¥
s 2p
where m is the mean and s is the standard deviation of the distribution. Construct a normal, convex membership
function from this distribution (select parameters m and s) that represents “real numbers close to 2”. Find its
support, and a-cut for a = 0.5. Show that the membership function
1
m A (x) =
1 + ( x - 2 )2
~
also represents “real numbers close to 2”. Find its support, and a-cut for a = 0.5.
12.3 Consider the piecewise-quadratic function
R| 0 ; x < a
x-a 2
2|| FH IK
b-a
;a £ x < a+b
2
f(x) = S| F I
x-b 2 ; a+b £ x < b
1- 2
|| H K
b-a 2
; b£ x <c
T1
Construct a normal, convex membership function from f (x) (select parameters a, b and c) that represents the set
“tall men” on the universe {3, 9}. Determine the crosspoints and support of the membership function.
12.4 (a) Write an analytical expression for the membership function m A(x) with supporting interval [–1, 9] and a-
~
cut interval for a = 1 given as [4, 5].
(b) Define what we mean by a normal membership function and a convex membership function. Is the function
described in (a) above (i) normal, (ii) convex?
12.5 (a) Let the fuzzy set A be the linguistic ‘warm’ with membership function
~
R| 0
x - a1
; x < a1
||
b1 - a1 ; a1 £ x £ b1
m A (x) = S|1 ; b1 £ x £ b2
~ x - a2
; b2 £ x £ a2
||
b2 - a2
; x ³ a2
T
0
a1 = 64ºF, b1 = 70ºF, b2 = 74ºF, a2 = 78ºF
A a normal fuzzy set?
(i) Is ~
(ii) Is A a convex fuzzy set?
A a singleton fuzzy set?
(iii) Is ~
742 Digital Control and State Variable Methods
If answer to one or more to these is ‘no’, then give an example of such a set.
(b) For fuzzy set A described in part (a), assume that b1 = b2 = 72ºF.
~
Sketch the resulting membership function and determine its support, crosspoints and a-cuts for a = 0.2 and 0.4.
12.6 Consider two fuzzy sets A and B; membership functions m A(x) and m B(x) are shown in Fig. P12.6. The fuzzy
~ ~ ~ ~
variable x is temperature.
Sketch the graph of m A(x), m A Ç B (x) and m A È B (x).
~ ~ ~ ~ ~
Which t-norm and t-conorm have you used?

A (x) B (x)

1 1

0 0
15 °C x 10 °C 20 °C x
Fig. P12.6
12.7 Consider the fuzzy relation R on the universe X ´ Y, given by the membership function
~
1
m R (x, y) = ,
~ [1 + 100( x − 3y ) 4 ]
vaguely representing the crisp relation x = 3y. All elements satisfying x = 3y have unity grade of membership;
elements satisfying, for example, x = 3.1y have membership grades less than 1. The farther away the elements
are from the straight line, the lower are the membership grades.
Give a graphical representation of the fuzzy relation R .
~
12.8 Assume the membership function of the fuzzy set ~ A, big pressure, is
R| 51 - x
; x ³ 5

~
m A (x) = S| 1 - 4
;1£ x £ 5
T 0 ; otherwise
Assume the membership function of the fuzzy set B , small volume, is
~
R| 1y - 1 ; y£ 1
m (y) = S 1 -
B
;1£ y£ 5
~ |T 0 4 ; otherwise
Find the truth values of the following propositions:
(i) 4 is big pressure,
(ii) 3 is small volume,
(iii) 4 is big pressure and 3 is small volume,
(iv) 4 is big pressure ® 3 is small volume.
Explain the conjunction and implication operations you have used for this purpose.
12.9 Consider the following statements:
Input : A′ is very small
~
Rule : IF A′ is small THEN B′
~ is large
~
Inference : B′ is very large
~
 Fuzzy Logic Models 743
If R A′
~ is a fuzzy relation from X to Y representing the implication rule, and ~ is a fuzzy subset of X, then the
fuzzy subset B¢ of Y, which is induced by A′ , is given by
~ ~
B¢ = ~ A′ o R
~ ~
where o operation (composition) is carried out by taking cylindrical extension of A¢ , taking the intersection with
~
R , and projecting the result onto Y.
~
Define cylindrical extension, intersection and projection operations that lead to max-min compositional rule of
inference.
12.10 Input : x is A′ and y is B¢
~ ~
Rule 1 : IF x is A1 and y is B1 THEN z is C1
~ ~ ~
Rule 2 : IF x is A2 and y is B2 THEN z is C2
~ ~ ~
Inference : ~¢
z is C
Taking arbitrary membership functions for A1, B1, C1, A2 , B2 and C2 , outline the procedure of determining C ¢
~ ~ ~ ~ ~ ~ ~
corresponding to the crisp inputs x = x0 and y = y0. Use t-norm ‘min’ for conjunction operation, Mamdani’s
implication operation and max-min compositional rule of inference.
12.11 Fig. P12.11 shows the fuzzy output of a certain control problem. Defuzzify by using the centre of area method,
to obtain the value of crisp control action.

Fig. P12.11

12.12 Consider the fuzzy system concerning the terminal voltage and speed of an electric motor, described by the
membership functions
x 100 150 200 250 300
m A(x) 1 0.8 0.5 0.2 0.1
~
y 1600 1800 2000 2200 2400
m B (y) 1 0.9 0.7 0.3 0
~

Input : Voltage is rather small (x is A¢ )

~
Rule : IF voltage is small (x is A ) THEN speed is small ( y is B )
~ ~
Inference : Speed is rather small (y is B¢)
~
 744 Digital Control and State Variable Methods

Assume that the input fuzzy set A¢ is a singleton at x0 = 125. Determine the inference fuzzy set B¢ of the fuzzy
~ ~
system. Defuzzify this set to obtain crisp value for speed.
Use piecewise continuous approximations of graphs of mA(x) and m B (y) to describe your solution.
~ ~
12.13 Consider the two-input, one output fuzzy system:
Input : x is A¢ and y is B¢
~ ~
Rule 1 : IF x is A1 and y is B1 THEN z is C1
~ ~ ~
Rule 2 : IF x is A2 and y is B2 THEN z is C2
~ ~ ~
Inference : z is C ¢
~
The fuzzy sets Ai , Bi and Ci ; i = 1, 2, have the membership functions
~ ~ ~
R| x - 2 ; 2£ x £ 5
R| x - 3 ; 3£ x £6
m (x) = S 8 -3 x
A1 m (x) = S 9 -3 x
A2
~ ~
|T 3 ; 5£ x £8 |T 3 ; 6£ x £ 9

R| y - 5 ; 5£ y£8
R| y - 4 ; 4 £ y£ 7
m (y) = S 11 3- y
B1 m (y) = S 10 3- y
B2
~ ~
|T 3 ; 8 £ y £ 11 |T 3 ; 7 £ y £ 10

R| z - 1 ; 1£ z £ 4
R| z - 3 ;3£ z £ 6
m (z) = S 7 3- z
C1 m (z) = S 9 -3 z
C2
~ |T 3 ; 4£ z £ 7 ~ |T 3 ;6£ z £ 9

Assume fuzzy sets ~ A¢ and B¢ are singletons at x0 = 4 and y0 = 8. Determine the inference fuzzy set C¢ of the
~ ~
fuzzy system. Defuzzify C ¢.
~
12.14 The control objective is to design an automatic braking system for motor cars. We need two analog signals:
vehicle speed (V), and a measure of distance (D) from the vehicle in the front. A fuzzy logic control system will
process these, giving a single output, braking force (B), which controls the brakes.
Term set for each of the variables (V, D, and B) is of the form:
{PS (positive small), PM (positive medium), PL (positive large)}
Membership functions for each term-set are given in Fig. P12.14.
Suppose that for the control problem, two rules have to be fired:
Rule 1: IF D = PS and V = PM THEN B = PL
Rule 2: IF D = PM and V = PL THEN B = PM
For the sensor readings of V = 55 km/hr, and D = 27 m from the car in front, find graphically
(i) the firing strengths of the two rules,
(ii) the aggregated output, and
(iii) defuzzified control action.
12.15 The control objective is to automate the wash time when using a washing machine. Experts select for inputs dirt
and grease of the clothes to be washed, and for output parameter the wash time, as follows:
Dirt =D {SD (small dirt), MD (medium dirt), LD (large dirt)}
Grease =D {NG (no grease), MG (medium grease), LG (large grease)}
Washtime =D {VS (very short), S (short), M (medium), L (long), VL (very long)}
 Fuzzy Logic Models 745

Fig. P12.14
The degrees of the dirt and grease are measured on a scale from 0 to 100; washtime is measured in minutes from
0 to 60.
50 - x 10 - z
m SD (x) = ; 0 £ x £ 50 mVS (z) = ; 0 £ z £ 10
~ 50 ~ 10
R| x R| z
50 ; 0 £ x £ 50 10 ; 0 £ z £ 10
m MD (x) =
~
S|
100 - x ; 50 £ x £ 100 ~
S|
m S (z) =
25 - z ; 10 £ z £ 25
T50 T 15
R| z - 10 ; 10 £ z £ 25
x - 50
m LD (x) = ; 50 £ x £ 100 m (z) = S 4015- z
M
~ 50 ~
|T 15 ; 25 £ z £ 40

R| z - 25 ; 25 £ z £ 40
50 - y
m NG (y) = ; 0 £ y £ 50 m (z) = S 15- z
L
~ 50 ~
|T6020 ; 40 £ z £ 60

R| y ; 0 £ y £ 50 z - 40
m MG (y) =
~ S| 10050- y ; 50 £ y £ 100
mVL (z) =
~ 20
; 40 £ z £ 60
T 50
y - 50
m LG (y) = ; 50 £ y £ 100
~ 50
 746 Digital Control and State Variable Methods
The selected rules are:
Grease ® NG MG LG
Dirt
B Find a crisp control output for
the following sensor readings:
SD VS M L Dirt = 60; Grease = 70
MD S M L
LD M L VL

12.16 A fuzzy controller is acting according to the following rule base (N = negative, M = medium, P = positive):
R1 : If x1 is N AND x2 is N, THEN u is N
R2 : If x1 is N OR x2 is P, THEN u is M
R3 : If x1 is P OR x2 is N, THEN u is M
R4 : If x1 is P AND x2 is P, THEN u is P
The membership functions of the input and output and variables are given in Fig. P12.16. Actual inputs are
x1 = 2.5 and x2 = 4. Which rules are active and what will be the controller action u? Find u by applying standard
fuzzy operations: min for AND, and max for OR.

N P P N M P
1 1 N 1

x1 x2 u
0 1 2 3 4 0 1 2 3 4 0 1 2 3 4
Fig. P12.16
12.17 Consider the following fuzzy model of a system with inputs x and y and outpur z:
Rule 1 : If x is A3 OR y is B1, THEN z is C1
Rule 2 : If x is A2 AND y is B2, THEN z is C2
Rule 3 : If x is A1, THEN z is C3
The membership functions of the input and output variables are given in Fig. P12.17. Actual inputs are
x1 and y1. Find the output z by applying standard fuzzy operation: min for AND, and max for OR.
m m m

A1 A2 A3 B1 B2 C1 C2 C3
1 1 1
0.7
0.5
0.2 0.1
x y z
x1 y1 0 20 35 55 70 100
25 30 60 65
Fig. P12.17
12.18 A fuzzy controller is acting according to the following rule base (N = negative, P = positive):
R1 : If x1 is N AND x2 is N, THEN u is k1
R2 : If x1 is N OR x2 is P, THEN u is k2
R3 : If x1 is P OR x2 is N, THEN u is k2
R4 : If x1 is P AND x2 is P, THEN u is k3
 Fuzzy Logic Models 747
The membership functions of the input variables are given in Fig. P12.16 and the membership functions of the
output variable (which is a controller action) u are singletons placed at k1 = 1, k2 = 2, k3 = 3. Actual inputs are
x1 = 2.5 and x2 = 4. Find u by applying standared fuzzy operations: min for AND, and max for OR.
12.19 Consider a two-dimensional sinc equation defined by
sin( x1 )sin( x2 )
y = sinc ( x1 , x2 ) =
x1x2
Training data are sampled uniformly from the input range [–10, 10] ´ [–10, 10]. With two symmetric triangular
membership functions assigned to each input variable, construct a Sugeno fuzzy model architecture for the sinc
function. Give defining equations for determination of the premise and consequent parameters of the model.
12.20 To identify the nonlinear system
2
(
y = 1 + ( x1) 0.5 + ( x2 ) −1 + ( x3 ) −1.5 )
we assign two membership functions to each input variable. Training and testing data are sampled uniformly
from the input ranges [1,6] × [1,6] × [1,6],and [1.5,5.5] × [1.5,5.5] × [1.5,5.5] , respectively. Extract Sugeno fuzzy
rules from the numerical input-output training data that could be employed in an ANFIS model.
12.21 Assume that a fuzzy inference system has two inputs x1 and x2, and one output y. The rule base contains two
Sugeno fuzzy rules as follows:
Rule 1: IF x1 is A11 and x2 is A21 THEN y(1) = a(1) (1) (1)
0 + a 1 x1 + a 2 x2
~ ~
Rule 2: IF x1 is A12 and x2 is A22 THEN y(2) = a(2) (2) (2)
0 + a 1 x1 + a 2 x2
~
~Aij are Gaussian functions.
For given input values x1 and x2, the inferred output is calculated by

μ (1) y (1) + μ (2) y (2)

yˆ =
μ (1) + μ (2)
where m(r), r = 1, 2 are firing strengths of the two rules. Product inference is used to calculate the firing strengths
of the rules.
Develop ANFIS architecture for this modelling problem, and derive learning algorithms based on least squares
estimation and the gradient-descent methods.
12.22 The objective is to minimize the function:
f(x1, x2) = ( x12 + x2 – 11)2 + ( x22 + x1 – 7)2
in the interval 0 £ x1, x2 £ 6. The true solution to the problem is [3, 2]T having a function value equal to zero.
Take up this problem to explain the steps involved in GA: maximizing the function
1.0
F(x1, x2) = ; 0 £ x1, x2 £ 6.
. + f ( x1 , x2 )
10
Step 1: Take 10 bits to code each variable. With 10 bits, what is the solution accuracy in the interval (0, 6)?
Step 2: Take population size equal to total string length, i.e., 20. Create a random population of strings.
Step 3: Consider the first string of the initial random population. Decode the two substrings and determine
the corresponding parameter values. What is the fitness function value corresponding to each string?
Similarly for other strings, calculate the fitness values.
Step 4: Select good strings in the population to form the mating pool.
Step 5: Perform crossover on random pairs of strings (the crossover probability is 0.8).
Step 6: Perform bitwise mutation with probability 0.05 for every bit.
The resulting population is the new population. This completes one iteration of GA and the generation count is
incremented by 1.
 References

APPLICATIONS

1. Siouris, G.M.; Missile Guidance and Control Systems; New York: Springer-Verlag, 2004.
2. Lawrence, A.; Modern Inertial Technology: Navigation, Guidance, and Control; 2nd Edition, New York:
Springer-Verlag, 1998.
3. Leonhard, W.; Control of Electric Drives; 3rd Edition, New York: Springer-Verlag, 2001.
4. Bose, B.K.; Modern Power Electronics and AC Drives; Englewood Cliffs, NJ : Prentice-Hall, 2001.
5. Sen, P.C.; Thyristor DC Drives; 2nd Edition, New York: Wiley-Interscience, 1991.
6. Seborg, D.E., T.F. Edgar, and D.A. Mellichamp; Process Dynamics and Control; 2nd Edition, New York:
John Wiley, 2004.
7. Bequette, B.W.; Process Control: Modeling, Design and Simulation; Upper Saddle Drive, NJ: Pearson
Education, 2003.
8. Marlin, T.E.; Process Control: Designing Processes and Control Systems for Dynamic Performance; 2nd
Edition, New York: McGraw-Hill, 2000.
9. Smith, C.A., and A.B. Corripio; Principles and Practice of Automatic Process Control; 2nd Edition, New
York: John Wiley, 1997.
10. Shinskey, F.G.; Process Control Systems; 4th Edition, New York: McGraw-Hill, 1996.
11. Astrom, K.J., and T. Hagglund; PID Controllers: Theory, Design, and Tuning; 2nd Edition, Seattle, WA:
International Society for Measurement and Control, 1995.
12. Corripio, A.B.; Tuning of Industrial Control Systems; Research Tringle Park, North Carolina: Instrument
Society of America, 1990.
13. Stephanopoulos, G.; Chemical Process Control—An Introduction to Theory and Practice; Englewood
Cliffs, NJ: Prentice-Hall, 1984.
14. Steven, B., and F. Lewis; Aircraft Control and Simulation; New York: Wiley Interscience, 2003.
15. Nelson, R.C.; Flight Stability and Automatic Control; 2nd Edition, New York: McGraw-Hill, 1997.
16. Etkin, B., and L.D. Reid; Dynamics of Flight: Stability and Control; 3rd Edition, New York: John Wiley,
1996.
17. Craig, J.J.; Introduction to Robotics: Mechanics and Control; 3rd Edition, Englewood Cliffs, NJ: Prentice-
Hall, 2003.
 References 749
18. Koivo, A.J.; Fundamental for Control of Robotics Manipulators; New York: John Wiley, 1989.
19. Asada, H., and K. Youcef-Toumi; Direct Drive Robots: Theory and Practice; Cambridge, Massachusetts:
The MIT Press, 1987.
20. Valentino, J.V., and J. Goldenberg; Introduction to Computer Numerical Control (CNC) 3rd Edition,
Englewood Cliffs, NJ: Prentice-Hall, 2002.
21. Groover, M.K; Automation, Production Systems, and Computer-Integrated Manufacturing; 2nd Edition,
Englewood Cliffs, NJ: Prentice-Hall, 2000.
22. Olsson, G., and G. Piani; Computer Systems for Automation and Control; London: Prentice-Hall Interna-
tional, 1992.
23. Hughes, T.A.; Programmable Controllers; 4th Edition, North Caroline: The Instrumentation, Systems, and
Automation Society, 2004.
24. Petruzella, F.D.; Programmable Logic Controllers; 3rd Edition, New York: McGraw-Hill, 2004.
25. Bolton, W.; Programmable Logic Controllers; 3rd Edition, Buslingfon, MA: Newnes Publication, 2003.
26. Beards, C.F.; Vibration Analysis and Control System Dynamics; 2nd Edition, Englewood Cliffs, NJ:
Prentice-Hall, 1995.

MATHEMATICAL BACKGROUND

27. Noble, B, and J.W. Daniel; Applied Linear Algebra; 3rd Edition, Englewood Cliffs, NJ: Prentice-Hall
1988.
28. Lancaster, P., and M. Tismenetsky; The Theory of Matrices; 2nd Edition, Orlando, Florida: Academic
Press, 1985.
29. Lathi, B.P.; Linear Systems and Signals; 2nd Edition, New York; Oxford University Press, 2005.
30. Oppenheim, A.V., R.W. Shafer, and J.R. Buck; Discrete-Time Signal Processing; 2nd Edition, Englewood
Cliffs, NJ: Prentice-Hall, 1999.
31. Oppenheim, A.V., A.S. Willsky and S. Hamid Nawab; Signals and Systems; 2nd Edition, Upper Saddle
River, NJ : Prentice-Hall, 1997.
32. Brown, J.W., and R.V. Churchill; Complex Variables and Applications; 7th Edition, New York: McGraw-
Hill, 2003.
33. Lefschetz, S.; Differential Equations: Geometric Theory; 2nd Edition, New York : Dover Publications, 1977.

DYNAMICAL SYSTEMS AND MODELLING

34. Palm, W.J., III; System Dynamics; New York: McGraw-Hill, 2004.
35. Zak, S.H.; Systems and Control; New York: Oxford University Press, 2003.
36. Ogata, K.; System Dynamics; 4th Edition, Englewood Cliffs, NJ: Prentice-Hall, 2003.
37. Clark, R.N.; Control System Dynamics; New York: Cambridge University Press, 1996.
38. Belanger, P.R.; Control Engineering: A Modern Approach; Orlando, Florida: Saunders College
Publishing, 1995.
39. Moschytz, G.S.; Linear Integrated Networks: Fundamentals; New York: Van Nostrand Reinhold, 1974.
40. Schwarz, R.J., and B. Friedland; Linear Systems; New York: McGraw-Hill, 1965.
41. Mason, S.J., “Feedback Theory: Further Properties of Signal Flow Graphs”, Proc., IRE, Vol.44, No. 7,
pp. 920-926, July 1956.
42. Mason, S.J., “Feedback Theory: Some Properties of Signal Flow Graphs”, Proc. IRE, Vol. 41, No. 9,
pp.1144–1156, Sept.1953.
 750 Digital Control and State Variable Methods

INDUSTRIAL CONTROL DEVICES

43. Necsulescu, D.; Mechatronics; Singapore: Pearson Education, 2002.

44. Gupta, S.; Elements of Control Systems; Upper Saddle River, N.J: Pearson Education, 2002.
45. Kilian, C.T.; Modern Control Technology: Components and Systems; 2nd Edition, Singapore: Delmar Pub-
lishers, 2002.
46. Morris, N.M.; Control Engineering; 4th Edition, London: McGraw-Hill, 1991.
47. De Silva, C.W.; Control Sensors and Actuators; Englewood Cliffs, NJ: Prentice-Hall, 1989.
48. Anderson, W.R.; Controlling Electrohydraulic Systems; New York: Marcel Dekker, 1988.
49. Parr, E.A.; Industrial Control Handbook, Vol.1-Vol.3; Oxford: BSP Professional Books, 1987.
50. Schuler, C.A., and W.L. McNamee; Industrial Electronics and Robotics; New York: McGraw-Hill,
1986.
51. Kenjo, T.; and S. Nagamori; Permanent-magnet and Brushless DC Motors, Oxford: Clarendon Press, 1985.
52. Kenjo, T.; Stepping Motors and their Microprocessor Controls; Oxford Clarendon Press, 1984.
53. Ahrendt, W.R., and C.J. Savant, Jr.; Servomechanism Practice; 2nd Edition, New York: McGraw-Hill,
1960.
54. Kuo, S., and W.S.Gan; Digital Signal Processors: Architectures, Implementations, and Applications;
Upper Saddle River, NJ: Pearson Education, 2005.
55. Pack, D., and S. Barrett; 68HCI2 Microcontroller: Theory and Applications; Upper Saddle River, NJ:
Prentice-Hall, 2002.
56. Wolf, W.; Computers as Components: Principles of Embedded Computing System Design; San Franscisco:
Morgan Kaufmann, 2001.
57. Mackenzie, I.; The 8051 Microcontroller; Upper Saddle River, NJ: Prentice-Hall, 1998.

FEEDBACK CONTROL THEORY

58. Franklin, G.F., J.D. Powell, and A. Emami-Naeini; Feedback Control of Dynamical Systems; 5th Edition,
Upper Saddle River, NJ: Pearson Education, 2005.
59. Dorf, R.C., and R.H. Bishop; Modern Control Systems, 10th Edition, Upper Saddle River, NJ: Pearson
Education, 2004.
60. Nise, N.S.; Control Systems Engineering; 4th Edition, Danvers, MA: John Wiley, 2003.
61. Kuo, B.C., and F. Golnaraghi; Automatic Control Systems; 8th Edition, Danvers, MA: John Wiley, 2003.
62. D’Azzo, J.J., C.H. Houpis, and S.N.Sheldon; Linear Control System Analysis and Design with MATLAB;
5th Edition, New York: Marcel Dakker, 2003.
63. Wilkie, J.,M. Johnson, and R. Ketabi; Control Engineering: An Introductory Course; New York: Palgrave,
2003.
64. Dorsey, J.; Continuous and Discrete Control Systems; New York: McGraw-Hill, 2002.
65. Stefani, R.T., B. Shahian, C.J. Savant, and G.H. Hostetter; Design of Feedback Control Systems;
4th Edition, New York: Oxford University Press, 2002.
66. Ogata, K.; Modern Control Engineering; 4th Edition, Englewood Cliffs, NJ: Prentice-Hall, 2001.
67. Goodwin, G.C., S.F. Graebe, and M.E. Salgado; Control System Design; Upper Saddle River, NJ: Pearson
Education, 2001.
68. Phillips, C.L., and R.D. Harbor; Feedback Control Systems; 4th Edition, Englewood Cliffs, NJ: Prentice-
Hall, 1999.
 References 751
nd
69. Shinners, S.M.; Modern Control System Theory and Design; 2 Edition, New York: John Wiley, 1998.
70. Raven, F.H.; Automatic Control Engineering; 5th Edition, New York: McGraw-Hill, 1995.
71. Wolovich, W.A.; Automatic Control Systems: Basic Analysis and Design; Orlando, Florida: Saunders
College Publishing, 1994.
72. Doebelin, E.O.; Control System Principles and Design; New York; John Wiley, 1986.
73. Truxal, J.G.; Automatic Feedback Control System Synthesis; New York: McGraw-Hill, 1995.

ROBUST CONTROL

74. Dullerud, G.E., and F. Paganini; A Course in Robust Control Theory; New York: Springer-Verlag, 2000.
75. Zhou, K., and J.C. Doyle; Essentials of Robust Control; Englewood Cliffs, NJ: Prentice-Hall, 1997.
76. Saberi, A., B.M. Chen, and P. Sannuti; Loop Transfer Recovery–Analysis and Design; London: Springer-
Verlag, 1993.
77. Doyle, J.C., B.A. Francis, and A.R. Tannenbaum; Feedback Control Theory; New York: MacMilian;
Publishing Company, 1992.
78. Francis, B.A.; A course in the Control Theory, Lecture Notes in Control and Information Seciences;
No. 88, Berlin: Springer-Verlag, 1987.
79. Rosenwasser, E., and R. Yusupov; Sensitivity of Automatic Control Systems; Boca Ratno, FL: CRC
Press, 2004.

DIGITAL CONTROL

80. Franklin, G.F., J.G. Powell, and M.L. Workman; Digital Control of Dynamic Systems; 3rd Edition, San
Diejo, CA: Addision-Wesley, 1997.
81. Astrom, K.J., and B. Wittenmark; Computer-Controlled Systems; 3rd Edition, Englewood Cliffs, NJ:
Prentice-Hall, 1996.
82. Santina, M.S., and A.R. Stubberud; Sample-Rate Selection; Boca Raton, FL: CRC Press, 1996.
83. Santina, M.S., and A.R. Stubberud; Quantization Effects; Boca Raton, FL: CRC Press, 1996.
84. Ogata, K.; Discrete-Time Control Systems; 2nd Edition, Upper Saddle River, NJ: Pearson Education, 1995.
85. Santina, M.S., A.R. Stubberud, and G.H. Hostetter; Digital Control System Design; 2nd Edition, Stamford,
CT: International Thomson Publishing, 1994.
86. Phillips, C.L., and H.T. Nagla, Jr.; Digital Control System Analysis and Design; 3rd Edition, Upper Saddle
River, NJ: Pearson Education, 1994.
87. Kuo, B.C.; Digital Control Systems; 2nd Edition, Orlando, Florida: Saunders College Publishing, 1992.
88. Houpis, C.H., and G.B. Lemont; Digital Control Systems: Theory, Hardware, and Software; 2nd Edition,
New York: McGraw-Hill, 1992.
89. Hristu-Varsakelis, D., and W.S. Levine(eds); Handbook of Networked and Embedded Control Systems;
Boston: Birkhauser Publishers, 2005.
90. Ibrahim, D.; Microcontroller Based Temperature Monitoring and Control; Woburn, MA: Newnes, 2002.
91. Chidambaram, M.; Computer Control of Processes; New Delhi: Narosa Publishers, 2002.
92. Ozkul, T.; Data Acquisition and Process Control using Personal Computers; New York: Marcel Dekkar,
1996.
93. Rigby, W.H., and T. Dalby; Computer Interfacing: A Practical Approach to Data Acquisition and
Control; Englewood Cliffs, NJ: Prentice-Hall, 1995.
94. Tooly, M.; PC-Based Instrumentation and Control; 2nd Edition, Oxford: Newnes: Publications, 1995.
 752 Digital Control and State Variable Methods
95. Gupta, S., and J.P. Gupta; PC Interfacing for Data Acquisition and Process Control; 2nd Edition, Research
Triangle Park, North Carolina : Instrument Society of America, 1994.
96. Gopal, M.; Digital Control Engineering; New Delhi: New Age International, 1988.
97. Rao, M.V.C, and A.K. Subramanium; “Elimination of singular cases in Jury’s test” IEEE Trans.Automatic
Control, Vol.AC-21, pp.1q14-115, 1976.
98. Jury, E.I., and J. Blanchard; “A Stability Test for Linear Discrete-time Systems in Table Form”, Proc. IRE.,
Vol.49, pp.1947-1948, 1961.

STATE SPACE AND LINEAR SYSTEMS

99. Tewari, A.; Modern Control Design with MATLAB and Simulink; Singapore: John Wiley, 2003.
100. Chen, C-T.; Linear System Theory and Design; 3rd Edition, New York: Oxford University Press, 1998.
101. DeRusso, P.M., R.J. Roy, C.M. Close, and A.A. Desrochers; State Variables for Engineers; 2nd Edition,
New York: John Wiley, 1998.
102. Szidarovszky, F., and A.T. Bahill; Linear System Theory; 2nd Edition, Boca Raton, FL: CRC Press, 1998.
103. Antsaklis, P.J., and A.N. Michel; Linear Systems; New York: McGraw-Hill, 1997.
104. Rugh, W.I.; Linear System Theory; 2nd Edition, Upper Saddle River, NJ: Prentice-Hall, 1995.
105. Gopal, M.; Modern Control System Theory; 2nd Edition, New Delhi: New Age International, 1993.
106. Brogan, W.L.; Modern Control Theory; 3rd Edition, Englewood Cliffs, NJ: Prentice-Hall, 1991.
107. Friedland, B.; Control System Design: An Introduction to State-Space Methods; New York: McGraw-
Hill, 1986.
108. Zadeh, L.A., and C.A. Desoer; Linear System Theory: A State Space Approach; New York: McGraw-
Hill, 1963.

MULTIVARIABLE AND OPTIMAL CONTROL

109. Bertsekas, D.P.; Dynamic Programming and Optical Control; 3rd Edition, Belnont: Athena Sientific, 2005.
110. Naidu, D.S.; Optimal Control Systems; Boca Raton, FL: CRC Press, 2003.
111. Camacho, E.F., and C. Bordons; Model Predictive Control; 2nd Edition, London: Springer-Verlag, 2003.
112. Bryson, A.E.; Applied Linear Optimal Control: Examples and Algorithms; Cambridge, UK : Cambridge
University Press, 2002.
113. Locatelli, A.; Optimal Control: An Introduction; Basel, Switzerland: Birkhauser Verlag, 2001.
114. Chen, T., and B. Francis; Optimal Sampled-Data Control Systems; 2nd Edition, London: Springer, 1996.
115. Lewis, F.L., and V.L. Syrmos; Optimal Control; 2nd Edition, New York: John Wiley, 1995.
116. Zhou, K., J.C. Doyle, and K. Glover; Robust and Optimal Control; Upper Saddle River, NJ: Prentice-
Hall, 1996.
117. Anderson, B.D.O., and J.B. Moore; Optimal Control: Linear Quadratic Methods; Englewood Cliffs, NJ:
Prentice-Hall, 1990.
118. Grimble, M.J., and M.A., Johnson; Optimal Control and Stochastic Estimation: Theory and Applications;
Vol.I, Chichester: John Wiley, 1988.
119. Albertos, P., and A. Sala; Multivariable Control Systems: An Engineering Approach; Springer, 2004.
120. Clarke, D.W., C. Mothadi, and P.S. Tuffs “Generalized predictive control–Part I. The basic algorithm”,
Automatica, vol. 23, No. 2, pp. 137–148; 1987.
121. Clarke, D.W., C. Mothadi, and P.S. Tuffs “Generalized predictive control–Part II. Extensions and interpre-
tations”, Automaticam vol. 23, No. 2, pp. 149–160; 1987.
 References 753
122. Fortman, T.E., and K.L. Hitz, An Introduction to Linear Control Systems, New York: Marcel Dekker, 1977.
123. Kautsky, J., N.K. Nichols, and P. Dooren; “Robust pole assignment by linear state feedback”, Int.,
J.Control, Vol.41, No.5, pp.1129-1155, 1985.
124. Doyle, J.C., and G. Stein; “Robustness with Observers”, IEEE Trans. Automatic Control, Vol.AC-24,
No.4, pp.607-611, 1979.

NONLINEAR CONTROL SYSTEMS

125. Khalil, H.K.; Nonlinear Systems; 3rd Edition, Upper Saddle River, NJ: Prentice-Hall, 2001.
126. Slotine, J-J.E., and W.Li.; Applied Nonlinear Control; Englewood Cliffs, NJ: Prentice-Hall, 1991.
127. Itkis,U.; Control Systems of Variable Structure; New York: John Wiley, 1976.
128. Atherton, D.P.; Nonlinear Control Engineering; London: Van Nostrand Reinhold, 1975.
129. Minorsky N.; Theory of Nonlinear Control Systems; New York: McGraw-Hill, 1969.

SYSTEM IDENTIFICATION AND ADAPTIVE CONTROL

130. Narendra, K.S., and A.M. Annaswamy; Stable Adaptive Systems; New York: Dover Publications, 2005.
131. Ljung, L.; System Identification: Theory for the User; 2nd Edition, Englewood Cliffs, NJ: Prentice-Hall,
1998.
132. Astrom, K.J., and B. Wittenmark; Adaptive Control; 2nd Edition, Reading, MA: Prentice-Hall, 1994.
133. Landau, I.D.; System Identification and Control Design; Englewood Cliffs, NJ: Prentice-Hall, 1990.
134. Sastry, S., and M. Bodson; Adaptive Control: Stability, Convergence and Robustness; Englewood Cliffs,
NJ: Prentice-Hall, 1989.
135. Grimble, M.J., and M.A. Johnson; Optimal Control and Stochastic Examination: Theory and Applica-
tions; Vol.II, Chichester: John Wiley, 1988.
136. Harris, C.J., and S.A. Billings (eds); Self-Tuning and Adaptive Control: Theory and Applications;
2nd Edition, London: Peter Peregrinus, 1985.

INTELLIGENT CONTROL

137. Negnevitsky, M.; Artificial Intelligence: A Guide to Intelligent Systems; 2nd Edition, Essex, UK: Pearson
Education, 2005.
138. Kccman, V.; Learning and Soft Computing; Cambridge, MA: The MIT Press, 2001.
139. Norgaard, M.O., Ravn, N.K. Poulsen, and L.K. Hansen; Neural Networks for Modelling and Control of
Dynamic Systems; London: Springer-Verlag, 2000.
140. Lewis, F.L., S. Jagannathan, and A. Yesildirek; Neural Network Control of Robot Manipulators and
Nonlinear Systems; London, Taylor & Francis, 1999.
141. Haykin, S.; Neural Networks: A Comprehensive Foundation, 2nd Edition, Prentice-Hall, 1998.
142. Jang, J-S.R., C-T. Sun, and E. Mizutani; Neuro-Fuzzy and Soft Computing: A Computational Approach to
Learning and Machine Intelligence; Upper Saddle River, NJ: Pearson Education, 1997.
143. Lin C.T., and C.S. George Lee; Neural Fuzzy Systems; Upper Saddle River, NJ: Prentice-Hall, 1996.
144. Ying, Hao; Fuzzy Control and Modelling: Analytical Foundations and Applications; New York: IEEE
Press, 2000.
145. Passino, K.M., and S. Yurkovich; Fuzzy Control; California: Addison-Wesley, 1990.
 754 Digital Control and State Variable Methods
146. Goldberg, D.E.; Genetic Algorithms in Search Optimization, and Machine Learning; Reading, Massachu-
setts, Addison-Wesley, 1989.
147. Werbos, P., A. Handbook of Learning and Approximate Dynamic Programming, New York: Wiley-IEEE
Press, August 2004.
148. Sutton. R.S., and A.G. Barto, Reinforcement Learning: An Introduction Cambridge, Mass.: MIT Press,
1998.
149. Mitchell, T.M., Machine Learning, Singapore: McGraw-Hill, 1997.
150. Bertsekas, D.P., and J.N. Tsitsiklis, Neruo-Dynamic Programming, Belmont, Mass: Athena Scientific,
1996.

TECHNICAL COMPUTING

151. National Programme on technology enhancement learnning

Course: Electrical Engineering (control Engineering)
Faculty Coordinator: Prof. M. Gopal
Web Content: Matlab Modules for Control System Principles and Design
www.nptel.iitm.ac.in
152. MATLAB & Simulink Software
www.mathworks.com
The MathWorks, Inc.
3 Apple Hill Drive
Natick, MA 01760-2098, USA
153. MATHEMATICA Software
www.wolfram.com
Wolfram Research, Inc.
100 Trade Center Drive
Champaign, IL 61820-7237, USA
154. MAPLESOFT Software
www.maplesoft.com
Maplesoft
615 Kumpf Drive
Waterloo, Ontario
Canada N2V1K8

COMPANION BOOK

155. Gopal, M.; Control Systems: Principles and Design; 3rd Edition, New Delhi: Tata McGraw-Hill, 2008.
 Answers to Problems

Caution
For some problems (especially the design problems) of the book, many alternative solutions are possible. The
answers given in the present section, correspond to only one possible solution for such problems.
2.1 (a) x1, x2: Outputs of unit delayers, starting at the right and proceeding to the left.

F=
LM 0 1 OP
;g=
0 LM OP
N−0.368 1.368 Q 1 NQ
c = [0.264 0.368]

(b) 0.368

1 z –1 z –1 0.264
R (z ) Y (z)

1.368

– 0.368
Y (z) 0.368z + 0.264
= 2
R( z ) z − 1.368z + 0.368
2.2 (a) y(k + 2) + 5y(k + 1) + 3y(k) = r(k + 1) + 2r(k)
(b) x1, x2: Outputs of unit delayers, starting at the right and proceeding to the left.

F=
LM 0 1OP ; g = LM 1OP ; c = [1 0]
N−3 −5Q N−3Q
Y (z) z+2
(c) = 2
R( z ) z + 5z + 3
756 Digital Control and State Variable Methods
1 Y (z) − z + 2
2.3 (a) y(k + 1) + y(k) = – r (k + 1) + 2r(k); =
2 R( z ) 1
z+
2
−1 ;k = 0
R| 1 k −1
2 5 1F I k
(b) y(k) = S| FH IK
5 (c) y(k) = – − ;k³0
− ;k ≥1 3 3 2H K
T
2 2

Aβ
2.4 (a) y(k) = Ab (a)k – 1; k ³ 1 (b) y(k) = [1 – (a)k]; k ³ 0
1−α
Aβ LM Aβ (α − e )OP k j Wk
(c) y(k) = [a k + (1 – a)k – 1]; k ³ 0 (d) y(k) = Re jW
(1 − α )2 Nα − e Q
k
1F 1 I kπ kπ O
2.5 (a) y(k) = (–1)k – (–2)k; k ³ 0 (b) y(k) = 1 + G J LMsin − cos ;k³0
2 H 2K N 4 4 QP
2.6 y(0) = 0, y(1) = 0.3679, y(2) = 0.8463, y(k) = 1; k = 3, 4, 5, ...
2.7 y(k) = 3(2)k – 2; k ³ 0
2.8 (a) y(k) = – 40d (k) + 20d (k – 1) – 10(0.5) k + 50(– 0.3)k; k ³ 0

(b) y(k) = – 16 + (0.56) k [7.94 sin(0.468k) + 16 cos(0.468k)]; k ³ 0

2.9 (a) y(k) = – 0.833(0.5)k – 0.41(– 0.3)k + 0.476(–1)k + 0.769; k ³ 0

(b) y(k) = –10k(0.5)k + 2.5(0.5)k – 6.94(0.1)k + 4.44: k ³ 0

2.10 y(¥) = K
2.13 (a) No (b) Yes
2.14 No
2.16 (b) T = p / 2

2.19

1 − e −T
2.20 G(z) = ; y(k) = 1 – e– kT; k ³ 0
z − e −T
 Answers to Problems 757

Y ( z ) 10 z + 0.76
LM OP
2.21 =
R( z ) 16 ( z − 1)( z − 0.46)
N Q
RSF1 + T + T I e(k ) – F1 + 2T I e(k − 1) + T e(k − 2) UV
2.25 u(k) – u(k – 1) = Kc D D D
|TGH T T JK H TK
I T W
L T F 1 I + T (1 − z )OP E(z)
U(z) = K M1 + D −1

N T H1 − z K T
c
I
−1
Q
z − 0.9391 I
2.26 (ii) D(z) = 0.4074 F
H z − 0.9752 K
L T F z + 1I + T F z − 1I OP E(z)
2.27 (a) U(z) = K M1 + D
N 2T H z − 1K T H z K Q
c
I

R| T e(i − 1) + e(i) T k U|
(b) u(k) = K Se( k ) + ∑
c + [e(k ) − e(k − 1)]V D

T| T 2 I i =1 T
W|
1 T
2.28 (a) y(k) = y(k – 1) + r(k) (b) y(k) = (1 – aT )y(k – 1) + Tr(k – 1)
1 + aT 1 + aT

2.29 F b + a + 1 I y(k) – F a + 2 I y(k – 1) + 1 y(k – 2) = 0; y(0) = a, y(–1) = a – Tb

H T T K HT T K 2
T
2 2

Y (z) Gh 0 G1 ( z ) Gh 0 G2 ( z )
3.1 =
R( z ) 1 + Gh 0G1( z ) Gh 0G2 H ( z )

Y (z) Gh 0G ( z )
3.2 =
R( z ) 1 + Gh 0G ( z ) H ( z )

Gh 0 G2 ( z )G1 R( z )
3.3 Y(z) =
1 + Gh 0 G2 HG1 ( z )

D( z )Gh 0 Gp ( z )
3.4 Y(z) = GpH2R(z) + [H1R(z) – GpH2R(z)]
1 + D( z )Gh 0G p ( z )

Y (z) D( z )Gh 0 G1G2 ( z ) X(z) D( z )Gh 0 G1 ( z )

3.5 = ; =
R( z ) 1 + D( z )[Gh 0 G1 ( z ) + Gh 0G1G2 ( z )] R( z ) 1 + D( z )[Gh 0 G1 ( z ) + Gh 0G1G2 ( z )]
GW ( z )
3.6 Y(z) =
1 + D( z )Gh 0G ( z )

3.7 Gh0G(z) = 0.0288

LM z + 0.92 OP ; θ L (z)
=
Gh 0 G( z )
N (z − 1) (z − 0.7788) Q θ R (z) 1 + Gh 0G ( z )

3.8
ω ( z) F K +K
= 159.97 G F IJ P
ωr ( z ) H z + 159 .97 K − 0.1353 K P
 758 Digital Control and State Variable Methods

Y (z) LM z( z + 0.92) OP
3.9 = 0.0095 3
R( z ) z − 2.81z 2 + 2.65z − 0.819
N Q
Y (z) 0.45z + 0.181 Y (z) 0.45z + 0.181
3.10 (i) = (ii) =
R( z ) z 2 + 0.081z + 0.181 R( z ) z 2 − 0.368z 2 + 0.45z + 0.181
3.13 0 < K < 4.293
3.14 (i) The system is stable for 0 < K < 4.723. (ii) The system is stable for 0 < K < 3.315.

3.15

3.16 0 < K < 0.785.

3.17 For T = 0.001 sec, the response y(k) is very close to y(t).

2.0

y(k)
1.5 T = 0.01 sec

1.0

0.5 y (t )

0 0.01 0.02 0.03 0.04 0.05

Time (sec)

3.18 y(k) = 1.02 (0.795)k sin(0.89k); k ³ 0

3.19 y(0) = 0; y(0.5T ) = 0.393; y(T ) = 0.632; y(1.5T ) = 0.528
y(2T ) = 0.465; y(2.5T ) = 0.493; y(3T ) = 0.509; y(3.5T ) = 0.502; L
 Answers to Problems 759
3.22 (a) K = 30ºC/(kg/min); tD = 5 min; t = 60 min (b) Kc = 0.545; TI = 13.75 min; tD = 2.2 min
3.23 Kc = 2.13; TI = 666.66 sec
4.1 Kp = ¥; Kv = K1/K2; Ka = 0.
25s + 1 z − 0.939
F I
4.2 D1(s) = ; D1(z) = 0.4
62.5s + 1 H
z − 0.975 K
Velocity error constants are equal
4.3 0, 1/3.041, ¥
4.4 Underdamped response with z = 0.199 and w n = 8.93.
[ D2 ( z ) + D1 ( z ) D3 ( z )]Gh 0 G( z ) GW ( z )
4.5 (a) Y(z) = R(z) +
1 + D1( z )Gh 0G( z ) 1 + D1 ( z )Gh 0 G( z )
GW ( z )
(b) Y(z) = D3(z)R(z) +
1 + D1 ( z )Gh 0 G( z )
(c) D1(z) can be made large to reject the disturbances
z − 0.607
4.7 S(z) = ; wb = 2 rad/sec
z − 0.214
4.8 GM = 8 dB; FM = 28º; nb = 1.6 rad/sec; w b = 1.35 rad/sec
4.9 (a) Increase plant gain by a factor of 10; F M = 30º

(b) D(z) = 4.2423 F z − 0.8187 I

H z − 0.2308 K
z − 0.98 I
(c) D(z) = 0.141 F
H z − 0.998 K
(d) nb1 = 4.8; nb2 = 9.8; nb3 = 1.04
(e) Yes

4.10 D(z) = 37.333 F z − 0.9048I ; K =¥

H z − 0.1111 K v

In the low-frequency range, Ð Gh0G(jn) is about –180º; therefore, a lag compensator cannot fulfil the
requirement of 50º phase margin.
4.11 (a) K = 50; F M = 20º; wb = 9.27 rad/sec.
z − 0.923
(c) With lag compensator D1(z) = 0.342 , F M = 54º, wb = 4.23 rad/sec
z − 0.973
z − 0.793
With lag-lead compensator D1(z)D2(z); D2(z) = 2.49 ; F M = 60º, wb = 7.61 rad/sec
z − 0.484
z − 0.6 F I
4.12 (a) K = 61.25 (b) Unstable (c) D(z) = 0.122K
z − 0.951 H K
4.13 (b) 0 < K < 2.1; K = 0.38
4.14 (a) K = 2.3925 (b) (i) K = 1.4557; (ii) K = 0.9653
4.15 (a) K = 0.88; 1.33 rad/sec (b) K = 0.072; t = 2.3 sec (c) K = 0.18; wn = 0.644 rad/sec
4.16 0 < A < 3.33
760 Digital Control and State Variable Methods
4.17 (a) t = 0.4854 (b) K = 5.1223

F z − 0.8187 I z − 0.94
4.18 (a) D1(z) = 13.934 (b) Kv = 3 (c) D2(z) =
H z − 0.1595 K z − 0.98
z − 0.84 I
4.19 D(z) = 1.91 F
H z − 0.98K
4.20 (a) z1, 2 = 0.78 ± j0.18 (b) Pole of D(z) at z = 0; (K/2) = 0.18

(c) Ka = 0.072 (d) z3 = 0.2; the third pole causes the response to slow down.

4.21 D(z) = 150 F z − 0.72 I

H z + 0.4 K
4.22 (a)
F (z − 0.9048) (z − 0.6135) IJ
D(z) = 135.22 G
H (z + 0.9833) (z − 0.7491) K
(b)
F (z − 0.9048) (z + 1) IJ ; 0.15
D(z) = 104.17 G
H (z + 0.9833) (z + 0.5) K
4.8 − 3.9z −1
4.23 D(z) =
1 − z −1
5.1 x1 = qM, x2 = θ& M , x3 = motor armature current ia, x4 = generator field current if ; y = qL

LM0 1 0 0 OP LM 0 OP
0 − 0.025 3 0 0
A= M
MM0 −12 −190 1000 PP MM 0 PP P ; b = M P ; c = [0.5 0 0 0]

N0 0 0 − 4.2Q N0.2Q
LM0 1 0 OP LM0 OP
A = 0 −1
5.2
MM0 0 − 520 PP ; b = MM02.5PP ; c = [1 0 0]
N Q N Q
5.3 x1 = qM, x2 = θ& M , x3 = ia

LM 0 1 0 OP
LM 0 OP
A= M 0
MM− k − 0.5
− (k2 + 0.5)
19 ; b =
−21
MM k0 PP ; c = LMN 201 0 0 OPQ
PP
1 1
N 40 2 2 PQ
MN 2 PQ
⎡ −B K T ⎤
⎢ J J ⎥ L0O
5.4 A= ⎢ ⎥ ; b = M k K P ; c = [1 0]
1 c
⎢ − (k K K + K ) − ( R + k K ) ⎥
1 t c b a 2 c MN L PQ
⎢ ⎥ a
⎣ L a L a ⎦

5.5 (a) A = M
L−11 6OP ; b = LM1OP ; c = [2 –1]
N−15 8Q N2Q
 Answers to Problems 761

(b) 1 s– 1 X2 1 s– 1
U X1 = Y

–3

–2

s– 1 X1

1 2

–11 –15
U Y
6
2 –1

s –1 X2

8
Y (s ) 1
(c) =
U ( s ) s 2 + 3s + 2

5.6 (a) A =
LM0 1OP ; b = LM0OP L 1 1OP ; b = LM0OP (c) |lI – A | = |lI – A | = l
(b) A = M 2
N0 0 Q N1 Q N−1 −1Q N1Q
1 M
Ls(s + 3) s + 3 1 OP
5.7 G(s) = −1 s(s + 3) s P
ΔM
MN −s −1 s PQ 2

1M P
L1O
3 2
H(s) = M s P ; D = s + 3s + 1
Δ
MNs PQ
2

5.9 x , x , x : outputs of integrators, starting at the right and proceeding to the left.
1 2 3

LM 0 1 0 OP LM0OP
A=
MM − 02 − 12 − 12PP ; b = MM01PP ; c = [2 – 2 1]
N Q NQ
5.10 x1, x2, x3, x4: outputs of integrators
Top row: x1, x2 = y1; Bottom row: x3, x4 = y2
0LM 0 0 −4 OP
3 0 LM
0 OP LM OP
1 −3 0 0 1 2 y1 (0) 0 1 0 0 LM OP
A= MM
0 −1 0
;B= PP ; x(0) = MM ;C= PP MM PP
MN 0 0 3
PQ 0
MN 0 0 0 1
PQ MN PQ N Q
0 0 1 −4 0 0 y2 (0)
s+3 1
5.11 (a) G(s) = (b) G(s) =
(s + 1)( s + 2 ) ( s + 1)( s + 2)
 762 Digital Control and State Variable Methods

1 −3s + 5
LM 4( s − 3) OP
5.12 G(s) = ; D = s3 – 4s2 + 6s – 5
Δ − s 2 + 2 s 2( s 2 − 3s + 1)
N Q
LM − 5 0.5 −3.5O L 0O
0PP ; b = MM 0PP ; c = [0
14
5.13 (a) A=
MM 40 −5 1 0] (b) G(s) =
( s + 1)( s + 2 )(s + 7)
N 1 0PQ MN −1PQ
5.14 (a) x1 = output of lag 1/(s + 2); x2 = output of lag 1/(s + 1)

A=
LM−2 1OP ; b = LM0OP ; c = [– 1 1]; d = 1
N −1 −1Q N1Q
(b) x1 = output of lag 1/(s + 2); x2 = output of lag 1/s; x3 = output of lag 1/(s + 1).
−2 1
LM 1 0 OP LM OP
A = −1 0 0 ; b = 1 ; c = [0 1 1]
MM
−1 0 −1
PP MM PP
N 1 Q NQ
5.15 x1 = output of lag 1/(s + 1); x2 = output of lag 5/(s + 5); x3 = output of lag 0.4/(s + 0.5); x4 = output of lag
4/(s + 2).
LM−1 − K − K 0
1 1 0 O
P LM K 0 OP 1
0 −5 − 5K − 5K 0 5K L1 1 0 0 O
A= M
MM − 0.4K − 0.4K − 0.5 0 PPP ; B = MMM0.4K 0 PPP ; C = MN0 0 1 1PQ
2 2 2

1 1 1

N 0 0 − 4 K −2 − 4 K Q
2 2 N 0 4K Q 2

L −1 0 O L1O
LM0 0 −2OP LM5OP
(i) A = M (ii) A = 1 0 −5 ; b = M0P ; c = [0
5.16
N 0 −2PQ ; b = MN1PQ ; c = [2 –1] MM0 1 −4PP M0P
N Q NQ
0 1]

L 0 1 0OP LM0OP
(iii) A = M 0
MM−6 −110 −61PP ; b = MM01PP ; c = [2 6 2]; d = 1
N Q NQ
LM0 0 0OP LM1OP
5.17 (i) A = 1 0 −2 ; b = 1 ; c = [0 0 1]
MM0 1 −3PP MM0PP
N Q NQ
L 0 1 0OP LM0OP
(ii) A = M 0
MM−6 −110 −61PP ; b = MM01PP ; c = [1 0 0]
N Q NQ
LM−1 0 0OP LM1OP
(iii) L = 0 −2
MM 0 0 −30PP ; b = MM11PP ; c = [– 1 2 1]; d = 1
N Q NQ
L0 1 0OP LM0OP
(a) A = M0
5.18
MM0 −1000 −521PP ; b = MM01PP ; c = [5000 1000 0]
N Q NQ
 Answers to Problems 763

LM
0 0 0 OP LM OP
50
(b) L = 0 −2 0 ; b = −31.25 ; c = [1 1 1]
MM 0 −50
PP MM PP
−18.75
0N Q N Q
L −1 1 0O L0O
5.19 (a) L=M 0 −1 0PP ; b = MM1PP ; c = [1 1 1] (b) y(t) = 2.5 – 2e – te – 0.5e –t –t – 2t
MM 0 0 −2PQ MN1PQ
N
5.20 1 2
L1O L1O
(i) l = 1, l = 2; v = M P ; v = M P 1 2
L1O L2O
(ii) l = –1, l = –2; v = M P ; v = M P 1 2 1 2
N0Q N1Q N1Q N1Q
LM 1OP LM 1OP LM 1OP
(iii) l = –1, l = –2, l = – 3; v = −1 ; v = M−2 P ; v = −3
1 2 MM−1PP M P MM 3PP
3 1 2
1
3

N Q N Q N Q 2

L 1O L 1 O L 1O
5.21 (b) l = –2, l = – 3, l = – 4; v = M−2 P ; v = M−3 P ; v = M−4 P
1 2 MM 4PP MM 9PP MM 16PP
3 1 2 3

N Q N Q N Q
LM 1 1 1O LM − j 0OP 1
2
1
2
5.22 (a) P = −1 + j1 −1 − j1 −1
MM − j 2 j 2 1PP P (b) Q = M j 0P 1
2
1
2
N Q M
N 0 0 1PQ
L 2e − e −e + e OP −t 2t −t 2t
5.23 1 2
L1O L1O
l = –1, l = 2; v = M P ; v = M P ; e = M
1 2
At
−t −t
2t 2t
NQ N Q
1 2 N2e − 2e −e + 2e Q
−t
L e − e − e + e OP −3 t −t L e − e −3t −t −3t −t −3t OP
At
3
2
1
2
3
2
3
2 e − e At
3
2
1
2
1
2
1
2
5.24 (a) e = M −t −3 t (b) e = M
−t −3t −t 3 −3t 1 −t −3 t
MN e − e − e + e PQ
1
2
1
2
1
2 MN− e + e − e + e
3
2
3
2 2 2
3
2 PQ
L 3e − 2e−2 t −3 t OP −2 t −3 t
L(1 + 2t ) e OP −2 t −2 t
At e −e 2t e At
5.25 (a) e = M −2 t −3 t
(b) e = M
−2 t −3 t − 2t −2 t
N−6e + 6e −2e + 3e Q N −2 t e (1 − 2t ) e Q
5.26 (a)
1
x02 x01

s –1 s –1

1 s –1 1 s –1
u x2 x1

–2 –2

1
764 Digital Control and State Variable Methods
X1( s ) 1/ 2 1/ 2 X1( s ) 1/ 2 −1/ 2
(b) = G11(s) = + ; = G12(s) = +
0
x1 s + 1 s + 3 x2 0
s+1 s +3
X2 (s) 1/ 2 −1/ 2 X2 (s) 1/ 2 1/ 2
= G21(s) = + ; = G22(s) = +
x10
s+1 s +3 x20
s +1 s +3
X1( s ) 1 X (s) 1
= H1(s) = ; 2 = H2(s) =
U (s ) s + 1 U (s ) s +1
1 e − ( x10 + x20 ) + e −3 ( x10 − x20 )
LM OP −t
LM1 − e OP
t t
(c) (i) x(t) = (ii) x(t) =
2 e − t ( x10 + x20 ) + e −3t (− x10 + x20 )
N Q −t
N1 − e Q
1 2
5.27 x1(t) = – e–2t + e–3t; x2(t) = 2(e–2t – e–3t ); x3(t) = –2(2e–2t – 3e–3t ); y(t) = x1(t)
3 3
1 3
5.28 (a) Asymptotically stable (b) y(t) = + 2e–t – e–2t
2 2
5 1
5.29 y1(t) = 3 – e– t – e –2t + e–3t; y2(t) = 1 + e–2t – 2e–3t
2 2
−6 0.5
LM 7 OP LM OP 28 1 1 LM OP
5.30 (a) A = ;b= ; c = [0 1] (b) y(t) = (1 − e −4 t ) − (1 − e −7t )
4 –5N 0 Q NQ 3 4 7 N Q
LM OP LM
x1 (1) 2.7183 − k OP
5.31 = for any k ¹ 0
N Q N
x2 (1) 2k Q
lt l t l t
k LM OP
5.32 (a) Modes: e 1 , e 2 , e 3 , l1 = –1, l2 = –2, l3 = –3 (b) x(0) = −3k ; k ¹ 0
MM PP
3k N Q
5.33 (b) x(0) =
k LM OP
;k ¹ 0
−k N Q
−t
− e −2 t
LM 2e e − t − e −2 t OP
5.34 eAt = −2 t
− 2e−t
N2e 2 e −2 t − e − t Q
A=
LM
0 1 OP
−2 −3
N Q
5.37 Controllable but not observable.
5.38 (i) Controllable but not observable (ii) Controllable but not observable
(iii) Both controllable and observable (iv) Both controllable and observable
(v) Both controllable and observable
5.39 (i) Observable but not controllable (ii) Controllable but not observable
(iii) Neither controllable nor observable
1
5.40 (i) G(s) = ; state model is not controllable
s+2
s+4
(ii) G(s) = ; state model is not observable
( s + 2)( s + 3)
1
5.41 (a) l1 = 1, l2 = – 2, l3 = –1; unstable (b) G(s) = ; stable
( s + 1)( s + 2)
 Answers to Problems 765

5.42 (a) A =
LM0 1OP ; b = LM0OP ; c = [10 0]
N0 −1Q N1Q
LM0 1 0OP LM0OP LM0 0 0 OP LM
20 OP
(b) A = 0
MM0 −20 −31PP ; b = MM01PP ; c = [20 10 0] (c) A= 1
MM0 0 −2 ; b = 10 ; c = [0
PP MM PP 0 1]
N Q NQ N 1 −3 Q 0N Q
1M
L z 2
z 1 OP
3
6.1 G(z) = M− (4 z + 1) z ( z + 3) z + 3 P z; D = z + 3z2 + 4z + 1
ΔM
N −z −1 z( z + 3) + 4 PQ

1 M
L −3z − 7z OP 2

2
H(z) =
Δ M
M−7z − 9z + 3PP
N 3z + 7 Q
2z + 2
6.3 G(z) =
z2 − z + 1
2

LM 2 z + 2 − 4 z − 14
4+
−30 OP
6.4 G(z) = M Δ Δ Δ ; D = z2 – z + 12
PP
MN z +Δ
1
2 −3z − 4
Δ
−2 +
−3z − 9
Δ Q
6.5 x1, x2, x3: Outputs of unit delayers, starting at the top of the column of unit delayers and proceeding to the
bottom.
LM1
2
1
4 2 1OP LM OP
F = M0 − 12−1 ; g = −1 ; c = [5
PP 6 –7]; d = 8
MN0 3 13
MM PP
2 Q N Q
6.6 x1, x2, x3: Outputs of unit delayers. x1 and x2 in first row, starting at the left and proceeding to the right.
LM 0 1 0OP LM1 0OP L0 2 0O L2 0O
F=
MM−123 −70 −62PP ; G = MM00 01PP ; C = MN0 0 1PQ ; D = MN0 1PQ
N Q N Q
6.7 (i) F = M
L0 1OP ; g = L0O ; c = [–1 –2]; d = 3
N − Q MN1PQ
2
3
1
3

LM0 0 OP L 0.5O 3
4
(ii) F = M1 0 1P ; g = M − 3P ; c = [0 0 1]; d = –2
MN0 1 −1PQ MMN 4PPQ
LM−1 0 0OP LM1OP
6.8 (i) F = 0 −2 0 ; g = 1 ; c = [–1 2 1]; d = 1
MM 0 0 −3PP MM1PP
N Q NQ
1
3
LM 1 0OP L0O
(ii) F = M0 1
1P ; g = M0P ; C = [5 –2 3]; d = 0
MN0 0 PQ MMN1PPQ
3
1
3
 766 Digital Control and State Variable Methods

0 0 −3
LM 0 OP LM OP
6.9 (i) F = 1 0 −7 ; g = 0 ; c = [0 0 1]; d = 0 (ii) F =
LM−1 0OP ; g = LM1OP ; c = [–2 7]; d = 0
0 1 −5
MM PP MM PP N 0 −2Q N1Q
N 0 Q NQ
L 0 1 0OP LM0OP
(iii) F = M 0
MM−3 −70 −51PP ; g = MM01PP ; c = [2 1 0]; d = 0
N Q NQ
6.10 k
F =M
L 1.5 − 0.5 (3) 0.5[(3) −1] OP
k k

k k
N−1.5[(3) − 1] − 0.5 + 1.5 (3) Q
17 22 25
6.11 y(k) = − (–0.2)k + (–0.8)k + ;k³0
6 9 18
k k k k
6.12 y1(k) = 5
F 1I + 10 − F 1I + 2; k ³ 0; y2(k) = 3
F 1I +2 −
F 1I + 1; k ³ 0
H 2K H 2K H 2K H 2K
6.13 (a) l1 = –1, l2 = –1, l3 = –2; Modes: (–1)k, k(–1)k – 1, (–2)k
LMk(−1) k −1 OP
k
(b) x(k) = M ( −1) PP
MN (−2) k
Q
0.2838z + 0.1485
6.14 (a) G(z) = Z [Gh0(s)Ga(s)] =
z 2 − 11353
. z + 0.1353

F=
LM0 1
;g=
0 OP LM OP
; c = [0.1485 0.2838]
−0.1353 11353
N . 1 Q NQ
(b) From controllable companion form continuous-time model:

F= M
L1 0.4323O
;g= M
L0.2838OP ; c = [1 0]
N0 0.1353 PQ N0.4323Q
6.15
L0.696
F= M
0.246 O
P ;g= M
L −0.021 OP ; c = [2 – 4]; d = 6
N 0.123 0.572 Q N 0.747Q
6.16 x(k + 1) = Fx(k) + g u(k) + g w(k); F = M
1
L1 0.1 OP ; g = LM0.005OP ; g = LM 0 OP
2 1 2
N0 0.99Q N0.1 Q N0.01Q
6.17 F= M
L0.741 0 OP ; G = LM259.182 0 OP ; C = LM1 0OP
N0.222 0.741Q N 36.936 259.182Q N0 1Q
L z + 0.7181 OP Y (z) 0.3679z + 0.2642
6.18 (a) 0.3679 M (b) = 2
N ( z − 1)( z − 0.3679 ) Q R ( z ) z − z + 0.6321

F= M
L 0 1OP ; g = LM0OP ; c = [0.2642 0.3679]
N−0.6321 1Q N1Q
0.4512z + 0.1809 L0 1 OP ; g = LM0OP ; c = [0.1809 0.4512]
6.19 (a) G(z) = Z [G (s)G (s)] =
h0 a ;F= M
2
z − 0.3679z N0 0.3679Q N1Q
(b) y& (t) = –y(t) + u(t – 0.4); x (k) = y(k); x (k) = u(k – 1); F = M
1 2
L0.3679 0.1809OP ; g = LM0.4512OP ; c = [1 0]
N 0 0 Q N 1 Q
 Answers to Problems 767

LM0.3679 0.2387 0.3935 0OP LM0OP

0 0 1 0 0
6.20 x (k) = x(k); x (k) = u(k – 3); x (k) = u(k – 2); x (k) = u(k – 1); F = M
1 2 3 4
MM0 0 0 1P
P ;g= M P
MM0PP
N0 0 0 0
P
Q N1 Q
LM1 T τ (T − τ /2 ) OP LM(T − τ ) /2 OP
D D D
2

6.21 x = y; x = y& ; x (k) = u(k – 1); F = 0 1 τ ;g= M T −τ PP ; c = [1 0 0] D

1 2 3 MM0 0 P
PQ N M 1
D

N 0 Q
6.23 (a) x1 = output of lag 1/s; x2 = output of lag 1/(s + 1); x3 = output of lag 1/(s + 2).

LM 7 − 1 T − e + 1 e −T
1−e −2 T1
(1 + e − 2e −T −2 T −T
) OP
4 2 4 2
M 1
x(k + 1) = M − + e − e −T 1 −2 T
e e −e −T −T −2 T
PP x(k)
MM 2 2 PP
1 1 −2 T −2 T
MN − 2 + 2 e 0 e
PQ
LM− 3 + 1 T + e − 1 e OP
−T −2 T
4 2 4
+M
M 1
−e + e
1
−T
PP r(k) −2 T

MM 2 1 1 2 PP
−2 T
− e
MN 2 2 PQ
6.24 x(k + 1) = M
L 0.4 0.233 OP x(k) + LM0.2 OP r(k); y(k) = [1 0]x(k)
N−0.698 −0.0972Q N0.233Q
LM 0.6 0.233 0.2 OP 0 LM OP
6.25 x(k + 1) = −0.465 −0.0972 0.233 x(k) + 0 r(k)
MM –1 PP MM PP
N 0 –2 Q 1 NQ
6.26 u(k) = 50e(k) – 41e(k – 1)
x1 = y; x2= y& ; x3(k) = e(k – 1)

L0.75 0.1 –0.205 0.25

OP LM OP
x(k + 1) = M –5 0.99 –4.1 x(k) + 5 r(k) MM PP
MM –1 PP
N 0 0 1Q N Q
6.27 (a) Both controllable and observable (b) Both controllable and observable
6.28 T = np ; n = 1, 2, ...
6.29 T ¹ n; n = 1, 2, 3, ...
1 1 1
6.30 (a) l1 = , l2 = (b) G(z) = (c) Controllable but not observable
4 2 1
z −
4
768 Digital Control and State Variable Methods

7.2 (a) k1 = 74, k2 = 25, k3 = 3 (b) x&$ = (A – mc) x$ + bu + my; mT = [3 7 –1]

(c) With reference to Fig. 7.7:

L x$ O La2 11 a1e OP = LM 00 1 0
b1
0
−2
OP L O LM OP LM OP
x$ e = M P; M 0 1 ; = 0 ;m=
PP MN PQ MM PP
N x$ Q Na3 e1 Aee Q MMN−6 −11 −6
be
1
17
Q N Q N Q
7.3 (a) k = [3 7 –1]; x& = (A – bk)x (b) mT = [74 25 3]
LM x& OP LM −6
3 0 1 OP LM x OP LM0OP La
3
11 a1e OP LM xx OP + LM b OP u
3
1
(c)
MM x&& PP = MM−−116
1 0 0 PP MM x PP + MM10PP u = MNa
1
e1 A ee Q MMN x PPQ Nb Q
1
e
Nx Q N
2 1 0 Q Nx Q N Q
2 2

With reference to Fig. 7.7:

x$ e =
LM x$ OP ; m = LM16OP
1

N x$ Q N 4Q
2

7.4 x&$ = (A – mc) x$ + Bu + m(y – du); mT = [–1 3]

7.5 With reference to Fig. 7.7:
LMa
OP LM OP LM OP LM OP
11 a1e 2 2 b1 1 y + z$2 LM OP
x$ e = z$2 ; = ; = ; m = 4.5; x$ =
Q N Q N Q NQ
Na e1 Aee −1 −1 be 0 y + 2 z$2 N Q
7.6
L OP ; b = LM9OP ; c = [0 1]
(a) A = M
0 9
(b) k = LM 2 3OP
N1 0 Q N0 Q N3 Q
k = LM
1 2O
(c) x&$ = (A – mc) x$ + bu + my; mT = [81 12] (d)
N 9 9 PQ
7.7 (a) A =
LM 0 2
1OP ; b = LM0OP ; c = [1 0]
2
(b) k1 = 3w 0 ; k2 = 4w0
N −ω 0 0Q N1 Q
(c) x&$ = (A – mc) x$ + bu + my; mT = [20w0
2
99w 0 ]
(d) With reference to Fig. 7.7:

x$ e = x$ 2 ;
LMa
11 a1e OP = LM 0 2
1 OP ; LMb OP = LM 0 OP ; m = 10w
1
0
Na e1 Aee Q N −ω 0 0 Q Nb Q N1Q
e
7.8 (a) k = [29.6 3.6]

(b) x&$ = (A – mc) x$ + bu + my; mT = [16 84.6]

(c) With reference to Fig. 7.9:
U ( s) . s + 3690.72
77816
= D(s) = 2
−Y ( s ) s + 19.6 s + 151.2
LM
x&1 0OP 1 0LM 0 x1 OP LM OP
x&2 20.6 0 −29.6 −3.6 x2
(d) & = MM
x$1
PP
16 0 −16
MM 1 x$1
PP MM PP
x&$
MN PQ MN
84.6 0 −93.6 −3.6 x$
2
PQ MN 2
PQ
 Answers to Problems 769

7.9 (a) A =
LM0 1OP ; b = LM0OP ; c = [1 0] (b) k = [1 2]
N0 0 Q N1 Q
(c) x&$ = (A – mc) x$ + bu + my; mT = [5 25]
(d) With reference to Fig. 7.9:
U ( s) 40.4( s + 0.619)
= D(s) = 2
−Y ( s ) s + 6.414 s + 33.07
(e) With reference to Fig. 7.7:

x$e = x$ 2 ;
LMa a OP = LM0 1OP
11 1e

Na A Q N 0 0 Q
e1 ee

LMb OP = LM0OP ; m = 5
1

Nb Q N1 Q
e
U ( s) 8.07( s + 0.62)
(f) = D(s) =
−Y ( s ) s + 6.41
7.10 (a) k1 = 4; k2 = 3; k3 = 1; N = k1 (b) mT = [5 7 8]
7.11 k = [–1.4 2.4]; ki = 1.6
7.12 k1 = 4; k2 = 1.2; k3 = 0.1
7.13 k1 = 1.2; k2 = 0.1; k3 = 4
7.14 mT = [5 6 5]
7.15 KA = 3.6; k2 = 0.11; k3 = 0.33
7.16 KA = 40; k2 = 0.325; k3 = 3
7.17 k1 = a2 /b; k2 = (a1 – a)/b; N = k1
7.18 k1 = – 0.38; k2 = 0.6; k3 = 6
7.19 (a) k1 = 3; k2 = 1.5
(b) For a unit-step disturbance, the steady-state error in the output is 1/7.
(c) k1 = 2; k2 = 1.5; k3 = 3.5, Steady-state value of the output = 0
7.20 (a) k = [3 1.5] (b) N = 7
(c) For a unit-step disturbance, the steady-state error in the output is 1/7.
(d) k1 = 2; k2 = 1.5; k3 = 3.5
10
7.21 (a) K = 0.095; N = 0.1 (b) For A + d A = – 0.6, w (¥) = r
10.1
(c) K1 = 0.105; K2 = 0.5
7.22 k1 = – 4; k2 = – 3/2; k3 = 0

7.23 x$ (k + 1) = (F – mc) x$ (k) + Gu(k) + m[y(k) – du(k)]; mT = LM 3 −

11
0 OP
N2 16 Q
7.24 k = [– 0.5 –0.2 1.1]; x(k + 1) = (F – gk)x(k)
 770 Digital Control and State Variable Methods

7.25 x (k + 1) = F x$ (k) + gu(k)

x$ (k + 1) = x (k + 1) + m[y(k + 1) – c x (k + 1)]; mT = [6.25 – 5.25]
LM x (k + 1)OP LM 0
2 0 1 OP LM x (k )OP L0O
2
Lf f1e OP LM xx ((kk))OP + L g O u(k)
2

PP MM x (k ) PP + MM01PP u(k) = MN f
11 1
7.26
MM x (k + 1) PP = MM−01.2
1 0 0
−0.5 11
1
e1 Fee Q MMN x (k )PPQ MNg PQ
1
e
N x (k + 1)Q N
3 . Q N x (k )Q MN PQ
3 3

x$ e (k) = [ x$1 (k) x$3 (k)]T

x$ e (k + 1) = (Fee – mf1e ) x$ e (k) + (ge – mg1)u(k) + (fe1 – m f11)y(k) + my(k + 1); mT = [0 1.1]

7.27 (a) k = LM111 −

18 OP
N 76 19 Q
(b) x$ (k + 1) = (F – mc) x$ (k) + bu(k) + m[y(k) – du(k)]; mT = [8 –5]
LM x (k + 1) OP LM 2
1 −1 −5.84 3.79OP LM x (k) OP
1

MM xx$ ((kk ++ 11))PP = MM−18

2 1 −4.38
8 −11.84
2.84
P M x (k )
2
−5.21P M x$ (k ) P
P
1 1
MN x$ (k + 1)PQ MN−5
2 −5 −0.38
PM P
8.84Q N x$ ( k )Q
2

7.28 (a) F =
LM 0 1O L0 O
; g = M P ; c = [1 0]
N−0.16 −1PQ N1 Q
(b) k1 = 0.36; k2 = –2.2
(c) x$ 2 (k + 1) = (–1 – m) x$ 2 (k) + u(k) – 0.16 y(k) + m y(k + 1); m = –1
(d)
z–2
–

7.29 (a) x(k) = Qz(k); Q =

LM1 −1OP ; u = – 0.36 z (k) + 2.2 z (k)
1 2
N0 1Q
(b) z$2 (k + 1) = (–1 – m) z$2 (k) + u(k) – 0.16 y(k) + m y(k + 1); m = –1
U (z) 2.56(1 + 0.1375z −1 )
(c) = D(z) =
−Y (z) 1 − 2.2 z −1

7.30 (a) F=
LM1 0.1 OP ; g = LM0.005OP
N0 1 Q N0.1 Q
(b) k1 = 13; k2 = 3.95
(c) x$ (k + 1) = (F – mc) x$ (k) + gu(k) + my(k); mT = [2 10]
U (z) 65.5( z − 0.802 )
(d) = D(z) = 2
−Y (z) z + 0.46z + 0.26
 Answers to Problems 771

7.31 (a) F =
LM1 0.0952 OP
;g=
LM
0.00484 OP
; c = [1 0]
N0 0.905 Q 0.0952N Q
(b) k1 = 105.1; k2 = 14.625
(c) x$ (k + 1) = (F – mc) x$ (k) + gu(k) + my(k); mT = [1.9 8.6]
(d) x$ 2 (k + 1) = (0.905 – 0.0952m) x$ 2 (k) + (0.0952 – 0.00484m)u(k) – my(k) + my(k + 1); m = 9.51
7.32 (a) y(k + 1) = 0.368y(k) + 0.632u(k) + 0.632w(k) (b) K = 0.3687; N = 1.37
(c) Steady-state error for a unit-step disturbance is 0.73. (d) K1 = 0.553; K2 = 2.013
8.1 Asymptotically stable in-the-large.
8.2 Unstable.
8.3 Equilibrium state xe = [2 0]T is asymptotically stable.
8.4 K > 0
8.5 0 < K < 8
8.6 Asymptotically stable.
8.7 Asymptotically stable.
8.8 K = [1 3]
8.9 K = [1 2]
8.10 Sufficient conditions not satisfied.
u = – [–1 4]x; optimal closed-loop system is asymptotically stable.
8.11 Asymptotically stable optimal solution does not exist.
8.12 u = – x1 – 0.23x2 + r; r = desired output yd = 1
8.13 u = – x1 – 6 x2
8.14 u = – x1 – x2; x$& = (A – MC) x$ + Bu + My; MT = [5 4]
8.15 k1 = 2 ; k2 = 0.275; J0 = 93.2375
8.16 r = 0.1; K = [3.1623 0.5968]; Poles: – 0.6377, – 4.9592
r = 0.01; K = [10 1.7082]; Poles: – 2.2361, – 4.4721
r = 0.001; K = [31.6228 4.3939]; Poles: – 4.6970 ± j3.0921

8.17 (a) K= 2 –1 (b) N = 2

(c) Steady-state error to unit-step disturbance is 1/ 2 (d) K = K1 = 1

(e)

+ + + +

– –
772 Digital Control and State Variable Methods
10
8.18 (a) K = 0.095; N = 0.1 (b) For A + dA = – 0.6, w(¥) = r (c) K = 0.105; K1 = 0.1
10.1
8.19 u(k) = – 0.178x(k)
8.20 (a) K = 0.277 (b) y(¥) = 0.217r (c) N = 1.277
0.9
8.21 (a) K = 0.2 (b) N = 0.45 (c) For F + dF = 0.3, x(¥) = r
11
.
8M − 8M
9.2 ; 1 rad/sec; y(t) = sin t
π π
9.3 0.3; 10 rad/sec
9.4 D < 0.131
9.6 4.25; 2 rad/sec, stable limit cycle
9.7 3.75; 1 rad/sec
9.8 (a) Stable node; (0, 0) point in (y, y& )-plane (b) Stable node; (1,0) point in (y, y& )-plane
(c) Unstable focus; (2,0) point in (y, y& )-plane
9.9 Singularity at (0, 0) is either a stable node or a stable focus, depending upon the magnitudes of a and K.
Singular point (p,0) is a saddle point.
9.10 (i) Singularity (1,0) in (y, y& )-plane is a centre
(ii) Singularity (1,0) in (y, y& )-plane is a stable focus.
x1 + x&1 + 7x1 = 0
9.11 (a) For – 0.1 < x1 < 0.1, && (b) Isocline equations:
–7x1
x1 + x&1 + 0.7 = 0
For x1 > 0.1, && x2 = ; – 0.1 < x1 < 0.1
m+1
– 0.7
x1 + x&1 – 0.7 = 0
For x1 > 0.1, && x2 = ; x > 0.1
m+1 1
0.7
x2 = ; x < – 0.1
m+1 1
(c) A singular point at the origin
x1 + x&1 = 0
9.12 (a) For – 0.1 < x1 < 0.1, && (b) Isocline equations:
x1 + x&1 + 7 (x1 + 0.1) = 0
For x1 < – 0.1, && m = – 1; – 0.1 < x1 < 0.1
–7x1 – 0.7
For x1 > 0.1, &&
x1 + x&1 + 7 (x1 – 0.1) = 0 x2 = ; x1 < – 0.1
m +1
–7x1 + 0.7
x2 = ; x1 > 0.1
m+1
(c) Singular point at (x1 = ± 0.1, x2 = 0)
x1 + x&1 + 0.7 sgn x1 = 0
9.13 (a) && (b) Isocline equations:
–0.7
x2 = ; x1 > 0
m+1
0.7
x2 = ;x <0
m+1 1
(c) No singular points
 Answers to Problems 773

x1 + x&1 = 0
9.14 (a) For – 0.1 < x1 < 0.1, && (b) Isocline equations:
For x1 > 0.1, &&
x1 + x&1 + 0.7 = 0 m=–1 ; – 0.1 < x1 < 0.1
– 0.7
For x1 < – 0.1, &&
x1 + x&1 – 0.7 = 0 x2 = ; x1 > 0.1
m +1
0.7
x2 = ; x < – 0.1
m+1 1
(c) No singular points
9.15 (a) &&
x1 + 0.1 sgn x&1 + x1 = 0 (b) Isocline equation:
– x – 0.1 sgn x2
x2 = 1
m
(c) Singular point a t (x1 = m 0.1, x2 = 0)
9.16 Steady-state error to unit-step input = – 0.2 rad; Maximum steady-state error = ± 0.3 rad.
9.17 Deadzone helps to reduce system oscillations, and introduces steady-state error.
9.18 Saturation has a slowing down effect on the transient.
9.20 The system has good damping and no oscillations but exhibits chattering behaviour. Steady-state error is
zero.
9.21 (b) (i) Deadzone provides damping; oscillations get reduced.
(ii) Deadzone introduces steady-state error; maximum error = ± 0.2.
(c) By derivative-control action, (i) settling time is reduced, but (ii) chattering effect appears.

9.23 x1 = e, x2 = e& ; Switching curve: x1 = – x2 +

x2 F
ln 1 + 2
x2 I
| x2 |
GH | x2 | JK
9.24 |x1| < 1; origin is the equilibrium state.
9.25 Asymptotically stable in-the-large; origin is the equilibrium state.
9.26 Asymptotically stable; origin is the equilibrium state.
9.27 1 > 2x1 x2; origin is the equilibrium state
10.1 KP = 100; KD = 14.14
10.4 a1 = 0.8187; b1 = 1.0877
10.5 a1 = 0.9845; a2 = – 0.1222; b1 = 0.0579; b2 = 0.1011
10.6 a1 = 1.5; a2 = 2; b1 = 1; b2 = 3
11.1 (a) w1 = 1, w2 = 3 (b) w1 = 0.86442, w2 = 2.8892 (c) 0.21626
11.4 w = 0.36; w0 = 0.666
11.6 (b) w(1) = [0.974 –0.948 0 0.526]T;
w(2) = [0.974 –0.956 0.002 0.531]T
w(3) = [0.947 –0.929 0.016 0.505]T
(c) w = [1.0518 –0.943 0.0155 0.5005]T
11.7 (b) w10 = 1.00043; w11 = 3.00043;
w12 = 4; w20 = – 5.9878; w21 = 6.0123;
w22 = 5; v0 = – 3.9078; v1 = 2.012;
v2 = 4.0061
(c) With initial weights, y$ = 0.51; With updated weights, y$ = 0.5239
 774 Digital Control and State Variable Methods
12.1 (a) Supporting Interval: [1 3] (b) Support: [1 3]
a-cut interval: 2 − 0.5 2 + 0.5 a-cut: [1.5 2.5]
1
12.2 (a) s = ; m = 2; Support; unbounded (– ¥ ¥); a-cut: [1.53 2.47]
2π
(b) Support: (– ¥ ¥); a-cut: [1 3]
12.3 a = 3 ft; b = 6 ft; c = 9 ft; Support: [3 9]; Cross point: 4.5
R| x +1
; −1≤ x ≤ 4
5
12.4 (a) Am = |S 1 ;4 ≤ x < 5 (b) It is normal and convex.
~ x−9
|| ;5≤ x ≤ 9
T −4
12.5 (a) (i) Yes (ii) Yes (iii) No
(b) Support: [64 78]; Cross points: 68, 75; a-cut α =0.2 : [65.6 76.8]; a-cut α =0.4 : [67.2 75.6]

12.8 (i) m A (4) = 0.75; (ii) mB (3) = 0.5; (iii) m ~A´ ~B (x, y) = 0.5; (iv) m A®B (x, y) = 0.5;
~ ~ ~ ~
12.11 z* = 6.76
12.12 1857.28
⎧ ⎛2 ⎞ ⎛1 ⎞⎫
12.13 mC ¢ (z) = max ⎨ min ⎜ , μC1 ( z ) ⎟ , min ⎜ , μC2 ( z ) ⎟ ⎬
~ ⎩ ⎝ 3 ~ ⎠ ⎝ 3 ~ ⎠⎭
z*COA = 4.7

12.14 (i) 0.25, 0.62

*
{ ( ~ ) (
(ii) magg(z) = max min 0.25, μ PL ( z ) , min 0.62, μ PM ( z )
~ )}
(iii) z = 53.18
12.15 34.40
12.16 Rules 2, 3, and 4 are active; u* = 2.663
12.17 67.4
12.18 2.3
 Index

A learning rate 649

momentum term 653
Acceleration error constant 198
multilayer network 651–653
Ackermann’s formula 401, 424
single-layer network 649–651
Activation function; 627
weight initialization 653
bipolar 629
Backward difference approximation of
Gaussian 632
derivatives 90–93
hyperbolic tangent 631
Bandwidth; 206–207, 209
linear 632
on Nichols chart 220
log-sigmoid 632
Batch-mode training 651
sigmoidal 631
Bell-shaped fuzzy set 699
tan-sigmoid 632
Berbalat’s lemma 584
unipolar 629
Bias (NN) 630
Adaptive control system;
BIBO stability 61–66, 329
model-reference 581–587, 599
Bilinear transformation; 95–98
self-tuning 593–594, 599
with frequency prewarping 213–214
A/D converter; 22, 26
Bode plots:
circuits 29–30
lag compensation 216–217
model 115
lag-lead compensation 217
Adjoint of a matrix 259
lead compensation 215–216
Aliasing 75–76
Alpha-cut; fuzzy set 700 C
Analytic function 51
Cancellation compensation 231, 237, 238–239
ANFIS 723–726
Canonical state models;
Antecedent; IF-THEN rule 690
controllability form 327
Anti-aliasing filter 23, 80
controllable companion form 283–285, 354–
Artificial neural network
355
(see Neural network)
first companion form 283–285, 354–355
Artificial neuron
Jordan form 287–291, 356–358
(see Neuron model)
observability form 328
Asymptotic stability 66, 329, 452, 549
observable companion form 285–286, 355–
Autonomous systems 547
356
B second companion form 285–286, 355–356
Cascade programming
Backlash nonlinearity; 510
of controllers 130–131
describing function 516–518, 526
Cartesian product 703
Backpropagation training; 649–654
Cayley-Hamilton theorem 281–282
batch-mode 651
Center of area defuzzification 718–719
gradient descent method 649
Centre point; phase portrait 538
incremental-mode 651
Characteristic equation 55, 294
 776 Index
Chattering 601 Crisp set 695
Chromosome (GA) 727 Crossover (GA) 727, 732–733
Classical logic 695 Cross point; fuzzy set 700
Coding 26 Cross product 703
Companion form of state model: Cross site (GA) 733
controllable 283–285, 354–355 Current state observer 426
first form 283–285, 354–355 Cylindrical extension; fuzzy relation 707
observable 285–286, 355–356
second form 285–286, 355–356 D
Companion matrices 285
D/A converter; 22, 26
Compensation:
circuits 27–29
cancellation 231, 237–239
model 115
lag on Bode plots 216–217
Damping ratio; correlation with
lag on root-locus plots 232–233
peak overshoot 202
lead on Bode plots 215–216
phase margin 209
lead on root-locus plots 230–232
resonance peak 209
Complement; fuzzy set 703
Data-based modelling
Complimentary strips in s-plane 84
fuzzy 719–723
Compositional rule of inference 708
NN 655–659
Composition; max-min 708
Deadbeat control systems 432
Computational time delay 24
Deadbeat state observer 432
Computer control systems (see Digital
Dead-time 122–124, 364–366
control systems)
Condition number of a matrix 262 Deadzone nonlinearity 509
Conclusion; IF-THEN rule 690 describing function 519
Conjunction; fuzzy set 704 phase portrait 565–566
Consequent; IF-THEN rule 690 Decoding 27
Constant-wn loci 87 Defuzzification 718–719
Constant-z loci 86–87 Describing function method 511–513
Controllability: stability analysis 519–523
definition 318, 372 table 519
tests 319–321, 324–325, 333, 372, 374, 377 Detectability 474
Controllability canonical form of state model 327 Determinant of a matrix 258
Controllability loss due Diagonal matrix 256
to sampling 375–376 Difference equations 38–39
Controllability matrix 320, 372, 377 Digital controller implementation 126–129
Controllable companion form cascade realization 130–131
of state model 283–285, 354–355 direct realization 129–131
Controllable eigenvalues (poles) 327 nonrecursive 128, 132
Controller tuning parallel realization 131–132
based on GA 735, 737–740 recursive 128, 129
based on process reaction curve 140–143 Digital control systems; 4, 24
based on ultimate gain and period 138–139 advantages of 21
digital PID 145–146 configuration 3–4, 23–24
Convex fuzzy set 699 implementation problems 22–23
Convolution sum 40 Digital PID controllers 106–107
Coulomb friction 510, 540–542 position algorithm 143
 Index 777
velocity algorithm 145 Feedback linearization 575–581
tuning 145–146 Feedback network (see Recurrent network)
Digital signals (see Discrete-time signals) Feedfarward action
Direct digital control 9 state-feedback servo 418–419, 428–429, 469
Direct digital design 114 Feedforward neural network 633
Direct method of Lyapunov 551, 552–554 dynamic map 657
Direct programming input-output map 635, 638
of controllers 129–131 multilayer 635–638
Discrete-time impulse 31, 44 single layer 633–635
Discrete-time signals Filter
sinusoidal sequence 32, 46 anti-aliasing 23, 80
unit-ramp sequence 45 finite impulse response 128
unit-sample sequence 31, 44 infinite impulse response 128
unit-step sequence 32, 45 low pass 77
Discretization 80–98 nonrecursive 128, 132
Distributed computer control system 10 recursive 128, 129
Disturbance rejection 209–211 zero-order hold 78–79
Dominant poles 203–206 Final value theorem
Duality 407 z-transform 51–52
Finite impulse response system 128
E Firing strength; IF-THEN rule 716, 721
Eigenvalue assignment First companion form of
(see Pole-placement by state feedback) state model 283–285, 354–355
Eigenvalues 262, 294 First-harmonic approximation 512
controllable 327 First method of Lyapunov 548
observable 329 First-order hold 70
Eigenvectors 297 Fitness function 730
computation 298–303 Focus; phase portrait 538
generalized 302 Forward difference approximation of
Eligibility trace 614 derivatives 91–92
Encoder; shaft 152–155 Fourier series 511–512
Encoding 26 Frequency folding (see Aliasing)
Epoch; NN training 651 Frequency prewarping 213–214
Equilibrium state 451, 536, 546 Frequency response 58–59
Equivalence transformation 274 specifications 206–209
Error constants Frequency warping 91, 98, 213–214
acceleration 198 Full-order state observer 404–408
position 197 current observer 426
velocity 198 prediction observer 425
Euclidean matrix norm 262 Function approximation (NN) 654
Euclidean vector norm 261 Function approximation (SVM) 676–679
Fuzzification 716
F Fuzzy cartesian product 704
Feedback control systems Fuzzy complement 703
nonunity feedback 15, 195 Fuzzy conjunction 704
unity feedback 16, 195 Fuzzy implication 705
 778 Index
Fuzzy inference; 706 Roulette wheel parent selection 731–732
compositional rule 708 Global stability 453, 549
Fuzzy intersection 703 Gradient descent method (NN) 649
Fuzzy logic 695 Grammian matrix 263, 264
Fuzzy logic control 709–719
GA-based tuning 735, 737–740 H
Fuzzy modelling 719–723 Hessian matrix 458
Fuzzy propositions 695 Hidden layer (NN) 633
Fuzzy relation 703 Hierarchical control systems 10–11
composition 707 Hold operation
cylindrical extension 707 first-order 70
projection 707 zero-order 70–72
Fuzzy rules (IF-THEN) Homogeneous state equations
Mamdani rules 706–708 solution 305, 368
singleton rules 708 Hyperbolic tangent activation (NN) 631
Sugeno rules 708–709
Fuzzy singleton 700 I
Fuzzy sets
IAE performance index 459
a-cut 700
Identification of models
bell-shaped 699
least squares method 587–593
convex 699
fuzzy–based 719–723
cross point 700
NN-based 655–659
Gaussian 699
Identity matrix 257
normal 699
IF-THEN rule
singleton 700
antecedent 690
support 700
conclusion 690
trapezoidal 699
consequent 690
triangular 699
firing 716
Fuzzy union 703
implication 705
G premise 690
Implication; fuzzy set 705
Gain margin 209 Impulse; discrete-time 31, 44
Gaussian activation (NN) 632 Impulse-invariance method for
Gaussian fuzzy set 699 discretization 82–86
Generalized eigenvectors 302 Impulse modulator model of sampler 41–43
Generalized predictive control 594–598 Impulse response model 39–41
Genetic algorithm (GA) Incremental-mode training (NN) 651
chromosome 725 Indefinite scalar function 552
coding 729 Inference; fuzzy system 706, 708
controller tuning 735, 737–740 Infinite impulse response system 128
crossover 727, 732–733 Inherent nonlinearities 510
cross site 733 Initialization (NN) 653
fitness function 730 Inner product of vectors 261, 264
mating pool 731 Instability theorem; Lyapunov 555–556
mutation 727, 734 Integral action
reproduction 732–734 state-feedback servo 420–421,429–430, 469
 Index 779
Intersample ripples 180–181, 375 Linear dependence of vectors 263, 264
Intersection; fuzzy set 703 Linear independence of vectors 263, 264
Inverse model (NN) 660–662 Linear system stability tests
Inverse of a matrix 259 Jury 67–69
Inverted pendulum 314–316 Lyapunov 452–457
ISE performance index 460 Local stability 453, 549
Isoclines method 532–535 Logic
ITAE performance index 459 classical 695
ITSE performance index 460 fuzzy 695
Log-sigmoid activation (NN) 632
J Lowpass filter 77
Jacobian matrix 558 Luenberger state observer 405
Jordan canonical form of state model 287–291 Lyapunov equations 455, 457
Jump resonance 508 Lyapunov functions
Jury stability criterion 67–69 for linear systems 454– 457
for nonlinear systems 556–561
K Lyapunov instability theorem 555–556
Lyapunov stability analysis
Kernal functions 675–676
direct method 551, 552–554
KKT conditions 668
first method 548
Kohonen’s SOM 647
non-autonomous systems 584–585
Krasovskii method 557–559
second method 548
L M
Ladder diagram 171–178
Machine learning 620–622
Lag compensation on
Mamdani architecture; FLC 709–719
Bode plots 216–217
Mamdani fuzzy rules 706–708
root-locus plots 232–233
Mapping of s-plane to z-plane 43
Lag-lead compensation 217
constant-wn loci 87
Lead compensation on
constant-z loci 86–87
Bode plots 215–216
Mapping of z-plane to w-plane 213
root-locus plots 230–232
Mapping of w-plane to z-plane 213
Learning; machine 620–622
Marginal stability 66
Learning in NN; 640–642
Mating pool (GA) 731
reinforcement 645–646
Matrix
supervised 642–645
adjoint 259
unsupervised 646–649
condition number 262
Least squares estimation 587–593
determinant 258
recursive 591–593
diagonal 256
Limit cycles 542
eigenvalues 262, 294
Linear activation (NN) 632
Grammian 263, 264
Linear SVM 665–674
Hessian 458
Linearization
identity 257
feedback linearization 575–581
inverse 259
first-harmonic approximation 512
Jacobian 558
Taylor’s series 270
negative definite 265
 780 Index
negative semidefinite 265 Negative semidefinite matrix 265
nonsingular 259 Negative semidefinite scalar function 552
norm; Euclidean 262 Neural Network
norm; spectral 262 dynamic map 657
null 257 feedforward 633–638
nullity 297 for control 662–665
orthogonal 262 for function approximation 654–655
partitioned 260 for model identification 655–659
positive definite 265 input output map 635, 638
positive semidefinite 265 multilayer perceptron 635–638
rank 259, 263 RBF 638–639, 648
singular 259 recurrent 639
singular values 262 single layer perceptron 633–635
skew-symmetric 258 Neural Network Modelling 655–659
symmetric 258 Neural Network Training
trace 259 (see Backpropagation training)
transpose 257 Neural Q-earning 611–613
triangular 257 Neuron:
unit 257
artificial 626
zero 257
biological 625
Matrix exponential 303–305
model 630
properties 304
Neuro-control
Matrix exponential evaluation by
feedforward-feedback 662
Cayley-Hamilton technique 309–310
inverse model 660–662
inverse Laplace transform 306
numerical algorithm 361 model-reference adaptive 662–665
similarity transformation 307–308 Neuro-fuzzy systems 723–726
Matrix Riccati equation 472, 482 Nichols chart; bandwidth
Max-min composition 708 determination 220
Measurement noise 211 Nodal point; phase portrait 538
Membership functions (see Fuzzy sets) Non–autonomous systems 584–585
MIMO systems; definition 13, 35–36 Nonhomogeneous state equations
Minor-loop feedback 7 solution 305, 368
Model reference adaptive Nonlinearities
control 581–587, 599 backlash 510
Modes 325 deadzone 509
Momentum gradient algorithm 653 describing function table 519
Multilayer NN 635–638 friction 510
Multiloop control systems 10 on-off 510
Multiple-rate sampling 24 saturation 509
Multivariable control Nonlinear SVM 674–676
systems 329–330, 376–377 Nonlinear system stability
Mutation (GA) 727, 734 describing function 519–523
Lyapunov functions 556–561
N Nonminimum-phase transfer
Negative definite matrix 265 function 215
Negative definite scalar function 552 Nonrecursive controller 128, 132
 Index 781
Nonsingular matrix 259 Parameter optimization 458
Nonsingleton fuzzy system 708 Partial state feedback 484–488
Nonunity feedback system 15, 195 Partitioned matrix 260
Norm: Peak overshoot 201
Euclidean; matrix 262 correlation with damping ratio 202
Euclidean; vector 261 Peak resonance 207
spectral; matrix 262 correlation with damping ratio 209
Normal fuzzy set 699 Peak time 201, 202
Nullity 297 Perceptron 630
Null matrix 257 Performance index (also see Quadratic
Nyquist stability criterion 207–208 performance index)
IAE 459
O ISE 460
Observability ITAE 459
definition 319, 373 ITSE 460
tests 322–325, 333–374, 377 Performance specifications
Observability canonical form of state model 328 frequency-response 206–209
Observability loss due to sampling 375–376 time-response 200–206
Observability matrix 323, 373, 377 Permanent-magnet stepping motors 157–159
Observable companion form of state Phase margin 209
model 285–286, 355–356 correlation with damping ratio 209
Observable eigenvalues (poles) 329 Phase-plane analysis 527–529
Observer (see State observer) Phase portraits 527
On-off controllers 510, 543–546 Construction by analytical method 529–532
describing functions 514–516, 523–526 Construction by isocline method 532–535
phase portraits 539–540 Phase trajectory 527
Optimal servo system; with integral control 469 PID controller analog, 134–137
Optimal state estimators (see Optimal PID controller, digital (see Digital PID
state observers) controller)
Optimal state observers 468–469 Pole-placement by state feedback 397–401
Optimal state regulator 470–475, 479–482 Ackermann’s formula 401
Optimization of parameters 458 multi-input systems 403–404
Order of a system 36, 54 Poles and zeros 44, 55
Orthogonal matrix 262 Pole-zero cancellation 231, 237, 238–239
Orthogonal vectors 262 Position error constant 197
Orthonormal vectors 262 Position form of digital PID algorithm 143
Output feedback (see Partial state feedback) Positive definite matrix 265
Output layer (NN) 633 Positive definite scalar function 552
Output regulator 466 Positive semidefinite matrix 265
Positive semidefinite scalar function 552
P Prediction state observer 425
Predictive control 594–598
Parallel programming of controllers 131–132 Premise; IF-THEN rule 690
Parameter estimation Prewarping 213–214
least squares method 589–591 Primary strip in s-plane 84
recursive 591–593 Process reaction curve 140–142
 782 Index
Programmable logic controller; 12, 163 correlation with damping ratio 209
applications 179 Riccati equation 472, 482
building blocks 166–171 Rise time 200, 201
ladder diagram 171–178 Robot manipulator control 576–581, 604–607
programming 179 Robust control systems 13, 211–212, 603
Projection; fuzzy relation 707 Robust observers 417
Proper transfer function 278 Root locus method 224–230
construction rules table 225–226
Q Root locus plots
Q-earning 608–613 lag compensation 232–233
Quadratic forms of scalar functions lead compensation 230–232
negative definite 265 Root sensitivity 182–183
negative semidefinite 265 Roulette-wheel parent selection (GA) 731–732
positive definite 265
positive semidefinite 265 S
Quadratic performance index Saddle point; phase portrait 538
ISE 460 Sampled-data control systems 4, 24
output regulator 466 state model 359–361
state regulator 466, 479 transfer function 115–120
Quantization errors 22, 24–27 Sample-and-Hold:
Quarter-decay ratio response 139 circuit 71–72
model 70–71
R Sampler impulse modulator model 41–43
Ramp sequence 45 Sampling
Rank of a matrix 259, 263 multiple rate 24
RBF network 638–639, 648 uniform 25
Realization of a transfer function: Sampling effects; 22–23, 80–81
cascade programming 130–131 on controllability and observability 375–376
direct programming 129–131 on stability 121–122
first companion form 283–285 on steady-state error 200
Jordan form 287–291 Sampling frequency 41
parallel programming 131–132 Sampling period; 41
second companion form 285–286 selection 81–82
Rectangular rules for integration 93 Sampling rate 41
Recurrent networks 639 Sampling theorem 77
Recursive controller 128, 129 Satellite attitude control system 395–396, 529
Recursive least squares estimation 591–593 Saturation nonlinearity; 509
Reduced-order state observer 410–412, 426–427 describing function 561–564
Regulator; definition 4 Scalar product of vectors 261, 264
Reinforcement learning control 608–615 Second companion form of state model
Relaxed system 39 285–286, 355–356
Reproduction (GA) 732–734 Second method of Lyapunov 548
Resolvent algorithm 280–281 Self-organizing map 647–648
Resolvent matrix 280 Self-tuning control 593–594, 599
Resonance frequency 207, 209 Sensitivity analysis 182–183, 211–212
Resonance peak 207 Separation principle 413–414, 427–428
 Index 783
Servo design with state feedback local 453, 549
with feedforward control 418–419, 428–429, marginal 66
469 zero-input 66–67
with integral control 420–421, 429–430, 469 Stability tests for linear systems
Servo system; definition 4 Jury 67–69
Set-point control system Lyapunov 452–457
definition 4 Stability tests for nonlinear systems
Settling time 201, 202 describing function 519–523
Shaft encoder 152–155 Lyapunov 551–554
S/H device Stabilizability 474
circuit 71–72 State diagram 275–276, 353
model 70–71 Statefeedback 394–395
Sigmoid activation (NN) 631 State model 35–37, 270, 329, 377
Similarity transformation 274 conversion to transfer
Single layer NN 633–635 function 276–278, 353, 377
Singleton equivalence with transfer
fuzzy set 700 function 325–374
rules 708 sampled plant 359–361
Singular matrix 259 system with dead-time 364–366
Singular points 536 State models; canonical (see canonical state
center 538 models)
focus 538 State observers 404, 468–469
node 538 current 426
saddle 538 deadbeat 432
vortex 538 full-order 404–408
Singular values of a matrix 262 prediction 425
Sinusoidal sequence 32, 46 reduced-order 410–412, 426–427
SISO systems; definition 12, 37 robust 417
Skew-symmetric matrix 258 State observer design through
Sliding-mode control 600–607 matrix Riccati equation 463–469
s-norm; fuzzy sets 702 State regulator 395, 432,467
Soft - computing 620–621 State regulator design through
Solution of matrix Riccati equation 470–475, 479–482
homogeneous state equations 305, 368 pole-placement 400–401, 423–424
nonhomogeneous state State transition equation 371
equations 312–313, 368 State transition matrix; 305, 368
Specifications (see Performance specifications) properties 305–306
Spectral norm of a matrix 262 State transition matrix evaluation by
s-plane to z-plane mapping 43 Cayley-Hamilton technique 309–310, 370
Stability definitions inverse Laplace transform 306
asymptotic 66, 329, 452, 549 inverse z-transform 368
BIBO 61–66, 329 numerical algorithm 361
global 453, 549 similarity transformation 307–308, 369
in-the-large 549 Steady-state error 196–199
in the sense of Lyapunov 452, 548 sampling effects 200
in-the-small 549 Steady-state error constants (see Error constants)
 784 Index
Step-invariance method for poles and zeros 44, 45
discretization 88–89 proper 278
Step motors (see Stepping motors) sampled-data systems 115–120
Stepper motors (see Stepping motors) systems with dead-time 122–124
Stepping motors zero-order hold 70–71
in feedback loop 8 Transportation lag (see Dead-time)
interfacing to microprocessors 161–163 Transpose of a matrix 257
permanent magnet 157–159 Trapezoidal fuzzy set 699
torque-speed curves 160–161 Trapezoidal rule for integration 96
variable-reluctance 159–160 Triangular fuzzy set 699
Step sequence 32, 45 Triangular matrix 257
Strictly proper transfer function
Tuning of process controller (see
Suboptimal state regulator 484–488
Controller tuning)
Sugeno architecture, data-based
Type number of a system 197
modelling 719–723
Type-1 system 199
Sugeno fuzzy rules 708
Type-2 system 199
Supervised learning (NN) 642–645
Support; fuzzy set 700 Type-0 system 198
Support vector machines 665
U
function approximation 676–679
hard-margin linear 665–671 Ultimate gain 138
nonlinear 674–676 Ultimate period 138
soft-margin linear 671–674 Uniform sampling 25
Sylvester’s test 266 Union; fuzzy set 703
Symmetric matrix 258 Unit circle in z-plane 43
System identification Unit delayer 33, 55
least squares method 587–593 Unit matrix 257
fuzzybased 719–723 Unit-ramp sequence 45
NN-based 655–659 Unit-sample sequence 31, 44
Unit-step sequence 32, 45
T Unit vector 262
Tan-sigmoid activation (NN) 632 Unity feedback systems 16, 195
Taylor series Universal approximation property (NN) 654
t-conorm; fuzzy sets 702, 708 Universe of discourse 695, 696
Temporal difference learning 613–615 Unsupervised learning (NN) 646–649
Time-response specifications 200–206
V
t-norm; fuzzy sets 702, 708, 721
Trace of a matrix 259 Variable gradient method 559–561
Tracking control systems Variable reluctance stepping motors 159–160
definition 4 Variable structure control 542–546, 600–604
Training NN (see Backpropagation training) Vectors
Transfer function inner product 261, 264
definition 53 linearly dependent 263–264
equivalence with state model 325–329 linearly independent 263–264
nonminimum-phase 215 norm; Euclidean 261
order 54 orthogonal 262
 Index 785
orthonormal 262 time-delay approximation 79, 146
scalar product 261, 264 transfer function model 70–71
unit 262 Zeros and poles 44, 55
Velocity error constant 198 Ziegler-Nichols tuning
Velocity form of digital algorithm 145 based on process reaction curve 140–143
Vender Pol’s oscillator 507, 527, 533 based on ultimate gain and period 138–139
Vortex point; phase portrait 538 z-plane
s-plane mapping 43
W
unit circle 43
Warping 91, 98, 213–214 w-plane mapping 213
Weights (NN) 630 z-plane synthesis 237–239
w-plane; 213 z-transfer function (see Transfer function)
z-plane mapping 213 z-transform
w-transform 213–215
definition 41, 43
Z final value theorem 51–52
inverse 50
Zero-input stability 66–67 pairs 47–48
Zero matrix 257
pairs for systems with dead-time 124
Zero-order hold
properties 45, 46
circuit 71–72
Shifting theorems 48
filtering characteristic 78–79