Foundations and Trends

R
in Systems and Control
Resilient Control in Cyber-Physical
Systems
Countering Uncertainty, Constraints, and
Adversarial Behavior

Suggested Citation: Sean Weerakkody, Omur Ozel, Yilin Mo and Bruno Sinopoli (2020),
“Resilient Control in Cyber-Physical Systems”, Foundations and Trends
R
in Systems and
Control: Vol. 7, No. 1-2, pp 1–252. DOI: 10.1561/2600000018.

Sean Weerakkody
Carnegie Mellon University
[email protected]
Omur Ozel
George Washington University
[email protected]
Yilin Mo
Tsinghua University
[email protected]
Bruno Sinopoli
Carnegie Mellon University
[email protected]

This article may be used only for the purpose of research, teaching,
and/or private study. Commercial use or systematic downloading
(by robots or other automatic processes) is prohibited without ex-
plicit Publisher approval.
Boston — Delft
 Contents

1 Introduction 3
1.1 Applications . . . . . . . . . . . . . . . . . . . . . . . . . 10

2 Modeling 21
2.1 System Identification . . . . . . . . . . . . . . . . . . . . 21
2.2 Challenges in Modeling Cyber-Physical Systems . . . . . . 24
2.3 State Space Models . . . . . . . . . . . . . . . . . . . . . 25
2.4 Hybrid Systems . . . . . . . . . . . . . . . . . . . . . . . 29

3 Networked Control Systems 33

3.1 Estimation in Lossy Networks . . . . . . . . . . . . . . . . 34
3.2 Control in Lossy Networks . . . . . . . . . . . . . . . . . . 49
3.3 Designing Sensor Network with Resource Constraints . . . 63
3.4 Event-based control . . . . . . . . . . . . . . . . . . . . . 86

4 Secure Cyber-Physical Systems 103

4.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . 104
4.2 Attacks on CPS . . . . . . . . . . . . . . . . . . . . . . . 111
4.3 Robust Structural Design of Control Systems . . . . . . . 142
4.4 Active Detection of Attacks on CPS . . . . . . . . . . . . 159
4.5 Resilient Estimation in CPS . . . . . . . . . . . . . . . . . 186
5 Privacy 210
5.1 Data Privacy . . . . . . . . . . . . . . . . . . . . . . . . . 211
5.2 Differential and Inference Privacy in Average Consensus . . 214
5.3 Cryptography Based Privacy . . . . . . . . . . . . . . . . 225
5.4 Further Reading . . . . . . . . . . . . . . . . . . . . . . . 227

6 Conclusions 229

References 232
Resilient Control in Cyber-Physical
Systems
Sean Weerakkody 1 , Omur Ozel2 , Yilin Mo3 and Bruno Sinopoli4
1 Carnegie Mellon University; [email protected]
2 George Washington University; [email protected]
3 Tsinghua University; [email protected]
4 Carnegie Mellon University; [email protected]

ABSTRACT
Cyber-Physical Systems (CPS), the amalgamation of sophis-
ticated sensing, communication, and computing technolo-
gies, applied to physical spaces, have become intrinsically
linked to society’s critical infrastructures. Indeed, CPS find
applications in energy delivery systems, intelligent trans-
portation, smart buildings and health care. Within these
systems, technological advances have enabled mankind to
improve their ability to both accurately monitor large scale
systems and precisely manipulate their behavior in order
to achieve complex local and global objectives. Nonetheless,
the opportunities created by CPS are met with significant
burdens and challenges, threatening the resilience of these
systems to both benign failures and malicious attacks.
In this monograph, we provide a comprehensive survey of
intelligent tools for analysis and design that take fundamen-
tal steps towards achieving resilient operation in CPS. Here,
we investigate the challenges of achieving reliable control
and estimation over networks, particularly in the face of
uncertainty and resource constraints. Additionally, we ex-
amine the threat of bad actors, formulating realistic models

Sean Weerakkody, Omur Ozel, Yilin Mo and Bruno Sinopoli (2020), “Resilient
Control in Cyber-Physical Systems”, Foundations and Trends R
in Systems and
Control: Vol. 7, No. 1-2, pp 1–252. DOI: 10.1561/2600000018.
2

to characterize adversaries as well as systematic tools to

detect and respond to attacks. Finally, we include a brief
introduction to the problem of privacy in CPS, providing
both measures to describe and techniques to preserve the
confidentiality of sensitive information.
 1
Introduction

Cyber-physical systems (CPS) are computationally capable systems

that directly interact with a physical environment and allow people
to intelligently and efficiently manage physical processes. CPS are the
foundation of key infrastructures such as the smart grid, water distri-
bution systems, and waste management. Their role in transportation,
smart buildings, and medical technologies are also burgeoning as new
application areas are discovered. We refer the reader to Lee (2008),
Rajkumar et al. (2010), Poovendran (2010), Kim and Kumar (2012),
and Johansson et al. (2014) for additional information on the reach of
CPS in today’s applications.
CPS are enabled by technologies which perform sensing, computing,
and communication. In particular, CPS leverage sensing technologies
to gather relevant data about physical systems. In transportation this
could for instance be the position and velocity of vehicles. Alternatively,
in medical technologies, this may be the heart rate or blood pressure of
a patient. Combined with a mathematical model of a system’s physical
dynamics, sensing can enable accurate state estimation and predic-
tion. This in turn allows the monitoring of physical processes. Sensing
technologies have significantly improved. We can sample systems more

3
4 Introduction

frequently and with less delay. Additionally, sensing devices are in many
cases cheap and economically viable. The availability of cheap and
accurate sensing allows the designer to better understand physical pro-
cesses by obtaining larger numbers of spatial and temporal samples. An
example in this regard is the increased presence of phasor measurement
units (PMUs) in the power grid (Abur and Exposito, 2004). We note
that modern PMU technology has significantly changed the operation of
the electricity grid. In particular, the high sampling rates and accuracy
of voltage phasor measurements have changed state estimation from a
static problem to a dynamic problem.
In addition to monitoring physical processes, it is typically desirable
to physically manipulate a system to achieve some objective. In a
waste management system, a relevant task would be to treat and
purify the wastewater. Alternatively, in smart buildings we wish to
regulate the environment (i.e. using HVAC systems) in an energy efficient
manner. Cyber-physical systems allow us in many cases to automate this
process using computing technologies. The intelligent control of physical
systems is generally a time sensitive task. Thus, a key to incorporating
CPS is improvement in the processing speed of our computers. Today,
programmable logic controllers (PLCs) and microcontrollers are able
to quickly process sensory information and automatically implement
an intelligent algorithm for control. The speed at which this can be
done has allowed humans to explore new frontiers. As an example, the
ability to safely incorporate safe driving cars to transportation systems
is in part a result of the vast computational abilities of the embedded
systems in today’s vehicles.
Finally, a sophisticated communication infrastructure allows oper-
ators to control cyber-physical systems remotely while also enabling
them to reliably control large scale systems. Many systems have transi-
tioned from wired to wireless communication technologies, which allows
for ease of maintenance and installation, lower costs, as well as au-
tomation in geographically disparate systems. As an example, wireless
communication technologies play a major role in supervisory control
and data acquisition (SCADA) systems, see, e.g., Cardenas et al. (2009).
A SCADA system is a hierarchical system, which enables the supervi-
sory management of a control system. The lowest layer consists of field
 5

devices such as sensors and actuators, which directly interact with the
physical environment. Remote terminal units (RTUs) and PLCs are
often used to implement autonomous local control. These units typically
interface with both field devices such as pumps, valves, and switches
as well as a centralized supervisory control layer which monitors the
system. SCADA systems are regularly seen in the smart grid as well
as water distribution and waste management systems. Communication
technologies allows RTUs to interface with human operators at SCADA
systems in real time. This allows operators to make high level control
decisions remotely in a timely fashion. This capability is especially
important when monitoring at the supervisory layer raises an alarm,
which requires immediate operator attention. Communication technolo-
gies not only allow devices and components to interface with central
operators, but also each other. Local communication among field devices
can enable distributed control. Here, autonomous controllers/agents
share information and act to achieve a larger task. Distributed control
can be used to achieve formations in aerial vehicles and platoons in
ground transport.
Unexpected challenges arise when accounting for the tight inter-
action of computing elements with the physical plant in CPS. Unlike
normal IT infrastructures, the operations of CPS are often safety critical
(Lee, 2008; Rajkumar et al., 2010; Giani et al., 2008). For example,
malfunctioning teleoperated robots in surgery may harm or possibly
kill patients. Likewise, blackouts on the electricity grid may disrupt
vital services. Thus, operators are obligated to ensure these systems
perform resiliently. Complicating the matter is the time sensitive nature
of CPS. To ensure that the dynamics of a physical process are well
regulated, CPS must be monitored and acted upon frequently. In this
monograph, we aspire to identify significant challenges, which hinder the
successful operation of cyber-physical systems. To this end, we consider
several proposed tools and methodologies aimed towards addressing
these fundamental problems.
First, in section 2 we consider the problem of modeling CPS.In
control systems, an accurate numerical representation of a plant is often
a crucial component to developing intelligent algorithms for automation,
with provable mathematical properties. These models can be developed
6 Introduction

from first principles. For instance, Newton’s laws can be used to de-
scribe the dynamics of vehicles while Maxwell’s equations can be used
to derive dynamical equations associated with mathematical genera-
tors. Alternatively, we can utilize big data and in particular system
identification/machine learning techniques to obtain effective models of
our systems. We briefly discuss system identification in subsection 2.1.
Cyber-physical systems pose a particular challenge due to the inherent
diversity of the systems being considered (Derler et al., 2012). They
not only contain a physical plant, which needs to be modeled like a
traditional control system, but also have heterogeneous hardware and
software systems which enable computing and data transfer. The chal-
lenges of modeling CPS are detailed in subsection 2.2. We then look
at specific classes of models. In addition to examining traditional state
space, LTI, and stochastic systems in subsection 2.3, we will address
modeling CPS through a brief discussion of hybrid systems in subsection
2.4.
Even with a precise and accurate model of CPS, operators must
account for sources of uncertainty and how they impact subsequent
analysis and design. As an example, in section 3 we will study net-
worked control systems, focusing on achieving feedback control over
stochastic, resource constrained, communication networks. While tran-
sitioning from wired to wireless communication technologies can reduce
costs and improve efficiency, reliability may be sacrificed. Packets con-
taining sensory or control data may be delayed or dropped over the
communication network. In a cyber-physical system, the availability of
real-time data is often essential for correct and reliable operation. Sen-
sor packet drops leads to inadequate monitoring and feedback control.
Input packet drops prevent corrective commands from being delivered
to the plant. As communication failures can significantly affect the
functionality of CPS, operators must carefully model and account for
their presence through robust analysis and design. We will discuss the
design of robust feedback controllers in CPS with sensor and input
drops respectively in subsection 3.1 and 3.2. In these cases, we will
additionally arrive at fundamental conditions on network reliability,
which allow the aforementioned algorithms to successfully stabilize CPS.
 7

Improvements in automation and efficiency are often cited as benefits

of incorporating cyber-physical systems to our long standing infrastruc-
tures. Even then, operators must be careful to ensure the economic
viability of these tasks. A traditional goal is to maximize the perfor-
mance of a system subject to a constraint on available resources. For
example, in subsection 3.3, we will briefly investigate resource con-
straints as it applies to sensors. Sensors in CPS are generally small
heterogeneous devices, which are subject to power constraints, band-
width constraints, and topology constraints. We will consider problems
of sensor scheduling and event triggered estimation with the objective
of maximizing system performance while meeting these constraints. In
addition to event triggered estimation, in subsection 3.4, we will explore
the dual problem of event based control.
At the heart of this monograph, in section 4, we will consider the
security of cyber-physical systems. While it is important to achieve
resilience to systematic and benign failures, which for instance can
occur due to operator error, normal wear and tear, or environmental
conditions, the bulk of our attention will be placed on malicious ad-
versarial scenarios. As cyber-physical systems are intrinsically linked
to our critical infrastructures, there exist ample motivation to target
them. Attacks on transportation CPS can lead to car accidents while
attacks on CPS associated with water treatment and management could
damage the environment or contaminate the water supply. Additionally,
attacks on the grid can disrupt vital services due to blackouts and
attacks on medical CPS can cause injury or even death to patients.
Next generation cyber-physical systems also create opportunities for
adversaries. Introducing wireless technologies into control systems allow
remote attackers to perform man in the middle attacks. Moreover, the
incorporation of heterogeneous subsystems and components provides
numerous attack surfaces for adversaries. The internet of things (IoT),
creates additional advantages for an attacker. CPS which leverage the
IoT utilize existing (and possibly insecure) networking infrastructures,
particularly the internet, to enable communication and remote pro-
cessing (for instance through cloud computing). Finally, there exists
precedence for attacks. Perhaps, the best known attack on a cyber-
physical system is the Stuxnet attack, which was a malicious worm
8 Introduction

which targeted uranium enrichment facilities in Iran and was able to

disable approximately one thousand centrifuges (Langner, 2011). In
subsection 4.1, we go into deeper detail regarding the motivation for
studying cyber-physical system security.
Next, in subsection 4.2, we will discuss common adversarial models
in CPS, describing potential attacker’s in terms of their knowledge,
capabilities, and potential strategies. Here, we pay special attention to
stealthy attack strategies, which allow an attacker to act on a system
without being recognized, thus eliminating reactive defensive counter-
measures. After, we describe potential mechanisms for achieving security
in CPS. The ultimate goal is for the system to remain operational, even
in the presence of an attacker. We argue the first step of this process
is detection. As an example, in subsection 4.3, we will evaluate how
sensor and link placement can be used structurally to ensure properties
of attack detectability and identifiability. Additionally, in subsection
4.4, we will introduce tools for active detection, which enable operators
to recognize and isolate classes of harmful and stealthy attacks, by
intelligently perturbing the system.
Beyond detection, we wish to recover from and resiliently respond to
attacks on our control system so that we can achieve graceful degradation
of system performance under attack. We remark that directly designing
resilient control laws to counter attacks is application dependent. Instead,
we argue that a necessary step to achieve desirable system performance
in the presence of an attacker is to perform resilient state estimation,
the subject of subsection 4.5. Indeed, resilient state estimation allows a
remote defender to maintain understanding of the system state under
attack, even when a subset of inputs and outputs are compromised. This
ability to perform resilient estimation in turn enables resilient control.
Specifically, a defender can incorporate reliable state information when
designing appropriate countermeasures (including a resilient feedback
control law) to remedy a cyber-physical system.
Finally, as noted in the title of this monograph, we aim to counter
adversarial behavior in CPS. While section 4 considers methods to
counter attacks which actively affect the operations of a control system,
passive adversaries in a CPS can also cause significant harm to society. In
particular, in the age of big data, copious amounts of information, much
 9

of which can be sensitive, is used to efficiently and effectively control

CPS. For instance, power consumption data aids in demand prediction,
transportation data reveals information about traffic patterns, and
medical information can enable preventive treatments. However, in the
wrong hands this type of data can reveal sensitive information about a
user’s routines, travel habits, and preexisting conditions. As a result, in
order to truly consider the impact of adversarial behavior in CPS, we
argue that one must also pay close attention to notions of privacy. An
introduction to some concepts in privacy is given in section 5. Here, we
wish to provide some intuition about how important and useful data
can be leveraged in a CPS without leaving citizens and users vulnerable
to the actions of a passive, information collecting, attacker. To begin,
we consider data privacy in subsection 5.1. We will discuss notions
of differential privacy and inference privacy. In this respect, we will
consider the problem of average consensus and discuss mechanisms that
achieve these notions of privacy in subsection 5.2. Finally, in subsection
5.3, we will give a brief overview of cryptography based privacy.
We remark that this is far from the only monograph to examine
cyber-physical systems. To date dozens of books on CPS have been
published. Many of these texts are for more detailed in their discussion
of applications, modeling, and specific architectures. Additionally, while
not a focus of this monograph, several books have studied concepts of
verification and validation in CPS. The main contribution of this text
relative to most other works is the highly mathematical, model aware
approach it takes to analysis and design when dealing with problems
of robust and resilient control in CPS. Our aim is to provide readers
with an introduction to challenges in this arena and discuss the basic
tools that have been used to address these problems. Of course, not all
concepts in resilient cyber-physical systems can be covered in this text.
However, to aid the interested reader, several further reading subsections
have been included to provide additional pointers to applicable and
related research.
In the rest of this section, we discuss several applications of CPS in
moderate detail. Here, we will emphasize application specific problems
that highlight challenges for ensuring resilience in CPS.
10 Introduction

1.1 Applications

In this subsection, we discuss several applications of cyber-physical sys-

tems, specifically the smart grid/energy management systems, medical
technologies, transportation, and water treatment/distribution. While
a comprehensive analysis of each infrastructure is out of scope, we aim
to highlight the role cyber-physical technologies play in these systems
and summarize key challenges which can threaten resilience.

1.1.1 Smart Grid and Energy Management CPS

The electric grid is a massive infrastructure, composed of a variety of
subsystems with different owners and a diverse range of regulators. This
large and complex system is inevitably prone to key challenges and
vulnerabilities. This includes withstanding the failure of components
and transmission lines, matching generation to demand, and preserving
the environment.
The development of a smart grid in particular aims to address
the major challenges and inefficiencies that exist in the current infras-
tructure through the use of advanced information, computing, and
communication technologies, smarter devices, and economically viable
renewable resources (Farhangi, 2010; Amin and Wollenberg, 2005; Fang
et al., 2012). For instance, the introduction of an advanced metering
infrastructure (Mohassel et al., 2014) and dynamic pricing will enable
demand response (Albadi and El-Saadany, 2008). This along with dis-
tributed generation can reduce the cost of electricity for consumers
as well as decrease peak demand. Additionally, the widespread use of
phasor measurement units (PMUs) allows for wide area monitoring via
dynamic state estimation as well as automatic control to improve real
time efficiency. Furthermore, increased information and better predictive
tools will help society in leveraging clean renewable resources such as
wind and solar power. The smart grid is a preeminent example of a CPS
where generation, transmission, and distribution subsystems comprise
the physical system, while sensors collecting data, networks routing
data, and computers processing data constitute the cyber system.
1.1. Applications 11

At a smaller scale, we consider energy management systems in build-

ings. Kleissl and Agarwal (2010) note that 70% of our total energy
consumption is spent in buildings, which also generate 40% of green-
house gases. Hence the application of information and communication
technologies to achieve smart buildings has significant potential. Modern
buildings could be viewed as cyber-physical systems that consist of heat
control, water distribution, airflow, and security subsystems interacting
closely via the usage of embedded sensing and control systems. Kleissl
and Agarwal (2010) in particular examine opportunities to optimize
energy consumption by both occupants and information processing
equipment and provides recommendations for buildings to achieve zero
net energy usage. The role of humans, especially in residential buildings
can not be underestimated. Information technologies can allow humans
to make better decisions in smart buildings. For instance Aksanli and
Rosing (2017), after obtaining a model to capture the relationship be-
tween activities of residents in a house and total power consumption,
use a human-behavior-centric scheduling method to achieve significant
energy savings and peak demand reduction in residential CPS.
As an aside, cyber-physical technologies play an important role in
managing energy usage in data centers. Data centers have shown rapid
growth in energy consumption (Koomey, 2011). With data collection and
storage only increasing, special care must be taken to efficiently manage
electricity usage in data centers. Parolini et al. (2012) considers the
problem of energy management in data centers using a cyber-physical
system approach. In particular, the authors provide a coordinated
strategy leveraging cooling and information technologies to achieve both
energy efficiency and a high quality of service.
Unfortunately their exists ample motivation for attackers to target
the smart grid. First, there exists economic benefits for potential at-
tackers. On one hand, an adversary can physically tamper with smart
meters in order to reduce electricity bills. Alternatively, attacker’s who
participate in the electricity market can elicit a profit by intelligently
compromising sensor measurements (Xie et al., 2010). Attackers may
also perturb the grid as a prank or for far more nefarious reasons in-
cluding terrorism. In particular, an attacker targeting the smart grid
will affect critical life-saving resources.
12 Introduction

There exists a precedent for attacks on the grid, perhaps most

notably the attack on the Ukraine power grid in 2015 (Pultarova,
2016). Here, hackers were able to deliver the BlackEnergy3 malware
to a SCADA system operating the grid months before commencing a
physical attack. The attackers were able to harvest valid credentials
and perform reconnaissance to ascertain appropriate targets. Finally, on
December 23, 2015, attackers remotely carried out an attack on the grid,
tripping breakers and blocking remote access from system operators.
As a result tens of thousands of customers lost power over a period
of several hours. The attackers also performed a telephone denial of
service to cut off communication between consumers and providers and
used the KillDisk malware to destroy data.
Mo et al. (2012a) mention confidentiality and privacy as another
relevent issue that arises due to the use of information technologies.
Energy use information stored in smart meters can leak personal informa-
tion about consumer habits and activities (McDaniel and McLaughlin,
2009). For instance, it is possible to intuit general information such as
when a user is at home or awake or even very specific information, such
as when a consumer is watching television. As many consumers consider
this information to be sensitive, we observe a critical tradeoff between
the benefits provided by data collection (improved demand prediction,
efficient use of resources), and the resulting loss in privacy (Le Ny and
Pappas, 2014). Differentially private filtering as discussed by Le Ny and
Pappas (2014) can help to address such tradeoffs by aggregating data
in a manner which provides strong privacy guarantees.

1.1.2 Medical CPS

Cyber-physical technologies have had a direct impact on medical sys-
tems. The management and operation of medical cyber-physical systems
have been positively influenced by miniaturized sensing implants and
actuating platforms, energy harvesting, in-body and on-body networks
and new fabrication methods such as 3D printing. Additionally, improve-
ments in communication and computing allow autonomous coordination
of medical devices, both microscopically via nanorobots and macro-
scopically in the operating room. Precise control also has enabled new
1.1. Applications 13

methods for device placement and drug delivery. We expand upon these
topics below.
Traditionally, intelligent sensing and actuation has found applica-
tions in scenarios that involve wearable devices and implantable devices
such as pacemakers and defibrillators. In particular, mobile monitoring
of vital signals and physical activities obviate the need of doctors to be
physically present to diagnose the health of individual patients. Schirner
et al. (2013) suggests that embedded sensors which measure human cog-
nitive activity are enablers of human in the loop CPS. Specifically, the
development of human and machine interfaces can improve interactions
with assistive robots, which perform actions for the benefit of a person
with a disability and allow for enhancements in intelligent prostheses,
restoring function to amputees.
Similarly, there is now also growing interest towards in-body and on-
body sensor networks that can measure activity and athletic performance
based on body state indicators such as heart and breathing rate, blood-
sugar level and skin temperature. In this respect, developing energy
harvesting technologies (such as RF energy harvesting or thermoelectric
energy harvesting using body heat) enable battery free operation and
ease of implementation in various types of body sensor applications. RF
energy harvesting is a well known technique for increasing the lifetime
of implantable devices (Ho et al., 2014). In addition, thermoelectric
generators, kinetic harvesters and solar technology are also being used
in body sensor networks to harvest energy in wireless bio-sensor devices
(Mitcheson, 2010).
It is argued by Lee and Sokolsky (2010) that monitoring and control
in medicine could greatly benefit from newly developed cyber-physical
technologies. Real time embedded closed-loop control could facilitate
immediate diagnostic evaluation of vital signals and make constant care
possible. For example, Lee and Sokolsky (2010) discuss how intelligent
coordination between x-ray machines and ventilators during an operation
can save patient lives. Specifically, to currently obtain good images
(without patient motion) during an operation, a ventilator must be
paused, thus preventing lung movement. However, patients have died in
cases where the ventilator would not restart. An intelligent alternative
involving precise control would be to enable automatic coordination
14 Introduction

between the x-ray and the ventilator. The x-ray would take images
when it detects the end of a breathing cycle. As a result, the ventilator
does not need to be turned off.
Additionally, the use of computing, sensing, and communication
technologies can reduce humans erros. Cyber-physical technologies
promise to minimize human mistakes by automating various medical
tasks both in clinical scenarios and operation room practice. For instance,
Lee and Sokolsky (2010) consider patient-controlled analgesia and argues
that it can benefit from feedback control. In this process, infusion pumps
are commonly used to deliver opioids for pain management before and
after surgery. Current technological safeguards suchs as drug libraries
and programmable limits can be insufficient in safely addressing pain
management. The authors propose a closed-loop control system with
a supervisor to monitor patient data for the early signs of respiratory
failure. The automated supervisor can stop infusions and sound an alarm
in case of an adverse event. We also remark the role of nanorobots in
the development of new drug delivery methods, see, e.g., Douglas et al.
(2012). This technology promises to deliver drugs to a targeted region in
the body and hence minimize the risks and possible side effects caused
by its use.
Unfortunately, without proper care, cyber physical technologies can
negatively impact the security and reliability of medical devices. First,
medical devices may be subject to a significant failure risk with poten-
tially catastrophic impacts on patients. Alemzadeh et al. (2012) argue
that faulty monitoring devices could cause serious injury and death.
An over reliance on autonomous monitoring and treatment in a faulty
scenario could result in harm to a patient, which could have otherwise
been prevented with a doctor in the loop. In addition, the dependence
of cyber-physical systems on information technology make them more
vulnerable to cyber attacks. Alemzadeh et al. (2013) report that tele-
operated robots are vulnerable to malicious adversaries. In particular,
this work considers attackers who install malware to strategically affect
robots during surgery. To detect and mitigate such attacks, Alemzadeh
et al. (2013) devises a model-based analysis framework using the dy-
namics of the surgical robot. This framework is utilized to determine if
a command is trustworthy before execution.
1.1. Applications 15

An enhanced information technology infrastructure also creates

significant privacy concerns. Patients often wish to keep medical infor-
mation private often due to a perceived stigma associated with various
health conditions. A release of such information can violate the trust
patients have in medical professionals and the system as a whole. As
such, the privacy of an individuals mental/physical health along with
the treatment they receive is mandated by law through the Health
Insurance Portability and Accountability Act (HIPPA). Unfortunately,
increased data collection in next generation and state of the art medical
systems have made personal medical information more vulnerable. The
research community has been active in attempting to prevent medical
information from leaking. As an example, Kocabas et al. (2016) pro-
vides a detailed survey of encryption schemes to enable privacy at data
collection, data aggregation, cloud storage, and action layers of medical
CPS.

1.1.3 Transportation related CPS

Transportation infrastructures, including both terrestrial and aerial
systems have been heavily influenced by CPS. Most obviously, improve-
ments in embedded sensing and control have allowed self driving cars
and unmanned aircrafts to surface. In addition, advanced wireless com-
munication methods made in-vehicle and vehicle-to-vehicle coordination
possible. This enhanced networking along with improvements in cloud
computing and cellular wireless technologies has opened up the possi-
bility of intelligent city wide and highway traffic control. With global
travel now a common necessity, the problem of intelligent aerial traffic
management has become increasingly important. On a smaller scale, as
advances are being made in drone technology, city wide aerial traffic
control may also pose a significant challenge.
Qu et al. (2010) argue that cyber-physical technologies have cre-
ated opportunities for intelligent transportation systems which reduce
traffic, improve safety, and increase sustainability. The authors envi-
sion a unified platform which integrates pedestrians, vehicles, roadside
infrastructures, traffic management centers, sensors, and satellites to
achieve safety and efficiency. Noting the capability of wireless commu-
16 Introduction

nication technology to transfer information rapidly in mobile systems,

Qu et al. (2010) analyze several candidate technologies for intra-vehicle
communication as well as vehicle-to-vehicle and vehicle-to-infrastructure
communication. Additionally, the future of transportation faces the chal-
lenge of integrating self-driving cars to traditional traffic. Self-driving
vehicles leverage precise sensing technologies such light detection and
radar (LIDAR) and GPS/INS and intelligent algorithms which perform
simultaneous localization and mapping (SLAM) (Wolcott and Eustice,
2014). Autonomous vehicles have the potential to improve safety and
increase efficiency.
Work and Bayen (2008) consider the role of mobile phones in the
way transportation CPS is evolving. It is argued in Work and Bayen
(2008) that cell phones can be used as traffic sensors in dynamic envi-
ronments. The utility of mobile devices is propelled by their ubiquity,
built infrastructure, and diverse capabilities. In particular, visualization
and computation platforms in cellphones enable crucial feedback in the
operation of transportation CPS. In Work et al. (2008), automotive
CPS are considered and in-vehicle and among vehicles data collection
and processing opportunities are set forth including forms of social net-
working and environmental monitoring. Possible benefits expected to be
gained by such integration opportunities, including more energy efficient
and human-centric operation, which remove a human from information
acquisition tasks and leave them with higher lever decisions.
Sampigethaya and Poovendran (2013) propose an aviation CPS
framework consisting of aircrafts, passengers, air traffic management,
and airports. The authors observe that advances and innovations
recorded in aviation design, flight operation, and airport management,
which mainly rely on information and computational capabilities on
the ground and during flight, will enable a new frontier for this infras-
tructure. As an example, aircrafts are beginning to employ integrated
modular avionics (IMA)-based architecture, which yield software sys-
tems with lower power consumption and higher integration. Moreover,
coupling higher level flight management systems with flight control en-
ables route optimization in the presence of uncertainty and constraints
while providing decision support for pilots. Improvements in air traf-
fic management will improve air-to-ground interactions. For instance
1.1. Applications 17

weather information can be collected by an aircraft, processed on the

ground, and delivered to other planes which will travel through the same
airspace. Tactical decisions can be made by the pilots of these aircrafts
accordingly. Finally, at airports, cyber-physical technologies will improve
surface operations, turnaround time at gates, and passage/baggage flow.
The resilience of transportation systems to failures and attacks is
critical for the safety of the public. To achieve widespread adoption
of next generation technologies (for instance autonomous vehicles that
leverage both vehicle to vehicle and vehicle to infrastructure communi-
cation), resilient architectures must be developed which can withstand
benign faults as well as malicious attacks. To investigate this matter
further, we consider the example of vehicular platoons. In a vehicle
platoon, several closely spaced vehicles follow a leader. The vehicles
leverage radar technology and (in certain cases) vehicle to vehicle com-
munication to share relative distances and velocities, as well as planned
accelerations. By autonomously reducing inter-vehicle distance and
relative velocities, platoons increase throughput and save fuel.
Nonetheless, platoons are vulnerable to attacks. Amoozadeh et al.
(2015) notes that messages between vehicles can be falsified, spoofed or
replayed by attackers while jamming attacks can disrupt communication
entirely. System level attacks can also tamper with vehicle hardware or
software. This can be done both at the manufacturing state or remotely
(Miller and Valasek, 2015). Gerdes et al. (2013) demonstrates how such
attacks can be subtly used to increase energy expenditures of vehicles
from anywhere between 20 − 300%. More malicious adversaries can use
control of a single vehicle to manipulate the actions of all other vehicles
in a stream and destabilize a platoon (Dadras et al., 2015). DeBruhl
et al. (2015) for instance demonstrates a particularly powerful attack
where a vehicle communicates that it is going to accelerate to the vehicle
behind it, only to brake suddenly. The authors demonstrate that careful
model-based detection and control schemes are needed to detect and
respond to such an attack safely while simultaneously benefiting from
the typical advantages of platooning in the absence of an attack.
The privacy of location data has been frequently emphasized in the
context of transportation CPS (Qu et al., 2010; Work and Bayen, 2008;
Work et al., 2008; Sampigethaya and Poovendran, 2013; Amoozadeh
18 Introduction

et al., 2015; Hoh et al., 2006). Hoh et al. (2006) for instance notes that
location monitoring services in next generation traffic systems can allow
drivers to be be tracked. Privacy can be corrupted by eavesdroppers
on the network, attackers who install spyware, or malicious insiders
with access to a traffic monitoring server. Significant information can
be gleaned from tracking a user. As noted by Hoh et al. (2006) ,
one can learn about the health of a driver if they frequently visit a
doctor/specialist or political leanings from visits to activist organizations.
Perhaps more worrisome is the home identification of particular drivers.
As such, the privacy of transportation data requires significant attention.

1.1.4 Water Based CPS

Sewage or wastewater treatment allows communities to remove contam-
inants from wastewater, enabling this water to be returned to nature
with minimal environmental consequence or in some cases, be reused.
A cyber-physical approach to water treatment improves automation
in this system (Department-of-Homeland-Security, 2015). Enhanced
sensing and monitoring will allow operators to anticipate failures and
thus increase reliability. Moreover, it will enable real time feedback
control at collection stations and pumping stations. As an example,
intelligent sensing and control can be utilized to monitor and finely tune
the environment of rotating biological contactors. Rotating biological
contactors consist of bacteria which can break down contaminants in
water, but require very specific environmental conditions (which can be
managed by SCADA systems) to function properly. Additionally, Konig
et al. (2015) discuss how SCADA and IoT based technologies will allow
cities to implement decentralized wastewater treatment, an initiative
which will significantly reduce energy consumption, decrease long term
costs, and increase the recycling of water.
Water distribution has also benefited from improvements in sensing,
computing, and control (Mutchek and Williams, 2014). Smart water
meters can monitor real time pressure and flow. This enables these
sensors to automatically detect costly leaks/breakages. Moreover, smart
meters enable consumers to control their water habits in much the way
that demand response has been considered in the smart grid. This can be
1.1. Applications 19

highly useful during droughts. Contamination sensors can additionally

be used detect impurities, which decay the quality of water. In addition
smart valves and pumps can control the flow of water in response to
environmental conditions. For instance, smart valves can reduce harmful
fluctuations in water pressure (Mistry, 2011) and isolate contaminated
water while smart pumps can detect and respond to clogs in pipes.
The resilience of smart water technologies, however, has been brought
to question. For instance, Amin et al. (2013) discuss relevant adversarial
models against an automated canal system. The authors also perform
tests on the Gignac canal system to demonstrate the effectiveness of
potential attacks. These attacks can occur at various levels of a hierar-
chical SCADA system. For instance, attacks may occur on the physical
infrastructure, the regulatory control layer (which interacts with the
canal network through sensing and actuation devices), the communica-
tion network, the supervisory control layer (which performs tasks such
state estimation/fault diagnosis/selection of control parameters), or the
corporate network. In water distribution systems, Laszka et al. (2017)
considers a cyber-physical attack model where the attacker introduces
contaminants into the water supply and disables a subset of sensors.
The authors recommend that operators intelligently add redundant
sensors, introduce diverse sensing devices, and increase device security
to achieve resilience.
The examination of the resilience of water based CPS has been in
part motivated by a precedent for attacks. Most notably, one can consider
the Maroochy Shire incident (Slay and Miller, 2007; Abrams and Weiss,
2008) an attack on a sewage treatment SCADA system in Queensland,
Australia. The system contained 142 pumping stations monitored by two
monitoring workstations. Radio communication was enabled between
pumping stations and central computers. An attack on this system
was carried out by a disgruntled former employee over a period of 2
months in the year 2000. The attack, which was done remotely using a
laptop and radio transmitter, led to communication failures among the
pumping stations and the central computer, unexpected pump behavior,
and a malfunctioning alarm system. Moreover, as a result of the attack,
800,000 liters of raw sewage spilled into the community. The attack
demonstrated the power of a malicious insider. Moreover, it revealed
20 Introduction

the vulnerability of remote control and sensing technologies when they

are used without adequate security defenses.
Privacy must also be accounted for in water distribution systems.
While water consumption may not release as much sensitive information
about users as electricity consumption, there exist avenues for adver-
sary’s to learn about the user. For instance Rottondi and Verticale (2016)
discuss how information can be leaked in gaming scenarios where users
are incentivized by operators to alter their water consumption habits.
In particular, it is argued that game actions can be related to physical,
social, and mental characteristics of the user. Thus, while cyber-physical
technologies such as smart water meters provide operators the ability
to increase efficiency by influencing resource consumption, collecting
the necessary data raises significant privacy concerns.
 2
Modeling

Obtaining reliable system models for CPS play an important role in

their design and operation. Models allow us to simulate behavior and
perform mathematical analysis offline. Moreover, they allow operators
to recognize flaws and defects in the design of our systems and po-
tentially measure robustness and resilience to disturbances, faults, or
malicious behavior without exposing a physical plant to such threats.
In this section, we briefly discuss modeling aspects of CPS. We begin
with system identification and its role in obtaining models of physical
processes in CPS (subsection 2.1). We then discuss challenges in CPS
modeling (subsection 2.2). Next, we discuss state space and LTI mod-
els (subsection 2.3). Finally, we investigate hybrid system models as a
means to capture complex behaviors in CPS (subsection 2.4).

2.1 System Identification

Determining accurate numerical models for physical systems, which is

often referred to as system identification, can be performed in multiple
ways. One can generally leverage basic physical principles to approx-
imately represent the dynamics of a system. As examples, Newton’s
laws of motion can describe the dynamics of vehicles and the swing

21
22 Modeling

equation can describe rotor dynamics of generators in power systems.

However, in many cases, basic principles are not sufficiently detailed
to obtain a working model of a physical control system. In such cases,
system identification can be performed using statistical tools.
The process of system identification can be divided into the following
steps (Ljung, 1998):
1. Collect data
2. Choose an appropriate class of models
3. Select criteria for model fit
4. Determine the best model in the chosen class according to criteria
5. Validate chosen model
Data collection for system identification requires one to intelligently
design the inputs of the system (Mehra, 1974; Wahlberg et al., 2010).
However, practical constraints may hinder the choice of inputs and limit
the effectiveness of experiments. In some cases, an operator can neither
design nor accurately measure the inputs and this requires system
identification from the outputs only (Peeters and De Roeck, 1999). For
instance, vibration tests using artificial inputs on bridges are often
expensive and difficult to carry out (Peeters and De Roeck, 1999). Big
data and associated technologies introduce new wrinkles to the problem
of system identification (Slavakis et al., 2014; Sharma et al., 2014). On
the positive side, high dimensional data can be leveraged to identify
more complex system structures (Sharma et al., 2014). On the other
side, the development of efficient algorithms is often challenging and
data collection also introduces significant privacy issues. For instance,
collecting power consumption data allows utilities to model demand
response, but can also reveal sensitive information about a person’s
daily routines and habits.
The choice of an appropriate model class significantly affects the
performance of system identification. Depending on the presence of some
intuition about the working principles of the system, grey box or black
box modeling can be used. In certain cases, linear models are sufficient
for describing a system’s dynamics. We consider an autoregressive
2.1. System Identification 23

moving average with exogenous input (ARMAX) model below:

Na
X Nb
X Nc
X
yk = aj yk−j + bi uk−i + cl ek−l + ek . (2.1)
j=1 i=1 l=1

The output, input and, disturbance at time j are denoted by yj , uj , and

ej , respectively. The choice of model class in part includes the dimension-
ality of the model. With respect to the ARMAX model provided in (2.1),
Na , Nb , Nc affect the model complexity. Additional complexity (larger
Na , Nb , Nc ) could potentially allow operators to model higher order
dynamics. However, this can come at the cost of overfitting. While an
ARMAX model generalizes simpler linear models (AR, ARX, ARMA),
it can still fail to capture important aspects of CPS. Significant work
has examined nonlinear system identification; for instance see Nelles
(2013) for a general treatment of nonlinear system identification, and
Giannakis and Serpedin (2001) for a list of references. The identification
of time varying systems has also been considered, for example by Tsat-
sanis and Giannakis (1993). Perhaps most noteworthy in the context of
CPS are developments in the identification of hybrid systems (Narendra
and Parthasarathy, 1990; Paoletti et al., 2007). As discussed later in
this section, hybrid system modeling is naturally equipped to model
many of the intricacies of CPS.
Typical criteria for a model’s fit aim to measure how close a model
reflects actual system behavior. As an example, one can consider a
quadratic function of the difference between the true outputs and
the expected outputs as derived by the system model. Once optimal
parameters for a given class of models are chosen, the chosen model
is evaluated using these criteria. In order to avoid a statistical bias,
the training data used for system identification is different from the
testing data used for validation. If the model closely captures the
dynamical behavior according to the chosen criteria, no further action
is required. However, if it does not, a designer may have to repeat
portions of this process. These steps could fail due to, e.g., deficient
data, inappropriate model class and/or criteria selection, and failures
in numerical algorithms.
We note that system identification continues to evolve, especially
in the age of big data. Consequently the field of system identifica-
24 Modeling

tion has been enhanced by tools from machine learning which offers a
plethora of tools and algorithms to efficiently process data and learn
models. For more information regarding the interactions between system
identification and machine learning, see Ljung et al. (2011)
The steps for system identfication described in this subsection often
allow designers to obtain models of physical processes. However, CPS
are comprised not only of physical systems, but also cyber technologies.
An in depth discussion of identifying models for computational and
networking systems in CPS is out of the scope of this monograph (see,
e.g., Lee (2006) for a more detailed discussion). In the next subsection,
we examine challenges that arise when attempting to model software
and communication systems along with physical processes in a CPS.

2.2 Challenges in Modeling Cyber-Physical Systems

Developing realistic applicable models of CPS can be a significant chal-

lenge due primarily to their inherent heterogeneity (Derler et al., 2012).
Appropriate abstractions must be made not only for physical processes,
but also for so called cyber technologies. In this regard, practical mod-
els must be rich enough to cover this diverse behavior. Derler et al.
(2012) recommends combining a multiplicity of domain specific mod-
els. A number of different modeling and simulation environments have
been used to capture the behavior of CPS including but not limited to
Ptolemy II (Eker et al., 2003), Simulink (Tariq et al., 2014), Modelica
(Fritzson, 2014), and Metro II (Davare et al., 2013). As noted by Derler
et al. (2012), several challenges arise in CPS modeling software develop-
ment due to nondeterminism in solvers, misconnected components, and
inconsistent component development.
The interaction and composition of heterogeneous systems must also
be accounted for. As noted by Rajkumar et al. (2010), physical processes
and software systems should be abstracted out in a composable manner
when modeling CPS. These abstractions should capture the degrees
of freedom in CPS in order to enable design. Moreover, rules defining
interactions at interfaces should carefully capture real system behav-
ior. Derler et al. (2012) observe that modeling tools are susceptible to
mismatched connections. Errors include inconsistent units, reversed con-
2.3. State Space Models 25

nections, and semantic errors. Additionally modelers of heterogeneous

subsystems and their interfaces must recognize and consider different
subsystem sizes, locations, and time scales.
For instance, Lee (2008) emphasizes the role of timing in the design
and modeling of CPS. Traditional models of software and computing
elements may not capture possible effects due to imperfections in timing
such as the delay in the arrival of program outputs. However, in physical
systems, unexpected behavior can arise if computational elements do
not produce outputs in a timely manner. As such designers must use
accurate models for timing in CPS. If a certain level of precision is
not possible, designs may need to be altered to obtain reliable timing
and/or synchronization (Henzinger et al., 2001; Edwards and Lee, 2007;
Johannessen, 2004).
In the ensuing sections, we arrive at several theorems and technical
results related to robustness and resilience of CPS. These theoretical
results rely in part on the selection of mathematical models as opposed
to real systems themselves. With this in mind, we recognize a fine line
between selecting models that enable thorough mathematical analysis
and ensuring that our modeling reflects realities of CPS. In the next
subsection, we will describe a class of models, which we will frequently
use in the rest of the document. While these models may lack suffi-
cient generality to address some of the challenges captured above, we
argue that they capture enough detail to perform initial mathematical
investigations centered around robustness and resilience in CPS.

2.3 State Space Models

A deterministic state space system model can be represented as follows:

ẋ(t) = f (x(t), u(t), t) y(t) = h(x(t), u(t), t) (2.2)

Here t refers to time. The control inputs u at time t or u(t) are assumed
to be in Rp . The outputs of the system, obtained from sensors at time t,
are given by y(t) ∈ Rm . The state x(t) ∈ Rn is used to summarize the
internal dynamics of a system. In a deterministic system, the current
state at time t along with present and future inputs completely describe
the evolution of a system. In particular, they allow us to determine
26 Modeling

future states as well as future outputs. An ordinary differential equation

(ODE) is used to describe the evolution of states. In many cases, linear
time invariant (LTI) systems serve as a valid first order approximation
of a general nonlinear system, especially in the region of interest for
control. LTI systems can be represented as follows:
ẋ(t) = Ac x(t) + Bc u(t), y(t) = Cx(t) + Du(t). (2.3)
The matrix Ac ∈ Rn×n captures effect of internal system dynamics, the
input matrix Bc ∈ Rn×p is used to describe the effect of the control
input on the state evolution, the output matrix C ∈ Rm×n determines
the portion of the state being observed and the feedthrough matrix
D ∈ Rp×n models the impact of the current input on the current output.
The matrix D is often identically 0 in many control systems and as
such is disregarded in most parts of this document. Nonetheless, much
of the analysis presented applies to the case when D is nonzero. Linear
systems allow for tractable analysis and can provide rich information
about a system’s behavior.
CPS are controlled through cyber components that operate in dis-
crete time intervals. A discrete LTI control system is presented below:

xk+1 = Ad xk + Bd uk , yk = Cxk . (2.4)

xk refers to the kth sample of the state, i.e., xk = x(kT ) where T
is a constant sampling interval. Likewise, yk and uk refer to the kth
sample corresponding to the output and input, respectively. A discrete
time control system model can be obtained from a continuous time
plant. The measured output y(t) is sampled using an analog-to-digital
(A/D) converter. A computer synthesizes a digital control input which is
implemented at the plant using a digital-to-analog (D/A) converter and
held constant until the next sampling interval. Note here that matrices
Ad and Bd differ from their continuous time counterparts Ac and Bc .
Fortunately, for a given constant sampling period T there exists closed
form transformations to obtain Ad and Bd .
Z T
Ad = exp(Ac T ), Bd = exp(Ac τ )Bc dτ. (2.5)
0
Due to disturbances, modeling errors, and sensor imperfections,
uncertainty has to be properly addressed in CPS modeling. One possible
2.3. State Space Models 27

way of modeling uncertainty in LTI systems is as follows:

xk+1 = Axk + Buk + wk , yk = Cxk + vk , (2.6)
where wk captures process noise and vk captures sensor noise. Given
no restrictions on wk and vk , the previous model is general enough to
completely describe the sampled dynamics of the system. However, typ-
ically certain restrictions are placed on these parameters. For instance,
we will typically model wk and vk as stochastic processes which are
IID and independent of each other. For instance, zero mean Gaussian
noise is a common assumption (wk ∼ N (0, Q) and vk ∼ N (0, R)) and
is particularly accurate in electrical components. Alternatively, one can
model wk and vk as bounded disturbances.
The prior model (2.6) fails to account for network imperfections. One
common phenomenon that we will frequently model is packet drops.
Packet drops can occur for a variety of reasons in a CPS including
congestion, delays, synchronization errors, faulty equipment, attacks,
or predefined user strategies. Packet drops which occur at the control
input can be modeled as
xk+1 = Axk + νk Buk + wk , (2.7)
where νk ∈ {0, 1}. When the control input is delivered on time to
the plant, νk = 1. A drop occurs when νk = 0. Implicitly, here we
assume that if no control input has been delivered, zero input will be
applied. However, this need not be the case. Typically, a system designer
has the degree of freedom to select an alternative local policy when
a control packet does not arrive. One possibility is to simply use the
previous control input, uk−1 (Nilsson, 1998). The operator may or may
not be aware of the packet drop at the control input. For instance, in
the Transmission Control (TCP) protocol, the operator will receive
an acknowledgement if a control input is received. Alternatively, in
the User Data (UDP) protocol, no acknowledgment is received. The
presence of an acknowledgement can fundamentally change the design
of both controllers and estimators (Schenato et al., 2007). For instance,
the absence of an acknowledgement eliminates the separation principle.
In a large scale systems, it may be common that multiple control
input packets must be delivered to multiple actuators. In the multi
28 Modeling

input-case, the dynamics can be described as follows

xk+1 = Axk + BVk uk + wk , (2.8)
where Vk is a diagonal matrix with binary diagonal elements. In the
single input case, Vk = νk I. For the multi-input case, we can assign
Vk = diag(νk,1 , . . . , νk,p ),
where νk,i is a binary variable to indicate whether the packet sent by
the operator at time k to actuator i is received (νk,i = 1) or dropped
(νk,i = 0). Notice that for simplicity, we assume that each packet has a
scalar input though extensions to vector inputs follow trivially.
Packet drops at the sensor measurement can be modeled in multiple
ways. For instance, Sinopoli et al. (2004) modeled a packet drop at
the output in the single sensor case as time varying sensor noise where
if γk = 1, indicating a measurement is sent, vk (γk = 1) ∼ N (0, R).
However, if a packet is dropped, γk = 0 and vk (γk = 0) ∼ N (0, σ 2 I).
Here, appropriate estimation filters are obtained by taking the limit
as σ goes to ∞. An equivalent formulation is given by Schenato et al.
(2007) where
yk = γk Cxk + vk . (2.9)
In this case γk results in a time varying observation matrix. In all cases it
is assumed, that the operator knows γk at time k. In other words, he can
distinguish a measurement packet from the characteristics of receiver
noise. The processes γk and νk are often modeled as IID Bernoulli
processes, independent of each other. This assumption enables tractable
mathematical analysis. Alternative models have been considered. For
instance, Mo et al. (2013) consider Markovian packet losses in a linear
quadratic Gaussian (LQG) setting.
More generally, one can consider packet drops in the multi-sensor
case. Here, the estimator receives
ỹk = Γk (Cxk + vk ),
where Γk is a diagonal matrix with binary diagonal elements. In the
single sensor case, Γk = γk I. For the multi-sensor case, we can assign
Γk = diag(γk,1 , . . . , γk,m ),
2.4. Hybrid Systems 29

where γk,i is a binary variable to indicate whether the packet from

sensor i at time k is received by the estimator (γk,i = 1) or dropped
(γk,i = 0). Again, for simplicity, we assume that each sensor makes a
scalar measurement though extensions to vector sensor measurements
follow trivially.
The models described above can effectively characterize many as-
pects of CPS. A fixed linear system often strongly approximate system
dynamics in a given region of interest. As discussed in the next subsec-
tion, hybrid system modeling allows us to mathematically generalize this
approach to systems which operate in multiple modes. Stochasticity in a
physical process is modeled through the inclusion of process and sensor
noise. A first order approximation of a computer controlled physical
system is captured using discrete time dynamics. Finally, imperfections
in communication are modeled using stochastic drops at both sensors
and actuators.

2.4 Hybrid Systems

While capturing sufficient generality for purposes of theoretical study,

LTI state space models often fall short of explaining the behavior of
certain classes of CPS. In many embedded systems, computational mod-
els have to address the specific behavior of software and programming
languages used to run micro-controllers. This includes paths for com-
munication and whether commands are run simultaneously, see Derler
et al. (2012). Precise modeling may also be necessary when multiple
heterogeneous micro-controllers are used in the system. As an example,
embedded systems run on different clocks with possibly unequal param-
eters. As these systems often interact and trigger each others actions,
unexpected behavior can arise if these systems (and their timing) are
not modeled carefully. In this regard, modular design with compatible
components and behavior is a major goal for CPS. We will not devote
our efforts to describe details of possible modeling approaches and
challenges for general CPS. Instead, in this subsection, we will explore
aspects of hybrid systems as a promising modeling framework known in
the literature.
30 Modeling

Hybrid system models are natural generalizations of continuous

time state-space models that involve discrete dynamical components.
In almost every modern CPS, actions made in software interact with
physical processes to be controlled. This requires a combination of
continuous and discrete variables where discrete elements often reflect
decision variables in software. Hybrid models find applications in hys-
teresis analysis, target tracking, system biology, temperature control
systems and power systems. While there is not a single definition for
hybrid systems, in general hybrid systems could be viewed as having
both analog and digital inputs and outputs. The interactions of the
analog and digital states determine the specific way a hybrid system is
defined.
A mathematical definition of a continuous time hybrid system in-
volves discrete and continuous state spaces, a continuous time variable
t, events that trigger jumps among discrete states and the evolution
descriptions for the times in between events. We refer the reader to,
e.g., Branicky et al. (1998) and Van Der Schaft and Schumacher (2000),
for a comprehensive mathematical treatment of hybrid systems with
abstract mathematical structures. For the purpose of this monograph,
we explain hybrid systems in terms of real numbers and differential
equations. A general class of hybrid systems is obtained by the following
set of equations:

ẋ(t) = f (x(t), q(t), u(t)), (2.10)

+
q (t) = g(x(t), q(t), u(t)), (2.11)
y(t) = h(x(t), q(t), u(t)), (2.12)

where f and g are arbitrary continuous functions of x ∈ X and u ∈ U

for each q ∈ Q. Here, the discrete set Q could be represented as an index
set and X ⊆ Rn and U ⊆ Rp . q + (t) denotes the value of the discrete
state q at a time t+ that is infinitesimally larger than t. Note that the
evolution of the discrete state q(t) follows an automaton structure where
the function g represents the events in which a jump from one state
to another occurs. The output is denoted by y(t). A special important
subclass of hybrid systems is obtained when we allow the function f
and h to be linear in x and u. These systems are often called jump
2.4. Hybrid Systems 31

linear systems and are represented as follows:

ẋ(t) = Aq x(t) + Bq u(t), (2.13)

y(t) = Cq x(t) + Dq u(t). (2.14)

Here, the jumps in the discrete variable q are induced by the events
described by g. For each q, the system acts as a LTI system.
A well known example of a hybrid system is the hysteresis loop
represented by the following equations:
(
1 + u, x≤∆
ẋ = . (2.15)
−1 + u, x > −∆

In this loop, the scalar state variable x is allowed to increase up to a

limit ∆. The derivative of x then switches to −1 + u. Note that the
state trajectory does not involve a jump; however, the dynamics are
determined by an internally induced jump. The hysteresis phenomenon
is used extensively to describe the behavior of magnetic recording
systems and more generally the memory a system retains due to the
initial state.
Another example of a hybrid system is an air conditioner with a
thermostat. The mechanical part of the system along with the heat flow
characteristics of the home, form a continuous-time dynamical system,
where temperature is to be kept at a desirable level using feedback
control. In a simple abstraction of this system, the difference between
the measured and intended temperature levels determine the effort
needed from the mechanical parts of the device. In this case, the set
temperature limit Tlimit determines the mode of operation for the air
conditioner. Depending on whether x(t) > Tlimit or x(t) ≤ Tlimit , the
dynamics follow:
(
f1 (x, u), x ≤ Tlimit
ẋ = . (2.16)
f2 (x, u), x > Tlimit

We note that a discrete time hybrid system model can be obtained

by replacing differential equations with difference equations. Similar
to LTI systems, a particularly important extension for the definition
of hybrid systems is stochastic linear hybrid systems. In this case,
32 Modeling

the evolution of the system state x is determined by an underlying

probability distribution (typically using a Markov chain). The following
equations represent such hybrid system models in discrete time:
(q)
xk+1 = Aq xk + Bq uk + wk , (2.17)
(q)
yk = Cq xk + Dq uk + vk , (2.18)
(q) (q)
where wk ∼ N (0, Wq ) and vk ∼ N (0, Rq ). We note that a prominent
example of a stochastic hybrid system is a system which accounts for
packet drops in the network (as modeled in the previous subsection).
Again, packet drops occurring at the control input are modeled as

xk+1 = Axk + νk Buk + wk , (2.19)

where νk ∈ {0, 1}. Similarly, packet drops at the sensor measurements

can be modeled as
yk = γk Cxk + vk . (2.20)
In this case γk effectively results in a time varying observation matrix
C. We observe νk and γk are discrete hybrid system states while xk
represents continuous system states.
 3
Networked Control Systems

In classical control theory, the communications between sensors, estima-

tors, controllers and actuators are often assumed to be perfect. However,
due to the wide usage of off-the-shelf low-cost devices, as well as energy
and bandwidth constraints, communications in CPS are subject to vari-
ous network effects such as packet drop, delay, quantization and fading.
Furthermore, it is usually not economical for all the devices in CPS to
communicate at each time instant, as it requires a large bandwidth and
drains the battery of wireless devices. Hence, it is important for each
node in CPS to only communicate the most “valuable” information to
ensure the performance of the system. In this section we analyze and
design CPS with imperfect communication.
The rest of the section is organized as follows: In subsection 3.1 we
derive the optimal estimator and analyze its performance when sensory
data are communicated over a lossy network. Subsection 3.2 looks into
the LQG optimal control problem when the control input may be lost.
Next we consider the problem of sensor and actuator scheduling in
subsection 3.3 and 3.4, in order to achieve the optimal trade-off between
frequency of communication and system performance.

33
34 Networked Control Systems

3.1 Estimation in Lossy Networks

We adopt a linear discrete-time state space model:

xk+1 = Axk + wk , yk = Cxk + vk , (3.1)

where wk captures process noise and vk captures sensor noise, which

are assumed to be IID zero mean Gaussian and wk ∼ N (0, Q) and
vk ∼ N (0, R). The initial state x0 is also assumed to be Gaussian with
mean x̄0 and variance Σ0 and is independent from the noise process.
Here we omit the control input uk as we will discuss it in the next
subsection. We further assume that the (A, C) is observable, since
otherwise we can perform a Kalman decomposition and only consider
the observable space.
It is assumed that the measurement yk is transmitted to the state
estimator via a lossy network. The packet containing the sensory data
yk will either be received by the estimator at time k or lost. In other
words, it means that at each time step, the estimator receives

ỹk = Γk yk ,

where Γk is a diagonal matrix with binary diagonal elements. For the

single sensor case, we assume that Γk = γk I, where γk = 0 indicates
the packet is lost while γk = 1 indicates the packet arrives successfully
at the estimator. For the multi-sensor case, we can assign

Γk = diag(γk,1 , . . . , γk,m ),

where γk,i is a binary variable to indicate whether the packet from

sensor i at time k is received by the estimator. Notice that for simplicity,
we assume that each sensor makes a scalar measurement. However, most
of the results presented in this subsection can be extended to vector
sensor measurements trivially. The diagram of the system is illustrated
in Figure 3.1.

Remark 3.1. It is worth noticing that a CPS designer could leverage

non-control theoretical methods to combat packet drops. For example,
the widely used TCP protocol could provide reliable communication
in an unreliable network. However, these methods in general are not
3.1. Estimation in Lossy Networks 35

γk,1
Sensor 1
γk,2
Sensor 2
γk,3
System Sensor 3 Estimator

..
.
γk,m
Sensor m

Figure 3.1: Diagram of a Sensor Network where the Measurements Go through a

Lossy Network

free, especially for low-cost devices. For example, for the TCP protocol,
two-way communication between the sender and the receiver is required.
Hence, in this section, we shall mainly investigate the impact of packet
drops on CPS in the absence of more sophisticated communication
protocols such as TCP. The CPS designer can then deside whether the
impact of packet drops is tolerable.
For the single sensor case, the random process {γk } is the packet
drop process and is typically assumed to be an IID Bernouli process
with
Pr(γk = 1) = γ̄.
Furthermore, the packet drop process {γk } is assumed to be independent
from the noise {vk }, {wk } and initial condition x0 . It is worth noticing
that other packet drop model exists, e.g. Markovian model, and some
results discussed in this subsection can be applied to a more general
packet drop process. It is further assumed, that the estimator can
distinguish a valid packet from a dropped packet. This can usually
be achieved via an error-detecting code. In other words, the estimator
knows Γk at time k. Therefore, the information available to the estimator
at time k can be written as
Ik = {Γ0 y0 , Γ0 , . . . , Γk yk , Γk } . (3.2)
36 Networked Control Systems

The goal of state estimation is to infer the current state xk given the
information set Ik . Here we use the MMSE (minimum mean squared
error) estimator. Define

x̂k|k = E[xk |Ik ], Pk|k = Cov(x̂k|k − xk |Ik ), (3.3)

x̂k|k−1 = E[xk |Ik−1 ], Pk|k−1 = Cov(x̂k|k−1 − xk |Ik−1 ).

Our first goal is to design an estimator to derive x̂k|k . We then character-

ize the performance of such an estimator by examining the estimation
covariance Pk|k .

3.1.1 Optimal Estimator

The system can be viewed as a time-varying linear system, with a time
varying C̃k = Γk C. As a result, the optimal filter is the Kalman filter,
which takes the following form:

• Initialization:

x̂0|−1 = x̄0 , P0|−1 = Σ0 . (3.4)

• Prediction:

x̂k+1|k = Ax̂k|k , Pk+1|k = APk|k AT + Q. (3.5)

• Correction:

x̂k|k = x̂k|k−1 + Kk (ỹk − C̃k x̂k|k−1 ), Pk|k = Pk|k−1 − Kk C̃k Pk|k−1 ,

(3.6)

where the Kalman gain is1

Kk = Pk|k−1 C̃kT (C̃k Pk|k−1 C̃kT + R̃k )−1 ,

and C̃k = Γk C, R̃k = Γk RΓk .

It is worth noticing that the estimation error covariances Pk|k are

stochastic matrices, which depend on the packet dropping process. (How-
ever, they are independent of the measurements yk ). This is different
1
The inverse of a singular matrix is defined as its Moore-Penrose inverse.
3.1. Estimation in Lossy Networks 37

from the classical Kalman filter case, where Pk|k is deterministic. As

a result, in order to characterize the performance of the Kalman fil-
ter, we need to derive the statistics of Pk|k . The rest of this section is
devoted to deriving statistics of Pk|k . We first prove that under very
mild assumptions, Pk|k converges to a unique distribution and under
strong conditions, the cumulative density function of the asymptotic
distribution can be derived. We then consider the moments of Pk|k ,
where we mainly focus on the first moment, i.e., the expected value of
Pk|k . We will provide upper and lower bounds for the expected value of
Pk|k as well as discuss whether it is uniformly bounded.

3.1.2 Asymptotic Distribution

In this subsection, we analyze statistical properties of Pk|k or Pk|k−1
in order to characterize the impact of packet loss on the performance
of Kalman filtering. We mainly focus on the single sensor case with
IID packet drops. To simplify notations, we will define the following
functions:

h(X) , AXAT + Q,
 −1
g0 (X) , X −1 + C T R−1 C = X − XC T (CXC T + R)−1 CX.

We further define function g = g0 ◦ h, where ◦ indicates function

composition. As a result, for the single sensor case, we have

h(P if γk = 0
k−1|k−1 )
Pk|k = .
g(P if γk = 1
k−1|k−1 )

Before continuing on, we would like to state several important

properties of function h and g, the proof of which can be found in
Sinopoli et al. (2004):

Lemma 3.1. h and g are monotonically increasing. In other words,

given X ≥ Y ≥ 0 2 , we have

h(X) ≥ h(Y ), g(X) ≥ g(Y ).

2
Note that all comparisons between matrices in this section are in the positive
definite sense unless otherwise stated.
38 Networked Control Systems

Moreover, the function g (g0 ) is a concave function. For any X, Y ≥ 0

and α, β ≥ 0 and α + β = 1, we have
g(αX + βY ) ≥ αg(X) + βg(Y ),
g0 (αX + βY ) ≥ αg0 (X) + βg0 (Y ).
The first question we have is whether Pk|k converges to some station-
ary distribution as k → ∞. For the single sensor case with IID packet
drops, Kar et al. (2012) shows that it is indeed the case:
Theorem 3.2 (Kar et al. (2012)). Assume Q > 0 to be positive definite.
Given an initial condition P0|0 , if the packet arrival probability γ̄ > 0,
then the following properties hold:
1. The process {Pk|k } is stochastically bounded, i.e.,
lim sup Pr(kPk|k k > N ) = 0.
N →∞ k=0,1,...

2. There exists a unique distribution µ (depends on γ̄), such that

the sequence {Pk|k } converges weakly to µ regardless of the initial
condition P0|0 .
A similar theorem can also be proved for Markovian packet drops.
Let us define the following distance function on the space of positive
definite matrices Sn++ :
( n )
X
2
d(X, Y ) , (log ρi ) , (3.7)
i=1

where ρ1 , . . . , ρn are the eigenvalues of XY −1 . The following theorem

establishes the contraction properties of h and g:
Theorem 3.3 (Censi (2011)). If A is non-singular, the functions h and
g are nonexpansive under the distance metric d defined in (3.7), i.e.,
d(h(X), h(Y )) ≤ d(X, Y ), d(g(X), g(Y )) ≤ d(X, Y ),
for any X, Y > 0. Furthermore, if (A, Q1/2 ) is controllable, then there
exists an integer K, such that g K = g ◦ · · · ◦ g is a strict contraction. In
other words, there exists ρ < 1, such that
d(g K (X), g K (Y )) < ρd(X, Y ).
3.1. Estimation in Lossy Networks 39

Using the contraction property, we can prove the following conver-

gence results for Markovian packet drop models:

Theorem 3.4 (Censi (2011)). Assuming A is non-singular, if (A, Q1/2 )

is controllable, then there exists a unique distribution µ , such that the
sequence {Pk|k } converges weakly to µ regardless of the initial condition
P0|0 given that the packet drop process {γk } is governed by a Markov
process:

Pr(γk+1 = j|γk = i) = pij ,

with p11 > 0.

Notice that the Theorem 3.4 requires an additional invertibility

assumption on A, while the convergence for the IID packet drop case
(Theorem 3.2) does not need such an assumption.
Now we have established that under very mild conditions, Pk|k
converges to a unique distribution. The next question would be to
compute the cumulative density function of Pk|k . However, this is in
general a very daunting task and can only be done under very specific
conditions. Censi (2009) considers systems satisfying the following non-
overlapping conditions:

Definition 3.1. Assuming that (A, Q1/2 ) is controllable, the functions

h and g are called non-overlapping if
 −1
sup g(X) = C T R−1 C < h(P∞ ),
X≥0

where P∞ is the unique solution of the Riccati equation:

P∞ = g(P∞ ).

Notice the non-overlapping condition actually implies that the matrix

C is full column rank, since otherwise g(X) will be unbounded. On the
set of all possible sequences of packet drop process γk:0 = (γk , . . . , γ0 ),
we can define a total order relation ., such that (γ0 ) . (γ00 ), if γ0 ≥ γ00 .
For sequences, . is recursively defined as

(γk , . . . , γ0 ) . (γk0 , . . . , γ00 )

40 Networked Control Systems

if and only if γk > γk0 , or γk = γk0 and (γk−1 , . . . , γ0 ) . (γk−1

0 , . . . , γ00 ).
Suppose that we have the initial condition Σ0 ≥ h(P∞ ). Then we
know that Pk|k is a function of γk:0 . In other words, we can write
Pk|k as Pk|k (γk , . . . , γ0 ). A key observation for a system satisfying the
non-overlapping condition is that

Pk|k (1, 0, . . . , 0) = g ◦ |h ◦ h ◦{z· · · ◦ h}(Σ0 )

k−1 times
≤ h(P∞ ) = h ◦ g ◦ g ◦ · · · ◦ g ◦g(P∞ )
| {z }
k−1 times
≤ h ◦ g ◦ g ◦ · · · ◦ g ◦g0 (Σ0 )
| {z }
k−1 times
= Pk|k (0, 1, . . . , 1). (3.8)

Combining (3.8) with the fact that g(X) ≤ h(X) and h and g are
monotonically increasing, we can prove the following lemma:

Lemma 3.5. Suppose the system satisfies the non-overlapping condi-

tion and the initial conditions Σ0 ≥ h(P∞ ). Pk|k is a monotonically
decreasing function of γk:0 , i.e.,
0
Pk|k (γk:0 ) ≤ Pk|k (γk:0 )
0 .
if and only if γk:0 . γk:0

Hence, in order to calculate the probability that Pk|k is no greater

∗
than some positive definite matrix N , we need to find the smallest γk:0
∗
such that Pk|k (γk:0 ) ≤ N . The probability can be derived as
∗
Pr(Pk|k ≤ N ) = Pr(γk:0 . γk:0 ).
3.1. Estimation in Lossy Networks 41

Such a sequence γk:0 ∗ can be found using the following branch and

bound method:
Algorithm 1: Branch and Bound Method
Data: N ≥ P∞
Result: γk:0∗ = (γ ∗ , . . . , γ ∗ )
k 0
Initialization: t ← k;
while t ≥ 0 do
if Pk|k (γk∗ , . . . , γt+1
∗ , 0, 1, . . . , 1) ≤ N then
∗
γt ← 0;
else
γt∗ ← 1;
end
t ← t − 1;
end
∗ , the probability can be then
For the IID case, once we derive γk:0
recursively calculated as

γ̄ + (1 − γ̄)Pr(γ ∗ if γk∗ = 0
∗ k−1:0 . γk−1:0 )
Pr(γk:0 . γk:0 )= ,
γ̄Pr(γk−1:0 . γ ∗ ) if γk∗ = 1
k−1:0

where γ̄ is the packet arrival probability. We can take the limit3 k → ∞

to get the cumulative density function of the stationary distribution of
Pk|k .
For general systems where the non-overlapping condition does not
hold, the matrix Pk|k is not necessarily a monotone function of γk:0 . As a
result, in general, it is computationally difficult to derive the cumulative
density function of Pk|k or the asymptotic distribution of Pk|k . A more
tractable question would be to calculate the expected value of Pk|k ,
which will be discussed in the next subsection.

3.1.3 Moments of the Error Covariance Matrix

In this subsection, we consider the moments of Pk|k . It is worth noticing
that in general, no analytical form of the moments exists. Hence, we
3
Notice that the asymptotic distribution is independent of the initial condition
Σ0 .
42 Networked Control Systems

will mainly focus on deriving bounds of the moments of Pk|k , as well as

discussing if the moments are uniformly bounded (w.r.t. k).
Notice that Pk|k satisfies the following recursive equation:

Pk+1|k+1 = (1 − γk+1 )h(Pk|k ) + γk+1 g(Pk|k ). (3.9)

Assuming that {γk } are IID, then γk+1 would be independent from Pk|k ,
since Pk|k is a function γk , . . . , γ1 . Taking the expectation on both sides
of (3.9), we get

E[Pk+1|k+1 ] = (1 − γ̄)E[h(Pk|k )] + γ̄E[g(Pk|k )]. (3.10)

As h(X) is an affine function of X, E[h(Pk|k )] = h(E[Pk|k ]). On the

other hand, g(X) is a concave function of X by Lemma 3.1. Therefore,
by Jensen’s inequality, we have

0 ≤ E[g(Pk|k )] ≤ g(E[Pk|k ]).

The following theorem provides an upper and lower bound for

E[Pk|k ]:

Theorem 3.6. Define the matrix Sk and Vk recursively as 4

Sk+1 = (1 − γ̄)h(Sk ), Vk+1 = (1 − γ̄)h(Vk ) + γ̄g(Vk ),

with initial condition S0 = (1 − γ̄)Σ0 and V0 = (1 − γ̄)Σ0 + γ̄g0 (Σ0 ).

Then E[Pk|k ] is bounded by

Sk ≤ E[Pk|k ] ≤ Vk , ∀k.

Proof. We prove the theorem by induction. Notice that

P0|0 = (1 − γ0 )Σ0 + γg0 (Σ0 ).

Taking expected value on both sides of the equation and by Jensen’s

inequality and the concavity of g0 , we have

S0 = (1 − γ̄)Σ0 ≤ E[P0|0 ] ≤ (1 − γ̄)Σ0 + γ̄g0 (Σ0 ) = V0

4
Notice that the definition of Sk and Vk are different from what is given by
Sinopoli et al. (2004). This is due to the fact that Sk and Vk here are lower and
upper bounds on Pk|k instead of Pk|k−1 .
3.1. Estimation in Lossy Networks 43

Now suppose that Sk ≤ E[Pk|k ] ≤ Vk , then we have

Pk+1|k+1 = (1 − γk+1 )h(Pk|k ) + γk+1 g(Pk|k ).

Taking expected value on both sides of the equation, we get

     
(1 − γ̄)h E[Pk|k ] ≤ E[Pk+1|k+1 ] ≤ (1 − γ̄)h E[Pk|k ] + γ̄g E[Pk|k ] .

Now using the fact that both h and g are monotonically increasing, we
get

Sk+1 ≤ E[Pk+1|k+1 ] ≤ Vk+1 ,

which finishes the proof.

Next we consider the uniform boundedness of E[Pk|k ]. Notice that if

the system is strictly stable, then the system is detectable even if no
observation is received. As a result, E[Pk|k ] will always be bounded for
strictly stable system. As a result, for the rest of the chapter, we only
consider unstable systems.
Assuming that the system is unstable, then clearly if the packet
arrival rate γ̄ = 0, then Pk|k is unbounded. On the other hand, if the
packet arrival rate γ̄ = 1, then the Kalman filter becomes a classical
Kalman filter without packet drops and Pk|k will be bounded given that
the system is observable. As a result, there exists some critical packet
arrival rate γ̄c , at which the system transitions from bounded E[Pk|k ]
to unbounded E[Pk|k ]:
1
Theorem 3.7 (Sinopoli et al. (2004)). If (A, Q 2 ) is controllable, and A
is unstable, then there exists a γ̄c ∈ (0, 1) such that 5

sup E[Pk|k ] = +∞ for 0 ≤ γ̄ < γc and ∃Σ0 ≥ 0,

k
E[Pk|k ] ≤ MΣ0 , ∀k for γc < γ̄ ≤ 1 and ∀Σ0 ≥ 0,

where MΣ0 > 0 depends on the initial condition Σ0 ≥ 0.

With the upper and lower bounds provided by Theorem 3.6, we can
easily derive upper and lower bounds for the critical value γc .
5
We use the notation supk Ak = +∞ when the sequence Ak ≥ 0 is not bounded;
i.e., there is no matrix M ≥ 0 such that Ak ≤ M, ∀k.
44 Networked Control Systems

Theorem 3.8 (Sinopoli et al. (2004)). Suppose A is unstable. Define γl

and γu to be

γl , inf{λ > 0 : ∃X > 0, s.t. X > (1 − λ)h(X)},

γu , inf{λ > 0 : ∃X > 0, s.t. X > (1 − λ)h(X) + λg(X)}.

Then the critical value is bounded by

γl ≤ γc ≤ γu .

Furthermore, the following statements hold:

1. γl = 1 − ρ(A)−2 , where ρ(A) is the spectral radius of A.

2. If rank(C) = 1, then
1
γu = 1 − Ql u 2
,
i=1 |ρi |

where ρu1 , . . . , ρul are all the unstable eigenvalues of A.

3. If C is full column rank, or if A only contains 1 unstable eigenvalue,

then γu = γl = γc .

It is worth noticing that, for general systems, the upper and lower
bounds provided by Theorem 3.8 are not tight. This is mainly due to
the “eigen-cycle” of the A matrix, which is a set of eigenvalues with
the same absolute value. If an eigen-cycle exists, then it is possible that
even if (A, C) is observable, (Aτ , C) is not observable for some τ . To
see this, consider the following system:
" #
0 2 h i
A= , C = 1 0 , Q = I, R = 1. (3.11)
2 0

The A matrix has two eigenvalues, 2 and −2, which form an eigen-cycle.
It is worth noticing that (A2 , C) is not observable. To see why this loss
of observability causes a problem, we illustrate the information flow
of the system in Figure 3.2. Intuitively speaking, if all measurements
made at the even (or odd) times are lost, then the estimator cannot
“reconstruct” the system state. In other words, we need at least one
measurement made at the odd time and one measurement made at
3.1. Estimation in Lossy Networks 45

the even time to have a bounded estimation of the system state. This
restriction will deteriorate the performance of our estimator.
The following theorem provides a quantitative result on the impact of
eigencycles on the critical value for diagonalizable second-order systems:

Theorem 3.9 (Mo and Sinopoli (2011)). For a system with

A = diag(ρ1 , ρ2 ), |ρ1 | ≥ |ρ2 |, |ρ1 | > 1 and R, Q, Σ0 > 0, the critical
value is
γc = fc (A, C) = 1 − |ρ1 |−2 , (3.12)
if either of the following conditions holds

1. |ρ1 | > |ρ2 |,

2. rank(C) = 2.

Otherwise the critical value of the system is given by

2
− 1−D
γc = fc (A, C) = 1 − |ρ1 | M (ϕ/2π) , (3.13)

where ρ1 = ρ2 exp(jϕ), and DM (x) is the modified Dirichlet function

defined as
(
0 for x irrational
DM (x) = (3.14)
1/q for x = r/q, r, q ∈ Z and irreducible.

Notice that if rank(C) = 1 and |ρ1 | = |ρ2 |, then the lower bound
and upper bound of the critical value given by Theorem 3.8 are:

γl = 1 − |ρ1 |−2 , γu = 1 − |ρ1 |−4 .

y1 y2 y3 y4 y5

x0,1 x1,1 x2,1 x3,1 x4,1 x5,1

x0,2 x1,2 x2,2 x3,2 x4,2 x5,2

Figure 3.2: Information Flow of System (3.11)

46 Networked Control Systems

Theorem 3.9 clearly shows that neither the lower nor the upper bound
is tight in general.
For general linear systems with diagonalizable A, we use linear
transformations to put A into the diagonal standard form. The system
can then be decomposed into subsystems with eigenvalues on the same
eigen-cycle. To this end, we first define an equivalence relation on C,
such that z and z 0 are equivalent if and only if there exists q ∈ N+ , such
that z q = (z 0 )q . A necessary and sufficient condition for z and z 0 to be
equivalent is that z and z 0 have the same absolute value and the angle
between them are a rational fraction of π. The proof that the relation
defined by us is indeed an equivalence relation is left to the readers.
Next we partition the eigenvalues of A into g equivalent classes.
As a result, A can be diagonalized as diag(A1 , . . . , Ag ), where each
Ai ∈ Cni ×ni is a diagonal matrix containing all eigenvalues in one
equivalent class. Hence, we can write Ai as
Ai = αi diag(exp(jφi,1 ), . . . , exp(jφi,ni )).
We can split the corresponding C matrix into
h i
C = C1 , . . . , C g ,

where Ci ∈ Cm×ni . The following theorem proves a necessary and

sufficient condition for the uniform boundedness of E[Pk|k ] for Markovian
packet drop cases in the multi-sensor setup:
Theorem 3.10 (Sui et al. (2015)). Assuming that {Γk }∞ k=0 is a stationary
Markov process and the transition probabilities of Γk from any state to
any state are strictly positive. Then a necessary and sufficient condition
for lim supk→∞ E[Pk|k ] to be bounded is that

αi2 lim sup (Pr(Rk,i ))1/k < 1, ∀i ∈ {1, . . . , g}, (3.15)

k→∞
where Rk,i is the following event:
  

 Γ0 Ci 

 

 Γ1 Ci Ai 



Rk,i =  ..  is not full column rank . (3.16)
.
   

   

k−1
 
Γk−1 Ci Ai
 
3.1. Estimation in Lossy Networks 47

It is worth noticing that although Theorem 3.10 provides the exact

characterization on the boundedness of E[Pk|k ], computing the prob-
ability of Rk,i is still a non-trivial task, especially for systems where
(Aτ , C) is unobservable for some τ . However, for linear systems, where
(Aτ , C) remains detectable for all τ > 0, Mo and Sinopoli (2012b) prove
that the critical value is indeed the lower bound given in Theorem 3.8
for the single sensor case with Markovian packet drops:

Theorem 3.11 (Mo and Sinopoli (2012b)). Assuming the matrix A is

diagonalizable, and (A, Q1/2 ) is controllable. Suppose the packet drop
process {γk }∞
k=0 follows a stationary and irreducible Markov chain, with
recovery probability p01 , i.e.,

Pr(γk+1 = 1|γk = 0) = p01 .

Further assume that the unstable part of the system is non-degenerate,

i.e., (Aτ , C) is detectable for any real τ > 0. Then the qth moment of
Pk|k , E[(Pk|k )q ] is bounded if

p01 > 1 − ρ(A)−2q ,

and it is unbounded if

p01 < 1 − ρ(A)−2q .

Furthermore, tr(Pk|k ) follows a heavy tail distribution, with the following

decay rate:
supk log Pr(tr(Pk|k ) > M ) log(1 − p01 )
lim = .
M →∞ log M 2 log ρ(A)

Example 3.1. In this example, we consider the tail distribution of Pk|k

for two different systems. For system 1, the parameters are chosen to be
" #
2 0
A= , C = Q = R = I.
0 −2
For system 2, the parameters are chosen to be
" #
2 0 h i
A= , Q = I, C = 1 1 , R = 1.
0 −2
48 Networked Control Systems

The packet loss rate is chosen to be 0.25 for both systems. The reader
can verify that both systems are detectable. However, system 2 is degen-
erate since (A2 , C) is not detectable, while system 1 is non-degenerate.
Figure 3.3 illustrates the tail distribution of Pk . Notice that for both
systems, the tail follows a power decay law. However, the tail converges
much quicker for non-degenerate system.

100
Pr(tr(Pk|k ) ≥ x)

10−3

10−6

10−9

10−12
100 101 102 103 104 105 106 107 108
x
Figure 3.3: The Tail Distribution of tr(Pk|k ). The systems are chosen so that
A = diag(2, −2) and Q = I. The dashed line corresponds to C = R = I and the
solid line corresponds to C = [1 1] and R = 1.

3.1.4 Further Reading

In this subsection, we mainly consider the problem of state estimation
over lossy networks. Throughout the subsection, we consider the case
where the sensors can only transmit raw measurements to the Kalman
filter. The main benefit of this framework is its simplicity. However, if
the sensors and estimator are more powerful and can do coding and
decoding, the performance of the system may be improved. In other
words, instead of sending the measurement yk at time k, the sensor
could send zk = f (yk , . . . , yk−T +1 ), using a coding scheme with window
size T . The references He et al. (2013), Sui et al. (2015), and Xu and
3.2. Control in Lossy Networks 49

Hespanha (2005) provide detailed discussions on using linear coding

with window size T = 2, T = m and T = ∞ respectively.
It is also worth noticing that packet drop is not the only form of
imperfect communication. Other network effects, such as quantization
and delay, can also have impacts on the quality of the state estimation.
For a more detailed discussion on delay, please refer to Schenato (2008)
and Shi et al. (2009). A summary on the feedback control system with
data rate constraints can be found in Nair et al. (2007).

3.2 Control in Lossy Networks

In this subsection, we consider a closed-loop system where the control

input is subject to packet loss. We adopt a linear discrete-time state
space model that is very similar to (3.1):

xk+1 = Axk + Buak + wk , yk = Cxk + vk , (3.17)

where wk captures process noise and vk captures sensor noise, which

are both assumed to be IID zero mean Gaussian and wk ∼ N (0, Q) and
vk ∼ N (0, R). The initial state x0 is also assumed to be Gaussian with
mean x̄0 and variance Σ0 and is independent from the noise process.
We will further assume that the (A, C) is observable and (A, B) is
controllable.
uak is the control input applied by the actuator. We assume that the
controller sends the desired control input uk through a lossy network,
and the actuator will use uk as the control input if it is successfully
received. Otherwise, the actuator will apply 0 as the control input. We
further assume that the system contains only one actuator. As a result,

uak = νk uk ,

where νk is a Bernoulli random variable denoting whether the control

input arrives or not. We focus on the IID packet drop case in this
subsection. Let us define the control packet arrival probability to be

Pr(νk = 1) = ν̄.

For simplicity, we assume that sensory data can be reliably received

by the estimator. The diagram of the system is shown in Figure 3.4.
50 Networked Control Systems

Actuator Plant Sensor

νk

Controller

Figure 3.4: Diagram of a Control System where the Control Input Goes Through
a Lossy Network.

Contrary to the estimation over lossy network problem, where the

estimator knows whether the measurement packet is dropped, the
controller may not be able to know whether the actuator receives the
control input uk or not. For example, if a UDP-like protocol is used,
then there is no acknowledgment (ACK) or negative-acknowledgement
(NAK) from the actuator to the controller and the controller is unaware
of the packet loss process νk . Therefore, the information set available
to the controller and the estimator at time k is

Gk , {y0 , y1 , . . . , yk , u0 , . . . , uk−1 } .

To simplify notation, we further define

Gk− , {y0 , y1 , . . . , yk−1 , u0 , . . . , uk−1 } .

On the other hand, if a TCP like protocol is implemented, then

an ACK or NAK will be sent from the actuator to the controller. In
practice, such packets may also be subject to packet loss. However, for
simplicity we assume that the ACK or NAK packet can be reliably
received by the controller, which implies that the information set at
time k is

Fk , {y0 , y1 , . . . , yk , ν1 , . . . , νk−1 , u0 , . . . , uk−1 } .

Similar to the UDP like case, we define

Fk− , {y0 , y1 , . . . , yk−1 , ν1 , . . . , νk−1 , u0 , . . . , uk−1 } .

3.2. Control in Lossy Networks 51

To simplify notations, let us define the following information set Ik to

be

G UDP like protocol
k
Ik , .
Fk TCP like protocol

Similarly we can define Ik− as


G − UDP like protocol
Ik− , k
.
F − TCP like protocol
k

We analyze the system performance under TCP-like protocol first

in the next subsection and the discussion on the UDP-like protocol can
be found in Section 3.2.2

3.2.1 TCP-like Protocol

In this subsection, we consider the estimator and controller design for
system with TCP-like protocol. It can be proved that the probabil-
ity distribution of xk conditioning on Fk , is a Gaussian distribution
with mean x̂k|k and covariance Pk|k , which can be computed using the
following Kalman filter Schenato et al. (2007):

• Initialization:

x̂0|−1 = x̄0 , P0|−1 = Σ0 . (3.18)

• Prediction:

x̂k+1|k = Ax̂k|k + νk Buk , Pk+1|k = APk|k AT + Q. (3.19)

• Correction:

x̂k|k = x̂k|k−1 + Kk (yk − C x̂k|k−1 ), Pk|k = Pk|k−1 − Kk Ck Pk|k−1 ,

(3.20)
where the Kalman gain is

Kk = Pk|k−1 C T (CPk|k−1 C T + R)−1 .

52 Networked Control Systems

We now consider the controller design. Suppose the goal of the

controller is to minimize the following finite horizon LQG cost:
"N #
xTk W xk + (uak )T U uak
X
JN = min E
u0 ,...,uN
k=0
"N #
X
= min E xTk W xk + νk uTk U uk (3.21)
u0 ,...,uN
k=0

where uk must be a measurable function of the information set Fk . The

LQG problem can be seen as a special case of a Partially Observed
Markov Decision Process (POMDP). Hence, it can be formulated as a
dynamic programming problem. Define the value function Vk to be
"N #
X
Vk (f (xk |Ik )) = min E xTk W xk + νk uTk U uk ,
uk ,...,uN
t=k

where we assume that xk conditioned on Ik follows the distribution

f (xk |Ik ). Notice that the value function Vk is a mapping from the set of
posterior probability distributions of xk to a scalar. Using the principle
of dynamic programming, Vk will satisfy the following Bellman equation:
Z
Vk = xTk W xk f (xk |Ik )dxk (3.22)
Rn Z
+ min ν̄uTk U uk + f (yk+1 |Ik− )Vk+1 (f (xk+1 |yk+1 , Ik− ))dyk+1 ,
uk Rm
(3.23)
with
Z
VN (f (xN |IN )) = xTN W xN f (xN |IN )dxN .
Rn

The optimal value can then be computed as:

Z
JN = f (y0 )V0 (f (x0 |y0 ))dy0 .
Rm

For the TCP case, we know that f (xk |Ik ) is a Gaussian distribu-
tion with mean x̂k|k and a deterministic covariance Pk|k , where Pk|k is
deterministic. Hence, instead of defining Vk on all possible probability
distributions, we only need to consider the value of Vk on Gaussian
3.2. Control in Lossy Networks 53

distribution N (x̂k|k , Pk|k ). With a slight abuse of notation, we will write

Vk (N (x̂k|k , Pk|k )) as Vk (x̂k|k )6 .
Therefore, the Bellman equation (3.23) becomes
Vk (x̂k|k ) = x̂Tk|k W x̂k|k + tr(Pk|k W ) + min ν̄uTk U uk
uk
Z
+ f (yk+1 |Fk− )Vk+1 (x̂k+1|k+1 )dyk+1 .
Rm
The following lemma is needed to solve the Bellman equation, whose
proof is left for the readers:
Lemma 3.12. For any y ∈ Rm and positive definite R, S ∈ Rm×m , the
following equality holds:
 
Z exp −(y − ȳ)T R−1 (y − ȳ)
p ydy = ȳ,
Rm (2π)m |R|
 
Z exp −(y − ȳ)T R−1 (y − ȳ)
p y T Sydy = ȳ T S ȳ + tr(RS).
Rm (2π)m |R|

The following theorem provides an explicit solution of Vk :

Theorem 3.13. The value function Vk is given by
Vk (x̂k|k ) = x̂Tk|k Sk x̂k|k + ck , (3.24)
where Sk and ck satisfy the following recursive equation
Sk = W + AT Sk+1 A − ν̄AT Sk+1 B(U + B T Sk+1 B)−1 B T Sk+1 A,
h i  
ck = ck+1 + tr (AT Sk+1 A + W )Pk|k + tr Sk+1 (Q − Pk+1|k+1 ) ,
with initial conditions SN = W, cN = tr(PN |N W ). Furthermore JN is
given by
JN = x̄T0 S0 x̄0 + tr(Σ0 S0 )
N
X −1 N
X −1  
+ tr(Sk+1 Q) + tr (AT Sk+1 A + W − Sk )Pk|k ,
k=0 k=0

and the optimal control law is uk = −(U + B T Sk+1 B)−1 B T Sk+1 Ax̂k|k .
6
We can omit Pk|k since it is deterministic and independent from the control
input u0 , . . . , uN .
54 Networked Control Systems

Proof. We prove the theorem by backward induction.

Clearly, VN (x̂k|k ) = x̂Tk|k W x̂k|k + tr(Pk|k W ). Now assume that (3.24)
holds for k + 1. By the Bellman equation, we have

Vk (x̂k ) = x̂Tk W x̂k + tr(Pk|k W )

Z
+ min ν̄uTk U uk + f (yk+1 |Fk− )Vk+1 (x̂k+1 )dyk+1 .
u k Rm

The probability density function yk+1 conditioning on Fk− is Gaussian

distributed with mean C x̂k+1|k and covariance CPk+1|k C T + R. On the
other hand,
 T
Vk+1 (x̂k+1|k+1 ) = ck+1 + x̂k+1|k + Kk+1 (yk+1 − C x̂k+1|k ) ×
 
Sk+1 x̂k+1|k + Kk+1 (yk+1 − C x̂k+1|k ) .

Therefore, one can prove that

Z
f (yk+1 |Fk− )Vk+1 (x̂k+1|k+1 )
Rm
h i
= ck+1 + x̂Tk+1|k Sk+1 x̂k+1|k + tr Sk+1 Kk+1 (CPk+1|k C T + R)Kk+1
T
h i
= ck+1 + x̂Tk+1|k Sk+1 x̂k+1|k + tr Sk+1 (Pk+1|k − Pk+1|k+1 )
= ck+1 + x̂Tk+1|k Sk+1 x̂k+1|k
h i
+ tr(AT Sk+1 APk|k ) + tr(Sk+1 Q) − tr Sk+1 Pk+1|k+1 .

Therefore, the Bellman equation can be simplified into

 T
Vk (x̂k|k ) = x̂Tk|k W x̂k|k + ck + min ν̄uTk U uk + x̂k+1|k Sk+1 x̂k+1|k
uk
 
= x̂Tk|k W + (1 − ν̄)AT Sk+1 A x̂k|k + ck
 
+ ν̄ min uTk U uk + (Ax̂k|k + Buk )T Sk+1 (Ax̂k|k + Buk )
uk
 
= x̂Tk|k W + (1 − ν̄)AT Sk+1 A x̂k|k + ck
 
+ x̂Tk|k ν̄AT Sk+1 A − ν̄AT Sk+1 B(U + B T Sk+1 B)−1 B T Sk+1 A x̂k|k
= x̂Tk|k Sk x̂k|k + ck ,

with minimizer uk = −(U + B T Sk+1 B)−1 B T Sk+1 Ax̂k|k .

3.2. Control in Lossy Networks 55

Now V0 (x̂0|0 ) = x̂T0|0 S0 x̂0|0 + c0 , with x̂0|0 = x̄0 + K0 (y0 − C x̄0 ).

Furthermore, y0 is Gaussian distributed with mean C x̄0 and covariance
CΣ0 C T + R.
h i
Z exp − 12 ky0 − C x̄0 k2(CΣ0 C T +R)−1
JN = q V0 (x̄0 + K0 (y0 − C x̄0 ))dy0
Rm (2π)m |CΣ0 C T + R|
= c0 + x̄T0 S0 x̄0 + tr(K0 (CΣ0 C T + R)K0T S0 )
 
= c0 + x̄T0 S0 x̄0 + tr (Σ0 − P0|0 )S0
= x̄T0 S0 x̄0 + tr(Σ0 S0 )
N
X −1 N
X −1  
+ tr(Sk+1 Q) + tr (AT Sk+1 A + W − Sk )Pk|k ,
k=0 k=0

which finishes the proof.

For the TCP case, one could also consider the infinite horizon LQG
problem, where
JN
J∞ = lim .
N →∞ N
The following theorem provides the optimal solution for the infinite
horizon LQG problem:

Theorem 3.14. If there exists a positive definite S, such that

S = W + AT SA − ν̄AT SB(U + B T SB)−1 B T SA, (3.25)

then

J∞ = tr(SQ + (AT SA + W − S)P ),

where P = limk→∞ Pk|k and the optimal control is

uk = −Lx̂k|k = −(U + B T SB)−1 B T SAx̂k|k . (3.26)

The theorem can be easily proved from Theorem 3.13 by letting N

go to infinity and the fact that Sk converges to S.
56 Networked Control Systems

Example 3.2. In this example, we consider an infinite horizon LQG

controller where the control packet is subject to packet drops. It is
assumed that a TCP-like protocol is used to transmit the control packet.
The system parameters are chosen to be

A = 1.5, B = C = Q = R = U = W = 1.

Figure 3.5 shows the optimal control gain L defined in (3.26) versus
the packet drop rate 1 − ν̄. Notice that (3.25) does not admit a solution
when the packet drop rate 1 − ν̄ ≥ 1/A2 = 4.9. One can see that as the
packet drop rate increases, the control gain L converges to A. Intuitively
speaking, this means that the controller is trying to drive the state xk
back to 0 in one step, instead of accomplishing it over multiple steps,
since future control packets may not be successfully delivered.
Figure 3.6 shows the tail distribution of |xk | for packet drop rates
of 1/4 and 4/9 respectively. For 1 − ν̄ = 1/4, the optimal control gain
from Theorem 3.14 is used. For 1 − ν̄ = 4/9, we use the limiting gain
L = A.

1.5

1.4

1.3
L

1.2

1.1

0 0.1 0.2 0.3 0.4

1 − ν̄

Figure 3.5: The optimal LQG gain L versus the packet drop rate. The system is
chosen so that A = 1.5 and B = C = Q = R = U = W = 1.
3.2. Control in Lossy Networks 57

100
10−1
10−2
Pr(|xk | ≥ x)

10−3
10−4
10−5
10−6

10−1 100 101 102 103

x
Figure 3.6: The tail distribution of |xk |. The system is chosen so that A = 1.5 and
B = C = Q = R = U = W = 1. The solid line corresponds to a control packet drop
rate of 1/4 and the dashed line corresponds to a control packet drop rate of 4/9.

3.2.2 UDP-like Protocol

In this subsection, we consider the system design problem under a
UDP-like protocol. The main difference between UDP-like and TCP-
like protocols are that the controller and estimator do not know if the
control input uk is received and applied by the actuator. As a result,
they have to infer the packet loss process from the sensor measurements,
which makes the analysis for the UDP-like protocol considerably more
difficult than the analysis for TCP-like protocol. We will first focus on
the estimator design, which is given by the following theorem:
Theorem 3.15. The conditional distribution of xk given Gk or Gk−1
under UDP-like protocol is a mixture of multivariate Gaussian distribu-
tions with different means and the same covariance, i.e.,
k −1
2X
f (xk |Gk ) ∼ φk,i N (ξk,i , Pk|k ), (3.27)
i=0
k −1
2X
f (xk |Gk− ) ∼ φ− −
k,i N (ξk,i , Pk|k−1 ). (3.28)
i=0
58 Networked Control Systems

φk,i , ξk,i , Pk|k and φ− −

k,i , ξk,i , Pk|k−1 satisfy the following equations:

φ−
k,i αk,i
φk,i = P2k −1 , (3.29)
i=0φ−k,i αk,i
1
 
− T T −1 −
with αk,i = exp − (yk − Cξk,i ) (R + CPk|k−1 C ) (yk − Cξk,i ) ,
2
(3.30)
− −
ξk,i = ξk,i + Pk|k−1 C T (CPk|k−1 C T + R)−1 (yk − Cξk,i ), (3.31)
Pk|k = (Pk|k−1 + C T R−1 C)−1 , (3.32)

and

φ− −
k+1,2i = (1 − ν̄)φk,i , φk+1,2i+1 = ν̄φk,i , (3.33)
− −
ξk+1,2i = Aξk,i , ξk+1,2i+1 = Aξk,2i + Buk , (3.34)
T
Pk+1|k = APk|k A + Q, (3.35)

subject to initial conditions

φ− −
0,0 = 1, ξ0,0 = x̄0 , P0|−1 = Σ0 .

Before proving the theorem, we would like to point out for each xk ,
the number of Gaussian distributions in the mixture is 2k . Conceptually,
this is due to the fact that there are 2k possible realizations of the
packet drop process ν0 , . . . , νk−1 . Therefore, a Kalman filter is built for
each possible realization. In fact, the update equation of ξk,i is exactly
the same as the Kalman filter for xk , assuming that νj = ij , where ij
is the jth digit in the binary representation of i. Similarly, Pk|k is the
corresponding estimation error. φk,i can be interpreted as how likely
this particular sequence of packet drop process ν0 , ν1 , . . . , νk−1 happens,
given our observation y0 , . . . , yk . It is not difficult to see that φk,i is
P k
positive and 2i=0−1 φk,i = 1.
The following lemma, which can be verified easily, is used to simplify
the proof:
3.2. Control in Lossy Networks 59

Lemma 3.16. The following equalities hold:

(x − x̂− )T P −1 (x − x̂− ) + (y − Cx)T R−1 (y − Cx)

 
= (x − x̂)T P −1 + C T R−1 C (x − x̂)
 −1
+ (y − C x̂− )T R + CP C T (y − C x̂− ),

where x̂ = x̂− + P C T (CP C T + R)−1 (y − C x̂− ).

We are now ready to prove Theorem 3.15.

Proof. We prove the theorem by induction. Clearly, f (x0 |G0− ) is the

prior distribution of x0 , i.e., N (x̄0 , Σ0 ). Now suppose f (xk |Gk− ) satisfies
(3.28), then by Bayes’ rule
f (xk |Gk− )f (yk |xk )
f (xk |Gk ) = , (3.36)
f (yk |Gk− )
where we use the fact that f (yk |xk , Gk− ) = f (yk |xk ).
For the denominator on the RHS of (3.36), yk conditioned on Gk− is
also a mixture of Gaussian distributions with
k −1
2X φ−
k,i αk,i
f (yk |Gk− ) = q .
i=0 (2π)m |CPk|k−1 C T + R|

Notice that f (yk |xk ) is given by

1 1
 
f (yk |xk ) = p m
exp − (yk − Cxk )T R−1 (yk − Cxk ) .
(2π) |R| 2
Therefore, the numerator on the RHS of (3.36) can be written as
k −1
2X φ−
k,i 1
 
f (xk |Gk− )f (yk |xk ) = q exp − ςk,i ,
i=0 (2π)m+n |Pk|k−1 R| 2

with
− T −1 −
ςk,i = (xk − ξk,i ) Pk|k−1 (xk − ξk,i ) + (yk − Cxk )T R−1 (yk − Cxk )
 −1
= (xk − ξk,i )T Pk|k (xk − ξk,i )
− T −
+ (yk − Cξk,i ) (R + CPk|k−1 C T )−1 (yk − Cξk,i ).
60 Networked Control Systems

Therefore, the numerator on the RHS is

f (xk |Gk− )f (yk |xk )

k −1
2X
φk,i 1
 
= f (yk |Gk− ) q −1
exp − (xk − ξk,i )T Pk|k (xk − ξk,i ) ,
i=0
m
(2π) |Pk|k | 2

which proves (3.29), (3.31), and (3.32).

Now suppose that f (xk |Gk ) satisfies (3.27). Since uak = νk uk , (3.33),
(3.34), and (3.35) can be trivially proved.

We now consider the LQG controller design problem for a UDP-like

protocol. Similar to the TCP-like protocol, we seek to minimize the
cost defined in (3.21), where uk must be a measurable function of the
information set Gk . For the UDP case, f (xk |Ik ) is a Gaussian mixture,
with parameters φk,0 , . . . , φk,2k −1 and ξk,0 , . . . , ξk,2k −1 . Hence, we only
need to consider Vk on Gaussian mixture distributions. For simplicity,
let us define φk as (φk,0 , . . . , φk,2k −1 ) and ξk as (ξk,0 , . . . , ξk,2k −1 ). With
slight abuse of notation, we will write Vk (f (xk |Ik )) as Vk (φk , ξk ). The
Bellman equation (3.23) can be written as
k −1
2X
T
Vk (φk , ξk ) = φk,i ξk,i W ξk,i + tr(Pk|k W )
i=0
 Z 
+ min ν̄uTk U uk + f (yk+1 |Gk , uk )Vk+1 (φk+1 , ξk+1 )dyk+1 ,
uk Rm

where
Z
f (yk+1 |Ik− )Vk+1 (f (xk+1 |yk+1 , Ik− ))dyk+1
Rm
 
2k+1
X−1 φ−
k+1,i αk+1,i
Z
=  q  Vk+1 (φk+1 , ξk+1 )dyk ,
Rm i=0 (2π)m |CPk+1|k C T + R|
3.2. Control in Lossy Networks 61

with

φ− −
k+1,2i = (1 − ν̄)φk,i , φk+1,2i+1 = ν̄φk,i ,
− −
ξk+1,2i = Aξk,i , ξk+1,2i+1 = Aξk,2i + Buk ,
φ−k+1,i αk+1,i
φk+1,i = P2k+1 −1 − ,
i=0 φ k+1,i α k+1,i
− −
ξk+1,i = ξk+1,i + Pk+1|k C T (CPk+1|k C T + R)−1 (yk+1 − Cξk+1,i ),

and αk+1,i given by

1
 
− −
exp − (yk+1 − Cξk+1,i )T (R + CPk+1|k C T )−1 (yk+1 − Cξk+1,i ) .
2
However, even though we have written down the Bellman equation
explicitly, it is non-trivial to solve the optimal LQG control problem
for a longer horizon. Next we will only show how to solve the Bellman
equation for VN and VN −1 . First, VN (φN , ξN ) is given by
N −1
2X
T
VN (φN , ξN ) = φN,i ξN,i W ξN,i + tr(PN |N W ).
i=0

As a result,
Z
− −
f (yN |IN −1 )VN (f (xN |yN , IN −1 ))dyN = tr(PN |N W )
Rm
N −1
2X
αN,i
Z
+ φ−
N,i q T
ξN,i W ξN,i dyN .
i=0 Rm (2π)m |CPN |N −1 C T + R|

Now using the fact that

− −
ξN,i = ξN,i + PN |N −1 C T (CPN |N −1 C T + R)−1 (yN − CξN,i ),

we can prove that

αN,i
Z
T
q ξN,i W ξN,i dyN
Rm (2π)m |CPN |N −1 C T + R|
 T h i
− −
= ξN,i W ξN,i + tr PN |N −1 C T (CPN |N −1 C T + R)−1 CPN |N −1 W
 T h i
− −
= ξN,i W ξN,i + tr (PN |N −1 − PN )W .
62 Networked Control Systems

Thus, the Bellman equation can be simplified to

VN −1 (φN −1 , ξN −1 )
−1 −1
2NX
T
= φN −1,i ξN −1,i W ξN −1,i + tr(PN −1|N −1 W )
i=0
h i
+ tr (PN |N −1 − PN |N )W
N −1
2X  T
+ min ν̄uTk U uk + φ− −
N,i ξN,i
−
W ξN,i
u k
i=0
To simplify notation, let us define the X-norm of a vector v to be
√
kvkX = v T Xv,
where X > 0, i.e, it is a positive definite matrix. Notice that
N −1
2X  T
ν̄uTk U uk + φ− −
N,i ξN,i
−
W ξN,i =
i=0
−1 −1
2NX
ν̄uTk U uk + ν̄φN −1,i kAξN −1,i + Buk k2W + (1 − ν̄)kAξN −1,i k2W ,
i=0
the minimum of which is
2
−1 −1
2NX 2N −1 −1
X
2

φN −1,i kAξN −1,i kW − ν̄
φN −1,i ξN −1,i

i=0 i=0
Z

(where Z = AT B(U + B T W B)−1 B T A) and is achieved when

−1 −1
2NX
T −1 T
uN −1 = −(U + B W B) B A φN −1,i ξN −1,i ,
i=0
As a result, we have
−1 −1
2NX
VN −1 = φN −1,i kξN −1,i k2W +AT W A
i=0
h i
+ tr (PN −1|N −1 + PN |N −1 − PN |N )W
2
2N −1 −1
X
− ν̄
φ ξ
N −1,i N −1,i
(3.37)
i=0 T
A B(U +B T W B)−1 B T A
3.3. Designing Sensor Network with Resource Constraints 63

Notice that the Bellman equation of VN −2 does not have an analytical

form due to the last term on the RHS of (3.37), since both φN −1,i and
ξN −1,i depend on uk−2 . Hence, in general, the optimal control problem
of the UDP case cannot be solved in a closed form.

3.2.3 Further Reading

The discussion in this subsection is mainly-based on the work of Schen-
ato et al. (2007). However, we have simplified the problem setting by
assuming that the sensor measurements can be transmitted perfectly.
The readers are encouraged to read Schenato et al. (2007) for a more
thorough treatment on the combined estimation and control over a lossy
network problem.
It is worth noticing that in Schenato et al. (2007), the estimator
for the UDP case is linear and hence it is not a minimum mean square
error estimator, since we have proved that the optimal Bayes estimator
is non-linear. Therefore, the UDP control law and the value function is
slightly different.
Finally, for the Markovian packet drop model, readers can refer to
Mo et al. (2013).

3.3 Designing Sensor Network with Resource Constraints

In this subsection we consider the problem of designing a sensor network

with resource constraints on sensors, such as:

• Bandwidth constraints. For example, a maximum of p out of a total

of m sensors can communicate to the fusion center simultaneously.

• Energy constraints. For example, sensor i cannot transmit more

than half of the time.

• Topological constraints. For example, sensor j needs to relay the

message from sensor i to the fusion center.

The ultimate goal is to generate a schedule describing which set of

sensors should communicate at each time step. In the literature, such a
schedule can be broadly classified into two categories:
64 Networked Control Systems

1. Off-line schedule: The schedule only depends on the statistics of

the sensory data instead of the real-time sensor measurements
and hence can be generated off-line.
2. Online schedule or event-based schedule: The schedule depends
on both the statistics and the realization of the sensory data and
hence can only be generated online.
In the next subsection, we first consider the offline schedule. The
online schedule will be discussed later in subsection 3.3.3.

3.3.1 Off-line Sensor Selection

Consider a sensor network illustrated in Figure 3.1. However, instead
of considering γk,i as random packet drops caused by imperfect com-
munication, in the sensor scheduling problem, we consider γk,i to be
a design variable, such that the system operator can choose whether
γk,i = 0 or 1.
At each time step, the set of sensors being selected to transmit is
defined as
σk , {i : γk,i = 1} .
We further define
Γk , diag{γk,1 , . . . , γk,m }, C̃k , Γk C, R̃k , Γk RΓk .
It is easy to see that Γk is a function of σk . Hence, we can write it as
Γk (σk ). However, we will write Γk for short if there is no confusion.
For an off-line scheduling, Γk is predetermined and independent of
the sensor measurements yk . Hence, the system can be treated as a
linear system with a time varying C̃k and the optimal filter is the same
as the Kalman filter described in subsection 3.1.1, where the estimator
covariance Pk|k satisfies the following recursive Riccati equation:
 −1
Pk+1|k+1 = (APk|k AT + Q)−1 + C̃kT R̃k−1 C̃k .

Let C̃ , ΓC, R̃ , ΓRΓ. With slight abuse of notations, we define7

 −1
g(X, Γ) , h(X)−1 + C̃ T R̃−1 C̃ ,
7
Notice that g(X) defined in subsection 3.1.2 equals to g(X, I).
3.3. Designing Sensor Network with Resource Constraints 65

A sensor schedule σ = (σ0 , σ1 , . . .) is an infinite sequence of σk s,

indicating the set of sensors being selected at time k. In this subsec-
tion, we attack the sensor scheduling problem using a moving horizon
approach, in the sense that we only try to optimize a finite sequence of
(σk , . . . , σk+T −1 ) inside a window of size T at time k. We call this opti-
mization problem over a finite horizon as the sensor selection problem.
Without loss of generality we can assume the window size is 1, since we
can stack the state vector to consider longer horizons.
The off-line sensor selection problem can be formulated as
minimize f (Pk|k )
Γk
subject to Pk|k = g(Pk−1|k−1 , Γk ), (3.38)
H~γk ≤ b,
γk,i = 0 or 1,
h iT
where ~γk = γk,1 · · · γk,m , H and b are matrix and vector of proper
dimensions respectively.
Next we shall explain the objective function and the constraints in
the Problem (3.38). Some possible choices of f include:
• f (X) = log det(X), which is related to the entropy of the estima-
tion error.

• f (X) = tr(F XF T ), where F is a real matrix of proper dimension.

• f (X) = kXk. An interesting example is the induced 2-norm, which

is the spectral radius of X for symmetric matrices.
Notice that all objective functions f (X) are non-decreasing with respect
to X, which ensures that we are minimizing the error.
The constraint H~γk ≤ b can be used to describe various resource
constraints:
• Bandwidth Constraints Assume at each time step we can select no
more than p < m sensors. Thus, the constraints can be written as
1T ~γk ≤ p,
where 1 is an all one vector.
66 Networked Control Systems

• Topological Constraints Consider that the sensor network has a

tree structure with the fusion center as the root. As a result, a
sensor needs to be selected if one of its child sensors is selected.

γi ≥ γj , if j is a child of i. (3.39)

Suppose we only consider the bandwidth constraint for the one

step selection problem. In other words, we only require 1T ~γk = p. The
following greedy approach Shamaiah et al. (2010), which selects one
sensor per iteration, can be adopted to solve Problem (3.38):
Algorithm 2: Greedy Algorithm for Scheduling Problem (3.38)
Result: Γk (σk )
σk ← ∅;
while |σk | < p do
s ← arg min1≤s≤m f (g(Pk−1|k−1 , Γk (σk {s}));
S

σk ← σk {s};
S

end
To prove a performance bound for the greedy algorithm, we need a
definition of submodular functions:

Definition 3.2. Let S be a finite set and 2S be the power set of S. A

function f : 2S → R is submodular if for all A, B ∈ 2S , the following
inequality holds:
[ \
f (A B) + f (A B) ≤ f (A) + f (B).

Now we have the following theorem (Shamaiah et al., 2010) to

characterize the submodularity of the objective function:

Theorem 3.17. Suppose that R is a diagonal matrix. The following

function φ is monotonically increasing and submodular:

φ(σk ) = − log det g(Pk−1|k−1 , Γk (σk )).

We now have the following theorem to provide a performance bound

on the greedy algorithm:
3.3. Designing Sensor Network with Resource Constraints 67

Theorem 3.18. Suppose that the optimal value of the following opti-
mization problem is ν∗

minimize log det(Pk|k )

Γk
subject to Pk|k = g(Pk−1|k−1 , Γk ), (3.40)
1T ~γk ≤ q,
γk,i = 0 or 1.

Suppose that the solution from the greedy algorithm is Γk (σk ) and
the corresponding log det Pk|k = ν. If R is diagonal, then the following
inequality holds:

0 ≤ ν − ν∗ ≤ (log det(h(Pk−1|k−1 )) − ν∗ )/exp(1),

Proof. Since φ is non-decreasing and submodular, the theorem can be

proved from Theorem 4.2 in Nemhauser et al. (1978) and the fact that
g(Pk−1|k−1 , Γk (σk )) = h(Pk−1|k−1 ).

Remark 3.2. Notice that the function − tr(g(Pk−1|k−1 , Γ(σk ))

and −ρ(g(Pk−1|k−1 , Γ(σk )) may fail to be submodular (Jawaid and
Smith, 2015). Hence, there is no known performance guarantee for the
greedy algorithm.

Next we shall consider using an optimization-based approach to

solve the scheduling Problem (3.38) in order to deal with more general
objective functions and constraints. We first focus on the case where
R is a diagonal matrix (Joshi and Boyd, 2009). The function g(X, Γ)
takes a simpler form if R is diagonal:
m
!−1
−1
X C T Ci
g(X, Γ) = (h(X)) + γi i ,
i=1
ri

where γi , ri are the ith diagonal entry of matrices Γ and R respectively

and Ci is the ith row of the matrix C.
68 Networked Control Systems

If the objective function f (X) = log det X, the scheduling prob-

lem (3.38) can be written as
m
!
C T Ci
− log det (h(Pk−1|k−1 ))−1 +
X
minimize γk,i i
γk,i ri
i=1
(3.41)
subject to H~γk ≤ b,
γk,i = 0 or 1.
For the case where f is a convex and non-decreasing function, using
the Schur complement, we can manipulate (3.38) into
minimize f (X)
γk,i
" #
X I
subject to 0≤ ,
I (h(Pk−1|k−1 )) + m
−1 T
P
i=1 γk,i Ci Ci /ri

H~γk ≤ b,
γk,i = 0 or 1.
(3.42)
In both Problem (3.41) and (3.42), the objective function and the
constraints are convex except for the binary constraints on γk,i . There-
fore, one way to relax the problem into a convex optimization problem
is to change the binary constraints γk,i = 0 or 1 to 0 ≤ γk,i ≤ 1. Notice
that the relaxation will increase the feasible space of γk and thus the
optimal value of the relaxed problem will be a lower bound for optimal
value of the original problem.
On the other hand, the optimal solution ~γk of the relaxed problem
is not necessarily a binary vector. One can quantize the solution to
get a binary vector (Joshi and Boyd, 2009). Alternatively, one can
adopt a stochastic scheduling strategy, where each sensor will transmit
with certain probability and ~γk can be interpreted as the transmitting
probability. For more details, please refer to Mo et al. (2011b).
Next we consider the case where R is not diagonal, which is consid-
ered in the multi-step sensor scheduling problem Rigtorp (2010). To
this end, the following lemmas are needed:
Lemma 3.19. Define the function L(K, X) as
L(K, X) = (I − KC)h(X)(I − KC)T + KRK T .
3.3. Designing Sensor Network with Resource Constraints 69

Then L(K, X) can be written as

L(K, X) = (K − K∗ (X))(Ch(X)C T + R)(K − K∗ (X))T + g(X),

where K∗ (X) = h(X)C T (Ch(X)C T + R)−1 .

Lemma 3.20. For any K of proper dimension, we have

g(X, Γ) ≤ L(KΓ, X)
= (KΓ − K∗ (X))(Ch(X)C T + R)(KΓ − K∗ (X))T + g(X).

Furthermore, the equality is achieved when

 −1
K = h(X)C̃ T C̃h(X)C̃ T + R̃ ,

with C̃ = ΓC and R̃ = ΓRΓ.

Proof. Similar to Lemma 3.19, we can see that

g(X, Γ) ≤ (I − K C̃)T h(X)(I − K C̃) + K R̃K T

= (I − KΓC)h(X)(I − KΓC)T + KΓR(KΓ)T = L(KΓ, X).

One can easily check the inequality is tight.

Let us denote KΓ as K, with Ki being the ith column of K. Then we

have the following theorem to convert the optimization problem (3.38)
into an almost convex form:

Theorem 3.21. If f is non-decreasing, then the one step optimization

problem (3.38) is equivalent to

minimize f (P ) (3.43a)
Γ, K, P
subject to P ≥ L(K, Pk−1|k−1 ), (3.43b)
H~γk ≤ b, (3.43c)
kKi k0 ≤ nγk,i , (3.43d)
γk,i = 0 or 1, (3.43e)

where kvk0 is the number of non-zero elements in v.

70 Networked Control Systems

Proof. Suppose Γk is the solution of the scheduling problem (3.38). One

can verify that the following (Γ, K, P ) are feasible for Problem (3.43)

Γ = Γk , K = h(Pk−1|k−1 )C̃ T (C̃h(Pk−1|k−1 )C̃ T + R̃)−1 Γ,

P = g(Pk−1|k−1 , Γ),

where C̃ = ΓC and R̃ = ΓRΓ. Therefore, the optimal value of (3.38) is

no less than the optimal value of (3.43).
On the other hand, suppose that Γ, K, P is the optimal solution for
(3.43). The constraint (3.43d) implies that there exists K, such that
K = KΓ. Hence, by Lemma 3.20, we have

P ≥ L(K, Pk−1|k−1 ) = L(KΓ, Pk−1|k−1 ) ≥ g(Pk−1|k−1 , Γ),

which implies that the optimal value of (3.38) is no greater than the
optimal value of (3.43), due to the non-decreasing property of f . As a
result, the optimal value of (3.38) equals the optimal value of (3.43).

The constraint (3.43b) can be written as a linear matrix inequality:

 
P − g(Pk−1|k−1 ) K − K∗ (Pk−1|k−1 )
 T  ≥ 0.
K − K∗ (Pk−1|k−1 ) Ch(Pk−1|k−1 )C T + R

Therefore, if the objective function f is convex, then we have two

non-convex constraints (3.43d) and (3.43e) in Problem (3.43), and a
reweighted L1 approximation (Candès et al., 2008) can be used to solve
the problem. For more details, please refer to Mo et al. (2011a).

3.3.2 Infinite Horizon Sensor Scheduling Problem

This subsection is concerned with designing infinite horizon offline
schedules, where instead of adopting a moving horizon approach to
optimize the performance of the system over the next T steps, we seek
to optimize the average performance of the system, which is defined as:

T
1 X
J (P0|−1 , σ) , lim sup f (Pk|k ), (3.44)
T →∞ T k=0
3.3. Designing Sensor Network with Resource Constraints 71

where Pk|k satisfies the recursive equation. Pk|k = g(Pk−1|k−1 , Γk (σk )).
It is clear that Pk|k is a function of both the initial condition P0|−1 and
the schedule σ and can be denoted as Pk|k (P0|−1 , σ). However, we will
write it as Pk|k for short if there is no confusion. We will assume that
the function f satisfies the following properties:
1. f is non-decreasing on the set of positive semidefinite matrices.

2. The preimage of any interval (−∞, φ] under f is bounded, i.e.,

{X : f (X) ≤ α} ≤ Mα I, ∀α > 0, (3.45)
where Mα depends on α.

3. For any  > 0, there exists a δ, such that for any X > 0
f ((1 + δ)X) ≤ (1 + )f (X). (3.46)

Remark 3.3. Notice that both the trace and maximum eigenvalue of X
satisfy (3.45). However, the log determinant fails to satisfy (3.45) since
the log determinant of diag(ρ, 1/ρ) is always 1. Equation (3.46) implies
that f is actually continuous.
J (P0|−1 , σ) can be seen as the average estimation error. Moreover,
let us define the average communication rate of sensor i as
N
1 X
ratei (σ) , lim sup 1s ∈I ,
N →∞ N k=1 i k
We further define feasible schedules:
Definition 3.3. A schedule σ is called feasible if for any initial condition
P0|−1 , J (P0|−1 , σ) is bounded (by a function of P0|−1 ).
Before continuing on, we need the following lemma by Mo et al.
(2014a) to establish another contraction property of g:
Lemma 3.22. For all ρ ≥ 0, X > 0, the following inequalities hold,
g(X, Γ) ≤ g((1 + ρ))X, Γ) ≤ (1 + ρ)g(X, Γ). (3.47)
Furthermore, if AXAT ≤ αQ, then
ρ
 
g(X, Γ) ≤ g((1 + ρ)X, Γ) ≤ 1 + g(X, Γ). (3.48)
1+α
72 Networked Control Systems

The contraction property of g enables us to show that J is actually

independent of the initial condition P0|−1 .

Theorem 3.23. If f is non-decreasing and satisfies (3.45) and (3.46),

Q > 0 and J (X, σ) < ∞ for some X > 0, then for all P0|−1 > 0, we
have
J (X, σ) = J (P0|−1 , σ) < ∞.

Proof. Due to symmetry, we only need to prove

J (Y, σ) ≤ J (X, σ) < ∞,

for any Y > 0. To this end, define ρk as

ρk , inf{ρ ≥ 0 : (1 + ρ)Pk|k (X, σ) ≥ Pk|k (Y, σ)}.

By the definition of Pk|k ,we have that

Pk+1|k+1 (Y, σ) = g(Pk|k (Y, σ), Γk+1 ) ≤ g((1 + ρk )Pk|k (X, σ), Γk+1 )
≤ (1 + ρk )g(Pk|k (X, σ), Γk+1 ) = (1 + ρk )Pk+1|k+1 (X, σ).

The second inequality is true due to (3.47). Therefore, we know that

(1 + ρk )Pk+1|k+1 (X, σ) ≥ Pk+1|k+1 (Y, σ), which implies that ρk+1 ≤ ρk .
As a result, ρk is monotonically non-increasing. At this point it remains
to prove that ρk → 0. To this end, let us select an α > 0, such that for
any X which satisfies f (X) < 2J (X, σ)

AXAT ≤ αQ.

Since we assume (3.45) holds and that Q > 0, we can always find such
an α. Now from the definition of J (X, σ), the following inequality holds
infinitely often (i.e. for an infinite number of integers k):

f (Pk|k (X, σ)) ≤ 2J (X, σ). (3.49)

Let ki be a time index when (3.49) holds. By (3.48), we have

1
ρki +1 ≤ ρk .
1+α i
3.3. Designing Sensor Network with Resource Constraints 73

Since (3.49) happens infinitely often and α > 0, it follows that ρk → 0.

Now by (3.46), we know that for any  > 0

f (Pk|k (Y, σ)) ≤ (1 + )f (Pk|k (X, σ)),

for sufficiently large k. Hence, J (Y, σ) ≤ J (X, σ), which finishes the
proof.

It can be seen that the average estimation performance depends on

the sensor schedule σ rather than the initial condition. In practice, it
is often desirable to apply periodic schedules since they require finite
memory to store, which leads to the following definition:

Definition 3.4. A schedule σ is called periodic if there exists T > 0,

such that σk+T = σk for all k.

The following theorem establishes the fact that periodic schedules

are “dense” in the set of all schedules:

Theorem 3.24 (Mo et al. (2014a)). Assume that f is non-decreasing

and satisfies (3.45) and (3.46) and Q > 0. For any feasible schedule
σ = (σ0 , σ1 , . . .) and for any ε, ε1 , . . . , εm > 0, there exists a periodic
schedule σ 0 , such that8

J (σp ) ≤ J (σ) + ε, (3.50)

and
ratei (σp ) ≤ ratei (σ) + εi , i = 1, . . . , m. (3.51)

Proof. The proof follows the same argument as Mo et al. (2014a).

As a result of Theorem 3.24, we only need to consider periodic

schedules. Suppose that the sensor network implements a schedule with
period 1. In other words, Γk = Γ is a constant matrix.9
Suppose that (A, Q1/2 ) is controllable, then if (A, ΓC) is detectable,
the fixed point of the Riccati recursion g(X, Γ) exists and is unique,
8
For simplicity, we write J (X, σ) as J (σ) due to Theorem 3.23.
9
This is also related to sensor placement problem, where we are given a large
number of possible sensor locations to place our sensors, which cannot move after
deployment.
74 Networked Control Systems

which we can define as P∗ (Γ). In other words, P∗ (Γ) is the unique

solution of following equation:

X = g(X, Γ).

On the other hand, if (A, ΓC) is not detectable, then we define P∗ (Γ) =
∞.
If f is continuous, then it is easy to prove that for the period 1
schedule, J = f (P∗ (Γ)). Similar to the schedule problem (3.38), the
periodic scheduling problem can be formulated as

minimize f (P∗ (Γ))

Γ
subject to H~γ ≤ b, (3.52)
γi = 0 or 1.

The following theorem is required to convexify (3.52):

Theorem 3.25. Assuming that Q is invertible, the following statements

are equivalent:

1. P∗ (I) ≤ P .

2. There exists an X ≤ P , such that g(X) ≤ X.

3. There exists a positive semidefinite matrix S ≥ 0, such that the

following linear matrix inequalities hold:
" #
S I
≥ 0, (3.53)
I P

and
" #
Q−1 − S + C T R−1 C Q−1 A
−1 ≥ 0. (3.54)
T
A Q S + AT Q−1 A

Proof. It is easy to see that statement 1 implies 2 since we can just

take X = P∗ (I). On the other hand, if g(X) ≤ X, by the monotonicity
of g(X), we have

P ≥ X ≥ g(X) ≥ g ◦ g(X) ≥ g ◦ g ◦ g(X) ≥ . . . .

3.3. Designing Sensor Network with Resource Constraints 75

The above non-increasing sequence is lower bounded by 0. Hence, the

above sequence converges to a fixed point of g(X). Due to the uniqueness
of the fixed point, we can conclude that P∗ (I) ≤ P .
To prove that statement 2 and 3 are equivalent, suppose that Q is
invertible, then g(X) is also invertible. Let us define S = X −1 .
Using the Schur complement, S −1 = X ≤ P is equivalent to (3.53).
On the other hand, g(X) ≤ X is equivalent to

(AXAT + Q)−1 + C T R−1 C ≥ X. (3.55)

Applying the matrix inversion lemma on the LHS of (3.55), we have

Q−1 − Q−1 A(X −1 + AT Q−1 A)−1 AT Q−1 + C T R−1 C ≥ X −1 ,

which is equivalent to (3.54).

Using Theorem 3.25, we know that if the objective function f is

convex and non-decreasing, then Problem (3.52) is equivalent to

minimize f (P ) (3.56a)
Γ
subject to P∗ (Γ) ≤ P, (3.56b)
H~γ ≤ b, (3.56c)
γi = 0 or 1, (3.56d)

where the constraint (3.56b) can be written as linear matrix inequalities:

" #
S I
≥ 0,
I P
and
" #
Q−1 − S + m T Q−1 A
P
i=1 γi Ci Ci /ri
−1 ≥ 0.
T
A Q S + AT Q−1 A

The binary constraint (3.56d) can be relaxed to make the problem

convex.
For the scheduling problem with correlated R, which typically arises
when we stack the state vector to consider the multi-step scheduling
problem, it is unknown how to directly optimize the fixed point P∗ (Γ)
in a form similar to (3.56).
76 Networked Control Systems

3.3.3 Event-based estimation

This subsection is devoted to the discussion of online schedules or event-
based schedules, which depend on the sensor measurements yk . We focus
on the scheduling of a single sensor subject to energy (communication
rate) constraints. For the extension to the multi-sensor case, please refer
to Weerakkody et al. (2016b).
Since only one sensor is used, let γk be a binary variable, such that
γk = 1 indicates the sensor measurement is transmitted at time k and
γk = 0 otherwise.
Wu et al. (2013) proposed the following deterministic event-triggering
scheme:

0 if kk k∞ ≤ δ,
γk = , (3.57)
1 otherwise

where δ is the pre-defined threshold and k = (CPk|k−1 C T +R)−1/2 (yk −

C x̂−
k ) is the normalized innovation vector. However, the optimal estima-
tor in this case is a Bayes filter, and hence it is difficult to implement.
In this subsection, we focus on a stochastic event-triggering scheme,
which leads to an optimal filter similar to the Kalman filter.
We assume that at every time step k, the sensor generates an IID
random variable ζk , which is uniformly distributed over [0, 1]. The
sensor then compares ζk with a function ϕ(yk ), which is defined as
1
 
ϕ(y) = exp − y T Y y , (3.58)
2
where Y is a pre-designed matrix. The sensor transmits if and only if
ζk > ϕ(yk ). In other words,

0, ζk ≤ ϕ(yk )
γk = . (3.59)
1, ζk > ϕ(yk )

Remark 3.4. Since ζk is uniformly distributed, one can interpret ϕ(yk )

as the probability of idle and 1−ϕ(yk ) as the probability of transmitting
for the sensor.
The choice of ϕ(y) = exp(−y T Y y/2) enables us to find a tractable
MMSE estimator.
3.3. Designing Sensor Network with Resource Constraints 77

Remark 3.5. Suppose that the system is unstable, then yk becomes

unbounded and the event-trigger will always trigger, which renders
it useless. Hence, we assume that A is stable. For unstable systems,
the event trigger needs to trigger on the innovations instead of raw
measurements, which requires either local state estimation or commu-
nication from the fusion center to the sensor. For more details, please
refer to Han et al. (2015).

From the choice of function ϕ, we know that if yk is small, then

with a large probability the sensor will be in the idle state. On the other
hand, if yk is large, then the sensor will be more likely to send yk . As a
consequence, even if the fusion center does not receive yk , it can still
exploit the information that yk is more likely to be small to update the
state estimate. This is the main advantage of an online schedule over an
offline sensor schedule, where no information about xk can be inferred
when yk is dropped.
The first question to address is what is the optimal (MMSE) estima-
tor for a system with our stochastic event trigger scheme. The answer
is given in the following theorem:

Theorem 3.26. Consider the problem of remote state estimation with

the event-triggered scheduler (3.59). Then xk conditioned on the infor-
mation set Ik−1 (defined in (3.2)) is Gaussian distributed with mean
x̂k|k−1 and covariance Pk|k−1 , and xk conditioned on Ik is Gaussian
distributed with mean x̂k|k and covariance Pk|k , where x̂k|k−1 , x̂k|k and
Pk|k , Pk|k−1 satisfy the following recursive equations:
Time update:

x̂k|k−1 = Ax̂k−1|k−1 , (3.60)

T
Pk|k−1 = APk−1|k−1 A + Q. (3.61)

Measurement update:

x̂k|k = x̂k|k−1 + γk Kk yk − Kk E[yk |Ik−1 ] (3.62)

= (I − Kk C)x̂k|k−1 + γk Kk yk , (3.63)
Pk|k = Pk|k−1 − Kk CPk|k−1 , (3.64)
78 Networked Control Systems

where
h i−1
Kk = Pk|k−1 C T CPk|k−1 C T + R + (1 − γk )Y −1 , (3.65)
with initial condition
x̂0|−1 = x̄0 , P0|−1 = Σ0 . (3.66)
Proof. We prove the theorem by induction. Since x0 is Gaussian, (3.66)
holds. We first consider the measurement update step. Assume that
xk conditioning on Ik−1 is Gaussian with mean x̂k|k−1 and covariance
Pk|k−1 . We consider two cases depending on whether the estimator
receives yk .
1. γk = 0:
If γk = 0, then the estimator does not receive yk . Consider the
joint conditional pdf of xk and yk ,
f (xk , yk |Ik ) = f (xk , yk |γk = 0, Ik−1 )
Pr(γk = 0|xk , yk , Ik−1 )f (xk , yk |Ik−1 )
=
Pr(γk = 0|Ik−1 ) (3.67)
Pr(γk = 0|yk )f (xk , yk |Ik−1 )
= .
Pr(γk = 0|Ik−1 )
The second equality follows the Bayes’ theorem and the last one
holds since γk is conditionally independent with (Ik−1 , xk ) given
yk . Let us define the covariance of [xTk , ykT ]T given Ik−1 as
" #
Pk|k−1 Pk|k−1 C T
Φk , (3.68)
CPk|k−1 CPk|k−1 C T + R
From the definition of ϕ, we have
 !
1  1
Pr(γk = 0|yk ) = Pr exp(− ykT Y yk )≥ζk yk = exp(− ykT Y yk ).

2  2

(3.69)
From (3.67), (3.68), and (3.69), we have
1
f (xk , yk |Ik ) = αk exp(− θk ),
2
3.3. Designing Sensor Network with Resource Constraints 79

where
1
αk = p
Pr(γk = 0|Ik−1 ) det(Φk )(2π)m+n
and
" #T " #
x − x̂k|k−1 xk − x̂k|k−1
θk = k Φ−1
k + ykT Y yk . (3.70)
yk − ŷk|k−1 yk − ŷk|k−1

Here, ŷk|k−1 = C x̂k|k−1 . Manipulating (3.70) using the matrix

inversion lemma, one has
" # " #
x − x̄k xk − x̄k
θk = k Θ−1
k + ck ,
yk − ȳk yk − ȳk

where

x̄k = x̂k|k−1 − Pk|k−1 C T (CPk|k−1 C T + R + Y −1 )−1 ŷk|k−1 ,

h i−1
ȳk = I + Y (CP C T + R) ŷk|k−1 ,
ck = (ŷk|k−1 )T (CPk− C T + R + Y −1 )−1 ŷk|k−1 ,

and " #
Θxx,k Θxy,k
Θk = ,
ΘTxy,k Θyy,k
with

Θxx,k = Pk|k−1 − Pk|k−1 C T (CPk|k−1 C T + R + Y −1 )−1 CPk|k−1 ,

h i−1
Θxy,k = Pk|k−1 C T I + Y (CPk|k−1 C T + R) ,
h i−1
Θyy,k = (CPk|k−1 C T + R)−1 + Y .

Thus, f (xk , yk |Ik ) is equal to

" # " #!
ck 1 xk − x̄k xk − x̄k
 
αk exp − × exp − Θ−1
k .
2 2 yk − ȳk yk − ȳk

Since f (xk , yk |Ik ) is a pdf,

Z Z
f (xk , yk |Ik )dxk dyk = 1,
Rn Rm
80 Networked Control Systems

which implies that

ck 1
 
αk exp − =p .
2 det(Θk )(2π)n+m
As a result, xk , yk are jointly Gaussian given Ik , which implies
that xk is conditionally Gaussian with mean x̄k and covariance
Θxx,k . Therefore, (3.62) and (3.64) hold when γk = 0.

2. γk = 1:
If γk = 1, then the estimator receives yk . Hence

f (xk |Ik ) = f (xk |γk = 1, yk , Ik−1 )

Pr(γk = 1|xk , yk , Ik−1 )f (xk |yk , Ik−1 )
=
Pr(γk = 1|yk , Ik−1 )
Pr(γk = 1|yk )f (xk |yk , Ik−1 )
= = f (xk |yk , Ik−1 ).
Pr(γk = 1|yk )
The second equality is due to Bayes’ theorem and the third equal-
ity uses the conditional independence between γk and (Ik−1 , xk )
given yk . Since yk = Cxk + vk and xk , vk are conditionally in-
dependently Gaussian distributed, xk and yk are conditionally
jointly Gaussian which implies that f (xk |Ik ) is Gaussian. As
f (xk |yk , Ik−1 ) represents the measurement update of the stan-
dard Kalman filter, following the standard Kalman filtering, we
have

f (xk |Ik ) ∼ N (x̂k|k−1 + Kk (yk − C x̂k|k−1 ), Pk|k−1 − Kk CPk|k−1 ).

Finally we consider the time update. Assume that xk conditioned

on Ik is Gaussian distributed with mean x̂k|k and covariance Pk|k .

f (xk+1 |Ik ) = f (Axk + wk |Ik ).

Since xk and wk are conditionally mutually independent Gaussian, we
have
f (xk+1 |Ik ) ∼ N (Ax̂k|k , APk|k AT + Q),
which completes the proof.
3.3. Designing Sensor Network with Resource Constraints 81

The next step is to analyze the performance of the event-based

schedule. Since Pk satisfies the following recursive equation
 −1
 h(Pk−1|k−1 )−1 + C T R−1 C

γ=1
Pk|k =  −1 ,
 h(Pk−1|k−1 )−1 + C T (R + Y −1 )−1 C
 γ=0

we can define the worst case estimation error covariance as P (Y ) as the

fixed point of the following Riccati equation:
h i−1
P (Y ) = h(P (Y ))−1 + C T (R + Y −1 )−1 C .

Since the system is assumed to be stable, P (Y ) exists and is unique as

long as Q > 0.
On the other hand, let us define the average communication rate to
be
−1
1 NX
rate , lim sup E[γk ]. (3.71)
N →∞ N k=0

The following theorem provides an explicit formula for the commu-

nication rate:

Theorem 3.27. Consider a system with an event-triggered scheduler

(3.59). If the system is stable, i.e., ρ(A) < 1, then the communication
rate is given by
1
rate = 1 − p , (3.72)
det(I + ΠY )
where Π = CSC T +R and S is the fixed point of the Lyapunov equation
S = h(S). Furthermore, the rate is bounded by
1
 
1
1 − (1 + tr(ΠY ))− 2 ≤ rate ≤ 1 − exp − tr(ΠY ) . (3.73)
2
82 Networked Control Systems

Proof. It is easy to see that yk is Gaussian distributed with zero mean.

Suppose the covariance of yk is Πk . From (3.59), we know that
1 1
     
Pr(γk = 0) = Pr ζk ≤ exp − ykT Y yk = E exp − ykT Y yk
2 2
 
Z exp − 12 ykT (Π−1
k + Y )yk
= p dyk
Rm det(Πk )(2π)m
 
1
Z exp − 12 ykT (Π−1
k + Y )yk
=p q dyk
det(I + Πk Y ) Rm det((Π−1 −1 m
k + Y ) )(2π)
1
=p ,
det(I + Πk Y )
where the last equality is due to the fact that the integration of a pdf
function over the entire space is equal to 1. From the definition of Π,
one can verify that
lim Cov(yk ) = Π.
k→∞
Hence,
1
rate = 1 − p ,
det(I + ΠY )
To prove the bound on the communication rate (3.73), we only need to
prove

1 + tr(ΠY ) ≤ det(I + ΠY ) ≤ exp(tr(ΠY ))). (3.74)

Since ΠY is similar to Π1/2 Y Π−1/2 , we know ΠY is diagonalizable and

all the eigenvalues are real and non-negative. Denote the eigenvalues of
ΠY to be ρ1 , . . . , ρm , then (3.74) is equivalent to
m
X m
Y m
Y
1+ ρi ≤ (1 + ρi ) ≤ exp ρi .
i=1 i=1 i=1

The first inequality can be easily proved by expanding m

Q
i=1 (1 + ρi ) and
the fact that ρi are non-negative. The second inequality is true since
1 + x ≤ exp x.

The final step is to design the event-trigger parameter Y in order to

optimize the estimation performance while satisfying the communication
3.3. Designing Sensor Network with Resource Constraints 83

rate constraint. This leads to the following optimization problem:

minimize f (P (Y )) (3.75a)
Y ≥0
subject to rate ≤ rate (3.75b)

The constraint (3.75b) is not convex in Y . However, if we replace

the communication rate by its upper bound, then (3.75b) is equivalent
to

tr(ΠY ) ≤ −2 log(1 − rate).

Assuming that f is non-decreasing and convex, we can relax (3.75)

to the following convex problem, using a similar procedure as in (3.52):

minimize f (P ) (3.76a)
Y ≥0
subject to P (Y ) ≤ P, (3.76b)
tr ΠY ≤ −2 log(1 − rate) (3.76c)

Constraint (3.76b) is equivalent to the following linear matrix in-

equalities
" # " #
S I Q−1 − S + C T (R + Y −1 )−1 C Q−1 A
≥ 0, ≥ 0.
I P AT Q−1 S + AT Q−1 A
The second linear matrix inequality can be further manipulated by
expanding (R + Y −1 )−1 with the matrix inversion lemma, which leads
to
" #
Q−1 − S + C T R−1 C Q−1 A
AT Q−1 S + AT Q−1 A
" #
C T R−1 h i
− (Y + R−1 )−1 R−1 C 0 ≥ 0, (3.77)
0

The equations (3.77) and Y + R−1 ≥ 0 are equivalent to

Q−1 − S + C T R−1 C Q−1 A C T R−1
 
T −1 T −1
A Q S+A Q A 0  ≥ 0. (3.78)
 

−1
R C 0 Y +R −1
84 Networked Control Systems

Example 3.3. We consider an example of stochastic event triggers.

Slightly different to what is shown here, we consider a multi-sensor
example where each sensor has an independent stochastic event trigger
(3.59). For more details on this implementation see Weerakkody et al.
(2016b). To assess performance, we consider a thermal model for data
centers, introduced in Parolini (2012). The size of data centers has been
growing both in number and capacity, resulting in rising energy costs.
To conserve energy, Parolini (2012) considers the following thermal
model for energy control.
Ṫsout Tsout
    
ks (Ψss − 1) ks Ψsc ks Ψso
 out     out 
Ṫc  =  kc Ψcs kc (Ψcc − 1) kc Ψco  Tc  + Bu,
Ṫoout ko Ψos ko Ψoc ko (Ψoo − 1) Toout

Tsin Ψss Ψsc Ψso Tsout

    
 in  
Tc  = Ψcs Ψcc Ψco  Tcout  + Du. (3.79)
 

Toin Ψos Ψoc Ψoo To out

Here the state x is a collection of output temperatures of devices while

the measured values y are the input temperatures of devices which
require multiple sensors. The subscripts represent different nodes under
consideration, where ‘s’ corresponds to servers, ‘c’ corresponds to air
conditioners, and ‘o’ corresponds to other devices. The inputs include a
reference temperature for the air conditioners, power consumed, and
temperature of heat sources. Ψ gives weight to how the temperature
output of each node affects the temperature into each node and k is a
set of thermal constants. Addressing the trade-off between estimation
and communication in this example will reduce energy expenditures
and data storage necessary for thermal control.
We linearize the system around its stable equilibrium, and assume
the inputs remain at or near their equilibrium values for all time, a
valid assumption during the night or backup periods. Furthermore, we
1
sample the system at a rate of 150 Hz. We consider a system with 16
servers, 3 air conditioners, and 1 other device. The matrices Q and R
are generated as a product of a random matrix with entries uniform
from 0 to 1 multiplied by its transpose. The matrices are scaled so
that the average magnitude of error in wk is 0.1 Kelvin and in vk is 0.5
Kelvin.
3.3. Designing Sensor Network with Resource Constraints 85

In Figure 3.7, we plot the apriori mean squared error in the state
estimate as a function of the average communication rate, where each
data point is obtained over a run of 10,000 trials. We consider 4 main
designs. We first consider a random design where for each sensor at
each time step, the probability of transmission is λavg . We also consider
a stochastic design where each sensor communicates at the same rate,
and an optimized design. In the optimized design, the communication
rate for each sensor is chosen so as to minimize an objective function
related to the total communication rate subject to an upper bound on
the covariance of the estimation error.
Finally, for comparison we include a deterministic trigger defined
(i)
where sensor i transmits if kyk k > δ (i) and does not transmit otherwise.
δ (i) is chosen so sensor i communicates at the same rate as sensor i in the
optimized stochastic trigger. A sub-optimal estimator is incorporated
here where a posteriori estimates and error covariances are obtained
using a Kalman filter for just the received measurements. Also shown
are upper and lower bounds for the un-optimized approach.
In Figure 3.8, we plot the percent improvement of the stochastic and
deterministic designs relative to the random design with regards to the
mean squared error plotted in Figure 3.7. An un-optimized stochastic
design provides as much as 15% improvement, a deterministic design
offers as much as 20% improvement, and the optimized stochastic design
offers as much as 30% improvement.

3.3.4 Further Reading

In this subsection, we consider the problem of scheduling sensors with
resource constraints. It is worth noticing that we assume the sensors
only transmit raw measurements, while in principle the sensors can use
coding to reduce the number of communications. One strategy is to
deploy smart sensors, where the sensor itself can perform the Kalman
estimation and transmit the estimated states back to the fusion center.
In that case, the problem can be formulated as a Markov decision
process with countably many states (Han et al., 2017a) and for some
special cases (Shi and Zhang, 2012), the optimal schedule can be solved
explicitly.
86 Networked Control Systems

1.5
Stochastic Un-Optimized
1.4
1.4 Stochastic Optimized
Random Offline
1.3 Deteministic Trigger
Upper Bound Un-Optimized X
1.2
1.2 Lower Bound X


X=
Mean Squared Error

1.1

11

0.9



0.8
0.8

0.7


0.6
0.6

0.5
00 0.1
0.2
0.2 0.3
0.4
0.4 0.5
0.6
0.6 0.7
0.8
0.8
λavg , Communication Rate
0.9
11
Communication Rate
avg

Figure 3.7: Mean square error (Kelvin2 ) for random, deterministic, stochastic, and
stochastic optimized strategies vs λavg , the communication rate

It is also worth noticing that in practice, the system may be resource

constrained as well as suffering from imperfect communication such as
packet drops. The readers can refer to Mo et al. (2012b) for a discussion
on scheduling with imperfect information.

3.4 Event-based control

Control systems typically update their measurements and inputs peri-

odically according to some chosen sample interval. Although periodicity
simplifies the analysis and digital implementation, it also leads to a
conservative usage of resources. In contrast, choosing update instants
intelligently can be used to improve the performance of a CPS and en-
sure more efficient resource utilization. This has been the main rationale
for what has been identified in the literature as event-based or event
triggered control. In event-based control, inputs are updated according
to event triggers-based on the sensor measurements. As opposed to
3.4. Event-based control 87

35
Stochastic Un-Optimized

3030 Stochastic Optimized
Deterministic Trigger

25

% Improvement Compared to Random

2020

15


1010

5


00

-5


00 0.1
0.2
0.2 0.3
0.4
0.4 0.5
0.6
0.6 0.7
0.8
0.8
λavg , Communication Rate
0.9
11

Communication Rate
avg

Figure 3.8: Percent improvement of designed stochastic triggers and deterministic

trigger compared to random triggers

changing inputs periodically, here inputs are altered at intervals which

are functions of the system state. For instance, inputs can be updated
when feedback is deemed necessary to correct discrepancies between
a desired state and the current state. Alternatively, updates may not
be needed when certain performance metrics are satisfied. Event-based
control can also make more efficient use of system resources. In particu-
lar, communication is triggered only when it is needed for performance
and this will reduce the cost of monitoring.
We start with considering the following continuous time linear system
model.
ẋ(t) = Ax(t) + Bu(t), u(t) = Lx(t). (3.80)
Here x(t) ∈ Rn and u(t) ∈ Rp represent the state and the control input,
respectively. Let us assume for now that perfect and complete state
observations are made. Here L is chosen so that the matrix (A + BL) is
Hurwitz and thus the resulting system is globally asymptotically stable.
88 Networked Control Systems

In practice, control inputs can not be changed continuously. Rather,

the state is sampled at discrete instants t1 , t2 , t3 , · · · . Typically, a zero
order hold strategy is employed, resulting in the control input being
kept constant in between update periods. In this case, we observe that
u(t) = u(ti ) = Lx(ti ), ti ≤ t < ti+1 (3.81)
The dynamics equation can, in turn, be rewritten as follows
ẋ(t) = (A + BL)x(t) + BLe(t), e(t) = x(ti ) − x(t), ti ≤ t < ti+1 .
(3.82)
Note here that since (A + BL) is Hurwitz, it can be shown that the
system (3.82) is input to state stable with respect to the measurement
error e(t). Traditionally, the control update instants are periodic, that
is ti+1 − ti = T . Our goal is to contrast this case to the case when these
instants are changed by using an event trigger, i.e., the case when the
control instants are determined by examining the state of the system.
In particular, we are interested in the case when update times are
chosen to maximize control performance while constraining the rate of
communication.
For instance, to construct a trigger, we can examine Lyapunov
stability. Since (A + BL) is Hurwitz, we know we can find positive
definite matrices P > 0 and Q > 0 such that
(A + BL)T P + P (A + BL) = −Q. (3.83)
As such, V (x) = xT P x is a Lyapunov function satisfying
λm (P )kxk22 ≤ V (x) ≤ λM (P )kxk22 ,
∂V (x(t))
= −xT Qx + xT P BLe + eT LT B T P x, (3.84)
∂t
≤ −λm (Q)kxk22 + 2kP BLk2 kek2 kxk2 , (3.85)
where λm (P ) is the smallest eigenvalue of positive definite matrix P and
λM (P ) is the largest eigenvalue of P . Note that the global asymptotic
stability of the system is guaranteed if 2kP BLk2 kek2 < λm (Q)kxk2 . To
attain global asymptotic stability, we can update the control input at
instants ti where
λm (Q)
ke(ti )k2 = σkx(ti )k2 , σ < . (3.86)
2kP BLk2
3.4. Event-based control 89

It is important to ensure that the update rule of an event trigger can

be physically implemented. In particular, we would like to assure that
time elapsed between two consecutive update instants ti+1 − ti is large
enough to be implemented using our sensing and actuation hardware
along with our communication network. In event triggered control, it is
possible that infinitely many update instants may be triggered over a
finite time in what is known in the literature as Zeno behavior. Naturally
this behavior is undesirable. Fortunately, we can characterize bounds
on inter-event times using the following result by Tabuada (2007).

Theorem 3.28. Consider the control system ẋ(t) = Ax(t) + Bu(t), with
controller u = Lx. Assume (A + BL) is Hurwitz, and event trigger
(3.86) is implemented. Then the inter-event times ti+1 − ti , i ∈ N are
bounded below by τ which satisfies,

φ(τ, 0) = σ, (3.87)

where φ(t, φ0 ) is the solution of

∂φ
= kA + BLk2 + (kA + BLk2 + kBLk2 )φ + kBLk2 φ2 (3.88)
∂t
satisfying φ(0, φ0 ) = φ0 .

Alternative triggers can be used. For instance, it may be a desirable

goal to ensure
∂V (x(t))
≤ −αxT Qx, (3.89)
∂t
where α ∈ [0, 1). From (3.84) the case with α = 1 would require
continuous sampling to ensure the error e(t) is identically zero. A
trigger which would guarantee the property (3.89) could implicitly
define triggering instants ti that satisfy
" #T " #" #
x(ti ) (α − 1)Q P BL x(ti )
= 0. (3.90)
e(ti ) LT B T P 0 e(ti )

It can be shown that such a trigger guarantees inter-event times which

are bounded below by a constant greater than 0. Nonetheless, in the
case of output feedback, a similar design can lead to Zeno behavior.
90 Networked Control Systems

Delay: The analysis presented above considers an idealized scenario

where perfect state observation is performed, complete model knowledge
is assumed, there are no disturbances, and no delay. We now begin
to examine scenarios which lift these assumptions. First, in practice,
there is some delay ∆ > 0 between the time a measurement is taken
and the time an associated control input is implemented. Designers
need to ensure that an event trigger accounts for this imperfection in
performance. In this case, we observe that

u(t) = u(ti + ∆) = Lx(ti ), ti + ∆ ≤ t < ti+1 + ∆. (3.91)

The dynamics equation can in turn be rewritten so that

ẋ(t) = (A+BL)x(t)+BLe(t), e(t) = x(ti )−x(t), ti +∆ ≤ t < ti+1 +∆.

(3.92)
For ∆ > 0, we would like to ensure we execute a control task
before kek2 = σkxk2 in order to account for the delay between taking a
measurement and updating the control input. According to Tabuada
(2007), we can do this by using the trigger

ke(ti )k2 = σ 0 kx(ti )k2 , (3.93)

where we choose σ 0 such that

h i
∆ A + BL BL (σ + 1)

h 2
i ≤ σ 0 ≤ φ(−∆, σ). (3.94)
1 − ∆ A + BL BL (σ + 1)

2

Given such a trigger, Tabuada (2007) shows that for all t

ke(t)k2 ≤ σkx(t)k2 .

Moreover, the inter-event times ti+1 − ti , i ∈ N for the trigger (3.93)

are bounded below by ∆ + τ where τ satisfies
 h i 
∆ A + BL BL (σ + 1)

φ τ, h 2
i  = σ0. (3.95)
1 − ∆ A + BL BL (σ + 1)

2
3.4. Event-based control 91

Modeling Uncertainties: We now consider a scenario where the

operator has imperfect model knowledge. In particular, we assume
ẋ(t) = Ax(t) + Bu(t), u(t) = Lx̂(t). (3.96)
Here we assume that instead of using a zero order hold in between
events, the operator attempts to maximize the interval between events
by incorporating a state estimate x̂(t), which leverages the operator’s
model knowledge.
˙
x̂(t) = (Â + B̂L)x̂(t), ti ≤ t < ti+1 . (3.97)
At update instants, ti , x̂(ti ) = x(ti ). The error is defined as e(t) =
x̂(t) − x(t). Here  and B̂ are the operator’s estimates of the model.
We assume that (Â + B̂L) is Hurwitz stable so that there exists positive
definite matrices P and Q satisfying
(Â + B̂L)T P + P (Â + B̂L) = −Q. (3.98)
If the deviations à , A −  and B̃ , B − B̂ are sufficiently small, the
operator can guarantee global asymptotic stability. In particular we
have the following result by Garcia and Antsaklis (2013).
Theorem 3.29. Assume k(Ã + B̃L)T P + P (Ã + B̃L)k2 ≤ δ < λm (Q)
and kB̃k2 ≤ β. Consider the following trigger with update instants ti
implicitly defined by the relation
σ(λm (Q) − δ)
ke(ti )k2 = kx(ti )k2 (3.99)
b
where
b = 2kP B̂Lk2 + 2βkP Lk2 , 0 < σ < 1.
Then system is globally asymptotically stable.
Quantization: We previously assumed our sensors have infinite pre-
cision, but in practice this is not the case. Measurements introduce
a quantization error that should be accounted for in event triggered
control. In this case the true estimation error and the true state are
not available and the trigger and its design must be adjusted accord-
ingly. We assume that a logarithmic quantizer is used. The quantizer
q : Rn → Rn is a function which we assume satisfies
kx − q(x)k2 ≤ γkxk2 , γ > 0. (3.100)
92 Networked Control Systems

At update instants ti , x̂(ti ) = q(x(ti )). We furthermore define the

quantized state estimation error as eq (t) = x̂(t) − q(x(t)). Since both
q(x) and x̂ are available to the operator, they can be used to define
a trigger. We define and characterize such a trigger from Garcia and
Antsaklis (2013) below.

Theorem 3.30. Consider the control system (3.96) with model-based es-
timator (3.97). Assume there exists a solution to the Lyapunov equation
(3.98). Moreover, assume k(Ã + B̃L)T P + P (Ã + B̃L)k2 ≤ δ < λm (Q)
and kB̃k2 ≤ β. Consider the following trigger with update instants ti
implicitly defined by the relation
ση
keq (ti )k2 ≥ kq(x(ti ))k2 (3.101)
γ+1
where η = (λm (Q) − δ)/b, 0 < σ < σ 0 < 1. Then,

ke(t)k2 ≤ σ 0 ηkx(t)k2 . (3.102)

Moreover, the system is asymptotically stable when δ ≤ (σ 0 − σ)η.

We note that there are examinations and results which account for
delays, quantization, and model uncertainty. For more information, see
Garcia and Antsaklis (2013).
Output Based and Decentralized Control: Typically, an operator
does not have access to the entire state during measurements. Attempts
to use similar triggers in this scenario can result in undesired Zeno
behavior. Additionally, we may need to consider multiple sensing nodes.
In such cases, evaluating a trigger may need to be a decentralized
process. Here, each sensing node must make its own decision in regards
to whether a measurement is sent. In this scenario, we model our control
system as follows

ẋp (t) = Ap xp (t) + Bp û(t) + Bw w(t), y(t) = Cp xp (t) (3.103)

ẋc (t) = Ac xc (t) + Bc ŷ(t), u(t) = Cc xc (t). (3.104)

Here, xp (t) ∈ Rn is the state of the physical system, û(t) ∈ Rp is the

applied control input, w(t) ∈ Rq is a disturbance, and y(t) ∈ Rm is the
sensor output. Additionally, xc (t) ∈ Rnc is the state of the controller,
ŷ(t) ∈ Rm is the input to the controller, and u(t) ∈ Rm is the desired
3.4. Event-based control 93

control input. We assume the controller is designed so that the system

is asymptotically stable when u(t) = û(t) and y(t) = ŷ(t).
In practice, however, û(t) and ŷ(t) are updated at discrete event
instants. Each node in a system with multiple sensing and control units
must independently evaluate it’s own event trigger. Here, we assume
there exists N nodes and the outputs of node j ∈ {1, · · · , N } is sent at
discrete instant tjij , where ij ∈ N. Thus at node j 0 s transmission instant,
it will send its respective entries in y and u. All nodes will then update
the corresponding entries in û and ŷ.
We let γjk , j ∈ {1, · · · , N } and k ∈ {1, · · · , m} be equal to 1 if
the kth entry of y is in node j and 0 otherwise. Similarly, we let γjk ,
j ∈ {1, · · · , N } and k ∈ {m+1, · · · , m+p} be equal to 1 if the (k−m)th
h iT
entry of u is in node j and 0 otherwise. Letting v(t) = y T (t) uT (t)
h iT
and v̂(t) = ŷ T (t) ûT (t) , we can describe event update instants as

v̂ + (tjij ) = Γi v + (tjij ) + (I − Γi )v + (tjij ). (3.105)

In between events for each node, a zero order hold is utilized
˙ j
v̂(t) = 0, for all t ∈ (0, ∞) − ∪N
j=1 {tij |ij ∈ N} (3.106)
We assume initially that v̂(0) = v(0), which can be done by transmitting
all inputs and measurements at time 0. In the event that multiple
transmissions are triggered at the same time, updates can be modeled
to take place simultaneously, or directly after each other in a negligible
amount of time. We now aim to construct a trigger for each node to
define the event times. An update will be triggered when the error
between the previous transmitted input/measurement and the desired
input/actual measurement in a node becomes too large. Particularly,
we have
tjij +1 = inf{t > tjij |keIj (t)k22 = σj kvIj (t)k22 + j } (3.107)

Since v̂(0) = v(0), we know tj0 = 0. Here the error is e(t) = v̂(t) − v(t).
We construct eIj (t) and vIj (t) by taking elements of e(t) and v(t) that
belong to Ij = {i ∈ {1, · · · , m+p}|γji = 1}. The structure of this trigger
ensures that for all nodes j ∈ {1, · · · , N } and all time t ≥ 0,
keIj (t)k22 = σj kvIj (t)k22 + j . (3.108)
94 Networked Control Systems

The remaining goal is to consider how to design parameters σj and j to

maximize the overall performance of the NCS. In particular, we desire
stability and adequate attenuation of the disturbance w(t). Moreover,
we want to minimize the frequency of events to reduce communication
and salvage the power supply of our sensing and actuation nodes. A
necessary condition here is to define a trigger that does not introduce
Zeno behavior.
To obtain results characterizing the stability of the system and
frequency of communication this NCS can be expressed as an impulsive
system
" # " #
˙ A + BC B E
x̄(t) = x̄(t) + w(t), x̄ ∈ C (3.109)
−C(A + BC) −CB −CE
" #
+ I 0
x̄ (t) = x̄(t), x̄ ∈ Dj , j ∈ {1, · · · , N }, (3.110)
0 Γj
C = {x̄ ∈ Rn+np +m+p |x̄T Qj x̄ ≤ j , ∀j ∈ {1, · · · , N }}, (3.111)
n+np +m+p T
Dj = {x̄ ∈ R |x̄ Qj x̄ = j }. (3.112)
h iT
where x̄ = xTp xTc eT and
" # " # " #
Ap 0 0 Bp Bw
A= , B= ,E = (3.113)
0 Ac 0 Bc 0
" # " #
Cp 0 −σj C T Γj C 0
C= , Qj = . (3.114)
0 Cc 0 Γj
The impulsive formulation given here completely and accurately
summarized the dynamics of the event triggered control system, in
particular by directly accounting for the dynamics of the error. By
presenting the model in this fashion, less conservative conditions for
stability and L∞ performance are obtained. For conciseness, we omit
detailed technical results stemming from this formulation. Instead we
briefly summarize several contributions that are found in (Donkers and
Heemels, 2012) and discussed in Heemels et al. (2012). To begin, given
a system where each node has inter-event times strictly larger than
0, sufficient conditions (in the form of linear matrix inequalities) are
provided which guarantee the global asymptotic stability of a set A and
3.4. Event-based control 95

an upper bound on the L∞ gain from the disturbance w to an output

that is linear in x̄ and w̄. The notion of global asymptotic stability for
a set is defined below.
Definition 3.5. A set A is stable with w = 0 if for each  > 0, there is a
δ > 0 such that minx∗ ∈A kx̄(0)−x∗ k2 ≤ δ =⇒ minx∗ ∈A kx̄(t)−x∗ k2 ≤ 
for all solutions x̄ to the impulsive system formulation with w = 0, and
all t where the solution is defined. Also, the set A is globally attractive
if each solution x̄ to the impulsive system formulation with w̄ = 0 has
limt→∞ minx∗ ∈A kx̄(t) − x∗ k = 0. The set A is globally asymptotically
stable with w = 0, if it is both globally attractive and stable.
By definition, globally asymptotic stable sets A contain 0 in their
interior, regardless of how the stabilizing trigger parameters are chosen.
It can be shown that j in the definition above is greater than zero for
each node j and if the LMIs are satisfied, the existence of a positive
lower bound on the inter-event times is guaranteed. However, when
j = 0 for at least one j, inter-event times can converge to zero and
Zeno behavior can occur. This motivates the design of strictly positive
j . The size of A is determined by parameters j and σj , the upper
bound on the L∞ gain, κ, and additional parameters defined in the
LMIs. However, satisfying the LMIs is independent of the parameter j .
Thus, parameters can be chosen independently of j to guarantee
global asymptotic stability of some set A, and some desired upper bound
on the L∞ gain, κ. From here, the j parameters for each node will
control the size of the set A. Smaller values of j will reduce the size A,
a desirable outcome for a designer who wishes to stabilize the system
to 0. However, decreasing j also increases the frequency of communi-
cation. In particular, when j → 0 (from above) for all j ∈ {1, · · · , N },
the stabilizing set also approaches the singleton {0}. However, again
undesired Zeno behavior, with infinitely many events in a finite interval
can occur. In the case when j > 0 for each node, a lower bound on
the inter-event times can be computed as a function of the magnitude
of the initial state and bounds on the disturbances. For more details
see Donkers and Heemels (2012). Output based event triggers may also
make use of observers. For more details see Heemels et al. (2012)
96 Networked Control Systems

Self Triggering: As proposed, an event trigger requires constant (or

in practice near constant) monitoring of the output for correct imple-
mentation. In many cases hardware restrictions prevent such rigorous
sensor monitoring. In this case, an alternative to event triggering is
self triggering, where in addition to the current control input, the next
control instant is also computed. In particular, consider the control
system with disturbances w(t),

ẋ(t) = Ax(t) + Bu(t) + Bw w(t). (3.115)

A state feedback controller with a zero order hold in between control

instants is implemented (3.81). The self trigger can be defined as a
function Γ : Rn → (0, ∞) which computes the duration for which the
control input remains constant and the sensors remain idle. That is
ti+1 − ti = Γ(x(ti )).
Suppose (A + BL) is Hurwitz, so that we can construct a Lyapunov
1
function V (x) = (xT P x) 2 which satisfies

V (x(t)) = V (x(0)) exp(−λ0 t), ∀t ≥ 0, x(0) ∈ Rn , (3.116)

for the ideal system ẋ = (A + BL)x. In the self triggered case, with
w = 0, we may instead desire the relaxed performance objective

V (x(t)) ≤ V (x(ti )) exp(−λ(t − ti )), (3.117)

where 0 < λ < λ0 is some desired slower rate of decay. Note that
implicitly defining a self trigger based on (3.117) is difficult as it re-
quires a continuous check across t. Instead, we will consider checks at
discrete instants of time, separated by a period ∆s . In the absence of a
disturbance, and for ti + n∆s ≤ ti+1 , we define
n−1
n
Ajd Bd L,
X
x(ti + n∆s ) = R(n)x(ti ), R(n) = Ad + (3.118)
j=0
Z ∆s
Ad = eA∆s , Bd = eA(∆s −τ ) Bdτ. (3.119)
0

Let tmin be the minimum time between updates, which will be implicitly
defined by our choice of λ, and let tmax be the maximum time, which
3.4. Event-based control 97

is selected as a design parameter. We define a self trigger as follows

Γ(x(ti )) = max(tmin , n(x(ti ))∆s ), (3.120)

n(x(ti )) = max{n ≤ btmax /∆s c|h(x(ti ), s) ≤ 0, s = 0, · · · , n},
n∈N
1
h(x(ti ), s) = kP 2 R(s)x(ti )k2 − V (x(ti )) exp(−λs∆s ).

The resulting trigger ensures an inter-event time that is larger than

tmin (Mazo and Tabuada, 2009). Moreover, using the results by Mazo
and Tabuada (2009) we can guarantee stability as follows.

Theorem 3.31. Consider the system (3.115) with controller (3.81) and
self trigger (3.120). The resulting system is exponentially input to state
stable. That is there exists constants λ > 0, κ > 0, and γ > 0 such that
for any essentially bounded function w and x(0) ∈ Rn , we have

kx(t)k2 ≤ κkx(0)k2 exp(−λt) + γkwk∞ . (3.121)

Periodic Event Triggered Control: Self Triggered Control can help

allow an operator to design a NCS that is efficient in its use of commu-
nication and monitoring resources by intelligently determining ahead
of time when to monitor and send measurements to maximize perfor-
mance. In fact, this is the only event triggered control model we have
seen thus far that does not assume constant monitoring. In reality,
constant monitoring of an output is impractical. As such we consider
the alternative of periodic event triggered control systems. Here, we
assume that the event triggering condition is evaluated periodically,
while control updates occur irregularly according to the event trigger.
The benefits compared to a pure self triggered approach here is that
measurements are triggered by events and disturbances in the system,
enabling a measure of feedback.
We first consider the system model with disturbance

ẋ(t) = Ax(t) + Bu(t) + Bw w(t). (3.122)

Conventional state feedback, in a fully observed, sampled data system,

would entail
u(t) = Lx(ti ), for t ∈ (ti , ti+1 ] (3.123)
98 Networked Control Systems

where the sampling times ti , i ∈ N are multiples of some constant

sampling interval h > 0, that is ti = ih. Instead, we now assume that
while a system is monitored at discrete event instants, the control input
is updated only when event triggering conditions are met. For instance,
given a quadratic event trigger, then
u(t) = Lx̂(t), (3.124)
where x̂(t) for t ∈ (ti , ti+1 ] is given by

x(t ),
i when ζ T (tk )Qζ(tk ) > 0
x̂(t) = , (3.125)
x̂(ti ), when ζ T (tk )Qζ(tk ) ≤ 0
h iT
and ζ = xT x̂T . Quadratic event triggers can be used to express a
variety of triggering conditions, including triggers based on state errors,
input errors, and Lyapunov functions. By construction we observe that
the inter-event times are always bounded below by the sampling time
h. In fact, one can pursue designs which make the inter-event times
strictly larger than the sampling time (see (Heemels et al., 2013) and
(Eqtami et al., 2010)).
The event triggered formulation can also be presented as an impulsive
formulation with
" # " #
ζ̇ Āζ + B̄w
= , τ ∈ [0, h], (3.126)
τ̇ 1
 
 J ζ
 1 
when ζ T (tk )Qζ(tk ) > 0

" #   ,
ζ +  0

=   ,
τ+ J2 ζ 

ζ T (tk )Qζ(tk )


 , when ≤0
 0


z(t) = C̄ζ(t) + D̄w(t)

where
" # " # " # " #
A BL Bw I 0 I 0
Ā = , B̄ = , J1 = , J2 = . (3.127)
0 0 0 I 0 0 I
Here z is some user defined performance indicator, while τ tracks the
amount of time since the last sampling instant. The overall performance
3.4. Event-based control 99

of the system can be analyzed again in terms of stability and disturbance

attenuation. Here, we introduce the notion of global exponential stability
and L2 disturbance gain.

Definition 3.6. A system (3.126) is globally exponentially stable if

there exists c > 0 and ρ > 0 such that for any initial condition ζ(0) =
ζ0 ∈ R2n , all solutions to (3.126) with τ (0) ∈ [0, h] and w = 0 have
kζ(t)k2 ≤ ce−ρt kζ0 k2 for all t ≥ 0. ρ is a lower bound on the decay rate.

Definition 3.7. A system (3.126) has an L2 gain from w to z that is

less than or equal to γ if there is a function δ : R2n → [0, ∞) such that
for any w ∈ L2 (that is w locally integrable with finite L2 norm given
by kwkL2 = ( 0∞ kw(t)k22 dt)1/2 ), any initial state ζ(0) = ζ0 ∈ R2n and
R

τ (0) ∈ [0, h], the solution to the system satisfies

kzkL2 ≤ δ(ζ0 ) + γkwkL2 (3.128)

Through Lyapunov analysis we can analyze system conditions that

guarantee performance. To obtain performance results, we introduce
matrix " #
F11 (τ ) F12 (τ )
F (τ ) = e−Hτ = (3.129)
F21 (τ ) F22 (τ )
where
" #
Ā + ρI + γ −2 B̄M D̄T C̄ B̄M B̄ T
H= (3.130)
−C̄ T K C̄ −(Ā + ρI + γ −2 B̄M D̄T C̄)T
K = (γ 2 I − D̄D̄T )−1 , , M = (I − γ −2 D̄T D̄)−1 (3.131)

We also let F̄11 = F11 (h), F̄12 = F12 (h), F̄21 = F21 (h), F̄22 = F22 (h).
For small enough sampling period h, it can be shown that F11 (τ ) is
invertible for all τ ∈ [0, h] and there exists a matrix S̄ that satisfies
−1
S̄ S̄ T = −F̄11 F̄12 . Given this we have the following result from Heemels
et al. (2013).

Theorem 3.32. Consider q the periodic event triggered system (3.126)

and let ρ > 0 and γ > λmax (DT D). Assume F11 (τ ) is invertible
for all τ ∈ [0, h]. Suppose there exists a matrix Ph > 0 and scalars
100 Networked Control Systems

ζj ≥ 0, j ∈ {1, 2}, such that for j ∈ {1, 2}, we have

−1 T −1 T −1 −1
Ph + (−1)j ζj Q JjT (F̄11 ) Ph S̄ JjT ((F̄11
 
) Ph F̄11 + F̄21 F̄11 )
∗ T
I − S̄ Ph S̄ 0 >0
 

−1 T −1 −1
∗ ∗ (F̄11 ) Ph F̄11 + F̄21 F̄11
(3.132)
Then, the periodic event triggered system (3.126) is globally exponen-
tially stable with convergence rate ρ when w = 0. Moreover, the system
has a L2 gain from w to z which is less than or equal to γ.

Additional extensions to periodic and discrete event triggered control

can be examined. For instance, Heemels et al. (2013) examine decen-
tralized and output based event triggers in this setting. Also, Eqtami
et al. (2010) consider self triggering in discrete time event systems.

3.4.1 Further Reading

Our treatment of event-based control were based on the results developed
by Heemels et al. (2013), Eqtami et al. (2010), Mazo and Tabuada
(2009), Donkers and Heemels (2012), Heemels et al. (2012), Garcia and
Antsaklis (2013) and Tabuada (2007). While our goal was to achieve a
general treatment of event-based control, the references covered here are
by no means a complete treatment. Below, we will attempt to provide
pointers to additional references that covers aspects in event-based
control.
We start with Anta and Tabuada (2010). This work considers self-
triggered event-based control for non-linear systems. In particular, the
work focuses on two classes of systems, namely, state-dependent ho-
mogeneous systems and polynomial systems, and develops self-trigger
conditions. These conditions are in the spirit of Tabuada (2007) and
are based on the properties of trajectories of homogeneous control sys-
tems. The results are then generalized by introducing the notion of
state-dependent homogeneity, which establishes a general self-triggering
condition based on the full state of the plant. As a next step, these
conditions are used to develop self-triggering conditions for polynomial
control systems. The self-trigger for both classes of systems reflect the
inherent tradeoff between the amount of resources allotted to a controller
3.4. Event-based control 101

and the achieved performance. Finally, application of the developed

conditions are shown on two examples from the literature: The control
of a jet engine compressor and the control of a rigid body.
Next, Borgers and Heemels (2014) present a general stability analysis
for a class of large-scale networked control systems with multiple com-
munication networks, allowing for the existence of both time-triggered
and event-triggered networks. The class of NCSs considered in this work
is modeled as an interconnection of ordinary nonlinear systems and
hybrid systems. By using a general small-gain theorem, it is shown that
Pareto optimality is achieved through updating scale functions. This
leads to a tradeoff between the scaling functions. In particular, it is
possible to increase the performance of one network while decreasing
the required performance of other networks.
Guinaldo et al. (2011) present a distributed event-based control
scheme for a NCS consisting of multiple linear time-invariant inter-
connected subsystems. Each subsystem broadcasts its state over the
network according to a trigger which depend on local information only.
It is shown that the system converges to an adjustable region around
the equilibrium point, and that the existence of a lower bound for the
broadcasting period is guaranteed. The region of convergence and this
lower bound are analyzed while a novel model-based approach is derived
to reduce communication. Later, Guinaldo et al. (2012) extend the work
in Guinaldo et al. (2011). The authors focus on the effect of network
delays and packet loss for each system. Specifically, bounds on the delay
and the number of successive packet dropouts which can still ensure
stability are derived while guaranteeing a certain level of performance.
Two different transmission protocols are proposed and a lower bound for
the inter-event times is derived. The first protocol is designed to preserve
state consistency. Here, a broadcast state is updated synchronously in
each neighboring agent. The second protocol is designed to remove
this restriction by allowing neighbors to use different versions of the
broadcast states.
Seyboth et al. (2013) consider a control strategy for multi-agent
coordination with event-based broadcasting. Three cases are considered:
networks of single-integrator agents with and without communication
delays, and networks of double-integrator agents. In these scenarios,
102 Networked Control Systems

each agent independently decides when to transmit its current state to

its neighbors and the local control laws depend on the sampled state
measurements. Time-dependent bounds for each agent’s measurement
error are derived for the proposed event-based scheduling strategy. Ad-
ditionally, the inter-event intervals are lower-bounded by a positive
constant. For each scenario it is shown that the proposed control strat-
egy guarantees either asymptotic convergence to average consensus or
convergence to a ball centered at the average consensus.
Finally, Dimarogonas et al. (2012) consider distributed event-driven
strategies for multi-agent systems. Assuming a fixed ratio of a certain
measurement error with respect to the norm of a function of the state,
input updates are generated. Initially, a centralized formulation of
the problem is considered and then, the results are extended to a
decentralized setting where agents use only the states of their neighbors
for determining the control input update. The results are then extended
to a self-triggered setup. In this case, each agent computes its next
update time at the current update instant, without having to keep track
of the state error that triggers the actuation between two consecutive
update instants.
 4
Secure Cyber-Physical Systems

The threat of attacks create new challenges for achieving reliable per-
formance in CPS. We note that CPS have become targets for malicious
adversaries. In this section, we will see that achieving security in CPS
requires tools which extend beyond what is offered in state of the art
software and cyber security. In particular, we will consider a science
of cyber-physical system security which combines tools from both cy-
ber security and system theory to defend against adversarial behavior.
We discuss realistic and intelligent attack models from an adversarial
perspective and then offer mechanisms for defenders to recognize and
respond to such malicious behavior in order to achieve resilience.
More specifically, the rest of the section is summarized as follows.
First, in subsection 4.1, we discuss the motivation for investigating
cyber-physical system security. Next, in subsection 4.2, we consider
possible attacks on cyber-physical systems. Here, we classify adversaries
by their knowledge, disclosure resources, and disruption resource before
placing a significant focus on undetectable integrity attacks. Then, in
subsection 4.3, we begin pursuing defensive strategies. In particular,
we discuss how the robust structural design of control systems can
prevent classes of stealthy attacks. Later, in subsection 4.4, we review

103
104 Secure Cyber-Physical Systems

how online random and secret physical perturbation can be used to

counter intelligent and resourceful adversaries. Finally, in subsection 4.5,
we take a step towards achieving resilience by providing strategies for
obtaining reliable state estimates in the presence of corrupted sensors.

4.1 Motivation

Pursuing cyber-physical system security requires us to evaluate threats

against system architectures. In this subsection, we first examine the
problem from the attacker’s perspective. We enumerate possible attack
surfaces and list opportunities to cause significant damage in CPS. We
then consider the problem from the defender’s perspective. We briefly
review existing mechanisms in cyber security before motivating the need
for a science of CPS security.

4.1.1 Attack Surfaces

We now consider attack surfaces available to adversaries in a CPS.
Increased automation and improved sensing have allowed system de-
signers to remotely monitor and control critical infrastructures. The
ability to perform sensing and control over a communication network
creates the opportunity for an attacker to cause damage via network
intrusions. Adversaries can find weaknesses in protocols including DNP3
and the older Modbus protocol or leverage poorly designed firewalls to
penetrate the network. Attackers can also target trusted peer utility
links. For instance, adversaries can attempt to hijack VPN connections.
Alternatively, adversaries can steal valid credentials allowing them un-
encumbered remote access to perform the same actions as a trusted
user as in the Ukraine power attack (Pultarova, 2016).
We remark that CPS operation relies on the use of small heteroge-
neous components and devices that are potentially prone to failure or
attack. For example, in the Stuxnet attack, introducing infected USB
devices into CPS allowed this malware to spread. Meanwhile in the
case of the Ukraine power attack, infected email attachments allowed
attackers access to system workstations. Attackers can also target field
devices such as sensors and actuators as well as networking devices
4.1. Motivation 105

which can interface with both field devices and the monitoring layer.
For instance, in SCADA systems, remote terminal units often allow for
dial up access and may not even require authentication. An attacker
can also take the initiative to introduce vulnerabilities to CPS devices
by targeting supply chains. If production is not performed securely,
adversaries can install backdoors in components, which can later be
leveraged to compromise the CPS.
Beyond attempting to access CPS through a network, an attacker
can simply attempt to target the physical plant itself. In many cases,
due to the scale of CPS it is impossible to physically monitor and protect
all devices and components. As an example, it is often the case that
substations as well as smart meters and PMUs are left unattended in
the electricity grid. Likewise, it is impractical to guard all the sensors,
pumps, and valves in a water distribution system or traffic lights and
vehicles in a transportation system. The defender must also account for
the actions of malicious insiders. Malicious insiders can leverage their
understanding of a CPS and their access to the system in order to target
the infrastructure. A notable example in this regard is the Maroochy
Shire incident (Slay and Miller, 2007) where a former employee was
able to hack a SCADA system performing waste management, causing
millions of gallons of sewage to leak. A malicious insider’s actions can
be amplified if intelligent access control policies are not implemented.

4.1.2 Attack Strategies

Unlike traditional software and computer systems, where attacks occur
only in the cyber realm, in cyber-physical systems, the defender must
counter attacks that occur and affect both the cyber and physical
portions of an infrastructure. An attacker can infiltrate a system’s
communication network, and use such access to manipulate transmitted
information including sensor measurements delivered to an operator and
control commands directed to the plant. Additionally, software attacks
can be used to target the supervisory layer of a SCADA system while
physical attacks on the infrastructure can compromise the plant. Given
these capabilities, we discuss several possible attack strategies on CPS.
106 Secure Cyber-Physical Systems

Secrecy Violations: The release of sensitive information in CPS

can have significant privacy repercussions (as will be further discussed
in the next section). In the smart grid, consumers do not want their
electricity consumption released, travelers in transportation systems do
not want their locations disclosed, and patients receiving health care do
not want their medical histories revealed. Additionally, eavesdropping
attacks on CPS can reveal information that an attacker can use to
construct stealthy attackers. In practice, attackers can utilize sensor
data and control commands to learn more about a CPS and its dynamics.
An accurate model of CPS provides an adversary with the chance to
synthesize attacks, which are stealthy and can negatively impact the
physical dynamics of CPS. Nonetheless, the desire to keep information
secret should be balanced with Kerckhoff’s principle, which discourages
ideas of security through obscurity.
Integrity Violations: In an integrity attack, the adversary typically
modifies control commands or sensor measurements transmitted in a
CPS. Misleading sensory information may cause operators to make
incorrect control decisions, which in turn leads to physical damage at
the plant. If done intelligently, harmful sensor attacks can be performed
in a stealthy manner, allowing an adversary to perturb a system for
long periods of time without defender interference. An example of this
is a replay attack, which was utilized in Stuxnet. Modified control
commands can also cause significant damage, for instance allowing an
attacker to cause blackouts on the grid by tripping breakers.
Both sensor and control command attacks can be realized through
the cyber realm or the physical realm. For example, these integrity
attacks can be performed in the cyber space via a man in the middle
attack occurring over the network. Here, the adversary can intercept true
data packets and replace them with his own falsified data packets before
forwarding the resulting message to the operator or plant. Alternatively,
a physical sensor integrity attack can occur if an adversary changes
the environment around a sensor. For instance, a temperature sensor
can be compromised through local heating and cooling. We also note
that control commands can be directly modified by an attacker who
has physical access to actuators in CPS. Alternatively, an attacker can
perform a topology attack where the plant itself is modified. As an
4.1. Motivation 107

example, in the smart grid, an attacker can destroy transmission lines

to change the structure of the plant. Last but not the least, integrity
attacks on software can be perpetrated by an attacker. This can be done
by introducing malware to the system. As most CPS rely on automation,
changes in the software can have deleterious effects on a control system.
Software attacks at supervisory control layer of a SCADA system can
be especially harmful as such attacks may prevent an operator from
performing any corrective measures remotely.
Availability Violations: Jamming or denial of service attacks can
be used to restrict the flow of information in a CPS. In a jamming
attack, an adversary emits a signal which interferes with the messages
being sent between the plant and SCADA operator, preventing the
receiving party from obtaining the proper message. In a similar vain,
a denial of service attack restricts availability by flooding a system
with requests. This can delay or prevent legitimate requests from being
addressed. While the availability of real time data streams is often
not critical in typical software systems, in CPS availability of sensor
information and control commands may be intrinsically linked to the
safe and reliable operation of the underlying control system. In the
absence of sensor measurements, a SCADA system fails to monitor the
plant. This in turn can prevent an operator from determining proper
corrective actions. Likewise, the absence of control commands prevents
proper control actions from being delivered to the plant. This results
in sub-optimal or possibly unsafe operation. Previous work (Schenato
et al., 2007; Sinopoli et al., 2004) has shown that open loop unstable
systems have critical packet delivery rates that are necessary to ensure
that the resulting closed loop system can be stabilized. Unlike integrity
violations, attacks that affect availability can often be easily detected.
Nonetheless, corrective actions need to be in place to ensure that CPS
can resiliently respond to such attacks.

4.1.3 Traditional Countermeasures

There are many traditional tools in cyber security that can be employed
to improve resilience in CPS. An immediate defense strategy is to limit
the available attack surfaces for potential adversaries. From a technical
108 Secure Cyber-Physical Systems

standpoint, well configured firewalls can limit an attacker’s access to

a network. Alternatively, implementing intelligent policies can reduce
attack surfaces. For instance, management can limit the transfer of usb
storage devices in and out of the network or require careful oversight of
supply chains to prevent backdoors from being installed.
Properties of secrecy are often guaranteed using cryptographic prim-
itives such as encryption. More generally, the framework of information
flow analysis can promote confidentiality (Denning, 1976). Information
flow analysis involves a set of tools in software security designed to
prevent illicit flows of information in computing systems. Information
flow and in particular the property of noninterference can be used
to express very general security policies including multilevel security,
discretionary access control, confinement, and channel control, all of
which can be used to promote confidentiality (Goguen and Meseguer,
1982). Various tools have been implemented to ensure that systems
have no valid information flow (Volpano et al., 1996) and to quantify
information leakage when there exists information flows (Smith, 2009).
Additionally, cryptographic primitives and protocols can detect
integrity attacks. Authentication protocols, for instance, can be used
to verify the identity of different devices, components, and operators.
Moreover, authenticated encryption simultaneously guarantees secrecy
and integrity in attacks from remote adversaries. The root of trust in
such a system is a set of secret keys. In public key cryptography each
object will have a public key used for encryption and a private key
used for decryption while in symmetric key cryptography each pair
of communicating objects has a shared key. Due to the sheer size of
many CPS, developing a key management system, which allow for key
generation, sharing, replacement, and recovery is a significant challenge
(Wu and Zhou, 2011). Responding to integrity attacks on code requires
alternative tools, for instance code attestation (Shi and Perrig, 2004).
Finally, significant work has examined means to ensure the availability
of data. These include preventing denial of service attacks as well as
malicious jamming. For brevity, we refer the reader to a survey of results
found in Peng et al. (2007) and Mpitziopoulos et al. (2009).
4.1. Motivation 109

4.1.4 Cyber-Physical System Security

Due to the strong coupling between cyber and physical domains, the
tools and methodologies developed to ensure cyber security are insuffi-
cient to secure CPS. For instance these methods can fail against purely
physical attacks. As an example, secrecy of encrypted sensor measure-
ments can be violated by placing unencrypted sensors in close proximity
to encrypted sensors. The integrity of sensor measurements can be
modified by changing a sensor’s local environment while control inputs
can be changed by directly manipulating system actuators. In such
a scenario, message authentication codes or digital signatures fail to
recognize an attack. We also note that availability can be compromised
by physically shielding sensors and actuators. In this case, anti-jamming
and denial of service techniques will fail.
Next, in general, it is often infeasible to remove all access points for
potential attackers. The large scale of a CPS means physical protection
is often impractical. Additionally, device heterogeneity provides ample
entry points for an attacker to leverage while system connectivity allows
adversaries to maximize the opportunities they receive. Last but not
the least, human error, which could allow corrupted devices to enter a
system (as in Stuxnet) or allow malware to infect workstations (as in the
Ukraine attack) can not be completely eliminated. Without the ability
to prevent all attacks, it is essential to respond to attacks once they
occur. However, techniques from cyber security are often insufficient in
addressing attacks at the physical level. For instance, upon detecting
an attack, one common countermeasure in cyber security is to take a
system offline. However, the inertia of CPS and the need to provide
continued service can make such a decision impractical. Moreover,
achieving resilience CPS requires a defender to design countermeasures
that preserve stability and control performance. This objective can be
achieved through a system theoretical treatment of CPS.
Overall, to develop a theory of CPS security, it is necessary to aug-
ment existing techniques in cyber security with tools from system theory.
In particular, the defender’s understanding of a system’s dynamics can
be used to detect, isolate, and respond to attacks. Here, sensor mea-
surements can be used as outputs, which allow a defender to evaluate a
110 Secure Cyber-Physical Systems

system’s health. For example, if the sensor measurements closely follow

expected behavior as determined by a physical model, a detector can
accept the hypothesis that the system is operating normally. However,
if the measurements significantly deviate from the model, the detector
may determine that their exists faults or malicious behavior in the CPS.
Additionally, as discussed later in the current section, control inputs
and other degrees of freedom can be used as active monitors, which can
be modified intelligently to allow the defender to better differentiate
observed behavior under attack and expected behavior under normal
operation. Finally, upon detecting attacks and identifying points of
failure, resilient algorithms can guide operators in performing corrective
actions. For example, we will specifically discuss how resilient estimation
algorithms will allow a defender to stabilize a control system in the
presence of sensor attacks.
We remark that system theory used on its own can fail to achieve
resilient CPS. As an example, system theoretic tools rely upon stochastic
models. Thus, finite time detection schemes will almost always be
probabilistic. This is often in stark contrast to methods that rely on
cryptographic primitives, where an attack without access to a key fails
with a probability near 1. Another potential challenge is that system
theory often introduces high level abstractions to describe devices and
software in CPS. Cyber and software security can help ease these
high level abstractions. In general, a layered approach to CPS security
combining system theory and cyber security is expected to be more
effective in responding to attacks.

4.1.5 Resilience vs. Robustness

We now address the need to distinguish the concepts of resilience and
robustness. A robust system is designed offline to remain functional even
in the presence of malicious or faulty behavior. The online operation of
a robust system does not change in the presence of an attack. Unlike a
robust system, a resilient system includes the notion of a response or
change in operation as a result of an attack. This requires a defender
to detect and identify attacks on a CPS as they occur. Upon detecting
malicious or faulty behavior and identifying nodes under an attack, a
4.2. Attacks on CPS 111

defender can change their control strategy in order to guarantee a level

of stability and performance.
In many instances, a resilient approach for CPS security is superior
to a robust approach. In particular, an argument can be made that a
resilient approach can allow a system to achieve better performance
and efficiency. For instance, it can be observed that optimal controllers
and estimators outperform robust estimators and controllers under
normal operation. Moreover, upon detection of an attack, conservative
countermeasures could yield comparable performance to that of robust
algorithms. Finally, upon attack identification, specially designed control
solutions can outperform more general robust control and estimation
techniques. Nonetheless, the ability to implement robust solutions offline
provides an avenue for robust techniques to surpass resilient counter-
measures. Notably, the need to implement resilient countermeasures
in a timely manner can preclude effective solutions. For instance, it is
unlikely a defender would have time to place additional sensors in a
CPS once an attack commences.
In the current section, we will largely focus on techniques for re-
siliency. In particular, we will place a large focus on the detection of
attacks, the first step in providing an attack specific response. Upon
recognizing attacks, we recommend that specific responses or corrective
actions be automatic and determined offline. This requires that the
defender determine a set of contingencies that are prioritized in terms
of risk. Performing such a risk analysis is often application specific and
as such is out of scope in this work. Nonetheless, we will conclude this
section by examining the problem of estimation and its role in achieving
resilient response in CPS.

4.2 Attacks on CPS

In order to achieve resilience, a defender needs to understand realistic

attack scenarios in a CPS. In this subsection, we describe several attack
scenarios in CPS with a special focus on integrity attacks.
112 Secure Cyber-Physical Systems

4.2.1 System Model

We consider the linear discrete stochastic CPS model used earlier
xk+1 = Axk + Buk + wk , yk = Cxk + vk , (4.1)
where xk ∈ Rn is the state vector, uk ∈ Rp is the control input, and
yk ∈ Rm is the sensor output. We assume the process noise wk ∈ Rn
and sensor noise vk ∈ Rm are IID and independent of each other with
wk ∼ N (0, Q) and vk ∼ N (0, R), We assume that R > 0, (A, C) is
1
detectable, and (A, Q 2 ) is stabilizable. Moreover, x0 is independent of
the noise processes and has distribution N (x̄0|−1 , Σ).

Estimation
We obtain a minimum mean squared error estimate by using a Kalman
filter as follows:
x̂k+1|k = Ax̂k|k + Buk , x̂k|k = x̂k|k−1 + Kk zk , (4.2)
T T −1
zk = yk − C x̂k|k−1 , Kk = Pk|k−1 C (CPk|k−1 C + R) , (4.3)
T
Pk|k = Pk|k−1 − Kk CPk|k−1 , Pk+1|k = APk|k A + Q, (4.4)
where we define
x̂k|k , E[xk |y0:k ], x̂k|k−1 , E[xk |y0:k−1 ], (4.5)
Pk|k , E[ek eTk |y0:k ], ek , xk − x̂k|k , (4.6)
Pk|k−1 , E[ek|k−1 eTk|k−1 |y0:k−1 ], ek|k−1 , xk − x̂k|k−1 . (4.7)
We observe that Pk|k−1 and Kk converge to unique matrices, which we
define as P and K respectively. Assuming that the system has been
running for a long time, we assume Pk|k−1 = P and Kk = K for all k.
Thus, here Σ = P .
For resilient operation of CPS, it is imperative for the operator
to recognize attacks as they occur. One possible method to achieved
resilience in attack detection is through control algorithm design. Here,
the defender uses the information Ik that includes model information
M = {A, B, C, Q, R, P, K}, the set of prior outputs, the set of prior
inputs, and an initial state estimate x̂−1|0 = x̄0|−1 . That is, Ik =
{M, y0:k , u0:k−1 , x̂−1|0 }.
4.2. Attacks on CPS 113

Control
A defender’s goal in CPS is to design a resilient controller C, which
under normal operation, allows the defender to successfully meet control
objectives. Under attack, the defender aims to achieve graceful degra-
dation of performance. Mathematically, a controller C is a sequence of
functions {Uk } which take a defender’s information Ik and obtains a
control input as uk = Uk (Ik ).

Example 4.1. In this example, we consider a linear quadratic Gaussian

or LQG controller. Here, we assume (A, B) is stabilizable, and the
defender wishes to minimize the following cost function
N −1
" #
1 X
J = lim E xTk W xk + uTk U uk , (4.8)
N →∞ N
k=0

where we assume W > 0, U > 0. It can be shown that the optimal

controller converges to a state feedback controller uk = Lx̂k|k where

L = −(B T SB + U )−1 B T SA, (4.9)

S = AT SA + W − AT SB(B T SB + U )−1 B T SA. (4.10)

When considering the LQG controller, we will typical assume that this
convergence has occurred.

4.2.2 Statistics for Attack Detection

Before describing our attack model in detail, we briefly review some sim-
ple concepts related to attack detection. Here, we begin by considering
the notion of a passive detector, defined below.

Definition 4.1. A passive detector Ψk implemented at time k is an

algorithm which processes the defender’s information Ik to make a
decision about the system.

In CPS security, we consider a binary decision, whether the system is

operating normally, or under attack. In particular, we associate the null
hypothesis H0 with the system operating normally, and the alternative
hypothesis H1 with the system is under attack. A passive detector
outputs either the null hypothesis H0 , or the alternative hypothesis H1 .
114 Secure Cyber-Physical Systems

Later in this section we will distinguish the process of passive detection

from active detection.
In a deterministic system without sensor and process noise, where the
defender has perfect model knowledge, attack detection itself becomes
very simple. For instance, if the defender knows the initial state, then
the entire output trajectory is deterministic from the defender’s point of
view. As such, any deviation from this deterministic trajectory should
be marked as an attack. If the defender receives the expected output
trajectory, there will be no reason to raise an alarm. In this case, we
note a defender would be unable to distinguish normal operation from
an attack which can generate the expected trajectory.
We will also consider the scenario where the defender has no knowl-
edge of the initial state. Assume without loss of generality that the
defender’s control inputs are identically 0. In this case, the set of valid
output trajectories y0:k−1 will be a lower dimensional subspace of Rmk
if mk > n. If the received output sequence does not lie in the proper
subspace, the system can be identified as under attack. If the defender
receives one of the feasible output trajectories, there will be no reason to
raise an alarm (unless partial knowledge of the initial state is available).
In this case, we note a defender would be unable to distinguish normal
operation from an attack which can generate a feasible trajectory.
In stochastic systems, statistical tests can be used to determine if a
system is under attack. Detection will be probabilistic. We define the
probability of detection βk and the probability of false alarm αk at time
k as follows

βk , Pr(Ψk (Ik ) = H1 |H1 ), αk , Pr(Ψk (Ik ) = H1 |H0 ). (4.11)

Designing an optimal detector is feasible if the adversary’s attack

strategy is known. In particular, let f (y0:k |H0 ) denote the distribution
of the outputs y0:k under normal operation and let f (y0:k |H1 ) denote
the distribution of outputs under attack. Then, the Neyman Pearson
lemma stated below allows us to construct an optimal detector.

Theorem 4.1. Consider the likelihood ratio test.

H1 f (y0:k |H1 )
Λ(y0:k ) ≷ τ, Λ(y0:k ) = . (4.12)
H0 f (y0:k |H0 )
4.2. Attacks on CPS 115

The likelihood ratio test is the most powerful test at size ᾱk where
ᾱk = Pr(Λ(y0:k ) > τ |H0 ). In particular, it is the statistical test which
maximizes the probability of detection βk for a given probability of false
alarm αk = ᾱk .

We remark that the threshold τ determines the probability of false

alarm. Though theoretically optimal, the likelihood ratio detector suffers
from several practical drawbacks. First, in practice a defender may not
know an adversary’s precise strategy, and as such would be unable
to determine f (y0:k |H1 ). If the defender has reason to believe that
the attacker will follow only a small number of strategies, a family of
Neyman Pearson detectors can be used. However, such a solution can
scale poorly if one wishes to detect many different contingencies.
Another potential challenge is that a defender does not know when
an attacker commences his or her attack. The time an attack begins
can significantly change the distribution of the sequence of sensor out-
puts. One consequence is that we recommend a defender use some
fixed bounded window of measurements to perform attack detection.
Alternatively, if the defender utilizes an entire measurement history, he
or she must be able to properly weigh the small fraction of recently col-
lected measurements during an attack and longer measurement histories
collected during normal operation.
To address some of these challenges, we next discuss the χ2 detector.
Unlike the likelihood ratio test, which requires the defender to identify
specific attack strategies, the χ2 detector attempts to recognize any
deviations between measured and expected behavior. Specifically, the
χ2 detector computes a quadratic function of the residue sequence:
k H1
zjT (CP C T + R)−1 zj ≷ τ.
X
(4.13)
j=k−WD +1 H0

The residue zk , yk − C x̂k|k−1 is the difference between observed be-

havior yk and expected behavior. A small difference indicates good
agreement between the model and the observations, while a large dif-
ference indicates a mismatch between observed and expected behavior.
We remark that under normal operation the left hand side of (4.13)
has a χ2 distribution with mWD degrees of freedom. The χ2 detector
116 Secure Cyber-Physical Systems

has two design parameters, the threshold τ and the window size WD .
The threshold captures tradeoffs between the probability of detection
and the probability of false alarm. Specifically, increasing the threshold
simultaneously reduces the probability of false alarm and detection.
The window size captures tradeoffs between time to detection and av-
erage detection performance. A larger window likely improves average
detection performance, as there is more evidence of an attacker’s ma-
nipulation. However, a large window also increases time to detection
since it will take a longer time to ignore older "normal" measurements
when computing a detection statistic.
Of course the preceding subsection provides only a brief overview of
attack detection in CPS. For a more detailed survey, see Giraldo et al.
(2018).

4.2.3 Attacker Capabilities

We now aim to describe a realistic attacker. Prior to developing possi-
ble attack strategies, it is important to determine the knowledge and
resources the attacker has at his disposal to carry out an attack. As
suggested by Teixeira et al. (2012a) and Teixeira et al. (2015), we
can describe an attacker and his capabilities in terms of three main
characteristics, his system knowledge, his disclosure resources, and his
disruption resources.

System Knowledge
An attacker’s system knowledge refers to his/her a priori understanding
of the dynamics of the system. Increased familiarity with a CPS can be
leveraged by an attacker to construct more sophisticated and effective
attacks. In the context of the model presented in subsection 4.2.1, an
attacker’s system knowledge may consist of some imperfect estimate of
the system model M̂, the controller C,ˆ and the detector D̂. Denoting
the adversaries information at time k as Ika , we observe that
ˆ D̂} ⊂ I a .
{M̂, C, (4.14)
k
4.2. Attacks on CPS 117

Disclosure Resources
Disclosure resources refer to the collection of data an adversary gathers
during the operation of a CPS. In our model, the attacker can potentially
observe a subset of control inputs and a subset of sensor measurements.
It can often be assumed that an attacker will only be able to observe
some constant fixed subset of inputs and outputs. We define the set
of readable inputs and outputs respectively as Sur , {i1 , · · · , ip0 } and
Syr , {j1 , · · · , jm0 }. Here j ∈ Sur and l ∈ Syr implies an attack can read
the jth entry of uk and lth entry of yk for all k.
Thus at each time step k an attacker is able to read Υy yk and
0 0
Υu uk−1 , where Υu ∈ {0, 1}p ×p , Υy ∈ {0, 1}m ×m . Moreover, Υu and Υy
are defined entrywise as
Υu (s, t) = 1is =t , Υy (s, t) = 1js =t . (4.15)
Here, Υu (s, t) refers to the entry of Υu at the sth row and tth column
and 1 is the indicator function. Thus, we observe that the attacker’s
information Ik grows as follows.
a
Ik−1 ∪ {Υy yk , Υu uk−1 } ⊂ Ika . (4.16)
Disclosure resources enable an attacker to improve upon prior knowl-
edge of a system’s model and controller, which in turn can be used
to construct an intelligent attack. The collection of data in a CPS by
passive observation is referred to as an eavesdropping attack. In certain
cases, disclosure resources can be directly used to construct an attack
without any need for additional data processing. One instance is a replay
attack, described later in this section.

Disruption Resources
An attacker’s disruption resources describe an attacker’s ability to act
on and corrupt a CPS under consideration. We examine three methods
through which an attacker can disrupt a CPS: integrity attacks, denial
of services attacks, and topology attacks. During an integrity attack, an
attacker is able to modify a subset of the defender’s inputs and outputs.
The dynamics of the attacked system are given by
xk+1 = Axk + Buk + B a uak + wk , yk = Cxk + Da dak + vk . (4.17)
118 Secure Cyber-Physical Systems

We make the assumption here that an attacker is restricted to manipulate

a fixed set of inputs and outputs described by the matrices B a and Da .
Unless otherwise stated, we without loss of generality, assume B a is
full column rank. We define the set of attackable inputs and outputs
respectively as Kua , {δ1 , · · · , δp∗∗ } and Kya , {η1 , · · · , ηm∗ }. In this
case, j ∈ Kua and l ∈ Kya implies an attack can modify the jth entry of
uk and lth entry of yk for all k. B a can be constructed as a matrix whose
columns are basis vectors of a subspace generated by {Bδ1 , · · · , Bδp∗∗ }
where Bj is the jth column of B. B a can be extended accordingly if an
attacker introduces additional actuators. We assume B a ∈ Rn×p∗ . The
sensor attack matrix Da ∈ Rm×m∗ can be defined entrywise as follows
Da (s, t) , 1s=ηi ,t=i . (4.18)
An integrity attack in a CPS can be performed both in the cyber and
physical realms. In the cyber realm, an attacker can simply modify
packets being delivered from the operator to the plant and vice versa.
In a physical attack, an adversary can leverage access to the plant
to alter the control input delivered by an actuator or manipulate the
environment around a sensor.
During a denial of service attack, an adversary is able to prevent
the operator from receiving sensor outputs or the plant from receiving
control inputs. If the attacker prevents the ith sensor from broadcasting
its information at time k, then (with some abuse of notation), the
ith measurement yk,r i received by the defender at time k will equal
∅ indicating the absence of information. Similarly, if the jth input is
jammed at time k, the jth input ujk,r received by the plant will equal ∅.
In many cases the absence of information is denoted using real numbers
with a 0. Let γk,i = 1 if the ith output measurement is received at time
k and let γk,i = 0 otherwise. Additionally, let νk,j = 1 if the jth input
is received at time k and let νk,j = 0 otherwise. Then
i
yk,r = γk,i yki , uik,r = νk,i uik . (4.19)
Finally, during a topology attack, the adversary is able to change pa-
rameters of the plant itself. This can be modeled as modifying (A, B, C)
to some Ã, B̃, C̃. In this case, the system dynamics are given by
xk+1 = Ãxk + B̃uk + wk , yk = C̃xk + vk . (4.20)
4.2. Attacks on CPS 119

In general an adversary can compose integrity, denial of service, and

topology attacks if he has the appropriate disruption resources. A full
analysis of compositions of attacks is being examined in current work.

4.2.4 Attack Strategies

We now describe several integrity attack strategies with a focus on
stealthy attacks. If an attack can be easily detected and isolated, a
defender can quickly deliver appropriate countermeasures, minimizing
the effect of an adversary. Alternatively, stealthy attacks allow adver-
saries to act on a system for long periods of time without a defender’s
response. This can potentially allow an attacker to maximize his or her
impact and significantly disrupt the correct operation of a CPS.

Zero Dynamics Attack

We commence the study of stealthy integrity attacks by examining the
set of stealthy attacks against a deterministic control system.
xk+1 = Axk + Buk + B a uak , yk = Cxk + Da dak . (4.21)
In this subsection, we will assume (A, C) is observable. Additionally,
assume that an attack commences at time 0 and that x0 is known to
the defender. Moreover, assume the defender’s control policy at time k
is a deterministic function of the model M = (A, B, C), the previous
inputs u0:k−1 , the previous outputs y0:k , and the initial state x0 so that
uk = Uk (A, B, C, u0:k−1 , y0:k , x0 ) (4.22)
For this system, it can be inductively shown that uk is deterministic.
As a result, yk is deterministic for all k. Let yk (x0 , u0:k−1 , ua0:k−1 , da0:k )
denote the output of yk as a function of the initial state, the defender’s
input, and the attacker’s output. Since yk is deterministic for all k, we
have the following definition:
Definition 4.2. A nonzero attack ua0:T −1 , da0:T on a deterministic sys-
tem (4.21) with controller (4.22) and known state x0 is stealthy or
undetectable up to time T if and only if
yk (x0 , u0:k−1 , ua0:k−1 , da0:k ) = yk (x0 , u0:k−1 , 0, 0), 0 ≤ k ≤ T. (4.23)
120 Secure Cyber-Physical Systems

By leveraging the linearity of the system, we arrive at the following

equivalent result, characterizing the set of stealthy attacks.

Theorem 4.2. An attack ua0:T −1 , da0:T on a deterministic system (4.21)

with controller (4.22) and known state x0 is stealthy up to time T if
and only if, there exists δx0 , · · · , δxT such that

δxk+1 = Aδxk + B a uak , 0 ≤ k ≤ T − 1, δx0 = 0, (4.24)

0 = Cδxk + Da dak , 0 ≤ k ≤ T. (4.25)

The proof follows from the definition of a stealthy attack and the
linearity of the system. Details are left to the reader. Note that the
stealthiness of an attacker’s inputs is independent of the defender’s
control strategy in the deterministic case. We next consider attacks that
are stealthy for all k ≥ 0. We define a perfect attack as follows.

Definition 4.3. A nonzero attack {uak }, {dak } is perfect if it satisfies

yk (x0 , u0:k−1 , ua0:k−1 , da0:k ) = yk (x0 , u0:k−1 , 0, 0), k≥0 (4.26)

In other words, the set of perfect attacks is the set of all attacks in
deterministic systems with known initial state that are stealthy for all
time k. We can relate perfect attacks to the fundamental property of
left invertibility.

Definition 4.4. Consider a system defined by (A, B, C, D), where

xk+1 = Axk + Buk , yk = Cxk + Duk ,

A system is left invertible if yk = 0, k ≥ 0 and x0 = 0 implies that

uk = 0, k ≥ 0.

Fundamentally, the left invertibility of a system implies that there

exists a unique input sequence generating every output sequence. This
is formalized below.

Theorem 4.3 (Pasqualetti et al. (2013)). There exists a perfect attack

on the system defined in (4.21) if and only if the system
(A, [B a 0n×m∗ ], C, [0m×p∗ Da ]) is not left invertible.
4.2. Attacks on CPS 121

The conditions for analyzing the left invertibility of a system can

be analyzed by looking at the matrix pencil. In particular, we have the
following based on results in (Trentelman et al., 2012).
Corollary 4.4. There exists no perfect attack on the system defined in
(4.21) if and only if for all but finitely many λ ∈ C, we have
" #!
λI − A −B a 0n×m∗
rank = n + m∗ + p∗ (4.27)
C 0m×p∗ Da
The existence of perfect attacks can also be described graphically
by considering the underlying structure of the inputs, outputs, and
state variables. This will be revisited later in the section. The set of
stealthy attacks in deterministic control systems can be increased if
the defender is unaware of the initial state. We assume now that the
defender’s control strategy satisfies
uk = Uk (A, B, C, u0:k−1 , y0:k ), (4.28)
where Uk is some deterministic function. It can be inductively shown
that a system with the same output history will have the same input
history. We can consequently define a stealthy or undetectable attack
as follows.
Definition 4.5. A nonzero attack ua0:T −1 , da0:T on a deterministic sys-
tem (4.21) with controller (4.28) and unknown state x0 is stealthy or
undetectable up to time T if and only if
yk (x0 , u0:k−1 , ua0:k−1 , da0:k ) = yk (x00 , u0:k−1 , 0, 0), 0 ≤ k ≤ T. (4.29)
for some x00 ∈ Rn . We refer to a nonzero attack that is stealthy for all
time k ≥ 0 as a zero dynamics attack.
We remark that perfect attacks are a subclass of zero dynamics
attacks. In practice, a defender may have some imperfect information
about x0 . Thus, x00 must be chosen carefully to avoid an alarm. The
existence of zero dynamics attacks is related to the strong observability
of a system.
Definition 4.6. Consider a system defined by (A, B, C, D), where
xk+1 = Axk + Buk , yk = Cxk + Duk ,
A system is strongly observable if yk = 0, k ≥ 0 implies that x0 = 0.
122 Secure Cyber-Physical Systems

We now aim to characterize systems that are vulnerable to zero

dynamics attacks. We have the following result.

Theorem 4.5. Suppose (A, C) is observable. The following statements

are equivalent.

1. There exists no zero dynamics attack on system (4.21).

2. There exists no nonzero inputs {uak }, {dak } satisfying

δxk+1 = Aδxk + B a uak , 0 = Cδxk + Da dak , δx0 ∈ Rn k ≥ 0.

(4.30)

3. (A, [B a 0n×m∗ ], C, [0m×p∗ Da ]) is strongly observable and left in-

vertible.
" #!
λI − A −B a 0n×m∗
4. rank = n + m∗ + p∗ , ∀ λ ∈ C
C 0m×p∗ Da
Sketch of Proof: The equivalence of statement 3 and 4 follows from
results in Trentelman et al. (2012). Statement 2 implies that if the output
is Cδxk + Da dak = 0 for all k, the attack inputs must be identically 0.
This implies left invertibility since the initial state is never specified.
Since the system is observable, this implies δx0 = 0, which implies strong
observability. Moreover, strong observability in statement 3 implies that
if Cδxk + Da dak = 0 in (4.30) for all k, δx0 = 0. Left invertibility
in statement 3 would then imply the inputs are necessarily 0. Thus
statement 2 and 3 are equivalent. The equivalence of statement 1 and 2
follows from the definition of a zero dynamics attack and the linearity
of the system. Details here are left to the reader.

Example 4.2. Consider the following control system

xk+1 = Axk + Buk , yk = Cxk ,

where
   
2 0 4 −3 2 1  
0 3 −2 1  1 −2 1 0 0 0
A= , B =   , C = 0 1 −1 0 .
     
3 −5 3 0 0 1 
0 0 −2 2
0 −1 −1 4 1 0
4.2. Attacks on CPS 123

Assume that the first actuator and first sensor are being attacked. In
h iT h iT
this case we have B a = 2 1 0 1 and Da = 1 0 0 . Moreover
m∗ = p∗ = 1.
From Corollary 4.4, we can see that there is no perfect attack. Indeed,
we have that for all but finitely many λ ∈ C.
λ−2 −4 −2 0
 
0 3
 0
 λ−3 2 −1 −1 0 

 −3 5 λ−3 0 0 0
 
 
rank  0
  1 1 λ − 4 −1 0  = 6 = n + m∗ + p∗ .

 1 0 0 0 0 1
 
−1
 
 0 1 0 0 0
0 0 −2 2 0 0
(4.31)
However, when λ = 1 or λ = 5, the rank of the preceding matrix pencil
drops to 5. As such, while there is no perfect attack, there does exist a
zero dynamics attack.
Before continuing, we remark that if B a and Da each have full
column rank (as currently constructed), then strong observability will
imply the left invertibility of a system. We now wish to assess the impact
of zero dynamics attacks. The true impact of the attack is dependent
on the control strategy Uk . For our purposes, we assume the defender’s
goal is to stabilize the system at 0. This can be accomplished even if x0
is unknown if (A, B) is stabilizable and (A, C) is detectable by using
state feedback and a stable observer.
Assume yk (x0 , u0:k−1 , ua0:k−1 , da0:k ) = yk (x00 , u0:k−1 , 0, 0) for all k ≥ 0.
Let xk (x0 , u0:k−1 , ua0:k−1 , da0:k ) denote the state xk generated by (4.21)
as a function of the initial state, the defender’s input, and the attacker’s
inputs. Under attack we observe that xk = xk (x0 , u0:k−1 , ua0:k−1 , da0:k ).
The defender, however has designed his feedback control inputs u0:k−1
so that he stabilizes a system with initial state x00 .
In this case, we make the assumption that
lim xk (x00 , u0:k−1 , 0, 0) = 0. (4.32)
k→∞
By the linearity of the system we see that
xk = xk (x00 , u0:k−1 , 0, 0) + xk (x0 − x00 , 0, ua0:k−1 , da0:k ). (4.33)
124 Secure Cyber-Physical Systems

Thus, if the attacker’s goal is to destabilize a control system, he may

wish to maximize kxk (x0 − x00 , 0, ua0:k−1 , da0:k )k2 . Using linearity, we
can show that 0 = yk (x0 − x00 , 0, ua0:k−1 , da0:k ). As a result, the attackers
perturbations on the state xk in our CPS can be approximately described
by the dynamics of δxk in (4.30). To understand the dynamics of δxk
we define the weakly unobservable subspace.
Definition 4.7. The weakly unobservable subspace Vu (A, B a , C, Da ) is
the set of δx0 ∈ Rn for which there exists {uak }, {dak } which allow (4.30)
to hold.
It can be shown that δxk ∈ Vu (A, B a , C, Da ) for all k ≥ 0. Moreover,
we have the following result from Trentelman et al. (2012) characterizing
the weakly unobservable subspace.
Lemma 4.6. Vu (A, B a , C, Da ) is the largest subspace of Rn for which
there exists linear maps F1 ∈ Rp∗ ×n and F2 ∈ Rm∗ ×n satisfying
(A + B a F1 )Vu ⊂ Vu , (C + Da F2 )Vu = 0 (4.34)
Methods to compute Vu as well as (non-unique) matrices F1 and
F2 are provided in Trentelman et al. (2012). We can now describe
the class of input strategies that allow an attacker to remain stealthy.
To begin we define the subspace (B a )−1 Vu = {u ∈ Rp∗ |Bu ∈ Vu }.
Moreover let L1 , L2 be a linear maps such that Im(L1 ) = (B a )−1 Vu
and Im(L2 ) = Ker(Da ) We have the following result based on the
characterization of inputs exciting a system’s zero dynamics found in
Trentelman et al. (2012).
Theorem 4.7. An attack {uak }, {dak } satisfies (4.30) if and only if
uak = F1 δxk + L1 ωk1 , dak = F2 δxk + L2 ωk2 (4.35)
where {ωk1 } and {ωk2 } are arbitrary sequences of real inputs of the proper
dimension and F1 , F2 satisfy (4.34).
We remark that since Da is full column rank, in practice L2 is an
empty matrix. We can see that δxk can be expressed as
k−1
X
δxk = (A + B a F1 )k δx0 + (A + B a F1 )k−1−j B a L1 ωj1 . (4.36)
j=0
4.2. Attacks on CPS 125

It is easy to imagine scenarios where an attacker can destabilize δxk

and thus the true state xk in the CPS. For instance, if there exists
v ∈ Vu satisfying lim supk→∞ k(A + B a F1 )k vk2 = ∞, the attacker can
cause the δxk to become unbounded. This can occur if the system has
an unstable invariant zero (a λ ∈ C, |λ| > 1 which causes the matrix in
(4.27) to drop rank). Alternatively, the set of feasible δxk can become
unbounded if (A + B a F1 , B a L1 ) has a nonempty reachable subspace.
Since B a and Da have full column rank this occurs if and only if the
(A, B a , C, Da ) is not left invertible.
The resources required by a zero dynamics attacker is also evident
from Theorem 4.7 and Lemma 4.6. In particular, the attacker’s system
knowledge must include (A, B a , C, Da ). The adversary, furthermore
requires disruption resources to insert an attack along B a and Da .
Finally, if an attacker can introduce additive perturbations, he or she will
require no disclosure resources. If additive perturbations are impossible,
then the attacker will need to be able to read the inputs and outputs of
the actuators and sensors he chooses to modify.
We next examine the effect of zero dynamics attacks on the stochastic
control system (4.17). In the case of a perfect attack, we can show that
an adversary remains stealthy. Specifically, we have the following.
Theorem 4.8. Suppose an attacker performs a perfect attack on (4.17).
Moreover, assume the defender’s control policy at time k is a deter-
ministic function of Ik = {M, y0:k , u0:k−1 , x̂−1|0 }. Then the probability
distribution of yk under attack f (yk |H1 ) is equal to the probability
distribution of yk under normal operation f (yk |H0 ).
Proof. yk (x0 , x̂−1|0 , u0:k−1 , ua0:k−1 , da0:k−1 , w0:k−1 , v0:k ) denote the
output as a function of the initial states, the defender’s and attacker’s
inputs, and the noise sequences. From the properties of a perfect attack
and the linearity of the system
yk (x0 , x̂−1|0 , u0:k−1 , ua0:k−1 , da0:k−1 , w0:k−1 , v0:k )
= yk (x0 , x̂−1|0 , u0:k−1 , 0, 0, w0:k−1 , v0:k ).
Using the fact uk is a deterministic function of Ik , we can inductively
show the sequence of control inputs remains the same both in the
presence and absence of an attack. The result follows.
126 Secure Cyber-Physical Systems

To understand the impact of general zero dynamics, we examine

attacker’s effect on the residue zk . Note under attack (4.17) applies to
the system dynamics. The Kalman filter equations (4.2),(4.3),(4.4) are
unchanged (though we assume Kk and Pk|k−1 have converge to K and
P ). Let zk (e0|−1 , v0:k , w0:k−1 , ua0:k−1 , da0:k ) be the residue zk generated
from (4.17),(4.2),(4.3),(4.4) due to the initial state estimation error, the
sensor noise, the process noise, and the attacker inputs. The attacker’s
bias on the residues is given by

∆zk , zk (e0|−1 , v0:k , w0:k−1 , ua0:k−1 , da0:k ) − zk (e0|−1 , v0:k , w0:k−1 , 0, 0).
(4.37)
We arrive at the ensuing result.

Theorem 4.9 (Weerakkody et al. (2017c)). Suppose an attacker performs

a zero dynamics attack on (4.17). Then, we have

∆zk = −C(A − AKC)k δx0 . (4.38)

From the stability of the Kalman filter the bias on the residue ∆zk
asymptotically approaches 0. In this case, we see that an attacker will be
asymptotically stealthy against a χ2 detector so that limk→∞ βk −αk = 0.
The prior result also applies to alternative continuous residue based
detectors with finite memory. We will soon demonstrate that small values
of ∆zk fundamentally lead to poor detection performance. Using the
same rationale as in the deterministic case, the impact of a zero dynamics
attack on the state xk in a stochastic system can be characterized using
the state δxk in (4.30). Consequently, in many scenarios, a zero dynamics
attack allows an attacker to surreptitiously destabilize a control system.
We conclude our study of zero dynamics attacks, by relating such
attacks to the class of unidentifiable attacks in control systems. We
assume an adversary is unable to insert their own actuators. Suppose an
attacker targets actuators Kua = {δ1 , · · · , δp∗ } ⊂ {1, · · · , p} and sensors
Kya = {η1 , · · · , ηm∗ } ⊂ {p + 1, · · · , p + m}. To write the corresponding
B a and Da uniquely as a function of their attack set we, without loss
of generality,
h assume all attacki sets are given in ascending order. Here,
B (Ku ) = Bδ1 · · · Bδp∗ where Bδi is the δi th column of B. Da (Kya )
a a

can be obtained entrywise as follows Da (s, t) , 1s=ηi −p,t=i . We assume

4.2. Attacks on CPS 127

that if a sensor or actuator is targeted in a window 0 ≤ k ≤ T , its value

has been modified by an attacker at least once during this time frame.
We let B a (K)ua0:k = {B a (K)ua0 , · · · , B a (K)uak }. Similarly, we have
D (K)da0:k = {Da (K)da0 , · · · , Da (K)dak }. Roughly speaking, we say an
a

attack is unidentifiable, if there exists an attack targeting a different

(but possibly intersecting) set of nodes with size less than or equal to
the original attack set. In other words, the nodes an adversary targets
provides the unique simplest explanation of an attack. Similar to the
notion of identifiability in Pasqualetti et al. (2013), we have the following
definition.

Definition 4.8. An attack input B a (Ku )ua0:T −1 , Da (Ky )da0:T on a deter-

ministic system (4.21) with controller (4.28) and unknown state x0 is
unidentifiable up to time T if and only if

1. there exists sets Ku0 ⊂ {1, · · · , p} and Ky0 ⊂ {p + 1, · · · , p + m}

with Ku 6= Ku0 or Ky 6= Ky0

2. |Ku0 | + |Ky0 | ≤ |Ku | + |Ky |.

3. there exists x00 ∈ Rn and inputs ūa0:T −1 , d¯a0:T satisfying.

yk (x0 , u0:k−1 , B a (Ku )ua0:k−1 , Da (Ky )da0:k )

= yk (x00 , u0:k−1 , B a (Ku0 )ūa0:k−1 , Da (Ky0 )d¯a0:k ), 0 ≤ k ≤ T (4.39)

Additionally, we say attack set Ku ∪ Ky is unidentifiable if there exists

an attack input targeting these nodes which is unidentifiable up to time
T = ∞. Otherwise we say Ku ∪ Ky is identifiable.

To be explicit here, when we write yk as a function, we must specify

the set of attacked sensors and inputs. We can easily see that un-
detectable attacks are also unidentifiable as the attack input can be
mistaken for a 0 attack. The class of unidentifiable attack inputs is
closely related to the class of zero dynamics attacks. For instance, we
have the following result.

Theorem 4.10. There exist an unidentifiable attack set of size q or less

if and only if there exists a zero dynamics attacks on a set of 2q or fewer
actuators or sensors.
128 Secure Cyber-Physical Systems

Proof. Suppose K = Ku ∪Ky is an unidentifiable attack set with |K| ≤ q

and Ku ⊂ {1, · · · , p} and Ky ⊂ {p + 1, · · · , p + m}. Then, there exists
K0 = Ku0 ∪ Ky0 with |K0 | ≤ |K|, K0 6= K, Ku0 ⊂ {1, · · · , p} and Ky0 ⊂
{p + 1, · · · , p + m} satisfying (4.39) for all k ≥ 0. This implies the
existence of a sequence of states {δxk }, and nonzero input sequence
{ũak }, {d˜ak }

δxk+1 = Aδxk + B a (Ku ∪ Ku0 )ũak , 0 = Cδxk + Da (Ky ∪ Ky0 )d˜ak . (4.40)

The input sequence is nonzero since K0 6= K and all sensors and actuators
are attacked. Thus, there exists a zero dynamics attack on a set of 2q or
fewer actuators or sensors. Now suppose there is a zero dynamics attack
on a set of 2q or fewer nodes K∗ . Assume, without loss of generality that
all nodes are attacked. In addition, without loss of generality assume
K∗ = K ∪ K0 where K = Ku ∪ Ky , K0 = Ku0 ∪ Ky0 , Ku0 , Ku ⊂ {1, · · · , p} ,
and Ky0 , Ky ⊂ {p + 1, · · · , p + m}. Moreover, without loss of generality,
assume K ≤ q, K0 ≤ q, K ∩ K0 = ∅, and |K0 | ≤ |K|. We know there
exists a zero dynamics attack {uak },{ūak },{dak },{d¯ak }, with each node
being attacked satisfying

δxk+1 = Aδxk + B a (Ku )uak − B a (Ku0 )ūak , (4.41)

0 = Cδxk + Da (Ky )da − Da (K0 )d¯a .
k y k (4.42)

Thus, for all k ≥ 0, we have an attack sequence {ūak },{dak } targeting all
sensors and actuators in K satisfying

yk (x0 , u0:k−1 , B a (Ku )ua0:k−1 , Da (Ky )da0:k )

= yk (x0 − δx0 , u0:k−1 , B a (Ku0 )ūa0:k−1 , Da (Ky0 )d¯a0:k )

As a result, preventing zero dynamics attacks coming from all sets

of 2q sensors and actuators will simultaneously prevent unidentifiable
attacks. This can be done by guaranteeing strong observability and left
invertibility for all sets of 2q sensors and actuators.

Corollary 4.11. Suppose (A, C) is observable. There exist no uniden-

tifiable attack set of size q or less if and only if for all K = Ku ∪ Ky
satisfying Ku ⊂ {1, · · · , p} and Ky ⊂ {p + 1, · · · , p + m} with |K| ≤ 2q,
4.2. Attacks on CPS 129

 
A, [B a (Ku ) 0n×|Ky | ], C, [0m×|Ku | Da (Ky )] is strongly observable and
left invertible.
This result follows immediately from Theorem 4.5 and Theorem
4.10.
Example 4.3. Consider the following control system
xk+1 = Axk + Buk , yk = Cxk , (4.43)
where
     
1 2 1 1 0 1 0 0
A = −2 3 1 B = 0 1 , C = 0 1 0 . (4.44)
     

3 −3 4 0 0 0 0 1
We would like to determine how many corrupted nodes can be identified.
By construction, since there are only 3 sensors, we know that there are
attacks on 3 nodes which can not be detected. Indeed, if all 3 sensors
are attacked, then for any eigenvalue λ(A) of A, we have
" #!
λ(A)I − A 0
rank < 6. (4.45)
C I
Next, it can be shown that for all feasible attacks K of size 2, and for
all λ ∈ C,
" #!
λI − A −B a (Ku ) 0n×m∗
rank = 5. (4.46)
C 0m×p∗ Da (Ky )
As a result, the attack system is strongly observable and left invertible
for any attack of size 2. Consequently, attacks of size 1 (a single corrupted
actuator or sensor) can be identified. Note, that as considered thus far,
the number of attack subsystems we have to potentially evaluate is
combinatorial. Simpler, graphical conditions for system verification will
be discussed in the next subsection.
We note that if B is not injective, this provides a path for an
adversary to generate unidentifiable attacks. For instance, if redundant
actuators are used and one or more are compromised, it would be
impossible for a defender to determine which if any actuators are secure.
While redundancy could compromise the ability to identify attacks, it
does not affect the ability to perform resilient estimation.
130 Secure Cyber-Physical Systems

Definition 4.9. Suppose an attacker can target up to q sensors and

actuators. We say that a defender can uniquely recover the state xj given
{yj , yj+1 , · · · } in the presence of attack input {B a (Ku )uak }, {Da (Ky )dak }
on a deterministic system (4.21) with controller (4.28) if there exists
no x0j ∈ Rn with x0j = 6 xj and sequences {B a (Ku0 )ūak }, {Da (Ky0 )d¯ak }
satisfying

yk (xj , uj:k−1 , B a (Ku )uaj:k−1 , Da (Ky )daj:k )

= yk (x0j , uj:k−1 , B a (Ku0 )ūaj:k−1 , Da (Ky0 )d¯aj:k ), k ≥ j (4.47)

where |Ku0 ∪ Ky0 | ≤ q.

In other words, we state that a defender can recover xj for a given

attack sequence, if there is no other state x0j and feasible set of attack
inputs that can generate the same output sequence. Similar, to Corollary
4.11, we can characterize systems for which the initial state is always
recoverable.

Theorem 4.12. Suppose an attacker can target up to q sensors and

actuators. A defender can recover the state xj for all feasible at-
tack sequences if and only if for all K = Ku ∪ Ky satisfying Ku ⊂
{1,
 · · · , p} and Ky ⊂ {p + 1, · · · , p + m} with |K| ≤ 2q, we have
A, [B a (Ku ) 0n×|Ky | ], C, [0m×|Ku | Da (Ky )] is strongly observable.

The proof is similar to the proof of Theorem 4.10 and is left to

the reader. Note that the index j is arbitrary. Thus, if the property
of strong observability is satisfied as stated in Theorem 4.12, then we
know that given the output sequence {y0 , y1 , y2 , · · · }, we can uniquely
recover the state sequence {x0 , x1 , x2 , · · · }.

False Data Injection Attack

In a false data injection attack, an adversary again introduces an additive
bias into the system. While certain works restrict their study of false data
injection attacks to only sensor corruption, we will analyze attacks which
occur on both a subset of inputs and a subset of outputs as in (4.17).
We will assume that the attacker is aware of (A, B, C, Q, R, K, P ) and
has disruption resources characterized by the matrices B a and Da . We
4.2. Attacks on CPS 131

moreover assume that inputs {dak } and {uak } are chosen independently
of the systems true inputs and outputs. Thus, they are independent of
the stochastic processes {wk } and {vk }.
In this case, we again assume the attacker wishes to remain stealthy.
As such the attacker designs his input sequence so that
q
∆zkT (CP C T + R)−1 ∆zk ≤ B, ∀k≥0 (4.48)

where we assume an attack begins at k = 0, ∆zk is defined in (4.37)

and B is some chosen bound for the attacker. We remark that due to
the stochastic nature of a system, an attacker practically does need not
choose the bound B = 0 to remain hidden, as long as the perturbations
introduced in the measurements are within the uncertainty of the system.
We can demonstrate that this false data injection attacker can impede
the performance of a passive detector.

Theorem 4.13 (Mo and Sinopoli (2016)). Consider a χ2 detector (4.13),

√
with window size WD = 1. Then for any B ∈ (0, τ ), if (4.48) holds,
then
√
βk ≤ (Γ(m/2))−1 Γ(m/2, ( τ − B)2 /2)), (4.49)
where Γ(s, x) , x∞ ts−1 e−t dt is the upper incomplete gamma function
R

and Γ(s) , Γ(s, 0) is the gamma function. Additionally,

√
lim (Γ(m/2))−1 Γ(m/2, ( τ − B)2 /2)) = αk , (4.50)
B→0+

where we note that the probability of false alarm αk for the χ2 detector
is a constant in k.

We can also make general statements relating the stealthiness of

false data injection attacks to all passive detectors using results from
Bai et al. (2017b) and Weerakkody et al. (2016c)

Theorem 4.14 (Weerakkody et al. (2016c)). For a false data injection

attack, suppose we have
−1
1 NX
lim sup ∆zkT (CP C T + R)−1 ∆zk > . (4.51)
N →∞ 2N k=0
132 Secure Cyber-Physical Systems

Then there exists some 0 < δ < 1 and a passive detector which satisfies
1
βk ≥ δ, ∀ k ≥ 0, lim sup − log(αk ) >  (4.52)
k→∞ k+1
Moreover, suppose yk collected under attack is ergodic and that
−1
1 NX
lim ∆zkT (CP C T + R)−1 ∆zk ≤ . (4.53)
N →∞ 2N
k=0

Then for any 0 < δ < 1, we have

1
βk ≥ δ ∀ k ≥ 0 =⇒ lim sup − log(αk ) ≤  (4.54)
k→∞ k+1
As seen here the bias introduced into the residues is related to the
best achievable decay rate in the probability of false alarm, with smaller
biases resulting in a smaller decay rate. We now aim to characterize the
impact of a false data injection attacker on a control system. As a first
case, a designer would like to guarantee that an adversary would not
be able to destabilize the CPS. In this case, we look at the attacker’s
impact on the state estimation error ek defined in (4.6). Here, we define

∆ek , ek (e0|−1 , v0:k , w0:k−1 , ua0:k−1 , da0:k ) − ek (e0|−1 , v0:k , w0:k−1 , 0, 0),
(4.55)
where ek is written as a function of the initial apriori estimation error,
the noise realizations, and the attacker’s inputs. It can be shown that

∆ek = (A − KCA)∆ek−1 + (B a − KCB a )uak−1 − KDa dak , (4.56)

∆zk = CA∆ek−1 + CB a uak−1 + Da dak . (4.57)

Stealthily destabilizing ∆ek for most feedback control policies allows

an attacker to destabilize the state in a control system. In particular,
∆ek = ∆xk − ∆x̂k|k where ∆xk is the bias the attacker introduces into
the state and ∆x̂k|k is the bias the attacker introduces into the state
estimate. If an attack is stealthy, ∆x̂k|k remains bounded due to the fact
that an unbounded state estimate always raises an alarm. Thus, ∆xk
is unbounded. For example, suppose the defender uses state feedback
4.2. Attacks on CPS 133

u = Lx̂k|k and (A + BL) is stable. Define ∆xk and ∆x̂k|k respectively

as

xk (x0 , x̂0|−1 , v0:k , w0:k−1 , ua0:k−1 , da0:k ) − xk (x0 , x̂0|−1 , v0:k , w0:k−1 ),
x̂k|k (x0 , x̂0|−1 , v0:k , w0:k−1 , ua0:k−1 , da0:k ) − x̂k|k (x0 , x̂0|−1 , v0:k , w0:k−1 ).

Here, xk and x̂k|k are written as a function of the initial state, the initial
apriori state estimate, the noise realizations, and the attack inputs. For
aesthetic purposes, we omit the attack inputs in the second term of
each expression, corresponding to the absence of an attack.
It can be shown that ∆xk is bounded if and only if ∆ek is bounded
when (4.48) is satisfied. Moreover, by similar methods the a priori
estimation error bias ∆ek|k−1 will be bounded if and only if ∆xk is
bounded. We now obtain the following result from Mo and Sinopoli
(2010), which characterizes when an attacker can destabilize ∆ek in the
sensor attack case.

Theorem 4.15 (Mo and Sinopoli (2010)). Consider a false data injection
attack where B a = 0. There exists a feasible attack input sequence
satisfying (4.48), which destabilizes ∆ek so that lim supk→∞ k∆ek k2 =
∞ if and only if A has unstable eigenvalue λ with eigenvector v, which
satisfies

1. Cv ∈ Im(Da ),

2. v is a reachable state of the system ∆ek = (A − KCA)∆ek−1 −

KDa dak .

Roughly speaking, in this scenario an attack excitets an unstable

mode in the system while simultaneously rendering it unobservable. In
the general case with nonzero B a a necessary condition for destabilizing
the CPS can be obtained.

Theorem 4.16. Consider a false data injection attack. There exists a

feasible attack input sequence satisfying (4.48), which destabilizes ∆ek
so that lim supk→∞ k∆ek k2 = ∞ only if there exists a real matrix La
and vector v ∈ Rn satisfying

1. Cv ∈ Im(Da ),
134 Secure Cyber-Physical Systems

2. v is an eigenvector of (A + B a La ).

The original result provided in Mo and Sinopoli (2012a) requires

first that v is an eigenvector of (A + B a La ) with eigenvalue λ and
second that Cv ∈ Im(Da ) or λ = 0. By performing similar analysis to
Mo and Sinopoli (2012a) on ∆ek|k−1 and demonstrating that ∆ek is
unbounded if and only ∆ek|k−1 is unbounded, we can remove the λ = 0
condition. We can leverage the prior theorem to relate the existence of
zero dynamics attacks to destabilizing integrity attacks in the following
original result.

Corollary 4.17. Consider a false data injection attack. Suppose (A, C)

is observable. There exists a feasible attack input sequence satisfying
(4.48), which destabilizes ∆ek so that lim supk→∞ k∆ek k2 = ∞ only if
there exists a zero dynamics attack.

Proof. Theorem 4.16 implies the existence of matrices F1 and F2 and

nonzero vector v ∈ Rn such that (A+B a F1 )v = λv and (C +Da F2 )v = 0.
From Lemma 4.6, this implies that the weakly unobservable subspace
Vu has nonzero dimension, which in turn implies the existence of zero
dynamics attacks.

In the event that an attacker is unable to destabilize a CPS using

false data injection attacks, we wish to characterize the extent to which
he can perturb a system, in particular the biases ∆ek and ∆xk the
attacker can inject while remaining stealthy. The attacker’s impact
on ∆xk depends on the controller. Here we assume uk = Lx̂k|k where
(A + BL) is Schur stable. In addition, the perturbations that an attacker
can introduce are dependent on the bound B. Since the system is linear,
without loss of generality, we take B = 1 Moreover, let P̄ = (CP C T +R).
It can be shown that
" # " # " # " # " #
∆xk+1 ∆xk ua 1 ∆xk ua
= Ā + B̄ a k , P̄ − 2 ∆zk+1 = C̄ + D̄ a k .
∆ek+1 ∆ek dk+1 ∆ek dk+1
(4.58)
4.2. Attacks on CPS 135

where ∆e0 , ∆x0 = 0 and

" # " #
A + BL −BL Ba 0
Ā = , B̄ = , (4.59)
0 A − KCA B − KCB −KDa
a a

1 1
h i h i
C̄ = P̄ − 2 0 CA , D̄ = P̄ − 2 CB a Da . (4.60)
Without loss of generality, it is assumed uak = 0, k ≤ −1 and dak =
0, k ≤ 0. We let ua∞ ,= {ua0 , da1 , ua1 , da2 , ua2 , · · · }.
Definition 4.10. An attacker’s actions ua∞ are feasible if (4.48) holds
for B = 1.
" #
∆xk
Definition 4.11. The reachable region Rk of is defined as
∆ek
" #
∆xk (ua∞ )
2n
Rk = {x̄k ∈ R , x̄k = for some feasible ua∞ }. (4.61)
∆ek (ua∞ )
The union of all Rk is defined as
∞
R = ∪ Rk . (4.62)
k=0
Thus, R characterizes all biases an attacker can inflict on the system.
We next provide a recursive definition of these reachability sets. In
order to do this, we must define the reach set, Rch, and the one step
set, Pre, operators.
Definition 4.12. Given a set S ⊂ R2n we define
Rch(S) , {x̄+ ∈ R2n : ∃ζ ∈ Rp∗ +m∗ , x̄ ∈ S (4.63)
s.t. Āx̄ + B̄ζ = x̄+ , kC̄ x̄ + D̄ζk2 ≤ 1},
Pre(S) , {x̄ ∈ R2n : ∃ζ ∈ Rp∗ +m∗ , (4.64)
s.t. Āx̄ + B̄ζ ∈ S, kC̄ x̄ + D̄ζk2 ≤ 1}.
The reach set of S characterizes the set of states that can be reached
while the current state is in S while ensuring the attacker is stealthy
at the current time step. Note that the reach set does not account
for stealthiness at future time steps. The one step set of S describes
the states that can be driven to a state in S in one time step, while
remaining stealthy at the current time step. Using the Rch and Pre
operators, the reachable set Rk can be recursively computed.
136 Secure Cyber-Physical Systems

Theorem 4.18 (Mo and Sinopoli (2016)). The reachability set Rk can
be computed as

Rk+1 = Rch(Rk ) ∪ C∞ , R0 = 0, (4.65)

∞
where C0 = R2n , Ck+1 = Pre(Ck ), and C∞ = ∩ Ck .
k=0

It can be shown that C∞ is the maximum controlled invariant set,

the largest set C ⊂ R2n for which for all x̄ ∈ C, there exists a ζ satisfying

Āx̄ + B̄ζ ∈ C, kC̄ x̄ + D̄ζ̄k2 ≤ 1.

We observe that the exact computation of the reachability sets as

currently defined is difficult. Ellipsoidal upper and lower bounds of
these reachability sets however are provided in Mo and Sinopoli (2016).
Moreover, exact computations of reachability sets in a slightly different
setting are provided in Kwon and Hwang (2017).

Example 4.4. We consider the following LTI system:

" # " #
1 0 1
xk+1 = xk + uk + wk , yk = xk + vk . (4.66)
1 1 0

The estimation and state feedback matrices are given by

" #
0.594 0.079 h i
K= , L = −1.244 −0.422 . (4.67)
0.079 0.694
h iT
In the first case we assume Da = 1 0 so that the first sensor is
attacked. An inner and outer bound approximation of R is given in
Figure 4.1. Here we project the ellipsoid defined in R4 along the state
in Figure 4.1(a) and the portion along the estimation error in Figure
4.1(b). Since R is bounded, the attacker can not destabilize the system
using the first sensor.
h iT
In the second case, we assume Da = 0 1 so that the second
sensor is attacked. The inner approximation Rk is found in Figure
4.2. We see the bounds grow in time. In fact, from Theorem 4.15, the
reachable set is unbounded.
4.2. Attacks on CPS 137

(a) State (b) Estimation Error

Figure 4.1: Inner and Outer Approximation of R, Attack on Sensor 1

(a) State (b) Estimation Error

Figure 4.2: Inner Approximation of R1 to R7 , Attack on Sensor 2

Replay Attack
During a replay attack, an attacker attempts to fool the defender into
believing a system is operating normally, by sending a prior sequence
of sensor measurements, collected during normal system operation. The
attacker follows the following steps.

1. (s)he records a long sequence of outputs y−T 0 :−T where (T 0 > T >
0). As a result, Υy = I.

2. Starting at time 0, (s)he replaces yk with yk0 , yk−T 0 . Thus,

Da dak = yk0 − Cxk − vk for 0 ≤ k ≤ T 0 − T .
138 Secure Cyber-Physical Systems

3. Starting at k = 0, the attacker may add some harmful input B a uak .

A replay attack requires a defender to have the disclosure resources

to observe all outputs and the disruption resources to modify all outputs
and potentially a subset of inputs. While, this is a relatively powerful
attacker, observe that a replay strategy does not require much system
knowledge. At the most, an attacker should have enough understanding
of the system to design harmful inputs uak .
Though an attacker can only record a finite sequence of outputs prior
to commencing an attack, in practice the sequence will be long enough
for an attacker to sustain significant damage to the CPS. The recording
can be made longer by looping previous outputs. For simplicity, we
assume the attacker has an infinite recording.
Without defender action, a replay attack can cause significant dam-
age. First, we remark that a replay attack prevents a defender from
performing proper closed loop feedback. This can have significant con-
sequences, especially if the system is open loop unstable. Moreover, the
attacker can inject an additive bias on state of the physical system along
the controllable subspace of (A, B a ). Assuming the reachable subspace
is nonempty, this is sufficient to destabilize the system.
Consequently, the only way to thwart a replay attack is to recognize
its presence. Unfortunately, in many scenarios, it can be shown that a
replay attack is asymptotically stealthy where limk→∞ (βk − αk ) = 0.

Example 4.5. Suppose a defender uses a χ2 detector. Moreover, assume

that the control input is a fixed function h of the current output so that
uk = h(yk ). Then, we have

x̂k+1|k = (A − AKC)x̂k|k−1 + AKyk0 + Bh(yk0 ), (4.68)

x̂0k+1|k = (A − AKC)x̂0k|k−1 + AKyk0 + Bh(yk0 ), (4.69)

where x̂0k|k−1 , x̂k−T 0 |k−T 0 −1 . Additionally, letting zk0 , zk−T 0 it can

be shown that zk = zk0 + C(A − AKC)k (x̂00|−1 − x̂0|−1 ). Letting ζ0 ,
4.2. Attacks on CPS 139

x̂00|−1 − x̂0|−1 , the χ2 statistic under replay attack is given

k
T
zj0 (CP C T + R)−1 (zj0 + 2C(A − AKC)j ζ0 )
X

j=k−WD +1
 T
+ C(A − AKC)j ζ0 (CP C T + R)−1 C(A − AKC)j ζ0 (4.70)

We note that zj0 is collected under normal operation so that zj ∼

N (0, CP C T + R). Noting that {zk } is an IID sequence under normal
operation, this implies that zj under normal operation has the same
distribution as zj0 . Moreover, due to properties of the Kalman filter,
1
(since (A, C) is detectable, (A, Q 2 ) is stabilizable, and R > 0), it can be
shown that (A − AKC) is Schur stable. As a result, for this controller
we have
lim βk − αk = 0
k→∞

Example 4.6. Suppose a defender uses a χ2 detector. Moreover, assume

the defender uses state feedback control where uk = Lx̂k|k−1 . Then, if
ρ((A + BL)(I − KC)) < 1, where ρ(·) is the spectral radius, it can be
shown that (Mo and Sinopoli, 2009)

lim βk − αk = 0.
k→∞

Alternatively, if ρ((A+BL)(I −KC)) > 1 and moreover, limk→∞ C((A+

BL)(I − KC))k (x̂00|−1 − x̂0|−1 ) 6= 0 (which occurs with high probability),
then
lim βk = 1
k→∞

A general condition applying to a larger class of systems, controllers,

detectors is obtained follows from Mo et al. (2014b). Let sk be the state
of an estimator evolving according to a function f¯ as follows

sk+1 = f¯(sk , yk ). (4.71)

We can additionally define the seminorm of f¯ as

kf¯(s, y) − f¯(s + ∆s, y)k

kf¯ks , sup (4.72)
∆s6=0,y,s k∆sk
140 Secure Cyber-Physical Systems

We assume some threshold detector is being used so that

k
X H1
g(sk , yk ) ≷ τ. (4.73)
j=k−WD +1 H0

where g is some arbitrary continuous function.

Theorem 4.19 (Mo et al. (2014b)). If kf¯ks ≤ 1, then limk→∞ βk − αk =

0.

We conclude this subsection on attacks in CPS by depicting the

cyber-physical attack space in Figure 4.3, which categorizes previously
discussed attacks in terms of disclosure resources, disruption resources,
and system knowledge. Beyond the attacks we have already discussed,
Figure 4.3 makes reference to a covert attack (Smith, 2015). In this
attack, an adversary can read and disrupt all input and output channels
and has full system knowledge. Here, the attacker leverages his ability
to modify all sensor measurements to cancel out the effect of his attack
in the measurements so as not to be detected by the defender. It is clear
that in such a scenario, little can be done to ensure resilience. In future
subsections, we will examine how system design can be used to limit
the likelihood that such an attacker can exist.

4.2.5 Further Reading

The classification of adversaries in this subsection is motivated by
the discussion of a secure framework for resource limited adversaries
presented in Teixeira et al. (2012a) and Teixeira et al. (2015). Along
with this framework, the authors include discussions of denial of service
attacks, eavesdropping attacks, a subclass of zero dynamics attacks,
bias injection attacks, and replay attacks. In this subsection, we did
not spend significant time discussing denial of service (DoS) attacks
Fortunately, resilient methods to counter DoS attacks have been well
studied for instance in Amin et al. (2009) and Yuan et al. (2013).
The class of zero dynamics attacks have been well studied. For
instance, the presence of zero dynamics was linked to attack detectability
4.2. Attacks on CPS 141

System Knowledge

Zero Dynamics
Attack

Covert
False Data
Attack
Injection
Attack

Eavesdropping
Attack
Disclosure
Resources
DoS Attack

Replay Attack
Disruption
Resources

Figure 4.3: Cyber-Physical Attack Space

and identifiability in Pasqualetti et al. (2013) and methods for revealing

zero dynamics attacks have been for instance examined by Teixeira et al.
(2012b) and Hoehn and Zhang (2016a). Classes of stealthy attacks when
the defender has side information about the initial state is considered
in Chen et al. (2017b).
False data injection attacks have been well studied, especially in
the context of the smart grid. Liu et al. (2011) in their seminal work
demonstrated how intelligently constructed attacks on sensors in the
smart grid can be used to mislead state estimators. Extensive studies
have followed, which for instance have considered attacks against AC
state estimation (Hug and Giampapa, 2012), examined the affect of
attacks on the electricity market (Xie et al., 2010; Tan et al., 2013),
and offered potential defense mechanisms (Bobba et al., 2010; Bi and
Zhang, 2014; Liu et al., 2014; Chaojun et al., 2015). The work in this
subsection is based off of analysis in general control systems (Mo and
Sinopoli, 2010; Mo et al., 2010). Though typical analysis of false data
injection attacks applies specifically to sensor attacks, we examined the
impact of attacks that can occur at both sensors and the control inputs
142 Secure Cyber-Physical Systems

as done in Mo and Sinopoli (2012a) and Mo and Sinopoli (2016) using

reachability sets. Similar reachability analysis is found in Kwon and
Hwang (2016) and Kwon and Hwang (2017).
Other notions of a deceptive attacker’ effect on a control system
has been characterized. In Bai et al. (2017a) and Bai et al. (2015), the
-stealthiness concept is introduced for a linear time-invariant systems
with a single sensor. This concept is then related to the Kullback–Leibler
divergence (KLD), which is well known in information theory. In Bai et
al. (2017a), the tradeoff between the stealthiness of the attacker and the
maximal mean squared error that an attacker can induce is characterized.
Later, Kung et al. (2017) extend the notion of -stealthiness to higher
order systems, and show how the performance of an attacker differs in
the scalar and vector cases. Moreover, Zhang and Venkitasubramaniam
(2016) considers a system in the same setting with a finite horizon linear
quadratic Gaussian cost. The authors quantify the trade-off between the
increase in this cost and the detectability of the input injected by the
adversary (as measured by the KL divergence). We also refer the reader
to Chen et al. (2017a) which derives optimal attack strategies with
respect to a linear quadratic cost that combines an attacker’s control
and stealthiness criteria.
The study of replay attacks in control systems was first carried out
in Mo and Sinopoli (2009) where the authors introduce the method of
physical watermarking to detect such attacks. Several extensions have
considered the design of watermarks to counter replay attack strategies
(Miao et al., 2013; Mo et al., 2014b; Mo et al., 2015). Additional analysis
and defense techniques against replay attacks are considered by Zhu
and Martinez (2014) and Hoehn and Zhang (2016b).

4.3 Robust Structural Design of Control Systems

In the previous subsection, we demonstrated the existence of damaging

attacks which can go unrecognized by the defender. In this subsection,
we begin investigating methods for a defender to prevent and respond
to failures in control systems through robust analysis and design. We
examine how structural properties of control systems, specifically the
links between attack inputs, control inputs, and sensor outputs can
4.3. Robust Structural Design of Control Systems 143

be used to characterize the vulnerability of a system to several class

of attacks and provide tools that allow defenders to design minimum
systems which can prevent these attacks
We place emphasis on distributed control systems (DCSs). A DCS is
a system where components such as sensors, actuators, and controllers
are separated over a large network. DCSs allow operators to control
multiple local environments while simultaneously meeting various global
objectives. The ability of a DCS to meet society’s demands for large
scale control has made such systems common in a variety of applica-
tions including sensor networks, the smart grid, vehicular systems, and
manufacturing.

4.3.1 Distributed Control System Model

Graphical Model: We will model a Distributed Control System (DCS)
both graphically and algebraically. We assume there are n agents,
X , {x1 , · · · , xn } that communicate with each other and are observed
by m sensors, Y , {y1 , · · · , ym } where we assume m ≤ n. We model
interactions using a directed graph G = (V, E) with vertices V , X ∪ Y.
The edges E ⊂ V × V capture agent/sensor interactions. If (xi , xj ) ∈ E,
agent xi sends messages to xj . If sensor yj measures state xi , (xi , yj ) ∈ E.
Each agent xi ∈ X has a self-loop, therefore (xi , xi ) ∈ E. The incoming
neighbors to a node vi or NvIi ⊂ V, and the outgoing neighbors NvOi ⊂ V
from vi are
NvIi , {vj | (vj , vi ) ∈ E}, NvOi , {vj | (vi , vj ) ∈ E}. (4.74)
Algebraic Model: We assume each agent xi has a scalar time depen-
dent state xi (k) with dynamics given as follows:
xi (k + 1) = aii xi (k) + ui (k). (4.75)
The input ui (k) is a linear function of the states of xi ’s incoming
neighbors and a centrally known input uif f (k) so
ui (k) = uif f (k) +
X
aij xj (k), (4.76)
j6=i

/ NxIi ∩ X =⇒ aij = 0. Without loss of generality uif f (k) =

where xj ∈
0. Each agent is assumed to have a scalar state. From a notational
144 Secure Cyber-Physical Systems

perspective, in this section on robust structural design, we write the

discrete time index k as function of the states, inputs, and measurements
as opposed to a subscript in order to distinguish vertices in graphs from
numerical parameters.
Remark 4.1. The state xi (k) can refer to a physical quantity such
as temperature or simply a quantity for distributed processing (e.g.
consensus). While it is assumed each agent has a scalar state in this
subsection, similar tools for DCS analysis and design can be incorporated
in the vector case. Examining the vector case is a subject of current
research.
A set of dedicated sensors Y measure the state of a subset of agents.
The outputs are sent to a central operator for estimation and detection.
A dedicated sensor measures the state of one agent and no two sensors
measure the state of the same agent. The output of sensor yi measuring
xj at time k is
yi (k) = xj (k). (4.77)
Remark 4.2. The assumption of dedicated sensors capture the dis-
tributed nature of the system. We ignore redundant sensors since an
attacker can typically corrupt all sensors measuring xi if he corrupts
one of them.
For simplicity, we concatenate state and output vectors
h iT h iT
x(k) , x1 (k) · · · xn (k) , y(k) , y1 (k) · · · ym (k) ,
so that the dynamics of the full control system are given by
x(k + 1) = Ax(k), y(k) = Cx(k). (4.78)
The pair (A, C) is assumed to be observable. Letting 1 be the indicator
function, A and C can be defined entrywise:
A(i, j) = aij , C(i, j) = 1(xj ,yi )∈E .
Since (A, C) is observable, the state can be estimated using a linear
filter.
x̂(k + 1) = (A − KCA)x̂(k) + Ky(k + 1), (4.79)
z(k + 1) = y(k + 1) − CAx̂(k). (4.80)
4.3. Robust Structural Design of Control Systems 145

Here, K is chosen so (A − KCA) is Schur stable. The residue z(k) is

a statistic often used to perform detection. Smaller residues are often
indicative of normal behavior while larger residues are associated with
faulty or malicious behavior.

4.3.2 DCS Attack Model

Graphical Model: We now define our DCS model under attack. At time
0 an unknown subset of the agents and sensors F are compromised.
No more than q agents and sensors can be corrupted. In other words,
the operator would like the system to be resilient to up to q malicious
failures. The set of all feasible sets of attacked nodes is given by F:

F = {F ⊂ V : |F | ≤ q}. (4.81)

We define the graph GFa = (VFa , EFa ) of a DCS when a set F of agents/
sensors is compromised.

F = {xl1 , · · · , xlp , ylp+1 , · · · , ylq∗ }, (q∗ ≤ q)

We introduce attack input vertices UFa = {ua1 , · · · , uaq∗ }. We assume

there exists directed edges from UFa to F given by

EUFa ,X , {(ua1 , xl1 ), · · · , (uap , xlp )}

EUFa ,Y , {(uap+1 , ylp+1 ), · · · , (uaq∗ , ylq∗ )}

We then define EFa , E ∪ EUFa ,X ∪ EUFa ,Y and VFa , V ∪ UFa .

Algebraic Model: We let xia (k) represent the state of xi under attack.
If (ual , xi ) ∈ EFa , then the dynamics are
X
xia (k + 1) = aii xia (k) + aij xja (k) + ula (k), (4.82)
j6=i

where ula (k) is an input from node ual at time k. If xi is secure then
ula (k) = 0. We define yia (k) as the output of yi at time k under attack.
If (ual , yi ) ∪ (xj , yi ) ⊂ EFa , then

yia (k) = xja (k) + ula (k). (4.83)

146 Secure Cyber-Physical Systems

If yi is secure then in (4.83), ula (k) = 0. Concatening xia (k), yia (k), and
uia (k) into xa (k), ya (k), and ua (k), we have :

xa (k + 1) = Axa (k) + BFa ua (k), xa (0) = x(0), (4.84)

a a
y (k) = Cx (k) + DFa ua (k), (4.85)

with BFa (i, j) , 1(uaj ,xi )∈EU a ,X , DFa (i, j) , 1(uaj ,yi )∈EU a ,Y . We assume the
F F
attacker knows (A, BFa , C, DFa ). The estimator policy remains unchanged
during an attack.

x̂a (k + 1) = (A − KCA)x̂a (k) + Kya (k + 1), (4.86)

a a a
z (k + 1) = y (k + 1) − CAx̂ (k). (4.87)

Note that there are some slight changes in notation compared to

subsection 4.2. In particular, the two separate attack inputs uak and dak
are concatenated here. The definition of matrices B a and Da change
accordingly.

Remark 4.3. Note that in this subsection, we assume that an attacker

can target both sensors and agents. However, in certain circumstances,
it may be more realistic to assume an attacker can only compromise
agents. In this case, the states delivered to an agent’s outgoing neighbors
and the central monitor will be the same. Analogous results can be
obtained in this circumstance.

4.3.3 Attack Strategy

As in the prior subsection we assume the attacker’s goal is to inject
inputs into the distributed control system in order to bias the state,
while remaining stealthy. Here, we use notions of stealthiness defined in
Definition 4.2 for known initial state x0 and Definition 4.5 for unknown
initial state x0 . From the previous subsection, the set of nonzero stealthy
attacks uka are the set of perfect attacks on the system (A, BFa , C, DFa )
when the initial state is known and the set of zero dynamics attacks
when the initial state is unknown.
It thus becomes a goal to design (A, BFa , C, DFa ) to be left invertible if
x(0) is known to the defender and strongly observable and left invertible
if (A, BFa , C, DFa ) if x(0) is unknown. This will ensure that an adversary
4.3. Robust Structural Design of Control Systems 147

will be unable to construct a stealthy attack. Of course, in practice, the

defender does not know the set F . In such a case, we wish to design (A, C)
so that (A, BFa , C, DFa ) remains strongly observable and/or left invertible
for all feasible attack vectors. We can use structural conditions to obtain
a graphical characterization of these properties. We associate GFa with
a tuple of structural matrices ([A], [BFa ], [C], [DFa ]). Here, for a matrix
M , [M ](i, j) 6= 0 implies M (i, j) is a free parameter. Alternatively,
[M ](i, j) = 0 implies M (i, j) = 0.
Observe that EFa = EX ,X ∪ EUFa ,X ∪ EX ,Y ∪ EUFa ,Y where EX ,X =
{(xi , xj ) : [A](j, i) 6= 0}, EUFa ,X = {(ui , xj ) : [BFa ](j, i) 6= 0}, EX ,Y =
{(xi , yj ) : [C](j, i) 6= 0}, and EUFa ,Y = {(ui , yj ) : [DFa ](j, i) 6= 0}.
We note here that G is associated with the structural system
([A], [C]). It can be shown that a system’s structure can be linked
to left invertibility and strong observability.

Definition 4.13. ([A], [BFa ], [C], [DFa ]) is structurally left invertible (or
strongly observable) if an admissible realization of (A, BFa , C, DFa ) is
left invertible (or strongly observable).

We remark that if ([A], [BFa ], [C], [DFa ]) is structurally left invertible

(strongly observable), then every valid realization of (A, BFa , C, DFa ) is
left invertible (strongly observable) except for a set of zero Lebesgue
measure. Additionally, if ([A], [BFa ], [C], [DFa ]) is not structurally left
invertible (strongly observable), no valid realization of (A, BFa , C, DFa ) is
left invertible (strongly observable). Thus, we aim to design DCSs that
are structurally strongly observable and left invertible for all feasible
attack sets F to almost surely prevent zero dynamics attacks when x(0)
is unknown. If x(0) is known, we only need to design DCSs that are left
invertible for all feasible attack sets F to almost surely prevent perfect
attacks
For ease of reference, we introduce the following definition to char-
acterize vulnerable systems.

Definition 4.14. A system (A, C) is discreetly attackable if there is a

set F ∈ F for which ([A], [BFa ], [C], [DFa ]) is not structurally strongly
observable and left invertible.
148 Secure Cyber-Physical Systems

Definition 4.15. A system (A, C) is perfectly attackable if there is a

set F ∈ F for which ([A], [BFa ], [C], [DFa ]) is not left invertible.
Of course as we saw earlier, preventing zero dynamics attacks can
simultaneously allow an attacker to perform attack identification and
resilient estimation in the presence of adversarial behavior. In particular,
ensuring a system is not discreetly attackable for 2q < m malicious
nodes almost surely allows the defender to identify attacks coming
from up to q sensors and actuators. Moreover, it allows the defender to
recover the initial state x(0) (and thus ensuing states since the system
is deterministic) in the presence of q malicious sensors and agents.

4.3.4 Graph Theory Preliminaries

In this subsection we introduce necessary preliminaries from graph
theory. Consider a graph G = (V, E). Two edges (v1 , v2 ) and (v10 , v20 ) are
vertex disjoint or v-disjoint if v1 6= v10 and v2 6= v20 . A set of edges are
v-disjoint if each pair are v-disjoint. Consider sets A ⊂ V and B ⊂ V .
An edge (v1 , v2 ) from A to B has v1 ∈ A and v2 ∈ B. We define
θ(A, B) , max number of v − disjoint edges from A to B.
A path from a set A ⊂ V to B ⊂ V , is a sequence v1 , v2 , · · · , vr where
v1 ∈ A , vr ∈ B, and (vi , vi+1 ) ∈ E for 1 ≤ i ≤ r − 1. A simple path has
no repeated vertices. Two paths are disjoint if they contain no common
vertices. Two paths are internally disjoint if they have no common
vertices except for possibly the starting and ending vertices. In general
l paths are (internally) disjoint if every pair of paths are (internally)
disjoint. A set of l disjoint and simple paths from A ⊂ V to B ⊂ V is
referred to as a linking of size l or a l-linking from A to B. We define
ρ(A, B) , size of the largest linking between A and B.
A vertex separator between a ∈ V and b ∈ V (with (a, b) ∈ / E)
is a set S ⊂ V \{a, b} whose removal deletes all paths from a to b.
As shorthand, we refer to S as a vertex separator between (a, b). A
minimum vertex separator S between (a, b) is a vertex separator between
(a, b) with the smallest size. The size of a minimum vertex separator
can be linked to the number of disjoint paths between 2 vertices.
4.3. Robust Structural Design of Control Systems 149

Theorem 4.20 ( Menger (1927)). The size of a minimum vertex sepa-

rator S between (a, b) is equal to the maximum number of internally
disjoint paths between a and b.
We define the set of essential vertices, Vess (A, B) ⊂ V :
Vess (A, B) , {x|x ∈ all ρ(A, B) − linkings from A to B}.
Suppose we add new vertices a and b to graph G where a has directed
edges to A and b has directed edges coming from B. Then, we have
Vess (A, B) = ∪S∈Sa,b S, where Sa,b is the set of all minimum vertex
separators between (a, b).

4.3.5 Structural Analysis

We now provide conditions which graphically characterize structural
left invertibility and strong observability for fixed (A, BFa , C, DFa ). The
ensuing two theorems make no assumptions regarding the structure
([A], [BFa ], [C], [DFa ]). In particular, we can remove prior assumptions
that the attack inputs are dedicated, the sensor outputs are dedicated
outputs, and each agent has a self loop. We have the following results.
Theorem 4.21 (Van der Woude (1991)). For fixed UFa , |UFa | = q, a
system is structurally left invertible if and only if for GFa , we have
ρ(UFa , Y) = q.
Theorem 4.22 (Boukhobza et al. (2007) and Boukhobza and Hamelin
(2009)). For fixed UFa , |UFa | = q, a system is structurally strongly
observable and left invertible if and only if for GFa
1. θ(X ∪ UFa , X ∪ Y) = n + q.

2. Every agent xi ∈ X has a path to Y.

3. ∆0 ⊂ Vess (UFa , Y)
where ∆0 = {x ∈ X |ρ(x ∪ UFa , Y) = ρ(UFa , Y)}.
As a result, structural left invertibility for a fixed attack strategy
requires the existence of a |UFa |-linking from the set of attack inputs to
the set of outputs. Moreover, stronger conditions are required to achieve
strong observability.
150 Secure Cyber-Physical Systems

Example 4.7. Consider a control system with the following structure.

   
∗ 0 ∗ ∗ ∗ 0
0 ∗ ∗ ∗ ∗ 0
[A] =   , [BFa ] =  ,
   
∗ ∗ ∗ 0 0 0
0 ∗ ∗ ∗ ∗ 0
   
∗ 0 0 0 0 ∗
a
[C] = 0 ∗ ∗ 0 , [DF ] = 0 0 .
   

0 0 ∗ ∗ 0 0
From Theorem 4.22, the system is structurally strongly observable and
left invertible.
First, the edges (u2 , y1 ), (u1 , x2 ), (x2 , y2 ), (x1 , x1 ), (x3 , x3 ), (x4 , x4 )
constitute a maximal matching so that θ(X ∪ UFa , X ∪ Y) = n + q = 6.
Next, each agent or state has a path to the outputs. In fact each
state has a directed edge to a sensor (x1 , y1 ), (x2 , y2 ), (x3 , y2 ), (x4 , y3 ).
Finally, we note that ∆0 = ∅. Indeed ρ(x1 ∪UFa , Y) = ρ(x2 ∪UFa , Y) =
ρ(x3 ∪ UFa , Y) = ρ(x4 ∪ UFa , Y) = 3 while ρ(UFa , Y) = 2.
We are careful to remark that just because this system is structurally
strongly observable and left invertible does not mean all valid realizations
are strongly observable and left invertible. In fact the system considered
in Example 4.2 is a valid realization which we have shown is vulnerable
to a zero dynamics attack. Nonetheless, almost all valid realization
except a set of zero Lesbesgue measure will be strongly observable and
left invertible. In particular, a structurally strongly observable and left
invertible system with independent system parameters (system matrix
entries) will almost surely be strongly observable and left invertible.

We now aim to describe sufficient and necessary conditions for a

defender to ensure a system is not perfectly attackable or discreetly
attackable. Here, to obtain the most general result, we make no as-
sumptions regarding the existence of self loops or dedicated outputs.
However, prior assumptions which dictate that an attacker can target
any q set of sensors and states with dedicated inputs remains in place.
Define the graph g(G) , (V ∪ {o}, E 0 ) by adding a node o with
incoming directed edges from all sensors Y to graph G. We have the
following:
4.3. Robust Structural Design of Control Systems 151

Theorem 4.23 (Weerakkody et al. (2016a)). A DCS is not perfectly

attackable if and only if for all xi ∈ X , the minimum vertex separator
Si between (xi , o) in g(G) has size |Si | ≥ q.

Theorem 4.24 (Weerakkody et al. (2017a)). A DCS is not discreetly

attackable if and only if

1. For all T ⊂ X ∪ Y with |T | = q, θ(X , (X ∪ Y)\T ) = n.

2. For all xi ∈ X , the minimum vertex separator Si between (xi , o)

in g(G) has size |Si | ≥ q + 1.

In view of these results, we conclude that avoiding perfect attacks is

equivalent to each agent having q disjoint paths to the output. Avoiding
zero dynamics attack will require one additional independent path to the
output and a maximum matching condition. We make the assumption
here that an attacker can insert dedicated attack inputs on any q
agents or sensors. Note that under this assumption, the prior results in
Theorems 4.23 and 4.24 are necessary and sufficient. Of course, if the
attacker can only inject dedicated inputs at a subset of insecure agents
and sensors, the conditions described in Theorem 4.23 and 4.24 remain
sufficient for preventing perfect and zero dynamics attacks respectively.
We now consider a more general case without the assumption of
dedicated inputs. Thus attack nodes in U a can have directed edges
to multiple agents and sensors. The corresponding edges are given by
EU a ,X = {(ui , xj ) : [B a ](j, i) 6= "
0} and a
# EU ,Y = {(ui , yj ) : [D ](j, i) 6= 0}.
a

B a
We make the assumption that is injective. This means the attack
Da
inputs need not be dedicated attack inputs. We attempt to make this
assumption structurally
" # with θ(U a , X ∪ Y) = |U a |. This is a necessary
Ba
condition for to be injective and it is a sufficient condition almost
Da
surely.
This assumption can be made without loss of generality, as the effect
of an attack on the state of a system where the input matrices are not
injective is equivalent to the effect of an alternative attack on a system
where the input matrices are injective.
152 Secure Cyber-Physical Systems

Corollary 4.25. For all feasible sets of attacks, assume that we have
|U a | ≤ q and θ(U a , X ∪ Y) = |U a |. Then, a system is structurally
left invertible for all feasible attack vectors if for all xi ∈ X , the
minimum vertex separator Si between (xi , o) in g(G) has size |Si | ≥ q.
Moreover, the system is structurally left invertible and structurally
strongly observable if

1. For all T ⊂ X ∪ Y with |T | = q, θ(X , (X ∪ Y)\T ) = n.

2. For all xi ∈ X , the minimum vertex separator Si between (xi , o)

in g(G) has size |Si | ≥ q + 1.

The proof is similar to the proofs of Theorem 4.23 and 4.24 and
is left to the reader. The conditions obtained to stop attackers with
dedicated inputs remain sufficient for security even when the attacker
has the freedom to inject inputs that are not dedicated. We note that in
practice, it may be difficult for a defender to verify the first condition
of Theorem 4.24 as the problem appears combinatorial. However, the
problem is greatly simplified if each agent has self loops.

Corollary 4.26 (Weerakkody et al. (2017a)). Suppose each agent xi ∈ X

has a self loop. A DCS is not discreetly attackable if and only if for all
xi ∈ X , the minimum vertex separator Si between (xi , o) in g(G) has
size |Si | ≥ q + 1.

If g(G) has self-loops at each agent, we can efficiently determine if a

system is discreetly attackable. To determine if a fixed agent (xi , o) has
minimum vertex separator Si of size q +1, we solve a 0−1 maximum flow
problem. We consider a graph hi (g(G)) = (VHi , EHi ), where |VHi | = 2|V|
and |EHi | ≤ |E 0 | + |V| − 1. Here, every v ∈ V\xi is converted to a pair
of nodes, vin and vout , where NvIin = NvI , NvOin = vout , NvIout = vin ,
NvOout = NvO . Moreover, all incoming edges to xi are removed. All edges
in EHi have capacity 1. (xi , o) has minimum vertex separator Si of size
at least q + 1 if and only if the maximum flow from source xi to sink o
in hi (g(G)) is at least q + 1. Using Dinic’s algorithm, (Dinic, 1970; Even
1
and Tarjan, 1975) this can be determined in O((2|V|) 2 (|E 0 | + |V| − 1))
time. Since, we must verify |Si | ≥ q + 1 for each of n agents, the
1
worst case computational complexity is O(n(2|V|) 2 (|E 0 | + |V| − 1)). This
4.3. Robust Structural Design of Control Systems 153

outperforms algebraic methods based on the matrix pencil (Trentelman

et al., 2012) and graphical methods based on Lemma 4.22 which verify
a system’s strong observability/left invertibility for fixed attack nodes.
This is a combinatorial task since there exists n+m


q possible attack
vectors.

4.3.6 Robust Structural Design

We now consider the problem of robustly designing the distributed con-
trol system. In this subsection, we assume that each agent has a self loop
and a subset of agents are observed using dedicated sensors. Moreover,
we assume that the defender wants to design the system so that it is
not discreetly attackable. The weaker case of solely preventing perfect
attacks has similar results. Before formulating our design problem, we
have the following Lemma, which follows trivially from Corollary 4.26.

Lemma 4.27. Suppose each agent has a self loop. Then, [A], [C] is not
discreetly attackable only if the out degree of each node xi ∈ X satisfies
|NxOi | ≥ q + 2.

Note that one outgoing edge is fixed to be a self loop by assumption.

The prior Lemma is obvious from Corollary 4.26 since each agent
requires q + 1 disjoint paths to o. Given this necessary requirement, we
can construct optimal robust networks which minimize communication.
Communication Design: We first assume the structure of C, or [C]
is given. Due to physical/cost constraints on communication, certain
agents can not communicate. This is encoded into [Ā], where [Ā]ji 6= 0
if and only if it is feasible for agent xi to send messages to agent xj .
Again, let Si be a minimal vertex separator between (xi , o) in g(G). We
have:

minimize kAk0 (4.88)

[A]

subject to |Si | ≥ q + 1, [A](i, i) 6= 0, i ∈ {1, . . . , n},

[Ā](u, v) = 0 =⇒ [A](u, v) = 0, u, v ∈ {1, . . . , n}.

The objective function represents the number of communication links in

our system. The first constraint ensures that our system is not discreetly
154 Secure Cyber-Physical Systems

attackable, while [Ā](u, v) = 0 =⇒ [A](u, v) = 0 ensures we only select

feasible links. Let, kAk∗0 denote the minimum value associated with
(4.88).
Theorem 4.28 (Weerakkody et al. (2017a)). Suppose Problem (4.88) is
feasible. Then in an optimal solution, ∀xi ∈ X , |NxOi | = q + 2. Therefore
kAk∗0 = (n − m)(q + 2) + m(q + 1) = (q + 2)n − m.
We note that (4.88) is solvable if ([Ā], [C]) is not discreetly attackable.
This can be checked by solving n maximum flow problems.
Remark 4.4. When Problem (4.88) is feasible, the optimal value of
Problem (4.88), (q + 2)n − m, is independent of the communication
constraints. Thus, a solution to Problem (4.88) with constraints defined
by [Ā] is also a solution to Problem (4.88) in the absence of constraints.
Theorem 4.29 (Weerakkody et al. (2017a) and Weerakkody et al. (2017b)).
Suppose Problem (4.88) is feasible. An optimal solution is found by
performing Algorithm 3 below.

Algorithm 3: DCS Network Design

Data: [Ā], [C]
Result: [A]
Let graph G be generated from [Ā], [C], [A] = [Ā];
for i = 1 : n do
if |NxOi | > q + 2 then
Solve maximum flow by using Dinic’s algorithm on
hi (g(G)) from source xi to sink o;
If xi is observed (or unobserved), keep q (or respectively
q + 1) neighbors in X through which ∃ a maximum flow.
Delete edges to other outgoing neighbors in X − xi ;
Update G, [A];
end
end
return [A];

Algorithm 3 can be performed using up to n maximum flow problems.

Note that Dinic’s algorithm to solve for the maximum flow has a worst
4.3. Robust Structural Design of Control Systems 155

1
case complexity of O(n(2|V|) 2 (|E 0 | + |V| − 1)) where V and E 0 are
associated with matrices [Ā], [C].
Joint Design: We now wish to minimize both sensing and communi-
cation costs through our choice of communication links and dedicated
sensor placement. Suppose the cost of a communication link is α1 ≥ 0
and the cost of a sensor is α2 ≥ 0. We wish to solve the following
problem:

minimize α1 kAk0 + α2 m (4.89)

[A],[C],m

subject to |Si | ≥ q + 1, [A](i, i) 6= 0, i ∈ {1, . . . , n},

[Ā](u, v) = 0 =⇒ [A](u, v) = 0, u, v ∈ {1, . . . , n},
C ∈ Rm×n , m ∈ {q + 1, · · · , n},
kCj k0 ≤ 1, j ∈ {1, . . . , n},
kC t k0 = 1, t ∈ {1, . . . , m}.

The last three constraints convey that [C] implements a set of m

dedicated sensors where m ∈ {q + 1, · · · , n}.

Theorem 4.30. Assume there exists a feasible solution to the joint

design problem (4.89). If α1 > α2 . Then every agent should be observed
(m = n). Alternatively, if α2 > α1 , then m = q ∗ , where q ∗ is the fewest
number of sensors for which Problem (4.88) is feasible.

Proof. kAk∗0 = (q + 2)n − m so the optimal value of (4.89) is (α2 −

α1 )m + α1 (q + 2)n. The result follows.

If α1 > α2 , then m = n and an optimal C ∗ satisfies Cij∗ 6= 0 ⇐⇒

i = j. Alternatively, when α2 > α1 we must first obtain a set of

∗
dedicated sensors [C ∗ ] with C ∗ ∈ Rq ×n which makes Problem (4.88)
feasible. Given C ∗ , Problem (4.89) can be solved using Problem (4.88).
We note that determining q ∗ is a combinatorial problem. Future work
aims to discover efficient solutions.

Example 4.8. We provide an illustrative example which shows how we

obtain the solution of Problem 4.88 based on Algorithm 3. Consider a
6-state system measured by 3 sensors, as depicted in Figure 4.4. The
graphical representation of the constraint matrix [Ā] is depicted in
156 Secure Cyber-Physical Systems

Figure 4.4(a) with self loops abstracted away. If [Ā](u, v) is not a fixed
zero, there exists an edge (xv , xu ). Suppose the goal is to design an
optimal communication network which prevents all perfect attacks when
q = 2 and all zero dynamics attacks when q = 1. Recalling Algorithm 3,
we start with the digraph associated with [Ā], and for each of the state
vertices xi we keep enough outgoing agent neighbors to ensure the size
of the minimum vertex separator between (xi , o) is q + 1 (to ensure the
system is not discreetly attackable) or q (to ensure the system is not
perfectly attackable). Figures 4.4(b), 4.4(c), and 4.4(d) show the results
of these iterations.

(a) Original graph, i.e., the (b) For x1 , x3 and x5 , delete

constraint matrix edge to x2 , x4 , and x6 , re-
spectively.

(c) For x2 , delete edge to x1 . (d) For x4 and x6 , delete

edge to x2 .

Figure 4.4: Process of Algorithm 3, starting with the constraint matrix in (a).
4.3. Robust Structural Design of Control Systems 157

n r k[A]k0 q m kAk∗0 Runtime (sec)

100 0.15 732 1 10 290 425.58
100 0.2 1080 1 10 290 776.31
100 0.3 2120 1 10 290 1766.97
100 0.2 1070 2 15 385 768.49
100 0.2 1038 3 20 480 682.13
50 0.2 232 1 10 140 25.11
150 0.2 2536 1 10 440 1.1430 × 104

Table 4.1: Runtime of Algorithm 3 for different n, q, r parameters

Example 4.9. Consider a multi-agent system with n agents, where the

agents are able to locally communicate with each other. The goal of
formation control could be organizing the agents according to certain
2-D formations. In the simulation, we generated an n × 2 matrix of
random variables under uniform distribution U [0, 1], which represent
the initial location of n agents.
Due to communication cost and noise, the communications between
agents are restricted to a certain radius r. As a result, we can compute
the constraint matrix [Ā] by enumerating the distance between every
pair of agents. More precisely, if the distance between the i-th agent
and j-th agent is less than r, then [Ā](i, j) = [Ā](i, j) 6= 0. Otherwise,
[Ā](i, j) = [Ā](i, j) = 0. Under such a constraint matrix, the goal is to
design a minimum communication network [A], which prevents zero
dynamics attacks (the defender does not know the initial state) from
q malicious sensors and actuators. To generate [C], we apply graph
clustering (Schaeffer, 2007) to the graph associated with [Ā] and group
the vertices into five clusters. In each cluster, we assign q + 1 sensors to
q arbitrary state vertices.
Note that the structural system ([Ā], [C]) constructed based on the
previous discussion is not necessarily left-invertible and strongly observ-
able with respect to q attacks. In other words, feasible solutions may not
exist for some of the randomly generated pairs ([Ā], [C]). The following
results only consider those ([Ā], [C]) pairs with a feasible solution. Table
4.1 lists the simulation results, where we consider different values of
158 Secure Cyber-Physical Systems

n, q and r, and record the runtime of Algorithm 3 using a Macbook Pro

running Ubuntu Linux with a 2.7 GHz Intel Core i5 processor. In order
to compute q + 1 essential neighbors of xi , corresponding to step 5-6 of
Algorithm 3, we incorporate the toolbox TOMLAB/CPLEX.

4.3.7 Further Reading

The results in this subsection are primary based on Weerakkody et al.
(2016a), Weerakkody et al. (2017b), and Weerakkody et al. (2017a). In
particular, Weerakkody et al. (2016a) and Weerakkody et al. (2017b)
develop graphical tools which allow a defender to determine if a system
is perfectly attackable and design systems that do not have viable perfect
attacks. Meanwhile, Weerakkody et al. (2017a) considers extensions to
prevent zero dynamics attacks.
Dion et al. (2003) provides a survey of generic properties related
to the structure of a linear control system. In particular, the authors
investigate properties of structured systems (defined by zero/nonzero
pattern in the system matrices) that are true for almost all values of
a systems free nonzero parameters. Properties such as observability
and controllability are linked to the structure of a system. Achieving
reliable structural observability through intelligent sensor placement
in the presence of sensor or link failures was the subject of Liu et al.
(2016).
In this subsection, we are interested in properties of left invertibility
and strong observability. Van der Woude (1991) provides work which
links the rank of a transfer function to the number of disjoint paths from
the inputs to the outputs, allowing a structural characterization of left
invertibility. Meanwhile, Van der Woude (1999) characterizes the generic
number of invariant zeros in a control system. The absence of invariant
zeros is equivalent to strong observability. These results were leveraged
by Boukhobza et al. (2007) and Boukhobza and Hamelin (2009) to
graphically characterize strong observability and left invertibility.
The structural notions of strong observability and left invertibil-
ity have been investigated from a security perspective. For example,
Pasqualetti et al. (2013) consider centralized control systems with a
fixed attack strategy (constant B a , Da ) and link structural left invert-
4.4. Active Detection of Attacks on CPS 159

ibility to the presence of undetectable attacks. Moreover, the authors

extend known results applied to nonsingular systems to general descrip-
tor systems. Additionally, Pasqualetti et al. (2015) provide sufficient
conditions for the presence of generically undetectable attacks when the
initial state is unknown.
Similar results have been achieved in the context of resilient dis-
tributed systems. For instance, Sundaram and Hadjicostis (2011) deter-
mine graphical conditions under which a set of agents can resiliently
compute an arbitrary function of their initial states. The ability to
compute an arbitrary function of the initial state is equivalent to each
agent being able to recover the entire initial state using observations
from neighbors. As seen in the previous subsection, this is linked to
strong observability. In addition, Pasqualetti et al. (2012) consider re-
silient consensus and use results related to strong observability and left
invertibility to graphically characterize attack and fault detectability
and identifiability in terms of the network’s connectivity.

4.4 Active Detection of Attacks on CPS

In subsection 4.2, we have seen that there are instances when passive
detection techniques provably fail, for instance during zero dynamics
attacks, false data injection attacks and replay attacks. One possible
method to counter stealthy attack scenarios is to consider offline robust
system design to reduce the space of stealthy attacks as considered in
subsection 4.3. In this subsection, we consider a different method for
attack detection. We consider how a defender can intelligently change
his policy online in order to better distinguish between normal and
attack scenarios. We refer to this process as active detection.
In active detection, the defender aims to authenticate the physical
dynamics of the system. In particular, the defender changes his policy
by introducing a physical secret into the system that is unknown to the
adversary. This physical secret enables a challenge response mechanism
in the CPS. Here, an attacker who targets a CPS must do so in a
manner that is consistent with a secret embedded in the dynamics
of the system to remain undetected. This serves as a response to the
defender’s challenge. Since the attacker is presumably unaware of the
160 Secure Cyber-Physical Systems

secret, active detection allows the operator to recognize the attacker’s

presence. The process of active detection is illustrated in Figure 4.5.
We remark that active detection can be combined with standard
cryptographic tools for authentication. The need for a physical means
for authentications stems from the fact that cryptographic tools alone
are often ineffective against physical attacks on a CPS. Together active
detection along with standard tools in cyber security will provide a
layered approach for authentication in a CPS, allowing a defender to
detect attacks in both the cyber and physical domains. As a sample,
in this subsection, we will consider two approaches for active detec-
tion: physical watermarking at the control input, and a moving target
approach for generating system dynamics.

uk Plant yk
A/ack Strategy
C
P

(independent
of secret)
Communica*on Network

uk uk
z 1

uk 1
S
x̂k State
Controller yk
Estimator

x
Failure Detector
Failure Detector
zk Ac$ve Detec$on:
Physical Secret is
introduced via
Recognizes a9ack Possible Detection Methods: Dynamics or Input
that doesn’t use Input: Physical Watermarking
secret Dynamics: Moving Target

Figure 4.5: Active Detection in Cyber-Physical Systems

4.4.1 Physical Watermarking

In this subsection, we introduce the approach of physical watermarking
for actively detecting attacks in control systems.

4.4. Active Detection of Attacks on CPS 161

Definition 4.16. A physical watermark is a secret noisy (random) con-

trol input inserted in addition to or in place of an intended control
input u∗k to authenticate the system.

The approach of physical watermarking has been shown to be effec-

tive in detecting replay attacks. For brevity in this subsection, we will
consider the use of watermarking against a replay attacker as seen in
subsection 4.2. We note that watermarking is in part motivated by the
use of nonces in cyber security described below.

Example 4.10. Let us consider the Needham Schroeder protocol in

Needham and Schroeder (1978), which establishes a session key between
2 users, Alice A and Bob B, by leveraging access to a trusted third
party, server S. In this protocol, Alice shares a session key KAB with
Bob by sending {KAB , A}KBS where KBS is Bob’s shared key with S
and {}K ∗ denotes encryption with key K ∗ . This message is vulnerable
to a replay attack. For instance, suppose Eve E recovers an old session
key KAB∗ . She can replay the message {K ∗ , A}
AB KBS to Bob. Bob now
∗
believes he shares key KAB with Alice, when he truly shares a key with
Eve. This lets Eve engage in a man in the middle attack.
To counter this attack, Alice receives a nonce or random number,
NB , from Bob encrypted with KBS . After communicating with S, Alice
sends {KAB , A, NB }KBS to Bob. The random nonce serves as a challenge
to Alice. By including the encrypted nonce in her response to Bob, Alice
proves that the message is fresh, and has not been replayed.

IID Gaussian Physical Watermarks

We consider the linear discrete stochastic CPS model used earlier

xk+1 = Axk + Buk + wk , yk = Cxk + vk , (4.90)

where xk ∈ Rn is the state vector, uk ∈ Rp is the control input, and

yk ∈ Rm is the sensor output. We assume the process noise wk ∈ Rn
and sensor noise vk ∈ Rm are IID and independent of each other with
wk ∼ N (0, Q) and vk ∼ N (0, R), We assume that R > 0, (A, C) is
1
detectable, and (A, Q 2 ) is stabilizable. Moreover, x0 is independent of
the noise processes and has distribution N (x̄0|−1 , Σ). A Kalman filter
162 Secure Cyber-Physical Systems

as described by (4.2), (4.3), (4.4) can be implemented and once more

we assume the apriori error covariance and Kalman gain have converged
to their steady state values P and K.
Physical watermarking was first introduced in Mo and Sinopoli
(2009) as an IID additive input sequence ∆uk ∼ N (0, J ) introduced on
top of an optimal control sequence u∗k . In particular, the overall control
input uk is given by
uk = u∗k + ∆uk . (4.91)
It is assumed that the adversary can not read the defender’s control
input uk or watermark ∆uk in this scenario. In particular, the control
input will serve as a secret in this approach for active detection. The
watermarks act as a cyber-physical nonce. Under normal conditions,
the watermark will be embedded in the sensor outputs due to the
system dynamics, a valid response to the defender’s challenge. However,
under replay attack, the measurements contain physical responses to
an earlier sequence of watermarks. Unable to detect recent watermarks
in the sensor outputs, the defender can not verify freshness of the
received sensor measurements. The process of physical watermarking
is pictorially illustrated in Figure 4.6. The first images represents the
CPS with an optimal control input. A watermark (the second image) is
embedded in the control input resulting in a noisy output (the third
image). The defender designs a detector that allows him to recognize
the presence of the watermark in the sensor outputs.
Because watermarking results in a sub-optimal input, this approach
can decay control performance. As an example we consider LQG con-
trol first discussed in example 4.1. Recall that the defender wishes to
minimize a cost function J given below
 
N 
1 X 
J = lim E xTk W xk + uTk U uk  , (4.92)
N →∞ 2N + 1
k=−N

where W, U are positive definite matrices. Here, we make the assumption

that the system has been running for a long period of time (since
k = −∞). Under the assumption that (A, B) is stabilizable and (A, C)
is detectable, the optimal LQG controller is the following fixed gain
4.4. Active Detection of Attacks on CPS 163

Physical Watermarking
a) Output with Optimal Input b) Add Watermark uk

c) Watermarked Output d) Perform Detection

Faulty/Attack Normal

Figure 4.6: Physical Watermarking in Cyber-Physical Systems

linear controller
u∗k = Lx̂k|k , (4.93)
where the control gain matrix L is defined as
 −1
L , − B T SB + U B T SA, (4.94)

and S satisfies the Riccati equation

 −1
S = AT SA + W − AT SB B T SB + U B T SA. (4.95)

Moreover, x̂k|k is the optimal a posteriori state estimate derived from

the Kalman filter (4.2). Given an LQG controller it can be shown that
optimal cost of control is given by

J ∗ = tr(SQ) + tr((AT SA − S + W )(P − KCP )), (4.96)

where P is the state state apriori state estimation error covariance and
K is the steady state Kalman gain. Due to watermarking, the LQG
cost increases to J = J ∗ + ∆J where from Mo and Sinopoli (2009), we
have
∆J = tr ((U + B T SB)J ) (4.97)
Note that cost of control increases linearly with the covariance of the
watermark. Thus, increasing the randomness of the watermark degrades
performance. However, this randomness also is the key mechanism
164 Secure Cyber-Physical Systems

for detection. For instance, consider the χ2 detector, introduced in

subsection 4.2. It can be shown that under normal operation denoted
by hypothesis H0
k h i
E zjT P̄ −1 zj = m(WD ).
X
(4.98)
j=k−WD +1

where P̄ , CP C T + R and zj is residue or innovation generated at

time j from the Kalman filter. However, under a replay attack, denoted
by hypothesis H1 , the expected value of the χ2 detector increases as
follows
k h i
E zjT P̄ −1 zj = m(WD ) + 2(WD ) tr(C T P̄ −1 CU),
X
lim
k→∞
j=k−WD +1
(4.99)
where U is the solution of the following Lyapunov equation: U −B T J B
=
AUAT and A , (A + BL)(I − KL). Note, here we implicitly assume
that A is Schur stable. As seen in example 4.6, this leads to asymptotic
stealthiness for the attacker in the absence of watermarking. Alter-
natively if A is unstable, the detection statistic diverges, leading to
deterministic detection even without a physical watermark. Given this
result, a convex semidefinite program can be formulated to design the
watermark. For instance, a defender can maximize the expected increase
in the detection statistic subject to a constraint on control performance
as seen below.
maximize tr(C T P̄ −1 CU) (4.100)
J 0

subject to U − B T J B = AUAT ,
tr ((U + B T SB)J ) ≤ δ,
Such a design balances the need to ensure that watermarking leads
to sufficient detectability without significantly affecting system perfor-
mance.

Stationary Gaussian Physical Watermarks

More recent work (Mo et al., 2015) has attempted to improve perfor-
mance by generalizing the design of the watermark to an additive zero
4.4. Active Detection of Attacks on CPS 165

mean stationary Gaussian watermark sequence ∆uk generated by the

following Hidden-Markov Model (HMM)
ξk+1 = Aω ξk + ψk , ∆uk = Ch ξk , (4.101)
where ψk ∈ Rnω is a sequence of i.i.d. zero-mean Gaussian random
variables with covariance Ψ. ξk ∈ Rnω is the hidden state. ω is a variable
that is used to characterize the autocovariance of ∆uk and its role will
be made clear in the ensuing framework. We let Γ(d) , E[∆uk ∆uTk+d ].
To make {∆uk } a stationary process, we assume that the covariance of
ξ0 is the solution of the following Lyapunov equation
Cov(ξ0 ) = Aω Cov(ξ0 )ATω + Ψ.
We assume that all the matrices are of proper dimensions and the
system matrices (A, B) is stabilizable and (A, C) is detectable.
Remark 4.5. Note that {∆uk } is completely described by its finite
dimensional distribution and hence the autocovariance function Γ. Even
though one may view the HMM assumption as a restriction on the space
of stationary Gaussian watermarks, we note that any autocovariance
function Γ can be approximated by an HMM with a sufficiently large
dimension of the hidden state. On the other hand, an HMM model with
low dimensionality is amenable for implementation.
Let us assume that Aω is strictly stable. In this case, the correlation
between the current watermark signal ∆k and the future watermark
signal ∆k0 decays to 0 exponentially as k 0 − k → ∞. We denote the
spectral radius of Aω as ρ(Aω ) and we assume ρ(Aω ) ≤ ρ̄, where ρ̄ < 1
is a design parameter. A value of ρ̄ close to 1 gives the system operator
more freedom to design the watermark signal, while a value of ρ̄ close
to 0 improves the freshness of the watermark signal by reducing the
correlation of ∆uk at different time steps. In this regard, we define the
feasible set G(ρ̄) as
G(ρ̄) = {Γ : Γ is generated by the HMM (4.101) with ρ(Aω ) ≤ ρ̄}.
(4.102)
As in the IID case, we can characterize the decay in control performance
due to a stationary Gaussian watermark. We explicitly characterize this
cost in terms of the autocovariance function Γ(d).
166 Secure Cyber-Physical Systems

T
Theorem 4.31 (Mo et al. (2015)). Let sym(X) = X+X
2 . The additional
LQG cost ∆J due to the additive watermark ∆uk is:
∞
( " #)
X
d
∆J = tr U Γ(0) + 2U sym L (A + BL) BΓ(1 + d)
d=0
h i
T
+ tr (W + L U L)Θ1 , (4.103)

where ∞
X h i
Θ1 , 2 sym (A + BL)d L1 (Γ(d)) − L1 (Γ(0)),
d=0
and L1 : Cp×p → Cn×n is a linear operator defined as
∞
X
L1 (X) = (A + BL)i BXB T ((A + BL)i )T
i=0
= (A + BL)L1 (X)(A + BL)T + BXB T . (4.104)

The defender can exploit the difference in distributions of the residue

zk under attack and normal operation. For instance, if u∗k = Lx̂k|k , then

zk ∼ N (0, P̄ ), H0 : no attack, (4.105)

zk ∼ N (µk , P̄ + Σ̄), H1 : attack, (4.106)

where µk , −C k−1 i=−∞ A

k−1−i B∆u and Σ̄ is a linear increasing func-
P
i
tion of the watermark covariance in the IID case, and a linear increasing
function of the autocovariances (in the positive semidefinite sense) in
the stationary case Mo et al. (2015). While a χ2 detector remains an
effective option, here we consider a Neyman Pearson detector, which
maximizes the probability of detection for a given probability of false
alarm. In the case the window size is 1, the optimal Neyman Pearson
detector is given below.

Theorem 4.32 (Mo et al. (2015)). The optimal Neyman-Pearson detec-

tor rejects H0 in favor of H1 if

zkT P̄ −1 zk − (zk − µk−1 )T (P̄ + Σ̄)−1 (zk − µk−1 ) ≥ τ. (4.107)

If this inequality holds otherwise, hypothesis H0 is accepted.

4.4. Active Detection of Attacks on CPS 167

A typical performance metric for a detector is the asymptotic de-

tection rate limk→∞ βk . We note that the asymptotic probability of
detection does not have a closed form solution. Therefore, we use the
Kullback-Leibler (KL) divergence to characterize detection performance.

Definition 4.17. KL divergence between two probability density func-

tions f0 and f1 is defined as:
Z ∞
f1 (x)
 
DKL (f1 ||f0 ) , f1 (x) log dx (4.108)
−∞ f0 (x)
The KL divergence provides a notion of distance between two given
distributions. The KL divergence is nonnegative, asymmetric, and is 0
if and only if two distributions are equal almost everywhere. Roughly
speaking, two distributions become easier to distinguish in a binary
hypothesis test as their KL divergence increases. In the next theorem,
we give the expected KL divergence between the distribution of the
residue zk under normal operation N0 and the distribution under attack
N1 .

Theorem 4.33 (Mo et al. (2015)). The asymptotic expected KL diver-

gence between N1 and N0 is
1
E[DKL (N1 kN0 )] = tr(Σ̄P̄ −1 ) − log det(I + Σ̄P̄ −1 ). (4.109)
2
Furthermore, the expected KL divergence satisfies the inequality
1 1 h i
tr(Σ̄P̄ −1 ) ≤ E[DKL (N1 kN0 )] ≤ tr(Σ̄P̄ −1 ) − log 1 + tr(Σ̄P̄ −1 ) ,
2 2
(4.110)
where the upper bound is tight if C is of rank 1.

As done in the IID case, to design a stationary Gaussian watermark,

one can formulate an optimization problem to select autocovariance
functions that maximize detection performance subject to a constraint
on control performance. It is worth noticing that the expected KL
divergence is a convex function of Σ̄. However, the upper and lower
bounds described in Theorem 4.33 are monotonically increasing in
tr(Σ̄P̄ −1 ) which is linear in Σ̄. In light of this result and the fact
tr(Σ̄P̄ −1 ) is concave, one mechanism to optimize detection performance
168 Secure Cyber-Physical Systems

is to maximize tr(Σ̄P̄ −1 ). This can be done while balancing the need to

limit the decay in control performance

maximize tr(Σ̄P̄ −1 )
Γ(d)∈G(ρ̄)

subject to ∆J ≤ δ, (4.111)

Both the objective function and ∆J are linear with respect to the
autocovariance functions Γ(d). Unfortunately, there are infinitely many
optimization variables Γ(d). Moreover, it is unclear how we can guarantee
that Γ(d) ∈ G(ρ̄). To address this we make the following additional
assumption.
Assumption 1: Γ̃(d) = ρ̄−|d| Γ(d) is an autocovariance function.
Intuitively if ρ(Aω ) is marginally smaller than ρ̄, Assumption 1
can be more easily satisfied. If ρ̄ = 1, the space is not constricted by
Assumption 1 and in fact one will be able to optimize over all stationary
Gaussian watermarks. Given this additional assumption, an equivalent
formulation can be obtained by converting the optimization problem to
the frequency domain using Bochner’s theorem. In particular, according
to Bochner’s theorem Γ(d) can be taken in the following form
Z 1
2
Γ(d) = exp(2πωdj)dν(ω) (4.112)
− 21

where ν(ω) is a positive semidefinite Hermitian measure of appropriate

size. An equivalent formulation is obtained below.

maximize tr(F2 (ω, H)C T P̄ −1 C) (4.113)

H∈Cp×p ,ω

subject to H ≥ 0, 0 ≤ ω ≤ 0.5
F1 (ω, H) ≤ δ,
(4.114)

Functions F1 and F2 are found in Mo et al. (2015). For fixed ω, the

optimization problem is a semidefinite programming problem in H,
which can be solved efficiently. Searching over ω allows the defender
to approximate a global solution. It can be shown there is a rank 1
4.4. Active Detection of Attacks on CPS 169

solution H ∗ = hhH where hH denotes the adjoint of h. Given the

optimal frequency ω ∗ , it can be shown that
" #
cos(2πω ∗ ) − sin(2πω ∗ ) √ h i
Aω = ∗ ∗ , Ch = 2 Re(h) Im(h) , (4.115)
sin(2πω ) cos(2πω )

with Ψ = (1 − ρ̄2 )I. Here, Re(h) and Im(h) refer to the the real and
imaginary potions of h respectively. Thus, the dimension of the hidden
state is always 2, and the resulting stationary Gaussian watermark is a
noisy sinusoid.

Example 4.11. We consider a control system with parameters

" # " #
1 1 0 h i
A= ,B= ,C= 1 0 . (4.116)
0 1 1

We let the cost matrices W = I and U = I, covariance matrices be

Q = 0.8I and R = I where I represents the identity matrix of proper
size for each case. As a result, the eigenvalues of A are -0.339 and -0.105
and hence A is stable. We consider a stationary Gaussian watermark
design. Designing a stationary Gaussian watermark requires solving a
semidefinite program for a set of frequencies sampled in 0 ≤ ω ≤ 0.5.
We take ρ̄ = 0.6 and a step size of 0.01 is chosen for this system, which
requires solving 51 semidefinite programs. Unless otherswise stated the
additional cost ∆J imposed by the watermark is 10 (which roughly
corresponds to 40 percent of the optimal cost J ∗ = 23.1) and the
probability of false alarm is 0.02.
Figure 4.7 illustrates the performance of the stationary watermark-
ing approach. We plot the asymptotic probability of detection vs the
probability of false alarm (when ∆J = 10) in Figure 4.7(a) while Figure
4.7(b) illustrates the relative improvement of the stationary approach
to the IID approach. We plot the asymptotic probability of detection
vs ∆J when α = 0.02 in Figure 4.7(c), illustrating the tradeoff between
detection and control performance. Here α corresponds to the proba-
bility of false alarm, which is constant in time due to the stationary
behavior of the state. Finally, in Figure 4.7(d), we plot the expected
time to detection as a function of the extra cost ∆J when α = 0.02. We
remark that the asymptotic probability of detection can be substantially
170 Secure Cyber-Physical Systems

improved by using a detector with a window size greater than 1. This

will however decrease time to detection.

(a) ROC Curve (b) Improvement Stationary vs IID

(c) Detection vs Cost (d) Time to Detection

Figure 4.7: Performance of Physical Watermarking

4.4.2 Moving Target

An alternative method for active detection in CPS is the moving target
approach. This approach is meant to counter an attacker with significant
disclosure and disruption resources. In particular, we consider a strong
adversary who can read and modify all input and sensor channels. If an
attacker has knowledge of the system dynamics he or she can arbitrarily
and stealthily perturb a system. To see this consider the following
example
4.4. Active Detection of Attacks on CPS 171

Example 4.12. Consider a control system under attack, where the

adversary can manipulate all sensors and actuators:

xk+1 = Axk + B(uk + uak ) + wk , yk = Cxk + vk + dak . (4.117)

The attacker can choose uak arbitrarily to perturb the system along the
controllable subspace (A, B). To perfectly avoid detection, the attacker
chooses dak as follows.

xak+1 = Axak + Buak , dak = −Cxak , xak = 0. (4.118)

Such an attack can be susceptible to a growing cancellation error, since

the attacker is subtracting his influence. Alternatively, the attacker can
use his or her knowledge of the inputs and simulate the output of the
system:

xak+1 = Axak +Buk +wka , dak = Cxak +vka −Cxk −vk , xa0 = x0 , (4.119)

where vka ∼ N (0, R), wka ∼ N (0, Q) are IID and independent processes.
Such an attack remains effective even if x0 is unknown to the attacker.
In both these cases, the attacker not only uses his or her access to the
channels, but also a detailed understanding of the system model. We
endeavor to limit the effectiveness of an adversary by removing their
system knowledge.

Our goal is to ensure the adversary is unaware of the full system

model. This can be challenging for several reasons. The dynamics of the
system may be well known for instance by physical laws. Alternatively,
an attacker can use his disclosure resource to learn the model through
passive observations.
As a result, we propose introducing extraneous states which are
causally affected by the ordinary states of the system. The extraneous
states are part of an authenticating subsystem, which has linear time-
varying dynamics, known to the system operator and hidden from
the adversary. The dynamics are designed so that an attacker who
impacts the original system will necessarily impact the authenticating
sybsystem. Moreover, the time varying dynamics ideally act as a moving
target, changing fast enough so the adversary does not have adequate
opportunity to identify the extraneous system. While essentially an
172 Secure Cyber-Physical Systems

attempt to prevent covert attacks, the moving target, by removing

an attacker’s model knowledge, can also defend against weaker zero
dynamics and false data injection attacks. The moving target approach
has been initially examined in Weerakkody and Sinopoli (2015).

Modeling the Moving Target

Mathematically, we introduce an authenticating subsystem with time
varying dynamics on top of the original system as follows:
" # " # " #
x̃k+1 x̃ w̃
= Ak k + Bk uk + k , (4.120)
xk+1 xk wk

where " # " #

A1,k A2,k Bk
Ak , , Bk , . (4.121)
0 A B
Moreover, we introduce additional sensors ỹk ∈ Rm̃ to measure the
extraneous states.
" # " # " # " #
ỹk x̃ ṽ Ck 0
= Ck k + k , Ck , . (4.122)
yk xk vk 0 C

The matrices are taken as IID random variables which are independent
of the sensor and process noise processes with distribution

A1,k , A2,k , Bk , Ck+1 ∼ fA1,k ,A2,k ,Bk ,Ck+1 (A1 , A2 , B, C). (4.123)

Furthermore, we also assume that

" # " #
w̃k ṽk
∼ N (0, Q) , ∼ N (0, R) , (4.124)
wk vk

where
" # " #
Q̃ Q̃12 R̃ R̃12
Q= > 0, R= > 0. (4.125)
Q̃T12 Q T
R̃12 R

Since the moving target system is linear and the noises remain
Gaussian, we can use a Kalman filter to still perform state estimation.
4.4. Active Detection of Attacks on CPS 173

" # " # " # " # " #

ˆk+1|k
x̃ ˆ
x̃ ˆ
x̃ ˆ
x̃ ỹ
= A k|k + Bk uk , k|k = (I − Kk Ck ) k|k−1 + Kk k ,
x̂k+1|k x̂k|k x̂k|k x̂k|k−1 yk
" # " #
ỹ ˆ
x̃
zk = k − Ck k|k−1 , Kk = Pk CkT (Ck Pk CkT + R)−1 , (4.126)
yk x̂k|k−1
Pk+1 = Ak Pk ATk + Q − Ak Pk CkT (Ck Pk CkT + R)−1 Ck Pk ATk

Here, Kk is the Kalman gain, Pk is the apriori state estimation error

ˆk+1|k , x̂k+1|k is the apriori state estimates and x̃
covariance, x̃ ˆk|k , x̂k|k
are the aposteriori state estimates.

Remark 4.6. While the system introduced above involves IID matrices
A1,k , A2,k , Bk , Ck+1 , the moving target design can still be effective in
other scenarios. For instance, the dynamics need not be linear as long
as the defender can accurately model the system. Moreover, the system
parameters can evolve at multiple time scales. In this case, the longer
the target remains in place, the easier it is for the adversary to identify
the system.

Remark 4.7. The defender must be able to introduce extraneous states

with time-varying dynamics correlated to the original state of the system.
The extraneous states are application dependent and are to be decided by
the system operator. Nonetheless, the system operator can leverage extra
products of the system, for instance the heat dissipated by a reaction or
process. The dynamics can be made time-varying by changing conditions
at the plant. Alternatively, the defender can introduce dynamics into
the system. For instance, the defender can introduce RLC circuits which
measure the states. Time varying dynamics can be incorporated by
including variable resistors or capacitors. By varying the components
of the circuit according to an IID distribution at each time step, the
defender can generate IID system matrices.

Remark 4.8. Unlike physical watermarking the moving target approach

does not need to result in suboptimal control performance. Specifically,
if we assume the defender does not care about controlling the extra
states, then no online performance has to be sacrificed. The cost of the
174 Secure Cyber-Physical Systems

moving target approach is likely primarily developmental. In particular,

a defender may have to expend financial resources along with man hours
to design, build, or purchase hardware which can be used to generate
an appropriate authenticating subsystem.

In the above formulation we assume that the defender is aware of

the real time system matrices although they are random. In general,
this information should not be sent over the network since doing so
amounts to the existence of a secure communication channel. The
secure communication channel could be leveraged to detect an attack
without considering a moving target approach. Alternatively, we can
generate pseudo random system matrices using a pseudo random number
generator (PRNG). In this case, the seed of the PRNG is known to
the defender and kept hidden from the attacker. The moving target
approach is illustrated in Figure 4.8

Original Original sensors

yk ✓ may not detect
Plant
a1ack
A1acker Performs
Stealthy A1ack Authenticating ỹk × Extra sensors
Subsystem detect a1ack
A1,k , A2,k , Bk , Ck+1

Pseudo-Random Random Seed, the

f (A1 , A2 , B, C) Number Random Seed root of trust,
Generator generates :me
varying dynamics

Figure 4.8: Moving Target for Active Detection in Cyber-Physical Systems

The moving target shares similarities with message authentication

codes or MACs used for authentication in cyber security. This is de-
scribed below.

Example 4.13. In cyber security, MACs can be used to verify the

integrity of a message. A message authentication code is computed by
computing a keyed pseudorandom function of the sender’s message.
The receiver obtains both the sender’s message and the MAC. The
receiver, using the secret key shared with the sender verifies that the
4.4. Active Detection of Attacks on CPS 175

MAC corresponds to the message. An attacker who attempts to modify

the message will almost certainly fail to generate an appropriate MAC
because he or she does not have access to the shared key.
We argue that the moving target approach allows us to introduce
a cyber physical MAC. In the context of the moving target approach,
suppose the message m corresponds to outputs yk while the MAC is ỹk .
The MAC ỹk is correlated to the message yk through the state xk−1 and
the input uk−1 . The key is the seed which determines the sequence of
system matrices. The defender uses knowledge of yk and the sequence
of system matrices to estimate ỹk . Under normal operation, ỹk and its
estimate ỹˆk closely agree, as seen by a residue based detector, and as a
result the MAC is verified.
On the other hand, suppose an adversary performs integrity attacks
using knowledge of (A, B, C, Q, R). The attacker could generate con-
vincing outputs yk , while biasing the states xk through a false data
injection or zero dynamics attack. At the same time, (s)he will also bias
the states x̃k and thus the MAC outputs ỹk if the time varying matrices
are properly chosen. Having no knowledge of the seed, the adversary
can not know the time varying matrices. Moreover, the time varying
dynamics act as a moving target, hindering system identification. As a
result, the attacker can not generate a convincing cyber-physical MAC
output ỹk .

Attacks on the Moving Target

In this subsection, we consider possible attacks on the moving target.
This will motivate an examination of bounds which characterize fun-
damental detectability with this approach. We assume the following
attacker capabilities:
1) The attacker can insert arbitrary inputs into the system and
can arbitrarily alter the sensor measurements. As a result, when under
attack, the system has dynamics given by
" # " # " #
x̃k+1 x̃ w̃
= Ak k + Bk (uk + uak ) + k , (4.127)
xk+1 xk wk
176 Secure Cyber-Physical Systems

" # " # " # " # " # " #

ỹka x̃k ṽk d˜ak ỹk d˜ak
= C k + + = + . (4.128)
yka xk vk dak yk dak
where uak is the attacker’s control input and d˜ak and dak are the biases
injected on the extraneous sensors and ordinary sensors respectively.
2) The attacker can read the true outputs of the system ỹk , yk and
the inputs being sent by the defender to the plant uk for all time k.
Note that this essentially corresponds to a man in the middle attack
between the plant and system operator so that he can manipulate and
read all communication channels arbitrarily.
3) The attacker has full knowledge of the system model M ,
{A, B, C, K, L, Q, R}. Moreover, the adversary knows the probability
density function (pdf) of random matrices A1,k , A2,k , Bk , Ck+1 . While
conservative, the adversary can obtain his knowledge of the system
model by observing the communication channels for an extended period
of time and performing system identification.
Note, we introduce some slight notational differences from the at-
tacks modeled in subsection 4.2. First, to emphasize the fact we can
consider an attacker who can modify all sensors, we omit the matrix Da .
Additionally, in order more easily distinguish the attacker’s information
and the defender’s information, we differentiate between the modified
outputs the defender receives ỹka , yka and the true outputs of the system
ỹk , yk .
Based on the above definitions we can define the private informa-
tion available to the attacker (IkA ) and defender (IkD ) and the public
information (IkP ) available to all parties at time k in the same order as
follows:

IkA , {ỹj , yj , d˜aj−1 , daj−1 , uaj−1 } ∀ j ≤ k, (4.129)

IkD , {A1,j−1 , A2,j−1 , Bj−1 , Cj } ∀ j, (4.130)
IkP , a
{M, f (A1 , A2 , B, C), uj−1 , ỹj−1 a
, yj−1 } ∀ j ≤ k. (4.131)

Thus the defender’s information is Ik , IkD ∪ IkP while the attacker’s

information is Ika , IkA ∪IkP . We now propose two main attack strategies.
Without loss of generality we assume any attack begins at k = 0.
4.4. Active Detection of Attacks on CPS 177

Attack Strategy 1 - Subtract Influence: In the first attack

strategy the attacker aims to estimate his influence on the control
system and subtract it. Define d¯ak , [d˜ak T dak T ]T . Observe that if

x̄ak+1 = Ak x̄ak + Bk uak , ∆ȳka = Ck x̄ak , (4.132)

with initial state x̄a0 = 0 and d¯ak = −∆ȳka , an attack is completely

stealthy. As the adversary does not know the time varying matrices, we
assume he computes an estimate of ∆ȳka and uses that to subtract his
influence on the sensor measurements. Thus, we would have

d¯ak = −E[∆ȳka |Ika ]. (4.133)

Observe that the adversary can exactly subtract his influence from
measurements yk due to his knowledge of the system model. However,
the adversary should be unable to completely subtract his bias from
the extraneous sensors ỹk .
Define ȳka , [ỹkaT ykaT ]T , x̄k , [x̃Tk xTk ]T , w̄k , [w̃kT wkT ]T , v̄k ,
[ṽk vkT ]T , and ȳk , [ỹkT ykT ]T . The adversary’s observations can be
T

formulated through the following linear time-varying system,

" # " #" # " #" # " #
x̄k+1 Ak 0 x̄k B Bk uk w̄
a = a + k a + k , (4.134)
x̄k+1 0 Ak x̄k 0 Bk uk 0
" #
h i x̄
k
ȳk = Ck 0 + v̄k . (4.135)
x̄ak

To estimate ∆ȳka at time k, assume the adversary has access to the

following distribution f (x̄k , x̄ak , Ck |Ika ). Then we have
Z Z Z
d¯ak = − Ck x̄ak f (x̄k , x̄ak , Ck |Ika )dx̄k dx̄ak dCk . (4.136)
x̄k x̄a
k
Ck

We show that the pdf can be recursively computed at each step. Letting
ζk+1 = {x̄k+1 , x̄ak+1 , Ck+1 } we have
a
f (ζk+1 |Ik+1 ) = f (ζk+1 |Ika , ȳka , ȳk+1 , d¯ak , uak , uk ),
= f (ζk+1 |Ika , ȳk+1 , uak , uk ),
f (ȳk+1 |Ika , ζk+1 )f (ζk+1 |Ika , uk , uak )
= . (4.137)
f (ȳk+1 |Ika , uk , uak )
178 Secure Cyber-Physical Systems

The second equality follows from the conditional independence of ζk+1

and ȳka , d¯ak given ȳk and uk . The last equality follows from Bayes rule
and the conditional independence of ȳk+1 and uk , uak given ζk+1 . We
note that this distribution can be theoretically computed given the
attacker’s information. That is, we know that
f (ȳk+1 |Ika , ζk+1 ) ∼ N (Ck+1 x̄k+1 , R) . (4.138)
Moreover, ζk+1 and ȳk+1 are deterministic functions of ζk , uk , uak and
random variables A1,k , A2,k , Bk , Ck+1 , w̄k , v̄k+1 , which are independent
of ζk given Ika . Thus, f (ζk+1 |Ik+1
a ) can be recursively computed from
a
f (ζk |Ik ).
Attack Strategy 2 - Estimate the Defender’s State Esti-
mate: In the next strategy, the adversary aims to track the system oper-
ator’s state estimate. Using the system operator’s state estimate, the ad-
versary attempts to generate stealthy outputs. Let x̄ ˆk = [x̃
ˆT T T
k|k−1 x̂k|k−1 ] .
The attacker’s observations and strategy can be formulated as follows
" # " #" # " #
x̄k+1 Ak 0 x̄k w̄
ˆ = ˆ + k ,
x̄k+1 0 Ak (I − Kk Ck ) x̄k 0
 
" # uk
Bk Bk 0  a
+ uk  , (4.139)
Bk 0 Ak Kk
ȳka
" #
h i x̄
k
ȳk = Ck 0 ˆk + v̄k , d¯ak = E[Ck x̄
ˆk |Ika ] − ȳk . (4.140)
x̄
ˆk , Ck , Pk }. The use of the
The attacker wishes to track ζk = {x̄k , x̄
preceding attack design is motivated by the following fact. Let Σ ≥ 0
be a positive semidefinite matrix.
ˆk |Ika ] − ȳk = arg min E[zkT Σzk |Ika ].
E[Ck x̄ (4.141)
d¯a
k

To determine d¯ak at time k assume the adversary knows f (ζk |Ika ).

As done before, the attacker can theoretically compute d¯ak by taking a
conditional expectation. Additionally, similar to (4.137) we have

a f (ȳk+1 |Ika , ζk+1 )f (ζk+1 |Ika , uk , uak , ȳka )

f (ζk+1 |Ik+1 )= . (4.142)
f (ȳk+1 |Ika , uk , uak , ȳka )
4.4. Active Detection of Attacks on CPS 179

Moreover, by similar analysis as in attack 1, we can demonstrate that

a ) can be recursively computed from f (ζ |I a ). The main
f (ζk+1 |Ik+1 k k
difference here is that the adversary must also estimate Pk .
In practice the proposed attacks are likely impossible to execute for
an adversary since it is numerically intractable to compute the necessary
distribution functions and expected values. This makes it difficult in
general to quantify the potential detectability of intelligent attackers.
As a result, we next aim to provide bounds on the attacker’s estimation
performance in terms of mean square error matrices.

Bounds on Attack Detectability

We now attempt to characterize lower bounds on the error matrices
associated with the states ζk defined in attack strategy 1 and 2. From
there, we can attempt to characterize how well the adversary can design
d¯ak to fool the bad data detector. We leverage conditional posterior
Cramer-Rao lower bounds for Bayesian sequences derived by Zuo et al.
(2011). The authors here make use of the Bayesian Cramer-Rao lower
bound or Van Trees bound derived in Van Trees (1968) which states
that for observations y and states ζ the mean squared error matrix is
bounded by the Fisher information as follows
h i
Ef (ζ,y) [ζ̂(y) − ζ][ζ̂(y) − ζ]T ≥ I −1 , (4.143)
where the Fisher information matrix I is given by
h i
I = Ef (ζ,y) −4ζζ logf (ζ, y) . (4.144)
Note that
4yx g(x, y) , Ox OTy g(x, y),
where O is the gradient operator. In Zuo et al. (2011), this result is
extended to nonlinear Bayesian sequences with dynamics given by
ζk+1 = Fk (ζk , ωk ), ȳk = Gk (ζk , v̄k ), (4.145)
where ωk and v̄k are independent process and sensor noise respectively.
In our case, we slightly adapt these results to account for the fact there
is feedback in our system so that
ζk+1 = Fk (ζk , ȳ1:k , ωk ), ȳk = Gk (ζk , v̄k ). (4.146)
180 Secure Cyber-Physical Systems

The inputs uk , uak and d¯ak are incorporated into the definition of Fk , while
uncertainty in the model (A1,k , A2,k , Bk , Ck+1 ) can be incorporated in
the process noise ωk . It can be shown that the following posterior
Cramer-Rao lower bound holds
h i
c
Efk+1 ē0:k+1 ēT0:k+1 |ȳ1:k ≥ I −1 (ζ0:k+1 |ȳ1:k ), (4.147)

where

ē0:k+1 , ζ0:k+1 − ζ̂0:k+1 (ȳk+1 |ȳ1:k ), (4.148)

c
fk+1 , f (ζ0:k+1 , ȳk+1 |ȳ1:k ), (4.149)
h i
ζ c
I(ζ0:k+1 |ȳ1:k ) , Efk+1
c −4ζ0:k+1
0:k+1
log fk+1 |ȳ1:k . (4.150)

Observe that (4.147) gives us an expected lower bound for the error
matrix associated with the entire state history ζ0:k+1 with knowledge of
measurements ȳ1:k . This expectation is taken over the state history as
well the measurement ȳk+1 so that ζ̂0:k+1 is a function of the measure-
ment ȳk+1 . Observe that unlike the traditional Cramer-Rao bound which
is limited to unbiased estimators, the Bayesian Cramer-Rao bound here
considers both biased and unbiased estimators ζ̂. While the lower bound
given here applies to the entire state history ζ0:k+1 , in practice we care
about estimating a lower bound on the current state ζk+1 . Nonetheless,
one can show that
h i
c
Efk+1 ēk+1 ēTk+1 |ȳ1:k ≥ I −1 (ζk+1 |ȳ1:k ), (4.151)

where I −1 (ζk+1 |ȳ1:k ) is the dim(ζk ) × dim(ζk ) lower right submatrix

of I −1 (ζ0:k+1 |ȳ1:k ). In practice, computing I −1 (ζk+1 |ȳ1:k ) directly from
the matrix I −1 (ζ0:k+1 |ȳ1:k ) is impractical since it requires computing
and taking the inverse of a Fisher information matrix which grows in
dimension at each time step. As a result, we would like a recursion to
compute I −1 (ζk+1 |ȳ1:k ). Due to the analysis in Zuo et al. (2011), we
have the following approximate recursion.
h i−1
I(ζk+1 |ȳ1:k ) = Dk22 − Dk21 Dk11 + IA (ζk |ȳ1:k ) Dk12 , (4.152)
4.4. Active Detection of Attacks on CPS 181

where
h i
Dk11 = Efk+1
c −4ζζkk log f (ζk+1 |ζk , ȳ1:k ) ,
h i
ζ
Dk12 = Efk+1
c −4ζk+1
k
log f (ζk+1 |ζk , ȳ1:k ) = (Dk21 )T ,
h i
ζ
Dk22 = Efk+1
c −4ζk+1
k+1
log f (ζk+1 |ζk , ȳ1:k )f (ȳk+1 |ζk+1 ) .

In addition,
 −1
IA (ζk |ȳ1:k ) = Ek22 − Ek21 Ek11 Ek12 , (4.153)

where
h i
ζ
Ek11 = Ef (ζ0:k |ȳ1:k ) −4ζ0:k−1
0:k−1
log f (ζ0:k |ȳ1:k ) ,
h i
Ek12 = Ef (ζ0:k |ȳ1:k ) −4ζζk0:k−1 log f (ζ0:k |ȳ1:k ) = (Ek21 )T ,
h i
Ek22 = Ef (ζ0:k |ȳ1:k ) −4ζζkk log f (ζ0:k |ȳ1:k ) .

We observe that it is still difficult to obtain matrices Ek11 , Ek12 , Ek21 , Ek22
so Zuo et al. (2011) introduces the following approximate recursion.
h i−1
T
IA (ζk |ȳ1:k ) ≈ Sk22 − Sk12 Sk11 + IA (ζk−1 |ȳ1:k−1 ) Sk12 , (4.154)

where
h i
ζ
Sk11 = Ef (ζ0:k |ȳ1:k ) −4ζk−1
k−1
log f (ζk |ζk−1 , ȳ1:k−1 ) ,
h i
Sk12 = Ef (ζ0:k |ȳ1:k ) −4ζζkk−1 log f (ζk |ζk−1 , ȳ1:k−1 ) ,
h i
Sk22 = Ef (ζ0:k |ȳ1:k ) −4ζζkk log f (ζk |ζk−1 , ȳ1:k−1 )f (ȳk |ζk ) .

We observe that in practice it may still be difficult to compute the exact

expectations because high dimensional integration is generally involved.
As alternatives, one can use particle filters (Arulampalam et al., 2002)
to approximate these expectations or approximate expressions for the
conditional posterior Cramer-Rao lower bound (Zheng et al., 2012).
The algorithm above enables the defender to compute an approxi-
mate lower bound on the mean square error matrix of the attacker’s
state ζk for a given set of inputs ua0:k , d¯a1:k and observation history ȳ1:k .
This allows us to obtain a lower bound on the expected value of the
squared 2-norm of our residue zk (defined in (4.126)) as follows.
182 Secure Cyber-Physical Systems

Theorem 4.34 (Weerakkody and Sinopoli (2015)). Consider the special

case that {Cj } is known to the adversary for all j ∈ Z. Suppose an
attacker attempts to estimate ζk = {x̄k , x̄ ˆk , Pk } as in attack strategy
ˆk (ȳk ) be an estimate of x̄
2. Let x̄ e ˆk as a function of ȳk given ȳ1:k−1 and
ˆ ˆ e
êk = x̄k − x̄k (ȳk ). Suppose a lower bound Zk on the error matrix of x̄ ˆk
is obtained so that h i
Efkc êk êTk ≥ Zk . (4.155)
Then we have h i
min
a
Ef ∗ zkT zk ≥ tr(Ck Zk CkT ), (4.156)
ȳk

where f ∗ = f (x̄ a , ua , d¯a , u

ˆk , ȳk |Ik−1 k−1 k−1 k−1 ).

In general, the adversary’s ability to estimate {ζk } is dependent on

the inputs {uak }, {d¯ak }. For instance, the more the adversary biases the
state away from its expected region of operation, the more challenging
it is to perform estimation. Thus, if the system operator wishes to
analyze how well an adversary can generate stealthy outputs, he must
consider a particular sequence of attack inputs uak , d¯ak . In practice, it
may be difficult to perform performance analysis when assuming Pk is
an unknown state. However, one can still approximate a lower bound
on the error matrix by assuming that the adversary has an oracle which
allows him to know Pk , Kk , I − Kk Ck .

Example 4.14. We test the moving target on the quadruple tank pro-
cess, a four state system Johansson (2000). The goal is to control the
water level of two of four tanks using two pumps. Two sensors mea-
sure water heights. We use an LQG controller with weights following
suggestions in Grebeck (1998). Q and R are created by generating a
matrix from a uniform distribution, multiplying it by its transpose, and
dividing by 100.
4 extra states and 2 extra outputs are added. The time varying ma-
trices A1,k , A2,k , Bk , Ck+1 are somewhat sparse (50% of entries nonzero).
The non-zero elements follow a multivariate Gaussian distribution with
means generated from U (−0.5, 0.5). The covariances of the nonzero pa-
rameters are created by generating a matrix from a uniform distribution,
multiplying it by its transpose, and dividing by 100.
4.4. Active Detection of Attacks on CPS 183

We consider an adversary who, starting at time 200 sec, adds a

constant input (in Volts) to the optimal LQG input and avoids detection
by trying to subtract his own influence (Weerakkody and Sinopoli, 2015)
from the measurements. First, in Figures 4.9(a), 4.9(c), we assume
the attacker knows the time varying system matrices. Secondly, we
assume the attacker does not know the realization of A1,k , A2,k , Bk , Ck+1 ,
but instead performs his attack by sampling the matrices from the
appropriate distribution (Figures 4.9(b), 4.9(d)). We plot a χ2 detector
statistic (window 10, α = 10−7 ) in Figure 4.9(a) and 4.9(b) and system
performance in Figure 4.9(c) and 4.9(d), both averaged over 1000 trials.
The asymptotic probability of detection vs false alarm is found in Figure
4.10. Here α corresponds to the probability of false alarm, which is
constant in time due to the stationary behavior of the state. Given full
knowledge of the system matrices, the attacker can significantly affect
water levels while remaining perfectly stealthy. However, with stochastic
knowledge of the system matrices, the attack is easily revealed, even for
small system perturbations and small α. In practice, the attack can be
improved by using the measurements ỹk to perform system identification.
We expect improvements to be marginal since the system changes at
each time step. Thus, it is important to analyze the effectiveness of an
attacker who performs machine learning in a scenario where the moving
target changes at a lower frequency.

4.4.3 Further Reading and Future Considerations

Physical watermarking and the moving target approach are not the
only active methods for detection that have been considered in the
literature. For example Teixeira et al. (2012b) considers how a one time
change in the system matrices, for instance through additional sensors
or changes in the plant can be used to change a systems zero dynamics.
Additionally, Miao et al. (2014) examine how a numerically efficient
coding scheme applied to sensor measurements can be used to detect
stealthy false data injection attacks on sensor measurements. Moreover,
Yuan and Mo (2015) examine how introducing suboptimal controller can
impede an attacker’s ability to perform system identification. Finally
Hoehn and Zhang (2016b) investigate how adding a nonlinear element
184 Secure Cyber-Physical Systems

(a) Full Knowledge: Det. Stat vs (b) Stochastic knowledge: Det. Stat
Time vs Time

(c) Mean absolute height deviation (d) Mean absolute height deviation
(cm): Full Knowledge (cm): Stochastic Knowledge

Figure 4.9: Performance of the Moving Target: Quadruple Tank

to a controller can detect replay attacks while Hoehn and Zhang (2016a)
propose using modulation matrices to remove an attacker’s system
knowledge and prevent zero dynamics and covert attacks.
The topic of physical watermarking has been investigated in detail
in the literature. Beyond the seminal work in Mo and Sinopoli (2009)
and extensions to stationary Gaussian watermarks Mo et al. (2015),
several other areas have been explored. For instance, in addition to the
Neyman Pearson Detector and χ2 detector, alternative detectors have
been explored. In particular, Satchidanandan and Kumar (2017) and
Hespanhol et al. (2017) provide asymptotic detectors, which guarantee
that zero average distortion power is injected into sensor measurements.
Moreover, Mo et al. (2014b) and Chabukswar et al. (2011) propose
the use of correlation detectors, which can be advantageous when the
4.4. Active Detection of Attacks on CPS 185

Figure 4.10: ROC Curve, Moving Target: Stochastic Attack

defender has model uncertainty. Alternative attack scenarios have been

considered, for instance situations where an attacker can read a subset
of inputs (Weerakkody et al., 2014).
In principle, there is no need to restrict watermarks to be Gaussian
or stationary. For example, results in Hosseini et al. (2016) suggest
that Gaussian watermarks could be optimal against Gaussian attackers
and vice versa. See also Rubio-Hernan et al. (2017) for non-stationary
and non-Gaussian watermark designs. Additional work (Ozel et al.,
2017; Weerakkody et al., 2017d) is also evaluating the effectiveness
of a watermark obtained by dropping the control input randomly ac-
cording to IID Bernoulli and Markovian strategies in combination with
a Gaussian additive input. Such a strategy illustrates that existing
uncertainty and randomness in a system can be utilized to obtain a
natural watermark. From an application perspective, implementations
are necessary to validate watermark designs. Ko et al. (2016) have
examined watermarking in vehicular systems while Rubio-Hernan et al.
(2017) have experimented on a SCADA testbed.
With respect to the moving target an alternative approach has been
previously considered. Here, the plant is modeled as a hybrid system
186 Secure Cyber-Physical Systems

which transitions across multiple modes:

xk+1 = Ak xk + Buk + wk , yk = Ck xk + vk . (4.157)

The authors provide design recommendations which enable an operator

to identify sensor attacks and perform resilient estimation. We refer the
reader to Weerakkody and Sinopoli (2016) for further details.
Future work should consider obtaining a unified approach for active
detection. We note that a defender is not restricted to use only one of
these strategies, especially since the proposed approaches vary in terms
of their effectiveness for detecting specific attacks. In this respect, it
is important to identify an adversarial model which characterizes the
resources and knowledge an operator anticipates an attacker could pro-
cure. Another direction to investigate is a game theoretic formulation of
the attack detection problem Miao et al. (2013) where an attacker wishes
to maximize performance loss while ensuring stealthiness, while the
defender wishes to limit the impact of a stealthy attacker through active
strategies. Analyzing potential equilibria can illustrate the effectiveness
of the proposed approaches against strategic, intelligent adversaries.

4.5 Resilient Estimation in CPS

The ability to detect attacks, a topic of major focus in subsections 4.3 and
4.4, is a necessary component to achieving resiliency in CPS. However,
the ultimate goal is to design systems and architectures which can
maintain performance in the presence of malicious behavior. To design
viable, resilient feedback control laws in the presence of an attacker, it
is important for the defender to have an understanding of the attacked
state. Indeed having a reliable estimate will allow a defender to better
understand the portions of a system that have been compromised and
design attack specific solutions to counter an adversary’s actions. The
nature of these attack specific solutions is system dependent and out of
scope for this monograph. Nonetheless, in order to begin the process
of achieving resilient system performance, this section investigates the
problem of resilient estimation.
To begin, we consider deterministic systems and consider integrity
attacks on sensors and actuators. In this scenario, we allow the errors
4.5. Resilient Estimation in CPS 187

injected to be of arbitrary magnitude and characterize resilience in

terms of the maximum number of attacked sensors/actuators that can
be tolerated while being able to achieve correct state estimation. We
later examine the problem of robust estimation in the presence of
sensor attacks in stochastic systems. Here, we will focus on fusion based
estimators which intelligently and robustly combine estimates generated
locally by individual or groups of sensors.

4.5.1 Resilient Estimation in Deterministic Systems

In this subsection we consider resilient estimation in deterministic
systems with results primarily from Fawzi et al. (2014). We consider
the following linear discrete deterministic CPS model

xk+1 = Axk + Buk + B a uak , yk = Cxk + Da dak , (4.158)

where xk ∈ Rn is the state vector, uk ∈ Rp is the control input of

the defender, and yk ∈ Rm is the sensor output. The attacker’s input
injections uak and dak result in an attack on some fixed subset of actuators
and sensors specified through matrices B a and Da , respectively. The
input is determined as a function of the past inputs and observations
as follows:
uk = Uk (A, B, C, u0:k−1 , y0:k ), (4.159)
where Uk is some deterministic function. We note that this model
is identical to the deterministic model we described in (4.21) in our
discussion on zero dynamics attacks. In the following, we first consider
attacks on sensors only before generalizing the problem to the case of
sensor and actuator attacks.

Sensor Attacks
Consider the resilient estimation problem under adversarial attacks at
the sensors only. In particular, the objective here is to estimate the
initial system state x0 from the corrupted observations y0:T −1 . In this
case, we assume B a = 0. Moreover, without loss of generality we are
able to assume uk = 0 as well since the impact of the control input on
188 Secure Cyber-Physical Systems

the sensor measurements is known exactly by the defender. We then

have
xk+1 = Axk , yk = Cxk + Da (K)dak , (4.160)
where K, the set of attacked sensors determines Da . Suppose an attacker
targets sensors K = {η1 , · · · , ηm∗ } ⊂ {1, · · · , m} where without loss of
generality we assume an attack set K is given in ascending order. Then
Da (K) can be obtained entrywise as follows Da (s, t) , 1s=ηi ,t=i . Note
that since the matrix A is deterministic, the problem of recovering the
state sequence x1:T −1 or the initial state x0 are equivalent. Therefore,
there is no loss of generality in focusing on the estimation of x0 instead
of the whole state sequence x0:T −1 . We also observe that this problem is
intimately related to error correction problem under dynamical system
operation, see Fawzi et al. (2014).
We say that the initial state can be recovered in the presence of q
attacks in T time steps if it is possible to exactly determine the initial
state x0 from an arbitrary attack on any q unknown sensors within T
time steps. We define this notion more formally as follows:

Definition 4.18. The initial state is recoverable under q sensor at-

tacks after T steps if for any x0 ∈ Rn and any inputs Da (K)da0 , · · · ,
Da (K)daT −1 ∈ Rm with |K| ≤ q, we can uniquely determine x0 from
y0:T −1

The definition above can be related to the notion of strong observ-

ability for linear systems. In particular, it can be shown that ability to
recover an initial state from a sequence of outputs in the presence of q
sensor attacks is equivalent to the strong observability of the system
(A, B = 0, C, Da (K)) for all |K| ≤ 2q.
Our pursuit of initial state recoverability is in essence an observability
analysis under sensor attacks. The classical observability analysis finds
a clear extension in the case of q sensor attacks. The following theorem
provides a necessary and sufficient condition for state recoverability
while under q sensor attacks.

Theorem 4.35 (Fawzi et al. (2014)). For any integer T > 0, the initial
state is recoverable under q sensor attacks after T steps if and only
if there does not exist x0 , x̄0 ∈ Rn with x0 6= x̄0 , and da0 , ..., daT −1 ∈
4.5. Resilient Estimation in CPS 189

Im(Da (K)) and d¯a0 , ..., d¯aT −1 ∈ Im(Da (K̄)) with |K| ≤ q, |K̄| ≤ q such
that CAk x0 + Da (K)dak = CAk x̄0 + Da (K̄)d¯ak for all k ∈ {0, ..., T − 1}.
Theorem 4.35 illustrates the close relationship between attack iden-
tifiability and state recovery. In particular, an initial state is not recover-
able if there exists an alternative initial state and a possible alternative
attack sequence on q or fewer sensors, which can lead to the observed
measurement sequence. The condition in Theorem 4.35 is also a stan-
dard extension for the property of observability to the case when q
attacks are present. In the following theorem, we provide a simpler
characterization of the recoverability of the initial state under q sensor
attacks.
Theorem 4.36 (Fawzi et al. (2014)). Define the support of a vector
v ∈ Rl as supp(v) = {i ∈ {1, · · · , l}, v(i) 6= 0}, where v(i) is the ith
entry of v. For any integer T > 0, the initial state is recoverable under
q sensor attacks after receiving measurements y0:T −1 if and only if for
any ν ∈ Rn − {0}, |supp(Cν) ∪ supp(CAν) ∪ . . . ∪ supp(CAT −1 ν)| > 2q.
Example 4.15. Let
   
0 3 1 −1 2 1 0 0 0 0
2 −3 0 0 1 0 1 0 0 0
   
   
A=
9 0 3 1 0  , C = 0
  0 1 0 0. (4.161)
0 0 1 1 0 0 0 0 1 0
 
 
1 2 −1 4 2 0 0 0 0 1
We want to know how many sensor attacks q we can withstand while
still recovering x0 given measurements y0:4 . From Theorem 4.36, we
know that 2q < m. As such, q ≤ 2. We next observe that for any
row C i , the system (A, C i ) is observable. Consequently, |supp(Cν) ∪
supp(CAν) ∪ . . . ∪ supp(CA4 ν)| = 5 for all ν =6 0. As such, q = 2. Up
to 2 sensor attacks can be tolerated with resilient state estimation.
We next consider the estimator that recovers the initial state x0
under the condition in Theorem 4.36.
min |K|
x̂0 ,K

s.t. yk − CAk x̂0 ∈ Im(Da (K))

for all k ∈ {0, 1, . . . , T − 1} (4.162)
190 Secure Cyber-Physical Systems

Note that the estimator finds the smallest set of attacked outputs K
which can explain the observed behavior, while simultaneously deter-
mining the initial state. If the initial state is correctly recovered, the
set of malicious sensors is easily determined by comparing the expected
outputs as derived by the true initial state to the received outputs.
Given the output sequence y0:T −1 , the preceding estimator is able to
correctly determine the initial state if the system can correct q sensor
attacks, and q or fewer sensor attacks have occurred.
Theorem 4.37 (Fawzi et al. (2014)). Whenever the initial state is recov-
erable under q sensor attacks after T time steps, solving problem (4.162)
recovers the initial state if q or fewer sensor attacks have occurred.
Despite recovering the initial state x0 , the estimator (4.162) intro-
duces high computational complexity since the optimization is combi-
natorial. A well known approach to tackle the computational burden of
this problem is to relax the zero norm by the one norm. This approach
finds its motivations from recent advances in compressive sensing. Let
us consider the `1 estimator that minimizes the sum of `r norm of errors
m
X
minimize kD̂i kr (4.163)
x̂0 ,dˆa
0:T −1 i=1
 
subject to D̂i = dˆa0 (i), · · · , dˆaT −1 (i)
yk = CAk x̂0 + dˆak , 0 ≤ k ≤ T − 1

Here, ||.||r denotes r norm and we assume r ≥ 1. Additionally, dˆak (i) is

the ith element of dˆak . The following result provides a sufficient condition
for this estimator to recover the initial state x0 (which will be equal to
x̂0 under q sensor attacks after T time steps).
Theorem 4.38 (Fawzi et al. (2014)). Let C i be the ith row of matrix
C. The estimator that solves (4.163) recovers the initial state x0 under
q sensor attacks after T time steps if for all z ∈ Rn − {0}
−1
TX
!1/r −1
TX
!1/r
X X
i k r i k r
|C A z| < |C A z| (4.164)
i∈K k=0 i∈Kc k=0

for all sets K ⊂ {1, . . . , m} with |K| = q and K ∪ Kc = {1, 2, . . . , m}.

4.5. Resilient Estimation in CPS 191

Here Kc denotes the complement of set K. We finally note that

local state feedback at the plant level can be performed to recover a
higher number of sensor attacks after T steps. More specifically, it can
be shown that if the pair (A, B) is controllable and B is a column
vector, then it is possible to choose a feedback gain K so that the initial
state can be recovered in the presence of a maximum number of attacks
(dm/2 − 1e malicious sensors) in the new system (A + BK, C). Moreover,
the eigenvalues of A + BK can be placed with a large degree of freedom.
For more details see Fawzi et al. (2014).

Sensor and Actuator Attacks

In this subsection, we now consider the problem of resilient state esti-
mation in the presence of both sensor and actuator attacks. Suppose an
attacker targets actuators Kua = {δ1 , · · · , δp∗ } ⊂ {1, · · · , p} and sensors
Kya = {η1 , · · · , ηm∗ } ⊂ {p + 1, · · · , p + m}. To write the corresponding
B a and Da uniquely as a function of their attack set we, without loss
of generality,
h assume all attacki sets are given in ascending order. Here,
B a (Kua ) = Bδ1 · · · Bδp∗ where Bδi is the δi th column of B. Da (Kya )
can be obtained entrywise as follows Da (s, t) , 1s=ηi −p,t=i .
To begin, we define the notion of state recovery in the presence of
sensor and actuator attacks.

Definition 4.19. A state sequence is recoverable under q sensor and

actuator attacks after T steps with a delay δ if for any x0 ∈ Rn and any
inputs Da (Ky )da0 , · · · , Da (Ky )daT −1 , B a (Ku )ua0 , · · · , B a (Ku )uaT −2 with
|Ku ∪ Ky | ≤ q, we can uniquely determine x0:T −1−δ from y0:T −1 .

First, we let

B a (K)ua0:k = {B a (K)ua0 , · · · , B a (K)uak },

Da (K)da0:k = {Da (K)da0 , · · · , Da (K)dak }.

Finally, we let yk (x0 , u0:k−1 , B a (Ku )ua0:k−1 , Da (Ky )da0:k ) be the output
yk due to the initial state, the defender’s inputs, and the attacker’s
inputs. We can now provide an equivalent characterization for being
able to recover a sequence of states.
192 Secure Cyber-Physical Systems

Lemma 4.39. A state sequence is recoverable under q sensor and actu-

ator attacks after T steps with a delay δ if and only if
yk x0 , 0, B a (Ku )ua0:k−1 , Da (Ky )da0:k = 0, 0 ≤ k ≤ T − 1

=⇒ x0 = 0, B a (Ku )ua0 = 0, · · · , B a (Ku )uaT −2−δ = 0

for all |Ku ∪ Ky | ≤ 2q.
The proof is similar to the proof of Theorem 4.10 and is left to the
reader. Note that Lemma 4.39 can be seen as a finite time interpretation
of Theorem 4.12, where it is shown that the entire sequence of states is
recoverable
 in the presence of q sensor and actuator
 attacks if and only
A, [B a (Ku ) 0n×|Ky | ], C, [0m×|Ku | Da (Ky )] is strongly observable for
all Ku ∪ Ky satisfying |Ku ∪ Ky | ≤ 2q.
The problem of state estimation here is closely related to attack
identification. Suppose B is full column rank. If x0:T −1−δ can be re-
covered, this means that we are equivalently aware of B a (Ku )ua0 , · · · ,
B a (Ku )uaT −δ−2 . Since B a consists of a subset of columns in B and B is
full column rank, a defender can determine the set of actuators attacked
in the period from k = 0 to k = T − δ − 2. Moreover, using knowledge of
the received measurement sequence y0:T −1−δ along with the recovered
state sequence, the defender can then determine the set of sensors that
have been attacked in the period of from k = 0 to k = T − δ − 1. The
ability to identify attacks allows the defender to implement an adaptive
policy. For instance, in the case of sensor attacks, the defender can
simply ignore measurements coming from sensors that are known to
be corrupt. With regards to corrupt actuators, the defender if possible
could try to remotely deactivate the attacked actuator. If the trusted
actuators and plant still form a controllable system while the trusted
sensors and plant form an observable system, robust control should still
be feasible.
As in the previous subsection, we now provide an optimization
problem, which allows the defender to recover the state sequence.
minimize |Ku | + |Ky | (4.165)
x̂0:T −1 ,Ku ,Ky

subject to yk − C x̂k ∈ Im(Da (Ky )), 0 ≤ k ≤ T − 1

x̂k+1 − Ax̂k − Buk ∈ Im(B a (Ku )), 0 ≤ k ≤ T − 2.
4.5. Resilient Estimation in CPS 193

The optimization problem can be interpreted as the defender’s

attempt to find the smallest set of attacked nodes which can explain
the observed behavior of y0:T −1 . If the state is recoverable after T steps
with delay δ under q attacks, and the attack policy indeed utilizes q
or fewer sensor and actuator nodes, then the preceding optimization
problem in fact allows the attacker to recover state x0:T −1−δ where
here xj = x̂j . Unfortunately, as in the sensor attack case, obtaining a
solution to the previous optimization problem can be computationally
difficult. As a result, we consider the following relaxed optimization
problem similar to the sensor attack only case.
m
X p
X
minimize kD̂i kr + λ kÛi kr (4.166)
x̂0:T −1 ,ûa ˆa
0:T −2 ,d0:T −1 i=1 i=1
 
subject to D̂i = dˆa0 (i), · · · , dˆaT −1 (i)
Ûi = ûa0 (i), · · · , ûaT −2 (i)

yk = C x̂k + dˆak , 0 ≤ k ≤ T − 1
x̂k+1 = Ax̂k + B(uk + ûak ), 0 ≤ k ≤ T − 2.

Again dˆak (i)) and ûak (i) are the ith entries of dˆak and ûak respectively. Thus,
kD̂i kr = 0 would imply the ith sensor is unattacked while kÛj kr = 0
would imply the jthe actuator has not been tampered with. λ controls
the relative weight of sensor and actuator attacks. The prior optimiza-
tion problem (4.166) is convex, introducing an `1 relaxation of the `0
optimization problem (4.165). While in general, this problem (4.166)
may not be able to recover and estimate and perform identification in
the presence of a maximum number of q sensor and actuator attacks as
defined by Proposition 4.39 (and obtained in theory by (4.165)), em-
pirical results obtained by Fawzi et al. (2014) indicate that for a small
enough number of malicious nodes, optimal estimation and identification
in the presence of malicious behavior can be performed.

4.5.2 Resilient Estimation in Stochastic Systems

In this subsection, we consider how the problem of resilient estima-
tion extends to stochastic systems. The relaxed optimization problems
presented in the deterministic case (Fawzi et al., 2014), in particular
194 Secure Cyber-Physical Systems

(4.166) and (4.163), can be modified for stochastic systems by relaxing

some of the equality constraints, for instance see Pajic et al. (2014).
The resulting estimator is essentially a moving horizon estimator. While
functional in the presence of both sensor and actuator attacks, the
moving horizon estimators suffer from some drawbacks, which include
ignoring past measurement histories and incurring some delay. In this
subsection, we will investigate alternative estimators in the presence of
sensor attacks. These estimators will be obtained as the fusion of many
local estimators, which use the entire measurement history. To begin,
we will present an estimator first proposed by Nakahira and Mo (2015).

Fusion of Local State Estimates

Before we continue, we consider the following notation and preliminaries.
The infinity norm of the finite sequence x0:T is defined as
kx0:T k∞ , max max |xik |.
0≤k≤T i

The infinity norm of an infinite sequence x = x0:∞ is defined as

kxk∞ , sup max |xik |.
k∈N i

We denote `n∞
as the space of infinite sequences of n-dimensional vectors
with bounded infinity norm. We will write `∞ when there is no confusion
on dimension of the vector. For any matrix A ∈ Rm×n . We denote its
induced norm as
kAxk∞ X
kAk∞ = sup = max |aij |.
x6=0 kxk∞ i
j

Next, consider the model

xk+1 = Axk + Bwk , x0 = 0 (4.167)
yk = Cxk + Dwk ,
with xk ∈ Rn , wk ∈ Rp and yk ∈ Rm . The matrices A, B, C, D are
real matrices with proper dimensions. Define the following function
H : N → Rm×p :
(
D k=0
H(k) , . (4.168)
CAk−1 B k ≥ 1
4.5. Resilient Estimation in CPS 195

Hence, y is the convolution of H(·) and w(·). For simplicity we denote

" #
A B
H, . (4.169)
C D
If A is strictly stable, then according to Dahleh and Diaz-Bobillo
(1994), H is a bounded operator on `∞ where its induced norm is given
as:
||y||∞
kHk1 , sup .
||w||∞ 6=0 ||w||∞

As a consequence, for any ||w||∞ ≤ ε, we have

||y||∞ ≤ ||H||1 ε. (4.170)
Moreover, if (A, C) is detectable, there exists a matrix K ∈ Rn×m ,
such that A + KC is strictly stable. We can construct the following
estimator
x̂k+1 = Ax̂k − K(yk − C x̂k ), (4.171)
with x̂0 = 0. The estimation error vector and residue vector associated
with this estimator is given by
ek , xk − x̂k , (4.172)
zk , yk − C x̂k . (4.173)
Using the bound in (4.170), it is easy to show that the estimation error
and residue sequence satisfy the following bounds.
Lemma 4.40 (Nakahira and Mo (2015)). Suppose ||w||∞ ≤ ε. For the
estimator defined in (4.171) with A + KC strictly stable, the following
inequalities hold:
||e||∞ ≤ ||E(K)||1 ε, (4.174)
||z||∞ ≤ ||G(K)||1 ε. (4.175)
where
" #
A + KC B + KD
E(K) = ,
I 0
" #
A + KC B + KD
G(K) = .
C D
196 Secure Cyber-Physical Systems

We are similarly able to provide a bound on the norm of a finite

sequence of the states in the following lemma:

Lemma 4.41 (Nakahira and Mo (2015)). Consider system (4.167) with a

detectable pair of (A, C) and ||w||∞ ≤ ε. If yk = 0 for all k = 0, 1, · · · , T ,
then

||x0:T ||∞ ≤ inf ||E(K)||1 ε (4.176)

K:A+KC strictly stable

We are now ready to consider the problem of estimation in the

presence of sensor attacks. To begin, we consider the following model.

xk+1 = Axk + Bwk , x0 = 0,

yk = Cxk + Dwk + Da (K)dak , (4.177)

As before, we let the state xk ∈ Rn . The vector yk = [yk1 , . . . , ykm ]T ∈ Rm

consists of m sensor measurements at time k, where we represent yki
as the measurement from sensor i. We denote the set of all sensors
as S , {1, . . . , m}. We use wk ∈ Rp to represent both process and
B
measurement noise. It is assumed that the matrix D is full row rank.
As a result, the noise wk is able to independently excite all measurements
and states. Finally Da (K)dak is the bias injected by the adversary. As
in the deterministic case, K, the set of attacked sensors determines Da .
Again an attacker targets sensors K = {η1 , · · · , ηm∗ } ⊂ S where without
loss of generality we assume an attack set K is given in ascending order.
Then Da (K) can be obtained entrywise as follows Da (s, t) , 1s=ηi ,t=i .
We make the following assumptions regarding the system noise and
attacker’s capabilities

A. The noise is `∞ bounded: kwk∞ ≤ ε.

B. The adversary can change the readings from at most q sensors. As a

result, |K| ≤ q. The set of “good” sensors is denoted as G , S\K.

We assume that the defender knows both ε and q. However, the set
of compromised sensors K is unknown. Note here that we will often
consider q as a design parameter of our system. In particular, we will aim
to design estimators that can tolerate up to q sensor attacks. Increasing
q will increase system resilience at the cost of computational complexity
4.5. Resilient Estimation in CPS 197

We define a causal state estimator to be an infinite sequence of

mappings h , (h0 , h1 , . . . ), where each hk maps past measurements
y0:k−1 to an estimate of the current state x̂k . As such, x̂k = hk (y0:k−1 ).
The estimation error ek for the given estimator is

ek , xk − x̂k , (4.178)
= xk − hk (y0:k−1 ). (4.179)

The sequence of estimation errors e depends on the system noise w, the

attacker’s input Da (K)da , and the choice of estimator h. Thus, ek can
be written as a function ek (w, Da (K)da , h). However, we will simply
write ek when there is no confusion. In this subsection, we will present
an estimate that can tolerate a worse case noise process w and attack
sequence Da da . To this end, let us define the worst-case estimation
performance as

κ(h) , sup kek (w, Da (K)da , h)k∞ . (4.180)

kwk∞ ≤ε, |K|≤q, k

Definition 4.20. An estimator h is a resilient estimator if κ(h) < ∞.

As a result, we claim that our estimator is resilient if for all feasi-

ble attack strategies and uncertainties, the estimation error remains
bounded. Unfortunately, there are scenarios where the construction
of such an estimator is impossible. Before arriving at this condition,
consider the following lemma.

Lemma 4.42 (Nakahira and Mo (2015)). There does not exist a resilient
estimator for system (4.177) if there exist infinite sequences x, x0 , w,
w0 , Da (K)da , Da (K0 )da0 , y and y 0 of proper dimension, which satisfy

1. x, Da (K)da , w, y satisfy (4.177), with kwk∞ ≤ ε and |K| ≤ q.

2. x0 , Da (K0 )da0 , w0 , y 0 satisfy

x0k+1 = Ax0k + Bwk0 , x00 = 0,

yk0 = Cx0k + Dwk0 + Da (K0 )dak 0 ,

with kw0 k∞ ≤ ε and |K0 | ≤ q.

3. yk = yk0 for all k and kx − x0 k∞ = ∞.

198 Secure Cyber-Physical Systems

To simplify notation, we introduce projection matrices below:

Definition 4.21. For any index set I = {i1 , . . . , il } ⊆ S, we define the

projection matrix P I to be
h iT
P I = e i1 . . . e il ∈ Rl×m ,

where ei is the ith canonical basis vector of Rm . We further define

the following vector ykI by selecting the entries of yk with indices in I.
Specifically,

ykI , P I yk .

Similarly, we define matrices C I , P I C, DI , P I D.

We are now able to characterize the existence of a resilient estimator.

Theorem 4.43 (Nakahira and Mo (2015)). Consider system (4.177) with

assumptions A and B. There does not exist a resilient estimator if
(A, C J ) is not detectable for some set J ⊂ S with cardinality |J | =
m − 2q.

We remark that moving horizon estimators such as those imple-

mented by Fawzi et al. (2014) (presented in the deterministic case) and
Pajic et al. (2014) require that a system remain observable after remov-
ing 2q sensors. By leveraging an infinite sequence of measurements in
the estimator to be proposed, it can be shown that it is indeed feasible
to construct a resilient estimator h if (A, C J ) is detectable for all sets
of m − 2q sensors J . By Theorem 4.43, we know that the following
assumption is necessary for the existence of a resilient estimator and
we assume that it holds throughout the rest of this subsection.

C. (A, C J ) is detectable for any J ⊂ S satisfying |J | = m − 2q

We propose the following estimator design for system (4.177) under

Assumptions A, B and C. Let I = {i1 , · · · , im−q } ⊂ S be an index set
with cardinality m − q. Denote the collection of all such index sets as

L , {I ⊂ S : |I| = m − q}.
4.5. Resilient Estimation in CPS 199

For any I ∈ L, Assumption C implies the existence of K(I) such that

A + K(I)C I is strictly stable. Note that we use parentheses for K(I)
in order to differentiate it from the projection, which is written using
a superscript. We can construct a stable local estimator, which only
uses the truncated measurement ykI to compute the state estimate. The
estimate at time k using the set of sensors in I is denoted by x̂k (I),
which has dynamics

x̂k+1 (I) = Ax̂k (I) − K(I)(ykI − C I x̂k (I)), (4.181)

with initial condition x̂0 (I) = 0.

Each local estimator has its own estimation error and residue given
below

ek (I) , xk − x̂k (I), (4.182)

zk (I) , ykI I
− C x̂k (I). (4.183)

These local errors and residues are associated with the following linear
operators
" #
A + K(I)C I B + K(I)DI
EI (K(I)) , , (4.184)
I 0
" #
A + K(I)C I B + K(I)DI
GI (K(I)) , . (4.185)
CI DI

By Lemma 4.40, we know that if K ∩ I = ∅ then the following

inequality holds:

||z(I)||∞ ≤ ||GI (K(I))||1 ε. (4.186)

Thus in order to recognize if a set I has a compromised sensor, we

assign each estimator a local detector, which determines whether the
following inequality holds at each time k:

||z0:k (I)||∞ ≤ ||GI (K(I))||1 ε. (4.187)

If (4.187) fails to hold, then we know the set I contains at least 1

compromised sensor and hence the local estimate x̂k (I) is corrupted
by the adversary. Alternatively, we call x̂k (I) a valid local estimate
200 Secure Cyber-Physical Systems

from time 0 to k if (4.187) holds at time k. Note that the sensors from
a valid estimator may still be under attack. However, the resulting
outputs could still have been generated by a valid noise sequence. One
can potentially design better local detectors to check if there exist
compromised sensors in the index set I. However, the local detector
based on (4.187) is sufficient for us to design a resilient estimator.
We further define the set Lk as
Lk , {I ∈ S : (4.187) holds at time k}. (4.188)
We will then fuse all the valid local estimates x̂k (I) at time k to
generate the state estimate x̂k . As a primary goal is to minimize the
infinite norm of the estimation error, we will use the following equation
to compute each entry of x̂k :
1 
x̂ik = min x̂ik (I) + max x̂ik (I) . (4.189)
2 I∈Lk I∈Lk

Here xik is the ith entry of xk . By fusing estimates where an adversary

has potentially introduced only bounded perturbations, we arrive at a
resilient estimator. Note that by identifying sets containing malicious
sensors, the defender is able to narrow down and in some cases able to
exactly identify a set of culprit sensors.
We next assert that under the provided system conditions the
proposed estimator is resilient, in particular by demonstrating the
existence of a finite upper bound κ(h).
Theorem 4.44 (Nakahira and Mo (2015)). Under Assumptions A,B and
C, the fusion state estimator (4.189) is a resilient estimator for system
(4.177). Furthermore, κ(h) is upper bounded by:
 1 
max kEI (K(I))k1 + α(I ∩ J )[βI (K(I)) + βJ (K(J ))]  (4.190)
I,J ∈L 2
where α(J ) is defined as
" h i #
A + KC J I K
α(J ) , inf ,


K:A+KC J strictly stable I 0
1
and βI (K(I)) is defined as
βI (K(I)) , max(kK(I)k∞ , 1) kGI (K(I))k1 .
4.5. Resilient Estimation in CPS 201

Remark 4.9. Observe that when I only contains secure sensors, then
minimizing the infinite norm of the local estimation error ek (I) is
equivalent to minimizing kEI (K(I))k1 . The second term on the RHS of
(4.190) exists since the estimator does not know which local estimate
can be trusted at the beginning.
Since we are able to achieve a resilient estimator when every sub-
system with m − 2q sensors is detectable, we arrive at the following
result.
Corollary 4.45 (Nakahira and Mo (2015)). A necessary and sufficient
condition for the existence of a resilient estimator is that (A, C J ) is
detectable for any index set J ⊂ S with cardinality m − 2q.
Example 4.16. We illustrate the estimator and its performance by
means of a numerical example. We take the matrices that define the
system as:
 
1 h i h i
A = 1, C = 1 , B = 1 0 0 0 , D = 0 I .
 

1
Additionally, we assume that ε = 1 and q = 1. First, we design a
linear estimator described in (4.171) for nonadversarial scenarios.
h Wei
utilize an estimator with the following symmetric gain K = θ θ θ ,
where θ ∈ R. The `1 norm of E(K) is (1 − |1 + 3θ|)−1 (1 + |3θ|) where
|1+3θ| < 1 in order to ensure the stability of the estimator. The optimal
θ, which minimizes kE(K)k1 , is given by θ = −1/3. As a result, when
no attacker is present
1h i
x̂k+1 = x̂k + 1 1 1 (yk − C x̂k ). (4.191)
3
In an adversarial scenario we additionally need the gains K(1, 2),
K(2, 3), K(3, 1). Once again, we consider the following symmetric gains:
h i
K(1, 2) = K(2, 3) = K(3, 1) = µ µ .
We observe that α(J ) = 2 for J = {1}, {2}, {3} and
1 + |2µ| 1 + |2µ|
kEI (K(I))k1 = , kGI (K(I))k1 = 1 + .
1 − |1 + 2µ| 1 − |1 + 2µ|
202 Secure Cyber-Physical Systems

Figure 4.11: Estimation Error for Estimator (4.191).

where µ ∈ (−1, 0) to ensure the stability of the local estimator. Hence,

we have

βI (K(I)) = 1 + (1 − |1 + 2µ|)−1 (1 + |2µ|),

and the upper bound on the worst-case estimation error is

κ(h) ≤ 2 + 3(1 − |1 + 2µ|)−1 (1 + |2µ|). (4.192)

The value of µ that minimizes the right hand side of (4.192) is µ∗ = −0.5
and the corresponding upper bound is 8.
We compare the nonadversarial estimator (4.191) and the resilient
estimator. Here, wk is generated from a uniform distribution on the
set kwk k∞ ≤ 1. The attacker targets only the first sensor with a bias
h iT
increasing in time: Da dak = 0.5k 0 0 . The trajectories for the
estimation error of the estimator (4.191) and the resilient estimator are
plotted in Figure 4.11 and Figure 4.12, respectively.

In Figure 4.12, we observe the estimation error of the resilient

estimator. The blue, teal and black lines correspond to the estimation
errors for 3 local estimators. The red line is the estimation error after
4.5. Resilient Estimation in CPS 203

Figure 4.12: Estimation Error for Resilient Estimator.

fusion. The teal line and black line terminate at time 4 and time 12,
respectively. These times correspond to the instants in which detection
of a violation of (4.187) occurs by local detectors. As a consequence,
sensor 1 is identified as a sensor under attack. Note that the error
for the estimator (4.191) grows linearly and hence, it is unbounded.
On the other hand, our resilient estimator detects that the index sets
{1, 2} and {1, 3} contain the compromised sensor and hence discard the
corresponding local estimates. As a result, the estimation error remains
bounded.

4.5.3 Resilient Estimation via Sensor Fusion

The estimator (4.189) is able to achieve resilience in the scenario where
the system is detectable for every set of m − 2q sensors. However, the
defender is required to potentially monitor a large family of estimators,
(every subset of m − q sensors). Moreover, performance is likely sacri-
ficed during normal operation. In this subsection we consider another
estimator, based on results by Liu et al. (2017), which combines efficient
sensor fusion based techniques along with convex optimization in order
204 Secure Cyber-Physical Systems

to obtain an optimal Kalman estimate under normal operation with

certain probability. Moreover, one can determine sufficient conditions
that allow the estimator to remain stable in the presence of q sensor
attacks.
Here, we consider a slightly different model
xk+1 = Axk + Bwk , yk = Cxk + vk + dk (4.193)
Here wk ∼ N (0, Q) is Gaussian process noise in Rp and vk ∼ N (0, R)
is IID Gaussian sensor noise (independent of the process noise) with
R > 0. In addition, the initial state x0 ∼ N (0, Σ) is independent of
the process and sensor. Moreover, (A, C) is assumed to be observable.
We make the additional assumption that A is invertible. For notational
simplicity we write the attacker’s input as a vector dk where we omit
the matrix Da and the superscript a from previous subsections. Each
sensor can be analyzed independently as follows.
yki = C i xk + vki + dik , i = 1, · · · , m (4.194)
The superscript i denotes the ith entry of a vector or ith row of a matrix.
We assume the system has been running for a long time and as a result
the optimal Kalman filter has converged. The resulting filter is given by
x̂k+1|k+1 = (A − KCA)x̂k|k + Kyk+1 , K = P C T (CP C T + R)−1
P = AP AT + BQB T − AP C T (CP C T + R)−1 CP AT .
The Kalman filter can be further decomposed as
m
X
i
x̂k+1|k+1 = (A − KCA)x̂k|k + Ki yk+1 (4.195)
i=1

where Ki is the ith column of K. We wish rewrite the Kalman filter as

a linear combination of state estimates generated by individual sensors.
To do this we make the Assumption that (A − KCA) and A share
no eigenvalues. Even if this is not the case for the Kalman filter, the
assumption that A is invertible implies that (A, CA) is observable.
Consequently the eigenvalues of (A − KCA) can be placed arbitrary.
Suppose the Jordan decomposition of (A − KCA) = VJ V −1 . We
assume (A − KCA) has s independent eigenvectors. The lth eigenvector
is associated with an eigenvalue λl and a Jordan block of size nl × nl .
4.5. Resilient Estimation in CPS 205

We wish to decompose the Kalman estimate as follows

m
X
x̂k|k = Fi ζki , (4.196)
i=1
i
ζk+1 = J ζki + 1n yki (4.197)

where ζki is an estimate generated by sensor i, which ideally is stable

along the sensor’s observable subspace and 1n ∈ Rn×1 is a vector of
ones. Fi can be computed by solving

Fi 1n = Ki , V −1 Fi J = J V −1 Fi . (4.198)

Beyond being able to decompose the Kalman filter as a fusion of

m estimates generated by each sensor, it can be shown that the
proposed estimator provides a stable estimate of Gi xk where Gi =
h iH
GH H H
1,i G2,i · · · Gs,i ∈ Cn×n for i = 1, 2, · · · , m, and

C A(A − λl I)−nl + · · · + C i A(A − λl I)−1

 i 
 .. 

Gl,i = 
 . 
 (4.199)
i −2 i
 C A(A − λl I) + C A(A − λl I) −1 

C i A(A − λl I)−1 nl ×n

for l = 1, 2, · · · , s. Here, operator H is used to denote the adjoint. The

error ik , Gi xk − ζ̂ki satisfies

ik+1 = J ik + (Gi B − 1n C i B)wk − 1n vk+1

i
− 1n dik+1 . (4.200)

Thus, this error is stable since (A − KCA) is stable. Note the estimation
error ik can be decomposed as the sum of an error φik due to system
noise and ψki due to an attack.

φik+1 = J φik + (Gi B − 1n C i B)wk − 1n vk+1

i
(4.201)
ϕik+1 = J ϕik − 1n dik+1 (4.202)

We concatenate ik , φik , ϕik to obtain ˜k , φ̃k , ϕ̃k respectively. Let W
f ,
limk→∞ Cov(ϕk ) which can be obtained by solving a Lyapunov equation.
The optimal Kalman estimate x̂k|k can be additionally found by solving
the following least squares problem
206 Secure Cyber-Physical Systems

1 H f−1
minimize ˇ W ˇk (4.203)
x̌k ,ˇ
k 2 k
   
ζ̂ 1 G1
 .k   . 
subject to  .  =  .  x̌k − ˇk .
 .   . 
ζ̂km Gm

We observe that x̂k|k is equal to the minimizer x̌k of (4.203). This

problem can be interpreted as the problem of finding an estimate x̌k
that minimizes a weighted least square of the error associated with
local estimates ζ̂ki . The weighting matrix is related to the covariance
of the error of the local estimates. Unfortunately this estimator is not
resilient as an adversary can potentially change the Kalman estimate
arbitrarily using his injected bias dik . In order to address this challenge,
an `1 penalty is added.

1 H f−1
minimize φ̌ W φ̌k + γkϕ̌k k1 (4.204)
x̌sk ,φ̌k ,ϕ̌k 2 k
subject to ζ̂ki = Gi x̌sk − φ̌ik − ϕ̌ik , ∀i ∈ {1, · · · , m},

Under normal operation x̌sk = x̂k|k if the following equality holds.

   

G1 i

f I −  ..  F1
−1   h
W Fm  ≤ γ.
 .  ...  ˜k (4.205)



Gm


∞

After a long period of time Cov(˜ k ) = Wf and the probability the

`1 estimator coincides with the Kalman estimate can be computed.
Increasing γ will increase this probability. Alternatively, under attack,
one can characterize resiliency.
Note that the proposed estimator could provide a basis to identify
malicious sensors. In particular, a large value for ϕ̌ik , (since it represents
the portion of an error due to the attack vector), could be indicative of
an attack. Note that the true value this error ϕik , is nonzero if and only
if sensor i has been compromised
4.5. Resilient Estimation in CPS 207

Theorem 4.46. A secure state estimate x̌sk is stable against q or fewer

sensor attacks if the following inequality holds for all u 6= 0:
X X
kGi uk1 < kGi uk1 , ∀I ∈ C,
i∈I i∈I c

where C = {I ⊂ {1, · · · , m} s.t. |I| = q}.

The proof is omitted here, but a similar result and procedures can
be found in Han et al. (2019)

Example 4.17. We demonstrate the proposed secure fusion sensor esti-

mator via a numerical example. We assume the following parameters
for our system:

A = diag{1, 1, −2}, B = I3 , Q = I3 ,
   
C1 1 0 0
C2   1 1 −1 
   
   
C=
C3  =  1
  2  , R = I5 .
1 
C
  
 4  1 −1 −0.5

C5 −0.5 1 1

Here, Ik is the k × k, identity matrix. We consider two scenarios: i) all

sensors are benign; ii) the first sensor is under the attack and d1k = 100
for all k. In the simulation, we compute the empirical Mean Squared
Error (MSE) of the secure estimator for each scenario and for different
choices of γ. When all sensors are benign, the optimal Kalman estimator
has an MSE equal to 0.8317. As a result, we define the normalized MSE
as the MSE divided by 0.8317. Figure 4.13 illustrates the normalized
MSE of the proposed secure estimator versus γ. It can be seen that
when γ ≥ 2, the secure estimator achieves roughly the same estimation
performance as the optimal Kalman estimator under normal operation.
On the other hand, if sensor 1 is malicious, then the MSE achieves a
minimum at around γ = 0.75.

4.5.4 Further Reading

The material in this subsection was based on the references Fawzi et al.
(2014), Nakahira and Mo (2015), and Liu et al. (2017). We observe that
208 Secure Cyber-Physical Systems

Figure 4.13: The normalized MSE of the secure estimator v.s. different choices of
γ.

Fawzi et al. (2014) presents a moving horizon approach to dynamic state

estimation. In particular a subset of measurements are use to perform
state estimation with some finite delay delay. In this case, the dynamic
estimation problem is essentially reduced to a static estimation problem.
Extensions to systems with modeling errors and noise are pursued by
Pajic et al. (2014) and Pajic et al. (2015). It is shown that the attacker
cannot destabilize the system by exploiting the difference between the
model used for state estimation and the real physical dynamics of the
system. Moreover the authors of Pajic et al. (2015) demonstrate how
their proposed attack resilient state estimators can be used for sound
attack detection and identification.
Chong et al. (2015) considers the problem of secure state estimation
in continuous systems. Here, the authors characterize the notion of
observability under attacks and propose two estimation schemes. One
estimator uses a finite window and observability Grammian to construct
the initial state while a second algorithm uses a switched observer.
In addition, Shoukry and Tabuada (2016) also consider estimation in
systems with arbitrary sparse sensor noise or attacks. The authors
4.5. Resilient Estimation in CPS 209

propose recursive estimators in the spirit of Luenberger observers and

utilize event triggered techniques to reduce complexity. To further
simplify the process of secure state estimation Shoukry et al. (2017)
propose the use Satisfiability Modulo Theory (SMT) solvers in secure
state estimation. The authors argue this approach is scalable while
having both memory and runtime efficiency. In deterministic systems
the solver provides a sound and complete solution to state estimation
and attack identification, while in stochastic systems bounds on the
estimation error are quantified.
Similar to Liu et al. (2017), the work of Lee et al. (2015) proposes
resilient estimation by constructing observers at each sensor, in this
case for a continuous time system. The state estimates are then intelli-
gently combined using error correction techniques. Yong et al. (2015)
propose a resilient state estimator for CPS with stochastic process and
measurement noise, actuator and sensor attacks, and switching attacks.
During a switching attack the system’s mode of operation or topology
is altered. A multiple model inference algorithms is used for estima-
tion. Asymptotic analysis and fundamental limitations to resilient state
estimation are provided.
We also refer the reader to Forti et al. (2016) for a joint attack
detection and resilient state estimation framework. In this work, three
different classes of adversarial attacks on CPS are considered: (i) signal
injection to corrupt sensor/actuator data, (ii) packet substitution and
(iii) extra packet injection. The problem considered here is that of
simultaneously detecting a signal attack while estimating the state
of the monitored system. A random set attack modeling approach is
undertaken by representing a signal attack’s presence/absence by means
of a Bernoulli random set. The joint attack detection-state estimation
problem is then formulated as a Bayesian problem. A hybrid Bernoulli
filter is used to estimate the probability of attack, the density function
of the state given that there is no signal attack, and a density function
of the state and attack input given that there is a signal attack.
 5
Privacy

In section 4, we consider mechanisms to detect and resiliently recover

from attacks on cyber-physical systems. Here, we considered attackers
who directly affected the operations of a CPS. However, we argue that
harmful adversarial behavior is not constrained to active actions against
the dynamics of a control system. Instead an attacker can be any
agent in a CPS who learns or propagates sensitive and/or confidential
information about users and other agents. To address this threat, this
section briefly considers the privacy aspect of cyber-physical Systems.
For large scale CPS such as power grid, there are usually multiple
actors involved in its operation, including citizen users. For example,
the successful operation of the power grid requires the participation
of generators, carriers and distributors, service providers, customers,
and operators (Fang et al., 2012). However, each participant may have
different interests and therefore do not want to reveal more information
than necessary to other players. For example, it is well known that
fine-grained power usage data, which can be used for demand response
purpose, also reveal information about the presence, absence and even
specific activities of a home’s occupants (Cortes et al., 2016). Addition-
ally, information collected in transportation CPS can reveal information

210
5.1. Data Privacy 211

about when and where a user travels, while data in a medical CPS can
include sensitive health information about specific patients. This section
aims to provide an initial discussion about how to strike a balance
between the utility achieved in algorithms that leverage big data in CPS
and the need to protect critical information from adversarial actors.
For the rest of the section, we first consider the notion of data
privacy, with an emphasis on differential privacy (subsection 5.1). We
then consider an example of average consensus and design a differentially
private mechanism to guarantee the privacy of the initial conditions
of consensus (subsection 5.2). Due to the limitation of the differential
privacy mechanism, e.g., it cannot achieve the exact average consensus,
we consider a different concept, inference privacy, and propose an
inferentially private mechanism that can achieve exact average consensus.
Finally, we give a brief introduction on cryptography based privacy
(subsection 5.3) before providing additional references for further reading
(subsection 5.4).

5.1 Data Privacy

In this subsection, we consider data privacy mechanisms. Suppose that

there is a data set d ∈ Rn and we would like to publish a function of
the data set q(d). However, as directly publishing q(d) may cause a
privacy breach, we need to distort q(d) before releasing it. A privacy
mechanism is thus defined as a probabilistic mapping u = M (d), which
is characterized by a transition probability pu|d . In other words, given
d, u, which is a distorted version of q(d), follows the distribution pu|d .
Differential privacy is a notion to quantify the privacy guarantees
provided by the probabilistic mapping M . To define differential privacy,
we first need to establish an adjacency relation on the data set, which
characterizes how close two data sets are. In general, any symmetric
relation can be defined as an adjacency relation. For data sets, usually
the following definition of δ-adjacency is used:

Definition 5.1. Two data sets d and d0 are called δ-adjacent if they
differ at no more than one entry and the difference is no greater than δ.

kd − d0 k0 ≤ 1, kd − d0 k1 ≤ δ.
212 Privacy

Notice that other definitions of adjacency exist. For example, we

can define d and d0 as adjacent if kd − d0 kp ≤ δ. However, we only focus
on δ-adjacency in this section.
A mechanism M is -differential private if for any two adjacent data
sets d and d0 , and any measureable set R, the following inequality holds:

Pr(u = M (d) ∈ R) ≤ exp()Pr(u = M (d0 ) ∈ R).

It is worth noticing that other metrics for data privacy exist. For
example, -identifiability ensures that for all neighboring d and d0 ,
Pr(d|u)
≤ exp().
Pr(d0 |u)
-mutual information privacy guarantees that the mutual information
between d and u is less than . However, both identifiability and mutual
information privacy require the prior distribution of the data base, while
differential privacy does not. For more detailed discussions on other
privacy metrics, please refer to Sun and Tay (2017).
As an example of differential privacy, let us consider a data base
d ∈ Rn with n entries, such that entry i represents the monthly power
consumption of household i. We would like to publish the average power
consumption q(d) = 1T d/n.
We next define our privacy mechanism. To this end, we say a
random variable w follows a Laplacian distribution with parameter b if
its probability density function satisfies:
1 |w|
 
f (w) = exp − .
2b b
We will write w ∼ Lap(b) for short.
In order to ensure -differential privacy, we can choose our mechanism
to be

u = M (d) = q(d) + w = 1T d/n + w, (5.1)

5.1. Data Privacy 213

where w ∼ Lap(b), with b = δ/(n). This mechanism is -differentially

private, since for any two δ-adjacent data set d and d0
!
1 |u − 1T d/n|
Z
Pr(M (d) ∈ R) = exp − du
R 2b b
!Z !
|1T (d − d0 )| 1 |u − 1T d0 /n|
≤ exp exp − du
nb R 2b b
0
≤ exp()Pr(M (d ) ∈ R).

The probability density function of u is illustrated in Figure 5.1. It

is worth noticing that a smaller  will result in better privacy, since
M (d) and M (d0 ) will be hardly distinguishable. However, a smaller 
also increases the variance of the noise w, which deteriorates the data.
Hence, it is important to balance the trade-off between privacy and the
utility of the data by choosing an appropriate privacy budget .

p(u|d)

1T d 1T d0 u
n n

Figure 5.1: The probability density function of the output of the Laplacian mecha-
nism M (d) defined in (5.1).

One of the main advantages of -differential privacy is the resiliency

to post-processing of the data, which is proved by the following theorem:

Theorem 5.1 (Post-processing Cortes et al. (2016)). Suppose that a

mechanism M preserves -differential privacy, then for any measur-
ablefunction f , the functional composition f ◦ M also preserves -
differential privacy.
214 Privacy

Proof. For any set R, the event f ◦ M (d) ∈ R is equivalent to M (d) ∈

f −1 (R), where f −1 (R) is the preimage of R under f defined as

f −1 (R) = {x : f (x) ∈ R} .

As a result, for any adjacent d and d0 , we have

Pr(f ◦ M (d) ∈ R) = Pr(M (d) ∈ f −1 (R))

≤ exp()Pr(M (d0 ) ∈ f −1 (R))
= exp()Pr(f ◦ M (d0 ) ∈ R),

which finishes the proof.

However, the post-processing theorem can be too restrictive for

certain applications as it forbids “accurate” inference of any function of
d from u. In the next subsection, we consider the problem of average
consensus and show that the concept of differential privacy may be too
strong for certain applications.

5.2 Differential and Inference Privacy in Average Consensus

To illustrate the differential privacy mechanism and its limitations, we

use the example of average consensus, where a group of agents, each
with an initial value, want to compute the average of their initial values.
However, the agents would like to achieve consensus in a manner that
preserves privacy, in the sense that their initial values are not revealed
to the other agents. Privacy preserving consensus can be used in power
grids, where smart meters can locally fuse their power consumption
data and report only the aggregated data to the central controller.
For the rest of the subsection, we first give a brief introduction on the
consensus problem and then discuss the differential privacy approach. A
weaker notion of privacy, namely inference privacy, is introduced later
to deal with the limitations of differential privacy.

5.2.1 Average Consensus

We model a network composed of n agents as a graph G = (V, E). V =
{1, 2, . . . , n} is the set of vertices representing the agents. E ⊆ V × V is
5.2. Differential and Inference Privacy in Average Consensus 215

the set of edges. (i, j) ∈ E if and only if agent i and j can communicate
directly with each other. The neighborhood of agent i is defined as

N (i) , {j ∈ V : (i, j) ∈ E, j 6= i}.

Suppose that each agent has an initial scalar state x0,i . At each itera-
tion, agent i will communicate with its neighbors and update its state
according to the following equation:
X
xk+1,i = aii xk,i + aij xk,j . (5.2)
j∈N (i)

Define xk , [xk,1 , . . . , xk,n ]T ∈ Rn and A , [aij ] ∈ Rn×n . The update

equation (5.2) can be written in matrix form as

xk+1 = Axk . (5.3)

Furthermore, define the average vector to be

1T x 0
x̄ , 1.
n
We say that (asymptotic) consensus is achieved if xk,i − xk,j converges
to 0 for any pair of i, j, i.e., the difference between the states of any
agent i and j converges to 0. The accuracy of the convergence is defined
as
h i
lim E kxk − x̄k22 .
k→∞

The consensus is exact if xk − x̄ converges to 0. In other words,

the state of each agent converges to the exact average of the initial
condition.
If we arrange the eigenvalues of A in the decreasing order as
|λ1 | ≥ |λ2 | . . . ≥ |λn |. It is well known that the following conditions are
necessary and sufficient in order to achieve average consensus from any
initial condition x0 :

(A1) λ1 = 1 and |λi | < 1 for all i = 2, . . . , n.

(A2) A1 = 1, and 1T A = 1T , i.e., 1 is both a left and right eigenvector

of A.
216 Privacy

5.2.2 Differential Privacy in Average Consensus

We propose the following consensus algorithm in order to preserve the
privacy of each agent:

Algorithm 4: Privacy Preserving Average Consensus

1. At time k, each agent adds a random noise wk,i to its state xk,i .
Define the new state to be x+k,i , i.e.,

x+
k,i = xk,i + wk,i . (5.4)

2. Each agent then communicates with its neighbors and update

its state to the average value by the following time invariant
equation: X
xk+1,i = aii x+
k,i + aij x+
k,j . (5.5)
j∈N (i)

3. Advance the time to k + 1 and go to step 1.

Define
wk , [wk,1 , . . . , wk,n ]T ∈ Rn , x+ + + T n
k , [xk,1 , . . . , xk,n ] ∈ R . (5.6)
We can write (5.4) and (5.5) in matrix form as
xk+1 = Ax+
k = A(xk + wk ). (5.7)
Let us define the adjacency relationship on the initial conditions:
Definition 5.2. x0 and x00 are δ-adjacent if only one entry of x00 is
different from x0 , and the difference is no greater than δ.
Let us denote the infinite sequence x+ as (x+ + +
0 , x1 , . . .). Clearly x is
a function of both the noise sequence w = (w0 , w1 , . . .) and the initial
condition x0 . Hence, we could write it as x+ (x0 , w). To ensure the
-differential privacy, one would need to design the noise sequence w,
such that
Pr(x+ (x0 , w) ∈ R) ≤ exp()Pr(x+ (x00 , w) ∈ R), (5.8)
for any measurable set R and any δ-adjacent initial condition x0 and
x00 .
5.2. Differential and Inference Privacy in Average Consensus 217

A simple way to ensure -differential privacy is to choose w0,i to be

independent noise following a Laplacian distribution Lap(δ/), with all
the future wk,i = 0 for k ≥ 1. Essentially it is equivalent to perturbing
the initial state x0 and doing the standard average consensus on the
perturbed initial state. To prove the mechanism is indeed -differentially
private, for any x0 and x00 that are δ-adjacent, we have
0
Pr(x+ +
0 = x0 + w0 ∈ R) ≤ exp()Pr(x0 = x0 + w0 ∈ R).

Furthermore, since all the future x+ +

k are functions of x0 , (5.8) holds by
the post-processing theorem.
Since xk will converge to x̄ + 1T w0 /n × 1, the accuracy of this
approach is given by
 
1T w 2 1 h i
0 
1 = E (1T w0 )2 = 2δ 2 /2 .

E 
n n
2

Notice that this adding one-shot noise scheme is optimal in the sense
that it provides the best accuracy within a certain privacy requirement.
For more details, please refer to Nozari et al. (2017).
We further provide an impossibility result on differentially private
average consensus:
Theorem 5.2. Suppose that the consensus algorithm is -differential
private, i.e., (5.8) holds, then xk cannot converge to x̄ in probability for
any initial condition x0 .

Proof. From (5.7), xk converges to x̄ in probability only if x+

k converges
to x̄ in probability, which implies that for any τ > 0:
lim Pr(kx+
k − x̄k1 < τ ) = 1.
k→∞
Now suppose we choose x0 and x00 that are δ-adjacent to each other.
Let us choose τ = kx0 − x00 k1 /2. By weak convergence, we have
lim Pr(kx+ T
k (x0 , w) − 11 x0 /nk1 < τ ) = 1.
k→∞
Now from -differential privacy
0
Pr(kx+ T
k (x0 , w) − 11 x0 /nk1 < τ )
≥ exp(−)Pr(kx+ T
k (x0 , w) − 11 x0 /nk1 < τ ),
218 Privacy

which implies that

0
lim inf Pr(kx+ T
k (x0 , w) − 11 x0 /nk1 < τ ) ≥ exp(−).
k→∞

However, this is impossible since

0
Pr(kx+ T
k (x0 , w) − 11 x0 /nk1 < τ )
0 T 0
≤ 1 − Pr(kx+
k (x0 , w) − 11 x0 /nk1 < τ ),

the RHS of which converges to 0 at k goes to infinity.

Notice that since Lp convergence and almost surely convergence

both imply convergence in probability, xk cannot converge to x̄ in the
Lp sense nor almost surely. Hence, it is impossible to achieve differential
privacy while achieving exact consensus. Roughly speaking, this is due
to the fact that the privacy guarantee provided by differential privacy
is too strong. In the next subsection, we consider a “weaker” inference
privacy guarantee and prove that we can achieve exact average consensus
while preserving inference privacy.

5.2.3 Inference Privacy in Average Consensus

In this subsection, we propose a mechanism to achieve exact average
consensus. Notice that such a mechanism cannot be differentially private
due to Theorem 5.2. Therefore, we need to develop a different privacy
metric to quantify the privacy guarantees of the mechanism. We want
to ensure that any agent participating in consensus cannot infer the
initial conditions of other agents, which is called “inference privacy”.
Throughout this subsection, we make an additional assumption that A
is symmetric.
We will still leverage Algorithm 4, but with a different noise process
{wk }. To this end, let us define vk,i to be an IID normal random variable
with mean 0 and variance σ 2 . The noise wk,i is chosen such that

v , if k = 0
0,i
wk,i = , (5.9)
ϕk v k,i − ϕk−1 vk−1,i , otherwise
5.2. Differential and Inference Privacy in Average Consensus 219

where 0 < ϕ < 1 is a constant for all agents. Notice that

k
X
wk,i = ϕk vk,i .
t=0

Hence, the noise added by the agent i sums to 0 asymptotically, which

is crucial for ensuring exact average consensus, as is proved by the
following theorem:

Theorem 5.3. Suppose that the matrix A is symmetric with eigenvalues

1 = λ1 > λ2 ≥ · · · ≥ λn > −1. For any initial condition x0 , xk
converges to x̄ in the mean square sense. Furthermore, the mean square
convergence rate ρ equals

ρ = max(ϕ2 , |λ2 |2 , |λn |2 ), (5.10)

where ρ is defined as
 !1/k
E kzk k22

ρ , lim sup 2 , (5.11)
k→∞ z0 6=0 kz0 k2

with zk = xk − x̄.

Proof. Since the RHS of (5.10) is strictly less than 1, we only need to
prove (5.10), since it implies mean square convergence. By (5.7),
k−1
X
x k = Ak x 0 + Ak−t wt
t=0
k−2
X
= Ak x0 + Aϕk−1 vk−1 + ϕt Ak−t−1 (A − I)vt .
t=0

Define matrix A to be

A , A − 11T /n.

The following equalities hold for all k ≥ 0

Ak (A − I) = Ak (A − I), (5.12)
Ak − 11T /n = Ak (I − 11T /n). (5.13)
220 Privacy

Since x̄ = (11T /n)x0 , we have

k−2
X
zk = Ak z0 + Aϕk−1 vk−1 + ϕt Ak−t−1 (A − I)vt .
t=0

Since {vk } are IID Gaussian vectors with zero mean and covariance
σ 2 I, the mean square error can be written as
h i
E zkT zk = z0T A2k z0 + σ 2 tr(A2 )ϕ2k−2 (5.14)
k−2
X h i
+ σ2 ϕ2t tr A2k−2t−2 (A − I)2 ) .
t=0

Since all the terms on the RHS of (5.14) are non-negative,

h i
E zkT zk ≥ z0T A2k z0 ,
h i
E zkT zk ≥ σ 2 tr(A2 )ϕ2k−2 ,

which implies that

ρ ≥ max(ϕ2 , |λ2 |2 , |λn |2 ).

On the other hand, since the eigenvalues of A are λ1 , . . . , λn , we have

k−2
X h i n k−2
X X 
2t
ϕ tr A 2k−2t−2
(A − I) ) =2
ϕ2t λ2k−2t−2
i (λi − 1)2
t=0 i=2 t=0
2k−2
≤ (n − 1)(k − 1) [max(ϕ, |λ2 |, |λn |)] (λn − 1)2 .

The last inequality is true due to the fact that for all t,
 
ϕ2t λi2k−2t−2 (λi − 1)2 ≤ [max(ϕ, |λ2 |, |λn |)]2k−2 (λn − 1)2 .

Combining with (5.14), we can prove that

ρ ≤ max(ϕ2 , |λ2 |2 , |λn |2 ),

which finishes the proof.

Next we consider the privacy guarantees provided by the proposed

algorithm. Without loss of generality, we consider the case where agent
5.2. Differential and Inference Privacy in Average Consensus 221

n wants to infer the other agents initial conditions. Suppose agent n

has m neighbors, the set of which is denoted as
N (n) = {j1 , . . . , jm }.
Define h iT
C , ej1 . . . e jm en ∈ R(m+1)×n , (5.15)
where ei denotes the ith canonical basis vector in Rn . The information
available to the agent n at time k contains its initial state and the
messages it sends and it receives up to time k, i.e.,
Ik , {x0,n , y0 , . . . , yk }, (5.16)
where
yk , Cx+
k = C(xk + wk ). (5.17)
Before continuing, we define à ∈ R(n−1)×(n−1) (C̃ ∈ R(m−1)×(n−1) ),
as a submatrix of A (C) by removing the last row and column. Define
the following matrices:
U , C̃ T C̃ ∈ R(n−1)×(n−1) , V , I − U. (5.18)
Further denote the eigenvectors of the symmetric matrix (I −
Ã)−1 U(I − Ã)−1 as ψ1 , . . . , ψn−1 ∈ Rn−1 . Without loss of generality,
we assume that {ψ1 , . . . , ψn−1 } forms an orthonormal basis of Rn−1 .
Furthermore, we assume that the eigenvalues corresponding to the eigen-
vectors {ψ1 , . . . , ψm } are non-zero and the eigenvalues corresponding
to {ψm+1 , . . . , ψn−1 } are zero. Define the matrix
h i
Q , ψm+1 . . . ψn−1 . (5.19)
The following theorem provides privacy guarantees for the proposed
algorithm:
Theorem 5.4. Suppose that A is symmetric. The variance of any un-
biased estimate of x0,i given the information set Ik is lower bounded
by Pii , where Pii is the ith diagonal entry of P given by the following
equality:
 h i−1 
Q QT (I − Ã)−1 Y (I − Ã)−1 Q QT 0
P = σ2  , (5.20)
0 0
222 Privacy

where Y = limk→∞ Yk is the limit of the following recursive Riccati

equations:

Y0 = ÃU Ã, (5.21)

  −1 
Yk+1 = ÃU Ã + ϕ−2 Ã Yk+ − Yk+ ϕ2 I + Yk+ Yk+ Ã,

where
Yk+ = VYk V. (5.22)

Proof. Notice that since the noise is Gaussian, the maximum likelihood
estimator of x0 given Ik is the unbiased estimator with the smallest
covariance, which is denoted as Pk .
It is easy to see that since agent n receives more information as k
increases, Pk is monotonically non-increasing. We then need to establish
that Pk converges to P , which is quite technical. Readers can refer to
Mo and Murray (2017) for the full details of the proof.

Notice that as long as Pii is non-zero, we can choose a large enough

σ2 to achieve the desired level of privacy. As a result, we need to derive
conditions on whether Pii is zero or not. To this end, let us define the
essential neighborhood Ne (i) of an agent i to be the set of neighboring
agents whose information is used to compute (5.2), i.e.,

Ne (i) , {j ∈ N (i) : aij 6= 0}. (5.23)

A agent j is called a super-neighbor of i if it is a neighbor of i and

all of i’s essential neighbors.

Theorem 5.5. Agent n can asymptotically infer x0,i with 0 variance if

and only if n is a super-neighbor of i.

For the detailed proof of the theorem, please refer to Mo and Murray
(2017). Notice that the condition is local in the sense that the privacy
of the initial condition of agent i can only be breached by its neighbors.
If we compare the differential privacy based approach and inference
privacy based approach, we see that differential privacy provides much
stronger privacy guarantees, due to the following reasons:
5.2. Differential and Inference Privacy in Average Consensus 223

1. Inference privacy assumes that the agent can only access the
messages from itself and its neighbors y(k), while differential
privacy does not.
2. To prevent a privacy breach, inference privacy requires that no
super-neighbors exists. However, this is not required by differential
privacy.
However, inference privacy mechanisms can achieve exact average
consensus, which is its main advantage over differential privacy mecha-
nisms.
It is also worth noticing that other inference privacy metrics exist,
e.g., -information privacy, -average information leakage. For a more
detailed discussion, please refer to Sun and Tay (2017).
Example 5.1. We consider the following network consisting of 5 agents,
whose topology is illustrated in Figure 5.2. We assume that the following
A matrix is used:
 
2 1 0 0 1
1 2 1 0 0
 
1 
A=  0 1 2 0 1 .
4 
0 0 0 3 1

1 0 1 1 1
We first implement the privacy preserving consensus protocol proposed

1 2

4 5 3

Figure 5.2: Network Topology

by Huang et al. (2012), by using independent and exponentially decay-

ing Laplacian noise as our w(k). To be specific, we assume that the
probability density function of wi (k) is given by
1 |wi (k)|
 
PDF(wi (k)) = exp − ,
2b(k) b(k)
224 Privacy

where b(k) = ϕk , with ϕ = 0.9. From Figure 5.3, it can be seen that
although consensus is achieved, the final result is not the original average,
which may not be desirable for certain applications.

x1 (k)
x2 (k)
x3 (k)
0 x4 (k)
xi (k)

x5 (k)

−2

0 10 20 30 40 50
k
Figure 5.3: The trajectory of each state xi (k) when using the privacy preserving
consensus protocol proposed by Huang et al. (2012). The blue, red, green, yellow and
purple lines correspond to x1 (k), x2 (k), x3 (k), x4 (k), x5 (k) respectively. The black
dashed line corresponds to the average value of the initial x(0).

Figure 5.4 illustrates the trajectory of xi (k), when the noise follows
(5.9). It is worth noticing that all xi (k)’s converge to the true average
of the initial condition x(0).
However, the first algorithm, although it does not converge to the
exact average, can potentially provide more privacy guarantees. For the
example discussed, the first algorithm can preserve the privacy of agent
4. On the other hand, we prove in Theorem 5.5 that the initial condition
of agent 4 will be leaked to agent 5 using the second algorithm, as 5 is a
super-neighbor of 4. Therefore, there is a trade-off between privacy and
the accuracy of the consensus. To verify that, Figure 5.5 shows Pii (k)
of the maximum likelihood estimate of agent 4 and the asymptotic Pii
derived by Theorem 5.4, when using the second algorithm. P33 (k) is
omitted since it equals P11 (k) due to symmetry. Notice that both P11
and P22 are greater than 0. As a result, agent 5 cannot infer the exact
initial condition of agent 1 or agent 2. On the other hand, P44 = 0.
Therefore, the initial condition of agent 4 is not private to agent 5.
5.3. Cryptography Based Privacy 225

2
x1 (k)
x2 (k)
1 x3 (k)
x4 (k)
xi (k)
x5 (k)
0

−1

0 10 20 30 40 50
k

Figure 5.4: The trajectory of each state xi (k). The blue, red, green, yellow and
purple lines correspond to x1 (k), x2 (k), x3 (k), x4 (k), x5 (k) respectively. The black
dashed line corresponds to the average value of the initial x(0).

5.3 Cryptography Based Privacy

In this subsection we give a brief overview of cryptography based

privacy. We first introduce the concept of homomorphic encryption. An
encryption scheme E is a mapping from the plaintext to the ciphertext.
It is homomorphic with respect to addition if there exists a binary
operation ⊕, such that
E(m1 ) ⊕ E(m2 ) = E(m1 + m2 ).
It is homomorphic with respect to multiplication if there exists a binary
operation ⊗, such that
E(m1 ) ⊗ E(m2 ) = E(m1 × m2 ).
Notice that if an encryption scheme is additively homomorphic, then
it can also support the multiplication of integer in plaintext with the
ciphertext:
E(m1 × m2 ) = E(m2 ) ⊕ · · · ⊕ E(m2 ) .
| {z }
m1 times

The key benefit of a homomorphic encryption scheme is that it

allows computation on the ciphertext instead of plaintext. As a result,
226 Privacy

15 P11 (k)
P11
P22 (k)

Pii (k) 10 P22

P44 (k)
P44
5

0
5 10 15 20
k

Figure 5.5: Pii (k) v.s. k. The blue solid and dashed line correspond to P11 (k) and P11
respectively. The red solid and dashed line correspond to P22 (k) and P22 respectively.
The black solid and dashed line correspond to P44 (k) and P44 respectively.

homomorphic encryption can be leveraged in cloud computing, where

the client uploads encrypted data to the cloud server. The cloud server
performs computation on the encrypted data and sends the results back
to the client. The client can then decrypt the message sent by the cloud
server. The privacy of the client’s data is protected during the entire
process, as the cloud server can only access the ciphertext.
If an encryption scheme is either additively homomorphic or mul-
tiplicatively homomorphic, but not both, then it is called a partially
homomorphic encryption. For example, RSA (Rivest et al., 1978) is
multiplicatively homomorphic and Paillier (Paillier, 1999) is additively
homomorphic. On the other hand, if an encryption scheme is both
additively and multiplicatively homomorphic, then it is called a fully ho-
momorphic encryption. The first fully homomorphic encryption scheme
was proposed by Gentry (2009). The main benefit of fully homomorphic
encryption is that it can support the evaluation of arbitrary functions
over ciphertext. However, it is usually computationally expensive to
implement a fully homomorphic encryption scheme, which prohibits its
adoption in cyber-physical systems, where the computational capability
of devices are limited.
5.4. Further Reading 227

In a distributed system with multiple agents, secure multi-party

computation can be used to preserve the privacy of the agents. Suppose
there are n agents, each with a value xi , and the goal is to compute a
certain function f (x1 , . . . , xn ).
Informally speaking, secure multi-party computation ensures that
the only information about the private data x1 , . . . , xn that can be
inferred from the messages sent during the execution of the protocol is
whatever could be inferred from seeing the output of the function alone.
There are multiple ways to achieve secure multi-party computation.
For example, oblivious transfer can be used for 2 party computation (Kil-
ian, 1988). For multiparty, secret sharing and homomorphic encryption
can be adopted (Damgård et al., 2012). The detailed discussion of these
approaches is out of the scope of this monograph.

5.4 Further Reading

In this section, we briefly discussed different ways to achieve privacy

in cyber-physical systems. We first consider the concept of differential
privacy, which is a data privacy approach. It is worth noticing that
other data privacy metrics, such as -identifiability and -mutual infor-
mation privacy, exist. Furthermore, these privacy metrics are sometimes
connected in the sense that one metric may imply the other. For more
discussion on data privacy, please refer to Wang et al. (2016). The
readers can also find examples on applying differential privacy to cyber-
physical systems in Le Ny and Pappas (2014), Cortes et al. (2016), and
Han et al. (2017b).
It is worth noticing that in general, the differential privacy provides
a very strong privacy guarantee, in the sense that no functions of the
original data can be calculated accurately. This property can be ben-
eficial to prevent attackers to obtain useful information. However, it
also limits the usefulness of the perturbed data. On the other hand,
inference privacy mechanisms, which can be tailored to allow the calcu-
lation of certain functions and disallow others, may be more suitable for
applications where accuracy is of importance. However, the inference
privacy mechanism needs to be carefully designed depending on which
function of the data is important to the system. Readers can find a
228 Privacy

comparison of inference privacy and data privacy in Sun and Tay (2017)
and Pin Calmon and Fawaz (2012). Liao et al. (2018) considers applying
inference privacy to hypothesis testing, which can also be interesting
for readers.
Finally we consider cryptography based approach to achieve pri-
vacy. At the time of writing, there are few results on applying fully
homomorphic encryption to cyber-physical systems, due to its com-
putational complexity. However, for certain applications, partially ho-
momorphic encryption has been sufficient. It has been shown in the
literature that additively homomorphic encryption can be used for
privacy preserving distributed optimization (Shoukry et al., 2016) or
average consensus (Ruan et al., 2019).
 6
Conclusions

In this monograph, we investigated the challenges of achieving resilient

control in cyber-physical systems, offering tools for analysis and design
that take fundamental steps towards achieving reliable, secure operation
in CPS. As an initial step, we identified the difficulties in modeling
modern CPS, which arise largely from the complex interactions between
a CPS’s diverse subsystems. We examined how traditional state space
and LTI systems can be used to describe phenomena in CPS, especially
when we add stochastic variables that model system uncertainty and
network imperfections. Additionally, we noted the utility of hybrid
system modeling in CPS due to its ability to simultaneously characterize
the discrete elements and decision variables in software systems as well as
the physical states of dynamical control systems. We next addressed the
challenges of reliable operation when decisions are made over networks.
Specifically, we considered the problem of performing estimation and
control over lossy networks, where sensory and input packets are delayed
or dropped altogether. Here, we summarized fundamental limitations on
network reliability which determine the ability of an operator to achieve
stabilization of CPS over networks with packet drops. We also studied
the design of CPS with resource constraints. Particularly, we noted

229
230 Conclusions

power and bandwidth limitations which impede constant communication

over networks and discussed how sensor scheduling and event based
estimation/control can address these issues.
We next considered the resilience of CPS against malicious adver-
saries. CPS are complex engineered systems associated with critical
infrastructures and present a variety of attack surfaces, offering both
motivation and opportunity for potential attackers. To begin, we dis-
cussed adversarial models in CPS, categorizing attackers in terms of
their knowledge, capabilities, and strategies. Next, we described mech-
anisms for achieving graceful degradation in CPS so that the system
remains operational, even in the presence of an attacker. In this context,
we compared the concepts of robustness and resilience in CPS. While
robustness involves sacrificing some measure of system performance
in order to tolerate malicious behavior if or when it occurs, resilience
incorporates the ability to take corrective actions after an attack occurs
to achieve system recovery. We considered robust structural sensor and
link placement to ensure properties of strong observability and attack
detectability in distributed control systems. To achieve resilience, we
initially investigated the problem of attack detection. Here, we intro-
duced tools for active detection/identification which enable operators
to recognize and isolate classes of intelligent and stealthy attacks, by
introducing physical perturbations in the system. We then focused on
resilient response. In particular, we delved into the task of resilient
estimation, which allows a defender to have an understanding of the
system state, even in the presence of an attacker, and thus to make
informed decisions.
Finally, we provided an introduction to the role of privacy for the
resilience of CPS. We acknowledged the reality that many control
tasks in our physical infrastructures require the collection of sensitive
information, whether it be power consumption of users in the grid, travel
habits of drivers in transportation system, or the medical records of
patients in a hospital. We investigated notions of data privacy in CPS,
with an emphasis on differential privacy. We then examined mechanisms
for guaranteeing these notions of privacy through the example of average
consensus. Finally, we gave a brief introduction to cryptography based
privacy and the potential useful role of homomorphic encryption.
 231

The main goal of this monograph was to provide an introductory

survey that allows students, researchers, and practitioners to gain a
fundamental understanding of the nature of resilience in cyber-physical
systems. Of course, there is much left to consider in the design of re-
silient CPS. For instance, while most of the concepts presented here
involved a central decision maker, there is a rich literature that ex-
amines distributed techniques for obtaining resilience. Additionally, in
order to present a broad set of tools and techniques, the models we
considered were often relatively simple. More detailed design will need
to consider the challenges for security introduced by the amalgamation
of a set of heterogeneous subsystems, which comprise CPS. A science
of compositional security needs to be considered in order analyze how
properties of security are preserved or destroyed when disparate systems
interact. Finally, our current treatment does not specialize in challenges
which may arise in specific applications such as transportation systems,
the smart grid, health care, and water distribution. For the results we
developed in this tutorial to have a direct impact on society, one has
to investigate how the presented tools can be applied to real world
scenarios. While it is unavoidable to reach different conclusions based
on more specific practical models, the mathematical frameworks in this
monograph offer methods to formally handle problems that emerge in
cyber-physical systems.
 References

Abrams, M. and J. Weiss. 2008. “Malicious control system cyber security

attack case study–Maroochy Water Services, Australia”. McLean,
VA: The MITRE Corporation.
Abur, A. and A. G. Exposito. 2004. Power system state estimation:
theory and implementation. CRC press.
Aksanli, B. and T. S. Rosing. 2017. “Human Behavior Aware Energy
Management in Residential Cyber-Physical Systems”. IEEE Trans-
actions on Emerging Topics in Computing. PP(99): 1–1.
Albadi, M. H. and E. El-Saadany. 2008. “A summary of demand response
in electricity markets”. Electric Power Systems Research. 78(11):
1989–1996.
Alemzadeh, H., C. D. Martino, Z. Jin, Z. T. Kalbarczyk, and R. K. Iyer.
2012. “Towards resiliency in embedded medical monitoring devices”.
In: Workshops of International Conference on Dependable Systems
and Networks. IEEE/IFIP. 1–6.
Alemzadeh, H., K. Ravishankar, Z. Kalbarczyk, and J. Raman. 2013.
“Analysis of Safety-Critical Computer Failures in Medical Devices”.
Security & Privacy. 11(4): 14–26.
Amin, S. M. and B. F. Wollenberg. 2005. “Toward a smart grid: power
delivery for the 21st century”. IEEE Power and Energy Magazine.
3(5): 34–41.

232
References 233

Amin, S., A. A. Cárdenas, and S. Sastry. 2009. “Safe and Secure Net-
worked Control Systems under Denial-of-Service Attacks”. In: Inter-
national Workshop on Hybrid Systems: Computation and Control.
Vol. 5469. Springer. 31–45.
Amin, S., X. Litrico, S. Sastry, and A. M. Bayen. 2013. “Cyber security
of water SCADA systems – Part I: Analysis and experimentation of
stealthy deception attacks”. IEEE Transactions on Control Systems
Technology. 21(5): 1963–1970.
Amoozadeh, M., A. Raghuramu, C.-N. Chuah, D. Ghosal, H. M. Zhang,
J. Rowe, and K. Levitt. 2015. “Security vulnerabilities of connected
vehicle streams and their impact on cooperative driving”. IEEE
Communications Magazine. 53(6): 126–132.
Anta, A. and P. Tabuada. 2010. “To sample or not to sample: Self-
triggered control for nonlinear systems”. IEEE Transactions on
Automatic Control. 55(9): 2030–2042.
Arulampalam, M. S., S. Maskell, N. Gordon, and T. Clapp. 2002. “A
Tutorial on Particle Filters for Online Nonlinear/Non-Gaussian
Bayesian Tracking”. IEEE Transactions on Signal Processing. 50(2):
174–188.
Bai, C. Z., V. Gupta, and F. Pasqualetti. 2017a. “On Kalman Filtering
with Compromised Sensors: Attack Stealthiness and Performance
Bounds”. IEEE Transactions on Automatic Control. 62(12): 6641–
6648.
Bai, C. Z., F. Pasqualetti, and V. Gupta. 2015. “Security in stochastic
control systems: Fundamental limitations and performance bounds”.
In: American Control Conference. IEEE. 195–200.
Bai, C.-Z., F. Pasqualetti, and V. Gupta. 2017b. “Data injection at-
tacks in stochastic control systems: Detectability and performance
tradeoffs”. Automatica. 82: 251–260.
Bi, S. and Y. J. Zhang. 2014. “Graphical methods for defense against
false-data injection attacks on power system state estimation”. IEEE
Transactions on Smart Grid. 5(3): 1216–1227.
Bobba, R. B., K. M. Rogers, Q. Wang, H. Khurana, K. Nahrstedt,
and T. J. Overbye. 2010. “Detecting false data injection attacks
on DC state estimation”. In: Workshop on Secure Control Systems,
CPSWEEK.
234 References

Borgers, D. P. and M. W. Heemels. 2014. “Stability Analysis of Large-

scale Networked Control Systems with Local Networks: A Hybrid
Small-gain Approach”. In: International Conference on Hybrid Sys-
tems: Computation and Control. ACM.
Boukhobza, T. and F. Hamelin. 2009. “State and input observability
recovering by additional sensor implementation: A graph-theoretic
approach”. Automatica. 45(7): 1737–1742.
Boukhobza, T., F. Hamelin, and S. Martinez-Martinez. 2007. “State and
input observability for structured linear systems: A graph-theoretic
approach”. Automatica. 43(7): 1204–1210.
Branicky, M. S., V. S. Borkar, and S. K. Mitter. 1998. “A unified
framework for hybrid control: model and optimal control theory”.
IEEE Transactions on Automatic Control. 43(1): 31–45.
Candès, E. J., M. B. Wakin, and S. P. Boyd. 2008. “Enhancing Sparsity
by Reweighted L1 Minimization”. Journal of Fourier Analysis and
Applications. 14(5-6): 877–905.
Cardenas, A. A., T. Roosta, and S. Sastry. 2009. “Rethinking security
properties, threat models, and the design space in sensor networks: A
case study in SCADA systems”. Ad Hoc Networks. 7(8): 1434–1447.
Censi, A. 2009. “On the performance of Kalman filtering with intermit-
tent observations: A geometric approach with fractals”. In: American
Control Conference. IEEE. 3806–3812.
Censi, A. 2011. “Kalman Filtering With Intermittent Observations:
Convergence for Semi-Markov Chains and an Intrinsic Performance
Measure”. IEEE Transactions on Automatic Control. 56(2): 376–
381.
Chabukswar, R., Y. Mo, and B. Sinopoli. 2011. “Detecting Integrity
Attacks on SCADA Systems”. In: IFAC World Congress. 11239–
11244.
Chaojun, G., P. Jirutitijaroen, and M. Motani. 2015. “Detecting false
data injection attacks in AC state estimation”. IEEE Transactions
on Smart Grid. 6(5): 2476–2483.
Chen, Y., S. Kar, and J. M. F. Moura. 2017a. “Optimal Attack Strategies
Subject to Detection Constraints Against Cyber-Physical Systems”.
IEEE Transactions on Control of Network Systems. PP(99): 1–1.
References 235

Chen, Y., S. Kar, and J. M. Moura. 2017b. “Dynamic attack detection

in cyber-physical systems with side initial state information”. IEEE
Transactions on Automatic Control. 62(9): 4618–4624.
Chong, M. S., M. Wakaiki, and J. P. Hespanha. 2015. “Observability
of linear systems under adversarial attacks”. In: American Control
Conference. IEEE. 2439–2444.
Cortes, J., G. E. Dullerud, S. Han, J. L. Ny, S. Mitra, and G. J. Pappas.
2016. “Differential privacy in control and network systems”. In:
Conference on Decision and Control. IEEE. 4252–4272.
Dadras, S., R. M. Gerdes, and R. Sharma. 2015. “Vehicular platooning
in an adversarial environment”. In: Symposium on Information,
Computer and Communications Security. ACM. 167–178.
Dahleh, M. and I. J. Diaz-Bobillo. 1994. Control of Uncertain Systems:
A Linear Programming Approach. Prentice-Hall, Inc.
Damgård, I., V. Pastro, N. Smart, and S. Zakarias. 2012. “Multi-
party Computation from Somewhat Homomorphic Encryption”.
In: Springer, Berlin, Heidelberg. 643–662.
Davare, A., D. Densmore, L. Guo, R. Passerone, A. L. Sangiovanni-
Vincentelli, A. Simalatsar, and Q. Zhu. 2013. “metroII: A design
environment for cyber-physical systems”. ACM Transactions on
Embedded Computing Systems. 12(1s): 49.
DeBruhl, B., S. Weerakkody, B. Sinopoli, and P. Tague. 2015. “Is your
commute driving you crazy?: A study of misbehavior in vehicular
platoons”. In: Conference on Security & Privacy in Wireless and
Mobile Networks. ACM. 22:1–22:11.
Denning, D. E. 1976. “A lattice model of secure information flow”.
Communications of the ACM. 19(5): 236–243.
Department-of-Homeland-Security. 2015. “The future of smart cities:
cyber-physical infrastructural risk”. Tech. rep. Department of Home-
land Security.
Derler, P., E. A. Lee, and A. S. Vincentelli. 2012. “Modeling Cyber-
Physical Systems”. Proceedings of the IEEE. 100(1): 13–28.
Dimarogonas, D. V., E. Frazzoli, and K. H. Johansson. 2012. “Dis-
tributed event-triggered control for multi-agent systems”. IEEE
Transactions on Automatic Control. 57(5): 1291–1297.
236 References

Dinic, E. A. 1970. “An algorithm for the solution of the max-flow

problem with the polynomial estimation”. Doklady Akademii Nauk.
194(4): 1277–1280.
Dion, J.-M., C. Commault, and J. Van Der Woude. 2003. “Generic prop-
erties and control of linear structured systems: a survey”. Automatica.
39(7): 1125–1144.
Donkers, M. and W. Heemels. 2012. “Output-Based Event-Triggered
Control With Guaranteed L∞ -Gain and Improved and Decentralized
Event-Triggering”. IEEE Transactions on Automatic Control. 57(6):
1362–1376.
Douglas, S. M., I. Bachelet, and G. M. Church. 2012. “A logic-gated
nanorobot for targeted transport of molecular payloads”. Science.
335(6070): 831–834.
Edwards, S. A. and E. A. Lee. 2007. “The case for the precision timed
(PRET) machine”. In: Design Automation Conference. ACM. 264–
265.
Eker, J., J. W. Janneck, E. A. Lee, J. Liu, X. Liu, J. Ludvig, S. Neuen-
dorffer, S. Sachs, and Y. Xiong. 2003. “Taming heterogeneity-the
Ptolemy approach”. Proceedings of the IEEE. 91(1): 127–144.
Eqtami, A., D. V. Dimarogonas, and K. J. Kyriakopoulos. 2010. “Event-
triggered control for discrete-time systems”. In: American Control
Conference. IEEE. 4719–4724.
Even, S. and R. E. Tarjan. 1975. “Network flow and testing graph
connectivity”. SIAM Journal on Computing. 4(4): 507–518.
Fang, X., S. Misra, G. Xue, and D. Yang. 2012. “Smart Grid – The
New and Improved Power Grid: A Survey”. IEEE Communications
Surveys & Tutorials. 14(4): 944–980.
Farhangi, H. 2010. “The path of the smart grid”. IEEE Power and
Energy Magazine. 8(1): 18–28.
Fawzi, H., P. Tabuada, and S. Diggavi. 2014. “Secure estimation and
control for cyber-physical systems under adversarial attacks”. IEEE
Transactions on Automatic Control. 59(6): 1454–1467.
Forti, N., G. Battistelli, L. Chisci, and B. Sinopoli. 2016. “A Bayesian
approach to joint attack detection and resilient state estimation”.
In: Conference on Decision and Control. IEEE. 1192–1198.
References 237

Fritzson, P. 2014. Principles of object-oriented modeling and simulation

with Modelica 3.3: A cyber-physical approach. John Wiley & Sons.
Garcia, E. and P. J. Antsaklis. 2013. “Model-based event-triggered
control for systems with quantization and time-varying network
delays”. IEEE Transactions on Automatic Control. 58(2): 422–434.
Gentry, C. 2009. “A fully homomorphic encryption scheme”. PhD thesis.
Stanford University. isbn: 978-1-109-44450-6.
Gerdes, R. M., C. Winstead, and K. Heaslip. 2013. “CPS: An efficiency-
motivated attack against autonomous vehicular transportation”. In:
Computer Security Applications Conference. ACM. 99–108.
Giani, A., G. Karsai, T. Roosta, A. Shah, B. Sinopoli, and J. Wiley.
2008. “A testbed for secure and robust SCADA systems”. ACM
SIGBED Review. 5(2): 4.
Giannakis, G. B. and E. Serpedin. 2001. “A bibliography on nonlinear
system identification”. Signal Processing. 81(3): 533–580.
Giraldo, J., D. Urbina, A. Cardenas, J. Valente, M. Faisal, J. Ruths,
N. O. Tippenhauer, H. Sandberg, and R. Candell. 2018. “A Survey of
Physics-Based Attack Detection in Cyber-Physical Systems”. ACM
Computing Surveys (CSUR). 51(4): 76.
Goguen, J. A. and J. Meseguer. 1982. “Security policies and security
models”. In: IEEE Symposium on Security and Privacy. 11–20.
Grebeck, M. 1998. “A comparison of controllers for the quadruple
tank system”. Department of Automatic Control, Lund Institute of
Technology, Lund, Sweden, Tech. Rep.
Guinaldo, M., D. V. Dimarogonas, K. H. Johansson, J. Moreno, and S.
Dormido. 2011. “Distributed event-based control for interconnected
linear systems”. In: Conference on Decision and Control held jointly
with the European Control Conference. IEEE. 2553–2558.
Guinaldo, M., D. Lehmann, J. Sánchez, S. Dormido, and K. H. Johans-
son. 2012. “Distributed event-triggered control with network delays
and packet losses”. In: Conference on Decision and Control. IEEE.
1–6.
Han, D., Y. Mo, J. Wu, S. Weerakkody, B. Sinopoli, and L. Shi. 2015.
“Stochastic Event-Triggered Sensor Schedule for Remote State Esti-
mation”. IEEE Transactions on Automatic Control. 60(10): 2661–
2675.
238 References

Han, D., Y. Mo, and L. Xie. 2019. “Convex optimization based state
estimation against sparse integrity attacks”. IEEE Transactions on
Automatic Control. 64(6): 2383–2395.
Han, D., J. Wu, H. Zhang, and L. Shi. 2017a. “Optimal sensor scheduling
for multiple linear dynamical systems”. Automatica. 75(Jan.): 260–
270.
Han, S., U. Topcu, and G. J. Pappas. 2017b. “Differentially Private
Distributed Constrained Optimization”. IEEE Transactions on Au-
tomatic Control. 62(1): 50–64.
He, L., D. Han, X. Wang, and L. Shi. 2013. “Optimal linear state
estimation over a packet-dropping network using linear temporal
coding”. Automatica. 49(4): 1075–1082.
Heemels, W. H., M. Donkers, and A. R. Teel. 2013. “Periodic event-
triggered control for linear systems”. IEEE Transactions on Auto-
matic Control. 58(4): 847–861.
Heemels, W., K. H. Johansson, and P. Tabuada. 2012. “An introduction
to event-triggered and self-triggered control”. In: Conference on
Decision and Control. IEEE. 3270–3285.
Henzinger, T., B. Horowitz, and C. Kirsch. 2001. “Giotto: A time-
triggered language for embedded programming”. In: Embedded soft-
ware. Springer. 166–184.
Hespanhol, P., M. Porter, R. Vasudevan, and A. Aswani. 2017. “Dynamic
Watermarking for General LTI Systems”. In: Conference on Decision
and Control. IEEE. 1834–1839.
Ho, J. S., A. J. Yeh, E. Neofytou, S. Kim, Y. Tanabe, B. Patlolla,
R. E. Beygui, and A. S. Y. Poon. 2014. “Wireless power transfer to
deep-tissue microimplants”. Proceedings of the National Academy
of Sciences. 111(22): 7974–7979.
Hoehn, A. and P. Zhang. 2016a. “Detection of covert attacks and zero
dynamics attacks in cyber-physical systems”. In: American Control
Conference. IEEE. 302–307.
Hoehn, A. and P. Zhang. 2016b. “Detection of replay attacks in cyber-
physical systems”. In: American Control Conference. IEEE. 290–
295.
References 239

Hoh, B., M. Gruteser, H. Xiong, and A. Alrabady. 2006. “Enhancing

security and privacy in traffic-monitoring systems”. IEEE Pervasive
Computing. 5(4): 38–46.
Hosseini, M., T. Tanaka, and V. Gupta. 2016. “Designing optimal
watermark signal for a stealthy attacker”. In: European Control
Conference. IEEE. 2258–2262.
Huang, Z., S. Mitra, and G. Dullerud. 2012. “Differentially private
iterative synchronous consensus”. In: Proceedings of the 2012 ACM
workshop on Privacy in the electronic society. ACM. 81–90.
Hug, G. and J. A. Giampapa. 2012. “Vulnerability assessment of AC
state estimation with respect to false data injection cyber-attacks”.
IEEE Transactions on Smart Grid. 3(3): 1362–1370.
Jawaid, S. T. and S. L. Smith. 2015. “Submodularity and greedy algo-
rithms in sensor scheduling for linear dynamical systems”. Automat-
ica. 61(Nov.): 282–288.
Johannessen, S. 2004. “Time synchronization in a local area network”.
IEEE Control Systems. 24(2): 61–69.
Johansson, K. H., G. J. Pappas, P. Tabuada, and C. J. Tomlin. 2014.
“Guest Editorial Special Issue on Control of Cyber-Physical Systems”.
IEEE Transactions on Automatic Control. 59(12): 3120–3121.
Johansson, K. H. 2000. “The quadruple-tank process: A multivariable
laboratory process with an adjustable zero”. IEEE Transactions on
control systems technology. 8(3): 456–465.
Joshi, S. and S. Boyd. 2009. “Sensor Selection via Convex Optimization”.
IEEE Transactions on Signal Processing. 57(2): 451–462.
Kar, S., B. Sinopoli, and J. M. F. Moura. 2012. “Kalman Filtering
With Intermittent Observations: Weak Convergence to a Stationary
Distribution”. IEEE Transactions on Automatic Control. 57(2): 405–
420.
Kilian, J. 1988. “Founding Cryptography on Oblivious Transfer”. In:
ACM Symposium on Theory of Computing. 20–31.
Kim, K. D. and P. R. Kumar. 2012. “Cyber-Physical Systems: A Per-
spective at the Centennial”. Proceedings of the IEEE. 100: 1287–
1308.
240 References

Kleissl, J. and Y. Agarwal. 2010. “Cyber-physical energy systems: Focus

on smart buildings”. In: Design Automation Conference. ACM. 749–
754.
Ko, W.-H., B. Satchidanandan, and P. Kumar. 2016. “Theory and imple-
mentation of dynamic watermarking for cybersecurity of advanced
transportation systems”. In: Conference on Communications and
Network Security. IEEE. 416–420.
Kocabas, O., T. Soyata, and M. K. Aktas. 2016. “Emerging Security
Mechanisms for Medical Cyber Physical Systems”. IEEE/ACM
Transactions on Computational Biology and Bioinformatics. 13(3):
401–416.
Konig, M., J. Jacob, T. Kaddoura, and A. M. Farid. 2015. “The role
of resource efficient decentralized waste water treatment in smart
cities”. In: International Smart Cities Conference. IEEE. 1–5.
Koomey, J. 2011. “Growth in data center electricity use 2005 to 2010”.
A report by Analytical Press, completed at the request of The New
York Times. 9.
Kung, E., S. Dey, and L. Shi. 2017. “The Performance and Limitations
of -Stealthy Attacks on Higher Order Systems”. IEEE Transactions
on Automatic Control. 62(2): 941–947.
Kwon, C. and I. Hwang. 2017. “Reachability Analysis for Safety As-
surance of Cyber-Physical Systems against Cyber Attacks”. IEEE
Transactions on Automatic Control. PP(99): 1–1.
Kwon, C. and I. Hwang. 2016. “Recursive reachable set computation
for on-line safety assessment of the Cyber-Physical System against
stealthy cyber attacks”. In: Allerton Conference on Communication,
Control, and Computing. IEEE. 1123–1128.
Langner, R. 2011. “Stuxnet: Dissecting a cyberwarfare weapon”. IEEE
Security & Privacy. 9(3): 49–51.
Laszka, A., W. Abbas, Y. Vorobeychik, and X. Koutsoukos. 2017. “Syn-
ergic security for smart water networks: redundancy, diversity, and
hardening”. In: International Workshop on Cyber-Physical Systems
for Smart Water Networks. ACM. 21–24.
Le Ny, J. and G. J. Pappas. 2014. “Differentially private filtering”. IEEE
Transactions on Automatic Control. 59(2): 341–354.
References 241

Lee, C., H. Shim, and Y. Eun. 2015. “Secure and robust state estimation
under sensor attacks, measurement noises, and process disturbances:
Observer-based combinatorial approach”. In: European Control Con-
ference. IEEE. 1872–1877.
Lee, E. A. 2006. “Cyber-physical systems-are computing foundations
adequate”. In: NSF Workshop On Cyber-Physical Systems: Research
Motivation, Techniques and Roadmap. Vol. 2.
Lee, E. A. 2008. “Cyber physical systems: Design challenges”. In: In-
ternational Symposium on Object Oriented Real-Time Distributed
Computing. IEEE. 363–369.
Lee, I. and O. Sokolsky. 2010. “Medical Cyber Physical Systems”. In:
Design Automation Conference. ACM/IEEE. 743–748.
Liao, J., L. Sankar, V. Y. F. Tan, and F. du Pin Calmon. 2018. “Hy-
pothesis Testing Under Mutual Information Privacy Constraints
in the High Privacy Regime”. IEEE Transactions on Information
Forensics and Security. 13(4): 1058–1071.
Liu, L., M. Esmalifalak, Q. Ding, V. A. Emesih, and Z. Han. 2014.
“Detecting false data injection attacks on power grid by sparse
optimization”. IEEE Transactions on Smart Grid. 5(2): 612–621.
Liu, X., S. Weerakkody, and B. Sinopoli. 2016. “Sensor placement for
reliable observability: a structured systems approach”. In: Conference
on Decision and Control. IEEE. 5414–5421.
Liu, X., Y. Mo, and E. Garone. 2017. “Secure Dynamic State Estimation
by Decomposing Kalman Filter”. IFAC-PapersOnLine. 50(1): 7351–
7356.
Liu, Y., P. Ning, and M. K. Reiter. 2011. “False data injection attacks
against state estimation in electric power grids”. ACM Transactions
on Information and System Security. 14(1): 13.
Ljung, L. 1998. System Identification: Theory for the User. Pearson
Education.
Ljung, L., H. Hjalmarsson, and H. Ohlsson. 2011. “Four encounters
with system identification”. European Journal of Control. 17(5-6):
449–471.
Mazo, M. and P. Tabuada. 2009. “Input-to-state stability of self-triggered
control systems”. In: Conference on Decision and Control. IEEE.
928–933.
242 References

McDaniel, P. and S. McLaughlin. 2009. “Security and Privacy Challenges

in the Smart Grid”. IEEE Security Privacy. 7(3): 75–77.
Mehra, R. 1974. “Optimal inputs for linear system identification”. IEEE
Transactions on Automatic Control. 19(3): 192–200.
Menger, K. 1927. “Zur allgemeinen kurventheorie”. Fundamenta Math-
ematicae. 1(10): 96–115.
Miao, F., M. Pajic, and G. J. Pappas. 2013. “Stochastic game approach
for replay attack detection”. In: Conference on Decision and Control.
IEEE. 1854–1859.
Miao, F., Q. Zhu, M. Pajic, and G. J. Pappas. 2014. “Coding sensor
outputs for injection attacks detection”. In: Conference on Decision
and Control. IEEE. 5776–5781.
Miller, C. and C. Valasek. 2015. “Remote exploitation of an unaltered
passenger vehicle”. Black Hat USA. 2015.
Mistry, P. 2011. “Pressure management to reduce water demand &
leakage”. Wide Bay Water Corporation, Australia.
Mitcheson, P. D. 2010. “Energy harvesting for human wearable and im-
plantable bio-sensors”. In: International Conference on Engineering
in Medicine and Biology Society. IEEE. 3432–3436.
Mo, Y., E. Garone, and B. Sinopoli. 2014a. “On infinite-horizon sensor
scheduling”. Systems & Control Letters. 67(May): 65–70.
Mo, Y., T. H. J. Kim, K. Brancik, D. Dickinson, H. Lee, A. Perrig,
and B. Sinopoli. 2012a. “Cyber-Physical Security of a Smart Grid
Infrastructure”. Proceedings of the IEEE. 100(1): 195–209.
Mo, Y., R. Ambrosino, and B. Sinopoli. 2011a. “Sensor selection strate-
gies for state estimation in energy constrained wireless sensor net-
works”. Automatica. 47(7): 1330–1338.
Mo, Y., R. Chabukswar, and B. Sinopoli. 2014b. “Detecting integrity
attacks on SCADA systems”. IEEE Transactions on Control Systems
Technology. 22(4): 1396–1407.
Mo, Y., E. Garone, A. Casavola, and B. Sinopoli. 2010. “False data in-
jection attacks against state estimation in wireless sensor networks”.
In: Conference on Decision and Control. IEEE. 5967–5972.
References 243

Mo, Y., E. Garone, A. Casavola, and B. Sinopoli. 2011b. “Stochastic

Sensor Scheduling for Energy Constrained Estimation in Multi-
Hop Wireless Sensor Networks”. IEEE Transactions on Automatic
Control. 56(10): 2489–2495.
Mo, Y., E. Garone, and B. Sinopoli. 2013. “LQG control with Markovian
packet loss”. In: European Control Conference. IEEE. 2380–2385.
Mo, Y. and R. M. Murray. 2017. “Privacy Preserving Average Consen-
sus”. IEEE Transactions on Automatic Control. 62(2): 753–765.
Mo, Y. and B. Sinopoli. 2009. “Secure control against replay attacks”. In:
Allerton Conference on Communication, Control, and Computing.
IEEE. 911–918.
Mo, Y. and B. Sinopoli. 2010. “False Data Injection Attacks in Control
Systems”. In: Workshop on Secure Control Systems.
Mo, Y. and B. Sinopoli. 2011. “Kalman Filtering with Intermittent
Observations: Critical Value for Second Order System”. IFAC Pro-
ceedings Volumes. 44(1): 6592–6597.
Mo, Y. and B. Sinopoli. 2012a. “Integrity attacks on cyber-physical sys-
tems”. In: International Conference on High Confidence Networked
Systems. ACM. 47–54.
Mo, Y. and B. Sinopoli. 2012b. “Kalman filtering with intermittent ob-
servations: Tail distribution and critical value”. IEEE Transactions
on Automatic Control. 57(3): 677–689.
Mo, Y. and B. Sinopoli. 2016. “On the performance degradation of cyber-
physical systems under stealthy integrity attacks”. IEEE Transac-
tions on Automatic Control. 61(9): 2618–2624.
Mo, Y., B. Sinopoli, L. Shi, and E. Garone. 2012b. “Infinite-horizon
sensor scheduling for estimation over lossy networks”. In: Conference
on Decision and Control. IEEE. 3317–3322.
Mo, Y., S. Weerakkody, and B. Sinopoli. 2015. “Physical authentication
of control systems: Designing watermarked control inputs to detect
counterfeit sensor outputs”. IEEE Control Systems. 35(1): 93–109.
Mohassel, R. R., A. Fung, F. Mohammadi, and K. Raahemifar. 2014. “A
survey on advanced metering infrastructure”. International Journal
of Electrical Power & Energy Systems. 63: 473–484.
244 References

Mpitziopoulos, A., D. Gavalas, C. Konstantopoulos, and G. Pantziou.

2009. “A survey on jamming attacks and countermeasures in WSNs”.
IEEE Communications Surveys & Tutorials. 11(4).
Mutchek, M. and E. Williams. 2014. “Moving towards sustainable and
resilient smart water grids”. Challenges. 5(1): 123–137.
Nair, G. N., F. Fagnani, S. Zampieri, and R. J. Evans. 2007. “Feedback
Control Under Data Rate Constraints: An Overview”. Proceedings
of the IEEE. 95(1): 108–137.
Nakahira, Y. and Y. Mo. 2015. “Dynamic state estimation in the
presence of compromised sensory data”. In: Conference on Decision
and Control. IEEE. 5808–5813.
Narendra, K. S. and K. Parthasarathy. 1990. “Identification and control
of dynamical systems using neural networks”. IEEE Transactions
on Neural Networks. 1(1): 4–27.
Needham, R. M. and M. D. Schroeder. 1978. “Using encryption for
authentication in large networks of computers”. Communications of
the ACM. 21(12): 993–999.
Nelles, O. 2013. Nonlinear system identification: from classical ap-
proaches to neural networks and fuzzy models. Springer Science &
Business Media.
Nemhauser, G. L., L. A. Wolsey, and M. L. Fisher. 1978. “An analy-
sis of approximations for maximizing submodular set functions-I”.
Mathematical Programming. 14(1): 265–294.
Nilsson, J. et al. 1998. “Real-time control systems with delays”. Lund
institute of Technology Lund, Sweden.
Nozari, E., P. Tallapragada, and J. Cortés. 2017. “Differentially private
average consensus: Obstructions, trade-offs, and optimal algorithm
design”. Automatica. 81(July): 221–231.
Ozel, O., S. Weerakkody, and B. Sinopoli. 2017. “Physical Watermarking
for Securing Cyber-Physical Systems via Packet Drop Injections”.
In: IEEE International Conference on Smart Grid Communications.
Paillier, P. 1999. “Public-Key Cryptosystems Based on Composite
Degree Residuosity Classes”. In: Advances in Cryptology – EURO-
CRYPT ’99. Springer Berlin Heidelberg. 223–238.
References 245

Pajic, M., P. Tabuada, I. Lee, and G. J. Pappas. 2015. “Attack-resilient

state estimation in the presence of noise”. In: Conference on Decision
and Control. IEEE. 5827–5832.
Pajic, M., J. Weimer, N. Bezzo, P. Tabuada, O. Sokolsky, I. Lee, and G. J.
Pappas. 2014. “Robustness of attack-resilient state estimators”. In:
International Conference on Cyber-Physical Systems. ACM/IEEE.
163–174.
Paoletti, S., A. L. Juloski, G. Ferrari-Trecate, and R. Vidal. 2007.
“Identification of hybrid systems a tutorial”. European Journal of
Control. 13(2-3): 242–260.
Parolini, L., B. Sinopoli, B. H. Krogh, and Z. Wang. 2012. “A Cyber-
Physical Systems Approach to Data Center Modeling and Control
for Energy Efficiency”. Proceedings of the IEEE. 100(1): 254–268.
Parolini, L. 2012. “Models and Control Strategies for Data Center
Energy Efficiency”. PhD thesis. Carnegie Mellon University.
Pasqualetti, F., A. Bicchi, and F. Bullo. 2012. “Consensus Computation
in Unreliable Networks: A System Theoretic Approach”. IEEE
Transactions on Automatic Control. 57(1): 90–104.
Pasqualetti, F., F. Dorfler, and F. Bullo. 2015. “Control-theoretic meth-
ods for cyberphysical security: Geometric principles for optimal
cross-layer resilient control systems”. IEEE Control Systems. 35(1):
110–127.
Pasqualetti, F., F. Dörfler, and F. Bullo. 2013. “Attack detection and
identification in cyber-physical systems”. IEEE Transactions on
Automatic Control. 58(11): 2715–2729.
Peeters, B. and G. De Roeck. 1999. “Reference-based stochastic subspace
identification for output-only modal analysis”. Mechanical systems
and signal processing. 13(6): 855–878.
Peng, T., C. Leckie, and K. Ramamohanarao. 2007. “Survey of network-
based defense mechanisms countering the DoS and DDoS problems”.
ACM Computing Surveys (CSUR). 39(1): 3.
Pin Calmon, F. du and N. Fawaz. 2012. “Privacy against statistical
inference”. In: Allerton Conference on Communication, Control,
and Computing. IEEE. 1401–1408.
246 References

Poovendran, R. 2010. “Cyber-Physical Systems: Close Encounters Be-

tween Two Parallel Worlds [Point of View]”. Proceedings of the IEEE.
98(8): 1363–1366.
Pultarova, T. 2016. “Cyber security-Ukraine grid hack is wake-up call
for network operators [News Briefing]”. Engineering & Technology.
11(1): 12–13.
Qu, F., F. Y. Wang, and L. Yang. 2010. “Intelligent transportation
spaces: vehicles, traffic, communications, and beyond”. IEEE Com-
munications Magazine. 48(11): 136–142.
Rajkumar, R. R., I. Lee, L. Sha, and J. Stankovic. 2010. “Cyber-
physical systems: the next computing revolution”. In: ACM Design
Automation Conference. 731–736.
Rigtorp, E. 2010. “Sensor Selection with Correlated Noise”. PhD thesis.
KTH Royal Institute of Technology.
Rivest, R. L., A. Shamir, and L. Adleman. 1978. “A method for obtaining
digital signatures and public-key cryptosystems”. Communications
of the ACM. 21(2): 120–126.
Rottondi, C. and G. Verticale. 2016. “Enabling privacy in a gaming
framework for smart electricity and water grids”. In: IEEE In-
ternational Workshop on Cyber-physical Systems for Smart Water
Networks. 25–30.
Ruan, M., H. Gao, and Y. Wang. 2019. “Secure and Privacy-Preserving
Consensus”. IEEE Transactions on Automatic Control.
Rubio-Hernan, J., L. De Cicco, and J. Garcia-Alfaro. 2017. “On the
use of watermark-based schemes to detect cyber-physical attacks”.
EURASIP Journal on Information Security. 2017(1).
Sampigethaya, K. and R. Poovendran. 2013. “Aviation Cyber-Physical
Systems: Foundations for Future Aircraft and Air Transport”. Pro-
ceedings of the IEEE. 101(8): 1834–1855.
Satchidanandan, B. and P. Kumar. 2017. “Dynamic Watermarking:
Active Defense of Networked Cyber–Physical Systems”. Proceedings
of the IEEE. 105(2): 219–240.
Schaeffer, S. E. 2007. “Survey: Graph Clustering”. Comput. Sci. Rev.
1(1): 27–64.
References 247

Schenato, L. 2008. “Optimal Estimation in Networked Control Systems

Subject to Random Delay and Packet Drop”. IEEE Transactions
on Automatic Control. 53(5): 1311–1317.
Schenato, L., B. Sinopoli, M. Franceschetti, K. Poolla, and S. S. Sastry.
2007. “Foundations of control and estimation over lossy networks”.
Proceedings of the IEEE. 95(1): 163–187.
Schirner, G., D. Erdogmus, K. Chowdhury, and T. Padir. 2013. “The
Future of Human-in-the-Loop Cyber-Physical Systems”. Computer.
46(1): 36–45.
Seyboth, G. S., D. V. Dimarogonas, and K. H. Johansson. 2013. “Event-
based broadcasting for multi-agent average consensus”. Automatica.
49(1): 245–252.
Shamaiah, M., S. Banerjee, and H. Vikalo. 2010. “Greedy sensor selec-
tion: Leveraging submodularity”. In: Conference on Decision and
Control. IEEE. 2572–2577.
Sharma, A. B., F. Ivančić, A. Niculescu-Mizil, H. Chen, and G. Jiang.
2014. “Modeling and analytics for cyber-physical systems in the age
of big data”. ACM SIGMETRICS Performance Evaluation Review.
41(4): 74–77.
Shi, E. and A. Perrig. 2004. “Designing secure sensor networks”. IEEE
Wireless Communications. 11(6): 38–43.
Shi, L., L. Xie, and R. M. Murray. 2009. “Kalman filtering over a
packet-delaying network: A probabilistic approach”. Automatica.
45(9): 2134–2140.
Shi, L. and H. Zhang. 2012. “Scheduling Two Gauss – Markov Systems:
An Optimal Solution for Remote State Estimation Under Bandwidth
Constraint”. IEEE Transactions on Signal Processing. 60(4): 2038–
2042.
Shoukry, Y., K. Gatsis, A. Alanwar, G. J. Pappas, S. A. Seshia, M.
Srivastava, and P. Tabuada. 2016. “Privacy-aware quadratic opti-
mization using partially homomorphic encryption”. In: Conference
on Decision and Control. IEEE. IEEE. 5053–5058.
Shoukry, Y., P. Nuzzo, A. Puggelli, A. L. Sangiovanni-Vincentelli, S. A.
Seshia, and P. Tabuada. 2017. “Secure state estimation for cyber
physical systems under sensor attacks: a satisfiability modulo theory
approach”. IEEE Transactions on Automatic Control.
248 References

Shoukry, Y. and P. Tabuada. 2016. “Event-triggered state observers

for sparse sensor noise/attacks”. IEEE Transactions on Automatic
Control. 61(8): 2079–2091.
Sinopoli, B., L. Schenato, M. Franceschetti, K. Poolla, M. I. Jordan, and
S. S. Sastry. 2004. “Kalman filtering with intermittent observations”.
IEEE Transactions on Automatic Control. 49(9): 1453–1464.
Slavakis, K., G. B. Giannakis, and G. Mateos. 2014. “Modeling and
optimization for big data analytics:(statistical) learning tools for
our era of data deluge”. IEEE Signal Processing Magazine. 31(5):
18–31.
Slay, J. and M. Miller. 2007. “Lessons learned from the maroochy water
breach”. Critical infrastructure protection: 73–82.
Smith, G. 2009. “On the foundations of quantitative information flow”.
In: International Conference on Foundations of Software Science
and Computational Structures. Springer. 288–302.
Smith, R. S. 2015. “Covert misappropriation of networked control
systems: Presenting a feedback structure”. IEEE Control Systems.
35(1): 82–92.
Sui, T., K. You, M. Fu, and D. Marelli. 2015. “Stability of MMSE state
estimators over lossy networks using linear coding”. Automatica.
51(Jan.): 167–174.
Sun, M. and W. P. Tay. 2017. “Inference and data privacy in IoT
networks”. In: IEEE International Workshop on Signal Processing
Advances in Wireless Communications. 1–5.
Sundaram, S. and C. Hadjicostis. 2011. “Distributed Function Calcu-
lation via Linear Iterative Strategies in the Presence of Malicious
Agents”. IEEE Transactions on Automatic Control. 56(7): 1495–
1508.
Tabuada, P. 2007. “Event-triggered real-time scheduling of stabilizing
control tasks”. IEEE Transactions on Automatic Control. 52(9):
1680–1685.
Tan, R., V. Badrinath Krishna, D. K. Yau, and Z. Kalbarczyk. 2013.
“Impact of integrity attacks on real-time pricing in smart grids”.
In: Conference on Computer & Communications Security. ACM.
439–450.
References 249

Tariq, M. U., J. Florence, and M. Wolf. 2014. “Design Specification

of Cyber-Physical Systems: Towards a Domain-Specific Modeling
Language based on Simulink, Eclipse Modeling Framework, and
Giotto.” In: ACESMB@ MoDELS. 6–15.
Teixeira, A., D. Pérez, H. Sandberg, and K. H. Johansson. 2012a.
“Attack models and scenarios for networked control systems”. In:
International conference on High Confidence Networked Systems.
ACM. 55–64.
Teixeira, A., I. Shames, H. Sandberg, and K. H. Johansson. 2012b. “Re-
vealing stealthy attacks in control systems”. In: Allerton Conference
on Communication, Control, and Computing. IEEE. 1806–1813.
Teixeira, A., I. Shames, H. Sandberg, and K. H. Johansson. 2015. “A se-
cure control framework for resource-limited adversaries”. Automatica.
51: 135–148.
Trentelman, H., A. A. Stoorvogel, and M. Hautus. 2012. Control theory
for linear systems. Springer Science & Business Media.
Tsatsanis, M. K. and G. B. Giannakis. 1993. “Time-varying system iden-
tification and model validation using wavelets”. IEEE Transactions
on Signal Processing. 41(12): 3512–3523.
Van Der Schaft, A. J. and J. M. Schumacher. 2000. An introduction to
hybrid dynamical systems. Vol. 251.
Van der Woude, J. 1999. “The generic number of invariant zeros of a
structured linear system”. SIAM Journal on Control and Optimiza-
tion. 38(1): 1–21.
Van der Woude, J. 1991. “A graph-theoretic characterization for the
rank of the transfer matrix of a structured system”. Mathematics of
Control, Signals, and Systems (MCSS). 4(1): 33–40.
Van Trees, H. L. 1968. Detection Estimation and Modulation Theory.
Vol. 1. New York: Wiley.
Volpano, D., C. Irvine, and G. Smith. 1996. “A sound type system for
secure flow analysis”. Journal of Computer Security. 4(2-3): 167–187.
Wahlberg, B., H. Hjalmarsson, and M. Annergren. 2010. “On optimal
input design in system identification for control”. In: Conference on
Decision and Control. IEEE. 5548–5553.
250 References

Wang, W., L. Ying, and J. Zhang. 2016. “On the Relation Between Iden-
tifiability, Differential Privacy, and Mutual-Information Privacy”.
IEEE Transactions on Information Theory. 62(9): 5018–5029.
Weerakkody, S., X. Liu, and B. Sinopoli. 2017a. “Robust Structural
Analysis and Design of Distributed Control Systems to Prevent Zero
Dynamics Attacks”. In: Conference on Decision and Control. IEEE.
Weerakkody, S., X. Liu, S. H. Son, and B. Sinopoli. 2016a. “A graph
theoretic characterization of perfect attackability and detection in
Distributed Control Systems”. In: American Control Conference.
IEEE. 1171–1178.
Weerakkody, S., X. Liu, S. H. Son, and B. Sinopoli. 2017b. “A Graph
Theoretic Characterization of Perfect Attackability for Secure Design
of Distributed Control Systems”. IEEE Transactions on Control of
Network Systems. 4(1): 60–70.
Weerakkody, S., Y. Mo, and B. Sinopoli. 2014. “Detecting Integrity
Attacks on Control Systems using Robust Physical Watermarking”.
In: Conference on Decision and Control. IEEE. 3757–3764.
Weerakkody, S., Y. Mo, B. Sinopoli, D. Han, and L. Shi. 2016b. “Multi-
Sensor Scheduling for State Estimation With Event-Based, Stochas-
tic Triggers”. IEEE Transactions on Automatic Control. 61(9): 2695–
2701.
Weerakkody, S., O. Ozel, P. Griffioen, and B. Sinopoli. 2017c. “Active
detection for exposing intelligent attacks in control systems”. In:
Conference on Control Technology and Applications. IEEE. 1306–
1312.
Weerakkody, S., O. Ozel, and B. Sinopoli. 2017d. “A Bernoulli-Gaussian
watermark design for detecting integrity attacks in control systems”.
In: Allerton Conference on Communication, Control and Computing.
IEEE.
Weerakkody, S. and B. Sinopoli. 2015. “Detecting integrity attacks on
control systems using a moving target approach”. In: Conference on
Decision and Control. IEEE. 5820–5826.
Weerakkody, S. and B. Sinopoli. 2016. “A moving target approach
for identifying malicious sensors in control systems”. In: Allerton
Conference on Communication, Control, and Computing. IEEE.
1149–1156.
References 251

Weerakkody, S., B. Sinopoli, S. Kar, and A. Datta. 2016c. “Information

flow for security in control systems”. In: Conference on Decision
and Control. IEEE. 5065–5072.
Wolcott, R. W. and R. M. Eustice. 2014. “Visual localization within
LIDAR maps for automated urban driving”. In: International Con-
ference on Intelligent Robots and Systems. IEEE. 176–183.
Work, D. and A. Bayen. 2008. “Impacts of the mobile internet on trans-
portation cyberphysical systems: traffic monitoring using smart-
phones”. In: National Workshop for Research on High-Confidence
Transportation Cyber-Physical Systems: Automotive, Aviation, &
Rail. 18–20.
Work, D., A. Bayen, and Q. Jacobson. 2008. “Automotive cyber physical
systems in the context of human mobility”. In: National Workshop
for Research on High-Confidence Transportation Cyber-Physical
Systems: Automotive, Aviation, & Rail. 3–4.
Wu, D. and C. Zhou. 2011. “Fault-tolerant and scalable key management
for smart grid”. IEEE Transactions on Smart Grid. 2(2): 375–381.
Wu, J., Q.-S. Jia, K. H. Johansson, and L. Shi. 2013. “Event-Based
Sensor Data Scheduling: Trade-Off Between Communication Rate
and Estimation Quality”. IEEE Transactions on Automatic Control.
58(4): 1041–1046.
Xie, L., Y. Mo, and B. Sinopoli. 2010. “False data injection attacks in
electricity markets”. In: International Conference on Smart Grid
Communications. IEEE. 226–231.
Xu, Y. and J. P. Hespanha. 2005. “Estimation under uncontrolled
and controlled communications in networked control systems”. In:
Conference on Decision and Control jointly held with the European
Control Conference. IEEE. 842–847.
Yong, S. Z., M. Zhu, and E. Frazzoli. 2015. “Resilient state estimation
against switching attacks on stochastic cyber-physical systems”. In:
Conference on Decision and Control. IEEE. 5162–5169.
Yuan, Y. and Y. Mo. 2015. “Security in cyber-physical systems: Con-
troller design against known-plaintext attack”. In: Conference on
Decision and Control. IEEE. 5814–5819.
252 References

Yuan, Y., Q. Zhu, F. Sun, Q. Wang, and T. Başar. 2013. “Resilient

control of cyber-physical systems against denial-of-service attacks”.
In: International Symposium on Resilient Control Systems. IEEE.
54–59.
Zhang, R. and P. Venkitasubramaniam. 2016. “Stealthy control signal
attacks in vector LQG systems”. In: American Control Conference.
IEEE. 1179–1184.
Zheng, Y., O. Ozdemir, R. Niu, and P. K. Varshney. 2012. “New
Conditional Posterior Cramér - Rao Lower Bounds for Nonlinear
Sequential Bayesian Estimation”. IEEE Transactions on Signal
Processing. 60(10): 5549–5556.
Zhu, M. and S. Martinez. 2014. “On the performance analysis of resilient
networked control systems under replay attacks”. IEEE Transactions
on Automatic Control. 59(3): 804–808.
Zuo, L., R. Niu, and P. K. Varshney. 2011. “Conditional Posterior
Cramér - Rao Lower Bounds for Nonlinear Sequential Bayesian
Estimation”. IEEE Transactions on Signal Processing. 59(1): 1–14.