Received March 25, 2022, accepted April 28, 2022, date of publication May 2, 2022, date of current version

May 9, 2022.
Digital Object Identifier 10.1109/ACCESS.2022.3171922

Binary Exploitation in Industrial Control Systems:

Past, Present and Future
QI LIU , KAIBIN BAO , AND VEIT HAGENMEYER
Institute for Automation and Applied Informatics, Karlsruhe Institute of Technology, 76344 Eggenstein-Leopoldshafen, Germany
Corresponding author: Qi Liu ([email protected])
This work was supported by funding of the Helmholtz Association (HGF) through the Energy System Design (ESD) program.
We also acknowledge support by the KIT-Publication Fund of the Karlsruhe Institute of Technology.

ABSTRACT Despite being a decades-old problem, binary exploitation still remains a serious issue in
computer security. It is mainly due to the prevalence of memory corruption errors in programs written with
notoriously unsafe but yet indispensable programming languages like C and C++. For the past 30 years, the
nip-and-tuck battle in memory between attackers and defenders has been getting more technical, versatile,
and automated. With raised bar for exploitation in common information technology (IT) systems owing to
hardened mitigation techniques, and with unintentionally opened doors into industrial control systems (ICS)
due to the proliferation of industrial internet of things (IIoT), we argue that we will see an increased number
of cyber attacks leveraging binary exploitation on ICS in the near future. However, while this topic generates
a very rich and abundant body of research in common IT systems, there is a lack of systematic study targeting
this topic in ICS. The present work aims at filling this gap and serves as a comprehensive walkthrough of
binary exploitation in ICS. Apart from providing an analysis of the past cyber attacks leveraging binary
exploitation on ICS and the ongoing attack surface transition, we give a review of the attack techniques and
mitigation techniques on both general-purpose computers and embedded devices. At the end, we conclude
this work by stressing the importance of network-based intrusion detection, considering the dominance of
resource-constrained real-time embedded devices, low-end embedded devices in ICS, and the limited ability
to deploy arbitrary defense mechanism directly on these devices.

INDEX TERMS Binary exploitation, industrial control systems, cyber incidents, attack and defense.

I. INTRODUCTION Windows operating systems (OS), and mostly in common

The recently increased prevalence of headline-making ran- information technology (IT) systems. However, the target
somware attacks on industrial control systems (ICS) is partic- of binary exploitation is about to gradually shift to ICS
ularly worrisome for the society. These attacks often leverage as a result of various factors. Firstly, with hardened miti-
program exploits as a stepping stone for getting the initial gation techniques in up-to-date commodity OS, successful
access. A remote exploit is perhaps the infection method binary exploitation is still possible, but becomes much harder.
hardest to develop and hardest to prevent, since it normally Secondly, the recent rapid integration of industrial internet of
does not need any human interaction, unlike other social things (IIoT) unintentionally increases the number of entry
engineering techniques including phishing emails. Binary points into ICS. Thirdly, many embedded devices are not only
exploitation has long been a major cybersecurity issue and lack of dedicated mitigation techniques because of intolerable
continues to pose a serious security threat to individuals performance overhead and/or unfulfilled hardware require-
and organizations around the world. Although a significant ments, but also inherently insecure due to vendors’ unjusti-
amount of efforts are dedicated to it, defeating and preventing fied prioritization of functionality over security. We argue that
successful binary exploitation remains an open problem. this will inevitably make attackers go for easier wins.
Historically, the main target of binary exploitation has been Apart from human factors, in general, security concepts in
devices running on Intel x86 architecture, predominantly ICS can be divided into a few classes: secure end devices,
secure communication protocols, and secure network archi-
The associate editor coordinating the review of this manuscript and tectures and configuration. In order to really thwart malicious
approving it for publication was Weizhi Meng . actors’ attempts to break into a system, all these three security

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
48242 VOLUME 10, 2022
Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

requirements must be guaranteed at the same time. To reach making it appear to be a topic exclusive for common IT
a secure network architecture and configuration, there are systems.
many practical recommendations to follow, e.g., [1], [2]. In this work, we analyze binary exploitation in the context
Those recommendations include robust network segmenta- of ICS, which consist of general-purpose computers, embed-
tion, e.g., deploying demilitarized zones (DMZ) and virtual ded automation devices, low-end IoT devices etc. At first,
local area networks (VLAN), strong access control, e.g., we provide a preliminary introduction of binary exploitation
implementing authentication and authorization mechanisms, in Section II, including its causes, types and consequences.
secure remote access, e.g., using virtual private network To show how and where it is used in real-world attacks,
(VPN) or secure shell (SSH). They are easy to implement, we present in Section III a brief overview and analysis of
i.e., even without assistance of any security professionals, major ICS cyber incidents leveraging binary exploitation
and also prove to be very effective especially against not very in the last decade. Then, we argue in Section IV why
skilled attackers. binary exploitation will likely become a more serious prob-
In the past, the lack of these basic security measures gave lem in ICS. Next, we summarize the contemporary miti-
even unskilled attackers an easy chance to break into a system gation techniques on both general-purpose computers and
and cause real damage. With raised security awareness, this embedded devices, respectively, in Section V. Lastly, we dis-
weakest link has been largely reinforced, making it much cuss heuristics-based detection and remote runtime attesta-
harder to attack an organization. However, a very skilled tion techniques in Section VI, and conclude this work in
attacker (group) can still break into a system, often by finding Section VII.
one or several software vulnerabilities and developing (zero-
day) exploits accordingly. This brings us back to the first two II. BINARY EXPLOITATION–A PRIMER
security concepts, i.e., secure end devices and secure com- Although memory corruption bugs in general become harder
munication protocols, and stresses their importance. Because to spot and much harder to exploit in modern software includ-
a vulnerability normally results from either design errors ing operating systems, there have always been examples
or implementation errors of the implemented OS, firmware, showing that binary exploitation is still possible in practice,
application software or communication protocols. Speaking and often easier than perceived by many security defenders.
of exploiting a communication protocol vulnerability, what It has been long the case that every time security experts
is really exploited during an attack and the direct target for claimed that with the newly designed mitigation technique,
an attacker is the vulnerable program that implements this binary exploitation is (almost) not possible anymore, and
protocol and runs on an end device. Hence, we can say that the then found out that their assertions are falsified by subse-
most important point of secure end devices and secure com- quent attacks [3]–[5]. The corresponding exploitation tech-
munication protocols is development and implementation of niques are often demonstrated and discussed by offensive
secure programs. security researchers in technical security conferences like
As it shows in the later sections of this article, in the early Black Hat1 or periodicals like Phrack Magazine.2 For exam-
days, i.e., before 2010, ICS were easily exploitable largely ple, as Intel presented its hardware-assisted Control-flow
due to terribly insecure network architectures and configura- Enforcement Technology (CET) [6] as a solution to prevent
tion, and of course, the presence of vulnerable programs was binary exploitation, it was probably believed that the problem
also critical. By analyzing the major cyber incidents in ICS is solved, only to find out there are still ways to achieve
in the last decade, we see that the sophistication of attacks control-flow hijacking in the presence of this modern mitiga-
increased very much, and threat actors seem to be willing tion technique, as it was demonstrated by security researchers
to invest lots of efforts to find software vulnerabilities and from the security company McAfee [7].
develop exploits and malwares. Undoubtedly, memory corruption exploits remain a clear,
Hence, we argue that both writing secure programs, present and persistent danger to modern software, as they are
and deploying program exploitation countermeasures, are still used as an entry point for the most recent and advanced
very important topics in ICS. When developing large pro- attacks [5]. Modern software written in unsafe programming
grams, oftentimes we can hardly verify that the programs languages like C and C++ is particularly concerning due to
are absolutely secure, which makes binary exploitation the requirement of manual memory management, which is
countermeasures become a crucially important second a very challenging task especially for large programs with
defense line. However, while this topic generate a very rich millions of lines of code. Any mistake in handling mem-
and abundant body of research in common IT systems, ory buffers could lead to a memory corruption vulnerability,
it is not well addressed in ICS. Instead, most studies on which consequently gives an attacker an opportunity to access
ICS defense strategies still either focus on how to prop- the otherwise inaccessible memory location of a vulnerable
erly isolate and configure networks, e.g., prevent network program. Besides, even memory-safe languages like Java rely
traffic from untrusted sources, or mainly tackle situations on virtual machines that are in turn implemented in C/C++
in which it is assumed that attackers already took foothold
in an ICS network, e.g., behavior-based intrusion detec- 1 https://www.blackhat.com/briefings/
tion. The initial infection methods are often not discussed, 2 http://phrack.org

VOLUME 10, 2022 48243

 Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

for performance reasons. As modern software becomes more used for a deleted object, but now may contain another (unre-
complex, it is less likely to see the end of memory errors in lated) object.
the near future [8]. It is worth noting that a single memory error can have
Moreover, in the world of embedded devices, memory cascade effect, i.e., invoking many more other memory
errors in programs and examples of binary exploitation are errors. Memory corruption errors should be prevented by
also observed in the wild. That is to say, except from enforcing both spatial and temporal memory safety policies.
x86-based platforms, a variety of other hardware platforms In memory-safe languages, spatial and temporal memory
can also be targeted by attackers, e.g., ARM [9], [10], safety are ensured by automatic bounds-checking of objects
MIPS [11], [12], PowerPC [13], [14], Atmel AVR [15], during memory access and garbage collection, respectively.
[16], SPARC [17], [18], Zilog Z80 [19], SuperH [20]. The
prevalence of binary exploitation in ICS will likely continue 2) SOFTWARE DEVELOPERS – THE UNINTENDED TRIGGER
to increase with accelerated integration of IIoT devices and Despite the above-mentioned memory safety issue, C and
augmented connectivity, and become a more serious problem. C++ still remain popular among many software develop-
Compared to general-purpose computers, embedded systems ers mainly for performance reasons. Programming in these
are way less protected, e.g., because of a lack of hard- memory-unsafe languages will often inevitably result in pro-
ware support required for modern defense techniques. Hence, gram bugs, especially when writing complex programs. It is
it can be confidently assumed that attackers will go for easier arguably nearly impossible to write memory-safe code using
wins by focusing more on embedded systems in ICS. C and C++ at scale due to the need of burdensome man-
In order to help to understand the attack techniques and ual memory management, meaning that software written in
defense strategies discussed in the following sections, a pre- unsafe languages is inherently error-prone.
liminary introduction of binary exploitation is given in this When writing software, software developers may often
section, including its causes, types and consequences. implicitly or explicitly make questionable assumptions. Both
design flaws and implementation flaws can appear in buggy
A. CAUSES OF BINARY EXPLOITATION
programs written by software developers who are often
The causes of a program being exploitable can be interpreted not security experts. The aforementioned invalid pointers
in several ways, and understood from different angles. can be easily produced through mistakes like incautious
pointer arithmetic. Historically, performance was probably
1) MEMORY UNSAFETY – A DANGEROUS CONDITION
the only key metric for many software programmers, mostly
A lack of memory safety in programming languages like C unconcerned about security. Today, the situation has greatly
and C++ is arguably the root cause of exploitable vulnerabil- improved, with raised security awareness in general, pro-
ities in software, and is deemed as the foundation of numerous posed secure coding guidelines, compilers-enforced safety
attack vectors. The need for manual memory management checks, and intense code review etc. However, some flaws
makes it appear to be a never-ending problem, in particular, still stay unnoticed, and some attack surface remains.
as modern software written in C and C++ becomes more
complex. Memory safety is a program property that guaran-
3) PROGRAM BUGS – THE OBSERVED AVALANCHE
tees objects in memory can only be accessed with the corre-
sponding capabilities, i.e., well-defined memory access [8]. We categorize frequently encountered program bugs into
That is to say, memory pointers should always point to valid two classes, namely spatial safety-related bugs and tempo-
allocated memory of the correct size and type, to prevent ral safety-related bugs. Whereas spatial safety-related bugs
unintended and/or undefined behavior. arise with a breach of memory objects’ boundaries, temporal
Memory safety includes spatial memory safety, temporal safety-related bugs occur when memory objects are accessed
memory safety, and type safety. They ensure that pointer at the wrong time due to incorrectly tracked memory usage.
dereferences are restricted to data inside the corresponding An avalanche of both kinds of exploitable bugs can originate
memory object, a pointer can only reference a currently from using memory-unsafe languages.
allocated memory object, and only pointers with the correct
type can access related memory objects, respectively [8]. Nor- a: SPATIAL SAFETY-RELATED BUGS
mally, an exploit starts by triggering a memory error, in which i) BUFFER OVERFLOW AND UNDERFLOW
it first makes a pointer invalid and then dereferences this As an instantiation of out-of-bounds write, a buffer overflow
pointer. A pointer becomes invalid, if it goes out of the bounds occurs when an input is allowed to be written beyond the
of its pointed object, or its pointed object is deleted, which is boundaries of an allocated buffer during program execution,
known as out-of-bounds pointer or dangling pointer, respec- which corrupts the data of adjacent objects. Buffer overflow
tively. While dereferencing an out-of-bounds pointer raises a bugs range from the classic stack-based buffer overflow bugs,
spatial memory error, dereferencing a dangling pointer results i.e., allowing copying a user-supplied over-sized input into a
in a temporal memory error [4]. That is to say, the first case fixed-size buffer on the stack, to more recent and complicated
makes accessing neighboring memory objects possible, and heap-based buffer overflow bugs. In fact, today, heap-based
the second case allows accessing memory location that was buffer overflows are more dominantly exploited, and they are

48244 VOLUME 10, 2022

Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

exploitable also due to intermingled user data and control data input potentially corrupt the stack base pointer close to the
in the heap as in the stack [21]. Whereas stack-based over- allocated buffer and hence creating an exploitable condition.
flows are often used to overwrite function return addresses,
stack pointers, or stack base pointers, heap-based overflows v) TYPE CONFUSION
are often used to overwrite function pointers [22], [23]. A type confusion bug [30], or type violation, appears
Another instantiation of out-of-bounds write is buffer when resources, e.g., objects in memory, are accessed using
underflows [24], a somewhat unfairly underestimated pro- an incompatible type. That is, during program execution,
gram bug due to too much attention given to buffer over- an object is first initialized using one type, but then accessed
flows. Whereas buffer overflows occur when memory access using another type. Type confusion often arises due to incor-
goes beyond the end of the targeted memory object, buffer rect downcasting of a base object to a subtype without proper
underflows happen if the boundary at the object’s beginning checking, in particular in C++ programs with complex inher-
is breached. itance hierarchies. This can result in logical errors, because
the object with a wrong type does not have the expected prop-
ii) BUFFER OVERREAD AND UNDERREAD
erties when accessed, and can be misinterpreted. Ultimately,
As the incarnation of out-of-bounds read, both buffer over- it could lead to out-of-bounds memory access, if the allocated
reads [25] and buffer underreads [26] can help attackers to buffer is smaller than an object with the correct type, or if
reveal sensitive information like cryptographic keys or mem- any data is misinterpreted as a pointer. Note that type-safe
ory addresses to bypass security mechanisms, e.g., address programming languages are practically also memory safe due
space layout randomization (ASLR), which will be discussed to static or dynamic error checks, i.e., not only spatial safety
in Section V. Similar to out-of-bounds writes, these bugs are errors but also temporal safety errors can be avoided.
typically caused by excessively incrementing or decrement-
ing an array pointer in a loop to a memory location after or
vi) FORMAT STRING BUG
before the valid buffer, respectively, or by erroneous pointer
arithmetic leading to a position beyond the bounds of the A format string bug [31] exists, when an externally submit-
buffer. ted input string can be interpreted as a command by some
functions like printf() that perform formatting and accept
iii) INTEGER OVERFLOW AND UNDERFLOW a variable number of arguments. As with many other bugs,
Once the classic buffer overflows were largely avoided, inte- format string bugs are exploitable due to directly accepting
ger overflows and underflows have become a serious security user-supplied inputs without validation or sanitization. A pro-
problem, and is frequently involved in binary exploita- grammer may omit a format string specifier, such as %s,
tion [21]. An integer overflow appears when an integer when using format functions in a program. An attacker can
arithmetic operation results in a value that is too large to exploit this weakness by inserting multiple format string
be stored in the declared variable, triggering a wraparound specifiers into a single input string, and supply it to a format
and making the value become very small or negative. On the function to subvert its normal behavior. When this input string
contrary, an integer underflow can turn a very small number is taken as an argument and parsed by a format function,
into a very large number by producing a value that is smaller it may mistakenly interpret this argument as instructions,
than the minimum allowable integer value during integer sub- ultimately leading to a so-called write-what-where condition,
traction. Both integer overflows and integer underflows can in which the attacker has the ability to write an arbitrary value
be very dangerous, especially when they can be invoked by to an arbitrary location. Fortunately, format string bugs are
user-supplied inputs, and attackers exploit the math involved now rare thanks to easy discovery and elimination, hence no
in resource management like calculating memory allocation longer pose a serious threat to security.
sizes. A buffer overflow can be introduced by an integer
overflow [27] or an integer underflow [28] during arithmetic b: TEMPORAL SAFETY-RELATED BUGS
operations of an array index, in which it causes less mem- i) USE-AFTER-FREE
ory or much more memory to be allocated than expected, The most common and infamous temporal safety-related bugs
respectively. are use-after-free bugs, in which a previously freed memory
location is accessed using a dangling pointer, i.e., a pointer
iv) OFF-BY-ONE ERROR that points to a deallocated object. This can happen due to
An off-by-one bug [29] is introduced when a maximum or some error conditions and/or when it is unclear which part
minimum value is incorrectly calculated to be one more of a program is responsible for freeing the memory loca-
or one less than the correct value. This often occurs when tion [32]. Sometimes, attackers can make a pointer become a
programmers fail to take into account that an array index dangling pointer by exploiting an incorrect exception handler,
starts at zero, making a loop operation iterate one more time which deletes an object, but does not reinitialize the corre-
than it should. Or they forget that a terminating null byte is sponding pointer [4]. Depending on the implementation of
automatically appended to a string variable taken as input memory management system and the program code, the con-
by some C library functions like strncat(), making the sequences of a use-after-free bug vary from data corruption to

VOLUME 10, 2022 48245

 Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

arbitrary code execution. After identifying or invoking a use- of the free() function. When the memory management
after-free bug, to exploit it, attackers could reallocate the freed functions are mismatched, the consequences can be as prob-
memory position to themselves and insert a crafted object lematic as those of above-mentioned temporal safety-related
into it, before this memory position will be accessed through bugs [38].
the dangling pointer. Later, the inserted object is accessed
through the dangling pointer, tricking the program into exe-
v) USE OF UNINITIALIZED VARIABLE
cuting attackers-specified code. Note that dangling pointers Use of uninitialized variables [39], or use of uninitialized
can point to not only heap but also other memory regions resources [40], may produce program bugs that cause unde-
like stack, but heap-based attacks are the most concerning sired effects. This happens either when a variable is accessed
ones [33]. before being assigned with a value, or when a variable is
accessed before it should be reinitialized. If a variable is
ii) DOUBLE FREE
accessed, but not assigned previously, it can result in program
Considered as a special case of use-after-free bugs, double- crash or invalid memory access. If a variable is accessed
free bugs appear in programs in which a previously freed prior to a necessary reinitialization, it can sometimes just
memory location is mistakenly freed again, instead of being return junk data, or, in other circumstances, leak sensitive
accessed using a dangling pointer. This potentially leads information.
to prematurely freeing a new object, or even worse, cor-
ruption of the program’s memory management data struc-
tures. To reliably exploit it, sometimes attackers leverage vi) WILD POINTER DEREFERENCE
a combination of double-free bugs and use-after-free bugs, Not only is use of uninitialized variables bad, but also
such as [34]. Like use-after-free bugs, double-free bugs use/dereference of uninitialized pointers. When pointers are
are also easy to create but difficult to spot during testing, created without necessary initialization, they are called wild
because, on the one hand, the first free() operation and pointers, because they simply point to arbitrary memory
the second free() operation or the dangling pointer deref- location, which can be valid or invalid. In many C and
erence are two separate events that may occur far apart in C++ programs, pointers are often declared as a wild pointer.
time and/or be invoked by code far apart in program space, If a wild pointer pointing to an invalid memory location is
and, on the other hand, the second free() operation or dereferenced, it could result in program crash. If the wild
the dangling pointer dereference may not appear in every pointer happens to point to the start of an arbitrary function
program execution [33], [35]. in memory, i.e., acting like a valid function pointer, this
function may get executed. If attackers can somehow access
iii) INVALID FREE the memory location to which the wild pointer points, this
Similar to double-free bugs, a free() operation can be may result in arbitrary code execution, making it particularly
wrongly used to introduce invalid-free bugs when trying to dangerous [41].
free an object in memory. But instead of freeing a memory
object twice, a vulnerable program with invalid-free bugs vii) Null POINTER DEREFERENCE
calls the free() function to free an object on the heap with To avoid wild pointers, programmers may initialize pointers
a pointer that does not point to the start of the object due to NULL when not possible to assign a more appropriate
to erroneous pointer arithmetic [36]. Or it frees an object value at the point of declaration. However, if a pointer is
not allocated on the heap at all, i.e., not allocated using initialized to NULL and dereferenced prior to being assigned
heap memory allocation functions like malloc(), but rather a proper value, it also introduces a bug. Whereas a wild
automatically allocated on the stack as a local variable or pointer has an undefined value, a NULL pointer keeps an
allocated on the data segment as a global variable [37]. implementation-dependent defined value, which is guaran-
Depending on the implementation of the free() function, teed not to be a valid memory address. Dereferencing a
the consequences range from simply crashing a program to NULL pointer means trying to access a memory location
corrupting the program’s memory management data struc- that does not exist, typically resulting in a crash or exit. This
tures, ultimately allowing attackers to modify critical pro- may be less harmful than dereferencing a wild pointer, and
gram variables and gain arbitrary memory access abilities. such a bug is easier to spot during testing. However, NULL
pointer dereference occurs sometimes due to race condition
iv) MISMATCHED FREE or other rarely encountered error conditions, making it less
When trying to return memory resource to the system, easier to detect [42]. Sometimes, attackers may leverage
it can go wrong not only by erroneous handling of the intentionally triggered NULL pointer dereference to bypass
free() function as in invalid-free bugs, but also by using security mechanisms. Under exceptional circumstances,
an incompatible memory deallocation function, producing malicious code execution may also be possible, if NULL
a mismatched-free bug. For instance, in a C++ program, pointers are defined to hold the 0 × 0 memory address,
a memory buffer is first allocated with the malloc() func- and attackers happen to be able to trick the kernel into
tion, but then released using the delete operation, instead accessing it.

48246 VOLUME 10, 2022

Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

viii) RACE CONDITION to execute attackers-injected code [49], attackers-selected

Another kind of bug arising only due to memory access at library functions [51], or attackers-selected small code snip-
a bad time is race condition bugs, which exist in different pets [52]. In data-oriented exploitation, attackers intend to
concrete forms with varying reasons, such as while using tamper with non-control but security-critical data, such as
signal handlers [43], in the middle of a switch statement [44], user ID, to bypass access control mechanisms and grant the
between time-of-check and time-of-use (TOCTOU) of a root privilege [50], to leak sensitive information [53], or to
resource [45], and during context switching [46]. A race con- indirectly influence the control flow [54], without even violat-
dition appears when two different execution contexts are able ing control-flow integrity polices in place. While this section
to affect a shared resource without proper synchronization, also briefly mentions some defense methods, more detailed
and hence interfere with each other. It is often caused by information about defense methods is provided in Section V
programmers thinking that system calls will execute atomi- and Section VI.
cally. But most system calls end up executing a large number
of instructions and hence having longer than expected exe- 1) CONTROL-ORIENTED EXPLOITATION
cution time, letting another concurrently running thread or The first known type of binary exploitation is control-
process have a chance to intervene [21]. A race condition oriented, which is still the one receiving most of the atten-
can have security implications, in particular when it takes tion in attacks exploiting memory bugs today. Beginning
place during execution of security-relevant code. It may be with the stack-smashing attacks [49], the early generation
exploited by attackers to corrupt or modify important state of control-oriented exploitation centered on code-injection
information, potentially leading to program crash or access attacks. Only few years later, the focus shifted to code-reuse
to resources otherwise unavailable to unauthenticated and attacks with a number of increasingly sophisticated variants
unauthorized users. Besides, in some cases, race condition surfacing one after another. In any of these cases, in order
bugs are the cause of other temporal safety-related bugs like to successfully exploit a program bug, an attacker needs
use-after-free, double-free, and NULL pointer dereference to be able to first inject code into a running program or
bugs. carefully pre-select existing code from that program, then
According to the MITRE CWE ranking [47], out-of- redirect the control flow to the injected or pre-selected code,
bounds writes and out-of-bounds reads bugs are among the hence obtaining the control of the process. The injected
top three most dangerous software weaknesses in 2021, while code or the pre-selected code is simply a user input con-
use-after-free bugs are among the top 10 and NULL pointer taining machine instructions or memory addresses of pre-
dereference bugs are among the top 15. That is to say, spa- selected instructions, respectively. By means of the program
tial safety-related bugs are still the most severe ones, but bug, part of the user input is used to change the target
the focus of memory corruption location has shifted from address of an indirect branch instruction, e.g., ret, jmp,
stack to heap [48]. Besides, it is worth noting that spatial call, essentially controlling which instruction is executed
safety-related bugs and temporal safety-related bugs are not next. Regarding to whether the control-flow diversion is
unrelated to each other, and in fact, temporal safety-related achieved through a return instruction, or an indirect jump
bugs can often result in spatial safety-related issues. For or call instruction, control-oriented exploitation is further
instance, out-of-bounds writes and reads are sometimes categorized into backward-edge control-flow hijacking and
enabled by use-after-free bugs. forward-edge control-flow hijacking, which differ very much
In summary, the fundamental problem is that a lack of in used methods for designing attacks.
memory safety in programming languages allows software
developers to introduce certain types of exploitable program a: BACKWARD-EDGE CONTROL-FLOW HIJACKING
bugs. Apart from invoking undefined behavior, i.e., mostly Backward-edge control-flow hijacking attacks include all
a stability problem, these program bugs can lead to leakage code-injection attacks [49] and some code-reuse attacks, i.e.,
of sensitive information from memory and/or arbitrary code return-into-libc attacks [51], [55], [56] and return-oriented
execution. programming (ROP) attacks [52], [57], due to their great
reliance on the return instruction ret.
B. TYPES OF BINARY EXPLOITATION
By leveraging one or more of the above-mentioned memory i) CODE INJECTION ATTACKS
corruption bugs, attackers can conduct different types of Initially, in order to execute arbitrary code of attackers’
attacks. To alter a running program’s behavior to the benefit choice, they solely relied on injection of simple code. The
of attackers, there are generally two types of exploitation, highly influential paper first to describe stack-smashing
i.e., control-oriented exploitation (also known as control-flow attacks [49] has spurred a considerable amount of research
hijacking [49]) and data-oriented exploitation (also known as both in attacks and in defenses over time. The described
non-control-data attacks [50]). In control-oriented exploita- stack-smashing attacks leverage a classic stack-based buffer
tion, attackers seek to modify control data of a running pro- overflow bug to inject code occupying the target buffer and
gram, i.e., the return address of a function call, or the code its neighboring memory area. The injected code is care-
pointer of an indirect call or jump, to redirect the control-flow fully chosen and arranged, so that part of it overwrites a

VOLUME 10, 2022 48247

 Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

return address, i.e., a saved instruction pointer pointing to the can move the stack pointer to the next stack frame, hence
next to-be-executed instruction, on the stack with a memory chaining multiple functions together [56]. That is to say, the
address of the attacker’s choice. Hence when the vulnerable attacker can first populate the stack with carefully crafted and
function returns, it points to the code injected by the attacker arranged malicious function call frames containing function
rather than returning to the caller that invoked the vulnerable entry points and parameters, then use the stack pointer to
function. The injected code that is executed after control-flow redirect the execution to the next function of the attacker’s
diversion launches a command shell, from which the attacker choosing.
can control the compromised machine, and hence is called However, the second generation of RILC attacks still can
shellcode.3 hardly support conditional branching, an essential operation
Code-injection attacks are possible due to the fact that, for a system to be Turing complete, meaning that they can-
on x86-based platforms with von Neumann architecture, code not freely alter the control flow during program execution.
and data are stored in the same memory space, and x86 Subsequently, the third generation of RILC attacks [56] were
instruction set architecture itself does not distinguish between presented to prove that Turing completeness is achievable
code and data, meaning that the same piece of raw bytes can with RILC attacks, and thereby attackers can perform arbi-
be referred to as instructions or data depending on the context. trary computations in the target program through selected and
arranged function calls. Some commonly available specific
ii) RETURN-INTO-LIBC ATTACKS functions, dubbed widgets, can be chosen and misused to
Not long after code-injection attacks had grown their popu- induce conditional branching and hence arbitrary behavior in
larity, the W⊕X (write xor execute) policy [59] was intro- the target program.
duced and has been widely enforced on modern operating
systems, asserting that a memory page can be either writable iii) RETURN-ORIENTED PROGRAMMING
or executable. By doing so, it marks all data regions, such As the x86-64 architecture becomes more prevalent, RILC
as the stack, as non-executable, and hence effectively thwarts attacks get more difficult to conduct, because most func-
all code-injection attacks. In response, attackers have turned tion arguments are passed into CPU registers instead of
to code-reuse attacks, i.e., reuse of existing legitimate code the stack, due to increased space available in registers in
in a vulnerable program or its linked libraries for malicious the x86-64 architecture. Moreover, removing certain func-
purposes. The first generation and also the simplest form tions from libc, as a defense strategy, may restrict the capa-
of code-reuse attacks are the return-into-libc (RILC) attacks bilities of RILC attacks. To overcome these restrictions,
initially introduced in [51], which leverage functions from the attackers moved to a more general and advanced attack
standard C library libc.4 This library is dynamically linked technique called return-oriented programming (ROP) [52].
to nearly all Unix programs and loaded in their memory ROP extends code-reuse attacks greatly, and the name results
spaces. Thus, instead of injecting malicious code into the from the fact that it is the return instruction triggering the
stack by means of a user input, attackers inject malicious data processor to continue executing what attackers put or spec-
interpreted as code pointers pointing to exported functions in ified in memory, like in code-injection attacks and RILC
libc during execution. attacks. Whereas the building blocks of RILC attacks are
The first generation of RILC attacks [51] used a single-call libc functions, ROP attacks take as building blocks only
to the system() function of libc, with specified function short instruction sequences ending with a return instruction,
arguments, to spawn a command shell. As a single func- dubbed gadgets. Such gadgets are discovered offline through
tion call is used to perform a specific operation, it alone static analysis of a vulnerable program, and allow an attacker
has limited ability or expressiveness. Afterwards, the second to carry out arbitrary computations possible with x86 code,
generation of RILC attacks [55] appear to be more advanced, i.e., Turing complete, when appropriately arranged [52].
i.e., capable of arbitrarily chaining multiple libc functions Each gadget performs a well-defined specific task, e.g.,
together and invoking them one after another for performing loading a value into a register, reading a value from memory,
a more powerful operation. A method called esp lifting [55] some arithmetic operation, or a conditional branching. To find
was introduced to glue multiple functions by means of com- these gadgets is not difficult in a large code base like libc.
mon short instruction sequences like pop esp; ret. The In particular on x86-based platforms, finding appropriate
stack pointer esp serves as a ‘‘virtual’’ program counter. gadgets is made even easier by the fact that instructions are
By overwriting the return address in the current stack frame of variable length and unaligned memory access is supported,
with the memory location of such a sequence, an attacker meaning that unintended instruction sequences can be found
3 The term shellcode is nowadays used to refer to exploit payload, irre-
and acquired by starting from an offset of some intended
spective of whether it launches a shell [58].
instructions. That is, every x86 program contains many unin-
4 Note that a C runtime library provides many useful low-level routines, tended instruction sequences which can be leveraged by
e.g., wrappers for system calls, that can be called by a compiled C program attackers. ROP is essentially all about finding useful gadgets
during execution. In most of research papers the GNU libc is used for located anywhere in memory, and connecting them sequen-
demonstration, but the presented techniques should be also applicable with
C runtime libraries implemented in other OS like Microsoft Windows OS, tially to perform desired operations. The location of every
as per [52], [56]. gadget is written into the stack, and the return instruction

48248 VOLUME 10, 2022

Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

at the end of each gadget helps to get the next gadget to be gadgets. The control transfer between functional gadgets is
executed, effectively allowing them to be chained. A perhaps achieved through maintaining an internal dispatch table by
more concrete way to view ROP is that the selected gadgets the dispatcher gadget and ensuring that the jump instruction at
form a ‘‘virtual’’ instruction set that can be used to write a the end of each functional gadget will always give the control
program of arbitrary complexity, and the stack pointer acts as back to the dispatcher gadget.
a ‘‘virtual’’ instruction pointer [57], [60].
ii) CALL-ORIENTED PROGRAMMING
b: FORWARD-EDGE CONTROL-FLOW HIJACKING Not only can an indirect jump instruction take the role of
Since the advent of ROP attacks, defenders have presented the return instruction in code-reuse attacks, but so does an
a variety of potent defense methods. Driven by the insight indirect call instruction, as it is proposed in call-oriented
that ROP has a great reliance on the return instruc- programming [60]. Similar to ROP and JOP, arbitrary com-
tion, such defense methods [61]–[63] either detect abnor- putations with malicious purposes are performed in COP
mal use of the return instruction, e.g., too frequent or by finding, chaining, and executing some useful gadgets
unusual according to calling convention, or prevent com- ending with an indirect call instruction. Even when some
pilers from producing code containing return instructions. control-flow integrity (CFI) [68] policies, a powerful defense
This, though, has prompted attackers to come up with technique discussed in detail in Section V, are enforced, COP
other code-reuse variations, extending the approach from may still be possible, given that indirect call instructions must
solely relying on return instructions to capable of lever- be always allowed to jump to certain functions [60].
aging any indirect branching instruction. Hence attackers
further generalized code-reuse attacks to include forward- c: HYBRID EXPLOITS
edge control-flow hijacking attacks such as jump-oriented Although aforementioned code-reuse attacks prove to be Tur-
programming (JOP) [64], [65] and call-oriented program- ing complete, the enforcement of CFI policies can make it
ming (COP) [60]. much more challenging to successfully conduct these type
of attacks. That is, when a CFI policy is enforced in a tar-
i) JUMP-ORIENTED PROGRAMMING get program, attackers have to find CFI-compliant gadgets,
The term of jump-oriented programming was coined by [65], which are usually much scarcer in the target program. This
while independent techniques leveraging the jump instruc- prompted attackers, again, to adapt their techniques, leading
tion jmp, instead of the return instruction ret, were to hybrid exploits, in which they combine code-reuse attacks
first presented in [66] and [64]. JOP attacks can bypass and code-injection attacks, such as [69] and [60]. In fact,
above-mentioned defense approaches against ROP attacks, hybrid exploits are shown to be more dominant than pure
because such obvious inherent characteristics of ROP attacks code-reuse attacks in real world [70]. Recall that the reason
like violation of calling convention, which can be taken as why attackers turned from carrying out code-injection attacks
a detection indicator, do not exist in JOP attacks. Given that to code-reuse attacks is that the enforcement of the W⊕X
certain instruction sequences behave like a return instruction, policy forbids injected code from being executed. However,
it is proposed in [64] that the ret instruction can be replaced the W⊕X policy only states that a memory page cannot be
with the pop x; jmp ∗x sequence. Such a sequence can both writable and executable at the same time, but still allows
be acquired from a target program, and is called trampoline. a memory page to be first only writable and afterward only
In this case, gadgets are certain selected instruction sequences executable. This lets attackers come up with a multi-stage
ending in an indirect jump instruction, whose target is the attack procedure. They first write shellcode into a buffer in a
trampoline. That is, the trampoline is responsible for chaining memory page that is now only writable. Subsequently, they
short instruction sequences and redirecting execution, essen- launch a code-reuse attack, in which they use gadgets to
tially behaving like the ‘‘glue’’ and thereby making JOP also invoke a function call or system call to change the permissions
Turing complete. of that memory page from only writable to only executable,
However, the techniques in [64] still rely on the stack effectively bypassing the W⊕X protection. Then, the shell-
to steer the control flow among gadgets, and the pop x; code can be executed, because neither the W⊕X policy nor
jmp ∗x sequence is rather rare [64], [65], i.e., not always the CFI policy will prevent it. Note that a CFI policy is
present in a target program. Another JOP approach [65] does enforced only in existing code, not in injected code, as CFI
not suffer from such limitations. A special kind of gadget checks are inserted in a binary program either during com-
is introduced to chain other gadgets ending with an indirect pilation by a compiler, or through binary rewriting following
jump, and to govern the control flow without relying on static analysis.
the stack. This kind of gadget is called dispatcher gadget,5
in order to differentiate it from other gadgets performing d: AUXILIARY TECHNIQUES
certain primitive operations such as arithmetic operations or To pave the way for a successful exploitation, except requir-
conditional branching, which are considered as functional ing one or more program bugs, attackers also need to make
use of some supporting techniques. As mentioned previously,
5 Sometimes it is also called gadget dispatcher like in [67]. heap is now the front-line of the battle in memory [21], [48].

VOLUME 10, 2022 48249

 Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

When exploiting heap-based vulnerabilities, attackers usually practicability and usability of AEG, another AEG approach
employ a number of techniques to reliably and successfully is demonstrated in [78], which automatically generates ROP
perform the exploitation, such as heap-spraying [71], [72], payloads. It is applicable even in the presence of loosely
heap-Feng-Shui [73], [74]. implemented defenses mechanisms W⊕X and ASLR, result-
Due to inherent randomness of memory allocation in the ing in hardened exploits. As an extension, during automatic
heap, and especially when the effective defense technique ROP payload generation, the presented solution in [79] can
ASLR is employed, it is not easy for an attacker to reliably also deal with gadgets containing pointer dereferences. Fur-
redirect the execution to attacker-supplied code. To increase thermore, AEG solutions are extended to include the one
the likelihood of jumping to the correct memory location working solely on raw binary programs without debugging
of attacker-supplied code, and hence facilitate a successful information or source code [80].
exploitation, the attacker makes lots of copies of the mali- Another group of AEG approaches are designed to abuse
cious code and ‘‘sprays’’ them into yet unoccupied memory heap-based program bugs, which are more common in mod-
space of the heap, thereby dubbed heap-spraying. Next, the ern software, and generally more difficult to exploit, e.g.,
attacker tries to overwrite a code pointer on the heap making due to involved interaction with the heap manager. Compar-
it point to the attacker-supplied code. ing to stack-based AEG, heap-based AEG is much harder
The heap-spraying technique is effective, but it alone might to realize, and yet remains an open challenge [74], [81],
not be sufficient to underpin a reliable heap-based exploita- [82]. That is, existing heap-based AEG solutions can only
tion [73]. Because, on the one hand, the state and layout of automate one or more steps of exploit generation, but still
the heap are dynamic and hard to predict, meaning that the fall short of a generic end-to-end fully AEG, even in the
overwritten code pointer is not guaranteed to always contain absence of modern defense mechanisms. Nonetheless, some
the attacker-controlled data. On the other hand, there might advancements were made in recent years by representative
not always be enough free space left in the heap for being heap-based AEG techniques. For instance, [83] presented
sprayed. Therefore, for a successful exploitation, it is often a framework for discovering exploit primitives6 in heap
necessary to control the heap state and layout before trigger- managers using symbolic execution, and generating usable
ing a program vulnerability. Heap-Feng-Shui is a technique exploits. Another framework demonstrated in [81] focuses on
for manipulating heap layouts by first finding some primitives automated exploitability assessment of heap-based program
to interact with heap allocators, and subsequently carefully bugs, in which it makes use of both fuzzing and symbolic exe-
assembling them through allocating or freeing objects of cution to explore exploitable states from a crashing input, i.e.,
selected sizes [73], [74]. an input which triggers a program bug, and to generate func-
tioning exploits. Note that transforming a crashing input into
e: AUTOMATIC EXPLOIT GENERATION an exploitable state is in general not an easy task. Similarly,
Carrying out a successful code-reuse attack is usually a based on bounded model checking and symbolic execution,
complex task, especially when defense mechanisms are in another tool is designed in [84] to assess the exploitabil-
place. In order to reduce time and human efforts, attackers ity of several heap allocators given a memory corruption
gradually managed to automate various steps of designing bug. Besides, the first AEG for heap overflows in language
a code-reuse attack, e.g., automatic discovery of gadgets, interpreters, e.g., PHP and Python interpreters, is proposed
ultimately leading to automatic exploit generation (AEG). in [82], which utilizes fuzzing-based input generation, rather
A variety of algorithms, techniques or tools are designed than relying on symbolic execution. Furthermore, automated
to semi-automatically or fully-automatically find program heap-Feng-Shui is proposed in [74] for automatically manip-
bugs and craft corresponding (defense-resilient) exploits, ulating heap layouts, hence to facilitate heap-based AEG,
respectively. as many heap-based program bugs are exploitable only given
Early attempts of AEG target stack-based program bugs. certain heap layouts.
Automatic patch-based exploit generation proposed in [75]
can generate an exploit for an unpatched program without 2) DATA-ORIENTED EXPLOITATION
first knowing the vulnerability, when provided with the patch As control-oriented exploitation gets harder, mostly due to
as a guidance. In [76], control-flow-hijacking exploits are widely deployed defense mechanisms like, in particular,
automatically generated also using dynamic taint analysis CFI, data-oriented exploitation gains popularity thanks to
paired with an algorithm that generates candidate exploits its immunity to CFI checks. Data-oriented attacks do not
from a constraint formula. Though, the first end-to-end fully violate the control-flow graph (CFG) of a target program
automatic exploit generation, i.e., from automatic vulnerabil- during its execution amid control-flow transfers, but still can
ities disclosure to generating exploits that spawn a command cause significant damage and pose a considerable threat [50],
shell, is considered to be the approach presented in [77], [53], [67]. This advantage is provided by the fact that some
which makes use of symbolic execution for exploring pro-
gram paths and checking their exploitability. However, these 6 Exploit primitives are basic operations like a write-what-where opera-
earliest AEG solutions do not assume any defense mecha- tion, and they are considered as exploitation building blocks used to achieve
nisms are deployed in the target systems. To enhance the arbitrary code execution.

48250 VOLUME 10, 2022

Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

program data is not directly used in control-flow transfer previously, a gadget dispatcher is a special kind of gadget
instructions, i.e., never loaded into the instruction pointer which selectively and consecutively invokes functional gad-
register, but can yet indirectly influence program’s execu- gets, allowing arbitrary recursive computations. An example
tion to the benefit of attackers. As per [67], data-oriented of a typical data-oriented gadget dispatcher can be acquired
exploitation can be grouped into two categories, namely from code which implements a loop statement [67].
direct-data-corruption attacks [50], [53], and data-oriented Another type of advanced data-oriented attack is
programming (DOP) attacks [67]. demonstrated in [85], in which the term block-oriented
programming (BOP) is introduced. BOP gadgets are com-
iii) DIRECT-DATA-CORRUPTION prised of entire basic blocks, instead of instructions. The
Direct-data-corruption attacks represent the first generation authors presented a framework for automatically constructing
of data-oriented exploitation, in which attackers directly defense-resilient exploits against programs hardened with
manipulate a variety of non-control, yet security-sensitive CFI policies, surpassing previous DOP attacks in the sense
data [50] like user ID or configuration data, in order that they heavily rely on manual analysis.
to mislead a program’s execution for malicious purposes. According to whether a bug in user-space programs or
A real-world example is the Heartbleed7 exploitation, which operating system kernels is exploited, we can categorize it
allows attackers to steal cryptographic keys and other creden- as user-space program exploitation or kernel exploitation,
tials from a vulnerable device, hence to eavesdrop on its com- respectively. Whereas user-space program exploitation rep-
munications or impersonate it. Early direct-data-corruption resents the majority of binary exploitation, due to easy acces-
attacks [50] are relatively straightforward, and can succeed sibility etc., kernel exploitation is less common and more
with only a single data-flow edge compromised. difficult, but very appealing to attackers. Real-life exam-
To further demonstrate the power of data-oriented exploita- ples like EternalBlue and EternalRomance [86] show that
tion, a systematic approach called data-flow stitching [53] kernel exploitation poses a serious and realistic threat. Note
is developed to automatically generate data-oriented exploits that commodity OS kernels are almost exclusively written
when provided with memory corruption bugs. Data-flow in the memory-unsafe C language, providing attackers
stitching aims to chain multiple existing data-flow edges a better chance to conduct runtime attacks against ker-
in a data-flow graph to create new unintended data-flow nels. Except above-mentioned software-based exploitation,
paths (from the perspective of normal program execution). attackers can also carry out sophisticated hardware-based
A further generalization of data-oriented attacks is proposed exploitation, which is particularly dangerous, e.g., the infa-
in [54], which introduces the notion control-flow bending mous Rowhammer—a hardware bug—exploitation on vari-
(CFB). In CFB, modifications of both control data and ous hardware platforms [87]–[89].
non-control data are allowed as long as the enforced CFI
policy is not breached. That is, CFB refers to any attack, C. CONSEQUENCES OF BINARY EXPLOITATION
in which the entire execution trace looks legitimate with Depending on specific program bugs, targeted applications,
respect to the control-flow graph. defense mechanisms in place etc., the consequences of binary
exploitation vary from the severest ones—arbitrary code
iv) DATA-ORIENTED PROGRAMMING executions—to the mildest ones—harmless program crashes
The aforementioned data-oriented attacks suffer from lim- of non-critical programs.
ited expressiveness, that is, they cannot provide attackers
the ability to perform arbitrary computations in a vulnerable 1) ARBITRARY CODE EXECUTION
program’s memory space. Besides, they can be easily pre- The ability to execute arbitrary code in a target program’s
vented by enforcing some access control policies in memory, memory space gives attackers the complete control of it, and
which largely restrict unauthorized access to security-critical the freedom of causing any possible damage to the system
data. With the advent of the second generation of data- if the controlled program has the highest privileges. This is
oriented exploitation, i.e., data-oriented programming [67], arguably attackers’ first objective. They can force the running
attackers can design Turing-complete data-oriented attacks, program to execute at their bidding by invoking arbitrary
also without relying on corrupting specific security-critical system calls, and behave differently from the programmer’s
data. Similarly to ROP, in DOP, the first step is to discover intention. Besides, they can leverage credentials stored in
useful gadgets, i.e., short sequences of instructions used to the victim system to further attack other devices. Even if
perform the basic operations like arithmetic operations and the controlled program does not have all the permissions
conditional branching. Note that data-oriented gadgets are, in the system, a limited set of system calls, i.e., confined code
by definition, CFI-compliant, and hence generally more dif- execution, may also allow attackers to do enough damage.
ficult to find. Next, gadget dispatchers need to be identified,
and are used to chain those functional gadgets, in order to 2) INFORMATION LEAKAGE
achieve desired functionality or computations. As mentioned Sometimes, an exploitable bug cannot directly grant an
attacker the right to take over the system and execute
7 https://heartbleed.com/ any code, but rather the ability to reveal some sensitive

VOLUME 10, 2022 48251

 Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

information, e.g., by copying it to an output stream. There needed to sabotage a system is embedded directly in the
are two types of information, which are of special interest corresponding exploit-based malware.
of attackers, i.e., security credentials and memory layout- When binary exploitation is used as an initial infection
related information. Leaking security credentials like pass- vector, it often employs one of the two most typical forms,
word hashes or cryptographic keys helps attackers to easily i.e., direct remote exploitation, and phishing emails with an
pass access controls, and eventually take over control of the attachment containing an exploit. Direct remote exploita-
target system. Disclosing memory layout-related informa- tion mostly targets network service programs, i.e., programs
tion, e.g., the memory address of a specific object, let an implementing communication protocols, e.g., Server Mes-
attacker reliably assess the memory layout of a target process. sage Block (SMB) protocol and Remote Desktop Protocol
As a result, it renders some deployed defense mechanisms (RDP). An exploit embedded in an attachment of a phishing
like ASLR useless, and helps the attacker to further exploit email typically targets a user application program, which is
another memory bug (or sometimes the same bug), then used to open and process the corresponding attachment, e.g.,
leading to arbitrary code execution. Microsoft Word document.

3) PRIVILEGE ESCALATION III. A BRIEF OVERVIEW OF MAJOR ICS CYBER INCIDENTS

If an exploited program itself does not have the root privilege In this section we aim to provide a brief overview and analysis
(in Unix parlance) or the administrator right (in Windows of major ICS cyber incidents leveraging binary exploitation
parlance), or the leaked credentials do not provide all the in the last decade. The past real-world examples demonstrate
permissions in the system, it may not satisfy an attacker’s that binary exploitation is a very important launch pad for
goal. The attacker may try to further leverage another bug to malware. For every cyber incident, we primarily discuss the
corrupt security-critical data [50], which can result in elevated consequence/damage of the attack, the technical attack pro-
privileges. cedure, and the exploits involved.
Note that this may only be a small sample of cyber inci-
4) DENIAL OF SERVICE dents, because, according to MITRE [90], the vast majority
Crashing a target program, as a result of binary exploitation, of discovered incidents are not reported publicly. It is also
is not uncommon during real-life attacks, not to mention worth mentioning that there are ICS cyber incidents that are
during the test of an exploit under development. Due to not listed in this paper, because, to the best of our knowl-
various factors, even a well-tested exploit may not work edge, there are no evidences published yet, which indicate
every time on every system containing the corresponding that binary exploitation is involved. This could be due to a
program bug. Causing this kind of denial-of-service (DoS) lack of forensic evidences in victim networks. For instance,
is sometimes intended, but at other times rather unintended. as per US ICS-CERT [91], [92], the initial access vectors
For instance, in a past ICS cyber attack, attackers exploited a in more than a third of total incidents reported in 2014 and
software vulnerability for causing DoS, as we will discuss it in 2015 were not identified, because of a lack of detec-
in Section III-F. Nonetheless, more often than not, attackers tion and monitoring capabilities within the compromised
would choose to make better use of a vulnerability like in the networks.
first three cases, when possible. Given that cyber incidents are usually characterized and
represented by the corresponding malware, and often even
D. EXPLOIT-BASED MALWARE named after these malware, and in order to avoid lengthy title
Traditionally, malware is encapsulated inside some exe- for each incident, we also use the malware name as the title
cutable needed to be manually executed, and requires no for each incident.
exploitation of program bugs. In today’s cyber attacks, the
target of an attacker is usually not a single computer, but A. STUXNET
rather a network with lots of computing devices. In order The attack against Iranian nuclear centrifuges uncovered in
to save time and have a higher impact on the target by 2010 [93] is widely perceived as a watershed moment in ICS
attacking as many devices as possible in a short period of security. This is not only due to the unprecedented use of four
time, an attack process is now largely automated by designing zero-day exploits [94], but also due to the willingness and
exploit-based malware, as it shows in Section III. In this way, the ability of the attackers to study the industrial devices and
user interaction is often not a prerequisite, and unnecessary the physical processes. The corresponding malware Stuxnet
intervention from command-and-control (C&C) servers can is deemed as the first confirmed example of ICS-tailored
also be reduced to evade network-based intrusion detection. malware. According to the security firm Symantec [94],
Besides, for systems that forbid outbound Internet access, it includes a Windows OS rootkit that hides its binaries, and
this would be the only way to attack them. That is to say, the first ever programmable logic controller (PLC) rootkit
nowadays, an exploit is mostly used in combination with that hides modified code on PLC.
some malware, which serves as payload following an exploit It is widely believed that the initial access was introduced
and/or has integrated exploit(s). In some cases, an attack can by removable USB drives. These removable drives contain
be completely automated, meaning that all the functionality a specially crafted Microsoft Windows shortcut file, which

48252 VOLUME 10, 2022

Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

exploits the then zero-day vulnerability CVE-2010-25688 of Windows OS, and helps to install Duqu onto targeted
(patched with the security update MS10-0469 ) in Windows computers unbeknownst to users. A malicious embedded
Explorer. When a removable drive is plugged into a Win- TrueType font file can allow code auto-execution in kernel
dows computer and viewed, the contained shortcut file will mode, because of erroneous font-parsing of the Windows
automatically execute the first hidden file in the removable kernel-mode driver [97].
drive. This hidden file will hook some important DLL10 API When such a Word document is opened on a computer,
and replace the original code of exported functions with some the exploit is triggered, and will first check if the computer
files-checking code. Then it loads the second hidden file from is already compromised by looking into the Windows Reg-
the removable drive, which contains the main Stuxnet DLL. istry. If the computer has already been compromised, it exits.
After the main DLL is extracted into memory and executed, Otherwise the shellcode contained in the exploit will decrypt
Stuxnet is installed. a driver file and an installer DLL file from within the Word
In the targeted system, many computers were non- document. The execution is then passed to the extracted driver
networked, and the data exchange between them were done file, which is signed with a stolen legitimate digital certificate
with removable drives. Hence one of Stuxnet’s propagating allowing it to bypass default restrictions on unknown drivers
ways is also through removable drives. Whenever an unin- and common security policies. The driver file injects the
fected removable drive is inserted into a already compro- installer DLL into a running benign services.exe process.
mised computer, Stuxnet will copy itself and its supporting Then the installer DLL gets executed, and starts extracting
files to the removable drive [94]. Other Stuxnet’s spreading other components from it. These components are injected
methods include exploiting the then zero-day vulnerability into other benign processes to hide Duqu’s activities, and to
CVE-2010-2729 (patched with MS10-061) in Windows Print bypass security programs. An information-stealer program is
Spooler, and exploiting the vulnerability CVE-2008-4250 installed by Duqu to collect system information [96].
(patched with MS08-067) in Windows Server Service.
Through the vulnerability in Windows Print Spooler, which C. FLAME
allows a file to be written to a system directory of a vulner- Another complex global cyber espionage targeting ICS espe-
able computer, Stuxnet spreads itself easily in a local area cially in Middle Eastern countries was revealed in 2012.
network (LAN), and executes itself in the infected machines. The corresponding malware, named Flame, was identified,
By means of the SMB protocol and the vulnerability in analyzed and disclosed by multiple entities including Ira-
Windows Server Service, Stuxnet is able to copy itself to nian National CERT, Kaspersky Lab [98] and the CrySyS
unpatched remote computers through sending a malformed Lab [99]. This modular malware shares many characteristics
path string that allows arbitrary code execution [94]. with the above-mentioned malware Stuxnet and Duqu, but its
The other two then zero-day exploits are used for privilege modules are different. Comparing with Duqu, Flame is not
escalation. The first one is used to exploit a vulnerability only a lot more complex, but also much more widespread,
in Windows Task Scheduler, which makes the main DLL making it redefine the notion of cyber espionage [98].
file run as a new process with Adminstrator rights. The sec- It is assumed that the initial access is achieved by
ond privilege escalation exploit targets the CVE-2010-2549 exploiting the CVE-2010-1879 vulnerability (patched with
vulnerability (patched with MS10-073) in the Windows MS10-033), and a then zero-day exploit is used to compro-
Win32k.sys kernel-mode device driver [94]. mise then fully-patched Windows 7 OS [98]. In order to
spread itself to non-infected computers, Flame includes two
B. DUQU exploits used by Stuxnet, i.e., for the CVE-2010-2729 vulner-
A global cyber espionage especially targeting ICS was dis- ability in Windows Print Spooler and for the CVE-2010-2568
closed in 2011 with the discovery of a new piece of malware, vulnerability in Windows Explorer [98], [99]. The first
which uses many of the same techniques from Stuxnet. exploit is used to spread itself through network shares, and
The malware, named Duqu and discovered by the CrySyS the second is used with removable drives for non-networked
Lab [95], targeted a number of organizations around the world computers.
for stealing information, which can be used for planning
another destructive attack against them. D. HAVEX
The initial access was likely introduced by spear-phishing The third confirmed widespread cyber espionage followed
emails with a Microsoft Word document as attachment con- in 2013 with the ICS-tailored malware Havex. It first tar-
taining a then zero-day kernel exploit [96]. This exploit geted defense and aviation companies in the US and Canada,
abuses the CVE-2011-3402 vulnerability (patched with and then shifted its focus to US and European energy
MS11-087) in the Win32k.sys kernel-mode device driver firms [100]. The stolen information gives the attackers the
ability to more easily sabotage operations in the targeted
8 https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2010-2568
9 https://docs.microsoft.com/en-us/security-
critical infrastructures.
updates/securitybulletins/2010/ms10-046
According to the security firm Symantec [100], the attacker
10 Dynamic-link library (DLL) is Windows OS’ implementation of the group has at least three infection tactics for getting initial
shared library concept. access to victim systems, i.e., spear phishing emails with

VOLUME 10, 2022 48253

 Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

malicious PDF attachment, watering hole attacks, and sup- 2 appears as a kernel-mode driver. In order to disable the
ply chain attacks. In the watering hole attacks, the attackers driver signature check in Windows OS and evade detec-
first compromised a number of energy-related websites, and tion, a tool called DSEFix13 is used, which exploits the
injected an iFrame11 into them. This iFrame redirects visi- CVE-2008-3431 vulnerability [103]. Besides, BlackEnergy
tors to another compromised website hosting the LightsOut 2 also contains exploits for human-machine interface (HMI)
exploit kit,12 or its updated version. The LightsOut exploit applications from ICS vendors like Siemens and General
kit abuses vulnerabilities in the Java Runtime Environ- Electric. Hence an Internet-connected HMI can easily give
ment (JRE) component in Oracle Java SE 7 and in Internet attackers an initial access and a foothold in the ICS central
Explorer browser to deliver the Havex malware, and in some location, for information gathering and conducting further
cases the Karagany malware [100]. attacks [104].
In the supply chain attacks, the attackers compromised a
number of legitimate software packages from three different F. INDUSTROYER
ICS equipment providers, by inserting the Havex malware Another large-scale cyber attack on Ukraine’s power grid
into them [100]. When users download these trojanized soft- took place in 2016 with a piece of even more advanced
ware packages from ICS vendor websites, they would then malware dubbed Industroyer. This malware is considered as
unknowingly download the malware themselves, allowing the the first malware designed and deployed specifically to target
malware to bypass some security measures. power grids. It has the ability to directly control switches and
circuit breakers, showing that the malware creators have a
E. BlackEnergy good knowledge of ICS and communication protocols used
In 2015, a major hours-lasting blackout in Ukraine was in power grids [104]. The initial infection vector remains
caused by a complex targeted attack consisting of a set of unknown [105].
actions, e.g., disrupting electricity distribution, destroying This modular malware has a core component being the
IT systems, flooding call centers, and inhibiting incident main backdoor, which receives commands from its remote
response [101]. The attackers deployed a piece of malware C&C server via HTTPS, and controls all other compo-
dubbed BlackEnergy 3, which is the second update of its nents [105]. A launcher is installed by the main backdoor as
original version BlackEnergy appeared in 2007. a program responsible for launching several payloads and a
To deliver the malware, various initial infection vectors data wiper module, which are DLL files and export a function
were employed [102]. The first kind is spear-phishing emails named Crash for the launcher. The payloads partly implement
with an executable file as attachment, which is disguised with the communication protocols specified in the standard IEC
a Microsoft Word document icon tricking receivers to click it 60870-5-101, IEC 60870-5-104, IEC 61850, and OPC Data
and execute it. The second kind is spear-phishing emails with Access specification, respectively [105]. By leveraging the
some Microsoft Power Point slides as attachment, which con- intended functionality in these protocols, the attackers are
tain hidden objects exploiting the CVE-2014-4114 vulnera- capable of enumerating and possibly taking over control of
bility (patched with MS14-060) in Windows Object Linking all remote terminal units (RTU), intelligent electronic devices
and Embedding (OLE). The last one is spear-phishing emails (IED), and open platform communications (OPC) servers etc.
with a Word document as attachment, which contains hidden The destructive data wiper module is used in the final stage of
objects exploiting the CVE-2014-1761 vulnerability (patched an attack, which hides its traces and makes recovery difficult,
with MS14-017) in Microsoft Word and Office Web Apps. e.g., by deleting all files.
After the initial infection and delivery of the core module, Besides, the CVE-2015-5374 vulnerability was exploited
a number of plug-ins, i.e., DLL files, are downloaded and by sending crafted packets to Siemens SIPROTEC digital
successively executed [103]. For instance, a file-system oper- relays rendering them unresponsive [105]. This denial-of-
ations plug-in is used for early reconnaissance. A network service (DoS) attack against SIPROTEC protective relays was
scanner plug-in explores the network perimeter, and disguises performed by the attackers right after opening circuit breakers
itself as a Windows service program. Credentials are stolen and removing operators’ visibility into system operations
with some password stealer and key logger plug-ins, and then through the data wiper module [106].
used to help attackers to move to other computers in the
network. G. WannaCry
After the attackers have gained the required credentials, In May 2017, numerous organizations across the world
they start infecting Windows servers using BlackEnergy 2, including railway companies, manufacturing companies, and
i.e., the direct predecessor of BlackEnergy 3. BlackEnergy energy firms, were attacked by a cryptoworm known as
WannaCry [107]. This fast-spreading malware continued
11 An iFrame is a HTML element allowing embedding documents, videos,
to infect thousands of computers globally for at least two
and interactive media within a web page. years [108]. WannaCry is a piece of ransomware, i.e.,
12 The LightsOut exploit kit contains exploits for the CVE-2012-1723,
CVE-2013-2465 and CVE-2013-1347. https://www.mcafee.com/enterprise/ malware containing an encryption plug-in and a ransom note.
de-de/threat-center/threat-landscape-dashboard/exploit-kits-details.lights-
out-exploit-kit.html 13 https://github.com/hfiref0x/DSEFix

48254 VOLUME 10, 2022

Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

The initial access into a network can be achieved by select- I. BAD RABBIT
ing random IP addresses across the Internet and exploit- In October 2017, another large-scale ransomware cam-
ing vulnerable Internet-facing Windows computers. It then paign took place, and affected organizations across Russia
spreads itself rapidly in local networks. It does this by and eastern Europe, e.g., the transportation sector in
exploiting the CVE-2017-0144 vulnerability (patched with Ukraine [111]. This, again, demonstrates that the histori-
MS17-010) in Windows SMB Server. Before infecting new cally ransomeware-free ICS networks are now a very focused
computers through an infected computer, it checks if they are target of ransomware.
already compromised. If not, it proceeds to use the infamous According to Kaspersky Lab [112], the ransomeware,
SMBv1 exploit EternalBlue [86] to drop its payload to those dubbed Bad Rabbit, also used a watering hole attack as initial
vulnerable computers. infection method. But in this case, no exploit was used to
According to Kaspersky Lab [107], during the investiga- gain initial access. Instead, when a target victim visits a com-
tion, WannaCry was able to infect computers in ICS due promised legitimate website, the malware is downloaded and
to the following facts: (1) use of dual-homed computers, pretends to be an Adobe Flash installer. This tricks the victim
e.g., engineering workstation,14 acting as a bridge between into executing the malware. After infecting one computer,
an enterprise network and a local industrial network; (2) a it starts to spread itself within the corporate network and
lack of properly configured secure network perimeter devices to the industrial network. It does so by using a modified
between segments of an industrial network, which are con- version of the EternalRomance exploit, which abuses the the
nected through the Internet via VPN channels15 due to large CVE-2017-0145 vulnerability in SMBv1 Server. Whereas in
distances; (3) use of devices, e.g., USB modem, for setting NotPetya this exploit is used to install the DoublePulsar16
up direct mobile Internet connections for computers on an backdoor, Bad Rabbit employs it to overwrite a kernel’s
industrial network, bypassing the network perimeter. session security context, which enables it to launch remote
services [114].
H. NotPetya
Another cryptoworm affecting ICS globally appeared only J. TRITON
one month later, in June 2017. According to Kaspersky ICS Another watershed moment in ICS security occurred with
CERT [109], at least half of the companies attacked by this an attack against a petrochemical plant in Saudi Arabian
malware, dubbed NotPetya, are manufacturing and energy reported in December 2017 [115]. This is considered as a
firms. The disruption caused by NotPetya resulted in a halt watershed moment, because the malware, named Triton, dis-
of operation. Despite appearing to be a piece of ransomware, tinguishes itself from other ICS-tailored malware like Stuxnet
encrypted files cannot be restored, even after victims have in two ways. Firstly, Triton targets a safety instrumented
paid the ransom. NotPetya is more like a piece of data-wiping system (SIS), which serves to put a critical infrastructure in a
malware pretending to be ransomware [109]. safe state or shut it down to prevent any physical harm. A SIS
One initial infection method is conducting watering hole is deemed as the last defense line for critical infrastructures.
attacks, i.e., first compromising websites presumably fre- It is often totally isolated from all other networks and hence
quently visited by targeted victims and starting infecting hard to reach. Secondly, sabotaging a SIS not only causes
victims from there. Once NotPetya has infected one computer economical loss for operators, but also directly risks human
in a network, it starts to spread itself to other computers lives.
inside the same network using several propagation meth- In this attack, the Triconex safety controllers made by
ods. Two infamous exploits are used for this purpose, i.e., Schneider Electric were specifically targeted [14]. Triton
EternalBlue and EternalRomance exploits [86], which abuse consists of two parts, i.e., a malicious Windows program
the CVE-2017-0144 vulnerability and the CVE-2017-0145 compiled from a Python script and a malicious program
vulnerability in SMBv1 Server, respectively. These exploits for the safety controllers. The attackers first compromised a
are used for different Windows OS versions [110]. Using the Windows computer within a SIS network, and the malicious
same exploits, NotPetya is able to propagate from corporate Windows program is executed on the compromised com-
networks to industrial networks. The exploitation process can puter. This program leverages a custom implementation of
be observed in network traffic, i.e., a series of specifically an internal TriStation protocol, through which the compro-
crafted SMB packets. mised computer connects to a safety controller. The malicious
program for the safety controllers includes an injector and a
backdoor, and both of them are downloaded into the safety
controller by the compromised computer.
14 Note that an engineering workstation is a device used to write program The injector executes automatically on the safety
and configuration data for PLC. Engineering workstations using Windows controller. After it verifies that the controller can be
OS are commonly seen in ICS.
15 Note that a VPN channel is used to prevent unauthorized eavesdropping
and data tampering in the data transfer channel. It cannot prevent a computer 16 DoublePulsar backdoor is a sophisticated SMB backdoor, and the
from being attacked, when another computer at the other end of the channel primary payload used in SMB and RDP exploits in the FuzzBunch
is already compromised. framework [113].

VOLUME 10, 2022 48255

 Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

TABLE 1. Summary of exploitation purposes of exploit-based Malware. privilege escalation, exploitation for detection evasion, and
exploitation for lateral movement. All these exploitation pur-
poses can be observed in the cyber incidents mentioned
above. Besides, we observed and add another exploitation
purpose after analyzing these cyber incidents, i.e., exploita-
tion for denial-of-service. It is worth mentioning that in this
section we analyze the purposes of binary exploitation from
a network point of view, whereas in Section II-C the attention
is solely on a single host.
A summary of exploitation purposes of above-mentioned
exploit-based malware is given in Table 1. Note that in the
case of protocol vulnerability, the same exploit could be
used both for getting initial access and for lateral move-
compromised, it exploits a then zero-day vulnerability in ment, e.g., the EternalBlue exploit, making the corresponding
the device firmware17 for privilege escalation [115] [14]. exploit-based malware spread very fast and hence especially
Before injecting the backdoor into the firmware memory dangerous.
region, it disables a firmware consistency check for detection
evasion. The backdoor is enabled by the injector through IV. ATTACK SURFACE ON ICS
changing a jump table entry to point to the injected code. The ICS landscape is very diverse in terms of involved
It gives the attackers the capability of reading, writing mem- physical processes, devices, software platforms, hardware
ory and executing arbitrary code on the controller [14]. platforms, communication protocols, vendors etc. The
increasing complexity and connectivity in these heteroge-
K. VPNFilter neous environments inevitably result in an increased attack
In 2018, at least half a million public-facing routers and net- surface. Devices commonly found in ICS encompass engi-
work attached storage devices across the globe were infected neering workstations, HMI, OPC servers etc. running com-
by a piece of malware known as VPNFilter [116]. This con- modity OS like Windows OS and based on x86 hardware
cerning malware has the capability of spying on traffic routed platforms, and specialized embedded devices such as PLC,
through infected devices, and has dedicated code for targeting RTU, IED, smart sensors running a (tiny) embedded OS
ICS. It is also destructive in that it can make infected devices or even bare-metal applications (i.e., without an OS), and
unusable. Besides, unlike most other internet of things (IoT) based on a variety of hardware platforms like ARM, MIPS,
malware, VPNFilter is capable of maintaining its persistence PowerPC, AVR etc. Evidences showing that all these plat-
even after a reboot [116], [117]. forms are potentially susceptible to binary exploitation are
As per security researchers in Cisco Talos [116], since all provided in Section V. As discussed in Section III, only one
of the affected devices have publicly known vulnerabilities, Internet-facing vulnerable device in an ICS suffices to grant
it is very likely that the initial infection vector is the exploits an attacker a strong initial foothold in its network and a great
targeting those vulnerabilities. This multi-stage, modular opportunity to further attack other devices in the network.
malware is used both for cyber espionage and for destructive Any compromised device could be utilized to provide rogue
attacks. The stage 1 module works as a backdoor providing services, such as disrupting network operation by masquerad-
a persistent foothold, and multiple redundant mechanisms ing as another device, eventually leading to safety issues.
are employed to connect it to its C&C server. The stage To promote a better understanding on why binary exploitation
2 module serves to gather and exfiltrate data, and overwrite a has become a more serious problem in ICS, we provide in
critical portion of the device’s firmware making it inoperable. this section primarily a brief overview of history of ICS
Additional functionality is provided by multiple stage 3 mod- network structure and a succinct analysis of its attack surface
ules, which include Modbus/TCP traffic monitor, exploits transition.
deliverer, man-in-the-middle (MitM) component capable of
intercepting and manipulating network traffic [116]–[118]. A. A BRIEF HISTORY OF ICS
Looking back in the history of ICS, we have witnessed
1) EXPLOITATION PURPOSES a transition from the very early air-gapped, well-isolated,
According to the MITRE ATT&CK for ICS Matrix [119], hardware-centric systems to some more connected, advanced,
the purposes of binary exploitation can be categorized into well-segmented systems with distinct IT and OT (operational
four groups: exploitation for initial access, exploitation for technology) networks based on the Purdue Model [120], and
then a drastic shift to systems with converged IT and OT
17 Note that Triconex firmware versions 10.0–10.4 running on a PowerPC
networks, driving the connectivity to a much higher level and
processor are vulnerable to Triton. The newer Triconex safety controllers
use ARM processors, meaning that a different version of exploit would be eventually breaking the (boundaries introduced in the) Purdue
required [14]. Model.

48256 VOLUME 10, 2022

Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

FIGURE 2. A modern IIoT-based ICS network.

for security purpose, the ISA-99 / IEC 62443 standard19

proposed a new level, which is a demilitarized zone (DMZ)
FIGURE 1. ISA’s representation of the purdue model.
and is positioned between level 3 and level 4, hence often
called level 3.5. The ISA’s representation of the Purdue Model
is shown in Figure 1.
1) HARDWARE-CENTRIC
The DMZ is deployed to better control information flow
Initially, physical security was more of a concern than digital
between the operation network and the enterprise network,
security in ICS due to their air-gapped nature, i.e., physically
or to re-introduce the air gap for the OT network by removing
completely isolated from other systems [121]. They were
the direct communication link between the OT network and
hardware-centric in the sense that there was not much soft-
the IT network. In some ICS environments, there may be
ware involved, but primarily hardware systems. The con-
an extra DMZ or firewall between any two adjacent levels.
trol was done mainly through electrical, pneumatic or other
However, a true air gap no longer exists, as a DMZ or firewall
physical signals instead of digital signals, and the reliance
does not fully stop information exchange between the OT
on human operator in the field was very high. Besides,
network and the IT network, but rather only mediate the
a reconfiguration of a control system would require re-wiring
communication between them, not to mention when a DMZ
of those hardware devices. Even after the introduction of
or firewall is not properly set up or configured.
remote control via a point-to-point connection, the increased
exposure was very low, because the physical medium for
3) THE PURDUE MODEL IS ‘‘BROKEN’’
communication was normally owned and exclusively used by
the same company and isolated from other networks. As new technologies progressively find their way into ICS,
it becomes, little by little, clear that the Purdue Model has
2) THE PURDUE MODEL WAS BORN been ‘‘broken’’, or at least the strict boundaries between
those levels start being bypassed, replaced, or disappea-
In order to fulfill the requirement for rapid information
ring [122]–[124]. This rigid, hierarchical model is greatly
exchange between production and business segments in
challenged by the proliferation of IIoT and cloud services,
an industrial sector company, and efficiently manage the
in which new untrusted third-party entities/levels are incorpo-
relationship between its ICS and enterprise resource plan-
rated in the network structure, all kinds of industrial devices
ning (ERP) system, a hierarchical model called the Purdue
are equipped with Internet connectivity and smart sensors,
Model [120] was created in the early 1990s. This model has
and data is transmitted directly or through an IIoT gateway
a high impact on the standard ISA-95 from the International
outside the company to a cloud service provider for big data
Society of Automation (ISA),18 and was extended by ISA-95,
analysis.
which results in a clear-cut, well-segmented network struc-
The reasons why we may now think that the Purdue Model
ture with six levels, i.e., from level 0 to level 5. This net-
was inevitably torn apart can be summarized into two key
work structure became highly influential for ICS design and
points. Firstly, it is due to performance and management
quickly found a broad acceptance in industry. Subsequently,
18 https://www.isa.org/standards-and-publications/isa-standards/isa- 19 https://www.isa.org/standards-and-publications/isa-standards/isa-
standards-committees/isa95 standards-committees/isa99

VOLUME 10, 2022 48257

 Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

requirements. Increased connectivity and intelligence in study [126], many Internet-connected vulnerable ICS devices
industrial devices (down to the level 0 of the Purdue Model) contained vulnerabilities, for which the exploits were already
allow improved data sharing both in quantity and in qual- present in online exploit databases.
ity within the OT network and to the enterprise network,
which results in a more detailed view of individual devices 2) THE LAST DECADE
and a more comprehensive overview of the entire indus- As discussed in Section III, the last decade has seen a number
trial ecosystem, enables continuous monitoring of the perfor- of ICS attacks with increased complexity. The entry points
mance and conditions of these devices and the ecosystem, were either an Internet-facing device in an enterprise net-
and eventually simplifies the management. Secondly, it is work or an ICS device with remote access functionality. The
caused by business requirements. The data captured from attack surface became bigger, especially because more and
everywhere in the ecosystem is analyzed with big data tech- more hardware products and software solutions from various
nologies, and used for predictive maintenance, optimization vendors were integrated in an ICS, and a direct or indirect
of operation conditions, fine-tuning of production processes, remote access to the network is required by those third-
and better market strategies. All these compelling benefits party vendors. This can also be reflected in the fact that
ultimately contribute to a huge cost saving [123], [125]. more watering hole attacks and supply chain attacks were
For instance, offshore oil and gas organizations can reduce observed in the last decade. Once attackers find an initial
more than a third of their losses due to unplanned downtime access point into a system, they may quickly pivot into a
with an agile information sharing, analysis, and response critical path in the network and reach all kinds of devices.
process [125]. This is made possible because of the reality that ICS networks
However, ICS operators often do not have the expertise have been increasingly upgraded to TCP/IP-backed routable
and resources for big data analytics, hence need to transmit networks.
their data to third-party cloud computing providers for timely Real-world examples analyzed in Section III show that
predictive analytics. The growing desire for real-time data the ‘‘IT conduit’’ [127] in both IT and OT networks is fre-
from (remote) industrial devices, and for timely control and quently exploited and traversed in order to reach the final
configuration of these devices, drives the convergence of IT targets. For instance, Stuxnet leverages an engineering work-
and OT networks, and thereby creates a modern IIoT-based station to get into a target PLC. BlackEnergy exploits an
ICS network like the one in Figure 2. Internet-connected HMI to get an overview of the target
ICS, and facilitate disrupting or destroying all its target
B. ATTACK SURFACE TRANSITION devices. Industroyer can take control of an application server
Due to raised security awareness in ICS in general, and to directly manipulate or damage its target RTU and IED.
more commonly applied good security practices, the attack An example of a complete attack path is shown in Figure 3,
surface should have been minimized. However, from the which is adapted from [128], [129].
perspective of network entry points, the attack surface on
ICS has inevitably steadily expanded with the transition 3) AFTER-2020
from hardware-centric systems to more connected, hierar- With continuous integration of IT technologies into OT net-
chical systems, and then to highly networked IIoT-based works, and the convergence of IT and OT networks, ICS
systems. The newly introduced initial access points into networks become increasingly connected. Remote access
ICS include any Internet-accessible hardware device, in par- directly to ICS is enabled for many participants like techni-
ticular when the OS or firmware, application software or cians, maintenance engineers, third-party vendors and con-
associated libraries running on it have unpatched program tractors. According to a survey by SANS in 2019 [130], about
vulnerabilities. 12% of ICS are directly connected to the Internet, 23% of ICS
are indirectly (through a DMZ) connected to the Internet, and
1) PRE-2010 around 10% are directly connected to a third-party private
Before the attack wave on ICS in the 2010s started with infrastructure. These numbers should increase in the near
the Stuxnet, attacks on ICS were possible mainly due to a future. In some cases, the attackers’ initial access into an
lack of security awareness, misconfigured network perimeter ICS is achieved through a compromised third-party vendor
devices, few to no security countermeasures in the network or contractor.
and on end devices. At that time, attacks weren’t really As a large amount of data is collected from field devices,
sophisticated, only required scanning the Internet for find- and pushed directly into the cloud, points of exposure spread
ing ICS devices, for which default or guessed credentials further down to the lowest level of the Purdue Model. The
would suffice to log into them, and using them to access and more interconnected devices there are, the more targets
manipulate other devices in the network, potentially causing attackers can have and the quicker they can compromise
serious damage very easily. As a little more technical as it devices. IIoT gateways clearly become an important target
could get, attackers would have to exploit a vulnerability to for attackers. Like other third-party partners, cloud service
gain the initial access into an ICS, yet would not necessarily providers are not immune to cyber attacks. A security breach
need to develop an exploit by themselves. As it shows in the in a cloud service provider may also provide an initial access

48258 VOLUME 10, 2022

Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

integrity-based, memory sanitization-based, and memory

isolation-based ones. Each type of defense technique intro-
duces some security policy enforced against a specific
exploitation stage [4], which serves to prevent attackers from
corrupting any code or data (cf. Section V-A1, Section V-A5),
from executing injected code (cf. Section V-A2), from finding
correct memory addresses (cf. Section V-A3), from accessing
critical memory regions (cf. Section V-A6), or from tamper-
ing program execution flow (cf. Section V-A4). Once a secu-
rity policy is violated at runtime, the process may be forced
to terminate. The reference monitor of a security policy can
be implemented either with hardware support, e.g., W⊕X,
or purely by embedding some integrity checks into program
code, e.g., flow-integrity checks [4].

1) PROGRAM INTEGRITY
Program integrity-based defense mechanisms prevent cer-
tain memory locations of a running program from being
tampered, including code integrity policies, return-address
integrity checking, and code-pointer integrity policies. These
defense mechanisms also provide an important support for
other more advanced defense mechanisms, e.g., CFI. For
instance, CFI could be easily bypassed, if memory regions
FIGURE 3. An example of a complete attack path.
containing code can be modified, meaning that CFI checks
can be removed by attackers [8].
into an ICS network. Besides, the growing popularity of
embedded web servers in ICS field devices or controllers a: CODE INTEGRITY
may provide additional attack vectors. All these added entry A code integrity policy prevents attackers from modifying
points in modern ICS, as shown in Figure 2, greatly expand code regions in memory by setting the corresponding memory
the attack surface. pages as not writable. Existing hardware support renders this
policy effective and efficient. However, it cannot be fully
V. ATTACK TECHNIQUES AND DEFENSE TECHNIQUES enforced for programs that have features like self-modifying
Since ICS consist of both general-purpose computers, e.g., code, dynamic library loading or just-in-time (JIT) compila-
engineering workstations, data servers, and a variety of tion. The necessity of changing, loading, or generating code
embedded systems, e.g., PLC, IED, IoT gateways, in this at runtime leaves attackers a small time window to corrupt
section we discuss the attack techniques and mitigation tech- code on a writable page [4], [8].
niques in both fields, respectively. In the arms race between
attackers and defenders till now, it seems that the attack- b: RETURN ADDRESS INTEGRITY
ers have always managed to successfully exploit computer If code cannot be overwritten, an attacker may instead try to
systems despite various defense mechanisms in place. The corrupt and control some security-critical data in the memory,
reason for it is that these adversaries not only keep explor- in particular a return address in the stack, which determines
ing and exploiting vulnerabilities in software, but also have what to be executed next when the called function returns.
evolved to exploit weaknesses in most of defense techniques, To prevent this from happening, several defense schemes
i.e., weaknesses in their design or in their implementation. were proposed, e.g., stack canaries [131], shadow stack [132],
Furthermore, a hard truth is that, to secure a system, the zipper stack [133].
developer or defender must eliminate all vulnerabilities and As one of the first solutions against the classic stack-based
consider all possible attacks, while to attack a system only buffer overflow attacks, a stack canary [131] is a secret value
one flaw may suffice. This makes it particularly challenging inserted between the return address and the local variables in a
or impossible to guarantee or formally prove the security of a call frame. The integrity of this secret value is checked before
system. All defense mechanisms, at the end, may only serve the return instruction. A tampered value signifies memory
to reduce the attack surface to a limited extent, or raise the corruption in the stack and will result in process termination.
bar to a certain degree. Despite being popular and widely implemented due to low
performance overhead, stack canaries can prohibit only the
A. ON GENERAL-PURPOSE COMPUTERS classic stack-based buffer overflow attacks, but not attacks
Defense techniques are often classified into program capable of overwriting a return address without touching the
integrity-based, W⊕X-based, randomization-based, flow secret value before it.

VOLUME 10, 2022 48259

 Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

A shadow stack [132] improves the defense effectiveness 3) RANDOMIZATION

by moving to directly ensure the integrity of a return address. A fundamental requirement of all types of binary exploita-
A shadow stack is a separate, safe memory region, onto tion is the ability to find the correct memory addresses of
which a copy of every saved return address is pushed dur- target code and/or data. For efficient execution, the memory
ing a function call. Before a return instruction, the saved addresses of some code and data were deliberately fixed,
return address on the stack is compared with its copy on meaning that they stay the same at every program execution
the shadow stack. A mismatch will cause process termi- on every host. This makes it straightforward for attackers
nation. Yet, the shadow stack’s own safety cannot always to find the correct memory addresses with binary analy-
be guaranteed, leaving attackers a harder but still possible sis, and thereby to successfully exploit a system. Hence,
way to successfully attack a system by corrupting a return to make binary exploitation much harder and more unreliable,
address. a variety of randomization-based defense techniques were
To overcome the reliance on safe memory isolation, zip- proposed.
per stack [133] instead makes use of message authentica-
tion codes (MAC) function for protecting return addresses. a: ADDRESS SPACE LAYOUT RANDOMIZATION
It authenticates a return address by a chain structure using A well-accepted and widely deployed countermeasure is
MAC, and provides a more robust return address protection. address space layout randomization (ASLR) [137], which
However, like the previous two defense schemes, this one is generates random start addresses of code segments, (e.g.,
also inherently ineffective against attacks that do not need to libraries,) and data segments, (e.g., stack and heap,) in mem-
manipulate return addresses and/or have no reliance on the ory address space, for each program execution. This increases
stack, e.g., heap-based code-reuse attacks. the resiliency against both code-injection and code-reuse
attacks, by making it more difficult for attackers to reliably
c: CODE-POINTER INTEGRITY find the location of injected code or existing program code.
Whereas above-mentioned techniques intend to only ensure Trying to access a wrong memory address will likely result
the integrity of return addresses on the stack, another defense in a segmentation fault and failed exploitation.
mechanism called code-pointer integrity (CPI) [134] aims to However, only randomizing the start address of a segment
guarantee the integrity of all code pointers including indi- means that, once attackers find this address, they can find
rect jump targets, indirect call targets and return addresses. the address of a target object (in this segment), which has a
A code-pointer integrity (CPI) policy can be enforced to fixed offset to the start address. Hence, ASLR can be defeated
prohibit (both stack-based and heap-based) control-oriented by information disclosure of only a few memory addresses.
exploitation (cf. Section II-B1), by protecting the integrity To make randomization-based defense more robust, more
of all code pointers as well as data pointers used to access fine-grained randomization techniques were proposed.
code pointers. These sensitive pointers can be identified by
a compiler, and then stored in a protected memory region b: INSTRUCTION LOCATION RANDOMIZATION
at runtime, which is isolated from the rest of the memory In order to thwart such (two-stage) attacks, in which an
address space [134]. The access to the protected pointers attacker first finds the memory addresses of target objects by
is only given to the program code that is supposed to have leaking the start address of the corresponding segment, and
the need. However, the memory isolation sometimes relies then conducts the real attack, instruction location randomiza-
on information hiding, which is fundamentally susceptible tion (ILR) [138] is introduced. ILR generates unpredictable
to information disclosure [135]. Furthermore, CPI does not code layout to create randomness in a code segment itself, i.e.,
prohibit data-oriented exploitation (cf. Section II-B2). instructions are not placed sequentially but rather distributed
randomly across the memory address space. To execute a
2) W⊕X program with randomly scattered instructions properly, ILR
Instead of preventing the corruption of code or data like creates, during static analysis, a so-called fallthrough map
program integrity-based defense mechanisms, another type specifying the execution order of those instructions. Despite
of defense allows memory corruption and code injection, but being more effective in hiding instructions locations, ILR
prevents its execution. The non-executable (NX) data pol- suffers from not only high performance overhead, but also
icy [59], or data execution prevention (DEP) policy [136] in code coverage deficiencies due to imprecision of the static
Windows parlance, makes all memory pages containing data analysis [70].
non-executable. Together with the code integrity policy, this
creates the widely enforced W⊕X (writable xor executable) c: DATA SPACE RANDOMIZATION
policy, meaning that a memory page is either writable or exe- Another approach providing a higher level of randomness
cutable. However, as mentioned in Section II-B1, the W⊕X and stronger resilience to information leak is data space
policy can be rendered useless by code-reuse attacks or hybrid randomization (DSR) [139]. Instead of randomizing the loca-
exploits. tion of some data, DSR randomizes the representation of all

48260 VOLUME 10, 2022

Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

data by encrypting it using different keys. When variables instruction, the control flow of the program execution will
are encrypted with different keys, a memory location (of be unknowingly diverted. Predetermining the control-flow
a variable) overwritten by an attacker through a different graph (CFG) of a program during static analysis and then
variable will have an unexpected value after decryption, mak- inserting a stateless check before each indirect control-
ing it useless for the attacker. As a result, DSR provides a flow transfer,21 can detect the deviation of the intended
strong protection against not only control-oriented exploita- control-flow at runtime.
tion but also data-oriented exploitation. Nevertheless, fine- CFI, proposed in [68], [158], is a key defense mechanism
grained DSR may cause high performance overhead [67]. that relies on a CFG, which can be seen as some form of
Furthermore, DSR is not binary compatible, meaning that finite state machine. It restricts the target of each indirect
instrumented binaries are not compatible with unmodified control-flow transfer to only a few locations, as these loca-
libraries [4]. tions are considered as legitimate during static analysis and
marked with a unique label. At runtime, before an indirect
d: AUTOMATED SOFTWARE DIVERSITY control-flow transfer, it is checked whether the transfer target
The essence of randomization-based approaches is protect- has the right label. If not, the process is forced to terminate.
ing programs by introducing program diversity,20 and this By doing so, most control-oriented exploitation will fail, as a
idea goes back to nearly 30 years ago [140]. The proposed control-flow hijacking attempt will most likely fail the CFI
program evolution aims to create syntactically different but check, and results in program execution termination. Note
semantically equivalent versions of an original program. that the effectiveness of CFI heavily relies on enforced W⊕X
A generalization of randomization-based approaches is auto- policy, as it ensures that code (including CFI checks) cannot
mated software diversity [141], [142], which presents var- be tampered, and injected code (for which there is no CFI
ious forms of randomization/diversification, with distinct check) cannot be executed.
diversification targets, at different levels of granularity, and
introduced in individual phases of the software life-cycle. i) FORWARD-EDGE CFI
Other influential research works on randomization-based Forward-edge CFI inserts checks for indirect jump and indi-
defense include [143]–[146] [147]–[150] [151], which fit to rect call instructions. Whereas an indirect jump instruction
one of above-listed types, but are not discussed in this paper can be introduced by a switch statement in source code,
due to space constraint. Nevertheless, like the discussed ones, an indirect call instruction is normally resulted from a func-
all randomization-based defense techniques suffer from a tion call. At runtime, for an indirect jump or call, there is only
fundamental problem. That is, they rely on secrets, which one correct jump target or call target, respectively. However,
can be eventually guessed (through brute-force) or leaked during static analysis, the jump target or call target cannot be
(through some flaws). Furthermore, even without directly completely determined, but rather restricted to a few possible
disclosing the memory layout, just-in-time ROP [152], just- locations. Hence, all these possible locations are considered
in-time spraying [153], [154], side channel attacks [155], as valid and receive the same unique label. This, though,
[156], and address-oblivious code-reuse attacks [157] etc. leaves an attacker a slight possibility to hijack the control
prove, again and again, that randomization-based defenses flow, when one of these locations happens to be the one to
cannot provide sufficiently strong protection as promised which the attacker aims to redirect the control flow. As a
or expected. That is to say, randomization-based defense result, the program execution is compliant to the enforced CFI
approaches, at best, add only an extra necessary hurdle for policy, but deviated from the expected program behavior. The
attackers to overcome. more statically determined valid targets an indirect jump or
call has, the better the attacker’s chance is.
4) FLOW INTEGRITY
Apart from program integrity-based defenses, there is a ii) BACKWARD-EDGE CFI
large body of research focusing on another type of integrity Backward-edge CFI aims to prevent a return instruction from
enforcement, i.e., flow integrity of a running program, includ- being used for control-flow hijacking. Similarly, CFI can-
ing control-flow integrity (CFI) and data-flow integrity (DFI). not guarantee that, at runtime, a return instruction really
returns to its true invoking site, but rather to one of its
a: CONTROL-FLOW INTEGRITY seemingly legitimate callers, as this function may be called
When a code pointer, e.g., a return address, is corrupted from (many) different places of the code at different times.
and then loaded into the instruction pointer register through The number of callers of a function may be noticeably higher
an indirect control-flow transfer instruction, e.g., a return than the number of locations to which an indirect jump/call
20 It is often said that security by obscurity is a bad strategy. However,
may lead, making backward-edge CFI even less precise than
the idea of program diversity is to introduce some obscurity and thereby to
enhance the complexity and cost for attackers. This may sound contradictory.
We stress that security by obscurity is a bad thing, only when the obscurity 21 In contrast, a direct control-flow transfer instruction has a fixed target
mechanism is the solely deployed countermeasure, and even not properly memory address/offset, which cannot be manipulated by attackers due to
designed and implemented. enforced code integrity.

VOLUME 10, 2022 48261

 Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

forward-edge CFI. That is, statically computed CFG alone fine-grained DFI may provide a stronger protection, but suf-
inevitably leads to coarse-grained CFI. fers from very high performance overhead and is not yet very
Coarse-grained CFI [159]–[162] are considered as practical [54], [67].
practical and suitable for real-world deployment due
to comparatively low overhead. Nevertheless, due to 5) MEMORY SANITIZATION
imprecision and inherent limitations of static analysis, Whereas above-mentioned defenses aim to detect or pre-
the number of allowed target locations for an indirect vent attacks at some late stage of a bug’s exploitation,
control-flow transfer is normally larger than necessary, result- another line of research focuses on fighting against the
ing in some degree of over-approximation [54]. The effec- underlying root cause, i.e., preventing memory corruption
tiveness of a coarse-grained CFI depends on the degree of from ever happening. This is done by memory sanitiza-
over-approximation. As shown in [54], coarse-grained CFI tion [174], which includes finding and removing vulner-
in general can be easily defeated by advanced code-reuse able code [175] or unnecessary code [176]; extending C
attacks. language to safer versions [177], [178]; integrating spatial
To enhance the precision of CFI, in the original CFI safety checks [179], temporal safety checks [180] to provide
works [68], [158], statically computed CFI checks are stronger policies. Like above-mentioned defenses, memory
coupled with an above-mentioned shadow stack, moving sanitizers also rely on program instrumentation, e.g., insert-
towards more fine-grained CFI. Whereas the statically ing reference monitors at some level. Memory sanitizers
inserted CFI checks are stateless, a shadow stack adds some normally provide stronger defenses and potentially stop any
state-awareness during program execution, and guarantees binary exploitation, at the cost of very high performance
that each return instruction really goes back to the initial overhead.
caller. However, having to keep a shadow stack for each exe- To uncover memory corruption bugs before a program’s
cution thread introduces non-negligible overhead. Further- release and hence to scale down the attack surface, defend-
more, problem may occur in certain programming constructs ers propose frameworks for conducting inter-procedural,
in which a return instruction does not always need to return context-sensitive analysis in not only user application pro-
to its direct caller, that is, correctly tracking a shadow stack grams but also operating system kernels [175]. To further
is not trivial [60], [70]. lower attackers’ chance to find and exploit a bug, some
To reduce performance degradation, hardware-assisted developer-intended program features and the code, which are
CFI [163]–[166] [6], [167]–[169], which requires extend- not necessarily useful for every program user or not used
ing an instruction set architecture, i.e., adding new CPU in a particular deployment context, can also be removed
instructions, becomes more relevant and attractive. Intel individually during program instrumentation and before the
CET [6], [169] showcases a real-world deployment of program’s deployment [176]. To make C programs safer,
hardware-assisted CFI. Although neither software-only a program transformation system [178] is proposed to extend
CFI nor hardware-supported CFI can completely prevent C with type safety, in which type safety is either statically
control-oriented exploitation [7], [170], they do make most verified or guaranteed by inserted checks at runtime. Another
code-reuse attacks highly unreliable and unsuccessful. As a C extension [177] also combines static analysis and runtime
result, data-oriented exploitation has attracted much attention checks to rule out (almost) all safety violations in a C pro-
of attackers in recent years [171]. gram, while preserving programmers’ control over low-level
details.
b: DATA-FLOW INTEGRITY A strong defense can also be provided by embedding
An effective way to defend against both control-oriented spatial memory-safety checks and temporal memory-safety
exploitation and data-oriented exploitation is enforcing a checks in a program. Complete memory safety can be ensured
DFI policy, which is originally proposed in [172] short after by maintaining and correctly tracking both bounds infor-
the proposal of CFI. To enforce a DFI policy, a data-flow mation and allocation information as metadata for every
graph (DFG) is constructed during static analysis for a pro- pointer. Whenever a pointer is dereferenced, this metadata is
tected program, and the program code is instrumented to used to examine whether the pointer stays inside the bounds
ensure that the data flow does not deviate from the DFG at of the pointed object, and whether the pointed object is
runtime. But instead of checking indirect branching instruc- still valid. The original pointer extended with the associated
tions, as it is done in CFI, DFI checks each read opera- metadata turns into a so-called fat pointer. With fat point-
tion, to prevent any corrupted data from being used. That ers, an instrumented program cannot interact with uninstru-
is, DFI aims to detect the corruption of the data that is mented libraries, as they cannot correctly interpret and update
the next to be accessed. A DFG may include all program fat pointers. To address this compatibility problem, the meta-
data for enhanced precision [172], or only consider a small data can also be separated from the original pointer and stored
portion of security-critical data for reduced performance in a disjoint metadata space like in SoftBound [179] and
overhead [173]. Like a statically computed CFG, a DFG CETS [180]. Whereas SoftBound ensures spatial memory
obtained from static analysis is only an over-approximation safety for a C program, CETS provides temporal memory
of a complete DFG, leading to coarse-grained DFI. More safety for it. When combined, a complete memory safety

48262 VOLUME 10, 2022

Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

can be guaranteed, but with a performance overhead over 1) CONSTRAINTS

100% [180]. Even on general-purpose computers, some effective defenses
are normally not implemented or enabled due to high
overhead. According to [4], defenses causing over 10% per-
6) MEMORY ISOLATION
formance overhead will unlikely get widely deployed. When
Lastly, memory isolation-based defenses are also widely applied to embedded systems with their tight constraints,
studied and implemented for countering binary exploita- the performance penalty issue is further exacerbated. Solely
tion, by restricting an attack to a confined memory address considering this, it makes many defenses developed for
space. Disjoint memory regions are used for trusted and general-purpose computers already practically infeasible.
untrusted program code, respectively, to contain the mem- Another important issue which makes many defenses
ory access and control-flow transfer behavior of untrusted technically infeasible is hardware limitation. That is, most
program code. Software-based Fault Isolation (SFI) tech- embedded systems (except a few high-end embedded sys-
niques [181]–[184] [185], [186], also referred to as sandbox- tems like smartphones) do not have a memory management
ing techniques, establish logical protection domains within a unit (MMU), and hence do not support virtual memory and
process, and constrain the untrusted code’s memory access memory paging, which underpin many necessary and univer-
and control-flow transfer to a designated logical memory sally deployed security mechanisms on general-purpose com-
address space, referred to as the fault domain. Similarly, puters. For instance, both the enforcement of code integrity
Intel Software Guard Extensions (SGX) [187] allows pro- policy and the enforcement of DEP policy rely on the pres-
gram code to designate a protected memory region, called ence of a MMU. Without these defenses, the most basic code
enclave, for hosting security-critical code and data, in order corruption attacks and code injection attacks become easily
to prevent compromised code from accessing it. However, achievable. Although CFI itself does not require a MMU,
memory isolation-based countermeasures can, at best, limit it can be rendered useless by code corruption attacks (through
the damage that an attacker can cause, but not prevent it manipulating CFI checks) and code injection attacks (since
from happening. Besides, these defense mechanisms are also for injected code there is no CFI check).
breakable [188], [189]. Above-mentioned randomization-based defenses like
ASLR also become much weaker and less relevant, when
B. ON EMBEDDED SYSTEMS it comes to embedded systems. Because embedded systems
According to Embedded Markets Study [190], [191], ICS typically run on 32-bit, 16-bit or even 8-bit architectures, for
are the largest host of embedded devices. As it shows in which the memory address space is much smaller and thereby
the past [9]–[12] [13]–[16] [17]–[20], despite their limited the entropy sources are very limited. This makes defenses
computation capabilities, embedded systems are also prone relying on information hiding, like randomization-based
to fall victim to binary exploitation. One of the main reasons defenses or shadow stack, generally more easily breakable.
is that embedded systems are predominantly developed in For example, as it shows in [192], the protection provided by
unsafe programming languages like C, C++ or assembly ASLR is limited on 32-bit architectures, not to mention on
languages [190], owing to the fact that they support high 16-bit and 8-bit architectures.
performance and low-level hardware control. As defense
mechanisms are widely deployed on modern general-purpose 2) AVAILABLE DEFENSE TECHNIQUES
computers, which can raise the bar for conducting binary Although defense techniques for general-purpose comput-
exploitation, attackers now typically focus on targets that ers are not directly applicable to embedded systems, they
either do not enable defense mechanisms or cannot imple- are highly influential for developing defenses for embed-
ment them. Embedded devices often cannot directly imple- ded systems. That is, as shown in the following, similar
ment many of above-mentioned defense mechanisms due approaches are proposed for embedded systems, but with the
to, in particular, performance overhead and hardware limi- limitations of embedded systems in consideration. Besides,
tation. Besides, embedded systems are normally difficult to in comparison to software-only solutions, hardware-based
update, and it is not uncommon that many of them contain solutions appear to be more favored in embedded systems due
unpatched publicly known software vulnerabilities for a long to relatively low overhead.
time. Hence, attackers will more likely shift their focus to
the low-hanging fruit—embedded devices in ICS—to yield a: CODE INTEGRITY
easy wins. In this section, we first discuss the constraints that The easiest way to carry out binary exploitation would be
prohibit the direct implementation of some above-mentioned directly corrupting program code in memory. On general-
defense techniques on embedded systems. Next, we review purposes computers, this is straightforwardly prevented by
some defense mechanisms designed for embedded sys- a code integrity policy, which leverages a MMU to real-
tems and with the limitations of embedded systems in ize access control, i.e., all memory pages containing code
mind. cannot be modified. However, a MMU causes unacceptable

VOLUME 10, 2022 48263

 Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

performance overhead, i.e., due to management of page guaranteed that this value is never spilled to the main memory.
tables, for most of embedded systems. To overcome this, As a result, it removes the need to ever store return addresses
a memory protection unit (MPU) [193], [194], often seen on the stack, and makes sure that return addresses are never
as a lightweight MMU, is implemented in a large number writable except by an authorized instruction.
of (high-end) embedded systems. A MPU does not support
virtual memory, but provides access control by arranging c: POINTER INTEGRITY
the memory address space into several memory regions with Enforcing return address integrity can only prevent
associated access permissions. Consequently, not only the backward-edge control-flow hijacking. In order to thwart
enforcement of a code integrity policy is feasible, but so are forward-edge control-flow hijacking, other code pointers
other defenses heavily relying on memory access control. like function pointers also need to be safeguarded. High-
end embedded systems running on ARM processors with
b: RETURN ADDRESS INTEGRITY ARMv8-A architecture [199] now have instructions for
At first glance, a return address is safely stored in many pointer authentication to resist attacks leveraging corrupted
embedded systems due to a dedicated register called link pointers. It makes use of cryptographic MAC, referred to
register. In many embedded devices employing RISC instruc- as pointer authentication codes (PAC), and places a PAC
tion set architectures, e.g., ARM, PowerPC, SPARC, a link into unused bits of a pointer value (in 64-bit architectures),
register is used to hold the return address of a function call. before the pointer is written to memory. Upon using this
This is not only more efficient (as accessing a register is pointer, its integrity is first verified. As a result, an attacker
much faster than accessing the stack on the main memory) has to find the correct PAC (in addition to a memory corrup-
but also safer, as an attacker cannot directly tamper a value tion bug), in order to corrupt a protected pointer. However,
on a register. However, nested function calls are very typical pointer authentication may be vulnerable to pointer substi-
in a program, and in this case some return addresses have to tution attacks, in which an authenticated function pointer
be stored on the stack. As a result, attackers typically try to is replaced with another authenticated pointer pointing to
overwrite a return address on the stack to hijack the control a different, attacker-intended memory location [200]. As a
flow. countermeasure, an enhanced scheme is proposed in [201],
As a countermeasure on embedded systems, defend- which enforces pointer integrity for all code and data pointers,
ers aim to protect return addresses by either implement- coupled with pointer type safety ensuring a pointer is of the
ing a dedicated return stack [195], or deploying a shadow correct type.
stack [196], [197], or directly ensuring the integrity of return
addresses [198]. To safeguard low-end embedded systems, d: eXecute-ONLY-MEMORY
a light hardware modification is introduced in [195] (for AVR By setting memory pages or regions as either writable (and
microcontrollers), which splits the stack into a normal data readable) or executable (and readable), i.e., W⊕X policy,
stack (without return addresses) and a return stack (contain- through a MMU or MPU, respectively, it prevents attack-
ing solely return addresses). The return stack is stored at a ers from both corrupting code and executing injected code,
different location in memory, and the access to the return thereby largely restricting attackers’ ability. However, this
stack is restricted to return and call instructions to prevent does not thwart memory disclosure attacks, in which attack-
unauthorized modification, which is realized by the hardware ers disclose sensitive information from code regions, e.g.,
modification. in order to conduct code-reuse attacks. To defeat memory dis-
Like on general-purpose computers, a shadow stack can closure attacks, eXecute-Only-Memory (XOM) is proposed,
be implemented to protect return addresses of a program which states that some code regions are solely executable,
running on embedded devices. But preventing a shadow i.e., neither writable nor readable. Although XOM is sup-
stack itself from being tampered is more difficult on embed- ported in high-end processors by extending memory access
ded systems due to limited entropy sources and memory permissions to include execute-only (XO) permission [202],
isolation. To make a shadow stack safer, an approach called it is not built into most of embedded systems. To enable this
Silhouette [197] is designed for various ARM architectures, feature in (low-end) embedded systems, researchers present
which enforces an efficient intra-address space isolation uXOM [203] for ARM Cortex-M22 —one of the most popu-
dubbed store hardening. It relies on some special store lar processors in low-end embedded devices. In addition to
instructions of ARM processors and a MPU to set memory a MPU, uXOM also leverages unprivileged load and store
access permissions. By doing so, it creates a logical sepa- instructions offered by ARM Cortex-M processors. It con-
ration between code associated to shadow stack operations verts most memory instructions into unprivileged ones and
and other program code. Another approach ensuring return sets code regions as privileged, so that these instructions can-
address integrity is µRAI proposed in [198], which turns a not access code regions. For memory instructions that cannot
general-purpose register into a dedicated register called status
register. The value in this register is used in combination 22 Note that XOM cannot be implemented simply by configuring the MPU
with (during program instrumentation) inserted direct jump in a Cortex-M processor, since read permission is needed for a memory
instructions to resolve the correct return addresses, and it is region to be executable [203].

48264 VOLUME 10, 2022

Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

be turned into unprivileged ones, they are instrumented with (TEE), a new security architecture [206] is proposed, which
verification routines ensuring that uXOM’s protection will suggests moving trusted programs to the normal world and
not be broken. introducing strongly isolated compartments in the normal
world.
e: RANDOMIZATION
Randomization-based defenses increase the resilience h: MEMORY TAGGING
against, in particular, code-reuse attacks. To address unique Another important hardware-based feature is memory tag-
challenges of implementing effective randomization mech- ging, which is designed to provide a very promising and
anisms on low-end embedded systems, a hardware-based robust defense against binary exploitation. Memory tagging
fine-grained randomization technique called MAVR [16] is is currently supported in some SPARC processors and ARM
proposed for 8-bit AVR microcontrollers. MAVR extends v8.5 architecture, in which the implementation is called
a main processor with an external flash memory and an SPARC ADI [207] and ARM MTE [208], respectively. For
additional low-cost processor dubbed master processor. The instance, two types of metadata are used in ARM MTE, i.e.,
external flash memory is used to store the unrandomized address tags and memory tags. Whereas an address tag is
program code and symbol information. The master processor a 4-bit value inserted to the topmost unused byte of each
is responsible for the randomization, and uploading random- pointer (only in 64-bit architecture), a memory tag is also
ized program code to the main processor. In order to make it a 4-bit value associated with every aligned 16-byte memory
harder for attackers to find out gadgets locations, it shuffles region [209]. These tags are handled by newly introduced
the function blocks within the code segment, instead of just instructions. When a heap region is allocated, a random 4-bit
start addresses of all segments. value is selected to mark both the pointer and the heap region.
Upon accessing the memory region, it is verified whether the
f: CFI
address tag and memory tag match. A mismatch will cause a
In order to enforce CFI on real-time embedded systems, hardware exception.
but not create extra unpredictable workload on the proces- Note that defenses like ARM pointer authentication, ARM
sor, a hardware-based solution called OCFMM is presented TrustZone and ARM MTE rely on advanced processor fea-
in [18]. OCFMM introduces a dedicated hardware module tures, and hence mostly apply to high-end (64-bit) embedded
to perform CFI checks for the program running on a 32-bit systems. Low-end embedded systems are in general less pro-
SPARC processor, in which it directly hooks into the pro- tected due to more strict resource constraints and hardware
cessor and tracks the control flow of the program. This limitations.
module has its own isolated memory unit for preventing
itself from being manipulated. Another hardware-assisted
VI. DETECTION TECHNIQUES AND EVASION TECHNIQUES
CFI approach for embedded systems proposed in [164] does
not add an additional hardware module, but rather introduces Fully enforced memory safety is rare in real-world systems,
new instructions through hardware modification on a 32-bit in particular due to performance and compatibility issues [8].
Intel processor designed for embedded systems. Besides, This often makes another line of defense become necessary,
software-only CFI, such as RECFISH [196] targeting ARM i.e., intrusion detection. Often used as a secondary defense,
Cortex-R processors, is also proposed for real-time embedded intrusion detection can lessen the trust needed in and the
systems. reliance on a front-line defense mechanism. In this section we
mainly discuss detection techniques against binary exploita-
g: MEMORY ISOLATION tion, and evasion techniques. They differ from defenses dis-
To constrain the ability of an attacker who already partly or cussed in Section V, in that they are either heuristics-based
completely exploited an unprivileged program, and to prevent detection, and/or the detection is conducted from a connected
the attacker from further exploiting other security-critical device in the same physical/virtual network (rather than on
programs and resources, memory isolation-based defenses the monitored device itself), i.e., remote runtime attestation.
can be deployed not only on general-purpose computers, Heuristics-based detection approaches have the advantage
e.g., with Intel SGX [187], but also on embedded devices, that they can apply machine learning techniques, e.g., rule
e.g., with ARM TrustZone [204]. However, ARM TrustZone learning based IDS [210]. Remote runtime attestation is in
offers a limited degree of isolation (and security) in com- particular suitable for (low-end) embedded systems due to
parison to Intel SGX, because it only separates programs little to no extra performance overhead on a monitored device
into two areas, i.e., a normal world and a secure world, but itself. Hence we see a great potential in these detection tech-
provides no strong compartmentalization within the areas. niques, especially when combined.
As a result, the attack surface of the secure world expands
with increased number of trusted (but potentially vulnerable) A. HEURISTICS-BASED DETECTION
programs [205]. To make ARM TrustZone more compara- 1) SHELLCODE DETECTION
ble to Intel SGX, i.e., establishing user-space enclaves and Early detection-based solutions focus on shellcode detec-
enabling unrestricted use of trusted execution environments tion, which allows detecting not only known exploits but

VOLUME 10, 2022 48265

 Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

also unknown exploits, irrespective of underlying program obtained from observed ROP attacks, e.g., an excessive use
bugs being exploited. The first generation of shellcode detec- of return instructions in a short time [61], [62], a viola-
tion bases on static analysis of network traffic [211], [212], tion of the last-in, first-out stack invariant [219]. To make
in which network input data is first disassembled and then these kind of techniques more generic and practical, i.e.,
compared with predefined pattern(s). A pattern can be a with the ability to detect also JOP and COP attacks, and
combination of heuristics-based features, e.g., NOP-sled, no reliance on side information and binary instrumentation,
identified through studying a large sample of shellcode. How- ROPecker [220] presents a new detection system, which
ever, static analysis-based detection approaches are limited in detects ROP attacks by leveraging a combination of offline
that they cannot correctly deal with code obfuscation tricks binary analysis results, runtime execution flow information,
employed by shellcode authors to evade detection. and records in last branch record (LBR) registers.
Hence, researchers later turned to dynamic analysis-based
detection techniques coupled with a processor emulator, 3) PROGRAM EXECUTION ANOMALY DETECTION
implemented either directly on the monitored/protected host In order not to be restricted to any specific type of program
[213] or on a network-level monitor/detector [214]–[216]. exploitation, another group of detection techniques aim to
As a result, obfuscated/encrypted shellcode can be detected, detect all kinds of attacks by focusing on a program’s normal
as the shellcode needs to be first deobfuscated/decrypted execution behavior, rather than relying on the characteristics
during its execution. For instance, in [216], each network of a specific kind of attack. A normal execution behavior
input, i.e., a byte sequence, is mapped to the memory space is defined in a normal program execution model, and any
and executed by the processor emulator for several times deviation from it, i.e., an anomaly, is deemed as malicious.
(each time beginning with a different byte in the input). The For example, a normal execution model is defined in [221]
execution results are matched against patterns based on run- as a set of conditions on program states. An execution
time heuristics and identified to be inherent in different types behavior is considered as benign, if the program counter
of shellcode. Nonetheless, the aforementioned approaches and the call stack prove to be valid and consistent with
may be still susceptible to evasion, due to insufficient con- each other for all program states. Nevertheless, data-oriented
text information only obtainable from a real target process. exploitation is generally harder to detect, as it may not
To solve this issue, a new approach [217] suggests frequently cause any control-flow anomaly during program execution.
taking a snapshot of a real target process’s memory space To improve the effectiveness of anomaly-based detection
containing a to-be-examined input, and sending the snapshot against data-oriented attacks on control programs in cyber-
to a shellcode detector for more realistic examination. physical systems, a new approach [222] also incorporates
runtime execution semantics checking, including event iden-
2) ROP DETECTION tification and dependence analysis with respect to physical
After the advent of code-reuse attacks, especially ROP environments.
attacks, shellcode detection-based approaches seem to
become much less relevant. That is, shellcode detection B. REMOTE RUNTIME ATTESTATION
may be effective against code-injection attacks and hybrid To detect the occurrence of a runtime attack on a (resource-
exploits, but not against pure code-reuse attacks, as they do constrained) embedded system, but without introducing
not contain any code. Consequently, defenders proceeded to non-negligible performance overhead on it, remote runtime
develop methods for detecting ROP exploits. One of these attestation is a popular choice, as the detection process, i.e.,
methods is ROPscan [218], which aims to catch ROP pay- the computation, is mainly performed on the remote server.
loads, i.e., a byte sequence consisting of gadgets addresses, Initially, remote attestation is introduced to detect code mod-
from monitored network traffic. The emulator in ROPscan ification or malware on an embedded device, in which the
is initialized with a snapshot of the memory space of a trusted server, called verifier, requires the prover running on
targeted/protected process. ROPscan speculatively drives the the embedded device to compute a checksum of its code
execution of code existing in the memory space, to which memory (with a nonce included to prevent replay attacks),
a to-be-examined network input (presumably interpreted as and the verifier will receive it and check it. Conventional
valid memory addresses) points to. The execution results are remote attestation is referred to as static attestation, because
examined according to some runtime heuristics, which will it only aims to detect the presence of modified code at load-
classify the input data as benign or malicious. Besides, such time, but is unable to detect binary exploitation at runtime.
a ROP detector can be integrated into a shellcode detector for To complement static attestation, a number of runtime attes-
creating a stronger defense. tation schemes are proposed to catch runtime attacks by
Some other runtime-behavior heuristics-based ROP detec- attesting not only control flow [223]–[225], but also data
tion techniques, like DROP [61], DynIMA [62], ROPde- flow [226] of a program running on a monitored/protected
fender [219], embed the checks directly in program code, embedded system.
instead of relying on a dedicated emulation-based (network) C-FLAT [223] is an early runtime attestation scheme that
monitor. An alarm is triggered, if a predefined condition is verifies the control-flow integrity of a running program on an
met. These detection techniques base on runtime heuristics embedded system. It requires the remote verifier to generate

48266 VOLUME 10, 2022

Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

the CFG of the attested program beforehand, and needs to emulation-based detection systems [228]. For instance, there
instrument all branch instructions of the program, to let them is always an emulation gap, i.e., some difference between
be intercepted by a runtime tracer on the prover during exe- a real target system and an emulator, which can abused to
cution. Besides, it has a trust anchor, i.e., ARM TrustZone, circumvent detection. An attacker may construct shellcode
on the prover to securely measure and report the program’s which will be executed completely and correctly only when
runtime behavior, i.e., its execution paths, to the verifier. it is on a real target system.
However, C-FLAT has limited practicality due to its reliance As it shows in [229], ROP detection techniques based
on the hardware extension ARM TrustZone and program on execution heuristics or anomalies [218]–[221] can also
instrumentation, and thereby noticeable performance penalty be evaded by a strong adversary. For instance, a length-
on the embedded system itself. To improve the efficiency and based detection approach classifies a code sequence as nor-
usability of runtime attestation, in particular by eliminating mal or ROP gadget depending on its length. A stealth ROP
the need of software instrumentation, a hardware-assisted attack will contain only code sequences that are longer than
approach LO-FAT [224] introduces dedicated hardware mod- expected, and hence would be incorrectly classified as nor-
ules, i.e., a branch filter and a loop monitor, to track the mal. Furthermore, dispatcher-like behavior or intensive use
attested program’s control flow in parallel to the main pro- of indirect jumps or calls may be considered as an execution
cessor. By doing so, it does not stall the attested program’s anomaly and taken as an indicator to detect JOP or COP
execution and allows efficient control-flow attestation. attacks. However, this kind of detection could be circum-
Nevertheless, both C-FLAT and LO-FAT are susceptible vented by selecting long-running functions and changing dis-
to two types of TOCTOU attacks, provided that the attacker patchers periodically [65]. Hence, evasion-resilient detection
has physical access to the embedded device [225]. This is due techniques are of special interest. It is undoubtedly critical to
to the fact that they only attest indirect control transfers, but identify properties that are truly representative and indispens-
instructions within a basic block are not verified, and hence able in an attack vector or during an attack, but non-existent
allowed to be replaced by malicious code. A more resilient in a benign program or during a normal execution. These
runtime attestation scheme ATRIUM [225] mitigates these fundamental differences need to be captured and used to
attacks by tracking and attesting both the executed instruc- design a reliable and sound detection scheme that cannot be
tions and the control flow of a target program, with an attesta- circumvented.
tion engine separated from the processor. Although ATRIUM
can detect additional memory manipulation attacks, it is inca- VII. CONCLUSION
pable of identifying DOP attacks, which do not change a With the aim of providing an appropriate assessment, and
victim program’s control flow. As a countermeasure against raising situation awareness adequately, we present a sys-
sophisticated DOP attacks, LiteHAX [226] suggests accu- tematic study concerning binary exploitation in ICS. In this
rately and continuously tracking not only control-flow but work, we first provide a comprehensive analysis of binary
also data-flow information of an attested program, which exploitation. We analyze its causes from different angles and
is realized by its introduced hardware modules tightly inte- give an overview of the most critical program bugs. The two
grated with the main processor. types of binary exploitation, i.e., control-oriented and data-
oriented, are extensively discussed, so are the varying con-
C. EVASION sequences. The connection to modern malware is explained
It is often said that the easiest way to break system security and illustrated with many real-world examples in the last
usually is to circumvent it rather than defeat it. A funda- decade, which serve to indicate the popularity and severity
mental problem of detection-based defenses is that they are of binary exploitation and exploit-based malware in ICS.
prone to evasion. The early generation of shellcode detec- Besides, a number of exploitation purposes are summarized
tion mechanism [211] aims to detect plain shellcode using for the past ICS cyber incidents.
static analysis. As attackers always seek a way to evade To further argue that binary exploitation will tend to be
detection, code obfuscation techniques such as polymor- a more serious problem, we reveal a drastic shift from a
phism and metamorphism [227] are employed to circumvent well-isolated and -segmented network structure to a more
such defense. Various forms of the same plain shellcode interconnected and complex structure with converged IT and
can be generated by encrypting it with individual random OT networks, and point out the added network entry points as
keys and appending a decryption routine to it, so that it well as the attack surface transition. Next, an extensive list of
can self-decrypt when executed. In the face of polymorphic available defense techniques for general-purpose computers
and metamorphic shellcode, defenders moved to developing is presented, as well as another list of defense mechanisms
dynamic analysis-based detection approaches, which run sus- designed for embedded systems and with the limitations of
picious code in an emulator and check the execution results embedded systems in mind.
with some runtime heuristics, and hence are not hindered by Lastly, we discuss detection-based defenses, often used
code obfuscation tricks. However, attackers, again, proved to as a secondary defense, as we see a great potential in
be capable of achieving a higher level of evasiveness, in which heuristics-based detection and remote runtime attestation
they exploit both design flaws and implementation flaws in techniques for ICS. In order to reduce the damage that an

VOLUME 10, 2022 48267

 Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

attacker can cause, it is ideal to catch and halt the attack [17] E. Buchanan, R. Roemer, H. Shacham, and S. Savage, ‘‘When good
as early as possible, i.e., detect and prevent the program instructions go bad: Generalizing return-oriented programming to RISC,’’
in Proc. 15th ACM Conf. Comput. Commun. Secur. (CCS), 2008,
exploits. An early-stage detection and mitigation of an attack pp. 27–38.
also means a less dramatic clean-up of the victim system. [18] F. A. T. Abad, J. V. D. Woude, Y. Lu, S. Bak, M. Caccamo, L. Sha,
In particular, ICS asset owners should prioritize employ- R. Mancuso, and S. Mohan, ‘‘On-chip control flow integrity check for
real time embedded systems,’’ in Proc. IEEE 1st Int. Conf. Cyber-Phys.
ing network-based detection techniques, as it is impractical Syst., Netw., Appl. (CPSNA), Aug. 2013, pp. 26–31.
to deploy resource-intensive defenses on many embedded [19] S. Checkoway, A. J. Feldman, B. Kantor, J. A. Halderman, E. W. Felten,
devices, especially when the systems are already under and H. Shacham, ‘‘Can DREs provide long-lasting security? The case of
return-oriented programming and the AVC advantage,’’ in Proc. Electron.
operation. However, the biggest challenge is still how to Voting Technol. Workshop/Workshop Trustworthy Elections, 2009.
identify fundamental properties in attack vectors and design [Online]. Available: https://www.usenix.org/conference/evtwote-09/can-
evasion-resilient detection techniques. dres-provide-long-lasting-security-case-return-oriented-programming-
and
[20] S. Checkoway, D. McCoy, B. Kantor, and D. Anderson, ‘‘Comprehensive
REFERENCES experimental analyses of automotive attack surfaces,’’ in Proc. 20th
[1] Homeland Security. (2016). Recommended Practice: Improving USENIX Conf. Secur., 2011. [Online]. Available: https://www.usenix.org/
Industrial Control System Cybersecurity With Defense-in-Depth conference/usenix-security-11/comprehensive-experimental-analyses-
Strategies. [Online]. Available: https://us-cert.cisa.gov/sites/default/files/ automotive-attack-surfaces
recommended_practices/NCCIC_ICS-CERT_Defense_in_Depth_2016_ [21] M. Howard, D. LeBlanc, and J. Viega, 24 Deadly Sins of Software
S508C.pdf Security: Programming Flaws and How to Fix Them. New York, NY,
[2] J. McCarthy, M. Powell, K. Stouffer, C. Tang, T. Zimmerman, W. Barker, USA: McGraw-Hill, 2010.
T. Ogunyale, D. Wynne, and J. Wiltberger. (2020). Securing Manufactur- [22] MITRE. CWE—CWE-121-Stack-Based Buffer Overflow.
ing Industrial Control Systems: Behavioral Anomaly Detection. [Online]. Accessed: Oct. 13, 2021. [Online]. Available: https://cwe.mitre.org/
Available: https://nvlpubs.nist.gov/nistpubs/ir/2020/NIST.IR.8219.pdf data/definitions/121.html
[3] V. van der Veen, N. D. Sharma, L. Cavallaro, and H. Bos, ‘‘Memory [23] MITRE. CWE—CWE-122-Heap-Based Buffer Overflow.
errors: The past, the present, and the future,’’ in Proc. 15th Int. Symp. Accessed: Oct. 13, 2021. [Online]. Available: https://cwe.mitre.org/
Res. Attacks, Intrusions Defenses, 2012, pp. 86–106. data/definitions/122.html
[4] L. Szekeres, M. Payer, T. Wei, and D. Song, ‘‘SoK: Eternal war in [24] MITRE. CWE—CWE-124-Buffer Underwrite (‘Buffer Underflow’).
memory,’’ in Proc. IEEE Symp. Secur. Privacy, May 2013, pp. 48–62. Accessed: Oct. 13, 2021. [Online]. Available: https://cwe.mitre.org/data/
[5] P. Larsen and A.-R. Sadeghi, Eds., The Continuing Arms Race: Code- definitions/124.html
Reuse Attacks and Defenses. New York, NY, USA: Association for Com- [25] MITRE. CWE—CWE-126-Buffer Over-Read. Accessed: Oct. 13, 2021.
puting Machinery and Morgan & Claypool Publishers, 2018. [Online]. Available: https://cwe.mitre.org/data/definitions/126.html
[6] Intel. (2017). Control Flow Enforcement Technology (CET). [Online]. [26] MITRE. CWE—CWE-127-Buffer Under-Read. Accessed: Oct. 13, 2021.
Available: https://www.intel.com/content/dam/develop/external/us/en/ [Online]. Available: https://cwe.mitre.org/data/definitions/127.html
documents/catc17-introduction-intel-cet-844137.pdf [27] MITRE. CWE—CWE-680-Integer Overflow to Buffer Overflow.
[7] B. Sun, J. Liu, and C. Xu. How to Survive the Hardware-Assisted Accessed: Oct. 13, 2021. [Online]. Available: https://cwe.mitre.org/data/
Control Flow Integrity Enforcement. Accessed: Dec. 7, 2021. [Online]. definitions/680.html
Available: https://i.blackhat.com/asia-19/Thu-March-28/bh-asia- [28] MITRE. CWE—CWE-191-Integer Underflow (Wrap or Wraparound).
Sun-How-to-Survive-the-Hardware-Assisted-Control-Flow-Integrity- Accessed: Oct. 13, 2021. [Online]. Available: https://cwe.mitre.org/data/
Enforcement.pdf definitions/191.html
[8] M. Payer, ‘‘How memory safety violations enable exploitation of pro- [29] MITRE. CWE—CWE-193-Off-By-One Error. Accessed: Oct. 13, 2021.
grams,’’ in The Continuing Arms Race, P. Larsen and A.-R. Sadeghi, Eds. [Online]. Available: https://cwe.mitre.org/data/definitions/193.html
New York, NY, USA: Association for Computing Machinery and Morgan
[30] MITRE. CWE—CWE-843-Access of Resource Using Incompatible Type
& Claypool Publishers, 2018, pp. 1–23.
(‘Type Confusion’). Accessed: Oct. 13, 2021. [Online]. Available:
[9] L. Davi, A. Dmitrienko, A.-R. Sadeghi, and M. Winandy.
https://cwe.mitre.org/data/definitions/843.html
Return-Oriented Programming Without Returns on ARM.
[31] MITRE. CWE—CWE-134-Use of Externally-Controlled Format
Accessed: Dec. 13, 2021. [Online]. Available: https://www.ais.ruhr-uni-
String (4.5). Accessed: Oct. 13, 2021. [Online]. Available:
bochum.de/media/trust/veroeffentlichungen/2010/07/21/ROP-without-
https://cwe.mitre.org/data/definitions/134.html
Returns-on-ARM.pdf
[10] L. Davi, A. Dmitrienko, A.-R. Sadeghi, and M. Winandy, ‘‘Privilege [32] MITRE. CWE—CWE-416-Use After Free. Accessed: Oct. 13, 2021.
escalation attacks on android,’’ in Proc. 13th Int. Conf. Inf. Secur., 2011, [Online]. Available: https://cwe.mitre.org/data/definitions/416.html
pp. 346–360. [33] J. Caballero, G. Grieco, M. Marron, and A. Nappa, ‘‘Undangle: Early
[11] A. Cui, J. Kataria, and S. J. Stolfo, ‘‘Killing the myth of Cisco detection of dangling pointers in use-after-free and double-free vul-
IOS diversity: Recent advances in reliable shellcode design,’’ in nerabilities,’’ in Proc. Int. Symp. Softw. Test. Anal. (ISSTA), 2012,
Proc. 5th USENIX Conf. Offensive Technol., 2011. [Online]. Avail- pp. 133–143.
able: https://www.usenix.org/conference/woot11/killing-myth-cisco-ios- [34] Blackngel, ‘‘Malloc des-maleficarum,’’ Phrack Mag., vol. 13, no. 66,
diversity-recent-advances-reliable-shellcode-design 2009. [Online]. Available: http://phrack.org/issues/66/10.html
[12] E. Itkin. (2020). Don’t be Silly—It’s Only a Lightbulb—Check Point [35] MITRE. CWE—CWE-415-Double Free. Accessed: Oct. 13, 2021.
Research. [Online]. Available: https://research.checkpoint.com/2020/ [Online]. Available: https://cwe.mitre.org/data/definitions/415.html
dont-be-silly-its-only-a-lightbulb/ [36] MITRE. CWE—CWE-761-Free of Pointer Not at Start of Buffer.
[13] F. Lindner. (2009). Router Exploitation. [Online]. Available: Accessed: Oct. 13, 2021. [Online]. Available: https://cwe.mitre.org/data/
https://www.recurity-labs.com/research/FX_Router_Exploitation.pdf definitions/761.html
[14] MAR-17-352-01 HatMan—Safety System Targeted Malware (Update B), [37] MITRE. CWE—CWE-590-Free of Memory Not on the Heap.
Cybersecur. Infrastruct. Secur. Agency, Arlington, VA, USA, 2019. Accessed: Oct. 13, 2021. [Online]. Available: https://cwe.mitre.org/data/
[15] A. Francillon and C. Castelluccia, ‘‘Code injection attacks on harvard- definitions/590.html
architecture devices,’’ in Proc. 15th ACM Conf. Comput. Commun. Secur. [38] MITRE. CWE—CWE-762-Mismatched Memory Management Routines.
(CCS), 2008, pp. 15–26. Accessed: Oct. 13, 2021. [Online]. Available: https://cwe.mitre.org/data/
[16] J. Habibi, A. Gupta, S. Carlsony, A. Panicker, and E. Bertino, ‘‘MAVR: definitions/762.html
Code reuse stealthy attacks and mitigation on unmanned aerial vehi- [39] MITRE. CWE—CWE-457-Use of Uninitialized Variable.
cles,’’ in Proc. IEEE 35th Int. Conf. Distrib. Comput. Syst., Jun. 2015, Accessed: Oct. 13, 2021. [Online]. Available: https://cwe.mitre.org/data/
pp. 642–652. definitions/457.html

48268 VOLUME 10, 2022

Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

[40] MITRE. CWE—CWE-908-Use of Uninitialized Resource. [65] T. Bletsch, X. Jiang, V. W. Freeh, and Z. Liang, ‘‘Jump-oriented program-
Accessed: Oct. 13, 2021. [Online]. Available: https://cwe.mitre.org/data/ ming: A new class of code-reuse attack,’’ in Proc. 6th ACM Symp. Inf.,
definitions/908.html Comput. Commun. Secur. (ASIACCS), 2011, pp. 30–40.
[41] MITRE. CWE—CWE-824-Access of Uninitialized Pointer. [66] S. Checkoway and H. Shacham. (2010). Escape From Return-Oriented
Accessed: Oct. 13, 2021. [Online]. Available: https://cwe.mitre.org/data/ Programming: Return-Oriented Programming Without Returns (on the
definitions/824.html X86). [Online]. Available: https://hovav.net/ucsd/papers/cs10.html
[42] MITRE. CWE—CWE-476-Null Pointer Dereference. [67] H. Hu, S. Shinde, S. Adrian, Z. L. Chua, P. Saxena, and Z. Liang,
Accessed: Oct. 13, 2021. [Online]. Available: https://cwe.mitre.org/data/ ‘‘Data-oriented programming: On the expressiveness of non-control
definitions/476.html data attacks,’’ in Proc. IEEE Symp. Secur. Privacy (SP), May 2016,
[43] MITRE. CWE—CWE-364-Signal Handler Race Condition. pp. 969–986.
Accessed: Oct. 13, 2021. [Online]. Available: https://cwe.mitre.org/data/ [68] M. Abadi, M. Budiu, U. Erlingsson, and J. Ligatti, ‘‘Control-flow
definitions/364.html integrity,’’ in Proc. 12th ACM Conf. Comput. Commun. Secur., 2005,
[44] MITRE. CWE—CWE-365-Race Condition in Switch. p. 340.
Accessed: Oct. 13, 2021. [Online]. Available: https://cwe.mitre.org/data/ [69] D. D. Zovi. (2010). Practical Return-Oriented Programming. [Online].
definitions/365.html Available: https://people.csail.mit.edu/hes/ROP/Readings/rop-talk.pdf
[45] MITRE. CWE—CWE-367-Time-of-Check Time-of-Use (TOCTOU) Race [70] L. V. D. Davi, ‘‘Code-reuse attacks and defenses,’’ Ph.D. dissertation,
Condition. Accessed: Oct. 13, 2021. [Online]. Available: https://cwe. Tech. Univ. Darmstadt, Darmstadt, Germany, 2015.
mitre.org/data/definitions/367.html [71] P. Ratanaworabhan, B. Livshits, and B. Zorn. (2008). Nozzle: A Defense
[46] MITRE. CWE—CWE-368-Context Switching Race Condition. Against Heap-Spraying Code Injection Attacks. [Online]. Available:
Accessed: Oct. 13, 2021. [Online]. Available: https://cwe.mitre.org/ https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/
data/definitions/368.html tr-2008-176.pdf
[47] MITRE. 2021 CWE Top 25 Most Dangerous Software Weaknesses. [72] F. Gadaleta, Y. Younan, and W. Joosen, ‘‘Bubble: A Javascript engine
Accessed: Oct. 13, 2021. [Online]. Available: https://cwe.mitre.org/ level countermeasure against heap-spraying attacks,’’ in Proc. Int. Symp.
top25/archive/2021/2021_cwe_top25.html Eng. Secure Softw. Syst., 2010, pp. 1–17.
[48] M. Miller, ‘‘Trends and challenges in the vulnerability mitigation [73] A. Sotirov. (2007). Heap Feng Shui in JavaScript. [Online]. Available:
landscape,’’ in Proc. 13th USENIX Workshop Offensive Technol., http://www.mathcs.richmond.edu/~dszajda/research/summer_2014/
2019. [Online]. Available: https://www.usenix.org/conference/woot19/ papers/sotirov_heap_feng_shui_javascript_paper.pdf
presentation/miller [74] Y. Wang, C. Zhang, Z. Zhao, B. Zhang, X. Gong, and W. Zou, ‘‘MAZE:
Towards automated heap Feng Shui,’’ in Proc. 30th USENIX Secur. Symp.,
[49] A. One, ‘‘Smashing the stack for fun and profit,’’ Phrack Mag.,
2021, pp. 1647–1664.
vol. 7, no. 49, pp. 14–16, 1996. [Online]. Available: http://phrack.org/
[75] D. Brumley, P. Poosankam, D. Song, and J. Zheng, ‘‘Automatic patch-
issues/49/14.html
based exploit generation is possible: Techniques and implications,’’ in
[50] S. Chen, J. Xu, E. C. Sezer, P. Gauriar, and R. K. Iyer, ‘‘Non-control-data
Proc. IEEE Symp. Secur. Privacy (SP), May 2008, pp. 143–157.
attacks are realistic threats,’’ in Proc. 14th Conf. USENIX Secur. Symp.,
[76] S. Heelan, ‘‘Automatic generation of control flow hijacking exploits for
2005, pp. 177–191.
software vulnerabilities,’’ M.S. thesis, Univ. Oxford, Oxford, U.K., 2009.
[51] Solar Designer. Getting Around Non-Executable Stack (and Fix).
[77] T. Avgerinos, S. K. Cha, B. L. T. Hao, and D. Brumley, ‘‘Automatic
Accessed: Sep. 18, 2021. [Online]. Available: https://seclists.org/bugtraq/
exploit generation,’’ in Proc. Netw. Distrib. Syst. Secur. Symp., 2011,
1997/Aug/63
pp. 74–84.
[52] H. Shacham, ‘‘The geometry of innocent flesh on the bone,’’ in Proc. 14th [78] E. J. Schwartz, T. Avgerinos, and D. Brumley, ‘‘Q: Exploit hard-
ACM Conf. Comput. Commun. Secur., 2007, pp. 552–561. ening made easy,’’ in Proc. 20th USENIX Conf. Secur., 2011.
[53] H. Hu, Z. L. Chua, S. Adrian, P. Saxena, and Z. Liang, ‘‘Automatic [Online]. Available: https://www.usenix.org/conference/usenix-security-
generation of data-oriented exploits,’’ in Proc. 24th USENIX Secur. Symp., 11/q-exploit-hardening-made-easy
2015, pp. 177–192. [79] A. Follner, A. Bartel, H. Peng, Y.-C. Chang, K. K. Ispoglou, M. Payer, and
[54] N. Carlini, A. Barresi, M. Payer, and D. Wagner, ‘‘Control-flow bending E. Bodden, ‘‘PSHAPE: Automatically combining gadgets for arbitrary
on the effectiveness of control-flow integrity,’’ in Proc. 24th USENIX method execution,’’ in Proc. Int. Workshop Secur. Trust Manage., 2016,
Secur. Symp., 2015, pp. 161–176. pp. 212–228.
[55] Nergal, ‘‘The advanced return-into-lib(c) exploits,’’ Phrack Mag., vol. 11, [80] S. K. Cha, T. Avgerinos, A. Rebert, and D. Brumley, ‘‘Unleashing may-
no. 58, 2001. [Online]. Available: http://www.phrack.org/archives/ hem on binary code,’’ in Proc. IEEE Symp. Secur. Privacy, May 2012,
issues/58/4.txt pp. 380–394.
[56] M. Tran, M. Etheridge, T. Bletsch, X. Jiang, V. Freeh, and P. Ning, ‘‘On [81] Y. Wang, C. Zhang, X. Xiang, Z. Zhao, W. Li, X. Gong, B. Liu, K. Chen,
the expressiveness of return-into-libc attacks,’’ in Proc. Int. Workshop and W. Zou, ‘‘Revery: From proof-of-concept to exploitable: (one step
Recent Adv. Intrusion Detection, 2011, pp. 121–141. towards automatic exploit generation),’’ in Proc. ACM SIGSAC Conf.
[57] R. Roemer, E. Buchanan, H. Shacham, and S. Savage, ‘‘Return-oriented Comput. Commun. Secur., 2018, pp. 1914–1927.
programming,’’ ACM Trans. Inf. Syst. Secur., vol. 15, no. 1, pp. 1–34, [82] S. Heelan, T. Melham, and D. Kroening, ‘‘Gollum: Modular and greybox
2012. exploit generation for heap overflows in interpreters,’’ in Proc. ACM
[58] I. Arce, ‘‘The shellcode generation,’’ IEEE Secur. Privacy Mag., vol. 2, SIGSAC Conf. Comput. Commun. Secur., Nov. 2019, pp. 1689–1706.
no. 5, pp. 72–76, Sep. 2004. [83] D. Repel, J. Kinder, and L. Cavallaro, ‘‘Modular synthesis of heap
[59] A. van de Ven. (2004). Exec Shield. [Online]. Available: https://people. exploits,’’ in Proc. 12th Workshop Program. Lang. Anal. Secur.,
redhat.com/mingo/exec-shield/docs/WHP0006US_Execshield.pdf Oct. 2017, pp. 25–35.
[60] E. Göktas, E. Athanasopoulos, H. Bos, and G. Portokalidis, ‘‘Out of [84] M. Eckert, A. Bianchi, R. Wang, Y. Shoshitaishvili, C. Kruegel, and
control: Overcoming control-flow integrity,’’ in Proc. IEEE Symp. Secur. G. Vigna, ‘‘Heaphopper: Bringing bounded model checking to heap
Privacy, May 2014, pp. 575–589. implementation security,’’ in Proc. 27th USENIX Secur. Symp., 2018,
[61] P. Chen, H. Xiao, X. Shen, X. Yin, B. Mao, and L. Xie, ‘‘Drop: Detecting pp. 99–116.
return-oriented programming malicious code,’’ in Proc. Int. Conf. Inf. [85] K. K. Ispoglou, B. AlBassam, T. Jaeger, and M. Payer, ‘‘Block oriented
Syst. Secur., 2009, pp. 163–177. programming,’’ in Proc. ACM SIGSAC Conf. Comput. Commun. Secur.,
[62] L. Davi, A.-R. Sadeghi, and M. Winandy, ‘‘Dynamic integrity measure- 2018, pp. 1868–1882.
ment and attestation: Towards defense against return-oriented program- [86] Microsoft Defender Security Research Team. (2017). Analysis of the
ming attacks,’’ in Proc. ACM Workshop Scalable Trusted Comput. (STC), Shadow Brokers Release and Mitigation With Windows 10 Virtualization-
2009, pp. 49–54. Based Security. [Online]. Available: https://www.microsoft.com/
[63] J. Li, Z. Wang, X. Jiang, M. Grace, and S. Bahram, ‘‘Defeating return- security/blog/2017/06/16/analysis-of-the-shadow-brokers-release-and-
oriented rootkits with ‘return-less’ kernels,’’ in Proc. 5th Eur. Conf. mitigation-with-windows-10-virtualization-based-security/
Comput. Syst. (EuroSys), 2010, p. 195. [87] Y. Kim, R. Daly, J. Kim, C. Fallin, J. H. Lee, D. Lee, C. Wilker-
[64] S. Checkoway, L. Davi, A. Dmitrienko, A.-R. Sadeghi, H. Shacham, and son, K. Lai, and O. Mutlu, ‘‘Flipping bits in memory without access-
M. Winandy, ‘‘Return-oriented programming without returns,’’ in Proc. ing them: An experimental study of dram disturbance errors,’’ ACM
17th ACM Conf. Comput. Commun. Secur. (CCS), 2010, pp. 559–572. SIGARCH Comput. Archit. News, vol. 42, no. 3, pp. 361–372, 2014.

VOLUME 10, 2022 48269

 Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

[88] M. Seaborn and T. Dullien, ‘‘Exploiting the dram rowhammer bug to gain [108] P. Mackenzie. (2019). Wannacry Aftershock. [Online]. Available:
kernel privileges: How to cause and exploit single bit errors,’’ Black Hat, https://www.sophos.com/en-us/medialibrary/PDFs/technical-papers/
vol. 15, 2015. [Online]. Available: https://www.blackhat.com/docs/us- WannaCry-Aftershock.pdf
15/materials/us-15-Seaborn-Exploiting-The-DRAM-Rowhammer-Bug- [109] Kaspersky ICS CERT. (2017). More Than 50% of Organizations Attacked
To-Gain-Kernel-Privileges-wp.pdf by Expetr (Petya) Cryptolocker are Industrial Companies. [Online].
[89] V. van der Veen, Y. Fratantonio, M. Lindorfer, D. Gruss, C. Maurice, Available: https://ics-cert.kaspersky.com/alerts/2017/06/29/more-than-
G. Vigna, H. Bos, K. Razavi, and C. Giuffrida, ‘‘Drammer: Deterministic 50-percent-of-organizations-attacked-by-expetr-petya-cryptolocker-are-
rowhammer attacks on mobile platforms,’’ in Proc. ACM SIGSAC Conf. industrial-companies/
Comput. Commun. Secur., Oct. 2016, pp. 1675–1689. [110] S. Hurley and K. Sood. (2017). Notpetya Technical Analysis Part II:
[90] B. E. Strom, A. Applebaum, D. P. Miller, K. C. Nickels, A. G. Pen- Further Findings and Potential for MBR Recovery. [Online]. Available:
nington, and C. B. Thomas. (2020). MITRE ATT&CKr: Design and https://www.crowdstrike.com/blog/petrwrap-technical-analysis-part-2-
Philosophy. [Online]. Available: https://attack.mitre.org/docs/ATTACK_ further-findings-and-potential-for-mbr-recovery/
Design_and_Philosophy_March_2020.pdf [111] M.-E. M. Léveillé. Bad Rabbit: Not-Petya is Back With Improved
[91] National Cybersecurity and Communications Integration Center. (2015). Ransomware. Accessed: Sep. 10, 2021. [Online]. Available:
ICS-CERT Monitor September 2014-February 2015. [Online]. Available: https://www.welivesecurity.com/2017/10/24/bad-rabbit-not-petya-back/
https://us-cert.cisa.gov/sites/default/files/Monitors/ICS-CERT_ [112] O. Mamedov, F. Sinitsyn, and A. Ivanov. (2017). Bad Rabbit
Monitor_Sep2014-Feb2015.pdf Ransomware. [Online]. Available: https://securelist.com/bad-rabbit-
[92] Homeland Security. (2015). NCCIC/ICS-Cert Year in Review 2015. ransomware/82851/
[Online]. Available: https://us-cert.cisa.gov/sites/default/files/Annual_ [113] Zerosum0x0. (2017). Doublepulsar Initial SMB Backdoor Ring 0
Reports/Year_in_Review_FY2015_Final_S508C.pdf Shellcode Analysis. [Online]. Available: https://zerosum0x0.blogspot.
[93] R. Langner. (2013). To Kill a Centrifuge: A Technical Analysis of What com/2017/04/doublepulsar-initial-smb-backdoor-ring.html
Stuxnet’s Creators Tried to Achieve. [Online]. Available: https://www. [114] C. Talos. (2017). Threat Spotlight: Follow the Bad Rabbit. [Online].
langner.com/wp-content/uploads/2017/03/to-kill-a-centrifuge.pdf Available: https://blog.talosintelligence.com/2017/10/bad-rabbit.html
[94] N. Falliere, L. O. Murchu, and E. Chien. (2010). W32.Stuxnet [115] A. Di Pinto, Y. Dragoni, and A. Carcano. (2018). Triton: The First
Dossier. [Online]. Available: https://www.wired.com/images_blogs/ ICS Cyberattack on Safety Instrument Systems. USA. [Online].
threatlevel/2010/11/w32_stuxnet_dossier.pdf Available: https://i.blackhat.com/us-18/Wed-August-8/us-18-Carcano-
[95] Laboratory of Cryptography and System Security. (2011). Duqu: TRITON-How-It-Disrupted-Safety-Systems-And-Changed-The-Threat-
A Stuxnet-Like Malware Found in the Wild. [Online]. Available: Landscape-Of-Industrial-Control-Systems-Forever-wp.pdf
https://www.crysys.hu/publications/files/bencsathPBF11duqu.pdf [116] C. Talos. (2018). New VPNFilter Malware Targets at Least
500K Networking Devices Worldwide. [Online]. Available:
[96] Symantec Security Response. (2011). W32.Duqu: The Precursor to the
https://blog.talosintelligence.com/2018/05/VPNFilter.html
Next Stuxnet. [Online]. Available: https://docs.broadcom.com/doc/w32-
duqu-11-en [117] Symantec Security Response. (2018). VPNFilter: New Router Malware
With Destructive Capabilities. [Online]. Available: https://symantec-
[97] Kaspersky Lab. (2015). The Duqu 2.0 Technical Details. [Online].
enterprise-blogs.security.com/blogs/threat-intelligence/vpnfilter-iot-
Available: https://media.kasperskycontenthub.com/wp-content/
malware
uploads/sites/43/2018/03/07205202/The_Mystery_of_Duqu_2_0_a_
[118] C. Talos. (2018). VPNFilter Update—VPNFilter Exploits Endpoints,
sophisticated_cyberespionage_actor_returns.pdf
Targets New Devices. [Online]. Available: https://blog.talosintelligence.
[98] A. Gostev. (2012). The Flame: Questions and Answers. [Online]. Avail- com/2018/06/vpnfilter-update.html
able: https://securelist.com/the-flame-questions-and-answers/34344/
[119] O. Alexander, M. Belisle, and J. Steele. (2020). MITRE ATT&CKr for
[99] Laboratory of Cryptography and System Security. (2012). Skywiper Industrial Control Systems: Design and Philosophy. [Online]. Available:
(a.k.a. Flame a.k.a. Flamer): A Complex Malware for Targeted https://collaborate.mitre.org/attackics/img_auth.php/3/37/ATT&CK_
Attacks. [Online]. Available: https://www.crysys.hu/publications/ for_ICS_-_Philosophy_Paper.pdf
files/skywiper.pdf [120] T. J. Williams, ‘‘The Purdue enterprise reference architecture’’ Comput.
[100] Symantec Security Response. Dragonfly: Cyberespionage Attacks Ind., vol. 24, no. 2, pp. 141–158, Sep. 1994.
Against Energy Suppliers. Accessed: Sep. 3, 2021. [Online]. Available: [121] E. D. Knapp, Industrial Network Security: Securing Critical Infrastruc-
https://docs.broadcom.com/doc/dragonfly_threat_against_western_ ture Networks for Smart Grid, SCADA, and Other Industrial Control
energy_suppliers Systems, 2nd ed. Waltham, MA, USA: Elsevier, 2014.
[101] J. Stycznski and N. Beach-Westmoreland. (2016). When the Lights [122] D. Peterson. (2019). Is the Purdue Model Dead? [Online]. Available:
Went Out: A Comprehensive Review of the 2015 Attacks on Ukrainian https://dale-peterson.com/2019/02/11/is-the-purdue-model-dead/
Critical Infrastructure. [Online]. Available: https://www.boozallen.com/ [123] M. Giles. Triton is the World’s Most Murderous Malware, and It’s
content/dam/boozallen/documents/2016/09/ukraine-report-when-the- Spreading. (2019). [Online]. Available: https://www.technologyreview.
lights-went-out.pdf com/2019/03/05/103328/cybersecurity-critical-infrastructure-triton-
[102] R. Lipovsky. (2016). CVE-2014-4114: Details on August Blackenergy malware/
Powerpoint Campaigns. [Online]. Available: https://www.welivesecurity. [124] Mission Secure. (2021). A Comprehensive Guide to Operational Technol-
com/2014/10/14/cve-2014-4114-details-august-blackenergy- ogy (OT) Cybersecurity. [Online]. Available: https://www.missionsecure.
powerpoint-campaigns/ com/resources/comprehensive-guide-to-operational-technology-
[103] A. Cherepanov and R. Lipovsky, ‘‘Blackenergy—What we really security-ebook
know about the notorious cyber attacks,’’ in Proc. Virus Bull. Conf., [125] K. G. Crowther, R. M. Lee, and K. R. Wightman. (2018). Build-
2016. [Online]. Available: https://www.virusbulletin.com/conference/ ing Security to Achieve Engineering and Business Requirements.
vb2016/abstracts/blackenergy-what-we-really-know-about-notorious- [Online]. Available: https://www.dragos.com/resource/design-and-build-
cyber-attacks/ productive-and-secure-industrial-systems-paper-2-of-3/
[104] Dragos. (2017). Crashoverride: Analyzing the Threat to Electric Grid [126] E. P. Leverett, ‘‘Quantitatively assessing and visualising industrial system
Operations. [Online]. Available: https://www.dragos.com/wp-content/ attack surfaces,’’ Ph.D. dissertation, Univ. Cambridge, Cambridge, U.K.,
uploads/CrashOverride-01.pdf 2011.
[105] A. Cherepanov. (2017). Win32/Industroyer: A New Threat for Industrial [127] S. Miller, N. Brubaker, D. K. Zafra, and D. Caban. (2019). TRITON
Control Systems. [Online]. Available: https://www.welivesecurity.com/ Actor TTP Profile, Custom Attack Tools, Detections, and ATT&CK Map-
wp-content/uploads/2017/06/Win32_Industroyer.pdf ping. [Online]. Available: https://www.mandiant.com/resources/triton-
[106] J. Slowik. (2019). Crashoverride: Reassessing the 2016 Ukraine Elec- actor-ttp-profile-custom-attack-tools-detections
tric Power Event as a Protection-Focused Attack. [Online]. Available: [128] B. Meixell and E. Forner, ‘‘Out of control: Demonstrating SCADA
https://www.dragos.com/wp-content/uploads/CRASHOVERRIDE.pdf exploitation,’’ Cimation, Kuala Lumpur, Malaysia, Tech. Rep., 2013.
[107] Kaspersky ICS CERT. (2017). Wannacry on Industrial Networks-Error [129] C. Foreman. (2015). Cyber-Security in Industrial Control Systems.
Correction. [Online]. Available: https://ics-cert.kaspersky.com/reports/ [Online]. Available: https://engineering.purdue.edu/VAAMI/ICS-
2017/06/22/wannacry-on-industrial-networks/ modules.pdf

48270 VOLUME 10, 2022

Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

[130] B. Filkins, D. Wylie, and J. Dely. Sans 2019 State of OT/ICS Cyberse- [153] D. Blazakis, ‘‘Interpreter exploitation,’’ in Proc. 4th USENIX Workshop
curity Survey. [Online]. Available: https://www.forescout.com/resources/ Offensive Technol., 2010, pp. 1–9.
2019-sans-state-of-ot-ics-cybersecurity-survey/ [154] R. Gawlik and T. Holz, ‘‘SoK: Make JIT-spray great again,’’ in Proc.
[131] C. Cowan, C. Pu, D. Maier, J. Walpole, P. Bakke, S. Beattie, A. Grier, 12th USENIX Workshop Offensive Technol., 2018. [Online]. Available:
P. Wagle, Q. Zhang, and H. Hinton, ‘‘Stackguard: Automatic adaptive https://www.usenix.org/conference/woot18/presentation/gawlik
detection and prevention of buffer-overflow attacks,’’ in 7th USENIX [155] R. Hund, C. Willems, and T. Holz, ‘‘Practical timing side channel
Secur. Symp., 1998. attacks against kernel space ASLR,’’ in Proc. IEEE Symp. Secur. Privacy,
[132] T.-C. Chiueh and F.-H. Hsu, ‘‘RAD: A compile-time solution to buffer May 2013, pp. 191–205.
overflow attacks,’’ in Proc. 21st Int. Conf. Distrib. Comput. Syst., 2001, [156] J. Seibert, H. Okhravi, and E. Söderström, ‘‘Information leaks without
pp. 409–417. memory disclosures,’’ in Proc. ACM SIGSAC Conf. Comput. Commun.
[133] J. Li, L. Chen, Q. Xu, L. Tian, G. Shi, K. Chen, and D. Meng, ‘‘Zipper Secur., Nov. 2014, pp. 54–65.
stack: Shadow stacks without shadow,’’ in Proc. 25th Eur. Symp. Res. [157] R. Rudd, R. Skowyra, D. Bigelow, V. Dedhia, T. Hobson, S. Crane,
Comput. Secur., vol. 12308, 2020, pp. 338–358. C. Liebchen, P. Larsen, L. Davi, M. Franz, A.-R. Sadeghi, and
[134] V. Kuznetsov, L. Szekeres, M. Payer, G. Candea, R. Sekar, and D. Song, H. Okhravi, ‘‘Address oblivious code reuse: On the effectiveness of
‘‘Code-pointer integrity,’’ in Proc. 11th USENIX Symp. Oper. Syst. leakage-resilient diversity,’’ in Proc. Netw. Distrib. Syst. Secur. Symp.,
Design, 2014, pp. 81–116. 2017. [Online]. Available: https://www.ndss-symposium.org/ndss2017/
[135] I. Evans, S. Fingeret, J. Gonzalez, U. Otgonbaatar, T. Tang, H. Shrobe, ndss-2017-programme/address-oblivious-code-reuse-effectiveness-
S. Sidiroglou-Douskos, M. Rinard, and H. Okhravi, ‘‘Missing the leakage-resilient-diversity/
point(er): On the effectiveness of code pointer integrity,’’ in Proc. IEEE [158] M. Abadi, M. Budiu, Ú. Erlingsson, and J. Ligatti, ‘‘Control-flow integrity
Symp. Secur. Privacy, May 2015, pp. 781–796. principles, implementations, and applications,’’ ACM Trans. Inf. Syst.
[136] S. Andersen and V. Abella. (2004). Changes to Functionality in Secur., vol. 13, no. 1, pp. 1–40, Oct. 2009.
Microsoft Windows XP Service Pack 2, Part 3: Memory Protection [159] Z. Wang and X. Jiang, ‘‘HyperSafe: A lightweight approach to provide
Technologies, Data Execution Prevention. [Online]. Available: lifetime hypervisor control-flow integrity,’’ in Proc. IEEE Symp. Secur.
https://docs.microsoft.com/en-us/previous-versions/windows/it- Privacy, May 2010, pp. 380–395.
pro/windows-xp/bb457155(v=technet.10) [160] C. Zhang, T. Wei, Z. Chen, L. Duan, L. Szekeres, S. McCamant, D. Song,
[137] PAX Team. (2003). PAX Address Space Layout Randomization (ASLR). and W. Zou, ‘‘Practical control flow integrity and randomization for
[Online]. Available: https://pax.grsecurity.net/docs/aslr.txt binary executables,’’ in Proc. IEEE Symp. Secur. Privacy, May 2013,
[138] J. Hiser, A. Nguyen-Tuong, M. Co, M. Hall, and J. W. Davidson, pp. 559–573.
‘‘ILR: Where’d my gadgets go?’’ in Proc. IEEE Symp. Secur. Privacy, [161] B. Niu and G. Tan, ‘‘Modular control-flow integrity,’’ in Proc. 35th
May 2012, pp. 571–585. ACM SIGPLAN Conf. Program. Lang. Design Implement., Jun. 2014,
[139] S. Bhatkar and R. Sekar, ‘‘Data space randomization,’’ in Proc. Int. Conf. pp. 577–587.
Detection Intrusions Malware, Vulnerability Assessment, 2008, pp. 1–22. [162] C. Tice, T. Roeder, P. Collingbourne, S. Checkoway, U. Erlingsson,
[140] F. B. Cohen, ‘‘Operating system protection through program evolution,’’ L. Lozano, and G. Pike, ‘‘Enforcing forward-edge control-flow integrity
Comput. Secur., vol. 12, no. 6, pp. 565–584, 1993. in GCC & LLVM,’’ in Proc. 23rd USENIX Secur. Symp., 2014,
[141] A. Homescu, S. Neisius, P. Larsen, S. Brunthaler, and M. Franz, ‘‘Profile- pp. 941–955.
guided automated software diversity,’’ in Proc. IEEE/ACM Int. Symp. [163] M. Budiu, Ú. Erlingsson, and M. Abadi, ‘‘Architectural support for
Code Gener. Optim. (CGO), Feb. 2013, pp. 1–11. software-based protection,’’ in Proc. 1st Workshop Archit. Syst. Support
[142] P. Larsen, A. Homescu, S. Brunthaler, and M. Franz, ‘‘SoK: Automated Improving Softw. Dependability (ASID), 2006, pp. 42–51.
software diversity,’’ in Proc. IEEE Symp. Secur. Privacy, May 2014, [164] L. Davi, P. Koeberl, and A.-R. Sadeghi, ‘‘Hardware-assisted fine-grained
pp. 276–291. control-flow integrity: Towards efficient protection of embedded systems
[143] S. Bhatkar, D. C. DuVarney, and R. Sekar, ‘‘Address obfuscation: An against software exploitation,’’ in Proc. 51st ACM/EDAC/IEEE Design
efficient approach to combat a broad range of memory error exploits,’’ Automat. Conf. (DAC), Jun. 2014, pp. 1–6.
in Proc. 12th USENIX Secur. Symp., 2003, pp. 105–120. [165] L. Davi, M. Hanreich, D. Paul, A.-R. Sadeghi, P. Koeberl, D. Sullivan,
[144] G. S. Kc, A. D. Keromytis, and V. Prevelakis, ‘‘Countering code-injection O. Arias, and Y. Jin, ‘‘HAFIX: Hardware-assisted flow integrity exten-
attacks with instruction-set randomization,’’ in Proc. 10th ACM Conf. sion,’’ in Proc. 52nd Annu. Design Autom. Conf., Jun. 2015, pp. 1–6.
Comput. Commun. Secur. (CCS), 2003, pp. 272–280. [166] D. Sullivan, O. Arias, L. Davi, P. Larsen, A.-R. Sadeghi, and Y. Jin,
[145] C. Kil, J. Jun, C. Bookholt, J. Xu, and P. Ning, ‘‘Address space layout ‘‘Strategy without tactics: Policy-agnostic hardware-enhanced control-
permutation (ASLP): Towards fine-grained randomization of commodity flow integrity,’’ in Proc. 53rd Annu. Design Autom. Conf., Jun. 2016,
software,’’ in Proc. 22nd Annu. Comput. Secur. Appl. Conf. (ACSAC), pp. 1–6.
Dec. 2006, pp. 339–348. [167] N. Christoulakis, G. Christou, E. Athanasopoulos, and S. Ioannidis,
[146] R. Wartell, V. Mohan, K. W. Hamlen, and Z. Lin, ‘‘Binary stirring: Self- ‘‘HCFI: Hardware-enforced control-flow integrity,’’ in Proc. 6th ACM
randomizing instruction addresses of legacy X86 binary code,’’ in Proc. Conf. Data Appl. Secur. Privacy, Mar. 2016.
ACM Conf. Comput. Commun. Secur. (CCS), 2012, p. 157. [168] J. Zhang, B. Qi, Z. Qin, and G. Qu, ‘‘HCIC: Hardware-assisted control-
[147] C. Giuffrida, A. Kuijsten, and A. S. Tanenbaum, ‘‘Enhanced operating flow integrity checking,’’ IEEE Internet Things J., vol. 6, no. 1,
system security through efficient and fine-grained address space random- pp. 458–471, Feb. 2019.
ization,’’ in Proc. 21st USENIX Secur. Symp., 2012, pp. 475–490. [169] V. Shanbhogue, D. Gupta, and R. Sahita, ‘‘Security analysis of processor
[148] V. Pappas, M. Polychronakis, and A. D. Keromytis, ‘‘Smashing the instruction set architecture for enforcing control-flow integrity,’’ in Proc.
gadgets: Hindering return-oriented programming using in-place code 8th Int. Workshop Hardw. Archit. Support Secur. Privacy, Jun. 2019,
randomization,’’ in Proc. IEEE Symp. Secur. Privacy, May 2012, pp. 1–11.
pp. 601–615. [170] V. van der Veen, D. Andriesse, M. Stamatogiannakis, X. Chen, H. Bos,
[149] D. Bigelow, T. Hobson, R. Rudd, W. Streilein, and H. Okhravi, ‘‘Timely and C. Giuffrdia, ‘‘The dynamics of innocent flesh on the bone,’’ in Proc.
rerandomization for mitigating memory disclosures,’’ in Proc. 22nd ACM ACM SIGSAC Conf. Comput. Commun. Secur., Oct. 2017, pp. 1675–1689.
SIGSAC Conf. Comput. Commun. Secur., Oct. 2015, pp. 268–279. [171] L. Cheng, H. Liljestrand, M. S. Ahmed, T. Nyman, T. Jaeger, N. Asokan,
[150] P. Chen, J. Xu, Z. Lin, D. Xu, B. Mao, and P. Liu, ‘‘A practical approach and D. Yao, ‘‘Exploitation techniques and defenses for data-oriented
for adaptive data structure layout randomization,’’ in Proc. 20th Eur. attacks,’’ in Proc. IEEE Cybersecur. Develop. (SecDev), Sep. 2019.
Symp. Res. Comput. Secur., 2015, pp. 69–89. [172] M. Castro, M. Costa, and T. Harris, ‘‘Securing software by enforcing data-
[151] S. Crane, C. Liebchen, A. Homescu, L. Davi, P. Larsen, A.-R. Sadeghi, flow integrity,’’ in Proc. 7th Symp. Oper. Syst. Design Implement., 2006,
S. Brunthaler, and M. Franz, ‘‘Readactor: Practical code randomization pp. 147–160.
resilient to memory disclosure,’’ in Proc. IEEE Symp. Secur. Privacy, [173] C. Song, B. Lee, K. Lu, W. Harris, T. Kim, and W. Lee, ‘‘Enforcing kernel
May 2015, pp. 763–780. security invariants with data flow integrity,’’ in Proc. Netw. Distrib. Syst.
[152] K. Z. Snow, F. Monrose, L. Davi, A. Dmitrienko, C. Liebchen, and Secur. Symp., 2016.
A. Sadeghi, ‘‘Just-in-time code reuse: On the effectiveness of fine-grained [174] D. Song, J. Lettner, P. Rajasekaran, Y. Na, S. Volckaert, P. Larsen, and
address space layout randomization,’’ in Proc. IEEE Symp. Secur. Pri- M. Franz, ‘‘SoK: Sanitizing for security,’’ in Proc. IEEE Symp. Secur.
vacy, May 2013, pp. 574–588. Privacy (SP), May 2019, pp. 1275–1295.

VOLUME 10, 2022 48271

 Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

[175] D. Gens, S. Schmitt, L. Davi, and A.-R. Sadeghi, ‘‘K-miner: Uncovering [199] ARM Limited. (2021). ARM Architecture Reference Manual, ARMv8,
memory corruption in Linux,’’ in Proc. Netw. Distrib. Syst. Secur. Symp., for ARMv8-A Architecture Profile. [Online]. Available: https://developer.
2018. arm.com/documentation/ddi0487/ga
[176] M. Ghaffarinia and K. W. Hamlen, ‘‘Binary control-flow trimming,’’ [200] Qualcomm Technologies Inc. (2017). Pointer Authentication on
in Proc. ACM SIGSAC Conf. Comput. Commun. Secur., Nov. 2019, ARMV8.3: Design and Analysis of the New Software Security
pp. 1009–1022. Instructions. [Online]. Available: https://www.qualcomm.com/media/
[177] T. Jim, G. Morrisett, D. Grossman, M. Hicks, J. Cheney, and Y. Wang, documents/files/whitepaper-pointer-authentication-on-armv8-3.pdf
‘‘Cyclone: A safe dialect of C,’’ in Proc. USENIX Annu. Tech. Conf., 2002, [201] H. Liljestrand, T. Nyman, K. Wang, C. C. Perez, J.-E. Ekberg, and
pp. 275–288. N. Asokan, ‘‘PAC it up: Towards pointer integrity using ARM pointer
[178] G. C. Necula, J. Condit, M. Harren, S. McPeak, and W. Weimer, ‘‘CCured: authentication,’’ in Proc. 28th USENIX Secur. Symp., 2019, pp. 177–194.
Type-safe retrofitting of legacy software,’’ ACM Trans. Program. Lang. [202] Y. Chen, D. Zhang, R. Wang, R. Qiao, A. M. Azab, L. Lu, H. Vijayakumar,
Syst., vol. 27, no. 3, pp. 477–526, May 2005. and W. Shen, ‘‘NORAX: Enabling execute-only memory for COTS bina-
[179] S. Nagarakatte, J. Zhao, M. M. K. Martin, and S. Zdancewic, ‘‘SoftBound: ries on AArch64,’’ in Proc. IEEE Symp. Secur. Privacy (SP), May 2017,
Highly compatible and complete spatial memory safety for c,’’ ACM pp. 304–319.
SIGPLAN Notices, vol. 44, no. 6, pp. 245–258, May 2009. [203] D. Kwon, J. Shin, G. Kim, B. Lee, Y. Cho, and Y. Paek, ‘‘uXOM: Efficient
[180] S. Nagarakatte, J. Zhao, M. M. K. Martin, and S. Zdancewic, ‘‘CETS: execute-only memory on ARM cortex-M,’’ in Proc. 28th USENIX Secur.
Compiler-enforced temporal safety for C,’’ ACM SIGPLAN Notices, Symp., 2019, pp. 231–247.
vol. 45, no. 8, pp. 31–40, 2010. [204] ARM Limited. (2009). ARM Security Technology: Building a Secure Sys-
[181] R. Wahbe, S. Lucco, T. E. Anderson, and S. L. Graham, ‘‘Efficient tem Using Trustzone technology. [Online]. Available: https://developer.
software-based fault isolation,’’ in Proc. 14th ACM Symp. Oper. Syst. arm.com/documentation/PRD29-GENC-009492/c/TrustZone-
Princ. (SOSP), 1993, pp. 203–216. Hardware-Architecture
[182] S. McCamant and G. Morrisett, ‘‘Evaluating SFI for a CISC architecture,’’ [205] G. Beniamini. (2017). Trust Issues-Exploiting Trustzone TEEs. [Online].
in Proc. 15th USENIX Secur. Symp., 2006, pp. 209–224. Available: https://googleprojectzero.blogspot.com/2017/07/trust-issues-
[183] U. Erlingsson, M. Abadi, M. Vrable, M. Budiu, and G. C. Necula, ‘‘XFI: exploiting-trustzone-tees.html
Software guards for system address spaces,’’ in Proc. 7th Symp. Oper. [206] F. Brasser, D. Gens, P. Jauernig, A.-R. Sadeghi, and E. Stapf, ‘‘SANC-
Syst. Design Implement., 2006, pp. 75–88. TUARY: ARMing TrustZone with user-space enclaves,’’ in Proc. Netw.
[184] M. Castro, M. Costa, J.-P. Martin, M. Peinado, P. Akritidis, A. Donnelly, Distrib. Syst. Secur. Symp., 2019. [Online]. Available: https://www.ndss-
P. Barham, and R. Black, ‘‘Fast byte-granularity software fault isolation,’’ symposium.org/ndss-paper/sanctuary-arming-trustzone-with-user-
in Proc. ACM SIGOPS 22nd Symp. Oper. Syst. Princ. (SOSP), 2009, p. 45. space-enclaves/
[185] B. Yee, D. Sehr, G. Dardyk, J. B. Chen, R. Muth, T. Ormandy, S. Okasaka, [207] K. Serebryany, E. Stepanov, A. Shlyapnikov, V. Tsyrklevich, and
N. Narula, and N. Fullagar, ‘‘Native client: A sandbox for portable, D. Vyukov, ‘‘Memory tagging and how it improves C/C++ memory
untrusted x86 native code,’’ in Proc. 30th IEEE Symp. Secur. Privacy, safety,’’ 2018, arXiv:1802.09517.
May 2009, pp. 79–93. [208] ARMV8.5—A Memory Tagging Extension, ARM, Cambridge, U.K.,
[186] G. Tan, ‘‘Principles and implementation techniques of software-based 2019.
fault isolation,’’ Found. Trends Privacy Secur., vol. 1, no. 3, pp. 137–198, [209] K. Serebryany, ‘‘ARM memory tagging extension and how it improves
2017. C/C++ memory safety,’’ Login USENIX Mag., vol. 44, no. 2, pp. 12–16,
[187] Intel. (2013). Software Guard Extensions Programming Reference. Summer 2019.
[Online]. Available: https://www.intel.com/content/dam/develop/ [210] Q. Liu, V. Hagenmeyer, and H. B. Keller, ‘‘A review of rule learning-
external/us/en/documents/329298-002-629101.pdf based intrusion detection systems and their prospects in smart grids,’’
[188] A. Biondo, M. Conti, L. Davi, T. Frassetto, and A.-R. Sadeghi, ‘‘The IEEE Access, vol. 9, pp. 57542–57564, 2021.
guard’s dilemma: Efficient code-reuse attacks against Intel SGX,’’ in [211] T. Toth and C. Kruegel, ‘‘Accurate buffer overflow detection via abstract
Proc. 27th USENIX Secur. Symp., 2018, pp. 1213–1227. pay load execution,’’ in Proc. 5th Int. Symp. Res. Attacks, Intrusions
[189] T. Cloosters, M. Rodler, and L. Davi, ‘‘Discovery and exploitation of Defenses, 2002, pp. 274–291.
memory corruption vulnerabilities in SGX enclaves,’’ in Proc. 29th [212] X. Wang, Y.-C. Jhi, S. Zhu, and P. Liu, ‘‘STILL: Exploit code detection
USENIX Secur. Symp., 2020, pp. 841–858. via static taint and initialization analyses,’’ in Proc. Annu. Comput. Secur.
[190] 2015 Embedded Markets Study: Changes in Today’s Design, Devel- Appl. Conf. (ACSAC), Dec. 2008, pp. 289–298.
opment & Processing Environments, UBM Electron. Group, Vienna, [213] M. Egele, P. Wurzinger, C. Kruegel, and E. Kirda, ‘‘Defending browsers
Austria, 2015. against drive-by downloads: Mitigating heap-spraying code injection
[191] AspenCore. (2019). 2019 Embedded Markets Study. [Online]. attacks,’’ in Proc. Int. Conf. Detection Intrusions Malware, Vulnerability
Available: https://www.embedded.com/wp-content/uploads/2019/11/ Assessment, 2009, pp. 88–106.
EETimes_Embedded_2019_Embedded_Markets_Study.pdf [214] M. Polychronakis, K. G. Anagnostakis, and E. P. Markatos, ‘‘Network–
[192] H. Shacham, M. Page, B. Pfaff, E.-J. Goh, N. Modadugu, and D. Boneh, level polymorphic shellcode detection using emulation,’’ in Proc. Int.
‘‘On the effectiveness of address-space randomization,’’ in Proc. 11th Conf. Detection Intrusions Malware, Vulnerability Assessment, 2006,
ACM Conf. Comput. Commun. Secur. (CCS), 2004, pp. 298–307. pp. 54–73.
[193] ARM Limited. (2014). ARMv7-M Architecture Reference Manual. [215] Q. Zhang, D. S. Reeves, P. Ning, and S. P. Iyer, ‘‘Analyzing network
[Online]. Available: https://developer.arm.com/documentation/ddi0403/ traffic to detect self-decrypting exploit code,’’ in Proc. 2nd ACM Symp.
eb/ Inf., Comput. Commun. Secur. (ASIACCS), 2007, pp. 4–12.
[194] Memory Protection Unit (MPU) for Keystone Devices User’s Guide, [216] M. Polychronakis, K. G. Anagnostakis, and E. P. Markatos, ‘‘Comprehen-
Texas Instrum., pp. 1–30. [Online]. Available: https://www.ti.com/lit/ sive shellcode detection using runtime heuristics,’’ in Proc. 26th Annu.
ug/sprugw5a/sprugw5a.pdf?ts=1638233271706 Comput. Secur. Appl. Conf. (ACSAC), 2010, pp. 287–296.
[195] A. Francillon, D. Perito, and C. Castelluccia, ‘‘Defending embedded [217] B. Gu, X. Bai, Z. Yang, A. C. Champion, and D. Xuan, ‘‘Malicious
systems against control flow attacks,’’ in Proc. 1st ACM Workshop Secure shellcode detection with virtual memory snapshots,’’ in Proc. IEEE Annu.
Execution Untrusted Code, 2009, pp. 19–26. Joint Conf. INFOCOM, Mar. 2010, pp. 1–9.
[196] R. Walls, F. N. Brown, T. Baron, C. Shue, H. Okhravi, and C. B. Ward, [218] M. Polychronakis and A. D. Keromytis, ‘‘ROP payload detection using
‘‘Control-flow integrity for real-time embedded systems,’’ in Proc. 31st speculative code execution,’’ in Proc. 6th Int. Conf. Malicious Unwanted
Euromicro Conf. Real-Time Syst., 2019, pp. 1–24. Softw., Oct. 2011, pp. 58–65.
[197] J. Zhou, Y. Du, Z. Shen, L. Ma, J. Criswell, and R. J. Walls, ‘‘Silhouette: [219] L. Davi, A.-R. Sadeghi, and M. Winandy, ‘‘ROPdefender: A detection
Efficient protected shadow stacks for embedded systems,’’ in Proc. 29th tool to defend against return-oriented programming attacks,’’ in Proc. 6th
USENIX Secur. Symp., 2020, pp. 1219–1236. ACM Symp. Inf., Comput. Commun. Secur. (ASIACCS), 2011, pp. 40–51.
[198] N. S. Almakhdhub, A. A. Clements, S. Bagchi, and M. Payer, ‘‘µRAI: [220] Y. Cheng, Z. Zhou, M. Yu, X. Ding, and R. H. Deng,
Securing embedded systems with return address integrity,’’ in Proc. Netw. ‘‘ROPecker: A generic and practical approach for defending against ROP
Distrib. Syst. Secur. Symp., 2020. [Online]. Available: https://www.ndss- attacks,’’ in Proc. 21st Netw. Distrib. Syst. Secur. Symp., 2014. [Online].
symposium.org/ndss-paper/murai-securing-embedded-systems-with- Available: https://www.ndss-symposium.org/ndss2014/programme/
return-address-integrity/ ropecker-generic-and-practical-approach-defending-against-rop-attacks/

48272 VOLUME 10, 2022

Q. Liu et al.: Binary Exploitation in Industrial Control Systems: Past, Present and Future

[221] E. R. Jacobson, A. R. Bernat, W. R. Williams, and B. P. Miller, ‘‘Detecting KAIBIN BAO received the Ph.D. degree from
code reuse attacks with a model of conformant program execution,’’ in the Karlsruhe Institute of Technology, Germany,
Proc. Int. Symp. Eng. Secure Softw. Syst., 2014, pp. 1–18. in 2021. His dissertation was on the topic
[222] L. Cheng, K. Tian, and D. Yao, ‘‘Orpheus: Enforcing cyber-physical of non-intrusive appliance load monitoring for
execution semantics to defend against data-oriented attacks,’’ in Proc. domestic buildings using convolutional neural net-
33rd Annu. Comput. Secur. Appl. Conf., Dec. 2017, pp. 315–326. works. He is currently the Head of the Working
[223] T. Abera, N. Asokan, L. Davi, J.-E. Ekberg, T. Nyman, A. Paverd, Group Robust Secure Automation (RSA), which
A.-R. Sadeghi, and G. Tsudik, ‘‘C-FLAT: Control-flow attestation for
is part of the Research Area Advanced Automation
embedded systems software,’’ in Proc. ACM SIGSAC Conf. Comput.
Technologies (A2T) with the Institute for Automa-
Commun. Secur., Oct. 2016, pp. 743–754.
[224] G. Dessouky, S. Zeitouni, T. Nyman, A. Paverd, L. Davi, P. Koeberl, tion and Applied Informatics (IAI), Karlsruhe
N. Asokan, and A.-R. Sadeghi, ‘‘LO-FAT: Low-overhead control flow Institute of Technology (KIT). His current research interests include center
ATtestation in hardware,’’ in Proc. 54th Annu. Design Autom. Conf., around cybersecurity for critical infrastructures and automation systems
Jun. 2017, pp. 1–6. using machine learning methods.
[225] S. Zeitouni, G. Dessouky, O. Arias, D. Sullivan, A. Ibrahim, Y. Jin,
and A.-R. Sadeghi, ‘‘ATRIUM: Runtime attestation resilient under mem-
ory attacks,’’ in Proc. IEEE/ACM Int. Conf. Computer-Aided Design
(ICCAD), Nov. 2017, pp. 384–391.
[226] G. Dessouky, T. Abera, A. Ibrahim, and A.-R. Sadeghi, ‘‘LiteHAX:
Lightweight hardware-assisted attestation of program execution,’’ in
Proc. Int. Conf. Comput.-Aided Design, Nov. 2018, pp. 1–8.
[227] P. Szor, The Art of Computer Virus Research and Defense. Upper Saddle
River, NJ, USA: Addison-Wesley, 2005.
[228] A. Abbasi, J. Wetzels, W. Bokslag, E. Zambon, and S. Etalle, ‘‘On
emulation-based network intrusion detection systems,’’ in Proc. Int.
Symp. Res. Attacks, Intrusions Defenses, 2014, pp. 384–404.
[229] N. Carlini and D. Wagner, ‘‘ROP is still dangerous: Breaking modern
defenses,’’ in Proc. 23rd USENIX Secur. Symp., 2014, pp. 385–399.
VEIT HAGENMEYER received the Ph.D. degree
from Universite Paris XI, Paris, France, in 2002.
QI LIU received the B.E. degree from Beijing He is currently a Professor of energy infor-
Information Science and Technology Univer- matics with the Faculty of Computer Science,
sity, China, in 2016, and the M.S. degree from and the Director of the Institute for Automation
the Karlsruhe Institute of Technology, Germany, and Applied Informatics, Karlsruhe Institute of
in 2019, where he is currently pursuing the Ph.D. Technology, Karlsruhe, Germany. His research
degree. His current research interests include interests include modeling, optimization and
industrial control systems (ICS) security, energy control of sector-integrated energy systems,
systems security, machine learning based intrusion machine-learning based forecasting of uncertain
detection systems (IDS), information and commu- demand and production in energy systems mainly driven by renewables, and
nications technologies in ICS, binary exploitation, integrated cybersecurity of such systems.
and malware analysis.

VOLUME 10, 2022 48273