Evaluating Susceptibility of VPN Implementations

to DoS Attacks Using Adversarial Testing

Fabio Streun, Joel Wanner, Adrian Perrig

Department of Computer Science
ETH Zurich

Abstract—Many systems today rely heavily on virtual private protocols contain built-in DoS defenses by design, such as
network (VPN) technology to connect networks and protect source-binding cookies and rate-limiting mechanisms. Wire-
their services on the Internet. While prior studies compare the Guard, a recently developed VPN protocol, claims to offer
performance of different implementations, they do not consider improved DoS resilience thanks to a novel cookie mechanism
adversarial settings. To address this gap, we evaluate the resilience that secures its handshake exchange [3]. However, designing,
arXiv:2110.00407v1 [cs.CR] 1 Oct 2021

of VPN implementations to flooding-based denial-of-service (DoS)

attacks.
implementing, and deploying such mechanisms is challenging.
While the performance of different VPN implementations has
We focus on a class of stateless flooding attacks, which are been tested in prior evaluations [3], [19], their measurements
particularly threatening to real connections, as they can be do not consider adversarial settings.
carried out by an off-path attacker that operates stealthily by
spoofing source IP addresses. We have implemented various at- We aim to address this lack of adversarial evaluation by
tacks to evaluate DoS resilience for three major open-source VPN stress-testing the DoS resilience of three widely used state-of-
solutions, with surprising results: On high-performance hard- the-art open-source VPN solutions: WireGuard, strongSwan
ware with a 40 Gb/s interface, data transfer over established (IPsec), and OpenVPN. Our methodology revolves around
WireGuard connections can be fully denied with 700 Mb/s of stateless flooding attacks, in which an off-path adversary
attack traffic. For strongSwan (IPsec), an adversary can block aims to disrupt VPN connections using a high-rate flood of
any legitimate connections from being established using only
packets that exhausts resources on the targeted endpoint. We
75 Mb/s of attack traffic. OpenVPN can be overwhelmed with
100 Mb/s of flood traffic, denying data transfer through the present a flexible framework that enables efficient testing and
VPN connection as well as connection establishment completely. optimization of flooding attacks in a high-performance testbed
Further analysis has revealed implementation bugs and major network.
inefficiencies in the implementations related to concurrency
aspects. These findings demonstrate a need for more adversarial
Our evaluation has exposed major weaknesses in state-
testing of VPN implementations with respect to DoS resilience. of-the-art VPN solutions, exploiting both protocol and im-
plementation aspects to disrupt existing tunnels and block
connection establishment. These attacks can be carried out
I. I NTRODUCTION in realistic situations with surprisingly low traffic volumes,
In today’s Internet ecosystem, enterprise networks are often demonstrating that today’s open-source implementations are
dispersed over several locations, such as corporate branches, highly vulnerable to flooding attacks. This indicates that VPNs
data centers, and infrastructure hosted by cloud providers. require more rigorous stress testing before they can reliably
VPN systems are an integral part of these setups, serving provide a first layer of defense against DoS attacks. By
as the glue that connects the different locations securely by providing an effective testing methodology that is not limited
encrypting and authenticating traffic that is tunneled between to the VPN domain but also generally applies to other types
pairs of endpoints over an untrusted network such as the of protocols, our framework represents a major step toward
Internet. Given this role, VPN protocols are as ubiquitous in improving DoS resilience of protocol implementations.
emerging SD-WAN deployments as they are in more traditional Our contributions are the following:
site-to-site connections operated by Internet service providers.
Moreover, the recent shift to remote working arrangements has • We propose a framework capable of launching and evaluating
led to a surge in VPN use [4]. In many scenarios, the endpoints flooding attacks on VPN implementations. The framework
of a VPN tunnel are the only attack surface of a corporate is implemented using DPDK and can be applied to new
network backbone exposed to the Internet. Combined with the protocols with limited user-defined changes. The source code
critical reliance on one or a few tunnels that connect individual of the framework will be made publicly available upon
sites, endpoints represent an attractive target for DoS attacks. acceptance of the paper.
• We evaluate three state-of-the-art VPN solutions on high-
To mitigate such threats, deployments commonly rely on
performance hardware, providing quantitative measurements
network-layer defenses in the form of firewall rules and
of their resilience under realistic adversarial conditions. Our
confidentiality of endpoint addresses. Moreover, all major VPN
measurements show that in this setup, an attacker can block
data transfer over WireGuard connections using 700 Mb/s
of attack traffic. For the IPsec-based strongSwan solution,
Network and Distributed Systems Security (NDSS) Symposium 2021 existing tunnels appear to be less susceptible to attacks,
21-24 February 2021
ISBN 1-891562-66-5 but connections can be denied from being established using
https://dx.doi.org/10.14722/ndss.2021.23xxx only 75 Mb/s of attack traffic. On OpenVPN, due to the
www.ndss-symposium.org single-threaded implementation, the data transfer can be fully
 blocked using a 100 Mb/s flood. 2) Rate Limiting: Since a DoS attack aims to consume
• Our analysis has revealed multiple design and implementa- excessive amounts of resources at the victim, the attacker will
tion vulnerabilities as well as exploitable default configura- commonly send requests at higher rates than legitimate clients.
tions. The vulnerabilities have been reported to the respective This observation can be leveraged to identify and mitigate
developers and are being addressed to improve the DoS malicious behavior through rate limiting. VPNs apply this
resilience of their VPN solutions. Additionally, one CVE technique often to handshake requests, which are not meant to
vulnerability identifier has already been published based on be sent at a high rate by a single peer. Usually, the IP address
our efforts.1 is used to identify the peer, but also other identifiers, such
as session IDs, can be used if available. However, attackers
II. VPN P ROTOCOLS AND D O S M ITIGATION masking the origin of their requests can not only circumvent
M ECHANISMS the rate limits but may also exceed the limits of legitimate
peers, causing their requests to be dropped.
VPN is a widely used technology to tunnel communication
over the public internet securely. VPNs are also frequently used 3) Address-Binding Cookies: While various approaches to
as an access control mechanism to protect network segments mitigate spoofing are deployed on the Internet [21], it is still
and services from malicious traffic, including DoS attacks. widely possible to forge source addresses [13]. Therefore,
However, as the outer layer of encapsulation, the VPN protocol source blocking and rate-limiting mechanisms based on source
itself becomes the main attack surface. For this reason, the IP addresses are ineffective against DoS attacks performed
protocol specifications and implementations contain dedicated by adversaries with spoofing capabilities. Address-binding
defense mechanisms against DoS attacks. This section will out- cookies, originally proposed for Photuris [10], an early key
line the basic design of VPN protocols as well as common DoS management protocol for Internet Protocol Security (IPsec),
mitigation mechanisms and practices used in today’s standard are a simple mechanism to provide protocol-level mitigation
open-source implementations of WireGuard [3], IPsec [6], and against spoofing by achieving a stronger binding between the
OpenVPN [9]. source IP addresses and the hosts that control them. The
key observation for address-binding cookies is the following:
while adversaries may be able to send packets with arbitrary
A. VPN Overview source addresses, they do not control them and cannot obtain
A VPN protocol session between a pair of peers is com- responses to these packets. A VPN endpoint can utilize this
posed of two main phases: the handshake and data transfer. by initially exchanging a cookie and only allocate resources
During the handshake, the two peers mutually authenticate for requests containing a valid cookie The cookie is usually a
each other and establish a shared key, which is then used to hash over the initiator’s IP address with a secret key known
encrypt and authenticate subsequent data transfer. One peer only to the responder and periodically changed. Such a cookie
takes the role of the initiator, while the other is called the derivation is computationally inexpensive and does not require
responder. Commonly, authentication is achieved through pre- the responder to keep any state for the initiator after replying
shared public keys or certificates, and key establishment is per- with the cookie. In summary, a cookie mechanism allows the
formed through a variant of the Diffie-Hellman (DH) exchange. responder to ensure that requests are processed only from hosts
Data transfer is protected by symmetric-key cryptography. that have received a cookie before, i.e., had control over the
used IP address recently. Since this mechanism adds a round
For an adversary attempting to disrupt a VPN connec- trip to the handshake protocol, it is preferable to activate it only
tion, the handshake phase is an attractive attack surface: when the responder suspects to be under attack with spoofed
authentication and key establishment rely on CPU-intensive IP addresses.
operations and allocate memory to store information about
newly established connections. Hence, if an adversary is able 4) Pre-Shared Static Key: Another common defense mech-
to provoke a large number of handshake executions at the anism to protect a server from malicious requests is the
server using fraudulent messages, the server’s resources may use of pre-shared static keys. By requiring requests to be
be exhausted, causing it to drop handshake or data transfer authenticated with a secret key shared between the peers,
packets from legitimate peers. To resist such attacks, VPN malicious invalid requests can be immediately discarded after
endpoints must therefore prevent attackers from taking up an only a lightweight verification using symmetric cryptography.
excessive share of resources, enabling them to serve legitimate However, keys shared between multiple peers could be leaked
peers. to an attacker in various ways. Furthermore, replaying pre-
viously captured packets sent to the server could circumvent
B. DoS Mitigation Mechanisms such a mechanism, as they may contain authentication codes
that are still valid.
1) Source Blocking: A primitive technique to mitigate
certain attacks is to filter requests based on their origin. This 5) Priority assignment: VPN implementations usually pri-
is usually achieved by providing a list of accepted sources oritize authenticated traffic over unauthenticated traffic to pro-
(allowlist), a list of denied sources (denylist), or a combination tect legitimate clients against potential attack traffic. This can
thereof. However, such a mechanism is only applicable to se- be achieved by classifying incoming packets before applying
tups with a limited number of static IP addresses. Furthermore, different rate limits or passing them to threads with specific
an attacker who is able to spoof an IP address can circumvent priorities. However, this prioritization can also be problem-
such filters. atic: not all legitimate requests are authenticated, and if the
priority assignment is too strict, they could be prevented from
1 The identifier is omitted to preserve anonymity. succeeding if there is a high load of authenticated traffic.

2
 6) Parallelism: To achieve high performance, some VPNs even using the same IP address as a legitimate VPN peer.
rely on a high degree of parallelism to utilize all available CPU This is possible for stateless attacks, as the adversary does not
resources available on multi-core systems. This principle can need to receive responses from the server. If the VPN setup
also improve DoS resilience by allowing the system to serve a is symmetric, such as in site-to-site setups, obtaining the IP
higher rate of requests. Operating system kernels usually make address of both peers is likely less difficult than in asymmetric
use of receive-side scaling (RSS), a mechanism that distributes setups with peers that use dynamic IP addresses.
received packets among multiple cores for processing, avoiding
We focus on off-path adversaries, who cannot intercept or
a single-core bottleneck in the packet processing. When a
modify traffic between the two endpoints and can only send
packet is received, RSS calculates a hash over some predefined
traffic from locations that are not on the path of the targeted
packet header fields, such as IP addresses and TCP/UDP ports.
VPN tunnel. Active on-path adversaries between the targeted
The hash is then used to determine which core will process the
endpoints are considered out of scope, as such a position would
packet further, and the packet is added to the core’s receiver
allow disruption of the connection through much simpler
queue. Packets originating from different sources, i.e., packets
means, i.e., dropping packets. Moreover, we assume that the
from different flows, are likely to be processed by different
adversary cannot break the cryptographic primitives used by
cores. On the other hand, packets of a single flow are processed
the VPNs.
by the same core throughout. The hash function is configured
using a secret value, such that adversaries cannot know the
mapping between the hash fields and the cores, which could B. Attack Impact Metrics
be abused to deteriorate the performance of a specific flow. While there are various metrics to describe an attack’s
7) Deferred resource allocation: To improve performance, impact on a VPN setup, we focus on the two most important
latency in particular, achieved by protocol implementations or metrics:
to simplify code structure, it can be desirable to compute some Throughput: The maximum rate at which data can be trans-
resources eagerly. For instance, the first data packet over a ferred over an established VPN connection between two
VPN tunnel can be decrypted more quickly by the responder endpoints.
if the session key has already been pre-computed during the Connection setup delay: The time required for two VPN
handshake. However, this could present an attack surface, as endpoints to establish a new connection.
adversaries may be able to consume more of the server’s
resources without fully executing the protocol themselves. For
IV. ATTACK I MPLEMENTATION
this reason, the IPsec standards recommend that the session
key derivation should be deferred until the key is used for the We have implemented a framework that enables efficient
first time [16]. testing of stateless flooding attacks on a high-performance
testbed. Two major challenges were addressed during de-
III. S TATELESS F LOODING ATTACK velopment: (a) generating attack traffic at a high rate, and
(b) systematic analysis through automated exploration of the
To define our problem statement, we introduce a class of attack space.
DoS attacks that will form the basis of our evaluation. In a
stateless flooding attack, an adversary attempts to exhaust a A. Line-Rate Traffic Generation
victim’s resources by flooding it with packets that do not have
a causal relation to one another (stateless). The rate may be In order to carry out flooding attacks, an attack tool must
limited by the adversary’s computational resources, network be able to achieve high packet rates while enabling low-layer
bandwidth, or stealth restrictions (i.e., the maximum rate that packet manipulation (e.g., source address spoofing). For this
is not detected by a network-level DDoS mitigation system). purpose, standard application-layer software sockets are not
flexible enough. Alternatively, raw sockets allow low-layer
packet manipulation, but the kernel networking stack would
A. Threat Model
limit the performance of this approach. To overcome such
In the context of VPNs, an adversary may target an limitations, we use the Dataplane Development Kit (DPDK)
endpoint or a specific connection between two peers (i.e., a framework [5], which offers more efficient packet processing
critical site-to-site tunnel between two corporate locations) by bypassing the Linux networking stack and allowing appli-
with the goal of degrading or denying data transmission, or cations in user space to control the networking devices directly.
denying connection establishment. Our threat model considers Since this low-level control requires a considerable amount of
adversaries located on the public Internet with minimal knowl- base code to set up the development environment, we built our
edge about the targeted infrastructure: we assume that the implementation on libmoon [7]. This library abstracts away
adversary knows the VPN protocol used and the location (i.e., some of the low-level setup steps needed for most applications,
IP address and port) of at least one endpoint. Such information greatly accelerating development. Our framework leverages
can be obtained in various ways: insider knowledge, traffic these advantages to enable the implementation of various
analysis, port scanning, or infiltrating a service (e.g., signing packet flooding attacks with minimal development overhead.
up as a legitimate customer of an SD-WAN service).
B. Automated Attack Exploration Algorithm
Since our evaluation aims to determine the resilience of
VPN peers exposed to the public Internet, it is a realistic The range of possible attacks an adversary can perform to
assumption that certain attackers can spoof their source ad- disrupt a VPN connection is extensive, and manually exploring
dress [13] to remain stealthy and circumvent defenses, possibly the entire attack space is impractical. Therefore, we define an

3
appropriate attack space and devise an automatic process to VPN Client VPN Server
Switch
explore it based on an optimization algorithm. In this process,
we are primarily interested to find the most efficient attacks,
i.e., attacks with a high impact but low cost.
A simple definition for the attack space consists of flooding BW:
attacks in which a single packet is sent repeatedly at a specific MTU:
rate. Given a list of possible packets known by the adversary,
the space could be explored exhaustively by determining the
most effective attack rate for each packet. We extend this
simple definition with attacks that use not just a single type of
packet, but flood the victim with a mix of packets. Such mixed Attacker
packet attacks may be able to circumvent simple threshold-
Fig. 1. The network topology used in our evaluation setup. Three high-
based detection mechanisms [1]. A mixed attack in the space performance servers are connected through a switch over 40 Gb/s links.
defined by n possible packets can be encoded as a vector
x = (x1 , . . . , xn ), where xi is the bit rate at which packet
i is sent to the victim. The attacker knows at least the server’s IP address and attempts
As the attack impact metric, we use the measured through- to disrupt the VPN connection by attacking the server.
put over the VPN connection tp(x). The attack cost is defined The VPN server, which is the the primary device under
as the attack’s total bit rate |x| = x1 +. . .+xn and is bounded test (DUT), and the attacker are commodity server machines
by the attacker’s bandwidth bw . The weighted sum of attack running Ubuntu 18.04.4 LTS2 with two 18-core CPUs3 at
impact and attack cost is used to describe an attack’s effec- 2.10 GHz, offering a total of 72 virtual cores using hyper-
tiveness. This results in the following optimization problem: threading. The machines are equipped with a 40 Gb/s net-
work card.4 RSS is activated with the maximum number of
minimize tp(x) + α · |x| supported queues on this system, i.e., 64 queues. For RSS,
subject to xi ≥ 0 for i = 1, . . . , n both IP addresses and port numbers in a packet are used as
|x| ≤ bw hash fields.

The constant α can be adjusted to control the trade-off between The VPN client is another commodity server machine
attack impact and attack cost. With a larger value of α, running Ubuntu 18.04.4 LTS5 with two 16-core CPUs6 at
the exploration algorithm favors more efficient attacks. If 2.80 GHz (64 virtual cores using hyper-threading) and a net-
α = 0 is chosen, the algorithm selects the attack with the work card7 that supports RSS and traffic up to 100 Gb/s.
highest impact, regardless of the required bandwidth. As the The maximum transmission unit (MTU) on the network
optimization algorithm, we select a differential evolutionary interface is set to 1500 B to simulate real-world limitations of
algorithm [22]. the public Internet. The receive and transmit buffer sizes are
set to 4096 each.
To instantiate and explore the attack space, it is necessary
to provide a list of packets that the attacker could potentially
send. The list might contain only generic protocol packets as B. Attack Impact Measurements
defined by the specification, but it is also possible to provide The client machine is primarily used to measure the attack
the attacker with packets that were intercepted from legitimate impact according to the metrics defined in section III-B.
peers. Initially, the optimal attack rates for the trivial mixes Additionally, we perform profiling on the server to obtain
(i.e., mixes comprised of only one packet) are determined as additional insights into the attacked system.
a starting point for the attack exploration. The results are then
given to the optimization algorithm, which optimizes the attack a) Throughput: iperf over a single TLS connection
mixes and attack rates over multiple cycles by combining them is used to measure the maximum achievable throughput from
and removing the worst-performing candidates. the client over the already established VPN connection. The
measurements are averaged over a duration of 10 seconds,
during which the attack is performed.
V. E VALUATION S ETUP
b) Connection setup delay: The Linux time command
In this section, we describe the high-performance evalua- in combination with ping is used to obtain the connection
tion setup we have applied to evaluate the VPNs. setup delay by measuring the time required by the VPN peers
to connect and transmit the first packet to each other. Attacks
A. Network Topology are performed over the first 25 seconds of a measurement,
which stops after 85 seconds if no connection has been
Figure 1 shows our setup consisting of three machines established.
connected over a 40 Gb/s star topology using a single switch, 2 GNU/Linux 5.10.11-051011-generic x86 64
modeling a VPN connection over the Internet with an off-path 3 Intel® Xeon® CPU E5-2695 v4
adversary. We refer to one endpoint as the server and to the 4 Intel® Ethernet Controller XL710 for 40 GbE QSFP+
other as the client. The server acts as a VPN endpoint with 5 GNU/Linux 5.4.0 x86 64
a static IP address to which the client connects using any IP 6 Intel® Xeon® Gold 6242 CPU

address. Hence, the server cannot apply any IP-based filtering. 7 Mellanox MT27800 ConnectX-5

4
TABLE I. A N OVERVIEW OF THE EVALUATION RESULTS . T HE VALUES RSS Kernel + WG Job List WG Workers
SHOW THE ATTACK BANDWIDTH IN Gb/s REQUIRED TO CROSS THE GIVEN WG Receive
VPN METRIC THRESHOLD .
core 1 core 1
≤ 5% throughput connection delay ≥ 25 s
multi-flow client-flow multi-flow client-flow
NIC
WireGuard 0.7 0.3 20 20
IPsec/strongSwan 15 15 0.1 0.1
OpenVPN 0.1 0.1 0.1 0.1 core N core N

Measurements are taken across five runs. The plots in the Handshake Packet Data Packet
following sections show mean and standard deviation values
across these runs. To eliminate carry-over effects across runs, Fig. 2. WireGuard’s packet processing pipeline. Packets are initially pro-
the VPN applications running on client and server machines cessed on the receiving core, which distributes work among dedicated workers
are always restarted before a run. spawned on all cores. Multiple queues are used for incoming packets: a
handshake list and a list for data packets for each established connection.
c) Server instrumentation: To further analyze an at-
tack’s impact on the DUT, we collect metrics about packets
dropped by the network interface controller (NIC) through
ethtool and instrument the system using perf sampling. VI. W IRE G UARD

WireGuard is a relatively new VPN protocol that aims to

C. Attack Traffic Flow Configurations differentiate itself from competing solutions through its sim-
plicity, high performance, and modern design [3]. WireGuard
On multi-core systems, traffic is distributed by RSS on is “cryptographically opinionated,” i.e., it only implements
a per-flow basis among multiple receiver queues and cores a single cipher suite and keeps configuration options to a
for the processing done by the kernel. Therefore, in our minimum. This allows its protocol specification and source
evaluation, we distinguish between the following attack traffic code to be significantly less complex than of other VPNs that
configurations: offer multiple cipher suites and a plethora of configuration
options, ultimately reducing the risk of implementation bugs
single-flow: The attacker uses a single source IP address and and possible misconfigurations that could lead to vulnerable
port such that all attack packets are mapped to the same deployments.
receiver queue and core. With high probability, this mapping
does not collide with the legitimate client’s mapping. In a WireGuard setup, each peer has a unique key pair for
multi-flow: The attacker uses multiple port numbers such identification and authentication. For two peers to establish a
that the attack packets are mapped to multiple receiver secure connection, both require the public key of the other
queues and cores (assuming RSS includes the source port peer. The handshake is an instance of the IK pattern defined
in its computation, which is a standard configuration). If the by the Noise protocol framework [17] and relies on DH
attacker uses multiple IP addresses, e.g., through spoofing operations for both authentication and key-derivation. Under
capabilities or control over many IP addresses, the same normal circumstances, the handshake completes in a single
effect is replicated. round trip, as shown in fig. 3. WireGuard uses UDP as its
client-flow: The attacker uses the same IP address and port as transport layer and listens on port 51820 by default.
the legitimate client such that attack packets and the client’s
packets are mapped to the same receiver queue and core. The implementation published for the Linux operating
This pattern might achieve a higher impact on the targeted system is a kernel module and was included in the Linux kernel
connection than on other connections. However, this requires tree with the Linux 5.6 release.8 To increase performance,
the adversary to know the client’s address and port and to the processing is distributed among all available CPU cores
have spoofing capabilities for that address. (fig. 2). When a packet is received, RSS assigns it to a
core, on which the kernel stack processes it and WireGuard
then classifies it: if the packet is a handshake packet, it is
D. Overview of Evaluation Results queued to the handshake list and assigned to one of the
handshake worker threads, which are always spawned on
Table I provides an overview of the main results presented all available cores. For initiation and response packets, the
in the following sections, comparing the attack bandwidth handshake worker then (a) dequeues the packet, (b) validates
required to disrupt VPN connections for the different protocols. the cryptographic message authentication codes (MACs) (de-
We indicate two different types of breaking points for the scribed later), (c) processes it according to the Noise protocol
connections: (a) the throughput reaches less than 5% of the framework, and finally (d) stores the connection state in
value achieved in the absence of attacks, and (b) the connection memory. The processing of initiation and response packets
cannot be established during the full duration of the attack, only differs in step (c).
i.e., 25 seconds. These thresholds are only chosen to provide
a quick point of comparison between the different implemen-
tations. For more in-depth results, we refer to the following 8 Besides the kernel module, a cross-platform Go implementation exists,
sections on WireGuard, IPsec, and OpenVPN. which runs in user space and is therefore significantly slower.

5
 Initiator Responder single-flow multi-flow client-flow
Initiation

Throughput [Gb/s]
3
optional
Cookie Reply 2

Initiation with mac2 1

0
Response 0 10 20 30 40
Attack rate [Gb/s]

Transport Data
Fig. 4. WireGuard performance under a basic UDP flood of 190 B packets.

Transport Data
B. Baseline: UDP Flood
As a baseline representing a theoretical upper limit on per-
Fig. 3. WireGuard protocol diagram, including a cookie mechanism that is formance under attack, we evaluate WireGuard’s performance
activated if the responder is under load. while the server is flooded with UDP packets of the same
size as a WireGuard initiation packet, i.e., 190 B, but with a
payload of only zeros. From the results in fig. 4, we observe
A. DoS Defense Mechanisms that a throughput of approximately 2.8 Gb/s is achieved in
WireGuard operates on the principle that memory is al- the absence of attack traffic. If the flood is performed over a
located or modified only for authenticated requests. Hence, single flow, the throughput is not negatively affected up to an
applying a per-peer rate limit is sufficient to mitigate memory attack rate of 30 Gb/s, at which point the network is nearing
exhaustion attacks. However, authentication is computationally saturation. However, if the flood is spread over multiple flows,
expensive and presents an attack surface. we observe that WireGuard’s throughput steadily decreases as
the attack rate increases.
As a first line of defense against CPU exhaustion attacks,
WireGuard leverages the peers’ public keys as pre-shared The connection setup delay is consistently around 4 ms
static keys. Each handshake packet contains a keyed-hash and is not significantly affected by UDP floods up to rates
MAC (HMAC) called mac1, which holds a hash over all of 30 Gb/s. For higher rates, the connection establishment
WireGuard packet fields (up to the mac1 field) using the sometimes requires one retransmission, i.e., succeeds after 5 s.
receiver’s public key as the hash key. The receiver drops any This is most likely due to initiation packets being dropped
incoming handshake packets without a valid mac1, ensuring occasionally by the NIC under high load.
that only handshake packets from senders with knowledge of
the public key are processed. C. Initiation Packet Flood

However, there are multiple possibilities an attacker could For the first attack, we consider a simple flood of ini-
flood the server with handshake packets containing a valid tiation packets from an outside attacker who has minimal
mac1: if the victim’s public key is publicly known or com- information, i.e., does not possess the victim’s public key and
promised, the attacker can generate packets with valid mac1 therefore cannot compute valid mac1 fields. It is expected that
fields. Alternatively, previously captured handshake packets WireGuard drops these packets before performing any CPU-
from legitimate peers could be replayed. intensive operations (only requiring a HMAC computation).

To defend against floods of handshake packets containing The throughput measurements (fig. 5) reveal that the attack
a valid mac1, WireGuard uses a cookie mechanism and IP- does not affect WireGuard’s performance for the single-flow
based rate limiting as a second line of defense. If a server is traffic pattern. However, if the attack traffic is spread over
under load, e.g., through a DoS attack, it requires a second multiple flows, the achievable throughput collapses at
HMAC called mac2, which is computed over the handshake attack bandwidths above 700 Mb/s and drops to zero
packet using a cookie value as the hash key. The cookie value for 1.6 Gb/s. If the client’s source IP and port number are
may be sent only if the responder receives a handshake packet used, the attack is even more effective: with just 600 Mb/s,
with a valid mac1 (but invalid or no mac2), adding a second the attack already stops practically all legitimate traffic through
packet exchange to the handshake (shown in fig. 3). Building the VPN connection.
on its cookie mechanism, WireGuard uses rate limiting based Connection setup delay (fig. 5), on the other hand, is not
on source IP addresses for packets that contain a valid mac2, as drastically impacted by the attack as the throughput. The
i.e., packets that are source-authenticated. attacker requires a bandwidth of at least 20 Gb/s to prevent
connection establishment entirely.
Additionally, the implementation assigns low thread pri-
ority to handshake workers such that a flood of handshake We have also evaluated the impact of initiation request
messages would not monopolize the CPU and prevent previ- floods with valid mac1 and mac2 fields (appendix A). How-
ously established connections from being served. To mitigate ever, these attacks are not significantly more effective
algorithmic complexity attacks, fast hash tables for initiator than the basic flooding, indicating that the processing of the
lookup and secret keys for hash tables are used. requests is not the limiting factor of performance in this case.

6
 single-flow multi-flow client-flow
Throughput [Gb/s]

Throughput [Gb/s]
3 3

2 2

1 1

0 0
0 5 10 15 20 25 30 35 40 0 0.5 1 1.5 2
Attack rate [Gb/s] Attack rate [Gb/s]
Connection setup [s]

Connection setup [s]

1
30
0.8
0.6 20
0.4
10
0.2
0 0
0 5 10 15 20 25 30 35 40 0 5 10 15 20
Attack rate [Gb/s] Attack rate [Gb/s]
Fig. 5. WireGuard performance under an initiation request flood with invalid cryptographic fields.

TABLE II. T HE MOST IMPACTFUL W IRE G UARD ATTACK TRAFFIC MIX

D. System Instrumentation FOUND BY THE EXPLORATION ALGORITHM . F OR EACH PACKET TYPE , THE
RELATIVE FREQUENCY, PACKET RATE (kp/s), AND BANDWIDTH (Mb/s)
During single-flow attacks, the overall CPU utilization ARE LISTED .
never exceeds 25%. Since RSS assigns all attack packets to
the same core, only a single receiver thread performs the Packet type (size) Frequency [%] kp/s Mb/s
initial processing and manages their distribution among the Handshake Initiation (190 B) 15.6 116.3 176.7
different handshake workers. For attack rates above 500 Mb/s, Handshake Response (134 B) 28.1 209.3 224.3
this thread requires the total capacity of the core it is running Handshake Cookie Reply (106 B) 53.1 395.3 335.2
on. Profiling reveals that the overloaded receiver thread spends Transport Data (170 B) 0.0 0.0 0.0
roughly 50% of its runtime on assigning packets to particular Transport Data (170 B) 0.0 0.0 0.0
workers and waking them up. This causes packets that are Transport Data Keepalive (74 B) 3.1 23.3 13.8
assigned to this core to be dropped at a high rate, while packets
mapped to other cores are still processed. With the single-flow
configuration, the impact on the client’s connection is therefore We consider an adversary with minimal knowledge of the
negligible, while client-flow attacks are devastating. setup and the multi-flow traffic configuration. We provide a list
of 6 generic WireGuard packets to the algorithm, containing
During multi-flow attacks, RSS distributes the packets all packet types as defined by the protocol specification. No
among multiple cores. Hence, multiple receiver threads per- special adversary capabilities are required to construct these
form the initial processing of the attack packets. For attack packets. The attack cost weight α is set to 0.001, such that the
rates above 1.5 Gb/s, the receiver threads and handshake search prioritizes attacks with high impact.
workers require 45% and 54%, respectively, of all CPU
resources, hence, completely exhausting system resources. After 58 evolution steps and a runtime of approximately
Surprisingly, approximately 90% of CPU time is spent in 1.5 hours, the exploration algorithm discovers a packet
spinlocks to add/remove packets to/from the handshake mix that completely disrupts the throughput with just
packet queue. 750 Mb/s of attack traffic. The mix, which is shown in
table II, clearly favors handshake packets over data packets.
These observations point toward the inter-thread communi- Out of the data packets, only the keepalive packet, which is
cation as the main bottleneck in the system: either a single data the smallest possible data packet, is represented in the mix.
structure is shared and regularly accessed by many threads, or Furthermore, the size of the handshake packets (including the
one process is responsible for distributing the entire workload. cookie packet) correlates with their share in the mix. Since
smaller packets achieve a larger number of packets at a given
attack bit rate, this distribution indicates that the handshake
E. Automated Search: Mixed Flood packet rate is the primary factor for an efficient flooding
attack. Similar observations have already been described in the
In the following, we analyze the effectiveness of other previous sections, in which we suggest that the bottleneck is
attack vectors on WireGuard and explore whether a mix of not the handshake processing but the distribution of it among
different packet types may be more impactful than a pure multiple cores, which is the same for all handshake packets.
flooding attack using initiation packets. For this purpose, we
employ the automated attack exploration algorithm described The measurements in fig. 6 show that the mixed-packet
in section IV-B. attack has a significantly stronger impact on the throughput

7
Throughput [Gb/s]
3 multi-flow cookie responses instead of initiation requests seems somewhat
client-flow
counter-intuitive, the resulting mix is useful to guide the
2 process toward a more effective stateless flooding attack on
WireGuard.
1
VII. IP SEC : STRONG S WAN
0
0 0.5 1 1.5 2 Internet Protocol Security (IPsec) is a secure network
Attack rate [Gb/s] protocol suite that is widely used in VPN solutions. The IPsec
standards have continuously been updated and published as
Fig. 6. Performance of WireGuard under an attack using the optimal traffic
mix determined by the optimization algorithm.
RFCs by multiple IETF working groups [6]. The relevant
pieces of the IPsec protocol suite for our analysis are Internet
Key Exchange Version 2 (IKEv2) [11] and Encapsulating
Throughput [Gb/s]

3 multi-flow Security Payload (ESP) [12]. IKEv2 provides a key exchange

client-flow protocol to mutually authenticate two peers and establish a
2 security association (SA) containing shared security attributes
such as encryption keys. ESP is used to provide encryption
1
and authentication of IP packets between two peers after an
0 SA has been established.
0 0.5 1 1.5 2
IKE and ESP can be used with a variety of cipher suites and
Attack rate [Gb/s] in different modes. Any negotiation, including the cipher suite
Fig. 7. Performance of WireGuard under a flooding attack consisting purely negotiation, is performed in the IKE exchanges. The IKEv2
of cookie reply packets. exchange consists of two phases (shown in fig. 9):
IKE_SA_INIT: In the initial exchange, usually consisting
than the flood of initiation requests (evaluated in VI-C). of one request packet and one response packet, the peers
establish a secure channel by negotiating cryptographic
algorithms, exchanging nonces, and performing a DH key
F. Optimal Attack: Cookie Reply Flood exchange. Both peers, i.e., initiator and responder, provide
Since our evaluation suggests that the handshake packet a randomly generated security parameter index (SPI), which
is the primary factor for an efficient flooding attack, we also each side uses to identify the connection. Information about
evaluate a flood based solely on the cookie response packet. the connection and key materials are stored in an IKE
The results in fig. 7 allow us to compare the pure cookie SA, which initially is declared as half-open since the peers
response flood to the mixed-traffic attack. The pure attack is have not yet been authenticated. Any subsequent packet is
slightly more effective for multi-flow traffic, and similarly identified with the SPIs, and its payloads are encrypted and
effective for client-flow traffic. integrity-protected using the established keys.
IKE_SA_AUTH: Authentication of the half-open IKE SAs
is performed in the second IKE message exchange. This
G. Summary of WireGuard Results
step can use different methods used such as pre-shared
The design of WireGuard appears to be sound against DoS keys, RSA certificates, or elliptic-curve-based certificates.
attacks: the protocol cleverly leverages shared public keys as an Because IKE_SA_AUTH packets can become larger than
additional layer of defense. Rate limiting and a sophisticated the underlying IP layer allows (e.g., when using certificates),
cookie mechanism provide strong defenses against resource they are sometimes fragmented before being sent over the
exhaustion. Furthermore, the multi-threaded implementation network.
aims to utilize all available computation resources, increasing
the number of requests, including malicious ones, it is able to While IKEv2 uses UDP as its transport layer, ESP func-
process. tions directly on top of the IP layer. Since this can cause
issues with NAT devices, which operate on TCP/UDP ports,
However, our evaluation has shown that there are signif- ESP supports encapsulation in UDP packets. IKEv2 can detect
icant weaknesses in the implementation that negate the DoS such a setting, and if required, ensures that the NAT devices
resilience properties of the design. The flooding attacks we open a session for the ESP encapsulated packets. While the
have studied cause WireGuard to spend most of its processing IKE_SA_INIT exchange uses port 500, the IKE_SA_AUTH
time in spinlocks, indicating that CPU resources are wasted exchange would switch to port 4500, which is by default
due to suboptimal thread synchronization. This has a severe subsequently used for the ESP-encapsulated UDP packets. This
impact on the data processing of legitimate connections. In- port separation has interesting implications for the behavior of
terestingly, disruption of connection establishment seems to IPsec under handshake flooding attacks, as will be demon-
require more attack volume than to disruption of data transfer. strated by our attacks.
We have applied the automatic attack exploration algorithm A widely used open-source VPN solution based on IPsec
to find impactful attacks beyond manual analysis. However, the is strongSwan [24], which provides an implementation of
mixed-packet attack found by the algorithm ultimately does the IKEv2 protocol consisting of a user-space daemon called
not have a significantly stronger impact than a pure cookie charon, which handles the IKE packets and distributes the
response flood. Nevertheless, considering that the choice to use processing of them among multiple threads. For the encryption

8
 RSS Kernel + strongSwan strongSwan
Initiator Responder
XFRM Layer Receiver Workers

core 1 core 1 core 1

IKE_SA_INIT Request

NIC
optional
IKE_SA_INIT Cookie
core N core N
IKE_SA_INIT Request with Cookie

IKE Packet ESP Packet

IKE_SA_INIT Response

IKE_SA_AUTH Request
Fig. 8. strongSwan’s IPsec packet processing pipeline
IKE_SA_AUTH Response

and authentication of data packets (i.e., ESP packets), the

IPsec stack implementation in the operating system kernel ESP
is responsible (called XFRM in Linux). Figure 8 shows a ESP
simplified overview of the packet processing pipeline.

A. Defense Mechanisms Fig. 9. IPsec protocol diagram, including a cookie mechanism that is activated
if the responder is under load.
Numerous RFCs have been published that update the IKE
protocol or recommend best practices for implementation and
configuration to increase DoS resilience. RFC 8019 [16] is In a more sophisticated attack, not only IKE_SA_INIT
particularly relevant, as it summarizes possible DoS attacks requests could be sent to the victim, but also IKE_SA_AUTH
and mitigation mechanisms. The strongSwan implementation requests to provoke the deferred CPU-intensive DH key com-
was created years before this document and has since not been putation. An IKE_SA_AUTH request must therefore contain a
updated to follow all of the recommendations. valid responder SPI. Otherwise, it is directly dropped. How-
Since the peers do not authenticate each other during the ever, for an attacker to obtain a valid responder SPI, an IKE
IKE_SA_INIT exchange, but establish an encrypted connec- SA must have been created before, i.e., an IKE_SA_INIT
tion, which requires processing-intensive operations and mem- exchange must have succeeded. Furthermore, the DH key only
ory, this phase of the protocol presents a considerable attack needs to be computed once for each (half-open) IKE SA.
surface to DoS attacks. By flooding a server with requests, an Hence, limiting half-open IKE SAs as described above also
adversary may attempt to exhaust its computational resources mitigates this attack.
through excessive amounts of DH operations or exhaust its
memory by creating many half-open IKE SAs. B. Setup and Configurations
CPU resources required in the IKE_SA_INIT phase For our IPsec setup with strongSwan (version 5.9.2), we
can be drastically decreased by deferring the key derivation choose IKEv2 as the initiation protocol and a certificate-
into the IKE_SA_AUTH phase. This is possible because the based authentication method. The recommended cipher set10
IKE_SA_INIT response requires only the generation of the is chosen [23], which makes use of the dedicated hardware
DH public key, which can be pre-computed by the responder. accelerators on the machines. Moreover, strongSwan is config-
The symmetric DH keys, on the other hand, are at the earliest ured to spawn a thread for each logical core, i.e., 72 threads
required for the decryption of the payload in the following in total, for IKE packet processing. The cookie threshold is
IKE_SA_AUTH request. However, strongSwan does not cur- set to 32, causing the cookie mechanism to activate if more
rently implement this technique. half-open IKE SAs exist. Additionally, a maximum of 128
concurrent half-open IKE SAs is allowed for a single IP
To mitigate memory exhaustion attacks, rate limits on before the address is blocked. Default values are used for all
initiation requests per source IP, or similarly, limits on the remaining configuration options. Hereinafter, we refer to our
number of half-open IKE SAs for a single initiator IP are IPsec setup with strongSwan simply as “IPsec” setup, unless
recommended. Since such IP-based limits can be circumvented specified differently.
by an attacker capable of spoofing their source address, IKEv2
makes use of a cookie mechanism. If an initiation request is C. Baseline: UDP Flood
received, the responder responds with a cookie, usually a hash
over the IP address and the initiator’s SPI.9 Half-open IKE To see how strongSwan handles a flood of IKE_SA_INIT
SAs are only created for requests with a valid cookie. The requests, we first consider a flood of UDP packets of the
combination of rate limits and cookies prevents adversaries same size, i.e., 306 B, but with only zeroes as payload. As
from exhausting amounts of memory disproportionately to the strongSwan directly drops these packets without much pro-
number of IP addresses they control. cessing, this allows us to determine the limitations introduced
by the system.
9 The IKEv2 specification [11] does not define the construction of the cookie,
but it offers a proposal. 10 aes128gcm16-prfsha256-ecp256

9
 single flow multi flow client flow degrades the connection’s throughput only slightly more than
the baseline UDP flood. If the attack is performed over multiple
Throughput [Gb/s]

3 flows, throughput decreases steadily with increasing attack

rate, similar to the baseline. With attack rates above 15 Gb/s,
2 data transfer between the peers is largely denied. If the attacker
uses the client’s address for the traffic, the attack starts to
1 impact the connection at much lower rates as with an unrelated
single flow: at an attack bandwidth of 1 Gb/s, throughput is
0
0 10 20 30 40 reduced by roughly 30%.
Attack rate [Gb/s] However, connection setup delay is drastically affected
Fig. 10. Throughput achieved by IPsec under a pure UDP flood.
by the attack, independent of the attack traffic configuration:
with just 75 Mb/s, the attack inhibits the client from
connecting to the server during its full duration.
single-flow multi-flow client-flow
1) Discussing Throughput: As expected, single-flow at-
Throughput [Gb/s]

3 tacks affect the throughput the least among all considered

flow constellations. Since the IKE_SA_INIT requests and
2 data transport packets use different port numbers, RSS assigns
them to different cores with high probability. Hence, the client-
1 flow attack configuration is not expected to have a significantly
higher impact than the single-flow constellation. Nevertheless,
0
0 10 20 30 40 the measurements show a surprising drop of roughly 30% for
Attack rate [Gb/s] attack rates above 1 Gb/s, for which we have not been able
to find an explanation. As the multi-flow attack configuration
Connection setup [s]

100 requires the resources of many cores, it also slows down

the processing of legitimate traffic. perf sampling during
the multi-flow attack with 20 Gb/s reveals that the Linux
50 kernel uses 65% of the total CPU capacity to process received
packets. Another 12% is used by strongSwan to process IKE
packets. However, we have not identified a specific bottleneck
0 that causes the throughput to drop to zero.
0 0.1 0.2 0.3 0.4
Attack rate [Gb/s] 2) Discussing Connection Setup Delay: Overall, the results
show that strongSwan’s implementation of the IKE handshake
Fig. 11. Performance of IPsec under a flooding attack consisting of is not resilient against floods of initiation requests. There are
IKE_SA_INIT requests. The attack lasted for 25 s, and unsuccessful client
handshake attempts were aborted after 85 s (indicated by the horizontal dashed multiple possible reasons for this. While strongSwan imple-
lines). ments some recommended DoS mitigation mechanisms (such
as rate limiting in combination with the cookie mechanism),
others (such as deferred key derivation) are missing. Further-
The results are depicted in fig. 10. The connection achieves more, even though the implementation is multi-threaded, the
a throughput of roughly 2.9 Gb/s in the absence of any attacks. initial processing of all IKE packets, including the cookie
For a single-flow UDP flood, the measurements show signif- validation, is performed by a single receiver thread. Hence,
icant variance up until 30 Gb/s, beyond which the throughput even initiation requests with an invalid cookie can overwhelm
falls sharply. If the UDP flood is performed over multiple this single thread, causing legitimate requests to be dropped.
flows, the impact is much more significant: the maximum Measurements reveal that the server sends at a maximum of
throughput between the peers is inversely proportional to the about 40 000 cookie replies, indicating that at most 40 000
attack rate. Connection setup delay for strongSwan remains at IKE_SA_INIT request packets per second can be processed.
roughly 200 ms during all attacks. Further tests revealed that This limit is reached with a 98 Mb/s flood, partially explaining
with multiple iperf connections in parallel, a higher total the client’s inability to connect to the server during larger
throughput is achievable. attacks.
3) Discovered Vulnerabilities: Problematic design choices
D. Initiation Packet Flood and implementation flaws cause the denial of connection
As a comparison to the initiation flood on WireGuard establishment for attack rates between 75 Mb/s and 200 Mb/s.
described in section VI-C, we evaluate an equivalent attack Moreover, an attacker with spoofing capabilities can exploit
on IPsec, i.e., we evaluate the impact of a IKE_SA_INIT IP-based limits even if the configurations suggest differently.
request flood on strongSwan. Such an attack does not require These security vulnerabilities are further described in ap-
any special knowledge from the attacker, as the peers do not pendix B.
yet authenticate each other at this stage of the protocol. For
this attack, we use a IKE_SA_INIT request packet containing VIII. O PEN VPN
the same payload structure as used by a legitimate peer.
OpenVPN is a VPN solution designed to be easily portable
The results in fig. 11 show that the single-flow attack and deployable on different platforms [8]. OpenVPN uses

10
 RSS Kernel OpenVPN
the wrapped client key to obtain the initiator’s secret key such
that the received message can be decrypted and authenticated.
core 1 With the wrapped client key, the server does not need to
store a list of client keys. Furthermore, the wrapped client
key can also contain metadata, such as the peer’s identifier
NIC or expiration date, facilitating key management. This method
scales better because it allows the revocation of single keys
using a revocation list.
core N
However, the decryption of the wrapped client key in the
first request requires more CPU resources compared to the
other methods, even if the request is bogus. Therefore, the
tls-crypt-v2 mechanism might also present an attack
Handshake Packet Data Packet
vector for CPU exhaustion attacks.
Fig. 12. The OpenVPN packet processing pipeline, which relies on a single
core for processing both handshake and data packets. B. Setup and Configuration
For our OpenVPN setup, we use the most recent version
available at the time of this writing.11 The peers authenticate
the TLS protocol for its handshake to authenticate peers and
each other using certificates over TLS 1.3 with the ciphers AES
securely derive key material. The back-end mainly relies on the
128 GCM and SHA256. For encryption of the data, AES 128
OpenSSL library for cryptographic algorithms. A wide variety
GCM is used as well.
of cipher suites are offered, such that users can choose them
depending on the given hardware and security requirements. Under normal conditions, the peers achieve a throughput
Supported authentication methods include pre-shared keys, of about 600 Mb/s between the peers. By increasing the
username/password, and certificates. MTU of OpenVPN’s virtual network interface to 60 000 B and
enabling IP fragmentation on the physical network interface,
OpenVPN supports both UDP and TCP as underlying
a throughput of up to 4 Gb/s can be achieved. The likely
transport protocols, although UDP is the recommended choice.
cause of this performance gain is that OpenSSL performs much
Since the TLS handshake assumes a reliable transport underlay,
better on larger packets. However, IP fragmentation also causes
OpenVPN implements a custom reliable transport layer on top
issues, such as low tolerance to packet loss, which could in
of UDP. By default, OpenVPN servers listen on port 1194.
turn reduce DoS resilience. Therefore, we do not consider this
On Linux, OpenVPN runs in user space and only utilizes approach in our evaluation.
a single CPU core (as depicted in fig. 12). Compared to other
VPN implementations, which run in kernel space and use C. Initiation Floods
multiple cores, this approach is expected to achieve weaker
performance. However, it is possible to mitigate this disadvan- In this section, we evaluate the resilience of OpenVPN
tage by running multiple OpenVPN instances in parallel and to a P_CONTROL_HARD_RESET_CLIENT_V2 packet flood.
deploying a load balancer to distribute traffic between them This packet is the first packet sent by a client to establish a new
(as discussed in section IX). connection. It contains a random session identifier, which the
client uses to identify the connection. On reception, the server
responds with a similar packet containing the client’s session
A. Defense mechanisms
identifier and a new session identifier used by the server.
OpenVPN offers three options to protect its TLS handshake Without any of the TLS protection mechanisms activated, any
against flooding attacks: tls-auth, tls-crypt, and tls- adversary can create valid initiation packets.
crypt-v2. These options all provide mechanisms to authen-
The measurements in fig. 13 show that the server is barely
ticate each packet using pre-shared keys.
able to process data packets when flooded with initiation
The tls-auth option relies on a single pre-shared key packets at a rate of 50 Mb/s, which corresponds to slightly
shared between all legitimate peers and uses it to calculate more than 100 000 packets per second. Starting at an attack
a HMAC for each TLS packet. With tls-crypt, the TLS bandwidth of 100 Mb/s, data transmission and connection
packets are also encrypted with the same shared key. This establishment is denied completely. With the client-flow at-
provides improved privacy, e.g., by hiding the used certificates. tack configuration, most of the time, connection establishment
However, none of these options scale well in setups with many does not succeed during the time of our measurements (85 s),
peers: if a single peer is compromised, the single secret key even though the attack stops after 25 s.
must be updated on all peers to restore the security properties.
A solution to this scaling problem is offered by tls- D. TLS Protection Mechanisms
crypt-v2. With this option, each peer receives a unique The TLS protection mechanisms offered by OpenVPN
secret key and a version of its key encrypted by the server’s enable authentication of initiation packets using a secret key
secret key, which is called the wrapped client key. The peer shared between the peers. An adversary without knowledge
uses its secret key for encryption and authentication of the of the key cannot create a valid handshake packet if such a
TLS messages, just like with tls-crypt, but also sends the
wrapped client key in the first packet. The receiver decrypts 11 Version 2.5.1 (git:release/2.5/f186691b32e68362) with OpenSSL 1.1.1

11
 single-flow multi-flow client-flow CPU exhaustion attacks than the single-threaded implementa-
800 tion of OpenVPN. Our measurements show that an initiation
Throughput [Mb/s]

request flood significantly impacts the data transfer as well as

600 connection setup delay of OpenVPN, even with a small volume
of attack traffic. WireGuard and IPsec/strongSwan can utilize
400
more available resources by distributing the processing among
200 multiple cores, improving performance and DoS resilience.
However, our evaluations reveal that the performance of multi-
0
0 20 40 60 80 100 120 140 160 threaded implementations can also be drastically impacted by
Attack rate [Mb/s] adversarial traffic.
When under attack, WireGuard attempts to distribute pro-
Connection setup [s]

80 cessing as much as possible among the cores. However,

most of the processing resources are spent on inter-thread
60 communication, i.e., in spinlocks. This highlights a frequent
40 issue in parallel programming, where a shared data structure
20 can become a bottleneck and nullify the benefits of a multi-
0 core infrastructure entirely.
0 20 40 60 80 100 120 140 160
strongSwan also offers improved performance by utilizing
Attack rate [Mb/s] multiple threads. However, since only a single thread performs
Fig. 13. Throughput and connection setup delay of OpenVPN under a the initial processing of all packets, which is not kept to the
flooding attack using initiation packets. TLS protection mechanisms are not minimum, an attack can easily overwhelm this single thread,
enabled. causing legitimate requests to be dropped. WireGuard avoids
this problem by maximizing the distribution of work among
800
Throughput [Mb/s]

no TLS protection multiple cores.

600 tls-auth Utilizing all available resources to handle handshake pack-
400 tls-crypt ets increases the DoS resilience of the handshake process.
tls-crypt-v2 However, there exists an inevitable trade-off between allocating
200 resources for handshake packet processing and data packet
0 processing. This trade-off is evident in WireGuard, which
0 20 40 60 80 100 120 140 160 assigns lower priority to handshake workers than to data packet
Attack rate [Mb/s] workers. However, our results show that this is insufficient to
protect the data flow.
Fig. 14. Throughput comparison of OpenVPN with the different TLS
protection mechanisms enabled, under a single-flow flooding attack using For single-threaded VPN implementations like OpenVPN,
invalid initiation packets. overall performance can be improved by running multiple
instances combined with a load balancer to distribute traffic
mechanism is activated. Hence, the server can quickly detect evenly across the processes. This approach has been shown to
the attacker’s packets and drop them before expending any scale linearly with the number of instances, as long as enough
more resources. However, these protections do not affect the physical cores are available on the system [19]. However, an
impact of an initiation packet flood significantly, as is shown attacker knowing the mapping performed by the load balancer
in fig. 14. The server is still overloaded by a 50 Mb/s flood might still be able to target individual connections.
of invalid initiation packets, and the options even appear to
degrade DoS resilience of the server slightly, especially for B. Network-Layer Mitigation
tls-crypt.
a) DDoS defenses: Our results for multi-flow con-
figurations have shown that attacks using many source IP
IX. P ROPOSED M ITIGATION
addresses are consistently successful. This type of attack traffic
Our evaluation has revealed significant limitations in the is challenging to mitigate on the protocol level [21] since
DoS resilience of the most popular open-source VPN imple- it is not effective to simply block single addresses, i.e., if
mentations. Moreover, it has also led us to discover multiple the attacker has extensive spoofing capabilities or controls a
flaws in the implementations (described in appendix B), which large botnet and can frequently change sources. For IKEv2, an
are exploitable for DoS attacks.12 In the following subsections, addition to the cookie mechanism is specified that addresses
we discuss root causes for the performance issues identified in such an attacker model [16]. If activated, initiation requests
our evaluation and propose possible mitigation techniques. are required to contain not only a valid cookie value but also
a solution to a client puzzle, which can only be generated by
A. Concurrency Aspects investing a moderate amount of computation power. Hence, in
addition to controlling the used IP, an attacker would need to
As expected, the multi-threaded implementations of Wire-
expend computational resources for every single request, vastly
Guard and IPsec/strongSwan are overall more resilient against
increasing the cost of a high-rate packet flood. At the time of
12 These findings have been reported to the respective developers ahead of this writing, the client puzzle mechanism is not implemented
submission. in strongSwan. As we only considered adversaries controlling

12
a single IP address, such a mechanism would not improve the b) DoS attacks on VPN: To the best of our knowledge,
results of our measurements. there exists limited literature about practical DoS attacks on
VPNs. The Deviation attack on IKEv2 is a recent attack [15]
b) Port randomization: We observed that for Wire- that has been implemented against strongSwan. The attack
Guard, client-flow attacks are more impactful than multi- builds on a vulnerability in the IKE handshake, which was
flow attacks when a specific connection between two peers discovered in 1999, but has been considered hard to exploit in
is targeted. However, this requires the adversary to direct its practice [14]. The Deviation attack can only be performed by
attack to traffic to the same RSS queue on the server as an attacker capable of intercepting legitimate packets, which
the targeted client. If the port randomization options offered assumes a much stronger position than our off-path attacker
by the evaluated VPN implementations are used, an off-path model.
adversary is less likely to predict the port used by the client.
c) Software testing: Our framework aims to find worst-
C. Specification and Implementation case traffic patterns that consume resources at the server. This
approach is related to fuzzing, a common software testing
strongSwan implements a highly complex protocol spec- technique that explores a large domain of possible inputs
ified across multiple standards documents. Moreover, many to provoke edge cases and reveal implementation bugs [18].
essential implementation and configuration recommendations This technique has been applied to OpenVPN to find logical
for DoS resilience are described in separate RFCs. Our analysis flaws in the state machine of implementations [2]. Similar
of strongSwan has revealed that the implementation is not com- to our approach, some fuzzing tools also apply evolutionary
pliant with many of the recommendations, e.g., RFC 8019 [16]. algorithms to effectively explore the input space of a given
These types of deviations make it very challenging to deploy black-box application [20]. Fuzzing is designed for a setting
and configure a DoS-resilient VPN, even for a user that has in which candidate inputs can be tested in rapid succession,
a thorough understanding of the specifications. WireGuard whereas in our approach, measuring the impact of a flooding
does not suffer from the same issues as it was designed and attack strategy is significantly more time-consuming.
implemented in conjunction.
XI. C ONCLUSION
D. Adversarial Testing Methodology We present a framework for generating and testing stateless
We envision developers to integrate adversarial tests such flooding attacks on VPN implementations. Our evaluation on
as ours into the development process to evaluate the DoS a real setup shows that state-of-the-art VPN solutions are
resilience of their designs and implementations. A high- vulnerable to well-orchestrated flooding DoS attacks. This
bandwidth testbed is beneficial for such evaluations, but as has important implications for real-world deployments: critical
our results have shown, many attacks clearly show their site-to-site connections commonly make use of several redun-
impact already at traffic rates below 1 Gb/s, which are easily dant VPN tunnels between different endpoints to achieve high
achievable on inexpensive commodity hardware. Moreover, availability. However, our results show that connections over
the attacks are not specific to a network topology and could as many as 10 tunnels could be brought down entirely using a
also be transferred to cloud-based testbeds. Our adversarial few gigabits of bandwidth, if the adversary is able to determine
testing framework can be applied to new protocols and can endpoint locations.
be extended and modified to perform other flooding attacks. These results highlight that rigorous adversarial testing is
Even providing just a simple list of generic protocol packets crucial for creating more DoS-resilient protocol implementa-
to the exploration algorithm can reveal interesting results, as tions. Our framework represents a major step toward making
we have observed in our analysis of WireGuard. this type of testing more accessible, and its source code will
be made available publicly.
X. R ELATED W ORK
In future work, our methodology could be applied to pro-
In this section, we review existing performance studies prietary (closed-source) VPN implementations, as the source
of the VPN implementations we have evaluated. Moreover, code is not required to carry out the adversarial testing.
we provide an overview of the literature on DoS attacks and Moreover, it would be useful to complement the evaluation
defenses on VPNs and relate our approach to standard software with other adversary models such as insider adversaries. Such
testing techniques. an attacker can establish a legitimate connection to the VPN
server, aiming to disrupt other clients connected to the same
a) VPN performance evaluation: In a recent evaluation, server. Finally, the automated attack space exploration could
Pudelko et al. [19] compare the performance of the same benefit from some additional extensions, e.g., by a combination
three open-source VPN implementations that we investigated. with traditional fuzzing techniques that enable a higher degree
Their findings are consistent with our baseline measurements of automation such that the framework can be applied more
(i.e., with no attack traffic) and highlight shortcomings in the easily to new protocols.
parallel implementations that are confirmed by our results. The
original WireGuard paper also includes a performance compar-
E THICAL C ONSIDERATIONS
ison against IPsec and OpenVPN [3], although the developers
state that these measurements are relatively dated [25]. Our We have carefully followed conventions for responsible
results complement these evaluations, which do not consider disclosure. More than a month prior to submission of this draft,
adversarial traffic and therefore provide no indication about we reported our findings according to the respective security
DoS resilience of different implementations. report policies of the studied VPN projects.

13
 R EFERENCES single-flow multi-flow client-flow

Throughput [Gb/s]
[1] Azeem Aqil, Ahmed O. F. Atya, Trent Jaeger, Srikanth V. Krishna-
murthy, Karl Levitt, Patrick D. McDaniel, Jeff Rowe, and Ananthram
3
Swami. Detection of stealthy TCP-based DoS attacks. In IEEE Military
Communications Conference (MILCOM), 2015. 2
[2] Lesly-Ann Daniel, Erik Poll, and Joeri de Ruiter. Inferring OpenVPN
state machines using protocol state fuzzing. In IEEE European 1
Symposium on Security and Privacy Workshops (EuroS&PW), 2018.
[3] Jason A. Donenfeld. WireGuard: Next generation kernel network tunnel.
0
0 0.5 1 1.5 2
In Network and Distributed System Security Symposium (NDSS), 2017.
Attack rate [Gb/s]
[4] Anja Feldmann, Oliver Gasser, Franziska Lichtblau, Enric Pujol, Ingmar

Connection setup [s]

Poese, Christoph Dietzel, Daniel Wagner, Matthias Wichtlhuber, Juan
Tapiador, Narseo Vallina-Rodriguez, Oliver Hohlfeld, and Georgios
30
Smaragdakis. The lockdown effect: Implications of the COVID-19
pandemic on Internet traffic. In Internet Measurement Conference 20
(IMC), 2020.
[5] Linux Foundation. Data plane development kit (DPDK), 2015. 10
[6] Sheila Frankel and Suresh Krishnan. IP security (IPsec) and internet
key exchange (IKE) document roadmap. Technical Report 6071, RFC 0
Editor, 2011. 0 5 10 15 20
[7] Sebastian Gallenmuller, Dominik Scholz, Florian Wohlfart, Quirin Attack rate [Gb/s]
Scheitle, Paul Emmerich, and Georg Carle. High-performance packet
processing and measurements. In International Conference on Commu- Fig. 15. Performance of WireGuard under an attack using initiation packets
nication Systems & Networks (COMSNETS), pages 1–8, 2018. that contain a valid mac1.
[8] OpenVPN Inc. OpenVPN. [Online]. Available: https://openvpn.net/.
[9] OpenVPN Inc. OpenVPN source code. [Online]. Available: https://
github.com/OpenVPN/openvpn. [23] strongSwan. strongSwan. [Online]. Available: https://www.strongswan.
org/.
[10] Phil R. Karn and William A. Simpson. Photuris: session-key manage-
ment protocol. RFC 2522, RFC Editor, 1999. [24] strongSwan. strongSwan security recommendations. [On-
line]. Available: https://wiki.strongswan.org/projects/strongswan/wiki/
[11] Charlie Kaufman, Paul E. Hoffman, Yoav Nir, Pasi Eronen, and Tero SecurityRecommendations.
Kivinen. Internet key exchange protocol version 2 (IKEv2). Technical
[25] WireGuard. WireGuard: fast, moder, secure VPN tunnel. [Online].
Report 7296, RFC Editor, 2014.
Available: https://www.wireguard.com/.
[12] S. Kent. IP encapsulating security payload (ESP). RFC 4303, RFC
Editor, 2005.
[13] Franziska Lichtblau, Florian Streibelt, Thorben Krüger, Philipp Richter,
and Anja Feldmann. Detection, classification, and analysis of inter- A PPENDIX
domain traffic with spoofed source IP addresses. In Internet Measure-
ment Conference (IMC). ACM, 2017. A. Additional Attacks on WireGuard
[14] C. Meadows. Analysis of the internet key exchange protocol using the
NRL protocol analyzer. In IEEE Symposium on Security and Privacy, Instead of flooding attacks composed of generic initiation
pages 216–231, 1999. requests, we are considering packets containing a valid cryp-
[15] Tristan Ninet, Axel Legay, Romaric Maillard, Louis-Marie Traonouez, tographic mac1. An adversary could generate such traffic by
and Olivier Zendra. The deviation attack: A novel denial-of-service replaying a previously captured legitimate request or using
attack against IKEv2. In IEEE International Conference on Trust, a pre-shared key obtained somehow. This attack circumvents
Security and Privacy in Computing and Communications (TrustCom),
2019. WireGuard’s first DoS mitigation mechanism to cause many
CPU-intensive DH calculations, triggering the cookie mecha-
[16] Y. Nir and V. Smyslov. Protecting internet key exchange protocol
version 2 (IKEv2) implementations from distributed denial-of-service nism.
attacks. RFC 8019, RFC Editor, 2016.
[17] Trevor Perrin. The Noise protocol framework. [Online]. Available:
The measurements in fig. 15 show that flooding the VPN
https://noiseprotocol.org, 2018. server with initiation requests containing a valid mac1 affects
[18] Theofilos Petsios, Jason Zhao, Angelos D. Keromytis, and Suman Jana. performance slightly more than an attack without a valid
SlowFuzz: Automated domain-independent detection of algorithmic mac1. During the multi-flow attack, the server activates the
complexity vulnerabilities. In ACM SIGSAC Conference on Computer cookie mechanism if the attack rate is above 350 Mb/s, i.e.,
and Communications Security (CCS), 2017. if the server receives more than 230 000 initiation requests per
[19] M. Pudelko, P. Emmerich, S. Gallenmüller, and G. Carle. Performance second. An attacker in possession of the server’s public key
analysis of VPN gateways. In IFIP Networking Conference, 2020. can extract the cookie value from the cookie responses and
[20] Sanjay Rawat, Vivek Jain, Ashish Kumar, Lucian Cojocar, Cristiano use it to generate initiation requests with a valid mac2.
Giuffrida, and Herbert Bos. VUzzer: Application-aware evolutionary
fuzzing. In Network and Distributed System Security Symposium Surprisingly, the measurements in fig. 16 show that a flood
(NDSS), 2017.
of initiation packets with both valid MACs is slightly less
[21] Kotikalapudi Sriram and Douglas C. Montgomery. Resilient interdo-
main traffic exchange: BGP security and DDoS mitigation. Technical
effective than the attack with just with a valid mac1. This
report, NIST, 2019. indicates that the rate-limiting mechanism is less resource-
[22] Rainer Storn and Kenneth Price. Differential evolution – a simple demanding than the creation and transmission of cookie re-
and efficient heuristic for global optimization over continuous spaces. sponses, which is done for each received handshake request
Journal of global optimization, 11(4):341–359, 1997. without a valid mac2.

14
 multi-flow to be the bottleneck, since the process runs at the full capacity
of its core.
Throughput [Gb/s]

3
single-flow multi-flow client-flow
2

Connection setup [s]

60
1
40
0
0 0.5 1 1.5 2
20
Attack rate [Gb/s]
0
Connection setup [s]

0 0.1 0.2 0.3 0.4

Attack rate [Gb/s]
20
Fig. 17. Connection setup delay of strongSwan with the fixes.
10
2) strongSwan – High Number of Half-Open IKE SAs:
0 Interestingly, even though the cookie threshold is set to 32
0 5 10 15 20 in our configuration and the initiation requests send by the
Attack rate [Gb/s] attacker do not contain any cookie, hundreds of half-open
IKE SAs are created during the attack. The number also
Fig. 16. Performance of WireGuard under an attack using initiation packets
that contain valid mac2, which is not possible with a spoofed IP address.
exceeds the limit of half-open IKE SAs for a single IP,
even though the attacker uses only a single IP address. The
reason for this behavior is a known race condition in the
B. Security Vulnerabilities implementation. When receiving an IKE_SA_INIT request,
strongSwan checks if the cookie threshold or the IP threshold
In the following, we describe some security vulnerabilities is reached and responds with a cookie or drops the packet if
that our adversarial testing has exposed. The vulnerabilities necessary. If the number of half-open IKE SAs is below the
have been reported to the respective developers at least a month thresholds, the packet is added to a job queue and processed
prior to submission. by another thread, creating a new half-open IKE SA for it.
Since the thresholds are only checked initially and hundreds
1) strongSwan – Cookie Reuse Limit: Further analysis of of jobs can be queued, the number of half-open IKE SAs may
the packet traces reveals that, as expected, the server does not exceed the thresholds specified in the configuration.
respond to some of the legitimate requests when under attack.
However, we also observe some exchanges where the client 3) OpenVPN – Forced Key Renegotiation: Throughout our
also receives cookie responses to requests already containing evaluation of OpenVPN, we discovered that an adversary capa-
a cookie, indicating that the server rejects some cookie values. ble of spoofing a legitimate client’s address could trigger a TLS
renegotiation. By injecting a single spoofed initiation packet,
strongSwan updates its secret cookie value after it is used the server believes the client requests a key renegotiation and
for 10 000 cookie responses.13 In our scenario, where the server responds to it. The client receiving the packet unexpectedly
generates approximately 40 000 cookie responses each second, accepts it, assumes the server demands a key renegotiation,
the secret is updated four times per second, causing the server and responds accordingly to it. Consequently, the client and
to reject most of the packets sent by the client due to an server perform an unnecessary TLS renegotiation, as shown in
outdated cookie. fig. 18.
Note that even though the attacks last for only 25 sec- While a forced key renegotiation is already highly prob-
onds, the connection is often not established at all, i.e., the lematic, the attack actually causes a more significant problem
measurement stopped after 85 seconds. This is mainly due to because the two peers are unable to complete the handshake.
strongSwan aborting the initiation process after receiving five This results in a complete DoS for the connection, which per-
different cookie responses.13 Manually restarting the initiation sists even through a restart of the client. Only after restarting
process right after completion of the attack usually results in OpenVPN on the server, the client can establish a connection
successful connection establishment. again. The attack is mitigated if one of the TLS-protection
mechanisms tls-auth, tls-crypt, or tls-crypt-v2
A possible fix to the exploitable design decision is to update is used. However, if the attacker is in possession of the
the cookie secret periodically after some seconds. Figure 11 secret key or a previously sent initiation packet, the protection
shows the impact of a IKE_SA_INIT flood on a patched mechanisms can be circumvented.
strongSwan implementation that updates the cookie secret
every 30 s.14 A significant improvement can be observed for We would expect the client to prevent the attack by drop-
attack rates between 75 Mb/s and 300 Mb/s. For higher rates, ping the initiation requests since it already contains a session
the single thread validating all of the initiation packets seems ID, which was not set by it. Interestingly, when receiving the
server’s packet, the client reports an error in the logs due to
13 This
value is hard-coded and cannot be configured. an unexpected acknowledgment of a packet, which is actually
14 The
developers of strongSwan implemented the improvement based on the packet sent by the attacker. Nevertheless, it proceeds with
our recommendations. the handshake.

15
 Client Server Attacker

P CONTROL HARD RESET CLIENT V2

P CONTROL HARD RESET SERVER V2

P CONTROL HARD RESET CLIENT V2

P ACK V1

TLS Handhsake

Fig. 18. Packet exchange of the forced key renegotiation attack on OpenVPN.

16