Testing of DDoS Protection Solutions
Testing of DDoS Protection Solutions
Abstract
Distributed Denial of Service (DDoS) attacks invade networks and web services
every day. Many current research projects and activities try to design various DDoS
protection solutions. Nevertheless, there are more and more advanced DDoS
attacks that are ingenious and powerful which may cause that many of these
comprehensive DDoS protection solutions are not so efficient and do not fully
mitigate advanced DDoS attacks. Accordingly, it is important to test DDoS
protection solutions and reveal their limitations and bottlenecks prior to employ
them into networks. This work deals with DoS and DDoS detection techniques
and presents the testing procedures of DDoS protection solutions. We describe
state of the art in detection techniques of current DDoS attacks. The techniques are
based on signature and anomaly detection. Other alternative approaches are also
evaluated and their advantages and drawbacks are discussed. Besides these detection
techniques, we survey the DDoS protection solutions and special DDoS protection
appliances and evaluate them.
Further, we introduce two testing procedures for observing the behaviour of
network security and DDoS protection appliances during the DDoS attacks. The
first testing procedure is based on a software DDoS generator that runs on
common server or personal computer. The paper also presents various software
DDoS generators and their specifications. The second testing procedure uses the
professional stress tester Spirent Avalanche which enables to generate various types
of DDoS attacks. This stress tester is able to mix legitimate traffic with DDoS
attacks and emulates various communication protocols and services. We evaluate
these testing procedures and present our experimental results of both approaches.
We focus on the performance and modularity of these testing procedures and the
range of possible DoS/DDoS attacks that can be generated.
Keywords: DoS Attacks, DDoS Attacks, DDoS protection, DDoS detection,
network, security, tests.
1 Introduction
Internet services, websites and web applications are frequently used by many clients
every day. These services must work correctly and must be available for users who
use them. Nevertheless, the Internet connection enables to various attackers to hit
these services and cause economic damages caused by the malfunction or
interruption of these services. Distributed denial of service attacks become very
frequent nowadays. Generally, a Denial of Service (DoS) attack is realized by one
host. Distributed DoS attacks are sent by more hosts or bots that are controlled by
an attacker. These attacks usually flood services at target devices connected to the
Internet. The basic principle of DDoS attacks is depicted in Figure 1. In the figure,
the combination of flood DDoS and amplification flood DDoS attacks is shown.
More information about types of DDoS attacks can be found in the paper [ 1 ].
DoS/DDoS attacks are threats especially for highly-profiled web services and sites
of financial institutions, government and large corporations. Many of these
institutions use data centers that are very often targets for sophisticated and
powerful attacks. There are many solutions, techniques and appliances that try to
mitigate DoS/DDoS attacks. The testing of these solutions and devices provides
important information about the defense of the sites and services. The test outputs
can help to better configure the employed devices and fix the bottlenecks in the
security solutions. There are many test appliances that can provide this testing.
Nevertheless, these appliances are usually expensive. Therefore, owners of websites
and services are not able to test their security solutions and perform the stress tests
to detect bottlenecks and the limits of their sites.
In this paper, we present some state of the art DDoS detection techniques (Section
2) and protection solutions and appliances (Section 3). Then, we describe popular
DDoS testing tools and appliances (Section 4). The main contribution of this work
can be found in Sections 5 - 7 where we introduce the DDoS testing procedures
which are based on software DDoS generator (Section 5) and hardware appliance
(Section 6). Section 7 discusses the pros and cons of these two procedures and
compares them.
2.1
The signature detection methods are based on the basic knowledge of DDoS
attacks patterns. These signatures/patterns are usually observed by security experts.
Then, the patterns are implemented into security network devices and IDS. These
devices must monitor packets and recognize the patterns of incoming DDoS
attacks. This type of the detection is fast but is effective only against already known
DDoS attacks. There are many DoS/DDoS attacks (e.g. TCP mixed flag attacks,
X-mas tree attacks) that can be easily detected by this technique. On the other
hand, the signature detection techniques are not able to recognize unknown
DoS/DDoS attacks. The more details about signature detection techniques can be
found in papers [ 3 ] and [ 4 ].
2.2
This type of the detection method detects and classifies attacks by anomalies caused
in network traffic. There are attacks such as flooding attacks that use a large
amount of TCP-SYN, UDP or ICMP packets. This increase can be observed as an
anomaly in the normal network traffic. The classic anomaly detection techniques
can be based on the observation of the dynamic statistical properties in network
traffic, e.g., time to live, IP header information and other data. Some of these
techniques are described in papers [ 5 ], [ 6 ], [ 7 ]. The paper [ 8 ] presents the
possibility of using Artificial Intelligence (A.I.) tools, e.g., neural networks and
genetic algorithms, to detect unusual network traffic and the classification of DDoS
attacks.
A.I. methods are able to learn how normal network traffic looks like, and then, the
methods can detect and classify anomalies in the traffic. The main disadvantage of
the anomaly detection methods is a larger number of false positive alarms. The
anomaly detection methods are usually slower than signature detection methods
due to the observation of larger samples of data from the network traffic.
Nevertheless, these methods might detect unknown and new types of DDoS/DoS
attacks.
2.3
(patterns) are extracted from the both traffics. Extracted variables can be calculated
by statistical tools, e.g., Granger Causality Test, Auto Regressive Model and so on.
Observed deviations from the normal profile then cause attack alarms.
3.1
3.2
The special DDoS/DoS protection appliances offer one single-box solutions that
can be plugged into networks or data centers to protect the services against the
various types of DDoS/DoS attacks. These special anti-DDoS appliances are
usually very computationally and memory powerful. They have good technical
support and can mitigate some unknown and large DDoS/DoS attacks.
Some of common DDoS/DoS protection appliances are shortly described in the
following text:
3.3
Nowadays, there are many cloud based DDoS protection providers who offer
DDoS/DoS protection as a service. This service is especially used by small-medium
businesses and enterprise-level companies who cannot afford the special anti-DDoS
appliances. When a DDoS/DoS attack is detected at the client side, whole inbound traffic is redirect to a cloud DDoS protection technology, more precisely,
the nearest cloud center of the provider, which employs DDoS filtering techniques
to remove the DDoS traffic and route the legitimate traffic back to the client.
The cloud DDoS protection services and providers such as Incapsula, Defense.net,
Prolexic DDoS Mitigation Services, Verising DDoS Protection Services,
CloudFlare Enterprise, Nexusguard and others rent their services usually for one
year per thousands to tens thousands euros. Nevertheless, using the cloud based
DDoS protection services can be less expensive for certain types of clients
(small/medium high-profiled ecommerce companies) than employing the special
anti-DDoS appliances. On the other hand, the detection and mitigation of the
DDoS/DoS attacks take longer time due to the routing.
4.1
Software DDoS generators and program tools are usually easy to acquire. These
tools can be often open source and can be downloaded for free. The tools can be
started on common computers and servers which are plugged to a target which is
testing.
Some popular software DDoS/DoS generators and tools are shortly described in
the following text:
Low Orbit Ion Canon (LOIC) this open source tool, which is written
in C#, provides stress testing and can generate various flooding HTTP,
TCP and UDP attacks. LOIC is easy to use due to the graphic interface
and enables DDoS attacks when is used by multiple users.
XOIC this tool is similar to LOIC. The tool provides DoS attacks based
on TCP, UDP, ICMP and HTTP protocols that is efficient against small
websites.
PyLoris this tool, which is written in Python, can be used for testing
servers. The tool provides a simple graphic interface and enables to
generate various DoS attacks based on protocols such as HTTP, FTP,
SMTP, IMAP and Telnet to hit the concrete service.
OWASP DOS HTTP POST this tool performs DoS attacks based on
the HTTP protocol. The tool has been developed by OWASP (Open Web
Application Security Project) group to provide a L7 DoS testing tool for
websites.
SlowLoris this DoS tool enables to generate only one type of a slow
denial of service attack. The tool poisons a HTTP server due to the
holding the connections open by sending partial HTTP requests. This
tool, which is programmed in Perl, does not provide TCP/UDP DoS
attacks and other flood attacks.
Others there are many tools that can be used for testing or for hacking,
such as GoldenEye HTTP Denial Of Service Tool, DAVOSET, HULK
(HTTP Unbearable Load King).
Many of described software DDoS/DoS tools focus solely on testing web servers
such as OWASP DOS HTTP POST tool, SlowLoris, R-U-D-Y, Tors Hammer,
HULK. Some tools such as LOIC, XOIC, DDOSSIM and PyLoris can be used to
test other services such as SMTP, FTP and can be used to flood servers and test
their limits.
4.2
There are appliances that can serve as hardware DDoS generators. These appliances
mainly serve as powerful stress testers, traffic and protocol emulators and enable to
test the network devices or whole network segments and solutions. These
appliances are usually based on multi-core processors, strong memory and network
interfaces with high throughput. These hardware based DDoS testers are very
powerful and can generate large traffic and DDoS attacks. The main disadvantage
of these appliances is their cost.
Common hardware DDoS generators and appliances are shortly described in the
following text:
5.1
The testing topology consists of two switches (Cisco Catalyst 2960 and Linksys
EG008W), a server/pc which generates DoS traffic a SW DoS generator,
a control terminal, service/site clients (a voluntary node which emulates clients or
routed real clients traffic) and a tested device. This testing topology which is based
on the software DoS generator is depicted in Figure 3.
The hardware of the SW DoS generator node should be powerful (strong CPU and
memory) to generate a large number of packets. Tested device can be a webserver,
a firewall, a router and so on. If we want test webservers or other services, we
should emulate website/service clients traffic by a client emulator application and
mix it with DoS traffic by using highly performed switch (Switch 2) to get real
results. If we test a firewall or a router performance and DoS mitigation functions,
we can generate DoS attacks directly (Switch 2 is not needed). The control terminal
is used for remote control and configuration of the nodes and devices in the testing
topology via Switch 1.
5.2
We test our procedure with two differently powerful hardware nodes (HW1: CPU
Intel Xeon E5310 @1,6 GHz RAM 2GB / 333MHz, , HW2: CPU Intel Xeon
E3440 @2,53 GHz RAM 8GB / 1 333MHz). Figure 4 shows how the hardware
specification of the SW DoS generator is important. The more powerful device
HW2 is able to generate more DDoS packets than device HW1 (HW2 around
204000 255000 packets per second and HW1 around 171000 238000 packets
per second). The most packets can be generated by using the ARP flood attack.
Nevertheless in practice, the number of packets can be limited by network interface
used (1 Gbps in this measurement).
6.1
The testing topology consists of one switch (Cisco Catalyst 2960), a test appliance
which generates normal traffic and DDoS/DoS traffic, a control terminal and
a tested device. This testing topology which is based on DDoS/DoS test appliance
is depicted in Figure 5.
Figure 6: Throughput of Cisco Firewall ASA 5510 with DDoS SYN flood attacks.
6.2
Spirent Avalanche 3100 B has several interfaces with 10 Gbps and 1 Gbps
throughput. The appliance by using 1 Gbps interface is able to generate huge
number DDoS packets (up to several million) per second until the link saturation.
By using one 10 Gbps interface, this appliance is able to generate around 7.5
million DDoS packets (SYN flood) per second. Avalanche 3100 B is able to mix
the normal and DDoS traffic. Further, we can configure many options of DDoS
attacks (rate, delay, iterance, duration and so on) and test more DDoS attacks in
one test scenario.
8 Conclusions
In this paper, we described and evaluated the basic DDoS/DoS detection
techniques (anomaly, signature and hybrid) and three DDoS/DoS protection
approaches (security network devices based, Anti-DoS appliance based and cloud
based). The cloud based DDoS mitigation solutions are more appropriate for small
and medium sized networks due to modest costs, a high percentage of the DDoS
mitigation and solid detection and mitigation response times (minutes).
Nevertheless, the anti-DDoS/DoS appliance based protection solutions are usually
more costly than cloud based protection solutions but they should be employ in
high-profiled large e-commerce and data centers due to faster DDoS/DoS detection
and mitigation and the higher frequency of attacks.
The paper also describes some common hardware and software based DDoS/DoS
generators and testers and their specifications and two DDoS/DoS testing
procedures are presented. The software based testing procedure is able to test some
basic DoS/DDoS attacks and flood less performed network devices to get their
limits. For example, the DDoS SYN attack is generated up to 208 000 packets per
second. The appliance based testing procedure is able to test this DDoS SYN attack
up to 7.5 million packets per seconds if Avalanche 3100B with 10 Gbps interface is
employed. For the professional testing of larger networks and some special security
devices, the appliance based procedure is more appropriate than software based
procedure due to their performance and configuration options.
Acknowledgements
Research described in this paper was financed by the National Sustainability
Program under grant LO1401, by the Czech Science Foundation under grant no.
14-25298P and the Technology Agency of the Czech Republic project
TA0301081. For the research, infrastructure of the SIX Center was used.
References
[1]
[2]
Alenezi, M., and Reed, M.: Methodologies for detecting DoS/DDoS attacks
against network servers, in ICSNC 2012, The Seventh International
Conference on Systems and Networks Communications, pp. 92-98, 2012.
[3]
Peng, T., Leckie, C., Ramamohanarao, K.: Survey of network based defense
mechanisms countering the DoS and DDoS problems, ACM Computing
Surveys (CSUR), vol. 39, p. 42 pages, 2007.
[4]
Kompella, R. R., Singh, S., Varghese, G.: On scalable attack detection in the
network, in Proceedings of the 4th ACM SIGCOMM Conference on Internet
Measurement. ACM Press, New York, pp. 187-200, 2004.
[5]
[6]
[7]
Kim, Y., Jo, J. Y., Suh, K. K.: Baseline profile stability for network anomaly
detection, International Journal of Network Security, vol. 6, No.1, pp. 60
66, 2008.
[8]
[9]
Blazek, R. B., Kim, H., Rozovskii, B., Tartakovsky, A.: A novel approach to
detection of denial-of-service attacks via adaptive sequential and batchsequential change-point detection methods, pp. 220-226, 2001.