A Survey of Software-Defined Networking: Past,

Present, and Future of Programmable Networks

Bruno Nunes Astuto, Marc Mendonça, Xuan Nam Nguyen, Katia Obraczka,
Thierry Turletti

To cite this version:

Bruno Nunes Astuto, Marc Mendonça, Xuan Nam Nguyen, Katia Obraczka, Thierry Turletti. A
Survey of Software-Defined Networking: Past, Present, and Future of Programmable Networks. 2013.
�hal-00825087v3�

HAL Id: hal-00825087

https://inria.hal.science/hal-00825087v3
Submitted on 29 Oct 2013 (v3), last revised 19 Jan 2014 (v5)

HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est

archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents
entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non,
lished or not. The documents may come from émanant des établissements d’enseignement et de
teaching and research institutions in France or recherche français ou étrangers, des laboratoires
abroad, or from public or private research centers. publics ou privés.
IN SUBMISSION 1

A Survey of Software-Defined Networking: Past,

Present, and Future of Programmable Networks
Bruno Astuto A. Nunes, Marc Mendonca, Xuan-Nam Nguyen, Katia Obraczka, and Thierry Turletti

Abstract—The idea of programmable networks has recently The idea of “programmable networks” has been proposed as
re-gained considerable momentum due to the emergence of a way to facilitate network evolution. In particular, Software
the Software-Defined Networking (SDN) paradigm. SDN, often Defined Networking (SDN) is a new networking paradigm
referred to as a “radical new idea in networking”, promises
to dramatically simplify network management and enable in- in which the forwarding hardware is decoupled from con-
novation through network programmability. This paper surveys trol decisions. It promises to dramatically simplify network
the state-of-the-art in programmable networks with an emphasis management and enable innovation and evolution. The main
on SDN. We provide a historic perspective of programmable idea is to allow software developers to rely on network
networks from early ideas to recent developments. Then we resources in the same easy manner as they do on storage
present the SDN architecture and the OpenFlow standard in
particular, discuss current alternatives for implementation and and computing resources. In SDN, the network intelligence is
testing of SDN-based protocols and services, examine current logically centralized in software-based controllers (the control
and future SDN applications, and explore promising research plane), and network devices become simple packet forwarding
directions based on the SDN paradigm. devices (the data plane) that can be programmed via an open
Index Terms—Software-Defined Networking, programmable interface (e.g., ForCES [44], OpenFlow [85], etc).
networks, survey, data plane, control plane, virtualization. SDN is currently attracting significant attention from both
academia and industry. A group of network operators, ser-
vice providers, and vendors have recently created the Open
I. I NTRODUCTION Network Foundation [13], an industrial-driven organization,
OMPUTER networks are typically built from a large to promote SDN and standardize the OpenFlow protocol [85].
C number of network devices such as routers, switches and
numerous types of middleboxes (i.e., devices that manipulate
On the academic side, the OpenFlow Network Research Cen-
ter [14] has been created with a focus on SDN research. There
traffic for purposes other than packet forwarding, such as a have also been standardization efforts on SDN at the IETF and
firewall) with many complex protocols implemented on them. IRTF and other standards producing organizations.
Network operators are responsible for configuring policies to The field of software defined networking is quite recent,
respond to a wide range of network events and applications. yet growing at a very fast pace. Still, there are important
They have to manually transform these high level-policies into research challenges to be addressed. In this paper, we survey
low-level configuration commands while adapting to changing the state-of-the-art in programmable networks by providing a
network conditions. And often they need to accomplish these historic perspective of the field and also describing in detail
very complex tasks with access to very limited tools. As a the SDN paradigm and architecture. The paper is organized
result, network management and performance tuning is quite as follows: in Section II, it begins by describing early efforts
challenging and thus error-prone. The fact that network devices focusing on programmable networks. Section III provides an
are usually vertically-integrated black boxes exacerbates the overview of SDN and its architecture. It also describes the
challenge network operators and administrators face. OpenFlow protocol. Section IV describes existing platforms
Another almost unsurmountable challenge network practi- for developing and testing SDN solutions including emulation
tioners and researchers face has been referred to as “Internet and simulation tools, SDN controller implementations, as
ossification”. Because of its huge deployment base and the well as verification and debugging tools. In Section V, we
fact it is considered part of our society’s critical infrastructure discuss several SDN applications in areas such as data centers
(just like transportation and power grids), the Internet has and wireless networking. Finally, Section VI discusses future
become extremely difficult to evolve both in terms of its phys- research directions related to SDN.
ical infrastructure as well as its protocols and performance.
II. E ARLY P ROGRAMMABLE N ETWORKS
However, as current and emerging Internet applications and
services become increasingly more complex and demanding, SDN has great potential to change the way networks oper-
it is imperative that the Internet be able to evolve to address ate, and OpenFlow in particular has been touted as a “radical
these new challenges. new idea in networking” [80]. The proposed benefits range
from centralized control, simplified algorithms, commoditizing
Bruno Astuto A. Nunes, Xuan-Nam Nguyen and Thierry Turletti network hardware, eliminating middleboxes, to enabling the
are with INRIA, France, {bruno.astuto-arouche-nunes, xuan-nam.nguyen, design and deployment of third-party ‘apps’.
thierry.turletti}@inria.fr
Marc Mendonca and Katia Obraczka are with UC Santa Cruz, {msm, While OpenFlow has received considerable attention from
katia}@soe.ucsc.edu industry, it is worth noting that the idea of programmable
IN SUBMISSION 2

networks and decoupled control logic has been around for switch between those controllers.
many years. In this section, we provide an overview of early d) 4D Project: Starting in 2004, the 4D project [105],
programmable networking efforts, precursors to the current [53], [31] advocated a clean slate design that emphasized
SDN paradigm that laid the foundation for many of the ideas separation between the routing decision logic and the pro-
we are seeing today. tocols governing the interaction between network elements.
a) Open Signaling: The Open Signaling (OPENSIG) It proposed giving the “decision” plane a global view of the
working group began in 1995 with a series of workshops network, serviced by a “dissemination” and “discovery” plane,
dedicated to “making ATM, Internet and mobile networks for control of a “data” plane for forwarding traffic. These ideas
more open, extensible, and programmable” [34]. They believed provided direct inspiration for later works such as NOX [54],
that a separation between the communication hardware and which proposed an “operating system for networks” in the
control software was necessary but challenging to realize; this context of an OpenFlow-enabled network.
is mainly due to vertically integrated switches and routers, e) NETCONF: In 2006, the IETF Network Configu-
whose closed nature made the rapid deployment of new ration Working Group proposed NETCONF [46] as a man-
network services and environments impossible. The core of agement protocol for modifying the configuration of network
their proposal was to provide access to the network hardware devices. The protocol allowed network devices to expose an
via open, programmable network interfaces; this would allow API through which extensible configuration data could be sent
the deployment of new services through a distributed program- and retrieved.
ming environment. Another management protocol, widely deployed in the past
Motivated by these ideas, an IETF working group was and used until today, is the SNMP [38]. SNMP was proposed
created, which led to the specification of the General Switch in the late 80’s and proved to be a very popular network
Management Protocol (GSMP) [43], a general purpose pro- management protocol, which uses the Structured Management
tocol to control a label switch. GSMP allows a controller Interface (SMI) to fetch data contained in the Management
to establish and release connections across the switch, add Information Base (MIB). It could be used as well to change
and delete leaves on a multicast connection, manage switch variables in the MIB in order to modify configuration settings.
ports, request configuration information, request and delete It later became apparent that in spite of what it was originally
reservation of switch resources, and request statistics. The intended for, SNMP was not being used to configure network
working group is officially concluded and the latest standards equipment, but rather as a performance and fault monitoring
proposal, GSMPv3, was published in June 2002. tool. Moreover, multiple shortcomings were detected in the
b) Active Networking: Also in the mid 1990s, the conception of SNMP, the most notable of which was its lack
Active Networking [115], [116] initiative proposed the idea of strong security. This was addressed in a later version of the
of a network infrastructure that would be programmable for protocol.
customized services. There were two main approaches being NETCONF, at the time it was proposed by IETF, was
considered, namely: (1) user-programmable switches, with in- seen by many as a new approach for network management
band data transfer and out-of-band management channels; that would fix the aforementioned shortcomings in SNMP.
and (2) capsules, which were program fragments that could Although the NETCONF protocol accomplishes the goal of
be carried in user messages; program fragments would then simplifying device (re)configuration and acts as a building
be interpreted and executed by routers. Despite considerable block for management, there is no separation between data
activity it motivated, Active Networking never gathered crit- and control planes. The same can be stated about SNMP.
ical mass and transferred to widespread use and industry A network with NETCONF should not be regarded as fully
deployment, mainly due to practical security and performance programmable as any new functionality would have to be
concerns [90]. implemented at both the network device and the manager so
c) DCAN: Another initiative that took place in the mid that any new functionality can be provided; furthermore, it is
1990s is the Devolved Control of ATM Networks (DCAN) [4]. designed primarily to aid automated configuration and not for
The aim of this project was to design and develop the enabling direct control of state nor enabling quick deployment
necessary infrastructure for scalable control and management of innovative services and applications. Nevertheless, both
of ATM networks. The premise is that control and management NETCONF and SNMP are useful management tools that
functions of the many devices (ATM switches in the case may be used in parallel on hybrid switches supporting other
of DCAN) should be decoupled from the devices themselves solutions that enable programmable networking.
and delegated to external entities dedicated to that purpose, The NETCONF working group is currently active and the
which is basically the concept behind SDNs. DCAN assumes latest proposed standard was published in June 2011.
a minimalist protocol between the manager and the network, f) Ethane: The immediate predecessor to OpenFlow was
in the lines of what happens today in proposals such as the SANE / Ethane project [36], which, in 2006, defined
OpenFlow. More on the DCAN project can be found at [87]. a new architecture for enterprise networks. Ethane’s focus
Still in the lines of SDNs and the proposed decoupling of was on using a centralized controller to manage policy and
control and data plane over ATM networks, amongst others, security in a network. A notable example is providing identity-
in the work proposed in [119] multiple heterogeneous control based access control. Similar to SDN, Ethane employed two
architectures are allowed to run simultaneously over single components: a controller to decide if a packet should be
physical ATM network by partitioning the resources of that forwarded, and an Ethane switch consisting of a flow table
IN SUBMISSION 3

and a secure channel to the controller.

Ethane laid the foundation for what would become
Software-Defined Networking. To put Ethane in the context of
today’s SDN paradigm, Ethane’s identity-based access control
would likely be implemented as an application on top of an
SDN controller such as NOX [54], Maestro [32], Beacon [1],
SNAC [20], Helios [6], etc.

III. S OFTWARE -D EFINED N ETWORKING

A RCHITECTURE
Data communication networks typically consist of end-
user devices, or hosts interconnected by the network infras-
tructure. This infrastructure is shared by hosts and employs
switching elements such as routers and switches as well as
communication links to carry data between hosts. Routers and
switches are usually “closed” systems, often with limited-
and vendor-specific control interfaces. Therefore, once de- Fig. 1. The SDN architecture decouples control logic from the forwarding
ployed and in production, it is quite difficult for current hardware, and enables the consolidation of middleboxes, simpler policy
network infrastructure to evolve; in other words, deploying management, and new functionalities. The solid lines define the data-plane
links and the dashed lines the control-plane links.
new versions of existing protocols (e.g., IPv6), not to mention
deploying completely new protocols and services is an almost
insurmountable obstacle in current networks. The Internet,
being a network of networks, is no exception. technically very different in terms of design, architecture,
As mentioned previously, the so-called Internet “ossifica- forwarding model, and protocol interface.
tion” [85] is largely attributed to the tight coupling between 1) ForCES: The approach proposed by the IETF ForCES
the data– and control planes which means that decisions about (Forwarding and Control Element Separation) Working Group,
data flowing through the network are made on-board each redefines the network device’s internal architecture having
network element. In this type of environment, the deployment the control element separated from the forwarding element.
of new network applications or functionality is decidedly non- However, the network device is still represented as a single
trivial, as they would need to be implemented directly into entity. The driving use case provided by the working group
the infrastructure. Even straightforward tasks such as config- considers the desire to combine new forwarding hardware with
uration or policy enforcement may require a good amount third-party control within a single network device. Thus, the
of effort due to the lack of a common control interface to control and data planes are kept within close proximity (e.g.,
the various network devices. Alternatively, workarounds such same box or room). In contrast, the control plane is ripped
as using “middleboxes” (e.g., firewalls, Intrusion Detection entirely from the network device in “OpenFlow-like” SDN
Systems, Network Address Translators, etc.) overlayed atop systems.
the underlying network infrastructure have been proposed and ForCES defines two logic entities called the Forwarding
deployed as a way to circumvent the network ossification Element (FE) and the Control Element (CE), both of which
effect. Content Delivery Networks (CDNs) [98] are a good implement the ForCES protocol to communicate. The FE
example. is responsible for using the underlying hardware to provide
Software-Defined Networking was developed to facilitate per-packet handling. The CE executes control and signaling
innovation and enable simple programmatic control of the functions and employs the ForCES protocol to instruct FEs on
network data-path. As visualized in Figure 1, the separation of how to handle packets. The protocol works based on a master-
the forwarding hardware from the control logic allows easier slave model, where FEs are slaves and CEs are masters.
deployment of new protocols and applications, straightforward An important building block of the ForCES architecture is
network visualization and management, and consolidation of the LFB (Logical Function Block). The LFB is a well-defined
various middleboxes into software control. Instead of enforc- functional block residing on the FEs that is controlled by CEs
ing policies and running protocols on a convolution of scat- via the ForCES protocol. The LFB enables the CEs to control
tered devices, the network is reduced to “simple” forwarding the FEs’ configuration and how FEs process packets.
hardware and the decision-making network controller(s).
ForCES has been undergoing standardization since 2003,
and the working group has published a variety of documents
A. Current SDN Architectures including: an applicability statement, an architectural frame-
In this section, we review two well-known SDN architec- work defining the entities and their interactions, a modeling
tures, namely ForCES [44] and Openflow [85]. Both Open- language defining the logical functions within a forwarding
Flow and ForCES follow the basic SDN principle of separation element, and the protocol for communication between the
between the control and data planes; and both standardize control and forwarding elements within a network element.
information exchange between planes. However, they are The working group is currently active.
IN SUBMISSION 4

can be exchanged between these entities over a secure channel.

CONTROLLER
Using the OpenFlow protocol a remote controller can, for
example, add, update, or delete flow entries from the switch’s
OpenFlow Protocol
flow tables. That can happen reactively (in response to a packet
arrival) or proactively.
OPENFLOW CLIENT
3) Discussion: In [126], the similarities and differences
OPENFLOW between ForCES and OpenFlow are discussed. Among the
FLOW TABLE SWITCH
Packet/byte counter,
RULE ACTIONSflow duration STATISTICS differences, they highlight the fact that the forwarding model
used by ForCES relies on the Logical Function Blocks (LFBs),
PORT PORT PORT while OpenFlow uses flow tables. They point out that in
1 2 N
OpenFlow actions associated with a flow can be combined
to provide greater control and flexibility for the purposes
Forward to port(s)
IP src/dst , MAC src/dst, Forward to the controller
Packets, Bytes, Duration
of network management, administration, and development. In
Transport Src/Dst, VLAN ... Modify header fields
Drop ForCES the combination of different LFBs can also be used
to achieve the same goal.
Fig. 2. Communication between the controller and the forwarding devices We should also re-iterate that ForCES does not follow the
happens via OpenFlow protocol. The flow tables are composed by matching same SDN model underpinning OpenFlow, but can be used
rules, actions to be taken when the flow matches the rules, and counters for
collecting flow statistics. to achieve the same goals and implement similar functional-
ity [126].
The strong support from industry, research, and academia
2) OpenFlow: Driven by the SDN principle of decoupling
that the Open Networking Foundation (ONF) and its SDN
the control and data forwarding planes, OpenFlow [85], like
proposal, OpenFlow, has been able to gather is quite impres-
ForCES, standardizes information exchange between the two
sive. The resulting critical mass from these different sectors
planes.
has produced a significant number of deliverables in the form
In the OpenFlow architecture, illustrated in Figure 2, the
of research papers, reference software implementations, and
forwarding device, or OpenFlow switch, contains one or more
even hardware. So much so that some argue that OpenFlow’s
flow tables and an abstraction layer that securely communi-
SDN architecture is the current SDN de-facto standard. In
cates with a controller via OpenFlow protocol. Flow tables
line with this trend, the remainder of this section focuses on
consist of flow entries, each of which determines how packets
OpenFlow’s SDN model. More specifically, we will describe
belonging to a flow will be processed and forwarded. Flow
the different components of the SDN architecture, namely:
entries typically consist of: (1) match fields, or matching
the switch, the controller, and the interfaces present on the
rules, used to match incoming packets; match fields may
controller for communication with forwarding devices (south-
contain information found in the packet header, ingress port,
bound communication) and network applications (northbound
and metadata; (2) counters, used to collect statistics for the
communication). Section IV also has an OpenFlow focus as it
particular flow, such as number of received packets, number
describes existing platforms for SDN development and testing,
of bytes and duration of the flow; and (3) a set of instructions,
including emulation and simulation tools, SDN controller im-
or actions, to be applied upon a match; they dictate how to
plementations, as well as verification and debugging tools. Our
handle matching packets.
discussion of future SDN applications and research directions
Upon a packet arrival at an OpenFlow switch, packet header is more general and is SDN architecture agnostic.
fields are extracted and matched against the matching fields
portion of the flow table entries. If a matching entry is
found, the switch applies the appropriate set of instructions, B. Forwarding Devices
or actions, associated with the matched flow entry. If the flow
table look-up procedure does not result on a match, the action The underlaying network infrastructure may involve a num-
taken by the switch will depend on the instructions defined ber of different physical network equipment, or forwarding
by the table-miss flow entry. Every flow table must contain a devices such as routers, switches, virtual switches, wireless
table-miss entry in order to handle table misses. This particular access points, to name a few. In a software-defined network,
entry specifies a set of actions to be performed when no such devices are often represented as basic forwarding hard-
match is found for an incoming packet, such as dropping the ware accessible via an open interface at an abstraction layer, as
packet, continue the matching process on the next flow table, the control logic and algorithms are off-loaded to a controller.
or forward the packet to the controller over the OpenFlow Such forwarding devices are commonly referred to, in SDN
channel. It is worth noting that from version 1.1 OpenFlow terminology, simply as “switches”, as illustrated in Figure 3.
supports multiple tables and pipeline processing. Another In an OpenFlow network, switches come in two vari-
possibility, in the case of hybrid switches, i.e., switches that eties: pure and hybrid. Pure OpenFlow switches have no
have both OpenFlow– and non-OpenFlow ports, is to forward legacy features or on-board control, and completely rely on a
non-matching packets using regular IP forwarding schemes. controller for forwarding decisions. Hybrid switches support
The communication between controller and switch happens OpenFlow in addition to traditional operation and protocols.
via OpenFlow protocol, which defines a set of messages that Most commercial switches available today are hybrids.
IN SUBMISSION 5

1) Processing Forwarding Rules: Flow-based SDN archi-

Applications
tectures such as OpenFlow may utilize additional forwarding
table entries, buffer space, and statistical counters that are
difficult to implement in traditional ASIC switches. Some Network OS
recent proposals [82], [88] have advocated adding a general-
Decoupled
purpose CPU, either on-switch or nearby, that may be used
Control Logic
to supplement or take over certain functions and reduce the
complexity of the ASIC design. This would have the added
benefit of allowing greater flexibility for on-switch processing Secure
as some aspects would be software-defined. Channel

In [83], network processor based acceleration cards were

Abstraction Layer
used to perform OpenFlow switching. They proposed and
described the design options and reported results that showed
a 20% reduction on packet delay. In [114], an architectural de-
sign to improve look-up performance of OpenFlow switching
in Linux was proposed. Preliminary results reported showed Flow Table
a packet switching throughput increase of up to 25% com-
pared to the throughput of regular software-based OpenFlow
switching. Another study on data-plane performance over SWITCH
Linux based Openflow switching was presented in [27], which
compared OpenFlow switching, layer-2 Ethernet switching
and layer-3 IP routing performance. Fairness, forwarding Fig. 3. The separated control logic can be viewed as a network operating
system, upon which applications can be built to “program” the network.
throughput and packet latency in diverse load conditions were
analyzed. In [69], a basic model for the forwarding speed
and blocking probability of an OpenFlow switch was derived, prioritized rules dictating, for example, access control and
while the parameters for the model were drawn from mea- load balancing, while viewing the whole network as a single
surements of switching times of current OpenFlow hardware, virtual switch. Routing policies, on the other hand, dictate
combined with an OpenFlow controller. through what paths traffic should flow in the network. The
2) Installing Forwarding Rules: Another issue regarding main idea in Palette is to partition end-to-end policies into
the scalability of an OpenFlow network is memory limitation sub tables and then distribute them over the switches. Their
in forwarding devices. OpenFlow rules are more complex algorithm consists of two steps: determine the number k of
than forwarding rules in traditional IP routers. They support tables needed and then partition the rules set over k tables.
more flexible matchings and matching fields and also different One Big Switch, on the other hand, solves the rule placement
actions to be taken upon packet arrival. A commodity switch problem separately for each path, choosing the paths based on
normally supports between a few thousand up to tens of network metrics (e.g. latency, congestion and bandwidth), and
thousands forwarding rules [110]. Also, Ternary Content- then combining the result to reach a global solution.
Addressable Memory (TCAM) has been used to support
forwarding rules, which can be expensive and power-hungry. C. The Controller
Therefore, the rule space is a bottleneck to the scalability of The decoupled system has been compared to an operating
OpenFlow, and the optimal use of the rule space to serve system [54], in which the controller provides a programmatic
a scaling number of flow entries while respecting network interface to the network. That can be used to implement
policies and constraints is a challenging and important topic. management tasks and offer new functionalities. A layered
Some proposals address memory limitations in OpenFlow view of this model is illustrated in Figure 3. This abstraction
switches. Devoflow [40] is an extension to OpenFlow for high- assumes the control is centralized and applications are written
performance networks. It handles mice flows (i.e. short flows) as if the network is a single system. It enables the SDN
at the OpenFlow switch and only invokes the controller in model to be applied over a wide range of applications and
order to handle elephant flows (i.e larger flows). The per- heterogeneous network technologies and physical media such
formance evaluation conducted in [40] showed that Devoflow as wireless (e.g. 802.11 and 802.16), wired (e.g. Ethernet) and
uses 10 to 53 times less flow table space. In DIFANE [132], optical networks.
“ingress” switches redirect packets to “authority” switches that As a practical example of the layering abstraction accessible
store all the forwarding rules while ingress switches cache through open application programming interfaces (APIs), Fig-
flow table rules for future use. The controller is responsible ure 4 illustrates the architecture of an SDN controller based on
for partitioning rules over authority switches. the OpenFlow protocol. This specific controller is a fork of the
Palette [71] and One Big Switch [70] address the rule Beacon controller [1] called Floodlight [5]. In this figure it is
placement problem. Their goal is to minimize the number possible to observe the separation between the controller and
of rules that need to be installed in forwarding devices and the application layers. Applications can be written in Java and
use end-to-end policies and routing policies as input to a rule can interact with the built-in controller modules via a JAVA
placement optimizer. End-to-end policies consist of a set of API. Other applications can be written in different languages
IN SUBMISSION 6

a single controller is able to handle a surprising number of new

Learning PortDown OpenStack flow requests, and should be able to manage all but the largest
Switch Reconciliation Quantum Plugin networks. Furthermore, new controllers under development
such as McNettle [123] target powerful multicore servers and
Firewall VNF Hub Circuit Pusher are being designed to scale up to large data center workloads
(around 20 million flows requests per second and up to 5000
switches). Nonetheless, multiple controllers may be used to
JAVA API reduce latency or increase fault tolerance.
A related concern is the controller placement problem [60],
which attempts to determine both the optimal number of
controllers and their location within the network topology,
Module Thread Packet Jython Unit often choosing between optimizing for average and worst
Web UI
Manager Pool Streamer Server Tests
case latency. The latency of the link used for communication
between controller and switch is of great importance when
Topology dimensioning a network or evaluating its performance [40].
Device Link Flow
Manager/ Storage Memory
Manager
Routing
Discovery Cache That was one of the main motivations behind the work in [100]
which evaluated how the controller and the network perform
with bandwidth and latency issues on the control link. This
Controller Counter
Switches PerfMon Trace Store work concludes that bandwidth in the control link arbitrates
Memory
how many flows can be processed by the controller, as well
OpenFlow Services as the loss rate when under saturation conditions. The switch-
to-control latency on the other hand, has a major impact on
Fig. 4. The Floodlight architecture as an example of an OpenFlow controller.
the overall behavior of the network, as each switch cannot
forward data until it receives the message from the controller
that inserts the appropriate rules in the flow table. This interval
and interact with the controller modules via the REST API. can grow with the link latency and impact dramatically the
This particular example of an SDN controller allows the performance of network applications.
implementation of built-in modules that can communicate Also, control modeling greatly impacts the network scala-
with their implementation of the OpenFlow controller (i.e. bility. Some important scalability issues are presented in [130],
OpenFlow Services). The controller, on the other hand, can along with a discussion about scalability trade-offs in software-
communicate with the forwarding devices via the OpenFlow defined network design.
protocol through the abstraction layer present at the forwarding 2) Control models: In the following, we go over some of
hardware, illustrated in Figure 3. these SDN design options and discuss different methods of
While the aforementioned layering abstractions accessible controlling a software-defined network, many of which are
via open APIs allow the simplification of policy enforce- interrelated:
ment and management tasks, the bindings must be closely • Centralized vs. Distributed
maintained between the control and the network forwarding Although protocols such as OpenFlow specify that a
elements. The choices made while implementing such layering switch is controlled by a controller and therefore ap-
architectures can dramatically influence the performance and pears to imply centralization, software-defined networks
scalability of the network. In the following, we address some may have either a centralized or distributed control-
such scalability concerns and go over some proposals that aim plane. Though controller-to-controller communication is
on overcoming these challenges. We leave a more detailed not defined by OpenFlow, it is necessary for any type of
discussion on the application layer and the implementation of distribution or redundancy in the control-plane.
services and policy enforcement to Section VI-C. A physically centralized controller represents a single
1) Control Scalability: An initial concern that arises when point of failure for the entire network; therefore, Open-
offloading control from the switching hardware is the scalabil- Flow allows the connection of multiple controllers to a
ity and performance of the network controller(s). The original switch, which would allow backup controllers to take over
Ethane [36] controller, hosted on a commodity desktop ma- in the event of a failure.
chine, was tested to handle up to 11,000 new flow requests per Onix [76] and HyperFlow [117] take the idea further
second and responded within 1.5 milliseconds. A more recent by attempting to maintain a logically centralized but
study [118] of several OpenFlow controller implementations physically distributed control plane. This decreases the
(NOX-MT, Maestro, Beacon), conducted on a larger emulated look-up overhead by enabling communication with local
network with 100,000 endpoints and up to 256 switches, found controllers, while still allowing applications to be written
that all were able to handle at least 50,000 new flow requests with a simplified central view of the network. The poten-
per second in each of the tested scenarios. On an eight- tial downside are trade-offs [78] related to consistency
core machine, the multi-threaded NOX-MT implementation and staleness when distributing state throughout the con-
handled 1.6 million new flow requests per second with an trol plane, which has the potential to cause applications
average response time of 2 milliseconds. As the results show, that believe they have an accurate view of the network to
IN SUBMISSION 7

act incorrectly.
A hybrid approach, such as Kandoo [58], can utilize local
controllers for local applications and redirect to a global
controller for decisions that require centralized network
state. This reduces the load on the global controller by
filtering the number of new flow requests, while also
providing the data-path with faster responses for requests
that can be handled by a local control application.
A software-defined network can also have some level of
logical decentralization, with multiple logical controllers.
An interesting type of proxy controller, called Flowvi-
sor [107], can be used to add a level of network virtualiza-
tion to OpenFlow networks and allow multiple controllers
to simultaneously control overlapping sets of physical
switches. Initially developed to allow experimental re-
search to be conducted on deployed networks alongside
production traffic, it also facilitates and demonstrates the
ease of deploying new services in SDN environments.
A logically decentralized control plane would be needed
in an inter-network spanning multiple administrative do-
mains. Though the domains may not agree to centralized
control, a certain level of sharing may be appropriate Fig. 5. A controller with a northbound and southbound interface.
(e.g., to ensure service level agreements are met for traffic
flowing between domains).
• Control Granularity
Traditionally, the basic unit of networking has been
in many cases, it may be a concern if the controller is
the packet. Each packet contains address information
geographically remote (though this can be mitigated by
necessary for a network switch to make routing decisions.
physically distributing the controller [117]) or if most
However, most applications send data as a flow of many
flows are short-lived, such as single-packet flows. There
individual packets. A network that wishes to provide
are also some scalability issues in larger networks, as the
QoS or service guarantees to certain applications may
controller must be able to handle a larger volume of new
benefit from individual flow-based control. Control can
flow requests.
be further abstracted to an aggregated flow-match, rather
Alternatively, proactive control approaches push policy
than individual flows. Flow aggregation may be based
rules from the controller to the switches. A good example
on source, destination, application, or any combination
of proactive control is DIFANE [132], which partitions
thereof.
rules over a hierarchy of switches, such that the controller
In a software-defined network where network elements
rarely needs to be consulted about new flows and traffic is
are controlled remotely, overhead is caused by traffic
kept within the data-plane. In their experiments, DIFANE
between the data-plane and control-plane. As such, using
reduces first-packet delay from a 10ms average round-trip
packet level granularity would incur additional delay as
time (RTT) with a centralized NOX controller to a 0.4ms
the controller would have to make a decision for each
average RTT for new single-packet flows. It was also
arriving packet. When controlling individual flows, the
shown to increase the new flow throughput, as the tested
decision made for the first packet of the flow can be ap-
version of NOX achieved a peak of 50,000 single-packet
plied to all subsequent packets of that flow. The overhead
flows per second while the DIFANE solution achieved
may be further reduced by grouping flows together, such
800,000 single-packet flows per second. Interestingly, it
as all traffic between two hosts, and performing control
was observed that the OpenFlow switch’s local controller
decisions on the aggregated flows.
implementation becomes a bottleneck before the central
• Reactive vs. Proactive Policies
NOX controller. This was attributed to the fact that com-
Under a reactive control model, such as the one proposed
mercial OpenFlow switch implementations were limited
by Ethane [36], forwarding elements must consult a con-
to sending 60-330 new flows requests per second at the
troller each time a decision must be made, such as when
time of their publication (2010).
a packet from a new flow reaches a switch. In the case
of flow-based control granularity, there will be a small
performance delay as the first packet of each new flow As shown in Figure 5, a controller that acts as a network
is forwarded to the controller for decision (e.g., forward operating system must implement at least two interfaces: a
or drop), after which future packets within that flow will “southbound” interface that allows switches to communicate
travel at line rate within the forwarding hardware. While with the controller and a “northbound” interface that presents
the delay incurred by the first-packet may be negligible an API to network control and high-level applications/services.
IN SUBMISSION 8

D. Southbound Communication: Controller-Switch requirements and architecture discussions on SDN signaling.

An important aspect of SDNs is the link between the The Software-Defined Networking Research Group (SDNRG)
data-plane and the control-plane. As forwarding elements are at IRTF [63] is also focusing on SDN under various perspec-
controlled by an open interface, it is important that this link tives with the goal of identifying new approaches that can be
remains available and secure. defined and deployed, as well as identifying future research
The OpenFlow protocol can be viewed as one possible im- challenges. Some of their main areas of interest include
plementation of controller-switch interactions, as it defines the solution scalability, abstractions, security and programming
communication between the switching hardware and a network languages and paradigms particularly useful in the context of
controller. For security, OpenFlow 1.3.0 provides optional SDN.
support for encrypted TLS communication and a certificate These and other working groups perform important work,
exchange between the switches and the controller(s); however, coordinating efforts to evolve existing standards and proposing
the exact implementation and certificate format is not currently new ones. The goal is to facilitate smooth transitions from
specified. Also outside the scope of the current specification legacy networking technology to the new protocols and archi-
are fine-grained security options regarding scenarios with tectures, such as SDN Some of these groups, such as ITU-T’s
multiple controllers, as there is no method specified to only SG13, advocate the establishment of a Joint Coordination Ac-
grant partial access permissions to an authorized controller. tivity on SDN (JCA-SDN) for collaboration and coordination
We examine OpenFlow controller implementation options in between standardizing efforts and also taking advantage of the
greater detail in Section IV. work performed by the Open Source Software (OSS) commu-
nity, such as OpenStack [66] and OpenDayLight [65] as they
start developing the building blocks for SDN implementation.
E. Northbound Communication: Controller-Service
External management systems or network services may IV. SDN D EVELOPMENT T OOLS
wish to extract information about the underlying network or
SDN has been proposed to facilitate network evolution and
control an aspect of network behavior or policy. Additionally,
innovation by allowing rapid deployment of new services and
controllers may find it necessary to communicate with each
protocols. In this section, we provide an overview of currently
other for a variety of reasons. For example, an internal control
available tools and environments for developing SDN-based
application may need to reserve resources across multiple
services and protocols.
domains of control or a “primary” controller may need to share
policy information with a backup, etc.
Unlike controller-switch communication, there is no cur- A. Emulation and Simulation Tools
rently accepted standard for northbound interactions and they Mininet [77] allows an entire OpenFlow network to be
are more likely to be implemented on an ad hoc basis for emulated on a single machine, simplifying the initial develop-
particular applications. We discuss this further in Section VI. ment and deployment process. New services, applications and
protocols can first be developed and tested on an emulation of
F. Standardization Efforts the anticipated deployment environment before moving to the
actual hardware. By default Mininet supports OpenFlow v1.0,
Recently, several standardization organizations have been
though it may be modified to support a software switch that
turning the spotlights towards SDN. For example, as previ-
implements a newer release.
ously mentioned, the IETF’s Forwarding and Control Element
The ns-3 [61] network simulator supports OpenFlow
Separation (ForCES) Working Group [44] has been working
switches within its environment, though the current version
on standardizing mechanisms, interfaces, and protocols aim-
only implements OpenFlow v0.89.
ing at the centralization of network control and abstraction
of network infrastructure. The Open Network Foundation
(ONF) [13] has been trying to standardize the OpenFlow B. Available Software Switch Platforms
protocol. As the control plane abstracts network applications There are currently several SDN software switches available
from underlying hardware infrastructure, they focus on stan- that can be used, for example, to run an SDN testbed or when
dardizing the interfaces between: (1) network applications and developing services over SDN. Table I presents a list of current
the controller (i.e. northbound interface) and (2) the controller software switch implementations with a brief description in-
and the switching infrastructure (i.e., southbound interface) cluding implementation language and the OpenFlow standard
which defines the OpenFlow protocol itself. Some of the Study version that the current implementation supports.
Groups (SGs) of ITU’s Telecommunication Standardization
Sector (ITU-T) [64] are currently working towards discussing
requirements and creating recommendations for SDNs under C. Native SDN Switches
different perspectives. For instance, the SG13 focuses on One of the main SDN enabling technologies currently being
Future Networks, including cloud computing, mobile and implemented in commodity networking hardware is the Open-
next generation networks, and is establishing requirements for Flow standard. In this section we do not intend to present a
network virtualization. Other ITU-T SGs such as the SG11 detailed overview of OpenFlow enabled hardware and makers,
for protocols and test specifications started, in early 2013, but rather provide a list of native SDN switches currently
IN SUBMISSION 9

Software Switch Implementation Overview Version

Open vSwitch [15] C/Python Open source software switch that aims to implement a switch platform v1.0
in virtualized server environments. Supports standard management
interfaces and enables programmatic extension and control of the
forwarding functions. Can be ported into ASIC switches.
Pantou/OpenWRT [16] C Turns a commercial wireless router or Access Point into an OpenFlow-enabled switch. v1.0
ofsoftswitch13 [12] C/C++ OpenFlow 1.3 compatible user-space software switch implementation. v1.3
Indigo [7] C Open source OpenFlow implementation that runs on physical switches and uses v1.0
the hardware features of Ethernet switch ASICs to run OpenFlow.
TABLE I
C URRENT SOFTWARE SWITCH IMPLEMENTATIONS COMPLIANT WITH THE O PEN F LOW STANDARD .

available in the market and provide some information about • Floodlight [5] is a Java-based OpenFlow controller, based
them, including the version of OpenFlow they implement. on the Beacon implementation, that works with physical-
One clear evidence of industry’s strong commitment to SDN and virtual- OpenFlow switches.
is the availability of commodity network hardware that are • SNAC [20] is an OpenFlow controller based on NOX-0.4,
OpenFlow enabled. Table II lists commercial switches that which uses a web-based, user-friendly policy manager
are currently available, their manufacturer, and the version of to manage the network, configure devices, and monitor
OpenFlow they implement. events.
• Ryu [18] is an SDN operating system that aims to
Maker Switch Model Version
Hewlett-Packard 8200zl, 6600, 6200zl, v1.0 provide logically centralized control and APIs to create
5400zl, and 3500/3500yl new network management and control applications. Ryu
Brocade NetIron CES 2000 Series v1.0
IBM RackSwitch G8264 v1.0 fully supports OpenFlow v1.0, v1.2, v1.3, and the Nicira
NEC PF5240 PF5820 v1.0 Extensions.
Pronto 3290 and 3780 v1.0
Juniper Junos MX-Series v1.0 • NodeFlow [10] is an OpenFlow controller written in
Pica8 P-3290, P-3295, P-3780 and P-3920 v1.2 JavaScript for Node.JS [11].
TABLE II • ovs-controller [15] is a simple OpenFlow controller ref-
M AIN CURRENT AVAILABLE COMMODITY SWITCHES BY MAKERS , erence implementation with Open vSwitch for managing
COMPLIANT WITH THE O PEN F LOW STANDARD .
any number of remote switches through the OpenFlow
protocol; as a result the switches function as L2 MAC-
learning switches or hubs.
D. Available Controller Platforms Included in Table III are also two special purpose controller
Table III shows a snapshot of current controller imple- implementations: Flowvisor [107], mentioned previously, and
mentations. To date, all the controllers in the table support RouteFlow [93]. The former acts as a transparent proxy be-
the OpenFlow protocol version 1.0, unless stated otherwise. tween OpenFlow switches and multiple OpenFlow controllers.
Below, we provide a brief overview of the controllers listed It is able to create network slices and can delegate control of
in Table III. each slice to a different controller, also promoting isolation
• POX [17] is a general, open-source SDN controller between slices. RouteFlow, on the other hand, is an open
written in Python. source project to provide virtualized IP routing over OpenFlow
• NOX [54] was the first OpenFlow controller written in capable hardware. It is composed of an OpenFlow Controller
Python and C++. application, an independent server, and a virtual network
• MUL [9] is an OpenFlow controller that has a C-based environment that reproduces the connectivity of a physical
multi-threaded infrastructure at its core. It supports a infrastructure and runs IP routing engines. The routing engines
multi-level north-bound interface (see Section III-E) for generate the forwarding information base (FIB) into the Linux
application development. IP tables according to the routing protocols configured (e.g.,
• Maestro [32] is a network operating system based on OSPF, BGP). An extension of RouteFlow is presented in [106],
Java; it provides interfaces for implementing modular which discusses Routing Control Platforms (RCPs) in the
network control applications and for them to access and context of OpenFlow/SDN. They proposed a controller-centric
modify network state. networking model along with a prototype implementation of
• Trema is a framework for developing OpenFlow con- an autonomous-system-wide abstract BGP routing service.
trollers written in Ruby and C.
• Beacon is a cross-platform, modular, Java-based Open-
Flow controller that supports event-based and threaded E. Code Verification and Debugging
operation. Verification and debugging tools are vital resources for
• Jaxon [8] is a Java-based OpenFlow controller based on traditional software development and are no less important for
NOX. SDN. Indeed, for the idea of portable network “apps” to be
• Helios [6] is an extensible C-based OpenFlow controller successful, network behavior must be thoroughly tested and
that provides a programmatic shell for performing inte- verified.
grated experiments. NICE [35] is an automated testing tool used to help uncover
IN SUBMISSION 10

Controller Implementation Open Source Developer

POX [17] Python Yes Nicira
NOX [54] Python/C++ Yes Nicira
MUL [9] C Yes Kulcloud
Maestro [32] Java Yes Rice University
Trema [21] Ruby/C Yes NEC
Beacon [1] Java Yes Stanford
Jaxon [8] Java Yes Independent Developers
Helios [6] C No NEC
Floodlight [5] Java Yes BigSwitch
SNAC [20] C++ No Nicira
Ryu [18] Python Yes NTT, OSRG group
NodeFlow [10] JavaScript Yes Independent Developers
ovs-controller [15] C Yes Independent Developers
Flowvisor [107] C Yes Stanford/Nicira
RouteFlow [93] C++ Yes CPQD
TABLE III
C URRENT CONTROLLER IMPLEMENTATIONS COMPLIANT WITH THE O PEN F LOW STANDARD .

bugs in OpenFlow programs through model checking and enterprise environments can have very different requirements,
symbolic execution. characteristics, and user population, For example, University
Anteater [84] takes a different approach by attempting to networks can be considered a special case of enterprise
check network invariants that exist in the data plane, such as networks: in such an environment, many of the connecting
connectivity or consistency. The main benefit of this approach devices are temporary and not controlled by the University,
is that it is protocol-agnostic; it will also catch errors that further challenging security and resource allocation. Addi-
result from faulty switch firmware or inconsistencies with the tionally, Universities must often provide support for research
control plane communication. VeriFlow [73] has a similar testbeds and experimental protocols.
goal, but goes further by proposing a real-time verification Adequate management is critically important in Enterprise
tool that resides between the controller and the forwarding environments, and SDN can be used to programmatically
elements. This adds the potential benefit of being able to halt enforce and adjust network policies as well as help monitor
bad rules that will cause anomalous behavior before they reach network activity and tune network performance.
the network. Additionally, SDN can be used to simplify the network by
Other efforts proposed debugging tools that provide insights ridding it from middleboxes and integrating their function-
gleaned from control plane traffic. OFRewind [127] allows ality within the network controller. Some notable examples
network events (control and data) to be recorded at different of middlebox functionality that has been implemented using
granularities and later replayed to reproduce a specific sce- SDN include NAT, firewalls, load balancers [57] [124], and
nario, granting the opportunity to localize and troubleshoot the network access control [94]. In the case of more complex
events that caused the network anomaly. ndb [56] implements middleboxes with functionalities that cannot be directly im-
breakpoints and packet-backtraces for SDN. Just as with the plemented without performance degradation (e.g., deep packet
popular software debugger gdb, users can pinpoint events that inspection), SDN can be used to provide unified control and
lead to error by pausing execution at a breakpoint, or, using management[51].
a packet backtrace, show the sequence of forwarding actions The work presented in [104] addresses the issues related
seen by that packet. STS [19] is a software-defined network to consistent network updates. Configuration changes are
troubleshooting simulator. It is written in python and depends a common source of instability in networks and can lead
on POX. It simulates the devices in a given network allowing to outages, security flaws, and performance disruptions. In
for testing cases and identifying the set of inputs that generates [104], a set of high-level abstractions are proposed that allow
a given error. network administrators to update the entire network, guaran-
teeing that every packet traversing the network is processed
V. SDN A PPLICATIONS by exactly one consistent global network configuration. To
Software-defined networking has applications in a wide va- support these abstractions, several OpenFlow-based update
riety of networked environments. By decoupling the control– mechanisms were developed.
and data planes, programmable networks enable customized As discussed in earlier sections, OpenFlow evolved from
control, an opportunity to eliminate middleboxes, as well Ethane [36], a network architecture designed specifically to
as simplified development and deployment of new network address the issues faced by enterprise networks.
services and protocols. Below, we examine different envi-
ronments for which SDN solutions have been proposed or
implemented. B. Data Centers
Data centers have evolved at an amazing pace in recent
A. Enterprise Networks years, constantly attempting to meet increasingly higher and
Enterprises often run large networks, while also having strict rapidly changing demand. Careful traffic management and
security and performance requirements. Furthermore, different policy enforcement is critical when operating at such large
IN SUBMISSION 11

scales, especially when any service disruption or additional required, could not be achieved by means of a traditional
delay may lead to massive productivity and/or profit loss. Due WAN architecture. A customized solution was proposed and
to the challenges of engineering networks of this scale and an OpenFlow-based SDN architecture was built to control
complexity to dynamically adapt to application requirements, individual switches. After three years in production, B4 is
it is often the case that data centers are provisioned for peak shown to be efficient in the sense that it drives many links
demand; as a result, they run well below capacity most of the at near 100% utilization while splitting flows among multiple
time but are ready to rapidly service higher workloads. paths. Furthermore, the experience reported in the work shows
An increasingly important consideration is energy consump- that the bottleneck resulting from control-plane to data-plane
tion, which has a non-trivial cost in large-scale data centers. communication and overhead in hardware programming are
Heller et al. [59] indicates that much research has been focused important issues to be considered in future work.
on improved servers and cooling (70% of total energy) through
better hardware or software management, but the data center’s
C. Infrastructure-based Wireless Access Networks
network infrastructure (which accounts for 10-20% of the total
energy cost) still consumed 3 billion kWh in 2006. They Several efforts have focused on ubiquitous connectivity in
proposed ElasticTree, a network-wide power manager that the context of infrastructure-based wireless access networks,
utilizes SDN to find the minimum-power network subset which such as cellular and WiFi.
satisfies current traffic conditions and turns off switches that For example, the OpenRoads project [129], [128] envisions
are not needed. As a result, they show energy savings between a world in which users could freely and seamlessly move
25-62% under varying traffic conditions. One can imagine that across different wireless infrastructures which may be man-
these savings can be further increased if used in parallel with aged by various providers. They proposed the deployment
server management and virtualization; one possibility is the of an SDN-based wireless architecture that is backwards-
Honeyguide[108] approach to energy optimization which uses compatible, yet open and sharable between different service
virtual machine migration to increase the number of machines providers. They employ a testbed using OpenFlow-enabled
and switches that can be shutdown. wireless devices such as WiFi APs and WiMAX base stations
However, not all SDN solutions may be appropriate in high controlled by NOX– and Flowvisor controllers and show im-
performance networks. While simplified traffic management proved performance on handover events. Their vision provided
and visibility are useful, it must be sensibly balanced with inspiration for subsequent work [79] that attempts to address
scalability and performance overhead. Curtis et al. [40] believe specific requirements and challenges in deploying a software-
that OpenFlow excessively couples central control and com- defined cellular network.
plete visibility, when in reality only “significant” flows need Odin[112] introduces programmability in enterprise wire-
to be managed; this may lead to bottlenecks as the control- less LAN environments. In particular, it builds an access point
data communication adds delay to flow setup while switches abstraction on the controller that separates the association state
are overloaded with thousands of flow table entries. Though from the physical access point, enabling proactive mobility
aggressive use of proactive policies and wild-card rules may management and load balancing without changes to the client.
resolve that issue, it may undermine the ability of the controller At the other end of the spectrum, OpenRadio [25] focuses
to have the right granularity to effectively manage traffic on deploying a programmable wireless data plane that provides
and gather statistics. Their framework, DevoFlow, proposes flexibility at the PHY and MAC layers (as opposed to layer-3
some modest design changes to keep flows in the data plane SDN) while meeting strict performance and time deadlines.
as much as possible while maintaining enough visibility for The system is designed to provide a modular interface that is
effective flow management. This is accomplished by push- able to process traffic subsets using different protocols such
ing responsibility over most flows back to the switches and as WiFi, WiMAX, 3GPP LTE-Advanced, etc. Based on the
adding more efficient statistics collection mechanisms, through idea of separation of the decision and forwarding planes, an
which “significant” flows (e.g. long-lived, high-throughput) are operator may express decision plane rules and corresponding
identified and managed by the controller. In a load-balancing actions, which are assembled from processing plane modules
simulation, their solution had 10-53 times fewer flow table (e.g., FFT, Viterbi decoding, etc); the end result is a state
entries and 10-42 times fewer control messages on average machine that expresses a fully-functional protocol.
over OpenFlow.
A practical example of a real application of the SDN
concept and architecture in the context of data centers was D. Optical Networks
presented by Google in early 2012. The company presented at Handling data traffic as flows, allows software-defined net-
the Open Network Summit [23] a large scale implementation works, and OpenFlow networks in particular, to support and
of an SDN-based network connecting its data centers. The integrate multiple network technologies. As a result, it is pos-
work in [68] presents in more detail the design, implementa- sible to provide also technology-agnostic unified control for
tion, and evaluation of B4, a WAN connecting Google’s data- optical transport networks and facilitating interaction between
centers world wide. This work describes one of the first and both packet and circuit-switched networks. According to the
largest SDN deployments. The motivation was the need for Optical Transport Working Group (OTWG) created in 2013
customized routing and traffic engineering and the fact that the by the Open Network Foundation (ONF), the benefits from
level of scalability, fault tolerance, cost efficiency and control applying SDN and the OpenFlow standard in particular to
IN SUBMISSION 12

optical transport networks include: improving optical trans- management to third-party experts, and that this could be
port network control and management flexibility, enabling accomplished successfully through the remote control of pro-
deployment of third-party management and control systems, grammable switches and the application of distributed network
and deploying new services by leveraging virtualization and monitoring and inference algorithms used to detect possible
SDN [24]. security problems.
There has been several attempts and proposals to control In contrast, Mortier et al. [91] believe that users desire
both circuit switched and packet switched networks using the greater understanding and control over their networks’ behav-
OpenFlow protocol. In [55] a NetFPGA [95] platform is used ior; rather than following traditional policies, a home network
in the proposal of a packet switching and circuit switched may be better managed by their users who better understand
networks architectures based on Wavelength Selective Switch- the dynamics and needs of their environment. Towards this
ing (WSS), using the OpenFlow protocol. Another control goal, they created a prototype network in which SDN is used
plane architecture based on OpenFlow for enabling SDN to provide users a view into how their network is being utilized
operations in optical networks was proposed in [109], which while offering a single point of control.
discusses specific requirements and describes implementation Mehdi et al. [86] argues that an Anomaly Detection System
of OpenFlow protocol extensions to support optical transport (ADS) implemented within a programmable home network
networks. provides a more accurate identification of malicious activity
A proof-of-concept demonstration of an OpenFlow-based as compared to one deployed at the ISP; additionally, the
wavelength path control in transparent optical networks is pre- implementation would be able to operate at line rate with no
sented in [81]. In this work, virtual Ethernet interfaces (veths) performance penalty, while, at the same time, offloading the
are introduced. These veths, are mapped to physical interfaces ISP from having to monitor these large number of networks.
of an optical node (e.g. photonic cross-connect - PXC), and The ADS algorithm could operate alongside other controller
enable an SDN controller (e.g. the NOX controller in this case) services, such as a HomeOS that may react to suspicious
to operate the optical lightpaths (e.g., via the OpenFlow pro- activity and report anomalies to the ISP or local administrator.
tocol). In their experimental setup, they quantitatively evaluate
network performance metrics, such as the latency of lightpath VI. F UTURE D IRECTIONS
setup and release, and verify the feasibility of routing and As SDN becomes more widely adopted and protocols such
wavelength assignment, and the dynamic control of optical as OpenFlow are further defined, new solutions are proposed
nodes in an OpenFlow-based network composed by four PXCs and new challenges arise.
nodes in a mesh topology.
A Software Defined Optical Network (SDON) architecture
is introduced in [99] and a QoS-aware unified control protocol A. Controller and Switch Design
for optical burst switching in OpenFlow-based SDON is devel- SDN raises significant scalability, performance, robustness,
oped. The performance of the proposed protocol was evaluated and security challenges. Below we review a number of re-
with the conventional GMPLS-based distributed protocol and search efforts focusing on addressing these issues at the
the results indicate that SDON offers an infrastructure to switch– and controller design level.
support unified control protocols to better optimize network In DIFANE [132], flow entries are proactively pushed to
performance and improve capacity. switches in an attempt to reduce the number of requests to
the controller. Devoflow [40] proposes to handle “short-lived”
flows in switches and “long-lived” flows in the controller to
E. Home and Small Business mitigate flow setup delay and controller overhead. The work
Several projects have examined how SDN could be used in proposed in [88] advocates replacing counters on ASIC by
smaller networks, such as those found in the home or small a stream of rule-matching records and processing them in the
businesses. As these environments have become increasingly CPU to allow efficient access to counters. FLARE [92] is a
complex and prevalent with the widespread availability of low- new network node model focusing on “deeply programmable
cost network devices, the need for more careful network man- networks” that provides programmability for the data plane,
agement and tighter security has correspondingly increased. the control plane, as well as the interface between them.
Poorly secured networks may become unwitting targets or The work presented in [28] discusses important aspects in
hosts for malware, while outages due to network configuration controller design including hierarchical control, data model,
issues may cause frustration or lost business. Unfortunately, it scalability, and extensibility.
is not practical to have a dedicated network administrator in As an example of performance and scalability, a study
every home and office. showed that one single controller can handle up to 6 million
Calvert et al. [33] assert that the first step in managing home flows per second [3]. A more recent study [47], focusing on
networks is to know what is actually happening; as such, they the Beacon controller, showed that a controller can handle
proposed instrumenting the network gateway/controller to act 12.8 million new flows per second in a 12 cores machine,
as a “Home Network Data Recorder” to create logs that may with an average latency of 24.7 us for each flow. How-
be utilized for troubleshooting or other purposes. ever, for increased scalability and especially for reliability
Feamster [48] proposes that such networks should operate and robustness purposes, it has been recognized that the
in a “plug in and forget” fashion, namely by outsourcing logically-centralized controller must be physically distributed.
IN SUBMISSION 13

Onix [76], Kando [58], and HyperFlow [117] use this approach associated controller in each domain are involved in inter-
to achieve robust and scalable control plane. In [78], trade- domain tasks, changes to inter-domain service models would
offs related to control distribution, such as staleness versus be limited to software modifications at the inter-domain con-
optimality and application logic complexity versus robustness trollers rather than the entire infrastructure. Examples of how
to inconsistency are identified and quantified. In [60], the this architecture could be used to realize new Internet services
controller placement problem is discussed in terms of the such as information-centric networking, and middlebox service
number of controllers needed and where to place them in the sharing are explored.
network. In more recent work on distributed control, the need Another approach to inter-AS routing [26] uses NOX and
for dynamic assignment of switches to controllers is addressed OpenFlow to implement BGP-like functionality. Alternatively,
in [42], which proposes an algorithm to increase or decrease an extensible session protocol [75] supports application-driven
the pool of controllers based on controllers’ load estimates. configuration of network resources across domains.
They also propose a mechanism to dynamically handover
switches from one controller to another as needed. C. Controller-Service Interaction
In [37] an SDN variant inspired by MPLS was proposed While controller-switch (“southbound”) interaction is fairly
along with the notions of edge controllers and fabric con- well defined in protocols such as OpenFlow and ForCES,
trollers: the former control ingress and egress switches and there is no standard for interactions between controllers and
handle the host-network interface, while the latter handle network services or applications (“northbound”). One possible
fabric switches and the operator-network interface. explanation is that the northbound interface is defined entirely
Although control and measurement are two important com- in software, while controller-switch interactions must enable
ponents of network management, little thought has gone into hardware implementation.
designing APIs for measurement. The work presented in [131] If we think of the controller as a “network operating
proposes a software-defined traffic measurement architecture, system”, then there should be a clearly defined interface
which separates the measurement data plane from the control by which applications can access the underlying hardware
plane. (switches), co-exist and interact with other applications, and
utilize system services (e.g. topology discovery, forwarding),
without requiring the application developer to know the im-
B. Software-Defined Internetworking
plementation details of the controller. While there are several
The Internet has revolutionized the way we, as individuals controllers that exist, their application interfaces are still in the
and as a society, live, work, conduct business, socialize, get early stages and independent from each other.
entertainment, etc. As a result, the Internet is now considered Some proposals (e.g., Procera [122], Frenetic [50],
part of our society’s critical infrastructure much like the power, FML [62], Nettle [121]) advocate the use of a network
water, and transportation grids. configuration language to express policies. For example, Pro-
Scalability and performance requirements from increasingly cera [122] builds a policy layer on top of existing controllers to
complex applications have posed a variety of challenges dif- interface with configuration files, GUIs, and external sensors;
ficult to address with the current Internet architecture. This the proposed policy layer is responsible for converting high-
has led the research community to examine “clean-slate” solu- level policies to flow constraints given to be used by the
tions [49]. As the Internet has grown beyond the point at which controller. In [74], network configuration and management
a “flag day”, such as the one used to “upgrade” the ARPANET mechanisms are proposed that focus on enabling changes to
with the TCP/IP protocol suite, would be realistic, another network condition and state, supporting network configuration
considerable challenge is evolving its physical infrastructure and policy definitions, and providing visibility and control
and protocols. A notable example is the deployment of IPv6: over tasks for network diagnostics and troubleshooting. The
despite over a decade in the standards track and two worldwide specification of a northbound interface via a policy layer and
deployment events, IPv4 still makes up the majority of Internet a high level language such as Procera is discussed.
traffic. Additionally, the northbound API should allow applications
Much of the current work on SDN examines or proposes to apply different policies to the same flow (e.g. forward-
solutions within the context of a single administrative domain ing by destination and monitoring by source IP). The work
which matches quite well SDN’s logically centralized con- in [89] proposed modularization to ensure that rules installed
trol model. However, environments whose administration is to perform one task do not override other rules. This was
inherently decentralized, like the Internet, call for a control accomplished by means of an abstraction layer implemented
plane that is logically distributed. This will allow participating with a language based on Frenetic.
autonomous systems (ASes) to be controlled independently Until a clear northbound interface standard emerges, SDN
by their own (logically centralized and possibly physically applications will continue to be developed in an “ad hoc”
distributed) controller. To-date, a few efforts have explored the fashion and the concept of flexible and portable “network
idea of a Software-Defined Internet. For example, the work apps” may have to wait.
in [101] proposed a software-defined Internet architecture
that borrows from MPLS the distinction between network D. Virtualization and Cloud Services
edge and core to split tasks between inter-domain and intra- The demand for virtualization and cloud services has been
domain components. As only the boundary routers and their growing rapidly and attracting considerable interest from in-
IN SUBMISSION 14

dustry and academia. The challenges it presents include rapid Traditional Networks: Host-centric, designed
provisioning, efficient resource management, and scalability for sharing resources between hosts
which can be addressed using SDN’s control model.
For example, FlowVisor [107] and AutoSlice [30] cre-
ate different slices of network resources (e.g., bandwidth,
topology, CPU, forwarding table), delegate them to different
controllers, and enforce isolation between slices. Other SDN
controllers can be used as a network backend to support
virtualization in cloud operating systems, such as Floodlight
for OpenStack [5] and NOX for Mirage [2]. FlowN [45] Information-Centric Networks: Content-centric,
designed for content access and distribution
aims to offer a scalable solution for network virtualization by
providing an efficient mapping between virtual and physical
networks and by leveraging scalable database systems.
In [52], an algorithm for efficient migration with bandwidth
guarantees using OpenFlow was proposed. LIME [72] is
an SDN-based solution for live migration of Virtual Ma-
chines, which handles the network state during migration and
automatically configures network devices at new locations.
NetGraph [102] provides a set of APIs for customers to access Fig. 6. ICN architecture addresses named content, rather than named hosts.
its virtual network functions such as real-time monitoring and
diagnostics.
F. Heterogeneous Network Support
On the context of cloud data centers providing Infrastructure
as a Service (IaaS), [125] presents a management framework Future networks will become increasingly more hetero-
for resources in cloud data centers and addresses multiple geneous, interconnecting users and applications over net-
management issues. In this paper, authors proposed a data- works ranging from wired, infrastructure-based wireless
centric and event-driven architecture with open management (e.g., cellular–based networks, wireless mesh networks), to
interfaces, that leverages SDN techniques to integrate network infrastructure-less wireless networks (e.g. mobile ad-hoc net-
resources into datacenter orchestration and service provision- works, vehicular networks). In the meantime, mobile traffic
ing with the aim of improving service-level agreements and has been increasing exponentially over the past several years,
faster service delivery. and is expected to increase 18–fold by 2016, with more
mobile-connected devices than the world’s population, which
is already a reality [22]. As mobile devices with multiple net-
E. Information-Centric Networking work interfaces become commonplace, users will demand high
quality communication service regardless of location or type
Information-Centric Networking (ICN) is a new paradigm of network access. Self-organizing networks (e.g., wireless
proposed for the future architecture of the Internet, which multi-hop ad-hoc networks) may form to extend the range
aims to increase the efficiency of content delivery and content of infrastructure-based networks or handle episodic connec-
availability. This new concept has been popularized recently tivity disruptions. Self-organizing networks may thus enable
by a number of architecture proposals, such as Content-Centric a variety of new applications such as cloud-based services,
Networking (CCN), also known as the Named Data Network- vehicular communication, community services, healthcare de-
ing (NDN) project [67]. Their driving motivation is that the livery, emergency response, and environmental monitoring, to
current Internet is information-driven, yet networking technol- name a few. Efficient content delivery over wireless access
ogy is still focused on the idea of location-based addressing networks will become essential, and self–organizing networks
and host-to-host communication, as illustrated in Figure 6. By may become a prevalent part of the future hybrid Internet.
proposing an architecture that addresses named data rather A major challenge facing future networks is efficient uti-
than named hosts, content distribution is implemented directly lization of resources; this is especially the case in wireless
into the network fabric rather than relying on the complicated multi-hop ad-hoc networks as the available wireless capacity
mapping, availability, and security mechanisms currently used is inherently limited. This is due to a number of factors
to map content to a single location. including the use of shared physical medium compounded,
The separation between information processing and for- wireless channel impairments, and the absence of managed
warding in ICN is aligned with the decoupling of the data infrastructure. Though these self–organizing networks can be
plane and control plane in SDN. The question then becomes used to supplement or “fill the gaps” in an overburdened
how to combine ICN with SDN towards “Software-Defined infrastructure [103], their lack of dedicated resources and
Information-Centric Networks”. A number of projects [97], shifting connectivity makes capacity sharing difficult. The
[120], [29], [111], [113], [96] have proposed using SDN heterogeneous characteristics of the underlying networks (e.g.,
concepts to implement ICNs. As OpenFlow expands to support physical medium, topology, stability) and nodes (e.g., buffer
customized header matchings, SDN can be employed as a key size, power limitations, mobility) also add another important
enabling technology for ICNs. factor when considering routing and resource allocation.
IN SUBMISSION 15

SDN has the potential to facilitate the deployment and [25] M. Bansal, J. Mehlman, S. Katti, and P. Levis. Openradio: a pro-
management of network applications and services with greater grammable wireless dataplane. In Proceedings of the first workshop
on Hot topics in software defined networks, pages 109–114. ACM,
efficiency. However, SDN techniques to–date, such as Open- 2012.
Flow, largely target infrastructure–based networks. They pro- [26] R. Bennesby, P. Fonseca, E. Mota, and A. Passito. An inter-as routing
mote a centralized control mechanism that is ill–suited to component for software-defined networks. In Network Operations and
Management Symposium (NOMS), 2012 IEEE, pages 138–145, 2012.
the level of decentralization, disruption, and delay present in [27] A. Bianco, R. Birke, L. Giraudo, and M. Palacin. Openflow switching:
infrastructure-less environments. Data plane performance. In Communications (ICC), 2010 IEEE
While previous work has examined the use of SDN in International Conference on, pages 1–5, May.
[28] R. Bifulco, R. Canonico, M. Brunner, P. Hasselmeyer, and F. Mir. A
wireless environments, the scope has primarily focused on practical experience in designing an openflow controller. In Software
infrastructure-based deployments (e.g., WiMAX, Wi-Fi access Defined Networking (EWSDN), 2012 European Workshop on, pages
points). A notable example is the OpenRoads project [129], 61–66, Oct.
[29] N. Blefari-Melazzi, A. Detti, G. Mazza, G. Morabito, S. Salsano,
which envisioned a world in which users could freely move and L. Veltri. An openflow-based testbed for information centric
between wireless infrastructures while also providing support networking. Future Network & Mobile Summit, pages 4–6, 2012.
to the network provider. Other studies such as [96], [39], [41] [30] Z. Bozakov and P. Papadimitriou. Autoslice: automated and scalable
slicing for software-defined networks. In Proceedings of the 2012 ACM
have examined OpenFlow in wireless mesh environments. conference on CoNEXT student workshop, CoNEXT Student ’12, pages
3–4, New York, NY, USA, 2012. ACM.
VII. C ONCLUDING R EMARKS [31] M. Caesar, D. Caldwell, N. Feamster, J. Rexford, A. Shaikh, and
J. van der Merwe. Design and implementation of a routing control
In this paper, we provided an overview of programmable platform. In Proceedings of the 2nd conference on Symposium on
networks and, in this context, examined the emerging field of Networked Systems Design & Implementation-Volume 2, pages 15–28.
USENIX Association, 2005.
Software-Defined Networking (SDN). We look at the history [32] Z. Cai, A. Cox, and T. Ng. Maestro: A system for scalable openflow
of programmable networks, from early ideas until recent control. Technical Report TR10-08, Rice University, December 2010.
developments. In particular we described the SDN architec- [33] K. Calvert, W. Edwards, N. Feamster, R. Grinter, Y. Deng, and
X. Zhou. Instrumenting home networks. ACM SIGCOMM Computer
ture in detail as well as the OpenFlow [85] standard. We Communication Review, 41(1):84–89, 2011.
presented current SDN implementations and testing platforms [34] A. Campbell, I. Katzela, K. Miki, and J. Vicente. Open signaling
and examined network services and applications that have been for atm, internet and mobile networks (opensig’98). ACM SIGCOMM
Computer Communication Review, 29(1):97–108, 1999.
developed based on the SDN paradigm. We concluded with a [35] M. Canini, D. Venzano, P. Peresini, D. Kostic, and J. Rexford. A nice
discussion of future directions enabled by SDN ranging from way to test openflow applications. NSDI, Apr, 2012.
support for heterogeneous networks to Information Centric [36] M. Casado, M. Freedman, J. Pettit, J. Luo, N. McKeown, and
S. Shenker. Ethane: Taking control of the enterprise. ACM SIGCOMM
Networking (ICN). Computer Communication Review, 37(4):1–12, 2007.
[37] M. Casado, T. Koponen, S. Shenker, and A. Tootoonchian. Fabric: a
R EFERENCES retrospective on evolving sdn. In Proceedings of the first workshop on
Hot topics in software defined networks, HotSDN ’12, pages 85–90,
[1] Beacon. https://openflow.stanford.edu/display/Beacon/Home. New York, NY, USA, 2012. ACM.
[2] Connected cloud control: Openflow in mirage. [38] J. D. Case, M. Fedor, M. L. Schoffstall, and J. Davin. Simple network
http://www.openmirage.org/blog/announcing-mirage-openflow. management protocol (snmp), rfc1157, 1990.
[3] Controller performance comparisons. [39] A. Coyle and H. Nguyen. A frequency control algorithm for a mobile
http://www.openflow.org/wk/index.php/ Con- adhoc network. In Military Communications and Information Systems
troller Performance Comparisons. Conference (MilCIS), Canberra, Australia, November 2010.
[4] Devolved Control of ATM Networks. [40] A. R. Curtis, J. C. Mogul, J. Tourrilhes, P. Yalagandula, P. Sharma, and
http://www.cl.cam.ac.uk/research/srg/netos/old-projects/dcan/#pub. S. Banerjee. Devoflow: scaling flow management for high-performance
[5] Floodlight, an open sdn controller. http://floodlight.openflowhub.org/. networks. SIGCOMM Comput. Commun. Rev., 41(4):254–265, Aug.
[6] Helios by nec. http://www.nec.com/. 2011.
[7] Indigo: Open source openflow switches. [41] P. Dely, A. Kassler, and N. Bayer. Openflow for wireless mesh net-
http://www.openflowhub.org/display/Indigo/. works. In Proceedings of 20th International Conference on Computer
[8] Jaxon:java-based openflow controller. http://jaxon.onuos.org/. Communications and Networks (ICCCN), pages 1–6. IEEE, 2011.
[9] Mul. http://sourceforge.net/p/mul/wiki/Home/. [42] A. Dixit, F. Hao, S. Mukherjee, T. Lakshman, and R. Kompella.
[10] The nodeflow openflow controller. http://garyberger.net/?p=537. Towards an elastic distributed sdn controller. In Proceedings of the
[11] Node.js. http://nodejs.org/. second ACM SIGCOMM workshop on Hot topics in software defined
[12] ofsoftswitch13 - cpqd. https://github.com/CPqD/ofsoftswitch13. networking, HotSDN ’13, pages 7–12, New York, NY, USA, 2013.
[13] Open networking foundation. https://www.opennetworking.org/about. ACM.
[14] Open Networking Research Center (ONRC). http://onrc.net. [43] A. Doria, F. Hellstrand, K. Sundell, and T. Worster. General Switch
[15] Open vswitch and ovs-controller. http://openvswitch.org/. Management Protocol (GSMP) V3. RFC 3292 (Proposed Standard),
[16] Pantou: Openflow 1.0 for openwrt. June 2002.
http://www.openflow.org/wk/index.php/ Open- [44] A. Doria, J. H. Salim, R. Haas, H. Khosravi, W. Wang, L. Dong,
Flow 1.0 for OpenWRT. R. Gopal, and J. Halpern. Forwarding and Control Element Separation
[17] Pox. http://www.noxrepo.org/pox/about-pox/. (ForCES) Protocol Specification. RFC 5810 (Proposed Standard), Mar.
[18] Ryu. http://osrg.github.com/ryu/. 2010.
[19] Sdn troubleshooting simulator. http://ucb-sts.github.com/sts/. [45] D. Drutskoy, E. Keller, and J. Rexford. Scalable network virtualization
[20] Simple Network Access Control (SNAC). in software-defined networks. Internet Computing, IEEE, PP(99):1–1.
http://www.openflow.org/wp/snac/. [46] R. Enns. NETCONF Configuration Protocol. RFC 4741 (Proposed
[21] Trema openflow controller framework. https://github.com/trema/trema. Standard), Dec. 2006. Obsoleted by RFC 6241.
[22] Cisco visual networking index: Global mobile data traffic forecast [47] D. Erickson. The beacon openflow controller, 2012.
update, 2011–2016. Technical report, Cisco, February 2012. [48] N. Feamster. Outsourcing home network security. In Proceedings of
[23] Inter-datacenter wan with centralized te using sdn and openflow. In the 2010 ACM SIGCOMM workshop on Home networks, pages 37–42.
Open Networking Summit, April 2012. ACM, 2010.
[24] Optical transport working group otwg. In Open Networking Foundation [49] A. Feldmann. Internet clean-slate design: what and why? SIGCOMM
ONF, 2013. Comput. Commun. Rev., 37(3):59–64, July 2007.
IN SUBMISSION 16

[50] N. Foster, R. Harrison, M. J. Freedman, C. Monsanto, J. Rexford, first workshop on Hot topics in software defined networks, HotSDN
A. Story, and D. Walker. Frenetic: a network programming language. ’12, pages 49–54, New York, NY, USA, 2012. ACM.
In Proceedings of the 16th ACM SIGPLAN international conference on [74] H. Kim and N. Feamster. Improving network management with
Functional programming, ICFP ’11, pages 279–291, New York, NY, software defined networking. Communications Magazine, IEEE,
USA, 2011. ACM. 51(2):114–119, February.
[51] A. Gember, P. Prabhu, Z. Ghadiyali, and A. Akella. Toward software- [75] E. Kissel, G. Fernandes, M. Jaffee, M. Swany, and M. Zhang. Driving
defined middlebox networking. 2012. software defined networks with xsp. In SDN12: Workshop on Software
[52] S. Ghorbani and M. Caesar. Walk the line: consistent network updates Defined Networks, 2012.
with bandwidth guarantees. In Proceedings of the first workshop on [76] T. Koponen, M. Casado, N. Gude, J. Stribling, L. Poutievski, M. Zhu,
Hot topics in software defined networks, HotSDN ’12, pages 67–72, R. Ramanathan, Y. Iwata, H. Inoue, T. Hama, et al. Onix: A distributed
New York, NY, USA, 2012. ACM. control platform for large-scale production networks. OSDI, Oct, 2010.
[53] A. Greenberg, G. Hjalmtysson, D. Maltz, A. Myers, J. Rexford, G. Xie, [77] B. Lantz, B. Heller, and N. McKeown. A network in a laptop: rapid
H. Yan, J. Zhan, and H. Zhang. A clean slate 4d approach to network prototyping for software-defined networks. In Proceedings of the Ninth
control and management. ACM SIGCOMM Computer Communication ACM SIGCOMM Workshop on Hot Topics in Networks, 2010.
Review, 35(5):41–54, 2005. [78] D. Levin, A. Wundsam, B. Heller, N. Handigol, and A. Feldmann.
[54] N. Gude, T. Koponen, J. Pettit, B. Pfaff, M. Casado, N. McKeown, Logically centralized?: state distribution trade-offs in software defined
and S. Shenker. Nox: towards an operating system for networks. ACM networks. In Proceedings of the first workshop on Hot topics in
SIGCOMM Computer Communication Review, 38(3):105–110, 2008. software defined networks, HotSDN ’12, pages 1–6, New York, NY,
[55] V. Gudla, S. Das, A. Shastri, G. Parulkar, N. McKeown, L. Kazovsky, USA, 2012. ACM.
and S. Yamashita. Experimental demonstration of openflow control [79] L. Li, Z. Mao, and J. Rexford. Toward software-defined cellular
of packet and circuit switches. In Optical Fiber Communication networks. In Software Defined Networking (EWSDN), 2012 European
(OFC), collocated National Fiber Optic Engineers Conference, 2010 Workshop on, pages 7–12, 2012.
Conference on (OFC/NFOEC), pages 1–3, 2010. [80] T. A. Limoncelli. Openflow: a radical new idea in networking.
[56] N. Handigol, B. Heller, V. Jeyakumar, D. Maziéres, and N. McKeown. Commun. ACM, 55(8):42–47, Aug. 2012.
Where is the debugger for my software-defined network? In Proceed- [81] L. Liu, T. Tsuritani, I. Morita, H. Guo, and J. Wu. Openflow-based
ings of the first workshop on Hot topics in software defined networks, wavelength path control in transparent optical networks: A proof-of-
HotSDN ’12, pages 55–60, New York, NY, USA, 2012. ACM. concept demonstration. In Optical Communication (ECOC), 2011 37th
[57] N. Handigol, S. Seetharaman, M. Flajslik, N. McKeown, and R. Jo- European Conference and Exhibition on, pages 1–3, 2011.
hari. Plug-n-serve: Load-balancing web traffic using openflow. ACM [82] G. Lu, R. Miao, Y. Xiong, and C. Guo. Using cpu as a traffic co-
SIGCOMM Demo, 2009. processing unit in commodity switches. In Proceedings of the first
[58] S. Hassas Yeganeh and Y. Ganjali. Kandoo: a framework for efficient workshop on Hot topics in software defined networks, HotSDN ’12,
and scalable offloading of control applications. In Proceedings of the pages 31–36, New York, NY, USA, 2012. ACM.
first workshop on Hot topics in software defined networks, HotSDN [83] Y. Luo, P. Cascon, E. Murray, and J. Ortega. Accelerating open-
’12, pages 19–24, New York, NY, USA, 2012. ACM. flow switching with network processors. In Proceedings of the 5th
[59] B. Heller, S. Seetharaman, P. Mahadevan, Y. Yiakoumis, P. Sharma, ACM/IEEE Symposium on Architectures for Networking and Commu-
S. Banerjee, and N. McKeown. Elastictree: Saving energy in data nications Systems, ANCS ’09, pages 70–71, New York, NY, USA,
center networks. In Proceedings of the 7th USENIX conference on 2009. ACM.
Networked systems design and implementation, pages 17–17. USENIX [84] H. Mai, A. Khurshid, R. Agarwal, M. Caesar, P. B. Godfrey, and S. T.
Association, 2010. King. Debugging the data plane with anteater. In Proceedings of the
[60] B. Heller, R. Sherwood, and N. McKeown. The controller placement ACM SIGCOMM 2011 conference, SIGCOMM ’11, pages 290–301,
problem. In Proceedings of the first workshop on Hot topics in software New York, NY, USA, 2011. ACM.
defined networks, HotSDN ’12, pages 7–12, New York, NY, USA, [85] N. McKeown, T. Anderson, H. Balakrishnan, G. Parulkar, L. Peterson,
2012. ACM. J. Rexford, S. Shenker, and J. Turner. Openflow: enabling innovation in
[61] T. Henderson, M. Lacage, G. Riley, C. Dowell, and J. Kopena. Network campus networks. ACM SIGCOMM Computer Communication Review,
simulations with the ns-3 simulator. SIGCOMM demonstration, 2008. 38(2):69–74, 2008.
[62] T. L. Hinrichs, N. S. Gude, M. Casado, J. C. Mitchell, and S. Shenker. [86] S. Mehdi, J. Khalid, and S. Khayam. Revisiting traffic anomaly
Practical declarative network management. In Proceedings of the 1st detection using software defined networking. In Recent Advances in
ACM workshop on Research on enterprise networking, WREN ’09, Intrusion Detection, pages 161–180. Springer, 2011.
pages 1–10, New York, NY, USA, 2009. ACM. [87] J. E. V. D. Merwe and I. M. Leslie. Switchlets and dynamic virtual atm
[63] http://irtf.org/sdnrg. Software-defined networking research group - networks. In Proc Integrated Network Management V, pages 355–368.
sdnrg at irtf, 2013. Chapman and Hall, 1997.
[64] http://www.itu.int/en/ITU T/sdn/Pages/default.aspx. Itu telecommuni- [88] J. C. Mogul and P. Congdon. Hey, you darned counters!: get off my
cation standardization sector’s sdn portal, 2013. asic! In Proceedings of the first workshop on Hot topics in software
[65] http://www.opendaylight.org/. Opendaylight, 2013. defined networks, HotSDN ’12, pages 25–30, New York, NY, USA,
[66] http://www.openstack.org/. Openstack, 2013. 2012. ACM.
[67] V. Jacobson, D. Smetters, J. Thornton, M. Plass, N. Briggs, and [89] C. Monsanto, J. Reich, N. Foster, J. Rexford, and D. Walker. Com-
R. Braynard. Networking named content. In Proceedings of the posing software-defined networks. In Proceedings 10th USENIX Sym-
5th international conference on Emerging networking experiments and posium on Networked Systems Design and Implementation, NSDI’13,
technologies, pages 1–12. ACM, 2009. 2013.
[68] S. Jain, A. Kumar, S. Mandal, J. Ong, L. Poutievski, A. Singh, [90] J. Moore and S. Nettles. Towards practical programmable packets.
S. Venkata, J. Wanderer, J. Zhou, M. Zhu, et al. B4: Experience with In Proceedings of the 20th Conference on Computer Communications
a globally-deployed software defined wan. In Proceedings of the ACM (INFOCOM). Citeseer, 2001.
SIGCOMM 2013 conference on SIGCOMM, pages 3–14. ACM, 2013. [91] R. Mortier, T. Rodden, T. Lodge, D. McAuley, C. Rotsos, A. Moore,
[69] M. Jarschel, S. Oechsner, D. Schlosser, R. Pries, S. Goll, and P. Tran- A. Koliousis, and J. Sventek. Control and understanding: Owning
Gia. Modeling and performance evaluation of an openflow architecture. your home network. In Communication Systems and Networks (COM-
In Teletraffic Congress (ITC), 2011 23rd International, pages 1–7, Sept. SNETS), 2012 Fourth International Conference on, pages 1–10. IEEE,
[70] N. Kang, Z. Liu, J. Rexford, and D. Walker. Optimizing the one big 2012.
switch abstraction in software-defined networks. [92] A. Nakao. Flare : Open deeply programmable network node architec-
[71] Y. Kanizo, D. Hay, and I. Keslassy. Palette: Distributing tables in ture. http://netseminar.stanford.edu/10 18 12.html.
software-defined networks. In INFOCOM, pages 545–549, 2013. [93] M. R. Nascimento, C. E. Rothenberg, M. R. Salvador, C. N. A.
[72] E. Keller, S. Ghorbani, M. Caesar, and J. Rexford. Live migration of Corrêa, S. C. de Lucena, and M. F. Magalhães. Virtual routers as a
an entire network (and its hosts). In Proceedings of the 11th ACM service: the routeflow approach leveraging software-defined networks.
Workshop on Hot Topics in Networks, HotNets-XI, pages 109–114, In Proceedings of the 6th International Conference on Future Internet
New York, NY, USA, 2012. ACM. Technologies, CFI ’11, pages 34–37, New York, NY, USA, 2011. ACM.
[73] A. Khurshid, W. Zhou, M. Caesar, and P. B. Godfrey. Veriflow: [94] A. Nayak, A. Reimers, N. Feamster, and R. Clark. Resonance: Dynamic
verifying network-wide invariants in real time. In Proceedings of the access control for enterprise networks. In Proceedings of the 1st ACM
IN SUBMISSION 17

workshop on Research on enterprise networking, pages 11–18. ACM, Systems, ANCS ’10, pages 13:1–13:2, New York, NY, USA, 2010.
2009. ACM.
[95] Netfpga platform. http://netfpga.org. [115] D. Tennenhouse, J. Smith, W. Sincoskie, D. Wetherall, and G. Minden.
[96] X. Nguyen. Software defined networking in wireless mesh network. A survey of active network research. Communications Magazine, IEEE,
Msc. thesis, INRIA, UNSA, August 2012. 35(1):80–86, 1997.
[97] X.-N. Nguyen, D. Saucez, and T. Turletti. Efficient caching in Content- [116] D. Tennenhouse and D. Wetherall. Towards an active network archi-
Centric Networks using OpenFlow. In 2013 IEEE Conference on tecture. In DARPA Active NEtworks Conference and Exposition, 2002.
Computer Communications Workshops (INFOCOM WKSHPS), pages Proceedings, pages 2–15. IEEE, 2002.
67–68. IEEE, Apr. 2013. [117] A. Tootoonchian and Y. Ganjali. Hyperflow: A distributed control plane
[98] A. Passarella. Review: A survey on content-centric technologies for the for openflow. In Proceedings of the 2010 internet network management
current internet: Cdn and p2p solutions. Comput. Commun., 35(1):1– conference on Research on enterprise networking, pages 3–3. USENIX
32, Jan. 2012. Association, 2010.
[99] A. Patel, P. Ji, and T. Wang. Qos-aware optical burst switching in [118] A. Tootoonchian, S. Gorbunov, Y. Ganjali, M. Casado, and R. Sher-
openflow based software-defined optical networks. In Optical Network wood. On controller performance in software-defined networks. In
Design and Modeling (ONDM), 2013 17th International Conference USENIX Workshop on Hot Topics in Management of Internet, Cloud,
on, pages 275–280, 2013. and Enterprise Networks and Services (Hot-ICE), 2012.
[100] K. Phemius and M. Bouet. Openflow: Why latency does matter. In [119] J. Van der Merwe, S. Rooney, I. Leslie, and S. Crosby. The tempest-
Integrated Network Management (IM 2013), 2013 IFIP/IEEE Interna- a practical framework for network programmability. Network, IEEE,
tional Symposium on, pages 680–683, 2013. 12(3):20–28, 1998.
[101] B. Raghavan, M. Casado, T. Koponen, S. Ratnasamy, A. Ghodsi, [120] L. Veltri, G. Morabito, S. Salsano, N. Blefari-Melazzi, and A. Detti.
and S. Shenker. Software-defined internet architecture: decoupling Supporting information-centric functionality in software defined net-
architecture from infrastructure. In Proceedings of the 11th ACM works. IEEE ICC Workshop on Software Defined Networks, June 2012.
Workshop on Hot Topics in Networks, HotNets-XI, pages 43–48, New [121] A. Voellmy and P. Hudak. Nettle: taking the sting out of programming
York, NY, USA, 2012. ACM. network routers. In Proceedings of the 13th international conference on
[102] R. Raghavendra, J. Lobo, and K.-W. Lee. Dynamic graph query Practical aspects of declarative languages, PADL’11, pages 235–249,
primitives for sdn-based cloudnetwork management. In Proceedings of Berlin, Heidelberg, 2011. Springer-Verlag.
the first workshop on Hot topics in software defined networks, HotSDN [122] A. Voellmy, H. Kim, and N. Feamster. Procera: a language for high-
’12, pages 97–102, New York, NY, USA, 2012. ACM. level reactive network control. In Proceedings of the first workshop on
[103] B. Rais, M. Mendonca, T. Turletti, and K. Obraczka. Towards truly het- Hot topics in software defined networks, HotSDN ’12, pages 43–48,
erogeneous internets: Bridging infrastructure-based and infrastructure- New York, NY, USA, 2012. ACM.
less networks. In Communication Systems and Networks (COMSNETS), [123] A. Voellmy and J. Wang. Scalable software defined network con-
2011 Third International Conference on, pages 1–10. IEEE, 2011. trollers. In Proceedings of the ACM SIGCOMM 2012 conference on
[104] M. Reitblatt, N. Foster, J. Rexford, C. Schlesinger, and D. Walker. Applications, technologies, architectures, and protocols for computer
Abstractions for network update. In Proceedings of the ACM SIG- communication, SIGCOMM ’12, pages 289–290, New York, NY, USA,
COMM 2012 conference on Applications, technologies, architectures, 2012. ACM.
and protocols for computer communication, SIGCOMM ’12, pages [124] R. Wang, D. Butnariu, and J. Rexford. Openflow-based server load
323–334, New York, NY, USA, 2012. ACM. balancing gone wild. In Workshop of HotICE, volume 11, 2011.
[105] J. Rexford, A. Greenberg, G. Hjalmtysson, D. Maltz, A. Myers, G. Xie, [125] X. Wang, Z. Liu, Y. Qi, and J. Li. Livecloud: A lucid orchestrator
J. Zhan, and H. Zhang. Network-wide decision making: Toward a for cloud datacenters. In Cloud Computing Technology and Science
wafer-thin control plane. In Proc. HotNets, pages 59–64. Citeseer, (CloudCom), 2012 IEEE 4th International Conference on, pages 341–
2004. 348, 2012.
[106] C. E. Rothenberg, M. R. Nascimento, M. R. Salvador, C. N. A. [126] Z. Wang, T. Tsou, J. Huang, X. Shi, and X. Yin. Analysis of
Corrêa, S. Cunha de Lucena, and R. Raszuk. Revisiting routing control Comparisons between OpenFlow and ForCES, March 2012.
platforms with the eyes and muscles of software-defined networking. [127] A. Wundsam, D. Levin, S. Seetharaman, and A. Feldmann. Ofrewind:
In Proceedings of the first workshop on Hot topics in software defined enabling record and replay troubleshooting for networks. In Proceed-
networks, HotSDN ’12, pages 13–18, New York, NY, USA, 2012. ings of the 2011 USENIX conference on USENIX annual technical
ACM. conference, USENIXATC’11, pages 29–29, Berkeley, CA, USA, 2011.
[107] R. Sherwood, M. Chan, A. Covington, G. Gibb, M. Flajslik, N. Hand- USENIX Association.
igol, T. Huang, P. Kazemian, M. Kobayashi, J. Naous, et al. Carving [128] K. Yap, M. Kobayashi, R. Sherwood, T. Huang, M. Chan, N. Hand-
research slices out of your production networks with openflow. ACM igol, and N. McKeown. Openroads: Empowering research in mo-
SIGCOMM Computer Communication Review, 40(1):129–130, 2010. bile networks. ACM SIGCOMM Computer Communication Review,
[108] H. Shirayanagi, H. Yamada, and K. Kono. Honeyguide: A vm 40(1):125–126, 2010.
migration-aware network topology for saving energy consumption in [129] K. Yap, R. Sherwood, M. Kobayashi, T. Huang, M. Chan, N. Handigol,
data center networks. In Computers and Communications (ISCC), 2012 N. McKeown, and G. Parulkar. Blueprint for introducing innovation
IEEE Symposium on, pages 000460–000467. IEEE, 2012. into wireless mobile networks. In Proceedings of the second ACM
[109] D. E. Simeonidou, R. Nejabati, and M. Channegowda. Software defined SIGCOMM workshop on Virtualized infrastructure systems and archi-
optical networks technology and infrastructure: Enabling software- tectures, pages 25–32. ACM, 2010.
defined optical network operations. In Optical Fiber Communication [130] S. Yeganeh, A. Tootoonchian, and Y. Ganjali. On scalability
Conference/National Fiber Optic Engineers Conference 2013, page of software-defined networking. Communications Magazine, IEEE,
OTh1H.3. Optical Society of America, 2013. 51(2):136–141, February.
[110] B. Stephens, A. Cox, W. Felter, C. Dixon, and J. Carter. Past: scalable [131] M. Yu, L. Jose, and R. Miao. Software defined traffic measurement with
ethernet for data centers. In Proceedings of the 8th international opensketch. In Proceedings 10th USENIX Symposium on Networked
conference on Emerging networking experiments and technologies, Systems Design and Implementation, NSDI’13, 2013.
CoNEXT ’12, pages 49–60, New York, NY, USA, 2012. ACM. [132] M. Yu, J. Rexford, M. Freedman, and J. Wang. Scalable flow-based
[111] J. Suh, H. Jung, T. Kwon, and Y. Choi. C-flow: Content-oriented networking with difane. In Proceedings of the ACM SIGCOMM 2010
networking over openflow. In Open Networking Summit, April 2012. conference on SIGCOMM, pages 351–362. ACM, 2010.
[112] L. Suresh, J. Schulz-Zander, R. Merz, A. Feldmann, and T. Vazao.
Towards programmable enterprise wlans with odin. In Proceedings of
the first workshop on Hot topics in software defined networks, HotSDN
’12, pages 115–120, New York, NY, USA, 2012. ACM.
[113] D. Syrivelis, G. Parisis, D. Trossen, P. Flegkas, V. Sourlas, T. Korakis,
and L. Tassiulas. Pursuing a software defined information-centric
network. In Software Defined Networking (EWSDN), 2012 European
Workshop on, pages 103–108, Oct.
[114] V. Tanyingyong, M. Hidell, and P. Sjödin. Improving pc-based
openflow switching performance. In Proceedings of the 6th ACM/IEEE
Symposium on Architectures for Networking and Communications