Reducing Network Energy Consumption

via Sleeping and Rate-Adaptation

Sergiu Nedevschi∗† Lucian Popa∗† Gianluca Iannaccone †
Sylvia Ratnasamy †
David Wetherall‡§

Abstract via standards such as EnergyStar. In fact, EnergyStar

We present the design and evaluation of two forms of standard proposals for 2009 discuss slower operation
power management schemes that reduce the energy of network links to conserve energy when idle. A new
consumption of networks. The first is based on putting IEEE 802.3az Task Force was launched in early 2007 to
network components to sleep during idle times, reducing focus on this issue for Ethernet [15].
energy consumed in the absence of packets. The second Fortunately, there is an opportunity for substantial re-
is based on adapting the rate of network operation to the ductions in the energy consumption of existing networks
offered workload, reducing the energy consumed when due to two factors. First, networks are provisioned for
actively processing packets. worst-case or busy-hour load, and this load typically
For real-world traffic workloads and topologies and us- exceeds their long-term utilization by a wide margin.
ing power constants drawn from existing network equip- For example, measurements reveal backbone utilizations
ment, we show that even simple schemes for sleeping under 30% [16] and up to hour-long idle times at access
or rate-adaptation can offer substantial savings. For in- points in enterprise wireless networks [17]. Second, the
stance, our practical algorithms stand to halve energy energy consumption of network equipment remains sub-
consumption for lightly utilized networks (10-20%). We stantial even when the network is idle. The implication
show that these savings approach the maximum achiev- of these factors is that most of the energy consumed in
able by any algorithms using the same power manage- networks is wasted.
ment primitives. Moreover this energy can be saved with- Our work is an initial exploration of how overall
out noticeably increasing loss and with a small and con- network energy consumption might be reduced without
trolled increase in latency (<10ms). Finally, we show adversely affecting network performance. This will
that both sleeping and rate adaptation are valuable de- require two steps. First, network equipment ranging
pending (primarily) on the power profile of network from routers to switches and NICs will need power man-
equipment and the utilization of the network itself. agement primitives at the hardware level. By analogy,
power management in computers has evolved around
1 Introduction
hardware support for sleep and performance states. The
In this paper, we consider power management for former (e.g.,C-states in Intel processors) reduce idle con-
networks from a perspective that has recently begun sumption by powering off sub-components to different
to receive attention: the conservation of energy for extents, while the latter (e.g.,SpeedStep, P-states in Intel
operating and environmental reasons. Energy consump- processors) tradeoff performance for power via operating
tion in network exchanges is rising as higher capacity frequency. Second, network protocols will need to make
network equipment becomes more power-hungry and use of the hardware primitives to best effect. Again, by
requires greater amounts of cooling. Combined with analogy with computers, power management preferences
rising energy costs, this has made the cost of powering control how the system switches between the available
network exchanges a substantial and growing fraction states to save energy with minimal impact on users.
of the total cost of ownership – up to half by some Of these two steps, our focus is on the network
estimates[23]. Various studies now estimate the power
protocols. Admittedly, these protocols build on hardware
usage of the US network infrastructure at between 5 support for power management that is in its infancy
and 24 TWh/year[25, 26], or $0.5-2.4B/year at a rate for networking equipment. Yet the necessary support
of $0.10/KWh, depending on what is included. Public will readily be deployed in networks where it proves
concern about carbon footprints is also rising, and stands valuable, with forms such as sleeping and rapid rate
to affect network equipment much as it has computers
selection for Ethernet [15] already under development.
∗
University of California, Berkeley For comparison, computer power management compat-
†
Intel Research, Berkeley ible with the ACPI standard has gone from scarce to
‡
University of Washingon widely deployed over the past five to ten years and is
§
Intel Research, Seattle now expanding into the server market. Thus our goal is

USENIX Association NSDI ’08: 5th USENIX Symposium on Networked Systems Design and Implementation 323
 to learn: what magnitude of energy savings a protocol a network element is:
using feasible hardware primitives might offer; what per-
formance tradeoff comes with these savings; and which E = pa T a + pi T i (1)
of the feasible kinds of hardware primitives would max-
imize benefits. We hope that our research can positively where pa , pi denote the power consumption in active
influence the hardware support offered by industry. and idle modes respectively and Ta , Ti the times spent
in each mode.
The hardware support we assume from network equip-
ment is in the form of performance and sleep states.
Performance states help to save power when routers Reducing power through sleep and performance
are active, while sleep states help to save power when states Sleep states lower power consumption by
routers are idle. The performance states we assume putting sub-components of the overall system to sleep
dynamically change the rate of links and their associated when there is no work to process. Thus sleeping reduces
interfaces. The sleep states we assume quickly power the power consumed when idle, i.e.,it reduces the pi Ti
off network interfaces when they are idle. We develop term of Eqn. (2) by reducing the pi to some sleep-mode
two approaches to save energy with these primitives. power draw ps where ps < pi .
The first puts network interfaces to sleep during short Performance states reduce power consumption by
idle periods. To make this effective we introduce small lowering the rate at which work is processed. As we
amounts of buffering, much as 802.11 APs do for elaborate on in later sections, some portion of both
sleeping clients; this collects packets into small bursts active and idle power consumption depends on the
and thereby creates gaps long enough to profitably frequency and voltage at which work is processed.
sleep. Potential concerns are that buffering will add Hence performance states that scale frequency and/or
too much delay across the network and that bursts will voltage reduce both the pa and pi power draws resulting
exacerbate loss. Our algorithms arrange for routers and in an overall reduction in energy consumption.
switches to sleep in a manner that ensures the buffering
We also assume a penalty for transitioning between
delay penalty is paid only once (not per link) and that
power states. For simplicity, we measure this penalty
routers clear bursts so as to not amplify loss noticeably.
in time, typically milliseconds, treating it as a period in
The result is a novel scheme that differs from 802.11
which the router can do no useful work. We use this as a
schemes in that all network elements are able to sleep
simple switching model that lumps all penalties, ignor-
when not utilized yet added delay is bounded. The
ing other effects that may be associated with switches
second approach adapts the rate of individual links based
such as a transient increase in power consumption. Thus
on the utilization and queuing delay of the link.
there is also a cost for switching between states.
We then evaluate these approaches using real-world
network topologies and traffic workloads from Abilene
and Intel. We find that: (1) rate-adaptation and sleeping Networks with rate adaptation and sleeping support
have the potential to deliver substantial energy savings In a network context, the sleeping and rate adaptation
for typical networks; (2) the simple schemes we develop decisions one router makes fundamentally impacts –
are able to capture most of this energy-saving potential; and is impacted by – the decisions of its neighboring
(3) our schemes do not noticeably degrade network routers. Moreover, as we see later in the paper, the
performance; and (4) both sleeping and rate-adaptation strategies by which each is best exploited are very
are valuable depending primarily on the utilization of different (Intuitively this is because sleep-mode savings
the network and equipment power profiles. are best exploited by maximizing idle times, which
implies processing work as quickly as possible, while
2 Approach performance-scaling is best exploited by processing
work as slowly as possible, which reduces idle times).
This section describes the high-level model for power Hence, to avoid complex interactions, we consider that
consumption that motivates our rate adaptation and the whole network, or at least well-defined components
sleeping solutions, as well as the methodology by which of it, run in either rate adaptation or sleep mode.
we evaluate these solutions.
We develop the specifics of our sleeping schemes in
2.1 Power Model Overview Section 3, and our rate adaptation schemes in Section 4.
Note that our solutions are deliberately constructed to
Active and idle power A network element is active apply broadly to the networking infrastructure – from
when it is actively processing incoming or outgoing traf- end-host NICs, to switches, and IP routers, etc. – so that
fic, and idle when it is powered on but does not process they may be applied wherever they prove to be the most
traffic. Given these modes, the energy consumption for valuable. They are not tied to IP-layer protocols.

324 NSDI ’08: 5th USENIX Symposium on Networked Systems Design and Implementation USENIX Association
 2.2 Methodology our default a traffic matrix whose link utilization levels
reflect the average link utilization over the entire day –
The overall energy savings we can expect will depend on
this corresponds to a 5% link utilization on average with
the extent to which our power-management algorithms
bottleneck links experiencing about 15% utilization.
can successful exploit opportunities to sleep or rate
We linearly scale TMs to study performance with
adapt as well as the power profile of network equipment
increasing utilization up to a maximum average network
(i.e.,, relative magnitudes of pa , pi and ps ). To clearly
utilization of 31% as beyond this some links reach very
separate the effect of each, we evaluate sleep solutions in
high utilizations. Finally, while the TMs specify the
terms of the fraction of time for which network elements
5-minute average rate observed for each ingress-egress
can sleep and rate-adaptation solutions in terms of the
pair, we still require a packet-level traffic generation
reduction in the average rate at which the network
model that creates this rate. In keeping with previous
operates. 1 In this way we assess each solution with
studies [18, 31] we generate traffic as a mix of Pareto
the appropriate baseline. We then evaluate how these
flows, and for some results we use constant bit-rate
metrics translate into overall network energy savings for
(CBR) traffic. As per standard practice, we set router
different equipment power profiles and hence compare
queue sizes equal to the bandwidth-delay product in the
the relative merits of sleeping and rate-adaptation
network; we use the bandwidth of the bottleneck link,
(Section 5). For both sleep and rate-adaptation, we
and a delay of 100ms.
calibrate the savings achieved by our practical solutions
by comparing to the maximum savings achievable by Intel As an additional real-world dataset, we collected
optimal, but not necessarily practical, solutions. topology and traffic information for the global Intel
In network environments where packet arrival rates enterprise network. This network connects Intel sites
can be highly non-uniform, allowing network elements worldwide, from small remote offices to large multi-
to transition between operating rates or sleep/active building sites with thousands of users. It comprises
modes with corresponding transition times can introduce approximately 300 routers and over 600 links with
additional packet delay, or even loss, that would have not capacities ranging from 1.5Mbps to 1Gbps.
otherwise occurred. Our goal is to explore solutions that To simulate realistic traffic, we collected unsampled
usefully navigate the tradeoff between potential energy Netflow records [7] from the core routers. The records,
savings and performance. In terms of performance, exported by each router every minute, contain per flow
we measure the average and 98th percentile of the information that allows us to recreate the traffic sourced
end-to-end packet delay and loss. by ingress nodes.
In the absence of network equipment with hardware 3 Putting Network Elements to Sleep
support for power management, we base our evaluations
In this section we discuss power management algorithms
on packet-level simulation with real-world network
that exploit sleep states to reduce power consumption
topologies and traffic workloads. The key factors on
during idle times.
which power savings then depend, beyond the details of
the solutions themselves, are the technology constants of 3.1 Model and Assumptions
the sleep and performance states and the characteristics Background A well established technique, as used by
of the network. In particular, the utilization of links microprocessors and mobiles, is to reduce idle power by
determines the relative magnitudes of Tactive and Tidle putting hardware sub-components to sleep. For example,
as well as the opportunities for profitably exploiting modern Intel processors such as the Core Duo [1] have
sleep and performance states. We give simple models for a succession of sleep states (called C-states) that offer
technology constants in the following sections. To cap- increasingly reduced power at the cost of increasingly
ture the effect of the network on power savings, we drive high latencies to enter and exit these states. We assume
our simulation with two realistic network topologies and similar sleep states made available for network equip-
traffic workloads (Abilene and Intel) that are summa- ment. For the purpose of this study we ignore the options
rized below. We use ns2 as our packet-level simulator. afforded by multiple sleep states and assume as an initial
Abilene We use Abilene as a test case because of the simplification that we have a single sleep state.
ready availability of detailed topology and traffic infor- Model We model a network sleep state as character-
mation. The information from [27] provides us with the ized by three features or parameters. The first is the
link connectivity, weights (to compute routes), latencies power draw in sleep mode ps which we assume to be a
and capacities for Abilene’s router-level topology. We small fraction of the idle mode power draw pi .
use measured Abilene traffic matrices (TMs) available The second characterizing parameter of a sleep state is
in the community [29] to generate realistic workloads the time δ it takes to transition in and out of sleep states.
over this topology. Unless otherwise stated, we use as Higher values of δ raise the bar on when the network

USENIX Association NSDI ’08: 5th USENIX Symposium on Networked Systems Design and Implementation 325
 element can profitably enter sleep mode and hence
δ critically affects potential savings. While network S R
interface cards can make physical-layer transitions in as
low as 10µs, transition times that involve restoring state
at higher layers (memory, operating system) are likely
to be higher [13]. We thus evaluate our solutions over a
S R
wide range values of transition times.
Finally, network equipment must support a mechanism
for invoking and exiting sleep states. The option that Figure 1: Packets within a burst are organized by destination.
makes the fewest assumptions about the sophistication of
hardware support is timer-driven sleeping, in which the packet arrives for some time. While very simple, such
network element enters and exits sleep at well-defined an approach can result in frequent transitions which
times. Prior to entering sleep the network element limits savings for higher transition times and/or higher
specifies the time in the future at which it will exit sleep link speeds. For example, with a 10Gbps link, even
and all packets that arrive at a sleeping interface are lost. under low utilization (5%) and packet sizes of 1KB, the
The second possibility, described in [12], is for routers average packet inter-arrival time is very small – 15µs.
to wake up automatically on sensing incoming traffic Thus while opportunistic sleeping might be effective in
on their input ports. To achieve this “wake-on-arrival” LANs [11, 21] with high idle times, for fast links this
(WoA), the circuitry that senses packets on a line is left technique is only effective for very low transition times
powered on even in sleep mode. While support for WoA δ (we quantify this shortly). In addition, opportunistic
is not common in either computers or interfaces today, sleep is only possible with the more sophisticated
this is a form of hardware support that might prove hardware support of wake-on-arrival.
desirable for future network equipment and is currently To create greater opportunities for sleep, we consider
under discussion in the IEEE 802.3az Task Force [13]. a novel approach that allows us to explicitly control
Note that even with wake-on-arrival, bits arriving during the tradeoff between network performance and energy
the transition period δ are effectively lost. To handle this, savings. Our approach is to shape traffic into small
the authors in [6] propose the use of “dummy” packets bursts at the edges of the network – edge devices then
to rouse a sleeping neighbor. A node A that wishes to transmit packets in bunches and routers within the
wake B first sends B a dummy packet, and then waits network wake up to process a burst of packets, and then
for time δ before transmitting the actual data traffic. The sleep until the next burst arrives. The intent is to provide
solutions we develop in this paper apply seamlessly to sufficient bunching to create opportunities for sleep if
either timer-driven or WoA-based hardware. the load is low, yet not add excessive delay. This is
Measuring savings and performance In this section, a radical approach in the sense that much other work
we measure savings in terms of the percentage of time seeks to avoid bursts rather than create them (e.g., token
network elements spend asleep and performance in terms buckets for QOS, congestion avoidance, buffering at
of the average and 98th percentile of the end-to-end routers). As our measurements of loss and delay show,
packet delay and loss. We assume that individual line our schemes avoid the pitfalls associated with bursts
cards in a network element can be independently put to because we introduce only a bounded and small amount
sleep. This allows for more opportunities to sleep than if of burstiness and a router never enters sleep until it has
one were to require that a router sleep in its entirety (as cleared all bursts it has built up. More precisely, we
the latter is only possible when there is no incoming traf- introduce a buffer interval “B” that controls the tradeoff
fic at any of the incoming interfaces). Correspondingly between savings and performance. An ingress router
our energy savings are with respect to interface cards buffers incoming traffic for up to B ms and, once every
which typically represent a major portion of the overall B ms, forwards buffered traffic in a burst.
consumption of a network device. That said, one could To ensure that bursts created at the ingress are retained
in addition put the route processor and switch fabric to as they traverse through the network, an ingress router
sleep at times when all line cards are asleep. arranges packets within the burst such that all packets
destined for the same egress router are contiguous within
3.2 Approaches and Potential savings the burst (see figure 1).
For interfaces that support wake-on-arrival, one ap- The above “buffer-and-burst” approach (B&B) creates
proach to exploiting sleep states is that of opportunistic alternating periods of contiguous activity and sleep
sleeping in which link interfaces sleep when idle – i.e.,a leading to fewer transitions and amortizing the transition
router is awakened by an incoming (dummy) packet and, penalty δ over multiple packets. This improvement
after forwarding it on, returns to sleep if no subsequent comes at the cost of an added end-to-end delay of up

326 NSDI ’08: 5th USENIX Symposium on Networked Systems Design and Implementation USENIX Association
 {S0 = 2} {S1 = 1} SJ SI 100

Time Asleep (%)

R0 R1 R3 R1 80
20 Idle (bound)
8 3 1 2 60 WoA(Pareto)
15
WoA(CBR)
R2 40
OptB&B
5 7 R4 R2 20
SI + 20 SI + 2 0
R3 R4
SJ + 1 SJ + 15 0 10 20
Average utilization (%)
a) b)

Figure 2: Examples of burst synchronization. Figure 3: Time asleep using optB&B and opportunistic
sleeping and compared to the upper bound (1 − µ).
to B ms. Note that because only ingress routers buffer network utilization. This upper bound is not achievable
traffic, the additional delay due to buffering is only by any algorithm since (unlike optB&B) it does not take
incurred once along the entire ingress-to-egress path. As into account the overhead δ due to sleep/wake transi-
importantly, this approach – unlike opportunistic sleep – tions. Nonetheless it serves to capture the loss in savings
can be used by interfaces that support only timer-driven due to δ and the inability to achieve perfect coordination.
sleep. A router R1 that receives a burst from upstream Any traffic shaping incurs some additional complexity
router R2 at time t1 knows that the next start-of-burst will and hence a valid question is whether we need any
arrive at time t1+B and can hence sleep between bursts. traffic shaping, or whether opportunistic sleeping that
The question then is how significant are the savings does not require shaping is enough? We therefore also
this approach enables for reasonable additional delay? compare optB&B to opportunistic sleeping based on
We note that the best possible savings would occur if wake-on-arrival (WoA). For this naive WoA, we assume
a router received the incoming bursts from all ingress optimistically that an interface knows the precise arrival
routers close in time such that it processes all incoming time of the subsequent packet and returns to sleep only
bursts and returns to sleep thus incurring exactly one for inter-packet arrival periods greater than δ. Because
sleep/wake transition per B ms. This might appear pos- the performance of opportunistic WoA depends greatly
sible by having ingress routers coordinate the times at on the inter-arrival times of packets we evaluate WoA for
which they transmit bursts such that bursts from different two types of traffic: constant bit rate (CBR) and Pareto.
ingresses arrive close in time at intermediate routers. For each of the above bounds, Figure 3 plots the
For example, consider the scenario in Figure 2(a) where percentage of time asleep under increasing utilization
ingress routers R0 and R1 are scheduled to transmit in Abilene. We use a buffer period of B = 10ms
traffic at times 2 and 1 respectively. If instead R1 were and assume a (conservative) transition time δ of 1ms.
to schedule its burst for time 7 instead, then bursts from Comparing the savings from optB&B to the utilization
R0 and R2 would align in time at R2 thus reducing bound, we see that a traffic shaping approach based on
the number of distinct burst times - and sleep-to-wake buffer-and-burst can achieve much of the potential for
transitions - at downstream routers R3 and R4. exploiting sleep. As expected, even at very low utiliza-
Unfortunately, the example in Figure 2(b) suggests this tion, WoA with CBR traffic can rarely sleep; perhaps
is unachievable for general topologies. Here, Si and Sj more surprising is that even with bursty traffic WoA
represent the arrival times of incoming bursts to nodes R3 performs relatively poorly. These results suggest that –
and R1 respectively and we see that the topology makes even assuming hardware WoA – traffic shaping offers a
it impossible to find times Si and Sj that could simulta- significant improvement over opportunistic sleep.
neously align the bursts downstream from R2 and R4.
We thus use a brute-force strategy to evaluate the 3.3 A Practical Algorithm
maximum achievable coordination. For a given topology We consider a very simple buffer-and-burst scheme,
and traffic workload, we consider the start-of-burst time called practB&B, in which each ingress router sends
for traffic from each ingress I to egress J (denoted Sij ) its bursts destined for the various egresses one after the
and perform an exhaustive search of all Sij to find a set other in a single “train of bursts”. At routers close to
of start times that minimizes the number of transitions the ingress this appears as a single burst which then
across all the interfaces in the network. We call this disperses as it traverses through the network.
scheme optB&B. Clearly, such an algorithm is not practB&B bounds the number of bursts (and corre-
practical and we use it merely as an optimistic bound on spondingly the number of transitions) seen by any router
what might be achievable were nodes to coordinate in R in an interval of B ms to at most IR , the total number
shaping traffic under a buffer-and-burst approach. of ingress routers that send traffic through R. In practice,
We compare the sleep time achieved by optB&B to the our results show that the number of bursts seen by R in
upper bound on sleep as given by 1 − µ, where µ is the time Bms is significantly smaller than this bound.

USENIX Association NSDI ’08: 5th USENIX Symposium on Networked Systems Design and Implementation 327
 100 60 130
120

Delay 98%ile (ms)

110

Avg. delay (ms)

Time Asleep (%) 80 50 100
90
40 80
60 70
30 60
B=5ms 50 B=5ms
40 20 40
optB&B B=10ms B=10ms
30
20 10 B=25ms 20 B=25ms
practB&B (pareto) 10
Default Default
practB&B (CBR) 0 0
0 0 10 20 30 0 10 20 30
0 10 20 30 Average utilization (%) Average utilization (%)
Average utilization (%)
(a) Average delay (practB&B) (b) Delay 98%ile
Figure 4: Time asleep using CBR and Pareto traffic.
Figure 5: The impact on delay of practB&B.
practB&B is simple – it requires no inter-router co-
ordination as the time at which bursts are transmitted is We see that for higher B, the delay grows slightly
decided independently by each ingress router. For net- faster with utilization (e.g.,compare the absolute increase
works supporting wake-on-arrival the implementation is in delay for B = 5ms to 25ms) because this situation is
trivial – the only additional feature is the implementation more prone to larger bursts overlapping at intermediate
of buffer-and-burst at the ingress nodes. For networks routers. However this effect is relatively small even
that employ timer-driven sleeping, packet bursts would in a situation combining larger B (25ms) and larger
need to include a marker denoting the end of burst and utilizations (20-30%) and is negligible for smaller B
notifying the router of when it should expect the next and/or more typical utilizations.
burst on that interface. We see that both average and maximum delays in-
crease abruptly beyond network utilizations exceeding
3.4 Evaluation 25%. This occurs when certain links approach full
We evaluate the savings vs. performance tradeoff utilization and queuing delays increase (recall that
achieved by practB&B algorithm and the impact of the utilization on the horizontal axis is the average
equipment and network parameters on the same. network-wide utilization). However this increase occurs
even in the default network scenario and is thus not
Savings vs. performance using practB&B We
caused by practB&B’s traffic shaping.
compare the sleep time achieved by practB&B to that
Finally, our measurements revealed that
achievable by optB&B. In terms of performance, we
practB&B introduced no additional packet loss (rela-
compare the end-to-end packet delay and loss in a net-
tive to the default network scenario) until we approach
work using practB&B to that of a network that never
utilizations that come close to saturating some links. For
sleeps (as today) as this shows the overall performance
example, in a network scenario losses greater than 0.1%
penalty due to our sleep protocols.
occur at 41% utilization without any buffering, they
Figure 4 plots the sleep time with increasing utilization
occur at 38% utilization with B = 10ms, and at 36%
on the Abilene network using a buffering interval
utilization with B = 25. As networks do not typically
B = 10ms. We plot the percentage sleep time under
operate with links close to saturation point, we do not
both CBR and Pareto traffic workloads. We see that even
expect this additional loss to be a problem in practice.
a scheme as simple as practB&B can create and exploit
In summary, the above results suggest that
significant opportunities for sleep and approaches the
practB&B can yield significant savings with a
savings achieved by the significantly more complex
very small (and controllable) impact on network delay
optB&B. As with opportunistic sleeping, we see that
and loss. For example, at a utilization of 10%, a buffering
practB&B’s savings with CBR traffic are lower than for
time of just B = 5ms allows the network to spend over
the more bursty Pareto workloads, but that this reduction
60% of its time in sleep mode for under 5ms added delay.
is significantly smaller in the case of practB&B than
with opportunistic sleeping (recall Figure 3). That Pareto Impact of hardware characteristics We now evaluate
traffic improves savings is to be expected as burstier how the transition time δ affects the performance of
traffic only enhances our bunching strategy. practB&B. Figure 6(a) plots the sleep time achieved by
Figures 5(a) and (b) plot the corresponding average practB&B for a range of transition times and compares
and 98th percentile of the end-to-end delay. As expected, this to the ideal case of having instantaneous transitions.
we see that the additional delay in both cases is propor- As expected, the ability to sleep degrades drastically
tional to the buffering interval B. Note that this is the with increasing δ. This observation holds across various
end-to-end delay, reinforcing that the buffering delay B buffer intervals B as illustrated in Figure 6(b) that plots
is incurred once for the entire end-to-end path. the sleep time achieved at typical utilization (≈5%) for

328 NSDI ’08: 5th USENIX Symposium on Networked Systems Design and Implementation USENIX Association
 100 100 For example, Ethernet links dissipate between 2-4W
90 90
when operating between 100Mbps-1Gbps compared

Time Asleep (%)

80 80
Time Asleep(%)

70
Ideal (δ=0)
70 to 10-20W between 10Gbps[3]. Second, operating at a
60 60
50 δ=10us 50 lower frequency also allows the use of dynamic voltage
B=5ms
40 δ=100us 40 scaling (DVS) that reduces the operating voltage. This
30 δ=1ms 30 B=10ms
20 20 B=25ms allows power to scale cubically, and hence energy con-
δ=10ms
10 10 B=50ms sumption quadratically, with operating frequency[32].
0 0
0 10 20 30 .1 .5 1 2.5 5 10
DVS and frequency scaling are already common in
Average utilization (%) Transition time (ms) microprocessors for these reasons.
We assume the application of these techniques to
(a) Impact of δ (b) Impact of B network links and associated equipment (i.e., linecards,
Figure 6: The impact of hardware constants on sleep time. transceivers). While the use of DVS has been demon-
strated in prototype linecards [22], it is not currently sup-
100 ported in commercial equipment and hence we investi-
gate savings under two different scenarios: (1) equipment
Time Asleep (%)

80
that supports only frequency scaling and (2) equipment
60
that supports both frequency and voltage scaling.
40 δ=0.1ms
δ=1ms Model We assume individual links can switch perfor-
20 mance states independently and with independent rates
0 for transmission and reception on interfaces. Hence the
0 10 20 30 40 savings we obtain apply directly to the consumption
Link Utilization (%) at the links and interface cards of a network element,
although in practice one could also scale the rate of
Figure 7: Time asleep per link
operation of the switch fabric and/or route processor.
We assume that each network interface supports
increasing transition times and different values of B.
N performance states corresponding to link rates
These findings reinforce our intuition that hardware
r1 , . . . , rn (with ri < ri+1 and rn = rmax , the default
support featuring low-power sleep states and quick
maximum link rate), and we investigate the effect that
transitions (preferably < 1ms) between these states are
the granularity and distribution (linear vs. exponential)
essential to effectively save energy.
of these rates has on the potential energy savings.
Impact of network topology We now evaluate The final defining characteristic of performance states
practB&B for the Intel enterprise network. The routing is the transition time, denoted δ, during which packet
structure of the Intel network is strictly hierarchical with transmission is stalled as the link transitions between
a relatively small number of nodes that connect to the successive rates. We explore performance for a range of
wide-area. Because of this we find a wide variation in transition times (δ) from 0.1 to 10 milliseconds.
link utilization – far more than on the Abilene network.
Over 77% of links have utilizations below 1% while Measuring savings and performance As in the case
a small number of links ( 2.5%) can see significantly of sleep we’re interested in solutions that reduce the rate
higher utilizations of between 20-75%. Correspond- at which links operate without significantly affecting
ingly, the opportunity for sleep also varies greatly across performance.
links. This is shown in Figure 7 – each point in the In this section, we use the percentage reduction in
average link rate as an indicative measure of energy
scatter-plot corresponds to a single link and we look at
sleep times for two transition times: 0.1ms and 1ms. We savings and relate this to overall energy savings in
Section 5 where we take into account the power profile
see that the dominant trends in sleep time vs. utilization
remains and that higher δ yields lower savings. of equipment (including whether it supports DVS or
not). In terms of performance, we again measure the
4 Rate-Adaptation in Networks average and 98th percentile of the end-to-end packet
This section explores the use of performance states to delay and packet loss.
reduce network energy consumption. 4.2 An optimal strategy
4.1 Model and Assumptions Our initial interest is to understand the extent to which
Background In general, operating a device at a lower performance states can help if used to best effect. For
frequency can enable dramatic reductions in energy a DVS processor, it has been shown that the most
consumption for two reasons. First, simply operating energy-efficient way to execute C cycles within a given
more slowly offers some fairly substantial savings. time interval T is to maintain a constant clock speed of

USENIX Association NSDI ’08: 5th USENIX Symposium on Networked Systems Design and Implementation 329
 to calibrate practical protocols.
We will evaluate the savings achieved by applying
the above per-link solution at all links in the network
and call this approach link optRA. One issue in
doing so is that the service curves at the different links
are inter-dependent – i.e.,the service curve for a link
l depends in turn on the service curves at other links
(since the latter in turn determine the arrival curve at
l). We address this by applying the per-link optimal
Figure 8: An illustration of delay-constrained service curves algorithm iteratively across all links until the service and
and the service curve minimizing energy.
arrival curves at the different links converge.
C/T [20]. In the context of a network link, this translates 4.3 A practical algorithm
into sending packets at a constant rate equal to the
average arrival rate. However under non-uniform traffic Building on the insight offered by the per-link opti-
this can result in arbitrary delays and hence we instead mal algorithm, we develop a simple approach, called
look for an optimal schedule of rates (i.e.,the set of rates practRA (practical rate adaptation), that seeks to nav-
at which the link should operate at different points in igate the tradeoff between savings and delay constraints.
time) that minimize energy while respecting a specified A practical approach differs from the optimum in that
constraint on the additional delay incurred at the link. (i) it does not have knowledge of future packet arrivals,
More precisely, given a packet arrival curve AC, we (ii) it can only choose among a fixed set of available
look for a service curve SC that minimizes energy con- rates r1 , . . . , rn , and (iii) at every rate switch, it incurs a
sumption, while respecting a given upper bound d on the penalty δ, during which it cannot send packets.
per-packet queuing delay. The delay parameter d thus While knowledge of the future arrival rate is unavail-
serves to tradeoff savings for increased delay. Figure 8(a) able, we can use the history of packet arrivals to predict
shows an example arrival curve and the associated latest- the future arrival rate. We denote this predicted arrival
departure curve (AC+d), which is simply the arrival rate as r̂f and estimate it with an exponentially weighted
curve shifted in time by the delay bound d. To meet the moving average (EWMA) of the measured history of past
delay constraint, the service curve SC must lie within arrivals. Similarly, we can use the current link buffer
the area between the arrival and latest-departure curves. size q and rate ri to estimate the potential queuing delay
In the context of wireless links, [19] proves that if so as to avoid violating the delay constraint.
the energy can be expressed as a convex, monotonically With these substitutes, we define a technique inspired
increasing function of the transmission rate, then the by the per-link optimal algorithm. In practRA, packets
minimal energy service curve is the shortest Euclidean are serviced at a constant rate until we intersect one of
distance in the arrival space (bytes × time) between the the two bounding curves presented earlier (Figure 8):
arrival and shifted arrival curves. the arrival curve (AC), and the latest-departure curve
In the scenario where we assume DVS support, the (AC+d). Thus, we avoid increasing the operating rate ri
energy consumption is a convex, monotonically increas- unless not doing so would violate the delay constraint.
ing function of link rate and thus this result applies to This leads to the following condition for rate increases:
our context as well. Where only frequency scaling is A link operating at rate ri with current queue size q
supported, any service curve between the arrival and δr̂f +q
increases its rate to ri+1 iff ( rqi > d OR ri+1 > d − δ)
shifted-arrival curves would achieve the same energy
savings and therefore the service curve with the shortest The first term checks whether the delay bound d would
Euclidean distance would be optimal in this case too. be violated were we to maintain the current link rate.
In summary, for both frequency-scaling and DVS, the The second constraint ensures that the service curve
shortest distance service curve would achieve the highest does not get too close to the delay-constrained curve
possible energy savings. which would prevent us from attempting a rate increase
Fig. 8 illustrates an example of such a minimal in the future without violating the delay bound. That is,
energy service curve. Intuitively, this is the set of lowest we need to allow enough time for a link that increases
constant rates obeying the delay constraint. Note that its rate to subsequently process packets that arrived
this per-link optimal strategy is not suited to practical during the transition time (estimated by δr̂f ) and its
implementation since it assumes perfect knowledge of already-accumulated queue. Note that we cannot use
the future arrival curve, link rates of infinite granularity delay constraints d smaller than the transition time δ.
and ignores switching overheads. Nonetheless, it is Similarly, the condition under which we allow a rate
useful as an estimate of the potential savings by which decrease is as follows:

330 NSDI ’08: 5th USENIX Symposium on Networked Systems Design and Implementation USENIX Association
 A link operating at rate ri with current queue size q 100
decreases its rate to ri−1 iff q = 0 AND r̂f < ri−1

Rate reduction (%)

80
First, we only attempt to switch to a lower rate when the
queue at the link is empty (q = 0). Intuitively, this corre- 60
sponds to an intersection between the arrival curve and
40 Utilization
the service curve. However, we don’t always switch to a
Expo rates (4)
lower rate ri−1 when a queue empties as doing so would 20
Uniform rates (4)
prevent the algorithm from operating in a (desired) Uniform rates (10)
0
steady state mode with zero queuing delay. Instead, the 0 10 20 30
desirable steady state is one where ri > rf > ri+1 Average utilization (%)
and we want to avoid oscillating between rates (which
would lead to larger average delay). For example, a Figure 9: Average rate of operation. The average is weighted
link that sees a constant arrival rate of 3.5Mbps might by the time spent at a particular rate.
oscillate between 3 and 4Mbps (incurring queuing
delays at 3Mbps), instead of remaining at 4Mbps (with 40 80
low average queuing delay). We thus use the additional 70

Delay 98%ile (ms)

Avg. delay(ms)
condition: r̂f < ri−1 to steer our algorithm toward the 30 60
50
desired steady state and ensure that switching to a lower Exponential rates (4) Exponential rates (4)
20 40
rate does not immediately lead to larger queues. Uniform rates (10)
30
Uniform rates (10)
In addition to the above conditions, we further dis- Uniform rates (4) Uniform rates (4)
10 20
Default Default
courage oscillations by enforcing a minimum time Kδ 10
between consecutive switches. Intuitively, this is because 0 0
0 10 20 30 0 10 20 30
rate switching should not occur on timescales smaller Average utilization (%) Average utilization (%)
than the transition time δ. In our experiments, we found
K = 4 to be a reasonable value for this parameter. (a) Average Delay (b) Delay 98%ile
Note that unlike the link optRA algorithm, the Figure 10: Average and 98th percentile delay achieved by
above decision process does not guarantee that the delay practRA for various available rates
constraints will be met since it is based on estimated
rather than true arrival rates. Similarly, practRA port on the energy savings due to link optRA in the
cannot guarantee that the rates used by the links match following section.) To measure performance, we measure
those used by the link-optimal algorithm. In Section 5, the end-to-end packet delay and loss in a network using
after discussing how power scales with rate, we use practRA to that in a network with no rate adaptation.
simulation to compare our approximate algorithm to the
optimal under realistic network conditions. We leave it Impact of hardware characteristics The main con-
to future work to analytically bound the inaccuracy due stants affecting the performance of practRA are (1)
to our approximations. the granularity and distribution of available rates, and
Finally, we observe that the above rate-adaptation is (2) δ, the time to transition between successive rates. We
simple in that it requires no coordination across different investigate the reduction in rate for three different dis-
nodes, and is amenable to implementation in high-speed tributions of rates r1 , . . . , rn : (i) 10 rates uniformly dis-
equipment. This is because the above decision making tributed between 1Gbps to 10Gbps, (ii) 4 rates uniformly
need not be performed on a per-packet basis. distributed between 1Gbps to 10Gbps and (iii) 4 expo-
nentially distributed rates (10Gbps, 1Gbps, 100Mbps,
4.4 Evaluation 10Mbps). We consider the latter case since physical layer
We evaluate the savings vs. performance tradeoff technologies for these rates already exist making these
achieved by our practical rate-adaptation algorithm and likely candidates for early rate-adaptation technologies.
the impact of equipment and network parameters on the Figure 9 plots the average rate reduction under increas-
same. We first evaluate the percentage reduction in av- ing utilizations, with a per-link delay constraint d = δ
erage link rate achieved by practRA for the Abilene + 2ms, and a transition time δ =1ms. We see that for 10
network. For comparison, we consider the rate reduction uniformly distributed rates, practRA operates links at
due to link optRA and the upper bound on rate reduc- a rate that approaches the average link utilization. With
tion as determined by the average link utilization. In this 4 uniformly distributed rates this reduction drops, but
case, since we found the reduction from link optRA not significantly. However, for exponentially distributed
was virtually indistinguishable from the utilization rates, the algorithm performs poorly, indicating that
bound, we only show the latter here for clarity. (We re- support for uniformly distributed rates is essential.

USENIX Association NSDI ’08: 5th USENIX Symposium on Networked Systems Design and Implementation 331
 Rate Reduction (%) 100 100 100
Uniform (10)

Rate Reduction (%)

Max Delay (ms)
80 80 80 Exponential (4)
Utilization Bound
60 60 60
40 40 40
5% utilization 5% utilization
20 12% utilization 20 13% utilization 20
24% utilization 23% utilization
0 0 0
0.10.5 1 2.5 5 10 0.10.5 1 2.5 5 10 0 10 20 30 40 50
Transition Time (ms) Transition Time (ms) Link Utilization (%)

(a) Rate reduction (time) (b) Max Delay (98th percentile) Figure 12: Rate reduction per link
Figure 11: Impact of Switch Time (δ) for practRA
5 Overall Energy Savings
Figure 10 plots the corresponding average and 98th In the previous sections we evaluated power-
percentile of the end-to-end delay. We see that, for all the management solutions based on their ability to increase
scenarios, the increase in average delay due to practRA sleep times (Section 3), or operate at reduced rates
is very small: ∼ 5ms in the worst case, and less than 2ms (Section 4). In this section, we translate these to overall
for the scenario using 4 uniform rates. The increase in energy savings and hence compare the relative merits
maximum delay is also reasonable: at most 78ms with of rate adaptation vs. sleeping. For this, we develop an
practRA, relative to 69ms with no adaptation. analytical model of power consumption under different
Perhaps surprisingly, the lowest additional delay operating modes. Our model derives from measurements
occurs in the case of 4 uniformly distributed rates. We of existing networking equipment [13, 5, 24]. At the
found this occurs because there are fewer rate transitions, same time, we construct the model to be sufficiently
with corresponding transition delays, in this case. . Fi- general that we may study the potential impact of future,
nally, we found that practRA introduced no additional more energy-efficient, hardware.
packet loss for the range of utilizations considered.
5.1 Power Model
Next, we look at the impact of transition times δ.
Figures 11 (a) and (b) plot the average rate reduction and Recall from section 2 that the total energy consump-
98th percentile of delay under increasing δ for different tion of a network element operating in the absence
network utilizations. For each test, we set the delay of any power-saving modes can be approximated as:
constraint as d = δ + 2ms, and we assume 10 uniform E = pa T a + pi T i .
rates. As would be expected, we see that larger δ lead We start by considering the power consumption when
to reduced savings and higher delay. On the whole we actively processing packets (pa ). Typically, a portion
see that, in this scenario, both savings and performance of this power draw is static in the sense that it does not
remain attractive for transition times as high as ∼ 2ms. depend on the operating frequency (e.g.,refresh power
in memory blocks, leakage currents and so forth) while
In summary, these results suggest that rate adaptation
the dominant portion of power draw does scale with
as implemented by practRA has the potential to offers
operating frequency. Correspondingly, we set:
significant energy savings with little impact on packet
loss or delay. In all our tests, we found practRA to have pa (r) = C + f (r) (2)
minimal effects on the average delay and loss and hence,
from here on, we measure the performance impact only Intuitively, C can be viewed as that portion of power
in terms of the 98th percentile in packet delay. draw that cannot be eliminated through rate adaptation
while f (r) reflects the rate-dependent portion of energy
Impact of network topology We now evaluate consumption. To reflect the relative proportions of C and
practRA applied to the Intel enterprise network. Fig- f (r) we set C to be relatively small – between 0.1 and
ure 12 plots the rate reduction across links - each point in 0.3 of the maximum active power pa (rn ). To study the
the scatter-plot corresponds to a single link, and we look effect of just frequency scaling alone we set f (r) = O(r)
at rate reduction for two rate distribution policies: 10 uni- and set f (r) = O(r3 ) to evaluate dynamic voltage scal-
formly distributed rates and 4 exponentially distributed ing (DVS). In evaluating DVS, we need to consider an
rates. Since these are per-link results, we see significant additional constraint – namely that, in practice, there is
variations in rate reduction for the same utilization, due a minimum rate threshold below which scaling the link
to specifics of traffic across various links. We also notice rate offers no further reduction in voltage. We thus define
that the dominant trend in reduction remains similar to a maximum scaling factor λ and limit our choice of avail-
that seen in the Abilene network (Figure 9). able operating rates to lie between [rn /λ, rn ], for scenar-

332 NSDI ’08: 5th USENIX Symposium on Networked Systems Design and Implementation USENIX Association
 ios that assume voltage scaling. While current transistor expected, increasing the range of supported rates (λ)
technology allows scaling up to factors as high as 5 [32], results in additional energy savings.
current processors typically use λ ∼ 2 and hence we Energy savings from sleeping To model the energy
investigate both values as potential rate scaling limits. savings with sleeping, we need to pin down the relative
Empirical measurements further reveal that the idle magnitudes of the sleep mode power draw (ps ) relative
mode power draw, pi , varies with operating frequency to that when idle (pi ). We do so by introducing a
in a manner similar to the active-mode power draw but parameter γ and set:
with lower absolute value[13]. Correspondingly, we
model the idle-mode power draw as: ps = γpi (rn ) (5)

pi (r) = C + βf (r) (3) where 0.0 ≤ γ ≤ 1.0. While the value of γ will depend
on the hardware characteristics of the network element
Intuitively, the parameter β represents the relative in question, empirical data suggest that sleep mode
magnitudes of routine work incurred even in the absence power is typically a very small fraction of the idle-mode
of packets to the work incurred when actively processing power consumption: ∼ 0.02 for network interfaces [13],
packets. While measurements from existing equipment 0.001 for RFM radios [11], 0.3 for PC cards [11] and
suggest values of β as high as 0.8 for network interface less than 0.1 for DRAM memory [8]. In our evaluation
cards [13] and router linecards [5], we would like to cap- we consider values of γ between 0 and 0.3.
ture the potential for future energy-efficient equipment With this, the energy consumption of an element that
and hence consider a wide range of β between [0.1, 0.9]. spends time Ts in sleep is given by:
Energy savings from rate adaptation With the above
E = pa (rn )Ta + pi (rn )(Ti − Ts ) + ps Ts . (6)
definitions of pa and pi , we can now evaluate the overall
energy savings due to rate adaptation. The total energy Our evaluation from Section 3 estimated Ts for
consumption is now given by: different scenarios. Figure 13(c) plots the corresponding
! overall energy savings for different values of γ for our
pa (rk )Ta (rk ) + pi (rk )Ti (rk ) (4) practB&B algorithm. We assume a transition time
rk
δ =1ms, and a buffering interval B=10ms. Again, our
Our evaluation in Section 4 yields the values of Ta (rk ) results confirm that sleeping offers good overall energy
and Ti (rk ) for different rk and test scenarios, while savings and that, as expected, energy savings are directly
Eqns. 2 and 3 allow us to model pa (rk ) and pi (rk ) for proportional to γ.
different C, β and f (r). We first evaluate the energy
5.2 Comparison: Sleep vs. Rate Adaptation
savings using rate adaptation under frequency scaling
(f (r) = O(r)) and DVS (f (r) = O(r3 )). For these We now compare the savings from sleeping vs. rate
tests, we set C and β to middle-of-the-range values adaptation by varying the two defining axes of our
of 0.2 and 0.5 respectively; we examine the effect of power model: C, the percentage of power that does not
varying C and β in the next section. scale with frequency, and β that determines the relative
Figure 13(a) plots the energy savings for our practical magnitudes of idle to active power draws. We consider
(practRA) and optimal (link optRA) rate adaptation two end-of-the-range values for each: C = 0.1 and
algorithms assuming only frequency scaling. We see C = 0.3 and β = 0.1 and β = 0.8. Combining the
that, in this case, the relative energy savings for the two gives us four test cases that span the spectrum of
different algorithms as well as the impact of the different hardware power profiles:
rate distributions is similar to our previous results (Fig. 9) • C = 0.1 and β = 0.1: captures the case where the
that measured savings in terms of the average reduction static portion of power consumption (that cannot
in link rates. Overall, we see that significant savings are be rate-scaled away) is low and idle-mode power is
possible even in the case of frequency scaling alone. significantly lower than active-mode power.
Figure 13(b) repeats the above test assuming voltage • C = 0.1 and β = 0.8: the static portion of power
scaling for two different values of λ, the maximum rate consumption is low and idle-mode power is almost
scaling factor allowed by DVS. In this case, we see comparable to active-mode power.
that the use of DVS significantly changes the savings • C = 0.3 and β = 0.1: the static portion of power
curve – the more aggressive voltage scaling allows consumption is high; idle-mode power is significantly
for larger savings that can be maintained over a wide lower than that in active mode.
range of utilizations. Moreover, we see that once again • C = 0.3 and β = 0.8: the static portion of power
the savings from our practical algorithm (practRA) consumption is high; idle-mode power is almost
approach those from the optimal algorithm. Finally, as comparable to active-mode power.

USENIX Association NSDI ’08: 5th USENIX Symposium on Networked Systems Design and Implementation 333
 70 80 80

Energy Savings (%)

Energy Savings (%)

Energy Savings(%) 70 70
60
50 60 60
50 50
40
40 40
30 link_optRA link_optRA, λ=2 γ=0
30 30
20 Expo rates (4) 20 link_optRA, λ=5 20 γ=0.1
10 Uniform rates (4) practRA,λ=2 γ=0.2
10 10
Uniform rates (10) practRA,λ=5 γ=0.3
0 0 0
0 10 20 30 0 10 20 30 0 10 20 30
Average utilization (%) Average utilization (%) Average utilization (%)

(a) practRA, freq scaling (b) practRA, DVS (c) practB&B

Figure 13: Total Energy Saving with Sleeping and Rate Adaptation
We evaluate energy savings for each of the above
scenarios for the case where the hardware supports DVS 100 100
Rate Adaptation
and when the hardware only supports frequency scaling.

Energy Savings(%)

Energy Savings(%)
80 80
Rate Adaptation
With DVS: f (r) = O(r3 ) Figures 14 plots the overall 60 60
energy savings for practRA and practB&B for the
40 40
different test scenarios. These tests assume 10 uniformly Sleeping
distributed rates and a sleep power ps = 0.1pi (rn ). 20 20
Sleeping
In each case, for both sleep and rate-adaptation, we 0 0
consider hardware parameters that reflect the best and 0 5 10 15 20 25 30 0 5 10 15 20 25 30

worst case savings for the algorithm in question. For Utilization Utilization
practRA, these parameters are λ (the range for voltage (a) C = 0.1, β = 0.1 (b) C = 0.1, β = 0.8
scaling) and δ (the transition time). For the best-case
100 100
results these are λ = 5 and δ = 0.1ms; for the worst
Energy Savings(%)

Energy Savings(%)
case: λ = 2, δ = 1ms. The parameter for practB&B 80 80
Sleeping Rate Adaptation
is the transition time δ which we set as δ = 0.1ms (best 60 60
case) and δ = 1ms (worst case).
40 40
The conclusion we draw from Figure 14 is that, in Sleeping
each scenario there is a “boundary” utilization below 20 20
which sleeping offers greater savings, and above which Rate Adaptation
0 0
rate adaptation is preferable. Comparing across graphs, 0 5 10 15 20 25 30 0 5 10 15 20 25 30
we see that the boundary utilization depends primarily Utilization Utilization
on the values of C and β, and only secondarily on the (c) C = 0.3, β = 0.1 (d) C = 0.3, β = 0.8
transition time and other hardware parameters of the
Figure 14: Comparison of energy savings between sleep
algorithm. For example, the boundary utilization for
and rate adaptation. Support for dynamic voltage scaling.
C = 0.1 and β = 0.1 varies between approximately
5-11% while at C = 0.3, β = 0.8 this boundary
utilization lies between 4% and 27%. We also evaluated Figure 15 represents the boundary utilization for this
savings under different traffic characteristics (CBR, idealized link as a function of C. In this idealized
Pareto) and found that the burstiness of traffic has a scenario, the dominant parameter is C because the link
more secondary effect on the boundary utilization. is never idle and therefore β has only a small, indirect
For further insight on what determines the boundary effect on ps . The gray zone in the figure represents the
utilization, we consider the scenario of a single idealized spread in boundary utilization obtained by varying β
link. The sleep-mode energy consumption of such an between 0.1 and 0.9.
idealized link can be viewed as:
With frequency scaling alone: f (r) = O(r) Fig-
Esleep = pa (rmax )µT + ps (1 − µ)T (7) ures 16 plots the overall energy savings for practRA
and practB&B for the different test scenarios in the
Similarly, the idealized link with rate adaptation is one more pessimistic scenario where voltage scaling is not
that runs with an average rate of µrmax for an energy supported. Due to lack of space, we only plot the com-
consumption of: parison for the first two test scenarios where C = 0.1;
at C = 0.3, the savings show a similar scaling trend but
Erate = pa (µrmax )T (8) with significantly poorer performance for rate-adaptation

334 NSDI ’08: 5th USENIX Symposium on Networked Systems Design and Implementation USENIX Association
 1
0.9 100 100
0.8 Sleeping is better

Energy Savings(%)

Energy Savings(%)
0.7 80 80 Rate Adaptation

C / pa(rn )
0.6
0.5 60 60
0.4 Sleeping
0.3
0.2 Rate Adaptation 40 40 Sleeping
0.1 is better
0 20 20
0 20 40 60 80 100 Rate Adaptation
Utilization 0 0
0 5 10 15 20 25 30 0 5 10 15 20 25 30

Figure 15: Sleeping vs. rate-adaptation Utilization Utilization

and hence add little additional information. (a) C = 0.1, β = 0.1 (b) C = 0.1, β = 0.8
The primary observation is that the savings from rate
Figure 16: Energy savings of sleep vs. rate adaptation,
adaptation are significantly lower than in the previous
β = 0.1, frequency scaling alone.
case with DVS and, in this case, sleeping outperforms
rate adaptation more frequently. We also see that – and OS techniques to extend battery lifetimes in
unlike the DVS case – network utilization impacts mobiles [10, 30].
energy savings in a similar manner for both sleeping and
Perhaps the first to draw attention to the problem
rate-adaptation (i.e., the overall “slope” of the savings-
of saving overall energy in the network was an early
vs-utilization curves is similar with both sleeping and
position paper by Gupta et al. [12]. They use data from
rate-adaptation while they were dramatically different
the US Department of Commerce to detail the growth
with DVS – see Fig. 14).
in network energy consumption and argue the case for
Once again, we obtain insight on this by studying the
energy-saving network protocols, including the possi-
the highly simplified case of a single idealized link. For
bility of wake-on-arrival in wired routers. In follow-on
this idealized scenario with f (r) = O(r), we find that
work they evaluate the application of opportunistic
the boundary condition that determines whether to use
sleeping in a campus LAN environment [21, 11].
sleep or rate adaptation is in fact independent of network
utilization. Instead, one can show that sleep is superior Other recent work looks at powering-down redundant
to rate-adaptation if the following inequality holds: access points (APs) in enterprise wireless networks [17].
The authors propose that a central server collect AP
γβ connectivity and utilization information to determine
c> (9)
1 − γ(1 − β) which APs can be safely powered down. This approach
is less applicable to wired networks that exhibit much
Otherwise, rate adaptation is superior. less redundancy.
In practice, network utilization does play a role
Sleeping has also been explored in the context of
(as our results clearly indicate) because the various
802.11 to save client power, e.g., see [2]. The 802.11
practical constraints due to delay bounds and transition
standard itself includes two schemes (Power-Save Poll
times prevent our algorithms from fully exploiting all
and Automatic Power Save Delivery) by which access
opportunities to sleep or change rates.
points may buffer packets so that clients may sleep for
In summary, we find that both sleeping and rate-
short intervals. In some sense, our proposal for bunching
adaptation are useful, with the tradeoff between them
traffic to improve sleep opportunities can be viewed as
depending primarily on the power profile of hardware ca-
extending this idea deep into the network.
pabilities and network utilization. Results such as those
presented here can guide operators in deciding how to Finally, the IEEE Energy Efficient Ethernet Task
best run their networks. For example, an operator might Force has recently started to explore both sleeping and
choose to run the network with rate adaptation during the rate adaptation for energy savings. Some initial studies
day and sleeping at night based on where the boundary consider individual links and are based on synthetic
utilization intersects diurnal behavior, or identify com- traffic and infinite buffers [4].
ponents of the network with consistently low (or high) In the domain of sensor networks, there have been
utilization to be run with sleeping (or rate-adaptation). numerous efforts to design energy efficient protocols.
Approaches investigated include putting nodes to sleep
6 Related Work using TDMA-like techniques to coordinate transmis-
There is a large body of work on power management in sion and idle times (e.g., FPS [14]), and distributed
contexts complementary to ours. This includes power algorithms for sleeping (e.g.,S-MAC [28]). This context
provisioning and load balancing in data centers[6, 9], differs from ours in many ways.

USENIX Association NSDI ’08: 5th USENIX Symposium on Networked Systems Design and Implementation 335
 7 Conclusion [14] B. Hohlt, L. Doherty, et al. Flexible Power Scheduling
for Sensor Networks. In IEEE and ACM Third Interna-
We have argued that power management states that slow tional Symposium on Information Processing in Sensor
down links and put components to sleep stand to save Networks (IPSN). April 2004.
much of the present energy expenditure of networks. [15] IEEE 802.3 Energy Efficient Ethernet Study Group.
At a high-level, this is apparent from the facts that http://grouper.ieee.org/groups/802/3/eee study/.
while network energy consumption is growing networks [16] Ipmon Sprint. The Applied Research Group.
http://ipmon.sprint.com/.
continue to operate at low average utilizations. We [17] A. Jardosh et al. Towards an Energy-Star WLAN
present the design and evaluation of simple power man- Infrastructure. In HOTMOBILE. 2007.
agement algorithms that exploit these states for energy [18] S. Kandula, D. Katabi, et al. Walking the Tightrope:
conservation and show that – with the right hardware Responsive Yet Stable Traffic Engineering. In ACM
support – there is the potential for saving much energy SIGCOMM 2005.
with a small and bounded impact on performance, e.g., a [19] M. Lin and Y. Ganjali. Power-Efficient Rate Scheduling
in Wireless Links Using Computational Geometric
few milliseconds of delay. We hope these preliminary Algorithms. In IWCMC. 2006.
results will encourage the development of hardware [20] J. Lorch. Operating Systems Techniques for Reducing
support for power saving as well as algorithms that use Processor Energy Consumption. In Ph.D. Thesis,
them more effectively to realize greater savings. University of California, Berkeley. 1994.
[21] M. Gupta and S. Singh. Dynamic Ethernet Link Shut-
Aknowledgments down for Energy Conservation on Ethernet Links. In
IEEE ICC. 2007.
We thank Robert Hays, Bob Grow, Bruce Nordman, [22] M. Mandviwalla and N.-F. Tzeng. Energy-Efficient
Rabin Patra and Ioan Bejenaru for their suggestions. We Scheme for Multiprocessor-Based Router Linecards. In
also thank the anonymous reviewers and our shepherd IEEE SAINT. 2006.
Jon Crowcroft for their useful feedback. [23] E. Miranda and L. McGarry. Power/Thermal Impact
of Networking Computing. In Cisco System Research
References Symposium, August, 2006.
[24] S. Nedevschi. Reducing Network Energy Con-
[1] Power and Thermal Management in the Intel Core Duo
sumption via Sleeping and Rate-adaptation,
Processor. In Intel Technology Review, Volume 10, Issue
http://www.ieee802.org/3/az/public/jan08/nedevschi 01
2, Section 4. 2006.
0108.pdf. In IEEE 802.3az Task Force Group Meeting.
[2] Y. Agarwal, R. Chandra, et al. Wireless Wakeups 2008.
Revisited: Energy Management for VoIP over Wi-Fi
[25] B. Nordman. Energy Efficient Ethernet, Outstanding
Smartphones. In ACM MobiSys. 2007.
Questions. 2007.
[3] C. Gunaratne, K. Christensen and B. Nordman. Man-
[26] K. W. Roth, F. Goldstein, et al. Energy Consumption by
aging Energy Consumption Costs in Desktop PCs and
Office and Telecommunications Equipment in Commer-
LAN Switches with Proxying, Split TCP Connections,
cial Buildings - Volume I: Energy Consumption Baseline.
and Scaling of Link Speed. In International Journal of
Tech. Rep. 72895-00, Arthur D. Little, Inc, Jan. 2002.
Network Management. October 2005.
[27] The Abilene Observatory. http://abilene.internet2.edu/.
[4] C. Gunaratne, K. Christensen et al. Reducing the Energy
observatory.
Consumption of Ethernet with Adaptive Link Rate
(ALR). In IEEE Transactions on Computers. April 2008. [28] W. Ye, J. Heidemann, et al. An Energy-efficient MAC
Protocol for Wireless Sensor Networks. In IEEE
[5] J. Chabarek, J. Sommers, et al. Power Awareness in
INFOCOM. 2002.
Network Design and Routing. In IEEE INFOCOM. 2008.
[29] Yin Zhang’s AbileneTM,http://www.cs.utexas.edu/ yzhang.
[6] J. S. Chase, D. C. Anderson, et al. Managing Energy and
/research/AbileneTM.
Server Resources in Hosting Centers. In ACM SOSP.
2001. [30] W. Yuan and K. Nahrstedt. Energy-efficient Soft Real-
time CPU Scheduling for Mobile Multimedia Systems.
[7] Cisco Systems. NetFlow Services and Applications.
In ACM SOSP. 2003.
White Paper, 2000.
[31] M. Yuksel, B. Sikdar, et al. Workload generation for ns
[8] X. Fan, C. S. Ellis, et al. Memory Controller Policies for
Simulations of Wide Area Networks and the Internet.
DRAM Power Management. In International Symposium
In Communication Networks and Distributed Systems
on Low Power Electronics and Design. 2003.
Modeling and Simulation Conference. 2000.
[9] X. Fan, W.-D. Weber, et al. Power Provisioning for a
[32] B. Zhai, D. Blaauw, et al. Theoretical and Practical
Warehouse-Sized Computer. In ACM ISCA. 2007.
Limits of Dynamic Voltage Scaling. In DAC. 2004.
[10] J. Flinn and M. Satyanarayanan. Energy-aware Adapta-
tion for Mobile Applications. In ACM SOSP. 1999. Notes
1
[11] M. Gupta, S. Grover, et al. A Feasibility Study for Power In reality the energy savings using rate-adaptation will
Management in LAN Switches. In ICNP. 2004. depend on the distribution of operating rates over time and the
[12] M. Gupta and S. Singh. Greening of the Internet. In corresponding power consumption at each rate. For simplicity,
ACM SIGCOMM, Karlsruhe, Germany. August 2003. we initially use the average rate of operation as an indirect
[13] R. Hays. Active/Idle Toggling with Low-Power Idle, measure of savings in Section 4 and then consider the complete
http://www.ieee802.org/3/az/public/jan08/hays 01 0108 distribution of operating rates in Section 5 when we compute
.pdf. In IEEE 802.3az Task Force Group Meeting. 2008. energy savings.

336 NSDI ’08: 5th USENIX Symposium on Networked Systems Design and Implementation USENIX Association