A Computer Scientist

Looks at the Energy Problem

Randy H. Katz
University of California, Berkeley

UCSB Computer Engineering Seminar

8 March 2010

“Energy permits things to exist; information, to behave purposefully.”

W. Ware, 1997
 Agenda
•  The Big Picture
•  IT as an Energy Consumer
•  IT as an Efficiency Enabler
•  Summary and Conclusions

2
 Energy “Spaghetti” Chart
Quads (1015 BTUs)

10-8-2008 3
Electricity is the Heart of the
Energy Economy

4
Evolution of the Grid

5
 The Big Switch:
Clouds + Smart Grids
Computing as a Utility

Energy
Efficient
Computing

Embedded
Intelligence in
Civilian
Infrastructures

Large-scale industrialization Computing in the Utility 6

of computing
 Agenda
•  The Big Picture
•  IT as an Energy Consumer
•  IT as an Efficiency Enabler
•  Summary and Conclusions

7
 2020 IT Carbon Footprint

820m tons CO2

360m tons CO2

2007 Worldwide IT
carbon footprint:
2% = 830 m tons CO2
Comparable to the
global aviation
industry

Expected to grow 260m tons CO2

to 4% by 2020

8
 Energy Proportional
Computing
“The Case for
Energy-Proportional It is surprisingly hard
Computing,” to achieve high levels
Luiz André Barroso, of utilization of typical
Urs Hölzle, servers (and your home
IEEE Computer PC or laptop is even
December 2007 worse)

Figure 1. Average CPU utilization of more than 5,000 servers during a six-month period. Servers
are rarely completely idle and seldom operate near their maximum utilization, instead operating 9
most of the time at between 10 and 50 percent of their maximum
 Energy Proportional
Computing
“The Case for
Energy-Proportional
Computing,”
Luiz André Barroso,
Urs Hölzle,
IEEE Computer
December 2007 Doing nothing well …
NOT!

Energy Efficiency =
Utilization/Power

Figure 2. Server power usage and energy efficiency at varying utilization levels, from idle to
peak performance. Even an energy-efficient server still consumes about half its full power
when doing virtually no work.
10
 Energy Proportional
Computing
“The Case for
Energy-Proportional
Computing,”
Luiz André Barroso,
Urs Hölzle,
IEEE Computer Doing nothing Design for
December 2007 VERY well wide dynamic
power range and
active low power
modes

Energy Efficiency =
Utilization/Power

Figure 4. Power usage and energy efficiency in a more energy-proportional server. This
server has a power efficiency of more than 80 percent of its peak value for utilizations of
30 percent and above, with efficiency remaining above 50 percent for utilization levels as
low as 10 percent. 11
Internet Datacenters

12
 “Doing Nothing Well”:
View from the Switch
•  Internet’s energy consumption is significant
(~$24 billion per year)
–  Switches consume ~10-15% of this
•  Design energy-efficient network
equipment
–  Energy Efficient Ethernet (EEE), Energy Star
•  Reduce energy consumption of network
switches
–  Idle power consumption is high
–  Traffic patterns are bursty and diurnal

Ganesh Ananthanarayanan, David Zats

 Shadow Ports

•  Ports consume ~40% of Switch energy

•  Shadow port backs multiple physical ports
•  Predict traffic at a port using past observations
•  Energy savings of ~30%, with <1% reduction in throughput
Ganesh Ananthanarayanan, David Zats
 Shadow Ports Evaluation
NetFPGA Implementation

NetFPGA Standard NIC vs. Shadow Port

15
Ganesh Ananthanarayanan, David Zats
 Elastic Switch
•  Diurnal variations in network traffic
–  e.g., number of requests to a web service, number of
users in an enterprise are fewer during nights
Can  we  design  a  switch  to  behave  as  a  slider?  

•  Adaptive switch that scales its performance and

energy needs according to the traffic load
Ganesh Ananthanarayanan, David Zats
 Modularized Design:
Sub-switches
•  Construct a modularized elastic switch out of
smaller sub-switches
•  Elasticity: Turn off sub-switches
•  Thin external layer as they are always on

Input   Intermediate   Output  

Ganesh Ananthanarayanan, David Zats

 Containerized Datacenter
Mechanical-Electrical Design
Google’s
Containerized
Datacenter

Microsoft
Chicago
Datacenter 18
Microsoft’s Chicago
Modular Datacenter

19
 Agenda
•  The Big Picture
•  IT as an Energy Consumer
•  IT as an Efficiency Enabler
•  Summary and Conclusions

20
 What if the Energy Infrastructure
were Designed like the Internet?
•  Information Age approach to Machine Age
infrastructure: bits follow current flow
–  Break synchronization between sources and loads:
energy storage/buffering is key
–  Lower cost, more incremental deployment, suitable for
developing economies
–  Enhanced reliability and resilience to wide-area
outages, such as after natural disasters
•  Exploit information to match sources to loads,
manage buffers, integrate renewables, signal
demand response, and take advantage of locality
•  LoCal Project
–  David Culler (Sensor Nets), Seth Sanders (Energy
21
Storage), Eric Brewer (Tech for Developing World)
 Prototype New Architecture
as Overlay to the Grid
Intelligent Energy Network
Source
IPS

energy
subnet
Load IPS
Intelligent
Power Switch

Generation
Transmission
Distribution
Load
Conventional Electric Grid
Conventional Internet
22
 Building Block of
the LoCal Energy Internet
Host Load

Intelligent
Power Power Switch Energy
Generation Storage
(IPS)
energy flows
PowerComm
Interface information flows

Energy Network

•  Locations within the Intelligent Grid for monitoring, load

and supply modeling, inter-IPS communications 23
 Multi-Scale Energy Internet
Price profile Load profile

w w CT
$ IPS
now
now

IPS AHU
comm
Internet Bldg IPS Power

power
IPS Energy
proportional
kernel
IPS Network
Grid IPS Datacenter

Actual load IPS IPS

IPS M/R
Chill Energy
w
Net IPS
now

Power Quality-
proportional Adaptive
service
Service
manager

24
Buildings as Loads
Buildings as a Load

HVAC & Plug Loads

Lighting

HVAC / CRU / PDU support

Servers / Clusters

26
Peak Reduction and Shifting

How can we make

this more fine
grained, exploting
intermittent sources?
27
 Datacenter as a
Supply-following Load
•  Datacenters/Machine Rooms as instance of a
sculptable load in a LoCal Energy Network
–  “Doing Nothing Well”: Better processing and
network node designs that exhibit more agile
transitions into lower energy states during idle
times
–  Scheduling: identify workloads time shifted to use
fewer resources at higher/more efficient levels of
utilization
–  Building-Machine Room Co-Design: Co-
management of building facilities (e.g., power,
cooling) given usage patterns
 “Supply-following”
Building Loads
Building
Loads

Datacenter
Loads
Non-dispatch

Dispatch Monitor – Model – Manage

 “Supply-following”
Building Loads
Available Energy

Wind
Supply

Dispatchable
Supply
Time

•  Reduce Demand for Conventional Supply

 “Supply-following”
Building Loads
Reduce the
Available Energy
Average

Wind
Supply

Defer to later
Perform sooner
Dispatchable
Supply
Time
•  “Make hay while the Sun shines”: Do more when
supply is available, defer when it is not
•  Workload awareness is essential
 Datacenter as a
Supply-following Load
Energy Supply Information
Building/Facility Manager
Tasks
Energy Aware
Workload Model/ Cluster
SLAs Workload B
Predictor Manager
Scheduler

Energy Consumption
Application Resource Footprint

•  Workload modeling and •  Low power processing and

scheduling network platform
–  When: peak shifting/filling –  Processing: Agility in
valleys of processing load entering low power states
–  Where: energy implications when idle
of topology, replicas, multi- –  Facility: Couple cooling with
DC distribution predicted DC peaks, e.g., in
advance of need
 Hadoop Performance
Modeling
Workload SYSTEM Behavior

Model

•  Which jobs should I schedule •  Approach: find relationships

together to avoid resource between workload and
contention? performance
•  What is the optimal number of •  Generate the model via
nodes on which to run? Machine Learning
•  Given observed behavior of a –  Kernel Canonical Correlation
job run at small scale, how will Analysis (KCCA)
the job behave when scaled •  Performance traces collected
up? from Facebook Extract-
Analyze-Load Hadoop jobs on
600 node cluster
Archana Ganapathi, Yanpei Chen
 KCCA
X Job Descriptor Performance Y
Vector Hadoop Vector

•  Job config parameters •  Map Time

‒  Number of Maps •  Reduce Time
‒  Number of Reduces
•  Total Time
•  Data characteristics
‒  Map Input Bytes
•  Map Output Bytes
‒  Locality

Archana Ganapathi, Yanpei Chen

 Predicting Job Performance

Archana Ganapathi, Yanpei Chen

 Model Successfully
Actual time (ms) Predicts Finishing Time

Predicted time (ms)

Archana Ganapathi, Yanpei Chen
Energy Efficient Cluster
Design (aka “LoClu”)

37
 Agenda
•  The Big Picture
•  IT as an Energy Consumer
•  IT as an Efficiency Enabler
•  Summary and Conclusions

38
 Summary and Conclusions
•  Energy Consumption in IT Equipment
–  Energy Proportional Computing and
“Doing Nothing Well”
–  Management of Processor, Memory, I/O,
Network to maximize performance subject to
power constraints
–  Internet Datacenters and Containerized
Datacenters: New packaging opportunities for
better optimization of computing +
communicating + power + mechanical
39
 Summary and Conclusions
•  Evolution of the Grid to the “Smart Grid”
–  Intelligence extends from supply to loads
–  From centralized to distributed/peer-to-peer
•  LoCal: a scalable energy network
–  Prototype as information overlay on the
existing Grid
–  Scalable IPS building blocks as points of
monitoring/modeling/managing of load, supply,
and storage
–  Energy matching and supply-following loads at
40
small, medium, large scale
 Thank You!

“We’re at the beginning of the information utility.

The past is big monolithic buildings. The future
looks more like a substation—the data center
represents the information substation of tomorrow.”
Mike Manos, Microsoft GM Datacenter Services 41