1

Distributed Demand Response and User Adaptation

in Smart Grids
Zhong Fan, Senior Member, IEEE

Abstract—This paper proposes a distributed framework for and operational reliability, there are also challenges ahead in
demand response and user adaptation in smart grid networks. implementing DR under smart grid and market paradigms.
In particular, we borrow the concept of congestion pricing in There are a few papers recently on smart grid DR using load
Internet traffic control and show that pricing information is very
arXiv:1007.5425v1 [cs.NI] 30 Jul 2010

useful to regulate user demand and hence balance network load. scheduling. In [3], user preferences are taken into account with
User preference is modeled as a willingness to pay parameter the concept of discomfort level and an optimization problem
which can be seen as an indicator of differential quality of service. is formulated to balance the load and minimize the user in-
Both analysis and simulation results are presented to demonstrate convenience caused by demand scheduling. Several ideas from
the dynamics and convergence behavior of the algorithm. the distributed computing area such as makespan have been
Index Terms—Smart grid, demand response, pricing, utility. introduced to energy consumption optimization. Similarly, in
[4], an energy consumption scheduling problem is established
to minimize the overall energy cost. Techniques similar to
I. I NTRODUCTION those used in wireless network resource allocation have been
applied here to solve the underlying optimization problem. In
A smart grid is an intelligent electricity network that in-
both works, the user demands are known beforehand and the
tegrates the actions of all users connected to it and makes
optimization problem is solved in numerical iterations.
use of advanced information, control, and communication
In this paper, we consider a fully distributed system where
technologies to save energy, reduce cost and increase relia-
the only information available to the end users is the current
bility and transparency. Recently, many countries have started
price which is dependent on the overall system load. Based on
massive efforts on research and developing smart grids. For
this information, the users try to adapt their demands so as to
example, the smart grid is a vital component of President
maximize their own utility. There is no central control entity.
Obama’s comprehensive energy plan: the American Recovery
Inspired by the well-established work on congestion pricing in
and Investment Act includes 11 billion USD in investments
IP networks, we propose a simple adaptation strategy based on
to “jump start the transformation to a bigger, better, smarter
price feedbacks and show that it is very effective in achieving
grid”.
demand response.
In electricity grids, demand response (DR) is a mechanism
The rest of the paper is organized as follows. Section 2
for achieving energy efficiency through managing customer
introduces our DR model and the adaptation algorithm. We
consumption of electricity in response to supply conditions,
present some simulation results in Section 3. Conclusions and
e.g., having end users reduce their demand at critical times
future work are presented in Section 4.
or in response to market prices. In the future smart grid, the
two way communications between energy provider and end
users enabled by advanced communication infrastructure (e.g., II. D EMAND RESPONSE MODEL
wireless sensor networks and power line communications) and A. Congestion pricing background
protocols will greatly enhance demand response capabilities
In this paper we propose to apply the principle of congestion
of the whole system. In contrast to the current simple time-
pricing in IP networks to demand response in the electricity
of-use (TOU) pricing (e.g. peak time vs. off-peak time), it
grid. In their seminal paper [5], Kelly et al. have proposed
can be envisaged that a more dynamic, real-time adaptation
the proportionally fair pricing (PFP) scheme in which each
to market prices would not only enable consumers to save
user declares a price per unit time that he is willing to pay
more energy and money, as well as manage their usage pref-
for his flow. In that sense the network capacity is shared
erences more flexibly, but also facilitate the grid move closer
among the flows of all users in proportion to the prices paid
towards its optimal operating point. For a recent overview of
by the users. It has been shown in [5] that in a weighted
challenges and issues of enabling communication technologies
proportionally fair system where the weights are the prices the
in this area, please refer to [1]. The authors of [2] argue that
users pay per unit time, when each user chooses the price that
demand response and distributed energy storage can be seen as
maximizes the utility she gets from the network, the system
distributed energy resources and are main drivers of smart grid.
converges to a state where the total utility of the network is
While DR can help the industry to achieve market efficiency
maximized. In other words, in an ideal environment, the PFP
proposal is able to decentralize the global optimal allocation
Z. Fan is with Toshiba Research Europe Limited, Telecommunications
Research Laboratory, 32 Queen Square, Bristol, BS1 4ND, UK. e-mail: of congestible resources. Another important result of [5] is
[email protected] that rate control (such as TCP) based on additive increase and
 2

multiplicative decrease achieves proportional fairness. It has Here the adaptation of x can be seen as an action of load
been proved in [6] that the decentralized congestion control scheduling: for example, re-scheduling a dryer operation from
mechanism is stable even under arbitrary network topologies time slot n = 1 to slot n = 3 leads to x(1) reduced by
and heterogeneous round trip times (feedback delays). 2.50 kW per hour and x(3) increased by 2.50 kW per hour.
In Kelly’s approach, the philosophy is that users who are Secondly, how to characterize user preference is an open issue.
willing to pay more should get more. As the network makes For instance, a user may prefer his washing done at 6pm which
no explicit promises to the user, there is no need for over is a typical peak time. To some extent, this preference can be
provisioning in the core of the network. One implementation reflected in the WTP parameter w in (4): when a user is willing
of PFP is to give control to end systems (users). In this scheme, to pay more, he/she can have a higher demand. However, the
the TCP algorithm is modified to incorporate congestion prices delay (or waiting time) incurred due to rescheduling is not
by means of protocols like explicit congestion notification considered in this model. Thirdly, as pointed out in [5], log-
(ECN) [7]. Upon receiving feedback signals, f (t), which are arithmic utility functions lead to proportional fairness. There
related to shadow prices (in terms of packet marks), the users are other types of utility functions available corresponding to
β
are free to react as they choose, but will incur charges when different fairness criteria, e.g. ui (x) = wi x β−1 , β < 1 as
resources are congested. An end system can adjust its rate x(t) proposed in [8]. How to choose a most suitable utility function
using a willingness to pay (WTP) parameter w: in DR applications is an open issue, e.g. how to factor in the
waiting time and user discomfort level [3].
x(t + 1) = x(t) + α(w − f (t)), (1)
User i adapts its demand according to the following equa-
where α affects the rate of convergence of the algorithm. tion:
In [8], explicit prices instead of marks are fed back to the xi (n + 1) = xi (n) + αi (wi − xi (n)p(n)), (6)
end users as incentives and users adapt their rates accordingly.
where αi is a parameter that controls the rate of convergence
It has been shown that the system converges to an optimal
of the algorithm. It is clear that the user adjusts her demand
allocation of bandwidth: the users’ price predictions converge
according to the price information (p(n)) and her own will-
to the actual price and their bandwidth allocations converge
ingness to pay preference (w).
to levels which equalize their marginal utility of bandwidth to
To show that the above adaptation converges to the user
the price of bandwidth.
optimum, let us assume that the equilibrium price is q. Then
by solving u′i (xi (n)) = q, we have the optimal demand x∗i as
B. The DR model and user adaptation wi
We consider a discrete time slot system where N users x∗i = . (7)
q
share some energy resources. In each time slot n, user i has a
demand of xi (n) (e.g. hourly energy consumption if the time Given (6), the error of demand estimate, ei (n + 1), is given
granularity is one hour). The unit price of energy in a time by
slot is a function of the aggregate demand: wi
ei (n + 1) = xi (n + 1) − x∗i = xi (n) + αi (wi − xi (n)q) − .
q
XN
(8)
p(n) = f ( xi (n)). (2) Then it follows that
i=1
wi
The price function (spot market price) can be of the following ei (n + 1) = (1 − αi q)(xi (n) − ) = (1 − αi q)ei (n). (9)
q
form [3][9]:
x Therefore ei (n) is a geometric series, and when |1 − αi q| <
f (x) = a( )k , (3)
C 1, limn→∞ ei (n) = 0. This has established that with properly
where a and k are constants, and C is the capacity of the chosen αi , the adaptation will converge to the optimum.
market. Following [5], it is also straightforward to establish the global
Each user i is associated with a utility function ui (xi (n)) stability of the algorithm in a differential equation form (10)
in time slot n, which is a concave, non-decreasing function of using an appropriate Lyapunov function.
its demand. A typical logarithmic utility function is given by d
[5]: xi (t) = αi (wi − xi (t)p(t)). (10)
dt
ui (x) = wi log x, (4)
where wi is the willingness to pay parameter. Hence user i C. Implementation considerations
chooses its demand xi (n) to seek to maximize In a residential energy management scenario, we envisage
ui (xi (n)) − xi (n)p(n). (5) that each user in our model is represented by an entity or
software agent called home energy manager (HEM) at a
We would like to elaborate on a few assumptions made consumer’s home. Appliances in the home are equipped with
in the above model. Firstly, in our model the demand x is a smart meters, and they communicate with HEM via low power
continuous variable, which may not be realistic in practice. For wireless such as ZigBee. HEM is further connected to the grid
example, in real life, the daily usages of a washing machine (supplier) via either wired or wireless links. Based on the price
and a dryer are (fixed as) 1.49 kWh and 2.50 kWh respectively. information it receives, HEM calculates demand in the next
 3

0.16

0.14

0.12

0.1

demand
0.08

0.06

0.04

0.02

0
0 10 20 30 40 50 60 70 80 90 100
time slot

Fig. 2. Simulation III-A: demand adaptation of 10 users

1.8

Fig. 1. Distributed demand response 1.6

1.4

time slot and distributes it to different appliances. The overall 1.2

architecture is shown in Figure 1.

1
We note that some appliances like refrigerator and heating price

have hard consumption scheduling requirements, while others 0.8

such as washing machine have soft requirements [4]. When 0.6

HEM has to shift the demand to another time slot, it may
apply only to soft appliances. For example, HEM obtains 0.4

x(n + 1) based on (6) with WTP parameter w. If it can 0.2

satisfy the demand from the hard appliances (denoted by h), it

0
can re-schedule the demand from some of the soft appliances 0 10 20 30 40 50 60 70 80 90 100
time slot
(denoted by s) so that x(n + 1) = h(n + 1) + s(n + 1).
On the other hand, if it cannot meet the demand from the Fig. 3. Simulation III-A: price evolution
hard appliances, HEM may have to increase w and recalculate
x(n + 1).
Summing over i on both sides of (11), it is easy to verify that
the price at equilibrium is
III. S IMULATION RESULTS
XN
k
In this section, we use simulations to study the behavior and p=( wi (n)) k+1 . (12)
dynamics of the proposed algorithm. There are N = 10 users i=1
and without loss of generality we assume that the capacity C In this case, p = 1.42 as shown in Figure 3.
is 1. For the price function (3), a = 1, k = 4.
B. The effect of α
A. Basic simulation In this simulation experiment we study the effect of α on
system performance. Figure 4 and Figure 5 depict the demand
Here all the users initiate their demands at 0.02, and their and price evolution versus time respectively for α = 0.17.
willingness to pay parameters range from 0.11 (user 1) to 0.20 Compared with Figure 2 and Figure 3, it can be seen that with
(user 10). All the users have the same adaptation parameter a larger α, it takes much longer to converge. Therefore α is
α of 0.1. Figure 2 shows the demand changes with time for an important system parameter that controls the convergence
10 users. After a short transient period, each user demand speed of the process.
converges to a stable value (determined by different w values).
It is also evident that w is a crucial factor in determining how C. Heterogeneous initial demands
aggressive a user should be responding to the price signals.
In this simulation experiment we study the effect of hetero-
Figure 3 clearly shows that the price converges to the optimal
geneity of initial demands, i.e., ten users start with demands
value. When the system reaches its equilibrium (assuming a =
ranging from 0.01 to 0.10 respectively. The results are shown
1, C = 1), we have
in Figure 6 and Figure 7. We observe that different initial
wi conditions do not affect the system stability and convergence
xi = PN . (11)
( i=1 xi (n))k to equilibrium.
 4

1.8

0.18 1.6

0.16 1.4

0.14 1.2

0.12 1

price
0.1 0.8
demand

0.08 0.6

0.06 0.4

0.04 0.2

0.02 0
0 5 10 15 20 25 30 35 40 45 50
time slot
0
0 50 100 150 200 250 300 350 400
time slot
Fig. 7. Simulation III-C: price evolution
Fig. 4. Simulation III-B: demand adaptation of 10 users
0.25

0.2

3 0.15

0.1
2.5 demand

0.05

0
price

1.5
−0.05

1
−0.1
0 10 20 30 40 50 60 70 80 90 100
time slot

0.5
Fig. 8. Simulation III-D: demand adaptation of 10 users

0
0 50 100 150 200 250 300 350 400
time slot
D. Heterogeneous initial demands and adaptation rates
Fig. 5. Simulation III-B: price evolution
In addition to heterogeneous initial demands as in last
simulation, here users also have different adaptation rates αi :
ranging from 0.11 to 0.20. The results are shown in Figure 8
and Figure 9, where we can see that the system still converges
0.16 to the equilibrium.
0.14

0.12
E. Time-varying w

0.1
To model the situation where users change their WTP w on-
the-fly to accommodate their energy needs, we change wi ’s at
demand

0.08 time slot 100 by adding a random number within the region
of (−0.05, 0.05). Figure 10 and Figure 11 clearly show that
0.06
after time 100, the system tracks the change nicely to a new
0.04
equilibrium.

0.02

F. The effect of C
0
0 5 10 15 20 25 30 35 40 45 50
time slot The energy provider can influence the price by adjusting
the capacity C. As shown in Figure 12 and Figure 13, when
Fig. 6. Simulation III-C: demand adaptation of 10 users k
C is doubled, the price will drop by 1 − C k+1 which is 43%,
and each user’s demand will increase accordingly.
 5

0.25

2.5

0.2

0.15

demand
1.5

0.1
price

0.05

0.5

0
0 10 20 30 40 50 60 70 80 90 100
time slot

0
0 10 20 30 40 50 60 70 80 90 100
time slot Fig. 12. Simulation III-F: demand adaptation of 10 users

Fig. 9. Simulation III-D: price evolution 1.4

1.2

1
0.25

0.8
0.2 price

0.6
0.15

0.4
0.1
demand

0.2
0.05

0
0 0 10 20 30 40 50 60 70 80 90 100
time slot

−0.05
Fig. 13. Simulation III-F: price evolution

−0.1
0 20 40 60 80 100 120 140 160 180 200
time slot
G. Inaccurate price signals
Fig. 10. Simulation III-E: demand adaptation of 10 users
The feedback price signals are transmitted via a communi-
cation network (e.g. GPRS) to the HEM, during which packet
loss and delay could occur. In this case users may have to
adapt their demands based on outdated or inaccurate price
2.5 information. We model this situation as a small perturbation
to the price signal and study its effect on the system behavior.
As shown in Figure 14 and Figure 15, the price and demands
2
still converge to the means of the equilibrium values, but with
small fluctuations. A more detailed perturbation analysis is
1.5 part of our future work.
price

1
IV. C ONCLUSION AND FUTURE WORK

This paper proposes a distributed framework for demand

0.5
response and user adaptation in smart grid networks. More
specifically, we have applied the concept of congestion pricing
in Internet traffic control to the DR problem and shown that it
0
0 20 40 60 80 100 120 140 160 180 200
is possible that the burden of load leveling can be shifted from
time slot
the grid (or supplier) to end users via pricing. Individual users
Fig. 11. Simulation III-E: price evolution
adapt to the price signals to maximize their own benefits. User
preference is modeled as a willingness to pay parameter which
can be seen as an indicator of differential quality of service.
 6

0.16
observation is that price anticipating users tend to pay less.
0.14 One important element of intelligence in the smart grid
is the learning capability of various components. In demand
0.12
response, if users can learn from past observations (e.g. prices
0.1
and load profiles), then they can predict the future load and
price and adjust their strategies accordingly (e.g. adjusting αi
demand

0.08 and wi ). In this context, Bayesian networks and reinforcement

learning are some of the powerful tools we can leverage to
0.06
enable learning in this highly dynamic environment.
0.04 As mentioned earlier, demand scheduling can be formulated
as a typical resource allocation problem, for which a wide
0.02
range of techniques (many of them have been applied in
0
the networking field) are available. We are currently inves-
0 50 100 150 200 250 300 350 400
time slot tigating the feasibility of applying some convex optimization
techniques such as water-filling to demand side management,
Fig. 14. Simulation III-G: demand adaptation of 10 users where a certain cost function is to be minimized subject to a
number of system constraints.
1.8

1.6
ACKNOWLEDGMENT
1.4
The author would like to thank his colleagues at Toshiba
1.2 Research Europe for helpful discussions and its Directors for
1
their support of this work.
price

0.8
R EFERENCES
0.6
[1] Z. Fan, G. Kalogridis, C. Efthymiou, M. Sooriyabandara, M. Serizawa,
0.4 and J. McGeehan, “The new frontier of communications research: Smart
grid and smart metering,” in ACM e-Energy, 2010.
0.2 [2] F. Rahimi and A. Ipakchi, “Demand response as a market resource under
the smart grid paradigm,” IEEE Trans. Smart Grid, Jun. 2010.
0
[3] C. Wang and M. de Groot, “Managing end-user preferences in the smart
−0.2 grid,” in ACM e-Energy, 2010.
0 50 100 150 200 250 300 350 400
time slot [4] A. Mohsenian-Rad, V. Wong, J. Jatskevich, and R. Schober, “Optimal
and autonomous incentive-based energy consumption scheduling algo-
Fig. 15. Simulation III-G: price evolution rithm for smart grid,” in IEEE PES Conference on Innovative Smart
Grid Technologies, 2010.
[5] F. Kelly, A. Maulloo, and D. Tan, “Rate control for communication
networks: shadow prices, proportional fairness and stability,” Journal of
the Operational Research Society, 1998.
The convergence of the algorithm has been demonstrated by [6] G. Vinnicombe, “On the stability of networks operating TCP-like
both analysis and simulation results. congestion control,” in IFAC, 2002.
This paper is just a first step towards our vision of fully [7] S. Floyd, “TCP and explicit congestion notification,” ACM CCR, Oct.
1994.
distributed demand response. There are a number of directions [8] A. Ganesh, K. Laevens, and R. Steinberg, “Congestion pricing and user
for future research. Firstly, the proposed model fits nicely into adaptation,” in IEEE Infocom, 2001.
the game theory framework. In fact, there is already a rich [9] H. Bessembinder and M. Lemmon, “Equilibrium pricing and optimal
hedging in electricity forward markets,” Journal of Finance, Jun. 2002.
literature in the networking community on game theoretical [10] P. Key and D. McAuley, “Differential QoS and pricing in networks:
analysis of congestion pricing, e.g. [10]. In Kelly’s framework, where flow control meets game theory,” IEE Proceedings Software, Feb.
the user and social optima coincide if the prices are right, 1999.
[11] P. Vytelingum, T. Voice, S. Ramchurn, A. Rogers, and N. Jennings,
and the social optimum is a Nash bargaining solution with “Agent-based micro-storage management for the smart grid,” in AAMAS,
the logarithmic utility function. More recently, researchers 2010.
have applied intelligent agents and game theory to micro- [12] R. Gibbens and F. Kelly, “Resource pricing and the evolution of
congestion control,” Automatica, 1999.
storage management for the smart grid [11], in which Nash
equilibrium is reached when the agents are able to optimize
the energy usage and storage profile of the dwelling and learn
the best storage profile given market prices at any particular Zhong Fan is a Research Fellow with Toshiba Research Europe in Bristol,
time. Similarly, based on our model, it would be interesting UK. He received his BSc and MSc degrees in Electronic Engineering
to study the system dynamics and user interaction in a large from Tsinghua University, China and his PhD degree in Telecommunication
Networks from Durham University, UK. His research interests are protocol
scale energy demand game context. There are two types of design and performance analysis of wireless networks, IP networks, and smart
user strategies: price taking users and price anticipating users grid communications.
[12]. A price taking user assumes that he has no effect on the
price of the energy, whereas a price anticipating user realizes
that his own choice of wi affects the price. An interesting