Computational Intelligence for Demand Side Management and

Demand Response Programs in Smart Grids

Taha Abdelhalim Nakabi, Keijo Haataja, and Pekka Toivanen
University of Eastern Finland, School of Computing, Kuopio Campus, P.O. Box 1627, 70211 Kuopio, Finland
[email protected], [email protected], [email protected]

customer with prices depending on the period of consumption

Abstract—The reliability of a power system relies more on its throughout the day. The price during the peak hour is high, and it
flexibility, which is no longer easy to achieve on the production side is low during the valley period. DAP is almost the same as TOU,
because of the large integration of renewable energy sources. With but the prices are announced for one day 24 hours before. The
the advances in smart grids, demand side management can achieve DAP scheme relies on short-term prediction of energy
this flexibility by making customers participate in demand- consumption in the next 24 hours to offer the optimal prices. The
response programs. In this paper, we mainly discuss a variety of RTP pricing scheme is different as it relies on the real time (or up
techniques mostly relying on computational methods to implement to 1 hour ahead) energy consumption. The prices depend on
efficient demand response programs. The discussed works used supply and demand laws and therefore will be dynamic and
several techniques from the fields of artificial intelligence and difficult to predict.
optimization algorithms to handle problems like optimal power
flow, optimal pricing, optimal device scheduling, economic In this paper, we review the major works in demand side
dispatch, and customer behavior learning. In this review, we show management and DR programs; we focus on the computational
that demand response programs that are based on computational aspect of the solutions, and discuss several methods for
methods can solve a variety of problems and can open many optimization and learning systems. Moreover, we will propose a
opportunities for electricity companies to make profit of the novel research idea to solve the problem of optimal pricing and
energy market. As an example of a potential opportunity to profit maximization. We combine two computational methods;
implement a demand response program, we propose a customer the first is a double learning model that will be trained to make
behavior learning model based on deep neural networks to learn predictions of the hourly energy consumption of each customer
the consumption patterns of a customer in response to a set of given the day ahead hourly prices, assuming that the customers
electricity prices. We also propose a profit maximization approach as are price sensitive. Then a genetic algorithm will be used to find
an application of the learning model. The optimization is based on the best prices to propose to customers in order to maximize the
Genetic algorithms. The models can used in a variety of cases retailer profit.
including reducing energy cost for customers as well as
maximizing the retailers profit. The rest of this paper is organized as follows: Section II
reviews and classifies several works dealing with DSM and DR
Keywords— artificial neural networks; computational programs. Section III proposes a research idea for optimal
intelligence; machine learning; optimization algorithms; smart grid. pricing and customer behavior learning. Finally, Section IV
I. INTRODUCTION concludes the paper and sketches some new research work ideas.
The concept of Smart grid have been first mentioned in [1] II. A LITERATURE REVIEW
as a response to the challenges that encountered the North
A. Optimal power flow and DSM
American power grid. Since then, several papers have been
published in the field, including solutions to integrate The optimal power flow (OPF) problem consists of finding an
distributed renewable energy generation, energy storage and optimal operational point of a power system that minimizes an
demand response optimization. There are many challenges to appropriate cost function, such as generation cost or transmission
implement a smart grid that can make extensive use of loss subject to certain constraints on power and voltage variables
renewable energy and maintain its reliability. Demand-side [3]. OPF started receiving big interest since 1962 by Carpentier
management is a key solution to tackle these challenges by in [4]. Since then several methods and algorithms in the literature
shifting the flexibility of the power system to the customer side. have approached this highly non- convex problem of OPF [5–7],
In this paper, we will discuss the features related to demand- including nonlinear programming [8], Newton-Raphson [9],
side management and more specifically demand response (DR) quadratic programming [10], Lagrange relaxation [11], and
programs. interior point methods [12]. Recently, better results have been
DR programs can be divided into two categories: price- achieved by approaching the problem from the perspective of
based DR and incentive-based DR [2]. In price-based DR the global optimization methods, such as genetic algorithm [13– 15],
customer responds to the varying prices during the day or price evolutionary programming [16–17], particle swarm optimization
signals in peak hours to reduce his consumption or shift his loads [18], simulated annealing [19], and tabu search [20]. In [21], an
from high price periods to lower price periods. In incentive- EMS was implemented together with a DC-OPF. Differential
based DR the customer is given incentives to reduce his evolution was used to solve the minimization problem of the total
consumption in response to a signal about energy shortage in a generation cost. However, these methods present a heavy
certain period. In this paper, we focus more on price-based DR computation load and require high computational capabilities,
as it enables a good energy management throughout the day and
which make their integration in EMS a complex task [22]. To
not just in peak hours. A price-based DR can rely on various
pricing schemes, such as time-of-used (TOU), day- ahead overcome this limitation, the authors in [22] propose a DSM
pricing (DAP), or real-time pricing (RTP). Each of these based on the combination of a fully connected neural network
schemes have its own properties, advantages, and challenges. (FCN) and OPF. They proposed a learning architecture to learn
the input/outputs and reproduce the behavior of an OPF that
In time-of-use pricing the utility company provides the
generates numerical data based on genetic algorithm and fuzzy programming and it requires a minimum user effort. In [29] an
logic. The benefit was the reduction of computational time optimization model for individual customers was developed to
necessary to find the optimal schedule of generation and price- adjust the customer’s decisions in response to time varying
responsive loads. electricity prices. Robust optimization approaches were
B. DR in Day-Ahead Market employed to model the price uncertainty in order to avoid
considerable distortion to the optimal solution that can occur due
The main objective of a demand response program is to to data uncertainty. In [30], authors proposed a real-time DR
enable the users to participate in the electric system stability by management model using scenario-based stochastic optimization
reducing their loads in response to power grid needs and and robust optimization approaches. The model can be
economic signals. As mentioned in Section I, this can be embedded into smart meters to automatically determine
achieved by incentive based or price based DR. The day-ahead the optimal operation in the next 5- minute time interval while
pricing scheme was addressed in the literature using various considering future electricity price uncertainties.
methods. In [23], authors presented a multi-objective
optimization for air-conditioning day-ahead control to minimize D. DR for User’s Cost Minimization
the electricity expenses and expected error for the desired The previous works focus only on users trying to minimize
indoor temperature while retaining the degree of comfort in a their cost in short period of time and did not mention how the
room. The optimization problem was solved by an Immune proposed scheduling algorithms can be used to minimize the
Clonal Selection algorithm. In [24], authors presented an long-term cost. Alternatively, [31–34] tackle this issue by
algorithm for the utility company to determine the optimal day introducing techniques for long-term cost minimization. In [31]
ahead price values for customers who participate in a DR a reinforcement learning approach was proposed to minimize a
program. The demand function is considered as an unknown bill payment and a discomfort cost function for a household
parameter. The problem was approached by a learning user. The model deals with stochastic demand arrivals and make
algorithm based on stochastic approximation. The behavior of a a demand scheduling based on the devices allowable delays. A
day-ahead electricity market was studied in [25] where a decentralized based heuristic method was implemented as well
hierarchical multi-agent framework was presented for Day- as an approximation approach based on Q-learning. A
Ahead planning and decision-making in retail electrical energy comparison of the results shows that each approach has
markets. On the retailer’s side, agents are using machine advantages over the other under different scenarios. Q-learning
learning techniques to model their environment and optimize approach works under more general settings, while the
their behavior to deal with interoperability and decision-making heuristic approach is better at delivering solutions in a much
under incomplete information in a system that maintains the faster manner for regular sized problems. In [32] a
data privacy of the customers. On the customer’s side air reinforcement learning was proposed to automatically
conditioning devices are controlled by agents that employ Q- schedule the appliances in a household. The system
learning algorithms [26] to optimize their consumption pattern. schedules the time of operation by calculating the customer’s
It is shown that this approach will reduce overall power trade-offs between delay and energy prices. It learns these
consumption cost, maximize retailer’s profit, and reduce peak trade-offs by observing energy customers’ behavior and
load. observing the patterns of energy pricing. Over time, the EMS
C. DR and Real-Time Pricing learns how to make the best decisions for energy customers in
the sense that it balances energy cost and the delay in energy
In a demand response program, users are required to usage in the same way that the customer would do. With the
schedule their energy consumption according to meet the same technique [33] introduced a dynamic pricing
energy availability and reduce consumption peaks. The energy algorithm based on reinforcement learning for the service
retailer provides the customers with a pricing scheme that provider to minimize the system cost. The model was then
reflects the energy cost and availability during the day. A extended by adopting multi-agent learning structure where each
comparison between real-time pricing and time-of-use pricing customer can decide its energy consumption scheduling based
schemes in [27] showed that real-time pricing is more beneficial on the observed retail price aiming at minimizing its expected
for both customers and retailers, because it enables the power cost. The problem was formulated as a dynamic pricing
system to flatten the load profile by providing enough problem in the framework of a Markov-Decision Process
information on the true time-variant electricity supply costs (MDP). This problem was solved using Q-learning algorithm
and enough financial incentives to end customers to adjust their with two improvements: alternative state definition and
energy consumptions. However, implementing a real-time price virtual experience [34]. Moreover, the algorithm is fast and
is a complex task, because the user may not know his energy dynamic and does not require information about the system
prices and demand ahead of time. It is even more challenging uncertainties. In [35], authors proposed a batch reinforcement-
because the users' scheduling decisions are coupled, since the learning algorithm for residential demand response to schedule
demand of a user affects the price that is charged to all users. controllable loads, such as water heater and heat- pump
To tackle these challenges, a DR management for real-time thermostat. The authors considered the situation when a forecast
price should be implemented to assist customers automatically of the exogenous data is provided and proposed a policy
by determining the optimal operations of appliances. In [28], an adjustment method that exploits general expert knowledge. A
automatic residential energy consumption scheduling model-free Monte-Carlo method was introduced to find a day-
framework was designed. It attempts to achieve a desired ahead consumption plan.
trade-off between minimizing the electricity payment and
minimizing the waiting time for the operation of each appliance E. DR for Multiple Users
in household in presence of a real-time pricing tariff combined However, these works only deal with one user without
with inclining block rates. The model was based on simple linear considering the decision of multiple users trying to optimize their
long-term costs. These interactions between users was captured rebuilt in [2] to make a common price based DRP, namely TOU,
by the authors in [36] where they studied the problem of instead of incentives. The objective is to minimize generation
electricity sharing among multiple foresighted users who cost and determine the optimal prices during different periods
participate in a demand response program with real-time simultaneously. The total cost was obtained by 4 meta-heuristic
pricing scheme. They consider an energy storage system shared algorithms namely PSO, GA, ABC, and BCO, but ICA
across a group of homes, to store energy at lower price and sell (Imperialist Colony Algorithm) was eventually adopted,
it back to the grid when the price is high. They focus on the because it gave better results.
centralized control of such a shared battery. The scheduling G. Customer Behavior Learning and DR
problem of each individual user was formulated as a Markov
decision process to study the optimal cost savings region for a In demand response, the information about customer energy
finite capacity battery assuming a zero tolerance for activity consumption is extremely valuable for optimization and pricing.
delay. The price dependence on real-time demand and The reaction of customers to the electricity price can enable the
renewable generation was modeled in [37] by a non- retailer to extract valuable insights about customer behavior by
cooperative game of incomplete information among the users recognizing different consumption patterns. In [44], authors
with heterogeneous, but correlated consumption preferences. proposed algorithms for learning the future price elasticity of
The Bayesian Nash Equilibria (BNE) in these games was customers based on their responses to previous pricing updates.
considered as the optimal user behavior. Given this anticipating The task was formulated as a linear regression problem and
behavior two pricing policies were proposed: the first aims to consider the aggregated changes in consumption over the
achieve a target return rate and the second aims to minimize distribution network as a weighted sum of all individual changes
consumption peak-to-average ratio (PAR). In [38] a load in consumption. In [45], authors studied the customer price
scheduling learning algorithm was designed for users who elasticity of demand using an agent-based model. The model
schedule their appliances in response to RTP information. The was used to demonstrate and quantify the economic impact of
study of long-term interactions among foresighted users enables price elasticity of demand in electricity markets. In [46] the
the authors to model the users’ decision making with problem of learning customers’ behavior was approached by
uncertainty about the price and load demand as a Markov short term load forecasting in a pricesensitive environment
decision process with different states for different possible with real-time pricing scheme, where customers are reacting to
scenarios. The distributed load scheduling learning algorithm price signals. The model used a neuro-fuzzy approach by
was based on actor-critic method [39–40] and converges to the building a two-stage forecaster consisting of an artificial neural
Markov Perfect Equilibrium policy. The algorithm is online and network load forecaster followed by a fuzzy logic system. All
model free, which enables the system to learn from the these models are built for aggregated customers whereas [47]
consequences of past decisions. presented a model for individual customers. This is important,
because it can identify valuable information about different
F. Economic Dispatch and DR behaviors and usage patterns between different customers in
While DR focus on the customer side and his reaction to response to the price and temperature signals. The proposed
prices and incentives to reduce load, economic dispatch (ED) model is based on probabilistic Bayesian behavior model to
focus on the supply to minimize the power generation cost learn the energy usage patterns of shiftable appliances and a
subject to some constraints. Combination of demand response price-demand model to predict the hourly energy consumption
and dynamic economic dispatch is of a big interest, because it of curtailable appliances. The authors also proposed a
does not just schedule the generation units optimally, but also distributed pricing optimization based on genetic algorithm for
the prices will be optimized at the same time, allowing a win- the utility company to maximize its profit based on the learning
win strategy and increasing the reliability of the power system results. The reviewed works tackle different types of problems
[27]. This concept was presented in [2, 41–42]. In [41] the in demand side management using various computation
combined model of ED and DR has been studied through a methods and approaches. Table 1 classifies these works in
model based on price signals sent to the customers at the peak addition to other works according to the issue and computation
hour to reduce their loads and shift their consumption to the method used.
off-peak hours, in exchange of incentives paid to them for III. OUR NOVEL RESEARCH PROPOSAL
their participation in the EMS. The method used to determine
the optimal price signal at peak hour is based on genetic The previously discussed works have approached the
optimization algorithm with a stochastic variable that problem of DSM and specifically the demand response problem
estimates the probability of the customer's participation. by valuable approaches. Most of works that tackle the issue of
However, this model is limited because it only considers the device scheduling assume that the household is equipped with a
peak hours and not the whole day. A new solution method for Home Energy Management System (HEMS) that automate the
solving the combined problem was presented in [42]. The task of turning on the appliances at lower price periods and
authors integrated DEED (Dynamic Economic Emission turning them off at high price periods. Despite these works have
Dispatch) and game theory based DR under a deregulated achieved valuable results, the requirements to implement it in a
environment. This method is able to give feedback and update large scale is too expensive for most practical situations.
inaccurate solutions. The results show that this model is Alternatively, smart meters infrastructure is already deployed in
superior to independent optimization of DR or DEED. In [43] a large scale around the world and still expanding. In 2015, U.S.
the integrated model of DED and EDRP (Emergency Demand electric utilities had about 64.7 million advanced (smart)
Response Program) was presented with non-linear responsive metering infrastructure (AMI) installations [48]. By 2020, it is
load models. The optimization problem of fuel cost and expected that almost 72% of European customers will have a
collaboration incentives was solved using RDPSO (Random smart meter for electricity [49]. It is expected that in the U.K., all
Drift Particle Swarm Optimization). This model was lately households will have smart meters installed by 2020 according
Table 1: CLASSIFICATION OF DIFFERENT WORKS AND TECHNIQUES the computational requirement without affecting the accuracy of
the model. We will implement and test 2 learning models; the
OPF Optimal Optimal DR and ED Customer first one will learn the consumption patterns from shiftable
device pricing combination behavior appliances during 24 hours. The second model will learn the
scheduling learning
Linear [28] [41] response of curtailable devices, namely air conditioning
programming systems, to an electricity price and a temperature corresponding
evolutionary [13], [53], [2] [47] [2] to a time slot of one hour. The model used for learning shiftable
optimization [14], appliances consumption patterns takes 24 inputs representing
[15], the day ahead hourly prices, and 24 outputs representing the
[16], hourly loads from shiftable appliances. This architecture is
justified by the fact that the customer is shifting his appliances
Particle [17],
[18] [2], [54] [43], [2] according to the whole set of prices over one day either using a
swarm [21]
optimization
scheduling system or manually. In the second learning model,
Neural [22] [46] the inputs are the electricity price and temperature. This is
networks justified by the assumption that air-conditioning systems’ loads
Fuzzy logic [22] [46] depend only on the electricity price and temperature at a certain
Immune [23] time slot. The learning approach in the first model will use deep
Clonal
selection NNs. This choice is justified by the ability of these models to
Stochastic [30] [24] learn the mapping between an input and output vectors with high
programming dimensions and complexity. In our case, we have an input and
Multi-agent [33] [25],[34] [45] an output with 24 entries, which is a complex and high
system
Machine [25] [55], [56]
dimensional learning problem. Fig.1 illustrate this architecture.
learning In the second learning model, we will implement different
Reinforcement [25], [31], regression algorithms, such as linear regression used in [47],
learning [32], [33], support vector machines regression, along with deep NNs, then
[35]
evaluate and compare the results in order to choose the best
Robust [29, [30]
optimization model for this problem. The two models will be trained on
Game theory [36], [37] [42] historical data, and updated every week with the new
Bayesian [47], [56], consumption data to track consumption changes. After
model [52] implementing the learning model, we will handle the
optimization problem to maximize the retailer’s profit without
to [50]. Therefore, it is important to develop methods and increasing the bill payment. We will evaluate 5 different meta-
techniques for demand side management to implement demand heuristics for combinatorial optimization namely: PSO, GA,
response programs that take advantage of this source of ACO, ABC, and ICA. Then we make a comparison of results
information by using the real-time data provided by the two- using real data from a local electricity retail company.
way communication channels in smart grid. The approach
proposed in [47], presented a valuable method in this matter, it Using the NILM we separate the data of shiftable and curtailable
tackles the problem of learning the customer behavior and devices. We train our models using these sets of data and then we
maximizing the retailer’s profit and the total energy cost use the trained models to predict the day ahead hourly energy
according to the learning model, without increasing the
electricity bill and without using any type of HEMS. The whole
approach is based on the smart meters infrastructure and day
ahead pricing.
Our idea is to tackle almost the same problem with
different methods and with different assumptions. In [47], the
pattern of energy usage learning approach was based on device
by device consumption data extracted using a nonintrusive
appliance load monitoring (NILM) to extract hourly
consumption and on/off time information of each appliance
from the entire electrical power consumption of each house by
using signal analysis algorithms [51]. The appliances were then
classified into shiftable and curtailable devices. In each
shiftable device, the learning is based on Bayesian probabilistic
model, whereas curtailable devices was modeled by linear
regression including historical data of hourly prices and
temperatures. However, the learning approach of shiftable
devices considers just the ranking of the cheapest periods
without considering the period itself. This information is
extremely valuable because without it we cannot know if the
user is price sensitive or is just using a device in an exact time
every day and it coincides with the cheapest period. In our novel
proposal, we will handle the aggregate shiftable devices instead Fig.1: Neural network architecture for shiftable devices
of learning the consumption of each device. This will reduce
Fig.2: Learning model for curtailable devices
consumption given the day ahead prices and temperatures. This
process will be executed each week using the most recent
consumption data.
Based on these models, an optimization algorithm will be
implemented to maximize the retailer’s profit. The aim of this
algorithm is to find the prices that will maximize the difference
between the energy cost and the benefit from all customers
considering economic and political constraints about the amount
of energy provided and the bill payment. We show here a GA
based optimization algorithm using results from the learning
models to find the optimal pricing strategy. Figure 3 shows the
process of finding the optimal pricing strategy. First, a population
of PN strategies is initialized randomly; each strategy is
represented by a 24-dimensional vector of hourly prices. Each
prices vector is used as an input in the neural networks for
shiftable and curtailable appliances. 24h temperature forecast for
the day ahead is also used for curtailable appliances model. The
energy consumption for each customer will be predicted then the
bill payment will be calculated. The benefit will be calculated by
summing the predicted bills of all customers in response to the
strategy. The cost of the total energy consumption will be
calculated. We calculate the predicted profit, which is the
difference between benefit and cost. We check the constraints
satisfaction and obtain the fitness function. We iterate this
process for all the strategies. Then generate a new population of
strategies by using the selection, crossover, and mutation. We
repeat this process until a stop criterion, i.e., the situation when
the solution is not improving anymore. At the end, we obtain an
optimal strategy that the retailer will announce via smart meters
IV. CONCLUSION AND FUTURE WORK
Demand response programs have a big potential in
increasing the power system’s flexibility and reliability by
involving the customer in the decision process. Several methods
from the literature for optimization and automation were
discussed in this paper. Computational methods from the field
of artificial intelligence and optimization, were used to solve the
problems of OPF, optimal pricing, optimal device scheduling,
DR with economic dispatch, and customer behavior learning.
Our novel proposal solves the combined problem of customer
behavior learning and optimal pricing. Two learning model
architectures were proposed for customer behavior learning as
well as a genetic algorithm for price optimization. The future
work will bring more details and numerical results about the
proposed idea.
Advances in artificial intelligence and machine learning
Fig.3: GA-based Optimization algorithm for retailer’s profit
are deeply influencing our life in many aspects. AI takes
maximization
advantage of the large availability of data nowadays and the big
computational power. Therefore, this huge power of AI and ML ML in the energy management process. Thanks to the two-ways
are likely to be the next revolution in the most vital aspect of communication infrastructure and the large data collected via
the modern life: energy. Smart grids are well appropriate and
smart meters. AI and ML techniques have a large potential in
much convenient to implement this idea of integrating AI and
taking advantage of this data and infrastructure to tackle the several researches are focusing on different aspects of the smart
challenges of the new smart grid allowing more integration of grid. The challenge in the future will be to implement a scalable
renewable energies, reducing carbon emission, saving system that can englobe different computation techniques and
generation cost, integrating electric vehicles, increasing general AI to handle different aspects like demand side
network reliability, and offering better products to the final management, Electric vehicles, virtual power plant, energy
customers at lower prices. To achieve these expectations, prosumers, and self- healing networks.
REFERENCES:
[1] S. Massoud Amin, B.F. Wollenberg, “Toward a smart grid: [15] M.S. Kumari, S. Maheswarapu, "Enhanced Genetic
power delivery for the 21st century”, IEEE Power and Algorithm based computation technique for multi- objective
Energy Magazine, vol. 3, pp. 34-41, Sept.-Oct. 2005 Optimal Power Flow solution", Electrical Power and Energy
[2] Ehsan Dehnavi, Hamdi Abdi, “Optimal pricing in time of Systems, vol 32, No. 6, pp. 736–742, July 2010.
use demand response by integrating with dynamic [16] J. Yuryevich , Kit Po Wong, "Evolutionary programming
economic dispatch problem”, Energy, 109 pp. 1086-1094, based optimal power flow algorithm", IEEE Trans.on
2016. Power Systems , Vol: 14, Issue: 4 pp. 1245 – 1250 , Nov
[3] J. A. Momoh, “Electric Power System Applications of 1999.
Optimization”, New York: Markel Dekker, 2001. [17] W. Ongsakul, T. Tantimaporn, "Optimal Power Flow by
[4] J. Carpentier, “Contribution to the economic dispatch Improved Evolutionary Programming", Electric Power
problem,” Bull. Soc. Francaise Elect., vol. 3, no. 8, pp. Components and Systems, vol. 34, pp. 79–95, 2006.
431–447, 1962 [18] M.A. Abido, "Optimal power flow using particle swarm
[5] M. Huneault and F. D. Galiana, “A survey of the optimal optimization", International Journal of Electrical Power &
power flow literature,”IEEE Trans. Power Syst, vol. 6, no. Energy Systems, Vol 24, Issue 7, pp. 563–571, October
2, pp. 762–770, May 1991. 2002.
[6] G. L. Torres and V. H. Quintana, “Optimal power flow by a [19] L. Chen, H. Suzuki, and K. Katou, “Mean-field theory for
nonlinear complementarity method,” IEEE Trans. Power optimal power flow,” IEEE Trans. Pwr. Syst., vol. 12, pp.
Syst, vol. 15, no. 3, pp. 1028–1033, Aug. 2000. 1481–1486, 1997.
[7] H. Wang, C. E. Murillo-Sanchez, R. D. Zimmerman, and [20] T. Kulworawanichpong and S. Sujitjorn, “Optimal power
R. J. Thomas, “On computational issues of market-based flow using tabu search,” IEEE Pwr. Engr. Rev., pp. 37–
optimal power flow, ”IEEE Trans. Power Syst, vol. 22, no. 40, 2002.
3, pp. 1185–1193, Aug. 2007. [21] P. P. Verma, P. Soumya, K. S. Swarup, “Optimal Day-
[8] M.Santos-Neitoand V.H. Quintana. "Linear Reactive Ahead Scheduling in Smart Grid with Demand Side
Power Studies for Longitudinal Power Systems," 9th Management”, 2016 IEEE 6th International Conference on
PSCC Conference, pp 783-787, 1987. Power Systems (ICPS), March 2016
[9] Rong-Mow Jan ; Nanming Chen. "Application of the fast [22] P. Siano, C. Cecati, H. Yu, and J. Kolbusz, “Real time
Newton-Raphson economic dispatch and reactive operation of smart grids via FCN networks and optimal
power/voltage dispatch by sensitivity factors to optimal power flow,” IEEE Trans. Ind. Informat., vol. 8, no. 4, pp.
power flow," IEEE Trans. on Energy Conversion , vol. 944 –952, Nov. 2012.
10, pp 293 - 301, 1995. [23] Y. Y. Hong, J. K. Lin, C. P. Wu, and C. C. Chuang, “Multi-
[10] A.D. Papalexopoulos, C.F. Imparato, and F. F. Wu, "Large objective air conditioning control considering fuzzy
Scale Optimal Power Flow: Effects of Initialization parameters using immune clonal selection programming,”
Decoupling and Discretization,"IEEE Trans.on Power IEEE Trans. Smart Grid, vol. 3, no. 4, pp. 1603–1610, Dec.
Systems, vol. PWRS-4, No.2, pp 748-739, May 1989 2012.
[11] K.H. Abdul-Rahman, S.M. Shahidehpour, M. Aganagic S. [24] L. Jia, Q. Zhao, and L. Tong, “Retail pricing for stochastic
Mokhtari, "A practical resource scheduling with OPF demand with unknown parameters: An online machine
constraints", IEEE Trans.on Power Systems, Vol. 11, learning approach,” in Proc. Of Allerton Conf. on
No.1, pp 254-259, February 1996. Communication, Control, and Computing, Monticello, IL,
[12] G. L. Torres, V. H. Quintana, "An interior-point method Oct. 2013.
for nonlinear optimal power flow using voltage [25] K. Dehghanpour, M. Hashem Nehrir, John W. Sheppard,
rectangular coordinate" IEEE Trans.on Power Systems, Nathan C. Kelly, “Agent-Based Modeling of Retail
Vol. 13, No. 4, pp. 1211-1218, November 1998, Electrical Energy Markets With Demand Response”,
[13] L. L. Lai, J. T. Ma, R. Yokoyama, and M. Zhao, “Improved Accepted for publication in IEEE Trans.on smart grid,
genetic algorithms for optimal power flow under both NOV. 13TH, 2016, pp 1-1
normal and contingent operation states,” J. El. Pwr. En. [26] Watkins, C.J.C.H. & Dayan, P. Mach Learn (1992) 8: 279.
Syst., vol. 19, no. 5, pp. 287–292, 1997. doi:10.1007/BF00992698
[14] A. G. Bakirtzis, P. N. Biskas, C. E. Zoumas, and V. [27] S. Borenstein, M. Jaske, and A. Rosenfeld, “Dynamic
Petridis, “Optimal power flow by enhanced genetic pricing, advanced metering and demand response in
algorithm,” IEEE Trans. Pwr. Syst., vol. 17, pp. 229–236, electricity markets,” Center for the Study of Energy
2002. Markets, UC Berkeley, 2002.
[28] H. Mohsenian-Rad and A. Leon-Garcia, “Optimal response programs. Energy 2015; vol. 91: pp. 404-419.
residential load control with price prediction in real-time [43] Abdi H, Dehnavi E, Mohammadi F. “Dynamic economic
electricity pricing environments,” IEEE Trans. Smart dispatch problem integrated with demand response
Grid, vol. 1, no. 2, pp. 120–133, Sep. 2010. (DEDDR) considering non-linear responsive load models.”
[29] A.J.Conejo, J.M.Morales,.Baringo,“Real-time demand IEEE Trans. Smart Grid 2015, issue 6, vol.7, pp. 2586 – 259.
response model,” IEEE Trans. Smart Grid, vol. 1, no. 3, pp. [44] V. Gómez, M. Chertkov, S. Backhaus, and H. J. Kappen,
236–242,Dec. 2010. “Learning price-elasticity of smart customers in power
[30] Z. Chen, L. Wu, and Y. Fu, “Real-time price-based demand distribution systems,” in Proc. IEEE 3rd Int. Conf. Smart
response management for residential appliances via Grid Commun. (SmartGridComm), Tainan City, Taiwan,
stochastic optimization and robust optimization,” IEEE 2012, pp. 647–652.
Trans. Smart Grid, vol. 3, no. 4, pp. 1822–1831, Dec. [45] P. R. Thimmapuram and J. Kim, “Customers’ price
2012 elasticity of demand modeling with economic effects on
[31] Y. Liang, L. He, X. Cao, and Z. J. Shen, “Stochastic control electricity markets using an agent based model,” IEEE
for smart grid users with flexible demand,” IEEE Trans. on Trans. Smart Grid, vol. 4, no. 1, pp. 390–397, Mar. 2013.
Smart Grid, vol. 4, no. 4, pp. 2296–2308, Dec. 2013 [46] A. Khotanzad, E. Zhou, and H. Elragal, “A neuro-fuzzy
[32] Z. Wen, D. O’Neill, and H. Maei, “Optimal demand approach to short-term load forecasting in a price- sensitive
response using device-based reinforcement environment,” IEEE Trans. Power Syst., vol. 17, no. 4, pp.
learning,” IEEE Trans. on Smart Grid, vol. 6, no. 5, pp. 1273–1282, Nov. 2002.
2312–2324, Sept. 2015. [47] Fan-Lin Meng, Xiao-Jun Zeng, “A Profit Maximization
[33] B. Kim, Y. Zhang, M. van der Schaar, and J. Lee, Approach to Demand Response Management with
“Dynamic pricing and energy consumption scheduling Customers Behavior Learning in Smart Grid,” IEEE
with reinforcement learning,” IEEE Trans. on Smart Grid, Trans. smart grid, VOL. 7, NO. 3, MAY 2016
vol. 7, no. 5, pp. 2187–2198, Sept. 2016 [48] The U.S. Energy Information Administration (EIA)
[34] N. Mastronarde and M. van der Schaar, “Joint physical- https://www.eia.gov/tools/faqs/faq.php?id=108&t=3
layer and system level power management for delay- [49]European Commission Energy,
sensitive wireless communications,” IEEE Trans. Mobile https://ec.europa.eu/energy/en/topics/markets-and-
Comput., vol. 12, no. 4, pp. 694–709, Apr. 2013. customers/smart-grids-and-meters.
[35] F. Ruelens, B. J. Claessens, S. Vandael, B. D. Schutter, [50] Department of Energy and Climate Change. (Mar. 2014).
R. Babuska, and R. Belmans, “Residential demand respo Helping Households to Cut Their Energy Bills. [Online].
nse of thermostatically controlled loads using batch Available:
reinforcement learning,” accepted for publication in IEEE https://www.gov.uk/government/policies/helping-
Trans. on Smart Grid, 2016. households-to-cut-theirenergy-bills/supporting-
[36] J. Yao, and P. Venkitasubramaniam, “Optimal end user pages/smart-meters,
energy storage sharing in demand response,” in Proc. of [51] G. W. Hart, “Nonintrusive appliance load monitoring,”
IEEE Smart GridComm, Miami,FL, Nov. 2015. Proc. IEEE, vol. 80, no. 12, pp. 1870–1891, Dec. 1992.
[37] C. Eksin, H. Delic, and A. Ribeiro, “Demand response [52] Mikhail V. Goubko, Sergey O. Kuznetsov. Alexey A.
management in smart grids with heterogeneous customer Neznanov. Dmitry I. Ignatov., “Bayesian Learning of
preferences,” IEEE Trans. On Smart Grid, vol. 6, no. 6, pp. Customer Preferences for Residential Demand Response”,
3082–3094, Nov. 2015 Volume 49, Issue 32, 2016, Pages 24-29.
[38] S. Bahrami, V. W.S. Wong, and J. Huang, “An Online [53] J. Khourya, R. Mbayedb, G. Salloumb, E. Monmassona,
Learning Algorithm for Demand Response in Smart Grid,” “Predictive demand side management of a residential
IEEE Trans. on Smart Grid (Volume: PP, Issue: 99), pp. 1- house under intermittent primary energy source
1, February 2017 conditions”, Volume 112, 15 January 2016, pp: 110–120
[39] V. Konda and J. Tsitsiklis, “On actor-critic algorithms,” [54] Sofana Reka S., Ramesh V. (2016) Demand Side Response
SIAM Journalon Control and Optimization, vol. 42, no. 4, Modeling with Controller Design Using Aggregate Air
pp. 1143–1166, Aug. 2003. Conditioning Loads and Particle Swarm Optimization. In:
[40] S. Bhatnagar, R. Sutton, M. Ghavamzadeh, and M. Lee, Choudhary R., Mandal J., Auluck N., Nagarajaram H.
“Natural actor-critic algorithms,” Automatica, vol. 45, (eds) Advanced Computing and Communication
no. 11, pp. 2471–2482, Nov.2009 Technologies. Advances in Intelligent Systems and
Computing, vol 452. Springer, Singapore.
[41] Ashfaq A, Yingyun S, Zia Khan A. Optimization of
economic dispatch problem integrated with stochastic [55] Dong Zhang, Shuhui Li, Min Sun, and Zheng O’Neill, “An
demand side response. In: IEEE International Conference Optimal and Learning-Based Demand Response and
on intelligent energy and power systems; 2014. pp. 116- Home Energy Management System”, IEEE Trans. on
121. smart grid, VOL. 7, NO. 4, JULY 2016, pp: 1790-1801.
[42] Nwulu NI, Xia X. Implementing a model predictive [56] Heng Shi, Minghao Xu, Ran Li, “Deep Learning for
control strategy on the dynamic economic emission Household Load Forecasting – A Novel Pooling Deep
dispatch problem with game theory based demand RNN,” IEEE Trans. on Smart Grid, Volume: PP:1-1, Issue:
99, 2017.