2964 IEEE TRANSACTIONS ON SMART GRID, VOL. 6, NO.

6, NOVEMBER 2015

A Micro-Grid Distributed Intelligent Control

and Management System
Wolf Kohn, Zelda B. Zabinsky, and Anil Nerode

Abstract—Micro-grids require active control to maintain qual- implementation of automated adaptive control strategies that
ity of service and to interface with the power grid in a are the result of multiobjective optimal control laws. A discus-
bi-directional manner. Further, micro-grids must be justified by sion of centralized and decentralized control issues is found
environmental, governmental, and economic viability. We present
a programmable architecture for active, optimal distributed con- in [5]. Some issues include scalability and accuracy of control
trol of elements of the grid to achieve desired behavior. A unique algorithms. Distributed algorithms have been developed for
aspect of this architecture is to include a distributed inductive wireless sensor networks [6] and macro-femto networks [7].
engine for learning the local dynamics of generators and loads Our approach is to be general in the type of micro-grid.
in the micro-grid. It generates feedback laws that are adapted Our intelligent control system learns the dynamics of the sys-
to the current status of the micro-grid, and responds to anoma-
lous events in a resilient manner. An important novelty is that tem through sensors and modifies the system to achieve the
control laws are extracted online for bidirectional discontinu- objectives.
ous nonlinear loads by mean field methods from physics outside Micro-grids with generation resources are connected to the
the standard design methodologies for piecewise linear quadratic general power grid to be able to satisfy the variable loads
controls. in a cost-effective manner. For micro-grids having genera-
Index Terms—Bidirectional power flow, computational and tion resources, the bidirectional exchange of power provides
artificial intelligence, decentralized control, distributed param- additional opportunities to optimize performance.
eter systems, learning systems, micro-grids, power engineering This paper describes a new architecture for micro-grid dis-
and energy, power system control, power system management,
smart grids. tributed intelligent control and management. The approach
we follow is based on principles of hybrid systems and
optimization [8]–[14]. We build a distributed network of intel-
I. I NTRODUCTION
ligent element controllers that can provide synchronization of
E CONSIDER a micro-grid that serves the energy
W needs for a building or a cluster of buildings, such
as a campus, a hospital complex, an industrial complex,
a micro-grid to the power grid, and provide feedback control
across the micro-grid to achieve optimal performance.

a residential neighborhood, or a military installation. The

classic approach to maintain and install such a micro-grid II. I NTELLIGENT C ONTROL AND M ANAGEMENT
involves a careful design based on detailed stochastic mod- A RCHITECTURE OF M ICRO -G RID
els of the dynamics of the loads and generators [1]–[4]. For the purposes of this paper, a micro-grid is a network
However, the dynamics of a micro-grid with renewable energy of electric devices connected by conductor lines. A device
sources and smart loads cannot be modeled by standard is attached to a bus and connected carrying loads, generat-
modeling of dynamical systems in meaningful ways. Our ing units, or a connection to another grid. Our micro-grid
approach uses logic rules to encode customizable dynamic architecture includes both controlled and uncontrolled devices.
principles of the loads and generation devices, including A micro-grid diagram is shown in Fig. 1. Controlled devices,
uncertainties, by learning and adaptation without using an as shown in Fig. 1, are directly connected in a feedback con-
explicit model. figuration to an intelligent control element, called an element
The high cost of energy and the need for high qual- controller. The purpose of these control elements is to drive the
ity of service and reliability of the micro-grid, suggest the grid to a desired dynamic behavior. This desired behavior is
Manuscript received June 26, 2014; revised November 25, 2014, effected by control actions generated by these intelligent con-
February 23, 2015, and June 10, 2015; accepted July 8, 2015. Date of publi- trol elements as a function of sensory data measuring the status
cation August 6, 2015; date of current version October 17, 2015. This work of the devices in the micro-grid, expressed desired behavior,
was supported in part by Atigeo, LLC, and in part by the National Science
Foundation under Grant CMMI-0908317. Paper no. TSG-00653-2014. a partial dynamic model of the grid and a corpus of behavior
W. Kohn and Z. B. Zabinsky are with Atigeo LLC, Bellevue, generated by active, inductive learning from the sensory data.
WA 98004 USA, and also with the Department of Industrial and Systems To a significant extent, this paper is concerned with estab-
Engineering, University of Washington, Seattle, WA 98195-2650 USA
(e-mail: [email protected]; [email protected]). lishing an implementable technology for micro-grid intelligent
A. Nerode is with Cornell University, Ithaca, NY 14853 USA (e-mail: control. Why does a micro-grid need intelligent control?
[email protected]). We explore four reasons.
Color versions of one or more of the figures in this paper are available
online at http://ieeexplore.ieee.org. 1) Conventional optimal control techniques require an
Digital Object Identifier 10.1109/TSG.2015.2455512 accurate model of the dynamic behavior of the loads,
1949-3053 c 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
KOHN et al.: MICRO-GRID DISTRIBUTED INTELLIGENT CONTROL AND MANAGEMENT SYSTEM 2965

Fig. 1. Micro-grid system.

such as an air conditioner, or a hospital MRI, in the relationship at the current time. Thus the response of
micro-grid. The main problem that we face today is that our intelligent control system also improves over time.
modeling the loads at sufficient accuracy is impractical. Furthermore, as the system evolves due to changes
Further, many current loads such as battery chargers, in the load profile or generation profile, our intelli-
high temperature crystal growing, MRI machines and gent control system adapts to those changes through
others have nonlinear and discontinuous dynamics. We learning.
propose a methodology based on machine learning, 2) Micro-grids are distributed dynamical systems. In the
load forecasting, and rule-based optimization to iden- past, micro-grids were designed with sufficient excess
tify the dynamic behavior of the loads using sensory capacity to satisfy peak demand with high probability,
data in real time. The methodology models near real- however, we can no longer afford to waste this excess
time power and reactive power, voltage, and current in capacity. A conventional centralized control design can-
the micro-grid. We model the dynamic behavior of loads not provide quality feedback control to the multiplicity
and generation using localized Hamiltonians [15] where of active devices of the micro-grid. Our distributed con-
the state variables include relationships between volt- trol manages the control action for each active device
age and current and the device, and operational cost. and is able to more accurately match the variability of
The Hamiltonian is constructed from rules describing the demand with the multiple sources of generation so
these relationships, by a conservation principle. For each that peak demand and base demand are satisfied. In our
device, the Hamiltonian associated with the device inter- distributed control, local intelligent feedback controllers
acts with the aggregated Hamiltonian representing the are connected to form a collaborative network with oper-
network at the ports of the device (see [9] and [16]). ational target settings for each element controller and
For each state variable in the Hamiltonian, there is system-wide optimization criteria.
a momentum (or co-state) variable. The dynamics of 3) Micro-grids exchange energy with the environment and
these variables are given in Section II-C. In addition are open systems. The nature of this bi-directional
the action variables representing voltage settings, or exchange is stochastic, and the dynamics of this stochas-
flow direction in the case of a battery, or inverter ticity has to be learned from observations. This exchange
settings, etc., are determined by an optimality condi- requires the intelligent control to provide active syn-
tion [represented by (3)]. The methodology we propose chronization control. This active synchronization control
includes a component that extracts structural charac- between the network of element controllers and the out-
teristics of the dynamics from historic and current side system is essential to maintain high quality of
data, and improves the accuracy of the model as time service in a cost-effective manner. The control is an
goes on. For example, the functional relation between electricity price-based control, where quality of service
voltage, current and phase in each DER is extracted is included as a component of the cost of the micro-
from data. This involves constructing polynomic forms, grid. The quality of service is specifically referring to
perhaps with saturation, that locally characterizes the voltage and frequency regulation. The synchronization
2966 IEEE TRANSACTIONS ON SMART GRID, VOL. 6, NO. 6, NOVEMBER 2015

mechanism of each device with respect to the micro- and frequency regulation, can be specified by rules with small
grid is achieved by establishing a Pareto equilibrium thresholds on variability. Catastrophic cascading power failures
between the Hamiltonian of the device and the aggre- of interconnected systems are also a driver for implementing
gated Hamiltonian at the ports of the device [15], [16]. power delivery as a network of micro-grids.
Pareto equilibrium is achieved when there it is near reso- Current software was designed to identify on the fly pos-
nance (high quality factor), between the desired dynamic sible cascading failure modes anywhere in the system and to
operation of the device and the grid dynamics at the port use this information to initiate isolation of failing systems and
represented by its grid. creation of a decomposition of working systems into indepen-
4) Micro-grids are distributed heterogeneous systems. We dently operating subsystems. This was a perspicacious and
apply a version of indirect synchronization between active laudable goal. But now a constantly increasing proportion of
controllers based on a notion of Pareto equilibrium [17] generated power comes from widely distributed nontraditional
between each control element and a virtual control ele- sources with different operating characteristics. The new dis-
ment called a mean field element, whose dynamics are parate sources of large irregular variations are very highly
encoded and transmitted from one controller to another, distributed, and were not factors in the design of current sys-
iteratively, establishing synchronization between each tem controllers for distribution of power. We see no way to
element controller and mean field. This concept gener- use traditional centralized control, due to random combinations
alizes Tellegen’s theorem [18] that states that in solving of these disparate power sources. A huge obstacle is figuring
distributed control by a network of agents, we can regard out the effect of a disturbance in one subsystem on a distant
an agent controlling a subsystem as interacting with an subsystem, and then communicating to the distant systems fast
aggregated agent, representing an average control for enough to disconnect, break into self-standing subsystems, and
the micro-grid. The interaction between an agent and stabilize them.
the average agent is formulated as an optimal control This paper proposes a global distributed control system with
system subject to constraints. We model this average no central arbiter, as opposed to a centralized architecture
agent opponent using a method from particle physics as in [3]. We formulate this system using measurements of
called mean field. A Pareto equilibrium is approximated deviations from “local invariants.” These invariants (gauges)
by iteratively playing a two-person Pareto game between characterize what should be happening if the subsystems are
a control element and the mean field element. By iterating interacting properly. The invariants are functions of voltage,
fast enough, a good approximation is achieved with com- frequency, VARS, and cost. These functions are called gauge
putation that is linear in the number of control elements. functions. Gauges have a constant value along the optimal tra-
Hence, the method is scalable. jectory. For each device, we construct the gauges that remain
Several sub-systems, each controlled by a separate intelli- constant. Large deviations from invariant behavior are what
gent control element, may have interrelated structures, such as constitute the warning signals for initiating safe disconnec-
physical proximity or similar structures, and are best analyzed tion into subsystems. So disconnection is to be based on the
as a lumped single system. A single local controller may best size of deviations from expected measured invariant behavior
control a town, factory, wind-farm, or solar complex, so far of interactions of subsystems. In other words, observed devi-
as interaction with a larger global grid is concerned. What ations from invariance can be used to drive the controllers
is extraordinarily useful in our model is that we can employ that actuate separation of the global system into self-standing
a “divide and conquer” strategy. We can interactively com- subsystems when subsystems collapse.
pute homogeneous subsystems and apply “divide and conquer” The control system is determined by a Hamiltonian for
strategies to simplify control and speed up response. each device under control. This Hamiltonian is extracted from
rules characterizing the device by a formal procedure. The
procedure involves, in addition to the construction of the
A. Economic Justification Hamiltonian, the generation of state, co-state, and control
Renewable sources and nontraditional loads have the potential trajectories.
to improve the economics of electric power delivery. However, The formulas for the gauge invariants and the Hamiltonians
their characteristics differ substantially from standard power for controlled elements establish the local state of the micro-
components, and new methodologies are necessary to integrate grid and its dynamics. Our hybrid systems methodology based
them in an economic and secure fashion. Because of their on constrained calculus of variations [12] gives an algorith-
nontraditional dynamics, we have to control the micro-grid to mic approach to compute the control system variables. We
achieve performance with regard to cost, resiliency, and quality construct input–output equations for suggested self-standing
of service. The cost can be provided by the designer and/or subsystems. These are coupled by the connections between
operator of the micro-grid in the form of rules that are included them, similar to a Tellegen decomposition [18]. These cou-
in the criterion. The cost at the device level is determined by the pled equations are used as constraints. To make this feasible,
contribution of the device energy generation or consumption to the equations for each putative subsystem are obtained by an
the global micro-grid cost. For instance, how much the cost of averaging method from particle physics. We design a cost
a solar panel contributes to the total cost of the global operation function which evaluates the risks of catastrophic failure
can be specified by a rule. Resiliency is incorporated in the based on a given control scheme as a function of measur-
subsystems, and the quality of service, with regard to voltage able states of subsystems. The controls for the system are
KOHN et al.: MICRO-GRID DISTRIBUTED INTELLIGENT CONTROL AND MANAGEMENT SYSTEM 2967

Fig. 2. Overview of our architecture for intelligent control and management.

chosen to minimize the cost subject to the constraints. With to a centralized system where a central server manages the
real time computation that we cannot describe here, failure activities of elements connected to it, the micro-grid manage-
to compute a controller of acceptable cost cause the separa- ment server allows all of the element controllers to operate
tion of the system into independent subsystems. Simulation independently, and only communicate with the server when
is to be used to test decompositions into subsystems for per- there is a reason (such as new data, or an update to the cri-
formance characteristics. There are many ways to break up terion). The distributed implementation is based on creating
the graph of interconnections of sources and sinks of power a processor for each element controller and each PSC. The
use into self-standing systems, which are best remains to be distributed network is implemented in the cloud.
analyzed. The element controllers are the active participants in imple-
menting the intelligence of the micro-grid. They implement
conventional feedback control, inductive active learning, and
B. Architecture of the Intelligent Feedback Control intelligent feedback control. The intelligent feedback control
Fig. 2 illustrates our proposed architecture for the network involves the implementation of rule-based control laws that
of element controllers connected to the micro-grid manage- are partially learned from the sensory data. They control one
ment server. The convention we use in the figure is that the or more devices whose dynamic load depends on variations in
programmable signal controller (PSC) boxes represent pro- the environment, or in the operation of the device. An example
grammable signal conditioners that transform digital signals of the first kind is an air conditioning/heating unit; an exam-
from a computer to device signals and vice versa. PSCs are ple of the second kind is equipment such as a crystal-growing
crucial components for our intelligent control architecture and furnace that draws loads from 10 to 100 kW hours during
must be designed with care to effectively interface the intelli- its operation. Their inputs and outputs are illustrated in the
gent control with the physical devices. In Fig. 2, shaded wide bottom left of Fig. 2.
arrows represent outside signals, and the thin arrows represent As an example of the inputs to the element controller, con-
computer signals. The fundamental functions of the micro-grid sider the gas turbine in the first micro-grid example, illustrated
management server are to convey the optimization criteria and in Fig. 6. The element criteria for this gas turbine includes
goal settings for each of the element controllers, and to receive a soft rule to keep the combined power from the solar panels
real-time data from the network. In Fig. 2, the micro-grid man- and the gas turbine within a specified range that reduces the
agement server abstracts data from the element network to use need for power from the utility by 15%. The element crite-
in learning, and makes it available back to the element network ria also includes cost information regarding turning on and off
for restarting operations. The micro-grid management server the gas turbine, so it can achieve a low cost operation. The
also schedules the interaction with the element network, com- element settings include the threshold that is needed to achieve
posed of multiple active element controllers. The micro-grid the 15% reduction, and the cost factors for the gas turbine. The
management server is implemented as an independent agent, aggregated state estimate includes the power level generated
so it maintains the distributed nature of the system. In contrast by the solar panels, the power level generated by the utility,
2968 IEEE TRANSACTIONS ON SMART GRID, VOL. 6, NO. 6, NOVEMBER 2015

dynamics model and state estimator with the dynamic ele-

ment rules. Most installations that are already in operation
will have an existing conventional loop, so the other loops
in the element controller will be compatible with the exist-
ing loop. For new installations, the element controller is
designed in a modular fashion, so it is possible to modify
loop components as necessary. An example of the need for
a learning loop is a crystal furnace in which the thermal
control is well-understood and implemented through a conven-
tional feedback loop, but the magnetic containment dynamics
of the plasma is exceedingly difficult to model with conven-
tional control techniques. However, the combination of sensory
data with inductive learning and Maxwell equations expressed
with rules may provide a dynamic model that is locally valid
and can be used to design on-line modifications to the con-
Fig. 3. Element controller architecture. troller so that the resulting feedback loop produces the desired
behavior.
This example points out the need for inductive learning as
and the power being delivered to the load. It also includes a means to implement effective feedback control when the
costs incurred by other aspects of the micro-grid. This infor- dynamics can only be partially identified from historic sensor-
mation allows the element controller to decide when to turn actuator data. We construct a local model in real-time using
on or off the gas turbine. The micro-grid manager status will our inductive active learning technique. Our inductive active
provide updated information on the status of the solar panels learning technique constructs operational rules that character-
and the load, when the element controller queries it. The sen- ize the dynamics model of the device under active control
sor data for the gas turbine may include its status (on or off), (see [20]). It modifies the state estimator, the dynamics model,
its range on voltage, and a measurement on capacity. and the controller to minimize error criteria between the
The sensory signals and actuator command signals are not actual sensor-actuator data and the modeled sensor-actuator
routed through the element network, to increase reliability of estimates. One of the fundamental hard rules in the induc-
the element. If the element controller is disconnected from tive active learning loop is to ensure that no changes to the
the network due to unwarranted interruption, it will continue model will lead to unstable behavior on the micro-grid. This
functioning as a conventional feedback loop that maintains the requirement implies that the inductive active learning con-
active device functionality and issues an alarm to the micro- troller must include constraining rules of adaptability authority
grid management server. For this reason, our design separates of parameters and structures of the controller, dynamics model,
the sensor and actuator signals from the communication bus. and state estimator that ensure input–output stability for the
A detailed block diagram describing the functional blocks micro-grid.
of the element controller is illustrated in Fig. 3. The archi- Awareness of the effect of the device on the micro-grid
tecture of the element controller, for the most part, can be via “awareness rules” allow the learning loop to participate
implemented using digital signal processing [19]. The control in a synchronization strategy with the amalgamator to achieve
functionality of the element controller is realized through five the desired micro-grid behavior.
feedback loops: 1) conventional feedback loop; 2) inductive Knowledge uncertainty is unavoidable, even in simple grid
active learning loop; 3) synchronization loop; 4) intelligent systems. The structural modifications of any one of the
controller loop; and 5) amalgamator controller loop. controller, dynamics model, and state estimator, have one
These loops are coupled and they synchronize to achieve of the fundamental purposes to improve the quality of the
the overall desired functionality. models. This requires increasing the input–output matching
1) Conventional Feedback Loop: This loop (connecting the of the impedance of the model with the actual impedance of
controller, actuator model, dynamics model and state estimator the device, and thus reducing structural uncertainty. In gen-
with inputs from sensors, through a PSC, and element settings eral, we will represent impedance with an operator, because,
and criteria, and outputs control action through the actuators) in the micro-grid domain when nontraditional elements are
implements the classical actuator-sensor control. In most oper- present, single frequency (ac) representation of impedance is
ations, it is implemented through one or more proportional- not enough to characterize the dynamics of the device.
integral-derivative controllers. The loop is designed to fulfill 3) Synchronization Loop: When multiple active devices
requirements such as rise-time, overshoot limitation, phase and form part of a micro-grid, the classical approach is to intro-
gain margins, and bandwidth signal-to-noise ratio specifica- duce interdevice constraints to the specification of each of
tions. Because we expect the sensory signals to be affected by the active controllers. This requires extensive modeling and
environmental noise, the state estimator is designed to generate is specific to each micro-grid. Our approach is substantially
estimates that respond to this uncertainty. different. We use a concept developed in particle physics in
2) Learning Loop: The inductive active learning loop which the interaction of a single particle with the rest of
adds an inductive learning controller to adapt the controller, the particles is mediated by an “average model” called the
KOHN et al.: MICRO-GRID DISTRIBUTED INTELLIGENT CONTROL AND MANAGEMENT SYSTEM 2969

mean-field approximation. In our case, each element interacts The dynamic element rule repository is composed of three
with the rest of the elements in the micro-grid by participat- types of rules: 1) absolute rules; 2) hard rules; and 3) soft
ing in a dynamic synchronization with the mean field element: rules. Absolute rules represent operational principles that must
the controller of the element participates in a synchronization be satisfied for the operation of the device under control.
event with the amalgamator controller representing the rest of Examples of absolute rules are Kirchhoff voltage and cur-
the micro-grid. rent laws, and Ohmic relations. Absolute rules having to do
The essence of this synchronization involves the construc- with physical principles are provided in the controller. Hard
tion of a variational model of the device under control rules are rules that must be satisfied for adequate performance,
representing the aggregated micro-grid. The synchronization and soft rules are acquired by inductive active learning during
controller reaches and maintains a steady interaction when an the operation of the micro-grid or from phenomenological or
invariance (gauge) condition between the controller dynam- empirical observations. An example of a hard rule for a bat-
ics and the amalgamator dynamics is attained and maintained. tery is that the charge never goes below a specified level. This
When external events occur and this invariance condition type of rule may be provided by the person familiar with the
fails, the interaction between the controller and amalgamator device. An example of a soft rule is to store the power from
controller changes the control signal to reacquire invariance. the solar panel in a battery when the price from the utility
The information needed to build and maintain the variational is low and use the battery during peak hours when the price
model of the amalgamator requires data from the network. from the utility is high. This type of rule would be provided
The network data is interpreted by the amalgamator as sen- by the micro-grid operator. It is also possible that rules may
sory data. The amalgamator constructs an aggregated state be learned during operation. For example, the price used to
estimate projection to the space of the device under control. indicate storing solar power may be adapted during operation.
The aggregated state estimate projection is computed from the In our approach, soft rules are transformed to inequality con-
micro-grid state and the element state. straints, while absolute and hard rules are transformed into
The synchronization loop is affected by uncertainty in the equality constraints. The truth-value of the rules is translated
controller, the state estimator, and the amalgamator controller. into numerical binary values: 0 for false, and 1 for truth. Soft
We impose a quality synchronization level, high enough to rules have values that can be scaled into a closed [0, 1] interval.
guarantee satisfactory operation, measured in terms of the Soft rules transform the problem from a pure logical infer-
spread around the mean field equilibrium. ence to a hybrid one involving both inference and probabilistic
4) Intelligent Controller Loop: The intelligent controller extraction of control laws for the control elements that are in
loop implements the learning loop procedure, the compat- synchrony with the other control elements. The synchrony is
ibility procedures with the synchronization loop, and with established continuously with respect to the amalgamator con-
the conventional loop. It involves the flow of signals for trol law. Kohn and Zabinsky [21] showed that this provides
knowledge acquisition, and the generation of incremental active robust synchronization of the element with the other
updates to the device model, controller, and state estima- control elements and the grid management system (see Fig. 2).
tor to respond to dynamic events. The mechanical aspect During operation, only the rules that are active are trans-
of the intelligent loop procedure is unique to us. Classical formed into functional constraints. This is essential for real-
learning loops are query-answering procedures implemented time computational purposes. An absolute or hard rule is
through inference mechanisms such as modus ponem and rule- active when it is violated (value of 0). A soft rule is active
chaining strategies. Our approach involves the transformation when its truth-value is below a preagreed threshold. The
of the active rules in the dynamic element rule repository threshold may vary among soft values. Soft rules that are
into constraints and the transformation of the current query learned through inductive learning include the functional struc-
to optimization-tracking criteria. ture and the logic value. The chaining of the active rules is
In a network of classical devices, loads and generators, established by the interaction in the functional representation.
the Kirchhoff and Ohms laws accurately describe the energy The inductive learning loop is realized dynamically by con-
flow and state of the devices in the network. Computing structing a Lagrangian that adjoins the functional relations of
dynamic currents and voltages in the micro-grid can be the active rules to the criterion. The off-diagonal elements
achieved by extracting the Lagrangian or equivalently the net- of the Hessian of the Lagrangian associated with the opti-
work Hamiltonian that are used in the element controllers. mization problem represent the chaining of the corresponding
However, with nonclassical devices or sources that are perva- active rules.
sive in a micro-grid, network Hamiltonian and the Hamiltonian The intelligent controller implements a discovery engine
devices are almost impossible to determine by classical net- (an automated search engine) for extracting the active rules
work procedures. Therefore, we use rules to characterize the as a function of sensor and actuator signals. It also gen-
dynamic behavior of the system. The rules that character- erates new rules using a repair mechanism that forms part
ize the dynamic behavior of nonclassical components are of of the inductive learning loop procedure. This repair mech-
three different types; they may come from physical principles, anism is unique to our approach [22]. It operates by adding
desired operational behavior for the multiobjective optimiza- corrective increments to the control signal as a function of
tion, and/or from sensory observations. For example, voltage detected operational error. This is one of the fundamen-
regulation requirements (1%–4%), frequency spreads, and PF tal aspects of the learning capabilities of our intelligent
ranges can be expressed with soft rules. controller.
2970 IEEE TRANSACTIONS ON SMART GRID, VOL. 6, NO. 6, NOVEMBER 2015

The energy and potential functions are obtained from all the
elements in the micro-grid

E(x, ẋ) = Ek (x, ẋ)
k

V(x) = Vk (x)
k

due to conservation of energy. The Lagrangian model of cir-

cuits is well-known. For standard elements, such as capacitors,
inductors, transformers, classic generators and classic loads,
the relationships between current and voltage across the ports
are known. However, the Lagrangian for a rule-based system
must be learned. The relationship between current and voltage
across the ports for nonstandard elements, such as wind tur-
bines, solar panels and smart loads, are only partially known
Fig. 4. Amalgamator controller architecture.
and must be estimated from sensory data.
We use a dynamic tomograph (see [23] and [24]), and an
identification process to improve our estimate of Ek (x, ẋ) and
5) Amalgamator Control Loop: The amalgamator con- Vk (x) for each k corresponding to a nonstandard element. The
troller, appearing as a box in Fig. 3, implements the projection tomograph provides functions of the form
of the micro-grid dynamic model into the domain of the
element controller. A more detailed architecture of the amal- Ek (xk , ẋk , other states)
gamator controller is illustrated in Fig. 4. The purpose of
Vk (xk , other states)
the amalgamator is to establish explicitly the constraint of
the device under control with respect to the micro-grid. The so that we are building the Lagrangian “on the fly.” This
unique aspect of our approach is to provide effective synchro- is what we refer to as structural adaptation. Once we have
nization and reacquisition of synchronization using an optimal an estimate of the Lagrangian, we construct the estimated
synchronization strategy between the amalgamator and the Hamiltonian
controller.
It is important to address the issue of how to characterize Ĥ(x(t), p(t)) = p(t)T x(t) − L(x(t), ẋ(t))
uncertainty. In most implementations of smart grid control and
management, uncertainty is described by assuming a probabil- where p(t) is the momentum in the micro-grid.
ity distribution on the state. We consider this unsatisfactory
because the probability distribution is unknown and does
D. Hamiltonian Dynamics
not satisfy the Hadamard condition; meaning that the dis-
tribution should be continuously dependent on the observed Once we have the estimated Hamiltonian, we used it to
data. Our approach is to represent uncertainty on the state, construct the control Hamiltonian
the structures, and the signals with mean and spreads that
are continuously computable from the data without assum- HC (x(t), p(t), u(t)) = Ĥ(x(t), p(t)) + VC (x(t), u(t))
ing any probability distribution. This approach requires the
where u(t) is the control signal, and VC (x(t), u(t)) is a known
propagation of sets representing the states and the structures.
potential function. The rules that reside in the “dynamic ele-
This method is referred to in the literature as differential set
ment rules,” and the Hamiltonians of individual devices are
inclusion.
maintained in the “inductive learning controller” (see Fig. 3).
The Hamiltonian model
C. Tomograph for Estimating the Lagrangian via Inferencing
Given a micro-grid, a tomograph is used to construct the HC (x(t), p(t), u(t))
Lagrangian for nonstandard elements in the network of the
micro-grid. We characterize the micro-grid with a Lagrangian is constructed from the rules, physical principles, historical
that is the difference between the energy and potential data, and adapted to current sensory data. The procedure in the
field inductive learning controller obtains the optimal control law
by constructing the state equation (1), co-state equation (2),
L(x, ẋ) = E(x, ẋ) − V(x) and meta-control equation (3), shown as follows:

where the state for the kth element is the current flowing the ∂x(t) ∂HC  ∂φi (x, p, u)
= + λi (1)
element in the network ∂t ∂p ∂p
i
 t ∂p(t) ∂HC  ∂φi (x, p, u)
xk (t) = ik (τ )dτ. =− − λi (2)
0
∂t ∂x ∂x
i
KOHN et al.: MICRO-GRID DISTRIBUTED INTELLIGENT CONTROL AND MANAGEMENT SYSTEM 2971

all of the element control systems must be synchronized for a

coherent global system. Fig. 5 illustrates the synchronization
strategy, which is carried out via a companion element that
aggregates the dynamics of all elements.

III. M ICRO -G RID C ONTROL E XAMPLES

The micro-grid intelligent control and management system
is applied to two examples. The first example, shown in Fig. 6,
is a micro-grid with three generators; two solar and one gas
turbine. The data for this example came from a hotel com-
Fig. 5. Element synchronization via mean field. plex center, which combines utility energy with their two solar
devices and a gas turbine to reduce the electric power cost.
 −1 The load is an air conditioning unit for an office building with
∂u(t) ∂ 2 HC  ∂ 2 φi (x, p, u) varying requirements for air conditioning based on historical
=− + λi
∂t ∂u2 ∂u2 weather conditions measured by sensors. The air conditioning
i
  load accounted for a high percentage of the total load, so the
∂HC  ∂φi (x, p, u)
× + λi . (3) rest of the typical loads were ignored in this example. Their
∂u ∂u goal is to reduce power from the utility by 15% at peak hours.
i
The algorithm represented by (1)–(3), is implemented with In the first example, the central objective of the simulation
efficient numerical integration routines. In most cases, we was to demonstrate the acquisition of the model of each of
use Runga–Kutta of the third order, specifically modified the electric devices in the micro-grid from sensory data. This
for [21], [25], and [26]. sensory data is represented in the data base of rules as soft
If the costate equation rules. For example, there is a rule that characterizes the load
as a function of time and temperature: if time is noon, and
∂L(x(t), ẋ(t)) temperature is greater than 20 ◦ C, turn on the gas turbine
p(t) = (4)
∂ ẋ and combine power with solar panels. Many rules are of this
is regular (Dirac), then we have a unique solution sort, including ranges of outside temperature, time, sunlight
levels, and rules to reduce transient behavior while charging
ẋ(t) = f0 (x(t), p(t)) (5)
electric cars.
but if the Hessian of the Lagrangian is not invertible (not A simulation was run for 10 000 min, with historic data for
regular Lagrangian), then from the inverse function theorem, all sensor data. The relative cost of energy at the load for the
we cannot solve ẋ in (4). This case is likely for our rule- controlled micro-grid, with respect to the open loop uncon-
based system. Then, we separate the state vector into two parts: trolled system, is shown in Fig. 7. Fig. 7 illustrates how the
1) the physical state components and 2) the constraint state control system learns about operation of the components of the
components. For every device in the network, energy must be grid. The line in the figure is the mean performance associated
conserved, so the flow (current) across the device is a function with conventional control, provided by the implementer of the
of voltage and is expressed in a Hamiltonian system. micro-grid. Notice that the performance, as measured by the
The inductive learning controller takes the rules from the cost, is initially higher than the average cost of the conven-
dynamic element rules repository, and transforms the logic tional system. The transients that are observed in the curve are
rules, denoted ϕQ (x) and ϕ (k) , into continuous functions, caused by high variations in the load and renewable generation.
denoted φ̆Q (x) and φ̆ (k) . The x vector contains the state The control system for each of the DERs is relatively smooth.
estimate and the variables associated with the rules. The Each of the four element controllers (associated with the
Lagrangian L(x; ϕ̆Q (x), ϕ̆˙ (k) , ω) is estimated using the contin- two solar devices, the gas turbine, and the load) is optimizing
ualized rules and the method of Lagrange multipliers. The ω locally with the mean field in the amalgamator as a constraint.
vector represents the level of unsatisfaction of the soft rules. As the controllers learn the characteristics of the dynamics,
In order to minimize the Lagrangian, the rates of ẋ(t) and Ṁ(t) they are updated through the mean field. The result of the
are computed, where M(t) is a positive semi-definite matrix sequential optimizations is to locally synchronize the four
used to estimate the inverse of the Hessian of the Lagrangian. controllers. This process is repeated iteratively. The central
For details, refer to [27] and [28]. Now, the dynamics of the idea here is that the update of the mean field is done fast
device is formulated as G(ẍ, ẋ, x) and the Hamiltonian Ĥ(x, p) enough (on the order of once every 4 ms) to minimize the
can be computed. Thus, we have estimated the Hamiltonian for effects of delay, which are included in the rules defining the
a nonclassical device from rules! Given the Hamiltonian, we Hamiltonian.
update according to changes in the system. Then, we use our At time zero, in Fig. 7, the relative cost of energy at the
meta-control approach (see [25]), to optimize the Hamiltonian load compared to the open loop system is zero. In the early
and determine optimal control actions. This provides the feed- minutes, the element controllers are actively learning the vari-
back control loop for the inductive learning controller. Once ations due to weather impact on the load and solar panels. At
the local feedback control loop for an element is designed, 10 min, the element controllers begin to improve performance
2972 IEEE TRANSACTIONS ON SMART GRID, VOL. 6, NO. 6, NOVEMBER 2015

Fig. 6. Micro-grid example with three generators, two solar and one gas turbine, and an air conditioning unit as the load.

Fig. 7. Relative cost of energy at the load of the controlled micro-grid, compared to the uncontrolled system.

due to the constructed Hamiltonian models. The cost savings The operational characteristics of the DERs are taken
decrease until approximately 100 min, and then stabilize at from [31]–[35]. Characteristics of the PSCs were provided
approximately 30% saving for the rest of the time horizon. by a communication cellular company, with average point to
The simulation demonstrates the value of synchronizing point delay of 3.6 ms and a standard deviation of 1.8 ms.
the generation facilities with the varying load, and adjusting The average latency of our control software was estimated to
the scheduling of their contribution to the load dynamically. be 100 μs on a 64 ARMA processor. The simulation manage-
A significant portion of the savings were realized by shutting ment of the network, run on a PC, had an average duty cycle of
down the gas turbine for periods where the demand for air 600 μs. Additional details on the numerical parameter settings
conditioning was slight. are available from the authors upon request.
The second example, illustrated in Fig. 8, represents a small A simulation was run for the network with no active control,
city with a distribution/transmission network in which some and again for the network with our controller, and the three
standard generation units have been replaced with DERs. To wind turbines included batteries that were under active control.
demonstrate scalability, the second example includes 30 buses, Table I presents summary results comparing the two sim-
six DER generators (three are wind turbines), and one non- ulations. The first row in Table I shows the average PF
standard load (see Fig. 8). This IEEE 30-bus network has been without control, and then the average PF over the four simu-
used in [29], and in others [30]. lated weeks. As the control system learns from the data, the
KOHN et al.: MICRO-GRID DISTRIBUTED INTELLIGENT CONTROL AND MANAGEMENT SYSTEM 2973

number of nodes increases. The control system has the ability

to develop models of the devices and the network that are valid
for the entire range of operation, even when analytical models
are nonexistent or difficult to approximate. Our approach of
using local models that are dynamically updated is practical.
The distributed nature of our control (at the edge) increases the
robustness, resiliency, and cost-effectiveness of the micro-grid.

R EFERENCES
[1] A. L. Dimeas and N. D. Hatziargyriou, “Operation of a multiagent sys-
tem for microgrid control,” IEEE Trans. Power Syst., vol. 20, no. 3,
1447–1455, Aug. 2005.
[2] H. Jiayi, J. Chuanwen, and X. Rong, “A review on distributed energy
resources and MicroGrid,” Renew. Sustain. Energy Rev., vol. 12, no. 9,
pp. 2472–2483, Dec. 2008.
[3] J. Jimeno, J. Anduaga, J. Oyarzabal, and A. G. de Muro, “Architecture
of a microgrid energy management system,” Eur. Trans. Elect. Power,
vol. 21, no. 2, pp. 1142–1158, Mar. 2011.
[4] A. Keyhani, Design of Smart Power Grid Renewable Energy Systems.
Hoboken, NJ, USA: Wiley, 2011, chs. 5–6.
[5] A. L. Dimeas, A. Tsikalakis, G. Kariniotakis, and G. Korres,
“Microgrids control issues,” in Microgrids Architectures and Control,
N. Hatziargyriou, Ed. Hoboken, NJ, USA: Wiley, 2014, ch. 2.
[6] M. Kubisch, H. Karl, A. Wolisz, L. C. Zhong, and J. Rabaey,
“Distributed algorithms for transmission power control in wireless sen-
sor networks,” in Proc. Wireless Commun. Netw., vol. 1. New Orleans,
LA, USA, Mar. 2003, pp. 558–563.
[7] T. Mao, G. Feng, L. Liang, S. Qin, and B. Wu, “Distributed energy-
efficient power control for macro-femto networks,” IEEE Trans. Veh.
Technol., to be published.
[8] W. Kohn, V. Brayman, P. Cholewinski, and A. Nerode, “Control in
hybrid systems,” Int. J. Hybrid Syst., vol. 3, no. 3, pp. 109–150, 2003.
[9] W. Kohn, V. Brayman, and A. Nerode, “Control synthesis in hybrid
Fig. 8. IEEE 30 bus network, from [26]. systems with Finsler dynamics,” Houston J. Math., vol. 28, no. 2,
pp. 353–375, Jan. 2002.
TABLE I [10] W. Kohn, V. Brayman, and J. B. Remmel, “Multi-echelon inventory plan-
S IMULATION R ESULTS ON THE 30 B US N ETWORK ning system: Multiple agent hybrid control (MAHCA) implementation,”
Int. J. Hybrid Syst., vol. 2, nos. 1–2, pp. 105–128, Jan. 2002.
[11] W. Kohn and B. Cummings, “Free flight ATC using hybrid agent
systems,” Int. J. Hybrid Syst., vol. 10, no. 9, pp. 10–17, 1997.
[12] W. Kohn, A. Nerode, and J. B. Remmel, “Hybrid systems as Finsler
manifolds: Finite state control as approximation to connections,” in
Hybrid Systems II (LNCS 999), P. Antsaklis, W. Kohn, A. Nerode, and
S. Sastry, Eds. Berlin, Germany: Springer, 1995, pp. 294–321.
[13] W. Kohn and J. B. Remmel, “Hybrid dynamic programming,” in Hybrid
and Real-Time Systems (LNCS 1201), O. Maler, Ed. Berlin, Germany:
Springer-Verlag, 1997, pp. 391–396.
PF increases significantly. The second row of Table I shows [14] W. Kohn, Z. B. Zabinsky, and V. Brayman, “Optimization of algorithmic
parameters using a meta-control approach,” J. Glob. Optim., vol. 34,
the voltage regulator, and illustrates how the control system no. 2, pp. 293–316, Feb. 2006.
improves efficiency as the voltage regulator decreases toward [15] B. Jacob and H. J. Zwart, Linear Port-Hamiltonian Systems on Infinite-
the ideal 1%. The third row shows the third harmonic, as dimensional Spaces. Berlin, Germany: Springer, 2012.
[16] W. Kohn, A. Nerode, and J. Remmel, “Agent-based velocity control
a metric representing loss. The results with the control system of highway systems,” in Hybrid Systems IV (LNCS 1273), Berlin,
indicate increased efficiency. Germany: Springer-Verlag, 1997, pp. 174–214.
The cost of the system, with and without control, was esti- [17] J.-P. Aubin, Mathematical Methods of Game and Economic Theory.
Mineola, NY, USA: Dover, 2007.
mated using terms for the cost of battery aging, cost for turbine [18] P. Penfield, Jr., R. Spence, and S. Duinker, Tellegen’s Theorem and
maintenance, and cost for doing control. The average simu- Electrical Networks. Cambridge, MA, USA: MIT Press, 1970.
lated cost for the network with control was 70% of the cost [19] T. B. Welch, C. H. G. Wright, and M. G. Morrow, Real-Time Digital
Signal Processing From MATLAB to C With the TMS320C6x DSPs,
with no active control. 2nd ed. Boca Raton, FL, USA: CRC Press, 2012.
[20] H. Liu, Z. B. Zabinsky, and W. Kohn, “Rule-based control system design
for smart grids,” in Proc. IEEE Power Eng. Soc. (PES) Gen. Meeting,
IV. C ONCLUSION Minneapolis, MN, USA, Jul. 2010, pp. 1–5.
The proposed architecture allows distributed control of [21] W. Kohn and Z. B. Zabinsky, “Hybrid dynamic programming for rule
constrained multi-objective optimization,” in Proc. 21st Int. Symp. Math.
devices with only local models built by active learning on Program. (ISMP), Berlin, Germany, Aug. 2012.
sensor information. The two examples demonstrate that the [22] W. Kohn, V. Brayman, and J. Littleton, “Repair-control of enter-
control methodology scales well. This is due to its fully dis- prise systems using RFID sensory data,” IIE Trans., vol. 37, no. 4,
pp. 281–290, 2005.
tributed design with self-synchronization (as given by rules), [23] S. Helgason, Integral Geometry and Radon Transforms. New York, NY,
where the computation per node remains the same as the USA: Springer, 2010.
2974 IEEE TRANSACTIONS ON SMART GRID, VOL. 6, NO. 6, NOVEMBER 2015

[24] A. Markoe, Analytic Tomography. Cambridge, U.K.: Cambridge Univ. Zelda B. Zabinsky received the Ph.D. degree
Press, 2006. in industrial and operations engineering from the
[25] W. Kohn, A. Nerode, and J. B. Remmel, “Continualization: A hybrid University of Michigan, Ann Arbor, MI, USA, in
systems control technique for computing,” in Proc. Symp. Control Optim. 1985.
Supervis. Multiconf. Comput. Eng. Syst. Appl. (CESA IMACS), Lille, She is currently a Professor with the Department
France, Jul. 1996, pp. 507–511. of Industrial and Systems Engineering, University
[26] Z. B. Zabinsky and W. Kohn, “Solving non-linear discrete optimization of Washington, Seattle, WA, USA, with adjunct
problems via continualization: An interior-point algorithm,” in Proc. 21st appointments in the Department of Electrical
Int. Symp. Math. Program. (ISMP), Berlin, Germany, Aug. 2012. Engineering, Mechanical Engineering, and Civil and
[27] W. Kohn and J. Remmel, “Hybrid dynamic programming,” Int. J. Hybrid Environmental Engineering. She teaches courses
Syst., vol. 4, no. 1, pp. 1–7, 1992. in operations research. She has published several
[28] H. Liu, Z. B. Zabinsky, and W. Kohn, “Rule-based forecasting and pro- papers in the areas of global optimization, algorithm complexity, and opti-
duction control system design utilizing a feedback control architecture,” mal design of composite structures, and the book Stochastic Adaptive Search
IIE Trans., vol. 43, no. 2, pp. 143–152, Feb. 2011. in Global Optimization (Kluwer Academic, 2003), which describes research
[29] H. Lui et al., “Social welfare maximization in transmission enhance- on theory and practice of algorithms useful for solving problems with multi-
ment considering network congestion,” IEEE Trans. Power Syst., vol. 23, modal objective functions in high dimension.
no. 3, pp. 1105–1114, Aug. 2008. Prof. Zabinsky was a recipient of the Annual Teaching Award in Industrial
[30] IEEE Power Systems Test Case Archive. [Online]. Available: Engineering at the University of Washington several times. She is active in
http://www.ee.washington.edu/research/pstca/pf30/pg_tca30bus.htm, Institute for Operations Research and the Management Sciences and is an
http://fglongatt.org/OLD/Test_Case_IEEE_30.html, accessed Feb. 19, Institute of Industrial Engineers Fellow.
2015.
[31] P. Jamieson, Innovation in Wind Turbine Design. Hoboken, NJ, USA:
Wiley, 2011.
[32] P. Weicker, A Systems Approach to Lithium-Ion Battery Management.
Norwood, MA, USA: Artech House, 2014.
[33] C. Glaize and S. Genies, Lithium Batteries and Other Electrochemical
Storage Systems. Hoboken, NJ, USA: Wiley, 2013.
[34] R. Teodorescu, M. Liserre, and P. Rodriguez, Grid Converters for
Photovoltaic and Wind Power Systems. Hoboken, NJ, USA: Wiley, 2011.
[35] M. K. Kazimierczuk and D. Czarkowski, Resonant Power Converters,
2nd ed. Hoboken NJ, USA: Wiley, 2011.

Wolf Kohn received the Ph.D. degree in elec-

trical engineering and computer science from Anil Nerode received the Ph.D. degree in mathemat-
the Massachusetts Institute of Technology (MIT), ical logic from the University of Chicago, Chicago,
Cambridge, MA, USA, in 1979. IL, USA, in 1963.
He co-founded Clearsight Systems, Inc., Bellevue, He is a Goldwin Smith Professor of Mathematics
WA, USA, and Kohn–Nerode LLC, Dover, DE, with Cornell University, Ithaca, NY, USA. He is cur-
USA. He is currently a Research Professor with rently collaborating with his long-term co-author in
the University of Washington, Seattle, WA, USA, hybrid optimal control, W. Kohn, developing quan-
in addition to being the Chief Scientist with Atigeo, tum hybrid control, and co-founded Kohn–Nerode
LLC, Bellevue. He has taught at City University of LLC, Dover, DE, USA. With J. Myhill, he proved
New York Graduate Center, New York, NY, USA; the Myhill–Nerode Theorem, specifying necessary
Courant Institute of Mathematical Sciences, New York; New York University, and sufficient conditions for a formal language to be
New York; Rice University, Houston, TX, USA; Wayne State University, regular. His current research interests include mathematical logic, the theory
Detroit, MI, USA; and the University of California–Berkeley, Berkeley, CA, of automata, computability and complexity theory, mathematics of artificial
USA. He has published several refereed papers and book chapters, and has intelligence, control engineering, and quantum control of macroscopic sys-
been a keynote speaker at several control conferences. tems.
Dr. Kohn was a recipient of the Engineer of the Year Award from Boeing, Prof. Nerode has been a recipient of numerous awards. He is an Editor
Seattle; Lockheed, Clearlake, TX; and NASA, Houston Space Center, TX, of several journals and is frequently a Keynote Speaker. In 2012, he became
and the Teaching Award from MIT. a Fellow of the American Mathematical Society.