Received 15 May 2022; revised 5 September 2022; accepted 25 October 2022.

Date of publication 26 October 2022; date of current version 12 January 2023.

Digital Object Identifier 10.1109/OAJPE.2022.3217601

Faster-Than-Real-Time Hardware Emulation

of Transients and Dynamics of a
Grid of Microgrids
SHIQI CAO (Graduate Student Member, IEEE), NING LIN (Member, IEEE),
AND VENKATA DINAVAHI (Fellow, IEEE)
Department of Electrical and Computer Engineering, University of Alberta, Edmonton, AB T6G 2V4, Canada
CORRESPONDING AUTHOR: S. CAO ([email protected])
This work was supported by the Natural Sciences and Engineering Research Council of Canada (NSERC).

ABSTRACT Enhanced environmental standards are leading to an increasing proportion of microgrids

(MGs) being integrated with renewable energy resources in modern power systems, which brings new
challenges to simulate such a complex system. In this work, comprehensive modeling of a grid of microgrids
for faster-than-real-time (FTRT) emulation is proposed, which can be utilized in the energy control center
for contingencies analysis and dynamic security assessment. Electromagnetic transient (EMT) modeling is
applied to the microgrid in order to reflect the detailed device processes of the converter and renewable energy
sources, while the AC grid utilizes the transient stability modeling to reduce the computational burden and
obtain a high acceleration value over real-time execution. Consequently, a dynamic power injection interface
is proposed for the coexistence of the two simulation types. The reconfigurability and parallelism of the
field-programmable gate arrays (FPGAs) enable the whole system to be executed in FTRT mode with 51 times
acceleration over real-time. Meanwhile, three case studies are emulated and the results are validated by the
off-line simulation tool Matlab/Simulink R .
INDEX TERMS Battery energy storage system, doubly fed induction generator, dynamic simulation, elec-
tromagnetic transients, faster-than-real-time, field programmable gate arrays, hardware emulation, hardware-
in-the-loop simulation, microgrids, parallel processing, photovoltaic array, predictive control, real-time
systems, transient stability simulation, wind turbine.

I. INTRODUCTION To deal with the complexity caused by the integration of

T HE penetration of microgrids (MGs) with renewable

energy resources has been increasing in the power sys-
tem to alleviate the energy crisis and environmental issues [1].
MGs, a variety of models and control strategies have been
developed and investigated in the literature. Most of the
existing microgrid technologies focus on one specific aspect,
The traditional centralized structure of the power systems is such as energy management, control methods [4], [5], [6],
not efficient to meet the growing electricity demand due to [7], optimal power flow [8], or protection schemes [9], [10].
the power loss in the transmission network [2]. Therefore, Meanwhile, some simulation models for analyzing the micro-
modern power systems are experiencing a shift from central- grids are also investigated, e.g., the detailed and simplified
ized generation to distributed generation [3]. The distributed models for the energy storage system (ESS) in MGs are
generation with multiple MGs integrated may bring new presented in [11] and [12]. Although significant progress has
challenges including designing, operating, and coordinating been made, these research results lack hardware support and
the complex system. Considering that in such an integrated the simulation models mentioned above are hard to realize
network a small disturbance may spread to other areas and in real-time, which is insufficient for modern energy control
cause severe damage to the whole system, it further increases center that requires taking remedial actions immediately after
the complexity of controlling the microgrid cluster. a disturbance. Furthermore, it falls short of revealing the

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
36 VOLUME 10, 2023
Cao et al.: Faster-Than-Real-Time Hardware Emulation of Transients and Dynamics of a Grid of Microgrids

impact of multiple MGs on the AC grid with which they are Recently, the FPGA-based platform has caught a lot of
integrated. attention due to its strong computation capability. The FPGA
While there is still further research needed related to is capable of performing high-speed real-time simulations
real-time simulation of the MGs, significant progress has with time-steps in the sub-microsecond range [32], [33],
been made in the literature which can be categorized into [29], however, a single FPGA board is limited by its finite
four aspects: 1) the design and modeling of MG compo- amount of logical resources to perform parallel computations
nents to achieve real-time simulation [13], [14], [15], [16], in solving a large-scale system. As presented in [34], a large
[17], [18], [19], [20]; 2) the utilization of existing com- system is deployed on a single FPGA board by utilizing
mercial real-time simulators [21], [22], [23], [24]; 3) the parallel computation method, and the real-time simulation
development of real-time virtual test bed for MGs [25], can be achieved. However, the steady-state estimation is taken
[26], [27]; 4) novel computational approaches for distribution into consideration in [34], which only provides a single state
grids [29], [30]. Real-time simulation prefers the models power estimation. The emulation of modern power grids
with lower computational burden, and therefore, the rela- requires each circuit component to be modeled in detail to
tively simpler models or equivalent dynamic models have properly reflect the dynamic processed and performance of
been developed. Machine learning (ML) based models have the system. Nevertheless, low latency and hardware resource
also been investigated to selectively model and simulate occupation are also critical for industrial practice. In this
MG components [15], [16], which significantly reduced the work, comprehensive modeling of microgrids for faster-than-
hardware consumption; however, the ML-based models may real-time (FTRT) emulation on multiple FPGA boards is
ignore some dynamics of the MG components to obtain a proposed, which can not only provide real-time HIL simu-
higher simulation speed. Thus, these modeling approaches lation services for testing local MG control and protection
still require further research to meet the requirements of functions, but also enable the energy control center with
modern power systems for dynamic security assessment. effective strategies to improve the stability and security of
Commercial real-time simulators are usually executed on the larger grid. It is quite challenging, even for real-time HIL
high-performance processors or supercomputers to realize simulation, to model and emulate a microgrid cluster.
the high-speed simulation for microgrids [35], which are able Compared with the available commercial real-time simu-
to conduct the simulation of MG components in real-time. lation tools, the FPGA-based FTRT emulation has following
Although their accuracy and efficiency can be guaranteed due advantages: firstly, both the FPGAs and RT tools are able to
to the high processing frequency, the cost of the computing realize the HIL emulation, however, the FPGAs can achieve
equipment is the main factor that limits their widespread the FTRT emulation with the help of a creative solution
application. Meanwhile, the hardware-in-the-loop (HIL) sim- algorithm, and efficient parallel implementation, which is
ulation is also utilized in modern power systems and has much faster than the RT tools. Secondly, the capability and
also been applied to investigating microgrids [26], [36]. scalability of the FPGA-based FTRT platform are better than
The hardware resources of the existing simulators restricts currently available commercial RT simulators. Take the case
the scale of MG or MG clusters. Only a relatively small study in [32] (which was done with RTDS Technologies Inc.,
scale microgrid is tested and neither power dispatch nor Winnipeg, Canada) for example, the 141-bus system with
interactions among grid components are investigated in the 38 generators is simulated using RTDS and 4 PB5 racks
above papers. Virtual testbeds for cyber and physical data were needed [32]. To reduce the hardware resource cost, only
acquisition have caught a lot of attention in MG simulation, 5 buses and 2 generators are simulated on RTDS, while the
which can also coordinate with other unconventional mod- rest of the system parts are simulated on FPGA boards. The
eling methods to realize real-time simulation [27]. Due to cost and the hardware resource occupation of an FPGA-based
their flexibility and scalability, virtual testbeds are conve- platform are much lower than the RT simulators if the sys-
nient to reconfigure and adapt to various systems without tem scale becomes larger. Due to its accelerated mode of
depending on specialized hardware; however, they have lim- execution, an FTRT emulator can conduct traditional control
ited real-time simulation capabilities, especially in applica- center functions such as dynamic state estimation, power
tions that require deterministic, low-latency feedback loops flow, and contingency analysis much faster to predict the
for real hardware [27]. The lack of efficient physical inter- system condition in response to adverse events. It can then run
faces is also another one of their drawbacks. Furthermore, multiple scenarios in parallel to devise and recommend viable
some novel approaches were also proposed for real-time solutions to dynamically restore the voltages and frequencies
simulation of MGs. Commonly, EMT simulation method to nominal values.
is adopted for detailed representation of MG components For a realistic power system, the role of the FTRT emula-
described by non-linear differential algebraic equations. Par- tion might be more prominent, as it is able to collect real-time
allel solution approaches were therefore put forward to data from the field and provide an optimal solution without
gain a high-speed execution. These methods usually require cutting off the fault area. FTRT emulation improves the grid
sufficient hardware resources for parallel processing, and stability by predicting the grid performance, which is beyond
multiple FPGA boards are utilized to realize real-time the capability of real-time (RT) simulation tools [37]. Further-
simulation [30], [31]. more, the FPGAs require lower cost and power consumption

VOLUME 10, 2023 37

 where the variables with the subscription ref are the ref-
erence values, G denotes the solar irradiance, αT refers to
the temperature coefficient, VT is the thermal voltage, and
T represents the absolute temperature. Meanwhile, I0 is the
diode saturation current.
In a practical PV array, a large amount of PV panels
are arranged in an array to provide sufficient energy. The
topology of a typical PV array with Np parallel strings and
each of them containing Ns series panels is given in Fig. 1 (c).
The equivalent circuit is still available in a PV array, where
the (1) can be expanded as
FIGURE 1. Configuration of a PV array model. Vpv (t)+Ns Np−1 Rs Ipv (t)
Ipv = Np iirr − Np I0 (e Ns VT − 1)
compared with RT tools, enabling them to be of service
Ipv (t)Rs + Np Ns−1 Vpv (t)
in dynamic security assessment (DSA). With the help of − . (4)
creative solution algorithm and efficient parallel implemen- Rp
tation, FTRT emulation provides sufficient time for DSA to The non-linear nature of the anti-parallel diode makes the
take remedial actions [38]. emulation of the solar cell complex. To reduce the compu-
Apart from the hardware, the solution strategy also has tational burden as well as shrink the emulation time. The
a significant impact on emulation efficiency. To reflect the equivalent circuit in Fig. 1 (c) can be further simplified by
detailed operating conditions of the components in MGs, the Norton’s Theorem, resulting in a two-node circuit as given in
electromagnetic transient (EMT) simulation is adopted, while Fig. 1 (d), where
transient stability simulation is applied to the host system to
reduce the hardware resource utilization. Due to the various Np (Gdio + Gp )
GPVarray = , (5)
simulation types, a dynamic voltage injection interface strat- Ns (Gdio Rs + Rs Gp ) + Ns
egy for hardware emulation with less data communication Np (iirr − IDeq )
JPVarray = , (6)
is also proposed, which significantly reduces the hardware Gdio Rs + Rs Gp + 1
resource utilization and accelerates the hardware emulation.
where Gp refers to the conductance of the parallel resistor, the
The rest of this paper is expanded as follows: Section II
Gdio and IDeq are given as follows:
introduces the detailed modeling of the components in a
typical microgrid. The interface strategy of the integrated AC vdio (t)

system and microgrids is specified in Section III. Section IV ∂idio I0 · e VT

Gdio = = , (7)
demonstrates the hardware design of proposed FTRT emula- ∂vdio VT
tion on FPGAs. The validation of EMT-dynamic emulation IDeq = idio − Gdio · vdio . (8)
results of MG cluster and case studies for power dispatch is
The operation of a microgrid under the islanded mode
given in Section V. Section VI presents the conclusion and
requires energy storage system to balance the generation
future work.
and demand as well as regulate the grid voltage. A battery
energy storage system (BESS) is applied in each microgrid,
II. DETAILED EMT MODELING OF MICROGRID
which contains a battery system, and a DC/AC converter.
COMPONENTS
To emulate the non-linear part of the battery system, the
A. PHOTOVOLTAIC (PV) AND BATTERY ENERGY
EMT simulation is applied for calculating BESS. The battery
STORAGE SYSTEM (BESS) EMT MODEL
is modeled [11] as an ideal controllable voltage source in
Fig. 1 (a) provides the equivalent circuit representation of a series with an equivalent internal resistance Rbatt , as given
solar cell, which consists of an irradiance-dependent current in Fig. 2 (a). The open-circuit voltage of the battery Voc can
source, an anti-parallel diode, shunt resistor (Rp ), and series be represented based on the actual battery charge (it) by a
resistor (Rs ) [39]. The output current of the single solar cell non-linear equation expressed as follows.
can be expressed based on Kirchhoff’s Current Law as:
Q
ipv = iirr − idio − ip , (1) Voc = V0 − K · it + A · exp(−B(it)), (9)
Q − it
where iirr and idio refer to the irradiance current and the where V0 refers to the battery constant voltage, K is the polar-
current flowing through the anti-parallel diode as given in (2) isation voltage, Q represents the battery capacity, and A and
and (3), respectively. B denote exponential zone amplitude and exponential zone
G time constant inverse, respectively. According to Kirchhoff’s
iirr = iirr,ref · [1 + αT · (T − T ref )], (2)
Gref laws, the battery voltage (Vbatt ) can be derived as:
vdio (t)
idio (t) = I0 · (e VT − 1), (3) Vbatt = Voc − Ibatt Rbatt . (10)

38 VOLUME 10, 2023

Cao et al.: Faster-Than-Real-Time Hardware Emulation of Transients and Dynamics of a Grid of Microgrids

machine can be expressed as:

dx(t)
= +A · +x(t) + B · +u(t), (11)
dt
y(t) = +C · +x(t), (12)

where x, y, and u are vectors that refer to the fluxes, currents,

and input voltages, respectively. The DAEs of the induction
machine contains 4 rotor and stator circuit equations in the
α-β frame [40], given as:
−Lr Rs λαs (t) Lm Rs λαr (t)
λ˙αs (t) = + + Vβs , (13)
Ls Lr − Lm2 Ls Lr − Lm2
−Lr Rs λβs (t) Lm Rs λβr (t)
λ˙βs (t) = + + Vαs , (14)
Ls Lr − Lm2 Ls Lr − Lm2
Lm Rr λαs (t) −Ls Rr λαr (t)
λ˙αr (t) = + − ωr λβr (t), (15)
Ls Lr − Lm2 Ls Lr − Lm2
L R λ (t) −Ls Rr λβr (t)
FIGURE 2. (a) Equivalent circuit of BESS, and (b) control
˙ (t) = m r βs +
λβr + ωr λαr (t), (16)
strategy of PV&BESS inverter.
Ls Lr − Lm2 Ls Lr − Lm2
where the λαs , λβs , λαr , and λβr refer to the fluxes of stator
and rotor in α-β frame, respectively, Vαs and Vβs are the input
voltages, Rs and Rr are the stator and rotor resistance, and
Ls , Lr , and Lm represent the stator, rotor, and magnetizing
inductance, respectively.
The 5th differential equation which describes the mechan-
ical dynamics is given as
P
ω˙r (t) = · (Te (t) − Tm (t)), (17)
2J
FIGURE 3. Topology of a typical DFIG. where P and J are constant values which refer to the poles and
inertia of the induction machine, ωr refers to the electrical
angular velocity. The electromagnetic torque Te and mechan-
The controllers of the PV system and BESS also share
ical torque Tm can be obtained by
some similarities since they are both based on the d-q frame,
as given in Fig. 2 (b). For a PV converter, the controller 3
Te (t) = P(iβs (t)λαs (t) − iαs (t)λβs (t)), (18)
regulates the DC voltage on the d-axis according to the ref- 2
erence voltage which is generated by the maximum power 1
Tm (t) = ρπrT3 v2w F(rT , vw , ωr ), (19)
point tracking (MPPT) algorithm, while in BESS, the DC 2
voltage or the active power is the control target. On the q-axis,
where ρ represents the air density, rT is the wind turbine
depending on the grid condition, the converters can control
radius, and F refers to a non-linear function of ωr , rT , and
either the PCC voltage or its reactive power.
the wind speed vw , where the detailed function F can be
found in [41]. The stator currents iαs and iβs are calculated
B. EMT MODEL OF WIND TURBINE from (12), where the vector y can be expanded as
A typical doubly-fed induction generator (DFIG) is applied in
the microgrid as shown in Fig. 3, which includes an induction y = [iαs (t), iβs (t), iαr (t), iβr (t)]T , (20)
machine, a grid side voltage source converter (GSVSC), and
the corresponding coefficient matrix C can be expressed as
a rotor side voltage source converter (RSVSC). The principle
of the DFIG is that the rotor windings are fed with a back-
 Lr −Lm 
2
Ls Lr −Lm
0 2
Ls Lr −Lm
0
to-back voltage source converter which controls the rotor Lr −Lm 
0 0

2 2 
and grid currents, while the stator windings are connected to Ls Lr −Lm Ls Lr −Lm . (21)

C =  −Lm

Ls
the grid. The converter controls the rotor currents, which is  Ls Lr −Lm2 0 2
Ls Lr −Lm
0 

possible to adjust the active and reactive power fed to the gird −Lm Ls
0 L L −L 2
0 L L −L 2
s r m s r m
under various wind speed and grid conditions.
For FTRT hardware emulation, the induction machine in The continuous differential equations should be dis-
the DFIG is represented by 5th order differential algebraic cretized before the numerical calculation. The correspond-
equations (DAEs). The state-space equation of the induction ing time-discrete for (11) can be obtained after utilizing

VOLUME 10, 2023 39

 FIGURE 5. MMC control strategy: (a) outer loop control, and
(b) inner loop control.
FIGURE 4. Illustration of MMC modeling: (a) three-phase
topology, (b) average value model. output voltage can be expressed as:

the trapezoidal rule: N N

Vdc X X
Vo = ·( Sui + Sli ). (25)
N
1t 1t i=1 i=1
x(t + 1t) = (I − A )−1 [(I + A )x(t)]
2 2
1t where Sui and Sli refer to the switching functions of upper and
+ B (u(t + 1t) + u(t))], (22) lower arms, respectively. The control strategy for the MMC
2
model is similar to the inverter controller without the MPPT
where 1t refers to the time-step of the EMT emulation part. The MMC adopts a two-loop control scheme where the
utilized in the wind turbine, which is defined as 50 µs, and outer-loop controller is in charge of converter functions which
x(t + 1t) denotes the vector of state variables of next time- regulate the active/reactive power and bus voltage, as given in
step. Fig. 5 (a). The reference current i∗d,q is calculated as:
Z
C. MMC AVERAGE VALUE MODEL i∗d,q = Kp (V ∗ − V ) + Ki (V ∗ − V )dt, (26)
The configuration of a MMC-based three-phase DC/AC
converter integrated with the renewable energy is provided where the Kp and Ki are constants, V ∗ and V refer to the
in Fig. 4 (a), where each phased contains 2N half-bridge control target reference and their feedback, respectively. The
submodules (HBSMs) and each arm has an arm inductor. MMC current controller in d-q frame is provided in Fig. 5 (b),
To avoid asynchronous data communication between BESS where the signals with the superscription ∗ represent the
and the converter system, the EMT simulation is also applied reference values. The current controller amplifies the current
in emulating the integrated DC/AC converter as well as the error to obtain the voltage Vd,q , which is converted into
MMC stations in the microgrids with a time-step of 200µs. three-phase signals mabc that are sent to the MMC inner-loop
The hardware-based FTRT emulation prefers the models controller employing phase shift strategy.
which require the least computational burden, and therefore,
the average value model (AVM) for MMCs is utilized. The
III. AC GRID MODELING AND INTERFACE STRATEGY
HBSMs can be simplified into the controlled voltages sources
Fig. 6 shows a modified IEEE 39-bus system [42] integrated
as given in Fig. 4 (b). When the upper switch K1 is turned on,
with seven DC microgrids, where the wind turbine (WT) and
the submodule is inserted, while the submodule is bypassed
PV-BESS system in each microgrid are linked to a five ter-
when the K2 is turned on. Assuming that the capacitor volt-
minal (5-T) LVDC system, where the 5-level MMC stations
ages are well balanced in the AVM, the average value of the
are utilized. Meanwhile, the voltage and current levels for
capacitors and the equivalent voltage of an arbitrary submod-
the MMC station are 1 kV and 3 kA, respectively. Under a
ule can be derived as.
base power of 1 MVA for the host grid, each wind turbine
Vdc has a rated 2 p.u. active power, both PV and BESS have
Vcap1 = Vcap2 = · · · = Vcap2 = . (23) a standard 1 p.u. rated power, and local loads of 500 kW
N
Vdc and 1 MW are connected with the PV-BESS system and WT,
VSMi = · Si . (24) respectively. When a microgrid operates under the islanded
N
mode, the generated power from the renewable energy is
where Si represents the switching function which yields 1 and utilized for supporting the local loads, while the extra active
0 when the submodule capacitor is inserted and bypassed, power is stored in the batteries. On the other hand, in the grid-
respectively. As a result of the well-balanced condition in connected mode, each microgrid provides up to 3 MW active
AVM, the circulating current can be neglected the AC side power to the host system.

40 VOLUME 10, 2023

Cao et al.: Faster-Than-Real-Time Hardware Emulation of Transients and Dynamics of a Grid of Microgrids

FIGURE 6. Topology of the host grid connected with multiple microgrids.

A. AC GRID MODELING
1) SYNCHRONOUS GENERATOR AND CONTROL
SYSTEM
Transient stability simulation is basically solving a series
of nonlinear differential algebraic equations (DAEs), which
can represent the detailed dynamics of synchronous genera-
tors. Meanwhile, the network equations containing transmis-
sion lines and loads are given in (28), which calculate the
non-generator bus voltages and generator output currents of
the AC system.
FIGURE 7. Excitation system of the synchronous generator.
Ẋ(t) = F(X, U, t), (27)
G(X, U, t) = 0, (28) electrical equations. Therefore, the excitation system con-
tributes 3 additional state variables in vector X and the 9th -
where U refers to the vector of inputs such as field voltages order differential equations are utilized for representing the
(Vfd ) and mechanical torque (Tm ), X represents the vector synchronous generators of the proposed AC system, which
of state variables of the synchronous machine. In this work, are solved by the explicit 4th -order Adams-Bashforth (AB4)
a detailed synchronous machine model is utilized, which given as:
contains two mechanical equations and four rotor electrical
h
equations with 2 windings on the d-axis and 2 damping x(t + h) = x(t) + · [55F(t) − 59F(t − h)
windings on the q-axis, and therefore, the vector X can be 24
+ 37F(t − 2h) − 9F(t − 3h)], (30)
expressed as
X(t) = [δ(t), 1ω(t), ψfd (t), ψd1 (t), ψq1 (t), ψq2 (t), . . .]T , where h refers to the time-step of AC grid, which is defined
as 1 ms.
(29)
where δ refers to the rotor angle, and 1ω represents the 2) AC NETWORK EQUATIONS
derivative of angular velocity. Meanwhile, an excitation sys- The AC network mainly contains the transmission lines,
tem with automatic voltage regulator (AVR) and power sys- transformers, and loads. In order to reduce the computa-
tem stabilizer (PSS) is also included in the synchronous tional burden during the hardware execution, the transmis-
generator as given in Fig. 7, where vt refers to the terminal sion lines and transformers are represented by the lumped π
voltage of the synchronous machine, TR , Kstab , Tw , T1 , and model. According to [43], only the fundamental frequency is
T2 are constant parameters, which can be found in [43], and considered, which means the other frequency ranges except
v1 , v2 , and v3 are the intermediate variables in the excitation for the system fundamental frequency between the micro-
system, which are solved together with the mechanical and grids and the AC network are ignored. The fixed loads

VOLUME 10, 2023 41

and shunt compensators are integrated with the admittance which significantly increases the scalability of the proposed
matrix, given as FTRT emulation. However, the current injection interface
PLoad + jQLoad is not suitable for the proposed FTRT emulation. Since the
YLoad = 2
, (31) synchronous generators are modeled as voltage sources and
Vpcc
integrated with the network equations, the current injection
where PLoad and QLoad represent the active and reactive interface will introduce additional calculations. Therefore,
power of the load, respectively, and Vpcc refers to the bus a voltage injection interface strategy is put forward in this
voltage at the point of common coupling. Following the intro- work to maintain a constant admittance matrix and accelerate
duction of the admittance matrix of the AC grid, the network the hardware emulation.
equations in (28) can be expanded as Under this scheme, a converter is taken as a simplified
  
In

Ynn Ynr Vn
 synchronous generator with a varying voltage that does not
= , (32) have detailed electrical and excitation circuits. The reduced
Ir Yrn Yrr Vr
network equation (33) can be further expanded as
where the subscript n refers to the generator nodes with     
current injection, and r represents the remaining nodes with- Ii Yii Yim Vi
= , (43)
out synchronous generators. Due to the absence of current Im Ymi Ymm Vm
injection in the non-generator buses, the current vector of the where the matrix with subscription i refers to the generator
remaining nodes Ir = [0] and In can be derived as node represented by the detailed Park’s equations, m denotes
In = Yreduced · Vn , (33) the buses where the converter stations connected to, and i + m
constitutes the n generator nodes in (32).
Yreduced = Ynn − Ynr Y−1rr Yrn , (34)
Since different synchronous machines and converter sta-
Although the generator voltages Vn are not directly known tions are interconnected together in a transmission network,
after solving the DAEs, the relationship between Vn and In the variables should be expressed in a common reference
can be expressed as frame, which is called a synchronously rotating D-Q frame.
The above complex matrix equation (43) yields 4 real matrix
VD = ID · u1 + IQ · u3 + u5 , (35)
equations in D- and Q-axis, given as
VQ = ID · u2 + IQ · u4 + u6 , (36)
IDi = Rii VDi − Bii VQi + Rim VDm − Bim VQm , (44)
where u1−6 can be obtained following the acquirement of new
IQi = Rii VQi + Bii VDi + Rim VQm − Bim VDm , (45)
state variables, which are given below.
IDm = Rmi VDi − Bmi VQi + Rmm VDm − Bmm VQm , (46)
u1 = −Ra , (37)
IQm = Rmi UQi + Bmi VDi + Rmm VQm − Bmm VDm , (47)
00
u2 = Xad sin2 (δ) + Xaq
00
cos2 (δ) + Xl , (38)
where R and B represent the real part and the imaginary
00
u3 = −(Xad cos2 (δ) + Xaq
00
sin2 (δ) + Xl ), (39)
part of the corresponding Y matrix. The values of VDi and
u4 = −Ra , (40) VQi associated with the detailed synchronous machines are
u5 = −cos(δ)Ed00 − sin(δ)Eq00 , (41) not directly known, but the voltages under the reference d-
u6 = cos(δ)Eq00 − sin(δ)Ed00 . (42) q frame can be represented by the state variables following
each step of integration, given as
Obviously, with the ascertained relationship between Vn
and In , all the unknown vectors in (32) can be solved directly. 00 00 ψq1 ψq2
vd = −Ra id + iq (Xaq + Xl ) − Xaq ( + ), (48)
Xq1 Xq2
B. PROPOSED MICROGRID AND AC GRID INTERFACE 00 00 ψfd ψd1
vq = −Ra iq − id (Xad + Xl ) + Xad ( + ), (49)
Since transient stability simulation is adopted for AC grid Xfd Xd1
analysis, an interface is proposed so that it is compatible Lad · Lfd · Ld1
with the aforementioned EMT models. As given in Fig. 1
00
Xad = Xl + , (50)
Lad · Lfd + Lad · Ld1 + Lfd · Ld1
(d), the PV arrays are modeled as the current sources, while Laq · Lq1 · Lq2
the wind turbine and BESS are modeled as voltage sources.
00
Xaq = Xl + , (51)
Laq · Lq1 + Laq · Lq2 + Lq1 · Lq2
As for the interface strategy, theoretically, there are three
available strategies that can be applied, which are P-Q inter- where Ra , Xl , Xq1 , Xq2 , Xfd , Xd1 , Lad , Lfd , Ld1 , Laq , Lq1 , and
face, voltage interface, and current interface. In the P-Q Lq2 refer to stator resistance, stator leakage reactance, reac-
interface, the converter stations can be treated as time-varying tance of damper winding q1, reactance of damper winding q2,
loads in transient stability simulation of a traditional AC/DC field winding reactance, reactance of damper winding d1, d-
grid, however, the admittance matrix needs to be updated axis mutual inductance, field winding inductance, inductance
every time-step, resulting in a heavy computational burden. of damper winding d1, q-axis mutual inductance, inductance
The utilization of dynamic voltage/current injection inter- of damper winding q1, inductance of damper winding q2,
face avoids solving the admittance matrix in every time-step, respectively, which are constant values of the synchronous

42 VOLUME 10, 2023

Cao et al.: Faster-Than-Real-Time Hardware Emulation of Transients and Dynamics of a Grid of Microgrids

generator. Meanwhile, the variables with superscription 00

represent the subtransient reactance and electromotive force.
In order to correlate the components of the voltages and
currents expressed in d-q frame to the rotating common
reference D-Q of the network, the following reference frame
transformation equations are used,
VD + jVQ = (vd + jvq ) · ejδ , (52)
ID + jIQ = (id + jiq ) · e , jδ
(53)
where the variable δ in (52) and (53) represents the syn-
chronous generator rotor angle given in (29). Therefore,
by combining (48)-(53), the values of VDi and VQi can be
calculated by the new state variables. Following the solution
of the microgrid functions, the components VDm and VQm
are available for solving IDm and IQm in (46) and (47). Since
distinct time-steps are adopted in TS and EMT emulation, the
data synchronization of the proposed interface is as follows,
the microgrid undergoing EMT emulation provides the volt-
ages in D-Q frame to the AC grid in every 5 EMT time-steps,
and the injected voltages are solved together with the network
equations in the AC system. Then the calculated currents at FIGURE 8. Hardware block design for FTRT emulation of the host
PCC are in return sent to microgrid systems to continue the system integrated with 7 microgrids.
EMT emulation, as shown in Fig. 6.
data exchange is realized by connecting the input/output ports
IV. FTRT HARDWARE EMULATION OF GRID OF among the hardware modules. According to the correlation
MICROGRIDS ON FPGAs among the subfunctions, the hardware modules are designed
The full deployment of the AC system integrated to be calculated in parallel or series. The hardware block
with the microgrid cluster requires two Xilinx Virtex R design and the data stream are provided in Fig. 8. After design
UltraScale+TM VCU118 boards containing XCVU9P FPGA synthesis and device mapping, the bitstreams were down-
which includes 6840 DSP slices, 1182240 look-up tables loaded to the target FPGA boards via the Joint Test Action
(LUTs), and 2364480 flip-flops (FFs). The host system as Group (JTAG) interface as given in Fig. 9. The Quad Small
well as the microgrid-1 (MG-1) to MG-3 in Fig. 6 are Form Pluggable (QSFP) interfaces are connected with cable
deployed on the VCU118 Board-1, and the remaining four and the build-in IP Aurora 66B/64B core is utilized to realize
microgrids are implemented on the VCU118 Board-2. The the data communication. Due to the little data required of
sufficient hardware resources enable the MGs to be executed the proposed AC/Microgrid interface, the PCC voltages and
on the FPGA board in FTRT mode. The reconfigurability currents in D-Q frame are chosen as the communication data
of FPGAs allows the hardware resources to be adjusted between the FPGA boards. Since the MG 1-3 are conducted
to accommodate and represent practical systems [44], and in Board-1, the data exchange only happens inside Board-
therefore it is suitable for the hardware-in-the-loop (HIL) 1. As for Board-2, the output voltages (VMG4−7,MMC,D , and
emulation and dynamic security assessment in the energy VMG4−7,MMC,Q ) of MG 4-7 are delivered to the build-in IP
control center. Aurora Core through the AXI bus, then the voltage data is
The hardware implementation can be conducted in two sent to Board-1 via the QSFP bidirectional cable. After solv-
ways: utilizing hardware programming language, and block ing the network equation in Board-1, the calculated currents
design. The former method models each circuit part in hard- (IMG4−7,MMC,D , and IMG4−7,MMC,Q ) are delivered to Board-2
ware language such as VHDL, which is complicated and through the QSFP cable and sent to the Aurora core. Finally,
requires hardware programming experience of the researcher. the current data is solved together with the VSC stations to
The Xilinx R high-level synthesis software Vivado HLS R keep the EMT simulation going on. Meanwhile, the output
is able to transform the C/C++ code into intellectual prop- digital data is transferred to analog data via the digital-to-
erty (IP) which contains corresponding input/output ports analogic converter (DAC) board, so that the waveforms can
in VHDL format. The circuit parts and subsystems which be displayed on the oscilloscope.
consist of the microgrid cluster are programmed in C/C++ Table 1 provides the hardware resource utilization and the
code before the hardware design. After IP generation, each latencies of the hardware modules in the FTRT emulation,
circuit component is converted to a hardware module and where the Tclk in Table 1 refers to the unit of the latency.
exported to Vivado R for block design,, which significantly For example, the 39 Tclk means the latency of the PVmodel
increases the flexibility and reduces the complexity of hard- part is 39 clock cycles. As mentioned, the components in the
ware implementation. In a practical real-time application, microgrid including the PV array, wind turbine, and LVDC

VOLUME 10, 2023 43

 FIGURE 10. FTRT emulation results of three-phase-to-ground
FIGURE 9. Experimental hardware-in-the-loop (HIL) emulation
fault: (a) rotor angles, (b) generator voltages (G6-G9),
platform.
(c) generator frequencies, (d) output power of MMCs in MG-1.
TABLE 1. Details of major microgrid cluster hardware modules.

FIGURE 11. Comparison of FTRT results and offline results:

(a) mechanical torque, (b) electrical torque, (c) generator output
id (G1-G10), and (d) generator output iq (G1-G10).

should wait for the wind turbine part to finish computation to

keep data synchronization. Therefore, the total FTRT ratio of
the proposed microgrid cluster is about 51.

V. FTRT EMULATION RESULTS AND VALIDATION

The hardware emulation of the AC grid integrated with micro-
system are modeled by EMT emulation. The hardware mod- grid cluster is conducted on the FPGA-based FTRT platform
ules of PV stations are fully parallelized, and therefore, the (Fig. 9), and the proposed FTRT emulation and the interface
total latency of a PV system is 41 Tclk. The execution time strategy are validated by comparing the results with those of
of the PV array is calculated as 41 × 10 ns = 0.41µs, the off-line simulation tool Matlab/Simulink R .
where the clock cycle is defined as 10 ns under the FPGA
frequency of 100 MHz. Then the FTRT ratio can be expressed A. CASE 1: THREE-PHASE-TO-GROUND FAULT
200µs
as 0.41µs = 487. Similarly, due to the parallelism, the FTRT At t = 5s, the three-phase-to-ground fault lasting 200 ms
ratios of the BESS, wind turbine, and DC system can be occurs at Bus 21 as given in Fig. 6. The rotor angles, output
200µs 50µs 200µs
solved as 73×10ns = 273, 97×10ns = 51, and 96×10ns = 208, voltages, and frequencies of the synchronous generators start
respectively. Meanwhile, the latency of the AC system is 33+ to oscillate immediately, as given in Fig. 10 (a)-(c), where the
196 + 29 + 21 = 279Tclk, thus the FTRT ratio reaches over dashed lines refer to the results from Simulink R and the solid
1ms
279×10ns = 358. Although a 358 FTRT ratio can be achieved lines represent the FTRT emulation results. Fig. 10 (d) pro-
in the transient stability emulation, the overall FTRT ratio is vides the output power of the VSCs in MG-1, which indicates
determined by the EMT emulation part, since the AC system that the waveforms from the FTRT emulation match with

44 VOLUME 10, 2023

Cao et al.: Faster-Than-Real-Time Hardware Emulation of Transients and Dynamics of a Grid of Microgrids

FIGURE 14. Impacts of excess power injection to the host grid:

FIGURE 12. Impact of lack of power generation in PV array: (a) generator relative rotor angles, (b) generator voltages,
(a) generator relative rotor angles, (b) generator voltages, (c) frequencies, (d) output power of MMCs in MG-1.
(c) frequencies, (d) output power of MMCs in MG-1.

FIGURE 15. Inter-MG coordination results: (a) generator relative

FIGURE 13. Microgrid internal power balance results: rotor angles, (b) generator voltages, (c) frequencies, (d) output
(a) generator relative rotor angles, (b) generator voltages, power of MMCs in MG-1.
(c) frequencies, (d) output power of MMCs in MG-1.
lack of power injection at Bus 39, as given in Fig. 12 (a).
those obtained from the off-line simulation. Since the output Although the rotor angles can restore to a new steady-state
power of the PV-BESS systems and wind turbines maintain as given in Fig. 12 (b), the impacts of reduced generation are
stability, there is no significant change of the active power severe, including the significant drop of the generator volt-
injections at the PCC, as given in Fig. 10 (d). ages and the unrecoverable generator frequencies as shown
To further demonstrate the accuracy of the proposed FTRT in Fig. 12 (c) and (d).
emulation, the electrical torques, generator output id , and iq With the 51 times faster than real-time execution, the
currents after the 200ms three-phase-to-ground fault are also FTRT emulation equipped in the energy control center comes
provided in Fig. 11. As illustrated in the zoomed-in plots up with an optimal power control strategy following the
in Fig. 11. (c), the accuracy of the FTRT emulation can be detection of the abnormal condition to mitigate the adverse
guaranteed as the waveforms from the proposed emulation impacts, as given in Fig. 13 (a). At t = 7.8 s, each BESS
are matched with the results from the offline simulation tool. in MG-1 provides extra 1 MW active power to the AC grid
in 1s and lasts 1.2s until t = 10.0 s, resulting in an extra
B. CASE 2: MICROGRID INTERNAL POWER BALANCE active power injection of 2 MW from MG-1 to Bus 39. As a
Since the output power of renewable energy is highly depen- result, the frequencies start to recover as given in Fig. 13 (d).
dent on the environment, such as irradiations or wind speed, After 10s, MG-1 reduces the power injection from 4 MW
low power generation may occur and last for a long period to 3 MW , then the generator frequencies restore to 60 Hz,
in the microgrids. This case study focuses on the microgrid meanwhile, the rotor angles and generator voltages recover
internal balance and the predictive control of the proposed to the previous working conditions. The zoomed-in plots
FTRT emulation. At the time of 5s, the output power of each in Fig. 12-13 (d) indicate that the FTRT emulation results
PV array in MG-1 decreases by 500 kW , which induces a matched with the off-line simulation tool.

VOLUME 10, 2023 45

 of EMT and TS models. Taking its inherent advantages of
reconfigurability and parallelism, the FPGA-based hardware
platform allows emulating the integrated microgrid cluster
with an execution speed over 51 times faster than real-
time. Meanwhile, an active power dispatch is emulated to
deal with the various working conditions of the microgrids.
The time-domain results indicate that the proposed FTRT
emulation is numerically stable and accurate compared with
the off-line simulation. Furthermore, the dynamic security
assessment can be carried out on the proposed FPGA-based
FTRT platform with significant acceleration. The detailed
comprehensive FTRT emulation is also suitable for analyzing
other severe disturbances, which is meaningful for modern
power systems with high penetration of renewable energy.
FIGURE 16. FTRT emulation results for an open circuit fault
occurs in DC grid. REFERENCES
[1] M. H. Saeed, W. Fangzong, B. A. Kalwar, and S. Iqbal, ‘‘A review
C. CASE 3: INTER-MG COORDINATION on microgrids’ challenges & perspectives,’’ IEEE Access, vol. 9,
In this case, assuming that the BESS in MG-1 is fully charged pp. 166502–166517, 2021.
[2] F. Blaabjerg, Y. Yang, D. Yang, and X. Wang, ‘‘Distributed power-
and cannot store the extra power generated from the PV array, generation systems and protection,’’ Proc. IEEE, vol. 105, no. 7,
so that the MG-1 injects 1 MW into the AC grid, as given pp. 1311–1331, Jul. 2017.
in Fig. 14 (a). It brings significant impacts on the stability [3] W. Hu, Z. Wu, X. Lv, and V. Dinavahi, ‘‘Robust secondary frequency con-
trol for virtual synchronous machine-based microgrid cluster using equiv-
of the host grid, including the rotor angles, bus voltages, alent modeling,’’ IEEE Trans. Smart Grid, vol. 12, no. 4, pp. 2879–2889,
especially the increasing generator frequencies, as shown in Jul. 2021.
Fig. 14 (b)-(d). After the detection of the abnormal frequency, [4] M. A. Ebrahim, R. M. A. Fattah, E. M. M. Saied, S. M. A. Maksoud,
and H. E. Khashab, ‘‘Real-time implementation of self-adaptive salp
the energy control center emulates several power control swarm optimization-based microgrid droop control,’’ IEEE Access, vol. 8,
contingencies and provides an optimal solution to mitigate pp. 185738–185751, 2020.
the increasing frequencies. [5] Y. Han, H. Li, P. Shen, E. A. A. Coelho, and J. M. Guerrero, ‘‘Review
of active and reactive power sharing strategies in hierarchical controlled
At t=8.0s, both MG-2 and MG-3 start to take action, microgrids,’’ IEEE Trans. Power Electron., vol. 32, no. 3, pp. 2427–2451,
and each microgrid absorbs 1 MW active power, As a Mar. 2017.
result, the generator frequencies start to decrease as given [6] S. D’silva, M. Shadmand, S. Bayhan, and H. Abu-Rub, ‘‘Towards grid
of microgrids: Seamless transition between grid-connected and islanded
in Fig. 15 (a) and (d). At the time of 15.2s, MG-2 and MG-3 modes of operation,’’ IEEE Open J. Ind. Electron. Soc., vol. 1, pp. 66–81,
reduce the absorption power to 0.5 MW , and the frequencies 2020.
start to recover and return to 60 Hz. Meanwhile, the rotor [7] J.-H. Jeon et al., ‘‘Development of hardware in-the-loop simulation system
for testing operation and control functions of microgrid,’’ IEEE Trans.
angles and the bus voltages of the AC grids stabilize in a Power Electron., vol. 25, no. 12, pp. 2919–2929, Dec. 2010.
new steady-state as shown in Fig. 15 (b) and (c). Meanwhile, [8] Y. Zhang, S. Shen, and J. L. Mathieu, ‘‘Distributionally robust chance-
the zoomed-in plots are also provided in Fig. 14-15 (d), constrained optimal power flow with uncertain renewables and uncertain
reserves provided by loads,’’ IEEE Trans. Power Syst., vol. 32, no. 2,
which thoroughly demonstrates the accuracy of the proposed pp. 1378–1388, Mar. 2017.
method. [9] M. Usama et al., ‘‘A comprehensive review on protection strategies to miti-
gate the impact of renewable energy sources on interconnected distribution
D. CASE 4: OPEN CIRCUIT FAULT IN LVDC SYSTEM networks,’’ IEEE Access, vol. 9, pp. 35740–35765, 2021.
[10] C. Yuan, M. A. Haj-ahmed, and M. S. Illindala, ‘‘Protection strategies for
The open circuit fault occurring in the 5-T LVDC line in medium-voltage direct-current microgrid at a remote area mine site,’’ IEEE
MG-1 is also emulated, where the fault location is shown in Trans. Ind. Appl., vol. 51, no. 4, pp. 2846–2853, Jul. 2015.
Fig. 6. The influence of the open circuit fault is similar to the [11] M. Farrokhabadi, S. König, C. A. Cañizares, K. Bhattacharya, and
T. Leibfried, ‘‘Battery energy storage system models for microgrid stability
low power generation fault due to the lack of power injection analysis and dynamic simulation,’’ IEEE Trans. Power Syst., vol. 33, no. 2,
at Bus 39. The zoomed-in plots in Fig. 16 (b) demonstrate pp. 2301–2312, Mar. 2018.
that the proposed FTRT emulation still has high accuracy [12] C. Lyu, N. Lin, and V. Dinavahi, ‘‘Device-level parallel-in-time simulation
of MMC-based energy system for electric vehicles,’’ IEEE Trans. Veh.
regarding the fault happens at one of the LVDC lines. Technol., vol. 70, no. 6, pp. 5669–5678, Jun. 2021.
[13] R. Rana, K. Berg, M. Z. Degefa, and M. Loschenbrand, ‘‘Modelling and
VI. CONCLUSION simulation approaches for local energy community integrated distribution
This paper proposed a comprehensive hardware-based faster- networks,’’ IEEE Access, vol. 10, pp. 3775–3789, 2022.
than-real-time dynamic emulation of a grid of microgrids [14] W. Chen et al., ‘‘DC-distributed power system modeling and hardware-
in-the-loop (HIL) evaluation of fuel cell-powered marine vessel,’’ IEEE
to study the impact of their integration on the host system. J. Emerg. Sel. Topics Ind. Electron., vol. 3, no. 3, pp. 797–808, Jan. 2022,
Since different emulation types and time-steps are adopted doi: 10.1109/JESTIE.2021.3139471.
for the AC grid and microgrids, a dynamic voltage injection [15] W. Chen, S. Zhang, and V. Dinavahi, ‘‘Real-time ML-assisted hardware-
in-the-loop electro-thermal emulation of LVDC microgrid on the interna-
interface strategy with less hardware utilization and lower tional space station,’’ IEEE Open J. Power Electron., vol. 3, pp. 168–181,
latency is therefore proposed to enable the compatibility Mar. 2022.

46 VOLUME 10, 2023

Cao et al.: Faster-Than-Real-Time Hardware Emulation of Transients and Dynamics of a Grid of Microgrids

[16] J. Zhao, F. Li, S. Mukherjee, and C. Sticht, ‘‘Deep reinforcement learning- [39] N. Lin, S. Cao, and V. Dinavahi, ‘‘Comprehensive modeling of large
based model-free on-line dynamic multi-microgrid formation to enhance photovoltaic systems for heterogeneous parallel transient simulation of
resilience,’’ IEEE Trans. Smart Grid, vol. 13, no. 4, pp. 2557–2567, integrated AC/DC grid,’’ IEEE Trans. Energy Convers., vol. 35, no. 2,
Jul. 2022, doi: 10.1109/TSG.2022.3160387. pp. 917–927, Jun. 2020.
[17] U. Tamrakar, D. A. Copp, T. A. Nguyen, T. M. Hansen, and R. Tonkoski, [40] N. Lin, S. Cao, and V. Dinavahi, ‘‘Adaptive heterogeneous transient anal-
‘‘Real-time estimation of microgrid inertia and damping constant,’’ IEEE ysis of wind farm integrated comprehensive AC/DC grids,’’ IEEE Trans.
Access, vol. 9, pp. 114523–114534, 2021. Energy Convers., vol. 36, no. 3, pp. 2370–2379, Sep. 2021.
[18] L. M. Tolbert et al., ‘‘Reconfigurable real-time power grid emulator for [41] G. Abad, J. López, M. Rodríguez, L. Marroyo, and G. Iwanski, Doubly Fed
systems with high penetration of renewables,’’ IEEE Open Access J. Power Induction Machine: Modeling and Control for Wind Energy Generation
Energy, vol. 7, pp. 489–500, 2020. Applications. Hoboken, NJ, USA: Wiley, 2011.
[19] F. Huerta, R. L. Tello, and M. Prodanovic, ‘‘Real-time power-hardware-in- [42] Texas A&M University College of Engineering. Accessed: Nov. 21, 2022.
the-loop implementation of variable-speed wind turbines,’’ IEEE Trans. [Online]. Available: https://electricgrids.engr.tamu.edu/electric-grid-test-
Ind. Electron., vol. 64, no. 3, pp. 1893–1904, Mar. 2017. cases/new-england-ieee-39-bus-system/
[20] N. Bazmohammadi et al., ‘‘Microgrid digital twins: Concepts, applica- [43] P. Kundur, Power System Stability and Control. New York, NY, USA:
tions, and future trends,’’ IEEE Access, vol. 10, pp. 2284–2302, 2022. McGraw-Hill, 1994.
[21] M. H. Cintuglu and D. Ishchenko, ‘‘Real-time asynchronous information [44] V. Dinavahi and N. Lin, Real-Time Electromagnetic Transient Simulation
processing in distributed power systems control,’’ IEEE Trans. Smart Grid, of AC-DC Networks. Hoboken, NJ, USA: Wiley, 2021.
vol. 13, no. 1, pp. 773–782, Jan. 2022.
[22] A. A. Memon and K. Kauhaniemi, ‘‘Real-time hardware-in-the-loop test-
ing of IEC 61850 GOOSE-based logically selective adaptive protection of
AC microgrid,’’ IEEE Access, vol. 9, pp. 154612–154639, 2021.
[23] H. F. Habib, N. Fawzy, and S. Brahma, ‘‘Performance testing and assess-
ment of protection scheme using real-time hardware-in-the-loop and IEC SHIQI CAO (Graduate Student Member, IEEE)
61850 standard,’’ IEEE Trans. Ind. Appl., vol. 57, no. 5, pp. 4569–4578, received the B.Eng. degree in electrical engineer-
Sep. 2021. ing and automation from the East China Univer-
[24] M. Manbachi et al., ‘‘Real-time co-simulation platform for smart grid volt- sity of Science and Technology, Shanghai, China,
VAR optimization using IEC 61850,’’ IEEE Trans. Ind. Informat., vol. 12, in 2015, and the M.Eng. degree in power system
no. 4, pp. 1392–1402, Aug. 2016. from Western University, London, ON, Canada,
[25] S. Nigam, O. Ajala, and A. D. Dominguez-Garcia, ‘‘A controller hardware- in 2017. He is currently pursuing the Ph.D. degree
in-the-loop testbed: Verification and validation of microgrid control archi- in electrical and computer engineering with the
tectures,’’ IEEE Electrific. Mag., vol. 8, no. 3, pp. 92–100, Sep. 2020. University of Alberta, Canada. His research inter-
[26] A. S. Vijay, S. Doolla, and M. C. Chandorkar, ‘‘Real-time testing ests include transient stability analysis, power
approaches for microgrids,’’ IEEE J. Emerg. Sel. Topics Power Electron.,
electronics, and field programmable gate arrays.
vol. 5, no. 3, pp. 1356–1376, Sep. 2017.
[27] D. L. Marino, C. S. Wickramasinghe, V. K. Singh, J. Gentle, C. Rieger, and
M. Manic, ‘‘The virtualized cyber-physical testbed for machine learning
anomaly detection: A wind powered grid case study,’’ IEEE Access, vol. 9,
pp. 159475–159494, 2021.
[28] C. Dufour and J. Belanger, ‘‘On the use of real-time simulation technology NING LIN (Member, IEEE) received the B.Sc.
in smart grid research and development,’’ IEEE Trans. Ind. Appl., vol. 50, and M.Sc. degrees in electrical engineering
no. 6, pp. 3963–3970, Nov. 2014.
from Zhejiang University, Hangzhou, China, in
[29] J. Xu, K. Wang, P. Wu, and G. Li, ‘‘FPGA-based sub-microsecond-level
2008 and 2011, respectively, and the Ph.D. degree
real-time simulation for microgrids with a network-decoupled algorithm,’’
IEEE Trans. Power Del., vol. 35, no. 2, pp. 987–998, Apr. 2020.
in electrical and computer engineering from the
[30] M. Milton, A. Benigni, and A. Monti, ‘‘Real-time multi-FPGA simulation University of Alberta, Edmonton, AB, Canada,
of energy conversion systems,’’ IEEE Trans. Energy Convers., vol. 34, in 2018. From 2011 to 2014, he was an Engineer
no. 4, pp. 2198–2208, Dec. 2019. of substation automation, flexible AC transmis-
[31] Y. Chen and V. Dinavahi, ‘‘Multi-FPGA digital hardware design sion systems, and high-voltage direct current trans-
for detailed large-scale real-time electromagnetic transient simulation mission control and protection. Currently, he is a
of power systems,’’ IET Gener., Transmiss., Distrib., vol. 7, no. 5, Senior Power Systems Consultant. His research interests include AC/DC
pp. 451–463, May 2013. grids, electromagnetic transient simulation, real-time simulation, transient
[32] C. Yang, Y. Xue, X.-P. Zhang, Y. Zhang, and Y. Chen, ‘‘Real-time stability analysis, heterogeneous high-performance computing of power sys-
FPGA-RTDS co-simulator for power systems,’’ IEEE Access, vol. 6, tems, and power electronics.
pp. 44917–44926, 2018.
[33] J. Xu et al., ‘‘FPGA-based submicrosecond-level real-time simulation of
solid-state transformer with a switching frequency of 50 kHz,’’ IEEE
J. Emerg. Sel. Topics Power Electron., vol. 9, no. 4, pp. 4212–4224,
Aug. 2021.
[34] J. Zhu, L. Pan, Y. Yan, D. Wu, and H. He, ‘‘A fast application-based supply VENKATA DINAVAHI (Fellow, IEEE) received
voltage optimization method for dual voltage FPGA,’’ IEEE Trans. Very the B.Eng. degree in electrical engineering from
Large Scale Integr. (VLSI) Syst., vol. 22, no. 12, pp. 2629–2634, Dec. 2014. the Visvesvaraya National Institute of Tech-
[35] R. Dai, G. Liu, and X. Zhang, ‘‘Transmission technologies and implemen- nology (VNIT), Nagpur, India, in 1993, the
tations: Building a stronger, smarter power grid in China,’’ IEEE Power M.Tech. degree in electrical engineering from the
Energy Mag., vol. 18, no. 2, pp. 53–59, Mar. 2020. IIT Kanpur, India, in 1996, and the Ph.D. degree
[36] Y. Huo and G. Gruosso, ‘‘Hardware-in-the-loop framework for validation
in electrical and computer engineering from the
of ancillary service in microgrids: Feasibility, problems and improve-
University of Toronto, ON, Canada, in 2000. Cur-
ment,’’ IEEE Access, vol. 7, pp. 58104–58112, 2019.
[37] S. Cao, N. Lin, and V. Dinavahi, ‘‘Faster-than-real-time hardware emu-
rently, he is a Professor with the Department of
lation of extensive contingencies for dynamic security analysis of large- Electrical and Computer Engineering, University
scale integrated AC/DC grid,’’ IEEE Trans. Power Syst., early access, of Alberta, Edmonton, AB, Canada. He is a fellow of the Engineering
Mar. 28, 2022, doi: 10.1109/TPWRS.2022.3161561. Institute of Canada. His research interests include real-time simulation of
[38] S. Cao, N. Lin, and V. Dinavahi, ‘‘Flexible time-stepping dynamic emu- power systems and power electronic systems, electromagnetic transients,
lation of AC/DC grid for faster-than-SCADA applications,’’ IEEE Trans. devicelevel modeling, large-scale systems, and parallel and distributed
Power Syst., vol. 36, no. 3, pp. 2674–2683, May 2021. computing.

VOLUME 10, 2023 47