22th International Conference on Renewable Energies and Power Quality (ICREPQ’24)

Bilbao (Spain), 26th to 28th June 2024

ISBN: 978-84-09-60656-6

A practical implementation of virtualized protection system with

IEC61850 under Docker
E. Torres, N. Escobar, P. Eguia, O. Abarrategi, D.M. Larruskain, V. Valverde and G. Buigues
Department of Electrical Engineering, School of Engineering of Bilbao, University of the Basque Country UPV/EHU
Plaza Ingeniero Torres Quevedo, nº 1, 48013 Bilbao (Spain)
Phone/Fax number: +0034 94 601 7332

Abstract. Substation Automation Systems (SAS) are operation of the substation, as well as gateways for
currently designed and operated based on IEC 61850 standard. the communication with the remote control center.
This standard along with the evolution of the information In an IEC 61850 substation, horizontal communication
processing technology brings the possibility of developing new (e.g. between IEDs in the bay level) and vertical
centralized SAS architectures based on virtualized protection and communication (e.g. between bay and station level)
control. However, communication latency must be evaluated as it between IEDs is Ethernet based, by means of the process
can limit the scalability of virtualized SAS. This paper explores
bus and the station bus, respectively.
via a practical simulation case the effects of the number of virtual
IEDs in the data traffic of a virtualized IEC61850 SAS based on
Docker. Simulation results have shown that the increase in the As an alternative to the typical distributed architecture,
network traffic due to a higher number of virtual IEDs produces a the centralized approach has gained renewed interest in
longer delay in tripping operation under a fault condition. recent years due to the development of high-performance
computing platforms. A CPC (Centralized Protection and
Key words. IEC 61850, virtualization, protection, Control) system consists of a high-performance
digital substation. computing platform capable of providing protection,
control, monitoring, communication and asset
1. Introduction management functions. It uses a lower number of
devices, which improves operational safety and
interoperability, and provides enhanced cybersecurity,
The Substation Automation System (SAS) plays an due to the limited number of access points for cyber
important role in substations as it performs essential threats [1]. Also, the potential of the application of
functions such as protection, automation and control. virtualization technologies in substations has recently
Earlier SAS used a centralized approach, although finally a started to be explored, in order to develop more flexible
distributed architecture was adopted due to limitations in substations to manage the growing variability in the
available technology in those days. In a typical distributed operation conditions of future electric systems [2, 3]. A
SAS, Intelligent Electronic Devices (IEDs) are linked to a virtualized SAS means a paradigm shift and the evolution
local processor, which controls one or more bays and is to fully digital substations, where substation
connected to a Human Machine Interface (HMI) or to a functionalities will be provided by software applications
SCADA system for overall control. The design of SAS is running autonomously in the same computing platform.
currently based on the functional requirements of the IEC As a result, hardware dependency will be reduced, as
61850 international standard to enable greater integration well as the time required for the commissioning of new
of protection, control, measurement and supervisory substations and the deployment of new functionalities in
functions in the substation and easier interoperability existent substations.
between devices from different manufacturers. The
standard establishes the following levels in the architecture Both approaches allow centralization of protection
of SAS: functions, typically carried out by multiple conventional
 Process level: It contains substation primary relays, in a single computing platform and so, a reduction
equipment (power transformers, current and voltage in the number of devices with regard to the traditional
transformers, switching equipment), and merging approach. However, as stated in [4], there is a difference
units (MU) which merge voltage and current signals regarding communication between different protection
from instrument transformers and transform them into functions: in a centralized approach it is device internal
digital signals. and not necessarily visible over the IEC 61850, whereas
 Bay level: It contains intelligent electronic devices in a virtualized approach it develops over a virtualized
(IEDs) for protection and control. communication network. As a result, in a virtualized
 Station level: It contains the Human Interface Unit approach, some extension is required to represent the
(HMI) and SCADA systems for monitoring and computer running the virtual IEDs in IEC 61850 SCL
(Substation Configuration Language) and latency issues The following protocols are specified in the IEC 61850
derived from virtual communication should be carefully for communication between IEDs:
examined. On this way, the application of virtualization in  MMS (Manufacturing Message Specification): It
substations poses several challenges that have to be further is used for application, configuration and
investigated, in order to guarantee correct real-time monitoring data exchange.
operation of the most critical SAS functionalities, as those  SMV (Sampled Measured Values): It is applied
related to protection and control. for communication of measurement values.
 GOOSE (Generic Object Oriented Substation
In this paper, communication performance requirements of Events): It allows fast and reliable
IEDs according to IEC 61850 standard are introduced in communication of critical events and states.
Section 2, and latency and scalability issues of
virtualization of protection IEDs are discussed in Section MMS protocol is intended for communications between
3. A practical simulation case study that analyzes the IEDs and higher level entities, such as a SCADA or a
influence of the number of virtual IEDs in the data traffic gateway. It applies a client-server approach, where a
of an IEC 61850 substation is described in Section 4. From client requests data from a server. In contrast, GOOSE
this study, conclusions are extracted regarding the and SMV are protocols for the operation of protection
influence of the number of IEDs in the scalability of real systems, which are based on a publisher-subscriber
virtualized SAS implementations. mechanism, where a subscriber receives all messages but
filters those messages is subscribed to. SMV messages
2. IEC 61850 standard are used in continuous real-time monitoring and control
for the transmission of voltage and current signal
IEC 61850 is the international Standard applicable to measurements to protection and control equipment, and
substation protection, automation and control systems. It data are sent as high speed streams of data set samples.
provides the engineering definitions and processes for GOOSE messages are used for communication of real-
configuration and parameterization of the functions time event-driven situations between IEDs. In GOOSE
required for digital communication between IEDs in the communication, the publisher sends messages
substation. The main purpose of the standard is the periodically but when an event happens, it sends a burst
interoperability between devices from different of messages to minimize the chance of message loss
manufacturers. With that purpose, an XML-based (Figure 2).
Substation Configuration Language (SCL) is specified in
the standard to create a set of files for the configuration of
an IEC 61850 substation, which facilitates the engineering
process and the integration of equipment from different
vendors. One of those files is the Substation Configuration
Description (SCD) file, which contains a description of the
switchyard equipment and the topology of the one-line
diagram, the capabilities of all protection and control IEDs
used in the substation, as well as the subnetworks, access Fig.2. GOOSE transmission [5]
points and communication connections between IEDs.
Operation of protection and control functions requires
An IEC 61850 IED is represented by means of a fast reliable communication of different type of
hierarchical data model, which is depicted in Figure 1. In messages. According to IEC 61850, the transfer time
this model, a physical device (PD) defined by its network involved in the transmission of information between two
IP address enables to identify the IED within the system. It devices includes the processing time within publisher, the
contains a logical device (LD) with one or more logical transmission time from publisher to subscriber, and the
nodes (LN) which represent the different functions processing time within subscriber to decode the received
performed by the IED. message (Figure 3). Also, different transfer time classes
are defined depending on the application (Table I). In the
case of protection functions, a transfer time class TT6 is
established, with a transfer time below 3 ms.

Fig.1. IEC 61850 data model [5] Fig.3. Overall transmission time between devices [6]
 Table I. – Classes for transfer times (IEC 61850-5) applied in substation automation, virtualization faces new
challenges in comparison to cloud environments, such as
Transfer Transfer time Application examples: limited compute, storage and network resources, mixed
time class (ms) Transfer of criticality of applications, as well as real-time and high
TT0 >1000 Files, events, log contents availability requirements of protection and control
TT1 1000 Events, alarms functions [9]. A critical challenge in virtualization of
TT2 500 Operator commands protection is guaranteeing real-time performance, which
TT3 100 Slow automatic interactions can be affected by different factors and is directly related
TT4 20 Fast automatic interactions to the scalability of virtualized protection and control
TT5 10 Releases, status changes solutions, i.e. the maximum number of IEDs that can be
TT6 3 Trips, blockings virtualized on the same computing platform, without
affecting correct operation. In [8], results of preliminary
3. Virtualization issues of protection and tests of virtualized protection in VMs show a variation in
the response time of overcurrent protection under
control functions different hardware allocation to VMs and fault
magnitude. In addition, real-time performance of a
Virtualization consists on the creation of a software-based protection and control application running in virtual
or virtual representation of a physical IT resource. It is machines and containers is evaluated in [10], being
widely applied in the IT sector and it is the basis of cloud concluded that primary causes of timing errors were poor
computing. It improves the scalability and flexibility of resource isolation and virtual network delays.
systems, which reduces capital and operating costs, so its
application in electric substations is being evaluated. There In a virtualized protection and control system,
are two main virtualization approaches: hardware-level communication between virtualized applications relies on
virtualization and operating system (OS) level a virtual network. The influence of networking across
virtualization (Figure 4). Hardware virtualization involves Docker containers in real-time automation systems is
virtualizing the hardware on a server and creating virtual investigated in [11] to determine whether communication
machines that provide the abstraction of a physical between Docker containers, running on the same or
machine. It uses a hypervisor, which emulates virtual different hosts, can be achieved within deadlines required
hardware (CPU, memory, etc.) for each virtual machine, by real time applications. The tests carried out using
which in turn runs its own operating system and several Docker-supported software networking solutions
applications. In contrast, OS virtualization encapsulates (Host, Bridge and MACvlan) and a hardware-assisted
standard OS processes and their dependencies to create solution (SR-IOV) under various workloads revealed
containers that share the underlying OS kernel [7]. some differences between them in terms of network
latency and missed packets.

In the literature, different approaches are considered to

emulate the network traffic in virtualized substation
applications. A first approach is the use of an application
that generates the different types of messages in a
substation. This approach was applied in [6], where a
laboratory virtual environment with 13 virtual machines
to analyze the performance of virtualization of power
(a) (b) system protection is described. Authors used Triangle
Fig.4. Hardware (a) and operating system (b) virtualization [7] MicroWorks 61850 Test Suite for running multiple IEC
61850 servers and clients on a single physical server, as
Virtual machines are typically more resource-intensive well as for the generation and traffic simulation of
than containers, but also provide a high level of isolation, GOOSE and SV packets. The research was later extended
which is important in substation environments for security in [12] to consider 171 virtual machines running on three
and compliance reasons. In contrast, containers are gaining physical servers under Windows and Linux operating
a great popularity as they are more portable and can isolate systems using the libIEC-61850 open source library. The
application environments from other applications as well developed platform was also tested with a physical
as from the host and, which facilitates the deployment of merging unit sending sampled values to a virtualized
applications, and will enable the development of more overcurrent protection with the GOOSE publisher service
agile and flexible substations. Virtualization of substation implemented. In [13], authors emulate a virtual
protection functionalities is analyzed in [8], where various centralized protection and control system workload,
possible approaches to develop virtual protection relays using MGEN network test tool for the simulation of
are identified, such as a straightforward approach based on multicast packets with size and arrival rate similar to
directly porting a complete IED into a VM, or an GOOSE and SV messaging. A different approach to
alternative approach where common modules are these studies is used in [14], where authors consider the
consolidated to a separate VM. Container technology may coupling of power grid simulation and virtualization of
be also applied to differentiate between protection network infrastructure. A reduced circuit is simulated
functions in the virtual IED and take advantage of its using HYPERSIM real-time simulation system, and a
scaling and portability characteristics. However, when virtualized network including a simplified overvoltage
protection (PTOV) is set up via Containernet, using open communicate, while providing isolation from containers
source library libIEC61850 as interface. An advantage of that are not connected to that bridge network [16]. In
this approach is the connection between energy and addition, the application includes a graphical interface to
communication systems, which enables mapping the browse the model and control the simulation. It has been
dependencies between them, providing online data for the used to modify a resistive load in order to create an
development and testing of substation process-level overcurrent condition and cause the operation of the
analysis algorithms. overcurrent protection, as shown in Figure 6, where
current signals at both sides of transformer T1 are
4. A case study of virtualization of protection represented.
and control IEDs
In this section, a simple case study of substation
virtualization under Docker is described, that considers the
coupling between the power and the communication
systems to analyze the influence of the number of virtual
IEDs in the data traffic of an IEC 61850 substation. The
case study is based on an application available in a public
repository [15], which simulates the operation of a simple
substation with various IEDs using the open library
libIEC61850. Although the application is not designed to
run in real time, it provides some insight about the
behavior of virtual IEDs in an IEC 61850 substation. A
simplified one-line diagram of the substation is shown in
Figure 5.
Fig.6. Transformer HV and LV current signals

The effect of virtualization on network traffic has been

analyzed by means of Wireshark software, which
required to modify the application to include a new
Docker container in order to execute Wireshark in the
same virtual environment (Figures 7 and 8).

Fig.5.Simplified one line diagram

An SCD configuration file of the substation includes the

information of substation primary equipment, IEDs and
Fig.7. Docker containers
communication network. There are four IEDs in the
substation: a circuit breaker (IED1_XCBR), a temporary
overcurrent protection (IED2_PTOC) and two merging
units (IED3_SMV, IED4_SMV) at the low and high
voltage sides of the power transformer. The overcurrent
protection is fed with sampled values from the merging
unit and activates a protection trip conditioning (PTRC)
logical node at the same logical device, publishing a
GOOSE message with a trip command. The logical node
XCBR is subscribed to GOOSE messaging from PTRC
and updates its state after receiving a trip command. The
application uses Docker, a tool for the automation and
deployment of applications in lightweight containers, to
create and run six Docker containers corresponding to the
four IEDs, to the IEC61850 python based client and to the
Fig.8. Capture of GOOSE and SMV messages with Wireshark
primary process simulator for the circuit simulation.
Container networking refers to the ability for containers to
Figures 9 and 10 show periodic GOOSE messaging from
connect and communicate with each other, and is
IED1_XCBR and SMV messaging from IED3_SMV,
configured in the application as “bridge”, which lets
respectively, in the base case under normal condition, as
containers connected to the same bridge network
captured by Wireshark.
 The occurrence of an overcurrent event produces the
activation of the overcurrent protection (IED2_PTOC)
and the operation of the circuit-breaker (IED1_XCBR).
IED1_XCBR is subscribed to GOOSE messages
published by the PTRC logical node of IED2_PTOC
when temporary overcurrent function of PTOC logical
node is activated, and GOOSE communication between
both IEDs was analyzed with Wireshark to evaluate the
time operation after a fault. The value of parameters
sqNum (sequence number) and stNum (state number)
Fig.9.GOOSE messages from IED1_XCBR and IED2_PTOC in was monitored, as for every GOOSE message published,
the base case, under normal condition sqNum is incremented by one, while stNum is updated
with an event. Figure 13 shows two captures of
Wireshark corresponding to two consecutive GOOSE
packets from IED2_PTOC. As it is shown, stNum
changes from 1 to 2, while sqNum is initialized to 0,
which means the operation of protection and the
publication of the corresponding trip message The delay
due to communication can be calculated as the difference
between the instants of operation of the protection and of
opening of the circuit breaker.

Fig.10.SMV messages from IED3_SMV in the base case, under

normal condition

A case study was developed to evaluate the influence of

virtualization scalability in an IEC 61850 substation. With
this purpose, the IEC 61850 SCD configuration file of the
substation was modified to include a higher number of
IEDs. In particular, the number of IEDs publishing SMV
messages was increased to boost the network traffic and
the consumption of hardware resources by virtualization
(Figure 11). Figure 12 shows an example of the network
traffic obtained using Wireshark, when 1 and 7 new
additional IEDs sending SMV were included, under
normal operation condition. As it is shown, when the
number of SMV publishers increases the network traffic
Fig.13.Identification of operation time with Wireshark
rises significantly, in comparison to the base case.
The communication delay was calculated in three
scenarios with different number of IEDs (A: base case
with 4 IEDs, B: case study with 1 additional IED, C: case
study with 7 additional IEDs) for different overcurrent
conditions. Results obtained in different tests are shown
in Figure 14, where a significant increase is observed in
the scenario C with 11 IEDs. In scenario A, an average
Fig.11. Resource usage by Docker containers
communication delay of 37 ms was obtained, which
increased around 3 ms in scenario B and 44 ms in
scenario C. As a result, performance requirements of the
most critical transfer time classes TT4, TT5 and TT6 of
IEC 61850 would not be fulfilled when increasing the
number of virtualized devices.

Fig.12.Increase in SMV network traffic in the base case (red) and

in the case study (blue), under normal operation condition Fig.14.Operation time in scenarios A, B and C
In addition, the use of Wireshark tool allowed to identify Acknowledgement
an increase in the amount of SMV messages in network
traffic (Figures 15 to 17) in scenarios B and C, in This work was supported by the Basque Government
comparison to scenario A, but also a decrease in the under GISEL Research Group grant “IT1522-22” and
frequency of SMV publishing. Elkartek VIRTGRID “KK-2022-00069”.

References
[1] M. Adamiak et al., Centralized Substation Protection and
Control, IEEE PES Power System Relaying Committee,
Report of Working Group K15 of the Substation Protection
Subcommittee, 2015.
[2] S. Dayabhai, J. Prestwich, A substation automation solution
that uses virtualization to reduce cost while ensuring
redundancy and security compliance, Power and Energy
Automation Conference, Washington (USA), March 2018.
[3] R. Hunt, B. Flynn, T. Smith, The substation of the future:
Fig.15. SMV messages in scenario A Moving toward a digital solution, IEEE Power & Energy
Magazine, Vol. 17, nº 4, pp. 47-55, 2019.
[4] C. Brunner, IEC 61850 and Virtualized Protection,
PacWorld, nº 66, December 2023.
[5] P. Bishop, N.K.C. Nair, IEC 61850 Principles and
applications to electric power systems, 2nd Edition,
Springer, 2022.
[6] R. Wójtowicz, R. Kowalik, D. Rasolomampionona, Next
generation of power system protection automation –
Virtualization of protection systems, IEEE Transactions on
power Delivery. Vol. 33(4), pp. 2002-2010, 2018.
[7] P. Sharma, L. Chaufournier, P. Shenoy, Y.C. Tay,
Containers and Virtual Machines at Scale: A Comparative
Study, 17th International Middleware Conference, Trento
Fig.16. SMV messages in scenario B (Italy), December 2016.
[8] D. Samara, G. McKenzie, P. Khajuria, R. Ariya, P.
Gopalakrishnan, V. Ravindran, Virtual protection relay – A
paradigm shift in power system protection, INTEL-
Kalkitech White paper. (Available at: www.intel.com/)
[9] S. Schönborn, T. Sivanthi, A. Kulmala, H. Nivery, R. Birke,
The virtues of -virtualization, ABB Review, nº 2, pp. 118 –
123, 2023.
[10] S. Schönborn, R. Birke, D. Kozhaya, T. Sivanthi, Real-
time performance of virtualised protection and control
software, 27th International Conference on Electricity
Distribution (CIRED), Paper nº 10702, Rome (Italy), June
2023.
[11] G. Albanese, R. Birke, G. Giannopoulou, S. Schönborn,
Fig.17. SMV messages in scenario C T. Sivanhi, Evaluation of networking options for
containerized deployment of real-time applications, 26th
IEEE International Conference on Emerging Technologies
5. Conclusion and Factory Automation (ETFA), Vasteras (Sweden), 2021.
[12] R. Wójtowicz, R. Kowalik, D. Rasolomampionona,
Virtualization of protection and control functions means a Virtualization of protection systems – Test performed on a
paradigm shift in SAS. Performance of virtualization of large environment base don data center solutions, IEEE
IEDs is analyzed through the evaluation of network traffic. Transactions on Power Delivery, Vol. 37(4), pp. 3401-3411,
In this paper, the effect on network traffic of virtualization 2022.
has been analyzed using a public application for the [13] R. Carvalho, M. Antunes, J.P. Barraca, D. Gomes, R.L.
simulation of a simplified IEC 61850 substation. A case Aguiar, Design and evaluation of a low-latency CPC
environment for virtual IEDs, IEEE 11th International
study with different number of virtual IEDs publishing Conference on Cloud Networking (CloudNet), Paris, 7-10
SMV messages to modify network traffic was used to November 2022.
evaluate the effect on the tripping time under fault. [14] D. Rösch, S.Nicolai, P. Bretschneider, Combined
Simulation results showed a higher delay in the publication simulation and virtualization approach for interconnected
of SMV and a higher latency in the tripping time when substation automation, 6th International Conference on
more virtual IEDs were included in the case study. Smart and Sustainable Technologies (SpliTech), Bol and
Although the application is not developed for real-time Split (Croatia), 2021.
simulation, it has allowed identifying latency issues due to [15] R. Massink, ied_61850_open_server, An open source
virtualization of IEDs that can affect scalability in real implementation of an IEC 61850 IED using lib61850.
Available at: https://github.com/robidev?tab=repositories
substations. [16] Docker Manuals, https://docs.docker.com/manuals/