CP018 LNETC WAMSComm
CP018 LNETC WAMSComm
Abstract - This paper presents a review of work per- the power system need to be improved. One develop-
formed in the field of communication systems requirements ment in this direction is the use of Wide Area Monitoring
for wide area monitoring and control applications. This in- and Control (WAMC) systems allowing real-time control
cludes studies performed both in the context of NASPI, the of the power system. Such systems use remote measure-
North American Syncrophasor Initiative, as well as other ments of high quality and advanced control algorithms to
studies in the US and Europe. Presently a relatively large control the whole or parts of a power system. A critical
body of work has been done regarding the requirements component in these WAMCs is the ICT infrastructure that
on the communication infrastructure. Among the work provides access to measurements and enables fast compu-
included, are surveys among Transmission System Opera- tation of control signals.
tors on the foreseen communication requirements, as well
1.1 Scope of the Paper
as other theoretical studies on acceptable delays and other
performance characteristics, such as data loss, that Wide The scope of this paper is to present a review of the
Area Control applications put on the Information and Com- state of the art regarding information and communication
munication System. Much of the above referenced work system architectures for Wide Area Monitoring and Con-
still needs verification either though experimental work or trol systems. The focus lies on the performance, in terms
through simulations. It is especially important to simu- of latency, of these systems in relation to the power sys-
late large scale communication infrastructures that consider tem requirements. The paper also presents some examples
background traffic under different operational conditions in of studies on the effect of communication system perfor-
the power system, such as under disturbances. The paper mance on the feasibility of power system functions and
is concluded with some remarks on future direction of work examples of measures taken to manage these effects.
within the field.
1.2 Layout of the Paper
Keywords - Wide Area Monitoring and Control, Pha- After this introductory section follows a section de-
sor Measurement Unit, Communication System Perfor- scribing the fundamental assumptions and definitions used
mance, ICT system Architecture, Quality of Service in this paper regarding Wide Area Monitoring and Control
systems, Power system applications and Quality of ICT
1 Introduction systems that are necessary for the presentation in the rest
of the paper. The ensuing two sections (sections 3 and 4)
LECTRICAL power networks constitute a critical in- contain the bulk of the presentation of presently ongoing
E frastructure in modern society. Our dependence and
demand on electricity has risen sharply while recently this
activities and their approaches. The paper is concluded
with a section discussing the future direction of research
rising demand for electricity has been met with a serious and development in the field.
strain in terms of production and expansion of transmis-
1.3 A note on terminology
sion capacity. This is, among other factors, due to in-
creasing environmental policies leading to large scale in- In this presentation, the term Power System Applica-
troduction of intermittent generation. Furthermore, the re- tion is used to denote a control or monitoring function im-
regulation of electrical markets and the connection of na- plemented in software fulfilling some purpose for moni-
tional grids with neighbouring nations have resulted in a toring, operation, control or planning of a power system.
more complex and dynamic environment in which mul- The core of the application is an algorithm using high
tiple organizations coordinate and cooperate in the oper- quality measurements as inputs and providing either out-
ation and control of the power system. To balance this put signals to be sent to actuators or processed data to be
th
17challenges
Power Systems
controlComputation
systems for Conference
operation and control of Stockholm
presented to a human Sweden
operator. - August
The term 22-26, ICT
Supporting 2011
system is used to denote the communications and com- from an external party (link B). In the figure, the high qual-
puting platform that the Power System Application relies ity measurements are assumed to come from Phasor Mea-
upon for receiving data from local or remote locations, surement Units (PMU) and data collection and sorting is
performing calculations and sending output signals. To- assumed to be performed in Phasor Data Concentrators
gether, these two constitute what we will refer to as a Wide (PDC). This choice of nodes is representative of present
Area Monitoring and Control System (WAMC). Finally, day solutions, but does not limit the architecture as such
the term Interconnected Power System will be used to de- to only this specific type of measurement.
note a single synchronous power system operated by sev- E %&
%&&'()*
eral organisational entities, typically ISOs or TSOs. The !"#$ External
+,-$
implication of this term is that several Supporting ICT Sys-
D
tems, owned and operated by separate ISOs/TSOs may be
needed to support one Power System Application that per- E
%&
%&&'()*
forms a function within an Interconnected Power System !"#$ Central
B +,-$
thereby adding a level of complexity from an ICT archi- C
tecture perspective.
E %&
%&&'()*
!"#$
+,-$ Local
2 Wide Area Monitoring and Control Systems
A
As implied above, Wide Area Monitoring and Control !./$ !./$
Systems (WAMC), are not singularly defined, but is often
interpreted as systems that use remote time-synchronised Figure 1: Generic architecture for Wide Area Monitoring and Control
measurements of high quality, generally from Phasor Systems
Measurement Units (PMU), for monitoring or control of The three layers in the architecture are local including
the power system. In many cases, it is assumed that the applications at the substation level using data from a sub-
control is done without a human in the loop. The term set of measurements points in the interconnected power
also include applications that are centralised and operate system. Measurements are transmitted locally or over high
on complete measurement sets from the power grid as well speed links A, e.g. Ethernet over fibre optics, the latencies
as applications that are tied to a specfic part of the system. of these links TA is assumed to be very low. For cases
The following three dimensions can be used to better un- where the power system application works across control
derstand the scope of the system. areas within the interconnected power system data is re-
quired from external parties, links B, here the links exist
• Whether the function relies on a complete set of through a wide area communication system spanning two
high quality measurements or if it is based on a sub- ISO/TSOs, resulting in latencies TB . The central level
set of measurement as input to its algorithm. applications are those that use a complete set of measure-
ments, transmitted over links C from the control area in the
• Whether the function is implemented centrally at a power system, the latency for this group is TC . Finally, ex-
control centre, or locally in a substation or a power ternal applications, are those that are built at a super-ISO
plant, close to either measurements, actuators or level requiring collection of data from several control ar-
both. eas as well as from within its own area. It is reasonable to
assume that TA < TC < TB , however, since applications
• Whether it is a closed-loop function without a hu-
can be built out of components requiring communication
man in the loop, or if it provides information for
through all three categories of links it is not sufficient to
human decision making.
dimension the support ICT system based only on one of
It is important to keep in mind that all types of functions these latencies.
can and will co-exist within a single Interconnected power 2.2 Power System Application Requirements
system. An optimal ICT system architecture must there-
fore be adaptable to several types of applications. This A number of research groups have conducted studies
observation, together with the fact that the WAMC sys- to characterise the types of power system applications in-
tem may require data from external parties, constitute the herent to WAMC systems and their requirements on the
main challenges to developing open, secure and reliable supporting ICT systems. An early paper on the topic
ICT architectures for WAMC systems. [3] identifies two categories of WAMC applications, those
that require complete network observability and those that
2.1 WAMC architectures
do not. Two applications discussed in [3] that require
The different categories of application naturally has complete network observability are frequency stability as-
different requirements on the supporting ICT systems. sessment and voltage stability assessment in meshed net-
Figure 1 below illustrates a generic architecture for works. Applications not requiring full network observ-
WAMC systems valid for the three types of power sys- ability are oscillation damping, voltage stability in trans-
tem applications listed above. The architecture also cap- mission corridors and line temperature assessment. The
th
17tures
Power
the Systems Computation
specific case, Conference are received
where measurements Stockholm
conclusion in the paper is thatSweden - August of
the performance 22-26, 2011
the sup-
porting ICT system will be critical for being able to close tional challenges posed by for instance inter-ISO commu-
the control loop. In [20] these applications have been nication with the additional considerations that need to be
used to determine more exact perforamnce requirements taken with regards to cybersecurity and interoperability.
on the supporting ICT system and compared these with
those presented in the NASPI project.
3 Quality of Information and communication
The most comprehensive work on power system ap- systems
plication requirements has been conducted as part of the
Gridstat.net project at Washington State University, and At the core of WAMCs are the power system appli-
the results are reported extensively in [18]. In this report, cations, implemented as algorithms in software. These
thirteen types of power system applications, ranging from algorithms, are often the result of research and develop-
Traditional State Estimation via Transient Stability control ment driven by the requirement to improve som aspect of
to Research applications are presented together with their the power system. The focus during the development is
requirements on the supporting ICT systems. The work is purely the function as such. The quality of the supporting
interesting since it brings the requirements a step beyond ICT system, is often expected to be sufficient, sometimes
merely latency and throughput, using the term QoS+ to leading to proper quality in the power system function as
denote a set of requirements including data criticality and such due to poor understanding of ICT limitations, see for
quantity. The term Quality of Service, or QoS in short, instance [2]. Needless to say the functional focus leads to
usually denotes a variety of non-functional aspects of a sub-optimal solutions when the entire ICT architecture is
communication service, such as latency, jitter, security and considered. Too large focus on the functions of ICT over
so on. the QoS+ requirements are classified and grouped their non-functional aspects leads to the stovepipe system
in 5 distinct levels as illustrated in Figure 2 below. architetures, see [22]. Furthermore, it can create a false
!"##"$%&'() 4.'15$() !1.0&"51)
sense of confidence, that one particular function’s ICT sys-
7.'1)*893) :/"'"$.&"'() ;%.5'"'() <1=>/.?-()
*+,-./012'3) *621$3) *@%&A)'/.#B3)
tem capabilities are sufficient for that of another function
56,788%-%3,29%7,% @!%8/6A%
+) !"#$% #&$"'#$(% )*+,-% ./,0%1234%
:;*+2<*/%=>?8% merely because they are similar in nature.
C) #$"!$% B#$"#&$% 1234*0% 1234%
C2+42D%-D%=>?E% B%:2DA% !"#$%#&%##&'()&$*+&',-).
FG?%
J/+K//D%-%L/K% B%4,A% 9/42$)%81*3.2/
D) !$"B$$% H$"B#$% I/92;:% I/92;:%
;+2*2+2/8%
C2+42D%-%82D3*/% B%9-0%
E) B$$"B$$$% B"H$% M7K% M7K%
;+2*2+0%
./,0%M7K% C2+42D%-% NB%9-0% %9/42$)% 9/42$)% 9/42$)%
F) NB$$$% @B% ./,0%M7K% B$3.*6.3.2/% &$'("')*+,$ ;+2$'"5$'*6.3.2/
O8/,2-*P% 8;Q8+-+27D%
thors also take this a step further and specify design guide-
<.+=%0*5*,.2/ 9"(2>*'$%
lines for Wide Area Monitoring and Control systems that 0"?$4.@$+$44 -((.,.$+,/%"(%
0*5*6.3.2/%"(%
7$)"'/
0"))1+.,*2."+%
consider these requirements. &'"2","34
In a study of similar scope, see [4], and specifically /0123#& 4.& 5*6"2#*7#& +0$13$8& 0""2%93$90*1& 7$2%$"& 3#"$90:*& ;#9<##*& $9930;29#%&
Figure 3: Influence Diagram illustrating the causal relation between at-
regarding communication latency requirements in [19], a tributes that constitute several non-functional aspects of ICT systems[21]
range of applications utilising high quality remote mea- Additionally, there is clear interdependencies between
surements and their associated ICT system requirements the non-functional aspects and the attributes of the ICT
are presented. The conclusions regarding data latency and system that comprise them, see [21]. In some cases there
volumes is similar to that of [18] and [20] with the most is even a negative coupling, for instance between inter-
strict requirements being in the range of 50 ms for data de- operability and performance. A more complete analysis
livery for most time critical applications, with more strict of ICT systems requires consideration of the fact that the
requirements for system integrity protection schemes. non-functional aspects of ICT are interrelated, see [23]
Combined, the above presented studies of WAMC ap- for an approach to this type of comprehensive analysis.
plications go a long way to categorise and describe the In the following listing, some aspects of how these non-
requirements on the supporting ICT systems put by power functional aspects are related, and how they impact the
system applications. The studies are very comprehensive quality of the supporting ICT systems are presented.
when it comes to latencies and data throughput, the most Performance It is clear that the overarching concern with
strict latency requirements are in the range of 5-20 ms data regards to supporting ICT systems is that of performance
delivery delays for transient stability applications allowing in the term of latency and throughput of data. The first
for control actions within 50-100 ms. Additionally, large order attempt to resolve this problem is to employ high
group of applications related to distributed control, island- capacity links at the physical level, i.e. fibre optics, be-
ing and special protection schemes are reported to require tween the points in the network requiring high speed ac-
data deliveries within 50 ms. Of the studies [18] provides cess. Disregarding the costs invovled for such efforts, this
an additional analysis beyond that merely of communi- effort is sometimes not sufficient if the causes for delays
cation system performance, by introduction of the QoS are not at the physical layer, but elsewhere in the com-
th
17concept.
Power Systems
However,Computation Conference
this still does not include the addi- Stockholm
munications solution Sweden
such as in - August
data sorting 22-26, 2011
or application
processing. For a study that includes these aspects please the ICT system’s capabilities.The second category are ac-
see [24]. In general, extended performance management tivites that aim at analysing and designing the ICT system
such as the QoS concepts presented in [15] are necessary to be able to fulfill stricter requirements. The third and
to ensure sufficient quality in the ICT systems. final category presented is that of new paradigms, for ded-
Cybersecurity Cybersecurity is perhaps the most dis- sign of WAMCs and their ICT systems.
cussed of the non-functional aspects of ICT systems, see
for instance [27] or [28] . It also provides the best exam- 4.1 Application development
ple of the dangers of focusing ICT development merely on
Fine tuning of the power system application to make
the function to be fulfilled. No organisation would con-
it less dependent on the capabilities of the ICT system is a
sider employing an ICT system completely without pro-
natural approaxh. This work normally includes compen-
tection against cyber attacks. Furthermore, adding these
sating for the expected delay by increased robustness in
protective measures once the system has been commis-
control algorithms or filtering of input signals to improve
sioned creates obvious risks to performance. The de-
measurement quality despite latency.
sign of ICT systems need to incorporate cybersecurity de-
For example in [10] the design of a robust TCSC con-
fence mechanisms from start, thereby putting additional
troller is presented, here the uncertainty due to ICT system
demands on understanding the performance implications
latency is represented by an inverse input multiplicative
at design time. A good example of such proactive de-
model. To enhance the robustness of the controller, its pa-
velopments can be found in the NITR program, see [29].
rameters are optimized using a genetic algorithm. Along
The cybersecurity issues is naturally more pressing for
the same lines, in [12] a number of approaches that work
WAMCs systems that act across control areas and ISOs.
to improve overall power system application functional-
Interoperability With interoperability we understand the
ity while increasing tolerance to latency are suggested and
ease by which a system can exchange information with
compared. The paper also presents a controller design
another system. As explained in [30] there are several lay-
technique, that allows the designer to consider varying
ers of interoperability, for our discussion here, it suffices
amounts of latency in the ICT system. In [13], the au-
to say that the interoperability issues increases the larger
thors present an alternative approach to managing latency
the distance between the measurement source and the ap-
by predictive control in which a recursive least-squares
plication. The distance is here not only geographical, but
algorithm is used to identify the predictive model. The
also has a context or organsiational dimension, meaning
method has been verified in simulations, but there seems
that if data transverses several networks within, or outside
to be little indication of experimental verification of the
of an ISO additional latency may be experienced due to
concept. Similarly in [14] predictive control using model
protocol conversions or communication gateways.
identification by means of a simultaneous recurrent neural
Reliability ICT system reliability differs significantly from
network is employed in the design of a WAMC system.
reliability of hardware devices most explicitly because it
In general,in all the presented projects, the proposed al-
does not exhibit the same type of fault probabilities. A
gorithms improvements are verified in simulation set ups
piece of software may function perfectly until a certain in-
and the application fulfills its function despite the mod-
put value is recieved and then stop functioning altogether.
eled ICT system limitations. The presented pieces of work
Studies on the reliability of WAMC systems are performed
should to be seen as good examples of the types of devel-
for example in [3] in which the individual component reli-
opments that are ongoing to adapt the applications to the
abilities are considered. In [2] the reliability analysis cov-
capabilities of ICT system. The lack of experimental or
ers the ability of the ICT system to deliver data in time as
empirical validation of the concepts does however make
a reliability issue, providing an expanded interpretation of
the approaches vulnerable to criticism regarding their va-
the term.
lidity for varied types of ICT system weaknesses beyond
In summary, the interdependecies between non-functional
latency.
attributes, and their impact on the critical aspect of latency
and throughput is difficult to predict at design time. The 4.2 ICT development
development of methods for analysis of how different ICT
system architectures that consider these non-functional as- An alternative to finetune the power system applica-
pects of ICT and the effect on latency and throughput these tion is to analyse and design the ICT system so that it
architectures have is necessary. Examples of such devel- provides the needed level of quality for the power system
opments are outlined in section 5. applications. There exist a number of examples of this
approach throughout literature. In an early paper [8], the
4 Ongoing Research and Development capabilities of an IP based network was evaluated and was
also through simulations verified to be sufficient for the
There is a multitude of activities ongoing to address intended applications at that time. In a completely differ-
the challenges with developing WAMC systems and their ent approach presented in [11], the characteristics of the
supporting ICT systems. In this section, we present a se- communication system, assumed to be a dedicated sensor
lection of these activities, grouped into three categories. network, is modeled using three different methods from
First those activities which work to adapt the power sys- signal processing and communications theory. The ap-
th
17temPower SystemsinComputation
application Conference
order to make it less dependent on Stockholm
proach is interesting Sweden
since it uses - August 22-26,
the characteristics of 2011
the
measurement data stream, and the requirements from the NASPInet bus adresses all interfaces discussed in the
power system function as a basis for defining generic char- WAMC architecture in section 2 above. The NASPInet
acteristics of the communication system. In [16] a method architecture also includes services of cybersecurity and re-
for analysis of dataflows in NASPInet is presented. Al- siliency intended to manage the non-functional aspects of
though developed specifically with NASPInet in mind, the ICT listed in section 3 above. At the same time, full scale
utilised framework, and the components developed for the implementation of the proposed architecture is still not
NS-2 simulator, can be used for other of WAMC architec- complete, and will require a significant effort. Also, the
tures as well, the challenge lies in developing simulator design of instances of the architecture will require analysis
models that represent components in these other architec- tools and simulations in order to verify the applicatibility
tures. of the design and its fulfilment of requirements.
In general, adressing the performance challenge from A similar concept to the NASPInet architecture is the
the ICT system perspective serves the same purpose as that gridstat.net middleware being developed at Washington
of finetuning the power system application. Which per- State University, see for instance [18]. The gridstat.net
spective to attack is very much dependent on the practical project adresses the same issues as NASPInet and through
situation faced, in situations with a fixed ICT system so- the use of the QoS+ concept described above, develops
lution it may be necessary to improve the power system implementation guidelines for Wide Area Monitoring and
application. While, if the ICT system is not yet deployed Control systems. The Gridstat architecture is based on
it may be worhtwile to apply a simpler power system ap- Quality of Serivce guarantees for Multicast communica-
plication and instead design the ICT system accordingly. tion, which is at the core of a publish-subscribe communi-
Support for such trade-offs need to be found in advanced cation model such as that described in the NASPI architec-
modeling and simulation tool platforms, see section 5. ture. The Gridstat architecture is based on a the concept of
a management plane in which meta-level communication
4.3 New Paradigms
attributes are communicated between participating nodes
From the above presentation of PMU based applica- in order to set up communication channels and manage
tions, it is apparent that for individual applications it is changes in the communication or power system configu-
possible to develop a sufficiently good ICT system. Keep- rations.
ing in mind the constant development of new applica-
tions, reducing the risk of building stovepipe architec-
tures is best done by a more comprehensive approach to
building ICT systems. The NASPInet project presented in
[31] is the most comprehenisive approach to these chal-
lenges. NASPIs ultimate objective is to decentralize, ex-
pand, and standardize the current synchro-phasor infras-
tructure through the introduction of a NASPI network
(NASPInet) that will be composed of Phasor Gateways
(PGs) and a Data Bus (DB), both of which shall, where ap-
plicable, utilize, be compatible with, and integrate within
Figure 5: Overview of Gridstat.net architecture[15]
the set of Common Services of the respective Requesters
enterprise IT infrastructures. As such, the NASPInet ar- A similar, but not as well developed proposals is the
chitecture adresses all of the non-functional aspects pre- PhasorNet outlined in [5] and the open system architecture
sented above in one comprehensive platform. presented in [6]. Both of these developments acknowledge
the many aspects of designing ICT systems for WAMCs,
and provides some examples of possible solutions to these
problems. Neither of them provide complete solutions to
the problem however, and do not provide the same depth
and scope as the two preceeding examples.
[11] S. Kirti, Z. Wang, A. Scaglione, and R. Thomas, [22] D.E. Bakken, R.E. Schantz, and R.D.
On the Communication Architecture for Wide-Area Tucker, Smart Grid Communications : QoS
Real-Time Monitoring in Power Network In pro- Stovepipes or QoS Interoperability, 2009.
ceedings of 2007 40th Annual Hawaii International http://gridstat.net/publications/TR-GS-013.pdf.
Conference on System Sciences (HICSS07), Jan.
2007. [23] P. Narman, P. Johnson, L. Nordstrom Enterprise Ar-
chitecture a framework supporting system quality
[12] H. Wu, K.S. Tsakalis, and G.T. Heydt, Evaluation of analysis, in Proceedings of 11th IEEE International
Time Delay Effects to Wide-Area Power System Sta- Enterprise Distributed Object Computing Confernce,
bilizer Design In IEEE Transactions on Power Sys- (EDOC), 2007, Annapolis, MD, USA, pp 130-141.
tems, vol. 19, Nov. 2004, pp. 1935-1941.
[24] M. Chenine and L. Nordstrom, Investigation of com-
munication delays and data incompleteness in multi-
[13] W. Yao, L. Jiang, Q.H. Wu, J.Y. Wen, and S.J.
PMU Wide Area Monitoring and Control Systems,
Cheng, Design of Wide-Area Damping Controllers
In proceeding of International Conference on Elec-
Based on Networked Predictive Control Consider-
tric Power and Energy Conversion Systems, 2009.
ing Communication Delays In proceedings of Power
EPECS 09. , 2009.
and Energy Society General Meeting, 2010 IEEE
[25] P.T. Myrda and K. Koellner, NASPInet - The Internet
[14] S. Ray and G.K. Venayagamoorthy, Real-time for Synchrophasors Proceedings of the 43rd Hawaii
implementation of a measurement-based adaptive International Conference on System Sciences - 2010
wide-area control system considering communica-
tion delays In Generation, Transmission & Distri- [26] M. Chenine, L. Vanfretti, S. Bengtsson, and L. Nord-
bution, IET, vol. 2, 2008, pp. 62- 70. strom, Implementation of an Experimental Wide-
Area Monitoring Platform for Development of Syn-
[15] H. Gjermundrod, D.E. Bakken, C.H. Hauser, and A. chronized Phasor Measurement Applications Sub-
Bose, GridStat : A Flexible QoS-Managed Data Dis- mitted to IEEE Power Engineering Society General
semination Framework for the Power Grid In IEEE Meeting, 2011
Transactions on Power Delivery, vol. 24, 2009, pp.
136-143. [27] NIST, Introduction to NISTIR 7628 Guidelines
for Smart Grid Cyber Security The Smart Grid
[16] R. Hasan, R. Bobba, and H. Khurana, Analyz- Interoperability Panel Cyber Security Working
ing NASPInet data flows,In proceedings of 2009 Group, September 2010, http://csrc.nist.
IEEE/PES Power Systems Conference and Exposi- gov/publications/nistir/ir7628/
17th Power
tion,Systems Computation Conference
IEEE, 2009. Stockholm Sweden - August 22-26, 2011
introduction-to-nistir-7628.pdf
[28] EU DG Energy, Taskfore Smartgrids, Expert [30] GridWise Architecture Council, GridWise In-
Group2, Regulatory Recommendations for Data teroperability Context-Setting Framework (v1.1),
Safety, Data handling and data protection. June March 2008. http://www.gridwiseac.org/
2010. http://ec.europa.eu/energy/ pdfs/interopframework_v1_1.pdf
gas_electricity/smartgrids/doc/
expert_group2.pdf [31] Data Bus Technical Specifications for North Ameri-
can Synchro-Phasor Initiative Network (NASPInet),
May 29, 2009, http://www.naspi.org/
[29] NITRD, The Networking and Information Tech- resources/dnmtt/naspinet/naspinet_
nology Research and Development, Moving databus_final_spec_20090529.pdf
Toward Trustworthy Systems: R&D Essentials,
http://www.nitrd.gov/about/about_ [32] http://openpdc.codeplex.com/x_
nitrd.aspx About
17th Power Systems Computation Conference Stockholm Sweden - August 22-26, 2011