Enterprise Architecture Blueprint For Utilities
Enterprise Architecture Blueprint For Utilities
ARCHITECTURE
BLUEPRINT FOR UTILITIES
– PART 1:
Technology, Dev Tales, News
“The Good, The Bad and the Ugly”
Some days ago, I was co-hosting a webinar on “Piecing together
the Enterprise Architecture Puzzle for Utilities”. During our
virtual roundtable, our guests including Klaus Wagner, (Enterprise
Architect, Netze-BW / EnBW) and Rinse Veltmann (Director
Solutions, Energyworx) discussed an enterprise architecture
blueprint for the Utility 4.0 future and highlighted specific
related topics including:
• Challenges or shortcomings with this kind of an “Accidental
IT”, that is quite often the As-Is architecture for many
utilities.
• Strategies such as “Make vs Buy”, “Cloud vs. Data Center”,
“Best-of-Breed vs. 1-Stop-Shop” or “Pull IT vs. Push IT”.
• Principles and concepts like Event Driven Architecture (EDA),
API-driven development, domain driven design (DDD), cloud
native, microservices and data mesh.
Our lively debate, in which the panelist shared their views on
what was good and bad architecture emphasized: What
characterizes a good or bad architecture? What does “good” and
“bad” mean? Does “good”, “bad” and even “ugly” mean the same
for all of us? And can we measure it? See it? Feel it? Or
otherwise sense it?
So let me start by sharing some of the typical indicators and
symptoms of a bad architecture from my point of view:
• Monoliths determine the processes, routines or services,
meaning the utility has to adapt the systems’ capabilities.
• Siloed applications offer no or minimal standardized
integration capabilities and make entirely digitized processes
impossible.
• Core systems are heavily customized and therefore expensive
to update or upgrade.
• Custom built or developed solutions serving “commodity”
functionality (i.e. CRM, ERP, etc.).
• Critical solutions need special knowledge or expertise to
operate, manage or adapt and create vendor lock in.
• Black-box applications nobody (internally or externally) wants
to touch anymore as no one knows how those work.
• Adaptions (i.e. some new fields in an API) take several
months, are costly and slow therefore down innovation.
• Applications provide overlapping or competing functionality
and lead with that to inconsistent decisions, processes or
data.
• Systems have been thickened, meaning new requirements are
always realized in the same application by the same vendor,
even the functionality would have belonged into the domain
of another.
• Underlying infrastructure (i.e. storage, messaging, integration)
cannot be scaled elastically or require substantial
investments to meet the upcoming requirements to handle
big energy data in almost real time.
• IT landscape is built on a huge variety of (partly aging)
technologies, different (maybe antiquarian) architecture styles
or (old-fashioned) proprietary integrations create enormous
operational complexity and fragility.
• Each solution or system uses different mechanism to manage
security or data privacy and its own logging & tracing
framework.
READ MORE IN OUR E-ZINE
“I am a Man of Principle”
“I am a man of fixed and unbending principles, the first of which
is to be flexible at all times” – a quote from former US Senator
Everett McKinley Dirksen. Most probably he thought about
politics in his role as senator, but the statement would equally
suit coming from enterprise architect, too!?
In part 1 – “The Good, the Bad and the Ugly” of our Enterprise
Architecture Blueprint, I focused on technical debt utilities have
built up over the years due to an “Accidental Architecture”. In
part 2 – “Bad choices make good stories,” I identified some KPIs
for a good architecture. And now, in this post, it’s time to move
on and talk about architecture principles.
Architecture Principles
According to the Open Group Architecture Framework better
known as TOGAF, principles are general rules and guidelines,
intended to be enduring and seldom amended, that inform and
support the way in which an organization sets about fulfilling its
mission. TOGAF details: Architecture Principles are a set of
principles that relate to architecture work. They reflect a level of
consensus across the enterprise, and embody the spirit and
thinking of existing enterprise principles. Architecture Principles
govern the architecture process, affecting the development,
maintenance, and use of the Enterprise Architecture.
TOGAF has laid out a set of 21 principles wherefrom some
experts would prioritize those 8 as the ones you need to know.
As a man of principles, let me introduce some of the most
important ones that, at least in my opinion, that are applicable
for the future digital utility.
Interoperability and Integration-friendly: The digital utility of the
future requires entirely digitized value chains and automated
processes. To achieve this, all applications and systems should
be built with integration and interoperability in mind and expose
documented interfaces using modern integration methods.
Standardization of Data and Meta Data: The Common
Information Model (CIM) emerges as a de facto standard for the
power industry and for transmission and distribution service
operators. In addition, we see more and more vendors are
leaning towards CIM as their de facto standard. Implementing a
unified information model inspired by CIM creates measurable
benefits for utilities.
Loose coupling: Utilities should get rid of technical debt as fast
as possible. To do so, dependencies to given applications,
vendors or proprietary platform technologies must be reduced or
minimized. Loose coupling between systems and services
increases exchangeability and flexibility.
Modularity: The future utility must be able to quickly adapt to
new opportunities, regulations, or market changes. As many
utilities still operate huge monolithic applications, they are
having to compromise and wait often several months for minor
changes or investing hundreds of thousands of Euros for
alterations or additional functionalities or fields in a user
interface. A modular application landscape creates the
foundation for the future success.
Event driven architecture: As more and more smart meters and
sensors are deployed in the electrical network and at the grid
edge, utilities must be able to handle a tsunami of big energy
data in real time. The traditional process driven integration
approach will fall short: Orchestrations get too complex; error or
exception handling gets extremely complicated; performance;
and message throughput will lag. Integration will be a nightmare
for both development, operations, and management.
Implementing the concept of an event driven architecture is key
to building a resilient and reactive integration system. A system
which is designed and built for scaling as needs change and
demands require.
API driven development: The entire industry is moving towards a
distributed energy system forcing utilities to operate a
distributed and modular IT and data architecture. API driven or
API-first development fosters a modern IT landscape with
holistically digitized and automated integrations.
Cloud native and containerized applications: Many utilities are on
their cloud journey. Most having set off with a single cloud
strategy. However, as every cloud provider (Microsoft Azure,
Google Cloud Platform, Amazon AWS, or other) has its own
strengths and drawbacks, utilities are recognizing the benefits of
embracing on a multi-cloud strategy with cloud native and
containerized applications: leveraging multiple cloud or
computing services (public cloud, private cloud), but in an
integrated and distributed cloud-mesh architecture.
Data Mesh: The future utility is data and analytics driven. Many
energy companies have recognized the value of data and
identified the potential benefits with implementing a data lake.
To avoid the same drawbacks experienced and typically
associated with an “Enterprise Warehouse” approach, that
included sizeable investments in terms of expenditures and
resources to build and manage a centralized and monolith data
warehouse platform, utilities should instead focus on
implementing a distributed data lake, also known as a data
mesh.
I am sure there are several other useful architecture principles.
So, I encourage getting a discussion started to establish a set of
best practice architecture principle for utilities.
ENTERPRISE
ARCHITECTURE
BLUEPRINT FOR UTILITIES
– PART 4:
Technology, Dev Tales, News
Source: Gartner
Source: Gartner
The “Pace layered Architecture for Utility 4.0” adds some specific
layers with defined characteristics:
System of Connectivity: A modern data integration layer based
on the concept of an Event Driven Architecture (EDA) and
reactive microservices handling all data flows and data exchange
between all the Systems of Record and between the layers to
establish the entirely digitized operational and business
processes that define the Utility 4.0.
System of Intelligence: A big energy data management and
analytics infrastructure layer used to deliver the cognitive
capabilities for the Utility 4.0: Smart grid edge analytics to create
insight and transparency in the network, intelligent grid
operations, asset health, predictive maintenance, demand / load
forecasting, EV charging optimizations, identification of
abnormalities and more, are just some examples of the cognitive
use cases.
System of Engagement: A digital user experience layer providing
the services and digital excellence both the digital native
consumers expect but also the utilities’ employees need to
handle the huge amount of data.
I feel fortunate to have been given the opportunity to implement
this architecture model as the foundation for a big
transformation program at one of the “Big Three” utilities in
Germany.
Introducing the “System of Connectivity” highlighted the value of
a modern data management approach and set the spotlight on
the importance of the “dirty job of data integration”. Placing the
“System of Intelligence” at the center of your architecture
fosters the cognitive capabilities the future utility needs to
achieve the sustainable development goals and business objects.
Plus, the “System of Engagement” improves the digital user
experience for the end-users, meaning for you and me as
consumers or prosumers. But maybe even more importantly, as
more and more data is digested in real time to drive faster
insights and decision-making, this applies to utility employees
too.
BUILDING BLOCKS OF AN
ADVANCED METERING
INFRASTRUCTURE - PART
1:
The Headend System
A recent study of AMI professionals at leading DSOs revealed
that regardless of the market set up or regulatory position, all
utilities are currently investigating the business case for
deploying modern smart meters to modernize the low voltage
grid and integrate the last mile of the smart grid. Driving the
rollout of next generation smart meters to improve customer
service and support advanced analytics in the smart grid at
minimal expenditure forms the backbone of Advanced Metering
Infrastructure in today’s context. But what makes up this
infrastructure? And which building blocks do utilities need to
understand and consider?
As industry specialists in utility data integration and enterprise
architecture, especially when it comes to Smart Metering, we
advise and consult utilities globally and support application and
system vendors implementing projects impacting millions of
consumers. As a result of this experience, we see a variety of
approaches to central architectural decisions that utilities are
taking. In this article we explore key factors and provide our
perspectives related to infrastructure cornerstones in the
advanced smart metering value chain, namely the headend
system. So what are the different types of headend systems on
the market today and what constitutes a good and future-
oriented integration architecture to support these systems?
The role of a Headend System
The role of the headend system or systems (HES) in the Smart
Metering architecture is typically two-fold. The main objective of
HES is to acquire meter data automatically, avoiding any human
intervention, and to monitor parameters acquired from meters.
In this case HES is managing the connectivity and scheduling the
collection of data from the metering infrastructure including
both the meter devices and communication. However, HES also
enables secure access to meters for configuration, software
updates and ad-hoc requests.
During the last years, our customer projects, ongoing tenders and
talks with partners supporting both HES and MDM solutions
around the world have indicated that the market is currently
undergoing a paradigm shift with regards to HES Northbound
integration. This paradigm shift is mainly based on two
developments:
1. The growing acceptance of IEC 61698-9 (CIM) as de-facto
standard and data model for all integration between HES and
MDM solutions;
2. Increased competence of customers around enterprise
integration, introducing concepts of loose coupling and event-
driven architecture (EDA), even in the smart metering value
chain.
The growing acceptance and implementation of CIM empowers
customers to be less dependent on the software suppliers’
services and makes it easier for third-party tools to take over
the responsibility for MDM-HES integrations.
In addition, growing competence and experience in the area of
enterprise integration makes customers realize that p2p
integrations in general are not sustainable and that the concept
of loose coupling and event-driven architecture is significantly
cheaper and more effective in the long run.
Another important factor is based on many lessons learned from
earlier AMI roll-outs where p2p integrations effectively prevented
quick and reliable error analysis and resolution, based on the
simple fact that no over-arching monitoring of all involved
integration flows was available, leading to an endless “finger-
pointing” game between involved vendors.
How to make the right architectural choices regarding headend
systems (HES)
All these factors have led to us recommending utilities (and
software vendors) implement a data integration layer, even on
the HES Northbound side, between HES and MDM. Ideally, this
integration layer supports CIM, loose coupling and event-driven
architecture out-of-the-box. It should also support real-time
streaming of data and be highly resilient.
When it comes to the role and integration of HES within the
Smart Metering value chain, the following overall highly effective
and resilient architecture is recommended for utilities:
Figure 4: Resilient and effective HES architecture
In this article we were only focusing on HES and its integration
into the Smart Meter value chain. We demonstrated that
universal HES may not be as promising as they seem, and we
explained why we think that Northbound HES integration should
go via an adequate integration bus.
In our next article we will describe the role of the Meter Data
Management System (MDM) and its integration into the Smart
Meter value chain. We will also give an outlook on the future of
MDM solutions as the core element of the Smart Meter value
chain based on different likely scenarios.