Cybersecurity Metrics

Supporting accurate and timely decision-making

November 2018

Anthony Muiyuro
Cybersecurity Leader, EY East Africa.
Contents

01
Business drivers
While more questions are being asked about cybersecurity, current reporting is not adequate.

02 Challenges
Organizations are struggling to determine what and how to report on cybersecurity.

03 Building an Effective metrics program

Improve cybersecurity reporting requirements.

04 The CISO Dashboard

Page 2 23 November 2018 Cybersecurity Metrics & Dashboards

Metrics story

"Measurement is the first step that leads to

control and eventually to improvement. If you
can't measure something, you can't understand
it. If you can't understand it, you can't control it.
If you can't control it, you can't improve it."

- H. James Harrington

Page 3 23 November 2018 Cybersecurity Metrics & Dashboards

Business drivers

Page 4 23 November 2018 Cybersecurity Metrics & Dashboards

It can be challenging to communicate the value of Cyber
Security in business terms

How does a CISO Are our security

defend the cyber investments paying
security budget? off?

Are cyber security

services delivered in a How secure is our
fashion that meets organization?
business needs?

?
Are our response
capabilities How well are we
adequately managing identifying and
the impact of responding to
incidents to the relevant threats?
organization?

How do we continue
to get support for
As the focus on cyber security has Is our security
continually increased, Information program on track to
cyber security efforts
achieve maturity
from executive Security functions are faced with a objectives?
leadership?
number of difficult questions

Page 5 23 November 2018 Cybersecurity Metrics & Dashboards

Telling “the Cyber Security story” is complicated for many
reasons
Information Security lacks a mature common language to describe its
Lack of common language
complex environment in terms of business value

Consistent, timely and relevant data to support reporting often is not

Difficulty in obtaining required data
readily available

Varying Information Security organizational structure and

Organizational differences
responsibilities make it difficult to standardize reporting focus areas

Lack of performance baselines There are no established widely accepted performance baselines

Legacy approach to security reporting is focused on tracking what is

Legacy thinking
being done vs. how well it is being done

Most traditional ways of reporting focus on available data rather than the needs of the reader

Page 6 23 November 2018 Cybersecurity Metrics & Dashboards

Stakeholders to ‘Manage’

Is the cybersecurity strategy Are security initiatives on track to remediate Is the organization
aligned with our business risks and improve security? complying with policies
strategy? and regulations?

Is the money spent

on cybersecurity creating
value?
How efficient have
our tools been in
protecting against
Chief Executive Functional Chief Compliance cyber attacks?
Officer Leads Officer

Board
Chief
Information Chief Risk Internal Audit Chief Information
Security Officer Officer Officer

Do we have real-time insights into

critical incidents, threats and
vulnerabilities impacting our What are our key risks Do we have appropriate
environment? and how can we mitigate and effective controls in
them? place?

Page 7 23 November 2018 Cybersecurity Metrics & Dashboards

Organizations are struggling to determine what and how to
report on cybersecurity.
Cybersecurity metrics are often presented
Cyber threats are just one of the many risks
as key risk indicators or key performance
that organizations face. most organizations
indicators that are accurately measurable;
struggle with fully understanding what they
however, these often tell “nothing but the
need to report on and to whom (e.g., to
truth,” but not the “whole truth” as they lack
boards, audit committees)
business context.

Legacy approach to security reporting is

Existing cybersecurity, governance risk and
focused on tracking what is being done
compliance (GRC), and service
versus how well risk is being reduced. As a
management technologies increasingly
result, current reporting does not provide
have dashboard and reporting capabilities
the insight needed to take risk-based
but are often not integrated.
business decisions.

Many executive cyber reports are largely

Existing reporting often lack actionable
manually compiled on an ad-hoc or
information that can be used to remediate
inconsistent frequency and require
issues quicker and more effectively.
significant effort and time to produce.

Page 8 23 November 2018 Cybersecurity Metrics & Dashboards

Developing an Effective Metrics Program

Page 9 23 November 2018 Cybersecurity Metrics & Dashboards

Well designed metrics support decision making

INFORMATION = DATA + VALUE + CONTEXT

Decisions that take into

Right consideration both
information Response external and internal
capabilities factors and
demonstrate response
capabilities
+
In the right Decision Strategic
Decisions that
strategically align with
format organization’s vision
support alignment
and objectives

+
Decisions that
Operational maximize operational
At the right excellence efficiency and
time effectiveness

Page 10 23 November 2018 Cybersecurity Metrics & Dashboards

Three categories of security measures are critical in
enabling decision making
Relative State Relative State of Security State of IS Operations
of IS Program Progress Posture Performance

Progress enabled with context from Processes evaluations against

Technical data contextualized
the broader Cyber Security program performance objectives (e.g.,
Reports… (e.g., counts, percentages, forecast
against internal and external
timeliness, quality, consistency,
relevant factors
to actual, burn rate, etc.) effectiveness, etc.)

What are we doing? Are we doing enough? How well are we doing?
Answers… (security projects & initiatives) (security controls) (security processes)

Supports… Strategic alignment

• Strategic alignment and Response capabilities Operational excellence
• Operational excellence
• Response capabilities
Operational excellence

Time-bound Outcome-based
Characterized as… and Outcome-based and
Outcome-based Quality-focused

Page 11 23 November 2018 Cybersecurity Metrics & Dashboards

Maturity Goal

Page 12 23 November 2018 Cybersecurity Metrics & Dashboards

Organizations cannot wait until they have reached their desired
maturity to begin measuring security
Many Cyber Security organizations erroneously opt to delay implementation of
Good performance management programs in order to allow their functions to mature. This
approach puts underdeveloped and unsophisticated cyber security organizations at
metrics drive greater risk of not getting the attention and investment they need to transform and
change develop as they lack the metrics and measurements necessary to demonstrate their
value to the overall business as well as the gaps that exist.

Security performance management enables organizations to improve

within and across maturity levels
INITIAL REPEATABLE DEFINED MANAGED OPTIMIZING

Metrics can help Metrics can assist Metrics can assist Metrics can be Metrics can drive
identify high-risk in obtaining in the formalization used to effectively continuous
areas for targeted visibility into some of security report on the security program
improvement and basic repeatable functions and performance of enhancements
support funding processes while services by operational and performance
requests for larger driving quantifying activities and the improvement
efforts by providing performance to performance quality of services towards strategic
visibility into desired levels expectations and delivered goals
“security” gaps reporting on
progress

Page 13 23 November 2018 Cybersecurity Metrics & Dashboards

Improving cybersecurity reporting requirements.

Page 14 23 November 2018 Cybersecurity Metrics & Dashboards

Cybersecurity reporting should enable accurate and timely
decision-making
Reporting must:
► Provide a realistic view of cyber risk posture
► Be readily available and produced consistently for all stakeholders
► Demonstrate analysis, knowledge and expertise

► Critical incidents
► Risk posture/trend
► Spend status/ROI
The
► Compliance
Board

► Portfolio status/health CISO, CIO,

► Financial and organizational health (e.g., budget, headcount) other C-level

► Control health (e.g., patching, malware protection)

► Mapping to controls (e.g., NIST, ISO)
Functional or
► Project status/health
domain leadership

► Operational risk (e.g., incidents, threats, vulnerabilities)

Operational leads
► Activities status

Page 15 23 November 2018 Cybersecurity Metrics & Dashboards

Improving the maturity of your cybersecurity reporting
*NIST - National Institute of Standards and Technology *KPI – Key Performance Indicator
*ISO - International Organization for Standardization *ISO – Key Risk Indicator
Optimizing
Managed ► Dashboards are
actively used in
Defined ► Strategic and tactical decision making.
metrics, KPIs* and
► Real-time dashboard
► Metrics cover most KRIs* to monitor
Repeatable with advanced
security domains. coverage and
analytics capabilities
effectiveness.
► Tactical metrics ► Metrics provide high- driven by threat
Initial ► Dashboards leveraged
focused on select level view of security intelligence and
security domains. risk across the for decision-making. predictive modeling.
► Ad hoc metrics;
created only when enterprise. ► Mostly automated ► Granular views and
► Manual dashboards
requested. created at some ► Dashboard dashboard; very broader audience
regular frequency. development is limited manual effort. (e.g., business units,
► Manual dashboards
(e.g., Excel, operationalized in a ► Most metrics pulled Human Resources,
► Effort still largely
PowerPoint). visualization tool with from source system in Privacy, regional
manual.
many manual near real-time. views, financial views).
► Highly manual effort. ► Metrics may be repeatable steps.
► Metrics measure ► Dashboard utilized for
influenced by industry
► Data is pulled from the health against industry communications and
frameworks.
source system for frameworks. awareness across
majority of metrics enterprise.
► Dashboard audience
(e.g., Splunk, Qualys).
expands to key
► Metrics are aligned to executives (e.g.,
leading industry Board, Audit
frameworks (e.g., Committee).
NIST*, ISO*).

Fig. Maturity model for cybersecurity reporting

Page 16 23 November 2018 Cybersecurity Metrics & Dashboards

Cybersecurity dashboards can help provide tangible
contributions to the organization.

Customizable dashboard and

Near real-time* insights into critical
reports to suit various reporting
threats and incidents
needs

Improved
Increased visibility into informed Integrated and
risk posture strategic and consolidated cyber
and control gaps financial dashboards
decision-
making

Predefined profiles to target Graphic and visual representation

specific organizational roles of actionable insights

*Depending on availability of data and capability of organizational tools

Page 17 23 November 2018 Cybersecurity Metrics & Dashboards

Developing a systematic framework to create relevant,
comprehensive, automated dashboards.
Metrics should be part of the life cycle with continuous assessment and improvement steps. The output has direct impact
for a business from financial to risk reduction.
Cybersecurity dashboard framework

Inputs Value
Posture
Assess gaps and risks,
and measure the coverage,
Portfolio/ effectiveness and impact of
Policies Framework
projects existing controls and
processes alignment

Security metrics
Stakeholder Improve Measure
information Processes
needs Initiatives
Assess progress and Risk Improved
Govern effectiveness of security reduction – spend
activities in improving posture informed allocation
and reducing risk decisions
Asset
Objectives
criticality

Report
Operations Improved
Measure critical operational capabilities
Strategy Controls activities, effectiveness and
performance

Industry trends Emerging technologies Threat landscape Laws and regulations

Page 18 23 November 2018 Cybersecurity Metrics & Dashboards

Sample dashboard artifacts

Page 19 23 November 2018 Cybersecurity Metrics & Dashboards

Demo dashboard: CISO executive overview
Target audience: CISO and the leadership team
Objective: cover key operational, controls health and project status metrics

Page 20 23 November 2018 Cybersecurity Metrics & Dashboards

Demo dashboard: CISO executive overview
Target audience: CISO and the leadership team
Objective: cover key real-time operational metrics for daily usage

Page 21 23 November 2018 Cybersecurity Metrics & Dashboards

Demo dashboard: business unit overview
Target audience: business unit IT leaders
Objective: highlight cyber risks for applications tied to a business unit and what risks to focus on first

Page 22 23 November 2018 Cybersecurity Metrics & Dashboards

Demo dashboard: cyber operations
Target audience: CISO and the leadership team
Objective: key metrics on incident, threat and vulnerability management

Page 23 23 November 2018 Cybersecurity Metrics & Dashboards

Demo dashboard: CISO overview mobile view
Target audience: CISO and the leadership team
Objective: key operational metrics

Page 24 23 November 2018 Cybersecurity Metrics & Dashboards

Lets Discuss…………….

Page 25 23 November 2018 Cybersecurity Metrics & Dashboards

EY | Assurance | Tax | Transactions | Advisory

About EY
EY is a global leader in assurance, tax, transaction and advisory services.
The insights and quality services we deliver help build trust and confidence in
the capital markets and in economies the world over. We develop outstanding
leaders who team to deliver on our promises to all of our stakeholders.
In so doing, we play a critical role in building a better working world for
our people, for our clients and for our communities.

EY refers to the global organization, and may refer to one or more,

of the member firms of Ernst & Young Global Limited, each of which is
a separate legal entity. Ernst & Young Global Limited, a UK company limited
by guarantee, does not provide services to clients. For more information
about our organization, please visit ey.com.

About EY's Advisory Services

EY Advisory believes a better working world means helping clients solve big,
complex industry issues and capitalize on opportunities to grow, optimize
and protect their businesses.

A global mindset, diversity and collaborative culture inspires EY consultants

to ask better questions, create innovative answers and realize long-lasting
results.

The better the question. The better the answer. The better the world works.

© 2018 EYGM Limited.

All Rights Reserved.

EYG no: 01799-183GBL

ED None
This material has been prepared for general informational purposes only and is not intended to be relied
upon as accounting, tax or other professional advice. Please refer to your advisors for specific advice.

ey.com