CFO

Cloud Finance Onboarding by

CLOUD FINANCE ONBOARDING (CFO): MODULE 5

Cloud Performance Indicators

© 2023 Amazon Web Services, Inc. (AWS) or its Affiliates. All rights reserved.
 Cloud Finance Onboarding (CFO)

Cloud Cloud Cloud Cloud

Visibility & Cloud Cost Financial Financial Performance
Accountability Optimisation Predictability Operations Indicators

© 2023 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 2
 1 Measuring Cloud Cost Efficiency: KPIs

AGENDA 2 Business Value Efficiency: Unit Economics KPIs

3 Improving Cloud Performance Indicators

© 2023 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 3
Measuring Cloud Spend Efficiency: KPIs

© 2023 Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Measuring Performance: Choose your KPIs
The best KPIs will simplify complex data to drive meaningful action towards your objectives

Essential Progressive Unique to YOU

(not comprehensive) (not overwhelming)
An initial set of KPIs that aims to answer the A KPI structure that takes into account the A proposal that provides flexibility to build
most important questions level of complexity of implementation your own dashboard based on your own
circumstances
Start here
What? Why?

AB001
What exactly are we Why is this important?
trying to measure?
KPI A ✓ Start point, not ‘set in stone’
Then add this one
What behaviours are we Low hanging fruit
How can we remove trying to incentivise? ✓ Specific vs. Thought-provoking
ambiguity around this
metric? ✓ Can be split into different levels
AB002 KPI B

Finally, add this one

Where? How to improve?
Next phase KPI ✓ Centralised vs. At-the-Edge
What data do we need? What can we do to ✓ Ultimately, different for each
AB003

improve this metric? KPI C (or improved KPI A) customer; unique to you
Where can we usually
find this data? Where should we focus
High maturity KPI
our CFM time to deliver
results?
© 2023 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 5
What makes a good KPI?
Characteristics of a valuable KPI

Relevant Understandable

Measurable Visible

Timely Comparative

Actionable Owned
© 2023 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 6
 On Measuring Performance

KPI (Key Performance Indicators) KMI (Key Movement Indicators)

KPI KMI

Example: Example:

(current’s month) (previous to current month)

EC2 Savings Plan Coverage%: EC2 Savings Plan Coverage%:
80% 90% - 80% = -10%

© 2023 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 7
 A Proposal: ‘SEE-SAVE-PLAN-RUN’ Dashboard

Cloud Visibility Cloud Cost Cloud Financial Cloud Financial

& Accountability Optimisation Predictability Operations

A framework that provides a template to measure and drive cloud financial

management (FinOps) excellence across its four main pillars / domains.

© 2023 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 8
 SEE (Cloud Visibility & Accountability)
What? Why? Where? How to improve?
This is a basic metric providing
SE001

AWS Spend Growth Rate AWS Cost Explorer: Observe evolution of SEE. SAVE, PLAN,
perspective on the trending of AWS
Bill Trend Rate (Daily, Weekly, Monthly,
spend, allowing as well to track
Total (or Workload-specific) Cloud RUN; to gain increased perspective into
3-Mo, 6-Mo, Annual) Spend for periods to be evaluated the nature of the observed growth.
unexpected behaviour.

AWS Organizations helps to Ensure that accounts that would

AWS Organizations:
SE002

Percentage of AWS accounts that maximise commitments utilisation by benefit from AWS Organizations are
Standalone Linked Accounts
do not belong to an AWS sharing, governance enforcement via added; and also setting an inclusive
Accounts Rate AWS Admin:
Organizations structure sharing guardrails, and minimise automated account creation process,
All Accounts
oversight via consolidated billing. to have them included by default.

In order to maximise adequate AWS

SE003

Percentage of AWS spend that spend allocation (e.g. minimising

Untagged AWS Cost Explorer:
does not correspond with any cost centralised charges), companies Improve KPI SE004.
Spend Rate Untagged Spend
allocation tag should maximise ratio of spend that
corresponds with cost allocation tags.

In order to maximise adequate AWS Manually tag untagged resources;

SE004

Untagged spend allocation (e.g. minimising apply proactive IAM policies to prevent
Number of AWS resources that do AWS Tag Editor:
centralised charges), companies untagged provisioning; configure
Resources not have a cost allocation tag
should maximise number of
Untagged Resources
reactive rules to automatically detect
resources with cost allocation tags. and fix untagged resources.

Adequate Cost Allocation provides AWS Cost Explorer: Improve KPI SE003; and properly
SE005

Percentage of total AWS spend

Unallocated collective accountability, a view on Tagged (and untagged) Spend define/refine allocation methods
that has not been allocated to a
Spend Rate profitability, and exhaustive AWS User: especially for both shared and
business unit, product line, etc. showback/chargeback of AWS spend. ‘untaggable’ sources of AWS spend.
Allocation Rules and Centralised View

© 2023 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 9
 SAVE (Cloud Cost Optimisation)
What? Why? Where? How to improve?
Virtualization provides multiple
Estimated monthly savings from: Ensure rightsizing recommendations
SA001

Compute compute choices for customers

AWS Cost Explorer:
A) Rightsizing underutilised are considered on monthly reviews,
Rightsizing resources; AND
(family, size), but it is also key to
Computing Rightsizing Opportunities and evaluated by tech teams, in order
Opportunities* ensure we are only using what we
B) Terminating idle resources to minimise compute usage waste.
need, since we pay for what we use.

Commitments provide drastic rate Pre-purchase: ensure to maximise

SA002

Percentage of utilisation of each reductions for steady-state flexibility possible; post-purchase: (if
Commitments
active SP/RI commitment workloads, but also imply 24/7 AWS Cost Explorer: SP / RI Utilisation KPI is not close to 100%) investigate
Utilisation Rate (monthly) usage: is key to maximise utilisation with all teams where estimated usage
of each purchased commitment. is not being met anymore.

Commitments provide drastic rate Consider coverage candidate resources

SA003

Percentage of all eligible on- reductions for steady-state on monthly reviews (with help from
Commitments
demand spend not covered by any workloads, but also imply 24/7 AWS Cost Explorer: SP / RI Coverage tech teams), and ensure to maximise
Coverage Rate active SP/RI commitment usage: is key to gradually maximise flexibility possible. Also leverage AWS
the coverage of eligible resources. Organizations to share commitments.

Non-critical EC2/RDS resources

AWS Cost Explorer: Improve KPI SE005 (proper cost
SA004

Non-Production Ratio of Non-Production EC2+RDS (dev/test/staging) can be scheduled

Actual and Applicable Spend allocation), and consider on quarterly
Cloud Actual Spend ($) / Total to run only during office hours (40
Elasticity Score Applicable 24/7 Spend ($) hours/week), paying only for what we
AWS User: reviews opportunities for improving
Allocation Rules savings on non-production resources.
need to use, and saving up to 76%.
Estimated monthly savings from On one hand, be aware of AWS’ latest
Customers should embrace all cost- AWS Cost Explorer:
SA005

migrating EC2, RDS, ElastiCache, generation announcements; and on

Modernisation efficiency opportunities stemming Current Applicable Spend
OpenSearch and EBS resources to the other hand, consider these
Opportunities their respective latest generation
from AWS evolving its offering via AWS User:
opportunities on quarterly reviews, and
resource modernisation. Latest Generation Pricing and Matching
versions evaluated by tech teams.

© 2023 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 10
 PLAN (Cloud Financial Predictability)
What? Why? Where? How to improve?
Customers should feel in control of Recalibrate based on observed
A: Percentage of set budgets that
PL001

their AWS spend and ideally avoid behaviour; identify and fix root cause
Bill Overage breach unwanted thresholds (a.k.a.
any surprises (e.g. Bill Overage), by
AWS Budgets:
for breaching unwanted thresholds
Frequency 'Bill overage') in a given period building on their cloud financial
Number of budget alerts
(AWS Cost Explorer, AWS Cost
B: Total number of 'Bill Overages' predictability. Anomaly Detection)

Recalibrate based on observed

PL002

Percentage of the delta between This provides context into the behaviour; identify and fix root cause
Bill Overage AWS Budgets:
total variance and total set budget severity of the Bill Overages that for breaching unwanted thresholds
Variance % Budget Variance (in Budget History)
amount (in the budget's period) actually occur. (AWS Cost Explorer, AWS Cost
Anomaly Detection)

Fine-tune your forecasting: method

Good forecasts are the basis for good AWS Cost Explorer:
PL003

Forecast Percentage of the delta between (simple trend-based, simple driver-

cloud financial predictability, and Actuals
Accuracy: actuals and forecast (on a monthly
helping avoid Bill Overages. It's all AWS User:
based, ML-based, unit-metrics-based),
Monthly basis) and external factors (customer growth,
about setting the expectations right. Forecasts
seasonality, workload roadmap)

Remediation speed is key to mitigate Implement automated remediation

AWS Budget Actions:
PL004

Bill Overage Average time to evaluate and collateral damage (e.g. endless actions in AWS Budget Actions; or
Automated remediation native tool
Remediation remediate any observed 'Bill Shock' Lambda functions, dev/test
External to AWS:
implement ticketing systems (e.g. JIRA)
Time in a given period environment compute/database to help drive awareness and speed in
Customer Ticketing System (e.g. JIRA)
resources, etc.) solving remediation requests.

Some use cases might require more Fine-tune your forecasting: method
AWS Cost Explorer:
PL005

Forecast Percentage of the delta between granularity in forecasting periods (simple trend-based, simple driver-
Actuals
Accuracy: variance and forecast (on a daily (workloads' earlier stages, or when
AWS User:
based, ML-based, unit-metrics-based),
Daily basis) cash preservation is crucial). This KPI and external factors (customer growth,
may need some automation support. Forecasts
seasonality, workload roadmap)

© 2023 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 11
 RUN (Cloud Financial Operations)
What? Why? Where? How to improve?
Cloud Financial Management is an
Percentage (or weighted average) Lead a process of organisational
RN001

evolving process that requires all

CFO Education of all relevant stakeholders that
stakeholders to achieve optimal
External to AWS: culture change, education evangelising,
Score have completed a defined set of Cloud fluency (how to operate in a
Customer Repository senior executive championing,
basic FinOps / Cloud certifications celebrating educational wins.
dynamic spend environment)

The idea is to encourage the

Percentage (or weighted average) AWS Console:
RN002

Revise the topic on a quarterly basis,

organisation to actively think how to
Governance of all governance methods (e.g. Service Configurations where the benefits of each option are
enact governance in their Cloud
Score AWS Tools) that are being actively CFM Stakeholders: evangelised, understood and planned
usage, with the aim to accelerate
used by the CFM program Who is doing it? for piloting and launch.
efficient Cloud adoption.

Especially in earlier stages, it is

RN003

Both act on the feedback received, as

Average NPS from all relevant important to get a view/opinion from
CFO CSAT External to AWS: well as communicate, communicate,
stakeholders on the actual CFM all the relevant stakeholders;
Score program including a numerical value plus a
Customer Repository communicate to avoid surprises and
maximise stakeholder alignment.
space for anecdotes.
A: Average time to evaluate Whilst optimisation is key, any
External to AWS:
RN004

Optimisation optimisation recommendations recommendation needs assessment; Implement ticketing systems to help
Customer Ticketing System (e.g. JIRA)
Evolving Score B: # of recommendations therefore it is recommended to start drive awareness and speed in
AWS Cost Explorer:
implemented with assessment speed, then implementing this type of request.
AWS (delta) Spend
C: $ of savings achieved implementation, then actual savings.

Finding the right balance between Identify FTE efficiencies along the
RN005

Total cost of running Cloud human effort and automation usually External to AWS: journey, or shift to automation as scale
CFO Running
Financial Operations (FTEs and involves scale as a key deciding Employee/Salary Data, justifies it. Also, on tools: reduce overall
Cost Automation Tools) factor, and this KPI helps assess and Cost of Building/Running Tools AWS bill (for tools charging based on
when automation makes sense. AWS bill).

© 2023 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 12
 KPI ownership

› Which stakeholders needs to be notified of a › Do we need to fix the problem to avoid it from
KPI variance? recurring?
› Which stakeholders/groups need to perform › What is the fix plan?
Root-cause-analysis? › What is the timeline to implement the fix
› What is the SLA for determining KPI root cause? plan?
› Which stakeholders need to be notified of a root › Who is going to implement the fix plan?
cause?
› Can we justify the KPI variance?
› Which stakeholder signs off on it?

© 2023 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 13
 Who is responsible for what (personas)
Finance FinOps Tech
Persona Persona Persona

✓ Validate and make available cloud

✓ Lead the process of defining a usage data (AWS) required for
Dashboard Roadmap, with all KPIs obtaining KPIs
✓ Validate and make available to be included (definition, ✓ Commit to the improvement of all
business data (outside AWS) calculation, data required, and KPIs, both directly (by
required for obtaining KPIs ownership) implementing optimisation
(especially for UNIT METRICS) ✓ Deliver a timely KPI reporting recommendations on your own
mechanism for all stakeholders, workloads) as well as indirectly (by
defined at the team level evangelizing KPI wins across the
organisation)

© 2023 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 14
Tying Cloud Spend to Business Value:
Unit Economics

© 2023 Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Cloud Unit Economics
A ‘Unit Metric’ is a measure of costs (or consumption) associated
with a particular business model (e.g. service, workload), expressed
on a per-unit basis

AWS spend $ Product owner

= incremental unit cost
Demand Driver
Engineer

AWS resource usage

= incremental unit consumption
Demand Driver
Finance
team member

© 2023 Amazon Web Services, Inc. or its Affiliates. All rights reserved. IT professional 16
 Example
If ‘Active Customers’ shows high Unit
AWS $ / Active Customer
correlation (R2) with AWS spend … Metric

Cloud Cost-Based KPIs

Total AWS invoice $
$ / Amazon EC2 hour
$ / GB storage
Active Customers AWS resources $ / GB of Data Xfer
consumption $ / AWS service consumption units

Value-Based KPIs
Total AWS $ / Active Customer
Demand Cloud Amazon EC2 $ / Active Customer
driver activity resource usage GB $ total storage / Active Customer
Data Xfer $ / Active Customer
Individual AWS services $ / Active Customer

© 2023 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 1 Amazon Elastic Compute Cloud 17
 The Cloud Spend Conundrum
As organisations increase their cloud footprint, each time the cloud bill
goes up, a debate typically opens about whether spend is good or bad
“AWS cost increased 10% MoM” (overall cost doesn’t provide any business context)

Cloud Spend
Was it because of inefficient usage
patterns being built up over time?
Was it because of acceleration in our
cloud migrations?
Was it actually due to organic
business growth?

Past Time Present

© 2023 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 18
 Unit Metrics: CFM/FinOps Nirvana
Unit Metrics help to identify the value of allowing a certain amount of
spend to capture additional revenue
“AWS cost increased 10% MoM…while transactions increased by 30%, number of
customers by 15%, and revenue by 25%”
Cloud Spend

Unit cost (i.e., cost/transaction)

A rising cloud spend pattern now put
into perspective with a declining ‘unit
cost’ trend, showing increased
economies of scale
(e.g. ‘a good thing’)

Past Time Present

© 2023 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 19
 Demand Drivers
Finding the right unit metrics is a fine art, and it relies on finding the
right ‘demand drivers’ (those most highly correlated to cloud spend)

Demand Cloud Demand Cloud

driver activity resource usage driver activity resource usage

Examples • Daily/weekly/monthly active • Page clicks

• Searches customers or users
• Concurrent logins
Demand Drivers should have a high
• Bookings • Seconds or GB of video streamed
• Incident reports correlation with cost and usage.
• Reports • Successful API responses
• Unit of Work “R squared” or “Pearson” coefficient.
• Batch Jobs • Order lines processed
• Unit of Production
• Backend Jobs • Line of code
© 2023 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 20
• Unit of Output
 Increasing granularity

AWS resources in their respective units of consumption

can also be used to quantify engineering efficiency when
expressed with the ‘unit metric’ as the denominator

Amazon EC2 GB Individual AWS

hours storage service unit of measure

Active Customer Active Customer Active Customer

© 2023 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 21
 Unit Metrics in Context (1/2)

When is it okay to increase unit cost or unit resource consumption?

Fixing a bug Implementing a Rolling out infrastructure for new

regulatory requirement features or products

© 2023 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 22
 Unit Metrics Benefits

Engage in corrective Quantifies the financial and Allows for the identification
actions earlier by identifying operational impact of architectural of impactful cost
negative variation and code changes* drivers—sensitivity analysis

Progress toward more accurate Identifies fixed operational costs Promotes data-driven
and agile driver-based forecasting from variable operational decision making
costs—improves your modeling

© 2023 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 23
 FLY (Unit Metric)
What? Why? Where? How to improve?
Begin here your Unit Metrics Easily affected by various business
UM001

Overall Spend AWS Cost Explorer:

monitoring journey. It provides a decisions (e.g. free tier offerings,
Ratio of Total Cloud Spend ($) to Total Cloud Spend ($)
to Revenue ‘quick and dirty’ way to determine if
External source:
marketing campaigns), compromising
Total Applicable Revenue ($) growth in Cloud spend is correlated the integrity of the KPI. Recommended
Ratio Total Revenue ($)
to profits for the organisation. as a temporary KPI.

Operational Unit Metric (engineering

UM002

EC2 Compute A combination of evolving SEE. SAVE,

Ratio of Total EC2 Compute Spend efficiency) to aim for the most cost AWS Cost Explorer:
PLAN, RUN; in particular Savings Plans,
Efficiency ($) to Total EC2 Running Hours efficient usage of EC2-based EC2 Spend ($),
elasticity, newer generation EC2,
Ratio (hrs) (in a given month) compute; one of the key drivers of and EC2 Running Hours (hrs)
containerisation, serverless compute.
AWS spend.
UM003

S3 Storage Operational Unit Metric (engineering A combination of evolving SEE. SAVE,

Ratio of Total S3 Storage Spend ($)
efficiency) to aim for the most cost AWS Cost Explorer: PLAN, RUN; in particular S3 storage
Efficiency to Total S3 Storage Size (GB)
efficient usage of S3-based storage, S3 Spend ($) and S3 Storage Size (hrs) tiers (Infrequent Access, Intelligent
Ratio (in a given month) one of the key drivers of AWS spend. Tiering, Glacier, Glacier Deep Archive).

An improvement from UM001. It AWS Cost Explorer: Easily affected by various business
UM004

Workload Ratio of Workload-specific Cloud provides increased granularity, and a Workload-specific Spend (account, tag, decisions (e.g. free tier offerings,
Spend to Spend ($) to Total Applicable KPI per individual service. Requires cost category) marketing campaigns), compromising
Revenue Ratio Revenue ($) proper Workload-specific cost External source: the integrity of the KPI. Recommended
allocation. Service-specific Revenue as a temporary KPI.

Workload AWS Cost Explorer: Scale usually brings a natural

UM005

Ratio of Workload-specific Cloud A further improvement from UM004.

Workload-specific Spend (account, tag, improvement, but also we can ensure
Spend to Spend ($) to identified demand Requires finding a suitable demand
cost category) to have the best possible demand
Transaction driver (e.g. # of transactions/API driver with a strong correlation to a
External source: driver (e.g. high correlation to
Ratio calls in the same period) Workload-specific Cloud spend.
Service-specific Demand Driver Workload-specific Spend)

© 2023 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 24
 Full Proposal: ‘SEE-SAVE-PLAN-RUN-FLY’ Dashboard
1 Start here, by focusing on 'Cloud Spend Efficiency'

Cloud Visibility Cloud Cost Cloud Financial Cloud Financial

& Accountability Optimisation Predictability Operations

+
2 Unit Metrics
… so you can progressively focus
here, and be able to ultimately
measure and drive 'Business Success’

© 2022 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 25
Improving Cloud Performance Indicators

© 2023 Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Where to start

- Customer’s degree of maturity +

Low Ideal
Effort Scenario

✓ Start with implementing all ✓ Implement orange KPIs ✓ Implement red KPIs
applicable green KPIs
✓ Try to achieve 100% dashboard ✓ Expand KPI coverage with
✓ Start defining your ideal KPI implementation across teams (in additional granularity dimensions
Dashboard (what KPIs will its simplest / actual form) (e.g. EC2 vs. RDS vs. Elastic
actually make it there and in Search; monthly vs. daily;
what form) ✓ Incorporate dashboards across all workload-specific vs. total AWS
teams into a unified spend)
✓ Start piloting a simple dashboard communications strategy (e.g.
in a few teams and on a celebrating wins, encouraging ✓ Become a Unit Metrics Master!,
centralised manner healthy competition) including a mechanism to
automatically collect data,
✓ Pilot your first Unit Metrics KPI, generate and track UM KPIs
including definition, testing and across all applicable services,
collection of demand drivers for products
one service
27
Thank you!
Q/A