0% found this document useful (0 votes)
69 views

Ebook Guide To Product Metrics

Uploaded by

ide santos
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
69 views

Ebook Guide To Product Metrics

Uploaded by

ide santos
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 17

A Guide to


Product Metrics
Fundamentals for defining key performance
indicators that enable growth
Introduction

What you’ll learn from

this guide

“What success metrics should I be tracking?”


If you build or market products, you've likely asked yourself this

question. In this guide, we'll help you find the answer.

We’ve partnered closely with hundreds of companies on their analytics strategies, and

have found there is simply no single set of metrics that works for everyone. Businesses

and products are all unique and have different goals based on stage and ambitions.

An ecommerce startup will want to leverage venture capital to acquire new customers

quickly, while a 100-year-old bank will be more focused on retention and upsell within an

already large customer base.

With these nuances in mind, we’ve put together a framework to help you find the right

metrics for your product. It also helps clarify relationships among all of your metrics, so

product managers, engineers, analysts, and growth marketers can see how their day-to-

day work on features and campaigns connects to the overall performance of the product.

When the system is working correctly, it looks like Hinge growing their user base 400%,

Viber increasing daily messages sent by 15% across a billion users, and Kaplan increasing

mobile users by 20%—just to name a few examples we’ve seen first-hand.

As you read on, you’ll learn

How to figure out your set of key metrics, as well as your focus metric

How active usage, reach, activation, engagement, and retention complement one

another in helping you measure the health of your product

How companies in different industries can customize the core measurement concepts

to fit their business

A simple method for building your own key metrics framework

Introduction A Guide to Product Metrics 01


Introduction

The framework

Every company we’ve worked with has a handful of metrics that sum up their product’s
overall performance. But among those, there is typically a metric that is slightly more
important than the rest, which we call a focus metric. 



Then there are more granular metrics that teams and individuals spend their time on to
drive movement in the focus metric. We call these level 1 and level 2 metrics. 


If a company’s metrics are set up well, there is a natural hierarchy and upward flow of
impact. An individual achieving their goal should advance the team toward its goal, which
should in turn get the department or company closer to its goals. 



We’ll dive deeper into the metrics categories and how to pick the right measurement
framework for your product, but here is a common setup we recommend:

Focus metric Active usage


Matters most to your business

Level 1 metrics Reach Activation Engagement Retention


Business-

Complement the focus metric specific

Level 2 metrics
More specific and drive the

L1 and focus metrics

Example: L1 broken down by platform, region, segment, or feature

Introduction The Guide to Product Analytics 02


O r ga n i z i n g Yo u r M e t r i c s

Focus metrics and what


matters most to your
business

If you’re experienced with analytics, you’ve likely heard the term


“North Star metric” before. The growth and analytics experts at
Reforge write well on the topic and explain why a company should
have a constellation of metrics, rather than one metric that
matters. A single North Star can be limiting, and we prefer the
softer framing of a focus metric that works in concert with other
To know whether key performance indicators (KPIs).


you have a good


focus metric, ask Single-mindedly maximizing your focus metric can hurt your
yourself this: product. For example, if a news site cares only about videos
watched, they might autoplay videos once a page loads. This

“If we improve 
 ultimately frustrates visitors who don’t want to watch the content
and drives them to a competitor’s site, reducing retention. Focus
this number will the
metrics should be the top priority, not the sole priority, and
product’s long-term
improving the focus metric should not be accomplished at the
performance expense of harming other KPIs.

improve?”
For this reason, we typically recommend choosing a focus metric
tied to active usage, such as weekly (WAU) or monthly (MAU)
active users. These metrics do a good job of summing up trends in
other metrics, like acquisition and retention, by showing whether
more people are using the product over time. We’ll dive deeper
into the topic later, but by “active users,” we mean people who go
beyond visiting or logging in and also take a key action that the
product was built for.

Chapter 01 The Guide to Product Analytics 03


Organizing Your Metrics

Metric levels

Level 1 metrics
Complement the focus metric

Level 1 (L1) metrics should either directly contribute to the focus metric or act as a check
to make sure the product is growing in a healthy direction. For example, if a product’s
focus metric is WAU, a good L1 metric would be 7-day retention to ensure you aren’t
spending precious marketing funds to acquire new users who leave after a day or two.

Level 2 metrics
More specific and drive the L1 and focus metrics

This type of metrics framework is infinitely customizable and you can continue adding as
many layers as you’d like. To take the retention example one step further, the level 2
metric could be iOS app retention, and nestled under that could be a level 3 metric such
as the retention of a regional iOS app or a specific feature within the app.

While it’s possible to keep adding layers of metrics, be careful about creating confusion
around what actually matters. Having too many goals can be just as ineffective as having
none because everyone will have different ideas about where to spend their time to make
the product successful.

Want expert guidance on your analytics strategy?


Mixpanel can recommend a custom measurement framework for your
product. Get in touch and we’d be happy to help.

[email protected] Mixpanel.com

Introduction The Guide to Product Analytics 04


DEFINING YOUR KPIs

The key metric categories

Reach
Reach is the total number of people who have used the product in a recent time period.
For consumer companies, it could be the number of paid accounts or users who have
made a purchase in the past three months. For B2B companies, this key metric is often
product install base or number of paid licenses within the past quarter or year. 


Reach is important because it represents the maximum amount of users who could
reasonably become active, whether organically or through re-engagement campaigns.

Activation
Activation is a foundational step that primes a new user to become an active user.
Famously, Facebook identified adding “7 friends in 10 days” as their activation metric
when they were a startup. They found it was a key milestone in driving long-term usage,
and thus made adding friends a central part of their onboarding experience.


For other types of products, activation could be defined as registering, making a first
purchase, viewing five videos, or making two deposits within a specified time period. 


We recommend viewing the metric as a percentage of new users rather than a count in
order to isolate it from natural user growth. That way you can know if you’re more
successful at activating users over time.

Active usage
Active users are people who have taken a key action and received value from your
product within a recent time period. Value could be defined as one action, like playing a
song, or a set of actions, such as playing three songs and creating a playlist. 


Products that promote habitual usage, such as Twitter or Instagram, look at DAU.
Business software is better viewed through the lens of WAU since it’s not always used
every day, especially not on the weekends. MAU would be a good fit for a bill payment
portal, since bills are usually due monthly. 


Active users is the most common focus metric within our customer base since it’s a great
snapshot of whether more, or fewer, people are getting value out of a product over time.

Introduction The Guide to Product Analytics 05


Engagement
Engagement is different than active usage because it measures a deeper level of
commitment to the product. It accounts for both the frequency and cadence of
completing key actions, answering the question: “How engaged are your active users?”

Engagement could be defined as the number of key actions taken, minutes of video
watched, or number of transactions completed. It’s important to divide this by your active
user count to measure the depth of engagement per user with your product. Otherwise, user
growth might mislead you into thinking your product is more sticky than it actually is.

Retention
Retention is the metric that shows whether your product has staying power. Think about
what drives retention in two ways: Are you bringing in the kinds of people who will stick
around? Are you giving those who have already come through the door enough reason to
come back?

When deciding on time frame for retention goals, pick a range that is long enough to
capture the reasonable repeat visit cycle of your customers yet short enough that teams
can get feedback to iterate quickly. We generally recommend having seven-day retention
as a leading indicator for 30- or 90-day retention.

Business-specific
If you have the sorts of metrics listed above, as well as L2 metrics that drill deeper in
those categories, there may still be some crucial gaps in your analytics strategy. These are
often specific to your business model.

For example, a dating site we work with uses the metric "good churn" to represent users
who find a lasting relationship and leave the app. While this involves losing users, it’s
good for business because satisfied customers will refer friends and come back
themselves if ever in the market again.

In contrast, a financial wellness company we work with tracks the percentage of users
whose account balances increase month over month. If that number goes up, it means the
product is helping users to budget successfully, which is their differentiating value in a
competitive industry.

Introduction The Guide to Product Analytics 06


Reach Weekly active users Engagement WAU retention
How many people Are people How engaged 
 How many of your
used the product performing a key are your active active users come
recently? action? users? back?

AA

AAA
AA
AA
50% 

12 users in last
 AAA (2/4)
6 months
4 users AA

2.5 key actions / WAU



(10A/4)

Eureka!
25%

(1/4) Drives active usage,


New users engagement and
retention

Activation
What % of new users

have experienced value?

Introduction The Guide to Product Analytics 07


Industry-specific metric examples

When it comes to the metrics that matter in each industry, here is what

we’ve most commonly seen.

Financial services SaaS Retail & ecommerce Media

Question
How many people have used the product in a recent time period?

Reach
Account holders, signed- Users from paid accounts, Active buyers (6-month), Paid subscribers, viewers
Sample in users (3-month active licenses paid subscribers (3-month window)

metric window), subscribers

Question What percentage of new users have onboarded and experienced your product’s

value?

Activation % of new users

who:

Sample
Made first deposit Completed Completed first Watched five videos
metric
within seven days registration within 30 purchase in the first seven

days days

Question Are people showing up regularly and performing a key

action?

Active 

usage
Sample Weekly active users (WAU), monthly active users

metric (MAU)

Question How engaged are your active

users?

Transactions, Viewed reports, Conversions, 
 Minutes watched,

deposits multi-feature users recommended product article reads

Engagement views
Sample
metric

Divided by active users (WAU or

MAU)

Question How many of your active users come

back?

Retention
S even- or 30-day
Sample
retention
metric

Question How else does your business deliver

value?

Business-
 Savings/debt ratio of 
 ARR/DAU, support 
 Average purchase Shares/WAU, 



Sample users, fraud events /user tickets/WAU price, 
 Multi-device users/WAU
specific
metric cart abandonment rate

Introduction The Guide to Product Analytics 08


Media Example

A closer look

From clothing to food delivery, subscription-based businesses are booming. Their long-

term health depends on their ability to attract the right audience, convert them into paid

subscribers, provide them with a great experience, and retain them as loyal users.

Subscription-based video streaming product

From clothing to food delivery, subscription-based businesses are booming. Their long-

term health depends on their ability to attract the right audience, convert them into paid

subscribers, provide them with a great experience, and retain them as loyal users.
ic

Owner
r

C ompany
o c u s Met

Weekly active subscribers (WAS) KPI


F

RE ACH Acti vation Eng agement Retention B u sines s -Specific


ic s

Marketing G rowt h Product & content Product & marketing Product


r

n
 O S

Met

Subscription i Minutes
 ne-week WA Avg. revenue



Subscribers

seven da y
s/users viewed/WAS retention /subscriber
L1

Action Completion
ic s
r

o
 Product Content Content


Met

Time t
Retained Reactivated New
subscription

Video starts/ Complete/ Same sho w



L2

WAS start retention

Introduction The Guide to Product Analytics 09


Media Metrics

Focus metric

Weekly active subscribers (WAS)

The crucial aspect of any active user metric is to define what makes a user active. Here,

expecting a paid subscriber to watch at least one video every week makes sense because

any fewer would mean they aren’t getting enough value to justify the cost of the

subscription.

Reach

Subscribers

The number of paid subscribers is the ceiling for weekly active subscribers, making it the

right reach metric. Subscriptions are the life-blood of this business because they provide

recurring revenue. In exchange for the monthly fees, subscribers expect to receive value

that matches or exceeds what they’re paying. Otherwise, they’ll churn.

For subscriber growth to be healthy, the retained and new subscriber L2 metrics should

also be growing. If new subscribers are increasing but retained are declining, then there is

a churn problem. If retained subscribers are growing but new subscribers are not, then

there is an acquisition issue.

Activation

Subscription in seven days/user

This tracks only the new user cohort and shows the percent that become a subscriber

within seven days of their first visit. The one-week timeframe is important because an

inability to drive upgrades early on means you need to improve your onboarding or trial

experience.

Introduction The Guide to Product Analytics 10


Enagement
Minutes viewed/WAS

This metric quantifies the value each user gets from the product. The more engaged users
are, the more value they get. Minutes viewed is a natural measure of value received in a
video product, and dividing that by weekly active subscribers (WAS) normalizes the
number.

The L2 and L3 metrics isolate the types of engagement so teams can focus on each factor
that drives the top line.

Video completions matter because when users repeatedly start new videos then move on,
they aren’t finding the content compelling and probably won’t stick around.

Retention
One-month buyer retention

The retention time frame of one week captures the typical viewing cycle. It’s long enough
to see accurate trends but short enough to make a change if the featured content or
product experience is driving down viewership. Monitoring same-show retention will
help measure the stickiness of each production.

Business-Specific
Average revenue/subscriber

This metric is important if you run pricing promotions or test different pricing structures.
It’s relatively easy to bring in new subscribers with a 50%-off coupon, so use this metric
to check whether your growth is healthy and sustainable.

Introduction The Guide to Product Analytics 11


ECOMMERCE EXAMPLE

Online clothing retailer

The company in this example sells affordable clothing, primarily through their website.

The product team wants a healthy pipeline of buyers coming through, so their primary

goal is to increase that number. By working closely with the growth and marketing teams

to increase the reach of the product, they can get the right people in the door, then

optimize the site experience to increase cart size and repeat purchases.
Fo c u s Metric

Pr od uct

Weekly active buyers

RE ACH Activation Eng agement Retention B u sines s


L1 Metric s

Marketing Growth Product Product & Marketing Product

Three-month
 First purchase in
 Bought
 One-month buyer
 Averag e



active users seven days/user items/WAB retention buying price
L2 Metric s

Shopping cart
 k active



z
% New with
 Bought
 One-wee
New Existing Total cart si e
same-day purchase items/searches abandonment user retention

Introduction The Guide to Product Analytics 12


Ecommerce metrics

Focus metric

Weekly active buyers (WAB)

In this case, active means making a purchase. Site visitors who browsed may come back

but aren’t included in the focus metric because they aren’t yet driving value for the

business.

Reach

Three-month active users

The reach metric also includes people who have searched or browsed recently. This gives

product owners a good sense for who may make a purchase in the near future. They’ve

broken this metric into new users and existing ones and, among those, sorted by

reactivated and retained.

Activation

First purchase in seven days/user

This tracks only the new user cohort and shows how many of them make a purchase

within seven days of first searching or browsing. The two sub-metrics track the health of

specific parts of the user journey, from showing up to purchasing.

Introduction The Guide to Product Analytics 13


Enagement
Bought items/WAB

Multi-item shopping trips are a fast way to increase revenue for this retailer. If the
shopper has already found something they like and inputted credit card information,
adding items to the cart is minimal incremental effort.


The L2 and L3 metrics dig into specific parts of the purchase funnel so the team can
identify problem areas. If cart abandonment is high, they know to focus on streamlining
the final stages of the buyer experience.

Retention
One-month buyer retention

This looks at whether buyers return to make another purchase the following month,
which is in line with this company’s customers’ buying habits.

To get more immediate feedback, the company tracks one-week retention of users who
search or browse. If they don’t find what they’re looking for, something is going wrong in
the period between searching and buying.

Business-specific
Average buying price

This metric measures the average price per item purchased and you should strive for
trends that match your pricing strategy. This company’s goal is to offer low prices, so they
want this number to go down and total cart size to go up. A luxury goods retailer would
want a higher average buying price but, in turn, wouldn’t expect 

a large number of items per purchase.

Introduction The Guide to Product Analytics 14


Me
W ot
rrkis
chset
eetm p l at e

Mixpanel measurement

framework

Company Name: Product:

Below is a blank metrics template so you can apply the framework to your product. 


Print this page out and fill in the blanks. Start with a focus metric, then add the level 1 and

2 metrics.

Focus metric

L1 metrics

Re ach activation Eng agement Retention B u sines s -

specific

L2 metrics

Introduction A Guide to Product Metrics 15


Outro

One final note


It’s imperative to remember the goal of measurement isn’t just to track change over time;
it is to effect change as well. To observe metrics over time requires rigor, discipline and
focus. To improve them requires all that plus ingenuity. And that’s what building and
managing products is all about.

At Mixpanel, we’re building a user analytics solution designed to track, manage, and
improve the metrics that matter most to businesses in all verticals and growth stages.

Want expert guidance on your analytics strategy?


Mixpanel can recommend a custom measurement framework for your
product. Get in touch and we’d be happy to help.

[email protected] Mixpanel.com

Introduction The Guide to Product Analytics 16

You might also like