Measuring Digital Advertising

in a Multi-Media Context
A Guide and Toolkit
 02 Background
03 What is Cross-Media Measurement and Effectiveness?
04 Key Principles for Measurement Success
06 Tools and Methodologies
07 Brand Studies
09 Econometrics / Marketing Mix Modelling
11 Attribution
13 Controlled Experiments
15 Emerging Trends
16 Recommendations and Takeaways

01
 When the IAB UK launched in 1997, For this reason, the IAB have
our mission was focused on driving worked with MTM, our research

Background
the growth of digital advertising. Two partner, and a number of experts
decades later, and with £11.55bn from across the industry to create
spent on digital marketing in the UK this guide and practical toolkit.
(IAB/ PwC Adspend 2017), growth is This guide consolidates current
no longer the most important factor. best practices and provides
Now it is about supporting the long- guidance on measuring digital
term health of the industry and advertising in the context of
ensuring it develops in a sustainable other media. It sets out the main
way. Born from this, is our vision to models and techniques that
“build a sustainable future for digital can be used to measure digital
advertising” and an industry that advertising, from big picture
works for everyone – businesses and approaches (e.g. econometrics)
consumers alike. Within this, one of to more granular analyses (e.g.
our key aims for 2019 is to tackle and attribution modelling), showing
address the big issues affecting the how they fit together and how to
digital landscape. use them. The guide concludes
with a set of practical templates
One of the key challenges for our and checklists for creating your
industry is how digital advertising is own measurement strategy.
measured. Our 2018 survey of brands
found that 83% of brands view cross-
media measurement as their single
biggest measurement challenge
(IAB UK, 2018). Advertisers are faced
with growing complexity and an
increasing number of methodologies.
This has created silos of expertise
in organisations based around
specific objectives and tools rather Special thanks to everyone
than supporting one unified, across for their time and
balanced approach. contributions.

02
 Advertising measurement can be challenging
given the broad array of topics and questions
Digital Media · What is being consumed?

it covers (as shown in Figure 1). Consumption

What is Measurement can mean different things to
different functions and roles. As such, there
Audience · Who is consuming what?

Cross-Media
is no silver bullet for ‘solving measurement’.
What matters is striving for continuous
improvement and giving brands clear insight
Measurement
and confidence on what their advertising

Measurement has achieved.

“The industry continues to face a

Advertising · What ads were served?

and Delivery & · What ads are seen across

significant challenge with measuring which channels?
effectiveness across a fragmented Exposure
media ecosystem. There isn’t a silver

Effectiveness? bullet but there are many helpful tools

and research available to guide us.”
Alastair Mack, European Media Manager, Honda
Advertising · What is the impact of the
campaign on brand metrics?
Effectiveness · What is the return
This guide focuses on how to measure
advertising effectiveness and business
& Business on investment?
outcomes across different media channels. Outcomes · What action does the
This is just one of the many aspects of business need to take?
measurement, as set out in Figure 1.

Taking a digital channel perspective –

it explains how to think about and measure Cross-Media · How did the different
channels perform
digital channels correctly within the context
of other media by using the best tools and
Measurement individually and as
methodologies available. a whole campaign?

Figure 1.
03 Different types of measurement
 • Be clear about what you are measuring –
map out the measurement activities already
Define
happening across channels and then define Campaign
specific objectives
Confronted by a wealth of data and available Objectives
tools, it is important for marketers to be • Clarify the goal and the target audience
e.g. are you building awareness, repositioning
strategic about what they are measuring.

Key Principles
the brand, driving sales, which demographic?
Careful planning needs to take place
• Review the balance of long term brand building
before selecting measurement tools and
and short term activation activities and the role
methodologies. Firstly, brands must take of each channel

for
time to define the overall campaign objectives
• Articulate the role and objectives of each
(e.g. whether it’s a new product launch, channel in the campaign
re-positioning the product or raising purchase
consideration or awareness). The specific

Measurement role and objectives of each channel within the

ad campaign should be set out, so it is clear
what the purpose of each media is. Following

Success
• Select KPIs and metrics that relate to the
this, KPIs can be set against each part of the
overall campaign goals – less is more
campaign and then metrics can be selected
• Set KPIs in advance – don’t be tempted
accordingly. Remember to consider what the
to retrofit them post campaign
long term goals are and how you will measure
Set KPIs • Understand what KPIs are provided by
them, as many measurement plans will focus
only on capturing the short term outcomes. to Reflect different tools and how different underlying
factors might impact their validity
For example, the IAB are passionate about Objectives e.g. sample sizes for surveys
removing the overuse of reporting click-through
rates, as they can be a misleading metric for
success and encourage excessive focus on the
short-term.

“Clients often ask us to retroactively

measure media – the problems and
misattributions this causes could • Be aware of information gaps and plan
measurement accordingly
Organise
be largely alleviated if media were
planned with a measurement plan • Make sure digital channels are properly
Data
from the start.” defined in measurement data (e.g. search, Requirements
social, online video etc) so it properly
Michael Salter, Head of Analytics, reflects your marketing activity
Havas Media Group • Aim to continuously improve granularity
and coverage (devices, platforms) -
Figure 2. meaningful results depend upon data
Five key principles accuracy and comprehensiveness
There are five key principles that can help
04 brands prepare for measurement success.
 • Explore how you could use a combination of
tools to provide an actionable insight at each
Integrate
step of your marketing journey Multiple Tools

Checklist
• Design a measurement approach that Organise Data Requirements
aggregates the best tools at each step rather
than relying on a single methodology What are the key categories of
• Integrating multiple tools helps counteract my marketing spend (particularly
biases and breaks down silos within digital)?

Where do I have gaps in my marketing

channel data coverage?

Where can I improve the granularity

and coverage of my data?

• Set expectations for initial results – you Define Campaign Objectives Integrate Multiple Tools
won’t get perfect data straight away and
should treat disappointing results as an What is my primary objective? What are the limitations of my
Test and Learn opportunity to learn
chosen tool and methodology?
• Use results as continuous feedback to How will my objective feed into
inform future marketing activity my long and short term goals? What other tools could provide
• Consider how to report results to different further insight?
teams and tailor the level of detail What do I need to measure
accordingly (e.g. senior management may across channels to address Who else in my organisation could be
only require the key insights)
the objective? working on a similar type of problem?

Set KPIs to Reflect Objectives Test and Learn

Do my KPIs relate directly Have I set expectations that
to my objective? measurement is a continuous
Figure 3. process?
Set the right foundations checklist
How is the KPI calculated
from the method I have What is the action(s) or decision
chosen (e.g.ROI)? I will take based on the results?

The checklist of questions across How will I calibrate KPIs What is the plan to report results
can be used to help make sure from different timeframes, and how will that feed into the next
that these principles are applied channels and tools? marketing activity?
in practice when planning cross-
media campaigns.
05
 This guide covers four key tools that “We are concerned that measurement
brands should consider including in is becoming more segmented and
their measurement approach – brand too granular to provide big picture

Tools studies, econometrics, attribution and

controlled experiments.
insight. We focus on combining tools
to provide clarity on key overarching
questions and tying everything back

and
There are multiple methodologies within
each tool but the focus of this guide is to a clear ROI.”
to explain the core methodology, provide Anthony Burt, Advertising Manager,
guidance on when the tool should be used,

Methodologies
Dixons Carphone
and outline the limitations along with a
practical example of the tool in action. Figure 4 gives an example of how the major Tools can be deployed across
tools can be combined to cover different the customer lifecycle and
When considering which tools to use, timeframes and provide insights for each for varied timeframes but offer
brands need to recognise that the right step in the customer lifecycle. the most actionable insights
mix will depend on the length and
at mapped touchpoints.
complexity of their product sales cycle,
the balance of media spend between
online and offline and the use of non-media
based promotional tools (e.g. price cuts).

Timeframe
It is also critical that the measurement
approach integrates different timeframes
Long Term
and covers the key steps in the customer months to years Econometrics /
lifecycle. Marketing Mix Modelling
(MMM)
Figure 4. Brand Brand
Explanatory unified Studies Studies
framework for tools Mid Term
“The crucial thing is making weeks to months
Controlled Experiments
sure that measurement tools
are used within one holistic
framework and that they are not
Near Term
talking different languages and days
Attribution
pushing out different numbers”
Nick Pugh, Head of Effectiveness UK, Ebiquity

Awareness Action Purchase Loyalty

06 Customer Lifecycle
 What are they? When should I use them?

Brand studies are a collection of tools Brand studies are unique for their flexibility. After establishing a baseline and testing in
used to measure brand metrics that They can provide insight on everything from specific campaigns, brand studies should
cover awareness, familiarity, favourability, creative design through to specific advert become part of the ongoing measurement
consideration and intent. They can also positioning on a webpage. Brand studies strategy and contribute to a culture of
cover claimed behaviours and attitudes. are most effective when used longitudinally continuous learning.
to provide quantitative evidence of the
These metrics are most commonly impact of longer-term brand activities. “A one-off brand study might
measured via responses to surveys provide insight on what drives
delivered across the life of a campaign, “Brand studies are especially brand equity but unless it is part
typically before and after a campaign useful for adding the “why?” to of a continuous process it won’t
(“pre and post”). the “what” of media measurement. be fed back into shaping the long
Econometrics can identify the term strategy”
Best practice surveys adopt a control and financial, ROI impact of advertising
exposed research design to quantify the Charlotte Diemer, Client Director,
(what happened) but we don’t Kantar Millward Brown
difference between those who did and
know why sales increased.
didn’t see a campaign (see more under

Brand Studies ‘Controlled Experiments’). Groups are

Brand studies can add this in.
also demographically matched to ensure For example, did sales increase
the only difference is the exposure to because the advertising made
advertising. people more favourable to the
product? Or simply because more
Figure 5. people were made aware of it?”
Example format for a
brand study survey George Hopkinson, Senior Research
and Project Manager, IAB UK

Split into groups Deliver survey Measure difference

and connected for completion in responses to
into survey panel by both groups calculate brand lift

Control:
Did not get exposed
to your advert → →
Y%

Exposed:
Did get exposed
to your advert → → X%
07
How does this fit in with What are the limitations? Brand Studies Case Study
measuring digital and
• Sample sizes and bias – panel sizes
offline channels together? inevitably compromise on scale when American vacuum manufacturer Shark A post-campaign total Defined Objectives

Source: Facebook Business, Nielson

attempting to capture the ‘true’ audience wanted to understand how combining audience rating estimate was
Brand studies are flexible and can be run response. When combined with the TV and social media could improve also deployed to provide a
individually across different ad channels fragmentation of media behaviour brand awareness. sense check of the targeting
to create a comparable set of metrics that across multiple devices, this can lead to efficiency of the campaign’s TV Reflective KPIs
assess the effectiveness of digital and unrepresentative results due to small To provide a comprehensive assessment, and social media investments.
offline channels side by side. Cross-media sample sizes. they chose to measure a range of
surveys that aim to track multiple channels supporting brand metrics such as ad Shark’s measurement
in one survey will also use external • Survey audience accuracy – even with a recall and purchase intent. approach identified that the
Prepared Data
benchmarks to estimate which channels large sample size, it is often challenging to largest incremental brand lift
respondents were most likely exposed to. accurately identify and segment groups of Shark built a poll upfront to assess their came when audiences were
This helps to prevent double counting and people who have been exposed to different audience’s likelihood to be exposed to its exposed to a combined TV
misattribution. media channels (particularly in digital). TV campaign. This provided a key input and social media campaign.
Integrated Tools
to the design of a robust, exposed / non
• Do not provide financial metrics for exposed measurement framework. This information allowed Shark
planning – brand studies provide a to design a media strategy
quantitative summary of a sample Shark conducted a pre and post campaign that most effectively combined
audience’s claimed attitude in the form of brand lift survey. By comparing the the wide audience reach of TV Test and Learn
brand metrics. They do not measure direct exposed and non exposed groups they with social’s targetability and
consumer actions (e.g. sales) that can be were able to isolate the incremental brand cost efficiency.
easily included in financial plans. effect of advertising in each channel.

• Limits to demographic matching – there

is no guarantee that a control audience is
identical to exposed so the change could be
due to factors other than advertising.
Brand metrics % lift 25

• Unreliable respondent recall – respondent after ad exposure 20

recall of where they saw (or heard) an
advert can be unreliable, with over-claim 15
to TV. To address this, some cross-media
brand surveys providers use external TV Only 10
benchmarks and other measurement tools
5
(e.g. econometrics) to calibrate responses. Social Media Only

0
TV and Social Media
08 Ad Recall Purchase Intent Combined
 What is it? When should I use it?

Econometrics are a set of statistical Incremental sales are those directly MMM is a well-established tool used
tools that aim to quantify the relationship attributable to marketing activity and by brands across all sectors to gain a
between cause and effect in economic enable marketers to identify and quantify top-down evaluation of how marketing
data. In marketing, this takes the form of the impact of their investments. and communications activities are
Marketing Mix Modelling (MMM) which impacting aggregate sales over a
predicts how all advertising activity Under the hood, MMM uses the principle period of months to years.
(e.g. TV, print, out of home, online video, of linear regression to create a sales
social media, and search) translates prediction equation (Figure 6) where the Traditionally MMM has been utilised
into incremental sales. Models can also dependent variable is sales (left hand of the to measure the impact of high-
include the impact of factors outside the equals sign) and the independent variables level channels (e.g. TV, out of home,
advertiser’s control such as the weather. are the marketing inputs (right hand of the radio) on predominantly offline
equals sign). sales. However, with the proliferation

Econometrics /
and growing prominence of digital
channels the technique is increasingly
being applied with greater granularity
and precision cross-media.

Marketing Mix Another common approach,

is to use MMM in conjunction

Modelling with attribution (see next section).

MMM provides insight on which
high-level channel is working best
(digital video vs TV vs social media)
and attribution modelling is then
used to define shorter term,
Figure 6. tactical marketing activities.
The sales prediction
equation behind MMM
Lastly, econometrics isn’t just for
predicting sales impact, it can also
be used on a wide range of B2C
and B2B KPIs including awareness,
consideration and brand equity
measurements.

09
“MMM is great for understanding
the bigger picture - like a plane
What are the limitations? Econometrics Case Study
flying across a city, it can see the
• Requires long term, granular data that
shape of the river running through
takes time to collect and analyse – MMMs Wall’s ice creams had lost market Social (Facebook and Twitter) Defined Objectives

Source: Wall’s, #IPA Social Works; 2016 IPA Effectiveness Awards

but is less helpful for giving run on weekly time series data that ideally share being eaten on average only generated almost half of these
directions to a restaurant.” covers a historic period of 2+ years. once per summer. sales from a media spend of
Harry Davison, UK Client Marketing Added to this, robust forecast results can just 17%.
Science Manager, Facebook take time to emerge, limiting the ability In 2014, launched cross-media campaign Reflective KPIs
to pick up short term markets signals. “Goodbye Serious” across out-of-home This provided quantitative,
(OOH), social and TV to increase awareness testable benchmarks to
• Measuring the contribution of digital – of Wall’s ice cream as an impulse purchase inform the planning of future
How does this fit in with MMMs assume that each input variable is and drive sales frequency. campaigns.
measuring digital and independent and so struggle to capture Prepared Data

synergies (e.g. where a digital campaign Wall’s wanted to understand the impact In a later phase, Wall’s designed
offline channels together? amplifies a TV advert). Large investment each channel would have on incremental a universal impressions metric to
in one major channel (e.g. TV) can drown sales whilst controlling for the effects of enable a more direct quantitative
MMM can directly compare the impact out the contribution of others (e.g. digital). good weather. comparison of reach. Integrated Tools

of digital and offline channels provided

sufficient historic cross channel sales • Collinearity – MMMs need variation Econometrics provided the ideal tool to This provided both supporting,
data is available. MMM can provide further across input variables in order to identify compliment the measurement of brand quantitative evidence for wider
insight across digital and offline by adding what impact each has. In practice, lift studies on awareness and appeal. brand lift studies and a sense Test and Learn

granularity to the advertising inputs that are advertising spend typically varies together check for econometric modelling
included across paid (e.g. TVRs, paid search), (“collinearity”) with campaign activities Econometrics identified that Wall’s had for incremental sales.
owned (e.g. website) and earned (e.g. social turned “on” and “off” at the same time. sold an additional 2.8m ice creams over
media engagement) sources. the summer period.
• Single KPI as an output – this means for
every KPI that is measured a new model
is required, adding cost and complexity
when analysing multiple KPIs.
Universal impressions comparison – volume and cost per impression

Impressions £ / Impressions

OOH Advert 455m - 460m £0.003

Facebook 155m - 60m £0.001

Twitter 25m - 30m £0.0013

10 TV Only 5m - 10m £0.0010

 What is it? Single Touch Attribution When should I use it?
Heavily simplified user journey

Attribution modelling is a Attribution is about conversions.

technique which evaluates how First Touch Conversions can be defined and
· Assigns all value to
different touchpoints contribute the first channel tracked across the customer journey
to a sale or action by assigning · Strong bias to initial
driver of action
by assigning KPIs to brand equity
credit based on their level of metrics through to sales.
involvement.
Last Touch
· Assigns all value to Attribution modelling is widely used in
the last channel
Statistical models are built · Strong bias to final digital marketing due to the volume,
with individual user level data trigger to action accessibility and granularity of data
across channels to analyse the available. Some platforms offer their
difference in media interactions Last Ad Event own attribution systems making
· Assigns all value to last
between “converters” and channel to show ad individual channel assessment straight
· Obscures channel
“non-converters”. These vs creative impact
forward. So, for deep-dives on individual
channels can then be assigned channel effectiveness, especially in fast
contribution ratios in close to evolving channels such as social media,
real-time which are used to attribution is a must-have tool.

Attribution
inform allocation of spend.
Multi Touch Attribution However, the greater value of attribution
More accurate model of user journey
Multiple techniques have been is in helping to determine the relative
developed within attribution to contributions of different types of activity
try and handle the complexity of Custom / Full Path *
Assigns a custom weight
(particularly digital) and enabling rapid
assigning credit across different to every channel based scheduling and creative alterations.
on bespoke journey
customer journeys (Figure 7).
Where attribution thrives is in its ability
Position Based *
Assigns value based
to provide immediately actionable
on purpose of activity insights on short term campaigns with
e.g. U-shaped first touch
and conversion touch lower historic data requirements (days
vs months to years in MMM). Attribution
Time Decay
can also provide a tactical compliment to
· Credits channel closer high-level aggregate forecasts provided
Figure 7. to conversion action
· Bias against early by MMMs.
Range of Attribution engagement channels
Techniques
Choice of model (e.g. last touch to full
Linear path) primarily depends on the nature
· Assigns value equally
· Arbitrary assignment of the product sales cycle and what you
obviating channel impact
are trying to understand. For example,
last touch models can still be helpful
11 * best practice
in identifying which touchpoints are most
powerful for closing conversions whereas
What are the limitations? Attribution Case Study: Clothing Retailer
a position based model (e.g. U shaped)
provides a more comprehensive view of • Risk of over-bias towards digital advertising
end-to-end performance. – attribution provides the most insight on A major clothing retailer with a A test and learn setup verified Defined Objectives

Source: Marketing Management Analytics (IPSOS)

navigation, not causality. Relying too heavily long heritage of driving sales through these insights against specific
on attribution models risks overstating the a direct mail catalogue wanted to thresholds, helping to develop
impact that both advertising and digital assess the potential sales contribution an effective channel strategy.
How does this fit in with channels have on conversions. of increasing investment in its digital Reflective KPIs
measuring digital and • Building a single view of the customer –
channels. The company found they
had been over-investing in
offline channels together? the quality of insight from attribution To understand the diversity of catalogue marketing and
models is heavily influenced by the customer purchase journeys, over 500 under estimating the role
Prepared Data
customer touchpoints tracked. Gaps can microsegments were created based on a digital played in attracting
Attribution is frequently used for digital
lead to incorrect weighting of channel combination of first-party CRM and third- existing customers.
channels, where granular individual actions
contributions (e.g. if a customer searches party audience data. Channels were shown to be
(e.g. clicks) can be tracked most easily.
and evaluates a product on Amazon’s most effective in combination.
Attribution models that combine digital
mobile app but completes the purchase These broke down the customer audience Integrated Tools
and offline channels exist but still face
via the owner’s website on desktop). by lifecycle and behaviour across digital By implementing these
significant challenges in accurately tracking
conversions across platforms. Moreover, search, digital display, mobile, catalogue insights into their media
• Access to data – the most commonly cited and other direct mail. strategy, the company was
there are additional challenges when the
challenge with multi-touch attribution is the able to drive 4.9% y-o-y Test and Learn
purchase occurs offline (in a store) and these
inability to tag campaigns across the major Customer conversions were tracked sales growth the next year.
should be addressed by using other tools in
internet platforms. Marketers need to be as they migrated across channels
combination (e.g. MMM).
realistic and not get lost trying to develop after being targeted by different kinds
the perfect multi-channel picture. of marketing.

“Multi Touch Attribution does not • Short timeframe of insights – an over Multi-touch attribution demonstrated
reliance on attribution can encourage an that each channel benefitted from actions
provide a single ‘true’ attribution
over investment on short term activation taken on other channels across the
- there’s just too much invisible
vs longer term brand building activities. customer journey.
data. Instead, it offers granular,
tactical insight that can be
Only 65% of catalogue sales were driven
a helpful addition to a wider solely by catalogue. The rest came from
measurement project” customers drawn to the catalogue via
Andy Bellis, Director of Performance digital channels that when viewed alone
Analytics, Mindshare had few direct conversions.

12
 What are they? When should I use them?

Controlled experiments randomly assign a group of people to a test or Controlled experiments represent the
control group to observe and quantify the impact of a change in media over most effective way to validate existing
a defined period of time. The test group is exposed to a change in media marketing activities and fill gaps in
(e.g. your new display advert) whilst the control group sees no change knowledge, particularly across high
(ideally users are shown a ‘ghost’ ad which presents a relevant competitive volume digital channels. Experiments
baseline ad). Experiments require a clear, testable hypothesis (e.g. paid can be designed for any campaign
social media will cause a 1% lift in search referrals.) with a hypothesis around a target KPI
where a control and a test group can
be practically exposed to different
Figure 8. Split into randomised Deliver advertising Measure conversions
Example format for a
media. Applying experiments should
groups
controlled experiment be approached as an ongoing process
Test: that can be continuously improved upon
with → Ad →
your advert Y% and extended across measurement
activities, rather than as a one-off tool.

Controlled
Control:
without Controlled experiments work best in
→ Ghost Ad →
your advert X% stress testing key touchpoints and
channels. For example, econometrics

Experiments
activity might have identified that
paid social media advertising is
Conversion results are then compared and quantified. This approach is triggering higher organic search
fundamental for accurately measuring incrementality – conversions due to resulting in increased sales. But
your advertising activity that would not otherwise have happened. before management will sign off on
further spend, they want more robust
(Test Media Conversion Rate - Control Media Conversion Rate) evidence of the uplift across social to
= Incremental Lift search. Controlled experiments offer
Control Media Conversion Rate
the perfect opportunity to adopt best
practice scientific methods to add
Controlled experiments can be utilised within other methodologies (e.g. confidence and precision to marketing
attribution) and are sometimes incorrectly referred to interchangeably investment decisions.
with AB tests and ‘test-and-learn’. AB tests compare two groups without
randomising the allocation of treatment and control. ‘Test-and-learn’ is a
reference to a general approach of doing something, measuring the results
and then trying again.

Controlled experiments are distinctive in the random assignment of control

and test groups which ‘design out’ other factors and isolate the impact of
13 specific media.
“Randomised, controlled
experiments are the gold
What are the limitations? Controlled Experiments Case Study
standard that the industry
• Clear testable hypotheses required –
should aim for. They should
if hypotheses are too generalised EE were aware that their typical Test regions were matched Defined Objectives

Source: Think with Google

be used to answer a small (e.g. do generic search ads drive overall customer’s purchase journey was to control regions with
number of key questions, and sales) that could lead to a poor allocation complex, with many in-store sales similar characteristics -
to calibrate other models like of marketing spend. A better framed preceded by extensive online research. ie. Edinburgh and Belfast -
attribution and econometrics” hypothesis would seek to test the differing to ensure comparability. Reflective KPIs
Matthew Taylor, Econometrics conversion impact of one tailored search They wanted a data-backed explanation
Program Lead, Google ad (“Fluffy pink socks”) vs a generic of the relationship between online Geo-testing enabled EE
search term (“Socks”) on driving sales media and offline sales to test whether to upweight search spend
to your website. increased digital media spend generated in the 12 test regions and
Prepared Data
an incremental sales lift. compare the incremental
• Sample size and scalability – to be
How does this fit in with statistically significant, controlled
sales lift against the
To quantify their ROI, they needed to comparable control group.
measuring digital and experiment groups need sufficient volume. measure variance of offline sales income
This becomes increasingly challenging Integrated Tools
offline channels together? the more channels and media options
across areas targeted with different levels As a result EE were
of digital ad spend. able to show the finance
are tested (e.g. adequately sized test and department a £2.32 ROI for
Controlled experiments are flexible and control groups for every combination of EE gathered historical offline sales data every £1 spent on search
can be deployed across digital and offline media and channel). for the whole UK. By assessing sales advertising, providing a Test and Learn
channels. They can be particularly helpful distribution and fluctuation over time key benchmark for future
in testing the interplay between digital • Limited cross channel applications – they were able to divide the nation into 24 marketing investment.
and offline. For example, geo-testing in controlled experiments combining online regions: 12 as tests, 12 as controls.
a controlled experiment design can be and offline channels remain challenging to
used to test the impact of online spend on conduct efficiently. Geo-testing (mapping
high-street sales. However, it is important online ad spend to offline sales activity
to recognise that ensuring a robust and across test and control regions) offers one
sufficiently large control and test group route, though scaling this to multiple media
Offline sales per 3

Offline Sales Income - £s

across multiple channels can quickly channels (e.g. TV, digital) quickly becomes
become very resource intensive. impractical when attempting to assign and £ of digital media
maintain a control and test group. 2
advertising spend
1

0
0 1 2 3

Digital Media Spend - £s

14
 Lastly, it is important to recognise two major
trends that are impacting the shape and
application of measurement and effectiveness
tools – machine learning and data privacy.

Machine learning algorithms ingest large

disaggregated data sets and identify patterns.
For example, you could feed in a dataset for
a known KPI (e.g. conversion uplifts from
campaign types) and train an algorithm to
predict how the KPI would perform based
purely on its input features.

This is a fast-moving space with live

applications across all the methodologies
discussed in this toolkit. However, the same
measurement principles apply, particularly
around understanding the scope and quality

Emerging Trends
of your data. Without large, high quality
datasets across your key KPIs and channels,
machine learning is a redundant tool.

In contrast to utilising increasingly large

datasets, is the rising awareness and
regulation of consumer digital privacy.

The full implications of GDPR remain to be

seen but future measurement frameworks
will have to respect and navigate the resulting
constraints on the granularity and coverage
of consumer data across channels.

15
Recommendations Summary
and Takeaways of Key Brand Studies Econometrics
/ MMM
Attribution Controlled
Experiments
Measurement
Cross-media measurement is not easy. Increasing Tools Overview • Measures how marketing
activities and channels
• Estimates the effect
of marketing mix on
• Assigns credit to
touchpoints across
• Quantifies impact
of change of media
channel complexity, competing and missing data impact consumer incremental sales. channels based on their between randomly
attitudes and brand contribution to an action. assigned test and
and rapidly evolving tools can make measurement perceptions. control groups.
and effectiveness feel like an insurmountable task.

Context for Use • Flexible survey format for • Long term evaluation • Measuring granular user • Stress testing key customer
But that is far from the truth. Whilst there’s no magic
diverse assessment of of historic aggregate level (particularly digital) touchpoints that have
formula, this toolkit aims to provide both practical brand building activities. data (usually weekly) tactical actions. sufficient volume.
tools and an understanding of how to improve cross- • Providing a “pulse- to assess past influences • Enables rapid scheduling • “Gold standard” approach
check” on short-term and predict future sales. and creative alterations. to target across all
media measurement efforts from today.
campaign activity. • Optimising high-level measurement activity.
channel mix.
Below is a pull-out summary of the four tools covered
that recaps the key components of each (see Figure 9).
Methodology • Survey selection bias • Requires 2yrs+ granular • Risk of over-bias towards • Requires clear testable
and sample sizes. data (e.g. weekly sales). digital advertising. hypotheses.
Figure 9.
Considerations
• Challenge of accurately • Struggles to capture • Challenge of building single • Requires large sample
Summary of key measurement tools segmenting audiences amplifier effect of digital. customer view (e.g. lack of sizes or significance.
by media channel. • Media spend timing has to universal IDs across major • Challenge of achieving
• Limited power as vary (avoid collinearity). internet platforms). scale and accurate
a direct input to • More KPIs = more models. group assignment
Create a Measurement Strategy financial planning. across-media.

This section provides three key templates to help • To what extent are • To what extent are non- • What is the length and • What are the volume of
Sector
set your measurement strategy on your next customers exposed media tools (discounting) complexity of customer interactions (particularly
Considerations to the brand e.g. relied upon for sales journey e.g. auto sales) by channel
campaign. They can be adapted to specific business
consumer retail (high) e.g. FMCG (high) vs (high) vs transactional e.g. FMCG (high) vs
requirements. Work through each of these, together vs manufacturing (low)? healthcare (low)? entertainment (low)? luxury goods (low)?
with the checklist on page 20.

Step 1. Set the right foundations

Expected • Qualitative analysis • Single KPI prediction – • Robust quantitative • Most robust
Step 2. Organise your data Outputs explaining drivers of
brand attitudes and
e.g. average effect on sales. models of channel
conversion rates.
quantification of
incrementality from
• Quantitative explanation
Step 3. Develop a unified framework perceptions. of supporting influences. • Quantitative explanation marketing activity.
• Quantification of change of best performing
(“lift”) in brand metrics customer journeys.
by channel.

16
 Define Campaign Set KPIs to Reflect Organise Data
Objectives Objectives Requirements

Key
Principles for What is my
primary objective?
Do my KPIs relate directly
to my objective?
What are the key categories of
my marketing spend (particularly
Measurement How will my objective feed into How is the KPI calculated
within digital)?

Success my long and short term goals? from the method I have
chosen (e.g.ROI)?
Where do I have gaps in my
marketing channel data coverage?
Checklist What do I need to measure across
channels to address the objective? How will I calibrate KPIs Where can I improve the
from different timeframes, granularity and coverage
channels and tools? of my data?

Integrate Multiple Tools Test and Learn

What are the limitations Have I set expectations

of my chosen tool and that measurement is
methodology? a continuous process?

What other tools could What is the action(s)

provide further insight? or decision I will take
based on the result?
Who else in my organisation
could be working on a What is the plan to report
similar type of problem? results and how will that
feed into the next
marketing activity?
17
 Channel Estimated Source Trackable ID Awareness Action Purchase
in use? Spend of data out of channel (e.g. impressions) (e.g. Clicks) (e.g. target sales
destination)

Organise Digital Search

your Data
Display

Social Media

Online Video

Owned
list of channels is Properties
non-exhaustive Email

Traditional TV

Radio

Outdoor

Print

Other

Device Mobile
Specific (e.g. in-app)
Activity
Tablet

Other
(e.g. smart
18 speakers
Develop
a Unified Timeframe
Framework

Long Term
months Econometrics /
to years Marketing Mix Modelling
(MMM)
Tools can be deployed across
the customer lifecycle and for
Brand Brand
varied timeframes but offer
Studies Studies
Mid Term
the most actionable insights weeks to
at mapped touchpoints. months Controlled Experiments

Near Term
days
Attribution

Awareness Action Purchase Loyalty

Customer Lifecycle

19
Top Tips Don’t operate in silos - understand the Create a measurement strategy – create a clear,
measurement activities already happening in your consistent plan for which tools you are selecting
Checklist organisation (as well as external companies) and why (use ‘develop a unified framework’ on
Plan measurement early - set clear objectives page 19 to help with this)
and KPIs upfront and avoid retrofitting (use ‘set Build benchmarks of key metrics internally
the right foundations’ on page 5 to help with this) – set targets that reflect the context of other
Think carefully about choosing metrics - don’t just measurement activities in your organisation
rely on what is most accessible (e.g. click through Review your measurement activity periodically
rates) and include long term as well as short term – What were the gaps and limitations?
metrics (use ‘organise your data’ on page 18 to Use results as continuous feedback to
help with this) inform future marketing activity.
Take a holistic approach - don’t look at one
channel in isolation and integrate multiple tools.
Assess the campaign as a whole and review
the role of each channel (e.g. strengths and
weaknesses) within this framework (use ‘develop
a unified framework’ on page 19 to help with this)

Visit www.iabuk.com/measurement for more information on the IAB’s latest work.

20