A Beginners Guide From The Pros

TESTING
A
Getting Started With
B
A Partner publication of:
1: Color Testing Page 3
2: Image Testing Page 7
3: Word Testing Page 11
4: Price Testing Page 15
5: Layout Testing Page 20
Table of Contents
GETTI NG STARTED WI TH A/B TESTI NG
SHARE
Uncertainty is a marketers best friend and worst enemy. It simultaneously
forces us to experiment with proven processes to drive success and instills us
with anxiety as we push the send button on that latest email campaign. A little
scary? Sometimes.
Thankfully, as marketing technologies continue to evolve, uncertainty is
gradually losing its anxiety-fueled grip on us marketers. How so? Through the
magic of A/B Testing! By simply testing a variable on marketing assets, one
can make better informed decisions to curb uncertainty and drive marketing
success.
However, A/B Testing isnt always as easy
as 1, 2, 3. It takes a little bit of experience
and know-how to execute correctly. Thats
where this ebook comes in. What follows is a
collection of A/B Tests conducted by HubSpot
partners, their results and key takeaways.
Youll learn everything you need to know to nip
uncertainty in the bud.
Introduction
Al Biedrzycki
Channel Marketer, HubSpot
@albiedrzycki
A
Chapter 1: Color Testing
B
GETTI NG STARTED WI TH A/B TESTI NG
SHARE
Color invokes feelings, which in turn sets the emotional mood
for acceptance or rejection by the viewer. Its a subjective and
culturally infuenced unconscious and involuntary response.
Colors, when used in pleasing combinations, can attract attention
and also make other elements easier or more difcult to
comprehend whether text, objects or images. Testing diferent
combinations allows you to maximize the results you want to get,
as in this following example.
Why Test Color?
GETTI NG STARTED WI TH A/B TESTI NG
SHARE
To get experience experimenting on calls-to-action to learn how
to efectively test and improve Click-Through-Rates (CTRs).
The background color of the call-to-action button for our Inbound
Marketing for Beginners ebook. We created two variations: an
orange background, and black one:
Goal
Variables Tested
GETTI NG STARTED WI TH A/B TESTI NG
SHARE
We speculated that the black background would have higher
conversions, as its appearance would be more of a contrast
against our white website pages.
The orange call-to-action had 10.9% higher
conversions than the black call-to-action.
Hyothesis
Result
Site visitor behavior is unpredictable. The only way to optimize is
with experiments!
Key Takeaway
Contributor
Jasmine Henry
Inbound Marketing
Agents
@InboundAgent
Learn more from
Inbound Marketing
Agents!
DOWNLOAD
A
Chapter 2: Image Testing
B
GETTI NG STARTED WI TH A/B TESTI NG
SHARE
Images convey more than mere text, shapes, and colors. They
help set the tone serious, playful, clinical, provocative
instantly establishing an emotional connection with the viewer.
Or setting up a barrier you may never overcome. Which is why
testing diferent images to see which ones yield the best results
can make the diference between a campaign that gets results or
a total dud. Dont just guess. Test.
Why Test Images?
GETTI NG STARTED WI TH A/B TESTI NG
SHARE
Its standard procedure to create A/B test groups for CTAs.
Because this ofer was for an Inbound Marketing Recipe Book,
our graphic designer came up with the idea to test out diferent
types of food to increase CTRs.
We tested two groups of images for a CTA candy and dessert
versus health food:
Goal
Variables Tested
GETTI NG STARTED WI TH A/B TESTI NG
SHARE
We thought the dessert CTA would get more clicks.
Our hypothesis was validated. The dessert CTA got more clicks.
However, the health food CTAs conversion rate was 5% higher
than the dessert CTAs!
Hyothesis
Result
Higher CTR doesnt always win. You need to look deeper and
analyze the more meaningful behavior to determine what really
works better for achieving your goals.
Key Takeaway
Contributor
Laura Hogan
OverGo Studio
@OverGoStudio
Learn more from
OverGo Studios!
DOWNLOAD
Chapter 3: Word Testing
B A
GETTI NG STARTED WI TH A/B TESTI NG
SHARE
Language is tricky. Shades of meaning hide around every corner.
Cultural nuances abound. Verbal fads and memes come and
go in the blink of an eye. Being sensitive to your ideal buyer
personas mindset is crucial for successful communication, as is
being clear about what you convey. Thats why testing diferent
words and phrases can be some of the most complex tests you
do in pursuit of achieving the greatest possible results.
Why Test Wording?
GETTI NG STARTED WI TH A/B TESTI NG
SHARE
We wanted to see how visitors would respond to specifc copy
on more middle to bottom-of-the-funnel CTAs. Would they be
turned of by seeing Pricing? Or would it more accurately set the
expectation and thus experience a higher CTR?
Originally when attempting to drive visitors to our pricing page, we
felt fewer words that clearly expressed what the visitor expected,
and wanted, would convert the most:
Goal
Variables Tested
We thought using the term Pricing by itself could insinuate a hard
sell to the visitor and not work as well as Plans and Pricing, which we
felt set expectations better. Our Solutions, our third test phrase we
felt would perform the worst because it was broad and unclear.
Hyothesis
GETTI NG STARTED WI TH A/B TESTI NG
SHARE
We were right. View Plans & Pricing had the highest CTR, while
Our Solutions was the lowest. Interestingly enough, See Pricing
didnt do badly, but not as well as View Plans & Pricing.
Result
Clearly set expectations. While
middle and bottom of the funnel
CTAs will generally see lower CTRs
than more top of the funnel material,
the CTR will remain consistent so
long as expectations are clearly set.
Key Takeaway
Contributor
John Bonini
Impact Branding
@ImpactBND
CTA Impressions CTR
Learn more
from Impact!
DOWNLOAD
Chapter 4: Price Testing
$
GETTI NG STARTED WI TH A/B TESTI NG
SHARE
In any ecommerce transaction, there are always key moments of
truth. Moments when the decision to buy is on the line. Factors
such as shipping rates, order placement deadlines, delivery time,
cross- and up-sells, discounts, rewards programs, bonuses, etc.
can make the diference between millions of dollars in revenue
being abandoned in shopping carts, or not. Testing such variables
is essential for increasing a companys revenue. And they often
present challenges that go well beyond just changing a few
words or images on a CTA button or landing page they can
also impact a companys fulfllment, order processing and/or
other systems. Successful A/B testing requires clear and constant
communication with your client at all times, so no one drops the
ball and no customer gets trapped in an unforeseen glitch.
Why Test Pricing?
GETTI NG STARTED WI TH A/B TESTI NG
SHARE
Our goal was to optimize conversion rates during the holiday
retail rush of 2013, without sacrifcing our clients proft margins.
After a comprehensive competitive analysis, we proposed to
split test more attractive shipping rates on orders under $150 to
measure the impact on conversion rates.
Goal
Variables Tested
GETTI NG STARTED WI TH A/B TESTI NG
SHARE
We hypothesized that a more attractive shipping rate ($5 fat rate,
$7.77 for speedier delivery) would boost checkout conversion
rates enough to overcome the lost shipping revenue, boosting
our clients bottom line.
After day 1 checkout conversion
surged 26% and promotion of the ofer
increased sales a whopping 66% - the
fourth biggest day our client ever had!
The 7 days of testing with $5 fat rate
shipping set sales records, claiming 7 of
the top 20 days theyve ever had! It also
contributed to the best revenue month in
our clients 11-year history!
Hyothesis
Result
GETTI NG STARTED WI TH A/B TESTI NG
SHARE
Contributor
Kevin Barber
Lean Labs
@Lean_Labs
Overcome hurdles to the decision to purchase and conversion
rates will soar.
Continuously test variations of your hypothesis to fnd the sweet
spot.
Monitor the conversion rate of customers who start the checkout
process, so you can more accurately test the impact of moment
of truth decision factors like shipping cost and estimated
delivery time.
Key Takeaways
Learn more
from Lean Labs!
DOWNLOAD
Chapter 5: Layout Testing
A
GETTI NG STARTED WI TH A/B TESTI NG
SHARE
The ages-old debate about whats better graphics or text
is tackled yet again in this example. No two cases are exactly
the same. Why? Because your personas, their expectations
about your brand, their experience on your web site they all
contribute to your visitors responses. Testing whether a graphics-
only layout or a text-only layout is a clean and prudent way to
gauge your visitors preferences for future CTAs that you will
create.
Why Test Layout?
GETTI NG STARTED WI TH A/B TESTI NG
SHARE
We wanted to see if a diferent layout (one with and without a
graphic) and diferent background colors would improve this
CTAs conversion rate. The copy was identical in each CTA.
We split-tested the layout of two
CTAs. Both linked to the same
landing page and included the
exact same copy. One CTA
included an ebook cover and the
other was text-only.
Goal
Variables Tested
Our hypothesis was that the CTA with a green background and
just text would perform the CTA with the ebook cover. We thought
the green background displayed the message more clearly by
avoiding the visual distraction of a graphic.
Hyothesis
GETTI NG STARTED WI TH A/B TESTI NG
SHARE
Contributor
Andrew Dymski
GuavaBox
@GuavaBox
We learned its better to include a representation of the ofer in our
CTAs and that background color matters. The text and button copy
was exactly the same, but people didnt seem to like the abrupt
change in background color.
Key Takeaway
We were dead wrong! The
CTA with the ebook cover
and a transparent background
outperformed the text on a
green background by 500%!
Result
Learn more
from GuavaBox!
DOWNLOAD
Ready to take your marketing
strategy to the next level?
Sign up for HubSpots Insiders Blog
SUBSCRIBE NOW!