Scribd
Upload
2 views

2sce5r6oetjh-TestingStudyGuide

The document serves as a study guide for A/B testing, outlining its purpose in identifying friction points in the customer journey and emphasizing the importance of a clear KPI. It details a six-step process for conducting A/B tests, including establishing objectives, analyzing results, and maintaining best practices. Additionally, it highlights the significance of testing variables that impact customer behavior and the need for consistent organization and long-term tracking of test results.

Uploaded by

charlene.rosella
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
2 views

2sce5r6oetjh-TestingStudyGuide

The document serves as a study guide for A/B testing, outlining its purpose in identifying friction points in the customer journey and emphasizing the importance of a clear KPI. It details a six-step process for conducting A/B tests, including establishing objectives, analyzing results, and maintaining best practices. Additionally, it highlights the significance of testing variables that impact customer behavior and the need for consistent organization and long-term tracking of test results.

Uploaded by

charlene.rosella
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

STUDY GUIDE

TESTING

Introduction to A/B Testing


● A/B testing can be used to identify surface points of friction, or areas where you may lose a customer,
across each touchpoint in the customer journey.
● Tests don’t need to be complex to provide valuable insights. Even a simple test can have major impacts.
○ Common variables to test include:
■ Copy
■ CTA placement
■ Pricing strategy
■ Images
■ Design and layout
■ The Six Steps of A/B testing
● Step 1: Establish a Key Performance Indicator, or KPI
○ Prioritize and select only one single KPI for each A/B test
○ Decide which metric is most important to you and then specify what change you want to see over
a set period of time
● Step 2: Decide What You Want to Learn
○ Choose a single question that you want to answer and make sure it’s clear, concise, and specific
● Step 3: Look at the Bigger Picture
○ Before you set up your test, you’ll need to answer two important questions:
■ What data do I already have available to me?
● Check for results from tests that have recently been run or are currently running
and consider how these might affect your own test
■ What constraints may limit the kind of test I want to run?
● Step 4: Write Out Your Testing Method
○ A well-crafted testing method should include:
■ The name of your test, something like “2022-11-10-XYZ Insurance- Landing Page Apply
For Quote Conversion A/B Test”
■ Your KPI
■ The question you’re attempting to answer
■ Your hypothesis
■ The amount of data you need to collect to make sure your results are statistically
significant
■ Documentation method ensures the outcomes of your A/B test are recorded and archived
for future reference
● Step 5: QA & Launch
○ Before you run your test, it’s critical that you quality-assure, or QA, your setup
■ Check that the testing software will work as expected and confirm
■ Make it live by serving your audience their respective versions of the test and then
continue to monitor it regularly
● Step 6: Analyze the Results
○ Compile the data and analyze the results
○ Document everything
■ Report what was tested and whether an event outside of your test may have
contaminated the results
■ Include your recommendations

Choosing a Test Metric


● The variable you test should always be informed by the following question: “What impacts customer
behavior?”
● If the tests you’ve been running haven’t been providing actionable insights, or any insights at all, it may
be time to consider re-evaluating your metric.

What Should You Test?


● The 40-40-20 Rule
○ 40 percent of the success of a campaign can be attributed to the targeting (i.e., the audience)
○ 40 percent can be attributed to the offer (i.e., what’s being promoted and how it’s worded)
○ 20 percent can be attributed to the content (i.e., the visual elements)

Best Testing Practices


● Have a Consistent Control
○ A/B tests typically involve testing an existing approach (the A test) against a new approach (the B
test).
● Be Organized
○ Whether your data set is documented in a spreadsheet or other type of software, keeping it
organized will help you and others compare it with other data sets more easily.
● Go for the Long Game
○ Track your tests over time. By doing so you can catch long-term trends, identify outliers that may
warrant further investigation, and provide benchmarks to compare against future data.
● Test Frequently
○ Organizations that run tests regularly are better positioned to leverage data in order to achieve
their objectives more quickly.
● Consider Environmental and Seasonal Influences
○ Test results may be heavily impacted by day of the week, time of the year (holidays, etc.), and
whether your visitors are new or returning. This is why it’s important to run tests for a long
enough period that these trends can be accounted for.

You might also like