Conversion Optimization Framework to Build Sustainable and Repeat Growth

ADVANT Multipurpose Presentation 2017
CONVERSION OPTIMIZATION
FRAMEWORK

Index
1.Conversion Optimization Process
2.Why Conversion Optimization & How?
3.Targeting Options
4.Targeting Methodology
5.Conversion Optimization Framework
i. Conversion Research
ii. Hypothesis Idea
iii. Architect idea
iv. Measuring A/B test
v. Global Rollout
vi. Sharing Results with wider groups

Tushar Purohit
Conversion Specialist
Hi, I’m Tushar. I co-founded
Growthvalleys.com
As Innovation happens at the edges & at the
intersections of product/marketing. I live at
the intersection.
On this premise, I founded Growthvalleys in
2016, with the aim of building and executing
conversion optimization strategies, creating
better user journeys and online experiences
using emotional targeting, qualitative &
Quant ITive analysis
I’ve over 10+ years of experience in AB-
testing from companies like
PremierFarnell.com, Infibeam.com and
Capillary Tech where creating high volume
conversion optimization teams.

Conversion Optimization Process
Growthvalleys.com @tushargp
Optimization is the ongoing, data driven process of continually discovering and
quantifying the most effective experience for your customers.
Testing and testing platforms coupled with processes, enable organizations to adopt
a culture of optimization.

Why Conversion Optimization & How?
• Checkout
• Homepages
• Brand pages
• Product Detail Page
• Shopping Cart
• Registration Pages
• My Account
• Brand Pages
WHERE ? PERSONALIZATIONWHY:-
• Improve Visitor
Experience
• Optimize the layout
• Experiment
• Identify Opportunity Areas
• Reduce Bounce / Exit
Rates
• Define right content
• Personas targeting
• Personalized Homepage
• High Value customers
• Abandoned cart basket
• Target users interested in
specific SKU’s or categories
• Target users based on referral
channel

Targeting Options
Geo
Country
State
City
Zip Code
Vestibulum congue tempus
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod
tempor.
Network
ISP
Domain Name
Connection Speed
tempor.
Mobile
Device Name
Device Model
Device vendor
in Mobile Device
tempor.
OS
Linux
Macintosh
Windows
Visitor
New Visitor
Returning Visitor
In other test
First Page of Session
Not in other test
tempor.
Browser
Type
Language
Version
tempor.
Site Pages
Current Page
Previous Page
Landing Page
HTTP Header
tempor.
Traffic Source
Baidu
Google
Facebook
Youtube
Time Frame
Flexibility to choose a time range and or apply week and day parting
Start Date End Date

Targeting Methodology
Customer/prospect
New/return visitor
Previous visit patterns
Previous Product interests
Searches
Previous online purchases
Previous Campaign exposure
Previous Campaign responses
Site Behaviour Variables
IP address
Country of origin
Time zone
Operating system
Browser type
Screen resolution
Environment Variables
Temporal Variables
Time of day
Day of week
Recency
Frequency
Referring domain
Campaign ID
Affiliate
PPC
Natural search
Direct/bookmark
Referrer Variables

Why We Need Framework
There is a problem with conversion rate optimization: It looks easy. Most of us
with some experience working online can take a look at a website and quickly
find problems that may prevent someone from converting into a customer.
There are a MANY such problems that are quite common.

Why We Need Framework (continue..)
A/B testing can impact major customer journey, therefor it is important to have the framework for
the following reasons :-
• To protect the overall customer experience
• Need to test/measure everything
• Testing can take time
• Not every test is a winner – Failing fast is a good thing
• There is no fail or pass but it’s a process to move temporary tests onto your website product
plan as its a continuous process of data collection
• Analytics should be fully integrated into the process
• Need lots of ideas – based on research and data
• Process should be developed and integrated in a agile & quick, and also balances in place
• Customers hate popups
• What are the rules for certain design patterns? Overlays; Toaster; Alerts etc
• Define espots and pages that marketing can update
• Promotions may conflict and overlap
• Ensure all ideas are well thought through with ROI

Conversion Optimization Framework to Build Sustainable and Repeat Growth

Step 1
CONVERSION RESEARCH

Conversion Research
In the next steps we’ll look at each step individually (first 2 process + big picture)
Conversion research is a strategic approach focused on identifying and
interpreting Quantitative/Qualitative data to find possible points of friction in a sales
funnel – and ultimately, to increase the overall conversion rate.
Primarily there are 2 steps of data gathering and analysis, followed by creating a master sheet of all found issues
that then turn into action items.

Qualitative Research – Listen your Customers
What do users like or dislike about your web site, mobile site, app? How easy
is it for users to perform core tasks? Would they recommend your site to
others? What are their recommendations for improving it?
Get survey responses, information architecture research, clickstream,
heatmaps from your users and keep your finger on the pulse of your audience.

Methodologies for Qualitative Research
• Surveys:- Helps companies to listen voice of customers and make informed
objective decisions based on what you know your users want.
• Information Architecture:- Leverage different research methods within the same
study for greater confidence in taxonomy like navigation structure
• Heatmaps :- Helps you to understand what users want, care about and interact with
on your site by visually representing their clicks, taps and scrolling behavior.

Surveys
Run better surveys by :-
• Logic, Conditions & Piping :- Allow you to screen
participants, show or hide questions, and dynamically carry
responses through to subsequent questions, based on
their responses.
• Live Intercepts :- Create targeted intercepts to get the
right people, at the right time, for projects or for your
private panel.
• User Feedback Monitoring for CX & VOC Programs
Continuously capture insights and track behavior to provide
the most valuable experience for your users.
Use NPS, ease of use, brand perception and other survey types
to add attitudinal data to your behavioral insights to monitor
overall user and customer satisfaction.

Information Architecture Research
Information architecture:- help you test your navigation and
validate that taxonomy is intuitive in the following way
• Design your IA
Run open and closed card sort studies for your users to
intuitively group items together so you can optimize your site
for findability.
• Test Your Navigation
Minimize risk and validate your IA by running screenshot
click tests, timeout tests and tree tests
• Combine Multiple Methods
Leverage different research methods within the same study
for greater confidence in your taxonomy
View and analyze what path participants are taking through your site to
complete their tasks and what percentage of them are successful, where
they complete, and where they abandon.

Heatmaps & UX Metrics
Heatmaps
See exactly how your users are behaving
and where they’re clicking on a page.
Analyst firms report as much as nine-fold
revenue improvements from the
application of behavioral data.
UX Metrics
Don’t guess – measure, but
gather quantitative UX metrics such
as success rates, time on task,
page views and number of clicks
with UX metrics

Centralization all Feedback & Findings
UX Feedback Issue
Issue Insight/Problem Status How can we fix it Latest update
Homepage
Natural Search
Traffic (SEO)
Richer dypages deemed more relevant
by search engines. The more
customized dynamic pages built, the
greater the impact.
Reviewed
We need to review how customers can
add new shipping / billing addresses
independantly
Redesigning checkout!
investigate options to manage this
situation if the 20 min period cannot be
extended
can we make this stand out more on
the cart?
Sitewide
Checkout
Search
PDP
PDP
Search
Login / Register
PDP
Checkout
Search
Paid Search Traffic
(SEM)
More mid-tail traffic will come from
natural search, reducing the amount of
clicks / traffic from paid search terms
Under Progress
independantly
extended
the cart?
Sitewide
Checkout
Search
PDP
PDP
Search
Login / Register
PDP
Checkout
Search
Shopping Cart
Add to shopping
cart rate
Merchants can build customized
dynamic pages to boost conversion on
lowest performing areas of site, or
highest value areas. Merchants can
also introduce richer content such as
reviews, customer profile information,
videos, buyer guides, etc. through pre-
built cartridges and CAS integration,
which increases conversion rate. Size
of impact relative to number of
customized pages
Cant test
independantly
extended
the cart?
Sitewide
Checkout
Search
PDP
PDP
Search
Login / Register
PDP
Checkout
Search
Cart to purchase
rate / increase
orders
Merchants can introduce content that
builds confidence, such as buyer
guides, expert reviews, or peer
reviews. For example, if someone was
able to isolate a product that is the top
selling and top rates among "people
like me", they are more likely to feel
confident that it's the right one, they
don't need to keep looking, and make
the purchase.
Reviewed
We need to review how customers
can add new shipping / billing
addresses independantly
situation if the 20 min period cannot
be extended
the cart?
Search
Average order size
(AOV)
Merchants can use Experience
Manager to deploy dynamic
merchandising zones to those areas
with good cross sell and up sell
opportunities. Merchants can also
introduce review and expert opinions
that help justify spending more on
certain brands and items.
Under process
Sitewide
Checkout
Search
PDP
PDP
Search
Login / Register
PDP
Checkout
Search
Product detail pages
Marketing capacity
Business user control and faster development of
websites and new pages
Centralize all your feedback in one place in order to
respond them quicker and efficient way:-
Give each feedback as priority score :-
• So that the most important issues bubble to the top.
• Priority is based on Frequency Raised and the impact
to the customer experience.
Launch Customer usability sessions
Dedicated session to understand customers in
focused groups
Launch Monthly session
Monthly session to be set up to review feedback with all
key stakeholder

Data & Analytics
You have a strategy? Great, measure it. Build a your site and then
complete picture. Gain deeper insights into how users from your Ads
campaigns engage with identify leaks to create custom site experiences
using the audiences you’ve already created in Analytics.

Metrics to consider for ?
• Where are we leaking sales
• Conversion funnels
• Deep dives – Laser focus on topics/trends
• Segmentation – Slicing the data
• Check the Funnel Performance
• Conversions per browser version
• Check conversions and bounce rate per device
• Identify high traffic & high bounce / high exit rate
pages
• High traffic/High bounce/Low Conversion Landing
• Which screen resolution boosts bounce rates?
• High traffic / Low speed Pages
• Check user flows
• Analyze new vs returning
Analytics Funnel Reports
Overall Website Revenue
FY17 FY18 Cumulative
$2,772,132 $4,631,676 $7,587,500
Details
Traffic to Site
Visits from Natural Search
24,159 48,318 72,477
On Site Conversion
Visits from Shopping Sessions
16,723 31,600 44,626
Overall Orders
4,905 9,524 13,855
Average Order Value
$0.68 $1.72 $4.30
Revenue Impact
$1,645,074 $3,495,541 $6,437,749
Supplier Advertising
Revenue Impact
$1,122,520 $1,622,520 $3,140,000
Paid Search Cost Reduction
Next Steps is to identify the best practices for Idea Generation

Best Practices for Idea Generation
• Browse to the page as a customer:- Ask the
below questions yourself and you will get the ideas.
• Content / Message - Am I seeing what I wanted to
see in the first 10 seconds?
• Information / Additional Specs - Are the
supporting details enough to let me decide on the
product / services?
• CTA - Is there any button which allows me to move
ahead for further details?
• Design – Is the website design / motion banners
appealing to my eyes to act?
• Time – Am I spending too much time to do what I
wanted (form fill, buy, login)?
• Trust – Does the page have enough information to
trust the site?
• Reach – Is there any contact us information
available on the page?
• Standards – Are there standard links like ‘About
Us’, ‘Social Media’, ‘Terms’ on the page?
• Headlines – Simple, Clear and Action Driven
• CTAs and Navigation Bars – Prominent Buttons,
Breadcrumbs, Menu Items.
• Search Functionalist and Placement – Always on
the top right section of the page
• Action driven content (Know More, View Details) –
Cut short the content and give CTAs
• Easy Access to Login / Cart/ Forms / Downloads –
One click to access the collaterals
• Layout based on Page Type – Standard templates
for Home, Product Detailed, Cart and Payment Pages
• Everything (almost) above the fold – Main CTA
should be accessible above the fold
• Balanced use of white space and colors – Optimal
use of white space and background colors to CTA
buttons

Step 2
PRIORTIZE & HYPOTHESIS

Step 2 :- Hypothesis Idea & Prioritization

Hypothesis Idea
A hypothesis is a prediction you create prior to running an
experiment. It states clearly what is being changed, what you believe
the outcome will be, and why you think that’s the case.
Running the experiment will either prove or disprove your
hypothesis.

Hypothesis Idea
• Identify goals and key metrics
• Create Hypothesis
• Estimate test duration with a
sample size
• Prioritize experiments with
projected ROI
• QA the experiment
• Set test live
• Record and share results
Company
Metric
Business
Unit
Optimization
Goals
Experiment
Goals
4. Company Metric-
Optimizing full customer lifecycle
value
3.Business Unit –
Strategies for enhancing
average order value
2.Optimization Goal –
Increase revenue engagement
per visitor
1. Experiment – Increase
Engagement on site (for
example with Community)

Prioritization
Growthvalleys.com @Tushargp
• A strong prioritization process arms a conversion optimization
research with the answer: “here’s what we’ll test, in what order, and
here’s why.”
• How big a difference:- that we can expect to see with the
proposed change compared to the status quo?
• Product :- Different areas of your product perform?
• Funnel :- How each part of your funnel converts which will
helps you decide of an effect you’d need to see for the new
change to be worth it.
• Technical :- How much development work is required to
graduate the test?
• How strategically important is it?
• Does this feature support future plans?
• What is the size of audience or action are we optimizing
for?
• Consider a retest?

Uplift Modelling
Uplift modeling, also known as incremental modeling, true lift modeling,
or net-lift modeling is a predictive modeling technique that directly models
the incremental impact of a treatment (such as a direct marketing action)
on an individual’s behavior.

Uplift Modelling
Once we have estimated an effect size for our test and
know the statistical standards we are going to use in
planning :-
• Power Calculation:- We can do a power calculation
to find out how big of a sample size we need for our
test. The point of power calculations like these is to
find out what sample size we need for our A/B test.
• Statistical Confidence :- How many views or users
or form submissions or other interactions we need in
each group to achieve the necessary power for our
test.
Then we can finally start our test! Time to wait for those
events to roll in.

Uplift Modelling Example
Below is the example demonstrating the uplift calculation.
We have taken comparison features analysis that allows users to compare and buy the product.
The metrics that we have considered while calculating the KPI is for:-
• Penetration on comparison feature
(Percentage of visits this feature have compared to the Overall visit)
• Monthly visit (Aggregated average based on past 6 months)
• Average conversion rate (based on last 6 months)
• Estimated improved CR (based on forecast)
• Estimated improved Orders (Improved CR * No. of visit)
• Incremental Improved Revenue (Incremented order- Actual Orders * AOV)

Idea list and scoring
• On the basis of test idea
hypothesis, uplift modelling and
scoring model –we built a scoring
model based on the impact on
each of its one.
• Also each item goes through a
scoring approach to understand
the potential impact of executing
this activity.
• Further it will allows you to ‘edge
case’ ideas and instead focus on
high potential activities backed by
data.
• Develop dedicated meetings
each month with Key stakeholders
to review all of these ideas

Step 3
ARCHITECT, DESIGN & BUILD

Step 3 :- Architect, Design & Build Idea

Architect design and build idea
•Customer Journey Mapping :- The concept behind journey mapping is to apply maximum data insights.
•Customer journey analysis :- Each step of customer with a process flow which helps to understand the
visualization in a better way.
•Functional document :- A functional document is nothing but a process document which defines what is
the problem statement,
Since conversion optimization is a continuous process of data collection
and learn more about the customers, that’s the reason the success of ab test
activity should not be measure based on pass or fail. But to architect the test
in such a way that each step of customer journey should be optimized

Customer Journey Mapping
• Successful journey mapping means mastering the
moment, and maintaining the attention of your
customers at every step in their buying journey.
• “With journey mapping and natural language
processing, automated systems can evaluate the
factors contributing to whether people are getting
through the process you outlined for them and then
determine the reasons why people are not continuing
on with that journey,”
• But before you can start developing an effective
customer journey map based on insights from your
data, you have to be able to access your data in an
integrated fashion.
Information Architecture
Site Maps
eSpot mapping
Naming conventions
Form design
Journey mapping
UI
Style Guides
Pattern Libraries
Page Templates/Wireframes
Prototypes
CSS
Regular sessions set up UX and Content
teams to align on UI guidelines.

Customer Journey Map of Community
Signup (Join the
community)
Signup with
credentials
1St interaction
Point
Profile Field*
Blue Box-
Existing
Red Box-
Proposed
Signup (Join the
community)
Signup with
credentials
1St interaction
Point
Profile Field*
Profile Field*
Signup (Join the
community)
Signup with
credentials
1St interaction
Point
Profile Field*
Profile Field*
Signup (Join the
community)
Signup with
credentials
1St interaction
Point
Profile Field*
Profile Field*
Signup (Join the
community)
Signup with
credentials
1St interaction
Point
Profile Field*
Profile Field*
Signup (Join the
community)
Signup with
credentials
1St interaction
Point
Profile Field*
Profile Field*

Conversion Research
Do start with critical
problems you want to solve
for your customers, mapped
to the measureable goals
that define success
Do not sneak solution into
your goal or problem
statement
• Customer service data
• Video sessions recording
• Drop off analysis
• Web analytics
• Customer services
feedback
• Customer survey data
• And more
• Effort to test
• Potential impact
• Likelihood belief is
true
• Ability to adopt
• UX
• Product
• Merchandizing
• Marketing
Goals & Problems
Establish
Hypothesis
Generation
Hypothesis
Prioritization
Investment

Step 4
MEASURE A/B TEST

Measuring AB Test
You have an A/B testing tool, a well-researched hypothesis and a
winning test with 95% confidence. The next step is to declare the winner
and push it live, right? Not so fast. There are factors which you need to
consider in order validate your test and if you not put focus on these are
there is a risk making decision based on the bad data…
Factors needs to consider while evaluating A/B test results
• Significance level
• Validity threat
• Data segmentation

Significance level
What percentage of the time users are willing to be fooled into seeing an
effect by random chance – this is called significance level (& more
precisely null hypothesis)
Here are a the great significance calculator tools you can use
• Use this A/B Significance Test :- To compare two ads or landing
pages and determine which performed better based on conversion
rate,.
• This PCC Ad Testing Tool allows you to input more variables for a
more concise analysis of click-through rate and conversion rate.
• If you want to stick to Excel, I recommend checking out
this video, which gives you more data surrounding I2C metrics for
your campaigns!

Post-Test Segmentation
Post-test segmentation is vital to gain insights and maximizing revenue
because statistical significance will only tell us the difference between
variations and control but not with the users, as users are different and
what resonates with one person, doesn’t work with another one.
Types of Segmentation Strategy you need to consider
• ‘What segments have true value to our business?’ (Returning visitor or new users)
• ‘What segments are actionable?’ (Desktop v/s Mobile)
• ‘Which are most likely to be impacted by the test we are running?’ (Category X v/s Category
Y)
• ‘What type of correction must I do to account for multiple observations of the treatment?’”
(chrome v/s IE)

Validity Threat
• Always integrate your testing tool with Analytics to see if the revenue numbers (mostly) match up.
• Always conduct quality assurance for every device and every operating system.
• Run tests separately for each type of device.
• Stop your test only when you have sampled the correct sample size.
• Take data fishing into account and adjust your significance level to compensate for the number of
tests you’re running.
• Run tests for as long as necessary (i.e. until your sample size is reached) and not a moment longer.
• When testing, use a representative sample and don’t acquire traffic from “unusual” sources.
• Run tests for full week increments so that you include data from every day of the week and every
time of day.
• If you run a test during a holiday season, know that the data you collect is only relevant to that
season.
• Look at your annual data and identify anomalies (in traffic and conversions). Account for this when
running your tests.
• Be aware of pop culture and the media. Document major news stories as they happen.
There are factors threatening the validity of your test (internal validity, external
validity, and ecological validity.)

Step 4
PLANNED ROLLOUT
& SCALE

Rollout Strategy
A large scale successful ab test global rollout strategy for a website can be
daunting, but the real payoff putting the effort for successful rollout is the one
that should engages and serves your customers better, has room to grow,
and holds the power to streamline your processes.
The payoff of getting goodly is not only a successful rollout, but a successful website too :-
OWN YOUR ROLLOUT STRATEGY :- by making the site with zero content mistakes, solving technical issues
and be more efficient.
COLLABORATE WITH LOCAL SITE TEAMS:- Mitigate the risk of overruns and
additional costs by appointing a central project manager on your side, who will be the
central point of contact for the whole rollout – running regular calls with the vendor and
local country teams in order to make sure vital deadlines are met and any issues are dealt
with..

Step 6
SHARE RESULTS

Sharing Results with Wider Groups
Analytics has always been a challenge for most digital marketers. It can
be confusing, overwhelming and, quite frankly, difficult for the ordinary
human to understand and decipher. In most of the cases, marketers don’t
even report back their clients’ analytics because they’re just not sure
where to start — partly because of information overload and difficult-to-
understand data.
Here are few example which helps you to get started sharing results with Wider Groups
• ‘Getting a complete view:- ‘What are the results that seems to be impacted with the test we perform?’ (Internal
Search X v/s Internal Search Y)
• ROI Framework Overview :- What type of KPI’s achieved apart from the revenue (Incremental uplift for
customers already using Internal Search )
• Data visualization :- The representation of information in the form of a chart, diagram, picture, etc.

Getting a Complete View
Incremental uplift for
customers already using
Internal Search
Incremental
uplift of 1%
This is the example of data visualization demonstrating Incremental Uplift by Enhancing Internal Search
Experience

Rationale
SEO + Experience Manager
• Expanded site footprint
• Site map generation
• URL beautification
• Richer landing pages in long-tail
Rationale
• Increasing site stickiness and engagement
• Spotlighting related content in user context
• Boosting confidence with relevant content and data
Rationale
• Quicker time-to-market for advertising offers / landing pages
• Better monetize search experience (ie boosting featured brands)
Rationale Mid-tail keywords boosted with SEO and Experience Manager richer pages and URL
beautification – less paid clicks
ROI Framework Overview

Summary: 1 and 3 Year Revenue Impact
ROI Year 1 – ALL REGIONS
Revenue Impact Summary
Conservative Pragmatic Aggressive
$2,772,132 $4,631,676 $7,587,500
Details
Traffic to Site
Add'l Visits from Natural Search
Conservative IMPACT Aggressive
24,159 48,318 72,477
On Site Conversion
Add'l Shopping Sessions
16,723 31,600 44,626
Add'l Orders
4,905 9,524 13,855
Add'l AOV
$0.68 $1.72 $4.30
Revenue Impact
$1,645,074 $3,495,541 $6,437,749
Supplier Advertising
Revenue Impact
$1,122,520 $1,622,520 $3,140,000
Paid Search Cost Reduction
Add'l Paid Search Clicks
9,175 27,526 55,053
Cost Savings
$4,539 $13,616 $27,231
YOY Incremental Impact prediction after enhancing internal
search results experience.

Data Visualization (Summary of the above data)
www.presentatiostemplate.com
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Conversion Engagement Performance Summary
Increase conversion
on Product detail
page by 45%
Engaging content with
over 23.8% more time
spent by users
Global Sales uplift by
1.5%
ROI Summary
45% 24% 1.5% 98%

About Growthavlleys
Growth Driven
Conversion Optimization
Optimization through research, fixing leaks in funnel &
Increase conversions with a meaningful conversion optimization process.
Get in touch

Healthcare IT Projects
B2B conversion
Optimization
B2B Business
Growth Hacking/CRO
Projects
E-Commerce Project
Growthavlleys Worldwide Footprints

Thank You

Conversion Optimization Framework to Build Sustainable and Repeat Growth

Recommended

More Related Content

What's hot (20)

Similar to Conversion Optimization Framework to Build Sustainable and Repeat Growth (20)

Recently uploaded (20)

Conversion Optimization Framework to Build Sustainable and Repeat Growth