SlideShare a Scribd company logo
Scaling your
Testing Program
Vivek Vilatia
Senior Product Manager, VWO
Anshul Gurbaxani
Associate Product Manager, VWO
Prem Kujur
Customer Success Manager, VWO
Ask Questions on Twitter
#AskVWO
What is A/B testing
In the most simplest form, A/B
Testing is comparing two versions
of a web page to see which version
performs better for a defined
conversion goal
Type of Tests
A/B Test
Change any element or section
of a webpage and test both
versions.
Split URL Test
Test performances of separate
URLs if the landing pages are
significantly different.
Multivariate Test
Run tests on multiple variation
combinations together. Needs
sufficient traffic.
Graduating to
Advanced tests
Simple A/B tests Advanced A/B tests
➔ Great starting point
➔ Smaller teams
➔ Easy to setup
➔ Quick wins
➔ Test changes like copy,
arrangement of elements,
CTA colors etc
➔ Test advanced use cases with
greater impact
➔ Conversion optimization program
➔ Self-sufficient team
➔ The right tool-set
➔ Management buy-in
➔ Tons of perseverance
Audience Changes MetricsPages
Key steps in setting up your tests
Test pages | Defining a set of similar pages for your test
Asterix * in the URL matches pattern would replace the variable part of the URL
Represented by * in the URL matches pattern
Audience | Complex targeting rules
Visitors logged in on
your website
Are either from India or have their
preferred currency as INR
Changes | Code editor for complex changes
Changes | Code editor for complex changes
Changes | Conditional Test Activation
Changes | Conditional Test Activation
SignUp Form Type Form
Measuring the
right metrics
Measuring the right metrics
Micro Conversions Macro Conversions
Measuring the right metrics
Never make your tests
live without proper
sanity checks
Quick previews
Cross Browser testing
Live previews
Check cookies using a plugin
Making sense of your
A/B testing reports
Making sense of your A/B testing reports
Run your test for at least a week.
Along with a primary goal, have related secondary goals.
Ensure that your primary goal represents the direct and
exclusive impact of the changes being tested.
Making sense of your A/B testing reports - Best Practices
Slice and Dice your test results to learn the trends and
behaviour of various user segments.
Testing radical changes at times might just do the trick
Identify micro-conversions and important correlative metrics
increase in a given variation, implement it.
Inconclusive Tests
Ensure that you don’t use segments with very less data.
Make sure there is no error in the test setup.
Ensure that you are not running multiple tests
simultaneously in the same part of the journey you are A/B
testing
Inconclusive Tests - Pitfalls
Thank You!
Q&A
Ad

More Related Content

What's hot (20)

Cultivating a Culture of Experimentation
Cultivating a Culture of ExperimentationCultivating a Culture of Experimentation
Cultivating a Culture of Experimentation
Optimizely
 
SplitMetrics answers burning questions on mobile A/B testing
SplitMetrics answers burning questions on mobile A/B testingSplitMetrics answers burning questions on mobile A/B testing
SplitMetrics answers burning questions on mobile A/B testing
SplitMetrics
 
How The Zebra Utilized Feature Experiments To Increase Carrier Card Engagemen...
How The Zebra Utilized Feature Experiments To Increase Carrier Card Engagemen...How The Zebra Utilized Feature Experiments To Increase Carrier Card Engagemen...
How The Zebra Utilized Feature Experiments To Increase Carrier Card Engagemen...
Optimizely
 
Making Your Hypothesis Work Harder to Inform Future Product Strategy
Making Your Hypothesis Work Harder to Inform Future Product StrategyMaking Your Hypothesis Work Harder to Inform Future Product Strategy
Making Your Hypothesis Work Harder to Inform Future Product Strategy
Optimizely
 
Optimizely & Photobox - DON'T PANIC: The No-Confusion Experimentation Startup...
Optimizely & Photobox - DON'T PANIC: The No-Confusion Experimentation Startup...Optimizely & Photobox - DON'T PANIC: The No-Confusion Experimentation Startup...
Optimizely & Photobox - DON'T PANIC: The No-Confusion Experimentation Startup...
Optimizely
 
Train-the-Trainer - Enabling your team
Train-the-Trainer - Enabling your teamTrain-the-Trainer - Enabling your team
Train-the-Trainer - Enabling your team
Optimizely
 
The Art of the Start --Success in the first 100 Days
The Art of the Start --Success in the first 100 DaysThe Art of the Start --Success in the first 100 Days
The Art of the Start --Success in the first 100 Days
Optimizely
 
Testing Your Testing Program
Testing Your Testing ProgramTesting Your Testing Program
Testing Your Testing Program
Jill Martay
 
10 Tactics for Building an Optimization Culture
10 Tactics for Building an Optimization Culture10 Tactics for Building an Optimization Culture
10 Tactics for Building an Optimization Culture
Optimizely
 
Apply A/B Testing with NGINX Routing Policy
Apply A/B Testing with NGINX Routing PolicyApply A/B Testing with NGINX Routing Policy
Apply A/B Testing with NGINX Routing Policy
Supachai Jaturaprom
 
The Science of Getting Testing Right
The Science of Getting Testing RightThe Science of Getting Testing Right
The Science of Getting Testing Right
Optimizely
 
Maximizing Optimizely Website Performance
Maximizing Optimizely Website PerformanceMaximizing Optimizely Website Performance
Maximizing Optimizely Website Performance
Optimizely
 
Opticon 2017 Pushing the Boundaries of Experimentation
Opticon 2017 Pushing the Boundaries of ExperimentationOpticon 2017 Pushing the Boundaries of Experimentation
Opticon 2017 Pushing the Boundaries of Experimentation
Optimizely
 
Lean Software Development: Validated Learning
Lean Software Development: Validated LearningLean Software Development: Validated Learning
Lean Software Development: Validated Learning
Òscar Vilaplana
 
The Finishing Line
The Finishing LineThe Finishing Line
The Finishing Line
Oban International
 
Opticon 2017 Advanced Program Management
Opticon 2017 Advanced Program ManagementOpticon 2017 Advanced Program Management
Opticon 2017 Advanced Program Management
Optimizely
 
Optimizely, HEMA & Farfetch - Unlock the Potential of Digital Experimentation...
Optimizely, HEMA & Farfetch - Unlock the Potential of Digital Experimentation...Optimizely, HEMA & Farfetch - Unlock the Potential of Digital Experimentation...
Optimizely, HEMA & Farfetch - Unlock the Potential of Digital Experimentation...
Optimizely
 
Introducing Program Management
Introducing Program ManagementIntroducing Program Management
Introducing Program Management
Optimizely
 
Test Everything: TrustRadius Delivers Customer Value with Experimentation
Test Everything: TrustRadius Delivers Customer Value with ExperimentationTest Everything: TrustRadius Delivers Customer Value with Experimentation
Test Everything: TrustRadius Delivers Customer Value with Experimentation
Optimizely
 
Create Successful Campaign with AB Testing
Create Successful Campaign with AB Testing Create Successful Campaign with AB Testing
Create Successful Campaign with AB Testing
NotifyVisitors
 
Cultivating a Culture of Experimentation
Cultivating a Culture of ExperimentationCultivating a Culture of Experimentation
Cultivating a Culture of Experimentation
Optimizely
 
SplitMetrics answers burning questions on mobile A/B testing
SplitMetrics answers burning questions on mobile A/B testingSplitMetrics answers burning questions on mobile A/B testing
SplitMetrics answers burning questions on mobile A/B testing
SplitMetrics
 
How The Zebra Utilized Feature Experiments To Increase Carrier Card Engagemen...
How The Zebra Utilized Feature Experiments To Increase Carrier Card Engagemen...How The Zebra Utilized Feature Experiments To Increase Carrier Card Engagemen...
How The Zebra Utilized Feature Experiments To Increase Carrier Card Engagemen...
Optimizely
 
Making Your Hypothesis Work Harder to Inform Future Product Strategy
Making Your Hypothesis Work Harder to Inform Future Product StrategyMaking Your Hypothesis Work Harder to Inform Future Product Strategy
Making Your Hypothesis Work Harder to Inform Future Product Strategy
Optimizely
 
Optimizely & Photobox - DON'T PANIC: The No-Confusion Experimentation Startup...
Optimizely & Photobox - DON'T PANIC: The No-Confusion Experimentation Startup...Optimizely & Photobox - DON'T PANIC: The No-Confusion Experimentation Startup...
Optimizely & Photobox - DON'T PANIC: The No-Confusion Experimentation Startup...
Optimizely
 
Train-the-Trainer - Enabling your team
Train-the-Trainer - Enabling your teamTrain-the-Trainer - Enabling your team
Train-the-Trainer - Enabling your team
Optimizely
 
The Art of the Start --Success in the first 100 Days
The Art of the Start --Success in the first 100 DaysThe Art of the Start --Success in the first 100 Days
The Art of the Start --Success in the first 100 Days
Optimizely
 
Testing Your Testing Program
Testing Your Testing ProgramTesting Your Testing Program
Testing Your Testing Program
Jill Martay
 
10 Tactics for Building an Optimization Culture
10 Tactics for Building an Optimization Culture10 Tactics for Building an Optimization Culture
10 Tactics for Building an Optimization Culture
Optimizely
 
Apply A/B Testing with NGINX Routing Policy
Apply A/B Testing with NGINX Routing PolicyApply A/B Testing with NGINX Routing Policy
Apply A/B Testing with NGINX Routing Policy
Supachai Jaturaprom
 
The Science of Getting Testing Right
The Science of Getting Testing RightThe Science of Getting Testing Right
The Science of Getting Testing Right
Optimizely
 
Maximizing Optimizely Website Performance
Maximizing Optimizely Website PerformanceMaximizing Optimizely Website Performance
Maximizing Optimizely Website Performance
Optimizely
 
Opticon 2017 Pushing the Boundaries of Experimentation
Opticon 2017 Pushing the Boundaries of ExperimentationOpticon 2017 Pushing the Boundaries of Experimentation
Opticon 2017 Pushing the Boundaries of Experimentation
Optimizely
 
Lean Software Development: Validated Learning
Lean Software Development: Validated LearningLean Software Development: Validated Learning
Lean Software Development: Validated Learning
Òscar Vilaplana
 
Opticon 2017 Advanced Program Management
Opticon 2017 Advanced Program ManagementOpticon 2017 Advanced Program Management
Opticon 2017 Advanced Program Management
Optimizely
 
Optimizely, HEMA & Farfetch - Unlock the Potential of Digital Experimentation...
Optimizely, HEMA & Farfetch - Unlock the Potential of Digital Experimentation...Optimizely, HEMA & Farfetch - Unlock the Potential of Digital Experimentation...
Optimizely, HEMA & Farfetch - Unlock the Potential of Digital Experimentation...
Optimizely
 
Introducing Program Management
Introducing Program ManagementIntroducing Program Management
Introducing Program Management
Optimizely
 
Test Everything: TrustRadius Delivers Customer Value with Experimentation
Test Everything: TrustRadius Delivers Customer Value with ExperimentationTest Everything: TrustRadius Delivers Customer Value with Experimentation
Test Everything: TrustRadius Delivers Customer Value with Experimentation
Optimizely
 
Create Successful Campaign with AB Testing
Create Successful Campaign with AB Testing Create Successful Campaign with AB Testing
Create Successful Campaign with AB Testing
NotifyVisitors
 

Similar to VWO Webinar: Scaling Your Testing Program (20)

How to Run Landing Page Tests On and Off Paid Social Platforms
How to Run Landing Page Tests On and Off Paid Social PlatformsHow to Run Landing Page Tests On and Off Paid Social Platforms
How to Run Landing Page Tests On and Off Paid Social Platforms
VWO
 
Data-Driven Decision Making by Expedia Sr PM
Data-Driven Decision Making by Expedia Sr PMData-Driven Decision Making by Expedia Sr PM
Data-Driven Decision Making by Expedia Sr PM
Product School
 
Ab testing
Ab testingAb testing
Ab testing
Fahad Zahid
 
AB testing.pptx
AB testing.pptxAB testing.pptx
AB testing.pptx
Mugabo4
 
Download Invesp’s The Essentials of Multivariate & AB Testing
Download Invesp’s The Essentials of Multivariate & AB TestingDownload Invesp’s The Essentials of Multivariate & AB Testing
Download Invesp’s The Essentials of Multivariate & AB Testing
Duy, Vo Hoang
 
Boost Your Conversion Rate
Boost Your Conversion RateBoost Your Conversion Rate
Boost Your Conversion Rate
✔ Antony Slabinck
 
introduction to Google Firebase and Ab testing
introduction to Google Firebase and Ab testingintroduction to Google Firebase and Ab testing
introduction to Google Firebase and Ab testing
Hamza Rehman
 
CRO Cheat Sheet
CRO Cheat SheetCRO Cheat Sheet
CRO Cheat Sheet
Chris Walton
 
A/B Testing with Website Optimizer
A/B Testing with Website OptimizerA/B Testing with Website Optimizer
A/B Testing with Website Optimizer
mm_merchant
 
A/B testing
A/B testingA/B testing
A/B testing
Sai krishna pramod Mandali
 
Shift Left - Approach and practices with IBM
Shift Left - Approach and practices with IBMShift Left - Approach and practices with IBM
Shift Left - Approach and practices with IBM
IBM UrbanCode Products
 
Webinar: Common Mistakes in A/B Testing
Webinar: Common Mistakes in A/B TestingWebinar: Common Mistakes in A/B Testing
Webinar: Common Mistakes in A/B Testing
Optimizely
 
A B testing introduction.pptx
A B testing introduction.pptxA B testing introduction.pptx
A B testing introduction.pptx
Ahmed Khaled
 
Introduction to Conversion Optimization - Nelio A/B Testing for WordPress
Introduction to Conversion Optimization - Nelio A/B Testing for WordPressIntroduction to Conversion Optimization - Nelio A/B Testing for WordPress
Introduction to Conversion Optimization - Nelio A/B Testing for WordPress
Nelio Software
 
Multivariable Testing White Paper 2006
Multivariable Testing White Paper 2006Multivariable Testing White Paper 2006
Multivariable Testing White Paper 2006
Rene Dechamps Otamendi
 
Making Use of A/B Testing Method for Digital Marketing
Making Use of A/B Testing Method for Digital MarketingMaking Use of A/B Testing Method for Digital Marketing
Making Use of A/B Testing Method for Digital Marketing
samkobe360
 
Website Optimization Without a Committee: Using Testing to Make Decisions
Website Optimization Without a Committee: Using Testing to Make DecisionsWebsite Optimization Without a Committee: Using Testing to Make Decisions
Website Optimization Without a Committee: Using Testing to Make Decisions
Earthbound Media Group
 
Resume
ResumeResume
Resume
vishalaxi sonnagi
 
Introduction to ab_testing_for_marketing_optimization
Introduction to ab_testing_for_marketing_optimizationIntroduction to ab_testing_for_marketing_optimization
Introduction to ab_testing_for_marketing_optimization
Turisticae
 
Web Analytics: A Measurement System of Website
Web Analytics: A Measurement System of WebsiteWeb Analytics: A Measurement System of Website
Web Analytics: A Measurement System of Website
ITDogadjaji.com
 
How to Run Landing Page Tests On and Off Paid Social Platforms
How to Run Landing Page Tests On and Off Paid Social PlatformsHow to Run Landing Page Tests On and Off Paid Social Platforms
How to Run Landing Page Tests On and Off Paid Social Platforms
VWO
 
Data-Driven Decision Making by Expedia Sr PM
Data-Driven Decision Making by Expedia Sr PMData-Driven Decision Making by Expedia Sr PM
Data-Driven Decision Making by Expedia Sr PM
Product School
 
AB testing.pptx
AB testing.pptxAB testing.pptx
AB testing.pptx
Mugabo4
 
Download Invesp’s The Essentials of Multivariate & AB Testing
Download Invesp’s The Essentials of Multivariate & AB TestingDownload Invesp’s The Essentials of Multivariate & AB Testing
Download Invesp’s The Essentials of Multivariate & AB Testing
Duy, Vo Hoang
 
introduction to Google Firebase and Ab testing
introduction to Google Firebase and Ab testingintroduction to Google Firebase and Ab testing
introduction to Google Firebase and Ab testing
Hamza Rehman
 
A/B Testing with Website Optimizer
A/B Testing with Website OptimizerA/B Testing with Website Optimizer
A/B Testing with Website Optimizer
mm_merchant
 
Shift Left - Approach and practices with IBM
Shift Left - Approach and practices with IBMShift Left - Approach and practices with IBM
Shift Left - Approach and practices with IBM
IBM UrbanCode Products
 
Webinar: Common Mistakes in A/B Testing
Webinar: Common Mistakes in A/B TestingWebinar: Common Mistakes in A/B Testing
Webinar: Common Mistakes in A/B Testing
Optimizely
 
A B testing introduction.pptx
A B testing introduction.pptxA B testing introduction.pptx
A B testing introduction.pptx
Ahmed Khaled
 
Introduction to Conversion Optimization - Nelio A/B Testing for WordPress
Introduction to Conversion Optimization - Nelio A/B Testing for WordPressIntroduction to Conversion Optimization - Nelio A/B Testing for WordPress
Introduction to Conversion Optimization - Nelio A/B Testing for WordPress
Nelio Software
 
Multivariable Testing White Paper 2006
Multivariable Testing White Paper 2006Multivariable Testing White Paper 2006
Multivariable Testing White Paper 2006
Rene Dechamps Otamendi
 
Making Use of A/B Testing Method for Digital Marketing
Making Use of A/B Testing Method for Digital MarketingMaking Use of A/B Testing Method for Digital Marketing
Making Use of A/B Testing Method for Digital Marketing
samkobe360
 
Website Optimization Without a Committee: Using Testing to Make Decisions
Website Optimization Without a Committee: Using Testing to Make DecisionsWebsite Optimization Without a Committee: Using Testing to Make Decisions
Website Optimization Without a Committee: Using Testing to Make Decisions
Earthbound Media Group
 
Introduction to ab_testing_for_marketing_optimization
Introduction to ab_testing_for_marketing_optimizationIntroduction to ab_testing_for_marketing_optimization
Introduction to ab_testing_for_marketing_optimization
Turisticae
 
Web Analytics: A Measurement System of Website
Web Analytics: A Measurement System of WebsiteWeb Analytics: A Measurement System of Website
Web Analytics: A Measurement System of Website
ITDogadjaji.com
 
Ad

More from VWO (20)

2025-04 - VWO Webinar - Alignment and Focus_ The Key to Delivering Business I...
2025-04 - VWO Webinar - Alignment and Focus_ The Key to Delivering Business I...2025-04 - VWO Webinar - Alignment and Focus_ The Key to Delivering Business I...
2025-04 - VWO Webinar - Alignment and Focus_ The Key to Delivering Business I...
VWO
 
The Experimenter’s Guide to eCommerce Growth
The Experimenter’s Guide to eCommerce GrowthThe Experimenter’s Guide to eCommerce Growth
The Experimenter’s Guide to eCommerce Growth
VWO
 
What 10,000 A/B Tests Reveal About Your Online Store
What 10,000 A/B Tests Reveal About Your Online StoreWhat 10,000 A/B Tests Reveal About Your Online Store
What 10,000 A/B Tests Reveal About Your Online Store
VWO
 
From Click to Call: Creating Landing Pages that Actually Convert to Sales Call
From Click to Call: Creating Landing Pages that Actually Convert to Sales CallFrom Click to Call: Creating Landing Pages that Actually Convert to Sales Call
From Click to Call: Creating Landing Pages that Actually Convert to Sales Call
VWO
 
Winners vs. Losers_ The New Rules of eCommerce Success - Webinar (1) (1).pdf
Winners vs. Losers_ The New Rules of eCommerce Success - Webinar (1) (1).pdfWinners vs. Losers_ The New Rules of eCommerce Success - Webinar (1) (1).pdf
Winners vs. Losers_ The New Rules of eCommerce Success - Webinar (1) (1).pdf
VWO
 
Real-time Behavior Prediction: The Missing Link in UX Experimentation
Real-time Behavior Prediction: The Missing Link in UX ExperimentationReal-time Behavior Prediction: The Missing Link in UX Experimentation
Real-time Behavior Prediction: The Missing Link in UX Experimentation
VWO
 
5 Myths that Stop you from running more experiments
5 Myths that Stop you from running more experiments5 Myths that Stop you from running more experiments
5 Myths that Stop you from running more experiments
VWO
 
VWO Webinar: Are You Covering the Basics?
VWO Webinar: Are You Covering the Basics?VWO Webinar: Are You Covering the Basics?
VWO Webinar: Are You Covering the Basics?
VWO
 
Por qué tus tests de e-commerce siguen fallando (y cómo solucionarlo)
Por qué tus tests de e-commerce siguen fallando (y cómo solucionarlo)Por qué tus tests de e-commerce siguen fallando (y cómo solucionarlo)
Por qué tus tests de e-commerce siguen fallando (y cómo solucionarlo)
VWO
 
From Data to Decisions: How AI is Reshaping User Behavior Analysis & Cont...
From Data to Decisions: How AI is Reshaping User Behavior Analysis & Cont...From Data to Decisions: How AI is Reshaping User Behavior Analysis & Cont...
From Data to Decisions: How AI is Reshaping User Behavior Analysis & Cont...
VWO
 
Cómo optimizar el embudo de conversión de tu tienda: Insights y estrategias p...
Cómo optimizar el embudo de conversión de tu tienda: Insights y estrategias p...Cómo optimizar el embudo de conversión de tu tienda: Insights y estrategias p...
Cómo optimizar el embudo de conversión de tu tienda: Insights y estrategias p...
VWO
 
EXT - Intersection Between AB Testing and Incrementality - VWO Webinar Dec 20...
EXT - Intersection Between AB Testing and Incrementality - VWO Webinar Dec 20...EXT - Intersection Between AB Testing and Incrementality - VWO Webinar Dec 20...
EXT - Intersection Between AB Testing and Incrementality - VWO Webinar Dec 20...
VWO
 
Supercharge Your HubSpot CRM: Unlocking Advanced CRO Capabilities with VWO In...
Supercharge Your HubSpot CRM: Unlocking Advanced CRO Capabilities with VWO In...Supercharge Your HubSpot CRM: Unlocking Advanced CRO Capabilities with VWO In...
Supercharge Your HubSpot CRM: Unlocking Advanced CRO Capabilities with VWO In...
VWO
 
La oleada de ventas navideñas: Una guía de experimentación para mayores conve...
La oleada de ventas navideñas: Una guía de experimentación para mayores conve...La oleada de ventas navideñas: Una guía de experimentación para mayores conve...
La oleada de ventas navideñas: Una guía de experimentación para mayores conve...
VWO
 
Copilot Webinar with product demo videos (1).pdf
Copilot Webinar with product demo videos (1).pdfCopilot Webinar with product demo videos (1).pdf
Copilot Webinar with product demo videos (1).pdf
VWO
 
Maximizing Revenue with A/B Testing During Black Friday & Cyber Monday: O...
Maximizing Revenue with A/B Testing During Black Friday & Cyber Monday: O...Maximizing Revenue with A/B Testing During Black Friday & Cyber Monday: O...
Maximizing Revenue with A/B Testing During Black Friday & Cyber Monday: O...
VWO
 
Use CRO Like the Pros_Modus_VWO_111324 (1).pdf
Use CRO Like the Pros_Modus_VWO_111324 (1).pdfUse CRO Like the Pros_Modus_VWO_111324 (1).pdf
Use CRO Like the Pros_Modus_VWO_111324 (1).pdf
VWO
 
Heatmaps: Visualizing User Engagement for Maximum Conversions | VWO Webinars
Heatmaps: Visualizing User Engagement for Maximum Conversions | VWO WebinarsHeatmaps: Visualizing User Engagement for Maximum Conversions | VWO Webinars
Heatmaps: Visualizing User Engagement for Maximum Conversions | VWO Webinars
VWO
 
Understanding What Your Visitors Truly Want Using VWO Insights - Web _ Webina...
Understanding What Your Visitors Truly Want Using VWO Insights - Web _ Webina...Understanding What Your Visitors Truly Want Using VWO Insights - Web _ Webina...
Understanding What Your Visitors Truly Want Using VWO Insights - Web _ Webina...
VWO
 
From OKRs To Strategic Experimentation - Masters of Conversion 2024 - Bjarn B...
From OKRs To Strategic Experimentation - Masters of Conversion 2024 - Bjarn B...From OKRs To Strategic Experimentation - Masters of Conversion 2024 - Bjarn B...
From OKRs To Strategic Experimentation - Masters of Conversion 2024 - Bjarn B...
VWO
 
2025-04 - VWO Webinar - Alignment and Focus_ The Key to Delivering Business I...
2025-04 - VWO Webinar - Alignment and Focus_ The Key to Delivering Business I...2025-04 - VWO Webinar - Alignment and Focus_ The Key to Delivering Business I...
2025-04 - VWO Webinar - Alignment and Focus_ The Key to Delivering Business I...
VWO
 
The Experimenter’s Guide to eCommerce Growth
The Experimenter’s Guide to eCommerce GrowthThe Experimenter’s Guide to eCommerce Growth
The Experimenter’s Guide to eCommerce Growth
VWO
 
What 10,000 A/B Tests Reveal About Your Online Store
What 10,000 A/B Tests Reveal About Your Online StoreWhat 10,000 A/B Tests Reveal About Your Online Store
What 10,000 A/B Tests Reveal About Your Online Store
VWO
 
From Click to Call: Creating Landing Pages that Actually Convert to Sales Call
From Click to Call: Creating Landing Pages that Actually Convert to Sales CallFrom Click to Call: Creating Landing Pages that Actually Convert to Sales Call
From Click to Call: Creating Landing Pages that Actually Convert to Sales Call
VWO
 
Winners vs. Losers_ The New Rules of eCommerce Success - Webinar (1) (1).pdf
Winners vs. Losers_ The New Rules of eCommerce Success - Webinar (1) (1).pdfWinners vs. Losers_ The New Rules of eCommerce Success - Webinar (1) (1).pdf
Winners vs. Losers_ The New Rules of eCommerce Success - Webinar (1) (1).pdf
VWO
 
Real-time Behavior Prediction: The Missing Link in UX Experimentation
Real-time Behavior Prediction: The Missing Link in UX ExperimentationReal-time Behavior Prediction: The Missing Link in UX Experimentation
Real-time Behavior Prediction: The Missing Link in UX Experimentation
VWO
 
5 Myths that Stop you from running more experiments
5 Myths that Stop you from running more experiments5 Myths that Stop you from running more experiments
5 Myths that Stop you from running more experiments
VWO
 
VWO Webinar: Are You Covering the Basics?
VWO Webinar: Are You Covering the Basics?VWO Webinar: Are You Covering the Basics?
VWO Webinar: Are You Covering the Basics?
VWO
 
Por qué tus tests de e-commerce siguen fallando (y cómo solucionarlo)
Por qué tus tests de e-commerce siguen fallando (y cómo solucionarlo)Por qué tus tests de e-commerce siguen fallando (y cómo solucionarlo)
Por qué tus tests de e-commerce siguen fallando (y cómo solucionarlo)
VWO
 
From Data to Decisions: How AI is Reshaping User Behavior Analysis & Cont...
From Data to Decisions: How AI is Reshaping User Behavior Analysis & Cont...From Data to Decisions: How AI is Reshaping User Behavior Analysis & Cont...
From Data to Decisions: How AI is Reshaping User Behavior Analysis & Cont...
VWO
 
Cómo optimizar el embudo de conversión de tu tienda: Insights y estrategias p...
Cómo optimizar el embudo de conversión de tu tienda: Insights y estrategias p...Cómo optimizar el embudo de conversión de tu tienda: Insights y estrategias p...
Cómo optimizar el embudo de conversión de tu tienda: Insights y estrategias p...
VWO
 
EXT - Intersection Between AB Testing and Incrementality - VWO Webinar Dec 20...
EXT - Intersection Between AB Testing and Incrementality - VWO Webinar Dec 20...EXT - Intersection Between AB Testing and Incrementality - VWO Webinar Dec 20...
EXT - Intersection Between AB Testing and Incrementality - VWO Webinar Dec 20...
VWO
 
Supercharge Your HubSpot CRM: Unlocking Advanced CRO Capabilities with VWO In...
Supercharge Your HubSpot CRM: Unlocking Advanced CRO Capabilities with VWO In...Supercharge Your HubSpot CRM: Unlocking Advanced CRO Capabilities with VWO In...
Supercharge Your HubSpot CRM: Unlocking Advanced CRO Capabilities with VWO In...
VWO
 
La oleada de ventas navideñas: Una guía de experimentación para mayores conve...
La oleada de ventas navideñas: Una guía de experimentación para mayores conve...La oleada de ventas navideñas: Una guía de experimentación para mayores conve...
La oleada de ventas navideñas: Una guía de experimentación para mayores conve...
VWO
 
Copilot Webinar with product demo videos (1).pdf
Copilot Webinar with product demo videos (1).pdfCopilot Webinar with product demo videos (1).pdf
Copilot Webinar with product demo videos (1).pdf
VWO
 
Maximizing Revenue with A/B Testing During Black Friday & Cyber Monday: O...
Maximizing Revenue with A/B Testing During Black Friday & Cyber Monday: O...Maximizing Revenue with A/B Testing During Black Friday & Cyber Monday: O...
Maximizing Revenue with A/B Testing During Black Friday & Cyber Monday: O...
VWO
 
Use CRO Like the Pros_Modus_VWO_111324 (1).pdf
Use CRO Like the Pros_Modus_VWO_111324 (1).pdfUse CRO Like the Pros_Modus_VWO_111324 (1).pdf
Use CRO Like the Pros_Modus_VWO_111324 (1).pdf
VWO
 
Heatmaps: Visualizing User Engagement for Maximum Conversions | VWO Webinars
Heatmaps: Visualizing User Engagement for Maximum Conversions | VWO WebinarsHeatmaps: Visualizing User Engagement for Maximum Conversions | VWO Webinars
Heatmaps: Visualizing User Engagement for Maximum Conversions | VWO Webinars
VWO
 
Understanding What Your Visitors Truly Want Using VWO Insights - Web _ Webina...
Understanding What Your Visitors Truly Want Using VWO Insights - Web _ Webina...Understanding What Your Visitors Truly Want Using VWO Insights - Web _ Webina...
Understanding What Your Visitors Truly Want Using VWO Insights - Web _ Webina...
VWO
 
From OKRs To Strategic Experimentation - Masters of Conversion 2024 - Bjarn B...
From OKRs To Strategic Experimentation - Masters of Conversion 2024 - Bjarn B...From OKRs To Strategic Experimentation - Masters of Conversion 2024 - Bjarn B...
From OKRs To Strategic Experimentation - Masters of Conversion 2024 - Bjarn B...
VWO
 
Ad

Recently uploaded (20)

Top 10 Search Engines in the World 2025, Learn form this PowerPoint
Top 10 Search Engines in the World 2025, Learn form this PowerPointTop 10 Search Engines in the World 2025, Learn form this PowerPoint
Top 10 Search Engines in the World 2025, Learn form this PowerPoint
kalyanrkm12
 
How to Integrate AI into Your Marketing Campaigns Today
How to Integrate AI into Your Marketing Campaigns TodayHow to Integrate AI into Your Marketing Campaigns Today
How to Integrate AI into Your Marketing Campaigns Today
NapierPR
 
CaseStudy of marketing strategy of himalyan java
CaseStudy of marketing strategy of himalyan javaCaseStudy of marketing strategy of himalyan java
CaseStudy of marketing strategy of himalyan java
PrashantShah565560
 
The SEO Lifecycle: Lessons from the Past, Strategies for the Future
The SEO Lifecycle: Lessons from the Past, Strategies for the FutureThe SEO Lifecycle: Lessons from the Past, Strategies for the Future
The SEO Lifecycle: Lessons from the Past, Strategies for the Future
Lily Ray
 
Beyond the Pain Points: A More Meaningful Customer Journey
Beyond the Pain Points: A More Meaningful Customer JourneyBeyond the Pain Points: A More Meaningful Customer Journey
Beyond the Pain Points: A More Meaningful Customer Journey
Odyyc
 
Unlocking Customer Engagement with WhatsApp Business API
Unlocking Customer Engagement with WhatsApp Business APIUnlocking Customer Engagement with WhatsApp Business API
Unlocking Customer Engagement with WhatsApp Business API
G Tech Web Marketing
 
Kamryn and Alex's Cowtown Brand Book Final
Kamryn and Alex's Cowtown Brand Book FinalKamryn and Alex's Cowtown Brand Book Final
Kamryn and Alex's Cowtown Brand Book Final
kamryn0009
 
Beyond ROAS: Aligning Google Ads With Your True Business Objectives
Beyond ROAS: Aligning Google Ads With Your True Business ObjectivesBeyond ROAS: Aligning Google Ads With Your True Business Objectives
Beyond ROAS: Aligning Google Ads With Your True Business Objectives
Search Engine Journal
 
Digital Reviver - Digital marketing agency in Noida
Digital Reviver - Digital marketing agency in NoidaDigital Reviver - Digital marketing agency in Noida
Digital Reviver - Digital marketing agency in Noida
Digital Reviver
 
The Power of Scheduling Videos - GudSho.
The Power of Scheduling Videos - GudSho.The Power of Scheduling Videos - GudSho.
The Power of Scheduling Videos - GudSho.
Gudsho
 
Twitch Growth Tips: Why View Count Still Matters
Twitch Growth Tips: Why View Count Still MattersTwitch Growth Tips: Why View Count Still Matters
Twitch Growth Tips: Why View Count Still Matters
officialishitamehra
 
The 10 Best Tips to Get SoundCloud Likes & Grow Your Profile Fast.pdf
The 10 Best Tips to Get SoundCloud Likes & Grow Your Profile Fast.pdfThe 10 Best Tips to Get SoundCloud Likes & Grow Your Profile Fast.pdf
The 10 Best Tips to Get SoundCloud Likes & Grow Your Profile Fast.pdf
Sociofire
 
REFERENCES GROUPS AND FAMILY REFERENCES SCHIFFMAN
REFERENCES GROUPS AND FAMILY REFERENCES SCHIFFMANREFERENCES GROUPS AND FAMILY REFERENCES SCHIFFMAN
REFERENCES GROUPS AND FAMILY REFERENCES SCHIFFMAN
FatimahAzzahro59
 
How to Use Roll-Up Banners for Brand Visibility at Events.pdf
How to Use Roll-Up Banners for Brand Visibility at Events.pdfHow to Use Roll-Up Banners for Brand Visibility at Events.pdf
How to Use Roll-Up Banners for Brand Visibility at Events.pdf
Ezybook
 
Chocki Relaunch Marketing Plan | Cadbury's Nostalgic Chocolate Comeback Strategy
Chocki Relaunch Marketing Plan | Cadbury's Nostalgic Chocolate Comeback StrategyChocki Relaunch Marketing Plan | Cadbury's Nostalgic Chocolate Comeback Strategy
Chocki Relaunch Marketing Plan | Cadbury's Nostalgic Chocolate Comeback Strategy
Azmeen Gadit
 
Byredo Makeup pitch (shortened version).pptx
Byredo Makeup pitch (shortened version).pptxByredo Makeup pitch (shortened version).pptx
Byredo Makeup pitch (shortened version).pptx
helloatveronika
 
Quality Analysis of the project and their requirement
Quality Analysis of the project and their requirementQuality Analysis of the project and their requirement
Quality Analysis of the project and their requirement
PrashantShah565560
 
How to Choose the Right Performance Marketing Agency to Drive Your Success
How to Choose the Right Performance Marketing Agency to Drive Your SuccessHow to Choose the Right Performance Marketing Agency to Drive Your Success
How to Choose the Right Performance Marketing Agency to Drive Your Success
Viacon
 
Get client sites EAA-ready: Accessibility best practices for agencies
Get client sites EAA-ready: Accessibility best practices for agenciesGet client sites EAA-ready: Accessibility best practices for agencies
Get client sites EAA-ready: Accessibility best practices for agencies
Anton Shulke
 
Alt Attribute SEO Guide: How to Optimize Image Alt Text for Search
Alt Attribute SEO Guide: How to Optimize Image Alt Text for SearchAlt Attribute SEO Guide: How to Optimize Image Alt Text for Search
Alt Attribute SEO Guide: How to Optimize Image Alt Text for Search
Sanjay Kumar Monu
 
Top 10 Search Engines in the World 2025, Learn form this PowerPoint
Top 10 Search Engines in the World 2025, Learn form this PowerPointTop 10 Search Engines in the World 2025, Learn form this PowerPoint
Top 10 Search Engines in the World 2025, Learn form this PowerPoint
kalyanrkm12
 
How to Integrate AI into Your Marketing Campaigns Today
How to Integrate AI into Your Marketing Campaigns TodayHow to Integrate AI into Your Marketing Campaigns Today
How to Integrate AI into Your Marketing Campaigns Today
NapierPR
 
CaseStudy of marketing strategy of himalyan java
CaseStudy of marketing strategy of himalyan javaCaseStudy of marketing strategy of himalyan java
CaseStudy of marketing strategy of himalyan java
PrashantShah565560
 
The SEO Lifecycle: Lessons from the Past, Strategies for the Future
The SEO Lifecycle: Lessons from the Past, Strategies for the FutureThe SEO Lifecycle: Lessons from the Past, Strategies for the Future
The SEO Lifecycle: Lessons from the Past, Strategies for the Future
Lily Ray
 
Beyond the Pain Points: A More Meaningful Customer Journey
Beyond the Pain Points: A More Meaningful Customer JourneyBeyond the Pain Points: A More Meaningful Customer Journey
Beyond the Pain Points: A More Meaningful Customer Journey
Odyyc
 
Unlocking Customer Engagement with WhatsApp Business API
Unlocking Customer Engagement with WhatsApp Business APIUnlocking Customer Engagement with WhatsApp Business API
Unlocking Customer Engagement with WhatsApp Business API
G Tech Web Marketing
 
Kamryn and Alex's Cowtown Brand Book Final
Kamryn and Alex's Cowtown Brand Book FinalKamryn and Alex's Cowtown Brand Book Final
Kamryn and Alex's Cowtown Brand Book Final
kamryn0009
 
Beyond ROAS: Aligning Google Ads With Your True Business Objectives
Beyond ROAS: Aligning Google Ads With Your True Business ObjectivesBeyond ROAS: Aligning Google Ads With Your True Business Objectives
Beyond ROAS: Aligning Google Ads With Your True Business Objectives
Search Engine Journal
 
Digital Reviver - Digital marketing agency in Noida
Digital Reviver - Digital marketing agency in NoidaDigital Reviver - Digital marketing agency in Noida
Digital Reviver - Digital marketing agency in Noida
Digital Reviver
 
The Power of Scheduling Videos - GudSho.
The Power of Scheduling Videos - GudSho.The Power of Scheduling Videos - GudSho.
The Power of Scheduling Videos - GudSho.
Gudsho
 
Twitch Growth Tips: Why View Count Still Matters
Twitch Growth Tips: Why View Count Still MattersTwitch Growth Tips: Why View Count Still Matters
Twitch Growth Tips: Why View Count Still Matters
officialishitamehra
 
The 10 Best Tips to Get SoundCloud Likes & Grow Your Profile Fast.pdf
The 10 Best Tips to Get SoundCloud Likes & Grow Your Profile Fast.pdfThe 10 Best Tips to Get SoundCloud Likes & Grow Your Profile Fast.pdf
The 10 Best Tips to Get SoundCloud Likes & Grow Your Profile Fast.pdf
Sociofire
 
REFERENCES GROUPS AND FAMILY REFERENCES SCHIFFMAN
REFERENCES GROUPS AND FAMILY REFERENCES SCHIFFMANREFERENCES GROUPS AND FAMILY REFERENCES SCHIFFMAN
REFERENCES GROUPS AND FAMILY REFERENCES SCHIFFMAN
FatimahAzzahro59
 
How to Use Roll-Up Banners for Brand Visibility at Events.pdf
How to Use Roll-Up Banners for Brand Visibility at Events.pdfHow to Use Roll-Up Banners for Brand Visibility at Events.pdf
How to Use Roll-Up Banners for Brand Visibility at Events.pdf
Ezybook
 
Chocki Relaunch Marketing Plan | Cadbury's Nostalgic Chocolate Comeback Strategy
Chocki Relaunch Marketing Plan | Cadbury's Nostalgic Chocolate Comeback StrategyChocki Relaunch Marketing Plan | Cadbury's Nostalgic Chocolate Comeback Strategy
Chocki Relaunch Marketing Plan | Cadbury's Nostalgic Chocolate Comeback Strategy
Azmeen Gadit
 
Byredo Makeup pitch (shortened version).pptx
Byredo Makeup pitch (shortened version).pptxByredo Makeup pitch (shortened version).pptx
Byredo Makeup pitch (shortened version).pptx
helloatveronika
 
Quality Analysis of the project and their requirement
Quality Analysis of the project and their requirementQuality Analysis of the project and their requirement
Quality Analysis of the project and their requirement
PrashantShah565560
 
How to Choose the Right Performance Marketing Agency to Drive Your Success
How to Choose the Right Performance Marketing Agency to Drive Your SuccessHow to Choose the Right Performance Marketing Agency to Drive Your Success
How to Choose the Right Performance Marketing Agency to Drive Your Success
Viacon
 
Get client sites EAA-ready: Accessibility best practices for agencies
Get client sites EAA-ready: Accessibility best practices for agenciesGet client sites EAA-ready: Accessibility best practices for agencies
Get client sites EAA-ready: Accessibility best practices for agencies
Anton Shulke
 
Alt Attribute SEO Guide: How to Optimize Image Alt Text for Search
Alt Attribute SEO Guide: How to Optimize Image Alt Text for SearchAlt Attribute SEO Guide: How to Optimize Image Alt Text for Search
Alt Attribute SEO Guide: How to Optimize Image Alt Text for Search
Sanjay Kumar Monu
 

VWO Webinar: Scaling Your Testing Program

  • 1. Scaling your Testing Program Vivek Vilatia Senior Product Manager, VWO Anshul Gurbaxani Associate Product Manager, VWO Prem Kujur Customer Success Manager, VWO
  • 2. Ask Questions on Twitter #AskVWO
  • 3. What is A/B testing In the most simplest form, A/B Testing is comparing two versions of a web page to see which version performs better for a defined conversion goal
  • 4. Type of Tests A/B Test Change any element or section of a webpage and test both versions. Split URL Test Test performances of separate URLs if the landing pages are significantly different. Multivariate Test Run tests on multiple variation combinations together. Needs sufficient traffic.
  • 6. Simple A/B tests Advanced A/B tests ➔ Great starting point ➔ Smaller teams ➔ Easy to setup ➔ Quick wins ➔ Test changes like copy, arrangement of elements, CTA colors etc ➔ Test advanced use cases with greater impact ➔ Conversion optimization program ➔ Self-sufficient team ➔ The right tool-set ➔ Management buy-in ➔ Tons of perseverance
  • 7. Audience Changes MetricsPages Key steps in setting up your tests
  • 8. Test pages | Defining a set of similar pages for your test Asterix * in the URL matches pattern would replace the variable part of the URL Represented by * in the URL matches pattern
  • 9. Audience | Complex targeting rules Visitors logged in on your website Are either from India or have their preferred currency as INR
  • 10. Changes | Code editor for complex changes
  • 11. Changes | Code editor for complex changes
  • 12. Changes | Conditional Test Activation
  • 13. Changes | Conditional Test Activation SignUp Form Type Form
  • 15. Measuring the right metrics Micro Conversions Macro Conversions
  • 17. Never make your tests live without proper sanity checks
  • 22. Making sense of your A/B testing reports
  • 23. Making sense of your A/B testing reports
  • 24. Run your test for at least a week. Along with a primary goal, have related secondary goals. Ensure that your primary goal represents the direct and exclusive impact of the changes being tested. Making sense of your A/B testing reports - Best Practices
  • 25. Slice and Dice your test results to learn the trends and behaviour of various user segments. Testing radical changes at times might just do the trick Identify micro-conversions and important correlative metrics increase in a given variation, implement it. Inconclusive Tests
  • 26. Ensure that you don’t use segments with very less data. Make sure there is no error in the test setup. Ensure that you are not running multiple tests simultaneously in the same part of the journey you are A/B testing Inconclusive Tests - Pitfalls