2025-04 - VWO Webinar - Alignment and Focus_ The Key to Delivering Business Impact Through Experimentation .pdf
0 likes84 views
On the one hand, it’s a simple ask: “Tell me the revenue impact from our experimentation program.” On the other hand…it’s just not that simple. In this webinar, Val and Tim will walk through an approach and mindset that puts business impact—measurable outcomes—at the core of an organization’s approach to research, experimentation, and analysis. And it starts long before a specific test idea or analysis project is identified. The approach is grounded in unambiguous alignment between the experimentation team and their business partners that focuses on: measurable outcomes, the problems most worth solving to ensure those outcomes come to pass, and treating ideas for addressing those problems as validatable hypotheses.
![factsandfeelings.io | April 2025
Ideas structured as
hypotheses
We believe [some idea]...
…because [some evidence or
observation].
If we are right, we will [some action].
solutions](https://image.slidesharecdn.com/2025-04-vwowebinar-alignmentandfocusthekeytodeliveringbusinessimpactthroughexperimentation-250430103942-d46db285/85/2025-04-VWO-Webinar-Alignment-and-Focus_-The-Key-to-Delivering-Business-Impact-Through-Experimentation-pdf-19-320.jpg)
2025-04 - VWO Webinar - Alignment and Focus_ The Key to Delivering Business Impact Through Experimentation .pdf
- 1. factsandfeelings.io | April 2025 Alignment and Focus The Key to Delivering Business Impact Through Experimentation
- 2. factsandfeelings.io | April 2025 Tim Wilson Valerie Kroll Co-founder, Head of Solutions Co-founder, Head of Delivery
- 3. factsandfeelings.io | April 2025 In spite of good intentions & significant investment many organizations struggle to achieve measurable business outcomes supported by experimentation, analytics, and research.
- 4. factsandfeelings.io | April 2025 We bring an always-on, evidence-based approach for identifying, prioritizing, and validating ideas that drive strategic growth.
- 5. factsandfeelings.io | April 2025 Experimentation Program Impact Isnʼt it just a mathematical formula?
- 6. factsandfeelings.io | April 2025 Where: n = the number of winning experiments in the year ΔCRi = the lift in the conversion rate from experiment i Vi = the baseline volume (number of visitors or sessions/year) relevant to the area affected by experiment i) AOVi = the average order value for experiment i ti = the time remaining in the year, in months, after implementation of experiment i D(ti ) = the decay function, representing the diminishing impact of the result of experiment i over time
- 7. factsandfeelings.io | April 2025 1 2 3 4 If only it were that simple The “observed lift” is not the actual lift The decay rate is real, but, in practice, it has to just be estimated Interaction effects—experiments upon experiments (even if not concurrently run)—aren’t accounted for “Savings”—experimentation preventing the rollout of worse experiences—are difficult to account for “And what about…” “And what about…” “And what about…”
- 8. factsandfeelings.io | April 2025 We’ve calculated that we’ve delivered $131,478,773 of impact this year! The entire website generated $110 million this year.
- 9. factsandfeelings.io | April 2025 A better way…
- 10. factsandfeelings.io | April 2025 Business Impact ~ Inform Decisions & Reduce Uncertainty & Deliver Learnings
- 11. factsandfeelings.io | April 2025 Clear ideas for shrinking key problems. Deep understanding of the problems hindering outcomes. Objective validation of which ideas work. Concise and measurable business outcomes. In theory, business is pretty simple
- 12. factsandfeelings.io | April 2025 “All” that is required is connecting the dots between business goals, the problems most hindering growth & the validated ideas driving measurable and meaningful outcomes. Easy to say. Harder to do!
- 13. factsandfeelings.io | April 2025 goals Start with business outcomes. Balanced, measurable & unambiguous.
- 14. factsandfeelings.io | April 2025 goals Outcome-Oriented Measurement Plan Focus and alignment among business partners as to what success looks like.
- 15. factsandfeelings.io | April 2025 goals Outcome-Oriented Dashboard Regular cadence for objectively checking in on performance
- 16. factsandfeelings.io | April 2025 problems Then focus on problems. Not just any problem, the problems most worth solving.
- 17. factsandfeelings.io | April 2025 Loss aversion is an impactful collaboration technique
- 18. factsandfeelings.io | April 2025 Now itʼs time for solutions. Hypothesized ideas. Validated with research, analysis, and experimentation. solutions
- 19. factsandfeelings.io | April 2025 Ideas structured as hypotheses We believe [some idea]... …because [some evidence or observation]. If we are right, we will [some action]. solutions
- 20. factsandfeelings.io | April 2025 Collect learnings & develop insights — that make you more likely to succeed and drive lasting organizational transformation. insights
- 21. factsandfeelings.io | April 2025 Priorities of the business used as ideation inspiration In-person collaboration… …captured as structured data… …used to power business activities.
- 22. factsandfeelings.io | April 2025 How are we tracking against our goals? What are we doing to continuously improve?
- 23. factsandfeelings.io | April 2025 This work is elegantly simple… …but brutally hard.
- 24. factsandfeelings.io | April 2025 Recovering CROs make the best strategists
- 25. factsandfeelings.io | April 2025 A Program Level Example
- 26. factsandfeelings.io | April 2025 Org Model Small centralized team supporting many LOBs Upskilling Investing in right practices & training Growth Expanding the experimenter user base Context
- 27. factsandfeelings.io | April 2025 How can we clearly demonstrate progress within this large change management initiative and highlight the business impact of experimentation? PRIMARY CLIENT QUESTION
- 28. factsandfeelings.io | April 2025 goals Start with program outcomes. Balanced, measurable & unambiguous.
- 29. factsandfeelings.io | April 2025 Business impact from experimentation is a combination of doing the right experiments and doing experimentation right.
- 30. factsandfeelings.io | April 2025 1 Supporting company strategy through experimentation 2 Improve experiment quality using best practices 3 Increase the velocity of experiments 4 Creating a culture of experimentation What are we trying to achieve? Experimentation Program Outcomes
- 31. factsandfeelings.io | April 2025 goals Outcome-Oriented Measurement Plan Focus and alignment among cross-functional team as to what success looks like.
- 32. factsandfeelings.io | April 2025 Leadership View
- 33. factsandfeelings.io | April 2025 1 Newly formed steering committee was made the clear owner 2 Measure progress and adherence to newly implemented right practices 3 Indicate where additional education and support is needed 4 Communicate the impact of experimentation to leadership Ongoing Utility
- 34. factsandfeelings.io | April 2025 In summary…
- 35. factsandfeelings.io | April 2025 1 2 3 4 Takeaways Think about “business impact” broadly: informing decisions, reducing uncertainty, delivering learnings Work towards unambiguous alignment on measurable business outcomes and the problems most worth solving Collaboratively identify and prioritize solution ideas— hypotheses—that can be validated Establish ongoing check-ins and to ensure alignment with business partners is maintained
- 36. factsandfeelings.io | April 2025 Thank you. factsandfeelings.io