SlideShare a Scribd company logo

12 Things to do Before Your Company Dies : Conversion Conference London - Oct 2013

Download as PPTX, PDF
Craig Sullivan, profile picture
Craig Sullivan

The document outlines essential actions for companies to avoid failure, emphasizing the importance of fixing analytics and gathering accurate insights. It advocates for investing in proper tools and methodologies to support user experience, usability testing, and continuous product improvement. Additionally, it stresses the need for agile execution, a collaborative culture, and effective segmentation for testing to adapt to market and customer needs.

TechnologyBusiness
1 of 84
Downloaded 48 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
12 things to do before your company dies 23rd Oct 2013 @OptimiseOrDie
Timeline + Tweet me! - 1998 1999 - 2004 2004-2008 2008-2012 @OptimiseOrDie
SEO PPC UX Analytics Design Development QA Performance A/B and Multivariate testing Customer Satisfaction 40+ websites, 34 countries, 19 languages, €1bn+ revenue 8 people @OptimiseOrDie
If you‟re not a part of the solution, there‟s good money to be made in prolonging the problem
Out of my comfort zone… @OptimiseOrDie
Behind enemy lines… @OptimiseOrDie
Nice day at the office, dear? @OptimiseOrDie
12 Things to do Before Your Company Dies : Conversion Conference London - Oct 2013
Competition… @OptimiseOrDie
Traffic is harder! SEO/PPC @OptimiseOrDie
Panguin tool…
#1 : Fix your broken Analytics! @OptimiseOrDie
#1 : Fix your broken Analytics! @OptimiseOrDie
#1 : Common problems (GA) • Dual purpose goal page – One page used by two outcomes – and not split • Cross domain tracking – Where you jump between sites, this borks the data • Filters not correctly set up – Your office, agencies, developers are skewing data • Code missing or double code – Causes visit splitting, double pageviews, skews bounce rate • Campaign, Social, Email tracking etc. – External links you generate are not setup to record properly • Errors not tracked (404, 5xx, Other) – You are unaware of error volumes, locations and impact • Dual flow funnels – Flows join in the middle of a funnel or loop internally • Event tracking skews bounce rate – If an event is set to be „interactive‟ – it can skew bounce rate (example) @OptimiseOrDie
#1 : Solutions • Get a Health Check for your Analytics – Try @prwd, @danbarker, @peter_oneill or ask me! • Invest continually in instrumentation – Aim for at least 5% of dev time to fix + improve • Stop shrugging : plug your insight gaps – Change „I don‟t know‟ to „I‟ll find out‟ • Look at event tracking (Google Analytics) – If set up correctly, you get wonderful insights • Would you use paper instead of a till? – You wouldn‟t do it in retail so stop doing it online! • How do you win F1 races? – With the wrong performance data, you won‟t @OptimiseOrDie
#2 : Get the right inputs Insight - Inputs Opinion Cherished notions Marketing whims Cosmic rays Not ‘on brand’ enough Ego IT inflexibility Panic Internal company needs #FAIL Competitor change An article the CEO read Some dumbass consultant Competitor copying Dice rolling Guessing Knee jerk reactons Shiny feature blindness @OptimiseOrDie
#2 : Get the right inputs Insight - Inputs Usability testing Forms analytics Search analytics Voice of Customer Market research Eye tracking Customer contact A/B and MVT testing Big & unstructured data Insight Social analytics Session Replay Web analytics Segmentation Sales and Call Centre Surveys Customer services Competitor evals @OptimiseOrDie
#2 : Solutions • Usability testing and User Centred design – If you‟re not doing this properly, you‟re hosed • Champion UX+ - with added numbers – (Re)designing without inputs + numbers is guessing • You need one team on this, not silos – Stop handing round the baby (I‟ll come back to this) • Ego, Opinion, Cherished notions – fill gaps – Fill these vacuums with insights and data • Champion the users – Someone needs to take their side! • You need multiple tool inputs – Let me show you my shortlist… @OptimiseOrDie
#3 : Get the right tools Insight - Inputs @OptimiseOrDie
3.1 - Session Replay • • • • • Vital for optimisers & fills in a ‘missing link’ for insight Rich source of data on visitor experiences Segment by browser, visitor type, behaviour, errors Forms Analytics (when instrumented) are awesome Can be used to optimise in real time! Session replay tools • Clicktale (Client) • SessionCam (Client) • Mouseflow (Client) • Ghostrec (Client) • Usabilla (Client) • Tealeaf (Hybrid) • UserReplay (Server) www.clicktale.com www.sessioncam.com www.mouseflow.com www.ghostrec.com www.usabilla.com www.tealeaf.com www.userreplay.com @OptimiseOrDie
21
22
3.2 - Feedback / VOC tools • • • • Anything that allows immediate realtime onpage feedback Comments on elements, pages and overall site & service Can be used for behavioural triggered feedback Tip! : Take the Call Centre for beers • Kampyle www.kampyle.com • Qualaroo www.qualaroo.com • 4Q 4q.iperceptions.com • Usabilla www.usabilla.com
3.3 - Survey Tools • Surveymonkey • Zoomerang • SurveyGizmo www.surveymonkey.com www.zoomerang.com www.surveygizmo.com (1/5) (3/5) (5/5) • For surveys, web forms, checkouts, lead gen – anything with form filling – you have to read these two: Caroline Jarrett (@cjforms) Luke Wroblewski (@lukew) • With their work and copywriting from @stickycontent, I managed to get a survey with a 35% clickthrough from email and a whopping 94% form completion rate. • Their awesome insights are the killer app I have when optimising forms and funnel processes for clients. @OptimiseOrDie
3.4 - UX Crowd tools Som, feedback Remote UX tools (P=Panel, S=Site recruited, B=Both) Usertesting (B) www.usertesting.com Userlytics (B) www.userlytics.com Userzoom (S) www.userzoom.com Intuition HQ (S) www.intuitionhq.com Mechanical turk (S) www.mechanicalturk.com Loop11 (S) www.loop11.com Open Hallway (S) www.openhallway.com What Users Do (P) www.whatusersdo.com Feedback army (P) www.feedbackarmy.com User feel (P) www.userfeel.com Ethnio (For Recruiting) www.ethnio.com Feedback on Prototypes / Mockups Pidoco Verify from Zurb Five second test Conceptshare Usabilla www.pidoco.com www.verifyapp.com www.fivesecondtest.com www.conceptshare.com www.usabilla.com 25
3.5 – Usability Recording / Testing • Some low budget testing tools: CamStudio (free) www.camstudio.org Mediacam AV (cheap) www.netu2.co.uk Silverback (Mac) www.silverbackapp.com Screenflow (Mac) www.telestream.net UX Recorder (iOS), Skype Hugging, Reflection www.uxrecorder.com & bit.ly/tesTfm & bit.ly/GZMgxR @OptimiseOrDie
3.5 – Test or Die! Email testing www.litmus.com www.returnpath.com www.lyris.com Browser testing www.crossbrowsertesting.com www.cloudtesting.com www.multibrowserviewer.com www.saucelabs.com Mobile devices www.perfectomobile.com www.deviceanywhere.com www.mobilexweb.com/emulators www.opendevicelab.com @OptimiseOrDie
3.6 – Collaboration tools Oh sh*t @OptimiseOrDie
3.6 – Basecamp (Project management) @OptimiseOrDie
3.6 – Join.me (free screen sharer)
3.6 – Trello (task & project) @OptimiseOrDie
3.6 – Pivotal Tracker (Agile, Lean) @OptimiseOrDie
3.6 – Cloud Collaboration (charting) • LucidChart @OptimiseOrDie
3.6 – Cloud Collaboration (feedback) • Webnotes & Protonotes @OptimiseOrDie
3.6 – Cloud Collaboration (feedback) • Conceptshare @OptimiseOrDie
3.7 – Split testing tools – Cheap! • Google Content Experiments bit.ly/Ljg7Ds • Multi Armed Bandit Explanation bit.ly/Xa80O8 • Optimizely www.optimizely.com • Visual Website Optimizer www.visualwebsiteoptimizer.com @OptimiseOrDie
3.7 – Split test funnies • Many tests fail due to QA or browser bugs – Always do cross browser QA testing – see resources • Don’t rely on developers saying ‘yes’ – Use your analytics to define the list to test • Cross instrument your analytics – You need this to check the test software works • Store the variant(s) seen in analytics – Compare people who saw A/B/A vs. A/B/B • Segment your data to find variances – Failed tests usually show differences for segments • Watch the test and analytics CLOSELY – After you go live, religiously check both – Read this article : stanford.io/15UYov0 @OptimiseOrDie
#3 : Summary – the minimum! • Session replay tools – Clicktale, Tealeaf, Sessioncam and more… • Cheap / Crowdsourced usability testing – See the list in this deck • Voice of Customer / Feedback tools – 4Q, Kampyle, Qualaroo, Usabilla and more… • A/B and Multivariate testing – Optimizely, Google Content Experiments, VWO • Email, Browser and Mobile Device Testing – You don‟t know if it works unless you check @OptimiseOrDie
#4 : Go Agile @OptimiseOrDie
Methodologies - Lean UX “The application of UX design methods into product development, tailored to fit Build-Measure-Learn cycles.” Positive – – – – – – Lightweight and very fast methods Realtime or rapid improvements Documentation light, value high Low on wastage and frippery Fast time to market, then optimise Allows you to pivot into new areas Negative – Often needs user test feedback to steer the development, as data not enough – Bosses distrust stuff where the outcome isn’t known @OptimiseOrDie
Agile UX / UCD / Collaborative Design “An integration of User Experience Design and Agile* Software Development Methodologies” *Sometimes Research Positive – User centric – Goals met substantially – Rapid time to market (especially when using Agile iterations) Negative – Without quant data, user goals can drive the show – missing the business sweet spot – Some people find it hard to integrate with siloed teams – Doesn’t’ work with waterfall IMHO Wireframe Concept Analyse Prototype Test @OptimiseOrDie
CRO @OptimiseOrDie
Lean Conversion Optimisation “A blend of User Experience Design, Agile PM, Rapid Lean UX Build-Measure-Learn cycles, triangulated data sources, triage and prioritisation.” Positive – A blend of several techniques – Multiple sources of Qual and Quant data aids triangulation – CRO analytics focus drives unearned value inside all products Negative – Needs a one team approach with a strong PM who is a Polymath (Commercial, Analytics, UX, Technical) – Only works if your teams can take the pace – you might be surprised though! @OptimiseOrDie
5 - Lean CRO Inspection Instrument Immersion Learn Identify Triage & Measure Triangulate Outcome Streams @OptimiseOrDie
#4 : Solutions – Agile • Design your own methodology Experiment and optimise with your team • Don’t be a slave The methodology is the slave, not your master • Ask me later… Questions – see me on Twitter, G+ or ask by mail • Collaborative working – Harvard study into teams – it‟s an „all the time’ thing @OptimiseOrDie
#5 : Build a volume testing culture @OptimiseOrDie
#5 : Common issues • How many tests do you complete a month? – Without volume, you won‟t get as much attention or traction • Vanity testing takes hold – Getting one test done a quarter? Still showing it a year later? • Not enough resource – You MUST hire, invest and ringfence time and staff for CRO • Testing has gone to sleep – Some vendors have a „rescue‟ team for these accounts • You keep testing without buyin at C-Level – If nobody sees the flower, was it there? • You haven’t got a process – just a plugin – Insight, Brainstorm, Wireframe, Design, Build, QA test, Monitor, Analyse. Tools, Process, People, Time -> INVEST • IT or release barriers slow down work – Circumvent with tagging tools – Develop ways around the innovation barrier @OptimiseOrDie
#5 : Testing Culture tips • Gamble in the office – Encourage betting in the office – Get people to make a prediction, get them hooked • Put BIG pictures on the wall – Make people stop and ask questions – Encourage betting in the office • Market the results like a pro – Market this stuff internally like a PR agency – Send emails around, put on the intranet, optimise internally • Offer a company wide prize – – – – – Offer a prize for best hypothesis that makes it to a test Do a monthly or quarterly award – make it visible A great incentive to come up with provable, winning ideas Do NOT limit to the testing team Run a TESTATHON to generate bulk ideas @OptimiseOrDie
#6 : Execute faster @OptimiseOrDie
#6 : Execution problems • Silo Mentality means pass the product – No „one team‟ approach means no „one product‟ • The process is badly designed – It‟s set up to introduce delays or loops of tweaking • People mistake hypotheses for finals – Endless argument, tweaking means NO TESTING – let the test decide, please! • No clarity : authority or decision making – You need a strong leader to get things decided • Signoff takes far too long – Signoff by committee is a velocity killer – the CUSTOMER and the NUMBERS are the signoff • You set your target too low – Aim for a high target and keep increasing it @OptimiseOrDie
#6 : Execution solutions • Agile, One Team approach – Everyone works on the lifecycle, together • Hire Polymaths – T-shaped or just multi-skilled, I hire them a lot • Use Collaborative Tools, not meetings – Speeds up time to decide, iterate, resolve, remove defects • Smash down silos – a special mission – – – – Involve the worst offenders in the hypothesis team “Hold your friends close, and your enemies closer” Work WITH the developers to find solutions Ask Developers and IT for solutions, not apologies @OptimiseOrDie
#6 : Product cycles are too long Conversion 0 6 12 18 Months @OptimiseOrDie
#6 : Solutions • Give Priority Boarding for opportunities – The best seats reserved for metric shifters • Release more often to close the gap – More testing resource helps, analytics „hawk eye‟ • Kaizen – continuous improvement – Others call it JFDI (just f***ing do it) • Make changes AS WELL as tests, basically! – These small things add up • RUSH Hair booking – Over 100 changes – No functional changes at all – 37% improvement • Inbetween product lifecycles? – The added lift for 10 days work, worth 360k @OptimiseOrDie
#6 : Make your own cycles @OptimiseOrDie
#7 – Get Experienced • Persuasion / Influence / Direction / Explanation • Helps people process information and stories • Vital to sell an „experience‟ • Helps people recognise and discriminate between things • Supports Scanning Visitors • Drives emotional response short.cx/YrBczl 24 Jan 2012
Photo UX • Very powerful and under-estimated area • I‟ve done over 20M visitor tests with people images for a service industry – some tips: • The person, pose, eye gaze, facial expressions and body language – cause visceral emotional reactions and big changes in behaviour • Eye gaze crucial – to engage you or to „point‟ 24 Jan 2012
Photo UX • Negative body language is a turnoff • Uniforms and branding a positive (ball cap) • Hands are hard to handle – use a prop to help • For Ecommerce – tip! test bigger images! • Autoglass and Belron always use real people • In most countries (out of 33) with strong female and male images in test, female wins • Smile and authenticity in these examples is absolutely vital • So, I have a question for you @OptimiseOrDie
@OptimiseOrDie
Terrible Stock Photos : headsethotties.com & awkwardstockphotos.com Laughing at Salads : womenlaughingwithsalad.tumblr.com BBC Fake Smile Test : bbc.in/5rtnv @OptimiseOrDie
SPAIN +22% over control 99% confidence @OptimiseOrDie
@OptimiseOrDie
#8 : Get Multichannel • Not using call tracking – Look at Infinity Tracking (UK) – Get Google keyword level call volumes! • You don’t measure channel switchers – People who bail a funnel and call – People who use chat, go to store, use ringback, email etc. • You ‘forget’ mobile & tablet journeys – Walk the path from search -> ppc/seo -> site – Optimise for all your device mix & journeys • You’re responsive – Testing may now bleed across device platforms – Changing in one place may impact many others – QA, Device and Browser testing even more vital @OptimiseOrDie
#8 : Multichannel – cross device • • • • • • • • • One reason conversion is lower is channel jumps You need to try and glue this together Don’t rely on amazing analytics promises If you don’t have a strategy here, you need one You need to think about identifying the customer At every touch point and micro or macro interaction Loyalty, marketing, wifi, incentives, geo, instore = glue What’s your strategy on post identification? Do you rewrite the data or just ignore it? @OptimiseOrDie
#9 : Get Segmented on Testing • Averages lie – What about new vs. returning visitors? – What about different keyword groups? – Landing pages? Routes? Attributes • Failed tests are just ‘averaged out’ – You must look at segment level data – You must integrate the analytics + a/b test software • The downside? – You‟ll need more test data – to segment • The upside? – Helps figure out why test didn‟t perform – Finds value in failed or „no difference‟ tests – Drives further testing focus @OptimiseOrDie
#10 : Get stats training • Many testers & marketing people struggle – – – – – How long will it take to run the test? Is the test ready? How long should I keep it running for? It says it‟s ready after 3 days – is it? Can we close it now – the numbers look great! • A/B testing maths for dummies: – http://bit.ly/15UXLS4 • For more advanced testers: – Read this : http://bit.ly/1a4iJ1H • A stats course would be handy: – A decent guide for testing or marketing pros – Watch this space… @OptimiseOrDie
#11 : Hire and retain great analysts… • • • • • • • • • • Long topic – mail me for killer questions Pay above market rates – 10k less is not ROI Don‟t make them into report monkeys Find a knowledge sharer and trainer! Let them solve business problems Invest in the analytics, people, continually Pay extra for specialist help Give staff direct outcome connections Freedom to grow and learn is vital Help them fix broken stuff! @OptimiseOrDie
SUMMARY : The best Companies…. • • • • • • • • • • • • See the Maturity Model in the resource pack Invest continually in analytics instrumentation, tools, people Use an Agile, iterative, cross-silo, one team project culture Prefer collaborative tools to having lots of meetings Prioritise development based on numbers and insight Practice real continuous product improvement, not SLED Source photos and content that support persuasion and utility Have cross channel, cross device design, testing and QA Segment their data for valuable insights, every test or change Continually reduce cycle (iteration) time in their process Blend ‘long’ design, continuous improvement AND split tests Make optimisation the engine of change, not the slave of ego @OptimiseOrDie
So you want examples? • • • • • • • Belron Dell Shop Direct Expedia Schuh TSR Group Soundcloud • Gov.uk – – – – – – – – Ed Colley Nazli Yuzak Paul Postance (now with EE) Oliver Paton Stuart McMillan Pete Taylor Eleftherios Diakomichalis & Ole Bahlmann Adam Bailin (now with the BBC) Read the gov.uk principles : www.gov.uk/designprinciples And my personal favourites of 2013 – Airbnb and Expensify #PRWDReveal
#12 : Do things that don‟t scale • World Economic Forum‟s Technology Pioneers 2014 • Zero to $3Bn in 5 years – servicing a Wembley every night @OptimiseOrDie • FREE Professional Photography
70 @OptimiseOrDie
71 @OptimiseOrDie
Do things that don‟t scale: “It‟s better to have 100 people love you experience than Create the perfect 1,000,000 need to do however you like you” it and Brian Chesky, CEO, experience. then scale thatAirbnb It‟s creating an experience. And then its multiplying them. Too many people start with „how many you sell‟ and then they try to make it better. Brian Chesky, CEO, Airbnb @OptimiseOrDie
If it isn‟t working, you‟re not doing it right @OptimiseOrDie
Optimise Or Die • Unless you have a dominant natural or physical advantage, the playing field is open to anyone. • Unfettered customer ratings and comments are here to stay : Think “Net Promoter Score on Steroids” • Market leaders like Nokia, Blackberry, Best Buy, Borders, The Washington Post and Blockbuster all got hit – why not you? • One investor asks new startups : “What if Amazon did your thing?” • Airbnb, Uber, Lyft, Lovefilm, Spotify and Google (Analytics) are excellent examples of innovation competition. • You must design and scale an amazing customer experience to win • If your competitor is doing this stuff, you‟ll have to spend MUCH more to beat them. @OptimiseOrDie
Conversion Heroes! Is there a way to fix this then? 75 @OptimiseOrDie
Download and freely use slides : slidesha.re/TeKiKr Email : sullivac@gmail.com Twitter : @OptimiseOrDie : linkd.in/pvrg14
RESOURCE PACK 77
RESOURCE PACK • • • • • Maturity model Triage and Triangulation The Bucket outcome methodology Belron methodology example CRO and testing resources 78
Level 1 Level 2 Starter Level Culture Early maturity Local Heroes Small team Low hanging fruit Chaotic Good Level 3 Serious testing Dedicated team Volume opportunities Level 4 Core business value Cross silo team Systematic tests Level 5 You rock, awesomely Ninja Team Testing in the DNA ________________________________________________________________________ Company wide Ad Hoc Well developed Streamlined Process _____________________ _ Outline process ________________________________________________________________________ +Spread tool use +Funnel +Cross channel _______________________+ Multi variate Guessing optimisation testing Testing focus A/B testing Basic tools Session replay No segments Call tracking Some segments Micro testing Integrated CRO and analytics Segmentation Dynamic adaptive targeting Machine learning Realtime ________________________________________________________________________ Multichannel + offline + Bounce rates + Funnel fixes ________________________Funnel analysis funnels integration Analytics Low converting focus Big volume landing pages & High loss pages Forms analytics Channel switches Single channel picture Cross channel synergy +User Centered Design Layered feedback Mini product tests scores tied to UX Rapid iterative testing and design of customer Driving offline using online All promotion driven by testing ________________________________________________________________________ ________________________ + All channel view +Regular usability Analytics + Customer sat Insight methods Surveys Contact Centre Low budget usability testing/research Prototyping Session replay Onsite feedback _________________________________________________________________________ Continual 79 Get buyin Scale the testing Mine value Mission _______________________ Prove ROI improvement
5 - Triage and Triangulation “This is where the smarts of CRO are – in identifying the easiest stuff to test or fix that will drive the largest uplift.” • • • • • • • • • • Starts with the analytics data Then UX and user journey walkthrough from SERPS -> key paths Then back to analytics data for a whole range of reports: Segmented reporting, Traffic sources, Device viewport and browser, Platform (tablet, mobile, desktop) and many more We use other tools or insight sources to help form hypotheses We triangulate with other data where possible We estimate the potential uplift of fixing/improving something as well as the difficulty (time/resource/complexity/risk) A simple quadrant shows the value clusters We then WORK the highest and easiest scores by… Turning every opportunity spotted into an OUTCOME 80
5 - The Bucket Methodology “Helps you to stream actions from the insights and prioritisation work. Forces an action for every issue, a counter for every opportunity being lost.”  Test  Instrument If there is an obvious opportunity to shift behaviour, expose insight or increase conversion – this bucket is where you place stuff for testing. If you have traffic and leakage, this is the bucket for that issue. If an issue is placed in this bucket, it means we need to beef up the analytics reporting. This can involve fixing, adding or improving tag or event handling on the analytics configuration. We instrument both structurally and for insight in the pain points we’ve found.  Hypothesise This is where we’ve found a page, widget or process that’s just not working well but we don’t see a clear single solution. Since we need to really shift the behaviour at this crux point, we’ll brainstorm hypotheses. Driven by evidence and data, we’ll create test plans to find the answers to the questions and change the conversion or KPI figure in the desired direction.  Just Do It JFDI (Just Do It) – is a bucket for issues where a fix is easy to identify or the change is a no-brainer. Items marked with this flag can either be deployed in a batch or as part of a controlled test. Stuff in here requires low effort or are micro-opportunities to increase conversion and should be fixed.  Investigate You need to do some testing with particular devices or need more information to triangulate a problem you spotted. If an item is in this bucket, you need to ask questions or do further digging. 81
5 - Belron example – Funnel replacement Final prototype Usability issues left Final changes Release build Legal review kickoff Instrument analytics Signoff (Legal, Mktng, CCC) Test Plan Marketing review Cust services review kickoff Instrument Contact Centre Offline tagging QA testing End-End testing Launch 90/10% Go live 100% Launch 50/50% Monitor < 1 week Launch 80/20% Monitor Analytics review Washup and actions New hypotheses New test design Rinse and Repeat!
6 - CRO and Testing resources • • • • • • • 101 Landing page tips 544 Optimisation tips 108 Optimisation tips 32 CRO tips 57 CRO books CRO article list Smashing Mag article : : : : : : : slidesha.re/8OnBRh bit.ly/8mkWOB bit.ly/3Z6GrP bit.ly/4BZjcW bit.ly/dDjDRJ bit.ly/nEUgui bit.ly/8X2fLk 83
END SLIDES Feel free to steal, re-use, appropriate or otherwise lift stuff from this deck. If it was useful to you – email me or tweet me and tell me why – I’d be DELIGHTED to hear! Cheers, Craig. 84

More Related Content

PPTX
Interact London - 21 Oct 2015 - Scaling Stupidity
Craig Sullivan
 
PPTX
Surviving the AB Testing Hype Cycle - Reaktor Breakpoint 2015
Craig Sullivan
 
PPTX
Surviving the hype cycle Shortcuts to split testing success
Craig Sullivan
 
PPTX
Web Analytics Wednesday - Session Replay Tools are Vital
Craig Sullivan
 
PPTX
Cross Device Optimisation - Google Analytics Shortcuts
Craig Sullivan
 
PPTX
20 top AB testing mistakes and how to avoid them
Craig Sullivan
 
PPTX
Conversion Research in One Hour
Craig Sullivan
 
PPTX
The Neuromarketing Toolkit - Chinwag Psych - 4 Feb 2014
Craig Sullivan
 
Interact London - 21 Oct 2015 - Scaling Stupidity
Craig Sullivan
 
Surviving the AB Testing Hype Cycle - Reaktor Breakpoint 2015
Craig Sullivan
 
Surviving the hype cycle Shortcuts to split testing success
Craig Sullivan
 
Web Analytics Wednesday - Session Replay Tools are Vital
Craig Sullivan
 
Cross Device Optimisation - Google Analytics Shortcuts
Craig Sullivan
 
20 top AB testing mistakes and how to avoid them
Craig Sullivan
 
Conversion Research in One Hour
Craig Sullivan
 
The Neuromarketing Toolkit - Chinwag Psych - 4 Feb 2014
Craig Sullivan
 

What's hot (20)

PPTX
Myths, Lies and Illusions of AB and Split Testing
Craig Sullivan
 
PPTX
#Measurefest : 20 Simple Ways to Fuck Up your AB tests
Craig Sullivan
 
PPTX
Mobile presentation - Sydney Online Retailer - 26 Sep 2011
Craig Sullivan
 
PPTX
Condensed testing syrup - @OptimiseorDie @sydney sep 2011 - 4 years of testin...
Craig Sullivan
 
PPTX
Myths and Illusions of Cross Device Testing - Elite Camp June 2015
Craig Sullivan
 
PPTX
#Measurecamp : 18 Simple Ways to F*** up Your AB Testing
Craig Sullivan
 
PPTX
Digital Impact 2014 - Oh Boy These AB tests sure look like Bullshit to me
Craig Sullivan
 
PPTX
Why do my AB tests suck? measurecamp
Craig Sullivan
 
PPTX
20 Ways to Shaft your Split Tesring : Conversion Conference
Craig Sullivan
 
PPTX
Brighton CRO Meetup #1 - Oh Boy These AB tests Sure Look Like Bullshit to Me
Craig Sullivan
 
PPTX
Product design is Poo - And how to fix it!
Craig Sullivan
 
PDF
Craig Sullivan - Oh Boy! These A/B tests look like total bullshit! MKTFEST 2014
Marketing Festival
 
PPTX
eMetrics London - The AB Testing Hype Cycle
Craig Sullivan
 
PPTX
Product Design is Poo - And we're all going to die
Craig Sullivan
 
PDF
Why Does My Conversion Rate Suck? Craig Sullivan, Senior Optimisation Consult...
PRWD
 
PPTX
Natural born conversion killers - Conversion Jam
Craig Sullivan
 
PDF
Onboard like a juggernaut - Elite camp 2015
Conversionista
 
PPTX
UXPA UK - Toolkits and Tips for Blending UX, Analytics and CRO
Craig Sullivan
 
PDF
Craig Sullivan - Keynote speaker summary & final thoughts - Conversion Hotel ...
Webanalisten .nl
 
PPTX
SAMPLE SIZE – The indispensable A/B test calculation that you’re not making
Zack Notes
 
Myths, Lies and Illusions of AB and Split Testing
Craig Sullivan
 
#Measurefest : 20 Simple Ways to Fuck Up your AB tests
Craig Sullivan
 
Mobile presentation - Sydney Online Retailer - 26 Sep 2011
Craig Sullivan
 
Condensed testing syrup - @OptimiseorDie @sydney sep 2011 - 4 years of testin...
Craig Sullivan
 
Myths and Illusions of Cross Device Testing - Elite Camp June 2015
Craig Sullivan
 
#Measurecamp : 18 Simple Ways to F*** up Your AB Testing
Craig Sullivan
 
Digital Impact 2014 - Oh Boy These AB tests sure look like Bullshit to me
Craig Sullivan
 
Why do my AB tests suck? measurecamp
Craig Sullivan
 
20 Ways to Shaft your Split Tesring : Conversion Conference
Craig Sullivan
 
Brighton CRO Meetup #1 - Oh Boy These AB tests Sure Look Like Bullshit to Me
Craig Sullivan
 
Product design is Poo - And how to fix it!
Craig Sullivan
 
Craig Sullivan - Oh Boy! These A/B tests look like total bullshit! MKTFEST 2014
Marketing Festival
 
eMetrics London - The AB Testing Hype Cycle
Craig Sullivan
 
Product Design is Poo - And we're all going to die
Craig Sullivan
 
Why Does My Conversion Rate Suck? Craig Sullivan, Senior Optimisation Consult...
PRWD
 
Natural born conversion killers - Conversion Jam
Craig Sullivan
 
Onboard like a juggernaut - Elite camp 2015
Conversionista
 
UXPA UK - Toolkits and Tips for Blending UX, Analytics and CRO
Craig Sullivan
 
Craig Sullivan - Keynote speaker summary & final thoughts - Conversion Hotel ...
Webanalisten .nl
 
SAMPLE SIZE – The indispensable A/B test calculation that you’re not making
Zack Notes
 
Ad

Similar to 12 Things to do Before Your Company Dies : Conversion Conference London - Oct 2013 (20)

PPTX
Toolkits and tips for UX analytics CRO by Craig Sullivan
UXPA UK
 
PPTX
Toolkits and tips of the conversion pros v 1.6
Craig Sullivan
 
PPTX
Confessions of an uber optimiser conversion summit - craig sullivan - v 1.9
Craig Sullivan
 
PPTX
3 Optimisation Decks : WAW Copenhagen - 27 Feb 2013
Craig Sullivan
 
PPTX
John lewis lego craig sullivan 1335
Charlie Lines
 
PPTX
Why does my Mobile Conversion rate suck? 19 Sep 2013 @ Conversion Thursday #...
Craig Sullivan
 
PDF
Conversion Hotel 2014: Craig Sullivan (UK) keynote
Webanalisten .nl
 
PDF
AB Testing and UX - a love story with numbers and people (by Craig Sullivan a...
Northern User Experience
 
PDF
What Your Customers Really Do Online: 5 Ways to Remove the Guesswork
Optimizely
 
PDF
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdf
VWO
 
PPTX
Measure camp tools of the cro rabble
Craig Sullivan
 
PDF
Conversion Rate Optimization Master Class - Marcus Cooke, Space Between
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
 
PPTX
Conversionista : Conversion manager course - Stockholm 20 march 2013
Craig Sullivan
 
PPTX
Lean UX and Optimisation - Userzoom : 24 jan 2012 - lean optimisation
Craig Sullivan
 
PDF
Conversion Optimisation
Lar Veale
 
PPTX
Elite Camp 2013 - Estonia
Craig Sullivan
 
PPTX
Emetrics - Oct 19 2011 - New York - X channel optimisation
Craig Sullivan
 
PDF
Conversion Rate Optimization Master Class - Richard Fenner, Space Between
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
 
PDF
Outperform Webinar Series: How to Optimize Your Checkout Flow & Engage Your C...
Optimizely
 
PPT
Designing Outcomes For Usability Nycupa Hurst Final
Marko Hurst
 
Toolkits and tips for UX analytics CRO by Craig Sullivan
UXPA UK
 
Toolkits and tips of the conversion pros v 1.6
Craig Sullivan
 
Confessions of an uber optimiser conversion summit - craig sullivan - v 1.9
Craig Sullivan
 
3 Optimisation Decks : WAW Copenhagen - 27 Feb 2013
Craig Sullivan
 
John lewis lego craig sullivan 1335
Charlie Lines
 
Why does my Mobile Conversion rate suck? 19 Sep 2013 @ Conversion Thursday #...
Craig Sullivan
 
Conversion Hotel 2014: Craig Sullivan (UK) keynote
Webanalisten .nl
 
AB Testing and UX - a love story with numbers and people (by Craig Sullivan a...
Northern User Experience
 
What Your Customers Really Do Online: 5 Ways to Remove the Guesswork
Optimizely
 
Research and Discovery Tools for Experimentation - 17 Apr 2024 - v 2.3 (1).pdf
VWO
 
Measure camp tools of the cro rabble
Craig Sullivan
 
Conversion Rate Optimization Master Class - Marcus Cooke, Space Between
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
 
Conversionista : Conversion manager course - Stockholm 20 march 2013
Craig Sullivan
 
Lean UX and Optimisation - Userzoom : 24 jan 2012 - lean optimisation
Craig Sullivan
 
Conversion Optimisation
Lar Veale
 
Elite Camp 2013 - Estonia
Craig Sullivan
 
Emetrics - Oct 19 2011 - New York - X channel optimisation
Craig Sullivan
 
Conversion Rate Optimization Master Class - Richard Fenner, Space Between
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
 
Outperform Webinar Series: How to Optimize Your Checkout Flow & Engage Your C...
Optimizely
 
Designing Outcomes For Usability Nycupa Hurst Final
Marko Hurst
 
Ad

Recently uploaded (20)

PDF
Sensors and Actuators in IoT Systems using pdf
jeffinmathew654
 
PDF
Machine learning based COVID-19 study performance prediction
IAESIJAI
 
PDF
Advanced Soft Computing BINUS July 2025.pdf
University of Hertfordshire
 
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
ReZa AdineH
 
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
PDF
GDG Cloud Iasi [PUBLIC] Florian Blaga - Unveiling the Evolution of Cybersecur...
athlonica
 
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
blackmambaettijean
 
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
PDF
Review of recent advances in non-invasive hemoglobin estimation
IAESIJAI
 
PPTX
Understanding_Digital_Forensics_Presentation.pptx
ImranKhan423233
 
PDF
NewMind AI Monthly Chronicles - July 2025
NewMind AI
 
PPT
Teaching material agriculture food technology
LiaRayya
 
PDF
Transforming Manufacturing operations through Intelligent Integrations
TGH Software Solutions Pvt. Ltd.
 
PDF
solutions_manual_-_materials___processing_in_manufacturing__demargo_.pdf
AbdullahSani29
 
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Peter Tan
 
DOCX
The AUB Centre for AI in Media Proposal.docx
GAONE9
 
PPTX
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
AVTRON Technologies LLC
 
PDF
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
Safe Software
 
PDF
Network Security Unit 5.pdf for BCA BBA.
Serpent6
 
PDF
The Rise and Fall of 3GPP – Time for a Sabbatical?
3G4G
 
Sensors and Actuators in IoT Systems using pdf
jeffinmathew654
 
Machine learning based COVID-19 study performance prediction
IAESIJAI
 
Advanced Soft Computing BINUS July 2025.pdf
University of Hertfordshire
 
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
ReZa AdineH
 
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
GDG Cloud Iasi [PUBLIC] Florian Blaga - Unveiling the Evolution of Cybersecur...
athlonica
 
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
blackmambaettijean
 
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
Review of recent advances in non-invasive hemoglobin estimation
IAESIJAI
 
Understanding_Digital_Forensics_Presentation.pptx
ImranKhan423233
 
NewMind AI Monthly Chronicles - July 2025
NewMind AI
 
Teaching material agriculture food technology
LiaRayya
 
Transforming Manufacturing operations through Intelligent Integrations
TGH Software Solutions Pvt. Ltd.
 
solutions_manual_-_materials___processing_in_manufacturing__demargo_.pdf
AbdullahSani29
 
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Peter Tan
 
The AUB Centre for AI in Media Proposal.docx
GAONE9
 
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
AVTRON Technologies LLC
 
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
Safe Software
 
Network Security Unit 5.pdf for BCA BBA.
Serpent6
 
The Rise and Fall of 3GPP – Time for a Sabbatical?
3G4G
 

12 Things to do Before Your Company Dies : Conversion Conference London - Oct 2013

  • 1. 12 things to do before your company dies 23rd Oct 2013 @OptimiseOrDie
  • 2. Timeline + Tweet me! - 1998 1999 - 2004 2004-2008 2008-2012 @OptimiseOrDie
  • 3. SEO PPC UX Analytics Design Development QA Performance A/B and Multivariate testing Customer Satisfaction 40+ websites, 34 countries, 19 languages, €1bn+ revenue 8 people @OptimiseOrDie
  • 4. If you‟re not a part of the solution, there‟s good money to be made in prolonging the problem
  • 5. Out of my comfort zone… @OptimiseOrDie
  • 7. Nice day at the office, dear? @OptimiseOrDie
  • 12. #1 : Fix your broken Analytics! @OptimiseOrDie
  • 13. #1 : Fix your broken Analytics! @OptimiseOrDie
  • 14. #1 : Common problems (GA) • Dual purpose goal page – One page used by two outcomes – and not split • Cross domain tracking – Where you jump between sites, this borks the data • Filters not correctly set up – Your office, agencies, developers are skewing data • Code missing or double code – Causes visit splitting, double pageviews, skews bounce rate • Campaign, Social, Email tracking etc. – External links you generate are not setup to record properly • Errors not tracked (404, 5xx, Other) – You are unaware of error volumes, locations and impact • Dual flow funnels – Flows join in the middle of a funnel or loop internally • Event tracking skews bounce rate – If an event is set to be „interactive‟ – it can skew bounce rate (example) @OptimiseOrDie
  • 15. #1 : Solutions • Get a Health Check for your Analytics – Try @prwd, @danbarker, @peter_oneill or ask me! • Invest continually in instrumentation – Aim for at least 5% of dev time to fix + improve • Stop shrugging : plug your insight gaps – Change „I don‟t know‟ to „I‟ll find out‟ • Look at event tracking (Google Analytics) – If set up correctly, you get wonderful insights • Would you use paper instead of a till? – You wouldn‟t do it in retail so stop doing it online! • How do you win F1 races? – With the wrong performance data, you won‟t @OptimiseOrDie
  • 16. #2 : Get the right inputs Insight - Inputs Opinion Cherished notions Marketing whims Cosmic rays Not ‘on brand’ enough Ego IT inflexibility Panic Internal company needs #FAIL Competitor change An article the CEO read Some dumbass consultant Competitor copying Dice rolling Guessing Knee jerk reactons Shiny feature blindness @OptimiseOrDie
  • 17. #2 : Get the right inputs Insight - Inputs Usability testing Forms analytics Search analytics Voice of Customer Market research Eye tracking Customer contact A/B and MVT testing Big & unstructured data Insight Social analytics Session Replay Web analytics Segmentation Sales and Call Centre Surveys Customer services Competitor evals @OptimiseOrDie
  • 18. #2 : Solutions • Usability testing and User Centred design – If you‟re not doing this properly, you‟re hosed • Champion UX+ - with added numbers – (Re)designing without inputs + numbers is guessing • You need one team on this, not silos – Stop handing round the baby (I‟ll come back to this) • Ego, Opinion, Cherished notions – fill gaps – Fill these vacuums with insights and data • Champion the users – Someone needs to take their side! • You need multiple tool inputs – Let me show you my shortlist… @OptimiseOrDie
  • 19. #3 : Get the right tools Insight - Inputs @OptimiseOrDie
  • 20. 3.1 - Session Replay • • • • • Vital for optimisers & fills in a ‘missing link’ for insight Rich source of data on visitor experiences Segment by browser, visitor type, behaviour, errors Forms Analytics (when instrumented) are awesome Can be used to optimise in real time! Session replay tools • Clicktale (Client) • SessionCam (Client) • Mouseflow (Client) • Ghostrec (Client) • Usabilla (Client) • Tealeaf (Hybrid) • UserReplay (Server) www.clicktale.com www.sessioncam.com www.mouseflow.com www.ghostrec.com www.usabilla.com www.tealeaf.com www.userreplay.com @OptimiseOrDie
  • 21. 21
  • 22. 22
  • 23. 3.2 - Feedback / VOC tools • • • • Anything that allows immediate realtime onpage feedback Comments on elements, pages and overall site & service Can be used for behavioural triggered feedback Tip! : Take the Call Centre for beers • Kampyle www.kampyle.com • Qualaroo www.qualaroo.com • 4Q 4q.iperceptions.com • Usabilla www.usabilla.com
  • 24. 3.3 - Survey Tools • Surveymonkey • Zoomerang • SurveyGizmo www.surveymonkey.com www.zoomerang.com www.surveygizmo.com (1/5) (3/5) (5/5) • For surveys, web forms, checkouts, lead gen – anything with form filling – you have to read these two: Caroline Jarrett (@cjforms) Luke Wroblewski (@lukew) • With their work and copywriting from @stickycontent, I managed to get a survey with a 35% clickthrough from email and a whopping 94% form completion rate. • Their awesome insights are the killer app I have when optimising forms and funnel processes for clients. @OptimiseOrDie
  • 25. 3.4 - UX Crowd tools Som, feedback Remote UX tools (P=Panel, S=Site recruited, B=Both) Usertesting (B) www.usertesting.com Userlytics (B) www.userlytics.com Userzoom (S) www.userzoom.com Intuition HQ (S) www.intuitionhq.com Mechanical turk (S) www.mechanicalturk.com Loop11 (S) www.loop11.com Open Hallway (S) www.openhallway.com What Users Do (P) www.whatusersdo.com Feedback army (P) www.feedbackarmy.com User feel (P) www.userfeel.com Ethnio (For Recruiting) www.ethnio.com Feedback on Prototypes / Mockups Pidoco Verify from Zurb Five second test Conceptshare Usabilla www.pidoco.com www.verifyapp.com www.fivesecondtest.com www.conceptshare.com www.usabilla.com 25
  • 26. 3.5 – Usability Recording / Testing • Some low budget testing tools: CamStudio (free) www.camstudio.org Mediacam AV (cheap) www.netu2.co.uk Silverback (Mac) www.silverbackapp.com Screenflow (Mac) www.telestream.net UX Recorder (iOS), Skype Hugging, Reflection www.uxrecorder.com & bit.ly/tesTfm & bit.ly/GZMgxR @OptimiseOrDie
  • 27. 3.5 – Test or Die! Email testing www.litmus.com www.returnpath.com www.lyris.com Browser testing www.crossbrowsertesting.com www.cloudtesting.com www.multibrowserviewer.com www.saucelabs.com Mobile devices www.perfectomobile.com www.deviceanywhere.com www.mobilexweb.com/emulators www.opendevicelab.com @OptimiseOrDie
  • 28. 3.6 – Collaboration tools Oh sh*t @OptimiseOrDie
  • 29. 3.6 – Basecamp (Project management) @OptimiseOrDie
  • 30. 3.6 – Join.me (free screen sharer)
  • 31. 3.6 – Trello (task & project) @OptimiseOrDie
  • 32. 3.6 – Pivotal Tracker (Agile, Lean) @OptimiseOrDie
  • 33. 3.6 – Cloud Collaboration (charting) • LucidChart @OptimiseOrDie
  • 34. 3.6 – Cloud Collaboration (feedback) • Webnotes & Protonotes @OptimiseOrDie
  • 35. 3.6 – Cloud Collaboration (feedback) • Conceptshare @OptimiseOrDie
  • 36. 3.7 – Split testing tools – Cheap! • Google Content Experiments bit.ly/Ljg7Ds • Multi Armed Bandit Explanation bit.ly/Xa80O8 • Optimizely www.optimizely.com • Visual Website Optimizer www.visualwebsiteoptimizer.com @OptimiseOrDie
  • 37. 3.7 – Split test funnies • Many tests fail due to QA or browser bugs – Always do cross browser QA testing – see resources • Don’t rely on developers saying ‘yes’ – Use your analytics to define the list to test • Cross instrument your analytics – You need this to check the test software works • Store the variant(s) seen in analytics – Compare people who saw A/B/A vs. A/B/B • Segment your data to find variances – Failed tests usually show differences for segments • Watch the test and analytics CLOSELY – After you go live, religiously check both – Read this article : stanford.io/15UYov0 @OptimiseOrDie
  • 38. #3 : Summary – the minimum! • Session replay tools – Clicktale, Tealeaf, Sessioncam and more… • Cheap / Crowdsourced usability testing – See the list in this deck • Voice of Customer / Feedback tools – 4Q, Kampyle, Qualaroo, Usabilla and more… • A/B and Multivariate testing – Optimizely, Google Content Experiments, VWO • Email, Browser and Mobile Device Testing – You don‟t know if it works unless you check @OptimiseOrDie
  • 39. #4 : Go Agile @OptimiseOrDie
  • 40. Methodologies - Lean UX “The application of UX design methods into product development, tailored to fit Build-Measure-Learn cycles.” Positive – – – – – – Lightweight and very fast methods Realtime or rapid improvements Documentation light, value high Low on wastage and frippery Fast time to market, then optimise Allows you to pivot into new areas Negative – Often needs user test feedback to steer the development, as data not enough – Bosses distrust stuff where the outcome isn’t known @OptimiseOrDie
  • 41. Agile UX / UCD / Collaborative Design “An integration of User Experience Design and Agile* Software Development Methodologies” *Sometimes Research Positive – User centric – Goals met substantially – Rapid time to market (especially when using Agile iterations) Negative – Without quant data, user goals can drive the show – missing the business sweet spot – Some people find it hard to integrate with siloed teams – Doesn’t’ work with waterfall IMHO Wireframe Concept Analyse Prototype Test @OptimiseOrDie
  • 43. Lean Conversion Optimisation “A blend of User Experience Design, Agile PM, Rapid Lean UX Build-Measure-Learn cycles, triangulated data sources, triage and prioritisation.” Positive – A blend of several techniques – Multiple sources of Qual and Quant data aids triangulation – CRO analytics focus drives unearned value inside all products Negative – Needs a one team approach with a strong PM who is a Polymath (Commercial, Analytics, UX, Technical) – Only works if your teams can take the pace – you might be surprised though! @OptimiseOrDie
  • 44. 5 - Lean CRO Inspection Instrument Immersion Learn Identify Triage & Measure Triangulate Outcome Streams @OptimiseOrDie
  • 45. #4 : Solutions – Agile • Design your own methodology Experiment and optimise with your team • Don’t be a slave The methodology is the slave, not your master • Ask me later… Questions – see me on Twitter, G+ or ask by mail • Collaborative working – Harvard study into teams – it‟s an „all the time’ thing @OptimiseOrDie
  • 46. #5 : Build a volume testing culture @OptimiseOrDie
  • 47. #5 : Common issues • How many tests do you complete a month? – Without volume, you won‟t get as much attention or traction • Vanity testing takes hold – Getting one test done a quarter? Still showing it a year later? • Not enough resource – You MUST hire, invest and ringfence time and staff for CRO • Testing has gone to sleep – Some vendors have a „rescue‟ team for these accounts • You keep testing without buyin at C-Level – If nobody sees the flower, was it there? • You haven’t got a process – just a plugin – Insight, Brainstorm, Wireframe, Design, Build, QA test, Monitor, Analyse. Tools, Process, People, Time -> INVEST • IT or release barriers slow down work – Circumvent with tagging tools – Develop ways around the innovation barrier @OptimiseOrDie
  • 48. #5 : Testing Culture tips • Gamble in the office – Encourage betting in the office – Get people to make a prediction, get them hooked • Put BIG pictures on the wall – Make people stop and ask questions – Encourage betting in the office • Market the results like a pro – Market this stuff internally like a PR agency – Send emails around, put on the intranet, optimise internally • Offer a company wide prize – – – – – Offer a prize for best hypothesis that makes it to a test Do a monthly or quarterly award – make it visible A great incentive to come up with provable, winning ideas Do NOT limit to the testing team Run a TESTATHON to generate bulk ideas @OptimiseOrDie
  • 49. #6 : Execute faster @OptimiseOrDie
  • 50. #6 : Execution problems • Silo Mentality means pass the product – No „one team‟ approach means no „one product‟ • The process is badly designed – It‟s set up to introduce delays or loops of tweaking • People mistake hypotheses for finals – Endless argument, tweaking means NO TESTING – let the test decide, please! • No clarity : authority or decision making – You need a strong leader to get things decided • Signoff takes far too long – Signoff by committee is a velocity killer – the CUSTOMER and the NUMBERS are the signoff • You set your target too low – Aim for a high target and keep increasing it @OptimiseOrDie
  • 51. #6 : Execution solutions • Agile, One Team approach – Everyone works on the lifecycle, together • Hire Polymaths – T-shaped or just multi-skilled, I hire them a lot • Use Collaborative Tools, not meetings – Speeds up time to decide, iterate, resolve, remove defects • Smash down silos – a special mission – – – – Involve the worst offenders in the hypothesis team “Hold your friends close, and your enemies closer” Work WITH the developers to find solutions Ask Developers and IT for solutions, not apologies @OptimiseOrDie
  • 52. #6 : Product cycles are too long Conversion 0 6 12 18 Months @OptimiseOrDie
  • 53. #6 : Solutions • Give Priority Boarding for opportunities – The best seats reserved for metric shifters • Release more often to close the gap – More testing resource helps, analytics „hawk eye‟ • Kaizen – continuous improvement – Others call it JFDI (just f***ing do it) • Make changes AS WELL as tests, basically! – These small things add up • RUSH Hair booking – Over 100 changes – No functional changes at all – 37% improvement • Inbetween product lifecycles? – The added lift for 10 days work, worth 360k @OptimiseOrDie
  • 54. #6 : Make your own cycles @OptimiseOrDie
  • 55. #7 – Get Experienced • Persuasion / Influence / Direction / Explanation • Helps people process information and stories • Vital to sell an „experience‟ • Helps people recognise and discriminate between things • Supports Scanning Visitors • Drives emotional response short.cx/YrBczl 24 Jan 2012
  • 56. Photo UX • Very powerful and under-estimated area • I‟ve done over 20M visitor tests with people images for a service industry – some tips: • The person, pose, eye gaze, facial expressions and body language – cause visceral emotional reactions and big changes in behaviour • Eye gaze crucial – to engage you or to „point‟ 24 Jan 2012
  • 57. Photo UX • Negative body language is a turnoff • Uniforms and branding a positive (ball cap) • Hands are hard to handle – use a prop to help • For Ecommerce – tip! test bigger images! • Autoglass and Belron always use real people • In most countries (out of 33) with strong female and male images in test, female wins • Smile and authenticity in these examples is absolutely vital • So, I have a question for you @OptimiseOrDie
  • 59. Terrible Stock Photos : headsethotties.com & awkwardstockphotos.com Laughing at Salads : womenlaughingwithsalad.tumblr.com BBC Fake Smile Test : bbc.in/5rtnv @OptimiseOrDie
  • 60. SPAIN +22% over control 99% confidence @OptimiseOrDie
  • 62. #8 : Get Multichannel • Not using call tracking – Look at Infinity Tracking (UK) – Get Google keyword level call volumes! • You don’t measure channel switchers – People who bail a funnel and call – People who use chat, go to store, use ringback, email etc. • You ‘forget’ mobile & tablet journeys – Walk the path from search -> ppc/seo -> site – Optimise for all your device mix & journeys • You’re responsive – Testing may now bleed across device platforms – Changing in one place may impact many others – QA, Device and Browser testing even more vital @OptimiseOrDie
  • 63. #8 : Multichannel – cross device • • • • • • • • • One reason conversion is lower is channel jumps You need to try and glue this together Don’t rely on amazing analytics promises If you don’t have a strategy here, you need one You need to think about identifying the customer At every touch point and micro or macro interaction Loyalty, marketing, wifi, incentives, geo, instore = glue What’s your strategy on post identification? Do you rewrite the data or just ignore it? @OptimiseOrDie
  • 64. #9 : Get Segmented on Testing • Averages lie – What about new vs. returning visitors? – What about different keyword groups? – Landing pages? Routes? Attributes • Failed tests are just ‘averaged out’ – You must look at segment level data – You must integrate the analytics + a/b test software • The downside? – You‟ll need more test data – to segment • The upside? – Helps figure out why test didn‟t perform – Finds value in failed or „no difference‟ tests – Drives further testing focus @OptimiseOrDie
  • 65. #10 : Get stats training • Many testers & marketing people struggle – – – – – How long will it take to run the test? Is the test ready? How long should I keep it running for? It says it‟s ready after 3 days – is it? Can we close it now – the numbers look great! • A/B testing maths for dummies: – http://bit.ly/15UXLS4 • For more advanced testers: – Read this : http://bit.ly/1a4iJ1H • A stats course would be handy: – A decent guide for testing or marketing pros – Watch this space… @OptimiseOrDie
  • 66. #11 : Hire and retain great analysts… • • • • • • • • • • Long topic – mail me for killer questions Pay above market rates – 10k less is not ROI Don‟t make them into report monkeys Find a knowledge sharer and trainer! Let them solve business problems Invest in the analytics, people, continually Pay extra for specialist help Give staff direct outcome connections Freedom to grow and learn is vital Help them fix broken stuff! @OptimiseOrDie
  • 67. SUMMARY : The best Companies…. • • • • • • • • • • • • See the Maturity Model in the resource pack Invest continually in analytics instrumentation, tools, people Use an Agile, iterative, cross-silo, one team project culture Prefer collaborative tools to having lots of meetings Prioritise development based on numbers and insight Practice real continuous product improvement, not SLED Source photos and content that support persuasion and utility Have cross channel, cross device design, testing and QA Segment their data for valuable insights, every test or change Continually reduce cycle (iteration) time in their process Blend ‘long’ design, continuous improvement AND split tests Make optimisation the engine of change, not the slave of ego @OptimiseOrDie
  • 68. So you want examples? • • • • • • • Belron Dell Shop Direct Expedia Schuh TSR Group Soundcloud • Gov.uk – – – – – – – – Ed Colley Nazli Yuzak Paul Postance (now with EE) Oliver Paton Stuart McMillan Pete Taylor Eleftherios Diakomichalis & Ole Bahlmann Adam Bailin (now with the BBC) Read the gov.uk principles : www.gov.uk/designprinciples And my personal favourites of 2013 – Airbnb and Expensify #PRWDReveal
  • 69. #12 : Do things that don‟t scale • World Economic Forum‟s Technology Pioneers 2014 • Zero to $3Bn in 5 years – servicing a Wembley every night @OptimiseOrDie • FREE Professional Photography
  • 72. Do things that don‟t scale: “It‟s better to have 100 people love you experience than Create the perfect 1,000,000 need to do however you like you” it and Brian Chesky, CEO, experience. then scale thatAirbnb It‟s creating an experience. And then its multiplying them. Too many people start with „how many you sell‟ and then they try to make it better. Brian Chesky, CEO, Airbnb @OptimiseOrDie
  • 73. If it isn‟t working, you‟re not doing it right @OptimiseOrDie
  • 74. Optimise Or Die • Unless you have a dominant natural or physical advantage, the playing field is open to anyone. • Unfettered customer ratings and comments are here to stay : Think “Net Promoter Score on Steroids” • Market leaders like Nokia, Blackberry, Best Buy, Borders, The Washington Post and Blockbuster all got hit – why not you? • One investor asks new startups : “What if Amazon did your thing?” • Airbnb, Uber, Lyft, Lovefilm, Spotify and Google (Analytics) are excellent examples of innovation competition. • You must design and scale an amazing customer experience to win • If your competitor is doing this stuff, you‟ll have to spend MUCH more to beat them. @OptimiseOrDie
  • 75. Conversion Heroes! Is there a way to fix this then? 75 @OptimiseOrDie
  • 76. Download and freely use slides : slidesha.re/TeKiKr Email : [email protected] Twitter : @OptimiseOrDie : linkd.in/pvrg14
  • 78. RESOURCE PACK • • • • • Maturity model Triage and Triangulation The Bucket outcome methodology Belron methodology example CRO and testing resources 78
  • 79. Level 1 Level 2 Starter Level Culture Early maturity Local Heroes Small team Low hanging fruit Chaotic Good Level 3 Serious testing Dedicated team Volume opportunities Level 4 Core business value Cross silo team Systematic tests Level 5 You rock, awesomely Ninja Team Testing in the DNA ________________________________________________________________________ Company wide Ad Hoc Well developed Streamlined Process _____________________ _ Outline process ________________________________________________________________________ +Spread tool use +Funnel +Cross channel _______________________+ Multi variate Guessing optimisation testing Testing focus A/B testing Basic tools Session replay No segments Call tracking Some segments Micro testing Integrated CRO and analytics Segmentation Dynamic adaptive targeting Machine learning Realtime ________________________________________________________________________ Multichannel + offline + Bounce rates + Funnel fixes ________________________Funnel analysis funnels integration Analytics Low converting focus Big volume landing pages & High loss pages Forms analytics Channel switches Single channel picture Cross channel synergy +User Centered Design Layered feedback Mini product tests scores tied to UX Rapid iterative testing and design of customer Driving offline using online All promotion driven by testing ________________________________________________________________________ ________________________ + All channel view +Regular usability Analytics + Customer sat Insight methods Surveys Contact Centre Low budget usability testing/research Prototyping Session replay Onsite feedback _________________________________________________________________________ Continual 79 Get buyin Scale the testing Mine value Mission _______________________ Prove ROI improvement
  • 80. 5 - Triage and Triangulation “This is where the smarts of CRO are – in identifying the easiest stuff to test or fix that will drive the largest uplift.” • • • • • • • • • • Starts with the analytics data Then UX and user journey walkthrough from SERPS -> key paths Then back to analytics data for a whole range of reports: Segmented reporting, Traffic sources, Device viewport and browser, Platform (tablet, mobile, desktop) and many more We use other tools or insight sources to help form hypotheses We triangulate with other data where possible We estimate the potential uplift of fixing/improving something as well as the difficulty (time/resource/complexity/risk) A simple quadrant shows the value clusters We then WORK the highest and easiest scores by… Turning every opportunity spotted into an OUTCOME 80
  • 81. 5 - The Bucket Methodology “Helps you to stream actions from the insights and prioritisation work. Forces an action for every issue, a counter for every opportunity being lost.”  Test  Instrument If there is an obvious opportunity to shift behaviour, expose insight or increase conversion – this bucket is where you place stuff for testing. If you have traffic and leakage, this is the bucket for that issue. If an issue is placed in this bucket, it means we need to beef up the analytics reporting. This can involve fixing, adding or improving tag or event handling on the analytics configuration. We instrument both structurally and for insight in the pain points we’ve found.  Hypothesise This is where we’ve found a page, widget or process that’s just not working well but we don’t see a clear single solution. Since we need to really shift the behaviour at this crux point, we’ll brainstorm hypotheses. Driven by evidence and data, we’ll create test plans to find the answers to the questions and change the conversion or KPI figure in the desired direction.  Just Do It JFDI (Just Do It) – is a bucket for issues where a fix is easy to identify or the change is a no-brainer. Items marked with this flag can either be deployed in a batch or as part of a controlled test. Stuff in here requires low effort or are micro-opportunities to increase conversion and should be fixed.  Investigate You need to do some testing with particular devices or need more information to triangulate a problem you spotted. If an item is in this bucket, you need to ask questions or do further digging. 81
  • 82. 5 - Belron example – Funnel replacement Final prototype Usability issues left Final changes Release build Legal review kickoff Instrument analytics Signoff (Legal, Mktng, CCC) Test Plan Marketing review Cust services review kickoff Instrument Contact Centre Offline tagging QA testing End-End testing Launch 90/10% Go live 100% Launch 50/50% Monitor < 1 week Launch 80/20% Monitor Analytics review Washup and actions New hypotheses New test design Rinse and Repeat!
  • 83. 6 - CRO and Testing resources • • • • • • • 101 Landing page tips 544 Optimisation tips 108 Optimisation tips 32 CRO tips 57 CRO books CRO article list Smashing Mag article : : : : : : : slidesha.re/8OnBRh bit.ly/8mkWOB bit.ly/3Z6GrP bit.ly/4BZjcW bit.ly/dDjDRJ bit.ly/nEUgui bit.ly/8X2fLk 83
  • 84. END SLIDES Feel free to steal, re-use, appropriate or otherwise lift stuff from this deck. If it was useful to you – email me or tweet me and tell me why – I’d be DELIGHTED to hear! Cheers, Craig. 84

Editor's Notes

  • #30: “A piece of paper with your design mockup. A customer in a shop or bookstore. Their finger is their mouse, the paper their screen. Where would they click? Do they know what these labels mean? Do they see the major routes out of the page? Any barriers.Congratulations, you just got feedback on your design, before writing a single freaking line of code or asking your developers to keep changing stuff.”
  • #56: This stuff is important. What do photographs do?Well they help me persuade people, influence their thinking, give them directions or cues and explain things – this is the scanning generation!And they’re very powerful when selling experiences, stories or using the power of social proofThey help people very quickly (more quickly than reading) discriminate, evaluate – work out what stuff is, how it’s organized, what the things are, what’s being shown to you.And most importantly, they drive emotional response in people. Whether you like being soggy, wet and without toilet paper for a 30 mile radius or not, a picture like this gets a RESPONSE! Work it!Lastly, a shout out to James Chudley, who’s book this example comes from.
  • #57: So this is quite a powerful area – what about people images?I’ve tested quite a few of these – in over 20 countries and over 15 languages. What did I find?Well - the person, pose, eye gaze, facial expressions and body language – cause visceral emotional reactions and big changes in behaviour. The difference between a crappy image and one optimised to get the right response is huge.And one interesting thing Eye gaze is pretty crucial – to engage you, the viewer, or to ‘point’ or draw eye gaze and attention to a product. I’ve tested all angles of viewing and in these people images, the best view is straight at the viewer or slightly away. Any further and the conversion rate drops. It makes a difference I can count.
  • #64: Add unique phone numbers to all your mobile sites and apps. That’s for starters.Then configure your analytics to collect data when people Click or Tap to make a phone call.Make sure you add other events like ringbacks, email, chat – any web forms or lead gen activity too.
  • #76: Tomorrow - Go forth and kick their flabby low converting asses