Iterate. Calculate. Communicate.

Oct 24, 20161 like393 views

This document discusses how to close the gap between measuring product metrics and learning from those metrics through an iterative process. It emphasizes establishing the right metrics to track, ensuring metrics are actionable by tying them to goals that change behaviors, and communicating metrics and plans to take action on results across teams. The document provides examples of metrics to track at different stages of product development and recommends establishing a single source of truth for metrics as well as communication checklists.

Linked In: /in/xplusx
Medium & Twitter: @laurex
LAURE PARSONS
SENIOR PRODUCT MANAGER

“THE GOAL OF BUILD-MEASURE-LEARN IS NOT
TO BUILD A FINAL PRODUCT TO SHIP OR EVEN
TO BUILD A PROTOTYPE OF A PRODUCT, BUT TO
MAXIMIZE LEARNING THROUGH INCREMENTAL
AND ITERATIVE ENGINEERING”
Steve Blank, author of The Four Steps to the Epiphany
BE ACTIONABLE!

1. WHAT SHOULD YOU TRACK?
2. WHAT MAKES A METRIC “GOOD?”
3. HOW CAN YOU GET ALIGNMENT WITH
YOUR TEAM AND LEADERSHIP?
4. WHAT SHOULD YOU DO TO ENSURE
YOU LEARN?
‣ CASE STUDY: BUILDING A NEW FEATURE!

ACQUISITION
ACTIVATION
REFERRAL
RETENTION
REVENUE
AARRR: PIRATE METRICS

ACQUISITION
ACTIVATION
REFERRAL
RETENTION
REVENUE
TEAM
AARRRT: TEAM METRICS

THE ONE METRIC
From Lean Analytics by Benjamin Yoskovitz & Alistair Croll

UX TESTS
MVP
A/B TESTSCLASSIC LEAN MEASUREMENT

WHO, WHAT, WHEN, WHERE, WHY & HOW
▸WHO CUSTOMERS ARE
▸WHAT CUSTOMERS DO AND/OR TRY TO DO
▸WHEN DO USERS DISENGAGE
▸WHERE CUSTOMERS COME FROM
▸WHY CUSTOMERS CONVERT
▸HOW MUCH YOU CAN BUILD

BE ACTIONABLE!
TWO TYPES OF ACTIONABLE DATA
▸“Accounting” metrics can make your
predictions more accurate
▸“Experimental” metrics help
optimize product, pricing, or market.
Agree on what will change before
you collect the data.

“IF YOU MEASURE SOMETHING AND IT’S
NOT ATTACHED TO A GOAL, IN TURN
CHANGING YOUR BEHAVIOUR, YOU’RE
WASTING YOUR TIME.”
LEAN ANALYTICS by Ben Horowitz and Alistair Kroll
BE ACTIONABLE!

THE DATA TOOL YOUR TEAM
WILL ACTUALLY USE

‣A single source of truth
‣Company-wide common understanding and deﬁnitions
‣Appropriate permissions
‣Transparency
‣Collaboration via shared data and commenting
‣Automated integration from your tools
‣Sharing in Slack, where your team is already talking
‣Dashboards to track the important things for each team
‣One-click reports to share with management and
stakeholders

CASE STUDY QUESTIONS
DISCOVERY & LEAN CUSTOMER
DEVELOPMENT
▸ Based on our problem
hypothesis, what
percentage of the 6-10
interviews we conduct
conﬁrm what we think is the
problem?

CASE STUDY QUESTIONS
MVP
▸ Solves a problem in the
simplest way possible
▸ Could actually be better than
the intended product, with
“concierge onboarding"
▸ Asks a question, which
should be clear and
measurable.

CASE STUDY QUESTIONS
EFFORT TO BUILD
▸ How much dev effort is
anticipated in points or cost per
developer hour?
▸ Reduce scope to the most
essential parts of the feature.
Build “nice to haves” when you
have signal
▸ Track velocity, throughput,
cycle times to anticipate &
understand building costs

CASE STUDY QUESTIONS
COSTS BEYOND DEVELOPMENT
▸ Track support requests
tagged with the feature
name
▸ Track marketing spend on
launching the feature
▸ Track escaped bugs

COMMUNICATION CHECKLIST
ARE YOU COMMUNICATING?
▸ Does your whole team and company know why
you are tracking these metrics?
▸ Do you have agreed-upon actions based on the
results?
▸ Do you have to track down people to get the
data?
▸ Does someone own the metric?

DON’T STOP NOW!
LEARN MORE
▸Lean Analytics by Benjamin Yoskovitz &
Alistair Croll
▸Getting Started with OKRs & Metrics
Guide
▸The School of Little Data
▸The AARRRT of Product Metrics

The document discusses some of the risks and challenges of data visualization and analytics programs in organizations. It argues that while complex data visualizations can work, they are difficult to implement successfully from scratch. Additionally, stakeholders may claim the benefits from outside ideas while only superficially complying with analytics recommendations. The document provides steps for organizations to truly realize change through data-driven insights, such as having leadership buy-in and starting with small, test-based implementations.

Agile bodensee - Agile Testing: Bug prevention vs. bug detectionMichael Palotas

[CXL Live 16] When, Why and How to Do Innovative Testing by Marie PolliCXL

The document discusses different strategies for innovative testing on websites, including iterative and innovative testing. Iterative testing involves changing small elements like button colors or copy, while innovative testing involves changing multiple elements or entire designs. It recommends using innovative testing when an iteration won't work, testing potential is small, or traffic is low. This allows uncovering bigger gains. Innovative testing carries more risk but also higher potential benefit than iterative testing. The document advises making sure the testing process is solid by asking questions like whether the test matches the hypothesis and goals are set up correctly. It also recommends continuing qualitative research for feedback.

Acceptance, accessible, actionable and auditableAlban Gérôme

Many a stakeholder presented with actionable insight have expressed doubts about data quality, its relevance or potential impact. In other cases, the stakeholders will want a data analyst who also commands knowledge of the business or a similar unicorn. The web analytics practitioners would rise to the challenge, and the stakeholders will then want their own Hans Rosling and his dazzling data visualisation and raconteur sills. Your stakeholders are silently experiencing the data transformation like a conservatorship. How can you help them perceive your efforts like a temporary guardianship from which they will emerge as ready to face your data-driven competitors?

Hyper Lean Startups- Innovation & Entrepreneurship in TanzaniaAlex Cowan

The document is from Cowan Publishing and discusses the process of science and experimentation for startups in Tanzania. It explains that science involves recognizing a problem, collecting data through observation and experiment, and testing hypotheses. It then outlines several steps for startups to vet ideas through small, corrective experiments rather than large plans, organize hypotheses by outlining assumptions and tests, focus on validating assumptions through customer feedback, make sure to periodically check progress on metrics, and conclude with deciding whether to pivot or persevere based on learning. The document encourages learning from short cycle experiments to allow for quicker success.

[CXL Live 16] Beyond Test-by-Test Results: CRO Metrics for Performance & Insi...CXL

Individual tests drive insights & ROI, but the most sophisticated optimizers look beyond what an individual test is telling them and use data to optimize their overall testing performance. In this talk, Claire will dive into the specifics of how to track, improve, and drive insight from performance metrics for your conversion program, so you can not only run better tests, but get more out of your investment in CRO.

Programmer's Bookshelf - Remote EditionLisa Backer

"The Pragmatic Programmer", "Clean Code", "Refactoring" - these are among the classic works that we are told should be on every programmer's bookshelf. Often, they make it to the shelf, but remain unread. Have you ever wondered why these books are considered such beacons to our community? Have you maybe even felt guilty for not reading them all? We'll go through the "cliff notes" edition of these classics, and maybe even find a few future classics to inspire you to check out what’s on your own bookshelf.

D. Aitcheson. How to make forecasts that are actually accurate.Agile Lietuva

Stop multiplying by 4: Practical Software EstimationChuck Reeves

The document provides tips and techniques for estimating software development timelines. It recommends doubling the initial time estimate and rounding up to the next unit to account for uncertainties. Key aspects to define include requirements, parameters to validate, and historical data. Estimates should have confidence intervals rather than precise figures. Prioritization involves assessing features' relative benefits, costs and risks to determine priority. Communication and removing personal biases are also important.

RecSysOps: Best Practices for Operating a Large-Scale Recommender SystemEhsan38

Ensuring the health of a modern large-scale recommendation system is a very challenging problem. To address this, we need to put in place proper logging, sophisticated exploration policies, develop ML-interpretability tools or even train new ML models to predict/detect issues of the main production model. In this talk, we shine a light on this less-discussed but important area and share some of the best practices, called RecSysOps, that we’ve learned while operating our increasingly complex recommender systems at Netflix. RecSysOps is a set of best practices for identifying issues and gaps as well as diagnosing and resolving them in a large-scale machine-learned recommender system. RecSysOps helped us to 1) reduce production issues and 2) increase recommendation quality by identifying areas of improvement and 3) make it possible to bring new innovations faster to our members by enabling us to spend more of our time on new innovations and less on debugging and firefighting issues. https://dl.acm.org/doi/10.1145/3460231.3474620

Information radiatorslheasman

This document discusses what makes a good information radiator for an agile team. It notes that information radiators publicly display information so people don't need to ask questions. Good radiators help teams self-organize, avoid surprises, assist with planning, and highlight work in progress. The document provides examples of simple and effective radiators like story walls, burn up charts, build lights, and riskometers. It emphasizes that radiators should focus on the work, be easily updated and understood, and reflect the team's changing work.

Lean Startup: How Development Looks Different When You're Changing the World ...Abby Fichtner

I updated my Lean Startup presentation for Agile 2011 to share what is happening in the startup community with the agile community and to show how Lean Startup is pushing Agile to the next level. Session Description: How does development look different when you're creating things that no one has ever created before? Lean Startup is about creating from a BIG VISION, where we want to change the world & do something really significant. It combines Agile Development with _Customer_ Development so we can be disciplined about how we create our startups. Learning (rather than working software) becomes our most important measure of progress and agile practices are pushed to the next level. Come get a glimpse of the next level of agile and discover how development looks different when you're changing the world.

You want it when? Probabilistic forecasting and decision makingLarry Maccherone

Before the space shuttle Challenger explosion, a group of engineers identified a potentially catastrophic risk. They brought the issue to NASA management attention but failed to influence the final decision enough to stop the launch. As a leader in your organization, your failure to influence may not cost lives but it could be “catastrophic” for your business. Learn how to get action and behavior change from your analysis. Steer the emotional elephant of your organization and appeal to the risk tolerance level of your stakeholders. Avoid your own cognitive biases and those of your executives.

Introducing HOSTING Labs - Ed Schaefer Hostway|HOSTING

HostingLabs was created by a small team at HOSTING to rapidly develop and test new technologies using an agile "Skunk Works" approach. It aims to foster innovation, thought leadership, staff development and community engagement through open source projects. Some examples of previous internal projects and current public projects are provided. The presentation encourages supporting employee satisfaction, learning, engagement and automating tasks to increase productivity. It promotes measuring the impact of changes using metrics.

A/B Testing and the Infinite Monkey TheoryUseItBetter

The Secret Sauce of Successful TeamsSven Peters

Every software team writes code, but some teams produce fewer bugs than others. Every software team creates new features, but some teams develop them faster than others. What do high performance teams do differently, and why are team members more focused, satisfied and relaxed? They truly work together. No 10x rockstar programmer can achieve what a well rounded, enthusiastic team can. Sven examines how the best software teams set and follow goals, integrate new members fast, ensure diversity, monitor and continually improve team health, embrace transparency, use a playbook to guide them through every phase of development and much more. He shares techniques including: bugfix rotations, OKRs, feature buddies, open demos, focus days and many more that help teams and team members to work more effectively together, and produce awesome results.

Agile basics Thoughtworks

Agile Basics for Government with ThoughtWorks Most people interested in the field of innovation have heard of agile innovation teams. These small, entrepreneurial groups are designed to stay close to customers and adapt quickly to changing conditions. When implemented correctly, they have a reputation for almost always result in higher team productivity and moral, faster time to market, better quality and lower risk than traditional approaches can achieve. But while agile methods caught on first in IT departments and are now widely used in software development, the agile approach has potential to transform the public sector in ways far beyond better bits and bytes. Conditions are ripe for agile teams in any situation where problems are complex, solutions are at first unclear, project requirements are likely to change, and close collaboration with end users is feasible: a description that matches many facing a wide variety of public sector activities. This session will provide participants with an opportunity to explore what the world of agile can teach them – about themselves, their work and their potential to serve their clients better, whatever their role. It will confront some of the common myths and misconceptions about agile, and demonstrate how an agile approach can enable teams to deliver sooner and scale faster through a proven learning culture that builds and strengthens the team and its capabilities.

[CXL Live 16] How to Utilize Your Test Capacity? by Ton WesselingCXL

Ton Wesseling gave a presentation at ConversionXL Live in Austin on March 31st 2016 about utilizing test capacity. He discussed optimizing conversions through the ROAR model of risk, optimization, automation and re-thinking. Wesseling emphasized fully using a company's test capacity for impactful A/B tests and separating that capacity for IT releases, campaigns and behavioral learning. He advised celebrating failures to encourage risk-taking and continuous learning.

Session 4: The Case for VR: Immersive Learning in the Workplace (Shelley Osbo...Udemy for Business

DOES SFO 2016 - Kaimar Karu - ITIL. You keep using that word. I don't think i...Gene Kim

Let’s get this straight. ITIL is not about implementing dozens of processes, or about establishing a CAB to review every change request, or about the never-ending story of creating a CMDB. The ITIL framework has been designed to help IT organizations to move from being a black box technology provider – often viewed as a disposable cost centre – to becoming a service provider, and a true partner for the rest of the business. We know – we own the framework. Unless your customer can achieve their objectives with the technology you run, and can get assistance when needed, no-one cares whether your architecture is built on a monolith, uses microservices, or can brag about being serverless. Agile as a mind-set covers the whole value chain, but common practices are limited to development only. DevOps as a philosophy covers the whole value chain, but common practices are limited to the deployment-focused intersection of development and operations only. Understanding the organisation's strategy, developing the product strategy, and dealing with customer issues are expected to be taken care of by someone else, as if by magic. Because of this, DevOps faces a risk of becoming the largest local optimisation exercise ever undertaken for way too many organisations In tens of thousands of companies around the world, ITIL has helped to develop an organizational capability that has provided them with a competitive advantage. More than three million people have been certified, and ten times as many trained over the years. Yet, we have all heard the horror stories, too. So what is it that separates a successful adoption of ITIL from an unsuccessful attempt at implementing the framework? What are the common problematic practices and anti-patterns we have seen in the wild, and what does the guidance in ITIL really say? How can you move from a broken approach to IT Service Management to one that delivers value. Can you still use ITIL in the DevOps world? Do you even need to? Or, perhaps, the questions is whether DevOps can survive (in the enterprise) without embracing the service mind-set.

What is the story with agile data keynote agile 2018 (Magennis)Troy Magennis

This document discusses using data to improve agile practices and outcomes. It argues that agile has lost the "data war" by not capturing and utilizing data from teams effectively. It suggests that data needs to be handled safely to avoid embarrassing people and destroying the utility of historical data. Better ways are needed to measure outcomes rather than just output, and to balance predictability with creativity. The document also discusses visualizing and managing dependencies, comparing performance across teams, and using the right metrics depending on a team's characteristics and challenges. The overarching message is that data needs to be used carefully and conversationally to drive the right actions and improve agile practices.

Stop multiplying by 4 LaraconChuck Reeves

Many developers are often asked by project owners to give time estimates for features or bug fixes. But how many developers have the ability to provide project owners a reasonable estimate? Many developers will just follow irrational formulas or arbitrary methods to create a number that is not only wrong, but costly. "Stop Multiplying by 4" will teach developers of all skill levels easy techniques to provide accurate estimations. We will start with a small calibration exercise to find out how good you are. We will then go over procedures to improve accuracy . At the end of the talk, you will possess the skills to get you started on improving the certainty of your estimates.

[CXL Live 16] Growth Hacking BS: Fixing Marketing One Truth at a Time by Morg...CXL

Growth hacking has exploded in popularity, with companies scrambling to find magical 'hackers' who create massive user bases and revenue out of thin air. Unfortunately, a closer look shows us that much of the hype is just that. To find growth, companies must stop searching for unicorns and do something much less sexy: get back to work. In this talk I'll cover: - how companies actually grow online based on studying dozens of fast-growing startups - why growth is based on companies finding winning tests faster - how we did that at growth hackers - how other companies do it (HubSpot, Twitter, more) - how to start hacking and start growing - things to avoid, etc.

AgileChina 2015: Agile Estimation WorkshopStephen Vance

Froomle Tech Webinar : Stand on the GiantsFroomle

I love the smell of data in the morning (getting started with data science) ...Troy Magennis

Slideshare 3: Master the Essentials of Conversion Optimization Ashley Enyeart

The document discusses various techniques for conversion optimization, including setting goals and measuring key metrics, conducting technical, heuristic and analytics-based research, using Google Analytics to identify opportunities, implementing A/B testing, and learning from test results even when hypotheses are not validated. Some specific techniques covered are mouse tracking and heatmaps to understand user behavior, on-page surveys, remote user testing, prioritizing high-impact opportunities, and ensuring tests are properly designed and sample sizes are sufficient. The overall goal is to follow a continuous process of gathering data, generating hypotheses, testing ideas, and using results to further optimize conversions and business performance.

Thinking Fast & Slow presentationLaure Parsons

This document summarizes key ideas from Daniel Kahneman's book "Thinking Fast and Slow" about two modes of thinking - System 1 thinking which is fast, automatic, and emotional and System 2 thinking which is slower, more deliberative, and logical. It describes various cognitive biases that result from System 1 thinking such as priming, familiarity, and the halo effect. It provides tips for avoiding cognitive biases in business contexts like hiring and planning. The overall message is that while intuitive thinking has its place, it's important to recognize cognitive biases and use deliberate, data-driven System 2 thinking when high stakes decisions are involved.

NC Kilkenny reference letterSiu-Hong Mok

More Related Content

What's hot (20)

D. Aitcheson. How to make forecasts that are actually accurate.Agile Lietuva

Stop multiplying by 4: Practical Software EstimationChuck Reeves

RecSysOps: Best Practices for Operating a Large-Scale Recommender SystemEhsan38

Information radiatorslheasman

Lean Startup: How Development Looks Different When You're Changing the World ...Abby Fichtner

You want it when? Probabilistic forecasting and decision makingLarry Maccherone

Introducing HOSTING Labs - Ed Schaefer Hostway|HOSTING

A/B Testing and the Infinite Monkey TheoryUseItBetter

The Secret Sauce of Successful TeamsSven Peters

Agile basics Thoughtworks

[CXL Live 16] How to Utilize Your Test Capacity? by Ton WesselingCXL

Session 4: The Case for VR: Immersive Learning in the Workplace (Shelley Osbo...Udemy for Business

DOES SFO 2016 - Kaimar Karu - ITIL. You keep using that word. I don't think i...Gene Kim

What is the story with agile data keynote agile 2018 (Magennis)Troy Magennis

Stop multiplying by 4 LaraconChuck Reeves

[CXL Live 16] Growth Hacking BS: Fixing Marketing One Truth at a Time by Morg...CXL

AgileChina 2015: Agile Estimation WorkshopStephen Vance

Froomle Tech Webinar : Stand on the GiantsFroomle

I love the smell of data in the morning (getting started with data science) ...Troy Magennis

Slideshare 3: Master the Essentials of Conversion Optimization Ashley Enyeart

D. Aitcheson. How to make forecasts that are actually accurate.Agile Lietuva

Stop multiplying by 4: Practical Software EstimationChuck Reeves

RecSysOps: Best Practices for Operating a Large-Scale Recommender SystemEhsan38

Information radiatorslheasman

Lean Startup: How Development Looks Different When You're Changing the World ...Abby Fichtner

You want it when? Probabilistic forecasting and decision makingLarry Maccherone

Introducing HOSTING Labs - Ed Schaefer Hostway|HOSTING

A/B Testing and the Infinite Monkey TheoryUseItBetter

The Secret Sauce of Successful TeamsSven Peters

Agile basics Thoughtworks

[CXL Live 16] How to Utilize Your Test Capacity? by Ton WesselingCXL

Session 4: The Case for VR: Immersive Learning in the Workplace (Shelley Osbo...Udemy for Business

DOES SFO 2016 - Kaimar Karu - ITIL. You keep using that word. I don't think i...Gene Kim

What is the story with agile data keynote agile 2018 (Magennis)Troy Magennis

Stop multiplying by 4 LaraconChuck Reeves

[CXL Live 16] Growth Hacking BS: Fixing Marketing One Truth at a Time by Morg...CXL

AgileChina 2015: Agile Estimation WorkshopStephen Vance

Froomle Tech Webinar : Stand on the GiantsFroomle

I love the smell of data in the morning (getting started with data science) ...Troy Magennis

Slideshare 3: Master the Essentials of Conversion Optimization Ashley Enyeart

Viewers also liked (19)

Thinking Fast & Slow presentationLaure Parsons

NC Kilkenny reference letterSiu-Hong Mok

KLON 103 Unit 8.pptx E-Gazarchin Online University

Rajiv Profile_Sept_2016Rajiv Joseph

Rajiv Joseph has over 8 years of experience in automation framework development using tools like Selenium, NUnit, JUnit, and Robot Framework. He has expertise in C# and developing test automation frameworks, and has experience working with build tools, SQL Server, and agile methodologies. Some of his roles and responsibilities include developing test suites, implementing continuous integration, and managing offshore teams.

Lia Nr. 141webmastercdu

Bharath_SiddaReddy_ResumeBharath Sidda Reddy

Xv jornada programaSoledad Romero

El documento describe un evento organizado por la Delegación de Igualdad del Ayuntamiento de Benalmádena sobre prevención de la violencia de género. El evento incluye conferencias sobre la violencia sexual y menores víctimas de violencia de género y abuso sexual. El alcalde y concejala de Igualdad inaugurarán el evento y clausurarán con la entrega de diplomas. El objetivo es concienciar sobre la violencia de género como un problema social y político que requiere atención y lucha contra la

Oil industry overviewJAYESH SAMTANI

The petroleum industry involves the global processes of exploring for, extracting, refining, transporting, and marketing petroleum products. It is usually divided into upstream (exploration and production), midstream (transportation, storage, and processing), and downstream (refining and marketing) sectors. Oil accounts for a large percentage of global energy consumption, ranging from 32-53% depending on the region. The largest consumers of oil are developed nations and transportation accounts for the majority of oil consumption. While oil will likely remain the dominant fuel source for decades, future supply is uncertain and production is becoming more difficult and costly, raising concerns about energy security and prices. Alternatives can only replace a small fraction of oil demand.

Pegasus slide power 16072013 (gold bar updated)miglobalnetwork

This document describes the Precious Metal Bank program offered by Pegasus Bullion HQ. The program allows individuals to purchase physical gold which is stored in their Precious Metal Bank account. Account holders receive monthly gains based on the quantity of gold stored. They also receive bonuses including sponsor bonuses, pairing bonuses, and gold mine bonuses based on their purchase amount and referrals. The bonuses are paid out in gold which can be withdrawn or kept in the account to earn monthly gains. The document provides examples of how the monthly gains, bonuses, and client package upgrades work.

How To Do Content Marketing So It WorksLaure Parsons

Using data to disrupt your brainLaure Parsons

Pegasus slide(web)miglobalnetwork

Pegasus Bullion offers a Precious Metal Bank (PMB) program that allows individuals to invest in physical gold bullion. Through the PMB, clients can purchase gold, store it, earn monthly gains, and withdraw funds or physical gold. The goal is to give mainstream investors easy access to the gold market. Pegasus Bullion is a subsidiary of Pegasus Investments that specializes in gold trading and retailing. The PMB program provides benefits like no lock-in periods, commissions paid in gold, and monthly gains paid three times per month.

MobileHCI_pie_1.0Hequan(Helen) Zhang

The document describes a pie menu augmented soft keyboard design for Chinese pinyin input methods. It found that only 10 letters ("e, r, u, i, o, a, g, h, v, n") are needed for characters after the first letter in pinyin syllables. The design integrates an 8-cell pie menu with these 10 letters on top of a quasi-QWERTY keyboard. A user study found that after 2 hours, users could enter Chinese characters at 25 characters per minute, slightly slower than without the pie menu but could remember the pie menu layout. The design aims to improve pinyin syllable input efficiency but has room for improvement.

The mystical world of Atlantisdouglas1441

Atlantis is a legendary island said to have existed some 11,000 years ago as described by the Greek philosopher Plato. While its exact location remains unknown, Plato placed it in the Atlantic Ocean off the coast of Africa. Many expeditions have searched for Atlantis in locations around the world without success. According to legend, Atlantis was a highly advanced civilization created by the god Poseidon for his mortal lover. However, the Atlantians grew greedy and powerful and were punished by Zeus, who destroyed the entire island continent by submerging it beneath the sea.

Success with business casesStuart Robb

The document provides guidance on writing successful business cases that can obtain approval and funding. It emphasizes that business cases should be measurable and backed by evidence. Key points include: - Business cases should justify a project's value, risks, priorities and benefits in a concise yet informative manner (typically 2-3 pages). - They must demonstrate how the project aligns with organizational strategy and quantify expected financial and non-financial impacts such as increased revenue, cost savings, risk reductions and compliance gains. - Assumptions should be supported by facts and benefits must be measurable both during and after project implementation through key performance indicators. - High-quality business cases will consider alternative options, include input from finance and procurement

Managing the portfolioStuart Robb

This document provides guidance on managing portfolio projects. It discusses key dimensions of project delivery including product, organization, process, and client/business needs. It emphasizes setting project levers in equilibrium to reduce risk and increase certainty. Successful delivery requires good processes, people, technology, and great leadership. The four essential disciplines of portfolio management are planning, resource management, financial management, and change management. Status reporting must be consistent to effectively manage projects. Challenges include applying a flexible rather than "one size fits all" methodology. Tips for programme managers include allowing time for decisions, surrounding oneself with good project managers, and clearly understanding goals and authority.

Talent Led ProgrammesStuart Robb

This document discusses talent-based or competency-led programmes, which organize workers into pools based on their skills rather than using a traditional hierarchical structure. It addresses how talent pools support both projects and steady-state work. Key aspects include competency leaders who manage resources and ensure quality, and project managers who are responsible for scope, time and budget. Decision-making is flattened, with few hierarchy levels. People progress based on skill levels rather than job titles. Subgroups within competencies are also possible. Objectives are evaluated using a balanced scorecard across skill development, utilization, delivery and client relationships.

Neuroscience & DesignLaure Parsons

The document discusses how neuroscience can help with design. It describes the different parts of the brain and their functions, such as the brain stem regulating basic functions and the prefrontal cortex giving humans reason. It discusses System 1 thinking as emotional and instinctive, and System 2 as logical. It introduces the SCARF model which says the brain desires status, certainty, autonomy, relatedness, and fairness. It also discusses concepts like cognitive dissonance, eye tracking patterns, and how stories engage different parts of the brain.

Target operating model definitionStuart Robb

Thinking Fast & Slow presentationLaure Parsons

NC Kilkenny reference letterSiu-Hong Mok

KLON 103 Unit 8.pptx E-Gazarchin Online University

Rajiv Profile_Sept_2016Rajiv Joseph

Lia Nr. 141webmastercdu

Bharath_SiddaReddy_ResumeBharath Sidda Reddy

Xv jornada programaSoledad Romero

Oil industry overviewJAYESH SAMTANI

Pegasus slide power 16072013 (gold bar updated)miglobalnetwork

How To Do Content Marketing So It WorksLaure Parsons

Using data to disrupt your brainLaure Parsons

Pegasus slide(web)miglobalnetwork

MobileHCI_pie_1.0Hequan(Helen) Zhang

The mystical world of Atlantisdouglas1441

Success with business casesStuart Robb

Managing the portfolioStuart Robb

Talent Led ProgrammesStuart Robb

Neuroscience & DesignLaure Parsons

Target operating model definitionStuart Robb

Similar to Iterate. Calculate. Communicate. (20)

Lean analyticsAndrii Gakhov

The document discusses lean analytics and lean principles for software development. It describes the main stages of product development - empathy, stickiness, virality, revenue, and scale. For each stage, it recommends key metrics to measure, such as daily/weekly active users for stickiness, viral coefficient for virality, and quarterly recurring revenue for revenue. It emphasizes focusing on one metric that matters the most for each stage, measuring assumptions in the right order, and using both qualitative and quantitative metrics for continuous learning and improvement.

Lean Stack - A Story Of Continuous ImprovementLukas Fittl

Testing workshop dayone final accelerator 26022019DayOne

This document summarizes a customer discovery workshop focused on developing and testing hypotheses for new products and services. The workshop covered topics such as conducting customer interviews and focus groups, developing minimum viable products, prioritizing critical hypotheses, designing hypothesis test cards, and planning experiments. Participants worked in groups to extract and refine hypotheses, develop MVPs, design test cards for important hypotheses, and track planned experiments and activities over the coming months. The goal was to help entrepreneurs learn quickly by getting feedback through the build-measure-learn process with minimal effort.

Creating a Product Roadmap - Product Strategy SeriesMike Biggs GAICD

Enabling Innovation using Lean StartupLukas Fittl

The document provides an overview of Lean Startup principles and techniques for enabling innovation. It discusses concepts like validated learning, falsifiable hypotheses, minimum viable products, and A/B testing to learn from customers and improve products iteratively. Key aspects covered include defining product ownership, establishing shared understanding across teams, ensuring accountability, and properly scoping and timeboxing experiments. Various tools and tactics are also presented to help operationalize the Lean Startup approach, such as experiment reports, metrics dashboards, cross-functional pairing, and weekly strategy meetings.

2011 10 12 eric ries lean startup web 2.0 expo ny keynoteEric Ries

The document discusses principles of the Lean Startup methodology. It emphasizes validating ideas with customers through minimum viable products and rapid experimentation (build-measure-learn loop), using metrics to determine when to pivot an idea that is not working or persevere with one that shows promise. The goal is to minimize the time and money spent on ideas that are not viable so entrepreneurs can focus on their most promising opportunities.

How To Build a Winning Experimentation Program & Team | Optimizely ANZ Webinar 8Optimizely

This document provides an overview of how to build a successful experimentation program within an organization. It discusses the importance of experimentation for innovation and outlines key areas to assess when developing an experimentation capability, including organizational culture. It then describes attributes of mature experimentation cultures and how most organizations are currently at an early stage. The rest of the document focuses on operationalizing experimentation through identifying a team, following an experimentation methodology, understanding metrics, and getting started with small changes.

2010 10 15 the lean startup at tech_hub londonEric Ries

The document discusses the key principles of the Lean Startup methodology for building startups under conditions of extreme uncertainty. It advocates for an approach of continuous experimentation through building minimum viable products, obtaining rapid customer feedback through metrics like split testing, and using this validated learning to iteratively pivot or evolve the product or business model. The goal is to minimize the time required to progress through the build-measure-learn feedback loop in order to increase the chances of success before running out of resources.

2010 10 28 the lean startup at ucsdEric Ries

This document discusses the Lean Startup methodology for building startups. It emphasizes using validated learning through experiments and customer feedback to reduce the time and resources wasted on products no one wants. Key principles include building minimum viable products to test hypotheses quickly and continuously deploying code to gather feedback to pivot the product as needed. This approach aims to maximize learning while minimizing wasted effort through practices like rapid A/B testing and measuring business metrics.

[Webinar] Visa's Journey to a Culture of ExperimentationOptimizely

Join us as we hear Ramkumar Ravichandran, the Director of A/B Testing at Visa Checkout, explain how he created a high impact experimentation program. Ram will take us through the growth of Visa’s program: from selling the value, to laying down the vision, the roadmap and success criteria, to creating the right team and driving engagement with the program. Attend this webinar to learn: -How an experimentation program drives business impact. -A model to drive continuous stakeholder engagement with the program. -How to build a roadmap that goes above and beyond simple UX optimization.

Curate Learning Experiences to Ensure Change SuccessANCILE Solutions, Inc.

This document discusses curating learning experiences to ensure change success. It defines curating as "to take charge of" or "to organize." It then outlines a four step process to curate: 1) inform users with content, 2) validate understanding with assessments, 3) analyze data to identify gaps, and 4) intervene with additional learning as needed. The document provides examples of when to curate, such as during a project rollout or for ongoing adoption. It also gives an example use case of curating learning for retail employees adopting new software. The overall message is that curating customized learning experiences can help drive adoption and ensure change success.

Summary of The Lean Startup (Eric Ries)Vinsol

Customer development and Agile developmentdchurchv

The document discusses how applying customer development principles can help improve agile projects. It suggests focusing on validating hypotheses about customers, problems, and solutions through experiments like minimum viable products and customer interviews before fully developing features. This helps minimize costs and risks until learning what customers really want. Product owners and developers are encouraged to prioritize learning over building by finding the fastest way to test hypotheses through the build-measure-learn process.

[Webinar] Scaling experimentation: 5 key pillars of maturity by Nick SoChris Goward

Go behind the curtain with WiderFunnel's Director of Strategy, Nick So, as he shares exclusive learnings from his experience building successful experimentation programs for companies like HP, Asics, Magento, and IBM. During this webinar, you’ll get an insider look at the frameworks and strategies that WiderFunnel developed to help clients scale experimentation and accelerate through the phases of maturity.

2010 10 19 the lean startup workshop for i_gap irelandEric Ries

The document discusses the Lean Startup methodology for building startups under conditions of extreme uncertainty. It advocates for an experimental, customer-focused approach where the minimum viable product is used to test hypotheses and gather customer feedback through rapid iteration. Key techniques include continuous deployment, rapid A/B testing, and using the five whys method to identify the root causes of problems. The goal is to minimize the time to validate learning about customers through frequent releases and measurement.

Gravity v0.2Harsh Jawharkar

The document discusses the experiences and lessons learned by Harsh Jawharkar as a "serial intrapreneur", launching new ventures within large organizations. Some key points discussed include building the right team with a mix of skills, focusing on solving the right problem by developing a minimum viable product, and being flexible to pivot if needed based on what is learned. It also emphasizes the importance of customizing the innovation process to fit the type of challenge and assets available, as well as designing for metrics from the beginning to understand what is working.

Agile Methods - 2 day workshopEvan Leybourn

This document provides an overview of agile project iterations in 3 parts. It discusses iteration planning, including prioritizing the backlog, estimating tasks, and creating an iteration backlog. It describes daily activities like scrums, testing, and continuous integration during the build phase. Metrics like burnup/down charts and cycle time are covered for monitoring progress. Retrospectives and continuous improvement are emphasized for inspecting and adapting between iterations.

2011 03 11 eric ries - the lean startup sxswEric Ries

The document discusses Lean Startup principles including building minimum viable products and using validated learning through continuous deployment, A/B testing, and metrics. It emphasizes releasing early and often to get fast customer feedback, pivoting if needed, and minimizing the time between learning cycles. The goal is to maximize learning while minimizing costs through iterative experimentation.

Charla Lean StartupSantiago Palladino

The document summarizes key concepts from Eric Ries' book The Lean Startup. It discusses the pillars of lean startup which are to validate learning through the build-measure-learn loop. Startups should define an MVP to test hypotheses, use innovation accounting to measure metrics, and be willing to pivot based on what is learned. The goal is sustainable growth through engines like viral, paid, or sticky customer acquisition and retention.

product development checklist manifestoJordan Berman

The document discusses how checklists can be used to improve product development processes. It provides examples of effective checklists from hospitals and Apple. Checklists increase "eptitude" by ensuring knowledge is consistently applied, diminish complexity, impose discipline, focus attention, aid judgment, and encourage efficiency and conversation. Product development checklists could focus on features, testing, and deployment. Teams should get buy-in, create checklists iteratively, and use data to identify areas to target.

Lean analyticsAndrii Gakhov

Lean Stack - A Story Of Continuous ImprovementLukas Fittl

Testing workshop dayone final accelerator 26022019DayOne

Creating a Product Roadmap - Product Strategy SeriesMike Biggs GAICD

Enabling Innovation using Lean StartupLukas Fittl

2011 10 12 eric ries lean startup web 2.0 expo ny keynoteEric Ries

How To Build a Winning Experimentation Program & Team | Optimizely ANZ Webinar 8Optimizely

2010 10 15 the lean startup at tech_hub londonEric Ries

2010 10 28 the lean startup at ucsdEric Ries

[Webinar] Visa's Journey to a Culture of ExperimentationOptimizely

Curate Learning Experiences to Ensure Change SuccessANCILE Solutions, Inc.

Summary of The Lean Startup (Eric Ries)Vinsol

Customer development and Agile developmentdchurchv

[Webinar] Scaling experimentation: 5 key pillars of maturity by Nick SoChris Goward

2010 10 19 the lean startup workshop for i_gap irelandEric Ries

Gravity v0.2Harsh Jawharkar

Agile Methods - 2 day workshopEvan Leybourn

2011 03 11 eric ries - the lean startup sxswEric Ries

Charla Lean StartupSantiago Palladino

product development checklist manifestoJordan Berman

Recently uploaded (20)

LLM finetuning for multiple choice google bertChadapornK

Just-In-Timeasdfffffffghhhhhhhhhhj Systems.pptssuser5f8f49

Data Science Courses in India iim skillsdharnathakur29

This comprehensive Data Science course is designed to equip learners with the essential skills and knowledge required to analyze, interpret, and visualize complex data. Covering both theoretical concepts and practical applications, the course introduces tools and techniques used in the data science field, such as Python programming, data wrangling, statistical analysis, machine learning, and data visualization.

04302025_CCC TUG_DataVista: The Design Storyccctableauusergroup

Ppt. Nikhil.pptxnshwuudgcudisisshvehsjkspanchariyasahil

Molecular methods diagnostic and monitoring of infection - Repaired.pptx7tzn7x5kky

EDU533 DEMO.pptxccccvbnjjkoo jhgggggbbbbJessaMaeEvangelista2

Ch3MCT24.pptx measure of central tendencyayeleasefa2

Medical Dataset including visualizationsvishrut8750588758

computer organization and assembly language.docxalisoftwareengineer1

Digilocker under workingProcess Flow.pptxsatnamsadguru491

03 Daniel 2-notes.ppt seminario escatologiaAlexander Romero Arosquipa

1. Briefing Session_SEED with Hon. Governor Assam - 27.10.pdfSimran112433

VKS-Python Basics for Beginners and advance.pptxVinod Srivastava

Minions Want to eat presentacion muy lindaCarlaAndradesSoler1

Deloitte Analytics - Applying Process Mining in an audit contextProcess mining Evangelist

Mieke Jans is a Manager at Deloitte Analytics Belgium. She learned about process mining from her PhD supervisor while she was collaborating with a large SAP-using company for her dissertation. Mieke extended her research topic to investigate the data availability of process mining data in SAP and the new analysis possibilities that emerge from it. It took her 8-9 months to find the right data and prepare it for her process mining analysis. She needed insights from both process owners and IT experts. For example, one person knew exactly how the procurement process took place at the front end of SAP, and another person helped her with the structure of the SAP-tables. She then combined the knowledge of these different persons.

Day 1 - Lab 1 Reconnaissance Scanning with NMAP, Vulnerability Assessment wit...Abodahab

Conic Sectionfaggavahabaayhahahahahs.pptxtaiwanesechetan

Cleaned_Lecture 6666666_Simulation_I.pdfalcinialbob1234

How iCode cybertech Helped Me Recover My Lost Fundsireneschmid345

I was devastated when I realized that I had fallen victim to an online fraud, losing a significant amount of money in the process. After countless hours of searching for a solution, I came across iCode cybertech. From the moment I reached out to their team, I felt a sense of hope that I can recommend iCode Cybertech enough for anyone who has faced similar challenges. Their commitment to helping clients and their exceptional service truly set them apart. Thank you, iCode cybertech, for turning my situation around! [email protected]

LLM finetuning for multiple choice google bertChadapornK

Just-In-Timeasdfffffffghhhhhhhhhhj Systems.pptssuser5f8f49

Data Science Courses in India iim skillsdharnathakur29

04302025_CCC TUG_DataVista: The Design Storyccctableauusergroup

Ppt. Nikhil.pptxnshwuudgcudisisshvehsjkspanchariyasahil

Molecular methods diagnostic and monitoring of infection - Repaired.pptx7tzn7x5kky

EDU533 DEMO.pptxccccvbnjjkoo jhgggggbbbbJessaMaeEvangelista2

Ch3MCT24.pptx measure of central tendencyayeleasefa2

Medical Dataset including visualizationsvishrut8750588758

computer organization and assembly language.docxalisoftwareengineer1

Digilocker under workingProcess Flow.pptxsatnamsadguru491

03 Daniel 2-notes.ppt seminario escatologiaAlexander Romero Arosquipa

1. Briefing Session_SEED with Hon. Governor Assam - 27.10.pdfSimran112433

VKS-Python Basics for Beginners and advance.pptxVinod Srivastava

Minions Want to eat presentacion muy lindaCarlaAndradesSoler1

Deloitte Analytics - Applying Process Mining in an audit contextProcess mining Evangelist

Day 1 - Lab 1 Reconnaissance Scanning with NMAP, Vulnerability Assessment wit...Abodahab

Conic Sectionfaggavahabaayhahahahahs.pptxtaiwanesechetan

Cleaned_Lecture 6666666_Simulation_I.pdfalcinialbob1234

How iCode cybertech Helped Me Recover My Lost Fundsireneschmid345

Iterate. Calculate. Communicate.

1. ITERATE. CALCULATE. COMMUNICATE.

2. CLOSING THE GAP BETWEEN MEASURE + LEARN

3. Linked In: /in/xplusx Medium & Twitter: @laurex LAURE PARSONS SENIOR PRODUCT MANAGER

4. “THE GOAL OF BUILD-MEASURE-LEARN IS NOT TO BUILD A FINAL PRODUCT TO SHIP OR EVEN TO BUILD A PROTOTYPE OF A PRODUCT, BUT TO MAXIMIZE LEARNING THROUGH INCREMENTAL AND ITERATIVE ENGINEERING” Steve Blank, author of The Four Steps to the Epiphany BE ACTIONABLE!

5. 1. WHAT SHOULD YOU TRACK? 2. WHAT MAKES A METRIC “GOOD?” 3. HOW CAN YOU GET ALIGNMENT WITH YOUR TEAM AND LEADERSHIP? 4. WHAT SHOULD YOU DO TO ENSURE YOU LEARN? ‣ CASE STUDY: BUILDING A NEW FEATURE!

6. ACQUISITION ACTIVATION REFERRAL RETENTION REVENUE AARRR: PIRATE METRICS

7. ACQUISITION ACTIVATION REFERRAL RETENTION REVENUE TEAM AARRRT: TEAM METRICS

8. THE ONE METRIC From Lean Analytics by Benjamin Yoskovitz & Alistair Croll

9. UX TESTS MVP A/B TESTSCLASSIC LEAN MEASUREMENT

10. WHO, WHAT, WHEN, WHERE, WHY & HOW ▸WHO CUSTOMERS ARE ▸WHAT CUSTOMERS DO AND/OR TRY TO DO ▸WHEN DO USERS DISENGAGE ▸WHERE CUSTOMERS COME FROM ▸WHY CUSTOMERS CONVERT ▸HOW MUCH YOU CAN BUILD

11. GOOD METRICS

12. GOOD METRICS COMPARABLE

13. RATIOS & RATES

14. GOOD METRICS UNDERSTANDABLE

15. GOOD METRICS ACTIONABLE

16. BE ACTIONABLE! TWO TYPES OF ACTIONABLE DATA ▸“Accounting” metrics can make your predictions more accurate ▸“Experimental” metrics help optimize product, pricing, or market. Agree on what will change before you collect the data.

17. “IF YOU MEASURE SOMETHING AND IT’S NOT ATTACHED TO A GOAL, IN TURN CHANGING YOUR BEHAVIOUR, YOU’RE WASTING YOUR TIME.” LEAN ANALYTICS by Ben Horowitz and Alistair Kroll BE ACTIONABLE!

18. ALIGNMENT & COMMUNICATION

19. OKRS

20. WHO OWNS THE DATA?

21. WHAT ACTIONS WILL YOU TAKE?

22. WHO NEEDS TO KNOW?

23. HOW WILL THEY KNOW?

24. THE DATA TOOL YOUR TEAM WILL ACTUALLY USE

25. ‣A single source of truth ‣Company-wide common understanding and deﬁnitions ‣Appropriate permissions ‣Transparency ‣Collaboration via shared data and commenting ‣Automated integration from your tools ‣Sharing in Slack, where your team is already talking ‣Dashboards to track the important things for each team ‣One-click reports to share with management and stakeholders

26. CASE STUDY

27. CASE STUDY QUESTIONS DISCOVERY & LEAN CUSTOMER DEVELOPMENT ▸ Based on our problem hypothesis, what percentage of the 6-10 interviews we conduct conﬁrm what we think is the problem?

28. CASE STUDY QUESTIONS MVP ▸ Solves a problem in the simplest way possible ▸ Could actually be better than the intended product, with “concierge onboarding" ▸ Asks a question, which should be clear and measurable.

29. CASE STUDY QUESTIONS EFFORT TO BUILD ▸ How much dev effort is anticipated in points or cost per developer hour? ▸ Reduce scope to the most essential parts of the feature. Build “nice to haves” when you have signal ▸ Track velocity, throughput, cycle times to anticipate & understand building costs

30. EFFORT TO BUILD

31. CASE STUDY QUESTIONS WHO CARES? ▸ Beta test the feature with customers who’ve requested it and/or with freemium or opt-in customers ▸ If you have a large enough customer base, survey your customers with a tool like Intercom or Qualaroo when they use the feature ▸ Measure feature adoption in Mixpanel or other event-based tools. See how often customers try the feature and how many times they re-use it.

32. CASE STUDY QUESTIONS COSTS BEYOND DEVELOPMENT ▸ Track support requests tagged with the feature name ▸ Track marketing spend on launching the feature ▸ Track escaped bugs

33. CASE STUDY QUESTIONS THE BOTTOM LINE ▸ Do a cohort analysis of customers before and after the feature was introduced. ▸ Track number of signups, churn, and referrals. ▸ If the feature is plan-gated, you can also track by pricing segments.

34. WHAT ACTION FOLLOWS?

35. COMMUNICATION CHECKLIST ARE YOU COMMUNICATING? ▸ Does your whole team and company know why you are tracking these metrics? ▸ Do you have agreed-upon actions based on the results? ▸ Do you have to track down people to get the data? ▸ Does someone own the metric?

36. DON’T STOP NOW! LEARN MORE ▸Lean Analytics by Benjamin Yoskovitz & Alistair Croll ▸Getting Started with OKRs & Metrics Guide ▸The School of Little Data ▸The AARRRT of Product Metrics

37. THANK YOU!

Iterate. Calculate. Communicate.

Recommended

More Related Content

What's hot (20)

Viewers also liked (19)

Similar to Iterate. Calculate. Communicate. (20)

Recently uploaded (20)

Iterate. Calculate. Communicate.