Your First Startup Experiment

Your First Startup Experiment

Luke Szyrmer

© 2016 - 2022 Luke Szyrmer

To every founder who has the courage to admit they could be wrong, and smarts to know how to deal with it in advance, and the grit to go ahead anyway

Table of Contents

Business plans are broken…

Postpone until certainty of success?

Why market testing velocity is a success secret hidden in plain sight

Why you always start with a vision

Choose your canvas

Why your riskiest assumption is a great place to start

Let’s look at AirBnB, to see why risky assumptions matter in practice

Proving the risky demand side assumption (as they were ignored)

Proving the supply side (as they were ridiculed)

Then AirBnB were attacked as they drew attention scaling up

Finally, they built monuments to AirBnB.

Key Takeaways

How to identify your riskiest assumption

With new products, your primary goal is to manage your risks

1. Intuition

2. Who is the customer and does the customer actually have problem you think?

3. Analogs

4. Antilogs (or Picnic in the Graveyard)

5. Risk Scorecard

6. Pre-Mortem

7. My ace in the sleeve questions

8. Talk to industry experts

9. Red team, Blue team

What you are after

Riskiest Assumption Test (RAT)

How to track your assumptions

Economic impact of risk factors

Approach 1: Tracking risk factors in spreadsheet

Approach 2: Business Modelling using a canvas tool

Approach 3: Going even wider doesn’t make sense at an early stage

Approach 4: The most important boxes according to published research

It doesn’t matter where you keep track of assumptions as long as you do

Key takeaways

How to construct your test backlog

How to verify your assumptions

Start With Assumptions About Your Market

Implicit Assumptions

How to test one specific hypothesis

Aim For Financial Significance With Pre-testing

The Painful Suck Of False Positives

Test One Assumption At A Time

How Much Does One Spend?

How Many Experiments Should You Run?

When Not to Test

Why customer outcome discovery trumps product discovery

When do you know you have a useful outcome?

1. A Specific Person, Company, or Role in a Narrow Market Segment

2. Their success metric

3. No side effects

4. Easy to execute or confirm

Case Study

What if my releases require Herculean efforts?

Why discovering outcomes first matters even more than discovering product

Key Takeaways

Why founder-market fit helps mitigate your risk

Hence the importance of founder-market fit

Strongly tied to how the founders expect to acquire customers

High growth markets will buoy up the business

Founder core competencies are surprisingly a strategic decision

What about industry outsiders as successful founders?

Summary

Key Takeaways

Epilogue: why minimum viable products are how to continue testing

A minimum viable product enables you to release earlier

Examples

Case Studies

A Marketing Experiment: The content upgrade

Take action now

Key takeaways

Additional resources

Books

Business plans are broken…

Analogy: Chris Rock and his audience {wdith=50%}

The Stress Factory.

It’s an ingenious name–for a place used to refine ideas. It’s also Chris Rock’s favorite place to try out new material. He comes over with a yellow legal pad, and performs for roughly 50 people. Peter Sims summarizes in Little Bets:

In sets that run around forty-five minutes, most of the jokes fall flat. [Chris Rock’s] early performances can be painful to watch. Jokes will ramble. He’ll lose his train of thought and need to refer to his notes. And some audience members site with their arms folded, noticeably unimpressed. The audience will laugh about his flops–laughing at him, not with him….He may think he has come up with the best joke ever, but if it keeps missing with audiences, that becomes his reality. Other times, a joke he thought would be a dud will bring the house down….For a full routine, Rock tries hundreds (if not thousands) of preliminary ideas, out of which only a handful make the final cut.

In short, Chris Rock stress tests the jokes, in order to figure out which ones work for his typical audience…long before he scales up. By the time he appears on HBO, Chris feels very comfortable that the material will really speak to his audience.

It’s common practice among standup comedians, to do dry runs of new material in an intimate club environment. Ideally, the audience will be friendly and willing to accept some untried jokes that flop, in exchange first dibs on new material.

Each joke at the Stress Factory is just a little bet. Rock is verifying his assumptions about what tickles his audience’s funny bone. Given that he stages multiple appearances, he iterates over many rough ideas, to figure out which ones are worth polishing. Once they’re fully polished, he’s ready to push record on TV cameras.

The same analogy holds true for product ideas. You need to interact with your audience, in order to find out what they want. Most breakthrough product ideas don’t start as breakthrough ideas. Most of them need to be discarded. Of the ideas that don’t, they often start life as dull but showing promise. Ultimately, they’re refined until they become great. Occasionally, there’s a fantastic idea right from the start, yet often the product creators can’t judge that effectively.

Just like with Chris Rock’s jokes, only the intended audience can.

In that context, understanding your audience and verifying your assumptions about their needs, is critical before you even start building a product or offering a service.

If you vet bad ideas immediately and only pursue really good ideas, you will spend most of your time and money identifying opportunities that were inherently excellent. Find the ideas for which your audience has latent demand–before committing to an idea. Understand your audience, and you’ll have a much easier time launching anything for them.

No entrepreneur ever writes a business plan saying that they’re going to fail. The main reason you’d want to write a long document like that is to convince yourself and others that you have a fantastic business idea (on paper). While it goes into how you plan to succeed with an idea, a business plan assumes you already have an idea that’s perfect. After all, the unique skill of an entrepreneur is to wonder, to ask what if?

Unfortunately that can lead to self-delusion. And self-delusion can easily be the your downfall, if you don’t think through your challenges thoroughly enough. In addition to speed, launching a business successfully requires significant self-honesty, such as truthful tracking of your costs and assumptions, in order to evaluate your ideas accurately.

To be clear, an assumption is just a statement of opinion about the occurrence of an event