The Lean Startup Process

The purpose of this page is to collect important information from the book The Lean Startup (http://lean.st). You can use these notes as inspiration when designing a suitable development process for a project.

The Lean Startup is the application of lean thinking to the process of innovation. It involves incremental improvements, extremely fast cycle times, a focus on what users want (without asking them), and use of the scientific method for decision making. The Lean Startup is said to improve the success rate of innovative products by eliminating waste.

Startup – a human institution designed to create new products and services under conditions of extreme uncertainty.

Startups exist to learn how to build a sustainable business around a company vision.

The five principles of the Lean Startup

• Entrepreneurs are everywhere
• Entrepreneurship is management
• Validated leaning
• Build-Measure-Learn
• Innovation accounting

Break down the vision into its major components. This includes at least the value hypothesis (value received from using the product or service) and the growth hypothesis (how new users discover the product or service).

Build a Minimum Viable Product. This could even be just a simple hyperlinked mockup with a signup button designed so as to enable measuring of the signup conversion rate.

Get user feedback. Don't neccesarily invest effort into building what users say they want – they probably don't know what they want in advance. Use the user feedback as one of several sources of information that you can consider when creating new hypotheses that you can attempt to validate.

Continuous deployment. Deploy new versions of the product extremely often (even many times a day) and expose changes to enough users to get accurate data. A software bug should not stop a deployment unless it prevents us from validating or invalidating the hypothesis we are testing.

Let the company vision guide the experimentation. With the Minimum Viable Product, we have entered the Build-Measure-Learn feedback loop. The goal of each experiment is to learn how to build a sustainable business around the company vision.

The Build-Measure-Learn feedback loop

• The Build-Measure-Learn feedback loop: Ideas → BUILD → Product → MEASURE → Data → LEARN

• Choose the path through the entire loop with the least amount of resources such as development time and money. Fix bugs that slows down the feedback loop. Leave other minor bugs for later, especially if many resources are required to fix them.

• Planned in reverse: 1) Figure out what we need to learn, 2) Use innovation accounting to figure out what we need to measure to know if we have gained validated learning, and 3) then figure out what product we need to build to run that experiment and get that measurement.

• Experiments should be isolated from one another. Users taking part in an experiment should not see changes due to other experiments until the experiment has ended.

• Test one hypothesis or several with the same prototype product.

• When improvements get so small as to not being worth the effort, optimize elsewhere or pivot.

Leap-of-faith assumptions – the riskiest elements of the strategy. Prioritize validation of the more risky hypotheses before the less risky.

Innovation accounting – a quantitative approach that allows us to see if engine-tuning efforts are yielding results.

Validated learning – a unit of progress used as part of doing innovation accounting. Learning milestone – a desired target of validated learning.

Metrics – actionable (demonstrate clear cause and effect), accessible (to the people of the organization), and auditable (testable by talking to users)

Examples of vanity metrics:

• Revenue
• Number of users

Examples of actionable metrics:

• Sign up rate (percentage of visitors that created an account and became users)

• Download rate (percentage of visitors that downloaded some software)

• Activation rate (percentage of users that logged in at least once)

• Paid rate (percentage of users that upgraded to a paid account)

• Retention rate (percentage of users that still login after some time (alternatively: continues to be a paying customer))

• Churn rate (percentage of users that in any period fail to remain engaged with the product (alternatively: stops being a paying customer) - inverse of the retention rate)

• New customer rate – Growth in number of new customers in any period.

• Customer Revenue – Average revenue per customer.

• Customer Variable Costs – Average variable costs per customer.

• Customer lifetime value (Customer Revenue – Customer Variable Costs)

• Viral coefficient - how many new users will use a product as a consequence of each new user that signs up.

Cohort analysis: Look at the measurements for each group independently. Use to track user flows (user behavior).

Pivot or Persevere

• Pivot – A structured course correction designed to test a new fundamental hypothesis about the product, strategy, and engine of growth. A sign of a successful pivot is that engine-tuning efforts result in better measurements than before the pivot. Abandon the pivot if measurements are worse than before the pivot.

• Persevere - Continue the current strategy and hope for better results through further optimization

Pivot or persevere meeting every two weeks to a few months. Bring all product optimization reports. Compare to expectations over time. Bring details of conversations with current and potential customers.

Common pivots

• Zoom-in pivot – Make a single feature of a product the whole product.

• Zoom-out pivot – Build a larger product keeping the current product as just one feature.

• Customer segment pivot – Change the customer archetype.

• Customer need pivot – Solve a newly identified customer problem.

• Platform pivot – Expand the application to a platform used by third parties (eg. like Facebook).

• Business architecture pivot – Change between High margin, Low volume (often B2B) and High volume, Low margin (often B2C).

• Value capture pivot – Change of how to generate money.

• Engine of growth pivot – Change of engine of growth (such as between sticky, viral and paid). Probably a value capture pivot is also needed.

• Distribution channel pivot – Change of distribution channel.

• Marketing channel pivot – Change of marketing channel.

• Technology pivot – A change of enabler technology.

Engine of growth: Derive the tuning variables of the engine of growth from the value and growth hypotheses. Then iteratively tune it using the Build-Measure-Learn feedback loop and if needed pivots. Improvements to product, marketing, or operations are all considered tuning the engine of growth.

Common engines of growth

• Sticky – Awareness spreads by word of mouth (active transmission). Growth is determined by the rate of compounding – New customer rate - churn rate during a period (>0 means growth during that period).

• Viral – Awareness spreads as a natural consequence of product usage (passive transmission). Viral loop speed and growth is determined by the viral coefficient – how many new users will use a product as a consequence of each new user that signs up (>1.0 means exponential growth).

• Paid - Awareness spreads due to paying marketing channel owners, direct sales personel, or stores with good placement. Growth rate is determined by the customer profitability coefficient – Customer lifetime value / customer acquisition cost.

Product immune system

• Automated tests.

• Verify business results after changes.

• Revert worse performing changes automatically, notify the whole team, block further changes until the root cause of the problem is found and removed.

The Five Whys

• Ask why five times to find the root cause of a problem (find the human problem behind the technical problem), then do incremental investments proportional to the severity of the problem at each level.

• Appoint a Five Whys master – senior, moderator, decision maker, assigns follow-up work, evaluates effectiveness of past investments.

• If a mistake happens, shame on us for making it so easy to make that mistake.

Create sandboxes for experimentation

• An experiment is also a product.

• Any team can create an experiment that only affects the sand boxed parts of the product or service or certain customer segments or territories (new product).

• One team must see the whole experiment through from end to end.

• No experiment can run longer than a specified amount of time (usually a few weeks).

• No experiment can affect more than a specified number of customers (usually expressed as a percentage of the company's total mainstream customer base).

• Every experiment has to be evaluated on the basis of a single standard report of five to ten (no more) actionable metrics.

• Every team that works inside the sandbox and every product that is built must use the same metrics to evaluate success.

• Any team that creates an experiment must monitor the metrics and customer reactions (support calls, social media reaction, forum threads, etc.) while the experiment is in progress and abort if something catastrophic happens.

• Start with a small sandbox and expand it over time.

• Report using the same actionable metrics and innovation accounting.

Other points

• Use small batches (since quality problems can be identified sooner).

• Use pull rather than push which together with small batches reduces work-in-progress at any time. The hypotheses does the pulling.

• The effort that is not absolutely necessary for learning what customers really want can be eliminated. When in doubt, simplify.

• Charge customers money from the beginning (if you are planning to do so later).

• Think Big. Start small.

• Newspaper headline: ”Inept Entrepreneurs Build Dreadful Product”? It propbably won't happen and even if it does, it doesn't matter. Stick to the process.

• Initial (baseline) results are probably poor. Stick to the process.

• Set quantitative (how many) targets that inspire qualitative (like, don't like) inquiry and act as guides for questions to ask.

• Use cross-functional teams held accountable to learning milestones.

• Analogs (existing products that suggests a hypothesis is true) and antilogs (existing products that suggests a hypothesis is false).

• Value-creating (sustainable) or value-destroying (unsustainable).

• Success theater: Using the appearance of growth to appear successful.

• Build a customer archetype over time using the validated learning.

• Early adopters desire to be first to use a new product or technology. Make the first versions specifically for them - rough on the edges. They want to tell/show their friends.

• Concierge Minimum Viable Product: Human automate the service. Very inefficient, but you will learn. Automate a process when you know it is viable and human resources need to be freed for other tasks.

• Are you making your product better? How do you know? Use innovation accounting.

• Rate of growth depends on: 1) Profitability of each customer, 2) Cost of new customer acquisition, 3) Repeat purchase rate.

• Stuck in the land of the living dead: When companies neither grow enough nor dying, consuming resources and commitment from employees and other stake holders, but not moving ahead.

• Runway length: The number of possible pivots left to build a sustainable business. Running out of resources? Get to each pivot faster.

• Evolve an employee training program.

Questions to ask

• Do consumers recognize that they have the problem you are trying to solve.

• If there was a solution, would they buy it?

• Would they buy it from us?

• Can we build a solution for that problem?

Additional method for innovation progress evaluation

• The number of products that didn't exist three years ago

• The percentage of revenue coming from products that didn't exist three years ago

• How many days does it take for a new product to reach X in revenue (correct for inflation)?

Addendum

The scientific method is a way to ask and answer scientific questions by making observations and doing experiments. The steps of the scientific method are to:

• Ask a Question

• Do Background Research

• Construct a Hypothesis

• Test Your Hypothesis by Doing an Experiment

• Analyze Your Data and Draw a Conclusion

• Communicate Your Results

It is important for your experiment to be a fair test. A "fair test" occurs when you change only one factor (variable) and keep all other conditions the same. More on The Scientific Method