How
to
Build
a
Lean
Startup,


step‐by‐step
#leanstartup


Eric
Ries
(@ericries)

h?p://StartupLessonsLearned.blogspot.com

 Agenda

•  What
is
a
startup?

•  Why
do
startups
fail?

•  IntroducJon
to
the
Lean
Startup

•  Product
Development
in
a
Lean
Startup

•  Specific
techniques

–  ConJnuous
Deployment

–  Rapid
Split‐tesJng

–  Five
Why’s

 What
is
a
startup?

•  A
startup
is
a
human
ins)tu)on
designed
to

deliver
a
new
product
or
service
under

condiJons
of
extreme
uncertainty.



•  Nothing
to
do
with
size
of
company,
sector
of

the
economy,
or
industry

 Why
do
startups
fail?

•  Rarely
fail
because
the
product
doesn’t
work

•  Usually
fail
because
there
are
no
customers

•  Quality
of
the
iniJal
idea
is
not
correlated
with

success

•  Startups
that
succeed
are
those
that
manage

to
iterate
enough
Jmes
before
they
run
out
of

resources

•  Time
between
these
iteraJons
is
fundamental

 The
Lean
Startup


•  Anything
we
can
do
to
shrink
the
Jme

between
major
iteraJons
will
increase
the

likelihood
of
success.

•  Speed
is
the
startup
compeJJve
advantage.

 Lean
Startups
Go
Faster

•  Commodity
technology
stack,
highly
leveraged

(free/open
source,
user‐generated
content,

SEM).

•  Customer
development
–
find
out
what

customers
want
before
you
build
it.


•  Agile
so]ware
development
–
but
tuned
to

the
startup
condiJon.

 Commodity
technology
stack

•  Leverage
=
for
each
ounce
of
effort
you
invest

in
your
product,
you
take
advantage
of
the

efforts
of
thousands
or
millions
of
others.

•  It’s
easy
to
see
how
high‐leverage
technology

is
driving
costs
down.

•  More
important
is
its
impact
on
speed.

•  Time
to
bring
a
new
product
to
market
is

falling
rapidly.


 Customer
Development

  ConJnuous
cycle
of
customer

interacJon

  Rapid
hypothesis

tesJng
about
market,

pricing,
customers,
…

  Extreme
low
cost,
low


burn,
Jght
focus

  Measurable
gates
for


h?p://bit.ly/tpTtE

investors

 A
tale
of
two
startups,
revisited

•  Mirrors
the
changes
in
development

methodologies
over
the
past
few
years.

•  Let’s
look
at
those
changes
schemaJcally.


•  These
examples
are
drawn
from
so]ware

startups,
but
increasingly:

–  All
products
require
so]ware


–  All
companies
are
operaJng
in
a
startup‐like

environment

 TradiJonal
Product
Development

Unit
of
progress:
Advance
to
Next
Stage


Waterfall

Requirements


Design


Implementa2on


Verifica2on


Maintenance


Problem: Known Solution: Known

 Agile

Unit
of
progress:
a
line
of
working
code


“Product Owner” or
in-house customer

Problem: Known Solution: Unknown

 Product
Development
at
Lean
Startup

Unit
of
progress:
validated
learning
about
customers
($$$)


Problem: Unknown Solution: Unknown

Minimize
TOTAL
Jme
through
the
loop


IDEAS


LEARN
 BUILD


DATA
 CODE


MEASURE

 How
to
build
a
Lean
Startup

•  Let’s
talk
about
some
specifics.
These
are
not

everything
you
need,
but
they
will
get
you

started


•  ConJnuous
deployment

•  Split‐test
(A/B)
experimentaJon

•  Five
why’s


 ConJnuous
Deployment


IDEAS


Learn
Faster
 Code
Faster

Five
Whys
Root

LEARN
 BUILD
 ConJnuous

Cause
Analysis
 Deployment


DATA
 CODE


Measure
Faster

MEASURE

Rapid
Split
Tests

Continuous Deployment
•  Deploy new software quickly
•  At IMVU time from check-in to production = 20 minutes

•  Tell a good change from a bad change (quickly)

•  Revert a bad change quickly

•  Work in small batches

•  At IMVU, a large batch = 3 days worth of work

•  Break large projects down into small batches

Cluster Immune System
What it looks like to ship one piece of code to production:
•  Run tests locally (SimpleTest, Selenium)
o  Everyone has a complete sandbox

•  Continuous Integration Server (BuildBot)

o  All tests must pass or “shut down the line”
o  Automatic feedback if the team is going too fast

•  Incremental deploy

o  Monitor cluster and business metrics in real-time

o  Reject changes that move metrics out-of-bounds

•  Alerting & Predictive monitoring (Nagios)


o  Monitor all metrics that stakeholders care about

o  If any metric goes out-of-bounds, wake somebody up

o  Use historical trends to predict acceptable bounds

When
customers
see
a
failure:

o  Fix the problem for customers
o  Improve your defenses at each level

 Rapid
Split
Tests


IDEAS


Learn
Faster
 Code
Faster

Five
Whys
Root

LEARN
 BUILD
 ConJnuous

Cause
Analysis
 Deployment


DATA
 CODE


Measure
Faster

MEASURE

Rapid
Split
Tests

 Split‐tesJng
all
the
Jme

•  A/B
tesJng
is
key
to
validaJng
your

hypotheses

•  Has
to
be
simple
enough
for
everyone
to
use

and
understand
it

•  Make
creaJng
a
split‐test
no
more
than
one

line
of
code:

if(
setup_experiment(...)
==
"control"
)
{





//
do
it
the
old
way

}
else
{




//
do
it
the
new
way

}

 The
AAA’s
of
Metrics

•  AcJonable

•  Accessible

•  Auditable

 Measure
the
Macro

•  Always
look
at
cohort‐based
metrics
over
Jme

•  Split‐test
the
small,
measure
the
large

Control
Group
(A)
 Experiment
(B)

#
Registered
 1025
 1099

Downloads
 755
(73%)
 733
(67%)

AcJve
days
0‐1
 600
(58%)
 650
(59%)

AcJve
days
1‐3
 500
(48%)
 545
(49%)

AcJve
days
3‐10
 300
(29%)
 330
(30%)

AcJve
days
10‐30
 250
(24%)
 290
(26%)

Total
Revenue
 $3210.50
 $3450.10

RPU
 $3.13
 $3.14

 Five
Whys


IDEAS


Learn
Faster
 Code
Faster

Five
Whys
Root

LEARN
 BUILD
 ConJnuous

Cause
Analysis
 Deployment


DATA
 CODE


Measure
Faster

MEASURE

Rapid
Split
Tests

 Five
Whys
Root
Cause
Analysis

•  A
technique
for
conJnuous
improvement
of

company
process.

•  Ask
“why”
five
Jmes
when
something

unexpected
happens.

•  Make
propor)onal
investments
in
prevenJon

at
all
five
levels
of
the
hierarchy.

•  Behind
every
supposed
technical
problem
is

usually
a
human
problem.
Fix
the
cause,
not

just
the
symptom.

 There’s
much
more…


IDEAS


Learn
Faster
 Code
Faster

Split
Tests

LEARN
 BUILD
 Unit
Tests

Customer
Interviews
 Usability
Tests

Customer
Development
 ConJnuous
IntegraJon

Five
Whys
Root
Cause
Analysis
 Incremental
Deployment

Customer
Advisory
Board
 Free
&
Open‐Source
Components

Falsifiable
Hypotheses
 Cloud
CompuJng

Product
Owner
Accountability
 Cluster
Immune
System

Customer
Archetypes
 DATA
 CODE
 Just‐in‐Jme
Scalability

Cross‐funcJonal
Teams
 Refactoring

Semi‐autonomous
Teams
 Developer
Sandbox

Smoke
Tests


Measure
Faster

MEASURE

Split
Tests
 Funnel
Analysis

Clear
Product
Owner
 Cohort
Analysis

ConJnuous
Deployment
 Net
Promoter
Score

Usability
Tests
 Search
Engine
MarkeJng

Real‐Jme
Monitoring
 Real‐Time
AlerJng

Customer
Liaison
 PredicJve
Monitoring

 The
Lean
Startup

•  You
are
ready
to
do
this,
whether
you
are:

–  Thinking
of
starJng
a
new
company,
but
haven’t

taken
the
first
step

–  Are
in
a
startup
now
that
could
iterate
faster

–  Want
to
create
the
condiJons
for
lean
innovaJon

inside
a
big
company

•  Get
started,
now,
today.

 Thanks!


•  Startup
Lessons
Learned
Blog

–  h?p://startuplessonslearned.blogspot.com/


•  The
Lean
Startup
Workshop


–  An
all‐day
event
for
a
select
audience

–  May
29
and
June
18,
2009
in
San
Francisco

–  h?p://training.oreilly.com/theleanstartup/


•  Other
upcoming
events

–  h?p://bit.ly/b9w1y