Growth Tribe invited Alistair Croll, author of the Lean Analytics to come to Amsterdam to give a workshop about Lean Analytics & Growth Hacking.
Livestream video part 1: https://www.youtube.com/watch?v=0cEfe9mSatM
Livestream video part 2: https://www.youtube.com/watch?v=muu0GENRkoI
Growth Tribe: http://www.growthtribe.nl/
7. Everyone’s idea is
the best right?
People love
this part!
(but that’s not always
a good thing)
This is where
things fall apart.
No data, no
learning.
8. Most startups don’t know what they’ll
be when they grow up.
Hotmail
was a
database
company
Flickr
was going to
be an MMO
Twitter
was a
podcasting
company
Autodesk
made
desktop
automation
Paypal
first built for
Palmpilots
Freshbooks
was invoicing
for a web
design firm
Wikipedia
was to be
written by
experts only
Mitel
was a
lawnmower
company
10. Analytics is the measurement of
movement towards your business
goals.
11. In a startup, the purpose of analytics is
to iterate to product/market fit
before the money runs out.
12. A good metric is:
Understandable
If you’re busy
explaining the
data, you won’t
be busy acting
on it.
Comparative
Comparison is
context.
A ratio or rate
The only way to
measure
change and roll
up the tension
between two
metrics (MPH)
Behavior
changing
What will you
do differently
based on the
results you
collect?
14. Metrics help you know yourself.
Acquisition
Hybrid
Loyalty
70%
of retailers
20%
of retailers
10%
of retailers
You are
just like
Customers that
buy >1x in 90d
Once
2-2.5
per year
>2.5
per year
Your customers
will buy from you
Then you are
in this mode
1-15%
15-30%
>30%
Low acquisition
cost, high checkout
Increasing return
rates, market share
Loyalty, selection,
inventory size
Focus on
(Thanks to Kevin Hillstrom for this.)
15. Qualitative
Unstructured, anecdotal,
revealing, hard to
aggregate, often too
positive & reassuring.
Warm and fuzzy.
Quantitative
Numbers and stats.
Hard facts, less insight,
easier to analyze; often
sour and disappointing.
Cold and hard.
16. Exploratory
Speculative. Tries to find
unexpected or
interesting insights.
Source of unfair
advantages.
Cool.
Reporting
Predictable. Keeps you
abreast of the normal,
day-to-day operations.
Can be managed by
exception.
Necessary.
17. MayAprMarFeb
Slicing and dicing data
Jan
0
5,000
Activeusers
Cohort:
Comparison of
similar groups
along a timeline.
(this is the April cohort)
A/B test:
Changing one thing
(i.e. color) and
measuring the
result (i.e. revenue.)
Multivariate
analysis
Changing several
things at once to
see which correlates
with a result.
☀
☁
☀
☁
Segment:
Cross-sectional
comparison of all
people divided by
some attribute (age,
gender, etc.)
☀
☁
19. January February March April May
Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50
Is this company
growing or stagnating?
Cohort 1 2 3 4 5
January $5 $3 $2 $1 $0.5
February $6 $4 $2 $1
March $7 $6 $5
April $8 $7
May $9
How about
this one?
20. Cohort 1 2 3 4 5
January $5 $3 $2 $1 $0.5
February $6 $4 $2 $1
March $7 $6 $5
April $8 $7
May $9
Averages $7 $5 $3 $1 $0.5
Look at the
same data
in cohorts
22. Eric’s three engines of growth
Virality
Make people
invite friends.
How many they
tell, how fast they
tell them.
Price
Spend money to
get customers.
Customers are
worth more than
they cost.
Stickiness
Keep people
coming back.
Approach
Get customers
faster than you
lose them.
Math that
matters
23. Dave’s Pirate Metrics
AARRR
Acquisition
How do your users become aware of you?
SEO, SEM, widgets, email, PR, campaigns, blogs ...
Activation
Do drive-by visitors subscribe, use, etc?
Features, design, tone, compensation, affirmation ...
Retention
Does a one-time user become engaged?
Notifications, alerts, reminders, emails, updates...
Revenue
Do you make money from user activity?
Transactions, clicks, subscriptions, DLC, analytics...
Referral
Do users promote your product?
Email, widgets, campaigns, likes, RTs, affiliates...
24. Stage
EMPATHY
I’ve found a real, poorly-met need that a
reachable market faces.
STICKINESS
I’ve figured out how to solve the problem in a
way they will keep using and pay for.
VIRALITY
I’ve found ways to get them to tell their friends,
either intrinsically or through incentives.
REVENUE
The users and features fuel growth organically
and artificially.
SCALE
I’ve found a sustainable, scalable business with
the right margins in a healthy ecosystem.
Gate
Thefivestages
25. Six business model archetypes.
E-commerce SaaS Media
Mobile
app
User-gen
content
2-sided
market
The business you’re in
26. (Which means eye
charts like these.)
Customer Acquisition Cost
paid direct search wom
inherent
virality
VISITOR
Freemium/trial offer
Enrollment
User
Disengaged User
Cancel
Freemium
churn
Engaged User
Free user
disengagement
Reactivate
Cancel
Trial abandonment
rate
Invite Others
Paying Customer
Reactivation
rate
Paid
conversion
FORMER USERS
User Lifetime Value
Reactivate
FORMER CUSTOMERS
Customer Lifetime Value
Viral coefficient
Viral rate
Resolution
Support data
Account Cancelled Billing Info Exp.
Paid Churn Rate
Tiering
Capacity Limit
Upselling
rate Upselling
Disengaged DissatisfiedTrial Over
27. Model + Stage = One Metric That Matters.
One Metric
That Matters.
The business you’re in
E-Com SaaS Mobile 2-Sided Media UCG
Empathy
Stickiness
Virality
Revenue
Scale
Thestageyou’reat
39. Baseline:
5-7% growth a week
“A good growth rate during YC
is 5-7% a week,” he says. “If
you can hit 10% a week you're
doing exceptionally well. If you
can only manage 1%, it's a sign
you haven't yet figured out
what you're doing.” At revenue
stage, measure growth in
revenue. Before that, measure
growth in active users.
Paul Graham, Y Combinator
• Are there enough people who really care
enough to sustain a 5% growth rate?
• Don’t strive for a 5% growth at the expense
of really understanding your customers
and building a meaningful solution
• Once you’re a pre-revenue startup at or
near product/market fit, you should have
5% growth of active users each week
• Once you’re generating revenues, they
should grow at 5% a week
40. Baseline:
10% visitor engagement/day
Fred Wilson’s social ratios
30% of users/month use web or mobile app
10% of users/day use web or mobile app
1% of users/day use it concurrently
41. Baseline:
2-5% monthly churn
• The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s
on the job.
• Get below a 5% monthly churn rate before you know you’ve got a
business that’s ready to grow (Mark MacLeod) and around 2%
before you really step on the gas (David Skok)
• Last-ditch appeals and reactivation can have a big impact.
Facebook’s “don’t leave” reduces attrition by 7%.
42. Baseline:
Calculating customer lifetime
25%
monthly churn
100/25=4
The average
customer lasts
4 months
5%
monthly churn
100/5=20
The average
customer lasts
20 months
2%
monthly churn
100/2=50
The average
customer lasts
50 months
43. Baseline:
CAC under 1/3 of CLV
• CLV is wrong. CAC Is probably wrong, too.
• Time kills all plans: It’ll take a long time to find
out whether your churn and revenue projections
are right
• Cashflow: You’re basically “loaning” the
customer money between acquisition and CLV.
• It keeps you honest: Limiting yourself to a
CAC of only a third of your CLV will forces you
to verify costs sooner.
Lifetime of 20 mo.
$30/mo. per
customer
$600 CLV
$200 CAC
Now segment
those users!
1/3 spend
45. Draw a new line
Pivot or
give up
Try again
Success!
Did we move the
needle?
Measure
the results
Make changes
in production
Design a test
Hypothesis
With data:
find a
commonality
Without data:
make a good
guess
Find a potential
improvement
Draw a linePick a KPI
46. Do AirBnB hosts
get more business
if their property is
professionally
photographed?
47. Gut instinct (hypothesis)
Professional photography helps AirBnB’s business
Candidate solution (MVP)
20 field photographers posing as employees
Measure the results
Compare photographed listings to a control group
Make a decision
Launch photography as a new feature for all hosts
51. Draw a new line
Pivot or
give up
Try again
Success!
Did we move the
needle?
Measure
the results
Make changes
in production
Design a test
Hypothesis
With data:
find a
commonality
Without data:
make a good
guess
Find a potential
improvement
Draw a linePick a KPI
52. “Gee, those
houses that do
well look really
nice.”
Maybe it’s the
camera.
“Computer: What
do all the
highly rented
houses have in
common?”
Camera model.
With data:
find a commonality
Without data: make a
good guess
53. Circle of Moms: Not enough engagement
• Too few people were
actually using the
product
• Less than 20% of any
circles had any activity
after their initial creation
• A few million monthly
uniques from 10M
registered users, but no
sustained traction
• They found moms were far more engaged
• Their messages to one another were on average 50% longer
• They were 115% more likely to attach a picture to a post they wrote
• They were 110% more likely to engage in a threaded (i.e. deep)
conversation
• Circle owners’ friends were 50% more likely to engage with the circle
• They were 75% more likely to click on Facebook notifications
• They were 180% more likely to click on Facebook news feed items
• They were 60% more likely to accept invitations to the app
• Pivoted to the new market, including a name change
• By late 2009, 4.5M users and strong engagement
• Sold to Sugar, inc. in early 2012
54. Landing page design A/B testing
Cohort analysis General analytics
URL shortening
Funnel analytics
Influencer Marketing
Publisher analytics
SaaS analytics
Gaming analytics
User interaction Customer satisfaction KPI dashboardsUser segmentation
User analytics Spying on users
56. The Attention Economy
“What information consumes
is rather obvious: it consumes
the attention of its recipients.
Hence a wealth of information
creates a poverty of attention, and a
need to allocate that attention efficiently
among the overabundance of
information sources that might
consume it.”
(Computers, Communications and the Public Interest, pages 40-41,
Martin Greenberger, ed., The Johns Hopkins Press, 1971.)Herbert Simon
67. 10:1 ROI on song
2:1 ROI on marketing
2:1 ROI on song
10:1 ROI on marketing
$100K on song
$1M on marketing
Your budget:
{
Artist A:
Awesome musician
Not very marketable
Artist B:
Average musician
Incredibly marketable
68. 10:1 ROI on song
2:1 ROI on marketing
2:1 ROI on song
10:1 ROI on marketing
$1M song ROI
$2M marketing ROI
$3M ROI total
2.73x ROI
$200K song ROI
$10M marketing ROI
$10.2M ROI total
9.27x ROI
3.4 times
better investment
$100K on song
$1M on marketing
Your budget:
{
77. Lagging
Historical. Shows you
how you’re doing;
reports the news.
Example: sales.
Explaining the
past.
Leading
Forward-looking.
Number today that
predicts tomorrow;
reports the news.
Example: pipeline.
Predicting the
future.
80. Correlated
Two variables that are
related (but may be
dependent on
something else.)
Ice cream &
drowning.
Causal
An independent variable
that directly impacts a
dependent one.
Summertime &
drowning.
81. • Growth hacking is simply what marketing should have been
doing, but it fell in love with Don Draper and opinions along the
way
• Optimize a factor you think is correlated with growth
The growth hack
83. A Facebook user reaching 7 friends within 10 days of signing up
(Chamath Palihapitiya)
If someone comes back to Zynga a day after signing up for a game,
they’ll probably become an engaged, paying user (Nabeel Hyatt)
A Dropbox user who puts at least one file in one folder on one device
(ChenLi Wang)
Twitter user following a certain number of people, and a certain
percentage of those people following the user back (Josh Elman)
A LinkedIn user getting to X connections in Y days (Elliot Schmukler)
Some examples
(From the 2012 Growth Hacking conference. http://growthhackersconference.com/)
85. Aunshul Rege of Rutgers University, USA in 2009
Experienced scammers expect a “strike rate” of 1 or 2 replies per 1,000 messages
emailed; they expect to land 2 or 3 “Mugu” (fools) each week.
One scammer boasted “When you get a reply it’s 70% sure you’ll get the money”
“By sending an email that repels all but the most gullible,” says [Microsoft Researcher
Corman] Herley, “the scammer gets the most promising marks to self-select, and tilts
the true to false positive ratio in his favor.”
1000 emails
1-2 responses
1 fool and their money, parted.
Bad language (0.1% conversion)
Gullible (70% conversion)
1000 emails
100 responses
1 fool and their money, parted.
Good language (10% conversion)
Not-gullible (.07% conversion)
This would be horribly
inefficient since
humans are involved.
86. Turns out the word “Nigeria” is the best
way to identify promising prospects.
105. I can’t tell you how to succeed
(If I could, I’d be starting your company instead of standing here.)
106. I can tell you how to avoid failure
(At least, some forms of it.)
107. Percent of businesses that fail
Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7 Year 8 Year 9 Year 10
71%69%66%63%60%55%50%44%36%25%
http://www.statisticbrain.com/startup-failure-by-industry/
108. Still operating after 4 years
Finance Insurance and Real Estate
Education and Health
Agriculture
Services
Wholesale
Mining
Manufacturing
Construction
Retail
Transportation Communication and Utilities
Information 37%
45%
47%
47%
49%
51%
54%
55%
56%
56%
58%
http://www.statisticbrain.com/startup-failure-by-industry/
109. Why they die
http://www.statisticbrain.com/startup-failure-by-industry/
Neglect, fraud, disaster
1%
Lack of industry experience
13%
Lack of managerial experience
34%
Incompetence
52%
Emotional Pricing
Living too high for the
business
Nonpayment of taxes
No knowledge of pricing
Lack of planning
No knowledge of financing
No experience in record-
keeping
Poor credit granting
practices
Expansion too rapid
Inadequate
borrowing practices
Carry inadequate inventory
No knowledge of suppliers
Wasted advertising budget
110. 1. An
entrepreneur
is (maybe) not
a founder The goal is to discover a new
business model in an uncertain
environment
Learning trumps planning
111. “A startup is an organization
designed to search for a sustainable,
repeatable business model.”
112. Entrepreneur
Known business model
Out-execute the
competition
Less risk, less reward
Steal and improve
Startup founder
New business model
Discover new product &
market first
Much risk, very reward
Define & invent
The goal is to discover a new
business model in an uncertain
environment. Learning trumps
planning.
113. Is your idea a roofrack or a steering wheel?
http://www.datacenterknowledge.com/archives/2010/03/02/cloud-computing-the-roof-rack-problem/
114. Degrees of uncertainty
Market with
unmet need
Job to be
done
Business
diagram
Business
model
Business plan
Really uncertain:
Does the market exist?
Is the need real and
known?
Really certain:
Can you do what you
said you would, when
you said you would?
117. It’s oxygen
You need customers to keep learning
It’s a substitute for solvency
PhotobyPaulMilleronFlickr.https://www.flickr.com/photos/94674772@N03/8788576498
140. 6. On advisors
and boards Why do you have advisors?
Why are investors sociopaths?
141. Why do you have advisors?
They need to earn their keep
Make expectations explicit
Give them options, and make sure you can fire them
142. Why investors are sociopaths
https://www.flickr.com/photos/peptravassos/10658028023
https://www.flickr.com/photos/46124960@N00/3471536479
143. 7. Marketing
is not a bad
word Sergio Zyman’s five mores
The attention economy
Risk is out of your control
Building message maps
How to construct a campaign
144. Sergio
Zyman’s
five
mores
More things
To more people
For more money
More often
More efficiently
Supply chain optimization
Per-transaction cost reduction
Loyal customer base that returns
Demand prediction, notification
Maximum shopping cart
Price skimming/tiering
Highly viral offering
Low incremental order costs
Inventory increase
Gifting, wish lists
145. Your growth problem is probably
a marketing problem.
(Things out of your control are riskier.)
146. How to build a message map.
http://solveforinteresting.com/building-a-message-map/
147. I need a carA.
I should buy
a car
B.
It should be
a hybridC.
I should buy
a Honda CivicD.
Everyone in the world
148. People who want to drive
Prospective car buyers
People looking for a hybrid
Honda Civic Hybrid owners
I need a carA.
I should buy
a car
B.
It should be
a hybridC.
I should buy
a Honda CivicD.
Everyone in the world
149. “Isn’t it time you got out of the
city?” campaign showing how cars
make nature accessible & ridiculing
urban hipsters.
Ads showing how cars are needed
any time (pregnancy, errands, urgent
business) and how a car is a
“personal assistant.”
Urgency (“every time you drive a
non-hybrid car you kill the planet a
little”) and testimonials from buyers
who’ve saved money.
Honda branding ads and model-
specific promotions.
Follow-up satisfaction campaign to
encourage buyers to tell their friends
People who want to drive
“I need a vehicle to get
around, be productive, and
enjoy my life.”
Prospective car buyers
“I want to own a car because it’s
convenient; it’s a personal
relationship; I don’t trust others.”
People looking for a hybrid
“I want to save money and fuel. I
also care about the environment
and want to be seen as ‘green’.”
Honda Civic Hybrid owners
I need a carA.
I should buy
a car
B.
It should be
a hybridC.
I should buy
a Honda CivicD.
Everyone in the world
150. People who want to drive
“I need a vehicle to get
around, be productive, and
enjoy my life.”
Prospective car buyers
“I want to own a car because it’s
convenient; it’s a personal
relationship; I don’t trust others.”
People looking for a hybrid
“I want to save money and fuel. I
also care about the environment
and want to be seen as ‘green’.”
Honda Civic Hybrid owners
Those who don’t need cars
• I’m too young to drive
• I’m too old to drive
• I can walk or take public
transit
Car users who won’t buy
• It’s too expensive for me
• I will use a shared car service
• It’ll get stolen
Those who won’t buy hybrids
• Hybrids are gutless
• Batteries are toxic & explosive
• In the end it costs more than
it saves
I will buy another brand
• I buy domestic
• I’ve always driven a VW
• Toyotas are reliable
• I want something prestigious
I need a carA.
I should buy
a car
B.
It should be
a hybridC.
I should buy
a Honda CivicD.
Everyone in the world
151. People who want to drive
“I need a vehicle to get
around, be productive, and
enjoy my life.”
Prospective car buyers
“I want to own a car because it’s
convenient; it’s a personal
relationship; I don’t trust others.”
People looking for a hybrid
“I want to save money and fuel. I
also care about the environment
and want to be seen as ‘green’.”
Honda Civic Hybrid owners
Those who don’t need cars
• I’m too young to drive
• I’m too old to drive
• I can walk or take public
transit
Car users who won’t buy
• It’s too expensive for me
• I will use a shared car service
• It’ll get stolen
Those who won’t buy hybrids
• Hybrids are gutless
• Batteries are toxic & explosive
• In the end it costs more than
it saves
I will buy another brand
• I buy domestic
• I’ve always driven a VW
• Toyotas are reliable
• I want something prestigious
Sponsor a driving school
“Give the gift of driving”
campaign for grandparents.
Financing, cashback
Sell to carshares;
underscore their limitations
PR on dangers of commuting,
pedestrian deaths
Theft warranty, tracking
services, high-end locks
Independent tests,
standard metrics (0-60 in X)
Lab research, studies
ROI calculator;
replacement programs
Prove Honda hires US workers
“Time to leave Germany” ads
Spontaneous accel. stories
Premium brand (Acura)
I need a carA.
I should buy
a car
B.
It should be
a hybridC.
I should buy
a Honda CivicD.
Everyone in the world
152. “Isn’t it time you got out of the
city?” campaign showing how cars
make nature accessible & ridiculing
urban hipsters.
Ads showing how cars are needed
any time (pregnancy, errands, urgent
business) and how a car is a
“personal assistant.”
Urgency (“every time you drive a
non-hybrid car you kill the planet a
little”) and testimonials from buyers
who’ve saved money.
Honda branding ads and model-
specific promotions.
Follow-up satisfaction campaign to
encourage buyers to tell their friends
People who want to drive
“I need a vehicle to get
around, be productive, and
enjoy my life.”
Prospective car buyers
“I want to own a car because it’s
convenient; it’s a personal
relationship; I don’t trust others.”
People looking for a hybrid
“I want to save money and fuel. I
also care about the environment
and want to be seen as ‘green’.”
Honda Civic Hybrid owners
Those who don’t need cars
• I’m too young to drive
• I’m too old to drive
• I can walk or take public
transit
Car users who won’t buy
• It’s too expensive for me
• I will use a shared car service
• It’ll get stolen
Those who won’t buy hybrids
• Hybrids are gutless
• Batteries are toxic & explosive
• In the end it costs more than
it saves
I will buy another brand
• I buy domestic
• I’ve always driven a VW
• Toyotas are reliable
• I want something prestigious
Sponsor a driving school
“Give the gift of driving”
campaign for grandparents.
Financing, cashback
Sell to carshares;
underscore their limitations
PR on dangers of commuting,
pedestrian deaths
Theft warranty, tracking
services, high-end locks
Independent tests,
standard metrics (0-60 in X)
Lab research, studies
ROI calculator;
replacement programs
Prove Honda hires US workers
“Time to leave Germany” ads
Spontaneous accel. stories
Premium brand (Acura)
I need a carA.
I should buy
a car
B.
It should be
a hybridC.
I should buy
a Honda CivicD.
Everyone in the world
153. How to build a marketing campaign
http://www.yearonelabs.com/three-questions-all-marketers-must-answer/
When will you decide if it worked, and adjust?
Who are you
targeting?
Size, reachability,
homogeneity.
What do you
want them to
do?
Specific, measurable
call to action.
Why should
they do it?
Laid, paid, made,
or afraid; message
fits their mindset.
How will you
know if they
did?
Analytics,
instrumentation.
154. “The most important figures that one
needs for management are unknown
or unknowable, but successful
management must nevertheless take
account of them.”
Lloyd S. Nelson
155. Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844