Presentation given to Tetuan Valley startup school students during the fall of 2013. Introduction to metrics for startups, with examples from 24symbols

1. Metrics:
because everything
counts
Justo Hidalgo
@justohidalgo
TAKE YOUR READINGS WHEREVER YOU GO

3. Hi!
■ Co-founder,
• Worry about metrics
■ Data Integration and Management,
Product Strategy and Innovation
■ Ph.D. in Computer Science on Data
Integration and Web Automation
■ Ergo: Love Data 
■ @justohidalgo

4. reading
+
discovery
+
any device
+
mobile
A service to read and discover digital books that
works on any device

5. @justohidalgo

6. My goals today
Metrics
A/B Testing (if we have time!)
Some tools
Conclusions
Eat pizza

7. What I won’t talk about
Qualitative measurement
• Competitive landscape, UX quality, …
Standard financial metrics
• Revenue, Balance, Burn rate, Runway, …
All possible metrics in the world
• As many as asses

8. Let’s talk about metrics and
startups!

9. Watch this…
500startups’ Dave McClure

10. … and this…
Carsonified’s Ryan Carson

11. … and read this!

14. AARRR
Acquire
Activate
SEO
SEM
Campaigns
Email
Blogs
…
Landing
Page
Product
Features
…
traffic
Retain
Content
(blogs,
articles,
…)
Emails
Alerts
…
Refer
Get Revenue
Campaigns
Emails
…
Shopping
cart
Subscriptions
Lead Gen
…
social
business

15. Traffic Metrics

16. Acquisition
How do people find your site?

18. @justohidalgo

20. 15 seconds in “La
1”

22. http://www.youtube.com/watch?v=YSRlDqRUizg

23. Metrics
Number of visits
Distributions (per browser, region,
…)
Bounce rate
Number of page views

24. AActivation
How sticky is your site?

29. Metrics
# Sign ins!!!
85000
Recency (time since user did
something meaningful)
80000
75000
70000
65000
60000
55000
50000
45000
40000
Time per page
Time on site
(and churn as well!!!)

30. Biggest Apple
Reseller in Spain
1-year
subscription to
24s per iPad/Mac
purchased
@justohidalgo

31. AARetention
Why do users come back?

35. Metrics. The standard ones…
Number of unique/returning visitors
Visits over time
Average session time

36. … engagement…
D/WAU: Daily/Weekly Active
Users
MAU: Monthly Active Users
Stickiness =
D/WAU
MAU

37. … but also domain-specific!!!
# Active readers
# Books read per user
Pages read per active
reader per month
Pages read per
paid user per
month

38. Social Metrics

39. AARReferral
Do people talk about your site?

40. Types of users
Visitor
Contributor
Distributor

42. Curation the unbeatable experience of a
bookstore

43. Curation bookshelves
Every user, a librarian

47. Virality
Amplification rate:
• (#shares, #RTs, …)
K-Factor = AR *
conversion rate
• = 1: steady state
• > 1: growth
➤ 1.4 < x < 2.1 => hot &
viral!!!
• < 1: decline

48. The Four Viral App Objectives
% viral hosts
Contact rate per active host
Duration of each active host’s
infectious time period
Infection conversion
Source: http://framethink.wordpress.com/2008/01/15/the-four-viral-appobjectives-aka-social-network-application-virality-101/

49. @justohidalgo

50. Business Metrics

51. AARRRevenue
Do you make money?

54. Source: Lean Analytics – Alistair Croll, Ben Yoskovitz

55. Churn

56. Lifetime Value =
100%
ARPU *
Churn rate
Source: http://blog.kissmetrics.com/how-to-calculate-lifetime-value/
LTV

57. LTV vs CAC
Of course, LTV > CAC
But, moreover: LTV > 3x CAC

58. Freemium. Conversion Rate
Free
Users
Delayed subscription
(test the service & engage)
Premium
Conversion
Rate
Time
@justohidalgo

59. Price/page
Freemium for Books. Price per page
Profit threshold
Growth
Consolidation
time/conversion rate
@justohidalgo

60. Source: Startup Metrics, a love story - @andreasklinger

61. Source: Lean Analytics – Alistair Croll, Ben Yoskovitz

63. One key question for the whole AARRR
cycle
■ How effective is a web page compared to another?
?

64. What’s A/B Testing?
Version A
Visitors are randomly distributed
Version B
time
1200 new users
1000 new users
Version A is better than B

65. What should you test?
 The call to action’s (i.e. the button’s) wording,
size, color and placement,
 Headline or product description,
 Form’s length and types of fields,
 Layout and style of website,
 Product pricing and promotional offers,
 Images on landing and product pages,
 Amount of text on the page (short vs. long)
… anything that you think it might affect your Key Metrics.

66. Is A/B testing perfect?
Very good for ONE-SINGLE-GOAL
projects where the KPI can be
measured by counting actions
• E.g. Downloads vs Quality perceived
Granularity of the A/B test is
paramount
• If “A” better than “B” or is it just that
the headline in B is horrible, but the
rest is much much much better than
“A”?
A/B testing is suboptimal*
* Recommended:
http://untyped.com/untyping/2011/02/11/stop-ab-testing-andmake-out-like-a-bandit/

68. Analytics
Heatmaps
Social media

77. Conclusions

79. @justohidalgo

80. 2. Have someone taking care of it

81. 3. Only measure what you truly care about
@justohidalgo

82. But again, choose right

83. 4. Beware vanity metrics…

84. … they may hide an awful truth

85. Thanks for your time!
Justo Hidalgo
http://www.loscuentosdelabuelo.com
jhidalgo@24symbols.com
@justohidalgo

86. Credits
Olivier Guin, from The Noun Project
Dima Yagnyuk, from The Noun Project
Redesign from Otl Aicher’s icon, from The Noun Project
Axsys, from The Noun Project
Monika Ciapala, from The Noun Project
Rob Schill, from The Noun Project
Martin Karachorov, from The Noun Project
Alexandre Mendes, from The Noun Project
All other icons from The Noun Project
Publishers trademarks, logos and brands belong to their
respective companies.