I gave this talk on how to use product analytics to optimize your product at Split Software's Decisions Product Management Conference on October 2, 2018.

1. Harnessing the Power of
Product Analytics
Dan Olsen 10-2-18

2. n Engineering background: submarine design
n Stanford MBA
n Product Management leader at Intuit & startups
n CEO & Cofounder, TechCrunch winner YourVersion
n Product Management consultant
n Founder: Lean Product Silicon Valley
Twitter: @danolsen
My slides: https://dan-olsen.com
My Background
Copyright © 2018 @danolsen

3. Product Manager’s motto:
Spiderman’s motto:
“With great power comes
great responsibility”
“With great responsibility
comes NO power”
Copyright © 2018 @danolsen

4. Product Teams Are Responsible For
Product-Market
Fit
AND Growth
Copyright © 2018 @danolsen

5. n Book giveaway on Twitter
n Tweet: include @danolsen
n Hashtags
#prodmgmt
#leanstartup
#Decisions2018
@SplitSoftware
That’s Why I Wrote
Copyright © 2018 @danolsen

6. Identify highest
ROI idea
Design and
Implement
Analyze How
the Metric
Changes
Brainstorm
Ideas to
Improve Metric
Copyright © 2018 @danolsen
Lean Product Analytics Process
Identify What
Your Metrics Are
Measure Metrics
Baseline Values
Evaluate Metrics
Upside Potential
Global
Level
Metric
Level
Select
Top Metric
Learn
& Iterate

7. At any point, one metric offers the
highest return-on-investment (ROI)
“Metric that Matters Most”
(MTMM)
Focus on right metric at right time
Copyright © 2018 @danolsen

8. If you just launched a new product,
which would you optimize first?
Acquisition: attracting prospects
Conversion: turning prospects into customers
Retention: ensuring customers remain active
Copyright © 2018 @danolsen

9. If you could track only 1 metric to measure
Product-Market Fit, which metric would it be?
Copyright © 2018 @danolsen

10. Retention Rate
n Retention rate tracks what % of your
customers are still active over time
~80%
never use
app again
Curve either
goes to zero
or flattens out

11. Cohort Analysis
Copyright © 2018 @danolsen

12. Improving Retention Rate Over Time=
Increasing Product-Market Fit

13. Improving Retention Rate Over Time=
Increasing Product-Market Fit

14. Improving Retention Rate Over Time=
Increasing Product-Market Fit

16. Unique Visitors in Last Period x Monthly Retention Rate
Profit = Revenue - Cost
Unique Visitors x Ad Revenue per Visitor
Impressions/Visitor x Effective CPM / 1000
Visits/Visitor x Pageviews/Visit x Impressions/PV
New Visitors + Returning Visitors
Define the Equation of your Business
Peeling the Onion
Advertising Business Model:
Copyright © 2018 @danolsen

18. View Each Metric as a Gauge
Copyright © 2018 @danolsen
Minimum
Possible
Value
Maximum
Possible
Value
Current
Value

19. Identifying Your Metric that Matters Most
n What is the upside potential of each metric?
n How much do we think we can “move the needle”?
n What would the revenue impact be?
n How many resources will it take to move the needle”?
n Developer-days, time, money
n Which metric offers best Return on Investment (ROI)?
Copyright © 2018 @danolsen

20. Copyright © 2018 @danolsen
Prioritizing Product Ideas by ROI
Investment (developer-weeks)
Return(ValueCreated)
Idea C
Idea B
Idea D
Idea A
Idea F
1
1
2 3 4
2
3
4
?

21. Types of Metric ROI SituationsReturn
Investment
Return
Investment
Return
Investment
Metric A
Good ROI
Metric B
Bad ROI
Metric C
Great ROI
Copyright © 2018 @danolsen

22. Case Study
Copyright © 2018 @danolsen

23. • Which metric has highest ROI opportunity?
Case Study:
Optimizing Friendster’s Viral Loop
Active
Users
Prospective
Users
Invite Click
Succeed
Invite
click-through rate
Conversion
rate
Don’t
Click
Fail
Invites per
sender
% of users
sending
invites
• Multiplied together, these metrics determine your viral ratio
Users
% of users
who are
active
= 15%
= 2.3
= 85%
Registration
Process
Copyright © 2018 @danolsen

24. The Upside Potential of a Metric
0
100%
0
100%
0
?
Registration
Process Yield
% of users sending
invitations
Avg # of invites
sent per sender
2.3
85%
15%
Max possible
improvement
0.15 / 0.85 = 18% 0.85 / 0.15 = 570% ? / 2.3 = ?%
Copyright © 2018 @danolsen

25. Is anyone feeling a
sense of déjà vu
right now?

26. The Upside Potential of a Metric
0
100%
0
100%
0
?
Registration
Process Yield
% of users sending
invitations
Avg # of invites
sent per sender
2.3
85%
15%
Max possible
improvement
0.15 / 0.85 = 18% 0.85 / 0.15 = 570% ? / 2.3 = ?%
Copyright © 2018 @danolsen
Metric B
Bad ROI
Metric A
Good ROI
Metric C
Great ROI

27. Okay, so how can we improve the metric?
n How do we increase the average number of
invites being sent out per sender?
n For each idea:
n What’s the expected benefit? (how much will it
improve the metric?)
n What’s the expected cost? (how many engineer-
days will it take?)
n You want to identify highest ROI idea
Copyright © 2018 @danolsen

28. After Launching Address Book Importer…
Copyright © 2018 @danolsen

29. After Launching Address Book Importer…
Copyright © 2018 @danolsen

30. After Launching Address Book Importer…
Copyright © 2018 @danolsen

31. Identify highest
ROI idea
Design and
Implement
Analyze How
the Metric
Changes
Brainstorm
Ideas to
Improve Metric
Copyright © 2018 @danolsen
Lean Product Analytics Process
Identify What
Your Metrics Are
Measure Metrics
Baseline Values
Evaluate Metrics
Upside Potential
Metric
Level
Select
Top Metric
Learn
& Iterate
Global
Level

32. Upping Your Experimentation Game
n Simultaneous tests vs. before-and-after
n Rollout to any % of users vs. only 100%
Feature flags
n Testing multiple variants vs. 1 alternative
n Experimentation platform: buy vs. build
n Experimentation velocity learning velocity
Copyright © 2018 @danolsen

33. I’m teaching a Lean Product
Management Workshop in SF
Oct 24th San Francisco
https://danolsen.eventbrite.com
Save 15% with code SPLIT

34. Thanks!
@danolsen
https://dan-olsen.com
Will notify book winner
on Twitter
Happy to connect on
LinkedIn