The document discusses the freemium business model and the importance of A/B testing for improving conversion rates. It provides an overview of how Webs uses a freemium model to attract millions of free users and then employs A/B testing and cohort analysis to optimize the user experience and increase monetization through paid upgrades. Key points include how testing small changes like buttons, copy, forms can have major impacts on signups and revenue, and how segmenting users allows testing effects on different groups to be analyzed over time. The document advocates that continuous testing is crucial for freemium companies to rapidly iterate and better serve their large free user bases.

1. The Freemium Model & A/B Testing HostingCon 2011 By Haroon Mokhtarzada CEO & Co-Founder, Webs.com

2. Introduction 2 Haroon Mokhtarzada Co-Founder and CEO Entrepreneur from the age of 10. Co-founded Webs in 2001 with brothers Zeki and Idris while studying Economics at the University of Maryland. Obtained J.D. from Harvard Law School in 2005 and thereafter joined Webs full-time. Founding Vision “Build a free website creation tool so easy that Mom can use it” Funding Raised $12.1 million from Novak Biddle, Columbia Capital and prominent angels in August 2006.

3. About Webs 3 Webs is a leading provider of online “front door” solutions for micro businesses across web, mobile and social platforms. 40 million websites created since inception Started as a tool for anyone to easily create a website for free and evolved to focus on online solutions for micro businesses Recently acquired and relaunchedPagemodo.com (Freemium Facebook page creation tool) We run an average of 3 different A/B tests at any given time

4. Freemium – A Success Story 4 As a disruptive business model, freemium has been the driving force behind many successful startups and $billions in value creation

5. Simple Fundamentals 5 Freemium is a business model that works well in scenarios where the marginal cost of users can be driven towards 0 over time or through economies of scale ARPU $ Per Active User Variable Cost Time (or # users)

6. What Is a Freemium Product/Service? Freemium is NOT… Free trial or sample – user forced to pay after period of time Shareware – user forced to pay to realize actual benefit of product 6

7. What Is a Freemium Product/Service? Freemium IS… A free, usable product/service That attracts users through its price tag and free user base And monetizes with optional paidpremium features(typically subscription based) 7

8. Freemium is merely one option: Popular web business models: Sell a product to customers (premium) Upsell a product to your users (freemium) Sell other people’s products to your users (ads) Sell your users’ data (ex: addthis) 8 Thanks!

9. Why we “chose” Freemium 10 yrs ago Broke - No marketing budget Clueless - Uncertainty about subscription/pricing model Frugal- Cost per user was low 9

10. Challenges for Freemium Players Low Conversion - People who may have been willing to pay normally do not pay because the free product is sufficient Low ARPU – makes it difficult to accelerate growth via traditional marketing methods Infrastructure– millions of free users can create non-trivial and costly infrastructure problems Customer Support – free users often still demand support 10

11. Benefits of the Freemium Model Viral – Rapid adoption, opens new, affordable, channels for getting users Disruptive – Compete with even the most entrenched premium competitors by giving it away for free Superior UI/UX – Offer a better product through rapid testing / customer development 11

12. The Freemium Offering on Webs 12

13. Leveraging our Free Users 13 Direct Marketing from Free Users Links on bottom of sites Branding - Logo on bottom of sites SEO value from websites back to Webs.com URL has webs.com in it Indirect Marketing from Free Users Word of mouth Size / Credibility

14. How Premium Businesses Can Leverage Freemium 14 Offer a limited version for free (but be careful NOT to cannibalize) Create new, related free or freemium product, and use as marketing funnel for your paid product Partner with a freemium product, offer their premium product to your audience

15. 15 Part II: Testing… Testing… 1 2 3 HostingCon 2011

16. The Freemium Conversion Funnel 16 “What gets measured gets improved”

17. Conversion Timeline 17 Understanding When Your Users Convert Is Key Our likelihood of conversion decreases over time Why? Users are publishing a new website and want all features at time of launch Implications: Focus on upselling EARLY in customer lifecycle Other models: User more likely to convert as time goes on: Dropbox (space fills up) Mailchimp (email list grows) Implications: Deemphasize upsell up front, and focus on getting as much usage as possible

18. Simple A/B Testing Define some goal event Create test treatments Test - randomly split visitors into test & control groups; measure success to goal Good for conversion events with immediacy Ex: Homepage, signup page, shopping cart 18

19. Simple A/B Testing What to test? Calls-to-action (buttons) Copy, Color, Size, Placement, Hover states Forms/Fields Descriptors, Size, Length/number, etc. Text Headlines, Value Props, Links, Amount (long/short) Imagery Segment results by source, use landing pages * Bonus Points 19

20. AbTests.com 20 Great site for testing inspiration

21. Case Study 21 Control Test Shortened Signup Flow Tested shorter 1-page flow against 2-page flow with more fields Shortened version created 10,000 more signups over 3 weeks Going 100% translates to over 25,000 more signups/month!

22. A/B Test Reporting (GWO) Confidence increases over time Low variation is good Google Website Optimizer is good because: Provides visitors a consistent experience Measures & reports confidence / significance Offers multivariate (combination) testing 22

23. Interpreting Results When A/B Testing is Not Enough Freemium Problem: Signup != $$$ A/B signup test only told us which produced more USERS, but does NOT tell us which produced more $$ Example: Asking more questions in signup will reduce signups but may increase user success rate and make more $$ 23

24. Database Driven Cohort Analysis Solution: A system that permanently fixes a user to a test group, allowing one to run results analysis on any variable that is also attached to that user (i.e. $ spent) Offers the ability to compare users who received different treatments through your entire funnel over time 24

25. Cohort Funnel Analysis Example A/B testing would have shown Test2 as the winner Test 1 makes more money. Tradeoff? 25 * Dummy data

26. How to Set Up Cohort Testing Thanks Eric Ries! Display Treatment Yes In Cohort? User Enters Assign to Cohort Coin Toss No Quick & Dirty Shortcut: Mod on userid(ex: show even users one treatment and odd users another) 26

27. Visualizing Results 27 Day to day reporting on metrics like Premium $ can often be lumpy. Looking at a cumulative view helps visualize more smoothly Diverging lines means one is superior Parallel lines apart means one got lucky;)

28. Segmentation & Cohort Analysis Advantages Integrated front- and back-end testing Complete ownership of user data Ability to track impact to user long after user-facing experience has gone away Provide users with consistent test experience that integrates your back-end - Great for pricing & promotional tests Ability to “slice” data by internal user variables 28

29. Advanced Segmentation 29 Total/Aggregate results can hide the real story Not all users are the same Segmenting data and “slicing” can produce surprising gains Treatments can be applied to different segments accordingly

30. Visualizing Advanced Segmentation 30 Cumulative Charts Different reports for each segment Can use end results to find interesting variable combinations, then check for significance

31. Why A/B Testing is Crucial 31 Lots of Leverage – since your free customers are often your largest referring source, improving your funnel can have compounding effects on both virality and conversion. Users constantly surprise – its impossible to guess what users will do. That’s why all great companies use testing. Validate ideas and time/energy spent on product efforts Better Serve Your Users – build a better product and improve your intuition about your customers

32. Summary 32 The freemium model is here to stay When implemented properly, freemium is a disruptive business model allowing rapid growth in new or existing markets Improving conversion is crucial to success, and requires rapid iteration and A/B testing Rapid iteration and testing cannot be done without large influx of free users, favoring those who have achieved scale Successful freemium products are always-improving, and tend to have the superior metrics & UX/UI (out of sheer necessity!) Premium players who do not want to disrupt legacy business can can partner with freemium players to leverage their assets

33. 33 Q&A Haroon Mokhtarzada, CEO & Co-Founder of Webs @haroon, haroon@webs.com