This presentation is from Affiliate Summit West 2017 (January 15-17, 2017 in Las Vegas, Nevada). Session description: Passive user testing is the best way to get info about users in the wild. You’ll learn what makes this type of test powerful, how to get this valuable info, & exactly how it can improve conversions.

1. Passive User Tests To
Improve Conversion Rates
Justin Rondeau
Director of Optimization

2. What do you think of when I say
‘User Testing’?

6. THAT’S JUST HALF
OF IT!

7. ACTIVE USER
TESTING
PASSIVE USER
TESTING

8. Active User Tests: A user test where they
user is aware they are in a testing
environemnt.

9. SURVEYS
WHAT YOUR CUSTOMERS ARE THINKING

10. Survey Pros
1. Quick Detailed Responses
2. Easy To Segment Data To
Find Trends
3. Know Exactly What Your
Customer Thinks

11. Survey Cons
1. Only a Subset of Audience
2. Inaccuracies Due to
Question Construction
3. Lots of External Variables

12. Tech To Use

13. Protip:
Trigger QUICK surveys at key times, e.g.,
abandonment, after purchase!
…and…

14. Use These Types Of Questions
• Was it was easy to find what you were looking for?
• Were the words/vocab used to define categories/sub-categories clear to you?
• Do you have any suggestions to improve our website navigation?
• Does our website look credible to you?
• Is our pricing clear?
• Is there anything else you’d like to know before signing up with us?
• Will you shop with us again? Why/why not?
• Do you think the form has any confusing/unnecessary input fields?

15. USABILITY TESTS
LITERALLY SIT DOWN AND WATCH THEM USE YOUR SITE

16. UsabilityTests Pros
1. Individual & Detailed
Responses
2. Relatively Easy To Recruit

17. UsabilityTests Cons
1. Expensive (If True Lab)
2. Small Sample Size (if on
Budget)
3. Time Consuming

18. Tech To Use

19. Tech To Use

20. Problems With Active
User Tests
1. Hawthorne Effect
2. Survey Bias
3. Researcher’s Bias
4. Non Response Bias

21. “Act as you normally
would…”

23. Problems With Active
User Tests
1. Hawthorne Effect
2. Survey Bias
3. Researcher’s Bias
4. Non Response Bias

25. Problems With Active
User Tests
1. Hawthorne Effect
2. Survey Bias
3. Researcher’s Bias
4. Non Response Bias

26. Research Question:
“How Much Do
You Love Star
Wars?”

27. Problems With Active
User Tests
1. Hawthorne Effect
2. Survey Bias
3. Researcher’s Bias
4. Non Response Bias

28. Target Market

29. Target Market

30. Biggest Problem With Active User Tests:
BIG DIFFERENCE BETWEEN
WHAT YOU SAY & WHAT YOU DO

31. Passive User Tests: A user test where they
user is unaware they are in a testing
environment.

32. HEATMAPS
WHERE YOUR USERS ARE CLICKING (ON AVERAGE)

33. What you can learn
1. Where your visitors are
clicking. (Aggregate)
2. Where your visitors are
scrolling.
…or more importantly where
they aren’t clicking or scrolling.

34. Example
40.9% Increase In Sales When
We Actually Made The Link
Clickable…
…Go Figure.

35. Tough Realization
Only 6% of traffic made it this
far…

36. RECORDINGS
WHAT YOUR VISITORS ARE ACTUALLY DOING

37. What you can learn
1. How individuals are using
your site.
2. Points of frustration on
your page.
3. Your visitors page journey.

38. What you can learn
1. We noticed visitors
consistently scrolled first instead
of filling out the form.
2. Realized they were looking for
price.
3. Came up with better designs
that addressed this issue.

39. FORM ANALYTICS
WHICH FIELDS ARE PROBLEMATIC

40. What you can learn
1. Which fields are causing form
drop off.
2. Which fields are confusing.
3. Which fields aren’t causing
problems.
4. Oh Yeah! Form Conversion
Rate :)

42. FUNNEL
VISUALIZATION
SEE WHERE VISITORS DROP OFF

43. What you can learn
1. Easily identify holes in
funnel
2. Pick your next page to
update, run a more in depth
test, or split test a new
hypothesis.

44. CUSTOMER SERVICE
QUESTIONS
WHAT THEY REALLY THINK

46. Top Customer Service Questions
1. What are the top 3 questions you get from potential customers?
2. What do you answer to these questions?
3. Are there any particular aspects of X that people don’t understand?
4. What aspects of X do people like the most/least?
5. Have we missed anything important? Got something to add?

47. Protip:
ASK CUSTOMER SERVICE EACH
WEEK THE TOP 5 ISSUES

48. SALES QUESTIONS
WHAT WILL CONVINCE YOUR CUSTOMER TO BUY

49. Top Sales Question
1. What are the top 3 questions you get from potential customers?
2. What do you answer to these questions?
3. What is the biggest barrier keeping people from buying your product?
4. Are there any specific selling points that work particularly well? If so,
which ones?
5. Have we missed anything important? Got something to add?

50. Passive
User Tests
Why?
1. An example of what your
customer actually does vs.
what they say.
2. Only limitation is traffic
3. Identifies structural issues
that bypass test periods.

51. QUESTIONS?
CONTACT INFO
EMAIL:
JUSTIN.RONDEAU@DIGITALMARKETER.COM
TWITTER:
@JTRONDEAU
LINKEDIN:
linkedin.com/in/jtrondeau