2,5-hour Workshop organized by Social UX at The Social Impact Factory in Utrecht (The Netherlands) Learning by doing interviewing and usability testing + lots of reference material

1. userneeds @anous
October 16, 2018
Workshop User Research
by Anouschka Scholten
userneeds
Success depends on
Your Relationship with
Your Users
Social UX presents

2. userneeds @anous

3. userneeds @anous
Make people’s
lives better.

4. userneeds @anous
Useful
Valuable
Accessible
Usable
More about this: Peter Morville Experience honeycomb http://semanticstudios.com/user_experience_design/
Solution ≠
{

5. userneeds @anous

6. userneeds @anous
… an app for everything?
‘Natural Cycles’ Contraceptive App Has People Furious After Causing 37 Accidental
Pregnancies https://twentytwowords.com/new-natural-cycles-contraceptive-app-has-people-furious-after-causing-37-
accidental-pregnancies/

7. userneeds @anous
Fill in destination
+ start location
Be on time for my
appointment-
Prepare my journey
How long does it take? Which route does it take?
I really have to be on time!!
Are there alternatives in case of traffic jams?
I have to do (task)
Is my goal
Are my needs
… also emotiona

8. userneeds @anous
Aaaaaarch….
Where do I fill in my
start location?
I do not want to share or
save or call the destination..
Let’s try again
Oh no, I have no more time, I
really need to leave now
How is this
possible??!!!
I’m so stupid!
Okay, I know the web
version does work … [Opens
the mobile browser]

9. userneeds @anous
Google
Maps
App

10. userneeds @anous
Google
Maps
Web

11. userneeds @anous
Disability isn’t
black & white
Source image https://www.microsoft.com/design/#toolkit
TIPS
Accessibility for UX
designers. A clear and
comprehensive presentation
with lots of examples, tips
and tooling:
https://deanbirkett.name/ac
cessibility-for-ux-designers
Dutch video about this topic
– fun and enlightening:
https://ischagast.nl/pr
esentatie/no-
accessibility

12. userneeds @anous
Try it out with Chrome
extension Funkify
Social Impact Factory site accessible?

13. userneeds @anous
UX Change Agent
Interaction Designer / User Experience Consultant / UX trainer en coach
I design with intention and I’m passionate to empower people and organizations
to make products and services people love to use.
I help to innovate and improve.
Anouschka Scholten
@anous – Userneeds
anouschka.scholten@userneeds.nl

14. userneeds @anous
USER CENTERED
Product or Service development
How?

15. userneeds @anous
https://www.youtube.com/watch?v=yY96hTb8WgI

16. userneeds @anous
Learn from your users

17. userneeds @anous
Measuring = Knowing?
Source: http://www.sitetuners.com/blog/weighted-sort-how-to-use-the-hidden-feature-in-google-analytics-to-find-pages-you-need-to-fix-
now/?utm_content=bufferef9a1&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer

18. userneeds @anous
What happens
Where
How often
Net Promotor Score
CRM Data
WHY
How better ?
Measuring = Knowing?

19. userneeds @anous
or
what the UX- or Conversion expert says?

20. userneeds @anous
ASSUMPTIONS
SUCCES

21. userneeds @anous
You can’t possibly predict/find/be aware of all the issues

22. userneeds @anous
Understand Users
Source: IDEO

23. userneeds @anous23
UX – U = X
user experience
without users is a no go
User Research is key

24. userneeds @anous
Focus
group
Surveys
Verify
test Remote
tracking Usability testing
User drivenMarketing driven
Observation
Interviewing
User Research?

25. userneeds @anous
People do not always do what
you think
you tell them
they think
they say
they do
Observing and asking why makes you
find out what people
really do and need

26. userneeds @anous
3 critical
user research methods
Observation
Interviewing
Usability testing

27. userneeds @anous
One-on-One
Meet your user in their
natural context –
Get out of the building

28. userneeds @anous
No excuses
everyone should and can do it

29. userneeds @anous
Observing behaviour See in context of users
How a Cola company turned falling bar and pub sales around via user
research. An anthropologist walk into a bar...
https://hbr.org/2014/03/an-anthropologist-walks-into-a-bar

30. userneeds @anous
Observations
Asking questions & listen
Doing
Thinking
Source: mbstudio
Interviewing
Foundation for UX insights

31. userneeds @anous
Open or closed
questions?

32. userneeds @anous
Open questions
Offer openings, insights into the user's experience
• Leave the word to the user, user can fill in the
answer himself
• Comprehensive own wordings of what users do and
need (how, when, where and why)
• A rich image, real connection with the user

33. userneeds @anous
Funnel questions
gets to the bottom of the users mind
• Subsequent inquiring questions, like
• Key question:? ”... How do you normally execute
X, how did you do that the last time?"
• Investigative Question 1: "What are you looking for
that kind of information?”
• Investigative Question 2 : "Why do you search for
that information in particular (as formulated by the
user)?”
• Research question 3: "Are there other ways in
which you come to this information?"

34. userneeds @anous
“The important thing is not to stop
questioning. Curiosity has its own
reason for existing”
― Albert Einstein

35. userneeds @anous
Listning with your mouth shut
35

36. userneeds @anous
Listen carefully
to what the user themselves describe things
• Which words they use (terminology usage)
• What considerations (motivations)
• What is very important to them and why
(certainties, control, trust)
• What do they want to prevent (obstacles,
frustrations, pain points)
• What kind of goal do they want to pursue, what
do they want to achieve and why

37. userneeds @anous
Your behaviour and position
influences the process
• Your tone: warm, quiet, soft, harmonious
• Your attitude: open, inviting, prone, quietly
looking, interested, familiar
• Your behavior: friendly, interested, involved,
curious, respectful
Everything the user says / does is okay!

38. userneeds @anous
Interview tips
1. Be deliberately quiet
2. Be open to every answer
3. Listen with attention
4. Ask for
- actual experiences from the past,
- explanation and details
- specific examples
5. Listen, summarize and ask a follow up question
Tips

39. userneeds @anous
A small exercise listening
with the one sitting next to you, in pairs
One interviewer
Asks 1 (open) question
I’ll state the subject, the interviewer translates this in
an open question.
After you formalated the question, you are quiet.
One interviewee
Answers for 3 minutes
Keep talking, no matter what happens.

40. userneeds @anous
Interview results process right after
• Write down the most memorable insights
• Use audio recording for reference
• Add a photo of the person in his / her
environment – write down memorable
quotes
AND Request permission before
making audio recording and photos:
Be clear: audio, photo and report is not
distributed to third parties solely for internal
use
Do you want to use it publicly, or
partly; explicitly ask for permission from the
user.

41. userneeds @anous
Usability Testing
See your user using your product or service

42. userneeds @anous
Usability testing
is about
observing behaviour
Watch the user do what comes naturally,
while performing a task (thinking aloud)
with a product or service.
Don't help.
Not asking about opinion.

43. userneeds @anous
1. Interview
2. Observe while doing the task (say nothing)
3. Evaluate afterwards (open follow up questions)
With actual users (no friends or family – no guerrilla or coffee bar testing!)
1 on 1 session of max. 1,5 hours
Preferably in context of use
Observing behavior

44. userneeds @anous
empathy
a key function of usability testing
to really see, hear and find out
what moves your users
the ability to see an experience
through another person’s eyes
Getting to

45. userneeds @anous
WHY Usability testing??!!!
Not about how often something is said or done –
reactive report
it’s not quantitative research, not about significance
or steering data for management
It’s about WHY things are done
it’s qualitative research
A usability test is mainly to inform us how to do it better –
to get to a better design (and ultimately better
quantitative data)
The $300 Million Button: https://articles.uie.com/three_hund_million_button/

46. userneeds @anous
Usability test to
Detect problems
Optimize
Innovate
Engage users and stakeholders
Convice
Focus
Reduce waste
Less support, development time better spend

47. userneeds @anousPicture credits: Johan Groenen http://www.jgroenen.nl/
NOT the waterfall way

48. userneeds @anousPicture credits: Johan Groenen http://www.jgroenen.nl/
The agile way!

49. userneeds @anous
user checks
& traditional
usability testing

50. userneeds @anous
User Checks –
agile usability testing

51. userneeds @anous
51

52. userneeds @anous
Usability Test traditional
1 round of testing
In a lab environment, eye tracking
Moderator is a UX researcher
Process based

53. userneeds @anous
Traditional Usablity test
drawbacks
• 1 time, at end design process
• An extensive report, little or no redesign
• No retest
• Not in user context
• High costs: lab, eye tracking, special UX
researcher
• Product team indirectly involved
• Schedule well in advance (at least 1 month)

54. userneeds @anous
http://uxpajournal.org/wp-content/uploads/pdf/JUS_Romano_Bergstrom_November_2011.pdf

55. userneeds @anous
Test and retest within 1-4 weeks
Preferably in the users context, quick and easy set up
Team member (Designer, PO,…) conducts the test, team
directly involved
Output is a better design/ better MVP
Value based
User Checks Agile Usability Test

56. userneeds @anous
User Checks Agile Usability Testing
6. Evaluate
Priortize
Improvements
2. User
Checks
round 1
3. Evaluate
Priortize
Improvements
5. User
Checks
round 2
1. Design /
MVP /
Site or App
4. Improved
Design / MVP /
Site or App
7. Improved
Design /
MVP / Site or
App
Repeat
or develop
Based on the RITE-model: https://uxmag.com/articles/the-rite-way-to-prototype

57. userneeds @anous
“Elaborate tests are a waste of resources”
Source: http://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/
75% of the problems found with 3 or 4 user tests

58. userneeds @anous
What to test ?

59. userneeds @anous
Cases

60. userneeds @anous
Design new site ggdhaaglanden.nl
User check with a doctor in his office
GGD Haaglanden project Angi Studio:
http://angistudio.com/cases/ggd-haaglanden.html

61. userneeds @anous
Ik weet niet wat OGGZ betekent, Milieu en
Gezondheid…is dit voor mij?
I don’t know what OGGZ means, Environment and
Health… is this for me?
From
GGD Haaglanden project Angi Studio: http://angistudio.com/cases/ggd-haaglanden.html
Menu indeling: geen overzicht, inzicht, herkenning, focus… voldoet niet aan verwachting, weg wordt niet gevonden
Menu: no overview,- insight, -regognition, -focus… doesn’t match expectations, user can’t find what they need

62. userneeds @anous
Natuurlijk, ook SOA en bezorgd over kinder-
mishandeling… ja, belangrijk!
Off course, also SOA and worried about child
abuse… yes, important!
To
GGD Haaglanden project Angi Studio: http://angistudio.com/cases/ggd-haaglanden.html
Wijzigingen op verschillende niveaus: wat waar onder valt, naamgeving, hiërarchie, toevoeging visuele cues (affordance),
less is more…
Changes on various levels: IA, labeling, hierarchy, visual cues (affordance), focus, less is more…

63. userneeds @anous
“ Bij Meiden verwacht ik 14
tot 21...oh, ik denk dat ik
toch wel in die doelgroep val
als ik het zo lees ? “
“LINDA.Girls.. I expect this is
for 14 till 21 year old… oh,
reading this subscription I
think it actually does fit me?”
Pop-up information about LINDA.meiden
Project Angi Studio: http://angistudio.com
Redesign
LINDA subscription

64. userneeds @anous
“Zo onder elkaar is duidelijk, je kunt er zo
doorheen”
“Very clear and easy to walk through this way,
underneath each other”
Sub home Subscription with
incentive - mobile
Lange lijst om te scrollen
vormde geen enkel probleem
Scrolling was no problem at all
on mobile
2e round: de footer met menu, werd wel
gebruikt
Footer with next step was used
Navigeren met hoofdmenu gebeurde eerst
ronde niet. Sticky menu bood ook geen
oplossing
Navigating via main menu on top wasn’t
used, it was ignored, even with a sticky
menu… dead end alert!
Project van Angi Studio: http://angistudio.com/

65. userneeds @anous
User Checks during redesign Greenchoice.nl, project Angi Studio:
http://angistudio.com/cases/greenchoice.html

66. userneeds @anous
ubsidie aanvraagsysteem Gemeente Den Haag, project Angi Studio:
tp://angistudio.com/cases/gemeentelijk-subsidie-loket.html
User Checks during completely new design

67. userneeds @anous
Outsourcing your user research work
is like outsourcing your vacation.
It gets the job done, but probably
won’t have the effects you were
seeking.
Jared Spool
http://www.uie.com/brainsparks/2011/08/02/outsourcing-your-user-research-is-like-outsourcing-your-vacation/
“

68. userneeds @anous
Let’s just do it

69. userneeds @anous
Live user check test session
additional participants in role ‘observer’

70. userneeds @anous
Collect en prioritize findings
learning by doing

71. userneeds @anous
Come up with solutions and
prioritize
learning by doing

72. userneeds @anous

73. userneeds @anous

74. userneeds @anous
The 4 basics of a User Check
1. Do a short interview upfront
Ask for experiences; how did they… the last time. Let the
user formulate the goal they have, the situation, the task.
Use that for the user check
2. Let your user experience the
prototype/MVP/site
Observe don’t tell. Put your prototype/site in the user’s
hands (or your user in the prototype) and give just the
minimum context so they understand what to do.
Give them the situation and task they formulated in the
interview

75. userneeds @anous
3. Have them talk through their experience
(talk aloud) & Observe actively
Watch how they use (and misuse) what you have given them.
Never immediately “correct” what your user is doing.
Only stimulate to talk aloud: “what are you thinking”
4. Evaluate afterwords: follow up with
questions
This is important; often this is the most valuable part of testing.
You mentioned… what did you mean with that? I saw you
doing… why? What did you expect? How would you call
it?Answer questions with questions (i.e “well, what do you
think that button does”)
The 4 basics of a User Check

76. userneeds @anous
I need 1 volunteer
the user J

77. userneeds @anous
I need 1 case
your website …

78. userneeds @anous
Tell us
• The main goal(s) of the site/application
• Target users
• Main tasks, main questions of the users
• Context of use
Do not tell us how you think the site works now!!!

79. userneeds @anous
Observe the user’s behaviour… you J
1 sticky per issue/observation
Note every occasion the user:
• hesitates, worries, or is forced to think
• misunderstands something
• gets frustrated or annoyed
• gives up
• is surprised
Also think of expressions of emotions that pass by
>> Do not judge, do not think of any solutions yet

80. userneeds @anous

81. userneeds @anous
The findings, issues
• Collect Findings
• Prioritize issues to be solved: prioritize from
the business/user goals perspective, rate them
Do this with thewhole team

82. userneeds @anous
• Collect all findings
• Re-arrange, categorize
them, label them
• Dot-vote (3 per person):
what is most important,
what should we tackle
The problem dot-voted
most: concentrate on that
one for the solutions…

83. userneeds @anous
Improvements
• Improvements on every level, note down
Every post it, a solution, as concrete as possible like
Labeling, content, tone of voice, usp’s, categorization, visual
assets, interaction, concept, flow, persuasion, features (chat …)
> but keep the business/user goals perspective in mind (focus)!
• Prioritize by Impact & Effort (matrix) or dot-
vote
Choose improvement(s) to be re-tested
Do this with thewhole team

84. userneeds @anous
Impact & Effort matrix
Impact: creates high/low value for users
effort: high/low effort in terms of time, knowledge, tools, people…
Source: http://www.innovationgames.com/impact-effort-matrix/
Do this with the
whole team
incl. decision
makers!

85. userneeds @anous
Source: https://www.flickr.com/photos/davegray/5172059225/in/gallery-markacarbone-72157625509228629/

86. userneeds @anous
• Tomorrow or next week:
Implement the solutions
that have the most
positive impact/value for
the user and take the
least effort to realize
• Put the rest of the
solutions on the backlog,
attach actions to it
And re-test !
with other users in 2 or 4
weeks

87. userneeds @anous
The more ‘exposure hours’ with users
(offline) the more improvements in
the design – every 6 weeks 2 hours,
every team member
Jared Spool
http://www.uie.com/brainsparks/2011/08/02/outsourcing-your-user-research-is-like-outsourcing-your-vacation/
“
Fast Path to a Great UX — Increased Exposure Hours

88. userneeds @anous
Lets make people’s lives
better.
Just do it!

89. userneeds @anous
Thank you!
anouschka.scholten@userneeds.nl
www.userneeds.nl
Twitter: @anous
LinkedIn: https://nl.linkedin.com/in/anouschkascholten

90. userneeds @anous
BONUS MATERIAL

91. userneeds @anous
User Checks set up
• Test setup with interview questions and scenario’s/real tasks
• Real target users as participants
• Idealy conducted in context user on their own device
• Record interview (in audio) and the user check with screen
capture
(Live) screen capture via http://www.lookback.io
http://www.airsquirrels.com/reflector/ of
http://www.telestream.net/screenflow/overview.htm of
https://www.techsmith.com/camtasia.html (ook voor Windows). Or
Skype or ather screensharing tool
Audio opname interview: your phone
• 1 test moderator, 1 observer
(more observers are possible, but not too much and really on
the background)

92. userneeds @anous
Key: moderating the test
If they… Say…
Are not talking “What are you thinking?”
Ask you a question (e.g. “Is that
what I should do here?”)
Rephrase the question (e.g “ What do
you think you should do?”)
Get a task right or wrong “Thank you , that is very helpful”
“Thanks for the feedback”
Mess up “Remember, you can’t make any
mistakes. You’re doing a great job.”
Are unsure if they have
completed a task and ask you
“Is this what you would do if you were
doing X at home?”
Criticize the design “Thanks for the feedback”

93. userneeds @anous
Test script-agenda
1. Introduction and questions up front
2. Situation sketch + think aloud instruction
3. Think aloud session
4. Evaluate session, open questions
5. End session - thank participant

94. userneeds @anous
Example Introduction
Thank you for participating, we really appreciate it!
I’m <name> and there is one person besides me <name> is also here. We’re helping
a company to develop a new service / to optimize a website. We are not part of the
company, we don’t have anything to do with the service/product/ website.
This company cares about their customers and thinks it’s very important to involve
potential customers when developing new services. They want the new service to be
valuable for you.
That’s why we want to ask you some questions and along the way we’d like show
you the service/site. You can walk through it.
The interview takes about 50 minutes. If you do not understand something, or
you want to add something, do not hesitate to interrupt me!
Do you mind if we record the session for future reference within our organisation?
Everything we discuss and record is confidential; we will not share things without
your permission!
Any questions?

95. userneeds @anous
Task-think aloud instruction
You are going to see: a draft proposal of the service, which has not
yet been developed/ the site now.
<if a prototype> The proposal is only images. It looks finished and
you can click through it, but most of it is not working yet. It’s also
not as fast as a normal app and you can’t do anything wrong.
<Situation sketch and task instructions>
Please open the link on your mobile phone and walk through the
app / You start at this page
Can you please say things aloud? Not to me, but for yourself?
Everything you do or say is fine, eg. Also "what the hell is this ?!“.
You can start now by clicking the link (or open the site page where
to start)

96. userneeds @anous
Evaluation questions
After the think aloud session, zoom in to what you observed, like:
• You mentioned… what did you mean with that?
• I saw you doing… why? What did you expect? How would you call it?
• You didn’t go to … why not? What do you expect to find?
Also ask service, product or content specific questions, e.g.:
• You read this text thoroughly, why?
• You were going back and fourth between product X and Y, why?
• You choose X and said something about .., what did you mean with
that ?

97. userneeds @anous
Some readings
How to Excite Stakeholders to Start Usability Testing
https://userbrain.net/blog/excite-people-about-usability-testing
Usability Test Demo video
Rocket Surgery Made Easy by Steve Krug: https://youtu.be/QckIzHC99Xc
About asking the right questions
https://blog.intercom.com/product-research-right-questions/
https://www.akendi.com/blog/the-art-of-asking-questions/
Fun facts
13 things Louis Theroux can teach us about user research https://medium.com/fluxx-
studio-notes/13-things-louis-theroux-can-teach-us-about-userresearch-
c80762d41583#.8y8qdrnbt

98. userneeds @anous
Where to find participants for User Research Interviews
Very practical video by Sarah Doody – 9 minutes
All her videos on UX are very enlightning
https://www.youtube.com/watch?v=RP84pQ-PqgA
In Dutch: Hoe werf je in ‘no-time ’respondenten voor jouw test?
https://medium.com/angi-studio/hoe-werf-je-in-no-time-respondenten-voor-
jouw-test-695d33d06885
Questions for user research interviews
Valuable set of questions, concerning:
1. Customer intro questions;
2. Topic specific questions;
3. Product opportunity questions;
4. Product reaction questions
https://www.dropbox.com/s/prza1xmywbuy0fc/BONUS%20Interview%20Q
uestions%20For%20User%20Research%20-
%20Copyright%20Sarah%20Doody.pdf?dl=0
TIPS