Healthcare companies should embrace iterative user research so that they may design products that aligns with their customers' wants and needs. UX research studies are not clinical trials - they are a means of learn how to best design a product for customers.
23. #HxR2014
Usability studies are qualitative
Positive: Users were able to find the
“Buy Now” button.
3 out of 8 participants mentioned
that the picture had no relevance to
them.
Consider: replacing this with an
infographic or another picture that
may draw in users.
6 out of 8 participants mentioned
that the text in the buttons was hard
to read.
Consider: increasing the contrast of
the text to ease readability.
24. #HxR2014
Finding Recommendation
Participants said that the text was hard to read Increase the contrast of the text
Participants were unable to locate the product; they
said they expected it under Products & Services
Move XYZ product to the Product & Services area
Participants were unable to discern the top from the
bottom of the vial
Include a visual cue as to which is the top and bottom
of the vial
Participants were able to figure out how to use the
insulin pump, but said that the instructions were very
hard to follow
Add detail throughout the instruction manual
Typical Usability Study Results
26. #HxR2014
Incremental Usability vs. For FDA Submission
• Small studies throughout the product lifecycle
• Qualitative; 5-8 participants per demographic
• Learn user needs and expectations and use
these to guide design
• Fewer, larger studies closer to product launch
• Quantitative and qualitative; >30 participants
• Validate that the product is indeed usable for
the target audience
A usability study is a one-on-one session between the participant and the moderator.
The participants perform a series of tasks on the interface while thinking-aloud. That is, telling us what they are thinking as they perform the task.
The moderator encourages think-aloud and asks probing questions where appropriate.
Most usability studies are screen based: websites and software.
There is typically a two-way mirror and a projection of the screen so observers can see what the participant is doing.
We can also perform usability studies on mobile phones, apps, and physical devices.
Instead of using a screen, we’ll setup a webcam to capture what the participant does.
Not using medicine during medical device testing, always water. Participants never use the devices on themselves.
Usability studies are only one type of UX research method. The wide array allows us to choose the right method for the right part of the project.
The goal is to get the right type of data at the right time: formative data at the beginning of the project, evaluative during, and summative at the end.
There are both qualitative and quantitative methods.
Little “e” ethnography is when we observe participants in their natural environment.
This allows us to see the environmental factors and how participants REALLY use the device.
Not to be confused with big “E” Ethnography, where researchers study cultures for years.
The moderator observes participants’ behavior, combines these with knowledge of design psychology, and determines design recommendations.
Based on participant behavior, the moderator asks probing questions to understand their motivation and expectations.
Most UX research is qualitative – no statistical significance. It’s not worth the cost.
UX research is about the getting the best bang for your buck – the best, most useful data, with as few participants as possible and in the least amount of time.
UX research is not definitive – we’re not “proving” anything.
We’re giving the design direction based on user input.
You never know what users are going to do unless you watch them use your product.
Designers are great – they making amazing interactions and visuals that make interfaces sing.
But, they are not your user. Don’t leave it up to the designers to determine how your product works.
Who knows more about grandma’s needs? The designer…
…or grandma?
When you don’t get input from users, you get a hybrid of needs, and it’s just ugly.
Say it with me: User Research studies are not clinical trials!
We’re not testing the efficacy of drugs or devices – we’re testing the usability and user experience.
We’re not aiming for statistically significant data – it’s not worth the cost.
We’re not proving anything, we’re getting design direction.
So don’t ask your agency/designer/researcher to get usability data from 30 participants per demographic. Save that for your FDA validation studies.
Don’t rely on marketing focus groups for design direction.
They are learning what product to sell, we are learning how to make the product align with customer expectations.
The fool’s errand of: “Let’s get as many people in the room at once so we can say that we had input for XX # of people.”
In the 90’s, Bob Virzi found that 5-8 participants will generally find 85% of your usability problems.
More complex now, but it’s still a good rule of thumb.
Minimum of 5 participants per demographic, if they truly represent different interaction/information needs.
Typically, 12-18 participants per usability study, depending on the number of demographics.
This is an example of what a usability report looks like.
Researchers will point out prioritized opportunities for improvement.
Reporting on the positive findings is important so we don’t break what is already working.
The overall goal is to report on the study findings and the associated recommendations.
Always report a recommendation for each finding.
Ensure the findings are prioritized, a 1-3 scale is typically sufficient.
Don’t be scared of the FDA, talk to them. They’re helpful.
Not every report and research study needs to be submitted to the FDA, only validation studies for product approval.
FDA validation studies are statistically significant studies conducted during and at the end of the product development lifecycle.
Incremental usability studies will help you make a product align with user expectations; validation usability studies will prove to the FDA that it meets usability requirements.
Incremental studies need only a few participants to maximize ROI; the goal of these is to guide the design.
FDA validation studies need a statistically significant number of participants to validate the product’s usability.
Again, it’s not just about usability. Choose the right UX research method for the right time in the project.
Lesson #1: Make information pop
Find the red circle
Find the red circle
Find the red circle
Pretty hard, eh?
Making information pop = taking advantage of how our mind works and bolstering its weaknesses.
We’re good at judging the length of a line (right), but not the area of a polygon (left)
The graphic on the right pops more, it requires less thinking.
SimIndia example – multiple areas of information popping
PregnantMe example – both information and interaction pop
CDC example – it’s not just about making information pop, interaction needs to as well
Lesson #2: Mobile First, Where Appropriate
But mobile isn’t always appropriate; people don’t need all of your functionality in the mobile environment.
Desktop is still important for privacy-related matters.
Plan out what is important to your users at different touch points.
Determine what functionality and information they need and want for the different platforms.
Lesson #3: Companies ask their users to do weird things.
Having to log into multiple systems.
I watched my Optician copy and paste information from one system to another.
Systems need to talk to each other.
This woman had to contort herself to reach the injection site, while maintaining the strength to press the button.
This set of patients typically used a rubber band to hold two pieces of this device together.
The inhaler that needed to be kept upright at all times.
Greatest myth: that we can’t talk to doctors and patients.
UX researchers take HIPAA very seriously – we will stop participants from divulging too much information and protect any collected data.
Don’t polish a turd by only conducting a single usability study at the end of the project.
Conduct iterative research to inform the project throughout its lifecycle.
Formative = at the beginning of the project.
Evaluative = after a design artifact is available.
Summative = after a high-fidelity prototype or finished product is available.
Many different research options for each stage of a project.
Use qualitative throughout the design project, but validate with quantitative at the end.
Do all of this to build empathy with your patients so you truly understand their wants and needs.