SlideShare a Scribd company logo
1 of 34
Unmoderated remote user
        testing:
    Do’s and don’ts



       2012 © Humix, all rights reserved
Remote User Testing




     2012 © Humix, all rights reserved   2
Synchronous vs asynchronous




         2012 © Humix, all rights reserved   3
Moderated vs unmoderated




        2012 © Humix, all rights reserved   4
Moderated usability testing




         2012 © Humix, all rights reserved   5
Unmoderated remote
  usability testing




     2012 © Humix, all rights reserved   6
Figures




Source: www.measuringusability.com/blog/ux-methods.php 2011 UPA Salary Survey (biennal)


          2012 © Humix, all rights reserved                                    7
ADVANTAGES


       2012 © Humix, all rights reserved   8
Test large groups




    2012 © Humix, all rights reserved   9
Real environments




    2012 © Humix, all rights reserved   10
Reduce costs




  2012 © Humix, all rights reserved   11
Recruitment




 2012 © Humix, all rights reserved   12
Organisation




  2012 © Humix, all rights reserved   13
Fast results




 2012 © Humix, all rights reserved   14
Quantitative & Qualitative




        2012 © Humix, all rights reserved   15
Intercept users




   2012 © Humix, all rights reserved   16
Self-reported feedback




      2012 © Humix, all rights reserved   17
DRAWBACKS


       2012 © Humix, all rights reserved   18
Gold diggers




  2012 © Humix, all rights reserved   19
No interaction




  2012 © Humix, all rights reserved   20
Only survey answers




     2012 © Humix, all rights reserved   21
False positives




   2012 © Humix, all rights reserved   22
Task time…




 2012 © Humix, all rights reserved   23
TOOLS


        2012 © Humix, all rights reserved   24
Choose the right tool
Functionality of tools has a
huge impact on the type of
info you’re able to gather




                2012 © Humix, all rights reserved   25
Choose the right tool
• What is your budget?
• What is your goal?
• How much support do you need?
• Will you be able to conduct competitive                research?
• Can you test prototypes?
• Can you export findings?
• How will the results be visualized?
• Does it come in different languages?
• Can you combine different methods
  using the same tool?
• Are there integrated recruiting tools?

                     2012 © Humix, all rights reserved               26
Information Architecture tools
Card sorting




               2012 © Humix, all rights reserved   27
Information Architecture tools
Tree testing




               2012 © Humix, all rights reserved   28
Click based feedback
Task & Interface testing




                2012 © Humix, all rights reserved   29
Feedback gathering




     2012 © Humix, all rights reserved   30
Unmoderated remote
usability testing tools




      2012 © Humix, all rights reserved   31
When to use URUT?
• Not a replacement for moderated testing
  or research
• Best used to complement
  your qualitative research




               2012 © Humix, all rights reserved   32
Interesting reading
• Remote Research: Real Users,
  Real Time, Real Research by Nate
  Bolt, Tony Tulathimutte, 2010

• Beyond the usability lab: Conducting
  Large-scale Online User Experience
  Studies, Bill Albert, Tom Tullis, Donna
  Tedesco, 2010



                   2012 © Humix, all rights reserved   33
www.humix.be
                   Lonneke.Spinhof@humix.be
                   @humix @lonnekespinhof




2012 © Humix, all rights reserved

More Related Content

Similar to Unmoderated Remote Usability Testing

Uxperts mobi 2013 what about testing
Uxperts mobi 2013   what about testing Uxperts mobi 2013   what about testing
Uxperts mobi 2013 what about testing Winnie Valbracht
 
A user centred CRM
A user centred CRMA user centred CRM
A user centred CRMHumix
 
Competitive Intelligence and Social Media
Competitive Intelligence and Social MediaCompetitive Intelligence and Social Media
Competitive Intelligence and Social MediaAlexandra Nelles
 
Taxonomy Design for the Short on Time
Taxonomy Design for the Short on TimeTaxonomy Design for the Short on Time
Taxonomy Design for the Short on TimeFred Leise
 
Cormas: Modelling for Citizens with Citizens. Building accessible and reliabl...
Cormas: Modelling for Citizens with Citizens. Building accessible and reliabl...Cormas: Modelling for Citizens with Citizens. Building accessible and reliabl...
Cormas: Modelling for Citizens with Citizens. Building accessible and reliabl...Oleksandr Zaitsev
 
UX and the path to success
UX and the path to successUX and the path to success
UX and the path to successOfir Rushinek
 
Improving the Mobile Application User Experience (UX)
Improving the Mobile Application User Experience (UX)Improving the Mobile Application User Experience (UX)
Improving the Mobile Application User Experience (UX)TechWell
 
Stop Talking About What You Do and Start Talking About What You Deliver - Pro...
Stop Talking About What You Do and Start Talking About What You Deliver - Pro...Stop Talking About What You Do and Start Talking About What You Deliver - Pro...
Stop Talking About What You Do and Start Talking About What You Deliver - Pro...Tim Creasey
 
Data Mining for Moderation of Social Data
Data Mining for Moderation of Social DataData Mining for Moderation of Social Data
Data Mining for Moderation of Social DataFernando G. Guerrero
 
4 philip lew - how to improve the mobile user experience
4   philip lew - how to improve the mobile user experience4   philip lew - how to improve the mobile user experience
4 philip lew - how to improve the mobile user experienceIevgenii Katsan
 
INSEMTIVES Tutorial ISWC2011 - Session5
INSEMTIVES Tutorial ISWC2011 - Session5INSEMTIVES Tutorial ISWC2011 - Session5
INSEMTIVES Tutorial ISWC2011 - Session5INSEMTIVES project
 
FluentConf 2016: Avoiding Critical UX Mistakes with Philip Lew
FluentConf 2016: Avoiding Critical UX Mistakes with Philip LewFluentConf 2016: Avoiding Critical UX Mistakes with Philip Lew
FluentConf 2016: Avoiding Critical UX Mistakes with Philip LewXBOSoft
 
Learning Strategies for a Transparent World: Achieving Results with Virtual a...
Learning Strategies for a Transparent World: Achieving Results with Virtual a...Learning Strategies for a Transparent World: Achieving Results with Virtual a...
Learning Strategies for a Transparent World: Achieving Results with Virtual a...Cynthia Clay
 
Hubscope te october 2015
Hubscope te october 2015Hubscope te october 2015
Hubscope te october 2015John Cant
 
Learning Anywhere: Mobile, Social and Virtual
Learning Anywhere: Mobile, Social and VirtualLearning Anywhere: Mobile, Social and Virtual
Learning Anywhere: Mobile, Social and VirtualCynthia Clay
 
Ensure network migration success with Change Management
Ensure network migration success with Change ManagementEnsure network migration success with Change Management
Ensure network migration success with Change ManagementComms Connect
 
Learning Strategies for a Transparent World
Learning Strategies for a Transparent WorldLearning Strategies for a Transparent World
Learning Strategies for a Transparent WorldCynthia Clay
 

Similar to Unmoderated Remote Usability Testing (20)

Uxperts mobi 2013 what about testing
Uxperts mobi 2013   what about testing Uxperts mobi 2013   what about testing
Uxperts mobi 2013 what about testing
 
Uxperts mobi 2013 what about testing
Uxperts mobi 2013   what about testing Uxperts mobi 2013   what about testing
Uxperts mobi 2013 what about testing
 
A user centred CRM
A user centred CRMA user centred CRM
A user centred CRM
 
Competitive Intelligence and Social Media
Competitive Intelligence and Social MediaCompetitive Intelligence and Social Media
Competitive Intelligence and Social Media
 
High Impact VILT
High Impact VILTHigh Impact VILT
High Impact VILT
 
Taxonomy Design for the Short on Time
Taxonomy Design for the Short on TimeTaxonomy Design for the Short on Time
Taxonomy Design for the Short on Time
 
Cormas: Modelling for Citizens with Citizens. Building accessible and reliabl...
Cormas: Modelling for Citizens with Citizens. Building accessible and reliabl...Cormas: Modelling for Citizens with Citizens. Building accessible and reliabl...
Cormas: Modelling for Citizens with Citizens. Building accessible and reliabl...
 
UX and the path to success
UX and the path to successUX and the path to success
UX and the path to success
 
Improving the Mobile Application User Experience (UX)
Improving the Mobile Application User Experience (UX)Improving the Mobile Application User Experience (UX)
Improving the Mobile Application User Experience (UX)
 
People and the Intranet
People and the IntranetPeople and the Intranet
People and the Intranet
 
Stop Talking About What You Do and Start Talking About What You Deliver - Pro...
Stop Talking About What You Do and Start Talking About What You Deliver - Pro...Stop Talking About What You Do and Start Talking About What You Deliver - Pro...
Stop Talking About What You Do and Start Talking About What You Deliver - Pro...
 
Data Mining for Moderation of Social Data
Data Mining for Moderation of Social DataData Mining for Moderation of Social Data
Data Mining for Moderation of Social Data
 
4 philip lew - how to improve the mobile user experience
4   philip lew - how to improve the mobile user experience4   philip lew - how to improve the mobile user experience
4 philip lew - how to improve the mobile user experience
 
INSEMTIVES Tutorial ISWC2011 - Session5
INSEMTIVES Tutorial ISWC2011 - Session5INSEMTIVES Tutorial ISWC2011 - Session5
INSEMTIVES Tutorial ISWC2011 - Session5
 
FluentConf 2016: Avoiding Critical UX Mistakes with Philip Lew
FluentConf 2016: Avoiding Critical UX Mistakes with Philip LewFluentConf 2016: Avoiding Critical UX Mistakes with Philip Lew
FluentConf 2016: Avoiding Critical UX Mistakes with Philip Lew
 
Learning Strategies for a Transparent World: Achieving Results with Virtual a...
Learning Strategies for a Transparent World: Achieving Results with Virtual a...Learning Strategies for a Transparent World: Achieving Results with Virtual a...
Learning Strategies for a Transparent World: Achieving Results with Virtual a...
 
Hubscope te october 2015
Hubscope te october 2015Hubscope te october 2015
Hubscope te october 2015
 
Learning Anywhere: Mobile, Social and Virtual
Learning Anywhere: Mobile, Social and VirtualLearning Anywhere: Mobile, Social and Virtual
Learning Anywhere: Mobile, Social and Virtual
 
Ensure network migration success with Change Management
Ensure network migration success with Change ManagementEnsure network migration success with Change Management
Ensure network migration success with Change Management
 
Learning Strategies for a Transparent World
Learning Strategies for a Transparent WorldLearning Strategies for a Transparent World
Learning Strategies for a Transparent World
 

More from Humix

The impact of GDPR on tracking (so far)
The impact of GDPR on tracking (so far)The impact of GDPR on tracking (so far)
The impact of GDPR on tracking (so far)Humix
 
Klipfolio - Your Swiss Knife on data
Klipfolio - Your Swiss Knife on dataKlipfolio - Your Swiss Knife on data
Klipfolio - Your Swiss Knife on dataHumix
 
Shopping behaviour of the future
Shopping behaviour of the futureShopping behaviour of the future
Shopping behaviour of the futureHumix
 
Why the "web analytics guy" is your key to digital leadership
Why the "web analytics guy" is your key to digital leadershipWhy the "web analytics guy" is your key to digital leadership
Why the "web analytics guy" is your key to digital leadershipHumix
 
Google Analytics Hacks for Magento
Google Analytics Hacks for MagentoGoogle Analytics Hacks for Magento
Google Analytics Hacks for MagentoHumix
 
It all starts with a measurement framework
It all starts with a measurement frameworkIt all starts with a measurement framework
It all starts with a measurement frameworkHumix
 
How to get the most out of on site questionnaires
How to get the most out of on site questionnairesHow to get the most out of on site questionnaires
How to get the most out of on site questionnairesHumix
 
Stay clear from report monkeys when making data driven decisions
Stay clear from report monkeys when making data driven decisionsStay clear from report monkeys when making data driven decisions
Stay clear from report monkeys when making data driven decisionsHumix
 
Everything they do can and will be used in favor of them.
Everything they do can and will be used in favor of them.Everything they do can and will be used in favor of them.
Everything they do can and will be used in favor of them.Humix
 
What's new in Adwords?
What's new in Adwords?What's new in Adwords?
What's new in Adwords?Humix
 
What about enhanced e-commerce?
What about enhanced e-commerce?What about enhanced e-commerce?
What about enhanced e-commerce?Humix
 
Trends in online behavior
Trends in online behaviorTrends in online behavior
Trends in online behaviorHumix
 
Online shopping behavior
Online shopping behaviorOnline shopping behavior
Online shopping behaviorHumix
 
More Efficiency with Dashboards
More Efficiency with DashboardsMore Efficiency with Dashboards
More Efficiency with DashboardsHumix
 

More from Humix (14)

The impact of GDPR on tracking (so far)
The impact of GDPR on tracking (so far)The impact of GDPR on tracking (so far)
The impact of GDPR on tracking (so far)
 
Klipfolio - Your Swiss Knife on data
Klipfolio - Your Swiss Knife on dataKlipfolio - Your Swiss Knife on data
Klipfolio - Your Swiss Knife on data
 
Shopping behaviour of the future
Shopping behaviour of the futureShopping behaviour of the future
Shopping behaviour of the future
 
Why the "web analytics guy" is your key to digital leadership
Why the "web analytics guy" is your key to digital leadershipWhy the "web analytics guy" is your key to digital leadership
Why the "web analytics guy" is your key to digital leadership
 
Google Analytics Hacks for Magento
Google Analytics Hacks for MagentoGoogle Analytics Hacks for Magento
Google Analytics Hacks for Magento
 
It all starts with a measurement framework
It all starts with a measurement frameworkIt all starts with a measurement framework
It all starts with a measurement framework
 
How to get the most out of on site questionnaires
How to get the most out of on site questionnairesHow to get the most out of on site questionnaires
How to get the most out of on site questionnaires
 
Stay clear from report monkeys when making data driven decisions
Stay clear from report monkeys when making data driven decisionsStay clear from report monkeys when making data driven decisions
Stay clear from report monkeys when making data driven decisions
 
Everything they do can and will be used in favor of them.
Everything they do can and will be used in favor of them.Everything they do can and will be used in favor of them.
Everything they do can and will be used in favor of them.
 
What's new in Adwords?
What's new in Adwords?What's new in Adwords?
What's new in Adwords?
 
What about enhanced e-commerce?
What about enhanced e-commerce?What about enhanced e-commerce?
What about enhanced e-commerce?
 
Trends in online behavior
Trends in online behaviorTrends in online behavior
Trends in online behavior
 
Online shopping behavior
Online shopping behaviorOnline shopping behavior
Online shopping behavior
 
More Efficiency with Dashboards
More Efficiency with DashboardsMore Efficiency with Dashboards
More Efficiency with Dashboards
 

Unmoderated Remote Usability Testing

Editor's Notes

  1. Remote usability testingdenotes a situationwhere “the evaluators are separated in spaceand/or time from users”Thisdefinitionemphasizestwogeneral types of methods: synchronousandasynchronous.
  2. Reducecosts, depending on the tool youuse
  3. It is easierto get participantstovolunteerbecausethey do not have totravel.
  4. Fewerlogisticsto manage and no needtoworryabout no-shows andgetting last minute replacements
  5. Partial automated results  faster resultsEasier to conduct multiple small tests
  6. Both quantitativeand (some) qualitative dataQuantitative:Taks-completionrateTime on taskTime on pageClickstreampathsSatisfaction ratings or opinion rankingsQualitative:Most of these tools canalsocapturequalitative feedback as users complete theirtasks - such as users’ suggestionsandcomments
  7. Someparticipantsmayonlybeinterested in earning the honorarium you’veprovided as an incentive.You have little or no contact and thus control over the motivation of participants
  8. Nothing beats watchingparticipants in real time andbeingabletoaskprobingquestionsaboutwhatthey are doing as it’s happening.
  9. Whatparticipants report on surveyscanbevery different in comparisontowhattheyactually do.
  10. It’s possibleforparticipantstothinkthey’vesuccessfullycompleted a taskwhentheyhaven’t
  11. Does it matter ifittooksomeonelongerto complete a task?