Slides used in workshop session B7 on "User Testing on a Shoestring Budget" at the IWMW 2006 event held at the University of Bath on 14 - 16 June 2006. See http://www.ukoln.ac.uk/web-focus/events/workshops/webmaster-2006/sessions/tonkin/

1. A centre of expertise in digital information management
www.ukoln.ac.uk
UKOLN is supported by:
Usability on a Shoestring
Budget (2)
Emma Tonkin & Adam Hardy
Software & Systems, UKOLN
HCI Department, University of Bath
www.bath.ac.uk

2. A centre of expertise in digital information management
www.ukoln.ac.uk
Analysing the results
• Data analysis:
– What problems appeared?
– Which were recurrent?
– Place them in context; why did they
appear?
– Consider personal variables; age,
experience, motivation, education, level of
interest, stress (especially if videoed)

3. A centre of expertise in digital information management
www.ukoln.ac.uk
Reporting the results
• Document what you did and why;
explain why you made your decisions
• Justify your interpretations
• Include output data (if possible;
remember data protection, etc!)
• Provide summaries of results

4. A centre of expertise in digital information management
www.ukoln.ac.uk
Now what?
• You’re aware that there are problems.
• What next?
• Consider the following:
– The stage the project is in
– The available resources
– The complexity of the problem & solutions
• Ideally, user testing should occur frequently
during a project’s lifecycle. (But this is not an
ideal world).

5. A centre of expertise in digital information management
www.ukoln.ac.uk
Why do projects fail?
Project Impaired Factors % of the Responses
1. Incomplete Requirements 13.1%
2. Lack of User Involvement 12.4%
3. Lack of Resources 10.6%
4. Unrealistic Expectations 9.9%
5. Lack of Executive Support 9.3%
6. Changing Requirements & Specifications 8.7%
7. Lack of Planning 8.1%
8. Didn't Need It Any Longer 7.5%
9. Lack of IT Management 6.2%
10. Technology Illiteracy 4.3%
11. Other 9.9%

6. A centre of expertise in digital information management
www.ukoln.ac.uk
Applying the results
Bug fixes
Feature requests
Major objections
Misnamed element
Confusing colours
It would be much easier if…
…‘this textbox autocompleted’
…’the system remembered my
email preferences’
I don’t like [type of application]
I prefer [totally different type of
application]
…(Oh)
Strange interaction flow
‘Low-
hanging
fruit’?

7. A centre of expertise in digital information management
www.ukoln.ac.uk
Applying the results
Bug fixes
Feature requests
Major objections
Misnamed element
Confusing colours
It would be much easier if…
…‘this textbox autocompleted’
…’the system remembered my
email preferences’
I don’t like [type of application]
I prefer [totally different type of
application]
…(Oh)
Strange interaction flow
‘Low-
hanging
fruit’?