Compliance with web accessibility standards can be assessed in several alternative ways. While all are valid, the different approaches offer distinct advantages and disadvantages depending on the product, the audience for which it is intended, the stage of the design process, the make-up of the project team, and the project budget. In this presentation, we will explore four main approaches for evaluating web accessibility, breaking down their pros, cons, and the scenarios in which they would be most appropriate. The four approaches are: 1. Enterprise-wide automated tools 2. Heuristic inspection by an accessibility specialist 3. Heuristic inspection by a pair of accessibility specialists, one sighted and one unsighted 4. Usability testing by participants with disabilities Attendees will gain familiarity with several accessibility evaluation methods. They will also be challenged to think about the most appropriate strategy to use in particular situations.

1. Pros and Cons of Alternative Strategies for Evaluating Web Accessibility
Dana Douglas, UserWorks, Inc.
Sarah Deighan, Next Century Corp.
Jennifer Sutton, Independent Consultant 1

2. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility
Web Accessibility Evaluations

Ensure ALL users have access to information

Typically based on set of guidelines

Can be conducted at any stage of development

Methods vary depending on many factors 2

3. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility
Enterprise-Wide Automated Tools

Software that scans large-scale websites for common accessibilityissues
3

4. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility
Enterprise-Wide Automated Tools
Pros:

Scalable

Track overall accessibility patterns and quantify progress

Train developers regarding best practices

Minimal personnel time 4

5. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility
Enterprise-Wide Automated Tools
Cons:

Only catch low hanging fruit

Can’t tell whether an element is actually usable, only whether it meets minimum requirements

Can miss actual problems and flag non-existent issues

Upfront costs can be expensive

Doesn’t always test the Document Object Model (DOM) 5

6. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility
Enterprise-Wide Automated Tools
Scenarios in which this method can work well:

Very large websites

Identifying an initial baseline of accessibility compliance

Maintaining accessibility among various content authors

Organization requires statistical evidence of progress 6

7. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility
Individual Heuristic Inspection

Use tools and assistive technologies to manually evaluate webpages 7
Sample WCAG 2.0 Checklist from WebAIM http://webaim.org/standards/wcag/checklist

8. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility
Heuristic Evaluation Tools
•
WAVE Toolbar (Firefox)
•
The Paciello Group Colour Contrast Analyser
•
Juicy Studio Accessibility Toolbar (Firefox)
•
NoCoffee vision simulator (Chrome)
•
Firefox Web Developer Toolbar (Firefox)
•
Internet Explorer and Chrome have built-in developer tools too
•
Firebug (Firefox) 8

9. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility
Individual Heuristic Inspection
Assistive Technology Demos:

Screen reader

Screen magnifier

Speech recognition software 9

10. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility
Screen Reader Demo
10
Demo of three different screen reader navigation methods on the Easy Checks website Courtesy of Jennifer Sutton

11. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility
Screen Magnifier Demo
11
Demo of screen magnifier on W3C website Windows Magnifier, Windows 7

12. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility
Speech Recognition Software Demo
12
Graham Armfield, Coolfields Consulting demo of Dragon Naturally Speaking http://www.coolfields.co.uk/2014/03/dragon-naturally-speaking-video-9-internet-forms/

13. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility
Individual Heuristic Inspection
Pros:

Comprehensive; identifies most, if not all, flaws

Accurate; flags only actual flaws

Inexpensive (often free!) tools 13

14. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility
Individual Heuristic Inspection
Cons:

Often doesn’t “touch” every page on the site

Specialist should learn how to use different assistive technologies

Can be time consuming

More prone to bias 14

15. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility
Individual Heuristic Inspection
Scenarios in which this method can work well:

Limited budget

Website with small number of page types

Website with primarily static webpages (little user interaction) 15

16. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility
Collaborative Heuristic Inspection

A team of two or more accessibility specialists, at least one of whom is also an assistive technology (AT) user

Each evaluates the site individually and then collaborates to discuss possible issues and brainstorm solutions
16

17. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility
Collaborative Heuristic Inspection
Pros:

Gets the perspective of an AT user who also knows what kinds of issues to look for

Have the benefit of seeing two sides of the site (one with AT and one without)

Can discuss issues and brainstorm possible solutions 17

18. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility
Collaborative Heuristic Inspection
Cons:

AT users who are accessibility experts can be difficult to find

Expert users may not be representative of typical users 18

19. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility
Collaborative Heuristic Inspection
Scenarios in which this method can work well:

Moderate budget

Website with interactive elements

Access to AT users with accessibility knowledge 19

20. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility
Usability Testing

Typical usability test with participants who are AT users

In-person or remote; moderated or un-moderated
20

21. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility
Usability Testing
Pros:

Gets feedback from representative users

May identify usability-related issues that are outside accessibility guidelines

Doesn’t require expert knowledge of AT

Doesn’t require expert knowledge of standards 21

22. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility
Usability Testing
Cons:

Can be expensive

May not identify all issues on the site

Logistics can be complicated 22

23. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility
Usability Testing
Scenarios in which this method can work well:

Moderate budget

Website with interactive elements

Team with limited knowledge of AT and standards 23

24. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility
Summary of Methods
Evaluation Method
Cost
Time
Expertise
Accuracy
Comprehensiveness
Automated tools
$$$$



%
Individual Heuristic Review
$



%%
Team Heuristic Review
$$



%%%
Usability Testing
$$$



%% 24

25. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility
Which method is best?

Combination of two or more is ideal

In all cases, evaluating early and often is best

Easy to take small steps to include accessibility evaluation as part of your other user research activities 25

26. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility 26
Resources

Rosenfield Media:

A Web for Everyone (Sarah Horton & Whitney Quesenbery)

A Podcast for Everyone

Accessibility Research Methods with Jonathan Lazar (podcast)

Choosing an Automated Accessibility Testing Tool: 13 Questions You Should Ask (Karl Groves)

WebAIM Screen Reader Survey

Easy Checks

27. July 22, 2014
UXPA 2014 LONDON
Pros and Cons of Alternative Strategies for
Evaluating Web Accessibility
Questions?
Sarah Deighan
Sarah.deighan@nextcentury.com 27
Dana Douglas ddouglas@userworks.com
Jennifer Sutton
@jsutt