Testing Techniques

Effective accessibility testing improves user experience, increases visitor traffic, protects a positive brand image, improves legal compliance, and delivers better search engine optimization.

WCAG

The Web Content Accessibility Guidelines (WCAG) are a widely-recognized set of technical standards and guidelines developed by the World Wide Web Consortium (W3C) to help ensure that web content is accessible to people with disabilities. WCAG covers various essential topics, including alternative text for non-text content, keyboard accessibility, color contrast, and clear and straightforward language.

By necessity, WCAG can only address the technical aspects of a webpage, but in reality, there will always be a range of factors that affect a web page's accessibility performance. These include the user, the different states of the application, and the technical platform the application is running on. Passing any set of automated tests run against the WCAG guidelines should only be considered a starting point; it does not ensure that the website will be accessible or usable in all, or even most, circumstances. Addressing these nuances will require human judgment: the experience and evaluation of a manual tester.

WCAG isn't the final word in making things accessible, but it is the baseline. [1]

Of course, having automated tests that run against the WCAG guidelines isn't a bad idea; starting points are always a good place to start, and automating tests so that manual testers can focus on what they're best at is a smart move. But basic manual tests for accessibility can and should be practiced by everyone involved in building any webpage.

First, validate

Validating the HTML of a webpage is a crucial step in ensuring that the page is functional, accessible, and optimized for both users and search engines. This should always be "Step Zero" of your accessibility checks. Valid HTML is the best way to ensure that a webpage will be correctly interpreted by assistive technologies, which rely on the HTML structure.

  • W3C Markup Validation Service: An online tool provided by the World Wide Web Consortium (W3C), which is the organization responsible for developing web standards. This tool checks your HTML code for correctness and provides suggestions for how to fix it. It can also check for accessibility and internationalization issues.
  • HTML5 Outliner: An online tool that analyzes the HTML code and generates an outline of the page's structure, making it easy to see the hierarchy of headings, sections, and other elements. This can be especially useful for ensuring accessibility and SEO.
  • Web Developer Extension: This is a browser extension available for Chrome, Firefox, and Edge that provides a suite of tools for web development, including an HTML validator that can check the code for errors and provide suggestions for fixes.
  • Visual Studio Code: This popular code editor includes settings to validate HTML syntax and can highlight errors and warnings in real-time as you edit your code, providing suggestions for how to fix issues. Extensions are also available, such as HTMLHint, which can analyze and lint your HTML in the editor.

Manual accessibility checks

What pages should you check?

Manually checking every page of your website isn't necessary or a good use of your resources. So get started by selecting what pages to include and exclude from your sample.

  • Include an array of critical and popular pages, such as the website homepage, section index pages, and the most highly-trafficked sub-pages.
  • Pages that include interactive elements which facilitate user input or transactions: web forms, login screens, shopping features, etc.
  • Pages that rely on non-text content such as images, animations, video, or audio.
  • Pages that include critical interactive components such as navigation menus, log-in forms, entry gates, etc.
  • Any pages required for legal or regulatory compliance, such as data protection, privacy, cookie notices, terms of service, or legal disclaimers.
  • All business-critical pages considered valuable or essential to the website's core business or organizational purpose: contact pages, e-commerce features, registration forms, marketing information, core content, etc.

It's worth choosing a random sample from a pool of these pages to test on a frequent basis if you can't afford the resources to test every page. It's better to test a few pages than none at all.

A checklist for basic manual accessibility testing

  • Review your webpage with keyboard-only navigation: Try navigating your website with a keyboard only, using no mouse or other pointing device. This can help you identify any keyboard accessibility issues, such as links or buttons that can't be reached or activated with the keyboard alone.
  • Try your webpage on a mobile device: Using only touchscreen interactions, it should be possible to use all core features and access all core content. Ensure that you don't require complex gestures: it should be possible to operate the webpage using only one finger.
  • Check color contrast: Ensure the color contrast between the text and the background is sufficient. You can use online tools such as WebAIM's Color Contrast Checker to check the contrast ratio.
  • Use descriptive text for links and images: Ensure that all links, form labels, and images have descriptive text, or alt text, which conveys the appropriate meaning or purpose to users who cannot see them. This helps users with screen readers and other assistive technology understand your website's content.
  • Use clear and simple language: this will make it easier for users with reading or cognitive disabilities to understand the content. Avoid jargon, abbreviations, and complex sentence structures.
  • Ensure video and audio content is accessible: Provide captions, transcripts, and audio descriptions for all video and audio content to make it accessible to users who cannot see or hear such content.
  • Build pages to be at least usable when JavaScript and stylesheets are disabled: Following a practice of progressive enhancement will ensure that the broadest possible range of assistive technologies will function for disabled users.
  • Don't rely only on color or graphics to communicate content: When activating High Contrast Mode and disabling images, you can get a sense of how a webpage will be experienced by users that cannot perceive color or images.
  • Review page title and headings for understanding: Having a clear, logical, and hierarchical outline of text content on your webpage is critical for many disabled users who rely on assistive technology or other tools to navigate content according to how it is organized.

Options for automated testing

Testing with a screen reader

Ultimately, to measure your webpage's performance when used with a screen reader, you must involve a person to actually use your webpage with a screen reader. This doesn't necessarily need to be a screen reader power-user; as with any group of users, not every assistive technology user is an expert, but people who rely on screen readers usually have a lot of experience learning and practicing using these tools and have developed their own personal patterns and habits that improve their efficiency.

Starting a screen reader for testing without any experience or practice in its use can be counterproductive. If you're unfamiliar with the usage of screen readers, it's worth training someone on your team to build up this skill. Each screen reader software has its own idiosyncrasies for configuration, integration with web browsers, and keyboard-combination or gesture-based shortcuts.

When testing, it makes sense to include the most-used screen readers. The most popular screen readers (in order) are:

  • JAWS (Job Access With Speech) is the oldest, most popular screen reader in use today. [2] It's designed to work with Windows, but unlike NVDA, it isn't free.
  • NVDA (NonVisual Desktop Access) is another popular screen reader designed to work with Windows. NVDA is free. It can access the text of a webpage to be translated to speech or for a Braille display.
  • VoiceOver Apple's built-in screen reader reads text on a MacOS or iOS device, including webpages and documents.
  • Other options Windows Narrator, Window-Eyes, Android TalkBack, ChromeVox, Orca (Linux)

It's worth pointing out that while most people associate screen reader software with speech synthesis, the screen reader itself is only part of a more complex process of interpreting digital content for users. In the case of a webpage, this will include the operating system and web browser software that provides the content, the screen reader that then makes that content available as text, and text-to-speech software that translates that text into a synthesized voice. However, not every screen reader user can hear speech; it is estimated that there are nearly 400,000 deafblind people in the UK. [3] These users are more likely to use another type of text interpretation, such as a Braille display. [4]

A minimum assistive technology test matrix:

These combination are based on the WebAIM survey of disabled users and assistive technologies. [5] You should adjust this if needed to address specific issues or demographics for your specific website.

  • JAWS - a screen reader for Windows, test with latest Chrome and Edge browsers on Windows.
  • NVDA - a screen reader for Windows, free and open source, test with latest Chrome and Firefox browsers on Windows.
  • VoiceOver - a screen reader which comes with Mac/iOS, test with latest Chrome and Safari browsers on Mac, and test with latest Mobile Safari on iOS.
  • ZoomText - a screen magnifier, test with latest Chrome and Edge browser on Windows.

Checkpoints for the Tester

  • Ensure the webpage passes HTML validation.
  • As a minimum aim for compliance with WCAG 2.1 level AA. Ensure you are testing that all core content and functionality is: perceivable, operable, understandable, and robust. [6]
  • Develop and document a plan for sampling webpages to test manually that maximizes test resources and provides the most benefit to users.
  • Define user testing profiles that include disabilities which affect: vision, hearing, mobility, and cognitive/understanding. Consider that these classes of disability may occur to greater or lesser degrees in individuals, from slight to severe, some will be temporary or based on situations, and that some users will be disabled in multiple ways.
  • If possible design your plan for accessibility testing from the start. Don't wait until the code is complete to start checking it's accessibility!
  • Work with designers, developers and testers to apply the appropriate automated testing checks throughout the entire workflow.
  • Ensure manual testers have access to the most common assistive technologies currently in use, and any training necessary to effectively use them.

Notes and references

  1. Quoted from Adrian Roselli

  2. A 2021 survey of Screen Reader Users by WebAim found that 53.7% used JAWS, and 30.7% used NVDA. 6.5% of respondents used VoiceOver.

  3. Statistics provided by the Deafblind UK organization.

  4. See this demonstration of refreshable Braille display on Youtube.

  5. WebAIM Screen Reader User Survey #9 Results

  6. The WCAG 2.1 Quick Ref lists specific checkpoints for each type of accessibility requirement.

Back to top