Insights

Accessibility testing

Published: 16.4.2023
Categories: Accessibility
Reading time: 23 min
Blindfolded person typing on a computer while other person is giving instructions

Whether you're a designer, developer, or content creator, accessibility testing should be part of your work process. Testing is needed to improve the accessibility level of service, and also to maintain said level as new features and content are introduced. Even if you're not an accessibility specialist, you can still learn some basic testing techniques, which can help you find common issues that impact a large number of users.

Automated testing

Value

Automated testing is a valuable tool; it can quickly find most occurrences of certain issues, with a thoroughness that would be very difficult to achieve manually.

Most of these issues are related to coding practices; for example, automated testing can quickly check if all image content (such as <img> elements) have text alternatives, which are needed by screen reader users.

Limitations

Unfortunately, automated testing can only detect 10 to 40% of accessibility issuesOpens in a new tab; furthermore, automated tools can lead to false negatives (not detecting an issue), as well as false positives (reporting something that's not actually a problem).

Since accessibility practices are designed to meet the needs of real people, most issues need to be manually tested by a human. For example, while automated testing can detect if text alternatives are present, only a person can check if these text alternatives are properly written and actually deliver the information they are supposed to. You can learn more about text alternatives on our Accessible UIs page.

Resources

Color testing

WCAG and colors

It's important to understand how WCAGOpens in a new tab talks about color in order to avoid confusion.

  • When they talk about color, they mean color hue (for example, red versus green).
  • When they talk about contrast, they are referring to differences in color lightness or luminance (for example, dark versus light).

Enough (color) contrast

WCAG defines the minimum and recommended contrast ratios between the color of the foreground elements (such as text, icons, and focus outlines) and the background. There are different requirements for different types of content and levels of compliance:

Although most color contrast issues can be evaluated via automated testing, manual testing is needed when dealing with interactive elements, as it is common for them to change colors when they are hovered, focused, or activated.

To make sure that there’s enough contrast between, for example, the color on hover and the color on active, you can use a developer tool or browser extension that updates in real-time as you interact with the site, such as the WCAG Color contrast checkerOpens in a new tab Chrome extension.

Please note that the WCAG contrast requirements aren’t always optimal; some combinations that pass the contrast check are still difficult to read for most people (for example, red text on a black background). Use your eyes and brain!

Never color (hue) alone

We need to make sure that information is never conveyed with color hue alone because some people may not be able to perceive it. For example:

  • A required field can be marked by a blue border and the word “required” next to it.
  • A button toggling between on and off states can be denoted with, for example:
    • A change in color hue between green and red, plus the words “ON”/”OFF” is toggled.
    • A change in lightness (black to grey), with or without a change in text.
    • A change in color hue and lightness (intense green to light red), with or without a change in text.

Customizing colors

Best practices recommend that the user is able to change the default foreground and background colors (for example, using their own CSS files to locally override the styles on the browser). You can test this with the developer tools or a browser extension that allows you to use custom color schemes, such as the Care Your Eyes ChromeOpens in a new tab extension.

Resources

Keyboard testing

Keyboards and beyond

Many people with disabilities do not use computer mice. Some navigate using a keyboard, and some use alternative input methods, many of which (such as switches, physical pointers, and eye and motion trackers) emulates the behaviour of a keyboard. Therefore, by making sure your website is keyboard-accessible, you are actually serving users of many other technologies. Read more about different digital assistive technologies here.

Basic keys

The basic keys you need to make sure are supported include:

  • Tab: to quickly jump through all focusable elements.
  • Enter: to activate links.
  • Space: to activate buttons and many <input> elements, and to choose an option from a <select> dropdown.
  • Arrows: to switch between options in <select> dropdowns and groups of radio buttons.
  • Escape: to close expanded <select> dropdowns.

All the native HTML elements mentioned above already come with the described functionality, which is implemented by the browser. If you create custom elements, they need to be keyboard-accessible in a way that is consistent with the usual conventions. For example, the user must be able to press Escape to close or dismiss custom elements, such as multilevel navigation menus or modal overlays.

Notes about forms:

  • If a button type=”submit” is inside a <form> element, pressing Enter while being focused on any element inside the form will trigger form submission, without activating the Submit button itself.
  • If a <form> element has no Submit button, the browser will look for the first button without an explicit type (so, with an empty or missing type attribute), but only inside the form. If none can be found, the form cannot be submitted via default browser methods.

Focus order and visibility

The default focus order follows the DOM (Document Object Model). This means that, as a user tabs through the page using the keyboard, interactive elements will receive focus in the order they appear on the HTML code file. It's possible to modify the focus order with the tabindex HTML attribute or using JavaScript methods, but you should only do this in specific cases when there is no alternative.

You can test focus order by simply tabbing through the page. Focus order should normally follow visual order, although there are exceptions. For example, if a language switcher is visually located at the end of the navigation bar, you can make it come first in the focus order so that keyboard and screen reader users can have the option of changing the language of the page as soon as possible.

All focusable elements must have a visible focus indicator. This is such a key accessibility feature that browsers must provide default focus styles; unfortunately, accessibility issues often arise because the default browser focus is insufficient or has been overridden by developers. You can learn more about good focus styling practices of Hover, focus, active in our Accessible UIs page.

Resources

Screen reader testing

Why you should learn

Many people with disabilities use text readers or screen readers; both types of software read out the text that is on the page. Additionally, screen readers provide further information for people with limited vision. For example, if the text is a link, the screen reader will announce that; if the screen reader encounters a non-text element, it will read its text alternative.

Screen-reading software can be quite complicated and may seem intimidating at first. However, here’s some good news for you:

  • If you’re sighted, you won’t be as disoriented as a fully blind person trying to learn it, because you can still see your screen and keyboard.
  • The basic keys you learned about in our keyboard testing section still apply here (for devices with keyboards).
  • If you’re just gonna use a screen reader to test websites, you can get by with just a few additional keystrokes and/or gestures, which we have compiled for you below.

Voices and languages

The screen reader voice is the voice synthesizer being used at any given time.

The language can be:

  • The language of your operating system.
  • The language of your screen reader UI.
  • The language of the web content.

You might use a website in multilanguage situations; for example, your operating system or screen reader UI might be in Finnish and the web content in English. Unfortunately, this leads to buggy language behaviour in all major screen reader, OS, and browser combinations, even when the language of a page or part of a page is set correctly. For example, the screen reader might read English text with a Finnish voice (or “accent”). Our former colleague Xurxe Toivo García performed experiments regarding this; check out his article, "The troubled state of screen readers in multilanguage situationsOpens in a new tab".

Reading cursor

The reading cursor is an indicator of where the screen reader “is at”. It often shows up as an outline around the item. When using a screen reader on a website, if might go like this:

  1. Cursor on the navigation bar, and the screen reader announcing "Navigation".
  2. Cursor on each link, announcing "Link" plus the link text.
  3. Cursor on the navigation bar again, and announcing "End, navigation".

Note: in the case of many (but not all) screen readers, as the reading cursor goes over a focusable element, it moves keyboard focus to it. The focus will remain on that element as the screen reader carries on until it encounters a new focusable element.

Reading modes

There are two general reading modes, which have to do with how actively the user is controlling the reading cursor:

  1. In continuous reading mode, the user initiates reading (by pressing specific keys or performing a specific gesture), and after that, the reader goes through the rest of the page by itself.
  2. In controlled reading mode, the user moves the reading cursor to the next or previous element, every time using the specific keystroke or gesture. Most screen readers allow users to control the reading granularity; this can mean, for example, choosing whether to read the next letter, word, sentence, or paragraph.

Additionally, if the user is tabbing to jump through the focusable elements on the page, the screen reader announces each element as it comes into focus.

Screen readers don't always announce the same element in the same way during different reading modes. As a result, it's important to test using the three methods we just described.

Tips for VoiceOver on macOS (desktop)

  • Learn how to turn it on and off without having to go to Preferences > Accessibility; the specific shortcut depends on your system (you can also set up a personalized shortcut).
  • If you might test content in several languages, set a different voice for each language on VoiceOver Tools.

Basic VoiceOver key presses for testing webpages on macOS

  • VO: VoiceOver modifier (Caps Lock and/or Control + Option, depending on your settings).
  • All the key presses you learned about in our keyboard testing section (Tab, Enter, Space, arrow keys, and Escape).
  • Caps Lock: pause/unpause reading.
  • VO + A: continue reading from the current location of the screen reader cursor.
  • VO + Right Arrow: move the reading cursor to the next item (useful, for example, when the reader encounters a long paragraph and you just wanna move on to the next one instead of listening to all of it).
  • VO + Left Arrow: move the reading cursor to the previous item.

Intermediate VoiceOver key presses for testing websites on macOS

  • VO + U: open VoiceOver rotor. On the desktop, the rotor is a menu that can help you choose how to navigate quickly around the page. You can already use tabbing to jump through all focusable elements, but the rotor might show you a list of, for example, links, landmarks, headings, articles… (you can customize this).
    • Right and Left arrows: switch between rotor lists.
    • Up and Down arrows: switch between items on a rotor list.
    • Enter: select an item on a rotor list.
    • Escape: close rotor menu.
  • VO + Fn + Right Arrow: move the reading cursor to the end of the current landmark region.
  • VO + Fn + Left Arrow: move the reading cursor to the beginning of the current landmark region.
  • VO + Space: activate the focused element.

Tips for VoiceOver on iOS (mobile)

  • Learn how to turn it on and off without having to go to Settings > Accessibility; the specific shortcut depends on your system (you can also set up a personalized shortcut).
  • If you might test content in several languages, set a different voice for each language on Settings > Accessibility VoiceOver > Speech > Rotor Languages.

Basic VoiceOver gestures for testing webpages on iOS

  • 1-finger single tap: select and read an item.
  • 1-finger double tap: activate an item.
  • 1-finger swipe right: move the reading cursor to the next item (useful, for example, when the reader encounters a long paragraph and you just wanna move on to the next one instead of listening to all of it).
  • 1-finger swipe left: move the reading cursor to the previous item.
  • 2-finger single tap: pause/unpause reading.
  • 2-finger swipe up: read everything from the beginning.
  • 2-finger swipe down: continue reading from the current location of the screen reader cursor.
  • 3-finger swipe: scroll (depending on the direction of the swipe).

Intermediate VoiceOver gestures for testing webpages on iOS

On mobile, the VoiceOver rotor is a menu that can help you:

  • Navigate faster: for example, if you choose Rotor > Headings, you can quickly jump through the headings on the page.
  • Change reading granulation: for example, if you choose Rotor > Words, you can read one word at a time.
  • Change screen reader settings: for example, if you choose Rotor > Language, you can change the screen reader voice (“accent”).

Here are some rotor-related gestures:

  • 2-finger rotate clockwise: next Rotor list.
  • 2-finger rotate counterclockwise: previous Rotor list.
  • 1-finger swipe down: next Rotor item.
  • 1-finger swipe up: previous Rotor item.

Other useful gestures:

  • 3-finger double tap: toggle mute (this mutes the screen reader speech completely until you repeat the gesture)
  • 2-finger scrub: Escape
  • 3-finger double tap, hold and flick up: copy
  • 3-finger double tap, hold and flick down: paste
  • 3-finger double tap, hold and flick left: undo
  • 3-finger double tap, hold and flick right: redo

Tips for NVDA on Windows

  • Launch the program like any other; for example, double-click on the shortcut on your desktop.
  • Here are some settings you should check and/or change. To open the settings dialogue, press NVDA + N (see next section to learn more about the NVDA modifier key), and go to Preferences > Settings. This will open the settings dialogue.
    • General > Language: change the language (of the screen reader UI) to match the language of your system.
    • Speech > Synthesizer: choose eSpeak NG or another synthesizer with multilanguage support.
    • Speech > Voice: change the main screen reader voice to match the language of your system.
    • Keyboard > Keyboard layout: you can make it match your machine, or you can choose “Laptop” if you want shortcuts to be a little more similar to VoiceOver on Mac.
    • Keyboard > Select NVDA modifier keys: this is up to you, but it might be a good idea to select all options for more flexibility.

Basic NVDA key presses for testing webpages

  • NVDA: NVDA modifier. By default, this is the Insert key, but you can change the settings so that it can be the Insert key and/or the Numpad Insert key and/or the Caps Lock key.
  • All the key presses you learned about in our keyboard testing section (Tab, Enter, Space, arrow keys, and Escape).
  • Control: stop reading.
  • Shift: pause/unpause reading.
  • NVDA + Down Arrow (desktop) or NVDA + A (laptop): continue reading from the current location of the screen reader cursor.
  • Numpad 9 (desktop) or NVDA + Down Arrow (laptop): move the reading cursor to the next item (useful, for example, when the reader encounters a long paragraph and you just wanna move on to the next one instead of listening to all of it).
  • Numpad 7 (desktop) or NVDA + Up Arrow (laptop): move the reading cursor to the previous item.
  • NVDA + Q: quit NVDA

Intermediate NVDA key presses for testing webpages

  • The following keys by themselves jump to the next available element of that type while adding the Shift key causes them to jump to the previous element of that type (there’s more, we listed just a few):
    • D: landmark
    • H: heading
    • 1 to 6: heading levels 1 to 6
    • K: link
    • U: unvisited link
    • V: visited link
    • L: list
    • I: list item
  • NVDA + Enter: activate focused element.
  • NVDA + Shift + Z (desktop) or NVDA + Shift + S (laptop): turn sleep mode on and off (this affects the current application only)

Tips for JAWS on Windows

  • JAWS is a paid program, and licenses can be over 1000 USD. However, you can download the software for free and use it in 40-minute sessions.
  • Launch the program like any other; for example, double-click on the shortcut on your Desktop. The first time you do this, you will get a dialogue where you can choose to use the 40-minute trial version (you can choose for this dialogue to not be shown in the future).
  • Here are some settings you should check and/or change. To see the JAWS window, simply click on the JAWS icon on your menu bar.
    • Language > JAWS Language: change the language (of the screen reader UI) to match the language of your system.
    • Options > Voices > Voice Adjustment > Profile Name: choose Eloquence or another synthesizer with multilanguage support.
    • Options > Voices > Voice Adjustment > Synthesizer Language: choose “Match JAWS UI Language”
    • Speech > Voice: change the main screen reader voice to match the language of your system.
    • Options > Basic > Use Keyboard layout: you can make it match your machine, or you can choose “Laptop” if you want shortcuts to be a little more similar to VoiceOver on Mac.
  • To close the program, simply close the JAWS window as you would any other program.

Basic JAWS key presses for testing webpages

  • JAWS: JAWS modifier. This is the Insert key on the desktop layout and the Caps Lock key on laptop layout.
  • All the key presses you learned about in our keyboard testing section (Tab, Enter, Space, arrow keys, and Escape).
  • NVDA or Control: stop reading.
  • NVDA + Down Arrow (desktop) or NVDA + A (laptop): continue reading from the current location of the screen reader cursor.
  • Down Arrow (desktop) or NVDA + O (laptop): move the reading cursor to the next item (useful, for example, when the reader encounters a long paragraph and you just wanna move on to the next one instead of listening to all of it).
  • Up Arrow (desktop) or NVDA + U (laptop): move the reading cursor to the previous item.

Intermediate JAWS key presses for testing webpages

The following keys by themselves jump to the next available element of that type while adding the Shift key causes them to jump to the previous element of that type (there’s more, we have only listed a few):

  • R: region (landmark)
  • H: heading
  • 1 to 6: heading levels 1 to 6
  • U: unvisited link
  • V: visited link
  • L: list
  • I: list item

Tips for TalkBack on Android

  • TalkBack may come from your device, or you may have to download it. You can download, for example, the Android Accessibility Suite from Google Play, which includes TalkBack.
  • Learn how to turn it on and off from Settings. It may be under Settings > Accessibility > Screen Reader, under Settings > Accessibility > Installed Services > TalkBack, or somewhere else, depending on your system.
  • Set up a shortcut. It may be under Settings > Accessibility > Advanced Settings, or somewhere else, depending on your system. The shortcuts available will also depend on your system, but they may include “Power and Volume up keys” and/or “Volume up and down keys”.

Basic TalkBack gestures for testing webpages

  • All TalkBack gestures use one finger: this means that you can perform all the other gestures you’re used to, as long as you use more than one finger. For example, to scroll down while TalkBack is on, use two or more fingers.
  • 1-finger single tap: select and read an item.
  • 1-finger double tap: activate an item.
  • 1-finger swipe right: move the reading cursor to the next item in the current navigation setting (useful, for example, when the reader encounters a long paragraph and you just wanna move on to the next one instead of listening to all of it).
  • 1-finger swipe left: move the reading cursor to the previous item in the current navigation setting.
  • Tap with any amount of fingers while reading: stop reading.
  • 1-finger swipe down and then right: open Global Context Menu. This reveals a dialogue with a number of options, including:
    • Read from top
    • Read from next item
    • TalkBack settings
    • Languages
    • etc.

Intermediate TalkBack gestures for testing webpages

  • 1-finger swipe down: switch to next navigation setting. Settings may include:
    • Default: swiping right will move the reading cursor to the next item, regardless of what it is.
    • Headings: swiping right will move the reading cursor to the next heading, skipping the content in between.
    • Links: swiping right will move the reading cursor to the next link, skipping the content in between.
    • etc.
  • 1-finger swipe up: switch to previous navigation setting.
  • 1-finger swipe up, then down: move the reading cursor to the first item on the screen.
  • 1-finger swipe down, then up: move the reading cursor to the last item on the screen.
  • 1-finger swipe right, then left: move the slider up (for example, a volume slider).
  • 1-finger swipe left, then right: move the slider down (for example, a volume slider).
  • 1-finger swipe up, then right: open Local Context Menu. This reveals a dialogue with a number of options, which will depend on the item that’s currently selected. For example:
    • In an editable text field, you’ll get some options related to text editing such as copying, pasting, etc.
    • In other items, you’ll get options related to reading granularity, controls, etc.
  • 1-finger swipe down, then left: go back

Resources

Screen readers and languages:

VoiceOver on macOS:

VoiceOver on iOS:

NVDA on Windows:

JAWS on Windows:

TalkBack on Android:

Size testing

Screen size and magnification

It's important to test your site using the developer tools on different devices, to make sure it behaves properly at different scales and proportions. We already do this as part of responsive design and development, but WCAGOpens in a new tab sets very specific thresholds for accessibility purposes.

You should:

  • Use developer tools to view on different screen sizes.
  • Use browser zoom to view at different magnifications on different screen sizes. The site should have no loss of content or functionality at least at 200% zoom, ideally up to 400% zoom.
  • Ensure that users viewing the site at a high magnification don't have to scroll in two directions (unless absolutely necessary). That means:
    • For vertical scrolling (most situations): ensure no loss of content or functionality (without requiring horizontal scroll) at a width equivalent to 320 CSS pixels. For example:
      • Width of 320 pixels at 200% zoom.
      • Width of 1280 pixels at 400% zoom.
    • For horizontal scrolling (rare): similar as above, but at a height equivalent to 256 CSS pixels.

Typography

You need to make sure that default typography sizes and spacing can be overridden by the user without loss of content or functionality. You can use developer tools to try the CSS code below and check if your site displays properly with an increased base font size and the minimum spacing required:

* {
 line-height: 1.5em !important;
 letter-spacing: 0.12em !important;
 word-spacing: 0.16em !important;
}

:root {
 font-size: 25px !important;
}

p {
 margin-top: 2em !important;
 margin-bottom: 2em !important;
}

Resources

Chrome:

Firefox:

Safari:

Edge:

Other:

Testing documents

Compliance

All documents should be tested before publication. They must comply with WCAGOpens in a new tab 2.1 and, in the case of PDFs, with PDF/UAOpens in a new tab (PDF/Universal Accessibility, or ISO 14289).

Testing methods

Since automated testing can only detect 10 to 40% of accessibility issuesOpens in a new tab, we should perform manual testing according to the methods we explained in these sections:

  • Color testing
  • Keyboard testing
  • Screen reader testing

As for automated testing, many authoring tools have built-in accessibility testing features. These features can help catch some accessibility issues and it is a good idea to use them as part of your workflow before publishing or exporting documents.

Testing PDFs

Acrobat Pro has a suite of accessibility tools that can be used to find and fix issues. To perform automated testing on a PDF, open the file and go to View > Tools > Accessibility. Choose "Full Check" to run the test and generate a report of the findings.

There are many other automated PDF accessibility testers; some are online services and some are downloadable apps. We have listed a few further down under Resources.

Please remember that automated tests have limitations; getting a passing score does not guarantee full compliance with accessibility standards, and manual testing is still necessary.

Testing Microsoft Office files

You can test the accessibility of certain Microsoft Office files by using the built-in Accessibility CheckerOpens in a new tab. This tool is included with Word, Excel, Outlook, OneNote, and PowerPoint (for Windows, Mac, and the cloud), as well as with Visio (for Windows).

To run the Accessibility Checker, go to Review > Check Accessibility. After the check is done, you'll get a list of Errors, Warnings, and Tips; you can then click on each issue to learn why it needs to be fixed, and how to do it. You can find out more about the rules for the Accessibility CheckerOpens in a new tab on the Office Support site.

In Windows, you can also select the "Keep accessibility checker running while I work" checkbox so that you get notified about accessibility issues in real-time while editing your document.

As with other automated testing tools, the Accessibility Checker will not catch all errors, so it is important to perform manual testing.

Testing Google Drive files

Google Drive is lacking many accessibility features, including automated testing tools and the ability to export accessible PDFs. It does have an Accessibility menu, which you can enable by going to Tools > Accessibility settings, and then selecting "Turn on screen reader support". After doing that, an item called "Accessibility" will be added at the end of the main menu.

Using the Accessibility menu, users can jump to specific elements in the document, including links, headings, lists, tables, graphics, etc. This feature can be used to check, for example, that headings are true headings and not just regular text made to look like a heading; this is, however, time-consuming, and not a meaningful substitute for real automated testing. As a result, the manual testing discussed earlier is absolutely essential when it comes to testing Google Drive documents.

Grackle SuiteOpens in a new tab is a series of tools that can remediate some of the shortcomings of Google Drive, by providing automated testing tools and the ability to export accessible PDFs. It's available as add-ons for Google Docs, Google Sheets, and Google Slides; there's also a Grackle Suite Chrome extensionOpens in a new tab that works with Google Docs and Google Sheets. The free version allows you to check for accessibility issues, but you need a subscription in order to get fixing tips and the ability to export PDFs.

Testing in OpenOffice and LibreOffice

OpenOffice and LibreOffice don't have automated testing tools. AccessODFOpens in a new tab is an open-source extension for LibreOffice Writer and OpenOffice Writer, developed to allow users to evaluate and repair accessibility issues in documents, including Open Document Format (ODF), Microsoft DOC, Microsoft DOCX and Office Open XML files. Unfortunately, AcessODF was released in beta in 2012 and has not been maintained since, so it can't be guaranteed to continue working properly as other technologies advance.

AccessODF can check for:

  • Insufficient color contrast
  • Missing text alternatives for images and other objects
  • Main document language and language changes inside the document
  • Headings
  • Tables
  • Compatibility of images with DAISY, the de-facto standard for digital audiobooks

Resources

Testing PDF documents:

Testing other formats:

 

If you are interested in hearing in detail about how to enhance accessibility in a cost-effective manner, order a recording of our webinar (in Finnish) where the CEO of Annanpurna and two experts from Wunder share their wisdom on this topic. By ordering the recording, you will also receive a link to presentation packed with links to useful tools and websites about the topic. 

Get your webinar and presentation links

Wish us to audit or enhance the accessibility of your digital environment?

Send a message to our expert or fill in the form below, and we will contact you!

Related content

Loading...