There are many tools to audit websites for accessibility issues that are fast, cheap, and provide immediate feedback to developers. WAVE and Lighthouse are two common software that can flag accessibility issues within a website in a few seconds, with fa ew clicks.
While these tools are great for general accessibility audit, it’s still extremely important to also run a manual accessibility test and check for issues. There are many issues that the automated tools do not catch on their own. The best way to know if something is accessible is to use it the way users who use accessibility features use it.
The tools themselves warn to manually test
A Lighthouse Chrome extension audit can be completed by opening Chrome Dev Tools and running a scan. Lighthouse will give a score up to 100 on the website page you are testing on four categories which are, Performance, Accessibility, Best Practices, and SEO. The higher the score, the better the website is in the categories they have tested.
If the website has any accessibility issues, Lighthouse will flag it, show the elements where it’s failing, and provide information on why the issue is failing and how to fix it. While Lighthouse tests for accessibility and gives immediate feedback, underneath the section where they identify failed elements, they often have an “Additional Items to manually check” section.
In this section, Lighthouse gives the developer a list of things to manually test to see if the site is accessible. Even if the accessibility score passes and shows 100, Lighthouse still tells the developer to manually test the website and check accessibility issues that their tools did not cover. Lighthouse recognizes that we must still manually test to see if the page is accessible.
Alt text Problem
A simple example of why manual accessibility testing is important is by looking at an alt text. One of the things that WAVE and Lighthouse check when running an accessibility audit is whether the image element is accessible. If an image has alt text associated with it or if there is an empty alt text to let the screen reader know to pass the image, the test will pass, stating that the image is accessible.
However, the alt text needs to be helpful and informational in order for it to be accessible. For example, let’s say there is an image of a dog on the website. The alt text should have information about this image so that when screen reader users focus on it, they know that the element is an image and are provided context about it. This is accessible.
If the image has a non-informative alt text, like an empty alt text, the screen reader will not know information about the image. The accessibility audit will pass, because the auditing tool thinks the image is a decorative image and is ok to not have an alt text, but it’s not accessible because there is no context of the image for screen reader users.
Dynamic Changes and Live Regions
Another area where context is vital is during interactive changes on a page. For example, if a user clicks a "Submit" button and an error message appears, an automated tool might not catch whether that message was actually announced to a screen reader.
While the error text might be technically "accessible" in the code, if the focus doesn't move to the error or if the page doesn't announce the change, the user is left wondering what happened. Manual testing is the only way to verify that these live updates are communicated clearly and in the right context.
Focus Order
Automated tools check if an element on the page can receive focus. However, they don’t tell whether the sequence of focus order actually makes sense. Developers must check the focus order and see if the focus is going through the page in a logical order.
If a developer uses flex-direction: reverse or absolute positioning, it’s possible that the focus order might jump from the header to the footer, then to the middle of the page.
Alternatively, it’s possible that focus might get trapped inside a section of the page without giving users the ability to leave. While the page visually looks great and the accessibility test passes, the page is not great for screen reader users if the focus order jumps all over the page or if the focus is trapped inside a modal. These are some things we must test manually and see if the page is accessible.
Conclusion
Many automated tools flag accessibility issues within websites. While these automated tools are great at catching many accessibility issues, developers must still run a manual test and audit their website for accessibility.
The images might pass accessibility with automated tools, but might have text that does not make sense for the image. Context is key on a website, and automated tools might miss it. The elements on the page might be focusable, but the focus order might not make sense.
It’s extremely important to run manual accessibility tests to catch these issues. We must test the pages with the same tools that accessibility users use in order to truly make sure pages are accessible.