Automated Accessibility Checking with aXe

How much time and effort did you spend planning the design of your last website to be accessible to people with special needs and disabilities? I have a hunch that the answer for many readers will be “None”. But hardly anybody would deny there are a considerable amount of Internet users who have trouble accessing sites, due to problems distinguishing colors, reading text, using a mouse, or just navigating a complex website structure.

Accessibility concerns are often ignored due to the efforts required to check them and implement solutions. Not only do developers have to get acquainted with the underlying standards but also constantly check that they are met. Can we make developing accessible websites easier by performing standard checks automatically?

In this article, I’m going to show you how to use the aXe library and some associated tooling to automatically check and report on potential accessibility problems in your sites and apps. By lowering the effort required for such activities and automating some of the manual work, we can achieve a better result for everyone who uses the things we create.

Introducing aXe

aXe is an automated accessibility testing library which has set out on the road to bring accessibility testing into mainstream web development. The axe-core library is open source and designed in a way to be used with different testing frameworks, tools, and environments. For example, it can be run in functional tests, browser plugins or straight in the development version of your application. It currently supports around 55 rules to check a website for various aspects of accessibility.

To give a quick demonstration of how the library works, let’s create a simple component and test it. We won’t create a whole page, but rather just a header.

See the Pen aXe Accessibility Check by SitePoint (@SitePoint) on CodePen.

During the creation of the header we’ve made some brilliant design decisions:

1. We’ve made the background light gray and the links dark gray because this color is classy and stylish;
2. We’ve used a cool magnifier glass icon for the search button;
3. We’ve set the tab index of the search input to 1 so that when a user opens a page he can press tab and instantly type a search query.

Neat, right? Well, let’s see how it looks from the accessibility point of view. We can add aXe from a CDN and log all of the errors to the browser console with the following script.

axe.run(function (err, results) {
  if (results.violations.length) {
    console.warn(results.violations);
  }
});

If you run the example and open the console you will see an array with six violation objects listing the problems that we have. Each object describes the rule that we’ve broke, references to the HTML elements to blame, as well as help information on how to fix the problem.

Here’s a sample of one of the violation objects, shown as JSON:

[  
   {  
      "id":"button-name",
      "impact":"critical",
      "tags":[  
         "wcag2a",
         "wcag412",
         "section508",
         "section508.22.a"
      ],
      "description":"Ensures buttons have discernible text",
      "help":"Buttons must have discernible text",
      "helpUrl":"https://dequeuniversity.com/rules/axe/2.1/button-name?application=axeAPI",
      "nodes":[  
         {  
            "any":[  
               {  
                  "id":"non-empty-if-present",
                  "data":null,
                  "relatedNodes":[  

                  ],
                  "impact":"critical",
                  "message":"Element has a value attribute and the value attribute is empty"
               },
               {  
                  "id":"non-empty-value",
                  "data":null,
                  "relatedNodes":[  

                  ],
                  "impact":"critical",
                  "message":"Element has no value attribute or the value attribute is empty"
               },
               {  
                  "id":"button-has-visible-text",
                  "data":"",
                  "relatedNodes":[  

                  ],
                  "impact":"critical",
                  "message":"Element does not have inner text that is visible to screen readers"
               },
               {  
                  "id":"aria-label",
                  "data":null,
                  "relatedNodes":[  

                  ],
                  "impact":"critical",
                  "message":"aria-label attribute does not exist or is empty"
               },
               {  
                  "id":"aria-labelledby",
                  "data":null,
                  "relatedNodes":[  

                  ],
                  "impact":"critical",
                  "message":"aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty or not visible"
               },
               {  
                  "id":"role-presentation",
                  "data":null,
                  "relatedNodes":[  

                  ],
                  "impact":"moderate",
                  "message":"Element's default semantics were not overridden with role=\"presentation\""
               },
               {  
                  "id":"role-none",
                  "data":null,
                  "relatedNodes":[  

                  ],
                  "impact":"moderate",
                  "message":"Element's default semantics were not overridden with role=\"none\""
               }
            ],
            "all":[  

            ],
            "none":[  
               {  
                  "id":"focusable-no-name",
                  "data":null,
                  "relatedNodes":[  

                  ],
                  "impact":"serious",
                  "message":"Element is in tab order and does not have accessible text"
               }
            ],
            "impact":"critical",
            "html":"<button>\n      <i class=\"fa fa-search\"></i>\n    </button>",
            "target":[  
               "body > header > div > button"
            ],
            "failureSummary":"Fix all of the following:\n  Element is in tab order and does not have accessible text\n\nFix any of the following:\n  Element has a value attribute and the value attribute is empty\n  Element has no value attribute or the value attribute is empty\n  Element does not have inner text that is visible to screen readers\n  aria-label attribute does not exist or is empty\n  aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty or not visible\n  Element's default semantics were not overridden with role=\"presentation\"\n  Element's default semantics were not overridden with role=\"none\""
         }
      ]
   },
]

If you just pick the descriptions of the violations, here’s that it says:

Ensures buttons have discernible text
Ensures the contrast between foreground and background colors meets WCAG 2 AA contrast ratio thresholds
Ensures every HTML document has a lang attribute
Ensures <img> elements have alternate text or a role of none or presentation
Ensures every form element has a label
Ensures tabindex attribute values are not greater than 0

It turns out that our design decisions weren’t so brilliant after all:

1. The two shades of gray that we’ve picked don’t have enough contrast and might be difficult to read for people with vision impairments
2. The magnifier icon of the search button provides no indication of the purpose of the button for someone using a screen reader
3. The tab index of the search input breaks the usual flow of the navigation for people who use screen readers or keyboards, and makes it harder for them to access the menu links.

It also noted several other things we hadn’t thought of. In total performs about 55 different checks including rules from different standard guidelines and best practices.

To see the list of errors we had to inject the script into the page itself. While perfectly doable, this is not very convenient. It would be better if we could perform these checks for any page without having to inject anything ourselves. Preferably using a well-known test runner. We can do this using Selenium WebDriver and Mocha.

Running aXe with Selenium WebDriver

To run aXe using Selenium we’ll use the axe-webdriverjs library. It provides an aXe API that can be used on top of WebDriver.

To set it up, let’s create a separate project and initialise an npm project using the npm init command. Feel free to leave default values for everything it asks for. To run Selenium, you’ll need to install selenium-webdriver. We’ll perform our tests in PhantomJS, so we’ll need to install that as well. Selenium requires Node version 6.9 or newer, so make sure you have it installed.

To install the packages run:

npm install phantomjs-prebuilt selenium-webdriver --save-dev

Now we’ll need to install axe-core and axe-webdriverjs:

npm install axe-core axe-webdriverjs --save-dev

Now that the infrastructure is set up, let’s create a script that runs tests agains sitepoint.com (nothing personal, guys). Create a file axe.js in the project folder and add the following contents:

const axeBuilder = require('axe-webdriverjs');
const webDriver = require('selenium-webdriver');

// create a PhantomJS WebDriver instance
const driver = new webDriver.Builder()
  .forBrowser('phantomjs')
  .build();

// run the tests and output the results in the console
driver
  .get('https://www.sitepoint.com')
  .then(() => {
    axeBuilder(driver)
      .analyze((results) => {
        console.log(results);
      });
  });

To execute this test, we can run node axe.js. We can’t run it from the console since we’ve installed PhantomJS locally in our project. We have to run it as an npm script. To do that, open your package.json file and change the default test script entry:

"scripts": {
    "test": "node axe.js"
},

Now try running npm test. In several seconds you should see a list of violations that aXe found. If you don’t see any, it might mean that SitePoint has fixed them after reading the article.

This is more convenient than our initial approach since we don’t need to modify the page we’re testing and we can handily run them using the CLI. The downside of this, however, is that we still need to execute a separate script to run the test. It would be better if we could run it together with the rest of our tests. Let’s see how to achieve this with Mocha.

Running aXe Using Mocha

Mocha is one of the most popular test runners out there, so it seems like a good candidate to try out with aXe. However, you should be able to integrate aXe with your favorite testing framework in a similar manner. Let’s build up our Selenium example project further.

We’ll obviously need Mocha itself and an assertion library. How about Chai? Install all of it using this command:

npm install mocha chai --save-dev

Now we’ll need to wrap the Selenium code that we’ve written in a Mocha test case. Create a test/axe.spec.js file with the following code:

const assert = require('chai').assert;
const axeBuilder = require('axe-webdriverjs');
const webDriver = require('selenium-webdriver');

const driver = new webDriver.Builder()
  .forBrowser('phantomjs')
  .build();

describe('aXe test', () => {
  it('should check the main page of SitePoint', () => {
    // a Mocha test case can be treated as asynchronous 
    // by returning a promise
    return driver.get('https://www.sitepoint.com/')
      .then(() => {
        return new Promise((resolve) => {
          axeBuilder(driver).analyze((results) => {
            assert.equal(results.violations.length, 0);

            resolve()
          });
        });
      })
      .then(() => driver.quit())
  })
  // The test might take some 5-10 seconds to execute, 
  // so we'll disable the timeout
  .timeout(0);
});

The test will perform a very basic assert by checking if the length of the results.violations array is equal to 0. To run the tests, change the test script to call Mocha:

"scripts": {
  "test": "mocha"
},

The logical next step of this exercise would be to produce a more detailed error report when the test fails. After that, it would also be useful to integrate it with your favorite CI environment to properly display the results of the page. I’ll leave both of these things as exercises for the reader and move on to some useful, additional aXe configuration options.

Advanced Configuration

By default, aXe will run all of the default checks against the whole page. But it’s sometimes more desirable to limit the area of a website being tests or the scope of checks performed.

Checking only parts of the website

You can limit, which parts of the website must be checked or skipped by using the include and exclude methods.

axeBuilder(driver)
  // check only the main element
  .include('main')
  // skip ad banners
  .exclude('.banner')
  .analyze((results) => {
    // ...
  });

This could be useful in case you would only like to check those parts of the website you’re currently working on, or excluding parts that you cannot fix or don’t have direct control of.

Selecting rules

Each rule in aXe is marked with a single or multiple tags, which group them together. You can also disable or enable some of the rules by using the withRules or withTags methods.

• withRules allows you to enable and configure specific rules by ID. For instance, the following example will only check for color contrast and link names.

  axeBuilder(driver)
  .withRules(['color-contrast', 'link-name'])
  .analyze((results) => {
    // ...
  });
  

• withTags allows you to enable rules that are marked with a particular tag (wcag2a in this case):

  axeBuilder(driver)
    .withTags(['wcag2a'])
    .analyze((results) => {
      // ...
    });
  

Conclusion

aXe allows you to partially offload the accessibility testing efforts to a machine and free up time for doing other things like overall project design and structure. While you still need to do some work to integrate it into your development and CI environment in a clean manner, it’s still better than doing it by hand. And when you do it for one of your projects, integrating it with the next should be a breeze.

There are other tools based on aXe. The aXe Chrome plugin allows you to quickly inspect any page in the browser. If you use Gulp, you’ll find a Gulp aXe plugin as well. For React-based projects, there’s a plugin for React StoryBook which allows you to test the accessibility of your React components. You might find that one of these is a better fit for your needs.

Hopefully, this will provide more teams with the motivation to start thinking about accessibility in their projects. What do you think: does aXe seem like a useful tool? Is this something you might try on your next project? Let me know in the comments.

This article was peer reviewed by Mallory van Achterberg, Dominic Myers, Ralph Mason and Joan Yin. Thanks to all of SitePoint’s peer reviewers for making SitePoint content the best it can be!

Frequently Asked Questions (FAQs) on Automated Accessibility Checking with Axe