A client recently asked me to produce an accessible video player, and one of the features she was very keen to have is audio descriptions. Audio descriptions are intended for people who are blind or have impaired vision, providing additional spoken information to describe important visual details.

Traditionally, audio-described videos have to be made specially, with the audio encoded in a separate track of the single video file. It takes pretty specialised video-editing equipment to encode these audio tracks, and that raises the bar for most content producers beyond a practical level.

All the audio-described content I’ve seen on the web is like this. For example, BBC iPlayer has a selection of such content, but the video player doesn’t give you control over the relative volumes, and you can’t turn the audio-descriptions off — you can only watch separate described or non-described versions of the program.

Enter HTML5

The HTML5 video specification does provide an audioTracks object, which would make it possible to implement an on/off button, and to control the audio and video volumes separately. But its browser support is virtually non-existent — at the time of writing, only IE10 supports this feature.

In any case, what my client wanted was audio descriptions in a separate file, which could be added to a video without needing to create a separate version, and which would be easy to make without specialised software. And of course, it had to work in a decent range of browsers.

So my next thought was to use a MediaController, which is a feature of HTML5 audio and video that allows you to synchronise multiple sources. However browser support for this is equally scant — at the time of writing, only Chrome supports this feature.

But you know — even without that support, it’s clearly not a problem to start two media files at the same time, it’s just a case of keeping them in sync. So can we use existing, widely-implemented features to make that work?

Video Events

The video API provides a number of events we can hook into, that should make it possible to synchronise audio playback with events from the video:

• The "play" event (which fires when the video is played).
• The "pause" event (which fires when the video is paused).
• The "ended" event (which fires when the video ends).
• The "timeupdate" event (which fires continually while the video is playing).

It’s the "timeupdate" event that’s really crucial. The frequency at which it fires is not specified, and practise it varies considerably — but as a rough, overall average, it amounts to 3–5 times per second, which is enough for our purposes.

I’ve seen a similar approach being tried to synchronise two video files, but it isn’t particularly successful, because even tiny discrepancies are very obvious. But audio descriptions generally don’t need to be so precisely in sync — a delay of 100ms either way would be acceptable — and playing audio files is far less work for the browser anyway.

So all we need to do is use the video events we have, to lock the audio and video playback together:

• When the video is played, play the audio.
• When the video is paused, pause the audio.
• When the video ends, pause the video and audio together.
• When the time updates, set the audio time to match the video time, if they’re different.

After some experimentation, I discovered that the best results are achieved by comparing the time in whole seconds, like this:

if(Math.ceil(audio.currentTime) != Math.ceil(video.currentTime))
{
  audio.currentTime = video.currentTime;
}

This seems counter-intuitive, and initially I had assumed we’d need as much precision as the data provides, but that doesn’t seem to be the case. By testing it using a literal audio copy of the video’s soundtrack (i.e. so the audio and video both produce identical sound), it’s easy to hear when the synchronisation is good or bad. Experimenting on that basis, I got much better synchronisation when rounding the figures, than not.

So here’s the final script. If the browser supports MediaController then we just use that, otherwise we implement manual synchronisation, as described:

var video = document.getElementById('video');
var audio = document.getElementById('audio');
if(typeof(window.MediaController) === 'function')
{
  var controller = new MediaController();
  video.controller = controller;
  audio.controller = controller;
}
else
{
  controller = null;
}
video.volume = 0.8;
audio.volume = 1;
video.addEventListener('play', function()
{
  if(!controller && audio.paused)
  {
    audio.play();
  }
}, false);
video.addEventListener('pause', function()
{
  if(!controller && !audio.paused)
  {
    audio.pause();
  }
}, false);
video.addEventListener('ended', function()
{
  if(controller)
  {
    controller.pause();
  }
  else
  {
    video.pause();
    audio.pause();
  }
}, false);
video.addEventListener('timeupdate', function()
{
  if(!controller && audio.readyState >= 4)
  {
    if(Math.ceil(audio.currentTime) != Math.ceil(video.currentTime))
    {
      audio.currentTime = video.currentTime;
    }
  }
}, false);

Note that the MediaController itself is defined only through scripting, whereas it is possible to define a controller using the static "mediagroup" attribute:

<video mediagroup="foo"> ... </video>
<audio mediagroup="foo"> ... </audio>

If we did that, then it would work without JavaScript in Chrome. It would sync the media sources, but the user would have no control over the audio (including not being able to turn it off), because the browser wouldn’t know what the audio represents. This is the case in which it would be better to have the audio encoded into the video, because then it could appear in the audioTracks object, and the browser could recognise that and be able to provide native controls.

But since we have no audioTracks data, that’s rather a moot point! So if scripting is not available, the audio simply won’t play.

Here’s the final demo, which will work in any recent version of Opera, Firefox, Chrome, Safari, or IE9 or later:

• Audio Descriptions Demo

This is just a simple proof-of-concept demo, of course — there’s no initial feature detection, and it only has the basic controls provided by the native "controls" attribute. For a proper implementation it would need custom controls, to provide (among other things) a button to switch the audio on and off, and separate volume sliders. The interface should also be accessible to the keyboard, which is not the case in some browsers’ native controls. And it would need to handle buffering properly — as it is, if you seek past the point where the video has preloaded, the audio will continue to play freely until the video has loaded enough to bring it back into sync.

I might also mention that the descriptions themselves are hardly up to professional standards! That’s my voice you can hear, recorded and converted using Audacity. But such as it is, I think it makes an effective demonstration, of how low the technical barrier-to-entry is with this approach. I didn’t have to edit the video, and I made the audio in an hour with free software.

As a proof of concept, I’d say it was pretty successful — and I’m sure my client will be very pleased!

On May 9, you are invited to participate in Global Accessibility Awareness Day (GAAD). The purpose of the day is to get people talking, thinking and learning about digital (web, software, mobile, etc.) accessibility and users with different disabilities.

The target audience of GAAD is the design, development, usability, and related communities who build, shape, fund and influence technology and its use. Yep, that’s us.

Our recent articles touching on web accessibility have made it apparent there are plenty of SitePoint readers interested in the topic of making technology accessible and usable by persons with disabilities, but they’re not always clear on why, or how, or where to start.

Awareness comes first. Read the blog post by Joe Devon that inspired GAAD http://mysqltalk.wordpress.com/2011/11/27/challenge-accessibility-know-how-needs-to-go-mainstream-with-developers-now.

I figure most of our readers will be coming at this from one of two perspectives:

1) I’m experienced with web accessibility issues, and I want to help others devs and designers get on board.

If this is you, consider holding a free talk/gathering/Meetup, organizing hands-on demos, or planning other activities that bring attention to some aspect of digital accessibility. You can make it big or small, virtual or in-person, formal, or less so. You can arrange something at work/school, or open it up to your local community. Get your event listed on the GAAD website so that others can join in.

 2) I’m keen to embrace web accessibility, but I’m not sure where to start.

On May 9, seek out a local public event or find a virtual GAAD activity and get involved. In addition to events, for an hour on that day, people are invited to experience digital accessibility by unplugging the mouse for an hour and using the keyboard alone, or turning on mobile devices’ accessibility features and surfing the web or using favorite mobile apps.

Ideas from last year’s GAAD and resources are on the event’s website. Full details are at http://www.globalaccessibilityawarenessday.org.

Other ways you can show support and keep up with developments are by Liking and Sharing the event’s Facebook page, and/or by following @gbla11yday on Twitter and tweet using the #GAAD hashtag.

You can reach Joe Devon and Jennison Asuncion, co-founders of this global effort at globala11yawarenessday@gmail.com.

We’ll try to point out some of the more interesting events here closer to May 9.

As a person with a disability I find navigating most websites fairly straightforward. I can use the keyboard or a mouse to scroll up and down to view content. With just the space bar I can scroll the page a frame at a time.

Until my browser is hijacked by elements on the page. Then I have to use my mouse to get control of the experience.

Here are four frustrating ways websites hijack my browser.

Search Bar

Hey Google (and now Yahoo!), I’m talking to you!

I get that you want to make it easy for someone to get going on that search, but does my cursor really have to jump to the search box as soon as the page loads?!

You can set up the tab order using tabindex to take the cursor to the box on the tab click.

<input type="text" id="search" tabindex="1">

I get that Google has built this into their system and it’s the expected behavior, but it is a nuisance for people like myself who navigate with our keyboards.

First, it interrupts the browsing flow because I am unable to follow my keyboard shortcut (just backspace on most browsers) to go back a page.

Second, when I start pressing the space bar and want to scroll the page I’m just filling in the search bar. It just doesn’t work.

Confirmation Alert Box

Every now and then I’m surfing the Web and end up on a site with a confirmation box. Sometimes the box pops up on arrival, other times it pops up when I try to leave the page.

I know enough about programming to recognize an opportunity to insert malware. So I never click the confirmation. I don’t know if pressing okay will close the tab or take me to russianhotties.xxx.

My remedy is shutting down my browser then reopening it and tromping through history to the sites I had open.

Popup Ads

This one drives me crazy and I’ll wager most developers are guilty of this one.

Attractive popup ads are no longer an annoyance, but part of the business model of many websites. I’m left to watch the popup obscure my content. I get it. The company has to generate revenue. But the ad should get out of my way.

The standard was to timeout the popup years ago. Some ads creatively move to the sidebar in a static ad. Regardless of the technology, this is available functionality. Another former standard was to tie a keyboard shortcut like esc or x to a close function on the popup. But rarely do I see a popup that I don’t have to click an action item or a tiny close link in the top corner with the mouse.

Static Navbar

Finally, there’s the static navigation bar across the top of the page. Sometimes these are three or four lines tall. They are convenient, giving users easy access to site information.

But go back to scrolling.

When I tap space bar on a typical webpage the window scrolls down one frame. This is such a convenient way to quickly browse a page. As I pointed out above, hijacking my cursor gets in the way. But so do those fancy navigation bars.

Some websites account for the navbar adding padding to accommodate the bar across the top. However, plenty of popular websites are oblivious. As the content scrolls, several lines are obscured by the handy bar.

Am I Just a Curmudgeon?

Our lives as developers are already full of UX demands.

Another accommodation now from this guy who just wants to click his spacebar to scroll is not floating to the top of my list.

I get it. But imagine how all of the little tweaks you build into your workflow save you steps and then having to do away with them because somebody else just changed the way things are done. Look at recent decisions by Adobe and Google to shut down services, for example.

For people with mobility impairments, added obstacles to the content are a real barrier. Just getting a mouse cursor in the precise spot can be a chore.

But thoughtful design and graceful transitions can give the browser back to users like myself to enjoy our convenient web experience.

When was the last time we received a new element? HTML5 introduced just nine new structural tags and they’ve been stable for several years: section, article, aside, hgroup, header, footer, nav, figure and figcaption. There are another 20 or so content elements including video, audio, canvas, progress etc.

Today, we have a new structural element to learn: <main> According to the recent W3C specification:

The main element represents the main content of the body of a document or application. The main content area consists of content that is directly related to or expands upon the central topic of a document or central functionality of an application.

Authors must not include more than one main element in a document.

Authors must not include the main element as a child of an article, aside, footer, header or nav element.

main marks the meaty content within your page, i.e. the target of a “skip to” link. You’ll probably use main where you had a content wrapper before; it will replace tags such as <div id="main">, <div id="page"> or <div id="wrapper">. If you’re using ARIA, you’d use it for the element where role="main" is defined…

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<title>Using main</title>
</head>
<body>
	<header>My page</header>
	<nav>
		<a href="#">Home</a>
	</nav>
	<main>
		<article>
			<h1>My article</h1>
			<p>Content</p>
		</article>
		<aside>
			<p>More information</p>
		</aside>
	</main>
	<footer>Copyright 2013</footer>
</body>
</html>

Browser Support

While main is new, most browsers support unrecognized tags. However, you will need to apply a block style in your CSS, i.e.

main
{
	display: block
}

This is may become unnecessary as browsers evolve (main is supported in Chrome and Firefox nightly builds), but there’s little harm retaining it.

The element’s also been added to the html5shiv so it will work in IE6, 7 and 8. You may need to download the new version if you’re using a locally-hosted file.

Can you use <main> today?

The element has been surprisingly controversial. It’s been argued that it’s unnecessary and pages should permit more than one main tag. But it’s here to stay and I think it will be useful. Just ensure you’ve thoroughly tested your new site prior to going live!

What’s a CAPTCHA?

CAPTCHA stands for Completely Automated Public Turing test to tell Computers and Humans Apart.

On the webpage, a CAPTCHA is a security measure designed to keep out robots by asking the user to key in characters displayed in a box.

Yes, that’s the one: where you have to decipher some squiggly words and enter them in a field before you can submit an online form.And often do it three or four times before you’re successful.

For example:

For more information on definitions, see the comprehensive Wikipedia article on CAPTCHAs.

As far as the real world goes, there are some real doozies out there, like the moving CAPTCHA we found recently in an audit (we’re rebuilding the site so it won’t be there long!)

John Foliot found some inexpressibly confusing CAPTCHAs, an article which is worth a read – please note there is a lot of movement in the article (and no it doesn’t fail the flickering accessibility requirements even if it looks like it)!

Why are there so many CAPTCHAs?

Really, the world would be a much easier place without CAPTCHAs. They are confusing and difficult and we are all time-poor. And surely people want us to use their web site / submit their form / sign up to their newsletter?

The reason that there are so many CAPTCHAs is that there is so much spam in the world. They are perceived as an effective way to prevent robots from, for example, posting comment spam on blogs.

Another common use is to prevent robots with more criminal intent from logging into online bank accounts and the like.

The CAPTCHA is, in reality, a reverse Turing test – performed by a machine to make sure the person filling out the form is, well, a person.

This is also why they are often difficult to interpret. If they were easy to read, then machines could read them, and that would defeat the point.

What about accessibility?

Not only are CAPTCHAs difficult for anyone to use, they are notoriously inaccessible to people with some types of disabilities.

In fulfilling their designated brief of keeping out machines, they keep out people using assistive technologies such as screen readers, thereby closing the door on millions of blind people. So, if you’re blind, use a screen reader and want to log into your CAPTCHA-protected bank account, well … bad luck. Isn’t there a law against that? There ought to be.

There is even a specific section in the Web Content Accessibility Guidelines, Version 2.0 about CAPTCHA, in which their inaccessibility is acknowledged, but the WCAG Working Group feel they can’t be too hard-line about it:

CAPTCHAs are a controversial topic in the accessibility community. As is described in the paper Inaccessibility of CAPTCHA, CAPTCHAs intrinsically push the edges of human abilities in an attempt to defeat automated processes. Every type of CAPTCHA will be unsolvable by users with certain disabilities. However, they are widely used, and the Web Content Accessibility Guidelines Working Group believes that if CAPTCHAs were forbidden outright, Web sites would choose not to conform to WCAG rather than abandon CAPTCHA. This would create barriers for a great many more users with disabilities. For this reason the Working Group has chosen to structure the requirement about CAPTCHA in a way that meets the needs of most people with disabilities, yet is also considered adoptable by sites. Requiring two different forms of CAPTCHA on a given site ensures that most people with disabilities will find a form they can use.

Because some users with disabilities will still not be able to access sites that meet the minimum requirements, the Working Group provides recommendations for additional steps. Organizations motivated to conform to WCAG should be aware of the importance of this topic and should go as far beyond the minimum requirements of the guidelines as possible. Additional recommended steps include:

1. Providing more than two modalities of CAPTCHAs
2. Providing access to a human customer service representative who can bypass CAPTCHA
3. Not requiring CAPTCHAs for authorized users”

http://www.w3.org/TR/UNDERSTANDING-WCAG20/text-equiv-all.html

The emphasis in the above quote is mine. When they talk about “two different forms of CAPTCHA”, they mean one that requires sight to complete plus one that relies on audio and should therefore be accessible to people with impaired vision. They then acknowledge that still won’t make it accessible to everyone.

In reality, the ones that rely on vision are so difficult to use for fully sighted people, while the audio versions use sounds so distorted that no-one can make them out.

So basically they are inaccessible, but the Working Group decided that if people had to choose between CAPTCHAs and WCAG2 they would choose CAPTCHAs, so they allowed for it anyway.

I believe there are some effective unique and most importantly, accessible, alternatives to CAPTCHA, but I’ll talk about that in a later article.

What about reCAPTCHA – it’s accessible isn’t it?

In a word, no.

I’m always asked about reCAPTCHA, or what about Accessible CAPTCHA? I have tested numerous CAPTCHAs and I have never come across an accessible CAPTCHA. Feel free to prove me wrong.

But I am also yet to find a CAPTCHA that complies to WCAG2 either.

There is a fundamental disconnect in intent that means it is highly unlikely that a universally accessible CAPTCHA, or even a set of different CAPTCHAs will ever be devised.

CAPTCHAs are, by definition, exclusive: they are are there to keep baddies out. Their way of testing “badness” does not allow for the legitimate use of machines. So they will tend to be inaccessible.

To understand how this becomes a negative spiral, you only have to look at the Google Account Sign Up process. In order to make it “accessible”, Google provide an audio version. A group of hackers was able to prove that it could pass the audio test robotically (read more about it in the article Google recaptcha brought to its knees).

Did Google concede the CAPTCHA was a failure and should be replaced by something more accessible? Not a bit of it. Instead, they made the audio more distorted so that a machione couldn’t possibly interpret it correctly – and nor could any human. Seriously. Try the Google CAPTCHA yourself.

One of the hackers pinpointed out the problem:

While the changes stymied the Stiltwalker attack, Adam said his own experience using the new audio tests leaves him unconvinced that they are a true improvement over the old system.

“I could only get about one of three right,” he said. “Their Turing test isn’t all that effective if it thinks I’m a robot.”

Couldn’t have said it better myself.

In my next article, I’ll explore how to replace CAPTCHAs with accessible options, while maintaining security and preventing spam.