Innovating on top of standards

Not around them

Kevin Yank

SitePoint Pty. Ltd.

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 License.

Hit 'T' to view with full notes. Download the presentation with examples

Web Standards and Innovation

A big part of my work at SitePoint involves staying on top of the progress of Web technology, and picking out the big ideas, improvements and trends to communicate to our audience in the form of books, articles, newsletters and blog entries.

When I think about the progress of Web standards in, say, the past year, I don't really think about new standards, or even new support for standards in software like Web browsers and development tools.

The truth is, the pace of standards development is really slow—CSS3 has been in development for as long as most designers have known about CSS. And the pace of standards adoption is downright glacial.

Nevertheless, standards are on the minds of everyday designers and developers more than ever before. Why is that? I believe it's because the big brains of Web design are increasingly looking to standards for opportunities for extending the capabilities of the Web.

So, hands up: who is writing XHTML today? Who is serving it so that browsers treat it as more than just HTML code? XHTML, like most Web standards, is a great idea, but without widespread browser support it's little more than a technical curiosity for mainstream Web developers.

No, the standards the brains have been looking at more and more are the established ones. Those with a nice layer of dust on them, like plain old HTML4. They're dusting them off and scouring them for opportunities to use them in new and unexpected ways, building on what's there to produce solutions that are both totally standards compliant but also totally new and exciting.

It's these kinds of advances that have been the most exciting for me this year, so I'd like to take you through a few of my favourites.

JavaScript's Back

Here's a simple example of what I'm talking about.

The use of JavaScript on the Web is seeing a huge resurgence, thanks in part to the ‘A’ word. But it's not all about AJAXremote scripting. We're also seeing a whole world of so-called “cinematic effects”, which are made possible through the use of DOM scripting. The Document Object Model (DOM) is one of those standards that has been around forever, and has been supported in mainstream browsers for years, now. So what is it that's got designers, who turned their backs on DHTML, so excited about “DOM scripting”?

One of the problems with DHTML was that scripts didn't always play well together. Let me show you an example. This page contains three JavaScript scripts of the sort that an inexperienced designer might download from HotScripts.com and chuck into a page together. We've got a slideshow flipping through a series of photos, a countdown timer showing the time before the user's session will expire, and when the user mouses over each of the navigation menu options on the left, a caption image will appear along the top of the page, here.

At a glance, everything seems to be working, but when we try to use the page, a few things go wrong. When I mouse over the links, the captions appear in the place of the slideshow images. If I try to stop the slideshow, the session timer stops instead. And the Play link causes the slideshow to start advancing erratically, instead of every five seconds.

Any experienced coders in the audience want to guess at what's gone wrong here? I'll give you a hint: each of the three problems is a separate case of the same coding issue.

What we're dealing with here are collisions of variable names. If we look at the slideshow script, we can see that it uses four global variables: timer for the timer that advances the slideshow, theImg for the image element in the page that should be used to display the slides, slides for the image files to display, and currentSlide for the current slide in that list that is displayed. As it turns out, our session timer script also uses a global variable timer to count down the seconds, and our link caption script uses a global variable theImg for the image element to display the link captions.

As you can see, this is a polluted namespace. All of our scripts' global variable and function names need to coexist in the same space, so as soon as two scripts use the same variable or function name for two different things, everything falls apart. This is what kept non-coders from taking advantage of DHTML back in the day, so any script that is to gain mass adoption today needs to overcome this issue.

And that's what unobtrusive scripting is all about.

Unobtrusive Scripting

Hands up everyone who understands and practices unobtrusive scripting.

With object oriented programming becoming commonplace on the server side beyond enterprise development, thanks to scripting languages like PHP, the more experimentally-minded are taking another look at the object-based programming capabilities to be found in JavaScript. For most Web scripting tasks, these capabilities are overkill, but it turns out they're really useful for addressing the type of issue we've just seen.

The whole point of objects, whether in JavaScript or in a general programming language like Java, is that they keep the data they work on private, so that only the functions (methods) that need to can access them. That's exactly the kind of thing we need to solve our variable collision issue. As it turns out, since Netscape 4 JavaScript has had a feature that makes it really easy to do this without getting bogged down in the complexities of full-on object-based programming: object literals.

Let's modify the scripts in our page, using object literals to create separate namespaces for each one.

Now each of the scripts in our page works as intended, without interfering with any other script. That's unobtrustive scripting. And it's all built on features of JavaScript that have been in browsers since 1998, and in the ECMAScript standard since 1999 (ECMAScript v3).

Microformats

Microformats are the poster child for innovation on top of existing standards. Tantek Çelik gave a great presentation on Microformats at Web Essentials 2005 in Sydney not long ago. Who here got to hear that presentation? Who here is familiar with Microformats? Who is already using them? Which ones?

In brief, Microformats are simple data formats based on existing Web standards like HTML. Microformats are generally visible to human beings, because the data they contain is written for people first, and computers second. That is, a normal user should be able to read the data right off the Web page, and as a bonus that data can also be read and used by computers to generate additional functionality.

Microformats are the poster child for building on top of established Web standards to produce exciting new behaviour. At this stage, all Microformats are built on plain, old HTML.

Now, if we're talking about building on top of HTML, we need to look at the points of extensibility in the language—those places where the standard allows us to include information for our own purposes without disrupting the content of a Web page. It turns out, there are three places where you can do this: <meta> tags, class attributes, and rel and rev attributes. Each Microformat relies on at least one of these to do its job.

The best way to get a feel for Microformats is to look at a couple, so let's do that. If you decide you want to dive in and see what Microformats exist that you might like to use, the Microformats website is the place to look.

relTag

The Microformat I deal with the most is relTag, which many people are familiar with as Technorati tags.

Can I assume that everyone here has heard of tags—not as in <table> tags, but as in Flickr, del.icio.us and, um, Flickrlicious?

Right, so all the cool kids are talking about “folksonomies” these days—where instead of setting up a category structure for your site, you get your users to assign tags, or keywords, to their bits of content and your site then allows them to browse through that jungle of keywords in, hopefully, a usable way. Flickr does this for photos; del.icio.us does it for bookmarks.

The relTag Microformat provides a tagging system for the entire Web, but unlike del.icio.us, where the visitors to a site assign it tags, relTag lets the author of a page tag its contents.

This Microformat couldn't be simpler. You just create a hyperlink with a rel attribute set to the value "tag". Then you point the hyperlink at a page that provides a description of the tag in question, with the last path segment in the URL containing the keyword for the tag. To be useful, that page should provide some means by which the user can navigate to other content that shares the same tag. The link text should make clear that this is a link to the tag page for the keyword in question.

So what, it's just a link, right? I mean, no browser today is going to treat this link any differently than a normal link. But that's exactly the point: Microformats do their thing without interfering with how Web content normally works.

What this Microformat gives us is a standard and—borrowing a buzzword from the JavaScript world—unobtrusive way to tag Web pages. You can set up a program to scour your site for these links and provide pages to navigate through your tag space.

This particular example links to technorati.com, a site that scours the entire Web for tags pointing to its site. Whereas del.icio.us lets you browse through pages from all over the Web tagged by the surfing public, Technorati lets you see pages from all over the Web tagged by their authors. Both have their uses.

Anyone who reads Wikipedia knows that many pages end with a list of relevant links on the Web. If I worked for Wikipedia, I'd retrofit the site so that it could act as a tag space just like Technorati. Web authors could use relTag to link to Wikipedia entries, and every page on Wikipedia might have a “Web links” tab listing sites that are tagged as having to do with that entry.

hCard

Another Microformat I quite like is hCard. It provides a standard way for programs to access contact information posted on the Web without interfering with how that information is presented in the content of the page.

The hCard Microformat is based heavily on the vCard format, a standard for exchanging contact information that is supported by many applications including Outlook and Outlook Express on Windows, and Mail on Mac OS. Sadly, Mozilla Thunderbird still has only very limited support for vCard, but it can't be too far off.

Basically, hCard uses the HTML class attribute to mark HTML elements so that their contents are interpreted as the values for standard vCard fields. You can then use a script, service, or browser extension to read the page, generate a vCard, and feed it to whatever program you have assigned to handle vCards.

Here's what an hCard might look like. Just some contact details marked up with some run-of-the-mill HTML and styled for display using CSS. The hCard button at the bottom isn't actually part of the hCard, but I'll get to that in a minute. For now, let's look at the source code of this hCard. As you can see, the HTML code is peppered with classes that indicate the meanings of each of the bits of information in the hCard.

So how do we use this? Well, the simplest way is to use the button I've provided here. It links to a script on the Internet that will read this page, parse the hCard data, and send back a vCard file for your contact manager application to open. If the page doesn't provide this convenient utility, you can instead use a bookmarklet that points to the same script—I've got it installed here in my bookmark toolbar. If you want to get even fancier, you can install a browser extension like Smartzilla that will let you right-click any hCard and convert it to a vCard directly, like this.

JEE: Server side standards

The last case of innovation on top of standards that I wanted to show off tonight comes from the server side of things, not the client side. Specifically, it comes from the world of Java Web development with Java Enterprise Edition (JEE). Have we got any Java Web developers in the audience?

JEE, or J2EE as it is still commonly known, is a standard initially produced by Sun but now developed under the auspices of the Java Community Process (JCP), an open consortium of Java developers much like the W3C. Just as W3C standards ensure that Web sites will work across browsers and devices, JEE dictates a common architecture for Web applications that must be supported by all Java Web application servers, ensuring that a Java Web application will work on all compliant servers.

Although the point is debatable, I would say that the Java Web application development world is a lot like the browser space would be without Microsoft—or, rather, what it would be like if Microsoft believed in developing its market-dominating browser to comply with standards developed by the developer community.

Thanks to JEE, all the “plumbing” of Java Web applications is set in stone. You can build whatever you want on top of it, but at some stage you know that you will have a Servlet class with a method named service that will process browser requests, with one server thread per active request, so that multiple requests can be going on at once. The service method does whatever processing it needs to do and sends a response to the browser, then finishes.

Just like standards like HTML, this architecture has been around forever. And at a glance it seems like a rigid standard with a lot of limitations that might hinder Java's ability to keep up with the evolving needs of Web development. But as we've seen, standards aren't always as limited as they may seem at first glance.

Let's take a look at one of these evolving needs and how JEE's architecture allows for it to be supported within established standards.

Receiving events with AJAX

Now I couldn't possibly show up without an AJAX demo for you, could I? What's the AJAX experience like here in the room?

So as you know, AJAX involves the browser making requests to the server in the background without requiring the user to navigate to a new page or submit a form. It can use the results of those requests to update the currently displayed page on the fly.

Compared to how Web applications have traditionally worked, this affords developers a lot of freedom in tailoring the experience to the users' needs. That said, at the end of the day, the browser still has to request everything it gets back from the server. The server can't unilaterally send things to the browser at any time. It's a pull model.

So what do we do when our application needs to respond to events that happen on the server in realtime? We could issue AJAX requests continuously, repeatedly asking the server if anything has happened. But in the programming trade we call this busy-polling, and it's a terrible thing to do. The browser is always busy sending and receiving requests, the server is always busy responding to these requests, and most of the time the response is just “nothing's happened yet”. It's like the two-year-old on a road trip: “Are we there yet?” “No.” “How about now?”

So instead we can use a standing request. Browsers a pretty patient—most will wait up to thirty seconds for a response before they give up on a Web server. So instead of answering the impatient two-year-old right away, the server can sit on the request until something does happen, and then send the response to the browser. If nothing happens for about thirty seconds, the server can respond with the old “nothing's happened yet” message to keep the browser from timing out.

Now this is all sounding pretty good until we look at the JEE specification, and remember how it says we need to process browser requests. It turns out that JEE is designed to handle a short request-response cycle. A compliant JEE server must create a thread to process each browser request before the application gets to look at that request. So if you're setting up, say, a Web chat application and you're going to have hundreds or thousands of users on the system at a time, if each one has a standing request sitting there waiting for an event to happen before sending the response back to the browser, that's a lot of threads with a lot of resources being held in memory to service them. Your server will likely collapse under that kind of load.

Find an extension point

So what have we learned from the other examples we've seen? Look for any spots in the standards that allow for extension and start considering some non-obvious ways of putting them to use.

According to the JEE specification, a Servlet that runs into a problem while processing the request can throw a ServletException, at which point the server will take over and handle the request itself, typically by showing an error message you have configured.

Now, the makers of a Java Web application server called Jetty saw a way to use this mechanism to solve the standing request problem. If your Web application sends a certain type of ServletException, the server terminates the thread that was created to process the request and sets the request aside in a queue. It's allowed to do this because, by the JEE spec, the request has failed with an exception, so the processing thread can end.

Now, when the Web application signals that an event has occurred, the server pulls all the held requests out of the queue and processes them again, as if they were new requests that had only just arrived. Let's look at a demo of this in action.

Okay, so some of you are probably two steps ahead of me. If the application is throwing ServletExceptions to put standing requests on hold, dooesn't that mean that these requests will be bombing out with errors on servers that don't support this special ServletException the way Jetty does?

Well, what Jetty does is it provides a library that you can use to handle standing requests on any JEE-compliant server. With this library, you can make a request wait for server events using an object called a Continuation. This object checks if the library is running on the Jetty server, and if so it throws that special ServletException to put that request on hold. If it's not running on Jetty, however, the library simply waits for an event without putting the request thread on hold, which it can do on any JEE-compliant server.

The way the library is written, other JEE servers can choose to support the special ServletException required for Continuations to put requests on hold too, and the library will use that support. The next step will be for Jetty to submit its work to the Java Community Process for consideration as an official enhancement to the JEE standard.

Though there's a lot of technological complexity going on here, the point that's important to take away here is that Jetty has added an impressive new feature without breaking the JEE standard by building on the points of extension that are there in the spec. On servers that support it, the AJAX applications that use standing requests will be able to handle a lot more users due to the efficient handling of waiting requests. On other servers, the application will still work—just not as efficiently.

I should point out for the sake of any .NET developers in the audience that ASP.NET has had efficient support for standing requests in it since day one. Of course, that's a lot easier to do when you don't have any cross-server compatibility standards to comply with.

For a more technical discussion of Continuations in Jetty 6, which is currently in early beta, see the blog post Jetty 6.0 Continuations - AJAX Ready!

In Summary

Thanks!