An interesting point was raised by Brendan Davis in my recent post “Responsive Web Design and Scrollbars: Is Chrome’s Implementation Better?”: are RWD breakpoints affected by high pixel-density screens?

The short answer is: no — but we need to delve a little deeper and look at the problems they can cause.

What is Retina?

“Retina” is Apple’s brand name for double-density screens but other manufacturers are creating similar displays. The technology is used in recent iPhones, iPads, MacBook Pros and other high-end devices.

For example, the MacBook Pro 15″ has a resolution of 2,880×1,800 or 220 pixels per inch. At this scale, most people are unable to notice individual pixels at typical viewing distances — applications and websites would be too small to use.

Therefore, the device reverts to a standard resolution of 1,440×900 but the additional pixels can be used to make fonts and graphics appear smoother.

What’s the Problem?

Standard-resolution bitmap images can look blocky on a Retina display. A 400 x 300 photograph is scaled to 800 x 600 pixels but there’s no additional detail. This can be noticeable when compared to smooth fonts and other high-resolution images.

Real-World Usage

If you look around the web, you’d be forgiven for thinking everyone has a Retina display. Currently, it’s only available in high-end devices, but these are coveted by developers so it leads to a disproportionate volume of online discussion. In the real world, the percentage of people using similar displays is in low single figures.

Let’s put it into context: if you’re not developing for the 1% of IE6/7 users, you probably shouldn’t be too concerned about people using Rentina — especially since they can still view your website.

That said, Retina-like screens will eventually migrate to all devices. There’s little reason to fret now, but there’s no harm in some forward planning. Let’s look at the options in order of recommendation…

1. Use SVGs and CSS3 Effects

The clue is in the name but Scalable Vector Graphics are … scalable! It doesn’t matter how big an SVG becomes — it will always be smooth because it’s defined using vectors (lines and shapes) rather than individual pixels.

SVG is not practical for photographs but is ideal for logos, diagrams and charts. The primary drawback is a lack of support in IE8 and below but you could always provide a PNG fallback or use a shim such as Raphaël or svgweb. See also: How to Add Scalable Vector Graphics to Your Web Page.

You may also be able to replace some images entirely. For example, titles, gradients, corners or shadows defined as graphics can be reproduced using CSS3 alone. They will render at a better quality, result in fewer HTTP requests and use less bandwidth.

2. Use Webfonts Icons

The more I use webfonts icons, the more I love them. Like SVGs, fonts are vectors so they’re scalable so you can use font sets which contain icons. They’re ideal for small, frequently used shapes such as email envelopes, telephones, widget controls and social media logos. They also work in every browser including IE6+.

There are plenty of commercial and free icon font sets available:

• Typicons
• Font Awesome
• Iconic
• Foundation

Or you can use a hosted font service such as We Love Icon Fonts.

I recommend creating your own small set of custom icons using online tools such as Fontello or IcoMoon.

3. Use High-Resolution Images When Practical

Retina has four times more pixels than standard screens. If you have a 400 x 300 image (120,000 pixels), you’d need to use an 800 x 600 alternative (480,000 pixels) to render it well on a high-density display.

However, the high-resolution file size may not necessarily be four times larger. Every image is different but if it contains solid blocks of color or details which can be omitted, it may be practical to use a 800 x 600 image and scale it in the browser.

Be pragmatic: if the standard image is 200Kb and the high-resolution version is 250Kb, there is negligible benefit using image replacement techniques. Use the better version throughout.

4. Use CSS Image Replacement

There will be times when high-resolution versions of your image are four times larger — or more. In those circumstances you may want to consider image replacement techniques, i.e. the standard image is replaced by larger alternative on Retina displays. The following media query code could be used:

#myimage {
	width: 400px;
	height: 300px;
	background: url(lo-res.jpg) 0 0 no-repeat;
}
@media
screen and (-webkit-min-device-pixel-ratio: 1.5),
screen and (-moz-min-device-pixel-ratio: 1.5),
screen and (min-device-pixel-ratio: 1.5) {
	#myimage {
		background-image: url(hi-res.jpg);
	}
}

The drawbacks:

1. You will need to create and maintain two sets of images.
2. Some browsers will download both images.

Remember that many of these users will be using smartphones or tablets on slower mobile networks. Detecting the connection speed would be more beneficial than determining the pixel density.

5. Use JavaScript Image Replacement

Retina display detection can be implemented using the following code:

var isRetina = (
	window.devicePixelRatio > 1 ||
	(window.matchMedia && window.matchMedia("(-webkit-min-device-pixel-ratio: 1.5),(-moz-min-device-pixel-ratio: 1.5),(min-device-pixel-ratio: 1.5)").matches)
);

Once a Retina display is determined, you could:

1. Loop through all page images and extract the URL.
2. Append ‘@2x’ to the file name and attempt to load the resulting image URL using Ajax.
3. If found, replace the current image with the high-resolution alternative.

Fortunately, the hard work’s been done for you at retinajs.com. While it only adds 4Kb weight, high-density display devices will download images twice — although the second time will occur as a background process after the page has loaded.

My advice: be practical and keep it simple. Don’t spend inordinate amounts of time attempting to solve minor rendering problems on devices with proportionally few users. Of course, none of that matters when your boss receives his new iPad and starts to complain about image quality…

Before You Get Started

Responsive Web design is intended to ensure that a site’s layout and content scale fluidly to the available screen real estate. This is a great approach for focusing your investments on improving site content and user functionality while ensuring that users have a good experience regardless of what device and screen size they use to visit your site. If you didn’t read the first article in this series, “Why the Web Is Ready for Responsive Web Design,” be sure to read it first.

It’s worth taking a step back, however, to think through your site’s experience and understand whether the device with which a user accesses your site changes the user’s expectations of the site’s functionality. Is the user checking your site for quick updates with her cellphone while she’s on the go? Is he sitting down, 10 feet away from a large TV screen, looking to immerse himself in a relatively passive consumption experience of rich content, videos and games? Are other users sitting down at their PCs, looking to get the most from your site content? Most of all, how do these expectations affect the site layout and functionality that you provide at those corresponding screen sizes?

What Kind of Site Is This?

Planning the content hierarchy for your site across different form factors is definitely the first step to having a great responsive-site experience. Consider the following examples, which evaluate and compare the top experiences that customers want to have when they access your site from a 4-inch phone while they walk or take public transportation, when they’re sitting at their computer desk, and when they’re lounging on their couches in their living rooms.

News Site (Content Consumption)

People visit ContosoNews.com primarily to do one thing—catch up on the day’s current affairs. When you see how this site is presented on a PC screen, it’s designed to have a layout like a newspaper. More important, the single home page is expected to attract and retain different kinds of readers, with interests in current affairs, business, sports, entertainment and other topics, and show them that ContosoNews has content that will interest them. The home page has a rich layout with slide shows, cycling of recommended articles, various categories of news available below the fold if you scroll down, recommended editorials and even the weather. Figure 1 shows a schematic illustration of the site at different resolutions.

Figure 1. Comparing Layouts for ContosoNews.com

If you visit this site on your mobile phone browser, you see a subset of the content, with menu and link navigation to the remaining content. The content that was available on the PC has been prioritized, and the top headline has been given focus above the fold. The slide show of recommended articles is replaced by a series of blurbs with links. The top articles from the Other Categories section are gone, replaced with a single category picker that navigates away from the home page.

In this way, users visiting the site on a phone can, in a cursory glance, become aware of the content available for consumption and dig deeper at their convenience.

Local Attraction (Hyper-Local Site)

Contoso Station is a hip new restaurant in Seattle. When people visit the restaurant’s site on their PC or TV screen, the restaurant proudly shows its latest Yelp reviews, news articles and tweets from users who add the hashtag #i&lt3contoso.

However, when you visit the site on a smartphone, the company makes a fair assumption that you’re visiting its site on the go with hopes to find its location, hours of operation and phone number. The phone might even request your location and show you a map with the quickest route to the restaurant. Some of the remaining content can be presented with much less detail—for example, the Yelp reviews are boiled down to one-line snippets—and the rest of the content (the Twitter feed, for example) can be hidden altogether for users visiting the site on their phones. Figure 2 shows an example of this scenario.

Figure 2. Comparing Layouts for Contoso Station

As seen in Figure 2, local businesses should prioritize and show an entirely different set of content to phone users and make their mobile experiences more sensitive to location.

Media Site (Rich Audiovisual Content)

ContosoTube is a popular Internet service where people share all kinds of videos. Users can see the latest top-rated and most frequently watched content. As they sign-in and explore the site, they can create and edit playlists of videos, get personalized recommendations, subscribe to other users’ playlists and even send each other messages.

The experience of ContosoTube on a phone is geared toward showing videos that a user has opened from other apps (instant messages, email, Twitter and so on), searching to view a video, and letting logged-in users access their existing subscriptions and playlists. Their experience is very limited for content curation.

What’s interesting about ContosoTube is that the Xbox site experience is similar to the phone experience from a user-functionality perspective, although the Xbox site is laid out differently based on screen real estate because even when ContosoTube users visit the site on their large screens, they are probably accessing it from their living room and doing so with controls less precise than a mouse. While the screen size of the TV might tempt developers to provide a more PC-like experience in terms of available functionality, it would be highly likely that users accessing ContosoTube on their TVs would focus primarily on watching content and not on creating it, managing it and messaging with others. Figure 3 compares site layouts for ContosoTube.

Figure 3. ContosoTube on a PC, TV and Smartphone.

HTML5 Games

On Build New Games, a website that explores HTML technologies for creating immersive gaming experiences on the browser, Jack Lawson provides a great discussion about what a gaming experience might be like for a responsively designed Web site.

A game is a great example of site design where users expect entirely different experiences based on the context in which they visit the site. For example, if a user visits the site WorldOfContosoCraft.com from his PC, he probably expects a full-fledged gaming experience—he can play the game himself, interact and communicate socially with other players through the in-game chat feature, make customizations and settings to his avatar and even participate in the in-game marketplace to buy upgrades, armor and other goodies.

On the console, this user might expect a similarly feature-rich experience, but he would also have expectations about being able to use his controller to drive the experience instead of the mouse and keyboard. (There are currently libraries for Chrome and Firefox—although with limited cross-browser support, as Nikhil Suresh points out in his discussion of controller support in JavaScript libraries.)

On the phone itself, the user might be looking to perform simpler actions, such as checking up on his inventory and gamer stats, performing some customizations on the avatar and maybe buying some add-ons from the in-game marketplace. Game developers, who can provide such a contextually relevant experience to users who visit their site from their cellphone for a few minutes, can keep their users engaged in the overall experience even when they can’t play the game.

Considerations for UI Design (aka Fat Fingers)

In addition to information design, you need to think about modes of user input. Today, first and foremost, this means that your site UI be touch-friendly. Visitors are not using touch for your Web site only on phones and tablets; they also use touch-screen-based PCs. Moreover, when you think about users on the Xbox, they’re interacting with the UI elements of your Web page by using a joystick, which is not as precise as a mouse.

Ideally, you do not want to design and code your user interface elements (buttons, links, form controls and so on) differently for touch (tablets and phones) than for PCs with traditional mouse-keyboard elements. In fact, Windows 8 makes this distinction nonexistent, with users able to run Microsoft Surface with a USB mouse as well as desktops with touch-screens. Moving forward, it’s reasonable to assume that more traditional PCs will be equipped with touch-screen functionality.

That’s why the best approach is to design a one-size-fits-all interface for user inputs that is comfortable for touch users to access. Mouse and keyboard users can still interact with these pages just fine.

To highlight some paradigm shifts in this approach, let’s take the example of one of the most common forms of navigation, the drop-down menu, on my favorite local radio station, Contoso Music. (See Figure 4.) This is just one example of a solution to links and navigation menus for touch, but it illustrates the most important considerations we need to take.

Figure 4. The Drop-Down Navigation Menu for Contoso Music

This navigation menu has a couple of issues that go beyond responsive layout, but they are still an integral part of building a unified site experience that scales across multiple devices.

1. First, a lot of sites use navigation menus on which links are revealed when a user mouses over the menu titles. This is absolutely unacceptable because a mouse-over does not translate well to touch browsers. In fact, touch-input aside, you shouldn’t rely on a mouse-over to reveal any useful information at all because it is not keyboard accessible and goes against W3C accessibility guidelines.
2. Second, look at the relative sizes of the links Playlists and DJs. These two pieces of information are supposed to be at the same level in the hierarchy. However, the size of the link is determined by the size of the text. This makes the DJs link less prominent, and also harder to precisely tap on a touch-screen. The DJs link could be as small as 20 px by 40 px, which is not accessible.
3. Another subtle problem, which you can see by glancing at the menu list items, is that only the text items themselves are hyperlinks. Here again, the touch user would be better served if the target for the link Foo was the entire width of the flyout menu instead of just the text width.

Moreover, users on all-in-one devices might utilize the same machine in different device configurations, in which case they might access your site with a mouse at one point and then revisit it later by using touch. It’s beneficial to provide the user with touch-friendly, well-spaced hyperlinks and navigation.

A common example of touch-friendly navigation that lots of sites use for their menus, especially on mobile apps or in a sidebar for tablets, is shown in Figure 5.

Figure 5. A Touch-Friendly Redesign of the Contoso Music Navigation Menu

The navigation menu utilizes touch, mouse or keyboard to expand and collapse the accordion-style submenus. All the links are the same width (even the submenu items), and for each link, the entire rectangle is clickable, not just the text.

A good example of a site that has made this transformation is MSN.com. The old MSN.com (shown in Figure 6) sports a significantly higher content density, with lots of text links (with smaller clickable areas) that are tightly packed (creating room for error when using touch and gaming joysticks), as well as a mouse-over to reveal the subcategories of news (see the menu under Entertainment).

Figure 6. The Old MSN.com

Figure 7 shows the new touch-friendly version of MSN.com. While currently offered only on Windows 8, the touch-friendly UI will be rolled out across the board for all browsers after testing. Notice the more spacious layout and larger hit targets.

Figure 7. The New Look for MSN.com

One Site Fits All

Responsive Web design should not only be about resizing the same content gracefully based on user screen sizes. To best connect with your users across multiple screens, your site should not only be aware of the device’s physical characteristics (such as screen size) but also infer the user’s physical circumstances, modes of input and the kind of information she is seeking.

In the next article in this series, I’ll cover some implementation techniques for responsive design.

I’ve never been particularly complimentary about the official jQuery Plugin Registry. The team are working on it, but perhaps the kindest observation is that it’s “ctional” — functional with all the “fun” removed. The system lacks an easy interface, good search facilities, online demonstrations, developer ratings and reviews.

As an alternative, why not try unheap.com — a new independent jQuery plugin repository…

The benefits of unheap.com become apparent the moment you use it:

• The site has a gorgeous dynamic and responsive interface — it’s a pleasure to use
• Plugins are categorized in sections and sub-sections for interface, inputs, media, navigation and miscellaneous code.
• The search facility works well.
• Plugin lists have direct links to demonstration pages and videos.
• Related plugins are listed.
• User views, votes and bug reports are recorded.
• Social media sharing facilities are provided.
• An RSS feed of new and updated plugins is available for you to check in Google Reader, erm, a compatible RSS Reader.
• The plugin submission process is easy and straight-forward.
• Almost 700 plugins are available at the time of writing.

I’m impressed. Unless the jQuery team can produce a system as nice as unheap.com, perhaps they should give up now and adopt it as the “official” repository?

Unless you’ve found something better?…

The Mobile Playing Field

Today, a large portion of site traffic comes from mobile devices—namely smart phones and tablets—in addition to traditional PCs. Across the globe, mobile devices now account for 12 percent of Internet traffic, and this is scaling up faster than desktop Internet traffic. The fraction of mobile Web traffic is sufficiently higher in nations with high smartphone penetration (for example, 20 percent of US-based Web traffic is via mobile browsing). What’s more, this figure is expected to grow significantly over the next 10 years, as smartphones evolve and mature in terms of hardware and software and adoption picks up in South America, Asia and Africa.

Site owners have begun to take advantage of this trend over the past several years and have primarily relied on native mobile apps for top sites such as Facebook, Hulu and the New York Times. Moreover, up-and-coming Web services such as Pulse, Flipboard and others have even taken to a mobile-first approach, by building apps for iOS and other ecosystems before building a Web site experience. Native apps allow developers to create unique phone-first, touch-optimized experiences for users to interact with their content to take advantage of features like camera integration, geo-location and offline data storage.

Targeting users on mobile natively makes good sense, especially within the US, where more than 50 percent of mobile users own a smartphone. While mobile apps offer site owners a way to connect with users on new form factors, new ways to monetize across platforms and new mobile-scenario-centric experiences to empower and delight their users, they offer an incomplete opportunity for developers compared to the ubiquity and reach of the Web. There are a couple of problems that affect a native mobile-only approach.

Problem 1: Cost of Supporting Multiple Platforms

Creating similar content and experiences across multiple platforms is costly and requires site owners to choose platforms for which to optimize. Additionally, this translates to a limited Web site experience for users who seek out your content from other platforms, especially when you need to prioritize your development investments.

Adopting a responsively designed Web site can help address the investment costs and ensure that your users across all the latest mobile operating systems are enjoying a consistently useful experience.

Scott Scazafavo, former vice president of product management at Allrecipes.com, whose responsibilities included mobile product development, puts it this way:

“To do a decent job developing a native mobile application that can compete with “best in class” offerings that are powered by live data or content (like we have at MSN and also at my previous employer, Allrecipes.com), it typically takes a minimum initial investment of about $250k to define, design and engineer that native application, and then an annual maintenance investment for that native app of $75K to $100K per platform to keep it evolving, feature wise, to keep consumers interested and adoption numbers healthy. That is over and above any internal work needed for design or engineering to create and maintain the services (APIs) that power those products.

The approach we have taken here at MSN for our TMX product (the new touch-first version of MSN.com currently available on IE10) with HTML5, as well as thin-shell apps to help deliver that product into app marketplaces, in addition to mobile browsers, comes only with a small incremental initial investment to what we do with internal resources to create that app product. [That figure is] probably a $25K to $50K initial investment per platform for each app, and a negligible maintenance cost thereafter to maintain those apps.”

Similarly, by using responsive Web design techniques, Clipboard.com was able to target many mobile, small device browsers like Internet Explorer 10 on Windows 8 and Safari on iPhone/iPad at half its expected costs to develop when they began the project.

Problem 2: Fragmented Ecosystems

Even within a given platform, a plethora of device geometries and sizes—as well as supported platform versions—exist. This requires site owners to not only design for near-similar display sizes and resolutions, but also to submit to multiple app stores (Kindle store, Google Play and Nook store, all on the Android platform). Managing multiple assets within the same platform adds complexity to the support matrix. Fix a layout bug in your native app for the Nexus 7, and you might have to fix it again for the Kindle Fire app. This means all your users are not on the same app version, with the same feature set and the same bug fixes.

Similarly (even within the iOS app ecosystem), top apps like ESPN, Spotify, Angry Birds Space and the App Store itself did not correctly occupy the full screen when the iPhone 5 was released, instead just showing users a black bar at the top and bottom of the app. The addition of iPhone 5 required developers to ship app updates to address this simple layout bug.

We’re also still at a stage where vendors are experimenting with new form factors, such as the big screen. For example, more than 25 million Xbox Live users now have access to Internet Explorer 10 on their living room TV screens and are interacting with it not just through a pointer but also through more human-centric mechanisms such as Kinect and Xbox SmartGlass. Today’s technical decision makers are facing a fragmented and very volatile landscape of devices that their users have integrated into their daily routines.

A Unifying Approach: Responsive Web Design

Responsive Web design aims to provide an optimal viewing/consumption experience—easy reading and navigation with minimum resizing, panning, and scrolling —across the gamut of devices that exist in the market, as well as to future-proof your site for those that are yet to come. There already exist different Web tutorials regarding individual techniques that help a site become more responsive. This series aims not only to provide a unified approach to responsive Web design, it aims to impress upon decision makers and developers the immediate need for adopting responsiveness as part of their reach strategy. According to a crawl of the top 5,000 Web sites by modern.IE, only about 14 percent of sites have some form of responsive design. It’s not difficult to see why developers think it’s a daunting task to consider.

Take a look at Figure 1. You can see the relative screen resolutions for the Web browsers on popular smartphones and tablets (the devices are identified in Table 1). The device resolutions, as well as ratio of CSS pixels to hardware pixels (a concept we’ll explain in part 3), are taken from Wikipedia. (Each square corresponds to 100 x 100 px of Web content, laid out at 1x optical zoom.)

Figure 1. Sampling of Resolutions for Several Current Devices

Table 1. Key to Figure 1

 
 
So is cross-browser, cross-device code the solution?

Traditionally, OS-specific apps have been able to provide more sophisticated user engagement because they have access to valuable user information, such as geo-location, offline storage and even custom font support for customized interfaces.

However, modern browsers such as Internet Explorer 10, Google Chrome (version 22), Safari 6 and Firefox (version 17) now offer the lion’s share of these experiences as part of their support for HTML5 and CSS3. HTML5 is not your grandpa’s HTML, which was originally designed to let people encode and deliver pieces of textual information across the Internet. HTML5 is intended for developers to write rich Web-based apps for the twenty-first century. Between HTML5 and CSS3, you get access to once-native features such as media queries, geo-location, custom font support, offline storage and even touch events! This way, your sites can have a different look and layout on hardware of different dimensions, provide users with location-aware services and even provide a valuable experience when the user is disconnected from the Internet.

HTML5 Myths

There are some common HTML5 myths out there. These include:

I can’t monetize HTML5.

HTML5 sites have arguably more monetization opportunities than their app equivalents. App monetization today includes app purchases (although most apps in the iOS apps store are in the free to $0.99 range). This is probably the only way in which HTML5 site experiences can’t be directly monetized. Otherwise, developers have a lot of control over advertising and in-app or in-site purchases. More importantly, a lot of apps tend to limit the amount of navigation a user can do. For example, most reader and newspaper/magazine apps offer textual content and don’t provide the “linky-ness” of the Web, which allows users to navigate to related content while consuming the current Web page.

The Web site experience, when responsively implemented, retains the “linky” nature of the Web and can lead to a higher number of user impressions.

HTML5 cannot be offline.

HTML5 has a couple of different solutions for ensuring that users have a great offline experience. First and foremost, Web pages can specify which of their assets should be made available to users when they are disconnected (using App Cache). This way, the user can still interact with the page even while offline. Additionally, HTML5 can locally store user information and input using Local Storage, as well as IndexedDB. This data persists even if the user closes the browser and can be synced back to the server at a later point in time when the user relaunches the Web page.

Check out the demo for this offline calculator. A user needs to be connected to the Web only the first time he visits it. Subsequently, he can access it offline. Moreover, the user’s calculation and results are stored via Local Storage so he can come back at a later time and continue a calculation.

The Mozilla hacks blog is a great start at busting some common myths about HTML5. It’s important to note here that native apps use APIs that are optimized for device-specific performance. However, HTML5 and CSS3 provide developers with tools to build engaging experiences across a variety of form factors and ensure that you are not missing out on users visiting from other platforms.

CanIUse.com is a great resource for understanding the available browser support for specific HTML5 and CSS features.

Media Queries and Responsive Design

One of the new tools in CSS3 to aid in responsive Web design is called media queries. Media queries allow you to offer your users the same HTML content but enable the browser to detect the size constraints of the device (in pixels) and layout the same content in a different, relevant manner. You can grow or shrink the width of your text and image content, increase or decrease the number of columns in your newspaper-style layout or even hide pieces of information entirely, depending on what you think the right consumption experience is for your user on a given device.

With a combination of media queries to dictate the layout of your content, as well as browser detection to identify additional constraints of the user experience (for example, if the user is interacting with a site via Xbox 360 on a large TV screen), you can identify your users’ needs and deliver the right experience for the current context in which a user has accessed your content—whether it be to consume it richly on a desktop, interact with it via touch on a slate or quickly skim through it on the go on a phone—and do so gracefully with Web technologies.

What’s best, most modern mobile devices support HTML5 and CSS3! This way, you can create near-native experiences directly within the browser. Short of DRM support or access to certain device-unique hardware, there’s no limit to the kinds of experiences you can offer through HTML5, CSS3 and JavaScript. Check out retro Atari video games to get an idea of the kind of cool experiences you can build purely with standards-compliant Web technologies.

It should be noted that using media queries alone to build three different fixed-width layouts for your Web site can definitely help you target common screen dimensions today (for example, desktop, tablet and phone). However, this is not truly responsive Web design. It does not provide an optimal experience for users visiting your site with a device that has an intermediate width, nor does it prepare you for the next wave of “it” devices with new geometries and dimensions.

Build Once! Deploy Once!

If you choose to invest in your site experience, you can design a single HTML5, CSS3 and JavaScript experience that can scale across form factors, from a small smartphone touchscreen to a large cinema display or TV set. We’ll go into the implementation details later in the series, but what’s great to note here is that you never have to choose which of your users you want to delight with that cool new feature, or protect with that high-security patch.

In addition to simplifying your code base and support matrix, this has the following advantages.

Benefit 1: Leave No Users Behind

Betting on powerful native apps for the top one or two mobile platforms can mean that some of your users migrate to competitors if they offer a useful Web experience, with more reach, on all platforms.

Benefit 2: Unified Ad Story

Often, when sites rely on advertising for revenue, they engage with their business partners and sell them advertising piecemeal, based on whether the users are experiencing the full-blown Web version or a limited app version. Also, click-through rates for ads on mobile devices are less than those on desktop PCs, in which case the extra cost of engaging with partners, creating advertising assets for native apps and selling app-specific ad real estate does not justify the additional gains. For example, MSN.com (which has now begun to roll out a unified, media-query-based HTML5 Web site across its international markets) can now unify its ad partnership story across all device types.

With a single HTML5 experience that responsively scales to different form factors, you can cater to a single ad customer with the same set of ad assets across a variety of devices—in the living room, on the work desk and on the go.

Benefit 3: Upgrade Your Site Experience Directly into Your App Experience

Occasionally, you might still hit a roadblock where you want to deliver a great mobile experience to your users that takes advantage of their unique hardware—for example, you want users to get new content from your site by shaking their phone. In this case, you need to access the device accelerometer.

Well, the great news is that you can create a native app by applying a wrapper around your site content and only write the necessary native app code to interact with the additional hardware on the phone. For example, you can host (the responsively scaled down view of) your site content within a WebViewController on an iPhone and just listen for the accelerometer event in your objective-C native code.

This means that for any fixes/features that you build within the Web layer, you don’t need to go through the trouble of shipping app upgrades!

“So, how do I start?”

At this point, we have yet to talk about the “how-tos” of responsive Web design. I’ll get to that in the next part of the series, but I hope you’ve had a chance to consider the long-term benefits of a solution for delivering your content to your users that consists of a single code base, written in familiar Web technologies, with ever-growing support of open JavaScript libraries, rich HTML5 device integration and high-quality CSS3 layout and graphic support. If not, you can always look back at the quickly increasing list of devices in Figure 1.

A client recently asked me to produce an accessible video player, and one of the features she was very keen to have is audio descriptions. Audio descriptions are intended for people who are blind or have impaired vision, providing additional spoken information to describe important visual details.

Traditionally, audio-described videos have to be made specially, with the audio encoded in a separate track of the single video file. It takes pretty specialised video-editing equipment to encode these audio tracks, and that raises the bar for most content producers beyond a practical level.

All the audio-described content I’ve seen on the web is like this. For example, BBC iPlayer has a selection of such content, but the video player doesn’t give you control over the relative volumes, and you can’t turn the audio-descriptions off — you can only watch separate described or non-described versions of the program.

Enter HTML5

The HTML5 video specification does provide an audioTracks object, which would make it possible to implement an on/off button, and to control the audio and video volumes separately. But its browser support is virtually non-existent — at the time of writing, only IE10 supports this feature.

In any case, what my client wanted was audio descriptions in a separate file, which could be added to a video without needing to create a separate version, and which would be easy to make without specialised software. And of course, it had to work in a decent range of browsers.

So my next thought was to use a MediaController, which is a feature of HTML5 audio and video that allows you to synchronise multiple sources. However browser support for this is equally scant — at the time of writing, only Chrome supports this feature.

But you know — even without that support, it’s clearly not a problem to start two media files at the same time, it’s just a case of keeping them in sync. So can we use existing, widely-implemented features to make that work?

Video Events

The video API provides a number of events we can hook into, that should make it possible to synchronise audio playback with events from the video:

• The "play" event (which fires when the video is played).
• The "pause" event (which fires when the video is paused).
• The "ended" event (which fires when the video ends).
• The "timeupdate" event (which fires continually while the video is playing).

It’s the "timeupdate" event that’s really crucial. The frequency at which it fires is not specified, and practise it varies considerably — but as a rough, overall average, it amounts to 3–5 times per second, which is enough for our purposes.

I’ve seen a similar approach being tried to synchronise two video files, but it isn’t particularly successful, because even tiny discrepancies are very obvious. But audio descriptions generally don’t need to be so precisely in sync — a delay of 100ms either way would be acceptable — and playing audio files is far less work for the browser anyway.

So all we need to do is use the video events we have, to lock the audio and video playback together:

• When the video is played, play the audio.
• When the video is paused, pause the audio.
• When the video ends, pause the video and audio together.
• When the time updates, set the audio time to match the video time, if they’re different.

After some experimentation, I discovered that the best results are achieved by comparing the time in whole seconds, like this:

if(Math.ceil(audio.currentTime) != Math.ceil(video.currentTime))
{
  audio.currentTime = video.currentTime;
}

This seems counter-intuitive, and initially I had assumed we’d need as much precision as the data provides, but that doesn’t seem to be the case. By testing it using a literal audio copy of the video’s soundtrack (i.e. so the audio and video both produce identical sound), it’s easy to hear when the synchronisation is good or bad. Experimenting on that basis, I got much better synchronisation when rounding the figures, than not.

So here’s the final script. If the browser supports MediaController then we just use that, otherwise we implement manual synchronisation, as described:

var video = document.getElementById('video');
var audio = document.getElementById('audio');
if(typeof(window.MediaController) === 'function')
{
  var controller = new MediaController();
  video.controller = controller;
  audio.controller = controller;
}
else
{
  controller = null;
}
video.volume = 0.8;
audio.volume = 1;
video.addEventListener('play', function()
{
  if(!controller && audio.paused)
  {
    audio.play();
  }
}, false);
video.addEventListener('pause', function()
{
  if(!controller && !audio.paused)
  {
    audio.pause();
  }
}, false);
video.addEventListener('ended', function()
{
  if(controller)
  {
    controller.pause();
  }
  else
  {
    video.pause();
    audio.pause();
  }
}, false);
video.addEventListener('timeupdate', function()
{
  if(!controller && audio.readyState >= 4)
  {
    if(Math.ceil(audio.currentTime) != Math.ceil(video.currentTime))
    {
      audio.currentTime = video.currentTime;
    }
  }
}, false);

Note that the MediaController itself is defined only through scripting, whereas it is possible to define a controller using the static "mediagroup" attribute:

<video mediagroup="foo"> ... </video>
<audio mediagroup="foo"> ... </audio>

If we did that, then it would work without JavaScript in Chrome. It would sync the media sources, but the user would have no control over the audio (including not being able to turn it off), because the browser wouldn’t know what the audio represents. This is the case in which it would be better to have the audio encoded into the video, because then it could appear in the audioTracks object, and the browser could recognise that and be able to provide native controls.

But since we have no audioTracks data, that’s rather a moot point! So if scripting is not available, the audio simply won’t play.

Here’s the final demo, which will work in any recent version of Opera, Firefox, Chrome, Safari, or IE9 or later:

• Audio Descriptions Demo

This is just a simple proof-of-concept demo, of course — there’s no initial feature detection, and it only has the basic controls provided by the native "controls" attribute. For a proper implementation it would need custom controls, to provide (among other things) a button to switch the audio on and off, and separate volume sliders. The interface should also be accessible to the keyboard, which is not the case in some browsers’ native controls. And it would need to handle buffering properly — as it is, if you seek past the point where the video has preloaded, the audio will continue to play freely until the video has loaded enough to bring it back into sync.

I might also mention that the descriptions themselves are hardly up to professional standards! That’s my voice you can hear, recorded and converted using Audacity. But such as it is, I think it makes an effective demonstration, of how low the technical barrier-to-entry is with this approach. I didn’t have to edit the video, and I made the audio in an hour with free software.

As a proof of concept, I’d say it was pretty successful — and I’m sure my client will be very pleased!

In part 1 we took a look at some of the UI layout implementation details of the InstaFuzz app.  You can get the source code for the app from here if you wish to run it locally.  In this installment we’ll take a look at some of the other bits such as how drag/drop, File API, Canvas and Web Workers are used.

Drag/Drop

One of the things that InstaFuzz supports is the ability to drag and drop image files directly on to the big blackish/blue box.  Support for this is enabled by handling the “drop” event on the CANVAS element.  When a file is dropped onto an HTML element the browser fires the “drop” event on that element and passes in a dataTransfer object which contains a files property that contains a reference to the list of files that were dropped.  Here’s how this is handled in the app (“picture” is the ID of the CANVAS element on the page):

var pic = $("#picture");
pic.bind("drop", function (e) {
    suppressEvent(e);
    var files = e.originalEvent.dataTransfer.files;
    // more code here to open the file
});
 
pic.bind("dragover", suppressEvent).bind("dragenter", suppressEvent);
 
function suppressEvent(e) {
    e.stopPropagation();
    e.preventDefault();
}

The files property is a collection of File objects that can then subsequently be used with the File API to access the file contents (covered in the next section).  We also handle the dragover and dragenter events and basically prevent those events from propagating to the browser thereby preventing the browser from handling the file drop.  IE for instance might unload the current page and attempt to open the file directly otherwise.

File API

Once the file has been dropped, the app attempts to open the image and render it in the canvas.  It does this by using the File API.  The File API is a W3C specification that allows web apps to programmatically access files from the local file system in a secure fashion.  In InstaFuzz we use the FileReader object to read the file contents as a data URL string like so using the readAsDataURL method:

var reader = new FileReader();
reader.onloadend = function (e2) {
    drawImageToCanvas(e2.target.result);
};
reader.readAsDataURL(files[0]);

Here, files is the collection of File objects retrieved from the function handling the “drop” event on the CANVAS element. Since we are interested only in a single file we simply pick the first file from the collection and ignore the rest if there are any.  The actual file contents are loaded asynchronously and once the load completes, the onloadend event is fired where we get the file contents as a data URL which we then subsequently draw on to the canvas.

Rendering the filters

Now the core functionality here is of course the application of the filters.  In order to be able to apply the filter to the image we need a way to access the individual pixels from the image.  And before we can access the pixels we need to have actually rendered the image on to our canvas.  So let’s first take a look at the code that renders the image that the user picked on to the canvas element.

Rendering images on to the canvas

The canvas element supports the rendering of Image objects via the drawImage method.  To load up the image file in an Image instance, InstaFuzz uses the following utility routine:

App.Namespace.define("InstaFuzz.Utils", {
    loadImage: function (url, complete) {
        var img = new Image();
        img.src = url;
        img.onload = function () {
            complete(img);
        };
    }
});

This allows the app to load up image objects from a URL using code such as the following:

function drawImageToCanvas(url) {
    InstaFuzz.Utils.loadImage(url, function (img) {
        // save reference to source image
        sourceImage = img;
 
        mainRenderer.clearCanvas();
        mainRenderer.renderImage(img);
 
        // load image filter previews
        loadPreviews(img);
    });
}

Here, mainRenderer is an instance created from the FilterRenderer constructor function defined in filter-renderer.js.  The app uses FilterRenderer objects to manage canvas elements – both in the preview pane as well as the main canvas element on the right.  The renderImage method on the FilterRenderer has been defined like so:

FilterRenderer.prototype.renderImage = function (img) {
    var imageWidth = img.width;
    var imageHeight = img.height;
    var canvasWidth = this.size.width;
    var canvasHeight = this.size.height;
    var width, height;
 
    if ((imageWidth / imageHeight) >= (canvasWidth / canvasHeight)) {
        width = canvasWidth;
        height = (imageHeight * canvasWidth / imageWidth);
    } else {
        width = (imageWidth * canvasHeight / imageHeight);
        height = canvasHeight;
    }
 
    var x = (canvasWidth - width) / 2;
    var y = (canvasHeight - height) / 2;
    this.context.drawImage(img, x, y, width, height);
};

That might seem like a lot of code but all it does ultimately is to figure out the best way to render the image in the available screen area considering the aspect ratio of the image.  The key piece of code that actually renders the image on the canvas occurs on the last line of the method.  The context member refers to the 2D context acquired from the canvas object by calling its getContext method.

Fetching pixels from the canvas

Now that the image has been rendered we will need access to the individual pixels in order to apply all the different filters that are available.  This is easily acquired by calling getImageData on the canvas’s context object.  Here’s how InstaFuzz calls this from instafuzz.js.

var imageData = renderer.context.getImageData(
    0, 0,
    renderer.size.width,
    renderer.size.height);

The object returned by getImageData provides access to the individual pixels via its data property which in turn is an array like object that contains a collection of byte values where each value represents the color rendered for a single channel of a single pixel.  Each pixel is represented using 4 bytes that specify values for the red, green, blue and alpha channels.  It also has a length property that returns the length of the buffer.  If you have a 2D co-ordinate you can easily transform that into an index into this array using code such as the following.  The color intensity values of each channel ranges from 0 through 255.  Here’s the utility function from filters.js that accepts as input an image data object along with 2D coordinates for the pixel the caller is interested in and returns an object containing the color values:

function getPixel(imageData, x, y) {
    var data = imageData.data, index = 0;
 
    // normalize x and y and compute index
    x = (x < 0) ? (imageData.width + x) : x;
    y = (y < 0) ? (imageData.height + y) : y;
    index = (x + y * imageData.width) * 4;
 
    return {
        r: data[index],
        g: data[index + 1],
        b: data[index + 2]
    };
}

Applying the filters

Now that we have access to the individual pixels, applying the filter is fairly straightforward.  Here, for instance is the function that applies a weighted grayscale filter on the image.  It simply picks intensities from the red, green and blue channels and sums them up after applying a multiplication factor on each channel and then assigns the result for all 3 channels.

// "Weighted Grayscale" filter
Filters.addFilter({
    name: "Weighted Grayscale",
    apply: function (imageData) {
        var w = imageData.width, h = imageData.height;
        var data = imageData.data;
        var index;
        for (var y = 0; y < h; ++y) {
            for (var x = 0; x < w; ++x) {
                index = (x + y * imageData.width) * 4;
                var luminance = parseInt((data[index + 0] * 0.3) +
                                         (data[index + 1] + 0.59) +
                                         (data[index + 2] * 0.11));
                data[index + 0] = data[index + 1] =
                    data[index + 2] = luminance;
            }
 
            Filters.notifyProgress(imageData, x, y, this);
        }
 
        Filters.notifyProgress(imageData, w, h, this);
    }
});

Once the filter has been applied we can have that reflected on the canvas by calling the putImageData method passing in the modified image data object.  While the weighted grayscale filter is fairly simple most of the other filters use an image processing technique known as convolution.  The code for all the filters is available in filters.js and the convolution filters were ported from the C code available here.

Web Workers

As you might imagine doing all this number crunching to apply the filters can potentially take a long time to complete.  The motion blur filter for instance uses a 9×9 filter matrix for computing the new value for every single pixel and is in fact the most CPU intensive filter among them all.  If we were to do all this computation on the UI thread of the browser then the app would essentially freeze every time a filter was being applied.  To provide a responsive user experience the app delegates the core image processing tasks to a background script using the support for W3C Web Workers in modern browsers.

Web workers allow web applications to have scripts run in a background task that executes in parallel along with the UI thread.  Communication between the worker and the UI thread is accomplished by passing messages using the postMessage API.  On both ends (i.e. the UI thread and the worker) this manifests as an event notification that you can handle.  You can only pass “data” between workers and the UI thread, i.e., you cannot pass anything that has to do with the user interface  – you cannot for instance, pass DOM elements to the worker from the UI thread.

In InstaFuzz the worker is implemented in the file filter-worker.js.  All it does in the worker is handle the onmessage event and apply a filter and then pass the results back via postMessage.  As it turns out, even though we cannot pass DOM elements (which means we cannot just hand a CANVAS element to the worker to have the filter applied) we can in fact pass the image data object as returned by the getImageData method that we discussed earlier. Here’s the filter processing code from filter-worker.js:

importScripts("ns.js", "filters.js");
 
var tag = null;
onmessage = function (e) {
    var opt = e.data;
    var imageData = opt.imageData;
    var filter;
 
    tag = opt.tag;
    filter = InstaFuzz.Filters.getFilter(opt.filterKey);
 
    var start = Date.now();
    filter.apply(imageData);
    var end = Date.now();
 
    postMessage({
        type: "image",
        imageData: imageData,
        filterId: filter.id,
        tag: tag,
        timeTaken: end - start
    });
}

The first line pulls in some script files that the worker depends on by calling importScripts.  This is similar to including a JavaScript file in a HTML document using the SCRIPT tag.  Then we set up a handler for the onmessage event in response to which we simply apply the filter in question and pass the result back to the UI thread by calling postMessage.  Simple enough!

The code that initializes the worker is in instafuzz.js and looks like this:

var worker = new Worker("js/filter-worker.js");

Not much is it?  When a message is sent by the worker to the UI thread we handle it by specifying a handler for the onmessage event on the worker object.  Here’s how this is done in InstaFuzz:

worker.onmessage = function (e) {
    var isPreview = e.data.tag;
    switch (e.data.type) {
        case "image":
            if (isPreview) {
                previewRenderers[e.data.filterId].
                    context.putImageData(
                        e.data.imageData, 0, 0);
            } else {
                mainRenderer.context.putImageData(
                    e.data.imageData, 0, 0);
            }
 
            break;
        // more code here
    }
};

The code should be fairly self-explanatory.  It simply picks the image data object sent by the worker and applies it to the relevant canvas’s context object causing the modified image to be rendered on screen.  Scheduling a filter for conversion with the worker is equally simple.  Here’s the routine that performs this function in InstaFuzz:

function scheduleFilter(filterId,
                             renderer,
                             img, isPreview,
                             resetRender) {
    if (resetRender) {
        renderer.clearCanvas();
        renderer.renderImage(img);
    }
 
    var imageData = renderer.context.getImageData(
        0, 0,
        renderer.size.width,
        renderer.size.height);
 
    worker.postMessage({
        imageData: imageData,
        width: imageData.width,
        height: imageData.height,
        filterKey: filterId,
        tag: isPreview
});
}

Wrapping it up

The source for InstaFuzz is available for download here.  We saw that fairly intricate user experiences are possible today with HTML5 technologies such as Canvas, Drag/Drop, File API and Web Workers.  Support for all of these technologies is quite good in pretty much all modern browsers.  One thing that we did not address here is the question of making the app compatible with older browsers.  That, truth be told, is a non-trivial but necessary task that I will hopefully be able to talk about in a future article.

When I started out on this app I was only really just interested in seeing if the web platform had really evolved to a point where an app like the hugely popular Instagram app could be built using just HTML, JavaScript and CSS.

As it turns out we can in fact do exactly that.  This article walks you through the technologies that make this possible and shows how it is entirely feasible today to build interoperable web applications that provide a great user experience no matter what brand of browser the user is running.

If you happen to be one of the two or so people who have not heard about Instagram then you might be pleased to hear that it is a hugely popular photo sharing and social networking service that allows you to take pictures, apply interesting digital filters on them and share them with the world.  The service became so popular that it was acquired by Facebook for a bag full of cash and stock in April of 2012.

InstaFuzz is the name of the app I put together and while I don’t expect to be acquired by Facebook or anybody else for a billion green it does, however, make the case that an app such as this one can be built using only standards compliant web technologies such as Canvas, File API, Drag/Drop, Web Workers, ES5 and CSS3 and still manage to run well on modern browsers such as Internet Explorer 10, Google Chrome and Firefox.  And you could easily use the code to build a Windows Store app too.

About the app

If you’d like to take a look at the app, then here’s where it is hosted:

http://blogorama.nerdworks.in/arbit/InstaFuzz/

As soon as you load it up, you’re presented with a screen that looks like this:

The idea is that you can load up a photograph into the app either by clicking on the big red “Add” button on the bottom left hand corner or drag and drop an image file into the blackish/blue area on the right. Once you do that you get something that looks like this:

You’ll note that a list of digital filters listed on the left of the screen shows a preview of what the image would look like if you were to apply the said filter.  Applying a filter is a simple matter of clicking on one of the filter previews on the left.  Here’s what it looks like after applying the “Weighted Grayscale” filter followed by a “Motion Blur”.  As you can tell filters are additive – as you keep clicking on filters, they are applied on top of what was applied earlier:

Let’s next take a look at how the UI layout has been put together.

UI Layout

The HTML markup is so little that I can actually reproduce the contents of the BODY tag in its entirety here (excluding the SCRIPT includes):

<header>
    <div id="title">InstaFuzz</div>
</header>
<section id="container">
    <canvas id="picture" width="650" height="565"></canvas>
    <div id="controls">
        <div id="filters-list"></div>
        <button id="loadImage">Add</button>
        <input type="file" id="fileUpload"
           style="display: none;"
           accept="image/gif, image/jpeg, image/png" />
    </div>
</section>
&nbsp;
<!-- Handlebar template for a filter UI button -->
&nbsp;
<script id="filter-template" type="text/x-handlebars-template">
    <div data-filter-id="{{filterId}}">
        <div>{{filterName}}</div>
        <canvas width="128" height="128"></canvas>
    </div>
</script>

There’s nothing much going on here.  Pretty much everything should be standard fare.  I will however draw attention to the fact that I am using the Handlebars JavaScript templating system here for rendering the markup for the list of filters on the left of the screen.  The template markup is declared in the HTML file (the SCRIPT tag in the snippet shown above) and then used from JavaScript.  The template markup is then bound to a JavaScript object that supplies the values for handlebars expressions such as {{filterId}} and {{filterName}}.  Here’s the relevant piece of JS from the app with a bit of DOM manipulation help from jQuery:

var templHtml = $("#filter-template").html(),
    template = Handlebars.compile(templHtml),
    filtersList = $("#filters-list");
 
var context = {
    filterName: filter.name,
    filterId: index
};
 
filtersList.append(template(context));

As you can tell from the HTML markup, all the filter preview boxes feature a CANVAS tag, as does the big box on the right where the final output is rendered.  We’ll go into a bit more detail later on in the article as to how canvas technology is used to achieve these effects.

The app also uses CSS3 @font-face fonts to render the text in the header and the “Add” button.  The fonts have been taken from the excellent Font Squirrel site and here’s what the declaration looks like:

@font-face {
    font-family: 'TizaRegular';
    src: url('fonts/tiza/tiza-webfont.eot');
    src: url('fonts/tiza/tiza-webfont.eot?#iefix')
           format('embedded-opentype'),
         url('fonts/tiza/tiza-webfont.woff') format('woff'),
         url('fonts/tiza/tiza-webfont.ttf') format('truetype'),
         url('fonts/tiza/tiza-webfont.svg#TizaRegular') format('svg');
    font-weight: normal;
    font-style: normal;
}

This directive causes the user agent to embed the font in the page and make it available under the name assigned to the font-family rule which in this case is “TizaRegular”.  After this we can assign this font to any CSS font-family rule like how we normally do.  In InstaFuzz I use the following rule to assign the font to the header element:

font-family: TizaRegular, Cambria, Cochin, Georgia, Times,
    "Times New Roman", serif;

You might also have noticed that there is a subtle shadow being dropped on the page by the container element.

This is made possible using the CSS3 box-shadow rule and here’s how it’s used in InstaFuzz.

-moz-box-shadow: 1px 0px 4px #000000, -1px -1px 4px #000000;
-webkit-box-shadow: 1px 0px 4px #000000, -1px -1px 4px #000000;
box-shadow: 1px 0px 4px #000000, -1px -1px 4px #000000;

This causes the browser to render a shadow around the relevant element.  Each comma separated section in the value specifies the following attributes of the shadow:

1. Horizontal offset
2. Vertical offset
3. Spread distance – positive values have the effect of softening the shadow
4. Shadow color

One can specify multiple shadow values separated by comma as in fact has been done above.  Note that I’ve also specified the shadow using vendor prefix syntax for Firefox and Chrome/Safari using the moz and webkit prefixes.  This causes the shadow to continue to work in versions of those browsers where support for this capability was provided using the vendor prefixed version of the rule.  Note that the W3C version of the rule – box-shadow – is specified last.  This is done deliberately to ensure that in case the browser supports both the forms then only the W3C behavior is actually applied to the page.

One often finds that web developers either fail to include vendor prefixed version of a given CSS3 rule for all the browsers that support that rule and/or fail to include the W3C version as well.  Often developers just put the webkit version of the rule ignoring other browsers and the W3C standard version.  This causes two problems – [1] poor user experience for users who are using non-webkit browsers and [2] it ends up resulting in webkit becoming a de-facto standard for the web.  Ideally we want W3C to be driving the future of the web and not one specific browser implementation.  So here are some things to remember when playing with experimental implementations of CSS features:

1. Use vendor prefixed versions of CSS rules by all means but remember to specify the rule for all supported browsers and not just the one that you happen to be testing the page in (if you’re using Visual Studio to edit your CSS then you might be interested in the supremely excellent extension for Visual Studio called Web Essentials that makes the job of managing vendor prefixes about as simple as it can possibly get).
2. Remember to specify the W3C version of the rule as well.
3. Remember to order the occurrence of the rules so that the W3C version shows up last.  This is to allow clients that support both the vendor prefixed version and the W3C version to use the W3C specified semantics for the rule.

That’s all for now.  In the next and final article in this series, we’ll take a look at how the app supports drag/drop of files, the use of File API, how the filters themselves work and how we prevent the UI thread from freezing by delegating the core number crunching work to web workers.

HTML5 introduced a few semantic elements that are not supported in older browsers. Some of these new elements are no different than generic block elements so they don’t pose any compatibility problems. All you need to ensure compatibility is to add a CSS rule to your website that causes the relevant elements to behave like block elements.

But Internet Explorer versions 8 and under pose a challenge. Any element not in the official roster of elements cannot be styled with CSS. That means we cannot make then behave like block elements or give them any formatting.

For example, the following code will not work.

<style>
section {display: block}
</style>
<section>This is on its own line.</section>
<section>This should appear on a separate line.</section>

But that’s not all. These new elements behave as if they don’t exist. For example, the following CSS won’t work, since the section element won’t match the universal selector.

<style>
body * span {color: red}
</style>
<body>
  <section>
    <span>This should be red, but won't be red in IE 8.</span>
  </section>
</body>

Fortunately for us, a workaround exists that allows Internet Explorer (IE) to recognize these new elements allowing them to be styled, and thus giving us full use of these new semantic tags. It’s a tool called HTML5Shiv.

As noted on the linked Google page, “shiv” and “shim” are interchangeable terms in this context.

But how did we go from IE not even acknowledging the existence of this element, to now being able to use it?

The trick is that calling document.createElement("section") will suddenly cause IE to recognize the section element. No one knows why, but it works and you don’t even need to use the node returned by that function.

But you need to make sure to call it early on in your website before any of those elements are used, otherwise it won’t work.

You will need to call it for each and every new HTML5 elements like so:

"abbr article aside audio bdi canvas data datalist details figcaption figure "+
  "footer header hgroup main mark meter nav output progress section " +
  "summary template time video"
  .replace(/\w+/g, function(a){ document.createElement(a) });

Notice we’re using the replace method of the string object to succinctly iterate over each contiguous length of characters matched by the regular expression and executing the callback function for each character block which in turn calls createElement.

Here on in, we’ll call this method, “shivving the document”, so that the document can render the new HTML5 elements.

Now our previous two HTML examples work. But that’s not all there is to it.

Pitfall 1: HTML5 and innerHTML

If HTML is being generated using innerHTML and it is called on a node not currently attached to a document (AKA an orphaned node), then it’s deja vu all over again. The following two examples will not render the section element, even though it’s run on a document already shivved.

var n1 = document.getElementById("n1");
n1.parentNode.removeChild(n1);
n1.innerHTML = "<section>Sect 1<\/section>";  //won't work

var n2 = document.createElement("div");
n2.innerHTML = "<section>Sect 2<\/section>";  //won't work

In the second example above, if we append the node to the document first before calling innerHTML, then it will work:

var n2 = document.createElement("div");
document.body.appendChild(n2);
n2.innerHTML = "<section>Sect 2<\/section>";  //works

We can conclude that although we shivved the document earlier on, orphaned elements do not benefit from the shiv when calling innerHTML.

What can we do? For starters, whenever we need to set innerHTML we should append it to the document first. An alternative is to first shiv the document property of the orphan before working with the orphan.

First let’s put our shiv in its own function.

function iehtml5shiv(doc) {
  "abbr article aside audio bdi canvas data datalist details " +
    "figcaption figure footer header hgroup main mark meter nav " +
    "output progress section summary template time video"
    .replace(/\w+/g, function(a){ doc.createElement(a) });
}

The next time we have an orphan element, we can do this:

var n1 = document.createElement("div");
iehtml5shiv(n1.document);
n1.innerHTML = "<section>Sect 1<\/section>";  //works

Notice how it’s just like shivving the document, but on the document property of the element. And notice we’re accessing document instead of ownerDocument. Both are different things as shown here:

alert(n1.document == document);  //false
alert(n1.ownerDocument == document);  //true

Now we have two methods to make sure our call to innerHTML works when handling HTML5 elements.

Pitfall 2: cloneNode

It appears our cousin cloneNode is also susceptible to losing its shiv. Any HTML5 elements which are cloned, or have had their parents cloned, will lose their identity.

Notice how the below element has colons in its nodeName, meaning it’s being confused for an element from another namespace.

var n2 = n1.cloneNode(true);
alert(n2.innerHTML);  //outputs: <:section>Sect 1</:section>

This happens even if the node was already attached to the document.

There isn’t much we can do here except roll out your own implementation of cloneNode, which is trivial enough.

Pitfall 3: Printing

Whenever you print a webpage, IE appears to generate a new document before printing which means all the shiv workarounds are not preserved.

There isn’t much you can do to mitigate this. The HTML5Shiv tool works around this by listening for the onbeforeprint event and replacing all the HTML5 elements on the page with normal elements and then doing the reverse on the onafterprint event.

Thankfully, the HTML5Shiv tool does that nicely for us.

References

• The HTML5Shiv tool: https://github.com/aFarkas/html5shiv
• The story of the HTML5 Shiv: http://paulirish.com/2011/the-history-of-the-html5-shiv/

The second branch of the web’s most popular JavaScript library was released on April 18, 2013. jQuery 2.0 is now available for download but don’t rush off and install it just yet — there is one major change…

No More Support for IE6/7/8

(or IE9 and IE10 if they’re used in “Compatibility View” mode).

I still think it’s a little premature to abandon IE8 but the team couldn’t wait any longer. jQuery 2.0 removes all the legacy IE code for node selection, DOM manipulation, event handling and Ajax.

This has resulted in a 11% file size reduction from 32,819 bytes to 29,123 bytes. That’s 3.6Kb for the gzipped minified version — it’s unlikely to be noticeable even on a dial-up connection. Admittedly, the team hoped to save more but discovered that Android/Webkit 2.x browsers still required many workarounds.

If you need to support IE8 and below, stick with jQuery 1.9.x for now. You could conditionally load version 2.0 in all other browsers, but:

1. conditional loading will offset any gains in file size reduction and processing, and
2. you may encounter differences between the two jQuery branches. The team has pledged to minimize API divergence but there will almost certainly be issues. jQuery 1.10 will address known compatibility problems.

Custom Builds

The custom build feature has been refined in version 2.0 so you can exclude any of 12 unused modules and shrink jQuery below 10Kb. The modules which can be omitted are:

• ajax: all Ajax functionality, transports, and event shorthands.
• ajax/xhr: XMLHTTPRequest Ajax transport only.
• ajax/script: <script> Ajax transport only.
• ajax/jsonp: JSONP Ajax transport only (depends on ajax/script).
• css: The .css() method plus non-animated .show(), .hide() and .toggle().
• deprecated: deprecated methods (currently .andSelf() only).
• dimensions: .width() and .height() methods, including inner- and outer- variations.
• effects: the .animate() method and its shorthands such as .slideUp().
• event-alias: event attaching/triggering shorthands such as .click().
• offset: the .offset(), .position(), .offsetParent(), .scrollLeft(), and .scrollTop() methods.
• wrap: the .wrap(), .wrapAll(), .wrapInner(), and .unwrap() methods.
• sizzle: the Sizzle selector engine. When this module is excluded, it is replaced by a rudimentary selector engine based on the native querySelectorAll method which does not support some advanced jQuery selectors.

For example, if you are using CSS3 animations rather than jQuery methods, you could omit the effects module and possibly dimensions, offset, wrap and sizzle.

Creating your own custom build is not for the faint-hearted and you’ll require some knowledge of Git, Node.js and Grunt. Full instructions are available but, longer term, I hope the team can implement an online build process similar to Modernizr.

Should I Upgrade?

It’s important to understand that jQuery 2.0 has API parity with jQuery 1.9. There are a small number of bug fixes but no new features.

However, if you’re one of those lucky developers who has dropped support for IE6/7/8, grab jQuery 2.0 and don’t look back.

JavaScript, HTML5, CSS3: it’s a changing landscape. Web Directions brings together practitioners and big thinkers at the leading edge to educate and inspire at a “festival of code”.

SitePoint is partnering with Web Directions Code 2013 in Melbourne, Australia, and we have two free tickets worth $999 each to give away.

To enter, all you need to do is let us know why would you like to be at this year’s conference. We will pick the two most creative answers on Wednesday 24th April.

Sorry, this promotion is now closed. Winners will be contacted directly.

You can still buy tickets to Web Directions Code 2013. Use SITEPOINT as a voucher code to take $150 off the full price.

Fine print:

• Two conference tickets to be given away to the two most creative answers.
• Conference to be held in Melbourne, Australia, 2-3 May, no transport or accommodation included.
• Promotion ends 23th April 2013.

I’d like to show you three cunning things you can do with regular expressions, that provide neat solutions to some very sticky problems:

1. Removing Comments
2. Using Replacement Callbacks
3. Working With Invisible Delimiters

1. Removing Comments

Regular expressions make light work of single-character delimiters, which is why it’s so easy to remove markup from a string:

str = str.replace(/(<[\/]?[^>]+>)/g, '');

It’s the negation in the character class that does the real work:

[^>]

Which means “anything except <”. So the expression looks for the starting tag-delimiter and possible slash, then anything except the closing tag-delimiter, and then the delimiter itself. Easy.

However comments are not so simple, because comment delimiters are comprised of more than one character. Multi-line comments in CSS and JavaScript, for example, start with /* and end with */, but between those two delimiters there could be any number of unrelated stars.

I often use multiple stars in comments, to indicate the severity of a bug I’ve just noticed, for example:

/*** this is a bug with 3-star severity ***/

But if we tried to parse that with a single negation character, it would fail:

str = str.replace(/(\/\*[^\*]+\*\/)/g, '');

Yet it’s not possible with regular expressions to say: “anything except [this sequence of characters]”, we can only say: “anything except [one of these single characters]”.

So here’s the regular expression we need instead:

str = str.replace(/(\/\*([^*]|(\*+[^*\/]))*\*+\/)/gm, '');

The expression handles unrelated characters by looking at what comes after them — stars are allowed as long as they’re not followed by a slash, until we find one that is, and that’s the end of the comment.

So it says: “/ then * (then anything except * OR one or more * followed by anything except * or /)(and any number of instances of that) then one or more * then /”.

(The syntax looks particular convoluted, because "*" and "/" are both special characters in regular expressions, so the ambiguous literal ones have to be escaped. Also note the "m" flag at the end of the expression, which means multi-line, and specifies that the regular expression should search across more than one line of text.)

Using the same principle then, we can adapt the expression to search for any kind of complex delimiters. Here’s another one that matches HTML comments:

str = str.replace(/(<!\-\-([^\-]|(\-+[^>]))*\-+>)/gm, '');

And here’s one for CDATA sections:

str = str.replace(/(<\!\[CDATA\[([^\]]|(\]+[^>]))*\]+>)/gm, '');

2. Using Replacement Callbacks

The replace function can also be passed a callback as its second parameter, and this is invaluable in cases where the replacement you want can’t be described in a simple expression. For example:

isocode = isocode.replace(/^([a-z]+)(\-[a-z]+)?$/i,
  function(match, lang, country)
  {
    return lang.toLowerCase()
      + (country ? country.toUpperCase() : '');
  });

That example normalizes the capitalisation in language codes — so "EN" would become "en", while "en-us" would become "en-US".

The first argument that’s passed to the callback is always the complete match, then each subsequent argument corresponds with the backreferences (i.e. arguments[1] is what a string replacement would refer to as "$1", and so on).

So taking "en-us" as the input, we’d get the three arguments:

0. "en-us"
1. "en"
2. "-us"

Then all the function has to do is enforce the appropriate cases, re-combine the parts and return them. Whatever the callback returns is what the replacement itself returns.

But we don’t actually have to assign the return value (or return at all), and if we don’t, then the original string will be unaffected. This means we can use replace as a general-purpose string processor — to extract data from a string without changing it.

Here’s another example, that combines the multi-line comment expression from the previous section, with a callback that extracts and saves the text of each comment:

var comments = [];
str.replace(/(\/\*([^*]|(\*+[^*\/]))*\*+\/)/gm,
  function(match)
  {
    comments.push(match);
  });

Since nothing is returned, the original string remains unchanged. Although if we wanted to extract and remove the comments, we could simply return and assign an empty-string:

var comments = [];
str = str.replace(/(\/\*([^*]|(\*+[^*\/]))*\*+\/)/gm,
  function(match)
  {
    comments.push(match);
    return '';
  });

3. Working With Invisible Delimiters

Extracting content is all very well when it uses standard delimiters, but what if you’re using custom delimiters that only your program knows about? The problem there is that the string might already contain your delimiter, literally character for character, and then what do you?

Well, recently I came up with a very cute trick, that not only avoids this problem, it’s also as simple to use as the single-character class we saw at the start! The trick is to use unicode characters that the document can’t contain.

Originally I tried this with undefined characters, and that certainly worked, but it’s not safe to assume that any such character will always be undefined (or that the document won’t already contain it anyway). Then I discovered that Unicode actually reserves a set of code-points specifically for this kind of thing — so-called noncharacters, which will never be used to define actual characters. A valid Unicode document is not allowed to contain noncharacters, but a program can use them internally for its own purposes.

I was working on CSS processor, and I needed to remove all the comments before parsing the selectors, so they wouldn’t confuse the selector-matching expressions. But they had to be replaced in the source with something that took up the same number of lines, so that the line-numbers would remain accurate. Then later they would have to be added back to the source, for final output.

So first we use a regex callback to extract and save the comments. The callback returns a copy of the match in which all non-whitespace is converted to space, and which is delimited with a noncharacter either side:

var comments = [];
csstext = csstext.replace(/(\/\*([^*]|(\*+([^*\/])))*\*+\/)/gm,
  function(match)
  {
    comments.push(match);
    return '\ufddf' + match.replace(/[\S]/gim, ' ') + '\ufddf';
  });

That creates an array of comments in the same source-order as the spaces they leave behind, while the spaces themselves take-up as many lines as the original comment.

Then the originals can be restored simply by replacing each delimited space with its corresponding saved comment — and since the delimiters are single characters, we only need a simple character class to match each pair:

csstext = csstext.replace(/(\ufddf[^\ufddf]+\ufddf)/gim,
  function()
  {
    return comments.shift();
  });

How easy is that!

I’d like to share with you a mysterious internal secret kept within Microsoft for a long time. It’s the real story behind the concept of the CSS Grid Layout imagined by Microsoft for IE10 and Windows Store Apps.

Most of you probably think that this specification was designed to give developers a better layout engine for their websites and applications. But the original motivation was completely different. The very first aim was to be able to create a Tetris-like game in an easy way!

I’m sure you’re not convinced yet. That’s why I’m going to prove it to you using Blend 5 as a companion. Ok, let’s go!

Pre-requisites: to follow this tutorial, you need first to:

1. Download/buy & install Windows 8 RTM on your machine: http://msdn.microsoft.com/en-US/windows/apps/br229516.aspx
2. Download & install the free edition of Visual Studio 2012 Express RTM for Windows 8: http://msdn.microsoft.com/en-US/windows/apps/br229516.aspx that includes Expression Blend 5 for Visual Studio or use the higher versions.

Step 1: discover the secret behind the CSS Grid Layout thanks to Blend 5

Launch Expression Blend 5 and create a new HTML (Windows Store) project of type Blank App. Name it “TheRealCSSGridStory”:

Replace:

<p>Content goes here</p>

With:

<div class="mainGrid">
</div>

Let’s create a grid containing 10 columns and 20 lines, whatever the screen’s resolution, by using fraction units. For that, add this CSS rule:

.mainGrid {
    display: -ms-grid;
    width: 100%;
    height: 100%;
    -ms-grid-columns: (1fr)[10];
    -ms-grid-rows: (1fr)[20];
}

Select in the Live DOM the <div> mainGrid element and you should obtain this:

Let’s draw a shape inside this beautiful grid. Add this block of HTML inside the main grid:

<div class="shape1">
</div>

And insert this CSS associated with it:

.shape1 {
    -ms-grid-column: 4;
    -ms-grid-row: 3;
    -ms-grid-column-span: 3;
    -ms-grid-row-span: 2;
    background-color: red;
}

You should now see that in Blend 5:

Cool, but nothing yet looks like to a Tetris gaming piece. Let’s work on that. Add these two DIVs inside the shape1:

<div class="line1shape1"></div>
<div class="line2shape1"></div>

and replace the previous .shape1 rule with this block of CSS:

.shape1 {
    -ms-grid-column: 4;
    -ms-grid-row: 3;
    -ms-grid-column-span: 3;
    -ms-grid-row-span: 2;
    display: -ms-grid;
    -ms-grid-columns: 1fr 1fr 1fr;
    -ms-grid-rows: 1fr 1fr;
    width: 100%;
    height: 100%;
}
.line1shape1 {
    -ms-grid-column-span: 2;
    background-color: red;
}
.line2shape1 {
    -ms-grid-column: 2;
    -ms-grid-row: 2;
    -ms-grid-column-span: 2;
    background-color: red;
}

The shape1 is currently spanning on three columns and two rows. I’ll then create a new grid inside this area defined by three columns and two rows in order to have cells having exactly the same size as the cells of the main grid, whatever the resolution might be.

Once done, I’ll create two lines in order to mimic the Z shape of the Tetris game. You should now have this result:

Even better, play with the various views available in the Device tab and you’ll see that our game is already implementing a responsive design! This is freaking cool, isn’t it?

Here, for instance, is the outputs for the snapped view and the portrait view:

Let’s now resolve another problem.

The Tetris grid gaming grid is composed of squares. Our current responsive design is stretching 100% width. Building a Windows 8 application for the Windows Store will most of the time meet widescreen monitors (current tablets are 1366×768 or 1920×1080 and most desktop PC have a 16/9 ratio). Let’s then assume that targeting a widescreen ratio is addressing almost all cases. To compute the proper responsive width, you need to do: 9/16 * 10/20 (the ratio of the main gaming grid) which equals to: 28.125%.

Add this rule to target the main grid in full screen landscape mode:

@media screen and (-ms-view-state: fullscreen-landscape) {
    .mainGrid {
        width: 28.125%;
        }
}

Let’s now center the gaming grid by using… the CSS Grid Layout again! (And you should now start to believe it was truly designed for Tetris!)

Switch the body element to –ms-grid made of one column and one row:

body {
    display: -ms-grid;
    -ms-grid-columns: 1fr;
    -ms-grid-rows: 1fr;
    width: 100%;
    height: 100%;
}

Now simply add this attribute to the CSS associated to the the main grid:

-ms-grid-column-align: center;

And the grid is now centered:

At this stage, you’re probably already shocked. “How could I have missed this incredible secret?” you’re wondering to yourself.

Take a breath.

Now that you know THE secret, let’s continue this tutorial together to discover other awesome possibilities delivered by the CSS specifications combined together.

Step 2: moving or rotating a shape

My first idea was to try avoiding JS by using as much CSS as possible. I then first tried to use CSS3 Animations to move and animate the shape on the various rows/columns. But the bad news is that you can’t change the –ms-grid-column or –ms-grid-row values via CSS3 animations. This will then be the job of some JavaScript code.

I then started to think about how I will rotate the shape. CSS Transforms seemed to be perfectly adapted for that. Let’s check that by doing some experiments. Blend 5 is really cool for that as you can directly see live the outcome of your changes.

Add a rotation of 90 degrees on shape1 by adding this class to its DIV element:

.shape1rotated {
    transform: rotate(90deg);
}

I’m sure you weren’t expecting this:

Problem: it’s not properly aligned to the gaming grid. To align our shape to the grid, we need some small adjustments:

.shape1rotated {
    transform-origin: 33% 50%;
    transform: rotate(90deg) translateX(-33%);
}

And we now have the same rotation as a Tetris-like game. Here are two screenshots before/after the rotation:

We can even go further than that by using a transition set on shape1 via this:

transition: all 1s ease-out;

And now, removing/adding the .shape1rotated class on the shape1 DIV will trigger a smooth rotation animation.

Check out the result inside Blend 5 thanks to this short video:

Download Video: MP4, WebM, HTML5 Video Player by VideoJS

At this stage, we could think that this approach is the good one to build our game. But this is unfortunately not yet the case. Here’s why. Try moving the shape by simply changing its –ms-grid-column property. Blend 5 will reflect the changes directly. When not rotated, the shape can be moved up to the 8th column:

So far so good. But when you’re rotating it by adding the .shape1rotated class to the DIV:

You see that there is still 1 row available on the right for the shape’s move. If you think we simply need to move it to the 9th row, you’re wrong! Indeed, here is what we’ll obtain on the 9th row:

You probably forgot that we’re currently moving a DIV element displaying a grid layout of three columns and two rows matching exactly the underlying gaming grid. When moving it, we really have the feeling that this is a block part of the main grid we’re moving. But for this trick to work, we need at least three columns available to contain our shape element. If it’s contained in two columns (when set to the 9th column) or less, it will be “compressed”, as in the screenshot.

There are two ways to resolve that.

1 – Stop using CSS Transforms and draw the rotated shape using another grid definition. For instance, using three divs inside the shape instead of two. But using such an approach will prevent us having the nice CSS Animations in place.
2 – Redefined the main grid to work on 12 columns instead of 10 and we’ll use only the columns from 2 to 11 (a sort of clipping area if you wish). This will resolve our “overflow” problem.

Let’s implement the second solution.

Redefine the main grid using this:

.mainGrid {
    display: -ms-grid;
    -ms-grid-columns: (1fr)[12];
    -ms-grid-rows: (1fr)[20];
    -ms-grid-column-align: center;
    width: 100%;
    height: 100%;
}

To also have the proper ratio, you need to update the associated media query:

@media screen and (-ms-view-state: fullscreen-landscape) {
    .mainGrid {
        width: 33.75%;
        }
}

33.75% = 9/16 *12/20

Let’s also add a “virtual grid” to delimitate the space where we will be able to move the shapes.  Inside the main grid DIV, insert this one:

<div class="virtualGrid">
</div>

Associated with this block of CSS:

.virtualGrid {
    -ms-grid-column: 2;
    -ms-grid-column-span: 10;
    -ms-grid-row-span: 20;
    border-right-style: dashed;
    border-left-style: dashed;
    background-color: #505A5A;
}

It will help to delimit the gaming area with a grey background and some dashed border lines.

Now, if I’m moving the Z shape on column 9, row 2, here is the result:

If I’m rotating it with CSS Transforms, I can move it correctly on column 10:

Bonus – handling the Portrait mode:

If you’d like to support the portrait mode (which is even better for the Tetris grid game), add this CSS Media Query definition:

@media screen and (-ms-view-state: fullscreen-portrait) {
        .mainGrid {
        width: 106.66%;
        }
}

As the ratio need to be computed as = 16/9 * 12/20 = 106,66%.

Step 3: adding some code to handle part of the game logic

Now that we’ve solved the graphics part of the game using only some pure CSS & HTML code, we need the help of JavaScript to move/rotate the shape in the gaming area. We’re going to re-implement the CSS logic via a JS object that’s going to be defined thanks to WinJS.Class.

Open the “TheRealCSSGridStory” in Visual Studio 2012.

Create a TetrisShapeZ.js file in the JS directory and copy/paste this code:

(function () {
    "use strict";
    var ShapeZ = WinJS.Class.define(
    /// Constructor - columnIndex is optional. If provided, defines the column the shape starts in
        function (columnIndex) {
            // We're creating the equivalent of this HTML block :
            // <div class="shape1 ">
            //     <div class="line1shape1"></div>
            //     <div class="line2shape1"></div>
            // </div>
            this._shape1 = document.createElement("div");
            var line1 = document.createElement("div");
            var line2 = document.createElement("div");
            this._shape1.className = "shape1";
            line1.className = "line1shape1";
            line2.className = "line2shape1";
            this._shape1.appendChild(line1);
            this._shape1.appendChild(line2);
            // Boolean to indicate if the shape is in its default orientation mode or not
            // True means not rotated, false means rotated
            this._defaultOrientation = true;
            // Setting the column position in the main grid
            if (columnIndex) {
                this._currentColPos = columnIndex;
                this._shape1.style.msGridColumn = this._currentColPos;
            }
            else {
                this._currentColPos = 1;
            }
            // We always start at line 1
            this._currentLinePos = 1;
            // Boolean to know if the shape can be move/rotate or not
            // If true, this means we've reached the last line possible
            this._fixed = false;
        },
        {
            /// Specify in which HTML element displayed in CSS Grid you'd like to work with
            /// width is the number of columns of the grid & height is the number of lines
            insertIntoGrid: function (element, width, height) {
                element.appendChild(this._shape1);
                this._gridWidth = width;
                this._gridHeight = height;
                // These are the left & bottom max limit for this shape
                // when displayed in default orientation mode
                this._maxLeft = width - 3;
                this._maxBottom = height - 1;
            },
            /// Rotate the Z shape 90 degrees anti/clockwise using CSS Transforms
            /// by simply removing/adding the shape1rotated class
            rotate: function () {
                if (!this._fixed && this._defaultOrientation) {
                    // rotating 90 degrees clockwise, it will trigger also the CSS Transition
                    WinJS.Utilities.addClass(this._shape1, "shape1rotated");
                    this._defaultOrientation = false;
                    // the left limit is now +1 vs the default orientation
                    this._maxLeft = this._gridWidth - 2;
                }
                else {
                    if (!this._fixed && this._currentColPos < this._maxLeft) {
                        // removing the shape1rotated will automatically reset the shape
                        // to its initial matrix and again the CSS Transition will do the
                        // animation for you
                        WinJS.Utilities.removeClass(this._shape1, "shape1rotated");
                        this._defaultOrientation = true;
                        this._maxLeft = this._gridWidth - 3;
                    }
                }
            },
            // Internal function called by public moveLeft/moveRight
            _moveHorizontally: function (direction) {
                if (!this._fixed && (this._currentColPos < this._maxLeft || direction === -1) &&
                (this._currentColPos > 2 || direction === 1)) {
                    this._currentColPos = this._currentColPos + direction;
                    this._shape1.style.msGridColumn = this._currentColPos;
                }
            },
            /// Move the shape on the immediate left column
            /// Test if you've reached the left limit or not
            moveLeft: function () {
                this._moveHorizontally(-1);
            },
            /// Move the shape on the immediate right column
            /// Test if you've reached the right limit or not
            moveRight: function () {
                this._moveHorizontally(1);
            },
            /// Move the shape down on the immediate below line
            /// Test if you've reached the bottom limit or not
            moveDown: function () {
                if (!this._fixed) {
                    this._currentLinePos = this._currentLinePos + 1;
                    this._shape1.style.msGridRow = this._currentLinePos;
                    if (this._currentLinePos === this._maxBottom) {
                        this._fixed = true;
                    }
                }
            }
        }
    );
    WinJS.Namespace.define("CSSTetris", { ShapeZ: ShapeZ });
} ());

Simply read the code to understand what it’s doing. It should be commented enough to be self-explicit.

Add a reference to this script file in default.html and keep only this block of HTML inside the body:

<div class="mainGrid">
    <div class="virtualGrid">
    </div>
</div>

Jump into default.js.

The cool part of having well-documented code is that we now have interesting IntelliSense details like for the constructor:

or the rotate function:

To properly use this code, add this block of JS just after processAll call:

document.addEventListener("keydown", OnKeyDown, false);
mainGrid = document.getElementsByClassName("mainGrid")[0];
myShape = new CSSTetris.ShapeZ(4);
myShape.insertIntoGrid(mainGrid, 12, 20);
init();

And add these two functions:

function init() {
    setInterval(function () {
        myShape.moveDown();
    }, 1000);
}
function OnKeyDown(event) {
    switch (event.keyCode) {
        case KEYCODE_X:
            myShape.rotate();
            break;
        case KEYCODE_LEFT:
            myShape.moveLeft();
            break;
        case KEYCODE_RIGHT:
            myShape.moveRight();
            break;
        case KEYCODE_DOWN:
            myShape.moveDown();
            break;
    }
}

And we’re done! We now have a very basic game using CSS Grid Layout coupled with CSS Transforms & Animations for the graphics part and a couple of JS lines of code to have the beginning of the basics of a Tetris-like game.

Here is a short video demonstrating the final result:

Download Video: MP4, WebM, HTML5 Video Player by VideoJS

You can download the final Visual Studio solution corresponding to the three steps of this tutorial here: http://david.blob.core.windows.net/win8/TheRealCSSGridStory.zip

So, are you now convinced that CSS Grid Layout was made to simplify the creation of Tetris-like games?

I met my two Skimr co-founders, Josf and Davd, at my current employer – Seznam.cz. It’s basically the Czech Yahoo – email, maps, news, search, classifieds, etc. We are part of a small team (skunk works or Google X, if you will) where we explore new product ideas, create prototypes, experiment … some of these make it into production.

BTW, my colleagues’ real names are Josef and David. The shortened versions, Josf and Davd, come from VentureBeat’s Dylan Tweney. He thought my name was Peter and I had shortened my name in the same way we shortened the Skimr name (instead of Skimmer). So then we applied that to my colleagues’ names. We think it’s pretty funny, so we keep referring to them as Josf and Davd.

I’ve been using Google Reader for a long time. Basically since the day it launched. I then moved to the popular Reeder app. My favorite subscriptions have always been TechCrunch, Techmeme, etc. However, due to the time difference between Europe and US, the majority of articles were published during the night. So in the morning, I had a gazillion of new unread posts – impossible to read them all.

So I was thinking of a way to literally skim through the headlines and quickly see what’s new. I realized the title alone was sometimes not enough, but a combination of the title plus the first one or two sentences was typically enough to get the basic story. If it was interesting enough, I could click on it and read the full article.

After I realized this, I asked Josf if we could take Techmeme’s RSS feed and display it on a web page in this format. A few days later, I was using this web page all the time. I showed it to my friends and they all asked if they could add their own feeds.

I put together some mockups, and Josf started coding them. Then, I asked my other colleague Davd, who is a great designer, to help me make the mockups look nice. You know, fonts, colors, proportions, etc. At this stage, we were still building a small tool for ourselves and our friends. So the backend was very basic, only able to handle tens of unique RSS feeds, max.

A few weeks later, Skimr was up and running and all our friends were using it. Daily. Many times a day. It was unbelievable. It encouraged me to post it on a few sites and see what other people had to say. And that’s when it started to get complicated …

We didn’t really know any journalists or bloggers outside of the Czech Republic who we could ask to write a review. So, I went to Quora and found a few posts there that were listing websites where people could submit their projects. After submitting Skimr to approx 20 websites, I decided I was not going to continue this way because it had no effect at all.

There was one site left on my list – Web.Appstorm.net. So, I pushed myself and submitted Skimr there too. Five minutes later, I hade an email from their editor, Matthew Guay, saying he was just looking for something interesting to write about for the upcoming weekend and out of nowhere, Skimr landed in his mailbox. He liked the service, so he immediately wrote a review.

And then on Monday, more reviews started to pop up.

I was extremely happy. Davd, too. Only Josf was furious. His backend was absolutely not built for a larger user base. Fortunately, we were running Skimr on AWS, so there was always an option to rent more instances. Also, we have learned there is a devastating conversion rate between tweet-article-visitor-registered user.

Let’s say a blogger writes an article and promotes it to his 10,000 followers on Twitter. 10% of them will click on the link and read the article. That’s 1,000 people. 10% of them will click on the link within the article and actually visit your site. That’s 100 people. And then 10% of them sign up, that’s 10 people.

So, a few weeks later, we had a small user base of people who really liked the service, were sending us emails about features they wanted us to add and telling their friends their friends about Skimr. They were adding a few unique feeds an hour, so no big deal.

But then, Google announced they were going to shut down their Reader. All of a sudden, people all around the world started looking for alternatives. Instead of a few unique RSS feeds being added to Skimr every hour, there were 100 unique feeds added every minute. And then 1,000 feeds every minute. We kept upgrading our AWS instances, launching more and more download robots, renting more storage, etc.

It was fascinating. But we knew it was only a matter of time when our tiny system would break under such a heavy load.

What’s more, journalists became very responsive. I started hearing back from them. Even Walt Mossberg replied to my email to him. Coming from a small country, it was amazing.

Fortunately, over the weekend, people calmed down a bit. We immediately contacted my long time friend Radm who is an excellent backend programmer to help us out. The following paragraphs come from him:

The current version is build on LAMP technology + JavaScript based front end. This solution works perfectly as a proof of concept and is able to handle traffic up to a certain degree.

After Google announced the end of its Reader, the traffic grew significantly and we handled this situation by adding new AWS servers. This solution had one big disadvantage – it was pretty expensive and the PHP backend was not really using the servers efficiently. To solve this challenge as a first step we decided to migrate the site from AWS to our own dedicated servers (based in Prague). This saved us approximately 2/3 of the operational costs.

As a next step we worked on a rewrite of the site to a more efficient technology – to the same technology we have been successfully using for several years now at www.agentka.cz (a popular local job site). Usually, one of the bottlenecks on heavily used sites is the DB which is under high load. This is a paradox as most of the sites are not really using advanced (sometimes even basic) SQL database features the way they were designed – heavy loaded databases are usually denormalized and serves only as a persistent storage. The situation is even worse when it is required to perform full-text searches over all stored data.

In the past, this observation led us to drop SQL database and use a special indexer that stores documents with a very loose structure and provides very fast full-text search features and, of course, provides support for atomic operations. This indexer is the heart of the site and it is used for storing all data, it also allows us to scale the site the way we need to. After we migrate the current Skimr implementation off LAMP to the described one we will be able to provide a lot of nice features to our users – for example full text search in the feeds. This work is in progress and we expect to have a new version in several weeks’ time.

Launching a web project always brings this dilemma – launch quickly and risk the system breaking if it becomes popular; or build it properly first and be ready. If we had chosen the second approach, we would have completely missed the window of opportunity that appeared when Google anounced the Reader closure.

Let’s go from the backend to seeing how the frontend works. It has two basic parts:

1. Static pages generated on the server via PHP, e.g. http://skimr.co/about
2. Dynamic pages generated on the client in javascript, e.g. http://skimr.co/techcrunch

Interesting is the second part – dynamic pages. The core of the application is just a vanilla JavaScript build with static object literals (single modules for homepage, articles, updater, etc). We don’t use any big JavaScript library like jQuery. We want to have full control with a focus on speed and simplicity.

Rendering of HTML is done strictly by Dust.js with precompiled templates on the server. JavaScripts don’t contain any HTML code or any direct manipulation of the DOM. We just supply data from the server via the JSON API, process them and send them to a render function that generates HTML for an active page.

Working with URLs is done by HTML5 function pushState() with a fallback to hash. This is very important and the main key to a really “responsive” application, because you are able to “switch” pages instantly.

All in all, the frontend is quite simple and completely responsive – works nicely on desktops, tablets and smartphones, out of the box. Everything is done just in CSS, we don’t have any application logic to support responsive function of the application. We are also using LESS in style sheets. We can definitely recommend LESS for any kind of web or application.

Regarding design, I wanted Skimr to be as simple as possible. I’ve always thought traditional RSS readers were a bit too complicated for the average users. But I liked the concept of pulling in content via RSS and showing it in a unified form. Yet, I thought it was fair for us to only show a small part of the original content, not the whole article. If people liked it, they could click and read it on the publisher’s website.

The biggest problem of traditional websites is the number of elements people try to put on them. All the menus, options, contextual content, etc. I find them quite distracting. The point of any website should be to solve a specific need and then let go.

When it comes to news sites, they should tell their users what’s new and that’s about it. So I made sure there was a clear purpose as to why Skimr existed – to quickly see what’s new. After a few iterations, I’ve realized there was a simpler way than a traditional RSS reader – just a simple feed. Latest entries on the top, older below. Nothing more, nothing less. It sounds so obvious now, but back then, it meant ditching the majority of features a good RSS reader would have.

Another interesting aspect of building Skimr is that of native vs. web apps. I had an interview with ZDNet’s Dave Morgenstern on this topic.

Basically, I think very soon, people won’t be able to tell the difference between native and web apps. The latest developments in CSS, JS and HTML 5 enable app designers to create almost identical user experiences to traditional native apps.

So, I thought web app would be totally enough. But I was wrong.

One of the main requests from our users are native apps. They think native apps offer better performance. Since it’s basically a distribution channel, we have decided to build these for them.

Speaking of new feature requests, folders are an interesting story. A long time ago, I used to be subscribed to many websites in Google Reader. Over time, I reduced the number to approximately 10. On that principle, we’ve designed Skimr for users with few feeds. It turned out very quickly that there are many people all around the world who still consume hundreds or maybe even thousands of feeds. In this scenario, folders are necessary to organize these feeds. So, lesson learned: we’ve developed folders for our users.

We have many ideas as to how to make Skimr even better for our users, for example native apps, search (within a user’s RSS feeds or even all the RSS feeds in the system). But it is very important to always think twice before implementing a user request.

Since we are not building a traditional RSS reader, some of our users (with all due respect), don’t quite get this, so they ask for features they are used to on Google Reader. Our path is different from Google Reader, Feedly, etc., so we might not make everyone happy. The benefit of this approach, however, is that the majority of our users are actually very happy we are offering something different. We have found a niche, where people are fed up by the complex readers out there and are happy with the simplicity Skimr offers. They are even sending us emails asking us to keep Skimr as is and not add any new features at all. Amazing!

Going forward, we can expect some spikes in traffic around June, when Google will definitely sunset Reader. We will do our best to prepare for them properly, mainly rewriting the backend to the no-DB stuff.

But, in general, we are more interested in organic growth, rather than overnight success. It helps to create a bond with the users, listen to what they have to say and carefully improve the product. After all, Skimr is a hobby project for us. We all have full time jobs.

Having said that, there are obvious business models we could try, especially Freemium (some special features for a small fee).

So far, the Skimr project has been full of surprises, so who know what comes next?

indexedDB, which is new in HTML5, allows developers to persist data within the web browser. As a result your app runs both online and offline with powerful query facilities. indexedDB is different from traditional relational databases in that it is an objectStore instead of a collection of rows and columns. You just create an objectStore in indexedDB and store JavaScript objects in that store. Furthermore, it’s very easy to perform CRUD operations on your stored objects. This tutorial gives an overview of indexedDB API and explains how to use this to create a simple notepad application.

Before we get started note that indexedDB API specification has not stabilized yet. But if you have the latest Firefox or Google Chrome installed on your machine you are good to go. To know which browser versions support the API see the compatibility table.

There are two types of APIs in indexedDB spec: synchronous and asynchronous. However, we will focus on the asynchronous API as currently this is the only API that’s supported by browsers. Asynchronous means you perform an operation on the database and receive the result in a callback through a DOM event.

In any note-making app there are four simple functions: Create, Read, Update and Delete. indexedDB provides very simple APIs to perform these operations. But before doing anything we need to create a database and open it.

Setting Up:

Since the specification has not stabilized yet, different browsers use prefixes in their implementations. So, you need to check correctly to ensure that a browser supports indexedDB. Use the following code to ensure that a browser supports indexedDB.

window.indexedDB = window.indexedDB || window.mozIndexedDB || window.webkitIndexedDB ||
    window.msIndexedDB;
window.IDBTransaction = window.IDBTransaction || window.webkitIDBTransaction ||
    window.msIDBTransaction;
window.IDBKeyRange = window.IDBKeyRange || window.webkitIDBKeyRange || window.msIDBKeyRange;
if (!window.indexedDB) {
    alert("Sorry!Your browser doesn't support IndexedDB");
}

Opening/Creating a Database:

Opening an indexedDB database is a matter of calling indexedDB.open().

var request = window.indexedDB.open("notepad",1);

indexedDB.open() accepts two parameters. The first one represents the database name. If the database does not already exist it creates a new one. The second parameter is the version number. If you need to update the database schema in future you need to call indexedDB.open()function with a version number higher than the previous one. In that case you will need to implement onupgradeneeded callback where you can update the database schema and create/delete objectStores.

Creating the database is the first step. But to store something you need to have an objectStore. When you create a database, you will probably want to create an objectStore as well. When a database is initially created onupgradeneeded callback is fired where you can create an objectStore.

var database;
var request = window.indexedDB.open("notepad",1);
request.onerror = function(event) {
console.log(event.target.errorCode);
};
request.onsuccess = function(event) {
    database=request.result;
};
request.onupgradeneeded = function(event) {
    var db = event.target.result;
    var objectStore = db.createObjectStore("notes", { keyPath:  "id",autoIncrement:true});
};

In the above code sample we call indexedDB.open() with database name notepad and version number 1. The method returns an  IDBOpenDBRequest. When the database open request succeeds the request.onsuccess callback is fired. The result property of request is an instance of IDBDatabase which we assign to variable database for later use.

Inside onupgradeneeded callback we get a reference to the database and use it to create a new objectStore called notes. createObjectStore() function also accepts a second parameter.  In this case we have defined a keyPath called id that uniquely identifies an object in our store. In addition we also want the id to be autoIncrementing.

Adding/Updating an Item in the objectStore:

Let’s say we want to persist a note in the store. The object should have fields like: title, body and a creation date. To store the object use the following code:

var note={title:”Test Note”, body:”Hello World!”, date:”01/04/2013”};
var transaction = database.transaction(["notes"], "readwrite");
var objectStore = transaction.objectStore("notes");
var request=objectStore.put(note);
request.onsuccess = function(event) {
    //do something here
};

database.transaction() takes an array as first parameter which represents the names of objectStores this transaction spans. The second parameter determines the type of transaction. If you don’t pass a second argument you will get a read only transaction. Since we want to add a new item we pass readwrite as the second argument. As a result of this call we get a transaction object. transaction.objectStore() selects an objectStore to operate on.   Finally objectStore.put() adds the object to the store. You can also use objectStore.add() to add an object. But the former will update an object in the store if we attempt to add a new object with an id the same as the id of an existing entry.

Deleting an Item from the store:

Deleting an object from the store is pretty simple and straightforward.

var request = database.transaction(["notes"], "readwrite") .objectStore("notes").delete(20);
request.onsuccess = function(event) {
    //handle success
};

The above code deletes an object from the store which has an id of 20.

Querying all objects in store:

For any database driven app it’s very common to display all the stored entries.  In indexedDB you can get the objects stored in the store and iterate through them with the help of a cursor.

var objectStore = database.transaction("notes").objectStore("notes");
objectStore.openCursor().onsuccess = function(event) {
    var cursor = event.target.result;
    if (cursor) {
        alert("Note id: "+cursor.key+", Title: "+cursor.value.title);
        cursor.continue();
    }
};

The above code is very simple to understand.  The openCursor() function can accept several arguments. You can control the range of results returned and the direction of iteration by passing appropriate parameters. The cursor object is the result of the request. cursor.continue() should be called if you expect multiple objects and want to iterate through them. This means as long as there is more data onsuccess callback is fired, provided you call cursor.continue().

So, this is all you should know before developing the notepad app using indexedDB.  Now, I will show how to create the app step by step.

Initial HTML markup:

<html>
<head><title>Simple Notepad Using indexedDB</title>
</head>
<body>
<div id="container">
    <h3 id="heading">Add a note</h3>
    <input type="hidden" value="0" id="flag"/>
    <a href="#" id="add"><img src="add.png" onclick="createNote(0)"/> New</a>
    <a href="#" id="back"><img src="back.png" onclick="goBack()"/></a>
    <div id="notes"></div>
    <div id="editor" contenteditable="true"></div>
</div>
</body>
</html>

Explanation:

We have two divs: notes and editor. The first one is used for showing a list of existing notes and the second one is used as an editor to write a note.  The editor div is initially invisible. When the user clicks on the add button we hide the notes div and show the editor div. You should keep in mind that by setting contenteditable="true" we are making a div editable. We also have a hidden input field with id flag. This will be used later in the tutorial.

The JavaScript:

<script type="text/javascript">
var database;
window.indexedDB = window.indexedDB || window.mozIndexedDB || window.webkitIndexedDB ||
    window.msIndexedDB;
window.IDBTransaction = window.IDBTransaction || window.webkitIDBTransaction ||
    window.msIDBTransaction;
window.IDBKeyRange = window.IDBKeyRange || window.webkitIDBKeyRange || window.msIDBKeyRange;
if (!window.indexedDB) {
    alert("Sorry!Your browser doesn't support IndexedDB");
}
function init(){
    var request = window.indexedDB.open("notepad",1);
    request.onerror = function(event) {
    console.log(event.target.errorCode);
};
request.onsuccess = function(event) {
    database=request.result;
    showNotes();
};
request.onupgradeneeded = function(event) {
    var db = event.target.result;
    var objectStore = db.createObjectStore("notes", { keyPath: "id",autoIncrement:true});
    };
}
function createNote(id){
    document.getElementById("editor").style.display="block";
    document.getElementById("editor").focus();
    document.getElementById("back").style.display="block";
    document.getElementById("add").style.display="none";
    document.getElementById("notes").style.display="none";
    if(parseInt(id)!=0){
    database.transaction("notes").objectStore("notes").get(parseInt(id))
    .onsuccess = function(event) {
document.getElementById("editor").innerHTML=event.target.result.body;
    document.getElementById("flag").value=id;
};
}
}
function goBack(){
    var note={};
    note.body=document.getElementById("editor").innerHTML;
    note.title=getTitle(note.body);
    note.date=getDate();
    var flag=parseInt(document.getElementById("flag").value);
    if(flag!=0)
      note.id=flag;
    if(note.title.trim()==="")
        window.location.href="index.html";
    else
        addNote(note);
    }
function getDate(){
    var date=new Date();
    var month=parseInt(date.getMonth())+1;
    return date.getDate()+"/"+month+"/"+date.getFullYear();
}
function getTitle(body){
    var body = body.replace(/(<([^>]+)>)/ig,"");
    if(body.length > 20) body = body.substring(0,20)+". . .";
        return body;
}
function addNote(note){
    var transaction = database.transaction(["notes"], "readwrite");
    var objectStore = transaction.objectStore("notes");
    var request=objectStore.put(note);
    request.onsuccess = function(event) {
        document.getElementById("flag").value="0";
        window.location.href="index.html";
        };
    }
function showNotes(){
var notes="";
var objectStore = database.transaction("notes").objectStore("notes");
objectStore.openCursor().onsuccess = function(event) {
    var cursor = event.target.result;
    if (cursor) {
        var link="<a class=\"notelist\" id=\""+cursor.key+"\" href=\"#\">"+cursor.value.title+"</a>"+"
        <img class=\"delete\" src=\"delete.png\" height=\"30px\" id=\""+cursor.key+"\"/>";
        var listItem="<li>"+link+"</li>";
        notes=notes+listItem;
        cursor.continue();
    }
    else
    {
    document.getElementById("notes").innerHTML="<ul>"+notes+"</ul>";
    registerEdit();
    registerDelete();
    }
};
}
function deleteNote(id){
var request = database.transaction(["notes"], "readwrite")
        .objectStore("notes")
        .delete(id);
request.onsuccess = function(event) {
    window.location.href="index.html";
};
}
function registerEdit(){
var elements = document.getElementsByClassName('notelist');
for(var i = 0, length = elements.length; i < length; i++) {
    elements[i].onclick = function (e) {
        createNote(this.id);
    }
}
}
function registerDelete(){
var deleteButtons = document.getElementsByClassName('delete');
    for(var i = 0, length = deleteButtons.length; i < length; i++){
        deleteButtons[i].onclick=function(e){
        deleteNote(parseInt(this.id));
        }
    }
}
window.addEventListener("DOMContentLoaded", init, false);
</script>

Explanation:

The init method does the necessary initialization. It creates/opens the database and also creates an objectStore when the database is first created.  After the database is successfully opened we get a reference to it and store it in a database variable.

showNotes() function displays a list of notes created by the user. We start a transaction and get the note objects that are present in the store. Then we create an unordered list of the note titles and finally display it in the div having id notes. We also call two functions registerEdit() and registerDelete(). The first function attaches a click event listener to the note titles which are simple links having class notelist so that the notes can be edited when someone clicks on the title. The latter function adds a click event listener to the delete buttons (simple images) that are present beside the note titles. By doing this we can delete a note when someone clicks on delete button. The deleteNote() function performs the delete operation.

The createNote() function displays an editor to create a new note or update an existing one . It accepts one argument. If it’s 0, we know that we want to create a new note. Otherwise we start a transaction to get the content of an existing note. We pass the id of the note to objectStore.get() and in onsuccess we fetch the body of the note. Then we populate the editor with the fetched note body. In addition we set our hidden input flag to the id which is used in goBack() function.  This method is fired when the user wants to go back after writing a note. This is where we save a note in the store.

In the goBack() function we create a new object and set its title, body and date property. The title is taken as the first 20 characters of the body. Then find out the value of hidden flag. If it’s not 0 we know we want to update an existing note. Therefore, we set the id property of the created object. Otherwise there is no need of an id property as the object is going to be a new entry in the store. In the end addNote() function is called with the note object as argument. The addNote() function simply starts a transaction that adds/updates an object in the store. If the transaction succeeds we take the user back where he/she can see a list of created notes.

You can try out the demo app here. I have tested the app in Chrome 25 and Firefox 20.

indexedDB is a great API in HTML5 and when used with app cache can be very powerful. Mozilla has some interesting and very useful information about indexedDB. Do check them out if you want to know more.

If you are not able to get something let me know in the comments.