HTML5 Browser Storage: the Past, Present and Future

I have slightly selfish reasons for writing this article; I can never remember all the client-side storage mechanisms available to HTML5 developers! I’ve possibly forgotten some now…

Why Store Data on the Client?

The main reason is practicality. JavaScript code running on the browser does not necessarily need to send all information to the server. There are several use cases:

  1. You want to increase performance. You can cache data client-side so it can be retrieved without additional server requests.
  2. You have a significant quantity of client-side-only data, e.g. HTML strings or widget configuration settings.
  3. You want you make your application work off-line.

Let’s browse the options.

JavaScript Variables (Past, Present and Future)

The simplest choice is JavaScript variables. It may be practical to create a single global variable to store application data, e.g.

// global application storage
var appDataStore = {};

// set values
appDataStore.hello = "Hello World!";
appDataStore.info = { a:1, b:2, c:3 };

// retrieve values
console.log(appDataStore.hello);
console.log(appDataStore.info.b);

It’s also possible to store values in the page DOM as node attributes or properties. This can be useful for widget-specific values, but it’s slower and riskier than JavaScript variables; future browsers or other libraries may interpret your data in unexpected ways.

// store data in node
var mywidget = document.getElementById("mywidget");
mywidget.setAttribute("data-myvalue", "Hello World!");

// retrieve values
mywidget.textContent = mywidget.getAttribute("data-myvalue");

The advantages of JavaScript variables:

  • the fastest and simplest solution
  • no need to serialize or de-serialize data
  • ideal for single-page applications

The disadvantages:

  • very fragile — linking elsewhere, refreshing or closing the tab will wipe all data
  • global variables can be overwritten and analyzed by third-party scripts.

Cookies (Past, Present and Future)

Cookies are domain-specific chunks of text data. They sound tasty, but cookie handling is awkward in JavaScript since the basic document.cookie string must be parsed, e.g.

// store cookie data
document.cookie = "hello=" + escape("Hello World") + "; path=/";

// retrieve value
cookies =
	document.cookie.split(";"),
	value = null,
	regexp = new RegExp("^\s*hello=(.*)$");
for (var c = 0; c < cookies.length && !value; c++) {
	var match = cookies1.match(regexp);
	if (match.length == 2) value = unescape(match[1]);
}

console.log(value);

The advantages of cookies:

  • a reliable method of retaining state between the client and server
  • by setting an expiry date, cookie data will persist beyond page refreshes and tab closing
  • cookies are supported in all modern browsers

The disadvantages:

  • clunky JavaScript implementation — you will need a cookie-handling library
  • values are strings only — other data must be serialized using methods such as JSON.stringify and JSON.parse
  • cookie storage space is limited — do not depend on having more than 20 cookies of 4KB each
  • cookies can be deleted or blocked
  • cookies were unfairly labeled as a threat to internet privacy; you may need to comply with bizarre regional rules and regulations.

The flip-side of cookie client/server sharing causes the biggest technical issue. Cookie data is sent in the HTTP header of every request and response. It’s therefore appended to every HTML page, image, CSS file, JavaScript file, Ajax call, etc. If you had 50Kb of cookie data and downloaded ten 1KB images, it would result in one megabyte of additional network traffic.

window.name (Past and Present)

The window.name property is a little odd. You can set a single string value which persists between browser refreshes or linking elsewhere and clicking back, e.g.

window.name = "Hello World!";
console.log(window.name);

The advantages of window.name:

  • simple to use
  • data is retained on the client only and never sent to the server
  • the property permits several megabytes of information
  • wide browser support

The disadvantages:

  • data is lost when the tab or browser is closed
  • only a single string value can be stored — serialization will be necessary
  • pages in other domains can read or change window.name data — never use it for sensitive information

window.name was never designed for data storage. It’s a hack and vendors could drop support at any time. For that reason, it’s best to use alternative storage options although the technique has been adopted within legacy browser shims and polyfills.

See also: How to Write a Cookie-less Session Library for JavaScript

HTML5 Web SQL Database (Past)

The Web SQL Database was an initial attempt by vendors to bring SQL-based relational databases to the browser. It has been implemented in Chrome, Safari and Opera 15+, but was opposed by Mozilla and Microsoft in favor of IndexedDB.

The advantages of Web SQL Database:

  • designed for robust client-side data storage and access
  • it uses SQL like many server side applications
  • some support on Webkit/Blink desktop and mobile browsers

The disadvantages:

  • SQL never seemed appropriate for client-side development
  • the database schema must be defined up-front
  • marginal browser support and the Webkit/Blink teams may eventually drop it
  • the W3C specification was abandoned in 2010

In summary: don’t use a Web SQL Database!

See also: W3C Web SQL Database Specification

HTML5 Web Storage (Present and Future)

Web Storage provides two objects with identical APIs: window.localStorage to retain persistent data and code.sessionStorage to retain session-only data which is lost when the tab is closed. Domain-specific strings are stored using name/value pairs. Unlike cookies, the storage limit is far larger (at least 5MB) and information is never transferred to the server.

// is localStorage available?
if (typeof window.localStorage != "undefined") {

	// store
	localStorage.setItem("hello", "Hello World!");

	// retrieve
	console.log(localStorage.getItem("hello"));

	// delete
	localStorage.removeItem("hello");
}

The advantages of Web Storage:

  • easy to use with simple name/value pairs
  • session and persistent storage options are available
  • an event model is available to keep other tabs and windows synchronized
  • wide support on desktop and mobile browsers including IE8+
  • Web Storage polyfills are available for older browsers which fall-back to cookie and windows.name storage methods

The disadvantages:

  • string values only — serialization may be necessary
  • unstructured data with no transactions, indexing or searching facilties
  • may exhibit poor performance on large datasets

See also:

HTML5 IndexedDB (Future)

IndexedDB provides a structured, transactional, high-performance NoSQL-like data store with a synchronous and asynchronous API. There’s too much to document here, but the API permits you to create databases, data stores and indexes, handle revisions, populate data using transactions, run non-blocking queries, and traverse data sets using cursors.

The advantages of IndexedDB:

  • designed for robust client-side data storage and access
  • good support in modern desktop browsers: IE10+, Firefox 23+, Chrome 28+ and Opera 16+

The disadvantages:

  • the API is very new and subject to revision
  • little support in older and mobile browsers
  • a large and complex API — it would be difficult and largely impractical to create a IndexedDB polyfill
  • like any NoSQL store, data is unstructured which can lead to integrity issues

See also:

HTML5 File API (Future)

The HTML5 File API is being extended to support writing as well as reading sequential data on the local file system. Your domain is provided with a complete sand-boxed hierarchical file system to use as it chooses.

The advantages of the HTML5 File API:

  • large text and binary files can be created and stored
  • performance should be good

The disadvantages:

  • a very early specification which is subject to revision
  • an obvious security risk unless file writing is restricted
  • little support in current browsers and polyfills may be impractical
  • unstructured data with no transactions, indexing or searching facilties

See also:

Summary

There we have it — no less than seven storage alternatives. My pragmatic recommendations at this point in time:

  1. JavaScript variables will always be necessary but only use them for volatile in-memory data. Avoid DOM attributes/properties when possible.
  2. Assuming it’s necessary, use cookies for retaining state between the client and server. But keep cookies small; a single token with a few characters should be enough.
  3. Use HTML5 Web Storage to store data off-line or when large volumes of client-side-only information is required.
  4. If you have to support older browsers such as IE6 and IE7, use a Web Storage polyfill which implements the API using cookies and window.name.
  5. Keep an eye on IndexedDB; it will eventually become a viable storage mechanism when legacy browsers die out.
  6. Similarly, the File API will become increasingly practical for storing unstructured data. It’s possibly the best solution for generated binary data such as images, audio, video and PDFs.

Or perhaps you have further thoughts?

Comments on this article are closed. Have a question about HTML5? Why not ask it on our forums?

Win an Annual Membership to Learnable,

SitePoint's Learning Platform

  • brothercake

    n.b. the EU privacy laws apply to all forms of client-side data storage, not just cookies.

    But the whole thing is ridiculous and unenforceable. I mean — if a client says they don’t want you to store data, how do you remember that? You have to keep asking them on every page, until eventually they’ll agree just to get rid of the notification!

    I’ve just been ignoring it.

    • Anonymous

      I think the law only applies when tracking is employed. If you created a to-do list which didn’t require a log-on and stored data on the client, there wouldn’t be any issues.

      Not that there are any issues … ignoring the daft rules seems to be the best policy.

  • Anonymous

    Useful article :-)
    A missing disadvantage with web storages:
    – no transaction support, so conflicts may occur when accessed from workers or multiple tabs/windows/frames

    Other used storage have been, and are also by some polyfills/storage libs (like lawnchair):
    – Google Gears SQLite
    – Adobe/Macromedia Flash Storage (http://www.macromedia.com/support/documentation/en/flashplayer/help/help02.html)
    – Internet Explorer User Data

    Not really used by polyfills, but still usable in Internet Explorer: ActiveX let’s use connectors to any SQL database (http://www.slideshare.net/alexandre_morgaut/nosql-and-javascript-a-love-story/11)

    Less known solutions include:
    – Java applet storage (http://ajaxian.com/archives/java-applet-storage-for-javascript-pages)
    – Silverlight Application Storage (http://www.microsoft.com/GetSilverlight/resources/documentation/AppStorage.aspx)

    • Anonymous

      All good points, but I don’t really think I could have included Gears, Flash, Silverlight or ActiveX options in an HTML5 storage article.

  • Anonymous

    ” no transaction support, so conflicts may occur when accessed from workers or multiple tabs/windows/frames”
    As just an FYI, there _is_ an event fired when DOMStorage changes in another tab. You could – in theory – handle this. I haven’t tried it in webworkers though.

    • Anonymous

      I’m aware of the “storage” event which is useful to synchronize data between windows / frames / workers from a central point, without the need to post messages to each of them (including some you might not be aware of)

      This miss of transaction is still an issue (and is mentionned in the specification)

    • Anonymous

      From the Web Storage specification:

      Because of the use of the storage mutex, multiple browsing contexts will be able to access the local storage areas simultaneously in such a manner that scripts cannot detect any concurrent script execution.

      Thus, the length attribute of a Storage object, and the value of the various properties of that object, cannot change while a script is executing, other than in a way that is predictable by the script itself.

      http://www.w3.org/TR/webstorage/#threads

      If in a script you get an item do some stuff then store it back once modified, the storage event will have hard time to interrupt this process if the same script is doing the same thing at the same time from another thread. Each step of the process may need to be executed independently from a setTimeout to let the storage event handle the fact the value changed, potentially requiring the process to restart

  • Anonymous

    My previous comment hasn’t been shown yet, but, it looks like you can’t access LocalStorage from Workers, so, not an issue.

  • Anonymous

    What about AppCache? It lets you store just about anything and gives your website app-like performance speed.

    • Anonymous

      AppCache is nice, and actually store html, css, js, image and potentially json files
      The performance speed should not in theory be much better than when you correctly use standard HTTP cache settings (Expires, last-modified, ETag), but is definitely the solution to go for offline usage of web apps

      I think that it was not mentioned because the JS API can’t be use to manually read or override the content of the cached files, all you can do is check (or be called) when an update is available and refresh the whole page…

  • Michael

    “SQL never seemed appropriate for client-side development”

    Says who?

    “the database schema must be defined up-front”

    So it’s bad to know what the schema is before you start writing an application?

    IndexDB looks great, HTML5 Web Storage is too much work for most use cases that I’ve seen (of course there are differing opinions).

    I for one will really miss WebSQL!

    • Anonymous

      Says me! JavaScript is slightly different to other languages because it runs in the browser. There’s no “installation” as such, so defining a DB schema is little unusual.

      The advantages of NoSQL over SQL is the flip-side of SQL over NoSQL. NoSQL is flexible, but you’re not defining rigid data structures so storage can become less efficient and messy.

      • Anonymous

        Installation of Web Apps actually exists for Chrome Web apps and Firefox OS apps. We may even extend it to Microsoft Windows / Windows Phone Apps developed in HTML5.

        I discussed this week-end with someone from Neo4J and we talked about NoSQL as it defined in a second time itself (Not only SQL). A interesting NoSQL approach is the multi-model one and with or without schema, relationships are useful either for performances and code boilerplate size, even for Web Applications ;-)

  • Simon

    Yes, it is not the best idea, but in addition, you can use hidden fields for client-side storing data.

    • Anonymous

      Very true. That’s very similar to using the DOM, but certainly safer.

  • Matthew Baker

    IE7 and I believe IE6 support localStorage… but in a non-standard way of course.

    Hopefuly the polyfills make use of the proprietary implementation rather than using window.name as you suggest(?)

    • brothercake

      No they don’t — neither of those browsers support localStorage.

      But it does work in IE8, and still works when IE8 is set to IE7 emulation mode, so this can give the false impression that it works in IE7. But it doesn’t actually work in a real version of IE7.

  • Anonymous

    One mistake here: HTML5 File API doesn’t not permit saving to the user filesystem. You’ve probably confused it with FileSystem API, which is only implemented by Chrome.

    • Anonymous

      Yes the FileSystem API is not yet really accepted by other implementors. Still, while Mozilla were apparently trying to promote of IndexedDB instead, it looks like they are finally more interested in FileSystem, with some proposals to make it better. Note that this API has been implemented on server-side in Wakanda (as sessionStorage).

  • matbrants

    Installation of Web Apps actually exists for Chrome Web apps and Firefox OS apps. We may even extend it to Microsoft Windows / Windows Phone Apps developed in HTML5.

    • Anonymous

      I actually wanted to mention it, thanks matbrants :-)
      And this will also be true for Firefox OS Apps
      A recent Working Group at W3C is called “System Applications” and work on the case of packaged Web Apps (which may include new specific APIs like one for low level sockets)

  • jhuesos

    IndexedDB specifications are now in candidate recommendation, which means, I don’t think there will be API changes from now on. Furthermore, in desktop its support it’s pretty good: IE10+, Firefox and Chrome supports it.

    I have never used it (I never needed it), but it looks to me that is in a pretty stable state right now.

  • Anonymous

    IE 6 & 7 can use #userData# which gets around the extra weight of unnecessary cookie transmission

  • Tobias Buschor

    There are already polyfills for IndexedDB, so browser support is not wo bad.
    http://caniuse.com/#search=IndexedDB

    • Anonymous

      There is one but it’s only for (Webkit) browsers which support Web SQL. Since those browsers now natively support IndexedDB, it’s become redundant.

  • Anonymous

    Excuse me? There are browsers out now (like mobile safari) which support WebSQL but not IndexedDB.

  • Tobias Buschor

    It targets safari 7 (mobile) and all androids
    http://caniuse.com/sql-storage
    http://caniuse.com/indexeddb

  • Anonymous
  • Andre

    Great article! :)
    By the way, there is a polyfill for IndexedDB and it’s quite good: https://github.com/axemclion/IndexedDBShim

  • Anonymous

    I’m dubious about claims that SQL is not ‘appropriate’ for client-side development. Some people like SQL, some people don’t — but a lot of devs who know about data storage know how to use it. Certainly more than know their way around IndexedDB.

    Moving between JavaScript objects and WebSQL storage can feel a bit clunky, but it’s straightforward. WebSQL is also by far the easiest option for text search: for example, find me all the books in my database that contain the word ‘web’ in the title.

    You also state that WebSQL support is ‘marginal’, whereas in fact it’s been supported for some time by Chrome (desktop and Android), Android Browser, Safari (iOS and desktop), Opera and Blackberry: http://caniuse.com/#search=websql. (Neither iOS/desktop Safari or Android Browser support IndexedDB.)

    • Anonymous

      I don’t think anyone’s said WebSQL can’t or shouldn’t be used. But — as you said — it feels a little clunky; it’s not an ideal fit.

      Yes, WebSQL is supported by the webkit/blink browsers. But, on average, that leaves a good 50% of browsers without it. In essence, it’s become a proprietary technology and has been abandoned as a W3C standard. Use WebSQL if you must, but that’s a risk I would be unwilling to take. There’s certainly no guarantee Chrome will continue to support it.

    • Anonymous

      There are wrappers that exist as a fallback until IndexDB is fully supported. I wish they would just leave WebSQL in. It’s better than nothing. And instead of deprecating it and pushing everything back another few years they could have said and could still say, use WebSQL until IndexDB is fully supported. It’s a pain to get support and then be told not to use it since it takes so long for adoption.