Up Close and Personal with HTML5 IndexedDB
Key Takeaways
- IndexedDB is a client-side storage solution under the HTML5 umbrella that allows significant amounts of structured data to be stored directly in the user’s browser, supporting both online and offline application functionalities.
- The technology supports asynchronous API calls, transactions, and the creation of multiple object stores (akin to database tables), enabling sophisticated data management and manipulation directly on the client-side.
- IndexedDB requires checking for browser support using simple JavaScript checks or libraries like Modernizr, and its use involves opening a database, creating object stores, and then performing CRUD operations within these stores.
- Object stores in IndexedDB hold records in key/value pairs, where keys can be arrays, strings, dates, or numbers, and can be auto-generated or specified using a “key path”.
- Advanced features of IndexedDB include versioning to manage database structure changes over time, and transactions that ensure atomic and durable interactions with the data stored within the browser.
What is IndexedDB?
An IndexedDB is basically a persistent data store in the browser—a database on the client side. Like regular relational databases, it maintains indexes over the records it stores and developers use the IndexedDB JavaScript API to locate records by key or by looking up an index. Each database is scoped by “origin,” i.e. the domain of the site that creates the database. IndexedDB is also a great example of how web standards evolve. Through standards working groups and HTML5 Labs (a site that publishes prototype implementations of various HTML5 specifications so you can try them out and provide feedback), IndexedDB will soon be ready for prime time site use. If you’re new to IndexedDB, start here:- Cookbook demo on IETestDrive
- Developers guide on MSDN
- Spec on W3C
Setting your development environment up
Begin with an install:- Download the prototype by clicking on the link “Download the Protoype now!” from here.
- Unzip the downloaded file.
- If you are running a 32-bit version of Windows then run vcredist_x86.exe.
- Register “sqlcejse40.dll” by running the following command from an elevated command prompt:

Building an offline note-taking app
We are going to build a client side data layer for a note-taking web app: From a data model point of view, it’s about as simple as it can get. The app allows users to write text notes and tag them with specific key words. Each note will have a unique identifier that will serve as its key, and apart from the note text, it will be associated with a collection of tag strings. Here’s a sample note object represented in JavaScript object literal notation:var note = { id: 1, text: "Note text.", tags: ["sample", "test"] };We’ll build a
NotesStore
object that has the following interface:
var NotesStore = { init: function(callback) { }, addNote: function(text, tags, callback) { }, listNotes: function(callback) { } };It should be obvious what each method does. All method calls execute asynchronously (that is, when results are reported via callbacks), and where a result is to be returned to the caller, the interface accepts a reference to a callback that is to be invoked with the result. Let’s see what it takes to efficiently implement this object using an indexed database.
Testing for IndexedDB
The root object that you deal with when talking to the IndexedDB API is called indexedDB. You can check for the presence of this object to see whether the current browser supports IndexedDB or not. Like so:if(window[“indexedDB”] === undefined) { // nope, no IndexedDB! } else { // yep, we’re good to go! }Alternatively, you can use the Modernizr JavaScript library to test for support for IndexedDB like so:
if(Modernizr.indexeddb) { // yep, go indexeddb! } else { // bleh! No joy! }
Asynchronous requests
The asynchronous API calls work through what are known as “request” objects. When an asynchronous API call is made, it would return a reference to a “request” object, which exposes two events—onsuccess
and onerror
.
Here’s what a typical call looks like:
var req = someAsyncCall(); req.onsuccess = function() { // handle success case }; req.onerror = function() { // handle error };As you work with the indexedDB API, it will eventually become hard to keep track of all the callbacks. To make it somewhat simpler, I’ll define and use a small utility routine that abstracts the “request” pattern away:
var Utils = { errorHandler: function(cb) { return function(e) { if(cb) { cb(e); } else { throw e; } }; }, request: function (req, callback, err_callback) { if (callback) { req.onsuccess = function (e) { callback(e); }; } req.onerror = errorHandler(err_callback); } };Now, I can write my async calls like so:
Utils.request(someAsyncCall(), function(e) { // handle completion of call });
Creating and opening the database
Creating/opening a database is done by calling theopen
method of the indexedDB
object.
Here’s an implementation of the NotesStore
object’s init
method:
var NotesStore = { name: “notes-db”, db: null, ver: “1.0”, init: function(callback) { var self = this; callback = callback || function () { }; Utils.request(window.indexedDB.open(“open”, this.name), function(e) { self.db = e.result; callback(); }); }, …The
open
method opens the database if it already exists. It is doesn’t, it will create a new one. You can think of this as the object that represents the connection to the database. When this object is destroyed the connection to the database is terminated.
Now that the database exists, let’s create the rest of the database objects. But first, you’ll have to get acquainted with some important IndexedDB constructs.
Object stores
Object stores are the IndexedDB equivalent of “tables” from the relational database world. All data is stored inside object stores and serves as the primary unit of storage. A database can contain multiple object stores and each store is a collection of records. Each record is a simple key/value pair. Keys must uniquely identify a particular record and can be auto-generated. The records in an object store are automatically sorted in ascending order by keys. And finally, object stores can be created and deleted only under the context of “version change” transactions. (More on that later.)Keys and Values
Each record in the object store is uniquely identified by a “key.” Keys can be arrays, strings, dates, or numbers. For comparison’s sake, arrays are greater than strings, which are greater than dates, which are greater than numbers. Keys can be “in-line” keys or not. By “in-line,” we indicate to IndexedDB that the key for a particular record is actually a part of the value object itself. In our notes store sample, for instance, each note object has anid
property that contains the unique identifier for a particular note. This is an example of an “in-line” key—the key is a part of the value object.
Whenever keys are “in-line,” we must also specify a “key path”—a string that signifies how the key value can be extracted from the value object.
The key path for “notes” objects for instance is the string “id” since the key can be extracted from note instances by accessing the “id” property. But this scheme allows for the key value to be stored at an arbitrary depth in the value object’s member hierarchy. Consider the following sample value object:
var product = { info: { name: “Towel”, type: “Indispensable hitchhiker item”, }, identity: { server: { value: “T01” }, client: { value: “TC01” }, }, price: “Priceless” };Here, the following key path might be used:
identity.client.value
Database versioning
IndexedDB databases have a version string associated with them. This can be used by web applications to determine whether the database on a particular client has the latest structure or not. This is useful when you make changes to your database’s data model and want to propagate those changes to existing clients who are on the previous version of your data model. You can simply change the version number for the new structure and check for it the next time the user runs your app. If needed, upgrade the structure, migrate the data, and change the version number. Version number changes must be performed under the context of a “version change” transaction. Before we get to that, let’s quickly review what “transactions” are.Transactions
Like relational databases, IndexedDB also performs all of its I/O operations under the context of transactions. Transactions are created through connection objects and enable atomic, durable data access and mutation. There are two key attributes for transaction objects:Scope
The scope determines which parts of the database can be affected through the transaction. This basically helps the IndexedDB implementation determine what kind of isolation level to apply during the lifetime of the transaction. Think of the scope as simply a list of tables (known as “object stores”) that will form a part of the transaction.Mode
The transaction mode determines what kind of I/O operation is permitted in the transaction. The mode can be:Read only
Allows only “read” operations on the objects that are a part of the transaction’s scope.Read/write
Allows “read” and “write” operations on the objects that are a part of the transaction’s scope.Version change
The “version change” mode allows “read” and “write” operations and also allows the creation and deletion of object stores and indexes.
- when they complete
- when they abort and
- when they timeout
Creating the object store
Our notes store database will contain only a single object store to record the list of notes. As discussed earlier, object stores must be created under the context of a “version change” transaction. Let’s go ahead and extend theinit
method of the NotesStore
object to include the creation of the object store. I’ve highlighted the changed bits in bold.
var NotesStore = { name: “notes-db”, store_name: “notes-store”, store_key_path: “id”, db: null, ver: “1.0”, init: function (callback) { var self = this; callback = callback || function () { }; Utils.request(window.indexedDB.open(“open”, this.name), function (e) { self.db = e.result; // if the version of this db is not equal to // self.version then change the version if (self.db.version !== self.version) { Utils.request(self.db.setVersion(self.ver), function (e2) { var txn = e2.result; // create object store self.db.createObjectStore(self.store_name, self.store_key_path, true); txn.commit(); callback(); }); } else { callback(); } }); }, …Object stores are created by calling the
createObjectStore
method on the database object. The first parameter is the name of the object store. This is followed by the string identifying the key path, and finally a Boolean flag indicating whether the key value should be auto-generated by the database when new records are added.
Adding data to object stores
New records can be added to an object store by calling theput
method on the object store. A reference to the object store instance can be retrieved through the transaction object. Let’s implement the addNote
method of our NotesStore
object and see how we can go about adding a new record:
… addNote: function (text, tags, callback) { var self = this; callback = callback || function () { }; var txn = self.db.transaction(null, TransactionMode.ReadWrite); var store = txn.objectStore(self.store_name); Utils.request(store.put({ text: text, tags: tags }), function (e) { txn.commit(); callback(); }); }, …This method can be broken down into the following steps:
- Invoke the
transaction
method on the database object to start off a new transaction. The first parameter is the names of the object stores that are going to be a part of the transaction. Passingnull
causes all the object stores in the database to be a part of the scope. The second parameter indicates the transaction mode. This is basically a numeric constant which we have declared like so:// IndexedDB transaction mode constants var TransactionMode = { ReadWrite: 0, ReadOnly: 1, VersionChange: 2 };
- Once the transaction has been created we acquire a reference to the object store in question through the transaction object’s
objectStore
method. - Once we have the object store handy, adding a new record is just a matter of issuing an asynchronous API call to the object store’s
put
method passing in the new object to be added to the store. Note that we do not pass a value for theid
field of the new note object. Since we passedtrue
for the auto-generate parameter while creating the object store, the IndexedDB implementation should take care of automatically assigning a unique identifier for the new record. - Once the asynchronous
put
call completes successfully, we commit the transaction.
Running queries with cursors
The IndexedDB way of enumerating records from an object store is to use a “cursor” object. A cursor can iterate over records from an underlying object store or an index. A cursor has the following key properties:- A range of records in either an index or an object store.
- A source that references the index or object store that the cursor is iterating over.
- A position indicating the current position of the cursor in the given range of records.
listNotes
method of our NotesStore
object and see what the code looks like.
listNotes: function (callback) { var self = this, txn = self.db.transaction(null, TransactionMode.ReadOnly), notes = [], store = txn.objectStore(self.store_name); Utils.request(store.openCursor(), function (e) { var cursor = e.result, iterate = function () { Utils.request(cursor.move(), function (e2) { // if "result" is true then we have data else // we have reached end of line if (e2.result) { notes.push(cursor.value); // recursively get next record iterate(); } else { // we are done retrieving rows; invoke callback txn.commit(); callback(notes); } }); }; // set the ball rolling by calling iterate for the first row iterate(); }); },Let’s break this implementation down:
- First, we acquire a transaction object by calling the database object’s
transaction
method. Note that this time we’re indicating that we require a “read-only” transaction. - Next we retrieve a reference to the object store via the
objectStore
method of the transaction object. - Then we issue an async call to the openCursor API on the object store. The tricky part here is that every single iteration over a record in the cursor is itself an async operation! To prevent the code from drowning in a sea of callbacks, we define a local function called
iterate
to encapsulate the logic of iterating over every record in the cursor. - This
iterate
function makes an async call to the cursor object’smove
method and recursively invokes itself again in the callback if it detects that there are more rows to be retrieved. Once all the rows in the cursor have been retrieved we finally invoke the callback method passed by the caller handing in the retrieved data as a parameter.
Dive even deeper!
This is, by no means, comprehensive coverage of the API, despite what you may think! I only covered:- Available options for implementing client-side storage today
- The various key aspects of the IndexedDB API, including:
- Testing whether the browser supports it
- Managing asynchronous API calls
- Creating/opening databases
- Key parts of the API including object stores, keys/values, versioning, and transactions
- Creating object stores
- Adding records to object stores
- Enumerating object stores using cursors
Frequently Asked Questions about HTML5 IndexedDB
What is the primary function of HTML5 IndexedDB?
HTML5 IndexedDB is a web standard that provides a robust database system for storing significant amounts of structured data in the user’s browser. It is designed to work well with online and offline applications, allowing for more complex client-side data manipulation. It is particularly useful for applications that require storing large amounts of data, such as games, or for applications that need to store data in a structured manner.
How does HTML5 IndexedDB differ from other client-side storage options?
Unlike other client-side storage options like LocalStorage and SessionStorage, IndexedDB allows for the storage of more significant amounts of data. It also supports transactions, which ensures data integrity even if a system fails. Moreover, IndexedDB provides a more advanced API that supports indexed and cursor-based access to stored data, unlike other storage options that only allow key-value pair storage.
How can I start using HTML5 IndexedDB in my web applications?
To start using IndexedDB, you first need to open a database using the indexedDB.open
method. This method takes two parameters: the name of the database and the version number. Once the database is open, you can create object stores, which are like tables in a relational database. You can then start adding, retrieving, updating, and deleting data in these object stores.
Can I use HTML5 IndexedDB with other web technologies?
Yes, IndexedDB works well with other web technologies. For instance, you can use it with Service Workers for offline functionality in your web applications. You can also use it with Web Workers to perform database operations in the background without blocking the user interface.
What are the limitations of HTML5 IndexedDB?
While IndexedDB is powerful, it also has some limitations. For instance, it does not support SQL or other query languages, which can make complex queries challenging. It also has a more complex API compared to other client-side storage options, which can make it harder to learn and use.
How secure is HTML5 IndexedDB?
IndexedDB follows the same-origin policy, meaning that web applications can only access databases that have the same origin. This ensures that malicious websites cannot access your data. However, it’s important to note that IndexedDB does not encrypt data by default, so sensitive data should be encrypted before storing.
Can I use HTML5 IndexedDB on all browsers?
Most modern browsers support IndexedDB, including Chrome, Firefox, Safari, and Edge. However, support may vary between different versions of these browsers, so it’s always a good idea to check the specific browser support before using IndexedDB.
How can I handle errors in HTML5 IndexedDB?
IndexedDB uses events to handle errors. For instance, you can listen for the error
event on a transaction to handle any errors that occur during the transaction. You can also use the onerror
handler on the database request to handle any errors that occur when opening the database.
How can I delete an HTML5 IndexedDB database?
You can delete an IndexedDB database using the indexedDB.deleteDatabase
method. This method takes one parameter: the name of the database you want to delete. It’s important to note that this operation is asynchronous and returns a request object that you can use to listen for success or error events.
Can I use HTML5 IndexedDB in a mobile web application?
Yes, you can use IndexedDB in a mobile web application. However, support for IndexedDB on mobile browsers may vary, so it’s important to check the specific browser support before using IndexedDB in a mobile web application.
Raj has been a part of the IT Industry for more than 9 years and works as a Developer Evangelist at Microsoft. He has worked on a wide range of Microsoft products & technologies ranging from VC++, MFC and Win32 programming to the latest developments on the web front, JavaScript and .NET. His current focus is on advances in web development, HTML5 and building rich internet applications.