Offline First: Your Next Progressive Enhancement Technique?

There are three primary reasons your client may demand a native phone app rather than a web app:

  1. Native apps are faster. This certainly matters if you’re creating the next Angry Birds, but few applications need game-like responsiveness. (That said, with a little care it is possible to create a fast action game using HTML5 technologies. Whether it would work well on a range of devices is another matter).
  2. The client doesn’t know any better: “Apps are cool! All our competitors have apps — we need one.” A little education solves that issue.
  3. Mobile apps work offline. But so can web apps — it’s just that the technologies are relatively new and few of us bother. Yet.

Adopting the AppCache to make a web application work offline has been possible for several years. The process defines which files should be cached so the browser can execute the application when Internet connectivity drops. It’s reasonably straight-forward, but:

  • Web developers shudder at the thought of connection failure. I’m writing this article on the train and it feels like I’ve lost several major organs. While connectivity is improving, it’s still an issue for commuters and many millions of people who live in remote locations and developing countries.
  • Adding offline capabilities to an existing app is difficult. You need to rework Ajax calls and network requests then consider connectivity status changes. But what if we considered it at the start?

Mobile-first is recognized as a good-practice technique. You start with a simple — perhaps linear — view of your site which works on all browsers regardless of age or device. More modern browsers then use media queries to apply styling enhancements and present a more typical desktop view on larger screens. In other words, the layout is progressively enhanced for better browsers using bigger displays.

Can a similar methodology be used for offline applications? The app would presume it was in offline mode and act accordingly. When connectivity resumed, the app would be progressively enhanced to retrieve additional data or save to the cloud.

Offline-first is a concept which has been discussed by several others although it’s not widely used. There are a few general concepts which make it viable.

1. Remove Server Reliance

The majority of application logic must be moved from the server to the client. The server would, in essence, become a lightweight data storage repository but — importantly — the client application should work regardless of the connection status.

2. Create a Client-Side Data Proxy

You cannot depend on Ajax calls. A data proxy would manage all routing, e.g.

  1. If a connection is available, an Ajax call to the live server can be made (assuming it’s necessary).
  2. If a connection is not available — or fails during an Ajax call — localStorage, IndexDB or another appropriate client-side storage mechanism is used.

Note that HTML5 Service Workers route all data requests through a named JavaScript file. While there is little browser support or standardization today, the technology is specifically designed for this purpose.

3. Synchronize When Possible

You will require a process to handle synchronization between the client and server when connectivity returns. This could be made efficient using Web Worker background processes and batch uploading/downloading during idle periods.

4. Consider Device Usage Factors

Mobile devices have further complications. For example:

  • The act of switching to another app could close the browser. Ideally, your web app should always save application state so the user can return to the place they left.
  • The Page Visibility API could be used reduce processing and bandwidth requirements when your app is not running within the opened browser tab.
  • Ideally, your app should use the Battery Status API to behave nicely. For example, it could revert to localStorage when battery levels drop below critical levels — even if a connection is available.

5. Test. Then Test Again.

Testing is difficult since your app needs to remain operable regardless of connectivity, e.g.

  • The app is installed on a device which doesn’t support localStorage or another essential technology.
  • Connectivity drops and restarts at random intervals.
  • The app is cached but connectivity drops before the first server communication can be made.
  • The app is run on two or more devices at the same time.

Rigorous testing on a range of devices appears to be the only option.

Not every application can take the offline-first plunge; a multi-player action game would be fairly pointless. However, the technique could be used by many web applications assuming it’s considered at the start. I like it, but I suspect the overheads of implementation in existing systems could make separate mobile apps look more cost-effective!

Have you considered offline-first? Are you doing it already? Were there any other complications? Or is it too much effort for too little benefit?

Free book: Jump Start HTML5 Basics

Grab a free copy of one our latest ebooks! Packed with hints and tips on HTML5's most powerful new features.

  • Matt Forrester

    I absolutely agree with this and thus have been working on a solution to this problem for roughly a year. What I have published on GitHub is basically a client side version control system so you can, through an easy well documented API see what is stored on the server and what still needs to be uploaded. It can also deal with conflict resolution too. The server side is in the process of getting support for MongoDB and I am also working on a piece of code to detect your connection status and sync when you are online. The code is located at https://github.com/forbesmyester/SyncIt

    • Anonymous

      Thanks Matt. That looks very comprehensive and impressive. I was hoping synchronization would be a simpler process, but you’ve devised a very robust approach.

  • Anonymous

    Zumero is a great possibility for syncing database from local to remote via an extension to SQLite. Created by makers of Vault … http://zumero.com/. I haven’t tried it yet, but would do as soon as I require such syncing

  • Vedran

    a superb point – i just shudder at the additional level of complexity…but when you think about it as an end user – an app that has this vs. an app that does not…the choice is simple and only one app wins.

    • Anonymous

      It could be more complex but, like most progressive enhancement techniques, it’s more of a development approach rather than additional effort.

  • Anonymous

    Hmmm… If you’re writing a bunch of logic into the client side instead of the server side and basically making the web client be able to be autonomous, aren’t you effectively writing an app, but doing it in a really complex way? I’d have thought that, at such a point, you might as well bite the bullet and write the app instead…

  • Anonymous

    I tried to use this approach a few years ago but the app cache was a bit flakey. An animation web app that I am building now uses local storage from the start but I use a lot of libraries, fabric.js, angularjs and bootstrap and wonder if this is going to be impacted by the iOS 25k cache limit.

    • Anonymous

      You are, but the key is that it’s a single-page web app. Many developers are doing that anyway. You still have the major advantage that it should/could work on multiple devices and you don’t need any app store approval shenanigans.

      Obviously, it still depends on what you want to achieve, but I suspect the process would be applicable to many applications.

  • Anonymous

    Great points and ones that I will save for future reference.

    I just did my first offline app and it works great. When I get time I will add the automatic stuff which is much more complicated, but then I guess it is just more of a learning process which is good. Right now it is a package delivery system designed to be used when delivering a package to a resident’s unit and get a signature on receipt. The process of send back to the server iss manual.

  • Jasdeep Khalsa

    I think one of the problems is which offline data storage is the best to use and if there is a library to help aid in this, and how then this offline data actually syncs with an online database. At the moment this is such a chore to develop. Does anyone have any repos that already achieve this?

    Also authentication does become a problem, where you want to login to a service, you kind of need this logic to only exist server-side so it reduces the use-cases of where offline first may be applicable. I’m sure though, the future is that the server-side is becoming more of just an api made available to a complex client-side responsive and offline-enabled application.

    • Matt Forrester

      I have connection monitoring working in SyncIt (https://github.com/forbesmyester/SyncIt), it has the client / server components done already.

      • Jasdeep Khalsa

        I took a look at your library – I think you’ve done really well! You’re covering all the main use cases that I would think a person developing a mobile app would be concerned about. Do you know how many browsers and devices your repo works for (cross-browser and cross-device compatibility?). That would be awesome to know. Also, is there any mechanism in place to fallback to the best possible local database using some kind of feature detection? For example, for <=IE7 or apps sending data larger than around 2MB LocalStorage cannot be used, but perhaps IndexedDB can or perhaps I may want to use a shim to get localStorage working across all browsers. Or if I’m in Chrome, I may want to use Web SQL. Is this level of flexibility possible within your API for SyncIt?

        • Matt Forrester

          One of SyncIt’s main aims was to be super portable so it’s wrote using very standard JS methods etc. The master branch is actively tested in Chrome/Firefox (with Dojo, jQuery and as pure JS) and Node.JS using Mocha, other branches will get the same treatment before integration. I __did__ test in IE and it does work, though it’s the poor cousin to be honest… I have to reboot to test! Having said that I will actively fix bugs as reported and will do the automated tests when releasing new “master” versions.

          As for support/fallback for things other than LocalStorage. If you look at the constructor for SyncIt you will see that it’s first parameter is a PathStore and the only current implementation for this is in Path/AsyncLocalStorage.js… Why is it AsyncLocalStorage when LocalStorage is not asynchronous at all, well it’s purely so completely different backends can be used, I was specifically thinking about IndexedDB but I believe anything will work.

    • Anonymous

      Authentication is a tricky issue. You don’t want to authenticate or store credentials on the client – it’s too insecure. You could implement some type of ongoing session which continues when offline and verifies on connection, but that adds its own layer of complexity. One to think about and discuss…

  • Anonymous

    Jasdeep, database syncing can be done with the excellent http://www.zumero.com. It’s an extension to SQLite so should work on all platforms and is written by SourceGear (makers of Vault Source Control)

    • Jasdeep Khalsa

      No offence, but I prefer a fully open source stack :)

  • Anonymous

    I think this library could be of use: http://github.hubspot.com/offline/docs/welcome/
    It “caches” AJAX requests when you are offline and calls them when the connection is restored.

    Unfortunately, it doesn’t appear to “activate” the callback associated with AJAX calls.

  • Erik Isaksen

    Great Article. Offline first is becoming so necessary for many projects out there. This was a gem to read. Thank you for posting.