Ajax and Web Service Data Formats Part 3: Custom Responses

Tweet

This is the last article in a series discussing web service data formats for Ajax. Part 1 examined XML, SOAP and HTML and Part 2 discussed JSON and JSONP. Today, we look at custom data formats.

The most efficient data transmission formats use a minimal number of structural elements to delimit the data. Consider our book data in XML and JSON. We’re always expecting six items of data per book, so we could simply use a format such as:


The Principles of Beautiful Web Design, 2nd Edition;http://www.sitepoint.com/books/design2/;Jason Beaird;SitePoint;39.95;USD
jQuery: Novice to Ninja;http://www.sitepoint.com/books/jquery1/;JEarle Castledine & Craig Sharkie;SitePoint;29.95;USD
Build Your Own Database Driven Website;http://www.sitepoint.com/books/phpmysql4/;Kevin Yank;SitePoint;39.95;USD

Essentially, our data is similar to comma-separated list. We’ve used carriage returns to delimit books and semi-colons to delimit individual data items (commas couldn’t be used because they appear in book titles and possibly numeric values). Choosing the right data delimiters is the most important decision we have to make.

Unlike JSON, we must extract and parse data from the returned string, but this can be achieved quickly and easily using JavaScript’s string split() method. The following code will convert the data above to an identical JSON format:


// convert custom data to an array of JavaScript objects
function ParseBookData(ajaxdata) {
	var book = [], bookData = ajaxdata.split("n"), bookItem;
	for (var b=0, bl=bookData.length; b < bl; b++) {
		bookItem = bookData[b].split(";");
		book[b] = {
			title: bookItem[0],
			url: bookItem[1],
			author: bookItem[2],
			publisher: bookItem[3],
			price: {
				amount: parseFloat(bookItem[4]),
				currency: bookItem[5]
			}
		};

	}
	return book;
}

var book = ParseBookData(xhr.responseText);
alert(book[0].title); // first book title
alert(book[1].url); // second book URL

JavaScript can process the data very quickly — even if thousands of books are returned. In most cases, you’ll find that total time for downloading and parsing the data is less than the equivalent JSON-powered Ajax code.

A custom data format offers several advantages:

  • It’s the most lightweight format and will result in faster Ajax responses.
  • The data can be generated quickly by any server-side language without additional formatting libraries.
  • It’s not easy to create a malicious payload.

But note the disadvantages:

  • This may not be a viable solution if the number of object properties are inconsistent. For example, assume our books had optional PDF URLs or prices in multiple currencies. We may require further delimiters and our parsing will become more difficult.
  • Your web service may be more limited in scope than one which returns XML or JSON. This won’t be an issue if you’re writing services for your own applications but it’s something to consider if you intend publishing the data for third parties.
  • A custom parser must be written for every type of data response.
  • The data is not necessarily human-readable.
  • You must be absolutely certain that delimiter characters do not appear in the original data. You could check for their existence or consider using delimiters toward the start of the ASCII character set.

I hope this series of articles has given you a number of Ajax data formatting options to consider. Have you used any other data formats?

Free JavaScript: Novice to Ninja Sample

Get a free 32-page chapter of JavaScript: Novice to Ninja

  • Anonymous

    A custom data format for every response type is a recipe for disaster. You really want to seal the programming for this with a one type fits all or you’ll have bugs every time you add a new response.

    I use JSON download but there’s no need to make this complex or use a library. A very short routine to join up strings with the appropriate JSON delimiters to support a sub set of the full JSON capability can provide good flexibility such as support for ordered lists in the download but without requiring heavy processing.

    You need to check in your download data for existing delimiters to avoid breakage but I don’t think there’s much risk of an exploit when sending back to the browser?

    I actually use a simple line based format for upload generated by the browser Javascript. I don’t see a need to avoid JSON on download.

    • http://www.optimalworks.net/ Craig Buckler

      It all depends on what you’re trying to achieve. For example, if you’re only using a single Ajax request but it retrieves a lot of data, a custom response might be a good way to tackle the problem.

      However, in most cases, I agree that JSON or XML will probably be the best solution.

  • Tim

    Sometimes the benefit in using xml or json is more about consistency and time to market. Not everyone has to deal with systems where a few bytes here and there make a tangible difference. Using a consistent format (custom or otherwise) across all requests during development can lead to other maintenance benefits like not having bugs in one packet format that aren’t in another. You should always be able to justify why you chose one format over another, even if the answer is simply that you’re lazy and didn’t want to think about it so you just went with an industry standard.

  • Another Tim

    For a long time I used ASCII characters 31 (unit separator) and 30 (record separator) as delimiters for custom download formats. Empty fields just didn’t have anything between the two unit separators, and I was pretty much guaranteed that these two characters weren’t ever going to be in the data being transferred.
    I recently have been moving more towards using JSON, simply because we’re now using Prototype on the client and PHP on the server, and they both have built-in JSON routines. I console myself with the thought that I’m becoming more “standardized”. :)

  • Snapey

    Thanks for the article Craig.
    An advantage of the XML and JSON formats over Custom is the ability to extend the schema of the server response without breaking the client. This could be very important if your ‘web service’ is used by third parties. Otherwise you end up building versioned requests so that the server knows to return the old format and not the new one.
    ps. Could do with a link from Part 2 to part 3.

    • http://www.optimalworks.net/ Craig Buckler

      Thanks Snapey. Absolutely: a custom data format will normally be useful when raw data transmission and processing speed is more essential than extendability.

      A link’s been added to part 2.

  • Michael Benin

    Thank you for the article!

    What brought me here was accessing Facebook’s graph which is in JSONP, I had no problem accessing it with AS3 from urlrequest and as3corelib, but when I tried with JS in an xmlhttprequest with eval() it was null.

    Thank you again !!!