Let's Talk: Efficient Communication for PHP and Android, Part 2

Part 1 of this series focused on getting the Android application set up to make an HTTP request. In part 2, we’ll focus on implementing the use of data serialization and compression on the Android and PHP sides of the request.

Determining the Data Serialization Format

Before data can be transferred to the client, it must be serialized into a data stream that the client can later convert back into a useful data structure. In your PHP REST service, you’ll need to determine the most appropriate data serialization format to use. This involves parsing the Accept header sent by the client to determine what server-supported format it prefers.

If you’re using a PHP framework to build your REST service, you should consult its documentation on how to do this, but here’s what the body of a method to determine the response format might look like:

<?php
// Define types supported by the server
$supportedTypes = array();
if (extension_loaded("json")) {
    $supportedTypes["application/json"] = "json_encode";
}
if (extension_loaded("msgpack")) {
    $supportedTypes["application/x-msgpack"] = "msgpack_pack";
}

// Get client-supported media types ordered from most to least preferred 
$typeEntries = array_map("trim", explode(",", $_SERVER["HTTP_ACCEPT"])); 
$acceptedTypes = array(); 
foreach ($typeEntries as $entry) { 
    $entry = preg_split('#;s*q=#', $entry); 
    $mediaType = array_shift($entry); 
    $qualityValue = count($entry) ? array_shift($entry) : 1; 
    $acceptedTypes[$mediaType] = $qualityValue; 
}
arsort($acceptedTypes); 

// Find the most preferred media type supported by client and server 
$supportedMediaTypes = array_keys($supportedTypes);
foreach ($acceptedTypes as $mediaType => $qualityValue) { 
    $pattern = "#" . str_replace("*", ".*", $mediaType) . "#"; 
    if ($matches = preg_grep($pattern, $supportedMediaTypes)) { 
        $supportedType = reset($matches); 
        return array($supportedType, $supportedTypes[$supportedType]);
    }
}
return null;

This code parses the value of the Accept header, sorts the MIME types it contains by their respective quality factors, and then loops through them in that order to determine which type is supported by the server and has the highest quality factor.

You can make use of this value in your response as in the example below where $contentType is the return value of your method. If no mutually supported format can be found, an appropriate response code is returned per RFC 2616 Section 10.4.7.

<?php
if ($contentType) {
    list ($name, $callback) = $contentType;
    header($_SERVER["SERVER_PROTOCOL"] . " 200 OK");
    header("Content-Type: " . $name);
    $data = call_user_func($callback, $data);
}
else {
    header($_SERVER["SERVER_PROTOCOL"] . " 406 Not Acceptable");
}

Determining the Data Compression Format

Once the data is serialized, it must be compressed to minimize the amount of bandwidth required to get the data from the server to the client. This process of determining the data compression format is similar to determining the data serialization format but instead parses the Accept-Encoding header, which does not use quality factors. This enables you to decide their precedence order if the client supports multiple compression formats.

Again, if you have a PHP framework of choice you should consult its documentation to see if there’s a more prevalent way to do this, but I’ve included a raw PHP sample below. Additional encodings can easily be added, but I’ve included the ones most commonly used in HTTP responses with bzip2 having the highest precedence because it has the best compression.

<?php
// Determine what encodings the client supports 
$clientEncodings = array_map("trim", explode(",", $_SERVER["HTTP_ACCEPT_ENCODING"])); 

// Determine what encodings the server supports 
$serverEncodings = array(); 
if (extension_loaded("bz2"))  {
    $serverEncodings["bzip2"] = "bzcompress"; 
} 
if (extension_loaded("zlib"))  {
    $serverEncodings["gzip"] = "gzdeflate"; 
    $serverEncodings["deflate"] = "gzcompress"; 
} 

// Return an encoding supported by both if one is found 
foreach ($serverEncodings as $encoding => $callback)  {
    if (in_array($encoding, $clientEncodings))  {
        return array($encoding, $callback); 
    } 
} 
return array();

In the example below, $encoding is the return value of this method. If a supported data compression format is found then its name is returned in the appropriate header per RFC 2616 Section 14.11.

<?php
if ($encoding) {
    list ($name, $callback) = $encoding;
    header("Content-Encoding: " . $name);
    $data = call_user_func($callback, $data);
}

While some data serialization formats may appear more efficient prior to compression being applied, that doesn’t necessarily mean this will also hold true after compression is applied. For example, MessagePack is often more compact than JSON prior to compression, but less compact in most cases after compression. As such, you should test different serialization and compression combinations on real data to determine which is optimal for your application. As a general rule of thumb, use JSON and either gzip or (preferably) bzip2 as a baseline for comparison.

Sending the Response

By this point, you’ve sent the response code and data serialization and compression formats. One other useful header you can send back indicates the amount of data in the response body. Finally, you’ll send the optionally compressed serialized data stream back.

<?php
header("Content-Length: " . mb_strlen($data));
echo $data;

Decompressing the Response

When last we left DataModel.getData(), it was executing the HTTP request to get the data we need. Now that our PHP REST service has returned a response, we need to decompress and deserialize it before we can do something useful with it.

The exact code needed to decompress the response varies based on the compression scheme used. gzip and deflate are natively supported in Android. Surprisingly, bzip2 is both the more efficient of the three decompression formats and the more difficult to use because it’s not natively supported. The easiest way to add support for it is through the Apache Commons Compress library. Here’s what code to handle each of these compression schemes might look like:

if (httpResponse.getStatusLine().getStatusCode() != 200) {
    // An error occurred, throw an exception and handle it on the calling end
}
String encoding = httpResponse.getFirstHeader("Content-Encoding").getValue();
java.io.InputStream inputStream = null;
if (encoding.equals("gzip")) {
    inputStream = AndroidHttpClient.getUngzippedContent(httpResponse.getEntity());
} else if (encoding.equals("deflate")) {
    inputStream = new java.util.zip.InflaterInputStream(httpResponse.getEntity().getContent(), new java.util.zip.Inflater(true));
} else if (encoding.equals("bzip2")) {
    inputStream = new org.apache.commons.compress.compressors.bzip2.BZip2CompressorInputStream(httpResponse.getEntity().getContent());
}
String content;
try {
    content = new java.util.Scanner(inputStream).useDelimiter("\A").next();
} catch (java.util.NoSuchElementException e) {
    // An error occurred, throw an exception and handle it on the calling end
}

If you choose to use bzip2, you have two options in how you use the Apache Commons Compress library. The first is to add the library JAR file to your project. This is less tedious to do and makes upgrades as easy as replacing the JAR file and rebuilding the project, but it results in a larger Android APK file because it’s including the entire JAR contents. Here’s how to use this method:

Download one of the Binaries archives for the library and extract it.
In Eclipse, click the Project menu and select the Properties option.
From the window that appears, select Java Build Path from the list on the left.
In the right pane, click the Libraries tab.
Click the Add External JARs button.
Locate and select the commons-compress-#.jar file where # is the current version.

The second option is to get the individual source files you need for bzip2 compression and copy those into your project manually. This is a more tedious process, but your Android APK file size will be significantly smaller. Here’s how to use this method:

Download the source tarball and extract it.
In Eclipse, right-click on your project’s src directory, select New > Package, and enter the package name org.apache.commons.compress.compressors.
Repeat step 2, but this time use a package name of org.apache.commons.compress.compressors.bzip2.
Navigate to the src/main/java directory of the extracted source tarball.
Copy these files into your project within their associated packages:

org/apache/commons/compress/compressors/CompressorInputStream.java
org/apache/commons/compress/compressors/bzip2/BZip2Constants.java
org/apache/commons/compress/compressors/bzip2/BZip2CompressorInputStream.java
org/apache/commons/compress/compressors/bzip2/CRC.java
org/apache/commons/compress/compressors/bzip2/Rand.java

Deserializing the Response

Once the response is decompressed, you’ll need to deserialize it into a usable data object. In the case of JSON, Android uses a Java implementation. The code to parse a JSON array of data might look like this:

import org.json.*;

// ...
// String content = ...
JSONArray contacts = (JSONArray) new JSONTokener(content).nextValue();
JSONObject contact = contacts.getJSONObject(0);
String email = contact.getString("email");

Caching Responses

The main focus of this article has been on minimizing the amount of data transferred during a request. In cases where the same resource may be requested multiple times, another way to make communication more efficient is to have the client (e.g. the Android application) cache responses and then indicate to the web service what version of the resource it last accessed.

There are two ways to do this, both described in subsections of RFC 2616 Section 14. The first method is time-based where the server returns a Last-Modified header (subsection 29) in its response and the client can send that value in a If-Modified-Since header (subsection 25) in a subsequent request for the same resource. The second method is hash-based where the server sends a hash value in its response via the ETag header (subsection 19) and the client may send that value in a If-None-Match header (subsection 26) in a subsequent requests. In both cases, if the resource has not been updated, it will return a 304 (Not Modified) response status.

Using the first approach, you’d simply include a line like the one below on the PHP side where $timestamp is a UNIX timestamp representing the time that the requested resource was last modified. In many cases, a UNIX timestamp can be obtained from a formatted date string using the strtotime() function.

<?php
header("Last-Modified: " . date("D, d M Y H:i:s", $timestamp) . " GMT");

On the Android side, DataModel.getData() would be modified to look like this:

// Fetch the Last-Modified response header value from a previous request from persistent storage if 
// available
// String lastModified = ... 
// HttpGet httpRequest = ...
httpRequest.addHeader("If-Modified-Since", lastModified);
HttpResponse httpResponse = this.httpClient.execute(httpRequest);
switch (httpResponse.getStatusLine().getStatusCode()) {
    case 304:
        // Use the cached version of the response
        break;
    case 200:
        // Handle the response normally
        lastModified = httpResponse.getFirstHeader(
            "Last-Modified").getValue();
        // content = ...
        // Store both of the above variables in persistent storage
        break;
}

Wrapping-Up

You now know how to implement content negotiation for serialization and compression purposes as well as response caching in both PHP and Android. I hope this article has proven useful to you and that your mobile applications are more efficient for it.

Image via Fotolia