Speeding up Existing Apps with a Redis Cache

We’ve gone through the basics of Redis in PHP before, but it’s time to cover a real life use case. In this tutorial, we’ll add it to an already deployed application to give the app the appearance of speed.

You can easily follow along by cloning the 0.6 release of the app.

The Problem

Before applying a solution, we need to have a clear definition of the problem.

The application in question, when executing a query, runs off to Diffbot’s API and makes it query the dataset. The subset is then returned and displayed. This can take up to 5 or so seconds, depending on the busyness of Diffbot’s servers. While the situation will undoubtedly improve as they expand their computational capacity, it would be nice if a query executed once were remembered and reused for 24 hours, seeing as the collection is only refreshed that often anyway.

“But what good is caching a single query?” you might wonder. It’s not like most people will search for one and the same thing often.

Well… as a matter of fact, not only has research shown that they will often search for one and the same thing (React is trending? Sudden influx of “react” queries), they will also very reliably search for prolific authors (or themselves). Considering the fact that implementing this cache costs us literally nothing (and actually reduces costs by reducing strain on the servers), adding it in is an easy win, even if it weren’t used as often as one would hope. There is no reason not to add it – it can only benefit us.

With the problem clearly defined, let’s handle the prerequisites.

Installing

First things first, we need to install Redis into both the development and production environment (note that if you’re using Homestead for local development, Redis is already installed, albeit v 3.0.1 at the time of writing).

We can do this via the operating system’s package manager like so:

sudo apt-get install redis-server

This is the simplest and recommended approach, but we can also install it from scratch and manually configure it. As per instructions on their website, this is done via:

sudo apt-get install gcc make build-essential tcl
wget http://download.redis.io/releases/redis-3.0.2.tar.gz
tar xzf redis-3.0.2.tar.gz
cd redis-3.0.2
make
make test
sudo make install

If you run into a fatal error mentioning jemalloc.h after running make just run make distclean and then make again. The make test command is optional, but helpful.

Note: If you’re reading this and version 3.0.2 is no longer newest, just adapt the commands to the newest version number.

To prevent some common warnings (on Ubuntu at least), we also preventatively run the following commands:

sudo sh -c 'echo "vm.overcommit_memory=1" >> /etc/sysctl.conf'
sudo sh -c 'echo "net.core.somaxconn=65535" >> /etc/sysctl.conf'
sudo sh -c 'echo "never" > /sys/kernel/mm/transparent_hugepage/enabled'

We also make sure the last command is added to /etc/rc.local just above exit 0, so that it’s re-issued on every server restart. Finally, we can reboot the server with sudo reboot and check if everything is fine by running Redis with sudo redis-server.

Finally, we need to make sure Redis starts after a server restart, so we follow their official instructions to get that accomplished.

Predis

We covered the basics of Predis before, and we’ll be using it in this case, too. Let’s install it with:

composer require predis/predis

Going further, the assumption is that we’ve absorbed the knowledge from the aforementioned introduction to Predis.

Since that post was published, minor differences were introduced (like a transition to namespaces) but the API we need is more or less the same.

Implementation

To make use of Redis in our app, we need to follow this procedure:

• see if results for the current query exist in the cache
• if yes, grab them
• if no, fetch them, store them, forward them to the rest of the app

Thus, the implementation is extremely simple: under the “form submitted” check (the one that looks for the “search” param), we instantiate the Predis client, calculate the md5 hash of the executed search query, and then check if the results for it are already cached. If false, the previous flow continues, only, rather than end in:

$result = ...
$info = ...

we serialize and save the results directly into the cache. Then, immediately outside the block, we grab them from the cache and the app’s flow continues as usual. The changed part of the index.php file therefore looks like this:

// Check if the search form was submitted
if (isset($queryParams['search'])) {

    $redis = new Client();
    $hash = md5($_SERVER['QUERY_STRING']);
    if (!$redis->get($hash . '-results')) {

        $diffbot = new Diffbot(DIFFBOT_TOKEN);

        // Building the search string
        $searchHelper = new SearchHelper();
        $string = (isset($queryParams['q']) && !empty($queryParams['q']))
            ? $queryParams['q']
            : $searchHelper->stringFromParams($queryParams);

        // Basics
        $search = $diffbot
            ->search($string)
            ->setCol('sp_search')
            ->setStart(($queryParams['page'] - 1) * $resultsPerPage)
            ->setNum($resultsPerPage);

        $redis->set($hash . '-results', serialize($search->call()));
        $redis->expire($hash . '-results', 86400);
        $redis->set($hash . '-info', serialize($search->call(true)));
        $redis->expire($hash . '-info', 86400);
    }

    $results = unserialize($redis->get($hash . '-results'));
    $info = unserialize($redis->get($hash . '-info'));

After testing, we can see that it works like a charm – a query executed once is instant if we refresh the page, or execute another query and then come back to the previous one. Finally, we can add, commit, and push to deploy:

git add -A
git commit -m "Added Redis cache [deploy:production]"
git push origin master

That’s it! Our app’s newest version is now live and Redis is serving the cached data. Test it here!

Note: If you’re wondering how we got from development mode to production deployment in a single commit, you should read this.

Fine Tuning

For an additional performance boost, Predis recommends the installation of phpiredis, a PHP extension to “to lower the overhead of serializing and parsing the Redis protocol“. Seeing as we’re in full control of the server, why not?

cd ~
git clone https://github.com/redis/hiredis
cd hiredis
make
sudo make install
cd ~
git clone https://github.com/nrk/phpiredis
cd phpiredis
phpize && ./configure --enable-phpiredis
make
sudo make install

sudo touch /etc/php5/mods-available/phpiredis.ini
sudo sh -c 'echo "extension=phpiredis.so" > /etc/php5/mods-available/phpiredis.ini'
sudo php5enmod phpiredis
sudo service php5-fpm restart

This installed the prerequisites and enabled the extension. Now all we have to do is configure the Predis client to use the phpiredis connection. We need to replace:

$redis = new Client();

with

$redis = new Client('tcp://127.0.0.1', [
        'connections' => [
            'tcp'  => 'Predis\Connection\PhpiredisStreamConnection',
            'unix' => 'Predis\Connection\PhpiredisSocketConnection',
        ],
    ]);

That’s it! Our Redis installation is now even faster!

Conclusion

In this tutorial, we used Redis in combination with the Predis library to give an already deployed app the appearance of speed. We leverage the available RAM of our DigitalOcean droplet to save the results of a query once per day, and then return those results from cache rather than doing the round-trip to their origin. This does mean the results aren’t always fresh, but as per this post, the results aren’t refreshed more often than that anyway.

Hopefully, this tutorial showed you how simple it is to add an in-memory caching layer to your application, and will be of use when you need to shorten those loading times and reduce those server costs.

Any additional advice? Tips? Comments? Leave them below!

Frequently Asked Questions (FAQs) about Speeding Up Existing Apps with a Redis Cache