Stress Testing with Loader.io

If you are working on a serious project, you want it to scale. The thing about scale is, you only focus on it once you really need it.

I’m the CTO of an soccer/futbol social network based in Brazil. To put it mildly, futbol is big in my country. This summer, we focused our marketing on the World Cup, preparing our application to support as many users as possible. To do that, we had to benchmark and improve, but how could we load test? What tool should we use? Those are just some questions that came up once we started to address this challenge.

This is when we found Loader.io.

Where Did It Come From?

Loader.io is a stress testing service. It’s not just a tool, as it helps you to focus on what to test. It’s developed by SendGrid, and they describe the service like so:

Loader.io is a free load testing service that allows you to stress test
your web-apps/apis with thousands of concurrent connections.

How to Setup

Loader.io tries to simulate users and their impact on your application, rather than focus on code. This gives a more realistic result, but it is not a guide that shows where you focus to increase performance. Loader.io will uncover scaling issues, but it’s up to you to find the fix. In our case, the results were compelling and we were able to hunt down our scaling issues.

First things first, you need an account on Loader.io (no worries, they won’t fill your inbox). When signing up, you need to select a plan, and the free one will work for your initial needs.

Once the account is created, Loader.io will ask you to confirm your domain. They can even help you, if you need it.

Heroku users are covered, Loader.io is a plugin, which makes it very simple to utilize the service.

Your First Test

Once you have added Loader.io to your application, use the web interface to add your first test:

There are a few, simple options to setup your test. The interface will guide you to scenario that best emulates your needs:

• Clients per test (Clients will be distributed evenly throughout the test duration.)
• Clients per second (Clients will be started each second)
• Maintain client load (A constant client count will be maintained throughout the test duration.)

The scenario you choose depends on the usual user behavior for your application. In most cases, you’ll want to create more than one test to cover multiple scenarios.

With the first test ready to go, Loader.io will run it and display the results. The whole process is simple and the results are easy to understand. I will run through some test results in a bit.

Stress Test as Culture

Today, we are used to writing tests/spec to our applications. It’s a part of Ruby culture. Stress testing, on the other hand, isn’t a common priority. More often than not, developers and companies only think about it after disaster occurs.

I see Loader.io as a facilitator in making stress tests a more common practice for application development. Besides everything I have shown so far, one of the best features of Loader.io is it’s API. It makes it possible to integrate your tests into your deployment routine. It’s a major win, helping you to keep your code’s quality and performance high and scalability-minded.

Real Case Example

Here are my first test results after deciding it was time to stress test our application for the World Cup (Needless to say, after this I got really worried):

Loader.io’s results pointed out that, when 15 requests hit in the same minute (1 for each 4”), only 3 received responses. The average time of each response was 9456ms (exactly, 9.5 seconds). Yikes!

I started by updating tiny details, trying to pinpoint the scaling issues. I then moved forward to refactoring code, adding indexes, and updating database structures. After this, I added caching and compost caching on everything possible.

I didn’t slept for 3 nights, but the results were amazing. This is the result of the test 3 days after the first one:

Instead of just 15 requests in one minute, this time it was 300 requests (5 for each second). The app now had 299 successful responses, with an average response time of 627ms. Yeah, it was awesome.

Wrap Up

If you are serious about your application, you need to work on scalability as soon as possible. Making stress tests a part of your development culture is the best way to guarantee that you will be ready to scale your application when needed.

There are many tools that help you to stress test your apps, but Loader.io is definitely one of the best I have used so far. Give it a try.