PHP

Running Tasks in the Cloud with IronWorker

Lukas White
Share

In this article I’m going to show you how we can use IronWorker to run code in the Cloud, just as if it were being run inside our PHP application’s code. There are a number of advantages to running tasks in the cloud, for example:

  • processor-intensive tasks can be offloaded from your web server
  • better fault tolerance
  • the execution of your code isn’t blocked waiting for long-running tasks

Here’s how IronWorker describes its service:

An easy-to-use scalable task queue that gives cloud developers a simple way to offload front-end tasks, run scheduled jobs, and process tasks in the background and at scale.

Glossary of Terms

Before we get started, let’s look at some of the terms used by IronWorker and this article which may be unfamiliar to you.

A worker is our code that performs a task. We can write and test it locally, but eventually we upload it to IronWorker’s servers and it will be executed there, rather than on on our web server.

A task is an execution of the code, either on demand (think of it like calling a function) or on a scheduled basis (like a cron job). We can delay execution, queue it to run a certain number of times, or run it repeatedly until a specific time and date.

The payload is the data used to execute a task (using the function analogy, the payload would be the arguments passed to the function).

The IronWorker environment is essentially a Linux sandbox in the Cloud that our tasks are executed on.

A project can be used to group workers together, perhaps for a particular application.

HUD is IronWorker’s web interface which we use to manage our projects and tasks, view the results, edit and view schedules, and keep track of our usage.

Installation and Configuration

There are a few steps involved in getting everything set up.

First, we need to create an account with IronWorker. It offers a free tier with up to 200 hours of processing time per month, which should be more than adequate to follow along with this article.

We also need the IronWorker CLI tool; visit the documentation for full installation instructions. But if you meet the pre-requisites for its installation – primarily you must have Ruby installed – you can install it in one line:

gem install iron_worker_ng

Next, we need to download the PHP client library for working with IronWorker, either from GitHub or with Composer.

{
    "require": {
        "iron-io/iron_worker": "dev-master"
    }
}

The easiest way to configure IronWorker is with a JSON file that looks like this:

{
    "token":      "YOUR-TOKEN",
    "project_id": "YOUR-PROJECT-ID"
}

The token identifies your account, and the project ID can be set to a default and overridden by a particular worker. You can find all of this information in HUD.

If you name the JSON file .iron.json (note the preceding dot) and place it in your home directory, it will be used to provide default values for any of your projects. Alternatively, you can name it iron.json (no preceding dot) and place it in the same directory as the worker. The file in the same directory will override values in the home file, so you can easily set global defaults for and then override them locally for specific projects as needed.

Alternatively, you can set these parameters using environment variables:

IRON_TOKEN=MY_TOKEN
IRON_PROJECT_ID=MY_PROJECT_ID

A Hello World Example

We’ll start with a very simple example to show what’s involved in getting tasks running in the Cloud. First, create a file called HelloWorld.php:

<?php
echo 'Hello World';

Although this doesn’t do anything remotely useful, it will demonstrate an important point that you’ll see when we come to run it – anything printed to STDOUT (i.e. using echo, print, etc.) is logged by IronWorker and made able in HUD.

Then create a file called HelloWorld.worker with the following:

runtime 'php'
exec 'HelloWorld.php'

As you can probably infer from the first line, we’re not restricted to just PHP; IronWorker supports a multitude of languages. Check the documentation for a list of supported languages.

Now we need to upload our code to IronWorker’s servers. Using the CLI tool, this is a breeze:

iron_worker upload HelloWorld

Behind the scenes, the CLI tool creates a ZIP file containing our code, uploads it to IronWorker, and registers it as an available task for our project. Any time you change your code and run the upload command again, it will re-upload the code and create a new revision. You can download any revision from HUD.

We can enqueue tasks to run from our PHP application like this:

<?php
require_once '../vendor/autoload.php';

$worker = new IronWorker();
$taskID = $worker->postTask('HelloWorld');

You can view the current state of all tasks in HUD. Depending on your account setup, you may also receive emails to keep you updated, particularly if a task fails for whatever reason.

An optional second argument to postTask() allows us to provide a payload if we want to send one.

$payload = array(
    'foo' => 'bar',
    'biz' => 'fuzz'
);

$taskID = $worker->postTask('MyTask', $payload);

The payload is encoded for us and sent to the IronWorker servers. In the task itself, we can receive the decoded payload simply like this:

$payload = getPayload();

We can set up a task to run on a scheduled basis, much like a cron job is run, using the postScheduleSimple() method. Simple scheduling is like saying “run this in x seconds time.”

// run the task in five minutes time (given in seconds)
$schedID = $worker->postScheduleSimple('HelloWorld', $payload, 5 * 60);

Advanced scheduling is done with postScheduleAdvanced() and allows us to set the time of the first run, the interval, and either the time it should stop or the number of times to run it. Because so many arguments are needed, it’s it’s helpful to look at the method’s definition:

/**
 * Schedules task
 *
 * @param string        $name       Package name
 * @param array         $payload    Payload for task
 * @param int|DateTime  $start_at   Time of first run in unix timestamp format or as DateTime instance. Example: time()+2*60
 * @param int           $run_every  Time in seconds between runs. If omitted, task will only run once.
 * @param int|DateTime  $end_at     Time tasks will stop being enqueued in unix timestamp or as DateTime instance format.
 * @param int           $run_times  Number of times to run task.
 * @param int           $priority   Priority queue to run the job in (0, 1, 2). p0 is default.
 * @return string Created Schedule id
 */
public function postScheduleAdvanced($name, $payload = array(), $start_at, $run_every = null, $end_at = null, $run_times = null, $priority = null){

Here are some examples of its use:

// Run the task immediately, then every five minutes – a total of ten times 
$worker->postScheduleAdvanced('HelloWorld', $payload, time(), 5 * 60, null, 10);

// run task in 3 minutes time, every two minutes, five times 
$worker->postScheduleAdvanced('HelloWorld', $payload, time() + (3 * 60), 2 * 60, null, 5);

// run the task every hour for the next 24 hours
$worker->postScheduleAdvanced('HelloWorld', $payload, time(), 60 * 60, time() + (60 * 60 * 24));

A More Useful Example – Sending Email

Let’s build something slightly more useful now, and in doing so demonstrate a slightly different approach to how we structure or project and upload our code. Rather than uploading the worker with the CLI tool, we’ll create a simple upload script.

Create a fresh directory called worker. Copy iron.json into it, and if necessary create a new project via HUD and update the project ID in your configuration file. Then, download PHPMailer and place it in the worker directory.

In the worker directory, create the file config.php which will store some configuration values for PHPMailer. It should look something like this:

<?php
return array(
    'smtp' => array(
        'host'        =>  'smtp.example.com',
        'port'        =>  465,
        'encryption'  =>  'ssl',
        'credentials' =>  array(
            'username' => 'username',
            'password' => 'password'
        )
    )
);

Now create the file Emailer.php:

<?php
require_once './phpmailer/class.phpmailer.php';

$payload = getPayload();
$config = require './config.php';

$mail = new PHPMailer();
$mail->IsSMTP();
$mail->Host = $config['smtp']['host'];
$mail->Port = $config['smtp']['port'];
$mail->SMTPAuth = true;
$mail->Username = $config['smtp']['credentials']['username'];
$mail->Password = $config['smtp']['credentials']['password'];              
$mail->SMTPSecure = $config['smtp']['encryption'];
$mail->From = 'no-reply@example.com';
$mail->FromName = 'SitePoint IronWorker Tutorial';
$mail->AddAddress($payload->recipient->email, $payload->recipient->name);
$mail->WordWrap = 100;
$mail->IsHTML(true);
$mail->Subject = $payload->subject;
$mail->Body    = $payload->body; 

if (!$mail->Send()) {
    print 'Could not send email: ' . $mail->ErrorInfo;                 
}

Create Emailer.worker:

runtime 'php'
exec 'Emailer.php'

And create upload.php:

<?php
require_once '../vendor/autoload.php';

$worker = new IronWorker();
$worker->upload('worker/', 'Emailer.php', 'Emailer');

That last line tells IronWorker to upload the whole of the worker directory, which of course includes not just the worker code but also theh configuration file and supporting libraries. The second argument is the name of the file to run to execute the task itself, and the third argument is simply an identifier for the worker.

Run the upload script from the command line:

php upload.php

We can run the task from our PHP application the same way as before:

<?php
require_once '../vendor/autoload.php';

$worker = new IronWorker();

$payload = array(
    'recipient' => array(
        'name'  => 'Joe Bloggs',
        'email' => 'joe.bloggs@example.com'
    ),
    'subject'   => 'This is a test',
    'body'      => 'This is the body'
);

$taskID = $worker->postTask('Emailer', $payload);

If you want to send a batch of emails, you can either do a simple loop to enqueue multiple tasks, or wait until the previous task has finished executing, like so:

// loop...
$taskID = $worker->postTask('Emailer', $payload);
$worker->waitFor($taskID);

More Information

You should refer to the documentation to understand the IronWorker environment your code will run in. Highlights include (at the time of this writing):

  • The operating system is Ubuntu Linux 12.04 x64.
  • The version of PHP is 5.3.10.
  • The php-curl, php-mysql, php-gdm and mongo modules are installed (athough you need to call require_once(‘module-name’) to use them).
  • ImageMagick, FreeImage, SoX, and cURL are available.
  • Tasks have access to up to 10Gb of temporary local disk space.
  • The maximum data payload is 64Kb.
  • Each worker has around 320Mb available memory, and a maximum run time of 60 minutes.
  • By default, you can have up to 100 scheduled tasks per project.

Although beyond the scope of this article, there are also a few features worth noting:

  • REST API – there is an API available whereby you can view or modify projects, tasks, and schedules.
  • Webhooks – you can set up your tasks to run when a POST request is made to a specified endpoint. For example, you could set up a task to run when you commit code to GitHub.
  • Status – you can call getTaskDetails($taskID) to get that task’s status – “queued”, “complete”, “error”, etc.
  • Progress – you can set the progress of a task in your worker using setProgress($percent, $message), which is made available to getTaskDetails() above.
  • Logs – as previously mentioned, anything printed to STDOUT is logged by IronWorker and is viewable in HUD. You can also access it from the worker with getLog($taskID).

Summary

In this article, we’ve looked at running tasks in the cloud using IronWorker. I’ve shown you how to upload workers and queue and schedule tasks, and I’ve also mentioned some of the additional features available. If you want to explore further, I’d recommend taking time to read the documentation and look at the repository of examples on GitHub.