IronMQ and Laravel: Setup

This entry is part 1 of 3 in the series IronMQ and Laravel

IronMQ and Laravel

This two-part article series aims to make a beginner understand using push queues with Laravel. To give a brief overview, the final solution which we will be looking at is a simple form to upload photos and resize them. Resizing images is a time consuming task, so instead of making a user wait until the image is resized we can do it in the background. At the same time, we’ll learn to use a tool called ngrok so that we can use queues in our local system.

The source code is provided at: https://github.com/phpmasterdotcom/laravel-queues

You can also try it in a proper live server.

Queues and Iron

A queue is nothing but a pipeline where we line up our jobs and they are executed in the order they are inserted. Let’s say currently our queue is empty and we put Job A into it, then we put Job B and Job C. We read the queue in the same order. First we take Job A from the queue and finish it, and then work on next one. If you focus on the “read” part, we always have to check our queue to see if any new job is posted. Wouldn’t it be better if the queue itself notifies us when a job is available ? Queues like those are called push queues, and that’s what we’ll be using with IronMQ.

Iron MQ is a service which makes our life easier when it comes to using queues. When you create a push queue you also need to mention a subscriber to that queue. A subscriber is nothing but a url which will be called when a job is available in the queue with the data originally provided to the job.

To read more about job queues and see comparisons between different solutions, see this article.

Setup and Installation

In this step we install Laravel and all required dependencies via composer. We also create an account at Iron and install a tool called ngrok.

Laravel

  1. Install Composer.

  2. Install Laravel

    composer create-project laravel/laravel --prefer-dist

    After this command you should see the folder “laravel”. Open the laravel folder in your favorite text editor or IDE.

    Go into the folder via the command line and run:

    php artisan serve

    At this point if you open http://localhost:8000 you should see the home page of your Laravel installation.

  3. Set up the Database : We will use MySQL for our database. Create a database via the command line, phpmyadmin, or whichever tool you are comfortable with. After you created a database, go to app/config/app.php and find the section similar to the following:

    'mysql' => array(
        'driver'    => 'mysql',
        'host'      => 'localhost',
        'database'  => 'database',
        'username'  => 'root',
        'password'  => '',
        'charset'   => 'utf8',
        'collation' => 'utf8_unicode_ci',
        'prefix'    => '',
    )
    

    Fill the required fields. Assuming you are on a local system, you need to provide the credentials of your MySQL server (username, password) and the name of the database you created.

    Run php artisan migrate:install. If you have put your credentials correctly it should say: Migration table created successfully.

  4. Let’s make a small edit. Open app/views/hello.php and find the following line:

    <h1>You have arrived.</h1>
    

    Edit it to (You can change it to anything!):

    <h1>I am learning how to use queues!</h1>
    

    If you open http://localhost:8000 you should be able to see the Laravel logo and some text under it reading: “I am learning how to use queues!”.

  5. Open composer.json, add iron (for iron push queues) and intervention (for image manipulation) libraries.

    "require": {
        "laravel/framework": "4.1.*",
        "iron-io/iron_mq": "1.4.6",
        "intervention/image": "dev-master"
    },
    

    Update the code

    composer update
    
  6. Follow instructions below to finish installation of Intervention.

    Open your Laravel config file config/app.php. In the $providers array add the service providers for this package: 'Intervention\Image\ImageServiceProvider', then add the facade of this package to the $aliases array: 'Image' => 'Intervention\Image\Facades\Image',

    Reference: http://intervention.olivervogel.net/image/getting_started/laravel

ngrok

In production users access your website via a url like http://example.com. But, you all know that our localhost is not available to the whole internet and as we are going to use Iron for implementing queues, we need to host our code in a server so that our website is accessible to Iron. Doing that means, every time we make a change to our code we have to upload to server and test to see if it is working as we intended. Wouldn’t it be better if we could test our code on a local system? That’s why we use a tool called ngrok which makes our localhost accessible to the whole internet via a special url. We will be installing and using it in a second.

  1. Go to ngrok.

  2. Download and Install ngrok. Follow – How to install ngrok.

    The steps are:

    • Download the zip file for your operating system.
    • Unzip the downloaded zip file
    • You should be able to run ./ngrok --help from the command line.
  3. Let’s make our localhost visible on the internet.

    For this step we assume Laravel is still running at http://localhost:8000. (If it’s not, run php artisan serve from the command line at the root of the Laravel folder)

    (You need to run the ngrok command through another terminal as in one terminal php artisan serve needs to be running)

    ./ngrok 8000
    

    I have given port as 8000 since our laravel is running on 8000. After I run the command I get a reply:

    Tunnel Status                 online                                            
    Version                       1.6/1.5                                           
    Forwarding                    http://953ffbb.ngrok.com -> 127.0.0.1:8000        
    Forwarding                    https://953ffbb.ngrok.com -> 127.0.0.1:8000       
    Web Interface                 127.0.0.1:4040                                    
    # Conn                        2                                                 
    Avg Conn Time                 167.38ms 
    

    Note: The response you get is different every time you run ngrok. So, the response which you see after running ./ngrok 8000 will be different from what I posted above.

    Observe the url under “Forwarding” : http://953ffbb.ngrok.com. If you open that url (which is a different url for you) you should be able to see your website with the Laravel logo and the text you have written, e.g.: “I am learning how to use queues!”.

    I suggest you keep ngrok running and note down the Forwarding url as we are going to use it for this tutorial.

    If you happen to restart ngrok in between, your url will change. It’s not at all a problem! Just make use of the different url whereever needed.

Iron MQ

We will use Iron for implementing queues in our project. To use Iron we first need to create an account.

  1. Go to http://www.iron.io/ and log in

  2. Create a new project. I have named my project “Laravel Queues”.

  3. You can see a “Key” button right next to your project name. Click on it and copy “Token” and “Project ID” to app/config/queue.php.

    • Change default to ‘iron’

      'default' => 'iron',
      
    • Find the below section:

      'iron' => array(
          'driver'  => 'iron',
          'project' => 'your-project-id',
          'token'   => 'your-token',
          'queue'   => 'your-queue-name',
      )
      

      Copy your Project ID and Token from the dashboard and paste them here, name your queue “laravel”.

      'iron' => array(
          'driver'  => 'iron',
          'project' => 'your-project-id',
          'token'   => 'your-token',
          'queue'   => 'laravel',
      )
      
  4. As discussed above we need to add a subscriber for our queue. You can go to Laravel Push Queues and check the official documentation which we are going to follow now:

    • We have to register a push queue subscriber using the following artisan command (We have to run this command from our project root folder)

      From above reference, we will run php artisan queue:subscribe laravel http://953ffbb.ngrok.com/queue/receive

      Note that the url I used was the Forwarding url which ngrok generated. You should see a reply similar to the one below:

      Queue subscriber added: http://953ffbb.ngrok.com/queue/receive

      If you check your dashboard and click on “MQ” you will go to the project page. Click on “Queues” and find a queue named “laravel”. If you click on a queue name, i.e. laravel, you can see a list of subscribers in the Push Queues tab and find http://953ffbb.ngrok.com/queue/receive under it.

    • Put this code in app/routes.php

      Route::post('queue/receive', function()
      {
          return Queue::marshal();
      });
      

From the official documentation:

The marshal method will take care of firing the correct job handler class. To fire jobs onto the push queue, just use the same Queue::push method used for conventional queues.

Wrapping Up

In this part:

  1. We have installed Laravel with Iron and Intervention libraries. Our laravel is running at http://localhost:8000

  2. We installed ngrok and we exposed our localhost to the whole internet via a Forwarding Url that was generated for us (let ngrok run in the terminal).

  3. We configured Iron with Laravel and created a queue with a subscriber.

In the next part, we’ll do the heavy lifting and get our hands dirty. We’ll be building our app and implementing the job logic fully. Stay tuned!

IronMQ and Laravel

IronMQ and Laravel: Implementation >>

Free book: Jump Start HTML5 Basics

Grab a free copy of one our latest ebooks! Packed with hints and tips on HTML5's most powerful new features.

  • MarkL

    You should address long running tasks with Iron. They timeout after 60s. Laravel still hasn’t fixed this.

    • http://www.bitfalls.com/ Bruno Skvorc

      Thanks for the heads up!

    • Chad Arimura

      Hi MarkL,

      We (Iron.io) are working on a few solutions around this. If the app knows it’ll take longer than 60s, you can do one of two things:

      1) Return a 202 http://dev.iron.io/mq/reference/push_queues/#long_running_processes__aka_202s
      2) Push to IronWorker which has 60 minute timeout (tutorials coming soon at http://blog.iron.io)

      Chad

      • MarkL

        Chad, thanks. There is nothing in 4.2 that would address this. Someone made a pull request though that hasn’t been accepted in months: https://github.com/laravel/framework/pull/3555

        • rajivseelam

          Bruno suggested that i write a tutorial on handling long running jobs. We can use IronMq class from iron-mq composer package directly and achieve it.

    • rajivseelam

      I think in Laravel 4.2 we will have option to pass timeout (and more options) when pushing to a Queue. I actually didn’t go much deeper as my intention was to help developers who are not using queues yet. Stay tunes for Part 2.

      • MarkL

        Looking forward to the second part. There’s nothing in 4.2 to pass options, unless Taylor will make a last minute change.

        • rajivseelam

          There is a pull request which makes passing options possible. But it’s not merged for sometime now.

  • http://harikt.com/ Hari K T

    It would have been better to make use of IlluminateQueue . ( Part of Laravel without making it Laravel specific )

    • rajivseelam

      You are right. But my focus was for beginners who are not making use of queues, and to help them start off immediately. Thanks for input.

  • rajivseelam

    composer require iron-io/laraworker

    Loving it!

  • Krzysztof Wolniak

    In Laravel 4.2 i have two additional options in iron config: ‘host’ => ” and ‘encrypt’ => true. What should i set in host?

    • rajivseelam

      ‘host’ was present in 4.1 too. By default it’s set to ‘mq-aws-us-east-1.iron.io’