Using Transloadit for Basic Picture Uploads

Glenn Goodrich
Ruby Editor

Recently, I had the need to investigate better methods of uploading pictures to our web application. The app is Rails-based and hosted on Heroku. Heroku, as I am sure many of you know, constrains its hosted applications with a read-only file system. In actuality, the read-only file system is an eventual constraint, meaning, Heroku will allow the application to write to a small, temporary space on the file system. However, that temp space is swept frequently and without predjudice, so any files in it don’t live very long.

Since Heroku does allow writing to this temp space, it’s possible to deploy an app that allows users to upload pictures, which your application can manipulate/resize to its needs. There are numerous examples of how to do this which gems like CarrierWave, which is where we started. Unfortunately, this approach does not scale with Rails. Uploading a file using Rails will consume a web dyno. If the file is large and takes more than 30 seconds to upload, then your app will start throwing timeout errors all over the place. This article goes into a bit more detail about this.

In short, if you have a serious Rails application hosted on Heroku that requires file uploads, you need to find an approach that uploads the file directly to S3 (or a dedicated image service) and performs image manipulations outside of your Rails app. Your life will be much better for it.

Options

In my research on this issue, I came across CarrierWave-direct, which allows direct uploading to S3. I tried, in earnest, to make it fit our existing flow and could not. Also, I beleive that Carrierwave will pull the file from S3 onto Heroku’s servers to perform its processing, which I wanted to avoid. I am not here to bash CarrierWave or CarrierWave-direct, but to bring attention to them before I go into how Transloadit fits into the picture.

Transloadit Basics

Along with every other developer on the planet, Node has caught my attention. I read about it daily, try to keep up with happenings in that community. This is where I first heard about Transloadit. I have heard many rave reviews of it’s service since then, so I decided to give it a go. The basics of Transloadit consist of four items: Assemblies, Robots, Steps, and Templates.

Assemblies

An assembly is, basically, an individual job on Transloadit servers. In our example today, an assembly will encompass the uploading and resizing of our image.

Robots

The Transloadit Robots (and I LOVE that they call them that) each have a special skill. Some robots resize images, some store files, etc. You bring the robots into your jobs as needed using Steps

Steps (or Assembly Steps)

Steps are one part of an assembly. Resizing an image is a step (performed by a robot) and writing the file to S3 is another step

Templates

Templates bring everthing else together. It is your job, as the developer, to create a template that includes the assembly steps, performed by robots, to satisfy the needs of your application. Templates are specified in JSON, so it should be in your wheelhouse. In other words, creating a template is very simple.

Example App: Uploading Pictures

Right, let’s bring this all together with a contrived application. This application will simply allow a user to upload a photo and create a thumbnail and “fullsize” version of the photo. I won’t bore you with the basic setup details of the Rails app, but the source is here

Setting Up Transloadit

First things first, go sign up for an account on Transloadit. They have a free plan that should get you through this demo (although, not much farther, unfortunately.)

Transloadit needs to know what to do with our pictures, so we need to go to our templates section and create a new template.

I mentioned above that we want a thumbnail and a fullsize version of the pictures, so we need to define a template that creates those versions. Looking through that more-than-adequate documention on Transloadit’s site, we can structure our template as:

Briefly running through that template, the first step creates our thumbnail image at 50X50 pixels. It uses the image/resize robot on the “:original” image. Specifying "use" : ":original" is important. By default, Transloadit will feed the output of one step into the next step, which in this case, would distort our full size image.

The second step, as you probably guessed, creates our fullsize image at 320X1000 pixels. The last step (“export”) writes our file to S3 using the /s3/store robot. This step has some interesting values.

The use parameter includes the two images we created plus the original, which means each assembly will write 3 files. Also, our AWS authentication (if you need to setup S3, this page will help) information is in this step. Finally, the path step defines how we will name the files. Each job will result in a folder named for the assembly ID and the files will be based on the step name that created them.

The template can be tested via Transloadit’s web site, which is really cool. Presuming you have S3 setup, go ahead and upload a file and check out the two versions it creates.

Workflow for Uploading a Picture

The diagram below shows a high-level flow of uploading a picture using Transloadit.

1) The user submits a form that contains a file input.
2) The file is uploaded to the transloadit service, and the robots do their thing.
3) Once the files are ready, they are moved to S3.
4) The form submission is redirected to the rails app, along with some extra parameters from Transloadit.
5) The request returns to the user.

Now, this is just ONE example of a workflow using Transloadit. There are many options, including NOT waiting on the file upload/processing to complete and just being notified when it’s all done. When looking into the service, explore all the options and use the best approach for your particular situation (duh).

The Code

For our application, we are going to use the minimal integration approach along with Transloadit’s jQuery plugin. Using this approach, we can take our vanilla web form:

and decorate it using Transloadit’s jQuery plugin, like so:

The wait parameter tells the plugin to not redirect to my redirect_url until the file processing is complete.

To make it work, we have to include some parameters in the form that specify our Transloadit API key as well as which template to use. The hidden parameters look like:

Here is our form with that information encoded and added to the form.

Obviously, you’ll need to change your key and template id information. Also, notice that the redirect_url is http://localhost:3000/pictures, which is where the form will be redirect to once the file upload and processing completes.

Handling the Redirect

With all the uploading goodness in place on the client, that last task is to handle the redirection of the form from Transloadit in our controller. Since each upload will have a unique assembly id, that value will be stored on the picture and used to generate a url for each photo.
The controller code itself is very simple:

Transloadit redirects the form with all of the paramters we need (assembly_id and file name information)  which we extract and then apply before the vanilla create method. The redirect includes the parameters from our own form as well, allowing us access to the data inputted by the user.  Pretty easy.

Gotcha

Looking back at the controller code, you may have noticed that I downcase the file extension when generating the file name. The URLs to resources on S3 are case-sensitive, and the Transloadit robots create their file versions with lowercase extension values. As a result, if you upload ‘IMAGE.JPG’ the thumbnail will be called ‘thumbnail_IMAGE.jpg’ and it will 404 if you don’t use ‘jpg’ in your urls.

Upload Away

I hope you found this article useful. Transloadit is a very powerful service that offers a slew of options for uploading assets (they do more than just photos) and allows you to keep the uploading process out of your main app. On hosts like Heroku, this can keep your processes free to handle other things, more core to your business and keep your users very happy.

Other Resources

There is an official Rails gem for using transloadit, that abstracts most of what you saw in this article. There is also a Ruby gem and SDKs for various other platforms.

Free book: Jump Start HTML5 Basics

Grab a free copy of one our latest ebooks! Packed with hints and tips on HTML5's most powerful new features.

  • Brandon

    Beautiful, thanks for sharing.

  • Zach

    Nice share, i am beginning to give Transloadit a go…

    But one question i have is that what if i have a as many relationship..
    ie. A blog post has many attachments. Each attachment comes with thumbnail, small, and large.

    Transloadit will be able to create these different sizes, but on my application end, i will need a nice way to manage these sizes so i can call attachment_url(:thumb) and so on (ala. Carrierwave)
    What is the best way to handle this after getting the callback from Transloadit? Combining it with Carrierwave on the app?

    I am thinking of using Transloadit to send the files to s3, then have Carrierwave do the processing in background. Then again i am limited by Heroku, and wonder if Carrierwave Backgrounder would work?

    • http://www.ruprict.net/ Glenn Goodrich

      Zach,

      Just saw this comment…sorry it’s taken me forever to respond. You can easily write your own method to build the url to each image (thumb or large), just like Carrierwave does. We store the images with the transloadit_assembly_id in the “path” (there is no path on S3, but you can fake it) and the file name should be thumb_.png

      You can set how the filename is generated in your transloadit assembly steps.

      Does that help?