Building a User Avatar Component With Node.js & TransloadIt

In the first part of this series, we looked at TransloadIt — a file-processing service which specialises in processing images, videos and audio. If you haven’t read it yet, I’d urge you to do so now as it covers a lot of background concepts which you’ll need to understand in order to follow along with this part.

But enough of the reasoning, background and theory — let’s go through a practical example of how to use the service to handle some images in your own application.

For the purposes of this tutorial, we’re going to implement a user profile photo feature for an arbitrary application. We’ll use TransloadIt to do the following:

1. Intercept the file upload, uploading it not to your application but to their servers.
2. Perform server-side file checking, to ensure it meets certain criteria, such as whether it is indeed an image.
3. Create a number of different derivatives — e.g. sizes — of the uploaded image, such as various sized thumbnails along with a “medium” and a “large” version for user profile pages.
4. Transfer the resulting derivatives to an Amazon S3 bucket.
5. Display a thumbnail of the newly-uploaded image in our application.
6. Use the information returned from TransloadIt to let our application know where to find the resulting images, so that we can store a reference to them in the user records.

The first step is to build some templates containing assembly instructions.

Getting Started with Templates

A template contains assembly instructions in JSON format. Fire up your favourite text editor, begin some JSON:

{

}

… and let’s dive in.

Filtering Files

First we’ll add a step which utilizes the /file/filter robot to check the uploaded file’s MIME type, to ensure that it is an image. Add the following to your empty JSON document:

"steps":
  "files": {
    "robot": "/file/filter",
    "accepts": [
      [
        "${file.mime}",
        "regex",
        "image"
      ]
    ],
    "error_on_decline": true
  },

Let’s break this down.

We’re beginning with a step identified by the key files. You can call it what you want, but files makes sense in most cases.

Next we’re telling TransloadIt to use the /file/filter/ robot, which is used to perform some checks on the incoming files. In this case, we’re telling it what we want to accept; we ask it to extract the file’s MIME type and run a regular expression on it (image).

In TransloadIt instructions, a variable is denoted using a dollar sign and curly brackets ${}. When specifying a regular expression, you only need specify the main body of it.

Should the test we’re setting fail, the error_on_decline parameter ensures an error is thrown rather than continuing to the next step.

In other words, we’re telling TransloadIt to reject any files which aren’t images.

Adding Resizing Steps

Now let’s create three more steps, each doing much the same thing — creating a derivative (that is, a particular size) of the incoming images.

We can call the steps whatever we like, so we’ll use names which provide some context to the derivatives — medium, large and thumbnail.

Let’s define the first of these steps:

"medium": {
  "use": ":original",
  "robot": "/image/resize",
  "width": 300,
  "height": 200,
  "resize_strategy": "fit"
},

Here we define a step named medium, which utilises the /image/resize robot. This takes a number of parameters, many of them optional, which are documented here.

The use parameter tells it to resize the original file.

In this case we’re providing the required dimensions — 300 by 200 pixels — as well as specifying a resize strategy. The available resize strategies are documented here, but essentially fit ensures that the image is resized to fit the specified dimensions, while preserving the aspect ratio.

The large step is virtually identical:

"large": {
  "use": ":original",
  "robot": "/image/resize",
  "width": 480,
  "height": 320,
  "resize_strategy": "fit"
},

Then the thumbnail step:

"thumbnail": {
  "use": ":original",
  "robot": "/image/resize",
  "width": 80,
  "height": 80,
  "resize_strategy": "crop"
},

This time we’re using the crop strategy to ensure we end up with an image which is perfectly square.

To make the process more efficient, there’s no reason why you can’t set the use parameter to tell TransloadIt to base the thumbnail on your already-processed large or medium versions.

At this stage, we’ve ensured that we’re dealing with an image, and we’ve resized it three times over to create three separate image derivatives. Next up, we’ll tell TransloadIt what to do with the newly-created derivatives.

Export

As noted previously, Transloadit won’t store our files for long — hosting isn’t what the service is all about — so we need to move the files somewhere more permanent.

We’re going to use the /s3/store file export robot to upload the files to an Amazon S3 bucket.

Here’s how you might configure that step:

"export": {
  "use": [
    "medium",
    "large",
    "thumbnail"
  ],
  "robot": "/s3/store",
  "path": "users/profiles/${fields.username}_${previous_step.name}.${file.ext}",
  "key": "YOUR-S3-AUTH-KEY",
  "secret": "YOUR-S3-AUTH-SECRET",
  "bucket": "YOUR-BUCKET-NAME"
}

You’ll need to replace your S3 credentials and bucket name with your own.

This is slightly more complex than our previous steps, so let’s break it down.

The use parameter tells the robot to run this step for each of our resized images, thus generating three files on S3 for every upload. As you can see, medium, large and thumbnail match the identifiers of our three previous steps.

Then we specify the key — S3’s term for a path — used to store the resulting files using the path configuration value. This, combined with the fully-qualified domain name of the bucket, later becomes the URI of the resulting derivative images.

In the example above, we’re using the following pattern:

users/profiles/${fields.username}_${previous_step.name}.${file.ext}

This pattern begins by prefixing the path with users/profiles/, then uses the value of a hidden form field named username which we’ll define shortly. It then concatenates that with the key which defines the previous step, which is the name of our derivatives. Finally, it adds the extension of the original file, which is accesible via the variable ${file.ext}.

That’s quite a mouthful, so perhaps it’s best illustrated with an example. Given a username of bob, this pattern will produce the following three paths:

users/profiles/bob_medium.jpg
users/profiles/bob_large.jpg
users/profiles/bob_thumbnail.jpg

There are all sorts of naming strategies you can employ, by chopping and changing the variables available to you. Foe example, consider the following pattern:

users/profiles/${fields.username}${file.meta.width}x${file.meta.width}.${file.ext}

This dynamically constructs a filename by concatenating the username, the resulting file’s width and height, and finally the file’s extension. This will result in something like this:

users/profiles/bob480x320.jpg

Note that if an image is smaller than a derivative’s target dimensions, these values will reflect the final image, rather than the configured dimensions.

To simply use the original filename:

${file.name}

To ensure uniqueness, the following variable provides a unique 32-character prefix:

${unique_prefix}

For a full list of the available variables, refer to the section in the documentation on assembly variables.

Uploading Templates

Putting all of these steps together, our assembly instructions — which make up our template — look like this:

{
  "steps": {
    "files": {
      "robot": "/file/filter",
      "accepts": [
        [
          "${file.mime}",
          "regex",
          "image"
        ]
      ],
      "error_on_decline": true
    },
    "medium": {
      "use": ":original",
      "robot": "/image/resize",
      "width": 300,
      "height": 200,
      "resize_strategy": "fit"
    },
    "large": {
      "use": ":original",
      "robot": "/image/resize",
      "width": 480,
      "height": 320,
      "resize_strategy": "fit"
    },
    "thumbnail": {
      "use": ":original",
      "robot": "/image/resize",
      "width": 80,
      "height": 80,
      "resize_strategy": "crop"
    },
    "export": {
      "use": [
        "medium",
        "large",
        "thumbnail"
      ],
      "robot": "/s3/store",
      "path": "users/profiles/${fields.username}_${previous_step.name}.${file.ext}",
      "key": "YOUR-S3-AUTH-KEY",
      "secret": "YOUR-S3-AUTH-SECRET",
      "bucket": "YOUR-BUCKET-NAME"
    }
  }
}

Enter your own S3 credentials in the appropriate place, and then we’re ready to upload the template to TransloadIt.

You’ll find the JSON above in the sample code repository which accompanies this tutorial, in a file named template.json.

If you haven’t already created an account with TransloadIt, you’ll need to do so now.

You’ll need to be logged in; then go to your dashboard (My Account). Under Integrations in the left sidebar, select Templates. Then click the New button in the top-right hand corner.

You’ll be asked to provide a name to identify your template — something like user_avatars should do just fine. Then paste in the JSON above (which you’ll also find at the root of the repository which accompanies this article) in — ensuring you’ve replaced the dummy S3 values with your own — and hit Save.

If you’d rather use an alternative storage mechanism, such as (S)FTP or Rackspace Cloud Files, you’ll find the relevant documentation here — just modify the final step accordingly.

You’ll be taken back to the main Templates file, and you’ll notice that the newly created template has been assigned a hash as a unique ID. Make a note of this, because you’ll need it later on.

With that done, let’s take a look at how you use TransloadIt from your application.

The Example Application

You’ll find an example application to accompany this tutorial on Github.

In order to run it, you’ll need to ensure you have the following pre-requisites installed:

• Node.js
• npm
• MongoDB
• Bower

Vagrant users will find a Vagrantfile in the repository for creating a VM which includes all of the listed dependencies.

Essentially it’s a simple Express application. There are a number of things we won’t cover here for brevity:

• It uses the config module to hold the application’s configuration in a .yaml file.
• It uses Mongoose with MongoDB to define a User model.
• It uses Passport with the Local strategy to provide a simple authentication mechanism.
• It provides middleware to securely hash passwords.
• It includes some simple middleware to restrict certain routes to authenticated users only.
• It uses Handlebars, along with the handlebars-layouts package to handle templating.

To get started, clone the application, then install the dependencies:

npm install
bower install

There are a couple of elements in the application worth covering briefly.

Here is the schema definition for the User model:

var userSchema = mongoose.Schema({
  username : { type: String, required: true, unique: true },
  email    : { type: String, required: true, unique: true },
  password : { type: String, required: true },
  avatar   : { type: mongoose.Schema.Types.Mixed, required: false }
});

Notice how we include an avatar field of type Mixed. This will allow us to specify the avatar as a hash, for example:

user.avatar = {
  thumbnail : 'http://your.bucket.name.aws.amazon.com/user/profile/bob_thumbnail.jpg',
  medium    : 'http://your.bucket.name.aws.amazon.com/user/profile/bob_medium.jpg',
  large     : 'http://your.bucket.name.aws.amazon.com/user/profile/bob_large.jpg'
};

Now that the basic structure is in place, let’s look at TransloadIt’s jQuery plugin.

The jQuery Plugin

The easiest way to integrate with TransloadIt on the client-side is to use the official jQuery plugin, although there are other alternatives, as we’ll see later in the article.

The latest version of the plugin is avaialble via the following URL:

https://assets.transloadit.com/js/jquery.transloadit2-latest.js

The minimal integration involves the following steps:

• You bind the plugin to your form
• The plugin “hijacks” the form submission, sending files directly to Transloadit
• The plugin waits until the files have been uploaded AND processed
• Transloadit returns a JSON object with the results, which will also include the URLs to the newly-generated files
• It creates a hidden textarea element containing the JSON from Transloadit
• The form is submitted to your application

Here’s a very simple example of initializing the plugin, telling it to use a template:

$(function() {
  $('#upload-form').transloadit({
    wait: true,
    params: {
      auth: {
        key: 'YOUR-AUTH-KEY'
      },
      template_id: 'YOUR-TEMPLATE-ID'
    }
  });
});

However as we noted in the previous part, exposing your authentication credentials in your client-side code is a bad idea. Instead, we’ll use a signature.

Signatures

Signatures are a more secure alternative to using authentication tokens, though they require some server-side work.

In essence using signatures requires that instead of sending a bunch of instructions to TransloadIt from your client-side app, you encode the instructions and encrypt them using the HMAC algorithm in conjunction with your private authentication key. A temporary token — i.e. signature — is generated as a result, which is restricted to that particular combination of instructions. Because it’s temporary, then if that token becomes compromised then it will very quickly become useless.

You needn’t worry about the ins-and-outs of generating the signature yourself, since we can use a third-party library to handle the process. If you’re using Node.js, the official SDK will take care of it for you.

To install the library:

npm install transloadit --save

You’ll need your Authentication Key and Authentication Secret, which you can obtain from the API Credentials section on the TransloadIt site. Put them in the relevant part of config\default.yaml.

You’ll need to create the default configuration file by renaming or copying RENAME_THIS_TO_default.yaml to default.yaml.

Now create an instance of the TransloaditClient class, providing it with your authentication details:

var TransloaditClient =   require('transloadit');
var transloadit       =   new TransloaditClient({
      authKey     : config.transloadit.auth_key,
      authSecret  : config.transloadit.auth_secret
    });

Next, define the parameters for the action you want to take. That can either be in the form of a set of assembly instructions:

var params = {
  steps: {
    // ...
  }
};

Or, in our case, we simply provide the ID of our template:

var params = {
  template_id: 'YOUR-TEMPLATE-ID'
};

To create the signature:

var sig = transloadit.calcSignature(params);

This results in a hash containing a signature — an access token, of sorts — as well as the parameters you’ll need to call the service. So our sig object will look something like this:

{
  signature: "fec703ccbe36b942c90d17f64b71268ed4f5f512",
  params: {
    template_id: 'YOUR-TEMPLATE-ID',
    auth: {
    	key: 'idfj0gfd9igj9dfjgifd8gfdj9gfdgf',
    	expires: '2015-06-25T10:05:35.502Z'
    }
  }
}

In order to pass this to our Handlebars templates so that our JavaScript can utilise it, we need to create a very simple helper:

app.engine('.hbs', exphbs(
  {
    extname: '.hbs',
    defaultLayout: 'default',
    helpers : {
      json : function(context) {
        return JSON.stringify(context);
      }
    }
  }
));

Let’s now put this together to define the account route, which will include our avatar upload form:

// The account page
app.get('/account', ensureAuthenticated, function(req, res){

  // Require the TransloadIt client
  var TransloaditClient = require('transloadit');
  
  // Create an instance of the client
  var transloadit       =   new TransloaditClient({
    authKey     : config.transloadit.auth_key,
    authSecret  : config.transloadit.auth_secret
  });

  // Build the Transloadit parameters...
  var params = {
    template_id 	: 	config.transloadit.template_id
  };

  // ...and generate the signature
  var sig = transloadit.calcSignature(params);  

  return res.render('account', {
    user: req.user,
    sig : sig
  });
});

Then, in the corresponding template (views/account.hbs), let’s start with some very simple HTML:

<h2>Hello, {{ user.username }}</h2>

{{# if user.avatar }}
<img src="{{ user.avatar.thumbnail }}" id="avatar">
{{else}}
<img src="/avatar.png" id="avatar">
{{/if}}

<form method="POST" action="/avatar" id="avatar-form">
  <input type="file" name="image" id="avatar-upload">
  <input type="hidden" name="username" value="{{user.username}}">
</form>

Note that we’re including a hidden field which contains the username. We’re going to send this to TransloadIt with our request, so that it can be used in our templates.

Now add the JavaScript, beginning with some variable initialization using our json Handlebars helper:

var sig = {{{ json sig }}};

Now we’ll bind the TransloadIt plugin to the upload form:

$(function() {
  $('#avatar-form').transloadit({
    wait: true,
    params: JSON.parse(sig.params),
    signature: sig.signature,
    fields: true,
    triggerUploadOnFileSelection: true,
    autoSubmit: false,
    onSuccess: function(assembly) {
      $('img#avatar').attr('src', assembly.results.thumbnail[0].url + '?' + (new Date()).getTime() );
      var derivatives = {
        thumbnail : assembly.results.thumbnail[0].url,
        medium : assembly.results.medium[0].url,
        large : assembly.results.large[0].url
      };
      $.ajax({
        type: 'post',
        url: '/avatar',
        data: derivatives,
        success: function(resp){
          console.log(resp);
        }
      })
    }
  });
});

This is more complex than the minimal integration initialization we looked at earlier, so let’s go through it a bit at a time.

We’re pulling in the params and signature from the sig variable, which we generated on the server and then encoded as JSON. Because the params part is nested, we use JSON.parse() to convert it back into an object, from which TransloadIt will extract the relevant parameters.

In the plugin initialization, wait is set to true, which means we wait until both the files have been uploaded and they’ve been processed.

Using Assembly notifications — which you can read about later in the Advanced Usage section — means that you won’t necessarily have to wait for the file to be processed, in which case you can set wait to false.

fields is set to true to tell the plugin that we want to include additional information when we send the files for processing; in our case that’s a hidden form field named username, which we populate with the authenticated user’s username.

triggerUploadOnFileSelection is used to send the file to Transloadit as soon as the user has selected a file, rather than when the form is submitted. autoSubmit prevents it from submitting the form once the result comes back from Transloadit, since we’re going to do that manually ourselves.

The onSuccess callback gets fired when the data comes back from Transloadit, which gives us a hash of data in assembly.

The assembly object contains a results property, which in turn contains properties for each of our “steps”. These contain an array of file objects. Since we’re only uploading one file, they’ll be arrays containing a single item. Each file object contains a number of properties including the original file name, meta information, unique IDs from Transloadit and other bits and pieces. To see the full range of information, you may wish to log it out into the console and take a look. However all we’re really interested in is the url property, which contains the URL of the generated image on S3.

Alternatively, you may wish to use the ssl_url property, which is identical to url but over HTTPS.

We’re simply extracting the three URLs by the corresponding derivative’s name, then creating a hash of the three derivatives and their corresponding URL’s.

To provide visual feedback to the user, we also grab the URL of the thumbnail and modify the avatar on the page to show the newly-uploaded image.

Finally, we use Ajax to POST that data silently back to our application.

Here’s the avatar route to capture that data:

// Ajax callback for setting the avatar
app.post('/avatar', ensureAuthenticated, function(req, res){
  req.user.avatar = req.body
  req.user.save(function(err) {
    if(err) {
      return res.send('error');
    }
    return res.send('ok');
  });
});

In production, you’ll probably want to sanitize and validate this.

As you can see, we take the hash of derivative images and their URLs, grab the currently authenticated user from req.user, set the avatar property to the provided hash and then update the user model.

This is just one possible approach. For quicker feedback, you might want to use the plugin’s onResult callback to obtain the thumbnail as soon as it’s been generated, rather than wait for all three derivatives. Instead of using an Ajax call from your client code to notify your server, you may instead prefer to use the Assembly notifications feature, which provides the additional benefit of running the assemblies in the background, rather than holding up execution on the client. Consult the plugin documentation for the full range of options.

That concludes our basic application. Don’t forget, all the source — including the authentication mechanism — is over on Github.

Advanced Usage

Before we wrap up, let’s just take a brief look at a couple of the more advanced aspects of TransloadIt.

Other Client-Side Options

You don’t have to use the provided jQuery plugin. In the Community Projects section of the documentation you’ll find a number of alternatives, including a plugin for Bootstrap, one for drag n’ drop, an Angular plugin or support for plain old XHR, among others.

The XHR one might be worth you looking at in more detail. It’s a bare-bones solution which offers plenty of flexibility, whilst requiring you to provide your own feedback — for example some sort of upload indicator. It’s also worth noting that once it has uploaded the file, it tries to determine when the assembly has been completed by polling the server at intervals of 1000ms.

Notifications

Rather than have users wait around for their uploads to be processed, you can use notifications to ping your application when the files are ready. Using this approach a user only need wait until the upload has completed.

Notifications are easy to implement from a consumer point-of-view; simply include a notify_url with your assembly instructions, for example:

{
  auth       : { ... },
  steps      : { ... },
  notify_url : "http://example.com/webhooks/incoming/transloadit"
}

When your URL gets pinged by Transloadit, the JSON provided will include a signature field which you can use to verify that the notification did indeed come from them. Simply decode the signature using your auth secret.

During development, you may wish to take advantage of this proxy package in order to test your assembly notifications, or use a tunnelling service such as ngrok.

Summary

In this two-part series we’ve taken a comprehensive look at TransloadIt, a file processing service.

In part one, we went through some of the advantages and disadvantages and then looked at the key concepts.

This part, we got our hands dirty and built a simple user avatar component using jQuery, Node.js and Express.

You’re not restricted to jQuery, and indeed you’re free to use a vanilla JavaScript solution or your favorite framework. You don’t even need to use it from a client-side application, and when it comes to server-side technologies you have a wide range of options. Hopefully, though, you’ve now got an appreciation of how it can be used for image-handling.

Are you using TransloadIt in your projects? Do you know of a better service? Let me know in the comments.

Frequently Asked Questions (FAQs) about User Avatar Component in Node.js with Transloadit