Key Takeaways
- TransloadIt is a file-processing service primarily used for processing images, videos, and audio, allowing various operations like resizing, optimizing, and format conversion. It enables file upload directly to their servers, bypassing your server, and transferring processed files straight to an Amazon S3 bucket, a Rackspace Cloud Files account, or your own server via (S)FTP.
- TransloadIt is beneficial as it allows you to focus on your application’s core functionality, reduces load on your servers, eliminates the need for additional libraries or programs on your server, and mitigates security risks associated with file uploads on your web server. However, it introduces an additional point-of-failure, entrusts sensitive files to a third party, and incurs financial costs.
- TransloadIt operates through assembly instructions that tell the service what actions to perform on a file. These instructions are performed by “robots”, which can import files from various sources, filter incoming files, perform tasks on image files, perform actions on video and audio files, and transfer the result of your assemblies to some form of permanent storage.
- TransloadIt offers a free sandbox account with a total of 1GB capacity and several paid plans. Assembly instructions are defined in JSON format or as a hash with various libraries, and are usually interacted with via the official jQuery plugin from your client application.
Transloadit is a file-processing service primarily designed for processing images, videos and audio.
It allows you to perform various operations — conducted by what it calls “robots” — on these files; for example resizing and optimizing images, extracting thumbnails from videos or converting files from one format to another.
TransloadIt also allows you to upload files directly to their servers for processing rather than via your application. Indeed you can bypass your server altogether, transferring the processed files straight to an Amazon S3 bucket, a Rackspace Cloud Files account or to your own server via (S)FTP. So in other words, your application need only be concerned with where to find the resulting files, without having to process or store them directly — great for running distributed applications or for those on clusters.
In this tutorial we’re going to take a detailed look at Transloadit, and in particular using it to handle images. We’ll look at some of the key concepts, such as constructing “assembly instructions” to tell it what we want to happen to our uploaded image files. We’ll “hijack” a file upload form to send a file directly to TransloadIt, run a set of instructions to generate a number of “derivatives” — different-sized images — and then save them to an Amazon S3 bucket. Finally, we’ll look at how we can use the results of these operations to notify our application about the newly generated files, so that we can update our database accordingly.
Before we look in detail at Transloadit, it’s worth taking a step back and discussing why you’d want to use a third-party service for something like this in the first place, rather than implement it all yourself.
Why Use a Third-Party Service?
Image handling isn’t all that difficult to implement yourself, and there are all sorts of libraries out there for just that. So why would you want to use a third-party service to handle it at all?
For one thing, it allows you to concentrate on what your application is really all about. Why spend additional time and effort on, say, user profile photo functionality when you could be concentrating your development efforts on mission-critical functionality?
Image processing can be quite resource intensive. In particular, resizing large images can eat up a lot of memory. Using a service such as Transloadit eases the load on your servers.
Processing files usually requires additional libraries or programs to be installed on your server; for example
gd
orImagemagick
for images orffmpeg
for videos and audio. By using a third-party service, there’s no need to worry about installing, configuring or maintaining them.Transloadit “jobs” can be run in the background, rather than hold up execution of your application. While there are solutions to get around this in your own application — for example queues such as RabbitMQ or Beanstalkd, or job servers such as Gearman, using TransloadIt frees you from the additional development effort and installation steps required to implement them.
The Transloadit servers are highly optimized to handle file uploads, so performance may well be much better than if you implement them yourself.
Permitting files to be uploaded to your web server also has all sorts of security implications which you must take into consideration. By using a third-party service, these risks are somewhat mitigated.
It’s decentralized. You can set it up such that uploaded files go via the service and straight to an external storage solution such as S3, without ever having to be stored on your web server. Whilst this might not seem a huge advantage in a small-scale application, you’ll reap the benefits as soon as it grows to the point where you have multiple services, for example a cluster.
Depending on your application, your preferred hosting solution, your expertise or the amount of time you happen to have on your hands, chances are not all of these will apply. Perhaps none; it’s not the optimum solution for all cases.
Naturally, there are disadvantages. Here are a few:
- It introduces an additional point-of-failure.
- It could mean entrusting sensitive files to a third-party.
- Finally, there’s the financial cost.
Bearing all of those pros and cons in mind, let’s start taking a detailed look at the service. Then, in part two, we’ll run through a practical example of how to use it.
Getting Started with Transloadit
Transloadit offer a free sandbox account that gives you a total of 1GB capacity to play around with — which will be more than enough to follow along with this tutorial, and then some. A number of paid plans are available, which give you varying amounts of capacity per month. Paid plans start at $19 per month, at time of writing.
The actual definition of capacity varies according to what you’re using the service for. To illustrate what that means in the context of image processing, you can determine the usage involved in processing one particular image upload by adding the file size of the original file, to the total size of the resulting processed images. So suppose you upload an 800kb image, which in turn is used to generate a large version of 400Kb, a medium version of 200Kb and a thumbnail of 30Kb. If you use the smallest possible version as the input for the next resize, then that would use (800Kb + 400Kb) + (400Kb + 200Kb) + (200Kb + 30Kb) = 2,030Kb towards your limit. These calculations become more complex when you start working with videos or audio, so you’ll probably want to consult their website.
If you plan to follow along and go through the practical example in part two, now might be a good time to sign up. Head on over to the site and hit “Get Started”.
Now let’s look at some of the concepts and the components that make up the service, in order to better understand the process.
The Concepts
Assemblies
An assembly is essentially a set of instructions which tells the Transloadit service what actions you want them to carry out on a particular file (or group of files).
In practice, you’d probably create different sets of assembly instructions for different types of files. Thereafter, you’d probably create subtly different assembly instructions for sub-types. For example, user profile pictures / avatars might require one particular set of sizes, and profile banners an altogether different set; so in this scenario you’d create a set of assembly instructions for each.
Assembly instructions contain a list of steps, defining precisely what processing to carry out and in what order. Each step is performed by a robot.
Robots
A “robot” is like a worker, which performs some sort of processing. There are a number of different types of robot:
Import Robots can take files from a variety of sources, and not just via form-based file uploads. For example, you can create a robot which imports files over HTTP, using (S)FTP or by downloading them from an Amazon S3 bucket.
Assembly flow logic robots can be used to filter incoming files — for example, you can set up a robot to reject files which are not images.
Image robots perform a number of different tasks on image files, most notably resizing. There are also robots for optimizing image files or taking screenshots.
Video and audio robots perform actions on those particular file types. Because this tutorial is focused on images, we won’t be looking at these in any great detail – but stay-tuned for a follow-up article later.
File Export robots are used to transfer the result of your assemblies to some form of permanent storage. It’s important to note that TransloadIt don’t host your files; although they make them available for download via S3 buckets after processing, they won’t be stored beyond 24 hours. It’s your responsibility to ensure your files are stored somewhere that you control. You can set up a robot to transfer your files to an Amazon S3 bucket, to Rackspace Cloud Files, or transfer them to your server using (S)FTP.
Building Assembly Instructions
Assembly instructions are defined in JSON format, or as a hash which you can use with one of the various available libraries. Most of the time you’ll probably use the official jQuery plugin to interact with the service from your client application, but there are other options available. You needn’t necessarily use it from a client-side application — there are server-side libraries available as well.
In part two, we’re going to look at how you construct assembly instructions in detail.
Templates
Assembly instructions tell the robots what to do with your files, but also include any additional information they’ll need to run them. This is a real issue when it comes to the file export steps in particular, since they’ll require credentials with which they can access your S3 bucket or (S)FTP service. Clearly defining these in client-side code would be disastrous, so for that reason you’d use templates.
Rather than define your assembly instructions when you call the TransloadIt API — for example, when using the jQuery plugin — you can instead create a JSON document containing the relevant assembly instructions on TransloadIt’s server, and then reference that template by its unique id. Templates are encrypted server-side for additional security.
You can also merge additional parameters, or override existing ones, into a template at runtime — all the while, keeping your sensitive information such as S3 credentials safe from prying eyes.
Signatures
Signatures get around the problem of exposing your TransloadIt authentication credentials in client-side code, in much the same way as templates protect your storage-related credentials.
A signature is essentially a short-lived access token, which you’d generate on the server, pass through to your client application and then use to authenticate with the service. If someone should obtain that value, then it would very quickly become useless. Signatures are also “bound” to a specific set of assembly instructions, or one particular template, making them even more secure.
Variables
You can use variables in your assembly instructions. These are particularly useful when determining file names of processed files. So for example, if you’re resizing an uploaded user avatar, you can provide the user id or username to form part of the resulting file name. Later we’ll see how the names of the various steps you can define, can also be used as variables.
Using Form Fields
When you’re using TransloadIt from a client application (e.g. an upload form) then variables can be derived from the form fields. So in the example above for uploading user avatars, you could place the user id in a hidden form field and then incorporate that into the instructions you send to TransloadIt’s servers.
Summary
In this first part, we’ve taken a high-level look at the TransloadIt service. We’ve looked at what it can do, and some of the pros and cons of using such a service to perform the sort of tasks that you might otherwise implement yourself.
We’ve looked at some of the concepts you need to understand in order to get the most out of TransloadIt and with that background knowledge, we’re ready to start building a practical example. Stay tuned for part two.
Frequently Asked Questions about Image Uploads and Manipulation with Transloadit
What is Transloadit and how does it work?
Transloadit is a versatile service that allows users to handle file uploads and manipulate data in various ways. It works by providing a robust API that developers can integrate into their applications. This API allows for the uploading, processing, and storing of files, including images, videos, and documents. Transloadit supports a wide range of operations, such as resizing images, encoding videos, and extracting metadata from files. It’s a cloud-based service, which means it can scale to handle large volumes of data and doesn’t require any server-side setup.
How can I integrate Transloadit into my application?
Integrating Transloadit into your application involves using the Transloadit API. This API provides a set of endpoints that you can use to upload and process files. You’ll need to send HTTP requests to these endpoints from your application. Transloadit provides SDKs for various programming languages, including JavaScript, Python, and Ruby, which can simplify the integration process.
What types of file operations does Transloadit support?
Transloadit supports a wide range of file operations. For images, it can resize, crop, and apply filters. For videos, it can encode, cut, and add watermarks. It can also extract metadata from files, convert documents to different formats, and much more. The exact capabilities depend on the specific operation and the type of file.
How does Transloadit handle security and privacy?
Transloadit takes security and privacy very seriously. All data transfers are encrypted using SSL/TLS, and files are stored securely in the cloud. Transloadit also complies with GDPR and other data protection regulations. Users have control over their data and can delete it at any time.
Can Transloadit handle large volumes of data?
Yes, Transloadit is designed to handle large volumes of data. It’s a cloud-based service, which means it can scale to meet the demands of your application. Whether you’re processing a few files or millions, Transloadit can handle it.
What programming languages does Transloadit support?
Transloadit provides SDKs for several programming languages, including JavaScript, Python, Ruby, Java, and PHP. This means you can integrate Transloadit into your application regardless of the language you’re using.
How much does Transloadit cost?
Transloadit offers several pricing plans, ranging from free to enterprise-level. The cost depends on the amount of data you need to process and the features you need. You can find more information on the Transloadit website.
Can I use Transloadit for video processing?
Yes, Transloadit supports a wide range of video processing operations. You can encode videos, cut them, add watermarks, and much more. Transloadit can handle videos in various formats, including MP4, AVI, and MOV.
Does Transloadit offer customer support?
Yes, Transloadit offers customer support. You can contact them via email or through their website. They also provide extensive documentation and a community forum where you can ask questions and get help from other users.
Can I use Transloadit on mobile devices?
Yes, Transloadit can be used on mobile devices. You can integrate the Transloadit API into your mobile application, allowing you to upload and process files directly from the device. Transloadit supports both iOS and Android.
Lukas is a freelance web and mobile developer based in Manchester in the North of England. He's been developing in PHP since moving away from those early days in web development of using all manner of tools such as Java Server Pages, classic ASP and XML data islands, along with JavaScript - back when it really was JavaScript and Netscape ruled the roost. When he's not developing websites and mobile applications and complaining that this was all fields, Lukas likes to cook all manner of World foods.