Better File Uploads with Dragonfly

June 18, 2014

libellula tattoo

File uploading is a vital part of the web. We upload our photos, videos, and music to share them. We send e-mails with a bunch of attachments; probably every active Internet user uploads something at least once a day (well, that’s my guess :)).

You’ve probably been given a task to create a system to allow users to upload files as a part of an app. If not – well, it will sooner or later :).

I am going to introduce you Dragonfly – a popular gem used for handling images and other attachments. Some of the features that will be covered are:

Integration with ImageMagick, image processing and analysis.
How to upload images to Amazon S3 instead of the local file system.

As a bonus, this can be done fairly quickly! So, let’s start, shall we?

The working demo can be found at http://sitepoint-dragonfly.herokuapp.com/.

The source code is available on GitHub.

Dragonfly and Alternatives

There are other solutions similar to Dragonfly that you might want to use in your projects. The most popular are Carrierwave and Paperclip. These are nice solutions and they are used on many websites.

What goodies does Dragonfly have and why should you try using it?

It is really easy to get started.
It integrates nicely with ImageMagick.
Ability to work with “magic” columns that can automatically store information about the attachment.
It allows the specification of validations and callbacks
It works with a wide range of data stores or allows you to create your own.
It has a framework for creating custom image analysers and generators, as well as plugins.
It has a bunch of extensions (every extension is provided in its own gem).
It works with Rails 3 and 4 as well as with Ruby 1.8.7, 1.9.2, 1.9.3, 2.0.0, jRuby 1.7.8 and Rubinius 2.2.

If you are not familiar with this gem, I hope that I’ve convinced you to (at least) take a look at it :)

Please note that, if you are still using Dragonfly 0.9 or earlier there is a new Dragonfly 1.0 that has some breaking changes. There is a detailed guide explaining how to migrate to the latest version – I was able to migrate in no time.

Preparing the Project

I will use Rails 4.1.1 for this demo, but you can implement the same solution with Rails 3

Let’s create a simple web app that will allows users to share (upload) their photos. For the first iteration, the app will be a very basic system that allows uploading of any kind of file. That will be followed by adding validations, integration with Amazon S3, and, finally, some optimizations.

Okay, create a new app without the default testing suite:

rails new uploader -T

Open your Gemfile and add the following gems:

Gemfile

gem 'dragonfly'
gem 'dragonfly-s3_data_store'
gem 'bootstrap-sass'
group :production do
  gem 'rack-cache', :require => 'rack/cache'
end

dragonfly is a highly customizable Ruby gem for handling images and other attachments created by Mark Evans, as well as our main subject today.

dragonfly-s3_data_store is an extension for dragonfly that allows storing files using Amazon S3 cloud services (we are going to talk about that later).

bootstrap-sass a gem that brings in Twitter Bootstrap. This if for styling purposes and not necessary to use Dragonfly

rack-cache is a simple solution for caching in production
environment that you can use for small projects (for large ones you should implement a reverse caching proxy).

Do not forget to bundle:

bundle install

Now, it is time to hook up some Bootstrap files

application.css.scss

@import 'bootstrap';

and create a PhotosController that will be empty for now:

photos_controller.rb

class PhotosController < ApplicationController
end

Integrating with Dragonfly

Generate the basic Dragonfly configuration:

rails generate dragonfly

It will create Dragonfly’s initializer file (that is located at config/initializers/dragonfly.rb).

The next thing we need is a table to store info about the uploaded photos. For now, this table will have only two columns (not to mention default ones – id, created_at and updated_at):

title (string) – title provided by the user.
image_uid (string) – path to the uploaded photo, generated by Dragonfly.

These commands will create and apply the migration:

rails g model Photo image_uid:string title:string
rake db:migrate

Now open your newly-generated model. We need to integrate it with Dragonfly:

photo.rb

[...]
dragonfly_accessor :image
[...]

Please note that your table must have a image_uid column if you specify dragonfly_accessor :image in the model. If, for example, you wish to call your attachment avatar, then your column needs to be named avatar_uid. The second thing to note is that you can ask Dragonfly to automatically store the original filename in a so-called “magic” column. In order to do that, add an image_name column with a string type to your table.

By default, Dragonfly integrates with ImageMagick, a very cool image processing library that allows the application of various modifications to the images. As long as we are going to use ImageMagick’s features, you should install it by choosing an appropriate method on this page. If you are not going to use ImageMagick for your project, comment out (or remove) this line:

config/initializers/dragonfly.rb

[...]
plugin :imagemagick
[...]

IF you are working with Rails 4, then open the production environment file and uncomment this line to enable rack cache:

config/environment/production.rb

config.action_dispatch.rack_cache = true

Cool! At this point we’ve set up Dragonfly to work with our app and are ready to move to the next part.

Uploading Photos

It is high time to flesh out the controller. We will add only a couple of simple methods:

photos_controller.rb

class PhotosController < ApplicationController
  def index
    @photos = Photo.all
  end

  def new
    @photo = Photo.new
  end

  def create
    @photo = Photo.new(photo_params)
    if @photo.save
      flash[:success] = "Photo saved!"
      redirect_to photos_path
    else
      render 'new'
    end
  end

  private

  def photo_params
    params.require(:photo).permit(:image, :title)
  end
end

If you are on Rails 3, then you won’t have the photo_params method. Instead, add this line to your model file:

photo.rb

[...]
attr_accessible :image, :title
[...]

Please note that you do not need to specify image_uid here – Dragonfly will take care of it.

Don’t forget to set up the routes:

routes.rb

[...]
resources :photos, only: [:new, :create, :index]
root to: 'photos#index'
[...]

The last point in our checklist is the views. Starting with the index:

index.html.erb

<h1>List of photos</h1>

<%= link_to 'Add your photo', new_photo_path, class: 'btn btn-lg btn-primary' %>

new.html.erb

<h1>New photo</h1>

<%= form_for @photo do |f| %>
  <div class="form-group">
    <%= f.label :title %>
    <%= f.text_field :title, class: 'form-control', required: true %>
  </div>

  <div class="form-group">
    <%= f.label :image %>
    <%= f.file_field :image, required: true %>
  </div>

  <%= f.submit 'Submit', class: 'btn btn-primary btn-lg' %>
<% end %>

At this point, you can launch your server and try to upload some files. The only problem is that you won’t see them on the main page, so let’s fix that:

index.html.erb

<h1>List of photos</h1>

<%= link_to 'Add your photo', new_photo_path, class: 'btn btn-lg btn-primary' %>

<ul class="row" id="photos-list">
  <% @photos.each do |photo| %>
    <li class="col-xs-3">
      <%= link_to image_tag(photo.image.thumb('180x180#').url, alt: photo.title, class: 'img-thumbnail'),
                  photo.image.url, target: '_blank' %>
      <p><%= photo.title %></p>
    </li>
  <% end %>
</ul>

Apply some styles as well:

application.css.scss

#photos-list {
  padding: 0;
  margin: 0;
  margin-top: 30px;
  clear: both;
  li {
    list-style-type: none;
    padding: 0;
    margin: 0;
    max-height: 250px;
    margin-bottom: 10px;
    text-align: center;
    p {
      margin: 0;
      margin-top: 10px;
      height: 40px;
      overflow: hidden;
    }
  }
}

Note the use of thumb('180x180#') method in the view. Thanks to ImageMagick, we can crop our images any way we like. The # symbol means that we wish to keep the central part of our image. If we specified 180x180#ne, then the north-east part of the image would remain. There are many other options that you can pass (for example, whether the aspect ratio should be maintained) – read about them here.

image.url provides the url to the original image. In our case, this image will open when clicking on the thumbnail.

Adding Validation

Right now, our users can upload any file – not only images. Moreover, these files can be any size. In most cases, this is not a good thing and you are probably wondering if there is a way to add some validations to the images. Of course!

Dragonfly equips our models with the required methods in the initializer:

# Add model functionality
if defined?(ActiveRecord::Base)
  ActiveRecord::Base.extend Dragonfly::Model
  ActiveRecord::Base.extend Dragonfly::Model::Validations
end

All we need to do is modify the model:

photo.rb

validates :title, presence: true, length: {minimum: 2, maximum: 20}
validates :image, presence: true

validates :image, presence: true
validates_size_of :image, maximum: 500.kilobytes,
                  message: "should be no more than 500 KB", if: :image_changed?

validates_property :format, of: :image, in: [:jpeg, :jpg, :png, :bmp], case_sensitive: false,
                   message: "should be either .jpeg, .jpg, .png, .bmp", if: :image_changed?

As you can see, our validations ensure the title is no less than 2 and no more than 20 characters . Also, image is required and its size (no more than 500 KB) and format (only “.jpg”, “.png”, “.bmp”) are validated. We are also using the image_changed? method to make sure that these validations fire only if the file was changed.

Dragonfly allows you to specify other validations. For example, you might want to check that the image’s width is no more than 400px and even provide a message about the actual width:

validates_property :width, of: :image, in: (0..400),
                           message: proc{ |actual, model| "Unlucky #{model.title} - was #{actual}" }

Image Processing and Analysis

Suppose we would like all uploaded files to be converted to JPEG. Dragonfly provides us with a convenient way to do that: a callback.

In our model:

photo.rb

[...]
dragonfly_accessor :image do
  after_assign { |img| img.encode!('jpg', '-quality 80') }
end
[...]

The above code converts all images to JPEG with the quality of 80 (note that if someone tries to upload a PNG file with transparency, that transparency will be lost).

If you are allowing your users to upload files that are not images, you will end up with an error in this callback. Fortunately, there is an easy way to check if an uploaded file is an image:

dragonfly_accessor :image do
   after_assign do |img|
    img.encode!('jpg', '-quality 80') if img.image?
   end
 end

You can do much more. For example, rotate the uploaded image:

dragonfly_accessor :image do
   after_assign do |img|
    img.rotate!(90) # 90 is the amount of degrees to rotate
   end
 end

Read more about Dragonfly’s callbacks here.

There are a handful of other methods to get information about the image. For example: width, height, aspect_ratio and others (full list is here).

Remember that all these methods are analyzing the image on the fly. There is a way to persist this info with the help of “magic” columns. Let’s, for instance, store the image size. To do that, create and apply this migration:

rails g migration add_image_size_to_photos image_size:integer
rake db:migrate

Then, tweak the view a bit:

index.html.erb

[...]
<p><%= photo.title %></p>
<small><%= number_to_human_size photo.image.size %></small>
[...]

Now, upload a new image and observe the result. Its size will be stored in the image_size column automatically! Pretty awesome, isn’t it?

Using Another Data Store

Dragonfly will store all uploaded images on your local hard drive inside the public/system/dragonfly directory. By the way, don’t forget to add this directory to the .gitignore file, otherwise all your local images will be pushed to the server when deploying (unless that is what you want).

To change this behavior, modify the config/initializers/dragonfly.rb file. For this demo, we are using Amazon S3 cloud file storage, but there are some other options available:

Dropbox by Daniel Leavitt
Cloudinary by Anton Dieterle
Couch by Mark Evans

There are others, but they seem to be outdated and do not work with Dragonfly 1+. You can even create you own data store.

To integrate with Amazon S3, add dragonfly-s3_data_store to the Gemfile (we already did that). A cool thing about Dragonfly is that changing the data store is very easy. All you have to do is to change the initializer file. You need, of course, an Amazon AWS account, which you can create here. When you’re done open your AWS management console, find S3 in the list and click on it.

Create a new bucket (read more about bucket naming and perfomance here) and remember its name.

Now open “YourAccount – Security Credentials” (in the right-top corner) and expand the “Access Keys” section. Create a new key pair and download it (you will have no option to download it later!)

Change the initializer file (the full list of available options can be found here):

config/initializers/dragonfly.rb

require 'dragonfly/s3_data_store'

[...]

url_format "/media/:job/:name"

if Rails.env.development? || Rails.env.test?
  datastore :file,
            root_path: Rails.root.join('public/system/dragonfly', Rails.env),
            server_root: Rails.root.join('public')
else
  datastore :s3,
            bucket_name: YOUR_BUCKET_NAME,
            access_key_id: YOUR_S3_KEY,
            secret_access_key: YOUR_S3_SECRET,
            url_scheme: 'https'
end
[...]

Be very careful with your key pair and never expose it to anyone. If you are deploying on Heroku, use its environment variables to store the key pair. It should not be present in version control!

Now all users’ files will be uploaded to Amazon in production. If you want to provide a link to those files, modify the index view as follows:

index.html.erb

<%= link_to image_tag(photo.image.thumb('180x180#').url, alt: photo.title, class: 'img-thumbnail'),
                  photo.image.remote_url, target: '_blank' %>

Note the use of remote_url instead of just url. It will lead to the file on Amazon.

A Bit of Optimization

Inside index.html.erb we are using the thumb method to display a thumbnail. That is okay, but thumbnail generation happens on the fly. We could have many images on one page and this generation happens for every image. How about trying to optimize that?

This issue can be solved quite easily. After the first on the fly processing, save the resulting thumbnail (to an Amazon storage in our case) and serve it directly for subsequent requests. Obviously, we need to store a reference to this thumbnail in the table. Let’s create that table:

rails g model Thumb uid:string job:string
rake db:migrate

Now add this code to your initializer:

config/initializers/dragonfly.rb

[...]
# Override the .url method...
define_url do |app, job, opts|
  thumb = Thumb.find_by_job(job.signature)
  # If (fetch 'some_uid' then resize to '180x180') has been stored already, give the datastore's remote url ...
  if thumb
    app.datastore.url_for(thumb.uid, :scheme => 'https')
    # ...otherwise give the local Dragonfly server url
  else
    app.server.url_for(job)
  end
end

# Before serving from the local Dragonfly server...
before_serve do |job, env|
  # ...store the thumbnail in the datastore...
  uid = job.store

  # ...keep track of its uid so next time we can serve directly from the datastore
  Thumb.create!(
      :uid => uid,
      :job => job.signature   # 'BAhbBls...' - holds all the job info
  )                           # e.g. fetch 'some_uid' then resize to '180x180'
end
[...]

As you can see, we are storing the thumbnail on the server and saving a reference to it inside the thumbs table. We also redefine the url method so that it tries to search our storage for an already generated image before generating it on the fly.

Now try to reload your main page and check thumbnail image’s src attribute – it should point to Amazon (if you are in the production environment, of course).

Conclusion

I hope you’ve learned couple of things and enjoyed reading it. What gem do you use to manage attachments in your apps? Share your experience in the comments!