Asynchronous File Uploads in Rails
A couple of weeks ago I penned an article called Better File Uploads with Dragonfly explaining how to set up Dragonfly – a gem for handling attachments. The post set up a demo app and integrated Dragonfly into, allowing our users to share (upload) their photos.
Today, we’ll explore a bit more about file uploading, such as how to implement asynchronous file uploading with Rails as well as uploading multiple files using AJAX.
We are going to take a look at two solutions: Remotipart and File Upload.
Some other things to be covered in this article:
- Creating a progress bar to track the file uploading progress and display bitrate.
- Implementing client-side validation.
- Adding a “dropzone” with some nice CSS3 effects, allowing users to drag & drop files to be uploaded.
All this will be done in five iterations. Let’s get started!
The source code is available on GitHub.
Preparing the Project
For this demo I am using Rails 4.1.1 but the same solution can be implemented with Rails 3.
In the first iteration, we will create a new project and integrate Dragonfly into it really quick (for a more detailed guide check out my previous article).
Create a new app without the default testing suite:
$ rails new async_uploader -T
Drop some gems into your Gemfile:
Gemfile
gem 'dragonfly'
gem 'dragonfly-s3_data_store'
group :production do
gem 'rack-cache', :require => 'rack/cache'
end
gem 'bootstrap-sass'
gem 'remotipart', '~> 1.2'
dragonfly
is the gem that will handle all the uploading magic and dragonfly-s3_data_store
will allow us to store images using Amazon S3 (we will need this in production). rack-cache
adds a simple caching mechanism for the production environment. bootstrap-sass
will equip our app with Twitter Bootstrap. remotipart
will be used in the next iteration.
Now hook up Bootstrap’s styles and scripts (don’t forget that, for real apps, pick only the required components):
stylesheets/application.css.scss
@import "bootstrap";
@import 'bootstrap/theme';
javascripts/application.js
[...]
//= require bootstrap
Now, run Dragonfly’s generator to create an initializer file:
$ rails g dragonfly
Open it up and replace the default datastore
settings with these:
config/initializers/dragonfly.rb
[...]
if Rails.env.development? || Rails.env.test?
datastore :file,
root_path: Rails.root.join('public/system/dragonfly', Rails.env),
server_root: Rails.root.join('public')
else
datastore :s3,
bucket_name: 'your_bucket_name',
access_key_id: ENV['AWS_KEY'],
secret_access_key: ENV['AWS_SEC'],
url_scheme: 'https'
end
For access_key_id
and secret_access_key
, you need to register at Amazon and then open up “YourAccount – Security Credentials”, expand the “Access Keys” section, and create a new pair. A bucket can be created using AWS management console.
This key pair should be kept safe, so don’t put it in version control. I’m using Heroku environment variables.
Also if you are on Rails 4, enable rack-cache
for the production environment:
config/environments/production.rb
[...]
config.action_dispatch.rack_cache = true
[...]
We are almost done setting up Dragonfly. The last thing to do is to create the model and add the file uploading requirements. Let’s say we want users to upload their photos. Apart from the default fields our photos
table will have these columns:
image_uid
(string
) – this column will store image’s URIauthor
(string
) – this column will contain author’s name
Great, now run these commands to create and apply the required migration:
$ rails g model Photo image_uid:string title:string
$ rake db:migrate
On to the model:
models/photo.rb
class Photo < ActiveRecord::Base
dragonfly_accessor :image
validates :image, presence: true
validates_size_of :image, maximum: 500.kilobytes,
message: "should be no more than 500 KB", if: :image_changed?
validates_property :format, of: :image, in: [:jpeg, :jpg, :png, :bmp], case_sensitive: false,
message: "should be either .jpeg, .jpg, .png, .bmp", if: :image_changed?
end
At this point we are done with Dragonfly. Of course, we will also need a controller and corresponding view. The controller will be dead simple for now:
controllers/photos_controller.rb
class PhotosController < ApplicationController
def new
@photos = Photo.order('created_at DESC')
@photo = Photo.new
end
def create
@photo = Photo.new(photo_params)
@photo.save
redirect_to new_photo_path
end
private
def photo_params
params.require(:photo).permit(:image, :title)
end
end
As you see, the “New photo” will not only allow to create a new photo, but will also list all uploaded photos.
The routes:
config/routes.rb
resources :photos, only: [:new, :create]
root to: 'photos#new'
The layout:
layouts/application.html.erb
[...]
<body>
<div class="navbar navbar-inverse" role="navigation">
<div class="container">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-collapse">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="#">Uploader</a>
</div>
<div class="collapse navbar-collapse">
<ul class="nav navbar-nav">
<li><%= link_to 'Upload one', root_path %></li>
</ul>
</div>
</div>
</div>
<div class="container">
<%= yield %>
</div>
</body>
[...]
And the view:
photos/new.html.erb
<h1>List of photos</h1>
<ul class="row" id="photos-list">
<%= render @photos %>
</ul>
<%= form_for @photo do |f| %>
<div class="form-group">
<%= f.label :author %>
<%= f.text_field :author, class: 'form-control' %>
</div>
<div class="form-group">
<%= f.label :image %>
<%= f.file_field :image, required: true %>
</div>
<%= f.submit 'Submit', class: 'btn btn-primary btn-lg' %>
<% end %>
I am using render @photos
to render all photos using the _photos.html.erb
partial, so create it as well:
photos/_photo.html.erb
<li class="col-xs-3">
<%= link_to image_tag(photo.image.thumb('180x180#').url, alt: photo.author, class: 'img-thumbnail'),
photo.image.remote_url, target: '_blank' %>
<p><%= photo.author %></p>
</li>
And, lastly, a bit of styling:
stylesheets/application.css.scss
#photos-list {
padding: 0;
margin: 0;
margin-top: 30px;
clear: both;
li {
list-style-type: none;
padding: 0;
margin: 0;
min-height: 272px;
max-height: 272px;
margin-bottom: 10px;
text-align: center;
img {
margin-bottom: 10px;
}
p {
margin: 0;
height: 40px;
overflow: hidden;
}
}
}
Phew, now we are done with the first iteration! At this point, launch the server and upload a couple of photos to make sure everything is working.
Making it Asynchronous
Our users can upload their photos, but the page is reloaded each time the form is submitted. How about making it asynchronous so the uploaded photo’s preview shows up instantly? This can be done easily.
You might think that all we need to do is add remote: true
to the form helper, like this:
photos/new.html.erb
[...]
<%= form_for @photo, remote: true do |f| %>
[...]
But that won’t work because Rails doesn’t know how to send multipart forms asynchronously. Luckily, the solution is simple. Remember, we added gem 'remotipart', '~> 1.2'
to our Gemfile. This gem, created by Steve Schwartz and Greg Leppert, enables AJAX file uploads with Rails 3 and 4 which exactly what we need.
The only thing to do is add a single line to the application.js
file:
javascripts/application.js
[...]
//= require jquery.remotipart
And now our app knows kung fu! Pretty awesome, isn’t it?
Well, the controller needs a minor tweak so it responds to AJAX requests:
controllers/photos_controller.rb
[...]
def create
respond_to do |format|
@photo = Photo.new(photo_params)
@photo.save
format.html { redirect_to new_photo_path }
format.js
end
end
[...]
Don’t forget to create the corresponding view:
photos/create.js.erb
<% if @photo.new_record? %>
alert('The photo could not be uploaded: <%= j @photo.errors.full_messages.join(', ').html_safe %>');
<% else %>
$('#photos-list').prepend('<%= j render @photo %>');
<% end %>
The if @photo.new_record?
condition actually checks whether the photo was successfully saved (we added some validations, remember?). If it wasn’t, then present the user with a basic alert box displaying the errors.
If the record was saved, use jQuery’s prepend
method to add another element to the #photos-list
unordered list. The j
method is required because, otherwise, all HTML from the _photo.html.erb
partial would be sanitized.
That’s all! Now, you can try some photos and check whether asynchronous uploading works.
Well, that was the easy part. Let’s move to the third iteration and implement functionality allowing users to upload multiple photos at once, asynchronously.
Uploading Multiple Photos Asynchronously
Create another view and the corresponding controller method so that we don’t mess up the previously written code.
controllers/photos_controller.rb
[...]
def new_multiple
@photos = Photo.order('created_at DESC')
@photo = Photo.new
end
[...]
As you can see, this method is actually the same as the new
. The difference is in the view:
*photos/new_multiple.html.erb
<h1>List of photos</h1>
<ul class="row" id="photos-list">
<%= render @photos %>
</ul>
<h3>Upload multiple photos</h3>
<%= form_for @photo do |f| %>
<div class="form-group">
<%= f.label :author %>
<%= f.text_field :author, class: 'form-control' %>
</div>
<div class="form-group">
<%= f.label :image %>
<%= f.file_field :image, required: true %>
</div>
<% end %>
Note that we do not need to provide remote: true
for the form helper in this case. Lastly, set up the route:
config/routes.rb
[...]
get '/photos/new_multiple', to: 'photos#new_multiple', as: :new_photo_multiple
To allow multiple file uploading, we use the jQuery File Upload plugin, created by Sebastian Tschan. This plugin has numerous options and cool features. I encourage you to check out its documentation.
We will also use jquery-fileupload-rails
by Tors Dalid to integrate this plugin into our app. Unfortunately, this gem hasn’t been updated since 2012, so let’s use a forked version instead:
Gemfile
gem 'jquery-fileupload-rails', github: 'Springest/jquery-fileupload-rails'
THe File Upload plugin comes in two versions: UI and basic. We are going to implement the basic version, as it provides more control and we can style the form the way we want.
javascripts/application.js
[...]
//= require jquery-fileupload/basic
Now, make some modifications to the view:
photos/new_multiple.html.erb
[...]
<div class="form-group">
<%= f.label :image %>
<%= f.file_field :image, required: true, multiple: true, name: 'photo[image]' %>
</div>
[...]
<script>
$(document).ready(function() {
$('#new_photo').fileupload({dataType: 'script'});
});
</script>
The multiple: true
option tells the file field that the user may choose multiple files by holding the CTRL (or Shift) key.
hard codes the field’s name because, otherwise, Rails will name this field to
and the files will be submitted as an array. We want them to be submitted one by one, and File Upload will take care that.
$('#new_photo').fileupload();
equips our form with File Upload’s magic and dataType: 'script'
indicates we are expecting javascript in the response.
Also, note that our form has no “Submit” button. This is because as soon as the user selects the files, the form is submitted automatically (you may want to write a short warning message for the users regarding that).
At this point, go upload a couple of photos at once – the controller’s create
method requires no changes! Everything should be working, apart from the author’s name. The photos will be uploaded, but the author
field will be empty. This is because File Upload submits only the files by default. To fix this, we need to tweak the script:
photos/new_multiple.html.erb
<script>
$(document).ready(function() {
var multiple_photos_form = $('#new_photo');
multiple_photos_form.fileupload({dataType: 'script'});
multiple_photos_form.on('fileuploadsubmit', function (e, data) {
data.formData = {'photo[author]': $('#photo_author').val()};
});
});
</script>
Here, we are listening for the fileuploadsubmit
event (the list of callbacks is available here) and attaching the author field’s value to the form data.
Adding Client-side Validation and a Progress Bar
We are ready to move to the fourth iteration. Right now, our form is fully functional, but it has some issues:
- Validation is only on the server side. Because of that, the user will have to wait for his file to be uploaded before he is told about any errors.
- The user is not given feedback about the uploading process. If the internet connection is slow, uploading may take a long time and the user won’t see anything happening at all.
The solutions are simple: Add a client-side validation and a progress bar.
Let’s start with validation. Tweak the script:
photos/new_multiple.html.erb
[...]
multiple_photos_form.fileupload({
dataType: 'script',
add: function (e, data) {
types = /(\.|\/)(gif|jpe?g|png|bmp)$/i;
file = data.files[0];
if (types.test(file.type) || types.test(file.name)) {
data.submit();
}
else { alert(file.name + " must be GIF, JPEG, BMP or PNG file"); }
}
});
[...]
We are checking if each file has the correct format. If it does, submit the form using data.submit();
, otherwise, present the user with an error message. Now, the user will know if the files are the an acceptable format.
On to the progress bar. Bootstrap already provides a nicely styled bar, so let’s use it:
photos/new_multiple.html.erb
[...]
<div class="form-group">
<%= f.label :image %>
<%= f.file_field :image, required: true, multiple: true, name: 'photo[image]' %>
</div>
<div class="progress-wrapper">
<div class="progress">
<div class="progress-bar" role="progressbar">
0%
</div>
</div>
</div>
[...]
We don’t want the progress bar to be displayed until the form is actually submitted, so tweak the styles a bit:
stylesheets/application.css.scss
[...]
.progress-wrapper {
display: none;
}
To track the overall file uploading progress, the script has to listen to the fileuploadprogressall
event, get the number of uploaded bites, and adjust progress bar. Also, the progress bar has to be shown once the uploading starts, complements of the fileuploadstart
event. Lastly, the bar should be hidden again after uploading finishes, and fileuploaddone
helps in this case.
Here is the modified script:
photos/new_multiple.html.erb
[...]
var wrapper = multiple_photos_form.find('.progress-wrapper');
var progress_bar = wrapper.find('.progress-bar');
multiple_photos_form.on('fileuploadstart', function() {
wrapper.show();
});
multiple_photos_form.on('fileuploaddone', function() {
wrapper.hide();
progress_bar.width(0); // Revert progress bar's width back to 0 for future uploads
});
multiple_photos_form.on('fileuploadprogressall', function (e, data) {
var progress = parseInt(data.loaded / data.total * 100, 10);
progress_bar.css('width', progress + '%').text(progress + '%');
});
[...]
data.loaded
gets the number of loaded bytes, whereas data.total
contains the total number of bites to be uploaded.
The last neat thing to do is to show the bitrate (upload speed) to the user. This is easy:
photos/new_multiple.html.erb
[...]
<div class="progress-wrapper">
<p>Bitrate: <span class="bitrate"></span></p>
<div class="progress">
<div class="progress-bar" role="progressbar">
0%
</div>
</div>
</div>
[...]
And the script:
[...]
var bitrate = wrapper.find('.bitrate');
multiple_photos_form.on('fileuploadprogressall', function (e, data) {
bitrate.text((data.bitrate / 1024).toFixed(2) + 'Kb/s');
var progress = parseInt(data.loaded / data.total * 100, 10);
progress_bar.css('width', progress + '%').text(progress + '%');
});
Our form looks pretty awesome and provides useful information to the user. Let’s move on to the fifth and final iteration.
Adding Dropzone
The File Upload plugin provides a way to setup a so-called “Dropzone” – an area where users may drag & drop their files to begin uploading immediately (check out the browser support).
This is a nice alternative to the default file dialog. It feels more natural to just drop some files to be uploaded.
Let’s add a new element to our markup:
photos/new_multiple.html.erb
[...]
<h3>Upload multiple photos</h3>
<div id="dropzone" class="fade">Drop files here</div>
<%= form_for @photo do |f| %>
<div class="form-group">
<%= f.label :author %>
<%= f.text_field :author, class: 'form-control' %>
</div>
<div class="form-group">
<%= f.label :image, 'Choose files here' %>
<%= f.file_field :image, required: true, multiple: true, name: 'photo[image]' %>
</div>
<div class="progress-wrapper">
<p>Bitrate: <span class="bitrate"></span></p>
<div class="progress">
<div class="progress-bar" role="progressbar">
0%
</div>
</div>
</div>
<% end %>
[...]
The script:
photos/new_multiple.html.erb
[...]
multiple_photos_form.fileupload({
dataType: 'script',
dropZone: $('#dropzone'),
add: function (e, data) {
types = /(\.|\/)(gif|jpe?g|png|bmp)$/i;
file = data.files[0];
if (types.test(file.type) || types.test(file.name)) {
data.submit();
}
else { alert(file.name + " must be GIF, JPEG, BMP or PNG file"); }
}
});
[...]
We can also apply some CSS3 transitions to the drop zone (File Upload’s events will help us here). Here are some styles that you might use:
stylesheets/application.css.scss
@import "compass/css3/transition";
@import "compass/css3/opacity";
@import "compass/css3/border-radius";
[...]
#dropzone {
background: palegreen;
width: 150px;
text-align: center;
font-weight: bold;
height: 50px;
line-height: 50px;
border: 1px solid darken(palegreen, 10%);
@include border-radius(10px);
}
#dropzone.in {
width: 600px;
height: 200px;
line-height: 200px;
font-size: larger;
}
#dropzone.hover {
background: lawngreen;
border: 1px solid darken(lawngreen, 10%);
}
#dropzone.fade {
@include transition-property(all);
@include transition-duration(0.5s);
@include transition-timing-function(ease-out);
@include opacity(1);
}
I am using some Compass utilities (I am too lazy to specify all the vendor prefixes for the CSS3 attributes), so don’t forget to drop these lines to the Gemfile:
Gemfile
[...]
gem 'compass-rails'
gem 'compass'
[...]
and run bundle install
.
Here are scripts that will take care of animating the dropzone:
photos/new_multiple.html.erb
$(document).bind('dragover', function (e) {
var dropZone = $('#dropzone'),
timeout = window.dropZoneTimeout;
if (!timeout) {
dropZone.addClass('in');
} else {
clearTimeout(timeout);
}
var found = false,
node = e.target;
do {
if (node === dropZone[0]) {
found = true;
break;
}
node = node.parentNode;
} while (node != null);
if (found) {
dropZone.addClass('hover');
} else {
dropZone.removeClass('hover');
}
window.dropZoneTimeout = setTimeout(function () {
window.dropZoneTimeout = null;
dropZone.removeClass('in hover');
}, 100);
});
You can read more about it here.
You can now test out the form and see all this stuff is working. :)
Conclusion
We’ve taken a look at two solutions for asynchronous file uploading. I hope you’ve found this article interesting and useful.
Have you ever tried to implement asynchronous file uploading in your apps? What solutions did you employ? What problems did you face? Share your experience in the comments!