Key Takeaways
- Bundler, the package and dependency manager for Ruby applications, uses a Gemfile to list the gems that an application needs to function. This Gemfile can be customized with a variety of options, such as group, path, and github.
- Gems can be specified in the Gemfile in several ways, including specific versions, version operators, the “twiddle-whacka” operator (which specifies any version greater than the one listed but less than the next minor revision), and compound requirements.
- Bundler allows for the grouping of dependencies into different groups, often used to install different gems in a development environment than those installed in a production environment. This can be specified on the same line as the gem or by placing the gem in a group block.
- The Gemfile offers various options for specifying gems, including Platforms (which allow for installation of a gem for a specific version of Ruby), install_if (which controls the installation of gems based on a boolean evaluation), Require (which specifies the name to require the gem by), and Override Source (which can override the source for individual gems served from a private gem server).
Bundler is fantastic, which is why it has become the de facto package and dependency manager for Ruby applications. I have used npm and golang vendoring and other language dependency managers, but none of them can even hold a candle to the simplicity Bundler offers.
As I am sure you know, at the root of Bundler’s goodness is the Gemfile. The Gemfile lists the gems that our application needs to get its work done. If you’re like me, I learned how to use the Gemfile and Bundler from various Rails (and other) tutorials. I saw people typing
gem "a_gem"
Then, running bundle install
and watching as all the dependencies were installed. Those two steps are so self-explanatory that I was never really pushed to investigate much more. Not to mention, Bundler was such a godsend after years of vendoring dependencies and other hair-pullingly frustrating approaches. rant about the old days and how easy you whippersnappers have it now
After the initial blindly-follow-the-steps Bundler tutorials, I’ve picked up on other Gemfile options, such as group
, path
, and github
, as needed. However, the other day I decided to read through the Bundler docs, specifically around the Gemfile and I was amazed at all the options I haven’t used. I learned a few things, so I figured others might too.
In today’s post, I am going mining for options in the Gemfile. There’s really nothing in here that you can’t see in the excellent Bundler documentation, but you haven’t read that, have you??? rant about how this is why we can’t have nice things
The Basics
Always a good place to start, this is the stuff you probably already know.
Source
At the top of all Gemfiles is a line like this one:
source "https://rubygems.org"
This line tells Bundler to use the Rubygems site as the one and only source for all gems listed herein. Here’s what you may not know:
- You can have multiple
source
lines, but it’s not recommended, and they are searched in the reverse order that they are listed. I don’t really like any of that, but it’s there. - If the
source
you use is a private gem server, you need to set the username and password usingbundle config
. DON’T PUT IT IN THE GEMFILE, because it will end up on Github and then your private gems will be in the open and chaos ensues. - OK, you can put the user and password into the URL itself (
https://user:pass@gem.server.com
), but don’t. You”re going to do it anyway, aren’t you? rant about how young people never listen
Gems
Right. We all know how to specify a gem in the Gemfile, right?
gem "name-of-gem"
That line will pull in the latest version of the gem. Obviously, you should lock your dependencies to versions that you know work with your application. There are several ways to specify gem versions:
- Use a specific version:
gem "name-of-gem", "1.0"
. You can find specific versions on Rubygems.org (provided that’s thesource
you”re using) by searching for your gem and looking at the “Versions” listed. - Use a version operator:
gem "name-of-gem", ">1.0"
. This pulls in any gem that is after 1.0. - Use that weird squiggly-stabby operator:
gem "name-of-gem", "~>1.3"
. This operator, which some call a “twiddle-whacka”, basically says “any version greater than this one, but less than the next minor revision”. Since I am sure you know what semantic versioning is, I don’t need to explain that the minor version is the 2nd number. In the example, it means “any version greater than or equal to 1.3, but less than 1.4”. It works with major versions, too, so~>1
will grab the latest version before2
.It gets a bit weird when you are using pre-release versions, like~>1.0.rc.1
, which means it will match all the1.0.rc
pre-release versions. - Use a compound requirement:
gem "name-of-gem", "~>2.2", ">=2.2.1"
. Here, we want a particular version, but are OK with newer minor releases. Nobody does this.
Groups
Bundler allows you to apportion your dependencies into different groups. This is used, much more often than not, to install different gems into a development environment than those installed to a production environment. You can specify the group(s) for a gem on the same line as the gem or by placing the gem in a group
block:
gem "my-dev-tool", group: :development
group :development, :test do
gem "my-test-tool"
end
You can then avoid installing a particular group (or groups) into a particular environment using the --without
command line option to bundle install
.
Rarer Finds
Let’s talk about other options for specifying gems that you may or may not have seen, but probably aren’t 100% clear about. At least, that’s how I feel about them.
Platforms
Platforms allow you to tell Bundler to install a gem for a specific version of Ruby. Platforms are basically groups, but Bundler handles the installation (or non-installation) of gems based on your current version of Ruby. The available platforms are listed here. Here’s how you use it:
gem "my-java-gem", platform: jruby
If I bundle
and my current Ruby is MRI, my-java-gem
won’t be installed.
install_if
Another way you can control the installation of a gem is install_if
. Passing a proc or lambda that evaluates to a boolean (meaning, “truthy” or “falsy”) will control the installation of the gems in the block. From the docs:
install_if -> { RUBY_PLATFORM =~ /darwin/ } do
gem "pasteboard"
end
Maybe you have another environment variable that determines if you install some gem or gems. I can’t speak intelligently on why you do things, all I can do is show you the tools.
Require
If you’ve ever seen the require
keyword in a gem line, like so:
gem "rspec", require: "spec"
That means that the gem name and the name you require the gem by are different. You see, when something calls Bundler.require
, Bundler will require all the gem dependencies. So, in the example, you want it to require "spec"
, not require "rspec"
.
Also we sometimes see a line like this:
gem 'rspec', require: false
That tells Bundler not to require rspec
when Bundler.require
is called. Rails, for example, will call Bundler.require
as a part of its magic, which is why all your gems are already required.
Bundler will, by default, install all gems in the Gemfile and load them into memory. If we don’t want to load a specific gem (maybe because we’ll be doing it dynamically from within our code) we can set the require keyword to false.
Override Source
I mentioned the source
keyword that lives in the top of the Gemfile, but if you happen to have gems that are served from a private gem server, you can override the source
for those gems individually. This can be done on a single gem basis or for multiple gems in a block:
gem "a-gem", source: "https://my-private-gem-server.com"
source "https://my-private-gem-server.com" do
gem "a-gem"
gem "b-gem"
end
Git
Another, more common, way to override where a gem comes from is to use the git
, github
, or path
options, which pull from a git repository, Github, or a file path, respectively.
When pulling a gem directly from git, it’s just a matter of specifying the path to the repository:
gem "my-git-gem", git: "https://github.com/me/my-git-gem.git"
You need to have access to the repo, so it needs to be public, on your network, etc. The block syntax works as well, if your git repository has more than one gemspec:
git "http://github.com/me/many_gems.git" do
gem "a-gem"
gem "b-gem"
end
By the way, those repositories must have .gemspec
files. Otherwise, you have to supply a version:
gem "my-git-gem-with-on-spec", "0.1", git: "https://github.com/me/my-git-gem-with-no-spec.git"
What if you want to pull in a branch, tag, or specific commit? Bundler lets you use the branch
, tag
, and ref
options, respectively:
gem "my-git-gem", git: "https://github.com/me/my-git-gem.git", branch: "the_new_stuff"
gem "my-git-gem", git: "https://github.com/me/my-git-gem.git", tag: "1.0.rc2"
gem "my-git-gem", git: "https://github.com/me/my-git-gem.git", ref: "5acdb"
By the way, the ref
option takes the first 5 characters (or more) of the commit hash.
Finally, if you are pulling in a git repository that has a ton of submodules (people do this?), there is a boolean submodule
option:
gem "my-gem-of-submodules", git: "https://github.com/me/my-gem-of-submodules.git", submodules: true
Custom Git Sources
As it turns out, you can define custom git sources. Bundler supplies 3 git sources out-of-the-box: github
, bitbucket
, and gist
. This provides some pretty cool shorthand:
gem "my-git-gem", github: "me/my-git-gem.git"
Just supply the path to the repo on Github (or Bitbucket). What if the owner (or organization) is the same name as the git repository? It gets even smaller:
gem "rails", github: "rails" # Works b/c the repo is https://github.com/rails/rails.git
All the options that work for the git
source are valid here as well. By the way, the github
option uses the git
protocol, which is not encrypted, so govern yourself accordingly.
If your repositories on Github are private, you can create a personal access token and use it in the Gemfile:
gem "my-private-gem", git: "https://<the_token>:x-oauth-basic@github.com/me/my-private-gem.git"
That makes using private git repos easy, but putting your token in the Gemfile is a recipe for disaster. To keep your token secure, set an environment variable (GITHUB_TOKEN
) and reference it in the git
option:
gem "my-private-gem", git: "https://#{ENV['GITHUB_TOKEN']}:x-oauth-basic@github.com/me/my-private-gem.git"
Much love to this gist for first showing this to me. Unfortunately, there is still a problem with this approach, as it will write the token to the Gemfile.lock file, which is likely under source control. So, what can we do?
There is also a way to use bundle config
to set up the auth. If you change the line in your Gemfile to:
gem "my-private-gem", git: "https://github.com/me/my-private-gem.git"
and then run
bundle config github.com <the_token>
Bundler will use the token to access github on your behalf. So, all your private repos will be accessible. Incidentally, that writes values into ~/.bundle/config
. Hmmm…maybe I need to do another article that dives into the Bundler CLI. rant about how my work is never done
Gist
I’ve never used a Gist as a gem source, so I wasn’t quite sure how it works. I made one to figure it out:
gem "gist-gem", gist: "e0bef10af2e0f9bb2ed703c721ca9d29"
That works. TIL this. I have no idea why someone would do this. But, I don’t get MySpace, either.
Finally, you can make custom git sources for other git servers, like Github Enterprise Server, Bitbucket Server (formally Stash), or other sources. Here’s an example from the docs:
git_source(:stash){ |repo_name| "https://stash.corp.acme.pl/#{repo_name}.git" }
gem "rails", :stash => "forks/rails"
Path
If you have a gem locally that you”re developing, you can pull it in to your bundler with the path
option:
gem "mine-mine-all-mine", path: "~/my-gems/mine-mine-all-mine"
Of course, there’s a block option:
path "~/my-gems" do
gem "mine-mine-all-mine"
gem "acts-as-myspace"
end
Ruby Version
It is possible to specify the Ruby version and interpreter (or engine) that your application requires. For example, if you are bound to Ruby 2.1 or above, but don’t want to presume 2.2 is OK, place this in the top of your Gemfile:
ruby "~>2.1"
That version obviously refers to MRI, by default. What if you want to grab JRuby?
ruby "~>2.1", engine: "jruby", engine_version: "9.0.3.0"
Which means you want JRuby 9000. The ruby
keyword also provides a patchlevel
option that takes all the same gem version operators, if you know what patch level you need. By the way, if I ever submit a patch, it will be patch level nicotine
, adams
, or eye
.
Some version managers, like RVM, also read the Ruby directive and install the appropriate Ruby version, so that Bundler will never complain that we’re using a different Ruby version.
All Done
So, that’s it. The Bundler Gemfile distilled. Who knew that there are so many options for installing gems in the Gemfile? I didn’t, so this effort taught me some new stuff and reaffirmed my love for Bundler, the best dependency manager in the world. I hope you learned something, too.
Here’s a question for you: Did I miss anything? Any other tricks that we should include here? What are the young folks doing with Bundler these days?
I would like to sincerely thank Enrique Gonzalez and Fred Heath for peer reviewing this article. Despite their youth, they are very useful members of society.
Frequently Asked Questions (FAQs) about Gemfile
What is the purpose of a Gemfile in Ruby on Rails?
A Gemfile is a crucial component in Ruby on Rails applications. It is a file where you specify the gem dependencies needed for your Ruby or Rails application to run. It’s like a shopping list for your application, detailing all the gems it needs to function correctly. The Gemfile is processed by Bundler, a Ruby dependency management tool, which then installs the specified gems in your application.
How do I create a Gemfile?
Creating a Gemfile is straightforward. In the root directory of your application, create a file named ‘Gemfile’ (without any extension). Inside this file, you list your gem dependencies in the following format: gem 'gem_name'
. For example, if your application requires the Rails gem, you would write gem 'rails'
in your Gemfile.
What does the source
command do in a Gemfile?
The source
command in a Gemfile specifies where Bundler should look for the gems listed in the Gemfile. The default source is RubyGems.org, the main repository for Ruby gems. You can specify this with the line source 'https://rubygems.org'
at the top of your Gemfile.
How do I specify gem versions in a Gemfile?
You can specify gem versions in your Gemfile using operators like >
, <
, >=
, <=
, ~>
, and =
. For example, gem 'rails', '~> 5.2.0'
will install the latest version of Rails that is greater than or equal to 5.2.0 and less than 5.3.
What is the difference between gem
and group
in a Gemfile?
The gem
command in a Gemfile is used to specify a gem dependency. The group
command, on the other hand, is used to specify groups of gems that should be bundled together. For example, you might group together all the gems needed for testing your application.
What is the role of the Gemfile.lock
file?
The Gemfile.lock
file is automatically generated by Bundler. It locks your gems to specific versions, ensuring that the same versions of the gems are used across all environments your application runs in. This helps prevent inconsistencies and potential bugs that could arise from different gem versions being used in different environments.
How do I update a gem using the Gemfile?
To update a gem using the Gemfile, you can change the version number specified next to the gem and then run bundle update gem_name
. This will update the gem to the specified version, as well as any dependencies it has.
Can I specify the source of individual gems in a Gemfile?
Yes, you can specify the source of individual gems in a Gemfile using the :git
, :path
, or :gist
options. For example, gem 'nokogiri', :git => 'https://github.com/tenderlove/nokogiri.git'
will fetch the Nokogiri gem directly from its GitHub repository.
What does the :require
option do in a Gemfile?
The :require
option in a Gemfile allows you to specify a different file to require than the default. For example, gem 'rack', :require => 'rack/session'
will require the ‘rack/session’ file instead of the default ‘rack’ file when the Rack gem is loaded.
How do I use environment variables in a Gemfile?
You can use environment variables in a Gemfile with the ENV
keyword. For example, gem 'rails', '3.0' if ENV['RAILS_VERSION'] == '3.0'
will only install Rails 3.0 if the ‘RAILS_VERSION’ environment variable is set to ‘3.0’.
Glenn works for Skookum Digital Works by day and manages the SitePoint Ruby channel at night. He likes to pretend he has a secret identity, but can't come up with a good superhero name. He's settling for "Roob", for now.