Unbundling Bundler: A Thorough Look at Bundler’s Utilities
In my previous article, I went into great detail about Bundler’s Gemfile and all the magic it contains. The Gemfile, however, is only a part of the Bundler ecosystem, so I went back to the docs to dig in other areas and landed on another part of Bundler that I didn’t know as well as I should.
This article will focus on the Bundler utilities, which are the commands that ship with Bundler and are not considered primary commands. Incidentally, here are all the Bundler commands, separated by whether they are primary or a utility:
- Primary: install, update, package, exec, config, help
- Utilities: check, list, show, outdated, console, open, lock, viz, init, gem, platform, clean, binstubs, inject
Today, I will be covering that second group in an effort to bring a bit more notice to the under-appreciated tools that Bundler provides. My guess is you will learn at least one new thing about Bundler. I know I did. If I am wrong, feel free to direct all complaints to
Let’s get started. For this article, I am going to start in an empty directory (and an empty gemset) and go through the various utilities as we see what they do.
If you find yourself creating one-off Ruby projects and manually writing out a new Gemfile, then you should leverage
bundle init. I do this a fair amount, where I create a Gemfile and manually add
source "https://rubygems.org", etc. This seems silly to do, since I can just type:
$ bundle init Writing new Gemfile to /path/Gemfile
This creates a Gemfile that looks like:
# A sample Gemfile source "https://rubygems.org" # gem "rails"
Nice. Now I can stop worrying about if I had a typo in
https:/roobygemmms.org and just get to work adding my gems to the file.
Also, if you have a
.gemspec that contains dependencies for your gem, you can specify
--gemspec=<name of gem>.gemspec and the generated Gemfile will have the gems from said gemspec. It even will create a
development group for your development dependencies.
OK, let’s uncomment that
gem "rails" line so we have something to play with.
Wanna know what’s in the current bundle, but you’re too lazy to open the Gemfile? Well then, type
bundle list and you’ll get something like:
$ bundle list Gems included by the bundle: * actioncable (5.0.0) * actionmailer (5.0.0) * actionpack (5.0.0) * actionview (5.0.0) * activejob (5.0.0) * activemodel (5.0.0) ....
Before you dismiss
bundle list as trivial, you should know that it uses the Gemfile.lock to tell you what’s what. However, if you remove that file and type
bundle list you get:
$ rm Gemfile.lock $ bundle list Starting resolution (2016-07-27 12:56:04 -0400) Creating possibility state for rails (1 remaining) Attempting to activate rails (5.0.0) Activated rails at rails (5.0.0) Requiring nested dependencies (activesupport (= 5.0.0), actionpack (= 5.0.0), actionview (= 5.0.0), activemodel (= 5.0.0), activerecord (= 5.0.0), actionmailer (= 5.0.0), activejob (= 5.0.0), actioncable (= 5.0.0), railties (= 5.0.0), bundler (< 2.0, >= 1.3.0), sprocket s-rails (>= 2.0.0)) Creating possibility state for sprockets-rails (>= 2.0.0) (1 remaining) Attempting to activate sprockets-rails (3.1.1) Activated sprockets-rails at sprockets-rails (3.1.1) Requiring nested dependencies (sprockets (>= 3.0.0), actionpack (>= 4.0), activesupport (>= 4.0)) Creating possibility state for activesupport (>= 4.0) (1 remai ....
So, Bundler goes through the resolution phase from scratch and builds a new Gemfile.lock. Sassy. Incidentally, there nothing in the latest Bundler docs about
bundler list, so I think it might be being phased out…
Sometimes when you return to an application to do some work, it’s a mystery as to whether or not
bundle install has been completed and all your dependencies are in order. Well, to solve this mystery, simply type:
$ bundle check The following gems are missing * rake (11.2.2) * i18n (0.7.0) * minitest (4.7.5) * multi_json (1.12.1) * thread_safe (0.3.5) * tzinfo (0.3.51) * activesupport (4.0.2) * builder (3.1.4) * erubis (2.7.0) * rack (1.5.5) * rack-test (0.6.3) * actionpack (4.0.2) * mime-types (1.25.1) * polyglot (0.3.5) * treetop (1.4.15) * mail (2.5.4) * actionmailer (4.0.2) * activemodel (4.0.2) * activerecord-deprecated_finders (1.0.4) * arel (4.0.2) * activerecord (4.0.2) * hike (1.2.3) * thor (0.19.1) * railties (4.0.2) * tilt (1.4.1) * sprockets (2.12.4) * sprockets-rails (2.0.1) * rails (4.0.2) Install missing gems with `bundle install`
There you go. Not only does it tell us the dependencies are not installed, but it tells us which ones are missing. After a
bundle install, we see:
$ bundle check The Gemfile's dependencies are satisfied
Cool. The check is run by looking at the various gemspecs in the
$GEM_HOME/specifications directory. So, it’s entirely possible to remove an installed gem from the gem cache, but have
bundle check say all is well. Granted, you’d have to do some silly stuff to get to that point, but there you are.
bundle check will also build the Gemfile.lock if it’s missing. The following options are available:
--dry-run: Tells you what is missing, but does not build the Gemfile.lock
--path=PATH: Uses a different path to look for the gems. This is a “remembered option.” (More on that in a sec.)
FILEinstead of the Gemfile.
Sidenote: Remembered Options
In some cases, a utility will change Bundler’s configuration going forward. In the case or
--path above, it tells Bundler to always use that path for gems for this project, up until the configuration is reset or changed by another remembered option. So, when I run
$ bundle check --path=./gems The following gems are missing * rake (11.2.2) * concurrent-ruby (1.0.2) * i18n (0.7.0) * minitest (5.9.0) ... Install missing gems with `bundle install`
It creates the
./gems directory and creates (or modifies)
./.bundle/config setting the
$ bundle config ...usual... bin Set for your local app (/path/.bundle/config): "./bin" path Set for your local app (/path/.bundle/config): "/path/gems"
Also, if you
$ cat ./.bundle/config --- BUNDLE_BIN: "./bin" BUNDLE_PATH: "/Users/ggoodrich/projects/bundler/gems" BUNDLE_DISABLE_SHARED_GEMS: true
You can see the local config is changed based on the value I used for
--path. This can be reset with
bundle install --system. I will denote remembered options with (RO) from here on out.
As you may or may not know, Bundler will (by default) install the current bundle into
$GEM_HOME. Unless, of course, you tell it to do otherwise. If you are unsure where a particular gem is located, do this:
$ bundle show tzinfo <YOUR $GEM_HOME>/gems/tzinfo-1.2.2
That can be useful if you want to go peruse the source for a given gem or whatever. Also, if you don’t supply a gem name, you’ll get the same output as
bundle list. However, if you add the
--paths option, it will show you the paths to ALL the gems.
$ bundle show --paths <YOUR $GEM_HOME>/gems/ruby-2.3.0@bundler-test/gems/actioncable-5.0.0 <YOUR $GEM_HOME>/gems/ruby-2.3.0@bundler-test/gems/actionmailer-5.0.0 <YOUR $GEM_HOME>/gems/ruby-2.3.0@bundler-test/gems/actionpack-5.0.0 <YOUR $GEM_HOME>/gems/ruby-2.3.0@bundler-test/gems/actionview-5.0.0 <YOUR $GEM_HOME>/gems/ruby-2.3.0@bundler-test/gems/activejob-5.0.0 .....
I didn’t know about the
bundle open is one of my favorite Bundler utilities. It opens the gem in your favorite editor. By “favorite editor”, I mean which ever editor you have setup in the
$BUNDLER_EDITOR environment variables.
$ export EDITOR=vim $ bundle open rake ...opens to the rake gem directory in vim...
This is very useful when I want to dig into a gem and put
puts to see what in heavens is happening in that gem.
I have seen a few posts about
bundle outdated, whose job is to go through the gems and tell you if a newer version of the gem has been released. Running this on one of my older projects, it took a few minutes (literally) and looked like:
$ bundle outdated Fetching gem metadata from https://rubygems.org/........... Fetching version metadata from https://rubygems.org/... Fetching dependency metadata from https://rubygems.org/.. Resolving dependencies.............................................(lots more dots) Outdated gems included in the bundle: * active_model_serializers (newest 0.10.2 4ff33a7, installed 0.10.0.rc5 63e9337) in group "default" * byebug (newest 9.0.5, installed 8.2.4) in groups "development, test" * committee (newest 1.14.1, installed 1.14.0) in group "default" * database_cleaner (newest 1.5.3, installed 1.5.1) in group "test" ....lots more outdated gems...
It lists the gem, the newest version, the installed version, and the group it which the gem resides. You can also pass the name of a specific gem, just to check that one gem:
$ bundle outdated rails Resolving dependencies... (again, takes a really long time)
There are three command line options:
--localwill check the gem cache without connecting to the remote sources (meaning, Rubygems).
--prewill include pre-release gems, which are excluded by default.
--sourcechecks against a “specific source”, which I presume you pass into the option. The docs don’t really say, and I don’t have a custom Rubygems server.
This is one of the utilities that I always forget exists. Running
bundle console will fire up an IRB session with all of the gems in the bundle already
required (unless, you specify not to require them in the Gemfile).
$ bundle console irb> Rails => Rails
That is very useful, especially if you are developing a gem and want a quick REPL with your gem and all its dependencies loaded. By the way, if you’re a lover of Pry (and, who isn’t?) you can use
bundle console config pry to have Bundler use Pry as the console, presuming you have installed Pry already.
You know how a couple of the commands
bundle list and
bundle show will rebuild the Gemfile.lock file? Well, that’s exactly what
bundle lock does. It rebuilds the lock file, but it does NOT download/install any gems:
$ bundle lock Fetching gem metadata from https://rubygems.org/........... Fetching version metadata from https://rubygems.org/... Fetching dependency metadata from https://rubygems.org/.. Starting resolution (2016-07-27 13:58:05 -0400) Resolving dependencies...Creating possibility state for rails (167 remaining) Attempting to activate rails (5.0.0) Activated rails at rails (5.0.0) Requiring nested dependencies (activesupport (= 5.0.0), actionpack (= 5.0.0), actionview (= 5.0.0), activemodel (= 5.0.0), activerecord (= 5.0.0), actionmailer (= 5.0.0), activejob (= 5.0.0), actioncable (= 5.0.0), railties (= 5.0.0), bundler (< 2.0, >= 1.3.0), sprocket s-rails (>= 2.0.0)) Creating possibility state for bundler (< 2.0, >= 1.3.0) (1 remaining) Attempting to activate bundler (1.11.2) Activated bundler at bundler (1.11.2) Requiring nested dependencies () ... Writing lockfile to /path/Gemfile.lock
There are a couple of options for
--lockfile=FILE: Write the lock file to
FILE. I couldn’t get this to work, btw.
--local: Do not connect to Rubygems, just use the local gem cache.
--update=[gems]: This will update the list of
gemsprovided, or all gems if none are specified. This is an aggressive update to the gems.
Here’s another utility I have never used.
bundle viz creates a dependency graph of your Gemfile. Let’s take a look:
$ bundle viz #<LoadError: cannot load such file -- graphviz> Make sure you have the graphviz ruby gem. You can install it with: `gem install ruby-graphviz`
Oh, I guess I need
gem install ruby-graphviz (which I do):
$ bundle viz /<path>/gem_graph.png
If we open that file, it looks like:
That’s for a vanilla Rails app…not bad. I ran it on one of my deployed projects and it looked more, um, involved, to put it mildly. Still, it also showed me that a gem we wrote is probably doing WAY too much based on what it is pulling in as dependencies. Useful and shaming, all at once.
There are a few options:
--fileallows you to specify the name of the output file.
-Fis how to choose the image format (png, jpg, svg, etc.)
--requirementsshows the versions of each required dependency, which are the edges of the graph.
--versionshows the versions of the gems, which are the nodes of the graph.
- And of course,
--withoutallows you to exclude gem groups from the image.
Of all the utilities,
bundle gem is probably the largest. It’s point is to generate all the files needed to start the development of a proper gem. In fact, using
bundle gem is THE way to start a new gem:
$ bundle gem bundler bundle gem my-new-gem Creating gem 'my-new-gem'... create my-new-gem/Gemfile create my-new-gem/.gitignore create my-new-gem/lib/my/new/gem.rb create my-new-gem/lib/my/new/gem/version.rb create my-new-gem/my-new-gem.gemspec create my-new-gem/Rakefile create my-new-gem/README.md create my-new-gem/bin/console create my-new-gem/bin/setup create my-new-gem/.travis.yml create my-new-gem/.rspec create my-new-gem/spec/spec_helper.rb create my-new-gem/spec/my/new/gem_spec.rb Initializing git repo in /path/my-new-gem
You can see all the things that the command created:
- Of course, we get a Gemfile
- We also get an initialized git repository, complete with
.gitignore. Oh, and all the new files are
git added to the index.
- Standard gem files, including a gemspec, Rakefile, version files, binaries for
- In my case, it generated RSpec files for testing. This means that, some time in the past, I have run
bundle gem, was prompted for which test framework (RSpec or Minitest) and chose
- A Travis CI configuration file that, frankly, I have no idea why it’s part of the default
bundle gemoutput. I feel like I must’ve done something to make this happen.
Also, there are a couple items not shown above: a code of conduct and license. Again, I must of opted out of those at some point when I
bundle gemed in the past.
From an options-perspective, most of the options are around overriding the creation (or omission) of parts of the gem. Options to create or omit the following exist: An executable for the gem (
--no-exe), code of conduct (RO) (
--no-coc), C extension boilerplate (
--no-ext), MIT license (RO) (
The only other two options are
-e to open the
.gemspec in your
--test=TEST (RO) to specify
bundle platform will tell you what Ruby platforms you have specified or what platforms your gem use:
$ bundle platform Your platform is: x86_64-darwin15 Your app has gems that work on these platforms: * ruby Your Gemfile does not specify a Ruby version requirement.
So, I am running on a Mac, using MRI. If I add
ruby "2.3.0" to my Gemfile, this is what happens:
$ bundle platform our platform is: x86_64-darwin15 Your app has gems that work on these platforms: * ruby Your Gemfile specifies a Ruby version requirement: * ruby 2.3.0 Your current platform satisfies the Ruby version requirement.
There is a
--ruby option that will simply return the Ruby version specified in the Gemfile, if any.
Sometime you install gems (either on accident or on purpose) that are not used by your application. Also, if you’re using RVM, I know at one point that unused gems, once installed, are never removed from a gemset. So,
bundle clean will take care of this:
$ bundle clean Cleaning all the gems on your system is dangerous! If you're sure you want to remove every system gem not in this bundle, run `bundle clean --force`.
Wow, that’s a scary message. closes eyes
$ bundle clean --force Removing ruby-graphviz (1.2.2)
Ah, that just removed the
ruby-graphviz gem from my current gemset, which I installed to show you
Binstubs are scripts that are used to wrap executables in order to set the environment for that executable. It’s like the Decorator Pattern, but for executables. If you’ve ever run
bundle exec <some command>, it’s pretty much the same thing. Bundler is kind enough to offer
bundler binstub <a gem in the bundle> to generate a file you can call that wraps things as needed:
$ bundle binstubs rack
This will create a
bin directory and write binstubs for all the executables in the rack gem. In this case, there is only one (
rackup). So, now we can call
bin/rackup and it will be run in the context of our bundle.
Incidentally, if you run
bundle install --binstubs (RO), then Bundler will generate binstubs for every gem with executables on every
This is a weird one.
bundle inject takes a gem name and version, and injects them into the Gemfile and Gemfile.lock:
$ bundle inject rspec ">0" ...performs resolution... Added to Gemfile: rspec (> 0)
rspec gem is not installed until
bundle install is run. I have no idea why someone would use this, but I bet there’s good reason (please tell us in the comments if you know what it is)
Just for completeness sake, there’s
$ bundle version Bundler version 1.12.5
By the way,
1.12.5 is the latest Bundler version as of the time of this writing? What version are you running? I bet some of you haven’t updated Bundler in a looooong time…
That’s the Bundler utilities in a nutshell. Well, if by “nutshell” I mean “kind of longer than I intended blog post.” Still, I learned quite a few things and probably got some of it wrong. So, if you see an error, let me know. Go forth and Bundle.