Ruby

Unbundling Bundler: A Thorough Look at Bundler’s Utilities

Glenn Goodrich
Share

Bundler

In my previous article, I went into great detail about Bundler’s Gemfile and all the magic it contains. The Gemfile, however, is only a part of the Bundler ecosystem, so I went back to the docs to dig in other areas and landed on another part of Bundler that I didn’t know as well as I should.

This article will focus on the Bundler utilities, which are the commands that ship with Bundler and are not considered primary commands. Incidentally, here are all the Bundler commands, separated by whether they are primary or a utility:

  • Primary: install, update, package, exec, config, help
  • Utilities: check, list, show, outdated, console, open, lock, viz, init, gem, platform, clean, binstubs, inject

Today, I will be covering that second group in an effort to bring a bit more notice to the under-appreciated tools that Bundler provides. My guess is you will learn at least one new thing about Bundler. I know I did. If I am wrong, feel free to direct all complaints to /dev/null.

Let’s get started. For this article, I am going to start in an empty directory (and an empty gemset) and go through the various utilities as we see what they do.

bundle init

If you find yourself creating one-off Ruby projects and manually writing out a new Gemfile, then you should leverage bundle init. I do this a fair amount, where I create a Gemfile and manually add source "https://rubygems.org", etc. This seems silly to do, since I can just type:

$ bundle init
Writing new Gemfile to /path/Gemfile

This creates a Gemfile that looks like:

# A sample Gemfile
source "https://rubygems.org"

# gem "rails"

Nice. Now I can stop worrying about if I had a typo in https:/roobygemmms.org and just get to work adding my gems to the file.

Also, if you have a .gemspec that contains dependencies for your gem, you can specify --gemspec=<name of gem>.gemspec and the generated Gemfile will have the gems from said gemspec. It even will create a development group for your development dependencies.

OK, let’s uncomment that gem "rails" line so we have something to play with.

bundle list

Wanna know what’s in the current bundle, but you’re too lazy to open the Gemfile? Well then, type bundle list and you’ll get something like:

$ bundle list
Gems included by the bundle:
  * actioncable (5.0.0)
  * actionmailer (5.0.0)
  * actionpack (5.0.0)
  * actionview (5.0.0)
  * activejob (5.0.0)
  * activemodel (5.0.0)
  ....

Before you dismiss bundle list as trivial, you should know that it uses the Gemfile.lock to tell you what’s what. However, if you remove that file and type bundle list you get:

$ rm Gemfile.lock
$ bundle list
Starting resolution (2016-07-27 12:56:04 -0400)
Creating possibility state for rails (1 remaining)
  Attempting to activate rails (5.0.0)
  Activated rails at rails (5.0.0)
  Requiring nested dependencies (activesupport (= 5.0.0), actionpack (= 5.0.0), actionview (= 5.0.0), activemodel (= 5.0.0), activerecord (= 5.0.0), actionmailer (= 5.0.0), activejob (= 5.0.0), actioncable (= 5.0.0), railties (= 5.0.0), bundler (< 2.0, >= 1.3.0), sprocket
s-rails (>= 2.0.0))
  Creating possibility state for sprockets-rails (>= 2.0.0) (1 remaining)
    Attempting to activate sprockets-rails (3.1.1)
    Activated sprockets-rails at sprockets-rails (3.1.1)
    Requiring nested dependencies (sprockets (>= 3.0.0), actionpack (>= 4.0), activesupport (>= 4.0))
    Creating possibility state for activesupport (>= 4.0) (1 remai
    ....

So, Bundler goes through the resolution phase from scratch and builds a new Gemfile.lock. Sassy. Incidentally, there nothing in the latest Bundler docs about bundler list, so I think it might be being phased out…

bundle check

Sometimes when you return to an application to do some work, it’s a mystery as to whether or not bundle install has been completed and all your dependencies are in order. Well, to solve this mystery, simply type:

$ bundle check
The following gems are missing
 * rake (11.2.2)
 * i18n (0.7.0)
 * minitest (4.7.5)
 * multi_json (1.12.1)
 * thread_safe (0.3.5)
 * tzinfo (0.3.51)
 * activesupport (4.0.2)
 * builder (3.1.4)
 * erubis (2.7.0)
 * rack (1.5.5)
 * rack-test (0.6.3)
 * actionpack (4.0.2)
 * mime-types (1.25.1)
 * polyglot (0.3.5)
 * treetop (1.4.15)
 * mail (2.5.4)
 * actionmailer (4.0.2)
 * activemodel (4.0.2)
 * activerecord-deprecated_finders (1.0.4)
 * arel (4.0.2)
 * activerecord (4.0.2)
 * hike (1.2.3)
 * thor (0.19.1)
 * railties (4.0.2)
 * tilt (1.4.1)
 * sprockets (2.12.4)
 * sprockets-rails (2.0.1)
 * rails (4.0.2)
Install missing gems with `bundle install`

There you go. Not only does it tell us the dependencies are not installed, but it tells us which ones are missing. After a bundle install, we see:

$ bundle check
The Gemfile's dependencies are satisfied

Cool. The check is run by looking at the various gemspecs in the $GEM_HOME/specifications directory. So, it’s entirely possible to remove an installed gem from the gem cache, but have bundle check say all is well. Granted, you’d have to do some silly stuff to get to that point, but there you are.

bundle check will also build the Gemfile.lock if it’s missing. The following options are available:

  • --dry-run: Tells you what is missing, but does not build the Gemfile.lock
  • --path=PATH: Uses a different path to look for the gems. This is a “remembered option.” (More on that in a sec.)
  • --gemfile=FILE: Use FILE instead of the Gemfile.

Sidenote: Remembered Options

In some cases, a utility will change Bundler’s configuration going forward. In the case or --path above, it tells Bundler to always use that path for gems for this project, up until the configuration is reset or changed by another remembered option. So, when I run

$ bundle check --path=./gems
The following gems are missing
 * rake (11.2.2)
 * concurrent-ruby (1.0.2)
 * i18n (0.7.0)
 * minitest (5.9.0)
 ...
 Install missing gems with `bundle install`

It creates the ./gems directory and creates (or modifies) ./.bundle/config setting the BUNDLE_PATH to ./gems:

$ bundle config
...usual...
bin
Set for your local app (/path/.bundle/config): "./bin"

path
Set for your local app (/path/.bundle/config): "/path/gems"

Also, if you cat ./.bundle/config:

$ cat ./.bundle/config 
---
BUNDLE_BIN: "./bin"
BUNDLE_PATH: "/Users/ggoodrich/projects/bundler/gems"
BUNDLE_DISABLE_SHARED_GEMS: true

You can see the local config is changed based on the value I used for --path. This can be reset with bundle install --system. I will denote remembered options with (RO) from here on out.

bundle show

As you may or may not know, Bundler will (by default) install the current bundle into $GEM_HOME. Unless, of course, you tell it to do otherwise. If you are unsure where a particular gem is located, do this:

$ bundle show tzinfo
<YOUR $GEM_HOME>/gems/tzinfo-1.2.2

That can be useful if you want to go peruse the source for a given gem or whatever. Also, if you don’t supply a gem name, you’ll get the same output as bundle list. However, if you add the --paths option, it will show you the paths to ALL the gems. 

$ bundle show --paths
<YOUR $GEM_HOME>/gems/ruby-2.3.0@bundler-test/gems/actioncable-5.0.0
<YOUR $GEM_HOME>/gems/ruby-2.3.0@bundler-test/gems/actionmailer-5.0.0
<YOUR $GEM_HOME>/gems/ruby-2.3.0@bundler-test/gems/actionpack-5.0.0
<YOUR $GEM_HOME>/gems/ruby-2.3.0@bundler-test/gems/actionview-5.0.0
<YOUR $GEM_HOME>/gems/ruby-2.3.0@bundler-test/gems/activejob-5.0.0
.....

I didn’t know about the --paths option.

bundle open

bundle open is one of my favorite Bundler utilities. It opens the gem in your favorite editor. By “favorite editor”, I mean which ever editor you have setup in the $EDITOR or $BUNDLER_EDITOR environment variables. 

$ export EDITOR=vim
$ bundle open rake
...opens to the rake gem directory in vim...

This is very useful when I want to dig into a gem and put puts to see what in heavens is happening in that gem.

bundle outdated

I have seen a few posts about bundle outdated, whose job is to go through the gems and tell you if a newer version of the gem has been released. Running this on one of my older projects, it took a few minutes (literally) and looked like:

$ bundle outdated
Fetching gem metadata from https://rubygems.org/...........
Fetching version metadata from https://rubygems.org/...
Fetching dependency metadata from https://rubygems.org/..
Resolving dependencies.............................................(lots more dots)
Outdated gems included in the bundle:
  * active_model_serializers (newest 0.10.2 4ff33a7, installed 0.10.0.rc5 63e9337) in group "default"
  * byebug (newest 9.0.5, installed 8.2.4) in groups "development, test"
  * committee (newest 1.14.1, installed 1.14.0) in group "default"
  * database_cleaner (newest 1.5.3, installed 1.5.1) in group "test"
  ....lots more outdated gems...

It lists the gem, the newest version, the installed version, and the group it which the gem resides. You can also pass the name of a specific gem, just to check that one gem:

$ bundle outdated rails
Resolving dependencies... (again, takes a really long time)

There are three command line options:

  • --local will check the gem cache without connecting to the remote sources (meaning, Rubygems).
  • --pre will include pre-release gems, which are excluded by default.
  • --source checks against a “specific source”, which I presume you pass into the option. The docs don’t really say, and I don’t have a custom Rubygems server.

bundle console

This is one of the utilities that I always forget exists. Running bundle console will fire up an IRB session with all of the gems in the bundle already required (unless, you specify not to require them in the Gemfile).

$ bundle console
irb> Rails
=> Rails

That is very useful, especially if you are developing a gem and want a quick REPL with your gem and all its dependencies loaded. By the way, if you’re a lover of Pry (and, who isn’t?) you can use bundle console config pry to have Bundler use Pry as the console, presuming you have installed Pry already.

bundle lock

You know how a couple of the commands bundle list and bundle show will rebuild the Gemfile.lock file? Well, that’s exactly what bundle lock does. It rebuilds the lock file, but it does NOT download/install any gems:

$ bundle lock
Fetching gem metadata from https://rubygems.org/...........
Fetching version metadata from https://rubygems.org/...
Fetching dependency metadata from https://rubygems.org/..
Starting resolution (2016-07-27 13:58:05 -0400)
Resolving dependencies...Creating possibility state for rails (167 remaining)
  Attempting to activate rails (5.0.0)
  Activated rails at rails (5.0.0)
  Requiring nested dependencies (activesupport (= 5.0.0), actionpack (= 5.0.0), actionview (= 5.0.0), activemodel (= 5.0.0), activerecord (= 5.0.0), actionmailer (= 5.0.0), activejob (= 5.0.0), actioncable (= 5.0.0), railties (= 5.0.0), bundler (< 2.0, >= 1.3.0), sprocket
s-rails (>= 2.0.0))
  Creating possibility state for bundler (< 2.0, >= 1.3.0) (1 remaining)
    Attempting to activate bundler (1.11.2)
    Activated bundler at bundler (1.11.2)
    Requiring nested dependencies ()
    ...
    Writing lockfile to /path/Gemfile.lock

There are a couple of options for bundle lock:

  • --lockfile=FILE: Write the lock file to FILE. I couldn’t get this to work, btw.
  • --local: Do not connect to Rubygems, just use the local gem cache.
  • --print: Print the lock file to STDOUT.
  • --update=[gems]: This will update the list of gems provided, or all gems if none are specified. This is an aggressive update to the gems.

bundle viz

Here’s another utility I have never used. bundle viz creates a dependency graph of your Gemfile. Let’s take a look:

$ bundle viz
#<LoadError: cannot load such file -- graphviz>
Make sure you have the graphviz ruby gem. You can install it with:
`gem install ruby-graphviz`

Oh, I guess I need gem install ruby-graphviz (which I do):

$ bundle viz
/<path>/gem_graph.png

If we open that file, it looks like:

bundle viz

That’s for a vanilla Rails app…not bad. I ran it on one of my deployed projects and it looked more, um, involved, to put it mildly. Still, it also showed me that a gem we wrote is probably doing WAY too much based on what it is pulling in as dependencies. Useful and shaming, all at once.

There are a few options:

  • -f or --file allows you to specify the name of the output file.
  • --format or -F is how to choose the image format (png, jpg, svg, etc.)
  • --requirements shows the versions of each required dependency, which are the edges of the graph.
  • --version shows the versions of the gems, which are the nodes of the graph.
  • And of course, --without allows you to exclude gem groups from the image.

bundle gem

Of all the utilities, bundle gem is probably the largest. It’s point is to generate all the files needed to start the development of a proper gem. In fact, using bundle gem is THE way to start a new gem:

$ bundle gem
 bundler bundle gem my-new-gem
Creating gem 'my-new-gem'...
      create  my-new-gem/Gemfile
      create  my-new-gem/.gitignore
      create  my-new-gem/lib/my/new/gem.rb
      create  my-new-gem/lib/my/new/gem/version.rb
      create  my-new-gem/my-new-gem.gemspec
      create  my-new-gem/Rakefile
      create  my-new-gem/README.md
      create  my-new-gem/bin/console
      create  my-new-gem/bin/setup
      create  my-new-gem/.travis.yml
      create  my-new-gem/.rspec
      create  my-new-gem/spec/spec_helper.rb
      create  my-new-gem/spec/my/new/gem_spec.rb
Initializing git repo in /path/my-new-gem

You can see all the things that the command created:

  • Of course, we get a Gemfile
  • We also get an initialized git repository, complete with .gitignore. Oh, and all the new files are git added to the index.
  • Standard gem files, including a gemspec, Rakefile, version files, binaries for console and setup.
  • In my case, it generated RSpec files for testing. This means that, some time in the past, I have run bundle gem, was prompted for which test framework (RSpec or Minitest) and chose rspec.
  • A Travis CI configuration file that, frankly, I have no idea why it’s part of the default bundle gem output. I feel like I must’ve done something to make this happen.

Also, there are a couple items not shown above: a code of conduct and license. Again, I must of opted out of those at some point when I bundle gemed in the past.

From an options-perspective, most of the options are around overriding the creation (or omission) of parts of the gem. Options to create or omit the following exist: An executable for the gem (--exe or --no-exe), code of conduct (RO) (--coc, --no-coc), C extension boilerplate (--ext, --no-ext), MIT license (RO) (--mit, --no-mit).

The only other two options are -e to open the .gemspec in your $EDITOR and --test=TEST (RO) to specify rspec or minitest.

bundle platform

bundle platform will tell you what Ruby platforms you have specified or what platforms your gem use:

$ bundle platform
Your platform is: x86_64-darwin15

Your app has gems that work on these platforms:
* ruby

Your Gemfile does not specify a Ruby version requirement.

So, I am running on a Mac, using MRI. If I add ruby "2.3.0" to my Gemfile, this is what happens:

$ bundle platform
our platform is: x86_64-darwin15

Your app has gems that work on these platforms:
* ruby

Your Gemfile specifies a Ruby version requirement:
* ruby 2.3.0

Your current platform satisfies the Ruby version requirement.

There is a --ruby option that will simply return the Ruby version specified in the Gemfile, if any.

bundle clean

Sometime you install gems (either on accident or on purpose) that are not used by your application. Also, if you’re using RVM, I know at one point that unused gems, once installed, are never removed from a gemset. So, bundle clean will take care of this:

$ bundle clean
Cleaning all the gems on your system is dangerous! If you're sure you want to remove every system gem not in this bundle, run `bundle clean --force`.

Wow, that’s a scary message. closes eyes

$ bundle clean --force
Removing ruby-graphviz (1.2.2)

Ah, that just removed the ruby-graphviz gem from my current gemset, which I installed to show you bundle viz.

bundle binstubs

Binstubs are scripts that are used to wrap executables in order to set the environment for that executable. It’s like the Decorator Pattern, but for executables. If you’ve ever run bundle exec <some command>, it’s pretty much the same thing. Bundler is kind enough to offer bundler binstub <a gem in the bundle> to generate a file you can call that wraps things as needed:

$ bundle binstubs rack

This will create a bin directory and write binstubs for all the executables in the rack gem. In this case, there is only one (rackup). So, now we can call bin/rackup and it will be run in the context of our bundle.

Incidentally, if you run bundle install --binstubs (RO), then Bundler will generate binstubs for every gem with executables on every bundle install.

bundle inject

This is a weird one. bundle inject takes a gem name and version, and injects them into the Gemfile and Gemfile.lock:

$ bundle inject rspec ">0"
...performs resolution...
Added to Gemfile:
  rspec (> 0)

The rspec gem is not installed until bundle install is run. I have no idea why someone would use this, but I bet there’s good reason (please tell us in the comments if you know what it is)

bundle version

Just for completeness sake, there’s bundle version:

$ bundle version
Bundler version 1.12.5

By the way, 1.12.5 is the latest Bundler version as of the time of this writing? What version are you running? I bet some of you haven’t updated Bundler in a looooong time…

Done

That’s the Bundler utilities in a nutshell. Well, if by “nutshell” I mean “kind of longer than I intended blog post.” Still, I learned quite a few things and probably got some of it wrong. So, if you see an error, let me know. Go forth and Bundle.

CSS Master, 3rd Edition