Unbundling Bundler: A Thorough Look at Bundler’s Utilities
In my previous article, I went into great detail about Bundler’s Gemfile and all the magic it contains. The Gemfile, however, is only a part of the Bundler ecosystem, so I went back to the docs to dig in other areas and landed on another part of Bundler that I didn’t know as well as I should.
This article will focus on the Bundler utilities, which are the commands that ship with Bundler and are not considered primary commands. Incidentally, here are all the Bundler commands, separated by whether they are primary or a utility:
- Primary: install, update, package, exec, config, help
- Utilities: check, list, show, outdated, console, open, lock, viz, init, gem, platform, clean, binstubs, inject
Today, I will be covering that second group in an effort to bring a bit more notice to the under-appreciated tools that Bundler provides. My guess is you will learn at least one new thing about Bundler. I know I did. If I am wrong, feel free to direct all complaints to /dev/null
.
Let’s get started. For this article, I am going to start in an empty directory (and an empty gemset) and go through the various utilities as we see what they do.
bundle init
If you find yourself creating one-off Ruby projects and manually writing out a new Gemfile, then you should leverage bundle init
. I do this a fair amount, where I create a Gemfile and manually add source "https://rubygems.org"
, etc. This seems silly to do, since I can just type:
$ bundle init
Writing new Gemfile to /path/Gemfile
This creates a Gemfile that looks like:
# A sample Gemfile
source "https://rubygems.org"
# gem "rails"
Nice. Now I can stop worrying about if I had a typo in https:/roobygemmms.org
and just get to work adding my gems to the file.
Also, if you have a .gemspec
that contains dependencies for your gem, you can specify --gemspec=<name of gem>.gemspec
and the generated Gemfile will have the gems from said gemspec. It even will create a development
group for your development dependencies.
OK, let’s uncomment that gem "rails"
line so we have something to play with.
bundle list
Wanna know what’s in the current bundle, but you’re too lazy to open the Gemfile? Well then, type bundle list
and you’ll get something like:
$ bundle list
Gems included by the bundle:
* actioncable (5.0.0)
* actionmailer (5.0.0)
* actionpack (5.0.0)
* actionview (5.0.0)
* activejob (5.0.0)
* activemodel (5.0.0)
....
Before you dismiss bundle list
as trivial, you should know that it uses the Gemfile.lock to tell you what’s what. However, if you remove that file and type bundle list
you get:
$ rm Gemfile.lock
$ bundle list
Starting resolution (2016-07-27 12:56:04 -0400)
Creating possibility state for rails (1 remaining)
Attempting to activate rails (5.0.0)
Activated rails at rails (5.0.0)
Requiring nested dependencies (activesupport (= 5.0.0), actionpack (= 5.0.0), actionview (= 5.0.0), activemodel (= 5.0.0), activerecord (= 5.0.0), actionmailer (= 5.0.0), activejob (= 5.0.0), actioncable (= 5.0.0), railties (= 5.0.0), bundler (< 2.0, >= 1.3.0), sprocket
s-rails (>= 2.0.0))
Creating possibility state for sprockets-rails (>= 2.0.0) (1 remaining)
Attempting to activate sprockets-rails (3.1.1)
Activated sprockets-rails at sprockets-rails (3.1.1)
Requiring nested dependencies (sprockets (>= 3.0.0), actionpack (>= 4.0), activesupport (>= 4.0))
Creating possibility state for activesupport (>= 4.0) (1 remai
....
So, Bundler goes through the resolution phase from scratch and builds a new Gemfile.lock. Sassy. Incidentally, there nothing in the latest Bundler docs about bundler list
, so I think it might be being phased out…
bundle check
Sometimes when you return to an application to do some work, it’s a mystery as to whether or not bundle install
has been completed and all your dependencies are in order. Well, to solve this mystery, simply type:
$ bundle check
The following gems are missing
* rake (11.2.2)
* i18n (0.7.0)
* minitest (4.7.5)
* multi_json (1.12.1)
* thread_safe (0.3.5)
* tzinfo (0.3.51)
* activesupport (4.0.2)
* builder (3.1.4)
* erubis (2.7.0)
* rack (1.5.5)
* rack-test (0.6.3)
* actionpack (4.0.2)
* mime-types (1.25.1)
* polyglot (0.3.5)
* treetop (1.4.15)
* mail (2.5.4)
* actionmailer (4.0.2)
* activemodel (4.0.2)
* activerecord-deprecated_finders (1.0.4)
* arel (4.0.2)
* activerecord (4.0.2)
* hike (1.2.3)
* thor (0.19.1)
* railties (4.0.2)
* tilt (1.4.1)
* sprockets (2.12.4)
* sprockets-rails (2.0.1)
* rails (4.0.2)
Install missing gems with `bundle install`
There you go. Not only does it tell us the dependencies are not installed, but it tells us which ones are missing. After a bundle install
, we see:
$ bundle check
The Gemfile's dependencies are satisfied
Cool. The check is run by looking at the various gemspecs in the $GEM_HOME/specifications
directory. So, it’s entirely possible to remove an installed gem from the gem cache, but have bundle check
say all is well. Granted, you’d have to do some silly stuff to get to that point, but there you are.
bundle check
will also build the Gemfile.lock if it’s missing. The following options are available:
--dry-run
: Tells you what is missing, but does not build the Gemfile.lock--path=PATH
: Uses a different path to look for the gems. This is a “remembered option.” (More on that in a sec.)--gemfile=FILE
: UseFILE
instead of the Gemfile.
Sidenote: Remembered Options
In some cases, a utility will change Bundler’s configuration going forward. In the case or --path
above, it tells Bundler to always use that path for gems for this project, up until the configuration is reset or changed by another remembered option. So, when I run
$ bundle check --path=./gems
The following gems are missing
* rake (11.2.2)
* concurrent-ruby (1.0.2)
* i18n (0.7.0)
* minitest (5.9.0)
...
Install missing gems with `bundle install`
It creates the ./gems
directory and creates (or modifies) ./.bundle/config
setting the BUNDLE_PATH
to ./gems
:
$ bundle config
...usual...
bin
Set for your local app (/path/.bundle/config): "./bin"
path
Set for your local app (/path/.bundle/config): "/path/gems"
Also, if you cat ./.bundle/config
:
$ cat ./.bundle/config
---
BUNDLE_BIN: "./bin"
BUNDLE_PATH: "/Users/ggoodrich/projects/bundler/gems"
BUNDLE_DISABLE_SHARED_GEMS: true
You can see the local config is changed based on the value I used for --path
. This can be reset with bundle install --system
. I will denote remembered options with (RO) from here on out.
bundle show
As you may or may not know, Bundler will (by default) install the current bundle into $GEM_HOME
. Unless, of course, you tell it to do otherwise. If you are unsure where a particular gem is located, do this:
$ bundle show tzinfo
<YOUR $GEM_HOME>/gems/tzinfo-1.2.2
That can be useful if you want to go peruse the source for a given gem or whatever. Also, if you don’t supply a gem name, you’ll get the same output as bundle list
. However, if you add the --paths
option, it will show you the paths to ALL the gems.
$ bundle show --paths
<YOUR $GEM_HOME>/gems/ruby-2.3.0@bundler-test/gems/actioncable-5.0.0
<YOUR $GEM_HOME>/gems/ruby-2.3.0@bundler-test/gems/actionmailer-5.0.0
<YOUR $GEM_HOME>/gems/ruby-2.3.0@bundler-test/gems/actionpack-5.0.0
<YOUR $GEM_HOME>/gems/ruby-2.3.0@bundler-test/gems/actionview-5.0.0
<YOUR $GEM_HOME>/gems/ruby-2.3.0@bundler-test/gems/activejob-5.0.0
.....
I didn’t know about the --paths
option.
bundle open
bundle open
is one of my favorite Bundler utilities. It opens the gem in your favorite editor. By “favorite editor”, I mean which ever editor you have setup in the $EDITOR
or $BUNDLER_EDITOR
environment variables.
$ export EDITOR=vim
$ bundle open rake
...opens to the rake gem directory in vim...
This is very useful when I want to dig into a gem and put puts
to see what in heavens is happening in that gem.
bundle outdated
I have seen a few posts about bundle outdated
, whose job is to go through the gems and tell you if a newer version of the gem has been released. Running this on one of my older projects, it took a few minutes (literally) and looked like:
$ bundle outdated
Fetching gem metadata from https://rubygems.org/...........
Fetching version metadata from https://rubygems.org/...
Fetching dependency metadata from https://rubygems.org/..
Resolving dependencies.............................................(lots more dots)
Outdated gems included in the bundle:
* active_model_serializers (newest 0.10.2 4ff33a7, installed 0.10.0.rc5 63e9337) in group "default"
* byebug (newest 9.0.5, installed 8.2.4) in groups "development, test"
* committee (newest 1.14.1, installed 1.14.0) in group "default"
* database_cleaner (newest 1.5.3, installed 1.5.1) in group "test"
....lots more outdated gems...
It lists the gem, the newest version, the installed version, and the group it which the gem resides. You can also pass the name of a specific gem, just to check that one gem:
$ bundle outdated rails
Resolving dependencies... (again, takes a really long time)
There are three command line options:
--local
will check the gem cache without connecting to the remote sources (meaning, Rubygems).--pre
will include pre-release gems, which are excluded by default.--source
checks against a “specific source”, which I presume you pass into the option. The docs don’t really say, and I don’t have a custom Rubygems server.
bundle console
This is one of the utilities that I always forget exists. Running bundle console
will fire up an IRB session with all of the gems in the bundle already require
d (unless, you specify not to require them in the Gemfile).
$ bundle console
irb> Rails
=> Rails
That is very useful, especially if you are developing a gem and want a quick REPL with your gem and all its dependencies loaded. By the way, if you’re a lover of Pry (and, who isn’t?) you can use bundle console config pry
to have Bundler use Pry as the console, presuming you have installed Pry already.
bundle lock
You know how a couple of the commands bundle list
and bundle show
will rebuild the Gemfile.lock file? Well, that’s exactly what bundle lock
does. It rebuilds the lock file, but it does NOT download/install any gems:
$ bundle lock
Fetching gem metadata from https://rubygems.org/...........
Fetching version metadata from https://rubygems.org/...
Fetching dependency metadata from https://rubygems.org/..
Starting resolution (2016-07-27 13:58:05 -0400)
Resolving dependencies...Creating possibility state for rails (167 remaining)
Attempting to activate rails (5.0.0)
Activated rails at rails (5.0.0)
Requiring nested dependencies (activesupport (= 5.0.0), actionpack (= 5.0.0), actionview (= 5.0.0), activemodel (= 5.0.0), activerecord (= 5.0.0), actionmailer (= 5.0.0), activejob (= 5.0.0), actioncable (= 5.0.0), railties (= 5.0.0), bundler (< 2.0, >= 1.3.0), sprocket
s-rails (>= 2.0.0))
Creating possibility state for bundler (< 2.0, >= 1.3.0) (1 remaining)
Attempting to activate bundler (1.11.2)
Activated bundler at bundler (1.11.2)
Requiring nested dependencies ()
...
Writing lockfile to /path/Gemfile.lock
There are a couple of options for bundle lock
:
--lockfile=FILE
: Write the lock file toFILE
. I couldn’t get this to work, btw.--local
: Do not connect to Rubygems, just use the local gem cache.--print
: Print the lock file to STDOUT.--update=[gems]
: This will update the list ofgems
provided, or all gems if none are specified. This is an aggressive update to the gems.
bundle viz
Here’s another utility I have never used. bundle viz
creates a dependency graph of your Gemfile. Let’s take a look:
$ bundle viz
#<LoadError: cannot load such file -- graphviz>
Make sure you have the graphviz ruby gem. You can install it with:
`gem install ruby-graphviz`
Oh, I guess I need gem install ruby-graphviz
(which I do):
$ bundle viz
/<path>/gem_graph.png
If we open that file, it looks like:
That’s for a vanilla Rails app…not bad. I ran it on one of my deployed projects and it looked more, um, involved, to put it mildly. Still, it also showed me that a gem we wrote is probably doing WAY too much based on what it is pulling in as dependencies. Useful and shaming, all at once.
There are a few options:
-f
or--file
allows you to specify the name of the output file.--format
or-F
is how to choose the image format (png, jpg, svg, etc.)--requirements
shows the versions of each required dependency, which are the edges of the graph.--version
shows the versions of the gems, which are the nodes of the graph.- And of course,
--without
allows you to exclude gem groups from the image.
bundle gem
Of all the utilities, bundle gem
is probably the largest. It’s point is to generate all the files needed to start the development of a proper gem. In fact, using bundle gem
is THE way to start a new gem:
$ bundle gem
bundler bundle gem my-new-gem
Creating gem 'my-new-gem'...
create my-new-gem/Gemfile
create my-new-gem/.gitignore
create my-new-gem/lib/my/new/gem.rb
create my-new-gem/lib/my/new/gem/version.rb
create my-new-gem/my-new-gem.gemspec
create my-new-gem/Rakefile
create my-new-gem/README.md
create my-new-gem/bin/console
create my-new-gem/bin/setup
create my-new-gem/.travis.yml
create my-new-gem/.rspec
create my-new-gem/spec/spec_helper.rb
create my-new-gem/spec/my/new/gem_spec.rb
Initializing git repo in /path/my-new-gem
You can see all the things that the command created:
- Of course, we get a Gemfile
- We also get an initialized git repository, complete with
.gitignore
. Oh, and all the new files aregit add
ed to the index. - Standard gem files, including a gemspec, Rakefile, version files, binaries for
console
andsetup
. - In my case, it generated RSpec files for testing. This means that, some time in the past, I have run
bundle gem
, was prompted for which test framework (RSpec or Minitest) and choserspec
. - A Travis CI configuration file that, frankly, I have no idea why it’s part of the default
bundle gem
output. I feel like I must’ve done something to make this happen.
Also, there are a couple items not shown above: a code of conduct and license. Again, I must of opted out of those at some point when I bundle gem
ed in the past.
From an options-perspective, most of the options are around overriding the creation (or omission) of parts of the gem. Options to create or omit the following exist: An executable for the gem (--exe
or --no-exe
), code of conduct (RO) (--coc
, --no-coc
), C extension boilerplate (--ext
, --no-ext
), MIT license (RO) (--mit
, --no-mit
).
The only other two options are -e
to open the .gemspec
in your $EDITOR
and --test=TEST
(RO) to specify rspec
or minitest
.
bundle platform
bundle platform
will tell you what Ruby platforms you have specified or what platforms your gem use:
$ bundle platform
Your platform is: x86_64-darwin15
Your app has gems that work on these platforms:
* ruby
Your Gemfile does not specify a Ruby version requirement.
So, I am running on a Mac, using MRI. If I add ruby "2.3.0"
to my Gemfile, this is what happens:
$ bundle platform
our platform is: x86_64-darwin15
Your app has gems that work on these platforms:
* ruby
Your Gemfile specifies a Ruby version requirement:
* ruby 2.3.0
Your current platform satisfies the Ruby version requirement.
There is a --ruby
option that will simply return the Ruby version specified in the Gemfile, if any.
bundle clean
Sometime you install gems (either on accident or on purpose) that are not used by your application. Also, if you’re using RVM, I know at one point that unused gems, once installed, are never removed from a gemset. So, bundle clean
will take care of this:
$ bundle clean
Cleaning all the gems on your system is dangerous! If you're sure you want to remove every system gem not in this bundle, run `bundle clean --force`.
Wow, that’s a scary message. closes eyes
$ bundle clean --force
Removing ruby-graphviz (1.2.2)
Ah, that just removed the ruby-graphviz
gem from my current gemset, which I installed to show you bundle viz
.
bundle binstubs
Binstubs are scripts that are used to wrap executables in order to set the environment for that executable. It’s like the Decorator Pattern, but for executables. If you’ve ever run bundle exec <some command>
, it’s pretty much the same thing. Bundler is kind enough to offer bundler binstub <a gem in the bundle>
to generate a file you can call that wraps things as needed:
$ bundle binstubs rack
This will create a bin
directory and write binstubs for all the executables in the rack gem. In this case, there is only one (rackup
). So, now we can call bin/rackup
and it will be run in the context of our bundle.
Incidentally, if you run bundle install --binstubs
(RO), then Bundler will generate binstubs for every gem with executables on every bundle install
.
bundle inject
This is a weird one. bundle inject
takes a gem name and version, and injects them into the Gemfile and Gemfile.lock:
$ bundle inject rspec ">0"
...performs resolution...
Added to Gemfile:
rspec (> 0)
The rspec
gem is not installed until bundle install
is run. I have no idea why someone would use this, but I bet there’s good reason (please tell us in the comments if you know what it is)
bundle version
Just for completeness sake, there’s bundle version
:
$ bundle version
Bundler version 1.12.5
By the way, 1.12.5
is the latest Bundler version as of the time of this writing? What version are you running? I bet some of you haven’t updated Bundler in a looooong time…
Done
That’s the Bundler utilities in a nutshell. Well, if by “nutshell” I mean “kind of longer than I intended blog post.” Still, I learned quite a few things and probably got some of it wrong. So, if you see an error, let me know. Go forth and Bundle.