Give Vagrant a Place in Your Workflow

Recently I’ve been heavily involved in dev-ops. It’s a bit of a guilty pleasure for me really. These days it’s almost expected that we developers know something about setting up boxes correctly for our applications. It makes sense, we spent a lot of time getting our local environments just the way we want it. If we decide we want to utilize Redis into our applications we install it, get to know it a little and then get back to being productive.

I’m by no means a “sys-admin” kind of guy and I had to google what a neckbeard was. But I do love fooling around with virtual machines, EC2 instances and the like. In the past I have always leaned toward cooking with chef. It is my personal opinion the developer should have an understanding of the infrastructure their application will live within. It may not be perfect, probably could do with optimization and more security on top, but it should be enough for the application to get its first foray into the wild. Then, if it looks successful, hire someone who knows what they are doing to solidify that infrastructure.

I have written before about using chef for this very purpose. I used chef solo to provision a virtual machine. That’s all well and good, but being honest it’s not ideal. Testing chef scripts is cumbersome, you need to snapshot, test, blow away, lather, rinse, repeat. It is very involved. Then, say you need to share the environment with a member of your team. They need to go through some degree of setup, get to know what the chef run is doing, then they get productive. If only there was a way to make all this setup trivial. If you have read this far, you already know I’m talking about Vagrant.

The Basics

Here is the whistle stop tour if you haven’t used Vagrant before. Basically it runs on top of VirtualBox. It’s a dependency, so if you haven’t got it installed, get it. Then download and install Vagrant for your system. I’m currently running Ubuntu 12.10 and after running the deb installation package I added /opt/vagrant/bin to my PATH. (I also use zsh, so for me I added PATH=/opt/vagrant/bin:$PATH line to my ~/.zshrc file).

Once you have it installed, you should be able to run vagrant -h and get a list of commands to run for vagrant.

Vagrant has a concept of “Boxes”. It’s best to think of these boxes as the base configuration for the environment you wish to configure. An example of a base box could be a Debian 64bit server with Apache installed, configured for virtual hosting.

A comprehensive repository of pre-built boxes is provided at vagrantbox.es. We will get into creating our own box later, but for now lets pick something comfortable. You add a box with the command:

vagrant box add precise64 http://files.vagrantup.com/precise64.box

It takes a while to download but you only have to do these kind of things once. Once it has downloaded and added the the box, you can create as many projects based on this base configuration (which we have aptly named “precise64”) as you wish.

Create a directory vagrant_test and execute vagrant init precise64. This will create a Vagrantfile. The Vagrantfile file contains tons of comments, all of which if you are new to vagrant will be somewhat alien. This is similar to the Rails routes.rbfile when creating a new application: You may only use some of it, but it will be a good reference down the line.

We are now ready to startup our first environment. Run vagrant up, let it boot up the virtual machine headless (i.e you will not see the Virtual Box GUI). Once it has completed, run vagrant ssh and, presto, you will be in the shell of our freshly created virtual environment. At this point you can exit out of the ssh session. If you now type vagrant status, you should see the virtual machine is running. To stop the virtual machine type vagrant halt. The halt command gracefully shuts down the VM ane running status again will show the VM is powered off.

Getting Something Running

Up to this point, all we have done is spin up a new VM, which is pretty pointless. Surely, we need some code on there. If you have used Virtual Box, in the past you will know you can share folders between the guest and host environments. In this instance we want our application code to be shared.

In the Vagrantfile look for the line config.vm.share_folder "v-data", "/vagrant_data", "../data". As the preceding comment tells us, the share_folder configuration takes 3 parameters, a name or identifier, the location on the VM the shared folder will reside, and the location on the host OS.

I intend to ship the Vagrant file with my application, and having used EngineYard for a few years I usually go for a shared folder mounted at location /data:

config.vm.share_folder "v-data", "/data", "./"

By using the above configuration, once we boot (vagrant up) and ssh into the VM, navigating to the /data directory will show that the current directory is shared. Go ahead, add/modify files and you will see it reflected on the VM. Pretty neat stuff, I think you will agree.

By now, you should begin to see the benefits of using something like Vagrant. I can now create a reproducible enviroment using nothing more than a text file included in the repository of my application. No more “works on my machine”, no more hair pulling. Looking around the open source projects on Github these days you will sometimes find the odd Vagrantfile in the top level of the project, I think this is an awesome practice.

Provisioning our Services

We have our base VM and the codebase installed on it, but it’s still pretty vanilla. At some point we will want to customize our environment by adding new packages such as MySQL or a ruby version manager like rbenv.

Vagrant, in my opinion, has really nailed this one. If you are a Chef fan, Puppet die hard or a simple shell purist, you can provision services with Vagrant. I’ll take a quick look at doing this with Chef Solo as that is my tool of choice.

Within the vagrant file we can configure where our cookbooks (Chef uses ‘cookbooks...) live. By default, Vagrant will look for a directorycookbooks` at the top level of the current directory, but it will take a relative path. At this point, it is worthwhile to set up some cookbooks.

I create a cookbooks project and submodule cookbooks I need from the Opscode repository. I then configure the chef run like so:

config.vm.provision :chef_solo do |chef|
    chef.cookbooks_path = "../cookbooks"
    chef.add_recipe 'apt'
    chef.add_recipe 'build-essential'
    chef.add_recipe 'mysql::client'
    chef.add_recipe 'mysql::server'
    chef.json = {
      "mysql" => {
          "server_debian_password" => "secret",
          "server_root_password" => "secret",
          "server_repl_password" => "secret"
      }
    }
  end

It’s very straightforward to follow, even if you are not all that familiar with Chef. All I have done here is specify where my cookbooks live add a couple of staple recipies (apt and build-essential). I’ve also added the MySQL client and server to the chef run, specifying some attributes that I want (passwords for root) If you look at the mysql cookbook these are explained in the README.

Now with our existing vagrant VM we can run vagrant reload if it is running or vagrant up and our VM is ready with MySQL installed and raring to go.

A small caveat when using Chef with vagrant is, just like Chef on it’s own you can pass a URL to a tarball like so:

config.vm.provision :chef_solo do |chef|
    chef.recipe_url = "http://github.com.com/company_name/cookbooks/archive/v0.1.0.tar.gz"
  end

In the example above I’m looking at GitHub. Every time you tag a project a tarball is created, perfect for vagrant setup.

Back to Base-ics

Now comes the real reason I love working with Vagrant. You remember way back at the start of the article we used a pre-made box from vagrantbox.es. Well we have just finished a base installation of all the things we may need for a project, or maybe just a base setup for Rails applications in general? Well we can create a “box” for this VM to take with us onto our next project or pass along to a friend.

In order to create the packent, use the vagrant package command. Simply running this in the project will package up the VM for easy distribution. I like to name the box file properly so run vagrant package --output base_rails.box

Once complete you will have a base_rails.box file in the root of your project. In future projects we can include the package in the Vagrantfile like so

config.vm.box = "base_rails"
# or if it is available for download

# config.vm.box_url = "http://domain/for/base_rails.box

The package command can also take a --vagrantfile option. Don’t be confused with the Vagrantfile we have been working with throughout the article. Instead this is a Vagrantfile.pkg that specifies configuration only for the package you wish to build.

For instance, if we wish to include the shared folder functionality we would create a Vagrantfile.pkg, add the following and run the vagrant package command.

Vagrant::Config.run do |config|
  config.vm.share_folder "v-data", "/data", "./"
end

Now once the VM has been packaged, the distribution will automatically share the current working directory with the VM.

Now anyone with access to the box file and can simply initialize and vagrant up the box. (vagrant init base_rails and vagrant up).

Wrapping Up

I hope this article has you pretty excited about Vagrant. I started using it as a way to provision test servers for my Chef runs. I have really seen the potential for sharing development environments, which I believe is the real reason Vagrant exists in the first place.

It really doesnt end there though. A recent release on the HashiCorp blog previewed Vagrant being used with AWS. I personally cannot wait for this to hit a release state so that I can get my mitts on it. Being able to provision environments outside my own development machine is a huge win. No longer do we worry about ensuring conformity across environments, instead we can use vagrant to test our applications where they will someday live.

Provisioning services like this is a real swiss army knife for a developer. We get some ‘best practice’ from the curators of the cookbooks we use for the chef run and a nice workflow with Vagrant. It is worth pointing out, and I may only be speaking for myself, I use the provisions and chef runs to only get me so far. It’s dev-ops. I have enough knowledge to be dangerous when it comes to provision servers for personal applications. I have no need or desire to go any further tweaking my boxes. Anything production worthy I recommend getting some expert advice from a sys-admin.

But that said, Vagrant has made it crazy fun. And it’s only going to get more fun when the AWS-VM-Ware integration is released on a MIT licence in the next month or so.