Chef is billed as “A systems integration framework, built to bring the benefits of configuration management to your entire infrastructure”. It doesn’t matter how many times I read that, I still hear a ‘whoosh’ over my head. Put simply, we can manage server configurations through good old familiar Ruby.
The gist of Chef is we create cookbooks, these cookbooks use the Chef DSL to install and configure packages we require for our servers. How many times have you looked up the same tutorial for installing and configuring something like MySQL? With Chef we have a cookbook, under source control that we can maintain and use to spin up new environments in no time. If you have ever had to create your own staging and production environments, and maintain consistency as your application evolves this should already be sounding like good news.
Chef comes in a couple of flavors. Today we will focus on Chef Solo, unlike Chef Server, Chef Solo works with everything “on disk”. The cookbooks, and anything else we require (which we will get into in a minute) have to be on the server we are configuring.
The Basics of Chef Solo
Before going any further it’s probably best we establish the components available to us when creating Chef scripts, previously we called this “anything else we require.”
Recipes for Resources
Quite simply the resource is what we are trying to configure on the server. Taking the example of MySQL, we write a recipe for the that resource. The Recipe is written in Ruby and establishes how we want the resource configured using block style syntax.
package "mysql-client" do
package_name value_for_platform("default" => "mysql-client")
action :install
endThe source above is a truncated example of the Opscode recipe for MySQL. Although it pretty much makes sense to read as is, you may wonder about the value_for_platform method. It simply selects the correct package name of the resource you are trying to configure based on the platform. I have condensed this for Debian based systems, but we can pass a plain old Ruby Hash to make Chef use the correct package. If you hadn’t clicked ahead to the original source yet:
package_name value_for_platform(
[ "centos", "redhat", "suse", "fedora", "scientific", "amazon"] => { "default" => "mysql" },
"default" => "mysql-client"
)How does Chef know what platform we are running on? The answer to that question leads nicely into Nodes.
Nodes
A Node is our server. Our Cookbook of Recipes are applied to a Node. We could end it here, but I recently found it handy to peel back the curtain on Nodes.
To find out everything it needs to know about our Node/Server, Chef uses a gem named Ohai. You can check out how Chef does this yourself.
require 'ohai'
o = Ohai::System.new
o.all_plugins
puts o[:platform] # => "linuxmint"Just Like Mama Used to Make it
That is quite enough theory, now it’s time to get our hands dirty. Grab your favorite server distro (I’m going to use Ubuntu on VirtualBox). I find VirtualBox great for its snapshot capabilities, allowing us to test out a Chef run and roll it back if something goes wrong.
The directory structure of the project is pretty simple. We will have a cookbook, recipes and support files.
mkdir cookbooks cookbooks/main cookbooks/main/recipes
touch solo.rb cookbooks/main/recipes/default.rb node.jsonThe next step is to open our solo.rb file and add the following:
cookbook_path File.join(File.dirname(File.expand_path(__FILE__)), "cookbooks")
json_attribs File.join(File.dirname(File.expand_path(__FILE__)), "node.json")As you probably guessed, all we are doing in the first line is telling Chef where to find our Cookbooks. We then pass the location of node.json to json_attribs. Our file node.json is where we will set specific attributes for our Chef run.
I showed earlier that Chef will set up a lot of information about our environment. Some of these attributes can be overwritten by our recipes. There is a precedence for these attributes. In this example, anything passed to json_attribs will take highest precedence.
It is time to look at our node.json file now. One thing I like to use is a common temp directory, it’s a handy place for downloading packages, files etc. Chef does not give us this so we can start by adding that. While we are doing that, we better setup our main recipe. This will simply include other recipes for our server, and makes a nice drop off point if we need to add/remove any recipes.
{
"temp_dir":"/tmp",
"run_list":["recipe[main]"]
}Now that we have some basic setup in place, let’s look at what we want our Chef run to do. For this example, we will keep it pretty short and sweet. There are a host of cookbooks already available for you to get a good starting point. Instead, we will look at installing a package using available cookbooks (Apache) and we will install RVM.
The first task for apache is simply a case of copying the existing cookbook into our cookbooks directory.
RVM on the other hand we will have to do a bit more work. First of all we create a rvm directory along with our other cookbooks and a recipes folder.
mkdir cookbooks/rvm cookbooks/rvm/recipes
touch cookbooks/rvm/default.rbIf we look at RVMs release notes we see there are a number of dependencies. Job number 1 for our RVM recipe is to meet these.
%w(build-essential openssl libreadline6 libreadline6-dev curl git-core zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-0 libsqlite3-dev sqlite3 libxml2-dev libxslt-dev autoconf libc6-dev ncurses-dev automake libtool bison subversion).each do |pkg|
package pkg
endI know some of these may already be in place on our system, but I like to treat each recipe in isolation. If the package is already there, what the hey? Using the package method we can ensure our dependencies are in place.
The next step is to install RVM itself. I’m guessing you will have already done this before and know about the great bin installer. Well, Chef supports these kind of installs using its bash provider. This essentially evaluates the block as a bash command.
bash "install RVM" do
user node[:user]
code "sudo bash < <(curl -s https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer )"
endThe Chef DSL is just great to follow, I setup a bash script to be run as the current user on the system and execute the RVM install script. Don’t get caught in the trap of executing this as user "root", as I did. Following the RVM documentation , there is a difference.
Garnish and Season
Now that we have a couple of recipes in place, it’s time to sort out the run order and how we get the Chef script onto the box/node. The first one is simple: In cookbooks/main/recipes/default.rb add your cookbooks.
include_recipe "apache2"
include_recipe "rvm"It’s just Ruby so we can add new cookbooks, comment out lines, or whatever you are used to doing in your ruby files. Now we have to look at getting the code onto the box we are interested in. The best way I have seen is to use a bash script. This was originally pointed out to me by my esteemed college Colin Gemmell, who done most of the grunt work for this bash script. Me? I just tweaked it a little.
#!/bin/bash
sudo apt-get update
sudo apt-get install git-core -y -q
sudo apt-get install build-essential -y -q
sudo apt-get install ruby -y -q
sudo apt-get install ruby-dev -y -q
sudo apt-get install libopenssl-ruby -y -q
sudo apt-get install rubygems -y -q
sudo apt-get install irb -y -q
sudo apt-get install rake -y -q
sudo apt-get install curl -y -q
sudo gem install chef --no-ri --no-rdoc
git clone git://github.com/bangline/demo_cookbooks.git /tmp/chef_run
cd /tmp/chef_run
sudo /var/lib/gems/1.8/bin/chef-solo -c /tmp/chef_run/solo.rb -j /tmp/chef_run/node.jsonIt’s pretty straight forward. All the dependencies I need are setup first (silenced so I an not asked “Is this OK? [Y/N]” for every package) before cloning the project and running chef-solo on it. The Chef script is passed in the configuration of solo.rb and the JSON attributes from node.json.
After the Chef script completes, testing it is easy. For Apache, just wget http://127.0.0.1 to download the “It works!” page. For RVM, you will have to logout and log back in again, after which rvm notes will produce the RVM documentation. The next step would to modify this recipe to install a default Ruby version of course.
Wrapping up
Chef is a huge topic and the documentation for it is improving over time. Hopefully this introduction will give you enough of an incentive to start using Chef in some way when thinking of starting some manual system ops. There is still so much to cover, setting up users, vhosts or even setting a default Ruby to use with RVM and install Passenger or something alongside so we have a complete hosting solution for our applications. If you go ahead and test out the demo (source can be found in the usual place), you will find a couple of annoying warnings when restarting Apache. This one is not a big deal, we should really have tinkered with the NameVirtualHost configuration. Nonetheless, considering this was just a quick copy/paste job, we do get a working Apache2 setup.
The greatest resource for me has been examining the existing cookbooks from Opscode and 37Signals. A lot of work has been done for a lot of common packages. These days I struggle to find a resource that does not have a cookbook available in some form to use “as is” or minimal customizations.
So, next time you are setting up an environment for your applications invest some time building it on a local virtual host using VirtualBox. Like most things it will take extra time, it will frustrate you, but what you end up with pays of ten fold. Now who else is hungry?
Dave KennedyDave is a web application developer residing in sunny Glasgow, Scotland. He works daily with Ruby but has been known to wear PHP and C++ hats. In his spare time he snowboards on plastic slopes, only reads geek books and listens to music that is certainly not suitable for his age.
