Anatomy of an Exploit: An In-depth Look at the Rails YAML Vulnerability

Tweet

yaml_hackedExploits happens, and this month the Rails and Ruby communities have seen no shortage. From a major exploit in Rails to a slightly different Rubygems.org attack, there has never been a better time to brush up on software security.

Maybe you’re wondering why these vulnerabilities happen in the first place, why they weren’t caught in the first place, or maybe you just want to know the specifics of this attack. We’ll start off by taking a look at the anatomy of a security exploit, and then dive into the gory details of the YAML issue.

Why Insecure Code Happens

No one intends to write insecure software. These vulnerabilities are bugs in the software that can be taken advantage of by others. Unlike a normal bug that will cause your software to not function as intended, a bug that opens up a security hole might still work fine for your task and never actually throw any errors. Often times this is due to side effects in your code. Let’s take a look at some code examples:

Unexpected Input

Let’s say you need to double a number for some reason, so you write a function like this:

def double(number)
  return number * 2
end

You test it out and it works fine:

double(10)
# => 20

While this isn’t an insecure function, it can be used in ways you didn’t intend. Check out what happens when we pass a string to the function:

double("foo")
# => "foofoo"

This isn’t what we wanted nor is it the original intention of the code. Regardless, it’s a side effect of our implementation, if you pass in a string the * operator will be called on it which duplicates the string.

This isn’t actually opening up any security holes, but it just goes to show you how software intended to do one thing can sometimes have unintended abilities.

A Real Example

We’re going to look at all the elements that enabled the YAML attack to happen on Rails system. You should know that all of this information is publicly available already and in the hands of the “bad guys” The hope is this info helps you to write better code and make better decisions in the future.

If you do discover a security vulnerability in a framework or platform, do the responsible thing and report it to the owners before making the information public.

To understand the attack you need to understand YAML, let’s do a quick refresher.

YAML Refresher

YAML (YAML Ain’t Markup Language) is often used by Rubyists to store configuration files. The most famous yml file is probably the config/database.yml used in Rails and it looks like this:

development:
  adapter: postgresql
  encoding: utf8
  database: example_development
  pool: 5
  host: localhost

And can be read in using YAML::load_file

require 'yaml'
database_config = YAML::load_file('database.yml')

puts database_config["development"]
# => {"adapter"=>"postgresql", "encoding"=>"utf8", "database"=>"example_development", "pool"=>5, "host"=>"localhost"}

YAML isn’t just for files, it’s a serialization format like JSON. We can use it to pass complicated objects like strings, numbers, and arrays. There is even support to pass arbitrary user defined objects like User or Product. This is where we get into trouble.

Ruby Objects in YAML

YAML allows us to represent ruby objects directly, the best way to understand how it works is to see it in action. Let’s see how to build a simple Array. You could put this in the top of a .yml file and read it in:

--- !ruby/array:Array
  - jacket
  - sweater

When you parse this it should produce ["jacket", "sweater"]. But we’re not limited to simple Ruby objects like arrays and strings, we can build any class that is in our project like User if we want to:

--- !ruby/hash:User
email: richard@example.com

Now when we load this YAML formatted string in:

string = "--- !ruby/hash:Usern" +
"email: richard@example.com"

YAML::load(string)
 => #<User id: 1, email: "richard@example.com">

We get a User object. Essentially what Ruby is doing is taking all the attributes on the left such as “email” and applying them as values on the right to a new object as if it was a hash. Like this:

user = User.new
user["email"] = "richard@example.com"

This is desired functionality by the creators of YAML, since it gives developers the ability to write and read Ruby objects to disk, like an object database. Unfortunately, this is an often overlooked ability of YAML. Furthermore, this ability has an unintended side effect, much like being able to pass a string into our double() method.

A Vulnerable Object

You might be tempted to think that since we’re only creating objects that have to be defined on our servers, this object instantiation ability wouldn’t be too bad. Unfortunately, let’s see how attackers used these functions to be able to execute arbitrary code.

To take advantage of this exploit, we need a class in our code that evaluates code either on create:

user = User.new

or when we’re setting values

user["email"] = "richard@example.com"

Since both “email” and “richard@example.com” are values we can manipulate through YAML that is the best place to look.

In this case a vulnerable class ActionDispatch::Routing::RouteSet::NamedRouteCollection was found to be exploitable via @lian and announced via Rapid7. To understand how this class is exploitable, let’s try to run arbitrary code on the class directly.

First we instantiate the class:

unsafe_object = ActionDispatch::Routing::RouteSet::NamedRouteCollection.new

Then make a value:

struct        = OpenStruct.new(defaults: {})

Now, we craft an exploit payload foo; eval(puts '=== hello there'.inspect); and set the attribute of the payload equal to the value like this:

unsafe_object["foo; eval(puts '=== hello there'.inspect);" ] = struct

Now when you run this code you will get an error, but before that error any code you put in that eval() will be executed:

# => "=== hello there"

This behavior isn’t inherently unsafe, after all we had to manually build our exploit string and manually instantiate our class. The problem only comes when we put all of these things together.

The Exploit

We know we have a class that runs arbitrary code:

unsafe_object = ActionDispatch::Routing::RouteSet::NamedRouteCollection.new
struct        = OpenStruct.new(defaults: {})
unsafe_object["foo; eval(puts '=== hello there'.inspect);"] = struct
# => "=== hello there"

And we know we can build objects like this using YAML:

--- !ruby/hash:ActionDispatch::Routing::RouteSet::NamedRouteCollection
 'foo; eval(eval(puts '=== hello there'.inspect);': !ruby/object:OpenStruct
   table:
    :defaults: {}

Even with both of these elements our Rails app is still safe, unless users are allowed to send the application arbitrary YAML that gets loaded. As you’ve likely guessed, there was a bug that allowed a malicious user to use an XML request to inject YAML into a Rails app. When you put the three elements together, you have a system that can be completely taken over by a malicious user. If someone can run arbitrary code on your server, you don’t own that server anymore, they do.

Hindsite 20/20

The holes in Rails XML and JSON parsers for different vulnerable versions have been fixed, and some have asked why they weren’t detected and patched earlier. The simple answer is: security is hard. These issues are only obvious in retrospect. Rails and Ruby aren’t any less secure than other frameworks and languages. Security vulnerabilities are bugs at their core, and very difficult to detect. There is almost guaranteed to be insecure software on your laptop/phone/server/garage-door-opener somewhere – it just hasn’t been discovered yet.

Hopefully you’ve gotten a taste for how difficult it can be to spot these vulnerabilities. We can arm ourselves with knowledge and a healthy distrust for user submitted information.

Knowledge is Power

The more people who understand how YAML can be used against a system, the easier it will be to detect a security hole before it’s put in production. When vulnerabilities are discovered, it is important that developers around that software understand the root causes and help spread knowledge to others. I’m not a security researcher or specialist, but just a guy who’s been writing Rails code for a few years. Until I started researching this attack, I didn’t know why, but now I’ll never forget.

Never Trust Your Users

The lesson isn’t that YAML is evil, it’s still as awesome as it ever was. The lesson is that you should never trust your user’s input. If an attacker can’t touch your code or use it in unexpected ways, they can’t exploit it. This is hard to do in practice, but a little extra attention can go a long way.

Stay Up to Date

Subscribe to mailing lists or groups that post security announcements for the major pieces of software you use such as the the Rails Security group. When an announcement goes out, upgrade immediately even if the threat is small. Hopefully you’ve seen how one unexpected use of a software can lead to another. The company I work for, Heroku, considered the threat from this issue so severe that we notified the people who are running vulnerable code.

Even with all the knowledge in the world, it doesn’t do you any good if you continue to run software with known exploits.

Recap

You might not be a security researcher, but hopefully you’ve learned a thing or two not only about this YAML attack, but also the nature of a security exploit. You’ve learned to distrust user input, and you’ve learned that knowledge is power. So go share your knowledge by sending this article out to another developer, they might appreciate it.

Get your free chapter of Level Up Your Web Apps with Go

Get a free chapter of Level Up Your Web Apps with Go, plus updates and exclusive offers from SitePoint.

  • http://n/a ckgagan

    Does this mean that if any site doesn’t take any yaml file as input from users, then the site is secure from this vulnerabilities?

    If you think I still didn’t understand what you explained can you please explain with example how database.yml file can be exploited by users.

    Thanks

    • HD

      The XML and JSON request parsing code within Rails was vulnerable because it loaded arbitrary YAML data during the decode process. The Rails application did not need to do anything specifically with YAML to be exploitable.

      Keep in mind however – if your Rails app does accept YAML from users, either directly or through something like file uploads/database field inject, you will need to take steps to prevent exploitation.

      I have a few apps that need to load untrusted YAML – there are a couple solutions to this (safe_yaml, using another parser, etc). My solution was to create psych_shield, which adds whitelisting to the default Psych parser.

      https://github.com/rapid7/psych_shield

  • bb

    “Rails and Ruby aren’t any less secure than other frameworks and languages.”

    It’s just not true. Take a simpler framework with a focus on security over features and you get a more secure product.

    You statement means that Rails is at least as secure as the best out there. Hard to believe.

    • Matt

      Sure, a “framework” that consisted of a single line:

      puts “Hello World”

      would be totally secure. It would also be TOTALLY USELESS.

  • Chris Jefferson

    Honestly, I think the answer is that YAML is evil. In my applications I currently use JSON as a data format. You seem to be saying I can never trust YAML from untrusted users. At that point, it seems to me it becomes much less useful.

  • http://philosopherdeveloper.com/ Dan

    For devs who may want to accept YAML user input without allowing arbitrary object deserialization (or even using YAML.load internally and wanting to play it safe), the safe_yaml gem provides a stopgap measure: https://github.com/dtao/safe_yaml

  • http://lou.io Lou Kosak

    Great writeup, Richard. Would be good to know a bit more about how this was exploited in practice–from the comments, it sounds like there’s still some confusion about where in a typical rails app this kind of YAML parsing could occur.

  • Prince

    Hi,

    From which versions of Rails, these vulnerabilities have been fixed?

    Also, you have said, JSON and XML parsers in Rails have these vulnerabilites. So, do you mean to say that, in the previous versions, if an attacker gives out a json which has some ruby code like ‘eval(some ruby code)’ as key or value, as the input for some process in that app, he might be able to execute that code?

  • http://rodmclaughlin.com Rod McLaughlin

    “From which versions of Rails, these vulnerabilities have been fixed?”

    And I wonder if anyone is aware of a fix for earlier versions. My hosting company told me to create initializers in config/initializers like this
    ActionDispatch::ParamsParser::DEFAULT_PARSERS.delete(Mime::XML)
    in Rails 3 apps and
    ActionController::Base.param_parsers.delete(Mime::XML)
    in Rails 2 apps.