Semi-Relational Data Modeling with Redis and Ohm

December 8, 2014

ohm

You probably know about Redis. If you’ve never used it, chances are you think of it as another in-memory data-store. But if you think of it like that, you’ll be doing it an injustice: Redis is much more than just a key-value store. It’s a data-structure server including an in-memory dataset. Redis goes beyond the capabilities of a mere key-value store by supporting:

Data structures such as Lists, Sets, and Hashes
Persistence, in both snapshot and log-based modes
Pub/Sub mechanism and -by extension- message queue implementations
Clustering, partitioning, and high availability (via [Sentinel] (http://redis.io/topics/sentinel))

In this tutorial, we’ll create an Object Model that wraps itself around Redis’s data structures, providing us with a way to seamlessly interact with Redis from within our Ruby code. This is provided by the excellent Ohm gem.

Furthermore, we’ll see how we can use this Object Model to emulate some ActiveRecord-like functionality. Before we begin though, let’s get one thing straight: Redis isn’t a relational database and Ohm isn’t an ORM. If you’re expecting to use Redis/Ohm for things like transactions and cascading referential integrity you’ll be bitterly disappointed.

If, however, you want to store and retrieve loosely related entities (think aggregates, not composites) in a blindingly fast manner with the added bonus of data partitioning, high availability, and pub/sub capabilities, then Redis and Ohm were made for you.

Ground Rules

This tutorial does not intend to cover the full range of Ohm or Redis features. It’ll cover a small subset needed to understand and implement the most common data modeling use cases. Ohm works on two facets:

Managing objects in the Ruby memory space
Managing Redis hashes, indices, and other data structures that correspond to those Ruby objects

The synchronization between the two isn’t always obvious, but Ohm offers a variety of methods to manage the synchronicity either implicitly or explicitly. In this tutorial, we’ll prefer the implicit methods, giving us the option of assuming that any operation on our Ohm objects will have an equivalent impact on the corresponding Redis data structures (unless otherwise stated). When, for instance, we update an object’s attribute, we expect the equivalent Redis entries to have been updated accordingly.

Setting Up

We obviously need to install Redis and ensure it’s up and running. Follow the instructions here and get Redis working on your machine.

Also, as mentioned, we’ll be taking advantage of the Ohm gem, so make sure it’s installed:

$ gem install ohm

Defining Classes

For the purpose of this tutorial, we’ll model an authoring system. Define two classes: an Author and a Book class, both derived from Ohm::Model. An Author may produce 0-to-many books. Assume that a Book can only belong to one Author.

A very basic implementation of the system looks as follows:

require 'ohm'

class Author < Ohm::Model
  attribute :f_name
  attribute :l_name
  attribute :email
  attribute :age
end

class Book < Ohm::Model
  attribute :title
end

Attributes

We defined our core class attributes using the attribute class macro. Attribute values are defined as strings, but we can also pass a lambda as a second argument to the macro which can serve as a way of simple typecasting:

attribute :age, lambda { |x| x.to_i }

The above ensures that the age attribute is always returned as an Integer, even if we set it as a String.

To set the value of an attribute, use the update instance method:

irb> author.update(email: 'fred@gmail.com')
#=> <Author>

To get the value of an attribute, use dot notation, as usual:

irb> author.email
#=> fred@gmail.com

The dot notation will get the attribute value from our locally cached Author object. To get the value directly from Redis, use the get instance method:

irb> author.get(:email)
#=> fred@gmail.com

Indexing

We can index attributes with the index class macro. This makes attributes searchable by using the find class method. Let’s make sure we can search for authors by their last_name:

class Author < Ohm::Model
  index :l_name
end

This is now possible:

irb> author = Author.create(f_name: "Joe", l_name: "Bloggs", age: 34, email: "jbloggs@gmail.com")
irb> Author.find(l_name: "Bloggs").size
#=> 1

The find method returns an Ohm::Set object. An Ohm::Set is an unordered list with external behavior similar to that of Ruby arrays. Just like arrays, information can be extracted using familiar methods like first:

irb> Author.find(l_name: "Bloggs").first.email
#=> "jbloggs@gmail.com"

Uniques

If we want an attribute to have a unique value, set it as unique. In our Author example, the email attribute is probably a good candidate to be marked as unique:

class Author < Ohm::Model
  unique :email
end

Now, creating an Author with the same email value as an existing Author will result in a Ohm::UniqueIndexViolation error.

Model Creation

Just like in ActiveRecord, we use the create class method to create a new model object:

irb> author = Author.create(f_name: "John", l_name: "Smith", age: 34, email: "jsmith@gmail.com")

which returns:

#=> #<Author:0x000000023c8118 @attributes={:f_name=>"John", :l_name=>"Smith", :age=>34, :email=>"jsmith@gmail.com"}, @_memo={}, @id="1">

One thing that’s immediately obvious is that Ohm added an id attribute to our new object. This is equivalent to a relational database’s primary key: a unique identifier for that type of object within the current Redis database. We can now use the id attribute value with the [] class method to create an instance with the same attributes whenever we need it:

irb> a = Author[1]
#=> #<Author:0x000000021db990 @attributes={:f_name=>"John", :l_name=>"Smith", :age=>34, :email=>"jsmith@gmail.com"}, @_memo={}, @id="1">

Note that each time we retrieve an Author object based on its id, a new Author object is being created with the same attributes as our targeted object. As such, we can have many Ruby Ohm::Model objects all referring to the same Redis-stored entity.

Deleting and Expiring Keys

Delete an object by calling its delete method, i.e. object.delete. Bear in mind two things when using #delete:

It will delete the object keys from Redis, including uniques, indices, and any sets the object may contain. It will not, however, delete the objects referenced by the object’s sets. These will continue to live happily in Redis’s memory until they either expire, are forced out by Redis’s eviction policy, or are deleted by having their #delete method called.

It will delete the object keys from Redis, but it will not delete the Ruby object from our application’s memory space. Furthermore, the remaining Ruby object can be used to “reanimate” the deleted Redis object keys, producing the following phenomenon:

irb> author = Author.create(f_name: "John", l_name: "Smith", age: 34, email: "abc@gmail.com")
=> #<Author:0x00000001e48e68 @attributes={:f_name=>"John", :l_name=>"Smith", :age=>34, :email=>"abc@gmail.com"}, @_memo={}, @id="13">


irb> Author[13] #check that new author is stored in Redis
=> #<Author:0x00000001e2f238 @attributes={:f_name=>"John", :l_name=>"Smith", :email=>"abc@gmail.com", :age=>"34"}, @_memo={}, @id=13>
irb> author.delete #delete author from Redis
=> #<Author:0x00000001e48e68 @attributes={:f_name=>"John", :l_name=>"Smith", :age=>34, :email=>"abc@gmail.com"}, @_memo={}, @id="13">
irb> Author[13] #author not present in Redis anymore
=> nil
irb> author.update(f_name: 'Jack') #we use the author object to update an attribute
=> #<Author:0x00000001e48e68 @attributes={:f_name=>"Jack", :l_name=>"Smith", :age=>34, :email=>"abc@gmail.com"}, @_memo={}, @id="13">
irb> Author[13] #surprise! the author has been re-created in Redis
=> #<Author:0x00000001cffe58 @attributes={:f_name=>"Jack", :l_name=>"Smith", :email=>"abc@gmail.com", :age=>"34"}, @_memo={}, @id=13>

As long as you’re aware of this behavior, you can use delete as needed. However, it’s safe practice to have a good eviction policy setting in the redis.conf file. A good bet would be:

maxmemory 512mb
maxmemory-policy volatile-lru

This causes Redis to start deleting keys as soon as the 512 (choose your own limit here) megabyte memory limit is reached. Keys will be deleted on a least-recently-used basis only if they have an expire set, which is useful if your Redis database holds mixed data (i.e. not only Ohm data but also some sort of caching data from other applications). If the Redis database’s only purpose is to keep relational data and the timing of the expires is critical, then the noeviction policy setting is recommended.

Associations

Ohm::Model allows us to to define relationships between objects using the #reference and #collection macros. The collection macro is equivalent to ActiveRecord’s has_many method, while reference roughly mimics ActiveRecord’s has_a or belongs_to methods. Add these macros to the existing classes:

require 'ohm'

class Author < Ohm::Model
  attribute :f_name
  attribute :l_name
  attribute :email
  attribute :age, lambda { |x| x.to_i }
  index :l_name
  unique :email

  collection :books, :Book
end

class Book < Ohm::Model
  attribute :title
  reference :author, :Author
end

Create an author and a couple of books:

irb> author = Author.create(f_name: "John", l_name: "Smith", age: 34, email: "jsmith@gmail.com")
=> #<Author:0x0000000279a480 @attributes={:f_name=>"John", :l_name=>"Smith", :age=>34, :email=>"jsmith@gmail.com"}, @_memo={}, @id="1">

irb> book1 = Book.create(title: 'Moby Dick')
=> #<Book:0x00000002776fa8 @attributes={:title=>"Moby Dick"}, @_memo={}, @id="1">

irb> book2 = Book.create(title: 'The Hobbit')
=> #<Book:0x000000026348c0 @attributes={:title=>"The Hobbit"}, @_memo={}, @id="2">

The reference macro has given our book objects an author attribute, which we can use to set the author of our books:

irb> book1.update(author: author)
=> #<Book:0x00000002776fa8 @attributes={:title=>"Moby Dick", :author_id=>"1"}, @_memo={}, @id="1">
irb> book2.update(author: author)
=> #<Book:0x000000026348c0 @attributes={:title=>"The Hobbit", :author_id=>"1"}, @_memo={}, @id="2">

Ohm has added a new author_id attribute to the book objects. This is a foreign key attribute referencing the author associated with this book. The collection macro used in the Author class ensures that the author can track all their books:

irb> author.books.each {|b| puts b.title}
=> Moby Dick
=> The Hobbit

In addition, we can use some of Ohm::Set‘s sorting and finding methods, like #include?, #find, #sort, and #sort_by to filter and sort the book collection.

Talking Directly to Redis

If you ever need to run a Redis command that isn’t abstracted by an Ohm method, use the redis.call method:

irb> Ohm.redis.call "FLUSHDB"
=> "OK"

Here, we’re telling Redis to clear the current database. We can run any of the Redis-cli commands in this manner from our Ruby code.

Summary

The Ohm – Redis combination offers great potential, of which only a small amount is described here. With very little effort we can leverage loads of powerful NoSQL features with a touch of relational goodness, a perfect combination for many modern applications.