Semi-Relational Data Modeling with Redis and Ohm
You probably know about Redis. If you’ve never used it, chances are you think of it as another in-memory data-store. But if you think of it like that, you’ll be doing it an injustice: Redis is much more than just a key-value store. It’s a data-structure server including an in-memory dataset. Redis goes beyond the capabilities of a mere key-value store by supporting:
- Data structures such as Lists, Sets, and Hashes
- Persistence, in both snapshot and log-based modes
- Pub/Sub mechanism and -by extension- message queue implementations
- Clustering, partitioning, and high availability (via [Sentinel] (http://redis.io/topics/sentinel))
In this tutorial, we’ll create an Object Model that wraps itself around Redis’s data structures, providing us with a way to seamlessly interact with Redis from within our Ruby code. This is provided by the excellent Ohm gem.
Furthermore, we’ll see how we can use this Object Model to emulate some ActiveRecord-like functionality. Before we begin though, let’s get one thing straight: Redis isn’t a relational database and Ohm isn’t an ORM. If you’re expecting to use Redis/Ohm for things like transactions and cascading referential integrity you’ll be bitterly disappointed.
If, however, you want to store and retrieve loosely related entities (think aggregates, not composites) in a blindingly fast manner with the added bonus of data partitioning, high availability, and pub/sub capabilities, then Redis and Ohm were made for you.
Ground Rules
This tutorial does not intend to cover the full range of Ohm or Redis features. It’ll cover a small subset needed to understand and implement the most common data modeling use cases. Ohm works on two facets:
- Managing objects in the Ruby memory space
- Managing Redis hashes, indices, and other data structures that correspond to those Ruby objects
The synchronization between the two isn’t always obvious, but Ohm offers a variety of methods to manage the synchronicity either implicitly or explicitly. In this tutorial, we’ll prefer the implicit methods, giving us the option of assuming that any operation on our Ohm objects will have an equivalent impact on the corresponding Redis data structures (unless otherwise stated). When, for instance, we update an object’s attribute, we expect the equivalent Redis entries to have been updated accordingly.
Setting Up
We obviously need to install Redis and ensure it’s up and running. Follow the instructions here and get Redis working on your machine.
Also, as mentioned, we’ll be taking advantage of the Ohm gem, so make sure it’s installed:
$ gem install ohmDefining Classes
For the purpose of this tutorial, we’ll model an authoring system. Define two classes: an Author and a Book class, both derived from Ohm::Model. An Author may produce 0-to-many books. Assume that a Book can only belong to one Author.
A very basic implementation of the system looks as follows:
require 'ohm'
class Author < Ohm::Model
attribute :f_name
attribute :l_name
attribute :email
attribute :age
end
class Book < Ohm::Model
attribute :title
endAttributes
We defined our core class attributes using the attribute class macro. Attribute values are defined as strings, but we can also pass a lambda as a second argument to the macro which can serve as a way of simple typecasting:
attribute :age, lambda { |x| x.to_i }The above ensures that the age attribute is always returned as an Integer, even if we set it as a String.
To set the value of an attribute, use the update instance method:
irb> author.update(email: 'fred@gmail.com')
#=> <Author>To get the value of an attribute, use dot notation, as usual:
irb> author.email
#=> fred@gmail.comThe dot notation will get the attribute value from our locally cached Author object. To get the value directly from Redis, use the get instance method:
irb> author.get(:email)
#=> fred@gmail.comIndexing
We can index attributes with the index class macro. This makes attributes searchable by using the find class method. Let’s make sure we can search for authors by their last_name:
class Author < Ohm::Model
index :l_name
endThis is now possible:
irb> author = Author.create(f_name: "Joe", l_name: "Bloggs", age: 34, email: "jbloggs@gmail.com")
irb> Author.find(l_name: "Bloggs").size
#=> 1The find method returns an Ohm::Set object. An Ohm::Set is an unordered list with external behavior similar to that of Ruby arrays. Just like arrays, information can be extracted using familiar methods like first:
irb> Author.find(l_name: "Bloggs").first.email
#=> "jbloggs@gmail.com"Uniques
If we want an attribute to have a unique value, set it as unique. In our Author example, the email attribute is probably a good candidate to be marked as unique:
class Author < Ohm::Model
unique :email
endNow, creating an Author with the same email value as an existing Author will result in a Ohm::UniqueIndexViolation error.
Model Creation
Just like in ActiveRecord, we use the create class method to create a new model object:
irb> author = Author.create(f_name: "John", l_name: "Smith", age: 34, email: "jsmith@gmail.com")which returns:
#=> #<Author:0x000000023c8118 @attributes={:f_name=>"John", :l_name=>"Smith", :age=>34, :email=>"jsmith@gmail.com"}, @_memo={}, @id="1">One thing that’s immediately obvious is that Ohm added an id attribute to our new object. This is equivalent to a relational database’s primary key: a unique identifier for that type of object within the current Redis database. We can now use the id attribute value with the [] class method to create an instance with the same attributes whenever we need it:
irb> a = Author[1]
#=> #<Author:0x000000021db990 @attributes={:f_name=>"John", :l_name=>"Smith", :age=>34, :email=>"jsmith@gmail.com"}, @_memo={}, @id="1">Note that each time we retrieve an Author object based on its id, a new Author object is being created with the same attributes as our targeted object. As such, we can have many Ruby Ohm::Model objects all referring to the same Redis-stored entity.
Deleting and Expiring Keys
Delete an object by calling its delete method, i.e. object.delete. Bear in mind two things when using #delete:
- It will delete the object keys from Redis, including uniques, indices, and any sets the object may contain. It will not, however, delete the objects referenced by the object’s sets. These will continue to live happily in Redis’s memory until they either expire, are forced out by Redis’s eviction policy, or are deleted by having their
#deletemethod called. -
It will delete the object keys from Redis, but it will not delete the Ruby object from our application’s memory space. Furthermore, the remaining Ruby object can be used to “reanimate” the deleted Redis object keys, producing the following phenomenon:
irb> author = Author.create(f_name: "John", l_name: "Smith", age: 34, email: "abc@gmail.com") => #<Author:0x00000001e48e68 @attributes={:f_name=>"John", :l_name=>"Smith", :age=>34, :email=>"abc@gmail.com"}, @_memo={}, @id="13"> irb> Author[13] #check that new author is stored in Redis => #<Author:0x00000001e2f238 @attributes={:f_name=>"John", :l_name=>"Smith", :email=>"abc@gmail.com", :age=>"34"}, @_memo={}, @id=13> irb> author.delete #delete author from Redis => #<Author:0x00000001e48e68 @attributes={:f_name=>"John", :l_name=>"Smith", :age=>34, :email=>"abc@gmail.com"}, @_memo={}, @id="13"> irb> Author[13] #author not present in Redis anymore => nil irb> author.update(f_name: 'Jack') #we use the author object to update an attribute => #<Author:0x00000001e48e68 @attributes={:f_name=>"Jack", :l_name=>"Smith", :age=>34, :email=>"abc@gmail.com"}, @_memo={}, @id="13"> irb> Author[13] #surprise! the author has been re-created in Redis => #<Author:0x00000001cffe58 @attributes={:f_name=>"Jack", :l_name=>"Smith", :email=>"abc@gmail.com", :age=>"34"}, @_memo={}, @id=13>
As long as you’re aware of this behavior, you can use delete as needed. However, it’s safe practice to have a good eviction policy setting in the redis.conf file. A good bet would be:
maxmemory 512mb
maxmemory-policy volatile-lruThis causes Redis to start deleting keys as soon as the 512 (choose your own limit here) megabyte memory limit is reached. Keys will be deleted on a least-recently-used basis only if they have an expire set, which is useful if your Redis database holds mixed data (i.e. not only Ohm data but also some sort of caching data from other applications). If the Redis database’s only purpose is to keep relational data and the timing of the expires is critical, then the noeviction policy setting is recommended.
Associations
Ohm::Model allows us to to define relationships between objects using the #reference and #collection macros. The collection macro is equivalent to ActiveRecord’s has_many method, while reference roughly mimics ActiveRecord’s has_a or belongs_to methods. Add these macros to the existing classes:
require 'ohm'
class Author < Ohm::Model
attribute :f_name
attribute :l_name
attribute :email
attribute :age, lambda { |x| x.to_i }
index :l_name
unique :email
collection :books, :Book
end
class Book < Ohm::Model
attribute :title
reference :author, :Author
endCreate an author and a couple of books:
irb> author = Author.create(f_name: "John", l_name: "Smith", age: 34, email: "jsmith@gmail.com")
=> #<Author:0x0000000279a480 @attributes={:f_name=>"John", :l_name=>"Smith", :age=>34, :email=>"jsmith@gmail.com"}, @_memo={}, @id="1">
irb> book1 = Book.create(title: 'Moby Dick')
=> #<Book:0x00000002776fa8 @attributes={:title=>"Moby Dick"}, @_memo={}, @id="1">
irb> book2 = Book.create(title: 'The Hobbit')
=> #<Book:0x000000026348c0 @attributes={:title=>"The Hobbit"}, @_memo={}, @id="2">The reference macro has given our book objects an author attribute, which we can use to set the author of our books:
irb> book1.update(author: author)
=> #<Book:0x00000002776fa8 @attributes={:title=>"Moby Dick", :author_id=>"1"}, @_memo={}, @id="1">
irb> book2.update(author: author)
=> #<Book:0x000000026348c0 @attributes={:title=>"The Hobbit", :author_id=>"1"}, @_memo={}, @id="2">Ohm has added a new author_id attribute to the book objects. This is a foreign key attribute referencing the author associated with this book. The collection macro used in the Author class ensures that the author can track all their books:
irb> author.books.each {|b| puts b.title}
=> Moby Dick
=> The HobbitIn addition, we can use some of Ohm::Set‘s sorting and finding methods, like #include?, #find, #sort, and #sort_by to filter and sort the book collection.
Talking Directly to Redis
If you ever need to run a Redis command that isn’t abstracted by an Ohm method, use the redis.call method:
irb> Ohm.redis.call "FLUSHDB"
=> "OK"Here, we’re telling Redis to clear the current database. We can run any of the Redis-cli commands in this manner from our Ruby code.
Summary
The Ohm – Redis combination offers great potential, of which only a small amount is described here. With very little effort we can leverage loads of powerful NoSQL features with a touch of relational goodness, a perfect combination for many modern applications.
