Using Wisper to Decompose Applications

Working on a new project is fun and feels super productive. Features can be added quickly and prototyping ideas leads to a tight feedback loop ideal for Agile/Lean-esque planning.

However, as time goes by, I found that the initial momentum enjoyed early on in the project decreases and doing test-first development of Rails applications becomes difficult. In contrast, when writing libraries a test-first approach seems natural.

The predominate voice in the Rails community advocates the so-called “Rails Way”. This is the golden prescribed path, the reason many of us, no doubt, took to Rails in the first place. It’s the beauty of Rails and its famed productivity boosts.

But somehow these gains seemed frustratingly short lived.

After working on several none-trivial Rails applications, I began to recognise the symptoms and by listening to others I began to piece together the causes of my frustration.

The Inspiration

I first heard of Hexagonal Rails in a talk by Matt Wynne at GoRuGo 2012. I immediately recognised the problems he described in my own code, particalluy controllers written in a procedural style, littered with if statements. Matt shows how to overcome this using the pub-sub pattern, which is what I will be demostrating in this article.

The idea behind Hexagonal Rails is the separation of the core application from the delivery mechanism, something echoed in the keynote talk, “Architecture the Lost Years”, by Robert Martin and “Objects on Rails” by Avdi Grimm.

The take away idea is your app is not a Rails app. Rails is nothing more than a framework used to wrap your app in the HTTP protocol. It’s a detail and we should be concerned with Object-Orientated development, not Rails-orientated development.

Yet Another Ebay Clone

Imagine we are working for a startup that is building an ebay clone for a niche market. On the product pages someone can bid on an item by entering an amount and pressing a “Bid Now” button.

In the beginning, the story is very simple; create a new bid and let the user know if it was successful or not. Easy, use the “Rails Way”, deliver and ship. Agile-ing-hell-fire.

Sometime later, new stories are added to the backlog:

• Email the seller of the item
• Update the UI of item watchers in realtime
• Store a statistic
• Update the activity feeds for all subscribers of the item

The code for these new stories needs to touch several models and hit an external API. Wherever it goes, (model, observer, or controller) it’s going to introduce dependencies that are not core concerns of “bidding on an item”. Testing and refactoring will become difficult and productivity slows.

Where else could this code go? We already have the words: “create bid” or CreateBid. It sounds like it would be a good candidate for a new object that models the creation of a bid.

This type of object is often refereed to as a service, use case, or command. I will use the word ‘service’.

Services are typically used to orchestrate interactions across multiple models and/or interact with external systems (e.g. HTTP and SMTP).

A service object seems like a good idea, but haven’t we just moved the problem?

Aren’t our services just going to get bloated like out models? Yes!

Wispered Service Objects

Wisper is a library which supplements Ruby’s Methods and Attributes with Events.

We will use it to allow our service objects can publish events to other objects in a decoupled manner.

The service object can perform its primary action and then broadcast the outcome, an event, to any listeners. Listeners are added to the service object within the context in which the service is executed, in this case, the context is the controller. Lets take a look at an example:

The controller:

class BidsController
  def new
    @bid = Bid.new
  end

  def create
    service = CreateBid.new
    service.subscribe(WebsocketListener.new)
    service.subscribe(ActivityListener.new)
    service.subscribe(StatisticsListener.new)
    service.subscribe(IndexingListener.new)
    service.on(:reserve_item_successfull) { |bid| redirect_to item_path(bid.item) }
    service.on(:reserve_item_failed)      { |bid| @bid = Bid.new(bid_params); render :action => 'new' }
    service.execute(current_user, bid_params)
  end

  private

  def bid_params
    params[:bid]
  end
end

The service:

class CreateBid
  include Wisper::Publisher  

  def execute(performer, attributes)
    bid = Bid.new(attributes)
    bid.user = performer
    if bid.valid?
      bid.save
      notify_seller_of(bid)
      broadcast(:bid_successful, performer, bid) 
    else 
      broadcast(:bid_failed, performer, bid)
    end
  end

  private
  
  def notify_seller_of(bid) 
    BidMailer.new_bid(bid).deliver 
  end 
end

Some listeners:

class WebsocketListener
  def bid_successful(performer, bid)
    Pusher.trigger("item_#{bid.item_id}", 'new_bid', :amount => bid.amount)
  end
end

class ActivityListener
  def reserver_item_successful(performer, bid)
   # ...
  end
end

The first thing you might notice is that the controller has no if‘s, is very readable, and has a pleasing shape.

We have moved the auxiliary concerns (websocket notifications, activity feed) in to seperate objects.

The controller (context) does the following:

• Creates a service object
• Wires up any listeners that need to know about the response
• Executes the service passing in data pulled from the HTTP request

Interestingly, CreateBid does not tell listeners to do something, its tells them something happened. This is subtle but important. CreateBid does not know the context in which it is being executed (it might not be a web app) or who is listening. Objects which trust, not control, others only need to make choices based on their own state. This creates a clear boundary between objects and their responsibilities and does not constrain choices we can make in the future about how use our domain objects.

If we need to add a new feature as part of the registration process we have a two choices:

• Add code to the service object. Appropriate if the feature is intrinsically part of the use case being expressed by the service. In my example I decided sending an email is part of registration process.
• Add a listener to the service object. Most appropriate if the feature is auxiliary to the use case expressed by the service, should not happen in all circumstances in which the service is executed and/or is specific to the delivery mechanism.

Publishing events

Simply include Wisper::Publisher in any object to allow it to publish events.

class MyPublisher
 include Wisper::Publisher

  def do_something
    publish(:done_something, 'hello', 'world')
  end
end

The first argument is the event to be broadcast followed by any number of arguments that should be passed to the listeners.

When publish is called done_something will be called on all listeners which have (respond_to?) this method. The method must have an argument signature which can accept the arguments being published, so in the above example a listener should have:

def done_something(greeting, location)
  # ...
end

The publish method is also aliased as broadcast and emit.

Subscribing to events

my_publisher = MyPublisher.new
my_publisher.subscribe(MyListener.new)

We can also optionally constrain which events are sent to a listener:

create_bid.subscribe(WebsocketListener.new, :on => :bid_successful)

You can also pass an array to on if you need to specific multiple events.

If the listener has a method which does not match the name of the event being broadcast you can map it a different method name using with.

create_bid.subscribe(BidListener.new, on => :bid_successful, :with => :successful)

We can also subscribe a block, this is used to great effect in controllers to allow them to subscribe to events and respond in the correct way. You could reasonably add the controller as a listener, e.g. create_bid.subscribe(self), but adding a block makes it look more Rails-ish.

create_bid.on(:reserve_item_successfull) do |reservation|
  redirect_to reservation_path(reservation)
end

Global Listeners

Wisper also allows adding of “global listeners” which are automatically subscribed to every publisher.

In a Rails application I would typically add global listeners in an initalizer.

Wisper::GlobalListeners.add_listener(StatisticsListener.new)

This can be convenient but this creates indirection. The execution path can be less obvious. I would suggest using global listeners with caution. You will also, like regular Rails observers, need to turn them off in your tests.

In Summary

By combining Service objects with Wisper we get the following benefits:

• Smaller objects with fewer responsibilities
• Objects can be wired up based on context in a lightweight manner
• The core application is not polluted with the delivery mechanism concerns
• Objects tell listeners what happened, not what to do (trusting, not controlling)
• Isolated testing is easier and because we can do away with requiring Rails tests run faster.
• Clear boundaries exist between objects in different layers.
• Moving to async is easier since we can execute from a background job without Rails loaded

Easier to test, means refactoring is easier, which means longevity increases, resulting in higher value for money and greater profit.

As with any such consideration, one size does not fit all. Personally I don’t use Wisper with simple CRUD apps. But if (when!) the CRUD app develops beyond the Rails sweet spot then I start using Wispered service objects.

It’s very easy to apply Wisper to an existing codebase. You can start with one controller or model.

Refactoring mature codebases

Luckily this concept is very easy to apply to a mature code base in stages, partcularly if you have good test coverage. Simply choose somewhere to start, like a bloated model or a crazy if laced controller.

As I developed Wisper, I tried many techniques for implementing Pub-Sub in my code and it still contains some of those experiments. I only replace them with Wisper when I need to touch the controller in question for some other reason.

Once a model or service object has Wisper in place, adding new listeners is very easy. For example, caching can be moved out of the model/observer/controller and in to a listener.

You can even apply Wisper to an ActiveRecord model as easily as any other object if you wish to remove some after_ callbacks.

Have a go, find one place in your Rails app that needs some TLC and create a branch. See if it works out for the better.

I’d love to hear how you get on, your thoughts and feedback on Github, in the comments or on Twitter.