Super Easy Activity Feeds with Stream

Previously I have written about how to create Activity Feeds in Rails using the public_activity gem. Today, I am going to introduce you to Stream, a platform providing an API to build complex scalable feeds with ease. Together, we will create a demo application and integrate it with Stream.

Stream is free up to 3 million updates per month. It has multi-region support and real-time updates. There are clients for Ruby, Python, Javascript, PHP, Java, C#, Scala, and Go, as well as integrations for the Rails, Django, and Laravel frameworks.

Currently this platform is actively evolving ($1.75 million was raised recently) and you certainly should give it a try.

A working demo is available at sitepoint-stream.herokuapp.com.

The source code for this post can be found on GitHub.

Special thanks to Tommaso Barbugli for providing valuable information for this article.

Stream: Past and Future

The Stream platform was created by Thierry Schellenbach and Tommaso Barbugli, maintainers of the open-source Stream-Framework, a Python library to build scalable news feeds and activity streams. They noted that, even using this library, many developers find it difficult to prepare the required infrastructure (Redis or Cassandra, RabbitMQ, and Python Celery are required). Also, many people asked for an HTTP API layer so that they can use it with other programming languages. This is how the idea of creating Stream was born. After six month of testing, the first beta was released.

Currently, the team consists of Thierry, Tommaso, and three engineers (Stream is hiring, if you are interested). Recently, they received a nice bit of investment, so the future of this company looks really bright. In the near future, the team plans to make it easy to add personalized feeds to applications. The approach is to combine a user-centric analytics platform with machine learning.

If you want to get the basic idea behind Stream, spend a bit of time and playing with this interactive tutorial. Also, there is a Rails demo app already built that you can use as an example.

Preparing the Demo App

As I already said, Stream provides integrations for three frameworks. Of course, we are going to pick our favorite, Rails, and use the stream-rails gem. In this article I will use Rails 4, but stream-rails works with ActiveRecord 3 as well.

Before proceeding, sign up here – it is free (for up to 3 million updates per month), however paid plans are available, as well.

Go ahead and create a new Rails app without the default testing suite:

$ rails new FeedMe -T

Drop in the following gems:

Gemfile

[...]
gem 'stream_rails'
gem 'devise'
gem 'bootstrap-sass'
[...]

and run

$ bundle install

I am going to use Devise to quickly build authentication, but you may use another solution, if you’re so inclined. (I recently covered some of them, so you have plenty of options from which to choose).

Let’s style the app a bit with our old friend, Bootstrap:

application.scss

@import 'bootstrap-sprockets';
@import 'bootstrap';

layouts/application.html.erb

[...]
<nav class="navbar navbar-inverse">
  <div class="container">
    <div class="navbar-header">
      <%= link_to 'FeedMe', root_path, class: 'navbar-brand' %>
    </div>
    <div id="navbar">
      <ul class="nav navbar-nav">

      </ul>
    </div>
  </div>
</nav>

<div class="container">
  <% flash.each do |key, value| %>
    <div class="alert alert-<%= key %>">
      <%= value %>
    </div>
  <% end %>

  <%= yield %>
</div>
[...]

Authentication

Run the Devise generators to install all required files and create a User model:

$ rails g devise:install
$ rails g devise User

Let’s give each user a name:

$ rails g migration add_name_to_users name:string

Now run the migrations:

$ rake db:migrate

If you are using protected attributes, the name attribute should be permitted upon sign up:

application_controller.rb

[...]
before_action :configure_permitted_parameters, if: :devise_controller?

protected

def configure_permitted_parameters
  devise_parameter_sanitizer.for(:sign_up) << :name
end
[...]

Copy the Devise views into the views folder for customization:

$ rails g devise:views

Tweak the sign up view to include the name field:

views/devise/registration/new.html.erb

[...]
<div class="field">
  <%= f.label :name %><br />
  <%= f.text_field :name %>
</div>
[...]

While we are working with the views, let’s also update the layout to allow users to sign out:

layouts/application.html.erb

[...]
<ul class="nav navbar-nav">
  <% if user_signed_in? %>
    <li>
      <a href="#" class="dropdown-toggle" data-toggle="dropdown">
        <i class="glyphicon"></i> <%= current_user.name %><b class="caret"></b>
      </a>
      <ul class="dropdown-menu">
        <li><%= link_to 'Log out', destroy_user_session_path, method: :delete %></li>
      </ul>
    </li>
  <% end %>
</ul>
[...]

Root Page

Now create the static pages controller, along with the view for the root page and the corresponding route:

pages_controller.rb

class PagesController < ApplicationController
  before_action :authenticate_user!

  def index
  end
end

authenticate_user! is the method provided by Devise that redirects users to the sign in page if they are not authenticated.

config/routes.rb

[...]
root to: 'pages#index'
[...]

views/pages/index.html.erb

<div class="page-header"><h1>Welcome!</h1></div>

Great, now it is time to add some “items” to our page that our users will be able to pin, just like in Pinterest.

Adding Items

It does not really matter how our items look, so, for the demo, they’ll have a title and a message. Each user will then be able to pin and unpin them – information about these actions will be displayed in the activity feed.

First of all, create the Item model:

$ rails g model Item user:references title:string message:text
$ rake db:migrate

Set up a one-to-many association on the user side:

models/user.rb

[...]
has_many :items
[...]

Now the controller:

items_controller.rb

class ItemsController < ApplicationController
  before_action :authenticate_user!

  def new
    @item = Item.new
  end

  def create
    @item = Item.new(item_params)
    @item.save
    flash[:success] = "Item created!"
    redirect_to root_path
  end

  private

  def item_params
    params.require(:item).permit(:message, :title)
  end
end

The routes:

routes.rb

[...]
resources :items, only: [:new, :create]
[...]

And the view:

views/items/new.html.erb

<div class="page-header"><h1>New item</h1></div>

<%= form_for @item do |f| %>
  <div class="form-group">
    <%= f.label :title %>
    <%= f.text_field :title, class: 'form-control' %>
  </div>

  <div class="form-group">
    <%= f.label :message %>
    <%= f.text_area :message, class: 'form-control' %>
  </div>

  <%= f.submit 'Create', class: 'btn btn-primary' %>
<% end %>

Let’s also display the list of items on the root page

pages_controller.rb

[...]
def index
  @items = Item.order('created_at DESC')
end
[...]

views/pages/index.html.erb

<div class="page-header"><h1>Welcome!</h1></div>

<%= render @items %>

views/items/_item.html.erb

<div class="well well-sm">
  <small class="text-muted"><%= time_ago_in_words item.created_at %> ago</small>
  <h3><%= item.title %></h3>
  <p><%= item.message %></p>
</div>

Brilliant! You may either create some items manually or employ seeds.rb to automate this task (which is the preferred way, of course).

Integrating Stream

Integrating Stream is really easy. Create a new initializer file:

config/initializers/stream_rails.rb

require 'stream_rails'

StreamRails.configure do |config|
  config.api_key     = ENV["STREAM_KEY"]
  config.api_secret  = ENV["STREAM_SECRET"]
  config.timeout     = 30
end

To get the key pair, you should log in at getstream.io and open up your Dashboard. Here an app will be created for you (make sure that the Status is set to “On”) and this is the place where you can grab the key and secret.

I am storing the key pair in environmental variables, but you may use another method, just make sure that those keys are not publicly accessible.

Pinning and Unpinning Items

Stream stores ActiveRecord models in the feed as activities. Activities are the objects that basically tell who performed which action on which object. In the simplest case, activity consists of an actor, a verb, and an object. Stream requires models to respond to the following methods:

• activity_object – returns the object of the activity (for example, ActiveRecord model).
• activity_actor – returns the actor performing the activity (defaults to self.user).
• activity_verb – returns the string representation of the verb (defaults to model’s class name).

First of all, we want to allow users to pin and unpin the items. For this we’ll need a separate Pin model equipped with Stream’s methods:

$ rails g model Pin user:references item:references
$ rake db:migrate

Tweak the model:

models/pin.rb

[...]
belongs_to :user
belongs_to :item

validates :item, presence: true, uniqueness: {scope: :user}
validates :user, presence: true, uniqueness: {scope: :item}

include StreamRails::Activity
as_activity

def activity_object
  self.item
end
[...]

include StreamRails::Activity and as_activity equip Pin with Stream’s functionality. Note that I do not use activity_actor because the default value (self.user) suits us perfectly. We are also leaving activity_verb to the default value of Pin.

Now, let’s add the pin and unpin buttons to the item partial:

views/items/_item.html.erb

<div class="well well-sm">
  <small class="text-muted"><%= time_ago_in_words item.created_at %> ago</small>
  <h3><%= item.title %></h3>
  <p><%= item.message %></p>

  <%= render "pins/form", item: item %>
</div>

Here is the pin form partial:

views/pins/_form.html.erb

<% if item.user_pin(current_user) %>
  <%= button_to "Unpin", pin_path(item.user_pin(current_user)), method: :delete, class: "btn btn-primary btn-sm btn-danger" %>
<% else %>
  <%= form_for :pin, url: pins_path do |f| %>
    <input class="btn btn-primary btn-sm" type="submit" value="Pin">
    <%= f.hidden_field :item_id, value: item.id %>
  <% end %>
<% end %>

As you can see, pinning simply means creating a new record in the pins table while providing the item and user’s id. Unpinning means, of course, deletion of that record. Here is the corresponding controller:

pins_controller.rb

class PinsController < ApplicationController
  before_action :authenticate_user!

  def create
    @pin = Pin.new(pin_params)
    @pin.user = current_user
    @pin.save!
    flash[:success] = "Pinned!"
    redirect_to root_path
  end

  def destroy
    @pin = Pin.find(params[:id])
    @pin.destroy
    flash[:success] = "Unpinned!"
    redirect_to root_path
  end

  private

  def pin_params
    params.require(:pin).permit(:item_id)
  end
end

As long as we’ve tweaked the model, those actions will be monitored by Stream.

Lastly, the routes:

routes.rb

[...]
resources :pins, only: [:create, :destroy]
[...]

Our next step is allowing users to follow each other. Only actions made by followed users will be displayed in the personalized feed.

Following and Unfollowing Users

Following, once again, means creating a simple record that tells whom a user has followed:

$ rails g model Follow target_id:integer user_id:integer
$ rake db:migrate

Modify the migration like this:

db/migrations/xxx_create_follows.rb

class CreateFollows < ActiveRecord::Migration
  def change
    create_table :follows do |t|
      t.integer :target_id, index: true
      t.integer :user_id, index: true

      t.timestamps null: false
    end

    add_index :follows, [:target_id, :user_id], unique: true
  end
end

Tweak the Follow model to add Stream’s functionality and establish associations:

models/follow.rb

[...]
belongs_to :user
belongs_to :target, class_name: "User"

validates :target_id, presence: true
validates :user_id, presence: true

include StreamRails::Activity
as_activity

def activity_notify
  [StreamRails.feed_manager.get_notification_feed(self.target_id)]
end

def activity_object
  self.target
end
[...]

This method

def activity_notify
  [StreamRails.feed_manager.get_notification_feed(self.target_id)]
end

is used to build the notification feed. This type of feed is useful to notify certain users about an action. In our case, we are notifying a user that someone has followed them.

Don’t forget to set up association on the other side:

models/user.rb

[...]
has_many :follows

def followed_by(user = nil)
  user.follows.find_by(target_id: id)
end
[...]

followed_by is a method to check whether a user follows someone. We are going to be using it shortly.

We need to display a list of users and provide a follow/unfollow button. Here is the UserController:

users_controller.rb

class UsersController < ApplicationController
  def index
    @users = User.all
  end
end

The routes:

routes.rb

[...]
resources :users, only: [:index]
[...]

And the views:

views/users/index.html.erb

<div class="page-header"><h1>Users</h1></div>

<%= render @users %>

views/users/_user.html.erb

<h3><%= user.name %></h3>

<% if user.followed_by(current_user) %>
  <%= button_to "Unfollow", follow_path(user.followed_by(current_user)), method: :delete, :class => "btn btn-primary btn-sm btn-danger" %>
<% else %>
  <%= form_for :follow, url: follows_path do |f| %>
    <%= f.hidden_field :target_id, value: user.id %>
    <input class="btn btn-primary btn-sm btn-default" type="submit" value="Follow">
  <% end %>
<% end %>

This is very similar to what we had with pin/unpin functionality.

Let’s update the top menu:

layouts/application.html.erb

[...]
<li><%= link_to 'Users', users_path %></li>
[...]

Of course, we’ll require a controller to manage follows, as well:

follows_controller.rb

class FollowsController < ApplicationController
  before_action :authenticate_user!

  def create
    follow = Follow.new(follow_params)
    follow.user = current_user
    if follow.save
      StreamRails.feed_manager.follow_user(follow.user_id, follow.target_id)
    end
    flash[:success] = 'Followed!'
    redirect_to users_path
  end

  def destroy
    follow = Follow.find(params[:id])
    if follow.user_id == current_user.id
      follow.destroy!
      StreamRails.feed_manager.unfollow_user(follow.user_id, follow.target_id)
    end
    flash[:success] = 'Unfollowed!'
    redirect_to users_path
  end

  private

  def follow_params
    params.require(:follow).permit(:target_id)
  end
end

StreamRails.feed_manager.follow_user(follow.user_id, follow.target_id) follows a user; actor’s and target’s ids have to be provided. unfollow_user works in exactly the same way.

Don’t forget to set up the routes:

routes.rb

[...]
resources :follows, only: [:create, :destroy]
[...]

Great, now make sure that everything is working properly. The last step is displaying the actual feed.

Rendering Feeds

We’re going to render three feeds:

• User’s personal feed. This feed, as the name implies, displays all actions for a certain user.
• Flat news feeds show what has happened recently. Flat feeds render activities without any grouping and this is the default type of feed in Stream.
• Aggregated news feeds allow the user to specify an aggregation format. We are going to display pins and follows separately using this feed.
• User’s notification feed are similar to aggregated feeds, however notifications can be marked as read and you can get a count of the number of unseen and unread notifications.

For these feeds, a FeedsController will be needed. I am going to start with user’s personal feed:

feeds_controller.rb

class FeedsController < ApplicationController
  before_action :authenticate_user!
  before_action :create_enricher

  def user
    @user = User.find(params[:id])
    feed = StreamRails.feed_manager.get_user_feed(@user.id)
    results = feed.get['results']
    @activities = @enricher.enrich_activities(results)
  end

  private

  def create_enricher
    @enricher = StreamRails::Enrich.new
  end
end

Using this line feed = StreamRails.feed_manager.get_user_feed(@user.id we are accessing user’s feed.

What is that create_enricher method? Raw data read from a feed looks like this:

{"actor": "User:1", "verb": "like", "object": "Item:42"}

This format is not ready to use in templates. Therefore, the enrichment mechanism prepares the data loaded from a feed to be used in templates. Inside the create_enricher we instantiate the StreamRails::Enrich class and then simply use enrich_activities to prepare our data.

Here is the route:

routes.rb

[...]
scope path: '/feeds', controller: :feeds, as: 'feed' do
  get 'user/:id', to: :user, as: :user
end
[...]

And the view:

views/feeds/user.html.erb

<div class="page-header"><h1>My feed</h1></div>

<% for activity in @activities %>
  <%= render_activity activity %>
<% end %>

render_activity is another special method to be used in the templates. This method expects to receive enriched data and is going to look for partials in either the activity (for flat feeds) or the aggregated_activity (for aggregated feeds) folders. Partials should be named after the action’s verb (pin, follow, etc.)

views/activity/_follow.html.erb

<div class="well well-sm">
  <p><small class="text-muted"><%= time_ago_in_words activity['time'] %> ago</small></p>

  <p><strong><%= activity['object'].name %></strong> and <strong><%= activity['actor'].name %></strong> are now friends</p>
</div>

render_activity automatically sends activity to the local scope of the partial. Here we are simply accessing object’s and target’s names. We can call activity['object'].name because activity['object'] returns an instance of the User class.

views/activity/_pin.html.erb

<div class="well well-sm">
  <p><small class="text-muted"><%= time_ago_in_words activity['time'] %> ago</small></p>
  <p>
    <strong><%= activity['actor'].name %></strong> pinned
    <strong><%= activity['object'].title %></strong>
  </p>
</div>

Here the process is the same: we are displaying who pinned which item.

Now provide a link in the top menu to access this newly created feed:

layouts/application.html.erb

[...]
<ul class="dropdown-menu">
  <li><%= link_to 'My feed', feed_user_path(current_user) %></li>
  [...]
</ul>
[...]

Also, modify user partial:

views/users/_user.html.erb

<h3><%= link_to user.name, feed_user_path(user) %></h3>
[...]

Now, let’s add the flat feed:

feeds_controller.rb

[...]
def flat
  feed = StreamRails.feed_manager.get_news_feeds(current_user.id)[:flat]
  results = feed.get['results']
  @activities = @enricher.enrich_activities(results)
end
[...]

feed = StreamRails.feed_manager.get_news_feeds(current_user.id) accesses the news feed and [:flat] indicates the flat feed should be fetched.

views/feeds/flat.html.erb

<div class="page-header"><h1>Flat feed</h1></div>

<% for activity in @activities %>
  <%= render_activity activity %>
<% end %>

Once again, we are passing enriched data to the render_activity. As long as we’ve already created the activity folder and the partials inside, we can proceed to the aggregated feed.

feeds_controller.rb

[...]
def aggregated
  feed = StreamRails.feed_manager.get_news_feeds(current_user.id)[:aggregated]
  results = feed.get['results']
  @activities = @enricher.enrich_aggregated_activities(results)
end
[...]

This time it’s [:aggregated] instead of [:flat].

views/feeds/aggregated.html.erb

<div class="page-header"><h1>Aggregated feed</h1></div>

<% for activity in @activities %>
  <%= render_activity activity %>
<% end %>

Here, require separate partials inside the aggregated_activity folder:

views/aggregated_activity/_pin.html.erb

<% if activity['actor_count'] == 1 %>
  <%= activity['activities'][0]['actor'].name %> pinned <%= pluralize(activity['activity_count'], 'item') %>
<% elsif activity['actor_count'] == 2 %>
  <%= activity['activities'][0]['actor'].name %> and <%= activity['activities'][1]['actor'].name %> pinned <%= activity['activity_count'] %> items
<% else %>
  <%= activity['activities'][0]['actor'].name %>, <%= activity['activities'][1]['actor'].name %> and <%= activity['actor_count'].name - 2 %> more pinned <%= activity['activity_count'] %> items
<% end %>

<div class="pull-right">
  <i class="glyphicon glyphicon-time"></i> <%= time_ago_in_words(activity['updated_at']) %> ago
</div>

<% for activity in activity['activities'] %>
  <%= render_activity activity %>
<% end %>

for activity in activity['activities'] takes each activity one by one and render_activity activity uses the same partials inside the activity folder that we’ve recently created. Note that you may pass additional arguments to that method in order to choose other partials, for example:

<%= render_activity activity, :prefix => "aggregated_" %>

This is going to look for partials with the aggregated_ prefix.

views/aggregated_activity/_follow.html.erb

<i class="glyphicon glyphicon-time"></i> <%= time_ago_in_words(activity['updated_at']) %> ago

<% for activity in activity['activities'] %>
  <%= render_activity activity %>
<% end %>

Lastly add the notification feed:

feeds_controller.rb

[...]
def notification
  feed = StreamRails.feed_manager.get_notification_feed(current_user.id)
  results = feed.get['results']
  @activities = @enricher.enrich_aggregated_activities(results)
end
[...]

views/feeds/notification.html.erb

<div class="page-header"><h1>Your notification feed</h1></div>

<% for activity in @activities %>
  <%= render_activity activity %>
<% end %>

Set up the routes:

routes.rb

[...]
scope path: '/feeds', controller: :feeds, as: 'feed' do
  get 'me', to: :user
  get 'flat', to: :flat
  get 'aggregated', to: :aggregated
  get 'notification', to: :notification
end
[...]

Also, update the top menu:

layouts/application.html.erb

[...]
<li><%= link_to 'Flat feed', feed_flat_path %></li>
<li><%= link_to 'Aggregated feed', feed_aggregated_path %></li>
<li><%= link_to 'Notification feed', feed_notification_path %></li>
[...]

Now boot up your server and check how this is all working! Just don’t forget that you have to follow a user to view his actions (you can follow yourself, as well).

Conclusion

In this article, we’ve discussed Stream, a platform to easily build scalable activity feeds. Feel free to browse its documentation and experiment with it further.

Have you ever tried using Stream? Would you consider using it in future? Share your opinion in the comments. Thanks for staying with me and see you soon!

Frequently Asked Questions about Super Easy Activity Feeds with Stream