Previously I have written about how to create Activity Feeds in Rails using the public_activity gem. Today, I am going to introduce you to Stream, a platform providing an API to build complex scalable feeds with ease. Together, we will create a demo application and integrate it with Stream.
Stream is free up to 3 million updates per month. It has multi-region support and real-time updates. There are clients for Ruby, Python, Javascript, PHP, Java, C#, Scala, and Go, as well as integrations for the Rails, Django, and Laravel frameworks.
Currently this platform is actively evolving ($1.75 million was raised recently) and you certainly should give it a try.
A working demo is available at sitepoint-stream.herokuapp.com.
The source code for this post can be found on GitHub.
Special thanks to Tommaso Barbugli for providing valuable information for this article.
Key Takeaways
- Stream simplifies the creation of complex, scalable activity feeds in Rails applications, providing a robust API and integrations for multiple programming languages and frameworks.
- Stream’s service is free for up to 3 million updates per month, with paid plans available for larger scale needs, making it accessible for both small projects and large enterprises.
- The platform supports real-time updates and multi-region deployment, ensuring high performance and availability across different geographical locations.
- Stream’s API allows for easy integration into Rails apps, with the `stream-rails` gem enabling quick setup and configuration to manage activity feeds effectively.
- The tutorial demonstrates how to build a demo Rails application that includes features like pinning/unpinning items and following/unfollowing users, showcasing Stream’s capabilities in a practical scenario.
Stream: Past and Future
The Stream platform was created by Thierry Schellenbach and Tommaso Barbugli, maintainers of the open-source Stream-Framework, a Python library to build scalable news feeds and activity streams. They noted that, even using this library, many developers find it difficult to prepare the required infrastructure (Redis or Cassandra, RabbitMQ, and Python Celery are required). Also, many people asked for an HTTP API layer so that they can use it with other programming languages. This is how the idea of creating Stream was born. After six month of testing, the first beta was released.
Currently, the team consists of Thierry, Tommaso, and three engineers (Stream is hiring, if you are interested). Recently, they received a nice bit of investment, so the future of this company looks really bright. In the near future, the team plans to make it easy to add personalized feeds to applications. The approach is to combine a user-centric analytics platform with machine learning.
If you want to get the basic idea behind Stream, spend a bit of time and playing with this interactive tutorial. Also, there is a Rails demo app already built that you can use as an example.
Preparing the Demo App
As I already said, Stream provides integrations for three frameworks. Of course, we are going to pick our favorite, Rails, and use the stream-rails gem. In this article I will use Rails 4, but stream-rails works with ActiveRecord 3 as well.
Before proceeding, sign up here – it is free (for up to 3 million updates per month), however paid plans are available, as well.
Go ahead and create a new Rails app without the default testing suite:
$ rails new FeedMe -T
Drop in the following gems:
Gemfile
[...]
gem 'stream_rails'
gem 'devise'
gem 'bootstrap-sass'
[...]
and run
$ bundle install
I am going to use Devise to quickly build authentication, but you may use another solution, if you’re so inclined. (I recently covered some of them, so you have plenty of options from which to choose).
Let’s style the app a bit with our old friend, Bootstrap:
application.scss
@import 'bootstrap-sprockets';
@import 'bootstrap';
layouts/application.html.erb
[...]
<nav class="navbar navbar-inverse">
<div class="container">
<div class="navbar-header">
<%= link_to 'FeedMe', root_path, class: 'navbar-brand' %>
</div>
<div id="navbar">
<ul class="nav navbar-nav">
</ul>
</div>
</div>
</nav>
<div class="container">
<% flash.each do |key, value| %>
<div class="alert alert-<%= key %>">
<%= value %>
</div>
<% end %>
<%= yield %>
</div>
[...]
Authentication
Run the Devise generators to install all required files and create a User
model:
$ rails g devise:install
$ rails g devise User
Let’s give each user a name:
$ rails g migration add_name_to_users name:string
Now run the migrations:
$ rake db:migrate
If you are using protected attributes, the name
attribute should be permitted upon sign up:
application_controller.rb
[...]
before_action :configure_permitted_parameters, if: :devise_controller?
protected
def configure_permitted_parameters
devise_parameter_sanitizer.for(:sign_up) << :name
end
[...]
Copy the Devise views into the views folder for customization:
$ rails g devise:views
Tweak the sign up view to include the name
field:
views/devise/registration/new.html.erb
[...]
<div class="field">
<%= f.label :name %><br />
<%= f.text_field :name %>
</div>
[...]
While we are working with the views, let’s also update the layout to allow users to sign out:
layouts/application.html.erb
[...]
<ul class="nav navbar-nav">
<% if user_signed_in? %>
<li>
<a href="#" class="dropdown-toggle" data-toggle="dropdown">
<i class="glyphicon"></i> <%= current_user.name %><b class="caret"></b>
</a>
<ul class="dropdown-menu">
<li><%= link_to 'Log out', destroy_user_session_path, method: :delete %></li>
</ul>
</li>
<% end %>
</ul>
[...]
Root Page
Now create the static pages controller, along with the view for the root page and the corresponding route:
pages_controller.rb
class PagesController < ApplicationController
before_action :authenticate_user!
def index
end
end
authenticate_user!
is the method provided by Devise that redirects users to the sign in page if they are not authenticated.
config/routes.rb
[...]
root to: 'pages#index'
[...]
views/pages/index.html.erb
<div class="page-header"><h1>Welcome!</h1></div>
Great, now it is time to add some “items” to our page that our users will be able to pin, just like in Pinterest.
Adding Items
It does not really matter how our items look, so, for the demo, they’ll have a title and a message. Each user will then be able to pin and unpin them – information about these actions will be displayed in the activity feed.
First of all, create the Item
model:
$ rails g model Item user:references title:string message:text
$ rake db:migrate
Set up a one-to-many association on the user side:
models/user.rb
[...]
has_many :items
[...]
Now the controller:
items_controller.rb
class ItemsController < ApplicationController
before_action :authenticate_user!
def new
@item = Item.new
end
def create
@item = Item.new(item_params)
@item.save
flash[:success] = "Item created!"
redirect_to root_path
end
private
def item_params
params.require(:item).permit(:message, :title)
end
end
The routes:
routes.rb
[...]
resources :items, only: [:new, :create]
[...]
And the view:
views/items/new.html.erb
<div class="page-header"><h1>New item</h1></div>
<%= form_for @item do |f| %>
<div class="form-group">
<%= f.label :title %>
<%= f.text_field :title, class: 'form-control' %>
</div>
<div class="form-group">
<%= f.label :message %>
<%= f.text_area :message, class: 'form-control' %>
</div>
<%= f.submit 'Create', class: 'btn btn-primary' %>
<% end %>
Let’s also display the list of items on the root page
pages_controller.rb
[...]
def index
@items = Item.order('created_at DESC')
end
[...]
views/pages/index.html.erb
<div class="page-header"><h1>Welcome!</h1></div>
<%= render @items %>
views/items/_item.html.erb
<div class="well well-sm">
<small class="text-muted"><%= time_ago_in_words item.created_at %> ago</small>
<h3><%= item.title %></h3>
<p><%= item.message %></p>
</div>
Brilliant! You may either create some items manually or employ seeds.rb to automate this task (which is the preferred way, of course).
Integrating Stream
Integrating Stream is really easy. Create a new initializer file:
config/initializers/stream_rails.rb
require 'stream_rails'
StreamRails.configure do |config|
config.api_key = ENV["STREAM_KEY"]
config.api_secret = ENV["STREAM_SECRET"]
config.timeout = 30
end
To get the key pair, you should log in at getstream.io and open up your Dashboard. Here an app will be created for you (make sure that the Status is set to “On”) and this is the place where you can grab the key and secret.
I am storing the key pair in environmental variables, but you may use another method, just make sure that those keys are not publicly accessible.
Pinning and Unpinning Items
Stream stores ActiveRecord models in the feed as activities. Activities are the objects that basically tell who performed which action on which object. In the simplest case, activity consists of an actor, a verb, and an object. Stream requires models to respond to the following methods:
activity_object
– returns the object of the activity (for example, ActiveRecord model).activity_actor
– returns the actor performing the activity (defaults toself.user
).activity_verb
– returns the string representation of the verb (defaults to model’s class name).
First of all, we want to allow users to pin and unpin the items. For this we’ll need a separate Pin
model
equipped with Stream’s methods:
$ rails g model Pin user:references item:references
$ rake db:migrate
Tweak the model:
models/pin.rb
[...]
belongs_to :user
belongs_to :item
validates :item, presence: true, uniqueness: {scope: :user}
validates :user, presence: true, uniqueness: {scope: :item}
include StreamRails::Activity
as_activity
def activity_object
self.item
end
[...]
include StreamRails::Activity
and as_activity
equip Pin
with Stream’s functionality. Note that I do not use activity_actor
because the default value (self.user
) suits us perfectly. We are also leaving activity_verb
to the default value of Pin
.
Now, let’s add the pin and unpin buttons to the item partial:
views/items/_item.html.erb
<div class="well well-sm">
<small class="text-muted"><%= time_ago_in_words item.created_at %> ago</small>
<h3><%= item.title %></h3>
<p><%= item.message %></p>
<%= render "pins/form", item: item %>
</div>
Here is the pin form partial:
views/pins/_form.html.erb
<% if item.user_pin(current_user) %>
<%= button_to "Unpin", pin_path(item.user_pin(current_user)), method: :delete, class: "btn btn-primary btn-sm btn-danger" %>
<% else %>
<%= form_for :pin, url: pins_path do |f| %>
<input class="btn btn-primary btn-sm" type="submit" value="Pin">
<%= f.hidden_field :item_id, value: item.id %>
<% end %>
<% end %>
As you can see, pinning simply means creating a new record in the pins
table while providing the item and
user’s id. Unpinning means, of course, deletion of that record. Here is the corresponding controller:
pins_controller.rb
class PinsController < ApplicationController
before_action :authenticate_user!
def create
@pin = Pin.new(pin_params)
@pin.user = current_user
@pin.save!
flash[:success] = "Pinned!"
redirect_to root_path
end
def destroy
@pin = Pin.find(params[:id])
@pin.destroy
flash[:success] = "Unpinned!"
redirect_to root_path
end
private
def pin_params
params.require(:pin).permit(:item_id)
end
end
As long as we’ve tweaked the model, those actions will be monitored by Stream.
Lastly, the routes:
routes.rb
[...]
resources :pins, only: [:create, :destroy]
[...]
Our next step is allowing users to follow each other. Only actions made by followed users will be displayed in the personalized feed.
Following and Unfollowing Users
Following, once again, means creating a simple record that tells whom a user has followed:
$ rails g model Follow target_id:integer user_id:integer
$ rake db:migrate
Modify the migration like this:
db/migrations/xxx_create_follows.rb
class CreateFollows < ActiveRecord::Migration
def change
create_table :follows do |t|
t.integer :target_id, index: true
t.integer :user_id, index: true
t.timestamps null: false
end
add_index :follows, [:target_id, :user_id], unique: true
end
end
Tweak the Follow
model to add Stream’s functionality and establish associations:
models/follow.rb
[...]
belongs_to :user
belongs_to :target, class_name: "User"
validates :target_id, presence: true
validates :user_id, presence: true
include StreamRails::Activity
as_activity
def activity_notify
[StreamRails.feed_manager.get_notification_feed(self.target_id)]
end
def activity_object
self.target
end
[...]
This method
def activity_notify
[StreamRails.feed_manager.get_notification_feed(self.target_id)]
end
is used to build the notification feed. This type of feed is useful to notify certain users about an action. In our case, we are notifying a user that someone has followed them.
Don’t forget to set up association on the other side:
models/user.rb
[...]
has_many :follows
def followed_by(user = nil)
user.follows.find_by(target_id: id)
end
[...]
followed_by
is a method to check whether a user follows someone. We are going to be using it shortly.
We need to display a list of users and provide a follow/unfollow button. Here is the UserController
:
users_controller.rb
class UsersController < ApplicationController
def index
@users = User.all
end
end
The routes:
routes.rb
[...]
resources :users, only: [:index]
[...]
And the views:
views/users/index.html.erb
<div class="page-header"><h1>Users</h1></div>
<%= render @users %>
views/users/_user.html.erb
<h3><%= user.name %></h3>
<% if user.followed_by(current_user) %>
<%= button_to "Unfollow", follow_path(user.followed_by(current_user)), method: :delete, :class => "btn btn-primary btn-sm btn-danger" %>
<% else %>
<%= form_for :follow, url: follows_path do |f| %>
<%= f.hidden_field :target_id, value: user.id %>
<input class="btn btn-primary btn-sm btn-default" type="submit" value="Follow">
<% end %>
<% end %>
This is very similar to what we had with pin/unpin functionality.
Let’s update the top menu:
layouts/application.html.erb
[...]
<li><%= link_to 'Users', users_path %></li>
[...]
Of course, we’ll require a controller to manage follows, as well:
follows_controller.rb
class FollowsController < ApplicationController
before_action :authenticate_user!
def create
follow = Follow.new(follow_params)
follow.user = current_user
if follow.save
StreamRails.feed_manager.follow_user(follow.user_id, follow.target_id)
end
flash[:success] = 'Followed!'
redirect_to users_path
end
def destroy
follow = Follow.find(params[:id])
if follow.user_id == current_user.id
follow.destroy!
StreamRails.feed_manager.unfollow_user(follow.user_id, follow.target_id)
end
flash[:success] = 'Unfollowed!'
redirect_to users_path
end
private
def follow_params
params.require(:follow).permit(:target_id)
end
end
StreamRails.feed_manager.follow_user(follow.user_id, follow.target_id)
follows a user; actor’s and target’s ids have to be provided. unfollow_user
works in exactly the same way.
Don’t forget to set up the routes:
routes.rb
[...]
resources :follows, only: [:create, :destroy]
[...]
Great, now make sure that everything is working properly. The last step is displaying the actual feed.
Rendering Feeds
We’re going to render three feeds:
- User’s personal feed. This feed, as the name implies, displays all actions for a certain user.
- Flat news feeds show what has happened recently. Flat feeds render activities without any grouping and this is the default type of feed in Stream.
- Aggregated news feeds allow the user to specify an aggregation format. We are going to display pins and follows separately using this feed.
- User’s notification feed are similar to aggregated feeds, however notifications can be marked as read and you can get a count of the number of unseen and unread notifications.
For these feeds, a FeedsController
will be needed. I am going to start with user’s personal feed:
feeds_controller.rb
class FeedsController < ApplicationController
before_action :authenticate_user!
before_action :create_enricher
def user
@user = User.find(params[:id])
feed = StreamRails.feed_manager.get_user_feed(@user.id)
results = feed.get['results']
@activities = @enricher.enrich_activities(results)
end
private
def create_enricher
@enricher = StreamRails::Enrich.new
end
end
Using this line feed = StreamRails.feed_manager.get_user_feed(@user.id
we are accessing user’s feed.
What is that create_enricher
method? Raw data read from a feed looks like this:
{"actor": "User:1", "verb": "like", "object": "Item:42"}
This format is not ready to use in templates. Therefore, the enrichment mechanism prepares the data loaded from a feed to be used in templates. Inside the create_enricher
we instantiate the StreamRails::Enrich
class and then simply use enrich_activities
to prepare our data.
Here is the route:
routes.rb
[...]
scope path: '/feeds', controller: :feeds, as: 'feed' do
get 'user/:id', to: :user, as: :user
end
[...]
And the view:
views/feeds/user.html.erb
<div class="page-header"><h1>My feed</h1></div>
<% for activity in @activities %>
<%= render_activity activity %>
<% end %>
render_activity
is another special method to be used in the templates. This method expects to receive enriched data and is going to look for partials in either the activity (for flat feeds) or the aggregated_activity (for aggregated feeds) folders. Partials should be named after the action’s verb (pin, follow, etc.)
views/activity/_follow.html.erb
<div class="well well-sm">
<p><small class="text-muted"><%= time_ago_in_words activity['time'] %> ago</small></p>
<p><strong><%= activity['object'].name %></strong> and <strong><%= activity['actor'].name %></strong> are now friends</p>
</div>
render_activity
automatically sends activity
to the local scope of the partial. Here we are simply accessing object’s and target’s names. We can call activity['object'].name
because activity['object']
returns an instance of the User
class.
views/activity/_pin.html.erb
<div class="well well-sm">
<p><small class="text-muted"><%= time_ago_in_words activity['time'] %> ago</small></p>
<p>
<strong><%= activity['actor'].name %></strong> pinned
<strong><%= activity['object'].title %></strong>
</p>
</div>
Here the process is the same: we are displaying who pinned which item.
Now provide a link in the top menu to access this newly created feed:
layouts/application.html.erb
[...]
<ul class="dropdown-menu">
<li><%= link_to 'My feed', feed_user_path(current_user) %></li>
[...]
</ul>
[...]
Also, modify user partial:
views/users/_user.html.erb
<h3><%= link_to user.name, feed_user_path(user) %></h3>
[...]
Now, let’s add the flat feed:
feeds_controller.rb
[...]
def flat
feed = StreamRails.feed_manager.get_news_feeds(current_user.id)[:flat]
results = feed.get['results']
@activities = @enricher.enrich_activities(results)
end
[...]
feed = StreamRails.feed_manager.get_news_feeds(current_user.id)
accesses the news feed and [:flat]
indicates the flat feed should be fetched.
views/feeds/flat.html.erb
<div class="page-header"><h1>Flat feed</h1></div>
<% for activity in @activities %>
<%= render_activity activity %>
<% end %>
Once again, we are passing enriched data to the render_activity
. As long as we’ve already created the activity folder and the partials inside, we can proceed to the aggregated feed.
feeds_controller.rb
[...]
def aggregated
feed = StreamRails.feed_manager.get_news_feeds(current_user.id)[:aggregated]
results = feed.get['results']
@activities = @enricher.enrich_aggregated_activities(results)
end
[...]
This time it’s [:aggregated]
instead of [:flat]
.
views/feeds/aggregated.html.erb
<div class="page-header"><h1>Aggregated feed</h1></div>
<% for activity in @activities %>
<%= render_activity activity %>
<% end %>
Here, require separate partials inside the aggregated_activity folder:
views/aggregated_activity/_pin.html.erb
<% if activity['actor_count'] == 1 %>
<%= activity['activities'][0]['actor'].name %> pinned <%= pluralize(activity['activity_count'], 'item') %>
<% elsif activity['actor_count'] == 2 %>
<%= activity['activities'][0]['actor'].name %> and <%= activity['activities'][1]['actor'].name %> pinned <%= activity['activity_count'] %> items
<% else %>
<%= activity['activities'][0]['actor'].name %>, <%= activity['activities'][1]['actor'].name %> and <%= activity['actor_count'].name - 2 %> more pinned <%= activity['activity_count'] %> items
<% end %>
<div class="pull-right">
<i class="glyphicon glyphicon-time"></i> <%= time_ago_in_words(activity['updated_at']) %> ago
</div>
<% for activity in activity['activities'] %>
<%= render_activity activity %>
<% end %>
for activity in activity['activities']
takes each activity one by one and render_activity activity
uses the same partials inside the activity folder that we’ve recently created. Note that you may pass additional arguments to that method in order to choose other partials, for example:
<%= render_activity activity, :prefix => "aggregated_" %>
This is going to look for partials with the aggregated_
prefix.
views/aggregated_activity/_follow.html.erb
<i class="glyphicon glyphicon-time"></i> <%= time_ago_in_words(activity['updated_at']) %> ago
<% for activity in activity['activities'] %>
<%= render_activity activity %>
<% end %>
Lastly add the notification feed:
feeds_controller.rb
[...]
def notification
feed = StreamRails.feed_manager.get_notification_feed(current_user.id)
results = feed.get['results']
@activities = @enricher.enrich_aggregated_activities(results)
end
[...]
views/feeds/notification.html.erb
<div class="page-header"><h1>Your notification feed</h1></div>
<% for activity in @activities %>
<%= render_activity activity %>
<% end %>
Set up the routes:
routes.rb
[...]
scope path: '/feeds', controller: :feeds, as: 'feed' do
get 'me', to: :user
get 'flat', to: :flat
get 'aggregated', to: :aggregated
get 'notification', to: :notification
end
[...]
Also, update the top menu:
layouts/application.html.erb
[...]
<li><%= link_to 'Flat feed', feed_flat_path %></li>
<li><%= link_to 'Aggregated feed', feed_aggregated_path %></li>
<li><%= link_to 'Notification feed', feed_notification_path %></li>
[...]
Now boot up your server and check how this is all working! Just don’t forget that you have to follow a user to view his actions (you can follow yourself, as well).
Conclusion
In this article, we’ve discussed Stream, a platform to easily build scalable activity feeds. Feel free to browse its documentation and experiment with it further.
Have you ever tried using Stream? Would you consider using it in future? Share your opinion in the comments. Thanks for staying with me and see you soon!
Frequently Asked Questions about Super Easy Activity Feeds with Stream
How does Stream compare to other Rails Activity Feed tools?
Stream is a powerful tool for creating activity feeds in Rails. Unlike other tools, Stream provides a robust and scalable solution that can handle high volumes of data. It offers a simple and intuitive API that makes it easy to create, update, and manage activity feeds. Additionally, Stream provides real-time updates, ensuring that your users always have the most current information.
Can Stream handle large-scale applications?
Yes, Stream is designed to handle large-scale applications. It is built to be scalable and can handle high volumes of data without compromising performance. This makes it an ideal choice for applications that require real-time updates and have a large user base.
How easy is it to integrate Stream into my Rails application?
Stream is designed to be easy to integrate into your Rails application. It provides a simple and intuitive API that allows you to quickly and easily create, update, and manage activity feeds. Additionally, Stream provides comprehensive documentation and support to help you get started.
Does Stream support real-time updates?
Yes, Stream supports real-time updates. This means that your users will always have the most current information. This is particularly useful for applications that require up-to-the-minute updates, such as social media platforms or news sites.
What kind of support does Stream offer?
Stream offers comprehensive support to help you get started and to assist you with any issues you may encounter. This includes detailed documentation, a dedicated support team, and a community of users who can provide advice and assistance.
How does Stream handle data security?
Stream takes data security very seriously. It uses industry-standard security measures to protect your data, including encryption and secure access controls. Additionally, Stream is GDPR compliant, ensuring that your users’ data is handled in accordance with European data protection regulations.
Can I customize my activity feeds with Stream?
Yes, Stream allows you to fully customize your activity feeds. You can define your own feed groups, choose which activities to include, and even customize the appearance of your feeds. This gives you complete control over how your activity feeds look and function.
How does Stream handle data redundancy?
Stream uses a distributed architecture to ensure data redundancy. This means that your data is stored in multiple locations, ensuring that it is always available and protected against loss.
Can I use Stream with other programming languages?
Yes, while this article focuses on using Stream with Rails, Stream provides SDKs for several other programming languages, including Python, PHP, Node.js, and more. This makes it a versatile solution that can be used in a variety of development environments.
Is Stream a cost-effective solution for creating activity feeds?
Stream offers a range of pricing options to suit different needs and budgets. It provides a free tier for small projects, as well as scalable pricing for larger applications. This makes it a cost-effective solution for creating activity feeds, regardless of the size of your project.
Ilya Bodrov is personal IT teacher, a senior engineer working at Campaigner LLC, author and teaching assistant at Sitepoint and lecturer at Moscow Aviations Institute. His primary programming languages are Ruby (with Rails) and JavaScript. He enjoys coding, teaching people and learning new things. Ilya also has some Cisco and Microsoft certificates and was working as a tutor in an educational center for a couple of years. In his free time he tweets, writes posts for his website, participates in OpenSource projects, goes in for sports and plays music.