Adapting RethinkDB For The Evented Web With Pusher

Share this article

Pusher_rethinkDB_demo

RethinkDB recently released version 2.0, and here at Pusher we’re all very excited about how creating real-time apps can now be even easier. Changefeeds, a feature introduced by RethinkDB a few versions ago, allows your system to listen to changes in your database. This new version has significantly improved, opening up interesting possibilities for real-time applications.

While RethinkDB covers listening to events on your server, there is still the issue of publishing these changes to your client, who can build anything from news feeds to data visualizations.

This is where a hosted message broker such as Pusher comes into play. By exposing RethinkDB changefeed changes as Pusher events, you can quickly achieve scalable last-mile delivery that instantly pushes database updates to the client. Not only that, but Pusher’s evented publish-subscribe approach fits the logic of real-time applications:

  • Channels identify the data, whether that’s a table in a database or, in the case of RethinkDB, a changefeed.
  • Events represent what’s happening to the data: new data available, existing data being updated or data being deleted.

As an added bonus, real-time features can be rolled into production fast, in the knowledge that Pusher will scale to millions of concurrent devices and connections, removing the pain of managing your own real-time infrastructure. To show you how this can be done, this post will guide you through how to create the type of activity streams found on the RethinkDB website. By creating a small Sinatra app, we’ll quickly build the JSON feed and high-scores list you can see in our demo. Note that while we are using Ruby and Sinatra, one of the great things about RethinkDB’s adapters is how similar they are across all languages. In other words, what we do here can easily be applied to the stack of your choice.

You can play with the demo here. If you get stuck at any point, feel free to check out the source code.

Step 1: Setting Up

First, if you don’t already have one, sign up for a free Pusher account. Keep your application credentials close by.

If you do not have RethinkDB installed, you can install it on a Mac using Homebrew:

$ brew install rethinkdb

Installation for other operating systems can be found in the documentation.

To start your RethinkDB server, type into the terminal:

$ rethinkdb

Now browse to https://localhost:8080/#tables, which leads to a Web UI where you can create your database. Create a database called game with a table called players.

In your project directory, add the Pusher and RethinkDB gems to your Gemfile, along with Sinatra for our web application:

gem 'pusher'
gem 'rethinkdb'
gem 'sinatra'

Bundle install your gems, and create app.rb for the route-handlers, and rethinkdb.rb for the database configuration.

In rethinkdb.rb, let’s connect to the database. Setup the Pusher instance we’ll need for later using your app credentials. You can get these on your dashboard.

require 'rethinkdb'
include RethinkDB::Shortcuts

require 'pusher'

pusher = Pusher::Client.new({
  app_id: ,
  key: 
  secret: })

$conn = r.connect(
  host: "localhost",
  port: 28015, # the default RethinkDB port
  db: 'game',
)

In app.rb, setup the bare-bones of a Sinatra application:

require 'sinatra'
require './rethinkdb'

get '/' do
  erb :index
end

Step 2: Creating Players

As you can see in the demo, whenever a player enters their name, a game of Snake starts. In the meantime, we want to create a player instance from the name the user has provided.

This demonstration will not go heavily into the HTML and jQuery behind the app. The snake game is based on borrowed code (that you can read up on here) and will detract from the purpose of the tutorial: last mile delivery in a few lines of code. But if you want to dig into it, feel free to check out the source code.

In the case of creating users, we simply want to send an AJAX POST to /players with {name: "the user's name"} to the Sinatra server. At this endpoint, the app runs a simple insert query into the players table:

post '/players' do
  name = params[:name]
  r.table("players").insert(name: name).run($conn) # pass in the connection to `run`
  "Player created!"
end

It’s as simple as that! If you run the app and browse to https://localhost:8080/#dataexplorer, running r.table("game").table("players"), you should see your brand new player document.

While this successfully creates a new player, we’ll probably want our server to remember them for subsequent requests, like submitting scores. We can just amend this endpoint to store the player’s ID in a session. Conveniently, a RethinkDB query response returns the instance’s ID in a "generated_keys" field.

post '/players' do
  name = params[:name]
  response = r.table("players").insert(name: name).run($conn)
  session[:id] = response["generated_keys"][0]
  "Player created!"
end

Step 3: Submitting Players’ Scores

For the purpose of making the demo more fun and interactive, I’ve added two jQuery events to the snake code. One to trigger the start of the game once the player has been created and a second to listen for the end of the game and retrieve the user’s score.

When the game ends, the score is passed to the jQuery event listener. Using this score, make a simple POST to /players/score with the params {score: score}. The route handler gets the player by their session ID and updates their score and high-score, accordingly:

post '/players/score' do
  id = session[:id]
  score = params[:score]

  player = r.table("players").get(id).run($conn) # get the player

  score_update = {score: score} # our update parameters

  if !player["score"] || score > player["high_score"]
    # if the player doesn't have a score yet
    # or if the score is higher than their highest score
    score_update[:high_score] = score
    # add the high-score to the query
  end

  r.table("player").get(id).update(score_update).run($conn) # e.g. .update(score: 94, high_score: 94)
  {success:200}.to_json
end

Now that we have a high_score key for players in our database, we can start rendering a static view of the leaderboard, before we make it realtime. In rethinkdb.rb, let’s build our leaderboard query:

LEADERBOARD = r.table("players").order_by({index: r.desc("high_score")}).limit(5)

In order for this to work, make sure you have created an index called high_score through which to order players. You can do this in your RethinkDB data explorer by running r.db("game").table("players").indexCreate("high_score").

The app needs a /leaderboard endpoint so that renders leaders to the DOM:

get '/leaderboard' do
  leaders = LEADERBOARD.run($conn)
  leaders.to_a.to_json
end

Using jQuery or your preferred Javascript framework, show a static list of the players with the highest scores:

$.get('/leaderboard', function(leaders){
  showLeaderboard(leaders); // showLeaderboard can render leaders in the DOM.
})

Step 4: Make Your Database Real-time

As you can see from the demo, we have two real-time streams involved: a raw JSON feed of live scores, and a live leaderboard. In order to start listening to these changes, we’ll use the RethinkDB gem’s adapter for EventMachine. To the top of rethinkdb.rb, add require 'eventmachine'. Sinatra lists EventMachine as a dependency, so it should already be available within the context of your bundle.

Seeing as we’ve already built our LEADERBOARD query above, let’s dive into how we can listen for changes regarding that query. All that is necessary is to call the changes method on the query, and instead of calling run, call em_run. Once we have the change, all we need is one line of Pusher code to trigger the event to the client.

EventMachine.next_tick do # usually `run` would be sufficient - `next_tick` is to avoid clashes with Sinatra's EM loop
  LEADERBOARD.changes.em_run($conn) do |err, change|
    updated_player = change["new_val"]
   pusher.trigger(“scores”, "new_high_score", update_player)
  end
end

An awesome thing about RethinkDB’s changefeed is that it passes the delta whenever there is a change concerning a query, so you get the old value and the new value. An example of this query, showing the previous and updated instance of the player who has achieved a high score, would be as follows:

{
  "new_val": {
    "high_score": 6 ,
    "id":  "476e4332-68f1-4ae9-b71f-05071b9560a3" ,
    "name":  "thibaut courtois" ,
    "score": 6
  },
  "old_val": {
    "high_score": 2 ,
    "id":  "476e4332-68f1-4ae9-b71f-05071b9560a3" ,
    "name":  "thibaut courtois" ,
    "score": 1
  }
}

In this instance, we just take the new_val of the change – that is, the most recently achieved high-score – and trigger a Pusher event called new_high_score on a channel called scores with that update. You can test that it works by changing a player’s high score, either in your app or the RethinkDB data explorer, then heading to your debug console on https://app.pusher.com to view the newly-created event.

The raw JSON feed of scores shown in our demo is also simple to implement. Let’s just build the query and place it in the same EventMachine block:

LIVE_SCORES = r.table("players").has_fields("score")
# all the players for whom `score` isn't `nil`

EventMachine.next_tick do

  ...

  LIVE_SCORES.changes.em_run($conn) do |err, change|
    updated_player = change["new_val"]
    pusher.trigger(“scores”, "new_score", updated_player)
  end
end

And now, in the browser debug console, whenever somebody’s score has changed, you should see an event like this:

{
  "high_score": 1,
  "id": "97925e44-3e8f-49cd-a34c-90f023a3a8f7",
  "name": "nacer chadli",
  "score": 1
}

Step 5: From DB To DOM

Now that there are Pusher events firing whenever the value of the queries change, we can bind to these events on the client and mutate the DOM accordingly. We simply create a new Pusher instance with our app key, subscribe to the scores channel, and bind callbacks to the events on that channel:

var pusher = new Pusher("YOUR_APP_KEY");

var channel = pusher.subscribe("scores");

channel.bind("new_score", function(player){
  // append `player` to the live JSON feed of scores
});

channel.bind("new_high_score", function(player){
  // append `player` to the leaderboard
});

And there you have it: a simple and efficient way to update clients in real-time whenever a change happens in the database!

Going Forward

Hopefully, we’ve given you insight into RethinkDB’s nice querying language and awesome real-time capabilities. We’ve also shown how Pusher can easily be integrated to work as ‘last-mile delivery’ from changes in your database to your client. Aside from setting up the app itself, there is not much code involved in getting changefeeds up and running.

The demo I have shown you is a fairly small and straightforward. Large, production apps are quite possibly where the benefits of integrating Pusher with RethinkDB are greater felt. RethinkDB allows you to easily scale your database, and Pusher handles the scalability of real-time messaging for you. Combining the two allows developers to build powerful and reliable applications that scale.

Frequently Asked Questions (FAQs) about Adapting RethinkDB for the Evented Web with Pusher

What is the main advantage of using RethinkDB with Pusher?

The main advantage of using RethinkDB with Pusher is the ability to create real-time, scalable applications. RethinkDB is a NoSQL database that stores JSON documents and allows for real-time changes to be pushed to applications. Pusher, on the other hand, is a simple hosted API for quickly, securely, and reliably adding real-time bi-directional functionality via WebSockets to web and mobile applications, or any other Internet-connected device. When combined, these two technologies allow developers to build applications that can instantly react to changes in the database.

How does RethinkDB differ from other NoSQL databases?

RethinkDB stands out from other NoSQL databases due to its real-time push architecture. Unlike traditional databases that require you to poll for changes, RethinkDB pushes changes to your application in real-time as they happen. This makes it an excellent choice for real-time applications like collaborative web apps, multiplayer games, and real-time analytics.

How can I start using Pusher with RethinkDB?

To start using Pusher with RethinkDB, you need to install both RethinkDB and Pusher. Once installed, you can use the Pusher library to connect to your RethinkDB instance. You can then use the Pusher API to send real-time updates to your application whenever changes occur in your RethinkDB database.

What programming languages are supported by Pusher?

Pusher provides libraries for a variety of programming languages including JavaScript, Ruby, PHP, Python, .NET, Java, and more. This makes it a versatile choice for developers working in different programming environments.

Can I use Pusher with other databases besides RethinkDB?

Yes, Pusher can be used with any database that supports WebSockets. This includes most modern relational and NoSQL databases. However, the real-time push functionality of RethinkDB makes it a particularly good match for Pusher.

What are some common use cases for RethinkDB and Pusher?

RethinkDB and Pusher are commonly used to build real-time applications. This includes collaborative web apps where multiple users can edit the same document simultaneously, multiplayer games that require real-time updates, real-time analytics dashboards that update in real-time as data changes, and more.

How does Pusher ensure secure data transmission?

Pusher uses secure WebSockets (WSS) for data transmission. This ensures that all data sent between your application and Pusher is encrypted and secure. Additionally, Pusher provides a variety of security features such as private channels and signed messages to further enhance security.

How scalable is a solution using RethinkDB and Pusher?

Both RethinkDB and Pusher are designed to be highly scalable. RethinkDB can be easily sharded and replicated across multiple nodes, while Pusher can handle millions of concurrent connections. This makes them a suitable choice for large-scale, high-traffic applications.

What is the pricing for using Pusher?

Pusher offers a variety of pricing plans to suit different needs. They offer a free plan for small applications, as well as several paid plans for larger applications with more demanding requirements. You can find more details on the Pusher website.

Where can I find more resources to learn about RethinkDB and Pusher?

Both RethinkDB and Pusher have extensive documentation available on their respective websites. Additionally, there are numerous tutorials, blog posts, and community forums available online where you can learn more about these technologies.

Jamie PatelJamie Patel
View Author

Jamie Patel is a Growth Engineer at Pusher. He loves playing with a range of technologies and working some realtime magic on them. Originally a Rubyist, he loves working with Go, Rust and Python programming languages. He is also contemplating the possibility to write a book, 'Realtime for Teenagers', no-doubt a future best-seller.

GlennG
Share this article
Read Next
Get the freshest news and resources for developers, designers and digital creators in your inbox each week
Loading form