Full-Text Search in Rails with ElasticSearch

In this article you will learn how to integrate ElasticSearch into a Rails application.

A full-text search engine examines all of the words in every stored document as it tries to match search criteria (text specified by a user) wikipedia. For example, if you want to find articles that talk about Rails, you might search using the term “rails”. If you don’t have a special indexing technique, it means fully scanning all records to find matches, which will be extremely inefficient. One way to solve this is an “inverted index” that maps the words in the content of all records to its location in the database.

For example, if a primary key index is like this:

article#1 -> "breakthrough drug for schizophrenia"
article#2 -> "new schizophrenia drug"
article#3 -> "new approach for treatment of schizophrenia"
article#4 -> "new hopes for schizophrenia patients"
...

An inverted index for these records will be like this:

breakthrough    -> article#1
drug            -> article#1, article#2
schizophrenia   -> article#1, article#2, article#3, article#4
approach        -> article#3
new             -> article#2, article#3, article#4
hopes           -> article#4
...

Now, searching for the word “drug” uses the inverted index and return article#1 and article#2 directly.

I recommend the IR Book, if you want to learn more about this.

Build an Articles App

We will start with the famous blog example used by the Rails guides.

Create the Rails App

Type the following at the command prompt:

$ rails new blog
$ cd blog
$ bundle install
$ rails s

Create the Articles Controller

Create the articles controller using the Rails generator, add routes to config/routes.rb, and add methods for showing, creating, and listing articles.

$ rails g controller articles

Then open config/routes.rb and add this resource:

Blog::Application.routes.draw do
  resources :articles
end

Now, open app/controllers/articles_controller.rb and add methods to create, view, and list articles.

def index
  @articles = Article.all
end

def show
  @article = Article.find params[:id]
end

def new
end

def create
  @article = Article.new article_params
  if @article.save
    redirect_to @article
  else
    render 'new'
  end
end

private
  def article_params
    params.require(:article).permit :title, :text
  end

Article Model

We’ll need a model for the articles, so generate it like so:

$ rails g model Article title:string text:text
$ rake db:migrate

Views

New Article Form

Create a new file at app/views/articles/new.html.erb with the following content:

<h1>New Article</h1>  

<%= form_for :article, url: articles_path do |f| %>

  <% if not @article.nil? and @article.errors.any? %>
  <div id="error_explanation">
    <h2><%= pluralize(@article.errors.count, "error") %> prohibited
      this article from being saved:</h2>
    <ul>
    <% @article.errors.full_messages.each do |msg| %>
      <li><%= msg %></li>
    <% end %>
    </ul>
  </div>
  <% end %>

  <p>
    <%= f.label :title %><br>
    <%= f.text_field :title %>
  </p>

  <p>
    <%= f.label :text %><br>
    <%= f.text_area :text %>
  </p>

  <p>
    <%= f.submit %>
  </p>
<% end %>

<%= link_to '<- Back', articles_path %>

Show One Article

Create another file at app/views/articles/show.html.erb:

<p>
  <strong>Title:</strong>
  <%= @article.title %>
</p>

<p>
  <strong>Text:</strong>
  <%= @article.text %>
</p>

<%= link_to '<- Back', articles_path %>

List All Articles

Create a third file at app/views/articles/index.html.erb:

<h1>Articles</h1>

<ul>
  <% @articles.each do |article| %>
    <li>
      <h3>
        <%= article.title %>
      </h3>
      <p>
        <%= article.text %>
      </p>
    </li>
  <% end -%>
</ul>
<%= link_to 'New Article', new_article_path %>

You can now add and view articles. Make sure you start the Rails server and go to http://localhost:3000/articles. Click on “New Article” and add a few articles. These will be used to test our full-text search capabilities.

Integrate ElasticSearch

Currently, we can find an article by id only. Integrating ElasticSearch will allow finding articles by any word in its title or text.

Install for Ubuntu and Mac

Ubuntu

Go to elasticsearch.org/download and download the DEB file. Once the file is local, type:

$ sudo dpkg -i elasticsearch-[version].deb

Mac

If you’re on a Mac, Homebrew makes it easy:

$ brew install elasticsearch

Validate Installation

Open this url: http://localhost:9200 and you’ll see ElasticSearch respond like so:

{
  "status" : 200,
  "name" : "Anvil",
  "version" : {
    "number" : "1.2.1",
    "build_hash" : "6c95b759f9e7ef0f8e17f77d850da43ce8a4b364",
    "build_timestamp" : "2014-06-03T15:02:52Z",
    "build_snapshot" : false,
    "lucene_version" : "4.8"
  },
  "tagline" : "You Know, for Search"
}

Add Basic Search

Create a controller called search, along with a view so you can do something like: /search?q=ruby.

Gemfile

gem 'elasticsearch-model'
gem 'elasticsearch-rails'

Remember to run bundle install to install these gems.

Search Controller

Create The SearchController:

$ rails g controller search

Add this method to app/controller/search_controller.rb:

def search
  if params[:q].nil?
    @articles = []
  else
    @articles = Article.search params[:q]
  end
end

Integrate Search into Article

To add the ElasticSearch integration to the Article model, require elasticsearch/model and include the main module in Article class.

Modify app/models/article.rb:

require 'elasticsearch/model'

class Article < ActiveRecord::Base
  include Elasticsearch::Model
  include Elasticsearch::Model::Callbacks
end
Article.import # for auto sync model with elastic search

Search View

Create a new file at app/views/search/search.html.erb:

<h1>Articles Search</h1>

<%= form_for search_path, method: :get do |f| %>
  <p>
    <%= f.label "Search for" %>
    <%= text_field_tag :q, params[:q] %>
    <%= submit_tag "Go", name: nil %>
  </p>
<% end %>

<ul>
  <% @articles.each do |article| %>
    <li>
      <h3>
        <%= link_to article.title, controller: "articles", action: "show", id: article._id%>
      </h3>
    </li>
  <% end %>
</ul>

Search Route

Add the search route to _config/routes.rb-:

get 'search', to: 'search#search'

You can now go to http://localhost:3000/search and search for any word in the articles you created.

Enhance the Search

You may notice that there are some limitations in your search engine. For example, searching for part of a word, such as “rub” or “roby” instead of “ruby”, will give you zero results. Also, it’d be nice if the search engine gave results that include words similar to your search term.

ElasticSearch provides a lot of features to enhance your search. I will give some examples.

Custom Query

There are different types of queries that we can use. So far, we are just using the default ElasticSearch query. To enhance search results, we need to modify this default query. We can, for example, give higher priority for fields like title over other fields.

ElasticSearch provides a full Query DSL based on JSON to define queries. In general, there are basic queries, such as term or prefix. There are also compound queries, like the bool query. Queries can also have filters associated with them, such as the filtered or constant_score queries.

Let’s add a custom search method to our article model in app/models/article.rb:

def self.search(query)
  __elasticsearch__.search(
    {
      query: {
        multi_match: {
          query: query,
          fields: ['title^10', 'text']
        }
      }
    }
  )
end

Note: ^10 boosts by 10 the score of hits when the search term is matched in the title

Custom Mapping

Mapping is the process of defining how a document should be mapped to the Search Engine, including its searchable characteristics like which fields are searchable and if/how they are tokenized.

Explicit mapping is defined on an index/type level. By default, there isn’t a need to define an explicit mapping, since one is automatically created and registered when a new type or new field is introduced (with no performance overhead) and has sensible defaults. Only when the defaults need to be overridden must a mapping definition be provided.

We will improve the search so that you can search for a term like “search” and receive results also including “searches” and “searching” ..etc. This will use the built-in English analyzer in ElasticSearch to apply word stemming before indexing.

Add this mapping to the Article class: at app/models/article.rb

settings index: { number_of_shards: 1 } do
  mappings dynamic: 'false' do
    indexes :title, analyzer: 'english'
    indexes :text, analyzer: 'english'
  end
end

It’s a good idea to add the following lines to the end of the file to automatically drop and rebuiled the index when article.rb is loaded:

# Delete the previous articles index in Elasticsearch
Article.__elasticsearch__.client.indices.delete index: Article.index_name rescue nil

# Create the new index with the new mapping
Article.__elasticsearch__.client.indices.create \
  index: Article.index_name,
  body: { settings: Article.settings.to_hash, mappings: Article.mappings.to_hash }

# Index all article records from the DB to Elasticsearch
Article.import

Search Highlighting

Basically, we’d like to show the parts of the articles where the term we are searching for appears. It’s like when you search in google and you see a sample of the document that includes your term in bold. In ElasticSearch, this is called “highlights”. We will add a highlight parameter to our query and specify the fields we want to highlight. ElasticSearch will return the term between an tag, along with a few words before and after the term.

Assuming we are searching for the term “opensource”, ElasticSearch will return something like this:

Elasticsearch is a flexible and powerful <em>opensource</em>, distributed, real-time search and analytics

Note that “opensource” is surounded by an tag.

Add Highlights to the SearchController

First, add the “highlight” parameter to the ElasticSearch query:

def self.search(query)
  __elasticsearch__.search(
    {
      query: {
        multi_match: {
          query: query,
          fields: ['title^10', 'text']
        }
      },
      highlight: {
        pre_tags: ['<em>'],
        post_tags: ['</em>'],
        fields: {
          title: {},
          text: {}
        }
      }
    }
  )
end

Show Highlights in View

It’s pretty easy show this highlight in the view. Go to app/views/search/search.html.erb and replace the ul element with this:

<ul>
  <% @articles.each do |article| %>
    <li>
      <h3>
        <%= link_to article.try(:highlight).try(:title) ? article.highlight.title[0].html_safe : article.title,
          controller: "articles",
          action: "show",
          id: article._id%>
      </h3>
      <% if article.try(:highlight).try(:text) %>
        <% article.highlight.text.each do |snippet| %>
          <p><%= snippet.html_safe %>...</p>
        <% end %>
      <% end %>
    </li>
  <% end %>
</ul>

Now add a style for in app/assets/stylesheets/search.css.scss:

em {
  background: yellow;
}

One last thing we need is the highlighted term returned by ElasticSearch to be surrounded by a few words. If you need to show the title from the beginning, add index_options: 'offsets' to the title mapping:

settings index: { number_of_shards: 1 } do
  mappings dynamic: 'false' do
    indexes :title, analyzer: 'english', index_options: 'offsets'
    indexes :text, analyzer: 'english'
  end
end

This was a quick example for integerating ElasticSearch into a Rails app. We added basic search, then mixed things up a little using custom queries, mapping, and highlights. You can download the full source from here

References

• elasticsearch/elasticsearch-rails
• rails-application-templates
• Mapping Guide
• Query-DSL Guide
• rubyonrails.org/getting_started.html