Full-Text Search in Rails with ElasticSearch
In this article you will learn how to integrate ElasticSearch into a Rails application.
A full-text search engine examines all of the words in every stored document as it tries to match search criteria (text specified by a user) wikipedia. For example, if you want to find articles that talk about Rails, you might search using the term “rails”. If you don’t have a special indexing technique, it means fully scanning all records to find matches, which will be extremely inefficient. One way to solve this is an “inverted index” that maps the words in the content of all records to its location in the database.
For example, if a primary key index is like this:
article#1 -> "breakthrough drug for schizophrenia"
article#2 -> "new schizophrenia drug"
article#3 -> "new approach for treatment of schizophrenia"
article#4 -> "new hopes for schizophrenia patients"
...
An inverted index for these records will be like this:
breakthrough -> article#1
drug -> article#1, article#2
schizophrenia -> article#1, article#2, article#3, article#4
approach -> article#3
new -> article#2, article#3, article#4
hopes -> article#4
...
Now, searching for the word “drug” uses the inverted index and return article#1 and article#2 directly.
I recommend the IR Book, if you want to learn more about this.
Build an Articles App
We will start with the famous blog example used by the Rails guides.
Create the Rails App
Type the following at the command prompt:
$ rails new blog
$ cd blog
$ bundle install
$ rails s
Create the Articles Controller
Create the articles controller using the Rails generator, add routes to config/routes.rb, and add methods for showing, creating, and listing articles.
$ rails g controller articles
Then open config/routes.rb and add this resource:
Blog::Application.routes.draw do
resources :articles
end
Now, open app/controllers/articles_controller.rb and add methods to create, view, and list articles.
def index
@articles = Article.all
end
def show
@article = Article.find params[:id]
end
def new
end
def create
@article = Article.new article_params
if @article.save
redirect_to @article
else
render 'new'
end
end
private
def article_params
params.require(:article).permit :title, :text
end
Article Model
We’ll need a model for the articles, so generate it like so:
$ rails g model Article title:string text:text
$ rake db:migrate
Views
New Article Form
Create a new file at app/views/articles/new.html.erb with the following content:
<h1>New Article</h1>
<%= form_for :article, url: articles_path do |f| %>
<% if not @article.nil? and @article.errors.any? %>
<div id="error_explanation">
<h2><%= pluralize(@article.errors.count, "error") %> prohibited
this article from being saved:</h2>
<ul>
<% @article.errors.full_messages.each do |msg| %>
<li><%= msg %></li>
<% end %>
</ul>
</div>
<% end %>
<p>
<%= f.label :title %><br>
<%= f.text_field :title %>
</p>
<p>
<%= f.label :text %><br>
<%= f.text_area :text %>
</p>
<p>
<%= f.submit %>
</p>
<% end %>
<%= link_to '<- Back', articles_path %>
Show One Article
Create another file at app/views/articles/show.html.erb:
<p>
<strong>Title:</strong>
<%= @article.title %>
</p>
<p>
<strong>Text:</strong>
<%= @article.text %>
</p>
<%= link_to '<- Back', articles_path %>
List All Articles
Create a third file at app/views/articles/index.html.erb:
<h1>Articles</h1>
<ul>
<% @articles.each do |article| %>
<li>
<h3>
<%= article.title %>
</h3>
<p>
<%= article.text %>
</p>
</li>
<% end -%>
</ul>
<%= link_to 'New Article', new_article_path %>
You can now add and view articles. Make sure you start the Rails server and go to http://localhost:3000/articles. Click on “New Article” and add a few articles. These will be used to test our full-text search capabilities.
Integrate ElasticSearch
Currently, we can find an article by id only. Integrating ElasticSearch will allow finding articles by any word in its title or text.
Install for Ubuntu and Mac
Ubuntu
Go to elasticsearch.org/download and download the DEB file. Once the file is local, type:
$ sudo dpkg -i elasticsearch-[version].deb
Mac
If you’re on a Mac, Homebrew makes it easy:
$ brew install elasticsearch
Validate Installation
Open this url: http://localhost:9200 and you’ll see ElasticSearch respond like so:
{
"status" : 200,
"name" : "Anvil",
"version" : {
"number" : "1.2.1",
"build_hash" : "6c95b759f9e7ef0f8e17f77d850da43ce8a4b364",
"build_timestamp" : "2014-06-03T15:02:52Z",
"build_snapshot" : false,
"lucene_version" : "4.8"
},
"tagline" : "You Know, for Search"
}
Add Basic Search
Create a controller called search
, along with a view so you can do something like: /search?q=ruby.
Gemfile
gem 'elasticsearch-model'
gem 'elasticsearch-rails'
Remember to run bundle install
to install these gems.
Search Controller
Create The SearchController
:
$ rails g controller search
Add this method to app/controller/search_controller.rb:
def search
if params[:q].nil?
@articles = []
else
@articles = Article.search params[:q]
end
end
Integrate Search into Article
To add the ElasticSearch integration to the Article model, require elasticsearch/model
and include the main module in Article
class.
Modify app/models/article.rb:
require 'elasticsearch/model'
class Article < ActiveRecord::Base
include Elasticsearch::Model
include Elasticsearch::Model::Callbacks
end
Article.import # for auto sync model with elastic search
Search View
Create a new file at app/views/search/search.html.erb:
<h1>Articles Search</h1>
<%= form_for search_path, method: :get do |f| %>
<p>
<%= f.label "Search for" %>
<%= text_field_tag :q, params[:q] %>
<%= submit_tag "Go", name: nil %>
</p>
<% end %>
<ul>
<% @articles.each do |article| %>
<li>
<h3>
<%= link_to article.title, controller: "articles", action: "show", id: article._id%>
</h3>
</li>
<% end %>
</ul>
Search Route
Add the search route to _config/routes.rb-:
get 'search', to: 'search#search'
You can now go to http://localhost:3000/search and search for any word in the articles you created.
Enhance the Search
You may notice that there are some limitations in your search engine. For example, searching for part of a word, such as “rub” or “roby” instead of “ruby”, will give you zero results. Also, it’d be nice if the search engine gave results that include words similar to your search term.
ElasticSearch provides a lot of features to enhance your search. I will give some examples.
Custom Query
There are different types of queries that we can use. So far, we are just using the default ElasticSearch query. To enhance search results, we need to modify this default query. We can, for example, give higher priority for fields like title over other fields.
ElasticSearch provides a full Query DSL based on JSON to define queries. In general, there are basic queries, such as term or prefix. There are also compound queries, like the bool query. Queries can also have filters associated with them, such as the filtered or constant_score queries.
Let’s add a custom search method to our article model in app/models/article.rb:
def self.search(query)
__elasticsearch__.search(
{
query: {
multi_match: {
query: query,
fields: ['title^10', 'text']
}
}
}
)
end
Note: ^10 boosts by 10 the score of hits when the search term is matched in the title
Custom Mapping
Mapping is the process of defining how a document should be mapped to the Search Engine, including its searchable characteristics like which fields are searchable and if/how they are tokenized.
Explicit mapping is defined on an index/type level. By default, there isn’t a need to define an explicit mapping, since one is automatically created and registered when a new type or new field is introduced (with no performance overhead) and has sensible defaults. Only when the defaults need to be overridden must a mapping definition be provided.
We will improve the search so that you can search for a term like “search” and receive results also including “searches” and “searching” ..etc. This will use the built-in English analyzer in ElasticSearch to apply word stemming before indexing.
Add this mapping to the Article class: at app/models/article.rb
settings index: { number_of_shards: 1 } do
mappings dynamic: 'false' do
indexes :title, analyzer: 'english'
indexes :text, analyzer: 'english'
end
end
It’s a good idea to add the following lines to the end of the file to automatically drop and rebuiled the index when article.rb is loaded:
# Delete the previous articles index in Elasticsearch
Article.__elasticsearch__.client.indices.delete index: Article.index_name rescue nil
# Create the new index with the new mapping
Article.__elasticsearch__.client.indices.create \
index: Article.index_name,
body: { settings: Article.settings.to_hash, mappings: Article.mappings.to_hash }
# Index all article records from the DB to Elasticsearch
Article.import
Search Highlighting
Basically, we’d like to show the parts of the articles where the term we are searching for appears. It’s like when you search in google and you see a sample of the document that includes your term in bold. In ElasticSearch, this is called “highlights”. We will add a highlight parameter to our query and specify the fields we want to highlight. ElasticSearch will return the term between an tag, along with a few words before and after the term.
Assuming we are searching for the term “opensource”, ElasticSearch will return something like this:
Elasticsearch is a flexible and powerful <em>opensource</em>, distributed, real-time search and analytics
Note that “opensource” is surounded by an tag.
Add Highlights to the SearchController
First, add the “highlight” parameter to the ElasticSearch query:
def self.search(query)
__elasticsearch__.search(
{
query: {
multi_match: {
query: query,
fields: ['title^10', 'text']
}
},
highlight: {
pre_tags: ['<em>'],
post_tags: ['</em>'],
fields: {
title: {},
text: {}
}
}
}
)
end
Show Highlights in View
It’s pretty easy show this highlight in the view. Go to app/views/search/search.html.erb and replace the ul
element with this:
<ul>
<% @articles.each do |article| %>
<li>
<h3>
<%= link_to article.try(:highlight).try(:title) ? article.highlight.title[0].html_safe : article.title,
controller: "articles",
action: "show",
id: article._id%>
</h3>
<% if article.try(:highlight).try(:text) %>
<% article.highlight.text.each do |snippet| %>
<p><%= snippet.html_safe %>...</p>
<% end %>
<% end %>
</li>
<% end %>
</ul>
Now add a style for in app/assets/stylesheets/search.css.scss:
em {
background: yellow;
}
One last thing we need is the highlighted term returned by ElasticSearch to be surrounded by a few words. If you need to show the title from the beginning, add index_options: 'offsets'
to the title mapping:
settings index: { number_of_shards: 1 } do
mappings dynamic: 'false' do
indexes :title, analyzer: 'english', index_options: 'offsets'
indexes :text, analyzer: 'english'
end
end
This was a quick example for integerating ElasticSearch into a Rails app. We added basic search, then mixed things up a little using custom queries, mapping, and highlights. You can download the full source from here