Key Takeaways
- The article explains how to integrate ElasticSearch into a Rails application to provide full-text search capabilities. This involves creating a Rails app with an Articles Controller, Article Model, and various views.
- ElasticSearch is installed and integrated into the app, allowing articles to be found by any word in its title or text. The article provides detailed instructions for Ubuntu and Mac installations.
- The search functionality can be enhanced by using custom queries and mappings. The article demonstrates how to give higher priority to certain fields, apply word stemming for more inclusive results, and implement search highlighting.
- The article also provides a FAQ section addressing various aspects of using ElasticSearch in Rails, including installation, indexing, complex search queries, performance optimization, error handling, data updates, index deletion, security, and performance monitoring.
In this article you will learn how to integrate ElasticSearch into a Rails application.
A full-text search engine examines all of the words in every stored document as it tries to match search criteria (text specified by a user) wikipedia. For example, if you want to find articles that talk about Rails, you might search using the term “rails”. If you don’t have a special indexing technique, it means fully scanning all records to find matches, which will be extremely inefficient. One way to solve this is an “inverted index” that maps the words in the content of all records to its location in the database.
For example, if a primary key index is like this:
article#1 -> "breakthrough drug for schizophrenia"
article#2 -> "new schizophrenia drug"
article#3 -> "new approach for treatment of schizophrenia"
article#4 -> "new hopes for schizophrenia patients"
...
An inverted index for these records will be like this:
breakthrough -> article#1
drug -> article#1, article#2
schizophrenia -> article#1, article#2, article#3, article#4
approach -> article#3
new -> article#2, article#3, article#4
hopes -> article#4
...
Now, searching for the word “drug” uses the inverted index and return article#1 and article#2 directly.
I recommend the IR Book, if you want to learn more about this.
Build an Articles App
We will start with the famous blog example used by the Rails guides.
Create the Rails App
Type the following at the command prompt:
$ rails new blog
$ cd blog
$ bundle install
$ rails s
Create the Articles Controller
Create the articles controller using the Rails generator, add routes to config/routes.rb, and add methods for showing, creating, and listing articles.
$ rails g controller articles
Then open config/routes.rb and add this resource:
Blog::Application.routes.draw do
resources :articles
end
Now, open app/controllers/articles_controller.rb and add methods to create, view, and list articles.
def index
@articles = Article.all
end
def show
@article = Article.find params[:id]
end
def new
end
def create
@article = Article.new article_params
if @article.save
redirect_to @article
else
render 'new'
end
end
private
def article_params
params.require(:article).permit :title, :text
end
Article Model
We’ll need a model for the articles, so generate it like so:
$ rails g model Article title:string text:text
$ rake db:migrate
Views
New Article Form
Create a new file at app/views/articles/new.html.erb with the following content:
<h1>New Article</h1>
<%= form_for :article, url: articles_path do |f| %>
<% if not @article.nil? and @article.errors.any? %>
<div id="error_explanation">
<h2><%= pluralize(@article.errors.count, "error") %> prohibited
this article from being saved:</h2>
<ul>
<% @article.errors.full_messages.each do |msg| %>
<li><%= msg %></li>
<% end %>
</ul>
</div>
<% end %>
<p>
<%= f.label :title %><br>
<%= f.text_field :title %>
</p>
<p>
<%= f.label :text %><br>
<%= f.text_area :text %>
</p>
<p>
<%= f.submit %>
</p>
<% end %>
<%= link_to '<- Back', articles_path %>
Show One Article
Create another file at app/views/articles/show.html.erb:
<p>
<strong>Title:</strong>
<%= @article.title %>
</p>
<p>
<strong>Text:</strong>
<%= @article.text %>
</p>
<%= link_to '<- Back', articles_path %>
List All Articles
Create a third file at app/views/articles/index.html.erb:
<h1>Articles</h1>
<ul>
<% @articles.each do |article| %>
<li>
<h3>
<%= article.title %>
</h3>
<p>
<%= article.text %>
</p>
</li>
<% end -%>
</ul>
<%= link_to 'New Article', new_article_path %>
You can now add and view articles. Make sure you start the Rails server and go to http://localhost:3000/articles. Click on “New Article” and add a few articles. These will be used to test our full-text search capabilities.
Integrate ElasticSearch
Currently, we can find an article by id only. Integrating ElasticSearch will allow finding articles by any word in its title or text.
Install for Ubuntu and Mac
Ubuntu
Go to elasticsearch.org/download and download the DEB file. Once the file is local, type:
$ sudo dpkg -i elasticsearch-[version].deb
Mac
If you’re on a Mac, Homebrew makes it easy:
$ brew install elasticsearch
Validate Installation
Open this url: http://localhost:9200 and you’ll see ElasticSearch respond like so:
{
"status" : 200,
"name" : "Anvil",
"version" : {
"number" : "1.2.1",
"build_hash" : "6c95b759f9e7ef0f8e17f77d850da43ce8a4b364",
"build_timestamp" : "2014-06-03T15:02:52Z",
"build_snapshot" : false,
"lucene_version" : "4.8"
},
"tagline" : "You Know, for Search"
}
Add Basic Search
Create a controller called search
, along with a view so you can do something like: /search?q=ruby.
Gemfile
gem 'elasticsearch-model'
gem 'elasticsearch-rails'
Remember to run bundle install
to install these gems.
Search Controller
Create The SearchController
:
$ rails g controller search
Add this method to app/controller/search_controller.rb:
def search
if params[:q].nil?
@articles = []
else
@articles = Article.search params[:q]
end
end
Integrate Search into Article
To add the ElasticSearch integration to the Article model, require elasticsearch/model
and include the main module in Article
class.
Modify app/models/article.rb:
require 'elasticsearch/model'
class Article < ActiveRecord::Base
include Elasticsearch::Model
include Elasticsearch::Model::Callbacks
end
Article.import # for auto sync model with elastic search
Search View
Create a new file at app/views/search/search.html.erb:
<h1>Articles Search</h1>
<%= form_for search_path, method: :get do |f| %>
<p>
<%= f.label "Search for" %>
<%= text_field_tag :q, params[:q] %>
<%= submit_tag "Go", name: nil %>
</p>
<% end %>
<ul>
<% @articles.each do |article| %>
<li>
<h3>
<%= link_to article.title, controller: "articles", action: "show", id: article._id%>
</h3>
</li>
<% end %>
</ul>
Search Route
Add the search route to _config/routes.rb-:
get 'search', to: 'search#search'
You can now go to http://localhost:3000/search and search for any word in the articles you created.
Enhance the Search
You may notice that there are some limitations in your search engine. For example, searching for part of a word, such as “rub” or “roby” instead of “ruby”, will give you zero results. Also, it’d be nice if the search engine gave results that include words similar to your search term.
ElasticSearch provides a lot of features to enhance your search. I will give some examples.
Custom Query
There are different types of queries that we can use. So far, we are just using the default ElasticSearch query. To enhance search results, we need to modify this default query. We can, for example, give higher priority for fields like title over other fields.
ElasticSearch provides a full Query DSL based on JSON to define queries. In general, there are basic queries, such as term or prefix. There are also compound queries, like the bool query. Queries can also have filters associated with them, such as the filtered or constant_score queries.
Let’s add a custom search method to our article model in app/models/article.rb:
def self.search(query)
__elasticsearch__.search(
{
query: {
multi_match: {
query: query,
fields: ['title^10', 'text']
}
}
}
)
end
Note: ^10 boosts by 10 the score of hits when the search term is matched in the title
Custom Mapping
Mapping is the process of defining how a document should be mapped to the Search Engine, including its searchable characteristics like which fields are searchable and if/how they are tokenized.
Explicit mapping is defined on an index/type level. By default, there isn’t a need to define an explicit mapping, since one is automatically created and registered when a new type or new field is introduced (with no performance overhead) and has sensible defaults. Only when the defaults need to be overridden must a mapping definition be provided.
We will improve the search so that you can search for a term like “search” and receive results also including “searches” and “searching” ..etc. This will use the built-in English analyzer in ElasticSearch to apply word stemming before indexing.
Add this mapping to the Article class: at app/models/article.rb
settings index: { number_of_shards: 1 } do
mappings dynamic: 'false' do
indexes :title, analyzer: 'english'
indexes :text, analyzer: 'english'
end
end
It’s a good idea to add the following lines to the end of the file to automatically drop and rebuiled the index when article.rb is loaded:
# Delete the previous articles index in Elasticsearch
Article.__elasticsearch__.client.indices.delete index: Article.index_name rescue nil
# Create the new index with the new mapping
Article.__elasticsearch__.client.indices.create \
index: Article.index_name,
body: { settings: Article.settings.to_hash, mappings: Article.mappings.to_hash }
# Index all article records from the DB to Elasticsearch
Article.import
Search Highlighting
Basically, we’d like to show the parts of the articles where the term we are searching for appears. It’s like when you search in google and you see a sample of the document that includes your term in bold. In ElasticSearch, this is called “highlights”. We will add a highlight parameter to our query and specify the fields we want to highlight. ElasticSearch will return the term between an tag, along with a few words before and after the term.
Assuming we are searching for the term “opensource”, ElasticSearch will return something like this:
Elasticsearch is a flexible and powerful <em>opensource</em>, distributed, real-time search and analytics
Note that “opensource” is surounded by an tag.
Add Highlights to the SearchController
First, add the “highlight” parameter to the ElasticSearch query:
def self.search(query)
__elasticsearch__.search(
{
query: {
multi_match: {
query: query,
fields: ['title^10', 'text']
}
},
highlight: {
pre_tags: ['<em>'],
post_tags: ['</em>'],
fields: {
title: {},
text: {}
}
}
}
)
end
Show Highlights in View
It’s pretty easy show this highlight in the view. Go to app/views/search/search.html.erb and replace the ul
element with this:
<ul>
<% @articles.each do |article| %>
<li>
<h3>
<%= link_to article.try(:highlight).try(:title) ? article.highlight.title[0].html_safe : article.title,
controller: "articles",
action: "show",
id: article._id%>
</h3>
<% if article.try(:highlight).try(:text) %>
<% article.highlight.text.each do |snippet| %>
<p><%= snippet.html_safe %>...</p>
<% end %>
<% end %>
</li>
<% end %>
</ul>
Now add a style for in app/assets/stylesheets/search.css.scss:
em {
background: yellow;
}
One last thing we need is the highlighted term returned by ElasticSearch to be surrounded by a few words. If you need to show the title from the beginning, add index_options: 'offsets'
to the title mapping:
settings index: { number_of_shards: 1 } do
mappings dynamic: 'false' do
indexes :title, analyzer: 'english', index_options: 'offsets'
indexes :text, analyzer: 'english'
end
end
This was a quick example for integerating ElasticSearch into a Rails app. We added basic search, then mixed things up a little using custom queries, mapping, and highlights. You can download the full source from here
References
- elasticsearch/elasticsearch-rails
- rails-application-templates
- Mapping Guide
- Query-DSL Guide
- rubyonrails.org/getting_started.html
Frequently Asked Questions (FAQs) about Full-Text Search in Rails with Elasticsearch
How can I install Elasticsearch in my Rails application?
To install Elasticsearch in your Rails application, you first need to add the Elasticsearch gem to your Gemfile. Run ‘bundle install’ to install the gem. Then, you need to configure Elasticsearch. Create a new initializer file in your config/initializers directory and add the necessary configuration code. You can then start the Elasticsearch server using the command ‘elasticsearch’.
How can I index my data in Elasticsearch?
Indexing your data in Elasticsearch involves creating an index and adding documents to it. You can use the ‘create_index!’ method to create an index and the ‘import’ method to add documents. Remember to specify the model you want to index in the ‘import’ method.
How can I perform a full-text search in Elasticsearch?
To perform a full-text search in Elasticsearch, you can use the ‘search’ method. This method takes a query string as a parameter and returns the matching documents. You can also use the ‘multi_match’ query to search across multiple fields.
How can I handle complex search queries in Elasticsearch?
Elasticsearch provides a powerful query DSL (Domain Specific Language) that allows you to build complex queries. You can use boolean operators, range queries, and more to refine your search results.
How can I optimize my Elasticsearch queries for performance?
Optimizing your Elasticsearch queries can involve several strategies. You can use the ‘explain’ method to understand how your query is being executed and identify potential bottlenecks. You can also use filters instead of queries for faster performance, as filters are cached by Elasticsearch.
How can I handle errors in Elasticsearch?
Handling errors in Elasticsearch involves catching exceptions and handling them appropriately. You can use the ‘rescue’ keyword in Ruby to catch exceptions and display a helpful error message to the user.
How can I update my Elasticsearch index when my data changes?
You can use callbacks in your Rails models to update your Elasticsearch index whenever your data changes. For example, you can use the ‘after_commit’ callback to reindex a document whenever it is updated.
How can I delete an index in Elasticsearch?
To delete an index in Elasticsearch, you can use the ‘delete_index!’ method. Be careful when using this method, as it will permanently delete the index and all its documents.
How can I secure my Elasticsearch server?
Securing your Elasticsearch server involves several steps. You can configure authentication and authorization, use HTTPS for secure communication, and restrict access to your server using firewalls or IP filtering.
How can I monitor the performance of my Elasticsearch server?
Elasticsearch provides several tools for monitoring performance, including the Elasticsearch-head plugin and the Kibana dashboard. These tools provide real-time information about your server’s health and performance.