Integrate Elasticsearch with Silex

Share this article

In the previous article I started exploring the integration between Drupal 7 and the Elasticsearch engine. The goal was to see how we can combine these open source technologies to achieve a high performance application that uses the best of both worlds. If you’re just now joining us, you should check out this repository which contains relevant code for these articles.

esdrupalsilex

We’ll now create a small Silex application that reads data straight from Elasticsearch and returns it to the user.

Silex app

Silex is a great PHP micro framework developed by the same people that are behind the Symfony project. It is in fact using mainly Symfony components but at a more simplified level. Let’s see how we can get started really quickly with a Silex app.

There is more than one way. You can add it as a dependency to an existent composer based project:

"silex/silex": "~1.2",

Or you can even create a new project using a nice little skeleton provided by the creator:

composer.phar create-project fabpot/silex-skeleton

Regardless of how your project is set up, in order to access Elasticsearch we’ll need to use its PHP SDK. That needs to be added to Composer:

"elasticsearch/elasticsearch": "~1.0",

And if we want to use Twig to output data, we’ll need this as well (if not already there of course):

"symfony/twig-bridge": "~2.3"

In order to use the SDK, we can expose it as a service to Pimple, the tiny Silex dependency injection container (much easier than it sounds). Depending on how our project is set up, we can do this in a number of places (see the repository for an example). But basically, after we instantiate the new Silex application, we can add the following:

$app['elasticsearch'] = function() {
  return new Client(array());
};

This creates a new service called elasticsearch on our app that instantiates an object of the Elasticsearch Client class. And don’t forget we need to use that class at the top:

use Elasticsearch\Client;

Now, wherever we want, we can get the Elasticsearch client by simply referring to that property in the $app object:

$client = $app['elasticsearch'];

Connecting to Elasticsearch

In the previous article we’ve managed to get our node data into the node index with each node type giving the name of an Elasticsearch document type. So for instance, this will return all the article node types:

http://localhost:9200/node/article/_search

We’ve also seen how to instantiate a client for our Elasticsearch SDK. Now it’s time to use it somehow. One way is to create a controller:

<?php

namespace Controller;

use Silex\Application;
use Symfony\Component\HttpFoundation\Response;

class NodeController {

  /**
   * Shows a listing of nodes.
   *
   * @return \Symfony\Component\HttpFoundation\Response
   */
  public function index() {
    return new Response('Here there should be a listing of nodes...');
  }
  
  /**
   * Shows one node
   *
   * @param $nid
   * @param \Silex\Application $app
   * @return mixed
   */
  public function show($nid, Application $app) {
    $client = $app['elasticsearch'];
    $params = array(
      'index' => 'node',
      'body' => array(
        'query' => array(
          'match' => array(
            'nid' => $nid,
          ),
        ),
      )
    );
    
    $result = $client->search($params);
    if ($result && $result['hits']['total'] === 0) {
      $app->abort(404, sprintf('Node %s does not exist.', $nid));
    }
    
    if ($result['hits']['total'] === 1) {
      $node = $result['hits']['hits'];
      return $app['twig']->render('node.html.twig', array('node' => reset($node)));
    }
  }
}

Depending on how you organise your Silex application, there are a number of places this controller class can go. In my case it resides inside the src/Controller folder and it’s autoloaded by Composer.

We also need to create a route that maps to this Controller though. Again, there are a couple of different ways to handle this but in my example I have a routes.php file located inside the src/ folder and required inside index.php:

<?php

use Symfony\Component\HttpFoundation\Response;

/**
 * Error handler
 */
$app->error(function (\Exception $e, $code) {
  switch ($code) {
    case 404:
      $message = $e->getMessage();
      break;
    default:
      $message = 'We are sorry, but something went terribly wrong. ' . $e->getMessage();
  }

  return new Response($message);
});

/**
 * Route for /node
 */
$app->get("/node", "Controller\\NodeController::index");

/**
 * Route /node/{nid} where {nid} is a node id
 */
$app->get("/node/{nid}", "Controller\\NodeController::show");

So what happens in my example above? First, I defined an error handler for the application, just so I can see the exceptions being caught and print them on the screen. Not a big deal. Next, I defined two routes that map to my two controller methods defined before. But for the sake of brevity, I only exemplified what the prospective show() method might do:

  • Get the Elasticsearch client
  • Build the Elasticsearch query parameters (similar to what we did in the Drupal environment)
  • Perform the query
  • Check for the results and if a node was found, render it with a Twig template and pass the node data to it.
  • If no results are found, abort the process with a 404 that calls our error handler for this HTTP code declared above.

If you want to follow this example, keep in mind that to use Twig you’ll need to register it with your application. It’s not so difficult if you have it already in your vendor folder through composer.

After you instantiate the Silex app, you can register the provider:

$app->register(new TwigServiceProvider());

Make sure you use the class at the top:

use Silex\Provider\TwigServiceProvider;

And add it as a service with some basic configuration:

$app['twig'] = $app->share($app->extend('twig', function ($twig, $app) {
  return $twig;
}));
$app['twig.path'] = array(__DIR__.'/../templates');

Now you can create template files inside the templates/ folder of your application. For learning more about setting up a Silex application, I do encourage you to read this introduction to the framework.

To continue with our controller example though, here we have a couple of template files that output the node data.

Inside a page.html.twig file:

<!DOCTYPE html>
	<html>
	<head>
	    {% block head %}
	        <title>{% block title %}{% endblock %} - My Elasticsearch Site</title>
	    {% endblock %}
	</head>
	<body>
	<div id="content">{% block content %}{% endblock %}</div>
	</body>
	</html>

And inside the node.html.twig file we used in the controller for rendering:

{% extends "page.html.twig" %}

{% block title %}{{ node._source.title }}{% endblock %}

{% block content %}

    <article>
        <h1>{{ node._source.title }}</h1>

        <div id="content">

            {% if node._source.field_image %}
                <div class="field-image">
                    {% for image in node._source.field_image %}
                        <img src="{{ image.url }}" alt="img.alt"/>
                    {% endfor %}
                </div>
            {% endif %}

            {% if node._source.body %}
                <div class="field-body">
                    {% for body in node._source.body %}
                        {{ body.value|striptags('<p><div><br><img><a>')|raw }}
                    {% endfor %}
                </div>
            {% endif %}
        </div>
    </article>


{% endblock %}

This is just some basic templating for getting our node data printed in the browser (not so fun otherwise). We have a base file and one that extends it and outputs the node title, images and body text to the screen.

Alternatively, you can also return a JSON response from your controller with the help of the JsonResponse class:

use Symfony\Component\HttpFoundation\JsonResponse;

And from your controller simply return a new instance with the values passed to it:

return new JsonResponse($node);

You can easily build an API like this. But for now, this should already work. By pointing your browser to http://localhost/node/5 you should see data from Drupal’s node 5 (if you have it). With one big difference: it is much much faster. There is no bootstrapping, theming layer, database query etc. On the other hand, you don’t have anything useful either out of the box except for what you build yourself using Silex/Symfony components. This can be a good thing or a bad thing depending on the type of project you are working on. But the point is you have the option of drawing some lines for this integration and decide its extent.

One end of the spectrum could be building your entire front end with Twig or even Angular.js with Silex as the API backend. The other would be to use Silex/Elasticsearch for one Drupal page and use it only for better content search. Somewhere in the middle would probably be using such a solution for an entire section of a Drupal site that is dedicated to interacting with heavy data (like a video store or something). It’s up to you.

Conclusion

We’ve seen in this article how we can quickly set up a Silex app and use it to return some data from Elasticsearch. The goal was not so much to learn how any of these technologies work, but more of exploring the options for integrating them. The starting point was the Drupal website which can act as a perfect content management system that scales highly if built properly. Data managed there can be dumped into a high performance data store powered by Elasticsearch and retrieved again for the end users with the help of Silex, a lean and fast PHP framework.

Frequently Asked Questions about Integrating Elasticsearch with Silex

How can I install Elasticsearch in Symfony?

Installing Elasticsearch in Symfony involves a few steps. First, you need to install Java, as Elasticsearch runs on the Java Virtual Machine (JVM). After installing Java, you can download and install Elasticsearch. Once installed, you can start Elasticsearch by running the command ‘./bin/elasticsearch’ in the terminal. To integrate Elasticsearch with Symfony, you can use the FOSElasticaBundle. This bundle provides a way to interact with Elasticsearch in Symfony applications.

What are the benefits of integrating Elasticsearch with Silex?

Integrating Elasticsearch with Silex offers several benefits. Elasticsearch is a powerful search engine that provides full-text search capabilities, making it easier for users to find relevant information. It also offers real-time analytics, allowing you to gain insights from your data quickly. Silex, on the other hand, is a lightweight web framework that is easy to use and highly customizable. By integrating these two technologies, you can create a robust and efficient web application.

How can I configure Elasticsearch in Silex?

Configuring Elasticsearch in Silex involves defining the Elasticsearch client and index in your Silex application. You can do this by adding the Elasticsearch service provider to your application and configuring the client and index settings. The client settings include the host and port of your Elasticsearch server, while the index settings define the name and settings of your Elasticsearch index.

How can I use Elasticsearch for data analysis in Silex?

Elasticsearch provides powerful data analysis capabilities through its aggregation framework. You can use this framework to perform complex data analysis tasks, such as calculating averages, sums, or counts, grouping data by certain fields, and creating histograms or date histograms. To use these features in Silex, you can use the Elasticsearch client to send aggregation queries to your Elasticsearch server.

How can I handle errors when using Elasticsearch in Silex?

Handling errors when using Elasticsearch in Silex involves catching exceptions thrown by the Elasticsearch client. These exceptions can occur when there is a problem with the Elasticsearch server, such as a connection error or a query error. To handle these errors, you can use a try-catch block to catch the exceptions and handle them appropriately.

How can I optimize the performance of Elasticsearch in Silex?

Optimizing the performance of Elasticsearch in Silex involves several strategies. These include optimizing your Elasticsearch queries, using the bulk API for large operations, and properly configuring your Elasticsearch server. You can also use the Elasticsearch monitoring features to monitor the performance of your Elasticsearch server and identify any potential issues.

How can I secure my Elasticsearch server in Silex?

Securing your Elasticsearch server in Silex involves several steps. These include configuring your Elasticsearch server to use HTTPS, setting up authentication and authorization, and securing your Elasticsearch data. You can also use the Elasticsearch security features to monitor and audit your Elasticsearch server.

How can I update my Elasticsearch index in Silex?

Updating your Elasticsearch index in Silex involves using the Elasticsearch client to send update requests to your Elasticsearch server. These requests can be used to update individual documents in your index, or to update multiple documents at once using the bulk API.

How can I delete data from my Elasticsearch index in Silex?

Deleting data from your Elasticsearch index in Silex involves using the Elasticsearch client to send delete requests to your Elasticsearch server. These requests can be used to delete individual documents from your index, or to delete multiple documents at once using the bulk API.

How can I troubleshoot problems with Elasticsearch in Silex?

Troubleshooting problems with Elasticsearch in Silex involves several steps. These include checking the Elasticsearch server logs for any error messages, using the Elasticsearch client to check the status of your Elasticsearch server, and using the Elasticsearch monitoring features to identify any potential issues.

Daniel SiposDaniel Sipos
View Author

Daniel Sipos is a Drupal developer who lives in Brussels, Belgium. He works professionally with Drupal but likes to use other PHP frameworks and technologies as well. He runs webomelette.com, a Drupal blog where he writes articles and tutorials about Drupal development, theming and site building.

BrunoSdrupal-7drupal-planetdrupal7ElasticsearchOOPHPPHPsilex
Share this article
Read Next
Get the freshest news and resources for developers, designers and digital creators in your inbox each week