Using Solarium with SOLR for Search – Solarium and GUI

This is the second article in a four part series on using Solarium, in conjunction with Apache’s SOLR search implementation.

In the first part, I introduced the key concepts and we installed and set up SOLR. In this second part we’ll install Solarium, start building an example application, populate the search index and get into a position where we can start running searches.

Creating the Application

Create a new Laravel application via Composer:

composer create-project laravel/laravel movie-search --prefer-dist

Make the storage directory app/storage directory writeable.

Installing Solarium

You’ll recall from the first part that the Solarium Project provides a level of abstraction over the underlying SOLR requests which enables you to code as if SOLR were a native search implementation running locally, rather than worry about issuing HTTP requests or parsing the responses.

By far the best way to install Solarium is using Composer:

"solarium/solarium": "dev-develop"

Alternatively you can download or clone it from Github.

If you’re following along and building the movie search application, this will go in your newly created project’s composer.json file just as you would any other third-party package; there’s no Laravel service provider though, in this case.

Configuring Solarium

The constructor to the Solarium client takes an array of connection details, so create a configuration file – app/config/solr.php as follows:

return array(
    'host'      => '127.0.0.1',
    'port'      => 8983,
    'path'      => '/solr/',
):

If you’ve installed SOLR as per the instructions in the first part these defaults should be just fine, although in some circumstances you may need to change the port number.

For simplicity, we’re simply going to create an instance of the Solarium client as a property of the controller (app/controllers/HomeController.php):

    /**
     * @var The SOLR client.
     */
    protected $client;

    /**
     * Constructor
     **/
    public function __construct()
    {
        // create a client instance      
        $this->client = new \Solarium\Client(Config::get('solr'));
    }

Normally in Laravel you’d create an instance in a service provider bound to the IoC container, but this way will do fine for what’s a pretty simple application.

Ping Queries

A ping query is useful for checking that the SOLR server is running and accessible, and therefore a good place to begin. Using Solarium it’s simple, so you may wish to test if everything is working by using the following:

// create a ping query
$ping = $client->createPing();

// execute the ping query
try {
    $result = $client->ping($ping);
} catch (Solarium\Exception $e) {
    // the SOLR server is inaccessible, do something
}

As the example illustrates, an inaccessible SOLR instance throws an exception, so you can act accordingly by catching it at this point.

Sample Data

For the purposes of this tutorial we’re going to build a simple movie search. I’ve created a CSV file containing around 2,000 movies around a bunch of keywords (for example space, night and house) which you can download, or if you want to create your own, you might want to check out the Rotten Tomatoes API. (As an aside, but one which is related, IMDB make their data available but spread over a number of CSV files – some of which are enormous – and only make them available via the website or over FTP.)

Before we write a command to import this data, let’s look at the basic create, update and delete operations on the SOLR search implementation using Solarium.

Adding Documents to the Search Index

To add a document to the search index, you first need to create an update query instance:

$update = $client->createUpdate();

Then create a document:

$doc = $update->createDocument();    

Now you can treat the document ($doc) as if it were a stdClass and simply assign data as public properties:

$doc->id = 123;
$doc->title = 'Some Movie';
$doc->cast = array('Sylvester Stallone', 'Marylin Monroe', 'Macauley Culkin');

…and so on, before adding the document to the update query:

$update->addDocument($doc);

Then commit the update:

$update->addCommit();

Finally, to actually run the query you call update() on the client:

$result = $client->update($update);

If you wish to verify that you’ve successfully indexed some documents, visit the SOLR admin panel in your browser – typically http://localhost:8983/solr and click Core Admin, you’ll find the number of documents in the index listed as numDocs in the Index section.

Updating Documents

If you need to update a document in the index, you simply need to “re-add” it and – assuming it has the same unique identifier – SOLR will be smart enough to update it, rather than add a new one.

Deleting Documents

You can delete a document from the index using an update query, using syntax not too dissimilar to WHERE clauses in SQL. For example, to delete a document uniquely identified by the ID 123:

// get an update query instance
$update = $client->createUpdate();

// add the delete query and a commit command to the update query
$update->addDeleteQuery('id:123');
$update->addCommit();

// this executes the query and returns the result
$result = $client->update($update);

Or you can be less specific:

// get an update query instance
$update = $client->createUpdate();

// add the delete query and a commit command to the update query
$update->addDeleteQuery('title:test*');
$update->addCommit();

// this executes the query and returns the result
$result = $client->update($update);

Note the use of a wildcard character – in other words: “delete all documents whose title starts with test”.

Populating the Search Index with Movies

Now we’ve looked at the fundamentals of indexing documents, let’s put some data into the index for our sample application.

Laravel makes it easy to build console commands, so let’s create one to import the contents of our movie CSV file and index them. We could also create corresponding database records at this point, but for the purposes of this exercise the indexed documents will contain everything we need.

To create the command, enter the following into the command line:

php artisan command:make PopulateSearchIndexCommand

In the newly-generated file – app/commands/PopulateSearchIndexCommand.php – set the command’s name (i.e., what you enter on the command-line to run it) and the description:

/**
 * The console command name.
 *
 * @var string
 */
protected $name = 'search:populate';

/**
 * The console command description.
 *
 * @var string
 */
protected $description = 'Populates the search index with some sample movie data.';

Now we’ll use the fire() method to create the Solarium client, open the CSV, iterate through it and index each movie:

/**
 * Execute the console command.
 *
 * @return void
 */
public function fire()
{       
    $client = new \Solarium\Client(Config::get('solr'));

    // open up the CSV
    $csv_filepath = storage_path() . '/movies.csv';

    $fp = fopen($csv_filepath, 'r');

    // Now let's start importing
    while (($row = fgetcsv($fp, 1000, ";")) !== FALSE) {

        // get an update query instance
    $update = $client->createUpdate();

    // Create a document
        $doc = $update->createDocument();    

        // set the ID
    $doc->id = $row[0];

    // ..and the title
        $doc->title = $row[1];

        // The year, rating and runtime columns don't always have data
        if (strlen($row[2])) {
            $doc->year = $row[2];
        }
        if (strlen($row[3])) {
            $doc->rating = $row[3];
        }
        if (strlen($row[4])) {
            $doc->runtime = $row[4];
        }

        // set the synopsis
        $doc->synopsis = $row[5];

        // We need to create an array of cast members
        $cast = array();

        // Rows 6 through 10 contain (or don't contain) cast members' names
        for ($i = 6; $i <= 10; $i++) {
            if ((isset($row[$i])) && (strlen($row[$i]))) {
                $cast[] = $row[$i];
            }
        }

        // ...then we can assign the cast member array to the document
        $doc->cast = $cast;

        // Let's simply add and commit straight away.
        $update->addDocument($doc);
    $update->addCommit();

    // this executes the query and returns the result
    $result = $client->update($update);

    }

    fclose($fp);
}

Whilst this isn’t the slickest or most resilient piece of code, it’s a fairly academic exercise which we’re only planning to run once anyway. Ensure the CSV file is in the correct place – app/storage/movies.csv and run it:

php artisan search:populate

All being well, the index should now contain ~2,300 movies. You can check this via the admin interface.

In the next section we’ll start building the basic search in.

The Search Form

Let’s create the search form using Laravel’s Blade templating engine, along with the form builder. So, in app/views/home/index.blade.php:

@extends('layouts.default')

@section('content')

<header>
    {{ Form::open(array('method' => 'GET')) }}
    <div class="input-group">
        {{ Form::text('q', Input::get('q'), array('class' => 'form-control input-lg', 'placeholder' => 'Enter your search term')) }}        
        <span class="input-group-btn">
            {{ Form::submit('Search', array('class' => 'btn btn-primary btn-lg')) }}            
        </span>
    </div>
    {{ Form::close() }}
</header>

@endsection

A basic page layout in app/views/layouts/default.blade.php:

<!doctype html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Movie Search</title>

    <link rel="stylesheet" href="//netdna.bootstrapcdn.com/bootstrap/3.0.3/css/bootstrap.min.css">

</head>
<body>
    <div class="container">
        <header id="header" class="row">        
            <div class="col-sm-12 col-md-12 col-lg-12">
                <h1>Movie Search</h1>
            </div>      
        </header>

        <div class="row">
            <div class="col-sm-12 col-md-12 col-lg-12">
                @yield('content')
            </div>
        </div>

        <hr />

        <footer id="footer">

        </footer>

    </div>

</body>
</html>

In app/controllers/HomeController.php:

public function getIndex()
{
    return View::make('home.index');
}

Finally, replace the contents of app/routes.php with the following, to tell Laravel to use HomeController:

<?php
/*
|--------------------------------------------------------------------------
| Application Routes
|--------------------------------------------------------------------------
*/
Route::controller('/', 'HomeController');

Now we’re set up and ready to implement the basic search mechanism, which will be the subject of the next part in the series.

Summary

In this second installment of a four part series, we’ve installed Solarium with a view to integrating Apache’s SOLR with a PHP application. We’ve set up an example application and indexed some data. In the next part we’ll start implementing the search mechanism.

Frequently Asked Questions (FAQs) about Using Solarium for Solr Search