Using Solarium with SOLR for Search – Implementation

Lukas White
Lukas White
Share

This is the third article in a four-part series on using Solarium, in conjunction with Apache’s SOLR search implementation.

In the first part I introduced the key concepts and we installed and set up SOLR. In part two we installed and configured Solarium, a library which enables us to use PHP to “talk” to SOLR as if it were a native component.

Now we’re finally ready to start building the search mechanism, which is the subject of this installment.

Let’s look at how to implement a really simple search:

$query = $client->createSelect();
$query->setQuery(Input::get('q'));

Input::get('q') is simply Laravel’s way of grabbing a GET or POST variable named q which, you’ll remember, is the name of our search form element.

Or better still, use a placeholder to escape the search phrase:

$query->setQuery('%P1%', array(Input::get('q')));

A placeholder is indicated by the % symbols. The letter “P” means “escape this as a Phrase”. The bound variables are passed as an array, and the number indicates the position in the array of the argument you wish to bind; bearing in mind that (perhaps unusually) 1 indicates the first item.

To run the search:

$resultset = $client->select($query);

You can now retrieve the number of results using the getNumFound() method, for example:

printf('Your search yielded %d results:', $resultset->getNumFound());

$resultset is an instance of Solarium\QueryType\Select\Result\Result, which implements the Iterator interface – so you can iterate through the results as follows:

foreach ($resultset as $document) {
    . . .
}

Each result is an instance of Solarium\QueryType\Select\Result\Document, which provides two ways in which you can access the individual fields – either as public properties, e.g.:

<h3><?php print $document->title ?></h3>

Or, you can iterate through the available fields:

foreach($document AS $field => $value)
{
    // this converts multi-value fields to a comma-separated string
    if(is_array($value)) $value = implode(', ', $value);

    print '<strong>' . $field . '</strong>: ' . $value . '<br />';        
}

Note that multi-value fields – such as cast – will return an array; so in the example above, it will simply collapse these fields into a comma-separated list.

Okay, so that’s an overview of how to do it – now let’s plug it into our example application.

We’ll make the search respond to a GET request rather than POST because it’ll make it easier when we come to look at faceted search – although it’s actually very common for site searches to use GET.

So the index route on the home controller (our application only has one page, after all) becomes the following:

/**
 * Display the search form / run the search.
 */
public function getIndex()
{

    if (Input::has('q')) {

        // Create a search query
        $query = $this->client->createSelect();

        // Set the query string      
        $query->setQuery('%P1%', array(Input::get('q')));

        // Execute the query and return the result
        $resultset = $this->client->select($query);

        // Pass the resultset to the view and return.
        return View::make('home.index', array(
            'q' => Input::get('q'),
            'resultset' => $resultset,
        ));

    }

    // No query to execute, just return the search form.
    return View::make('home.index');

}

Now let’s modify the view – app/views/home/index.blade.php – so that it displays the search results, as well as a result count, by adding this below the search form:

@if (isset($resultset))    
<header>
    <p>Your search yielded <strong>{{ $resultset->getNumFound() }}</strong> results:</p>
    <hr />
</header>

@foreach ($resultset as $document)

    <h3>{{ $document->title }}</h3>

    <dl>
        <dt>Year</dt>
        <dd>{{ $document->year }}</dd>

        @if (is_array($document->cast))
        <dt>Cast</dt>
        <dd>{{ implode(', ', $document->cast) }}</dd>              
        @endif

        </dl>

        {{ $document->synopsis }}

@endforeach
@endif

Try running a few searches. Quite quickly, you might notice a major limitation. As an example, try searching for “Star Wars”, note the first few results and then do a search for “Mark Hamill”. No results – looks like the search is only taking into account the title attribute, but not the cast.

To alter this behavior we need to use the DisMax component. DisMax is an abbreviation of Disjunction Max. Disjunction means it searches across multiple fields. Max means that if a query matches multiple fields, the maximum scores are added together.

To indicate that we wish to perform a DisMax query:

$dismax = $query->getDisMax();

Then we can tell the search to look in multiple fields – separate them with a space:

$dismax->setQueryFields('title cast synopsis');

Now, if you try searching for “Mark Hamill” again, you’ll see that the search picks up the cast, as well as the title.

We can take our DisMax query one step further by attaching weights to fields. This allows you to prioritize certain fields over others – for example, you probably want title matches to give you a higher score than matching words in the synopsis. Take a look at the following line:

$dismax->setQueryFields('title^3 cast^2 synopsis^1');

This indicates that we wish matches on the cast field to be weighted much higher than the synopsis – by a magnitude of two – and the title field further still. For your own projects, you’ll probably want to play around and experiment with various queries to try and work out the optimum weightings, which are likely to be very specific to the application in question.

So just to sum up, we can implement searching over multiple fields by modifying app/controllers/HomeController.php as follows:

        // Set the query string      
  $query->setQuery('%P1%', array(Input::get('q')));

        // Create a DisMax query
  $dismax = $query->getDisMax();

  // Set the fields to query, and their relative weights
  $dismax->setQueryFields('title^3 cast^2 synopsis^1');

  // Execute the query and return the result
  $resultset = $this->client->select($query);

Specifying Which Fields to Return

If you run the search, then for each resultset document iterate through the fields, you’ll see that by default every field we’ve added to the index gets returned. In addition, SOLR adds the _version_ field, and the score associated with the search result, along with the unique identifier.

The score is a numeric value which expresses the relevance of the result.

If you wish to change this behavior, there are three methods you can use:

$query->clearFields(); // return no fields

$query->addField('title');  // add 'title' to the list of fields returned

$query->addFields(array('title', 'cast')); // add several fields to the list of those returned

Note that you’ll probably need to use clearFields() in conjunction with addField() or addFields():

$query->clearFields()->addFields(array('title', 'cast'));

Just as in SQL, you can use an asterisk as a wildcard – meaning select all fields:

$query->clearFields()->addFields('*');

Sorting Search Results

By default, search results will be returned in descending order of score. In most cases this is probably what you want; “best matches” appear first.

However, you can change this behavior if you wish as follows:

$query->addSort('title', 'asc');

The syntax will probably look familiar; it’s very similar to SQL.

Pagination

You can specify the start position – i.e., where to start listing results – and the number of rows to return. Think of it as being like SQL’s LIMIT clause. So for example, to take the first hundred results you’d do this:

$query->setStart(0);
$query->setRows(200);

Armed with the result of getNumFound() and these functions, it should be straightforward to implement pagination, but for brevity I’m not going to go over that here.

Faceted search essentially allows you to “drill down” through search results based on one or more criteria. It’s probably best illustrated by online stores, where you can refine a product search by things like category, format (e.g. paperbacks vs hardback vs digital books), whether it’s currently in stock or by price range.

Let’s expand our movie search with a basic facet; we’ll allow people to narrow down their movie search by its MPGG rating (a certificate specifying the appropriate age-range for a movie, e.g. “R” or “PG-13”).

To create a facet based on a field, you do this:

$facetSet = $query->getFacetSet();

$facetSet->createFacetField('rating')
    ->setField('rating');

Upon running the search, the result-set can now be broken down based on the value of the field – and you can also display a count for that particular value.

$facet = $resultset->getFacetSet()->getFacet('rating');
foreach($facet as $value => $count) {    
    echo $value . ' [' . $count . ']<br/>';
}

This will give you something along these lines:

Unrated [193]
PG [26]
R [23]
PG-13 [16]
G [9]
NC-17 [0]

A facet doesn’t have to use single, distinct values. You can use ranges – for example, you might have price ranges in an e-commerce site. To illustrate facet ranges in our movie search, we’re going to allow people to narrow their search to movies from particular decade.

Here’s the code to create the facet:

$facet = $facetSet->createFacetRange('years')
    ->setField('year')
    ->setStart(1900)
    ->setGap(10)
    ->setEnd(2020);

This indicates that we want to create a range-based facet on the year field. We need to specify the start value – the year 1900 – and the end; i.e., the end of the current decade. We also need to set the gap; in other words we want increments of ten – a decade. To display the counts in our search results, we could do something like this:

$facet = $resultset->getFacetSet()->getFacet('years');
foreach($facet as $range => $count) {    
    if ($count) {
        printf('%d's (%d)<br />', $range, $count);
    }
}

This will result in something along these lines:

1970's (12)
1980's (6)
2000's (8)

Note that the facet will contain every possible value, so it’s important to check that the count is non-zero before displaying it.

Faceted Search: Filtering

So far we’ve used facets on the search results page to show the counts, but that’s of limited use unless we can allow users to filter their searches on them.

In the search callback, let’s first check whether the MPGG rating filter has been applied:

if (Input::has('rating')) {
    $query->createFilterQuery('rating')->setQuery(sprintf('rating:%s', Input::get('rating')));     
}

Actually, just as with the main search query, we can yet Solarium escape the search term rather than use sprintf:

if (Input::has('rating')) { 
    $query->createFilterQuery('rating')->setQuery('rating:%T1%', array(Input::get('rating')));    
}

Remember, the 1 indicates that we wish to use the first element of the array of arguments – it’s not a zero-based array. The T indicates we wish to escape the value as a term (as opposed to P for phrase).

Filtering on decade is slightly more complicated, because we’re filtering based on a range rather than a discreet value. We only have one value specified – in Input::get('decade') – but we know that the upper bound is simply the start of the decade plus nine. So, for example, “the ‘Eighties” is represented by the value 1980, and the range 1980 through (1980 + 9) = 1989.

A range query takes the following form:

field: [x TO y]

So it would be:

year: [1980 TO 1989]

We can implement this as follows:

if (Input::has('decade')) {
    $query->createFilterQuery('years')->setQuery(sprintf('year:[%d TO %d]', Input::get('decade'), (Input::get('decade') + 9)));            
}

Alternatively we can use a helper instead. To get an instance of the helper class:

$helper = $query->getHelper();

To use it:

if (Input::has('decade')) {     
    $query->createFilterQuery('years')->setQuery($helper->rangeQuery('year', Input::get('decade'), (Input::get('decade') + 9)));
}

Whilst this may seem fairly academic, it’s worth knowing how to create an instance of the Solarium helper because it’s very useful for other things, such as geospatial support.

Faceted Search: The View

Now that we’ve covered how to set up faceted search, how to list the facets and how to run filters based on them, we can set up the corresponding view.

Open up app/views/home/index.blade.php and modify the search results section to include an additional column, which will contain our facets:

@if (isset($resultset))    
<div class="results row" style="margin-top:1em;">
    <div class="col-sm-4 col-md-4 col-lg-3">
        <?php $facet = $resultset->getFacetSet()->getFacet('rating'); ?>
        <div class="panel panel-primary">
            <div class="panel-heading">
                <h3 class="panel-title">By MPGG Rating</h3>
            </div>
            <ul class="list-group">
                @foreach ($facet as $value => $count)
                    @if ($count)
                    <li class="list-group-item">
                        <a href="?{{ http_build_query(array_merge(Input::all(), array('rating' => $value))) }}">{{ $value }}</a>
                        <span class="badge">{{ $count }}</span>
                    </li>
                    @endif
                @endforeach
            </ul>    
        </div>

        <?php $facet = $resultset->getFacetSet()->getFacet('years'); ?>
        <div class="panel panel-primary">
            <div class="panel-heading">
                <h3 class="panel-title">By Decade</h3>
            </div>
            <ul class="list-group">
                @foreach ($facet as $value => $count)
                    @if ($count)
                    <li class="list-group-item">
                        <a href="?{{ http_build_query(array_merge(Input::all(), array('decade' => $value))) }}">{{ $value }}'s</a>
                        <span class="badge">{{ $count }}</span>
                    </li>
                    @endif
                @endforeach
            </ul>    
        </div>
    </div>
    <div class="col-sm-8 col-md-8 col-lg-9">

        <!-- SEARCH RESULTS GO HERE, EXACTLY AS BEFORE -->

    </div>
</div>
@endif

We’re doing as we discussed in the section on facetted search; grabbing the facet set, iterating through each item and displaying it along with a count of the number of results for that particular value.

Each facet item is a link, which when clicked will refresh the page but with that filter applied. It does this by merging in the appropriate value to the currently “active” set of GET parameters; so if you’ve already filtered on one facet, clicking an item in a different facet-set will maintain that filter by including the appropriate query parameters. It will also maintain your original query, which is set as “q” in the input array.

This approach has some limitations – for one thing, there’s no way to “reset” the filters, except to manually alter the query parameters in the address bar – but its aim is to demonstrate using multiple facets. I’ll leave improving it to you as an additional exercise!

Frequently Asked Questions (FAQs) on Using Solarium Solr Search Implementation

How Can I Integrate Solarium Solr with Laravel?

Integrating Solarium Solr with Laravel involves a few steps. First, you need to install Solarium using composer. Run the command composer require solarium/solarium. Next, you need to configure Solarium in your Laravel application. Create a new service provider and register it in your config/app.php file. In the service provider, you can set up Solarium client with the necessary configurations. Finally, you can use the Solarium client in your Laravel application to perform various Solr operations.

What Are the Benefits of Using Solarium Solr with Laravel?

Solarium Solr provides a powerful search functionality that can be integrated into Laravel applications. It offers features like faceted search, filtering, and sorting which can greatly enhance the user experience of your application. Moreover, Solarium is easy to use and provides a fluent API that fits well with Laravel’s elegant syntax.

How Can I Perform a Basic Search Using Solarium Solr?

To perform a basic search using Solarium Solr, you first need to create a select query. You can do this using the createSelect method of the Solarium client. Next, you can set the query term using the setQuery method of the select query. Finally, you can execute the query using the execute method of the Solarium client. The result will be a result set object that you can iterate over to access the individual documents.

How Can I Implement Faceted Search Using Solarium Solr?

Faceted search can be implemented using the createFacetSet method of the select query. You can add facets to the facet set using the createFacetField method. Each facet field represents a field in your Solr index that you want to facet on. Once you have set up the facet set, you can execute the query as usual and access the facet counts in the result set.

How Can I Handle Errors in Solarium Solr?

Solarium Solr throws exceptions when an error occurs. You can catch these exceptions and handle them appropriately in your Laravel application. For example, you might want to log the error and show a user-friendly message to the user. Solarium provides detailed error messages that can help you diagnose and fix the problem.

How Can I Optimize My Solr Index for Better Search Performance?

There are several ways to optimize your Solr index for better search performance. One way is to use the optimize command of the Solarium client. This command merges all segments of the index into a single segment, which can improve search speed. However, it can be resource-intensive and should be used sparingly. Other optimization techniques include using appropriate field types, enabling caching, and tuning the Solr configuration.

How Can I Update My Solr Index When My Data Changes?

You can update your Solr index by sending update requests to Solr. These requests can be created using the createUpdate method of the Solarium client. The update request can contain add, delete, and commit commands. The add command adds or updates documents in the index, the delete command removes documents from the index, and the commit command makes the changes visible to search.

How Can I Use Solarium Solr for Autocomplete Functionality?

Solarium Solr can be used to implement autocomplete functionality by using the suggester component. The suggester component can provide suggestions based on the indexed data. You can configure the suggester in your Solr schema and use the createSuggester method of the Solarium client to create a suggester query. The result of the query will be a list of suggestions that you can use for autocomplete.

How Can I Secure My Solr Server?

Solr provides several security features that you can use to secure your Solr server. These include basic authentication, rule-based authorization, and SSL encryption. You can configure these features in your Solr configuration. In addition, you should follow best practices for securing web applications, such as keeping your software up to date, using strong passwords, and limiting access to your server.

How Can I Monitor the Performance of My Solr Server?

Solr provides a web-based admin interface that you can use to monitor the performance of your Solr server. The admin interface provides information about the status of your Solr cores, the performance of your queries, and the usage of your system resources. In addition, you can use the stats component of Solr to collect detailed statistics about your Solr operations.