Sending PHP Event Messages to Remote Logstash on Windows
This article was peer reviewed by Wern Ancheta, Scott Molinari, and Jeff Madsen. Thanks to all of SitePoint’s peer reviewers for making SitePoint content the best it can be!
By opening this article you’ve endeavored yourself to expanding your knowledge of PHP applications as part of event-based distributed systems. You’ll be given a quick intro into what we are referring to when we say event messages, what Logstash is, and why it is so cool.
If you’ve already heard of Beats or understand you can run Logstash locally to ship logs to another Logstash instance or directly to a datastore such as Elasticsearch, this article is still for you and will show you an easy-to-configure-and-run, hopefully more effective and certainly fun-to-use alternative.
Quick Intro into Event Messages and Logstash
With event messages, we gather information about events that occur in our applications, be it business-oriented decisions of the applications’ users, decisions made by the applications themselves, or their failures. Each event, besides the message it conveys, is typically determined by a timestamp and a type such as informational, warning or error. A record of an event is an event log.
Additionally, there’s also Event Sourcing – a somewhat different but also somewhat similar concept which you may want to check out.
There are many tools built specifically for the purpose of shipping logs to datastores for later analysis and making knowledge-based decisions. Logstash is one of them, and because of the vast number of input, output, codec and filter plugins it offers, the most popular. Out of the box, it can read from Heroku app logs, GitHub webhooks or Twitter Streaming API, create new events and send them to Graylog, IRC, or JIRA.
The event messages would ordinarily be of interest to the users of your applications, too. In an application, one page would generate events and another one would display them in an aggregated form.
Let’s consider an example where the first page publishes new blog posts and the other one lists all blog posts related to PHP that have been published in the last month. The application could have talked to a relational database directly for both read and write. But with event messages it is decoupled from the database so other subscribers can be added easily, e.g. an email list or a more performant datastore like Elasticsearch.
Publishing Events
For quick comparison, let’s first consider event publishing on Linux with Rsyslog, the favorite syslog of many computer systems.
Running this simple oneliner will write “Hello Wold!” to syslog.
php -r "openlog('greeting', LOG_NDELAY, LOG_USER); syslog(LOG_INFO, 'Hello World!');"
Since both Rsyslog and Logstash use RELP, a TCP based protocol for reliable delivery of event messages, sending that message to Logstash requires adding only two short statements to the Rsyslog configuration file.
$ModLoad omrelp
if $source == 'PHP-5.5.37' then :omrelp:centralserv:2514
provided that Logstash is listening on centralserv, port 2514.
In this example, we are also filtering messages by source so that only those made with our PHP will be sent over.
We won’t be giving you more details here. This is not supposed to be that kind of party. Instead, let’s switch over to Windows for a while. We’ll give you step-by-step instructions on how to do the same thing from there.
But just in case you want to learn more about this Linux setup, here are two quality links.
Publishing Events on Windows
We’ll be getting instructional at this point. Find yourself a Windows box and run the oneliner from before in the Command Prompt. PHP is cross-platform. This time, it means the greeting is recorded by the Windows Event Log service and you are able to see it in the Event Viewer.
Event Viewer can be found from the Cortana Search box. Alternatively, press Windows key + R to open the Run box, type eventvwr and click OK to open the Event Viewer.
Once the Event Viewer is open, expand Windows Logs in the Console Tree view on the left, click on the Application and scroll down the displayed logs at the central part of the window if necessary to reach the log you previously made.
At this point you should be getting something like this:
Level Information matches the first argument LOG_INFO we’ve passed into the syslog() call and the Source PHP-5.5.37 matches the version of the PHP CLI we ran (update to match yours). Knowing this, you may also want to filter the Application log or create a Custom View by clicking on the corresponding actions from the Actions Pane to the right.
Note: If you are also seeing a message like this:
The description for Event ID 2 from source PHP-5.5.37 cannot be found.
then your PHP installation did not add the required information to the Windows Registry so you have to do it manually. Download /win32/syslog.reg from the PHP source repository, open it in a text editor, adjust the PHP version and the path to the dll file e.g. change key name from HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\EventLog\Application\PHP-5.3.99-dev to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\EventLog\Application\PHP-5.5.37 and EventMessageFile value from g:\\test\\srctrunkinstall\\php7ts.dll to C:\\xampp\\php\\php5ts.dll and import the key into the registry by double-clicking the file icon.
Streaming Event Messages to Logstash with Rsyslog Windows Agent
The remaining step is to stream such messages to the Logstash. We will show you how to do it with the Rsyslog Windows Agent, a Windows service that comes from the creators of Rsyslog and RELP. The agent will automatically monitor the Event Log and let you define rules and actions.
Download the agent from here, install it, then run the Configuration Client that comes with the service in order to tell the service which messages to send and where to.
Filters
We are going to send only messages generated by PHP, so, once the Configuration Client is open, go to the filters dialog by clicking on the Filters label in the tree view on the left side of the window. Click on the AND node of the central tree view, that’s the graphical representation of the boolean expression we are going to make.
Now click on Add Filter > button on the right. From the dropdown menus choose Add Filter > > Event Log Monitor V2 > Event Source. Click to focus the newly created expression and in the Details tab below change Compare Operation from contains to is equal and in the Set Property Value field type the source name of your logs. In our example, it’s PHP-5.5.37.
We’ve made a rule. Next, we are going to create an action.
Actions
Remove the default action under the Actions label in the tree view on the left by right-clicking on it, then selecting Delete Action from the dropdown menu. Then, add a new action by right-clicking on the Actions label and choosing Add Action > Send RELP from the context menu.
Now that the Send RELP action is focused, you are able to set RELP Servername and RELP Port. Also, as a start, change Message Format to %msg%. This will pass the messages to the RELP server just as they are, without modifying them.
Starting the Service & Debugging
There’s only one more thing to do: click on the blue triangle to start the service. From this point onward the messages should be flying over to Logstash and you won’t even have to start the service on the next Windows boot, it will start up automatically.
While you are confirming the setup really works, it’s good to know that the agent emits an error message on its own in case a message, your application’s message, could not be received by the RELP server. You can see the service messages as you would expect in the Event Viewer, but also by clicking on the Service Events node of the tree view on the left side of the Configuration Client window.
Conclusion
By now you’ve gained a basic understanding of what Logstash and Rsyslog are, learned that they can be used to make an event-based distributed system and acquired some hands-on experience.
And now let’s hear what you have to say. Do you develop on Windows and use Logstash in production or have interest in using it now? Great! We’d also love to hear about your use cases or questions in the comments below.
References
