Key Takeaways
- MongoDB is a highly efficient tool for logging due to its fast data writing capabilities and easy-to-use, flexible query interface. This makes it a fitting choice for those looking to migrate their applications to a document database.
- Analog, a micrologging tool found on GitHub, can be easily integrated with MongoDB to create a robust error logging system. Analog’s extensibility and simplicity make it a preferred choice for this purpose.
- MongoDB allows for various logging levels, ranging from urgent (0) to debugging messages (7). This flexibility allows developers to categorize their logs based on the severity of the error, making the logs easier to filter and analyze.
- MongoDB’s powerful query interface allows developers to filter and summarize logs efficiently. This is particularly useful when dealing with large datasets, making it easier to spot trends and troubleshoot issues.
Setting Up The Logger
Before writing a MongoDB logging class for PHP, we took a quick look to see what else was out there already and found a nice micrologging tool on GitHub called Analog. In the interest of not reinventing the wheel, we’ll use this in our examples. What we really liked about Analog is the simplicity of its code and the number of things you can log to! It’s designed to be extensible, so you should be able to easily build on in to anything specific you may need for your own project. The logger is fairly self-contained, so all you’ll need to do to make its functionality available is to include its main fileAnalog.php
. This takes care of the autoloading and namespace registration needed for it to find its dependencies. Since it uses spl_autoload_register()
, it will happily co-exist alongside any other autoloading arrangements you already have in place.
To start using the logger, you’ll need to intialize the logging handler you want to use and then pass it to the main logging class. There are some examples included with the project which makes it easy to see what you need for a specific platform. For MongoDB, we have the following:
<?php
Analog::handler(AnalogHandlerMongo::init(
"localhost:27017",
"testing",
"log"));
All we have to do here is to point Analog at our MongoDB installation (ours is on the same machine as the web server and uses the default port), tell it to use the testing
database, and write to the log
collection. With this included somewhere at the top of our script, probably along with various other bootstrapping tasks, we’re ready to go.
Logging Errors
At this point we can use the logging functionality anywhere we want it in our application. To log an error, simply do:<?php
Analog::log("Oh noes! Something went wrong!");
To see what’s in the database, open the mongo shell.
lorna@taygete:~$ mongo type "help" for help > use testing > db.log.find(); { "_id" : ObjectId("4f268e9dd8562fc817000000"), "machine" : "localhost", "date" : "2012-02-29 11:11:16", "level" : 3, "message" : "Oh noes! Something went wrong!" }As you can see this gives us the error message, the severity, the date and time that the error was created, and the machine from which it came. The machine identifier comes from
$_SERVER["SERVER_ADDR"]
if set, otherwise “localhost” is used.
Logging Levels
The Analog library comes with a great set of constants that you can use to set the level of each error. Here’s a snippet from the class to showing them:<?php
...
class Analog {
/**
* List of severity levels.
*/
const URGENT = 0; // It's an emergency
const ALERT = 1; // Immediate action required
const CRITICAL = 2; // Critical conditions
const ERROR = 3; // An error occurred
const WARNING = 4; // Something unexpected happening
const NOTICE = 5; // Something worth noting
const INFO = 6; // Information, not an error
const DEBUG = 7; // Debugging messages
...
The default is level 3 to denote an error. To log an error of any other level, pass the desired level as a second parameter to the log()
method:
<?php
Analog::log("FYI, a log entry", Analog::INFO);
Looking in the database now, we can how our log messages collection will grow.
> db.log.find(); { "_id" : ObjectId("4f268e9dd8562fc817000000"), "machine" : "localhost", "date" : "2012-02-29 11:11:16", "level" : 3, "message" : "Oh noes! Something went wrong!" } { "_id" : ObjectId("4f268e9dd8562fc817000001"), "machine" : "localhost", "date" : "2012-02-29 12:35:41", "level" : 6, "message" : "FYI, a log entry" }Although (as with all logs) in a real application we’ll be building up a large set of data, using a database means we can easily generate summary information or filter the data to find only the important entries.
Filtering And Summarizing MongoDB Logs
Using database storage means the ability to search results, and MongoDB is designed to be easy for developers to use even with large datasets. The days of grep’ing enormous flat-file logs are over! We can very easily filter the data to show only what we’re interested in.> db.log.find({level: 3}); { "_id" : ObjectId("4f268e9dd8562fc817000000"), "machine" : "localhost", "date" : "2012-02-29 11:11:16", "level" : 3, "message" : "Oh noes! Something went wrong!" }There are some higher-level entries also in the database since we have many different levels of logging. To show everything of error severity and above (a lower error level constant), we can query with the operator
$lte
:
> db.log.find({level: {$lte: 3}}); { "_id" : ObjectId("4f268e9dd8562fc817000000"), "machine" : "localhost", "date" : "2012-02-29 11:11:16", "level" : 3, "message" : "Oh noes! Something went wrong!" } { "_id" : ObjectId("4f26aaafd8562fcb27000009"), "machine" : "localhost", "date" : "2012-02-29 13:01:04", "level" : 0, "message" : "To the lifeboats!" }We can also look for date ranges, for example, using a
$gt
comparison to pull the most recent few log entries from my database:
> db.log.find({date: {$gt: "2012-02-29 14:35:30"}}); { "_id" : ObjectId("4f26aaafd8562fcb2700000a"), "machine" : "localhost", "date" : "2012-02-29 14:35:31", "level" : 4, "message" : "Empty variable $a on line 127" } { "_id" : ObjectId("4f26aaafd8562fcb2700000b"), "machine" : "localhost", "date" : "2012-02-29 14:35:35", "level" : 4, "message" : "Empty variable $a on line 93" } { "_id" : ObjectId("4f26aaafd8562fcb2700000c"), "machine" : "localhost", "date" : "2012-02-29 14:35:40", "level" : 4, "message" : "Empty variable $a on line 277" } { "_id" : ObjectId("4f26aaafd8562fcb2700000d"), "machine" : "localhost", "date" : "2012-02-29 14:35:45", "level" : 6, "message" : "FYI, it seems to be snowing" }If you commonly query data on a particular field, you can speed up your queries by adding an index. For example, if you frequently query on
level
and date you can create a compound index:
> db.log.ensureIndex({ date : -1, level : 1 } );The above line will create a single index if it doesn’t already exist. There’s a couple things worth noting here, however. First, we placed
date
first as it will have the largest variation and therefore the index will do the most good. We also created date
as a reverse index as we commonly want to query for the most recent entries. Secondly, we added level
as part of the index. This compound index will make any query on date and any query on date
and level
more efficient. It will not be able to be used for queries on just level
and not date
.
Sometimes you’ll want to look for overall trends in your logs, so you’ll group how many of a particular error happens. In this example, we’ve grouped the error set by the error level to show how many there are of each:
> db.log.group({key: {level: true}, initial: {count: 0}, reduce: function (obj, prev){prev.count++}}); [ { "level" : 3, "count" : 1 }, { "level" : 6, "count" : 4 }, { "level" : 4, "count" : 8 }, { "level" : 0, "count" : 1 } ]You can use the
group()
function to count errors per day, or from a particular machine, as you so choose. Do take care though as this approach is only useful on small data sets. If you have over 10,000 results then you’ll want to use map/reduce to generate the results.
Summary
It makes sense to start small when looking at adding MongoDB to an existing application, and logging is an ideal candidate. Different types of errors can include different types of information and you can also save the current object or any other information to MongoDB since it has a flexible schema. Any new technology can be a bit of a learning curve but hopefully the command line examples help you to get quite close to what you are working on. Implementing just one piece of functionality in something new can be a great way to get your feet wet – hope you enjoy MongoDB as much as we do! Image via mama-art / ShutterstockFrequently Asked Questions (FAQs) about Error Logging with MongoDB and Analog
How can I set up MongoDB for error logging?
Setting up MongoDB for error logging involves a few steps. First, you need to install MongoDB and Analog on your system. Once installed, you can configure MongoDB to log errors by modifying the MongoDB configuration file. This file is usually located in the /etc/mongodb.conf directory. In the configuration file, you can specify the log file path, log format, and log level. After setting up the configuration, you can start MongoDB with the –logpath option followed by the path to your log file.
What are the different log levels in MongoDB?
MongoDB supports several log levels that determine the type of information recorded in the log files. The log levels include:
- 1. Error (0): Only error messages are logged.
- 1. Warning (1): Includes Error level and additional warning messages.
- 1. Informational (2): Includes both Error and Warning levels, plus informational messages.
- 1. Debug (3-5): Includes all previous levels, plus debug messages. The debug level has three tiers, with each subsequent tier logging more detailed information.
How can I use Analog with MongoDB for error logging?
Analog is a powerful tool that can be used with MongoDB for error logging. After setting up MongoDB for logging, you can use Analog to analyze the log files. Analog can be configured to read the MongoDB log files and provide detailed reports about the errors. This can help you identify patterns, troubleshoot issues, and improve the performance of your MongoDB database.
How can I view the MongoDB log files?
MongoDB log files can be viewed using any text editor. The location of the log files is specified in the MongoDB configuration file. By default, MongoDB writes logs to a file in the /var/log/mongodb directory. You can change this location by modifying the logpath option in the MongoDB configuration file.
Can I use MongoDB for logging application data?
Yes, MongoDB can be used for logging application data. MongoDB’s flexible schema and high write throughput make it an excellent choice for logging. You can create a separate collection for logs and write log data directly to MongoDB. This can be useful for tracking user activity, debugging, and monitoring application performance.
How can I rotate MongoDB log files?
MongoDB supports log rotation, which can be useful for managing log files. You can rotate logs by sending a logRotate command to the admin database. This will close the current log file and open a new one. You can also configure MongoDB to automatically rotate logs at a certain size or after a certain period of time.
How can I filter MongoDB log messages?
MongoDB log messages can be filtered using the systemLog.component verbosity settings in the MongoDB configuration file. Each component has a verbosity level that determines the amount of information logged. By adjusting these settings, you can control the type of information that is logged for each component.
Can I use MongoDB Atlas for logging?
Yes, MongoDB Atlas, the cloud version of MongoDB, also supports logging. MongoDB Atlas logs can be viewed in the Atlas UI or downloaded for offline analysis. Atlas logs include information about database operations, performance, and errors.
How can I troubleshoot issues using MongoDB logs?
MongoDB logs can be a valuable resource for troubleshooting issues. The logs contain detailed information about database operations, errors, and performance. By analyzing the logs, you can identify patterns, spot anomalies, and diagnose issues. Tools like Analog can help you analyze the logs and generate reports.
What is the performance impact of logging in MongoDB?
Logging can have a performance impact on MongoDB, especially at higher log levels. Writing to the log file is a synchronous operation, which means it can block other operations. However, the impact is generally minimal at lower log levels. You can mitigate the performance impact by rotating logs regularly and monitoring the size of your log files.
Lorna Jane Mitchell is an independent PHP consultant, developer, and trainer based in Leeds, England. Code is her passion, and she loves to share her ideas and experiences with technology with others, so much so that she co-authored PHP Master: Write Cutting Edge Code published by SitePoint. Lorna writes regularly for her own site lornajane.net about all kinds of topics, mostly technical ones. When she's not writing either code or words, you can usually find her cooking or eating; Lorna love food as much as she loves code! Author pic magicmonkey