Error Condition Testing with PHPUnit

Share this article

Let’s say you’re maintaining code that uses PHP’s native trigger_error()
function to log error information. Let’s also say that you’re in the process of using PHPUnit to write unit tests for that code. If you refer to the PHPUnit manual, there’s a section that deals with testing for error condition. It describes how PHPUnit implements its own error handler that converts errors, warnings, and notices into exceptions and that catching those exceptions is how you should handle testing for these types of errors. However, depending on what your code looks like, it’s possible that you’ll run into a problem with PHPUnit’s approach to this. This article will detail what this problem is, how it impacts your ability to test your code, and how to go about solving it.

What’s the Problem?

Errors and exceptions behave in fundamentally different ways. Of particular relevance to this article is the fact that code execution can continue at the point immediately after trigger_error() if the error level constant passed to it is not indicative of a fatal error. When an exception is thrown, execution will continue at the beginning of a catch block found to correspond to the class of that exception, which may or may not be immediately after the point at which the exception is thrown. Let’s look at some examples of these behaviors. First, errors.
<?php
error_reporting(E_ALL | E_STRICT);
echo "Before warningn";
trigger_error("Danger Will Robinson!", E_USER_WARNING);
echo "After warningn";
You’ll get the following output if you run the above code:
Before warning
PHP Warning:  Danger Will Robinson! in /home/matt/error_handler.php on line 4
After warning
From this we see that the echo statement after the trigger_error() call is executed. Now, exceptions.
<?php
try {
    echo "Before exceptionn";
    throw new Exception("Danger Will Robinson!");
    echo "After exceptionn";
}
catch (Exception $e) {
    echo "In catch blockn";
}
And the output:
Before exception
In catch block
In contrast to the example case for errors, the code after the exception was thrown is not executed. Because PHPUnit converts errors to exceptions, errors behave the same way in unit tests as exceptions do. Any code that follows an error being triggered will not executed while it is being tested. Here’s another example:
<?php
function foo($param) {
    if (is_string($param)) {
        trigger_error(__FUNCTION__ . " no longer supports strings, pass an array", E_USER_NOTICE);
    }
    // do useful stuff with $param
    ...
}
With error-to-exception conversion, there’s no way to test if useful stuff is done with $param because that code will never be executed when the error is converted into an exception.

Side Effects of PHPUnit’s Behavior

This error-to-exception conversion causes differences from how the code will behave in development and testing than how it will behave in production. Here’s an example:
<?php
function error_handler($errno, $errstr) {
    throw new Exception($errstr);
}
set_error_handler("error_handler");

try {
    trigger_error("Danger Will Robinson!", E_USER_WARNING);
}
catch (Exception $e) {
    var_dump(error_get_last());
}
restore_error_handler();
trigger_error("Danger Will Robinson!", E_USER_WARNING);
var_dump(error_get_last());
Here’s its output:
NULL
PHP Warning:  Danger Will Robinson! in /home/matt/exception_converter.php on line 16
array(4) {
  ["type"]=>
  int(512)
  ["message"]=>
  string(21) "Danger Will Robinson!"
  ["file"]=>
  string(59) "/home/matt/exception_converter.php"
  ["line"]=>
  int(14)
}
The first var_dump() call, during which the custom error handler that converts errors to exceptions is in effect, outputs NULL. The second var_dump() call, during which PHP’s default error handler is in effect, outputs information about the error that was triggered. Note that it’s not because a custom error handler is used that the first var_dump() call outputs NULL, but because that error handler throws an exception. If the error handler shown in this example did not do that, the first var_dump() call would have the same output as the second.

The Solution

We need a solution that allows for the execution of code being tested to continue while still allowing us to check that an error condition was raised. As above examples showed, allowing code execution to continue can be done using a custom error handler that doesn’t convert errors to exceptions. What this error handler should do instead is capture error information for later analysis with assertions. Here’s what this might look this:
<?php
class MyTest extends PHPUnit_Framework_TestCase
{
    private $errors;

    protected function setUp() {
        $this->errors = array();
        set_error_handler(array($this, "errorHandler"));
    }

    public function errorHandler($errno, $errstr, $errfile, $errline, $errcontext) {
        $this->errors[] = compact("errno", "errstr", "errfile",
            "errline", "errcontext");
    }

    public function assertError($errstr, $errno) {
        foreach ($this->errors as $error) {
            if ($error["errstr"] === $errstr
                && $error["errno"] === $errno) {
                return;
            }
        }
        $this->fail("Error with level " . $errno .
            " and message '" . $errstr . "' not found in ", 
            var_export($this->errors, TRUE));
    }

    public function testDoStuff() {
        // execute code that triggers a warning
        $this->assertError("Message for the expected error",
            E_USER_WARNING);
    }
}
setUp()
, which is run before each test method, handles setting up the error handler which is just another method in the same class that stores information about each error in an array. Other method like assertError() are then used by test methods like testDoStuff() to perform assertions against that error information and output relevant debugging information, like what errors were triggered compared to what errors were expected. Other useful types of assertions include logical inversions (i.e. asserting that a specific error was not triggered), checking for errors with messages that match regular expressions, or checking the number of errors triggered.

Conclusion

In instances where you don’t care about testing that the logic following a triggered error is still executed, PHPUnit’s default behavior is perfectly suitable for your needs. However, it’s important that you be aware of the implications of that behavior. In cases where you do care about the execution of such logic, it’s equally important that you know how to supplement PHPUnit’s functionality to facilitate accurate testing of your code is conditions as close to those of your production environment as is feasible. Image via Fotolia

Frequently Asked Questions (FAQs) about Testing Error Conditions with PHPUnit

Why doesn’t PHPUnit show any errors in the console?

PHPUnit is designed to handle errors and exceptions in a way that allows for effective testing. If you’re not seeing any errors in the console, it’s likely that PHPUnit is catching them and treating them as failed tests. To see the details of these errors, you can use the –debug option when running your tests. This will provide more detailed output, including any errors or exceptions that were caught during testing.

How can I assert that an exception was thrown in PHPUnit?

PHPUnit provides a set of assertions specifically for dealing with exceptions. The most commonly used is expectException(), which you can use to specify the type of exception you expect to be thrown. If the specified exception is thrown during the test, the test will pass. If not, the test will fail. This allows you to write tests that specifically check for the correct handling of error conditions.

How does error reporting work in PHP?

PHP’s error reporting feature allows you to control which errors are reported and how they are handled. By default, all errors are reported and displayed. However, you can change these settings using the error_reporting() function and the display_errors ini directive. This allows you to hide certain types of errors, or to log errors instead of displaying them.

How can I test exceptions in PHPUnit?

PHPUnit provides several methods for testing exceptions. The simplest is to use the expectException() method, which allows you to specify the type of exception you expect to be thrown. If the specified exception is thrown during the test, the test will pass. If not, the test will fail. This allows you to write tests that specifically check for the correct handling of error conditions.

How do I write tests for PHPUnit?

Writing tests for PHPUnit involves creating a new test case class that extends the PHPUnit\Framework\TestCase class. Each test is a public method in this class that starts with the word “test”. Inside each test method, you can use PHPUnit’s assertions to check that your code behaves as expected. For example, you might use the assertEquals() method to check that a function returns the expected result.

How can I handle errors in PHPUnit?

PHPUnit provides a set of assertions specifically for dealing with errors. The most commonly used is expectError(), which you can use to specify the type of error you expect to be triggered. If the specified error is triggered during the test, the test will pass. If not, the test will fail. This allows you to write tests that specifically check for the correct handling of error conditions.

How can I debug tests in PHPUnit?

PHPUnit provides several options for debugging tests. The –debug option provides more detailed output, including any errors or exceptions that were caught during testing. The –stop-on-error, –stop-on-failure, and –stop-on-risky options can be used to stop the test run as soon as a certain type of problem is encountered. This can make it easier to identify and fix problems.

How can I test error conditions in PHPUnit?

PHPUnit provides several methods for testing error conditions. The expectError() method allows you to specify the type of error you expect to be triggered. The expectWarning() method allows you to specify the type of warning you expect to be triggered. If the specified error or warning is triggered during the test, the test will pass. If not, the test will fail.

How can I handle warnings in PHPUnit?

PHPUnit provides a set of assertions specifically for dealing with warnings. The most commonly used is expectWarning(), which you can use to specify the type of warning you expect to be triggered. If the specified warning is triggered during the test, the test will pass. If not, the test will fail. This allows you to write tests that specifically check for the correct handling of warning conditions.

How can I use data providers in PHPUnit?

Data providers are a powerful feature of PHPUnit that allow you to run a test multiple times with different sets of data. To use a data provider, you create a method that returns an array of arrays. Each inner array is a set of parameters for the test. You then annotate your test method with @dataProvider followed by the name of your data provider method. PHPUnit will then run the test once for each set of parameters, passing the parameters to the test method.

Matt TurlandMatt Turland
View Author

Matthew Turland has been working with PHP since 2002. He has been both an author and technical editor for php|architect Magazine, spoken at multiple conferences including Confoo and php|tek, served as an instructor for php|architect training courses, and contributed to Zend Framework. He holds the PHP 5 and Zend Framework ZCE certifications and is the author of "php|architect's Guide to Web Scraping with PHP." He currently works as a Senior Engineer for Synacor. In his spare time, Matt likes to bend PHP to his will to scrape web pages and run IRC bots.

Advanced
Share this article
Read Next
Get the freshest news and resources for developers, designers and digital creators in your inbox each week
Loading form