Simple Test Remote Testing

Continue to be blown away by just how good Simple Test is.

For reasons explained by Jeff here, we’ve been looking for ways to run WACT’s test suite in a way that won’t run into PHP’s memory limit and the obvious solution is execute the tests in smaller groups over the network, meaning Apache will create a new child for each group, each child having a fresh chunk of memory to gobble up.

At first glance looks like a big mission, first to generate the test results in some form thats easy to parse (i.e. XML) then to have some client capable to acting as a “browser”, to run and parse the test output.
Turns out it’s not so hard after all; Simple Test is ahead of the game, packing a “reporter” that delivers test results in an XML format and a RemoteTestCase class, primed to connect to a URL and parse output from the XMLReporter.

Here’s a quick example (I assume you’ve read the excellent documentation);

/**
* The class I've written that I want to test
*/
class Example {
    var $message = NULL;

    function setMessage($message) {
        $this->message = $mssage; // Whoops
    }

    function getMessage() {
        return $this->message;
    }
}

if ( !defined ('SIMPLE_TEST') ) {
    define ( 'SIMPLE_TEST','path/to/simpletest/' );
}

// Include the main unit tester
require_once SIMPLE_TEST . 'unit_tester.php';

// Include the XMLReporter
require_once SIMPLE_TEST . 'xml.php';

/**
* The test class
*/
class TestOfExample extends UnitTestCase {

    var $Example;

    function TestOfExample() {
        parent::UnitTestCase('Test Of Example');
    }

    function setUp () {
        $this->Example = & new Example();
    }

    function tearDown() {
        unset($this->Example);
    }

    function testEmptyMessage() {
        $this->assertNull($this->Example->getMessage());
    }

    function testMessage() {

        $this->Example->setMessage('Hello World!');
        $this->assertEqual($this->Example->getMessage(),'Hello World!');

    }
}

// Create an instance of the test class
$T = &new TestOfExample();

// Run it using the XMLReporter
$T->run(new XMLReporter());
?></code><p>The test output from running this looks like;</p>

<code>
< ?xml version="1.0"?>
<run>
  <case>
    <name>Test Of Example</name>
    <test>
      <name>testemptymessage</name>
      <pass>[NULL] should be null at line 38</pass>
    </test>

    <test>
      <name>testmessage</name>
      <fail>
        Equal expectation fails as [NULL]
        does not match [String: Hello World!] at line 44
      </fail>
      <exception>
        Unexpected PHP error [Undefined variable:  mssage]
        severity [E_NOTICE] in [/home/hfuecks/public_html/test.php]
        line [6]
      </exception>
    </test>
  </case>
</run>

That’s already a big step forward in making your test results available to other programs and environments but Simple Test doesn’t stop there. There’s also a RemoteTestCase class that deals with converting output from the XMLReporter back into a normal HTML report;

if ( !defined ('SIMPLE_TEST') ) {
    define ( 'SIMPLE_TEST','/path/to/simpletest/' );
}

// Include the RemoteTestCase
require_once SIMPLE_TEST . 'remote.php';

// Include the normal HTML reporter
require_once SIMPLE_TEST . 'reporter.php';

$T = &new RemoteTestCase('http://localhost/~hfuecks/test.php');
$T->run(new HTMLReporter());

If you’re maintaining sites for your customers, this could be very useful. For example you might test PHP’s configuration for the settings your application requires, and catch problems caused by your client’s hosting provider (who just upgraded PHP) as they happen, using a test case like;

if ( !defined ('SIMPLE_TEST') ) {
    define ( 'SIMPLE_TEST','/path/to/simpletest/' );
}

// Include the main unit tester
require_once SIMPLE_TEST . 'unit_tester.php';

// Include the XMLReporter
require_once SIMPLE_TEST . 'xml.php';

/**
* The test class
*/
class TestOfPHPConfiguration extends UnitTestCase {

    var $Example;

    function TestOfPHPConfiguration() {
        parent::UnitTestCase('Test Of PHP Configuration');
    }

    // Test magic_quotes_gpc is off
    function testMagicQuotesGPC() {
        $this->assertEqual(ini_get('magic_quotes_gpc'),0);
    }

}

// Create an instance of the test class
$T = &new TestOfPHPConfiguration();

// Run it using the XMLReporter
$T->run(new XMLReporter());

With a little cron automation, (and perhaps your own EMailReporter) you could have your tests notify you when there’s a problem. Nice way to prevent hair loss.