Unit Testing a Data Mapper

[Brenden Vickery]

Ive been using simpletest for a little while now and am very impressed with it however I havent been able to get my head around mock objects.

I have a Data Mapper that is nearly identical to the Data Mapper in PoEAA pg. 171-175. What would be the best way to test this?

Ive read somewhere that when testing data access code some people will use a testing database, in setUp() they will create all needed records in the database, then run the tests, and on teardown delete all records from the database.

So, should I be using Mock Objects (if so is there a kind soul out there who would give me an example) to test a Data Mapper or create the db on setUp and destroy the db on tearDown or is there another better method?

Heres an example of the code I want to test, If more of the code is needed for mock objects let me know and Ill post it.

class PersonMapper extends AbstractMapper {
    // Public
    function findByLastName($name) {
        $stmt = $this->db->prepare($this->findLastNameStatement());
        $stmt->setString(1, $name);
        $rs = $stmt->executeQuery();
        return $this->loadAll($rs);
    } 
    function update($subject) {
        $updateStatement = $this->db->prepare($this->updateStatement());
        $updateStatement->setString(1, $subject->getLastName());
        $updateStatement->setString(2, $subject->getFirstName());
        $updateStatement->setInt(3, $subject->getNumberOfDependents());
        $updateStatement->setInt(4, $subject->getID());
        $updateStatement->execute();
    } 
    // Private
    function &doLoad($row) {
        $person = new Person();
        $person->setID($row['id']);
        $person->setLastName($row['lastname']);
        $person->setFirstName($row['firstname']);
        $person->setNumberOfDependents($row['number_of_dependents']);
        return $person;
    } 
    function doInsert(&$subject, &$stmt) {
        $stmt->setString(1, $subject->getLastName());
        $stmt->setString(2, $subject->getFirstName());
        $stmt->setString(3, $subject->getNumberOfDependents());
    }
    function columns() {
        return ' id, lastname, firstname, number_of_dependents ';
    } 
    function findStatement() {
        return 'SELECT ' . $this->columns() . 
               'FROM people' . 
               ' WHERE id = ?';
    } 
    function findLastNameStatement() {
        return 'SELECT ' . $this->columns() . 
               ' FROM people ' . 
               ' WHERE lastname LIKE ? ' . 
               ' ORDER BY lastname';
    } 
    function updateStatement() {
        return 'UPDATE people ' . 
               ' SET lastname = ?, firstname = ?, number_of_dependents = ? ' . 
               ' WHERE id = ?';
    } 
    function insertStatement() {
        return 'INSERT INTO people ' . 
               ' (lastname, firstname, number_of_dependents) ' . 
               ' VALUES (?, ?, ?)';
    }      
} 
class AbstractMapper {
    var $db;
    // Public
    function AbstractMapper($db) {
        $this->db = $db;
    } 
    function insert($subject) {
        $insertStatement = $this->db->prepare($this->insertStatement());
        $this->doInsert($subject, $insertStatement);
        $insertId = $insertStatement->execute();
        return $insertId;
    }
    function find($id) {
        $findStatement = $this->db->prepare($this->findStatement());
        $findStatement->setInt(1, $id);
        $rs = $findStatement->executeQuery();
        $result = $this->doLoad($rs->fetch());
        return $result;
    } 
    // Protected
    function loadAll($rs) {
        $result = array();
        while ($row = $rs->fetch()) {
            $result[] = $this->doLoad($row);
        } 
        return $result;
    } 
    // Abstract
    function findStatement() { die(); } 
    function &doLoad($row) { die(); } 
    function insertStatement() { die(); } 
    function doInsert(&$subject, &$insertStatement) { die(); }      
} 

[sike]

Ive been using simpletest for a little while now and am very impressed with it however I havent been able to get my head around mock objects.

I have a Data Mapper that is nearly identical to the Data Mapper in PoEAA pg. 171-175. What would be the best way to test this?

Ive read somewhere that when testing data access code some people will use a testing database, in setUp() they will create all needed records in the database, then run the tests, and on teardown delete all records from the database.

So, should I be using Mock Objects (if so is there a kind soul out there who would give me an example) to test a Data Mapper or create the db on setUp and destroy the db on tearDown or is there another better method?

Heres an example of the code I want to test, If more of the code is needed for mock objects let me know and Ill post it.

i would vote for mock objects because using databases in test cases will
give you more trouble than you would like . try to mock the db object and check what queries it receives. i am a bit in a hurry but if no one could help you i will post a unit test later...

Sike

[lastcraft]

Hi...

Ive read somewhere that when testing data access code some people will use a testing database, in setUp() they will create all needed records in the database, then run the tests, and on teardown delete all records from the database.

It depends .

When you are testing failure conditions, use mocks. You simply won't be able to reliably do it any other way.

If you are testing query construction at a low level, possibly use mocks. I would move this functionality into a separate Sql class in your example and test it directly though. That way you can mock your Sql class in your data mapper, otherwise you will be doing a lot of fiddly string matching tests. These will be fragile once you start refactoring or changing database schemas unless if it isn't abstracted.

If what you are doing is a fairly simple mapping, then you can use mocks. This will be much faster and the tests will be more precise. Back it up with some kind of end to end sanity test just to make sure you are not building castles in the air.

What is left are situations where the mapping is complex and so sensitive to details. Mocks can actually get in the way for this. Here is an example...

class TestOfPersonSaving ...
    function testSaveAndReloadNew() {
        $connection = &new MockConnection($this);
        $connection->expectOnce(
                'execute',
                array("insert into people (name) values ('fred')"));

        $person = &new MockPerson($this);
        $person->setReturnValue('getName', 'fred');

        $mapper = &new PersonMapper($connection);
        $mapper->save($person);

        $connection->tally();
    }
} 

A very precise test. Too precise as the slightest variation of the SQL (the order of the fields for example) will break the tests. Mocks are very effective at exposing interactions, but here they are likely too effective.

I like to think of Mocks as horizontal isolation. You tend to mock layers, such as the database connections and SQL, or the data mapping layer if higher up. With something like data mapping, quite a simple initial query, such as "save me", can produce a very specific query lower down. When testing the act of saving, you are actually not interested in this degree of specifics, only that once saved, you can get it back. You would like to write a test such as...

class TestOfPersonSaving ...
    function testSaveAndReloadNew() {
        $mapper = &new PersonMapper();
        $person = &$mapper->create();
        $person->setName('fred');
        $mapper->save($person);

        $found = &$mapper->findByName('fred');
        $this->assertIdentical($person, $found);
    }
} 

This test would be robust in the face of refactoring, but you have to have setUp()/tearDown() code to manage a test database.

I call this a vertical test as the executed code spans all the levels under test. With a horizontal test, only the current layer is tested, but the boundary is exposed in the test. With a vertical test, the test code is written purely in the language of the layer under test.

A vertical test is cleaner and more robust to refactoring, but if something breaks, likely the whole test suite will go red. Not very helpful. The way I have dealt with this in the past, is to test thin vertical slices. Here I would create a mapped test class that had only a single field and then save it. Then one with two fields. Then one with a join, and so on. We actually code generate our data access and so this is simpler than when hand coding mappers.

Vertical tests are more of a blunt instrument. I would mix and match depending on the situation. Start with the mocks whilst you are writing code as they are more informative and quicker to write, and then switch to non-mock tests once the code is mostly working and you just want to refactor.

I don't think anyone currently has a good answer to this.

yours, Marcus

[Brenden Vickery]

Thanks for the replies. Lastcraft that is exactly the response I was looking for .

Heres what I have so far:

function testInsertAndReloadNewHorizontal() {
      $preparedStatement = &new MockPreparedStatement($this);
      $preparedStatement->expectArgumentsAt(0, 'setString', array(1, 'Vickery'));
      $preparedStatement->expectArgumentsAt(1, 'setString', array(2, 'Brenden'));
      $preparedStatement->expectArguments('setInt', array(3, 5));
      
      $conn = &new MockDataConnection($this);
      $conn->setReturnReference('prepare', $preparedStatement);
      
      $person = &new Person();
      $person->setFirstName('Brenden');
      $person->setLastName('Vickery');
      $person->setNumberOfDependents(5);
      
      $mapper = &new PersonMapper($conn);
      $mapper->insert($person);            
    }
    function testInsertAndReloadNewVertical() {
      $person = &new Person();
      $person->setFirstName('Brenden');
      $person->setLastName('Vickery');
      $person->setNumberOfDependents(5);
      
      $conn = &new DataConnection('localhost', 'user', 'pass', 'dbName');
      $mapper = &new PersonMapper($conn);
      $person->setID($mapper->insert($person));
      
      $foundPerson = $mapper->find($person->getID());
      
      // Will use assertIdentical once I pull 
      // correct data types out of database
      $this->assertEqual($person, $foundPerson);
    } 

From what I got from Lastcrafts post, when testing a data mapper horizontally you make sure that the correct query is contructed and that that query is that same as expected.

The horizontal test above is the best I could come up with for my prepared statement class. Would you find this an equal test to the horizontal test you posted Lastcraft?

This is the bare bone prepared statement class:

class PreparedStatement {
    var $conn;
    var $sql;
    var $params = array();
    // Public
    function PreparedStatement($conn, $sql) {
        $this->conn = $conn;
        $this->sql = $sql;
    } 
    function &executeQuery() {
        $resource = mysql_query($this->prepareSql(), $this->conn);
        return new Result($resource);
    } 
    function execute() {
        mysql_query($this->prepareSql(), $this->conn);
        return mysql_insert_id($this->conn);
    } 
    function setString($parameterIndex, $str) {
        $this->params[$parameterIndex] = "'" . mysql_real_escape_string($str) . "'";
    } 
    function setInt($parameterIndex, $int) {
        $this->params[$parameterIndex] = (int)mysql_real_escape_string($int);
    } 
    // Private
    function prepareSql() {
        $sqlSections = explode('?', $this->sql);
        $preparedSql = $sqlSections[0];
        for ($i = 1, $max = count($sqlSections); $i < $max; $i++) {
            $preparedSql .= $this->params[$i] . $sqlSections[$i];
        } 
        return $preparedSql;
    } 
} 

[lastcraft]

Hi...

From what I got from Lastcrafts post, when testing a data mapper horizontally you make sure that the correct query is contructed and that that query is that same as expected.

Please, call me Marcus .

The horizontal/vertical is a temporary naming scheme that arose out of an e-mail correspondence with Jeff and is complete pants. Hopefully you can come up with something better.

Would you find this an equal test to the horizontal test you posted Lastcraft?

Yes. Would it be possible to make the PreparedStatement interface a bit more intuitive by removing the parameter position?

yours, Marcus

[Brenden Vickery]

The horizontal/vertical is a temporary naming scheme that arose out of an e-mail correspondence with Jeff and is complete pants. Hopefully you can come up with something better.

Well, I cant come up with something better right now but Ive come across a couple of people who talk about this.

Lasse Koskela calls it The "Sandbox" Approach and terry chay says its not unit testing its Integration Testing.

Yes. Would it be possible to make the PreparedStatement interface a bit more intuitive by removing the parameter position?

Looking through the rest of the code this would mean I have to be careful about the order I call setString() etc. and the order of the ?
s in a query. I do that already however and this is probably a good spot to refactor.

Just changes a couple functions in PreparedStatement:

function setString($str) {
        $this->params[] = "'" . mysql_real_escape_string($str) . "'";
    } 
    function setInt($int) {
        $this->params[] = (int)mysql_real_escape_string($int);
    } 
    // Private
    function prepareSql() {
        $sqlSections = explode('?', $this->sql);
        $preparedSql = $sqlSections[0];
        foreach($this->params as $index => $param) {
          $preparedSql .= $param . $sqlSections[++$index];
        }
        return $preparedSql;
    } 

Here is where I am now:

    function &getPerson() {
        $person = &new Person();
        $person->setFirstName('Brenden');
        $person->setLastName('Vickery');
        $person->setNumberOfDependents(5);
        return $person;
    } 
    function testInsert() {
        $preparedStatement = &new MockPreparedStatement($this);
        $preparedStatement->expectArgumentsAt(0, 'setString', array('Vickery'));
        $preparedStatement->expectArgumentsAt(1, 'setString', array('Brenden'));
        $preparedStatement->expectArguments('setInt', array(5));

        $conn = &new MockDataConnection($this);
        $conn->expectOnce('prepare', array('INSERT INTO people ' . 
                                           ' (lastname, firstname, number_of_dependents) ' . 
                                           ' VALUES (?, ?, ?)'));
        $conn->setReturnReference('prepare', $preparedStatement);

        $mapper = &new PersonMapper($conn);
        $mapper->insert($this->getPerson());
    }
    function testInsertAndReloadNewVertical() {
        $person = $this->getPerson();

        $conn = &new DataConnection('localhost', '', '', 'DataMapper1');
        $mapper = &new PersonMapper($conn);
        $person->setID($mapper->insert($person));
        $foundPerson = $mapper->find($person->getID()); 
        
        $this->assertEqual($person, $foundPerson);
    } 

Would it be good practice if I used the private function PersonMapper::insertStatement() in $conn->expectOnce() instead of writing out the sql? I would say yes.. anyone disagree?

[pointbeing]

I'm looking at Simpletesting a Mapper or two right now, hence finding this thread.

It strikes me that I don't actually want to test for exact SQL fragments? That seems kind of brittle, when I only care that objects are getting stored or retrieved correctly, regardless of SQL implementation, no?

And what strategies are people using in terms of setting up a test database? Do you have a clone of your devel DB? And if so, how do you keep the two structurally in sync without duplication?

Or do you just fire the tests at your devel DB, and maybe do as Brenden mentions - using setUp and tearDown to keep things in order?

Slightly vague questions I know - I'm just keen to know how others are doing this sort of thing, get some dialogue going on the subject.

Cheers,
SImon

[McGruff]

It strikes me that I don't actually want to test for exact SQL fragments? That seems kind of brittle

That crops up a lot when you're testing data access classes - time to use a real db.

I use a fixture-creation tool to setup/teardown data. This implements a database interface (execute, getError, etc) with a few extras. The main one being a teardown method which drops everything created during the test run, leaving the database exactly as it started out.

We've been discussing this recently and not everyone agrees it's a problem, but I think it's important to ensure that the fixture creation code does not use the same database connection as the class under test. Connection state (currently used db, current error, etc) could contaminate the tests.

Do you have a clone of your devel DB? And if so, how do you keep the two structurally in sync without duplication? Or do you just fire the tests at your devel DB, and maybe do as Brenden mentions - using setUp and tearDown to keep things in order?

If you run tests on the live site, you'll definitely want some kind of sandboxing. You can decorate your Database object of choice adding some code to:
(a) take a snapshot of all databases (or tables) present at the start of the test
(b) block any queries which try to manipulate these (read-only is fine)

The fixture creation code and the classes under test would use sandboxed database objects so there is no risk of altering live data. Some copyTableStructure and copyDatabaseStructure methods are handy to have in the fixture tool. Usually you get finer control if you use your own minimalist sample data in tests but you could also copy real data as well as the db structure if that's useful.

You can even build a sandbox within a single database if that's all you've got. In fact, if you're working on an open source library I think you have to test like this as a matter of course. Some users might only have a single database to work with and won't be able to run the tests if they assume create db privileges.

Finally, some custom assertions can make the tests cleaner: assertDatabases, assertNoTable, assertEqualTableRows - whatever you need for your own tests.

[kyberfabrikken]

I use a fixture-creation tool to setup/teardown data. This implements a database interface (execute, getError, etc) with a few extras.
(...)

That sounds like something, which could be helpfull to include with SimpleTest, so that everybody use the same fixturetool, rather than creating their own variations. If you have one done already, perhaps you could post it here ? I'd be curious to have a look at it.

[McGruff]

Give me a day or two to make it presentable and I'll post the code.

[pachanga]

Here's one of the test cases from a real world application we're developing:

require_once(LIMB_DIR . '/tests/cases/orm/AppPersistenceTest.class.php');
require_once(COCKTAIL_DIR . '/Cocktail.class.php');
require_once(COCKTAIL_DIR . '/Unit.class.php');
require_once(COCKTAIL_DIR . '/Ingredient.class.php');

class CocktailAndRecipeTest extends AppPersistenceTest
{
  function _defineMapperName()
  {
    return 'CocktailMapper';
  }

  function _defineCleanupTables()
  {
    return array('sys_uid',
                 'cocktail',
                 'ingredient',
                 'unit',
                 'recipe_item');
  }

  function testCocktailRecipe()
  {
    $unit = new Unit();
    $unit->setName("kg");
    $unit->setFullName("Kilogram");

    $ingr1 = new Ingredient();
    $ingr1->setName("A Frog");
    $ingr1->setUnit($unit);

    $ingr2 = new Ingredient();
    $ingr2->setName("A Fox");
    $ingr2->setUnit($unit);

    $cocktail = new Cocktail();

    $cocktail->addIngredient($ingr1, $amount1 = 0.25, $order1 = 15);
    $cocktail->addIngredient($ingr2, $amount2 = 1.15, $order2 = 10);
    $this->_checkCocktailRecipe($cocktail,
                                array(array($ingr1, $amount1, $order1),
                                      array($ingr2, $amount2, $order2)),
                                __LINE__);

    $this->uow->register($cocktail);
    $this->_commitAndReset();

    $this->_checkDbRecordsAmount(1, 2, 2, __LINE__);
    $cocktail_loaded = $this->mapper->findById($cocktail->getID());


    $this->_checkCocktailRecipe($cocktail_loaded,
                                array(array($ingr1, $amount1, $order1),
                                      array($ingr2, $amount2, $order2)),
                                __LINE__);

    $cocktail_loaded->removeIngredient($ingr2);
    $this->_commitAndReset();
    $this->_checkDbRecordsAmount(1, 1, 2, __LINE__);

    $cocktail_reloaded = $this->mapper->findById($cocktail->getID());
    $this->_checkCocktailRecipe($cocktail_reloaded,array(array($ingr1, $amount1, $order1)),__LINE__);

    $this->uow->delete($cocktail_reloaded);
    $this->uow->commit();

    $this->_checkDbRecordsAmount(0, 0, 2, __LINE__);
    $this->assertFalse($this->uow->isRegistered($cocktail_reloaded));
  }

  function _checkCocktailRecipe($cocktail, $ingredients_array, $line)
  {
    $cocktail = ProxyResolver :: resolve($cocktail);
    $recipe = $cocktail->getRecipe();

    $i = 0;
    foreach($recipe as $recipe_item)
    {
      $this->assertEqual($recipe_item->getIngredient()->getId(), $ingredients_array[$i][0]->getId(), '%s ' . $line);
      $this->assertEqual($recipe_item->getAmount(), $ingredients_array[$i][1], '%s ' . $line);
      $this->assertEqual($recipe_item->getPriority(), $ingredients_array[$i][2], '%s ' . $line);
      $i++;
    }
    $this->assertEqual($i, sizeof($ingredients_array), '%s' . $line);
  }

  function _checkDbRecordsAmount($cocktails, $recipe_items, $ingredients, $line)
  {
    $result = $this->db->select('cocktail');
    $this->assertEqual($result->getTotalRowCount(), $cocktails, '%s' . $line);

    $result = $this->db->select('recipe_item');
    $this->assertEqual($result->getTotalRowCount(), $recipe_items, '%s' . $line);

    $result = $this->db->select('ingredient');
    $this->assertEqual($result->getTotalRowCount(), $ingredients, '%s' . $line);

  }
}
?> 

We try to stick to vertical tests(using Marcus' terminology) as much as possible when developing end applications based on our own simple ORM solution. At the same time we have very detailed horizontal tests for base datamapping classes(UnitOfWork, AbstractDataMapper, etc). However sometimes we mix high level checks with low level ones, e.g we check UnitOfWork for registered objects and at the same time we check raw amount of inserted db records just to be on the safe side.

[Brenden Vickery]

Ive gained a fair bit of experience with this in the year or so since I posted that.

If youre hand-rolling your mappers I would still unit test it right down to the actual SQL being passed into your connection object. You will still need to write some integration tests so you know the right objects are being inserted, retrieved etc from your database. If you have a fully unit tested ORM then you shouldnt need to do any testing at all for persistence, you already know it works.

If you have to do itegration testing make sure you keep refactoring your tests. You will start to see obvious duplication that can be refactored into a base test class.

Always use a clean (No tables) test database for integration tests. When writing your first integration test create the tables and populate them (if population is needed) that you need for that test and remove the tables at the end of your test. When you write your next test create/remove the tables again and refactor the creation of tables into setUp and the removing of tables into tearDown.

If you need integration tests to run on different databases pass the connection object into the test when it is instantiated. An example group test from perdure that runs on different databases:

class LibraryPersistenceIntegrationTests extends GroupTest {
    function LibraryPersistenceIntegrationTests() {
        $this->GroupTest('Perdure Integration tests');
        $this->addIntegrationTest(
            new SqliteTestEnvironment());
        if (extension_loaded('pdo_sqlite')) {
            $this->addIntegrationTest(
                new PdoSqliteTestEnvironment());
        }
        if(file_exists(dirname(__FILE__) . '/config/mysql.php')) {
            $this->addIntegrationTest(
                new MysqlTestEnvironment());
            if (extension_loaded('pdo_mysql')) {
                $this->addIntegrationTest(
                    new PdoMysqlTestEnvironment());
            }
            if (extension_loaded('mysqli')) {
                $this->addIntegrationTest(
                    new MysqliTestEnvironment());
            }
        }
    }
    function addIntegrationTest($environment) {
        $this->addTestCase(
            new ConnectionIntegrationTest($environment));
        $this->addTestCase(
            new MetaDataIntegrationTest($environment));
        $this->addTestCase(
            new RecordSetIntegrationTest($environment));
        $this->addTestCase(
            new SQLStatementIntegrationTest($environment));
        $this->addTestCase(
            new PersistTest($environment));
        $this->addTestCase(
            new MapperTest($environment));
        $this->addTestCase(
            new OQLQueryTest($environment));
        $this->addTestCase(
            new OQLOperatorsTest($environment));
    }
} 

[Brenden Vickery]

If you have a fully unit tested ORM then you shouldnt need to do any testing at all for persistence, you already know it works.

Im rethinking this statement. You do need to do some testing for a fully unit tested ORM. Im less experienced here since we (PHP people) dont have a full featured/tested ORM to use at this point. Youll need to test your OQL/Query object/Active Record or whatever youre using. You may also need to test your basic insert/update etc functions to make sure there arent errors in your XML/Annotation configurations.

[kyberfabrikken]

You may also need to test your basic insert/update etc functions to make sure there arent errors in your XML/Annotation configurations.

The XML should be possible to test with DTD/schema.

[McGruff]

That sounds like something, which could be helpfull to include with SimpleTest, so that everybody use the same fixturetool, rather than creating their own variations. If you have one done already, perhaps you could post it here ? I'd be curious to have a look at it.

Zip file attached. See notes in read_me file.

First, a warning: the classes TransientTables and TransientDatabases track "transient" tables/dbs created purely for testing and will drop these with calls to the tearDown() method (their own tearDown() method that is, not the simple test tearDown() ). They are supposed to detect tables/dbs which pre-exist the test run and leave these untouched. However, if something goes wrong, you could potentially clean out everything on the db host.

I've never encountered this problem when using the classes, and not even while writing them, but it's worth pointing out. This is a beta version which has developed far enough to be usable for my own needs. Use at your own risk.

If you decide there's something in there worth working on, be very careful when you edit the classes...

[pointbeing]

Amazing how productive it can be to revive a year-old thread, plenty there to work through, thanks guys

I definitely like the idea of a tool to automate some of the nuts and bolts.

For example, what's especially fun is that if a test generates a Fatal error (not entirely rare, since we write tests for methods before we write the methods themselves, right testers?), then tearDown() won't get called, potentially leaving the test DB all out of whack.

[McGruff]

That will happen with the code I posted. You have to manually delete testing databases after a fatal error. As you can see from the tests, the aim was simply to take a snapshot of anything on the host which pre-exists the test run and protect it from any changes - any sample data which was not dropped in a previous test run will be on the wrong side of the sandbox next time around. I'm not sure whether to look at that as a problem or a feature.

[Brenden Vickery]

For example, what's especially fun is that if a test generates a Fatal error (not entirely rare, since we write tests for methods before we write the methods themselves, right testers?), then tearDown() won't get called, potentially leaving the test DB all out of whack.

Yeah, that can be a problem. Since you usually start out with a clean existing database (no tables), I would iterate over all the existing tables and drop them before and after each test. You could get away with deleting only the specific tables created for each test in tearDown but in setUp its best to delete all exisiting tables in my opinion. I use some variation of the code below usually.

class PolymorphicIntegrationTest extends UnitTestCase {
    function setUp() {
        $this->dropDatabaseTables();
        ...
    }
    function tearDown() {
        $this->dropDatabaseTables();
        ...
    }
    function dropDatabaseTables() {
        $dialect = $this->connection->getDialect();
        $metaData = $this->connection->createMetaData();
        foreach($metaData->getTables() as $table) {
            $sql = $dialect->getDropTableSql($table->getName());
            $this->connection->update($sql);
        }
    }
} 

[kyberfabrikken]

For example, what's especially fun is that if a test generates a Fatal error (not entirely rare, since we write tests for methods before we write the methods themselves, right testers?), then tearDown() won't get called, potentially leaving the test DB all out of whack.

This is actually a more generic problem with simpletest, although it's probably most notable with databases. The same applies to files however, and other resources. Also, if issuing remote tests, the result won't be wellformed xml, as expected.

I'm not sure how much though Marcus have put into this issue, but after thinking about it for some time, I realized a way around it. Basically, if the each test is run in a separate process, a fatal error can be cought because it will only terminate the executing thread.

For proof of concept, I stuck some code together. I'm not entirely sure how to unittest a unittester, so there is no automated test for it. (I suppose I could just write a webtester for it or something come to think of it).

You'll have to adjust the path to php-bin (php.exe) to reflect your system. Apart from that, it should run out of the box.

The whole package could probably be somewhat better integrated with RemoteTestCase and simpletest in general.

[lastcraft]

Hi...

I'm not sure how much though Marcus have put into this issue,

Lot's!

A lot of refactoring has gone into SimpleTest to make this feature at least possible, but I was waiting for PHP 4.3 minimum support (SimpleTest 1.1+) for the streams interfaces. Having a group test that runs in a separate process is very much on the roadmap, not least because it gets around the PHP memory limit. I was going to combine an upgraded (using streams) shell class with the XML feed for this, but just haven't got around to doing it yet .

Catching the failed tearDown() from a separate process is trickier, and I hadn't got that far in my thinking. A parse time fatal won't run the setup, so the act of running setUp() is an event that has to be recorded I guess. One solution I am leaning toward is to have setUp() and tearDown() capability in the group test classes. This can act as a safety net for really serious screw ups.

So much to do, so little time. Always open to offers of help though .

yours, Marcus

[kyberfabrikken]

One solution I am leaning toward is to have setUp() and tearDown() capability in the group test classes. This can act as a safety net for really serious screw ups.

Yes, this was the route I was aiming at with the previous post. Something along the lines of :

class DbFixtureTestCase extends IsolatedTestCase
{
    function setUp() {
        // backup the db
    }

    function tearDown() {
        // restore the db
    }
} 

Catching the failed tearDown() from a separate process is trickier, and I hadn't got that far in my thinking.

So you're suggesting to track the setups/teardowns of each of the individual testcases, rather than just having one such for the entire group ?
I suppose this is a better solution overall. Not just for the added granularity, but also for keeping responsibilities focused around the testcase.

So much to do, so little time. Always open to offers of help though

You're most welcome to use my post, although it's rather sketchy it does work. I kind of got a bit of track a time or two, simply because I didn't fully understood all of the internals of simpletest. Theese things would have to be cleaned up if it should be of any serious use. Perhaps I'll have a more in-depth look at it when I get some time between the boring stuff I do for a living.

[OfficeOfTheLaw]

I've recently been doing what marcus mentioned earlier... when you save an object, all you really care about is if you can get the same object back from a finder or loader method. For awhile I'd commonly serialize both and compare:

function testSave()
  {
   $foo = &new DealerEntity();
   $foo->setName('Fred');
   $foo->setBudget(20.00);
   FooMapper::save($foo);

   $bar = &FooMapper::loadById($foo->getId());
   $this->assertTrue(serialize($bar) == serialize($foo));
  } 

This kind of sucks though if you have composite objects that are lazy loaded, or have to deal with php's typecasting everything as strings (that 20.00 above will be a float in $foo, a string in $bar). So I normally just save, then load and test each getter method for equality.

Recently I read http://www.theserverside.com/news/th...hread_id=36502 and was curious a bit... if interacting with a database is considered bad form in a unit test, how are we supposed to test that aspect?

[McGruff]

If a class has to interact with some kind of service, you have to create a realistic version of the service in the fixture. With classes which interact with a db, the only realistic way I can think of to do that is to setUp/tearDown a real database in the tests.

Some classes don't depend on any particular db structure - eg a php implementation of a boolean search script - so anything will do. For others, such as Finder classes, I think you should use an exact copy of the target db structure to ensure that queries issued by the classes do actually make sense in the context of the real database. There could be errors in dynamically generated sql, the query might refer to a field which doesn't exist, or the sql might not be valid in any circumstances at all.

The bottom line is that you don't know that the queries issued by the class will do what you think they should until you run them past a db.

[pointbeing]

Seconded, I think - if tests are about confidence, then you have to run them against a real db at some point. Haven't yet read the article OOtL mentions, though.

Glad to know I'm not the only one to have come up against certain issues anyway. I can live with it all anyway - a bigger problem is getting some of my team to write tests in the first place...

[sweatje]

a bigger problem is getting some of my team to write tests in the first place...

Not migrating code without them is a big incentive...