Ideas for lightweight ORM implementation

[allspiritseve]

Being able to map a handwritten query into objects

I've been playing around with this as well. Mostly it involves using the load() method of related mappers so that, for example, you could load a Post and it's related User in one query, but delegate the mapping into a User object to the UserMapper. I think that's a fine solution (the alternative is mapping both objects in the same class, which would lead to duplication of mappings) for objects that are directly associated with the main object, but I don't know how things would work if you're querying for data for objects several relations down the line.

Also, oddz is right in the use of aliases. Our current mapper code allows use of aliases for selected columns on joined data. I wonder if it might be useful to specify a prefix for all columns from a joined table? That might make writing mappings that span several tables a little easier.

[Ren]

I've been playing around with this as well. Mostly it involves using the load() method of related mappers so that, for example, you could load a Post and it's related User in one query, but delegate the mapping into a User object to the UserMapper. I think that's a fine solution (the alternative is mapping both objects in the same class, which would lead to duplication of mappings) for objects that are directly associated with the main object, but I don't know how things would work if you're querying for data for objects several relations down the line.

Also, oddz is right in the use of aliases. Our current mapper code allows use of aliases for selected columns on joined data. I wonder if it might be useful to specify a prefix for all columns from a joined table? That might make writing mappings that span several tables a little easier.

If you know the order of the tables in the select, and the set of column names for each table, then its possible without needing to know aliases, I think.

<?php

    $pdo = new PDO('sqlite::memory:', '', '');
    $pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);


    $pdo->exec('CREATE TABLE users(id INTEGER PRIMARY KEY, name)');
    $pdo->exec('CREATE TABLE messages(id INTEGER PRIMARY KEY,
                    sender INTEGER REFERENCES users(id),
                    receiver INTEGER REFERENCES users(id),
                    text)');

    $pdo->exec('INSERT INTO users VALUES(1, \'ren\')');
    $pdo->exec('INSERT INTO users VALUES(2, \'bob\')');


    $pdo->exec('INSERT INTO messages VALUES(1, 1, 2, \'Hello World!\')');


$tables = array(
    'users' => array('id', 'name'),
    'messages' => array('id', 'sender', 'receiver', 'text')
);

function extractFromRow(&$row, $tableColumns)
{
    $r = array();
    foreach($tableColumns as $column)
        if (is_array($row[$column]))
            $r[$column] = array_shift($row[$column]);
        else
        {
            $r[$column] = $row[$column];
            unset($row[$column]);
        }
    return $r;
}

    $rs = $pdo->query('SELECT * FROM messages m
                        INNER JOIN users s ON s.id = m.sender
                        INNER JOIN users r ON r.id = m.receiver', PDO::FETCH_NAMED);

    foreach($rs as $row)
    {
        $message = extractFromRow($row, $tables['messages']);
        $sender = extractFromRow($row, $tables['users']);
        $receiver = extractFromRow($row, $tables['users']);

        var_dump($message, $sender, $receiver);
    }

[allspiritseve]

If you know the order of the tables in the select, and the set of column names for each table, then its possible without needing to know aliases, I think.

I would much prefer aliases though. Your method hinges on each column only being used once. The way we currently have mappings set up, each one gets the whole result array and the object instance so they are not limited at all. One property could map two two or more columns, and one column could map to two or more properties (if needed).

[Ren]

I would much prefer aliases though. Your method hinges on each column only being used once. The way we currently have mappings set up, each one gets the whole result array and the object instance so they are not limited at all. One property could map two two or more columns, and one column could map to two or more properties (if needed).

I understand the latter points, I dont understand why you'd need to use a column more than once.

[allspiritseve]

I understand the latter points, I dont understand why you'd need to use a column more than once.

Well, you don't always have the luxury of complete control over your DB. There might be denormalized fields like John_Smith in which the first name maps to one property and the last name maps to another property. I'm sure there are other scenarios... but in general we'd like to keep the possibility open for developers if they ever need such a feature.

[Ren]

Well, you don't always have the luxury of complete control over your DB. There might be denormalized fields like John_Smith in which the first name maps to one property and the last name maps to another property. I'm sure there are other scenarios... but in general we'd like to keep the possibility open for developers if they ever need such a feature.

If have use a column value more than once in a model, what happens if its altered in on location, and not in the other?

Denormalizing is for improving select performance, but at the cost of update & insert performance.

[allspiritseve]

If have use a column value more than once in a model, what happens if its altered in on location, and not in the other?

Well, in the example above there's still no overlap-- so naming John to Jim would result in Jim Smith, and renaming Smith to Jones would result in John Jones.

If there is some overlap, then either both locations need to be updated, or (and?) the domain object needs to be refactored. I'd rather offer the possibility of mapping one column to two methods than force a user to modify their domain object, though.

[Ren]

Not sure if this is going to be much use to you, but this is what my solution amounts to:

Might help might not…

Decided on a collection object to track changes...

class Collection
{
    private $added;
    private $removed;

    function attach($object)
    {
        if (!$this->added)
            $this->added = new SplObjectStorage();
        $this->added->attach($object);
    }

    function detach($object)
    {
        if (!$this->removed)
            $this->removed = new SplObjectStorage();
        $this->removed->attach($object);
    }

    function commit($add, $remove)
    {
        if ($this->added)
        {
            foreach($this->added as $object)
                $add($object);
            $this->added = null;
        }

        if ($this->removed)
        {
            foreach($this->removed as $object)
                $remove($object);
            $this->removed = null;
        }
    }
} 

And defining a relationship is just a matter of hooking into a few mapper events...

$class String name of the class in the collection
$getter Closure to retrieve a collection give an object instance.
$insert Closure to insert a row into the association table.
$delete Closure to delete a row from the association table.
$deleteAll Closure to delete all associated rows.

protected function hasMany($class, $getter, $insert, $delete, $deleteAll)
{
    $fn = function(Transaction $transaction, Connection $connection, $owner) use ($getter, $insert, $delete)
    {
        $transaction->onBeforeCommit(
            function() use ($owner, $getter, $insert, $delete, $transaction, $connection)
            {
                $collection = $getter($owner);
                $collection->commit(
                    function($add) use ($owner, $insert, $transaction, $connection)
                    {
                        $insert($transaction, $connection, $owner, $add);
                    },
                    function($remove) use ($owner, $delete, $transaction, $connection)
                    {
                        $delete($transaction, $connection, $owner, $remove);
                    }
                );
            }
        );
    };

    MapperEvent::append($this->afterInsert, $fn);
    MapperEvent::append($this->afterUpdate, $fn);
    MapperEvent::append($this->beforeDelete, $deleteAll);
} 

[Ezku]

Decided on a collection object to track changes...

Isn't that what a UnitOfWork is for?

[Ren]

Isn't that what a UnitOfWork is for?

Can't see how its supposed to work with many to many associations.

By the time you registerDirty() on a object, its too late to determine what alterations have been done. Unless use registerClean() to take a snapshot of the collections, then when its do diffs at commit time... but doesn't sound particularly appealing.

Or could run a select and to do the diff at commit time, might be better.

[Ren]

Doctrine appears to take the snapshot route, with a collection class, PersistentCollection.

Puts objects into it at hydration time, snapshot, then at UoW commit get it to diff itself to work out whats been added or removed.

[Peter Goodman]

I would distinguish between destructive (INSERT, UPDATE, DELETE) and non-destructive queries (SELECT). I would implement all non-destructive queries using lazy evaluation.

[Ren]

Another thought... if using autoincrements as identity/primary key, why do domain objects need an "id" property?

class Book
{
public $id;
public $name;
} 

vs

class Book
{
public $name;
} 

Obviously the autoincrement value has to be held somewhere (within the Mapper), but it would eliminate the need for setter and getter on domain objects. And given can use object instances as indexes with SplObjectStorage...

$identities = new SplObjectStorage();
if ($identities->contains($book))
{
update books set name = $book->name WHERE id = $identities[$book]
}
else
{
insert into books values($book->name) 
$identities->attach($book, lastInsertId);
} 

[Czaries]

Whoa, guys. You maybe getting a little too carried away for the topic at hand:
"Ideas for a lightweight ORM implementation"

I'm sure everyone's definition of lightweight will differ a bit, but I really think object hydration and tracking changes with snapshots is in the middle/heavyweight territory. If the goal is to make a truly lightweight ORM, Doctrine shouldn't even be mentioned in this thread. Doctrine is probably the heaviest PHP ORM in existence.

[Ren]

Whoa, guys. You maybe getting a little too carried away for the topic at hand:
"Ideas for a lightweight ORM implementation"

I'm sure everyone's definition of lightweight will differ a bit, but I really think object hydration and tracking changes with snapshots is in the middle/heavyweight territory. If the goal is to make a truly lightweight ORM, Doctrine shouldn't even be mentioned in this thread. Doctrine is probably the heaviest PHP ORM in existence.

I don't think it'd be an ORM without hydration.
Also fail to see how 30odd lines for snapshotting code is going to tip the balance.

<?php

class Snapshot
{
    private $snaps;

    function __construct()
    {
        $this->snaps = new SplObjectStorage();
    }

    function take(SplObjectStorage $collection)
    {
        $this->snaps->attach($collection, clone $collection);
    }

    protected function diff(SplObjectStorage $a, SplObjectStorage $b, Closure $f)
    {
        foreach($a as $o)
            if (!$b->contains($o))
                $f($o);
    }

    function runChanges(SplObjectStorage $changed, Closure $add, Closure $remove)
    {
        if ($this->snaps->contains($changed))
        {
            $original = $this->snaps[$changed];
            $this->diff($changed, $original, $add);
            $this->diff($original, $changed, $remove);
        }
    }
}


class O
{
    function __toString() { return spl_object_hash($this); }
}


$col1 = new SplObjectStorage();
$col2 = new SplObjectStorage();


$snapshot = new Snapshot();

$col2->attach($o = new O());

$snapshot->take($col1);
$snapshot->take($col2);

$col1->attach(new O());

$col2->detach($o);

$snapshot->runChanges($col1,
    function($item) { echo 'ADD ', $item, "\n"; },
    function($item) { echo 'REM ', $item, "\n"; }
);

echo "\n\n";

$snapshot->runChanges($col2,
    function($item) { echo 'ADD ', $item, "\n"; },
    function($item) { echo 'REM ', $item, "\n"; }
);

[Peter Goodman]

Obviously the autoincrement value has to be held somewhere (within the Mapper),

I disagree. An auto-increment value should not be presupposed for tables, especially where it makes no sense as a primary key.

Auto-incrementing keys as primary keys (surprise, an auto-incrementing key isn't always a primary key, nor is it necessarily easy to port of other databases) should be explicit. Even if you use them incorrectly, you should allow yourself to use composite primary keys correctly, or no primary key at all.

Please put a lot of thought into how you manage keys, primary or not, as they are an important part of database design and most ORM implementations that I have seen fail miserably at them.

[old_iron]

I would implement all non-destructive queries using lazy evaluation.

Why do you recommend lazy evaluation for SELECT queries?

[Peter Goodman]

Why do you recommend lazy evaluation for SELECT queries?

It will allow you to build queries/modify 'run' queries incrementally. The semantics of queries change when you know they are lazily evaluated. Be it by seeing SELECT queries more as query-building objects than as functions with an immediate side-effect.

Lazily evaluated queries will also make sure that even if you 'query' the database, the query will never actually be performed until the time when you actually need the result. This is useful in its own right.

Also note that the code I linked to is a means to emulate a sort of lazy-evaluation. The usual means might be to call on a function that either gets or returns an existing result set every time the result is needed. This has the same effect as the code I posted above; however, the usual method imposes the penalty of needless function calls and checks if the query has been executed for every time when the query is needed after it has been executed.

[Czaries]

Why do you recommend lazy evaluation for SELECT queries?

I recommend it too - execute the query at the last possible second, like when put in a foreach() or count(). That way additional conditions can be added dynamically or by chaining functions together. It also allows you to copy/clone and modify the exact query conditions in code since it's not automatically run when called, enabling you to do things like pagination more easily since you're modifying code instead of some generated SQL to add a total row count and limit conditions.

[Ren]

I disagree. An auto-increment value should not be presupposed for tables, especially where it makes no sense as a primary key.

Auto-incrementing keys as primary keys (surprise, an auto-incrementing key isn't always a primary key, nor is it necessarily easy to port of other databases) should be explicit. Even if you use them incorrectly, you should allow yourself to use composite primary keys correctly, or no primary key at all.

Please put a lot of thought into how you manage keys, primary or not, as they are an important part of database design and most ORM implementations that I have seen fail miserably at them.

I understand that an autoincrement and identity keys aren't necessarily the same. And the Mapper code posted previously allows any combination of possibilities.

// Have an auto increment property, and here is the getter & setter for it...
$this->autoIncrement(function (User $u) { return $u->id; }, function(User $u, $id) { $u->id = $id });

// Have a primary key/identity with single column in columnName => getter format.
$this->identity(array('id' => function(User $u) { return $u->id; })); 

The point was that with object's identity fields are not needed... so why include them in the domain model?

The mapper then doesn't need to know how out get/set an object autoincrement property, as its managing it entirely.

class Book { public $id; public $name; } vs class Book { public $name; } 

[Czaries]

I don't think it'd be an ORM without hydration.
Also fail to see how 30odd lines for snapshotting code is going to tip the balance.

It depends on how far you take the hydration. If you're just setting data/values on row objects, sure. But if you're keeping objects hydrated across different queries with identity maps, etc. then I don't think it belongs in a lightweight solution.

As with anything, you have to draw the line somewhere. I think lightweight should include only the features that you really can't do without, plus maybe a few more to make life easy. So in my opinion, I don't think snapshots fits in there, no matter how small the actual code is. But of course, everyone is entitled to their own opinion.

[allspiritseve]

As with anything, you have to draw the line somewhere. I think lightweight should include only the features that you really can't do without, plus maybe a few more to make life easy. So in my opinion, I don't think snapshots fits in there, no matter how small the actual code is. But of course, everyone is entitled to their own opinion.

Well... I agree, the things being discussed are probably a bit much for a lightweight solution. But as I've said a couple of times, we have more important goals than staying lightweight. I don't want anyone to get the impression we're trying to write a replacement for Doctrine, because we're not. But we're not opposed to discussing and potentially including features that may place us more at a medium weight ORM. Worst case we all learn a little bit, best case we end up improving the library.

[Ren]

It depends on how far you take the hydration. If you're just setting data/values on row objects, sure. But if you're keeping objects hydrated across different queries with identity maps, etc. then I don't think it belongs in a lightweight solution.

As with anything, you have to draw the line somewhere. I think lightweight should include only the features that you really can't do without, plus maybe a few more to make life easy. So in my opinion, I don't think snapshots fits in there, no matter how small the actual code is. But of course, everyone is entitled to their own opinion.

What I've chosen to do away with,

No modelling of the database objects (tables & columns) and so on.. Mappers provide closures which perform the required operations (insert, update, delete)

No bulk optimization, things like combining insert operations into one multi-insert statement, combing deletes, DELETE FROM things WHERE id = 1 and DELETE FROM things WHERE id = 2 into DELETE FROM things WHERE id IN(1,2).

As I don't think its worth the effort in time/code for web applications which would rarely use such features. And if you intend to do bulk operations, an ORM is not the thing to use anyway.

What would you leave out? As it sounds you'd leave out dealing with relationships/assocations... which is what snapshotting/tracking changes to collection is for.

$post = new Post();
$post->tags[] = $tag;
$post->save(); 

[allspiritseve]

What I've chosen to do away with, No modelling of the database objects (tables & columns) and so on.. Mappers provide closures which perform the required operations (insert, update, delete)

What do you mean by 'no modeling of the database objects'?

[Ren]

What do you mean by 'no modeling of the database objects'?

Stuff like...

http://www.getdorm.com/api/Dorm_Data...ase_Table.html

http://www.getdorm.com/api/Dorm_Data...ble_Field.html

Having to describe your schema in such detail first doesn't seem very lightweight.