Ideas for lightweight ORM implementation

[allspiritseve]

The problem with hard coding different find methods is that the model file must be changed.

I generally have one mapper per domain object, so that's not a problem.

By providing a interface to do anything at run-time the model never needs to be touched. Furthermore, routing everything to one method to find everything supports the DRY principle.

Really? Lets say on one page you need to find a user by ID:
user::find(array('id'=>89)); And then on another page, you need to find a user by another ID. How many times did you write this?

array('id'=>) 

Twice. Even in a simple case like this, I'd rather duplicate a method name (getById()) than an array key. If I ever need to rename that method, I'm going to get a bunch of errors telling me exactly what code where used a nonexistent method. Yeah, it's a simple example, but the more you do at runtime, the more duplication you have.

How the model finds what it is looking for is best a decided at run-time in my opinion. Otherwise, you end up with a bunch of similar methods that aren't very flexible.

I disagree. A datasource object should only need enough information to correctly identify what it needs. In the case above, the id is all it needs. Considering how common finding by an ID is, I don't need to tell it what field to match.

Actually the finder does have a interface. The array is converted by the system though. As does the model

Internally. That doesn't help me out when trying to figure out what to pass to your system to get it to do what I want. If I can't use your methods, what good is the interface to me? I'm stuck working with nested arrays. An array inside an array is probably fine, but any more than that is a pretty big indicator you have some refactoring to do.

[oddz]

Really? Lets say on one page you need to find a user by ID:
user::find(array('id'=>89)); And then on another page, you need to find a user by another ID. How many times did you write this?

find() is one generic method that handles ALL circumstances.

    public static function _find($pClassName,$pOptions) {
        
        if(self::isConnected()===false) throw new Exception('A database Adaptor has not been set for the '.__CLASS__.' Class.');
    
        $model = ActiveRecordModelConfig::getModelConfig($pClassName);
        
        $mode = !empty($pOptions) && is_array($pOptions[0])===false?array_shift($pOptions):self::findAll;
        
        $node = new ActiveRecordSelectNode($model,new ActiveRecordFindConfig(!empty($pOptions)?$pOptions[0]:array()));
        $select = strcasecmp($mode,self::findCount)==0?new ActiveRecordCount($node,$pOptions):new ActiveRecordSelect($node,$pOptions);
        
        if(strcmp($mode,self::findSelect)==0) return $select; // return without reseting count
        
        //echo '<p>',$select->toSql(),'</p>';
        //echo '<pre>',print_r($select->getBindData()),'</pre>';
        //return;
        
        $stmt = $select->query(self::$_db);
        ActiveRecordSelectNode::resetCount();
        $collectionAgent = new ActiveRecordCollectionAgent($select);
        
        if(strcmp($mode,self::findCount)==0) { return $stmt->fetchColumn(); }

        while($row = $stmt->fetch(PDO::FETCH_ASSOC)) {
            $collectionAgent->process($row,$node);
        }

        $records = $collectionAgent->getRecords();
        
        return strcmp($mode,self::findOne)==0?count($records)!=0?$records[0]:null:$records;
        
    } 

Twice. Even in a simple case like this, I'd rather duplicate a method name (getById()) than an array key. If I ever need to rename that method, I'm going to get a bunch of errors telling me exactly what code where used a nonexistent method. Yeah, it's a simple example, but the more you do at runtime, the more duplication you have.

There isn't any duplication. One find, save and delete method that are generic enough to handle most if not all circumstances.

I disagree. A datasource object should only need enough information to correctly identify what it needs. In the case above, the id is all it needs. Considering how common finding by an ID is, I don't need to tell it what field to match.

Correct but what happens when you need to filter and add many more conditions? Either you would need to write a separate find method or make the finder generic. I prefer the ladder.

Internally. That doesn't help me out when trying to figure out what to pass to your system to get it to do what I want. If I can't use your methods, what good is the interface to me? I'm stuck working with nested arrays. An array inside an array is probably fine, but any more than that is a pretty big indicator you have some refactoring to do.

The interface supports a very straightforward interface while also providing a very complex one. There aren't many system out there that support calculated columns, aggregates and subqueries. All of which are things that I have managed to integrate on the generic level without any hard coding or repetition. All ActiveRecord instances/classes inherit these functions.

[allspiritseve]

find() is one generic method that handles ALL circumstances. There isn't any duplication. One find, save and delete method that are generic enough to handle most if not all circumstances

user::find(array('id'=>89));
user::find(array('id'=>89));

The text in red is duplicated. Again, simple example but the more you do at runtime the more code is duplicated when you grab the same object twice in different parts of your application.

Correct but what happens when you need to filter and add many more conditions? Either you would need to write a separate find method or make the finder generic. I prefer the latter.

Some sort of generic finder methods needs to be included that either use arrays, as yours does, an Object Query Language (OQL), or straight SQL.

The most common use cases should probably be methods, the edge cases can be handled by the generic finder. That strikes a good balance between too specific and too generic.

The interface supports a very straightforward interface while also providing a very complex one. There aren't many system out there that support calculated columns, aggregates and subqueries. All of which are things that I have managed to integrate on the generic level without any hard coding or repetition. All ActiveRecord instances/classes inherit these functions.

That's great... I don't want to specify calculated columns, aggregates, and/or subqueries more than once per domain object unless I have to. If the system does that by default, all I have to do is tell it how to find the right object. I don't even have to worry about SQL.

[oddz]

The text in red is duplicated. Again, simple example but the more you do at runtime the more code is duplicated when you grab the same object twice in different parts of your application.

That has to do with cache management. I'll admit that my system does not have a cache system yet. That is one thing I've began thinking about recently. So that isn't something I'd be comfortable with commenting about.

The most common use cases should probably be methods, the edge cases can be handled by the generic finder. That strikes a good balance between too specific and too generic.

The only common case is filtering by the primary key.

That's great... I don't want to specify calculated columns, aggregates, and/or subqueries more than once per domain object unless I have to. If the system does that by default, all I have to do is tell it how to find the right object. I don't even have to worry about SQL.

The SQL would need to written in the model. Also, unless your unconcerned about duplicate data the parsing of the result set into a meaningful hierarchy would need to be handled separately also. That would be at least 100 lines of code that could be eliminated with a generic mechanism.

[allspiritseve]

That has to do with cache management. I'll admit that my system does not have a cache system yet. That is one thing I've began thinking about recently. So that isn't something I'd be comfortable with commenting about.

Man, if you're thinking about caching this stuff, you're really making things hard on yourself.

The only common case is filtering by the primary key.

I don't mean common to all domain objects... just commonly used. Finding a user by username, for example.

The SQL would need to written in the model. Also, unless your unconcerned about duplicate data the parsing of the result set into a meaningful hierarchy would need to be handled separately also. That would be at least 100 lines of code that could be eliminated with a generic mechanism.

The ORM should handle the SQL, and parsing the result set into a meaningful hierarchy. SQL queries aren't generic, nor are result sets nor hierarchies. At some point you need specific code to handle specific use cases, and that 100 lines of code becomes very useful (potentially saving you thousands of lines later on).

[oddz]

I don't mean common to all domain objects... just commonly used. Finding a user by username, for example

That isn't a generic case though. A Category won't have a user name. Username in this instance would only apply to a User.

The ORM should handle the SQL, and parsing the result set into a meaningful hierarchy. SQL queries aren't generic, nor are result sets nor hierarchies.

Queries have rules which define them. By understanding those rules a generic system of generating them is possible. The same for a relational hierarchy. If object A is a child of B and A can be related to B via A.id = B.a_id then it can be said that A has B and B is dependent on A. Knowing that a hierarchy can constructed on a generic level that mimics that relationship A->B or B->A.

[allspiritseve]

That isn't a generic case though. A Category won't have a user name. Username in this instance would only apply to a User.

Right. So it would either go in the User class that extends your AR base class, or in my case, in UserMapper.

[oddz]

I don't see the purpose behind abstracting that with a method. The same thing is just as simple using a generic filter in either case of a Gateway or ActiveRecord.

$userMapper->find(array('username'=>'tom')); 

=

User::find(array('username'=>'tom')); 

[wysiwyg]

/.../ For instance, I could make a properly separated Mapper look like AR by passing it in the constructor of a domain object and delegating finders and save() to it /.../

No... That wouldn't be a properly separated Data Mapper, since the Domain Object knows about the Data Mapper. It's some separation though, since you can pass any type of Data Mapper (XmlMapper, DbMapper, what ever) but the Domain Object still knows about its own persistance.

Not trying to be a wiseguy, but in Fowlers definition of a Data Mapper, it says: "A layer of Mappers (473) that moves data between objects and a database while keeping them independent of each other and the mapper itself."

[allspiritseve]

No... That wouldn't be a properly separated Data Mapper, since the Domain Object knows about the Data Mapper. It's some separation though, since you can pass any type of Data Mapper (XmlMapper, DbMapper, what ever) but the Domain Object still knows about its own persistance.

Maybe it was poor word choice on my part, but I meant starting with a properly separated Mapper, you can make the interface similar to a non-ORM ActiveRecord implementation. I'm not claiming it stays properly separated, just that the interfaces are similar. I'm also not claiming it's anything I would do in practice, it was just an example. No quoting Fowler needed, we are on the same page.

[Czaries]

user::find(array('id'=>89));
user::find(array('id'=>89));

The text in red is duplicated. Again, simple example but the more you do at runtime the more code is duplicated when you grab the same object twice in different parts of your application.

I don't see the difference between duplicating the code with an array and duplicating in a function name. It's still going to have to be duplicated somewhere:

user::findById(89);
user::findById(89);

It all depends on how you want to solve the problem. I like doing as little work as possible, which means as few specific custom methods on my mapper as possible. The find() method is made to be as generic and re-usable as possible. The point is to make it so you never HAVE to code functions for each mapper. If you want to though, you still can with one line of code:

public function findById($id) {
    return self::find(array('id' => $id));
} 

It makes it easier all the way around. I don't get why you're against doing less work.

[allspiritseve]

I don't see the difference between duplicating the code with an array and duplicating in a function name. It's still going to have to be duplicated somewhere

That's why it's a simple example... but if the method is what's duplicated, I have the freedom to change the underlying field name without breaking the system, and I only have to change it in one place. Obviously this example doesn't work as well because an id is probably the least likely field to change, but you get the idea. The same concept can apply for longer methods:

$users = User::find(
    array(
        'dynamic'=>array(
            'user_year'=>'YEAR({this}.created)'
        )
        ,'sort'=>array(
            'user_year'=>'DESC'
        )
    )
); 

Compared with:

$users = $userMapper->findByYearCreated (2009, 'asc'); 

It all depends on how you want to solve the problem. I like doing as little work as possible, which means as few specific custom methods on my mapper as possible. The find() method is made to be as generic and re-usable as possible. The point is to make it so you never HAVE to code functions for each mapper. If you want to though, you still can with one line of code. It makes it easier all the way around. I don't get why you're against doing less work.

I as well like doing as little work as possible, but that means I spend a little time setting up my mappers so I don't have to do much at runtime. I do think the method you showed is a step in the right direction, and I also agree flexible finders are necessary. They should not be the only source for ORM functionality, though.

[Czaries]

That's why it's a simple example... but if the method is what's duplicated, I have the freedom to change the underlying field name without breaking the system, and I only have to change it in one place. Obviously this example doesn't work as well because an id is probably the least likely field to change, but you get the idea.

I completely agree, and that is a good thing in general. I also agree that longer more complex code needs to be moved into it's own function - that's just basic programming 101. But my point is that if your field name changes, then your function name will be wrong too, so it's still not eliminating the problem.

You could solve the problem by re-mapping the field to a different one, like an override. So maybe you changed the 'fullname' field to just 'name'. So you can map the 'fullname' column to 'name' in your mapper and your old queries will still run just fine. You still end up having to make changes if you used verbose functions instead, like 'findByFullname' would now have to be 'findByName', and 'findByFullname' would have to delegate to the new 'findByName' function for backwards compatibility.

I as well like doing as little work as possible, but that means I spend a little time setting up my mappers so I don't have to do much at runtime. I do think the method you showed is a step in the right direction, and I also agree flexible finders are necessary. They should not be the only source for ORM functionality, though.

I agree again, and obviously I would never say they are the only source for ORM functionality. That's just ridiculous. I'm just implying that I don't want to have to make a bunch of finder functions just to start using the mapper. That's way too much work. I want to be able to make custom functions if I need to, but I never want to have to, and I won't until I need to.

[allspiritseve]

But my point is that if your field name changes, then your function name will be wrong too, so it's still not eliminating the problem.

I agree with what you're saying, and I think it depends on the type of change. For example, working with a database in which you don't have control over schema. Or optimizing the SQL used in a complex query.

I agree again, and obviously I would never say they are the only source for ORM functionality. That's just ridiculous. I'm just implying that I don't want to have to make a bunch of finder functions just to start using the mapper. That's way too much work. I want to be able to make custom functions if I need to, but I never want to have to, and I won't until I need to.

I think one of the central arguments between oddz and I is whether to set up joins, subqueries, calculated columns, (and in my case mappings) at runtime or when coding the classes. I prefer setting up defaults when I first code a mapper, and then using custom methods when I need to vary from those defaults. That should cover most use cases, and then for the complex edge cases that are complex and probably only used once, I can customize a find method. I don't think a system that allows you do that can't also allow you to do everything with find methods, but I'd like the option to do either.

[oddz]

"A layer of Mappers (473) that moves data between objects and a database while keeping them independent of each other and the mapper itself."

So would the following constitute as a mapper? There is a separation between the mapper (config) and data container (entity) – is there not?

$config = new ModelConfig('User');

$sql = 'SELECT '.implode(',',$config->getFields()).' FROM '.$config->getTable().';';

$result = mysql_query($sql);

$entities = new DataCollection();

while($row = mysql_fetch_assoc($result)) {

      $entiity = new DataEntity();
      $entities->add($entity);
      
       foreach($config->getFields() as $field) $entity->setProperty($field,$row[$field]); 
} 

I agree again, and obviously I would never say they are the only source for ORM functionality. That's just ridiculous. I'm just implying that I don't want to have to make a bunch of finder functions just to start using the mapper. That's way too much work. I want to be able to make custom functions if I need to, but I never want to have to, and I won't until I need to.

exactly

I completely agree, and that is a good thing in general. I also agree that longer more complex code needs to be moved into it's own function - that's just basic programming 101. But my point is that if your field name changes, then your function name will be wrong too, so it's still not eliminating the problem.

Field names will always be tightly coupled to the related table. Whether you place them inside the Mapper or outside or abstract them – in the end they need to be converted to the table equivalent.

[Czaries]

I agree with what you're saying, and I think it depends on the type of change. For example, working with a database in which you don't have control over schema. Or optimizing the SQL used in a complex query.

I think we're pretty much in agreement here. I would probably never make custom functions for things like 'findById' or 'findByName', but I generally always make custom functions for anything even remotely complex, like 'findInvoicesInDateRange' or 'findContactsByUsername'.

I prefer setting up defaults when I first code a mapper, and then using custom methods when I need to vary from those defaults. That should cover most use cases, and then for the complex edge cases that are complex and probably only used once, I can customize a find method. I don't think a system that allows you do that can't also allow you to do everything with find methods, but I'd like the option to do either.

I really think we're on the same page here .

[allspiritseve]

What different ways of hydrating an object with data should we support? We have currently discussed:

Setting a property:

$object->color = 'Red'; 

Setting methods:

$object->setColor('red'); 

Setting generic methods (set() and get()):

$object->set('color', 'Red'); 

Setting a constructor parameter:

$object = new Object('Red'); 

Anything else we should support?

[Ren]

Ok, been asked to if I'd share some ideas/code. So here is what I have atm.

It requires PHP 5.3, currently has no support for SELECTs, building SQL dynamically, or building/hydrating.

What it does start to implement the Data Mapper, Identity Map & Unit of Work patterns.

Whilst dealing with relationships aren't complete, the UnitOfWork is aware and will arrange the IUD operations in an order that can be run correctly.

Warning code is a bit gnarly in places.
All classes and example are all in the same PHP file, which requires PDO & pdo_sqlite.
Also echoing of SQL happens.

[oddz]

Your may need custom getter and setters for adding none primitive data such as other data containers (records). I'm currently using two methods called addRecord() and getRecord().

The getRecord() method takes three arguments:
1.) property name
2.) primary key value
3.) primary key field

It will go through a collection of associated records and return the record that that matches the primary key at the specified primary key field.

The other method addRecord() takes three arguments:
1.) property name
2.) record
2.) collection or object (bool)

These methods are used to add records to records. Something could prove useful if you need that type of management system.

Here is a snippet of those two methods just to give you idea if its something your interested in supporting:

    public function getRecord($pPropertyName,$pPrimaryKey,$pField) {
    
        if(!array_key_exists($pPropertyName,$this->_data)) return false;
        
        if(!($this->_data[$pPropertyName][0] instanceof IActiveRecordDataEntity) && $this->_data[$pPropertyName][0] instanceof arrayaccess) {
        
            foreach($this->_data[$pPropertyName][0] as $record) {
            
                if($record->$pField == $pPrimaryKey) return $record;
            
            }
        
        } else {
            
            if($this->_data[$pPropertyName][0]->$pField == $pPrimaryKey) {
                
                return $this->_data[$pPropertyName][0];
            
            }
            
        }
        
        return false;
    
    }
    
    public function addRecord($pPropertyName,IActiveRecordDataEntity $pRecord,$pArrayByDefault=false) {
    
        if(array_key_exists($pPropertyName,$this->_data)===true) {
        
            if($this->_data[$pPropertyName][0] instanceof arrayaccess) {
            
                $this->_data[$pPropertyName][0][] = $pRecord;
            
            } else {    
            
                //$this->_data[$pPropertyName][0] = array($this->_data[$pPropertyName][0]);
                $this->_data[$pPropertyName][0] = new ActiveRecordCollection($this->_data[$pPropertyName][0]);
            
            }
        
        } else {
            
            // for a hasMany relationship with only one item. Otehrwise if something
            // only has one item in its result set but has a hasMany relationship
            // a array would not exists which seems wrong.
            if($pArrayByDefault === true) {
                $this->_data[$pPropertyName] = array(new ActiveRecordCollection($pRecord));
            } else {
                $this->_data[$pPropertyName] = array($pRecord);
            }
        
        }
    
    } 

[allspiritseve]

Ok, been asked to if I'd share some ideas/code. So here is what I have atm.

Cool, I'll check it out.

Your may need custom getter and setters for adding none primitive data such as other data containers (records). I'm currently using two methods called addRecord() and getRecord().

Cool, thanks for that. I think for something nonstandard like that we may need a way to specify any number of parameters for a method. As a last resort, we'll also offer the possibility of overwriting the mapping, so you can customize it as you see fit.

[Ren]

What different ways of hydrating an object with data should we support? We have currently discussed:

Setting a property:

$object->color = 'Red'; 

Setting methods:

$object->setColor('red'); 

Setting generic methods (set() and get()):

$object->set('color', 'Red'); 

Setting a constructor parameter:

$object = new Object('Red'); 

Anything else we should support?

Well using PHP5.3, I'm probably going down this route...

function createFromRow($row)
{
      $object = new ... ();
      foreach($this->setter as $name => $setter)
           $setter($object, $row[$name]);
      return $object;
} 

$setter can do any of first 3, or set up lazy loading for other objects.

[oddz]

Its going to be a while before 5.3 becomes wide spread… a while. I wouldn't hold your breath and for that reason I would recommend making your code compatible with <=5.3.

[Ren]

Its going to be a while before 5.3 becomes wide spread… a while. I wouldn't hold your breath and for that reason I would recommend making your code compatible with <=5.3.

Yeah, I understand that.

The only reason I'm toying with DataMappers and so forth is to exercise the new features in 5.3, and how can improve, simplify and reduce the LOC.

[allspiritseve]

Well using PHP5.3, I'm probably going down this route... $setter can do any of first 3, or set up lazy loading for other objects

Yeah we've got something similar, though non PHP5.3:

    public function load($array)    {
        $object = $this->create($this->getParams($array));
        if (empty ($this->mappings))    {
            foreach (array_keys ($array) as $column)    {
                $this->map($column);
            }
        }
        foreach ($this->getMappings() as $mapping)    {
            $mapping->loadObject ($object, $array);
        }
        return $object;
    } 

Basically, create() instantiated the target object from an array of parameters (only possible using reflection, I believe). If no mappings are set for the object, then default mappings are given based on the array keys. Finally, each mapping object maps from db to object (seems very similar to your $setters). The mapping objects do double duty though, as they're also used for mapping from an object back to an SQL query.

[allspiritseve]

Another question: I'm trying to get the minimum config use case worked out. That is:

$mapper = new A_Orm_DataMapper('Post', 'posts'); 

This will allow you to interact with a Post object in which its properties map 1:1 to fields in the posts table. No more config needed than that, unless you want more

Can I assume a field named 'id' is the primary key for this table? Or would you guys prefer I did a DESCRIBE query every time the mapper was loaded? I'm trying to assume as little as possible about these objects, but it's tough because I'm forced to check everywhere and see if mappings exist, otherwise handle the 1:1 scenario. I may end up extracting all of that code into a class that extends the main class, so the core stays light and the developer can choose whether to allow no-config mapping by which class they instantiate or extend.