PHP Hibernate, OQL and other goodies

[lastcraft]

Hi...

I guess the topic to start would be the format of the metadata. We'd need a DTD or XMLSchema to define how the mapping will be described and furthermore a parser for it. And a strategy to cache the metadata (we don't want to parse the XML-File on every request).

It's a bit too early for this as it would evolve with teh project. For example I doubt inheritance would be implemented at the early stages (if at all) and there are several ways to do it. IT would take a crystal ball to have any idea what would be needed from the metadata. Beware the BDUF demon (big dedsign up front).

As for caching it, there is actually no point. By code generating the data mappers all of the information is converted to code anyway. You will need a generation step anyway to generate the schema scripts, so why not do as much processing as possible at that time ("currying"). Most of the example code posted on Sitepoint so far has used this process.

yours, Marcus

[lsmith]

Well my argument was that even if you generate all the code you may not want all the associated code to be loaded until you actually use it. Therefore it may make sense to not have to load the rest of the code to handle the given entity until you have determined you need it to make an instance of it.

[lastcraft]

Hi...

Well my argument was that even if you generate all the code you may not want all the associated code to be loaded until you actually use it. Therefore it may make sense to not have to load the rest of the code to handle the given entity until you have determined you need it to make an instance of it.

Then just put each mapper into it's own file and use the usual PHP require_once() trickery to get it in efficiently. Or better yet, use a byte code cache. This really is a minor issue in the scheme of things.

The real problems are determining how pure the object model will be and how much of the system tuning is exposed to the user. That dictates the complexity of the project. The level of sophistication if you will. Without some sort of mission statement to this effect you will run into a swamp of minor disagreements on how to do things. You cannot even write the first prototype without this.

yours, Marcus

[lsmith]

I agree .. that when it comes down to it what we are talking about atm are minor features that require no key structural decisions and as such qualify as premature optimization.

[sike]

hi,

I guess the topic to start would be the format of the metadata. We'd need a DTD or XMLSchema to define how the mapping will be described and furthermore a parser for it. And a strategy to cache the metadata (we don't want to parse the XML-File on every request).

I think we need a mechanism to translate the metadata information into instances of certain descriptor classes (e.g. ClassDescriptor, AttributeDescriptor, AssociationDescriptor, ...) and cache those instances.

The strategies for caching could be: cache the serialized object on the disk or
use shared memory, if available.

Furthermore we need a kind of Handler class that will detect whether the XML-file changed (if it changed, it needs to be parsed again) and lazy load the cached descriptor data.

for getting started we should really skip the xml formats discussion and related stuff and work on the internal meta model instead (eg. using object graphs or a more array based solution).

cheers
Chris

[hantu]

It's a bit too early for this as it would evolve with teh project.

Ok, I totally agree. I just wanted to throw another point into the discussion.

By code generating the data mappers all of the information is converted to code anyway

Would it be a viable option to only convert the metadata into an easy accessible format and do the mapping based on this information and reflection (e.g. dynamic method calls)?

...

function generateInsertStatement ($object, $metaHandler) {
  $desc = $metaHandler->getClassDescriptor(get_class($object));
  
  $sql = 'INSERT INTO '.$desc->getTable();
  $sql.= ' ('.$desc->getColumnList.') VALUES (';
  // generate placeholders (?,?,?,?) 
  $sql.= implode(',', array_pad(array(), $desc->getColumnCount(), '?'));
  $sql.=')';
 
  $fieldDescs = $desc->getFieldDescriptors();
  $i = 1;

  $stmt = new PreparedStatement($sql);

  // fill statement with object's values
  foreach ($fieldDescs as $fieldDesc) {
    $setMethod = 'set'.$fieldDesc->getType();
    $getMethod = 'get'.$fieldDesc->getName();

    $stmt->$setMethod($i++, $object->$getMethod());
  }

  return $stmt;
}
... 

Just some food for thoughts ...

[lsmith]

I got that part nicely covered inside LiveUser_Admin .. although I dont use prepared queries to handle proper quoting (I do ensure proper quoting though!).

Actually aside from the deficiencies I mentioned earlier about the limitations in regards to WHERE statement flexibility (I also mentioned this could be fixed by letting people pass Criteria Objects instead of simply arrays as filters) I think I have the query generation part done quite nicely there (well things like GROUP BY, self joins etc are also missing).

So maybe it would be a good idea to define what kind of queries we think would need to be possible to generate?

[arborint]

how pure the object model will be and how much of the system tuning is exposed to the user

This is a very interesting statement from the YAGNI point of view.

Do we know up front that we want/need a pure the object model or little/no system tuning is exposed to the user?

If we start by saying that we can tolerate an messier object model and a good amount system tuning exposed to the user, does that get us farther faster?

Are we headed toward one or more prototypes? Or a toward what we think is the best shot?

[ponticelli]

I don't know if that has been discussed before and I apologize in this case. My question is: have you considered to store serialized objects instead of ORM?

[hantu]

Without some sort of mission statement to this effect you will run into a swamp of minor disagreements on how to do things. You cannot even write the first prototype without this.

I found an interesting slide summarizing the goals/features of Hibernate. Maybe it's a help for discussion and sorting out unwanted stuff:


Modern ORM Solutions

• Transparent Persistence
• Automatic dirty checking
• Transitive Persistence
• Inheritance mapping strategies
• Smart fetching and caching
• Development tools

Defining Transparent Persistence

• Any class can be a persistent class
• No interfaces have to be implemented
• No persistent superclass has to be extended
• Persistent classes can be used outside of the “persistence” context (Unit Tests, Batch Processing)
• Full portability without any dependency

Persistent classes

• JavaBean specification (or POJOs)
• Accessor methods for properties
• No-arg constructor
• Collection property is an interface (I don't understand this ...)
• Identifier property optional

Hibernate Query Language

• Make SQL “object-oriented”
• classes and properties instead of tables and columns
• supports polymorphism
• automatic association joining
• much less verbose than SQL
• Full support for relational operations
• inner/outer/full joins, cartesian product
• projection, ordering, aggregation and grouping
• subqueries and SQL functions

The slide can be found here: http://www.hibernate.org/hib_docs/on...atljug2004.pdf

[Dr Livingston]

Yer, I got something much like what you've gone and posted about data persistence in a PDF?

The file is way too large for attachment thus I've not already attached it you see I'll hunt it out again for anyone who wants me to email them it yes? Below is an actual copy and paste of some paragraphs from the PDF to clarify

A persistence layer encapsulates the behavior needed to make objects persistent, in other words to read,
write, and delete objects to/from permanent storage. A robust persistence layer should support:
1. Several types of persistence mechanism. A persistence mechanism is any technology that can be used
to permanently store objects for later update, retrieval, and/or deletion. Possible persistence
mechanisms include flat files, relational databases, object-relational databases, hierarchical databases,
network databases, and objectbases. In this paper I will concentrate on the relational aspects of a
persistence layer.
2. Full encapsulation of the persistence mechanism(s). Ideally you should only have to send the
messages save, delete, and retrieve to an object to save it, delete it, or retrieve it respectively. That’s it,
the persistence layer takes care of the rest. Furthermore, except for well-justified exceptions, you
shouldn’t have to write any special persistence code other than that of the persistence layer itself.
3. Multi-object actions. Because it is common to retrieve several objects at once, perhaps for a report or
as the result of a customized search, a robust persistence layer must be able to support the retrieval of
many objects simultaneously. The same can be said of deleting objects from the persistence
mechanism that meet specific criteria.
4. Transactions. Related to requirement #3 is the support for transactions, a collection of actions on
several objects. A transaction could be made up of any combination of saving, retrieving, and/or
deleting of objects. Transactions may be flat, an “all-or-nothing” approach where all the actions must
either succeed or be rolled back (canceled), or they may be nested, an approach where a transaction is
made up of other transactions which are committed and not rolled back if the large transaction fails.
Transactions may also be short-lived, running in thousandths of a second, or long-lived, taking hours,
days, weeks, or even months to complete.
5. Extensibility. You should be able to add new classes to your object applications and be able to change
persistence mechanisms easily (you can count on at least upgrading your persistence mechanism over
time, if not port to one from a different vendor). In other words your persistence layer must be flexible
enough to allow your application programmers and persistence mechanism administrators to each do
what they need to do.
6. Object identifiers. An object identifier (Ambler, 1998c), or OID for short, is an attribute, typically a
number, that uniquely identifies an object. OIDs are the object-oriented equivalent of keys from
relational theory, columns that uniquely identify a row within a table.
7. Cursors. A persistence layer that supports the ability to retrieve many objects with a single command
should also support the ability to retrieve more than just objects. The issue is one of efficiency: Do you
really want to allow users to retrieve every single person object stored in your persistence mechanism,
perhaps millions, all at once? Of course not. An interesting concept from the relational world is that of
a cursor. A cursor is a logical connection to the persistence mechanism from which you can retrieve
objects using a controlled approach, usually several at a time. This is often more efficient than
returning hundreds or even thousands of objects all at once because the user many not need all of the
objects immediately (perhaps they are scrolling through a list).
8. Proxies. A complementary approach to cursors is that of a “proxy.” A proxy is an object that
represents another object but does not incur the same overhead as the object that it represents. A
proxy contains enough information for both the computer and the user to identify it and no more. For
example, a proxy for a person object would contain its OID so that the application can identify it and the
first name, last name, and middle initial so that the user could recognize who the proxy object
represents. Proxies are commonly used when the results of a query are to be displayed in a list, from
which the user will select only one or two. When the user selects the proxy object from the list the real
object is retrieved automatically from the persistence mechanism, an object which is much larger than
the proxy . For example, the full person object may include an address and a picture of the person. By
using proxies you don’t need to bring all of this information across the network for every person in the
list, only the information that the users actually want.
9. Records. The vast majority of reporting tools available in the industry today expect to take collections
of database records as input, not collections of objects. If your organization is using such a tool for
creating reports within an object-oriented application your persistence layer should support the ability
to simply return records as the result of retrieval requests in order to avoid the overhead of converting
the database records to objects and then back to records.
10. Multiple architectures. As organizations move from centralized mainframe architectures to 2-tier
client/server architectures to n-tier architectures to distributed objects your persistence layer should be
able to support these various approaches. The point to be made is that you must assume that atsome
point your persistence layer will need to exist in a range of potentially complex environments.

There is more besides btw

[arborint]

I think there are three things here: ORM, Persistence and HQL. I think the Persistence part is where the questions are at this point. The thing I am having a hard time clarifying is whether Hibernate style Transparent and Transitive Persistence is so much a part of JavaBeans and the application server that instead of trying to go that way in PHP you should just use Java? And if that is the case, then is there a "third way" to design a useful, PHP-style persistence layer on which to build the ORM/HQL parts?

[Dr Livingston]

Number 7 (Cursors) is off interest I'd think

[sike]

Number 7 (Cursors) is off interest I'd think

are you talking about the papers from scott ambler (www.agiledata.com)?
if not most of his writings are very usefull. in one pdf (and i thinks its the one you are speaking about) he describes the whole mechanics behind a persistence framework in enough details to base a framework on it. maybee its a good starting point.

cheers
Christian

ps. just finished a prototype of a query parser and the necessary meta descriptors including a sql builder. anyone interested in reviewing the code ?
if we work on some prototypes we might get a better grasp on what fits php best.

[hantu]

The thing I am having a hard time clarifying is whether Hibernate style Transparent and Transitive Persistence is so much a part of JavaBeans and the application server that instead of trying to go that way in PHP you should just use Java?

As I see it the way of persistence Hibernate offers does explicitly NOT require an application server (JavaBeans doesn't mean EJB, if this is unclear). The main goal is to persist idiomatic java, which means (almost) any class can be persisted.

For most things I develop, I consider PHP much more appropriate than Java because of its ease of use (no 'ant install' ... wait ... open browser ... wait until jsps are compiled ...).
The main disadvantage is that there is no comfortable way of persistence in PHP (I don't like handcoding SQL or using generated classes like propel's). And a transparent persistence tool would wipe out this last disadvantage.

[Dr Livingston]

Sike,

Material for this White Paper has been excerpted from Scott W. Ambler’s books
Building Object Applications That Work,
Process Patterns,
The Object Primer 2nd Edition
and the
Design of a Persistence Layer Series
Software Development magazine, January through April 1998

Yes, must be the same PDF we are referring to I think? I agree with you on the point about the quality as well, worthwhile taking a nose over to the site once in a while in my view

Instead of emailing the PDF, I think there is a link to get a copy

http://www.ambysoft.com/persistenceLayer.pdf

Enjoy.

[arborint]

The main disadvantage is that there is no comfortable way of persistence in PHP (I don't like handcoding SQL or using generated classes like propel's). And a transparent persistence tool would wipe out this last disadvantage.

I guess I'm trying understand the distinctions here. You want a "comfortable way of persistence" which means (to you) no "handcoding SQL or using generated classes," then jump to "transparent persistence" as the solution.

If this thing provided an HQL system and dealt with smart fetching and dirty cache writing, would that be enough? I see things like Propel as more of a convenience than an improvement.

Going back to Brenden's example (http://www.sitepoint.com/forums/show...9&postcount=39) is "persistence" really needed at all for the short PHP request/response cycle where the main goal is to just make sure that all data is read/written back.

[hantu]

If this thing provided an HQL system and dealt with smart fetching and dirty cache writing, would that be enough?

I'd even be content with a good Query/Criteria implementation instead of HQL/OQL, yet OQL support would be a very cool and mighty feature.

Going back to Brenden's example (http://www.sitepoint.com/forums/show...9&postcount=39) is "persistence" really needed at all for the short PHP request/response cycle where the main goal is to just make sure that all data is read/written back.

What do you see as the difference between "all data is read/written back" and "persistence"? For me this means quite the same.

[arborint]

What do you see as the difference between "all data is read/written back" and "persistence"? For me this means quite the same.

I think in PHP they are the same. However in a long running program written in Java is a different and much more difficult problem. In PHP we really just want to go from:

all data is read -> manipulate data -> all data is written back

To:

needed data is read -> manipulate data -> changed data is written back

For me this question is boiling down to: how to create a "comfortable" system that does not require generated classes. Comfortable means that creating Persisent classes is not too different from regular classes and not too much work. Can we agree on a definition of "too"?

[hantu]

In PHP we really just want to go from:

all data is read -> manipulate data -> all data is written back

To:

needed data is read -> manipulate data -> changed data is written back

I totally agree.

For me this question is boiling down to: how to create a "comfortable" system that does not require generated classes. Comfortable means that creating Persisent classes is not too different from regular classes and not too much work. Can we agree on a definition of "too"?

So what do you consider a "regular" PHP class?

I'd say requirements for a persistent class should be a no-arg constructor and getters/setters for the fields including a field holding the PrimaryKey. And it shouldn't have to implement a certain interface or extend a certain base class to be persistent.

Would that be a definition you could agree on or do you imagine something different?

[arborint]

Once you set some requirements you get rid of "every class can be persisted" which is fine my me. This gets back to my question that if you have to do this:

class Employee {
private $Pay;
    function setPay($pay) { ... }
    function getPay() { ... }
    function addYearlyRise() {
        $this->setPay($this->getPay() * 1.05);
    }
} 

Then is something like this that might allow you to have args in the constructor and no gettors/settors like:

class Employee {
private $persistent_Pay;
   function addYearlyRise() {
        $this->persistent_Pay *= 1.05;
    }
} 

Both provide info to the Persistence code.

[Brenden Vickery]

Both provide info to the Persistence code.

The ideal way, I think, would be to not force the use a constructor or setters/getters. Take some of the work done in the comments on get_object_vars. This way the only thing forced on a persistence object is that the property name is equal to the property named described in the xml map, everything else is done with casts (I assume this becomes more difficult if you have, for example an array of objects as a property). Have a constructor or setters if you want, it becomes irrelevant. Of course you have to weigh the performance cost/benefits.

Does anyone know of another way to set/get access to a private variable?

The only reasons, IMO, to force a persistent object to implement an interface or extend a base class is if that is the only way to get an acceptance / performance test to pass. If either of these situations come into play, again the cost / benfits of the feature needs to be weighed against being able to persist any object.

[hantu]

@arborint:

Sorry, but I couldn't get the point of your previous post. Could you provide some more details on what you're trying to accomplish?

[hantu]

Have a constructor or setters if you want, it becomes irrelevant.

The reason for the constraint of having a no-arg constructor (or none) is that the mapper will need to be able to instantiate objects of persistent classes. This might be very difficult if your constructor has arguments.

[Brenden Vickery]

The reason for the constraint of having a no-arg constructor (or none) is that the mapper will need to be able to instantiate objects of persistent classes. This might be very difficult if your constructor has arguments.

Yes, in the getter/setter situation. I was bringing up the option of casting the objects to/from arrays to get at properties. Im not sure how viable this is (performance wise) but it would be ideal from a design point of view.

Quick example :

class Person {
   private $firstname;
   private $lastname;
   public function __construct($f, $l) {
       $this->firstname = $f;
       $this->lastname = $l;
   }
}

$person = array();
$person['firstname'] = 'Brenden';
$person['lastname'] = 'Vickery';

$person = cast($person, 'Person');

if($person instanceof Person) {
   echo 'true';
}

// this echos true 

The "cast" function above is equivalent to the bless function in the link I gave in the last post.