PHP DataCache 1.0-alpha1 release!

[sweatje]

I wrote a class to implement server side data caching that I have used for several projects at work. I decided to give a presentation on data caching (Day 3-9:30) and consequently rewrote the class. I ended up putting enough effort into getting the class right for the presentation that I thought it merited release as an open source project (I hope you will agree ).

Please have a look at the project homepage and especially the project documentation.

The idea here is that the DataCache object represents a very simple API to persist data.

Here is a sample example of how the cache object can be used:

$o_data =& new DataCache(MY_APPLICATION);



// if data is cached and more recent than one hour old

if ($o_data->IsCached(DATA_ID) &&

    $o_data->GetCacheTime(DATA_ID) > mktime()-3600) {

    $a_data = $o_data->GetCache(DATA_ID);

} else { //data is stale, refresh

    $o_rs =& $o_conn->Execute(SQL);

    if ($o_rs) {

        $a_data = $o_rs->GetArray();

        $o_data->SetCache(DATA_ID, $a_data);

    } else {

        trigger_error('A database error has occurred: '.

            $o_conn->ErrorMsg());        //use old data instead

        $a_data = $o_data->GetCache(DATA_ID);

    }

}

// $a_data is now valid from either path 

This was my first attempt at implementing DAOs (for which I have one File based DAO and one PostgreSQL base DAO) and any comments on these would be welcome.

SourceForge's file upload was broken, so I will attached a tarball to this post for people interested in the source.

Thanks.

[mlemos]

The problem of many data cache solutions including yours is that they are not robust enough to work in a multi-user environment. What happens is that if two users try simultaneously to access the cache this is outdated or non-existing, chances are that both will corrupt the data that each other tries to store in the cache.

The correct solution employs locking schemes that prevent more than one cache writer or that the cache is updated while there are any cache reader processes.

You may want to look at this file data cache class that uses file locks to prevent the problems above.

[ZangBunny]

You may want to look at this file data cache class that uses file locks to prevent the problems above.

That class is listed at phpclasses.org, but it's not written in PHP but in "MetaL", so I guess it's kinda off-topic for this forum?

[sweatje]

Originally Posted by mlemos

You may want to look at this file data cache class that uses file locks to prevent the problems above.

The code I posted already had file locks in the DataCacheDaoFile.php :

/**

 * cache a particular piece of data

 *

 * @param       string  $psId           identifier for the data being cached

 * @param       mixed   $pmValue        value to cache

 * @param       boolean $pbZip          optional - compress the data, defaults to true

 * @return      boolean                 sucess

 */

function SetCache($psId, $pmValue, $pbZip=true)

{

    $s_file = $this->_GetFileName($psId);

    $s_template = ($pbZip) ? DATACACHE_ZIP_TEMPLATE : DATACACHE_TEMPLATE;

    $s_data = ($pbZip)      ? base64_encode(gzcompress(serialize($pmValue)))

                : addslashes(serialize($pmValue));

    umask(0117);

    $r_fh = fopen($s_file, 'w');

    flock($r_fh, LOCK_EX);  //here

    $b_ret = fwrite($r_fh,

            sprintf($s_template, $s_data)

            );

    flock($r_fh, LOCK_UN);  //and here

    fclose($r_fh);

    return $b_ret;

} 

Or did I misunderstand you comment?

[mlemos]

Originally Posted by ZangBunny

That class is listed at phpclasses.org, but it's not written in PHP but in "MetaL", so I guess it's kinda off-topic for this forum?

Duh? What is the relevance of that? Isn't the class made of PHP code? Don't it do proper file locking unlike many other that are not suitable for use in busy sites?

What is your point? Censorship?!?!

[mlemos]

Originally Posted by sweatje

Or did I misunderstand you comment?

You are only trying to address part of the problem. The verify/update/retrieve cycle should be atomic.

If you verify that the cache needs to be update, you should lock it from update before you start writing to it. Your code has several problems.

Your verification code consists just of a file_exists() call. Two simultaneous processes can verify that the cache needs to be updated and so both will try to write it.

Furthermore, if for some reason your process exits during the cache update, it will leave a corrupt cache file that with a single file_exists() call will validate as if the cache file is not corrupt.

There are probably more problems that I did not analyse.

[ghurtado]

That class is listed at phpclasses.org, but it's not written in PHP but in "MetaL", so I guess it's kinda off-topic for this forum?

I followed the link, and looked at the source, and as far as I can tell it is all written in PHP. Am I Missing something?

[ZangBunny]

Originally Posted by mlemos

What is your point? Censorship?!?!

I will not have you or anyone accuse me of censorship.
I will not lead any discussion on this basis.

[ZangBunny]

Originally Posted by ghurtado

...as far as I can tell it is all written in PHP. Am I Missing something?

Originally Posted by phpclasses.org

The code and the documentation of this class were written in the MetaL meta-programming language.

The PHP code that can be downloaded (after registering on the site) is not "written" in PHP, but generated from "MetaL" code. To modify the class (should the author have overlooked anything despite his great wisdom), one would do that in the source code (i.e. in MetaL) which is a) not included in the package and b) in a language that not many PHP programmers (this is a PHP forum) will know.

I might have a look at this language when I have some time to kill, but after seeing several multi-million-dollar attempts at "meta-programming" fail, I beg to be allowed to have my doubts, preferably without being accused of censorship.

[mlemos]

Originally Posted by ZangBunny

The PHP code that can be downloaded (after registering on the site) is not "written" in PHP, but generated from "MetaL" code. To modify the class (should the author have overlooked anything despite his great wisdom), one would do that in the source code (i.e. in MetaL) which is a) not included in the package and b) in a language that not many PHP programmers (this is a PHP forum) will know.

I don't think you know how MetaL generated code works to make a qualified judgement.

MetaL is a PHP application but that is not relevant either. What is relevant that it can generate standalone PHP code that is exactly what I would produce if I would write it directly in PHP. This means that for those that use it, it is irrelevant how I created the class code. It also means that the generated code does not depend on any external libraries. Any PHP version would do.

Even if you want to change the code, nothing stops you for editing the class PHP code. The license is BSD and by that license you are free to modify and distribute modified versions of the class.

As a matter of fact I have received many suggestions to improve the class in the form changed versions of the class in PHP.

Anyway, I just mentioned the class as an example of doing proper cache locking to prevent access concorrency problems. I do not see how the access to the original MetaL source code would help, but if you think that is relevant, MetaL is a fully Open Source project and its source is available for download, including the MetaL source for the file cache class.

[sweatje]

Originally Posted by mlemos

Anyway, I just mentioned the class as an example of doing proper cache locking to prevent access concorrency problems.

I think you are correct that the locking code can be improved in the FileDAO. After this point however, I believe the intent of our two classes differ. Your class apears to deal directly with the file data itself as the cached data, whereas the intent of my FileDAO class is to create a new PHP script that can be included in the application to return the cached data in a native PHP format (array, object, etc.). Also, I included a Database based DAO (with intent to develop more) so that this datacache mechanism could work in a server farm environment. Moving a file system only based cache to a server farm would require some kind of a network mounted file system, that could introduce additional latency and locking issues in and of itself.

[mlemos]

Originally Posted by sweatje

I think you are correct that the locking code can be improved in the FileDAO. After this point however, I believe the intent of our two classes differ.

My point was not to compare classes but rather give you an example of how to do cache locking properly.

In the PHP Classes site there is a whole section for PHP cache classes but most of them have the same reentrancy problem as yours.

Originally Posted by sweatje

Your class apears to deal directly with the file data itself as the cached data, whereas the intent of my FileDAO class is to create a new PHP script that can be included in the application to return the cached data in a native PHP format (array, object, etc.).

That is not a limitation of my class. All you need to do is to [http://www.php.net/serialize]serialize()[/url] the variables that you want cache and pass the resulting string a data to the class. I do that sometimes.

However, better than caching variables eventually taken from database query results, I recommend that you cache the actual HTML pages or excerpts that you would generate from those variables values. This way, you also avoid recomputing the PHP code that generates that HTML.

Originally Posted by sweatje

Also, I included a Database based DAO (with intent to develop more) so that this datacache mechanism could work in a server farm environment. Moving a file system only based cache to a server farm would require some kind of a network mounted file system, that could introduce additional latency and locking issues in and of itself.

With databases you also have take care of locking. Usually this means using transactions.

Anyway, I don't see much sense in caching data in databases unless you are caching something much slower to compute than a database query. As a matter of fact, usually you use file or shared memory based caches to avoid database accesses.

I see your point regarding server farms but often is better to rely on the database server caches, like MySQL 4 has now.