PHPBench.com: Live PHP benchmarks, demystifying “best practices”
As a new contributor to the SitePoint blogs, I’ll be covering PHP web development, JavaScript and general web tech.
When it comes to optimizing PHP for performance, there’s no end of resources available, and no end of conflicting opinions either. Everyone seems to have their own approach to writing “fast” PHP code; using single quotes, avoiding require_once() and using isset() before is_array() are some of the most common. But with reliable benchmarks thin on the ground, how do we know if any of these techniques – often touted as “performance best practices” – actually deliver benefits? Chris Vincent’s new PHP benchmark suite at PHPBench.com aims to “set the record straight” on PHP performance techniques, with a simple, comprehensive view of how various approaches actually stack up.
The benchmark suite covers all the usual bases, taking a simple task — like iterating over an array — and speed testing almost every possible way to achieve it. Most importantly, however, Chris takes raw numbers out of the spotlight and instead focuses on how the options compare with each other. Each test takes the fastest technique as the base value for execution time; all the other options are measured as percentages in relation to this. For example, foreach($aHash as $val) has script execution time of 558% compared to while(list($key) = each($aHash)) $tmp[] = $aHash[$key] with 100%.
The test scores are also generated live; refreshing the page will produce a slightly different set of results. Chris uses the header of the relatively uncomplicated results page to recommend refreshing a few times, just to ensure consistency and to avoid a one-off impact on any particular test. Personally, I’d have preferred the tests be carried out in a controlled environment and regenerated every time a new test is added; anomalies don’t help anyone and live benchmarking offers little real benefit besides operational simplicity. The code for each test is supplied for transparency.
His current set of tests are very comprehensive; no less than 13 tests compared the performance of various echo and print calls, with enough variety to check for any usage scenario. He’s also accepting suggestions for new tests. Most importantly, however, many of his tests help identify the value of best practices. For example, PHP is often criticised for the vague results of == comparisons; the control structures tests show us that not only is === safer, it also takes about 2/3 as long to execute.
While most of the tests show trivial performance differences, there are some interesting results. For example, iterating over an array and modifying each element is 100 times as fast using while(list($key) = each($aHash)) than a simple foreach ($aHash as $key=>$val), while array_keys() is about 20 times slower than while(list($key,$val) = each($aHash));.
The next step, of course, would be to download the test suite to your local machine or server and run the tests yourself, to see how the factors of your production environment affect the results; Chris aims to make this possible in the near future. Many of the techniques listed are clearly going to be either memory-efficient or CPU cycle efficient, and the limits of your infrastructure will determine which way you want to go before scaling out. Nevertheless, managing bottlenecks in your code can really help you get the most out of your servers — after all, if you can achieve something 100 times as fast, why not? — and PHPBench.com could become the definitive reference on PHP performance.
Update: Many commenters have noted issues with the usefulness and reliability of the tests. It’s important to consider Chris’ intentions while creating the site, as he’s described in the comments. With all the “use this syntax because it’s slightly faster” posts, PHPBench.com was built to see if there is any real material benefit to using a particular syntax; most of the tests show that there clearly isn’t. I’ve done a few tests of my own and seen similar results, but if something looks amiss, feel free to point it out to Chris. The site is not trying to identify immaterial performance benefits from e.g. single quotes vs. double quotes. Moving further, PHP Bench could be used to measure algorithm performance and other more intensive procedures; for now, the basic syntax tests simply reflect the potential of the system. Hopefully this clears things up.
