I’m getting ready to do a few benchmarks for a project I’m working on (PHPRouter). I mainly use it for RESTful APIs. I don’t expect any crazy results, it will obviously have a little less overhead than a full fledged framework, but I want to put it up against ZF1, ZF2, CI and a few others. I’d like suggestions on other frameworks I should include and also, more importantly, what tool should I use to create these benchmarks? I know Apache has a benchmarking tool that I will be looking in to, should I use this (seems like the default)? I’m mainly after page loads per second.
There’s an app for t—err, I mean, a Symfony Component for that. 
But if instead you’re just interested in the final response time, then yes, the Apache Benchmark (ab) tool is perfectly good for that. Or if you want another choice to consider, then siege is a good tool also.
But be careful to compare apples with apples. Is your PHPRouter supposed to be a full-stack framework? If not, then it probably doesn’t make much sense to compare it against other full-stack frameworks. It may be more useful to compare against just the router component of those frameworks.
It isn’t, however I still want to show common frameworks vs PHPRouter vs no routing.
I am not an expert on this topic, but there’s one tiny advice I’d like to make, you better actually build a simple but useful application(maybe an e-commerce site) to run the benchmark tests. Some benchmarking articles use simple ‘hello world’ as the source for benchmarking test but the problem is, why do you even need PHP at all if all your program does is to print hello world on the screen(I can achieve this with HTML lol)? Some go a little bit further by executing several if conditions, loops or creating objects, even so they may be biased since in real applications you do not just create like say a loop that executes 1000 times that accomplish nothing. To produce a reasonable benchmarking result, you need to be a bit more realistic but creative.
Thanks for your input Hall of Famer. I will actually be using a “hello world” in json. The idea of this is to show the overhead difference between a full framework vs just the router for when you are creating something as simple as a RESTful api. Benchmarking how many requests you can complete in a period of x seconds with a hello world accomplishes quite a lot, actually. It is the easiest way to determine the true overhead of the framework / language. Many do the same (and rightfully so) when comparing web services across languages (go, python, node.js) because certain libraries such as an html or db package might be causing slow downs which can be addressed separately.
What you need to remember is that tools like “ab” and “siege” only give a static overview of the possible load.
The numbers you get from using them normally does not propagate to real world scenarios.
In your case I don’t think this matters, since you just want to “showcase” results vs. other frameworks, so its purely statistic and not for instance configuration.
Though, for anyone that want to test their new application/website, I would highly recommend one of the services which offer real traffic during the test (most just spawn small instances and then use test cases to do specific things on your system). As this will give you much more valuable information on how much your system can handle at the current server/cluster configuration.
I use Xdebug to create files for Wincachegrind to analyze those files and use it mainly to check for bottlenecks in my applications.
You might want to add Slim & Silex in the frameworks you compare against - these 2 are micro frameworks for making very lightweight apps & APIs - the real competition for what you’re making rather than full stack frameworks like ZF2, Symfony etc. Also, throw in Phalcon in the pot as well, would be interesting to see if you can beat it.
Ah nice. I had stumbled across Slim but wasn’t sure of how widely it was used.
I’ve looked into Slim a bit (haven’t made any app/API in it yet), haven’t looked into Silex. Not sure if there are any other micro frameworks in PHP.