Evaluating PHP Applications

Following on from here, perhaps the two most common questions I’ve seen people ask, when it comes to evaluating PHP applications are;

  • Does it loook good?
  • Is it easy to install?

Now not everyone is a programmer or a system administrator – “normal human beings” rank these highly because they relate directly to the two most pressing problems they’re facing: they want a site which is visually attractive and, with limited technical expertise, installation can be a significant hurdle to overcome.

But when it comes to security or maintenance, those requirements rank pretty low down. So here’s some different things to think about, following on from this talk (PDF) on page 19, which I’d argue rank much higher when evaluating a project you plan to use (further suggestions appreciated).

Note that in an ideal world you’d have time and expertise on hand to do a full code review but in reality that’s not going to happen so what I’m suggesting here is meant as a reasonable compromise to help you build up a “ballpark” feeling for an application without making a huge effort.

What’s the security record like? The obvious place to find out is via Google with some searches like “appName exploit”, “appName vulnerability”, “appName security”. A place to get a better impression is searching the Bugtraq mailing lists.

Of course you have to bear in mind that quality of information may vary – simply finding a random online opinion that “appName rox / sux” is not enough. Also newer or less popular applications won’t have attracted enough attention to form valid opinions this way. And you have to bear in mind that pretty much every application that’s been around and has real users will have problems at one time or other but comparing this to this, it’s easy to spot the difference.

As a side note there, I’d recommend registering on this mailing list – pretty much all security issues with well known (and less well known) PHP Open Source code bases get announced here.

What’s the code like? Although a complete code review is not realistic, with a little effort and knowhow, you can get a good idea of how the code smells.

Number 1 tool here is phpxref, which makes it very easy to identify use (or lack of) of PHP functions – run the source code through it that check the results. For example you might look for use of eval (and friends) – in general there’s zero valid reason to use eval so if you find it, query the developers on exactly why they used it. You might also find the absence of some functions indicators – if the app uses MySQL at the backend, do you find any of mysql_escape_string, mysql_real_escape_string or addslashes to escape parameters to SQL queries. Are htmlspecialchars or htmlentities being used to escape output? Is there any use of the PCRE or POSIX extended regular expressions functions for stuff like validation?

Otherwise, what does the code look like to you? This is highly subjective and depends on your experience but does it look “sane”?

How is the code being managed? Another area to investigate is how the project is actually run. How many people are involved and are they active? Do they have sense making release / upgrade policies – clear version numbering, good documentation on how to upgrade, are they using version control, what are their communication channels etc.?

Chris Kunz made a wry remark while giving this talk. He helps run a shared hosting company and pointed out most of their users were extremely happy when they could install an application in the first place – once installed there was no way they were going to risk breaking it with an upgrade.

As a user of an application, you have to be aware that it is really your responsibility to keep pace with new releases, especially when they contain bug or security fixes. As an example of a project that does a good job here, check out Serendipity’s upgrade docs. The question you need to ask yourself is “can I do this?”. You’re also going to need to make the effort to stay informed – subscribe to the relevant mailing list / RSS feed etc., so you hear about new releases.

Does it scale? More on the maintenance front, what’s the application like after you’ve been using it for a while and you’ve collected a volume of data and a crowd of active users? Can that forum cope with a large number of posts and concurrent users? How does that wiki handle a large number of documents? Is using the packaged RSS feed like volunteering for a DOS attack? How easy is it to backup / restore the data? Is a shared host account with nothing but FTP access adequate to maintain this application? Does the admin interface allow you to cope with 20,000 registered users?

Some of those kind of questions can be answered by talking to other users. Others can be determined by seeing what the developers are doing for example are they benchmarking / profiling their code?

Who’s using it? That Mediawiki is the code behind Wikipedia is obviously a very good indicator. Meanwhile Zend use fudforum. I’m not suggesting blindly following here BTW – the reasons for selection may not match situation (you could always ask) but this does serve as a useful indicator.

You should also be careful about “following the herd”. Just because “everyone” uses it, doesn’t always mean it’s the smartest choice. There may also be a specific benefit to not using the same as everyone else – big installed bases make attactive targets.

Who’s got an opinion? There are a lot of people “out there” with knowledge of PHP, so getting opinions isn’t a problem. At the same time, it’s worth considering where an opinion is coming from and bearing in mind it’s just an opinion. Sometimes even the most experienced disagree. So this path can be as misleading as it is useful but shouldn’t be ignored.

Anyway – that’s off the top of my head. Anything else?

Win an Annual Membership to Learnable,

SitePoint's Learning Platform

  • http://boyohazard.net Octal

    Great and informative post Harry, I can see me referring to this one time and again. Thank you.

  • http://diigital.com cranial-bore

    Good comments. Scanning for (in|ex)clusion of particular functions is a smart idea.
    Also is the code written for an old version of PHP?
    Does it depend on widely known bad practices such as register globals or magic quotes?
    Does the code use deprecitated functions in place of newer alternatives (e.g $HTTP_POST_FILES instead of $_FILES)

    btw, a lot is two words (last paragraph). Sorry, I am a picky bastard ;)

  • http://www.phppatterns.com HarryF

    Does the code use deprecitated functions in place of newer alternatives (e.g $HTTP_POST_FILES instead of $_FILES)

    Good point. Really need to do a long piece of using phpxref for doing basic analysis

    btw, a lot is two words (last paragraph).

    Thanks – fixed.

  • matthijsA

    Good write up Harry. In line of what Cranial-bore says:
    Does it depend on setting permissions for files and directories to 777?

  • willthiswork

    Also is the code written for an old version of PHP?
    Does it depend on widely known bad practices such as register globals or magic quotes?
    Does the code use deprecitated functions in place of newer alternatives (e.g $HTTP_POST_FILES instead of $_FILES)

    btw, a lot is two words (last paragraph). Sorry, I am a picky bastard ;)

    I guess phpxref is written in Perl (http://phpxref.sourceforge.net/faq.php)

    Does the code use deprecitated functions

    BTW, did you mean deprecated?

  • Anonymous

    > in general there’s zero valid reason to use eval

    Executing plugin/user-editable php-code, but preventing the page to “just die” if there is a parse error.

  • McGruff

    My first question would always be is it fully-tested OOP? I think testing is a good indicator of the developers’ ability. I’ll be looking for tests being used to drive the design and not just the odd unit test stuck on after the fact.

    Without tests I’m going to have some major maintenance problems. Right out the box I don’t know what the code is supposed to do or if it actually does it. If I make an edit how can I tell if it broke something? Unless you simply plan to install and use as is it’s effectively useless.

    Design wouldn’t necessarily matter so much to me. Not in an early version anyway: you kind of expect TDD’d code to be a bit rough at first. Provided the developers know how to knock it into shape that’s OK.

  • zonked

    Having recently survived the nightmare that is Drupal (mainly by dumping it completely for a home grown solution) I have to say that there are a few other red flags I’ve identified for PoS software:

    1. Assigning values to superglobals. If you see something like $_POST['foo'] = 'bar' run for your life. It makes absolutely no sense to ADD anything to the list of POST variables (which PHP packages for you in $_POST), therefore the whole program is probably horribly designed.
    2. ‘Magically’ assigned variables. Savant templates or whatnot, if you are working in a template and have no idea where certain variables are assigned….that’s really not a good thing for extenibility.
    3. No easy way to configure error handling. Does the software you want to use allow you to determine how errors are logged and where they’re logged to? It’s not that fun trying to track down a bug when the program uses output buffering so nothing is sent to STDOUT but also doesn’t leave a trace in syslog, error_log, …

  • willthiswork

    @cranial-bore

    I just realised I made non-sense irony. Sorry man. Very useful post and comments. That’s what I meant anyway.

  • Buddha443556

    After security, I am usually concerned about the footprint (memory and file I/O) per request. The bigger the footprint the more limited it’s usefulness on a shared server. Something those on dedicated servers may not even need to consider.

  • Peter

    Very good post. This is not only helpful too people who need to evaluate others code but is indeed good for programmers as well to ensure that they are covering all bases.

    1 point of contention I have tho is your comparison of phpbb and fudforum on the bugtraq site. lthough yes I agree phpBB has been plagued with bad coding and exploits I would just like to add if you compare how many people use phpBB and how many use fudforum you will see that phpBB is of far greater numbers. This being the case it should be a case that exploits are found much quicker. Just because fudforum only has had 1 exploit found does not mean others do not exist.

    Your comparison in a sense gives the wrong impression that if few bugs have been found then it must be better which quite frankly is not necessarily the case.

  • whacky2002

    Brilliant topic, nice discussion and comments, all being are very helpful!


    PHP Expert > Developer

  • http://www.phpsimplicity.com NeverMind

    1. Assigning values to superglobals. If you see something like $_POST['foo'] = ‘bar’ run for your life. It makes absolutely no sense to ADD anything to the list of POST variables (which PHP packages for you in $_POST), therefore the whole program is probably horribly designed.

    I wouldn’t agree here.. because sometimes you don’t want to initilize a new variable just because you are going to use it once and never use it again!
    I agree that creating a new element in a superglobal manually (e.g. $_POST['I_AM_NEW_NOT_POSTED'])is wrong but if you already have a variable (e.g. $_POST['foo']) I think it’s ok to manipulate it and reassign it to the same superglobal (i.e. $_POST['foo'])

  • Ren

    @zonked,

    Whats wrong with assigning things to superglobals? Absolutely nothing as far as I can see. Alot reassign variables to superglobals just to undo magic_quotes. I’d worry more about the superglobals being used directly in more than one class/file.

    As for deprecated functions, Reflection in 5.1.3 has a isDeprecated() method. So imagine tools could report if any are used.

    Its a shame about phpBB and the apparent attitude the developers have toward bugs. As judging from http://www.big-boards.com, its handling the scalability issue pretty well with half a billion posts, and 3.5 million members as the top rated board.

  • http://www.phpsimplicity.com NeverMind

    @Ren,
    I suppose you mean Gaia Online but I don’t have a second thought that this forum is heavily moded that it’s now maintained by the site team who update it as per their needs.

  • sectic

    Nice article.
    But I have one question regarding PHPXref and its interpretation. Do you think that it is enough for a first impression of the code to just check out if there where calls to function that generally sanitize user input to e.g. mysql-databases?
    I asked because I don’t see a way to get a greater impression of the code using PHPXref and this may – within evalution purposes where I can’t have a look at all parameters but only a few – be wrong as there are a number of parameters which might affect database calls.

    Relying on sanitizing functions only works if you really know, that every single parameter goes through these functions. And I think PHPXref (or any other automated software) doesn’t give you this possibility. So maybe its not that much worth in application’s evaluating that one may first think.

  • Ren

    @NeverMind,
    phpBB2 is GPL, so therefore all Gaia’s modifications should be available.

  • Pingback: Wisiwip » Blog Archive » Evaluating PHP Application

  • Jeewhizz

    As a side note there, I’d recommend registering on this mailing list—pretty much all security issues with well known (and less well known) PHP Open Source code bases get announced here.

    Shame really…

    phpsec-subscribe@phparch.com
    Delay reason: SMTP error from remote mail server after RCPT TO::
    host mail2.tabini.ca [72.51.34.155]: 450 :
    Recipient address rejected: User unknown in local recipient table

  • Pingback: PHPHound » Blog Archive » Evaluating PHP Applications and Scripts

  • malikyte

    Since Xdebug is probably one of the best profilers out there right now (with APD and DGB coming close), it’s a shame there are no tutorials on getting it to work on a Windows’ system within a PHP environment (i.e.: without using Wincachegrind). I can’t, for the life of me, figure out why my installation of it will not output any information when calling the internal PHP functions of the script…and there are no tutorials.

    Anyone for creating a simple tutorial on any profiler/debugger? :(

  • http://www.greenash.net.au/ Jaza

    @zonked: I’m sorry to hear that your experience with Drupal was a “nightmare”. I’ve been developing with Drupal for over a year now, and have found it to be an extremely well-built and extensible system.

    1. Assigning values to superglobals. If you see something like $_POST['foo'] = ‘bar’ run for your life. It makes absolutely no sense to ADD anything to the list of POST variables (which PHP packages for you in $_POST), therefore the whole program is probably horribly designed.

    Many users have already commented on how this technique is sometimes necessary and justified. Also, the latest version of Drupal (4.7 – currently in RC) has a new forms API, meaning that you very seldom need to access $_POST et al directly anymore.

    2. ‘Magically’ assigned variables. Savant templates or whatnot, if you are working in a template and have no idea where certain variables are assigned….that’s really not a good thing for extenibility.

    Template variables are all clearly defined in one function, using Drupal’s PHPTemplate theme system (other supported theme systems work similarly). I don’t see what’s so magical or mysterious about that.

    3. No easy way to configure error handling. Does the software you want to use allow you to determine how errors are logged and where they’re logged to? It’s not that fun trying to track down a bug when the program uses output buffering so nothing is sent to STDOUT but also doesn’t leave a trace in syslog, error_log, …

    Drupal does not use output buffering (under most circumstances). It also has a fairly good error handling system, called the ‘watchdog’, which logs all errors to a database table, and allows error messages to be browsed through an admin interface. So I don’t know what you’re getting at here.

    All up, I think your criticism of Drupal is unjustified. It has an excellent security record. It uses recommended validation techniques consistently. It is modular and stable. The code is well managed. It scales very well. And a number of large and high-profile sites are using it. It is generally considered a much more developer-friendly app than its competitors in the open-source PHP CMS market, such as Mambo, PHP-Nuke, and Xoops.

    I don’t mean to start a flame war, but I think people should hear both sides of the story.

  • http://doitslower.com/ lartexpert

    Manipulating the values of superglobals isn’t in and of itself a bad thing, but it tends to indicate that data isn’t being validated or having sanity checks applied to it before being used.

    If you’re accessing and using $_POST['somevar'] without checking and cleaning the value first, then you are asking for trouble. It is perfectly possible to write code without modifying the superglobals themselves – e.g.

    $cleanvars['sortorder'] = addslashes($_POST['sortorder']);

    You do all further evaluation using your value in $cleanvar – code that pumps superglobal values directly into databases, etc, is what gives rise to all the nasty SQL injection vulnerabilities that come up in popular forum apps all the time.

    Have a look at how Perl handles taint checking – it’s a good way to do things.

  • Pingback: SitePoint Blogs » Tomcat sucks… Is Apache flawed?

  • http://macosbrain.com macosbrain

    this is a very nice tool to find exploits by black-box-testing your php applications. just give it a try and you will love it(even if it is commercial).