Magic Quotes Headaches

I am frequently amazed and/or frustrated when I encounter online applications which have problems with Magic Quotes and string escaping. Even commercial PHP applications I use on a daily basis have such problems. The most common symptom is that slashes (/) end up appearing throughout content. For example:

What’s the slash for?

However, there are other, less obvious problems caused by bad handling of quotes and escaping that can be more serious. Not escaping code that is sent in a database query can, of course, equate to an SQL injection vulnerability, one of the most common and more severe security holes in PHP applications.

For the uninitiated, Magic Quotes is a feature of PHP that can automatically escape strings that are input to PHP. For example, quotes (‘ or “) are escaped
(‘ or “). Other characters such as NULL characters and backslashes are also escaped.

I believe that the introduction of Magic Quotes was a mistake made by the developers of PHP – a mistake that developers are still paying for. While many PHP installations now have turned off all Magic Quotes functionality, some PHP installations still enable them for backwards-compatibility, so the problems persist.

So, what exactly are my problems with Magic Quotes, anyway?

The Name

The name ‘Magic Quotes’ is hardly self-explanatory. ‘Magic’ doesn’t help describe its meaning at all, and the ‘quotes’ part is misleading. A more appropriate name for this troublesome feature would be ‘Automatic String Escaping’. It’s not just quotes that are escaped by the process.

Magic Quotes were intended to make scripts safer for beginners. However, the name ‘Magic Quotes’ is mystifying to beginners. Its existence also helps prevent beginners from learning why strings must be escaped in SQL.

Different Types

As if it wasn’t confusing enough, there are three different types of ‘Magic Quotes’ settings in PHP, described in the PHP manual.

The most common type is magic_quotes_gpc, which adds slashes to variables submitted to the PHP application via GET, POST or a cookie.

Application Developers Need to Know if it’s Turned On

The fact that the Magic Quotes settings can be turned on and off is a major problem. If it were either always on or always off, the problem would not be as severe.

It is crucial for application developers to know whether magic_quotes_gpc or magic_quotes_runtime are turned on when coding. If magic_quotes_gpc is turned on, and a programmer assumes it is turned off and does the string escaping himself, he will end up with the dreaded ‘extra slashes’ in his content. If magic_quotes_gpc is off, and the programmer assumes it is turned on, he leaves himself vulnerable to SQL injections.

There is no way to disable magic_quotes_gpc in a script, because the damage has already been done. However, it is possible to detect whether it is turned on or off and act accordingly. In a previous blog post (as well as in his book), Harry describes a way for developers to create code that works whether magic_quotes_gpc is turned on or off. This code can be inserted somewhere in your application (near the start):


// Is magic quotes on?
if (get_magic_quotes_gpc()) {

// Yes? Strip the added slashes

$_REQUEST = array_map('stripslashes', $_REQUEST);
$_GET = array_map('stripslashes', $_GET);
$_POST = array_map('stripslashes', $_POST);
$_COOKIE = array_map('stripslashes', $_COOKIE);

}

Rather than increasing security or simlifying the process of escaping strings for use in SQL commands, Magic Quotes gives developers additional headaches and can, in some situations, create annoying bugs or cause security vulnerabilities in applications when the script is run on a PHP installation which uses a different setting than the developer expected.

In this article, Harry Fuecks examines the problem in depth. In a glorious irony, the application hosting the article has its own problems related to Magic Quotes – the slashes in the code samples (see for example the green text) are missing!

For now, I would recommend that developers do their own string escaping, as if magic quotes is off, and use the above code to strip the additional slashes resulting from a PHP installation where magic quotes is on.

Free book: Jump Start HTML5 Basics

Grab a free copy of one our latest ebooks! Packed with hints and tips on HTML5's most powerful new features.

  • Dan

    A little more in-depth treatment of this topic, including code to detect and reverse the effects of all three magic_quotes settings, can be found here:

    http://education.nyphp.org/phundamentals/PH_storingretrieving.php

  • Alan Knowles

    Yes Magic Quotes are evil :)
    The example code introduces nighmares for using any library that may interact with input variables (which is not usually a good idea anyway).

    It’s far better to add this to the start of the application, than even bothering trying to deal with them..
    if (get_magic_quotes_gpc()) {
    trigger_error(“Turn of magic quotes in php.ini / .htaccess or apache config”, E_USER_ERROR);
    }

  • http://simon.incutio.com/ Skunk

    The library thing is the killer as far as I’m concerned. Basically, the magic quotes issue makes it all but impossible to write code for other people to re-use (unless the code has no interactions at all with the outside world). If you write it expecting a specific setting for magic_quotes and the end user has a different setting you’ll get horrible problems.

    Avoiding user input isn’t a very pretty option either, since your library functions may be passed strings from input by the user which are in an “unknown” state – they might be escaped, they might not be. Alan’s suggestion of dying if magic qutoes are on isn’t a terrible idea, but if a user already has a large code base that expects the feature to be on they will be unable to use your library without a major rewrite (although maybe that’s not a bad thing). That said, many users on shared hosting don’t have access to php.ini OR htaccess files, and it’s a little harsh expecting them to change hosts just to reuse your code.

    The most annoying thing about this all is that magic quotes is actually a very poor solution to the database escaping problem. Firstly, different databases have different escaping rules (I think SQL server requires quotes to be doubled up rather than backslash escaped). Secondly, a far, far more reliable way of safely escaping database variables used be pretty much databases access libraries for loads of other languages is to use something like this:

    $query = sql_query(“select * from table where tag = ? and section = ?”, $tag, $section);

    In magic quotes defence, if the above were used there would always be utterly clueless newbies who still stuck everything together using string concatenation and opened them up to vulnerabilities, but a decent sized warning against this on the manual page for the function would probably be enough to save all but the most hopeless of cases.

    This turned in to a bit of a rant, but magic quotes is one of the principle things that turned me away from PHP for large web application development (I use Python now) so it’s something of a pet peeve!

  • http://www.phpism.net Maarten Manders

    Skunk, at http://www.php.net/mysql_real_escape_string#AEN89967 there is an example that does exactly what you suggested. Another (more tedious but faster) way would be formatting any data type manually with sprintf and mysql_real_escape_string.

  • http://www.jeroenmulder.com/ JMulder

    Haha, I remember those days where I had a truckload of addslashes / stripslashes all through my code — so many it’d make me cry.

    Now I just always detect if magic_quotes_gpc is turned on and stripslashes if necessary. I am recommending this to everyone writing PHP and hopefully it’ll become a sort of unwritten standard some day.

    PS: You have a very clever method to use array_map, it never occured to me. :)

  • Ned Baldessin

    I totally agree, this is one of the very “messy” parts of PHP. Personnally I never uses addslashes() directly anymore, I use
    function magicaddslashes($t) {
    return (get_magic_quotes_gpc()==1) ? $t : addslashes($t);
    }
    but array_map() is a nice trick.

  • Ren

    magic_quotes was a poor method of solving the problem.

    Finally PDO is here/coming, which does do parameterised queries.


    $query = $pdo->prepare("select * from table where tag = ? and section = ?");

    $query->bindParam(1, $tag, PDO_PARAM_STR);
    $query->bindParam(2, $section, PDO_PARAM_STR);

    PS. array_map() implementation is buggy if you use [] in form elements, as it doesnt recurse correctly.


    function array_map_recursive($callback, $array)
    {
    $r = array();
    if (is_array($array))
    foreach($array as $key => $value)
    $r[$key] = is_scalar($value) ? $callback($value) : array_map_recursive($callback, $value);
    return $r;
    }

    function array_stripslashes($array)
    {
    return array_map_recursive('stripslashes', $array);
    }

    if (get_magic_quotes_gpc())
    {
    $_GET = array_stripslashes($_GET);
    $_POST = array_stripslashes($_POST);
    $_COOKIE = array_stripslashes($_COOKIE);
    }

  • http://www.sitepoint.com/ mmj

    PS. array_map() implementation is buggy if you use [] in form elements, as it doesnt recurse correctly.

    That’s a good point, Ren.

    It looks like PHP 5 has an array_walk_recursive function which can accomplish this, and the PEAR package PHP_Compat contains a PHP 4 version.

  • http://www.deanclatworthy.com Dean C

    Magicquotes to me is no big deal, you just have to be sure to have a little chunk of code in a global file you include on all pages and you’re fine :) It’s part of the standard global file I use on all my applications.

  • md2perpe

    array_map() isn’t buggy. That’s how it’s supposed to work. The function you want should rather be called array_map_recursive() or tree_map().

  • MiiJaySung

    It is all fair and well the PHP developers going ahead and putting in “Magic Quotes”, which as a dumb idea on it’s own. What’s even ***DUMBER*** is that they don’t even escape the quotes to the ANSI standard. The ANSI standard says that quotes should be paired together (str_replace(“‘”, “””, $string)), not addslashed.

    Of course, MySQL being querky thing it is didn’t support the ANSI standards so you were forced to using backticks, and silly quoting till recently.

    Thankfully, with Creole and PDO, we no longer have these issues in modern PHP coding

  • Ian Bicking

    Like MiiJay said, backslashes are not the way strings are escaped in standard SQL. MySQL and PostgreSQL both allow this escaping, but SQLite (last I looked) does not; which means that people who rely on magic quoting will leave themselves open to SQL injection bugs when using SQLite.

  • http://www.deanclatworthy.com Dean C

    md2perpe – Yep, I pass a recursive stripslashes function to it :) Works a charm!

  • dusoft

    What about mysql_escape_string() ???

  • Ren

    I didnt say array_map() was buggy.

    I said using array_map with form elements with [] was buggy.

  • http://www.enthropia.com AhmedF

    On webmasterstop.com, its not the backend is having problems with quotes – the import from the old CMS seems to have killed it :)

  • Anonymous

    I think the PHP setting magic_quotes is in fact a useful setting: in certain circumstances. For the sakes of portability, it can be quite annoying (though solutions are generally simple, as shown) but when the project is one dedicated website, I would assume that the magic_quotes code is faster than PHP-based implementations of the same functionality. So, it could be a perfomance boost for some who wish to addslashes() (or similar) to all incoming GET/POST/COOKIE variables.

    I think it should be turned off by default, and many of the problems that people have experienced would have not occured. It has to be said that I myself have seen this setting save many potentially vulnerable applications – well-written or not.

  • fabeddie

    Thanks for the info…..very interesting to one new to PHP development. I’ve already run into the problems.
    Ed Clark

  • matthijsA

    Ok, so if I understand correctly: what you do with the function mentioned above (if magic-quotes is enabled, stripslashes) is strip all slashes. But if data from for example user input from a form goes in to a database, it has to be escaped, right? So then, you’ll still have to do addslashes to the data. If you would forget that, you would be in trouble.
    Wouldn’t it be more logical to turn around the function: if magic-quotes is off, then addslashes? That way, you’re sure data is allways escaped.
    Ok, I know I’m probably not correct. I’m just hoping for some more insight here! Thanx.

  • matthijsA

    To clarify my comment/question a bit: if all slashes are removed, what’s the best way to escape data before entering it into a database? In Harry Fuechs book, there is a chapter about escaping data. However, in all following examples there’s no escaping done. (i’m starting to develop a headache here ;)

  • http://www.sitepoint.com/ mmj

    Wouldn’t it be more logical to turn around the function: if magic-quotes is off, then addslashes?

    matthijsA, your idea would certainly work and would be useful when the script itself is written with the assumption that magic_quotes_gpc will be on. However, if the script is written with the assumption that magic_quotes_gpc will be off, then the method of Harry’s that I posted would be appropriate.

    Being that magic_quotes_gpc is off by default on current versions of PHP, it is generally a better idea to code assuming that magic_quotes_gpc is off, and use addslashes whenever sending a string to the database.

  • Rick

    I’ve realized that putting our own code is the best. Since all my DB uses mySQL, I’m fine with magic quotes. However, we must follow the #1 rule in PHP Security: DO NOT TRUST THE INPUT. And the input, in this case, is get_magic_quotes.

    I add the code to my prepend config. I could port it to postgresql, etc.

    So in the end, don’t make such fuzz about it. Just treat it as another config parameter that MUST be preset by YOU, the programmer, and then forget about it.

  • philip
  • rob

    I’ve had problems with using array_map() on form input; instead, I use this method:


    function add_magic_quotes($array) {
    foreach ($array as $k => $v) {
    if (is_array($v)) {
    $array[$k] = add_magic_quotes($v);
    } else {
    $array[$k] = addslashes($v);
    }
    }
    return $array;
    }
    if (!get_magic_quotes_gpc()) {
    $_GET = add_magic_quotes($_GET);
    $_POST = add_magic_quotes($_POST);
    $_COOKIE = add_magic_quotes($_COOKIE);
    }

  • NoOne

    From PHP Manual recursive stripslashes code:

    NoOne,

  • NoOne

    From PHP Manual recursive stripslashes code:

    if (get_magic_quotes_gpc()) {
    function stripslashes_deep($value)
    {
    $value = is_array($value) ?
    array_map('stripslashes_deep', $value) :
    stripslashes($value);

    return $value;
    }

    $_POST = array_map('stripslashes_deep', $_POST);
    $_GET = array_map('stripslashes_deep', $_GET);
    $_COOKIE = array_map('stripslashes_deep', $_COOKIE);
    }

    NoOne,

  • John

    Stellar! Saved my ass, thanks for the stripslashes code.

  • Dan

    PHP is one of the most incompetently designed languages ever. Sad but true.

  • SoreGums

    Man I’ve run into major problems with magic quotes, thanks to the comments above from Ren its all fixed!

    Thanks…

    Man I can’t wait to get my head around JBoss Seam – I’ve always tried to not do anything in PHP but this project needed to be producing results straight away and I had no choice but to start using PHP :(

  • phpdevel

    A direct quote from PHP.net:
    “An example use of stripslashes() is when the PHP directive magic_quotes_gpc is on (it’s on by default), and you aren’t inserting this data into a place (such as a database) that requires escaping. For example, if you’re simply outputting data straight from an HTML form.”
    http://us3.php.net/stripslashes

    NOTICE: magic_quotes_gpc is on BY DEFAULT!!!! A word of advice, do not make assumptions, check your PHP.ini using phpinfo() to determine your PHP configuration. Site Admins aren’t necessarily PHP pros, and therefore may not understand PHP directives, and unwittingly make alterations to a default PHP.ini. As a developer every site is unique, look at this site, for example, they explicitly ask that you escape a post to this forum. (smart or not) security through obscurity? Check your own input… don’t assume yours will behave the same as Joe’s or Eddie’s because they said thats how theirs was set up, so thats how yours will be set up. This is exactly my problem now, I was assuming my input was handled one way when in fact it was being handled another… didn’t notice it until someone asked me about a formatting problem. oops (take my own advise)

  • eddie thieda

    Thanks for the quick fix it!

    I really appreciate your time to share this valuable information.

    [E]