I am frequently amazed and/or frustrated when I encounter online applications which have problems with Magic Quotes and string escaping. Even commercial PHP applications I use on a daily basis have such problems. The most common symptom is that slashes (/) end up appearing throughout content. For example:
What\’s the slash for?
However, there are other, less obvious problems caused by bad handling of quotes and escaping that can be more serious. Not escaping code that is sent in a database query can, of course, equate to an SQL injection vulnerability, one of the most common and more severe security holes in PHP applications.
For the uninitiated, Magic Quotes is a feature of PHP that can automatically escape strings that are input to PHP. For example, quotes (’ or “) are escaped
(\’ or \”). Other characters such as NULL characters and backslashes are also escaped.
I believe that the introduction of Magic Quotes was a mistake made by the developers of PHP – a mistake that developers are still paying for. While many PHP installations now have turned off all Magic Quotes functionality, some PHP installations still enable them for backwards-compatibility, so the problems persist.
So, what exactly are my problems with Magic Quotes, anyway?
The Name
The name ‘Magic Quotes’ is hardly self-explanatory. ‘Magic’ doesn’t help describe its meaning at all, and the ‘quotes’ part is misleading. A more appropriate name for this troublesome feature would be ‘Automatic String Escaping’. It’s not just quotes that are escaped by the process.
Magic Quotes were intended to make scripts safer for beginners. However, the name ‘Magic Quotes’ is mystifying to beginners. Its existence also helps prevent beginners from learning why strings must be escaped in SQL.
Different Types
As if it wasn’t confusing enough, there are three different types of ‘Magic Quotes’ settings in PHP, described in the PHP manual.
The most common type is magic_quotes_gpc, which adds slashes to variables submitted to the PHP application via GET, POST or a cookie.
Application Developers Need to Know if it’s Turned On
The fact that the Magic Quotes settings can be turned on and off is a major problem. If it were either always on or always off, the problem would not be as severe.
It is crucial for application developers to know whether magic_quotes_gpc or magic_quotes_runtime are turned on when coding. If magic_quotes_gpc is turned on, and a programmer assumes it is turned off and does the string escaping himself, he will end up with the dreaded ‘extra slashes’ in his content. If magic_quotes_gpc is off, and the programmer assumes it is turned on, he leaves himself vulnerable to SQL injections.
There is no way to disable magic_quotes_gpc in a script, because the damage has already been done. However, it is possible to detect whether it is turned on or off and act accordingly. In a previous blog post (as well as in his book), Harry describes a way for developers to create code that works whether magic_quotes_gpc is turned on or off. This code can be inserted somewhere in your application (near the start):
// Is magic quotes on?
if (get_magic_quotes_gpc()) {
// Yes? Strip the added slashes
$_REQUEST = array_map('stripslashes', $_REQUEST);
$_GET = array_map('stripslashes', $_GET);
$_POST = array_map('stripslashes', $_POST);
$_COOKIE = array_map('stripslashes', $_COOKIE);
}
Rather than increasing security or simlifying the process of escaping strings for use in SQL commands, Magic Quotes gives developers additional headaches and can, in some situations, create annoying bugs or cause security vulnerabilities in applications when the script is run on a PHP installation which uses a different setting than the developer expected.
In this article, Harry Fuecks examines the problem in depth. In a glorious irony, the application hosting the article has its own problems related to Magic Quotes – the slashes in the code samples (see for example the green text) are missing!
For now, I would recommend that developers do their own string escaping, as if magic quotes is off, and use the above code to strip the additional slashes resulting from a PHP installation where magic quotes is on.
A little more in-depth treatment of this topic, including code to detect and reverse the effects of all three magic_quotes settings, can be found here:
http://education.nyphp.org/phundamentals/PH_storingretrieving.php
March 2nd, 2005 at 9:41 pm
Yes Magic Quotes are evil :)
The example code introduces nighmares for using any library that may interact with input variables (which is not usually a good idea anyway).
It’s far better to add this to the start of the application, than even bothering trying to deal with them..
if (get_magic_quotes_gpc()) {
trigger_error(”Turn of magic quotes in php.ini / .htaccess or apache config”, E_USER_ERROR);
}
March 3rd, 2005 at 12:08 am
The library thing is the killer as far as I’m concerned. Basically, the magic quotes issue makes it all but impossible to write code for other people to re-use (unless the code has no interactions at all with the outside world). If you write it expecting a specific setting for magic_quotes and the end user has a different setting you’ll get horrible problems.
Avoiding user input isn’t a very pretty option either, since your library functions may be passed strings from input by the user which are in an “unknown” state – they might be escaped, they might not be. Alan’s suggestion of dying if magic qutoes are on isn’t a terrible idea, but if a user already has a large code base that expects the feature to be on they will be unable to use your library without a major rewrite (although maybe that’s not a bad thing). That said, many users on shared hosting don’t have access to php.ini OR htaccess files, and it’s a little harsh expecting them to change hosts just to reuse your code.
The most annoying thing about this all is that magic quotes is actually a very poor solution to the database escaping problem. Firstly, different databases have different escaping rules (I think SQL server requires quotes to be doubled up rather than backslash escaped). Secondly, a far, far more reliable way of safely escaping database variables used be pretty much databases access libraries for loads of other languages is to use something like this:
$query = sql_query(”select * from table where tag = ? and section = ?”, $tag, $section);
In magic quotes defence, if the above were used there would always be utterly clueless newbies who still stuck everything together using string concatenation and opened them up to vulnerabilities, but a decent sized warning against this on the manual page for the function would probably be enough to save all but the most hopeless of cases.
This turned in to a bit of a rant, but magic quotes is one of the principle things that turned me away from PHP for large web application development (I use Python now) so it’s something of a pet peeve!
March 3rd, 2005 at 3:39 am
Skunk, at http://www.php.net/mysql_real_escape_string#AEN89967 there is an example that does exactly what you suggested. Another (more tedious but faster) way would be formatting any data type manually with sprintf and mysql_real_escape_string.
March 3rd, 2005 at 4:01 am
Haha, I remember those days where I had a truckload of addslashes / stripslashes all through my code — so many it’d make me cry.
Now I just always detect if magic_quotes_gpc is turned on and stripslashes if necessary. I am recommending this to everyone writing PHP and hopefully it’ll become a sort of unwritten standard some day.
PS: You have a very clever method to use array_map, it never occured to me. :)
March 3rd, 2005 at 5:20 am
I totally agree, this is one of the very “messy” parts of PHP. Personnally I never uses addslashes() directly anymore, I use
function magicaddslashes($t) {
return (get_magic_quotes_gpc()==1) ? $t : addslashes($t);
}
but array_map() is a nice trick.
March 3rd, 2005 at 6:38 am
magic_quotes was a poor method of solving the problem.
Finally PDO is here/coming, which does do parameterised queries.
$query = $pdo->prepare("select * from table where tag = ? and section = ?"); $query->bindParam(1, $tag, PDO_PARAM_STR); $query->bindParam(2, $section, PDO_PARAM_STR);PS. array_map() implementation is buggy if you use [] in form elements, as it doesnt recurse correctly.
function array_map_recursive($callback, $array) { $r = array(); if (is_array($array)) foreach($array as $key => $value) $r[$key] = is_scalar($value) ? $callback($value) : array_map_recursive($callback, $value); return $r; } function array_stripslashes($array) { return array_map_recursive('stripslashes', $array); } if (get_magic_quotes_gpc()) { $_GET = array_stripslashes($_GET); $_POST = array_stripslashes($_POST); $_COOKIE = array_stripslashes($_COOKIE); }March 3rd, 2005 at 7:03 am
That’s a good point, Ren.
It looks like PHP 5 has an array_walk_recursive function which can accomplish this, and the PEAR package PHP_Compat contains a PHP 4 version.
March 3rd, 2005 at 7:12 am
Magicquotes to me is no big deal, you just have to be sure to have a little chunk of code in a global file you include on all pages and you’re fine :) It’s part of the standard global file I use on all my applications.
March 3rd, 2005 at 9:59 am
array_map() isn’t buggy. That’s how it’s supposed to work. The function you want should rather be called array_map_recursive() or tree_map().
March 3rd, 2005 at 12:30 pm
It is all fair and well the PHP developers going ahead and putting in “Magic Quotes”, which as a dumb idea on it’s own. What’s even ***DUMBER*** is that they don’t even escape the quotes to the ANSI standard. The ANSI standard says that quotes should be paired together (str_replace(”\’”, “\’\'”, $string)), not addslashed.
Of course, MySQL being querky thing it is didn’t support the ANSI standards so you were forced to using backticks, and silly quoting till recently.
Thankfully, with Creole and PDO, we no longer have these issues in modern PHP coding
March 3rd, 2005 at 12:57 pm
Like MiiJay said, backslashes are not the way strings are escaped in standard SQL. MySQL and PostgreSQL both allow this escaping, but SQLite (last I looked) does not; which means that people who rely on magic quoting will leave themselves open to SQL injection bugs when using SQLite.
March 3rd, 2005 at 1:12 pm
md2perpe – Yep, I pass a recursive stripslashes function to it :) Works a charm!
March 3rd, 2005 at 1:42 pm
What about mysql_escape_string() ???
March 3rd, 2005 at 3:24 pm
I didnt say array_map() was buggy.
I said using array_map with form elements with [] was buggy.
March 3rd, 2005 at 7:00 pm
On webmasterstop.com, its not the backend is having problems with quotes – the import from the old CMS seems to have killed it :)
March 4th, 2005 at 10:32 am
I think the PHP setting magic_quotes is in fact a useful setting: in certain circumstances. For the sakes of portability, it can be quite annoying (though solutions are generally simple, as shown) but when the project is one dedicated website, I would assume that the magic_quotes code is faster than PHP-based implementations of the same functionality. So, it could be a perfomance boost for some who wish to addslashes() (or similar) to all incoming GET/POST/COOKIE variables.
I think it should be turned off by default, and many of the problems that people have experienced would have not occured. It has to be said that I myself have seen this setting save many potentially vulnerable applications – well-written or not.
March 7th, 2005 at 3:56 am
Thanks for the info…..very interesting to one new to PHP development. I’ve already run into the problems.
Ed Clark
March 10th, 2005 at 1:34 pm
Ok, so if I understand correctly: what you do with the function mentioned above (if magic-quotes is enabled, stripslashes) is strip all slashes. But if data from for example user input from a form goes in to a database, it has to be escaped, right? So then, you’ll still have to do addslashes to the data. If you would forget that, you would be in trouble.
Wouldn’t it be more logical to turn around the function: if magic-quotes is off, then addslashes? That way, you’re sure data is allways escaped.
Ok, I know I’m probably not correct. I’m just hoping for some more insight here! Thanx.
March 11th, 2005 at 4:19 am
To clarify my comment/question a bit: if all slashes are removed, what’s the best way to escape data before entering it into a database? In Harry Fuechs book, there is a chapter about escaping data. However, in all following examples there’s no escaping done. (i’m starting to develop a headache here ;)
March 11th, 2005 at 5:31 am
matthijsA, your idea would certainly work and would be useful when the script itself is written with the assumption that magic_quotes_gpc will be on. However, if the script is written with the assumption that magic_quotes_gpc will be off, then the method of Harry’s that I posted would be appropriate.
Being that magic_quotes_gpc is off by default on current versions of PHP, it is generally a better idea to code assuming that magic_quotes_gpc is off, and use addslashes whenever sending a string to the database.
March 15th, 2005 at 12:49 am
I’ve realized that putting our own code is the best. Since all my DB uses mySQL, I’m fine with magic quotes. However, we must follow the #1 rule in PHP Security: DO NOT TRUST THE INPUT. And the input, in this case, is get_magic_quotes.
I add the code to my prepend config. I could port it to postgresql, etc.
So in the end, don’t make such fuzz about it. Just treat it as another config parameter that MUST be preset by YOU, the programmer, and then forget about it.
March 17th, 2005 at 12:39 am
Read: http://php.net/manual/en/security.magicquotes.php
March 25th, 2005 at 7:32 pm
I’ve had problems with using array_map() on form input; instead, I use this method:
function add_magic_quotes($array) { foreach ($array as $k => $v) { if (is_array($v)) { $array[$k] = add_magic_quotes($v); } else { $array[$k] = addslashes($v); } } return $array; } if (!get_magic_quotes_gpc()) { $_GET = add_magic_quotes($_GET); $_POST = add_magic_quotes($_POST); $_COOKIE = add_magic_quotes($_COOKIE); }April 6th, 2005 at 1:14 am
From PHP Manual recursive stripslashes code:
NoOne,
July 11th, 2006 at 2:15 pm
From PHP Manual recursive stripslashes code:
if (get_magic_quotes_gpc()) { function stripslashes_deep($value) { $value = is_array($value) ? array_map('stripslashes_deep', $value) : stripslashes($value); return $value; } $_POST = array_map('stripslashes_deep', $_POST); $_GET = array_map('stripslashes_deep', $_GET); $_COOKIE = array_map('stripslashes_deep', $_COOKIE); }NoOne,
July 11th, 2006 at 2:19 pm
Stellar! Saved my ass, thanks for the stripslashes code.
March 22nd, 2007 at 2:30 pm
PHP is one of the most incompetently designed languages ever. Sad but true.
March 27th, 2007 at 5:19 pm
Man I’ve run into major problems with magic quotes, thanks to the comments above from Ren its all fixed!
Thanks…
Man I can’t wait to get my head around JBoss Seam – I’ve always tried to not do anything in PHP but this project needed to be producing results straight away and I had no choice but to start using PHP :(
July 9th, 2007 at 4:42 pm
A direct quote from PHP.net:
“An example use of stripslashes() is when the PHP directive magic_quotes_gpc is on (it’s on by default), and you aren’t inserting this data into a place (such as a database) that requires escaping. For example, if you’re simply outputting data straight from an HTML form.”
– http://us3.php.net/stripslashes
NOTICE: magic_quotes_gpc is on BY DEFAULT!!!! A word of advice, do not make assumptions, check your PHP.ini using phpinfo() to determine your PHP configuration. Site Admins aren’t necessarily PHP pros, and therefore may not understand PHP directives, and unwittingly make alterations to a default PHP.ini. As a developer every site is unique, look at this site, for example, they explicitly ask that you escape a post to this forum. (smart or not) security through obscurity? Check your own input… don’t assume yours will behave the same as Joe’s or Eddie’s because they said thats how theirs was set up, so thats how yours will be set up. This is exactly my problem now, I was assuming my input was handled one way when in fact it was being handled another… didn’t notice it until someone asked me about a formatting problem. oops (take my own advise)
January 29th, 2008 at 8:42 am
Thanks for the quick fix it!
I really appreciate your time to share this valuable information.
[E]
October 18th, 2008 at 12:02 am