Unblocking Adblock

Tweet

If you’re using Firefox with the Adblock Plus extension installed … have you noticed anything different on sitepoint.com?

A couple of weeks ago I implemented a site-wide script that unblocks ads which were blocked by Adblock Plus.

See, a while ago, when the whole firefox is blocked debacle was raging, I made some hand-waving statement about how easy it would be to circumvent ad-blocking software (though of course, I wouldn’t say how). Then a little while later (not quite sure when … I was probably drunk) I lost a bet, and had to prove it. The stakes were high, and I felt a little dirty, but I’m a man of my word, so I had to do it. And as it happened, it turned into a fun and interesting challenge, with a relevant point to make …

To ad or not to ad

I’m no fan of ads myself, and I really don’t have an issue with people using ad-blocking software. I know there are some who think that blocking ads is tantamount to theft, but I think their arguments are entirely specious. Any part of the web is customisable for you — you can make sites have a high-contrast color scheme because you need that to read, or you can filter out all images because you don’t have the bandwidth to load them, or you can selectively remove particular types of content because you simply don’t want to see it. It’s all the same, and objecting to one is objecting to all of them (which is why the law would never be able to intervene — making the activity of ad-blocking illegal would make all user-modification illegal, including the use of assistive technologies, user CSS and user scripting, and other extensions).

In any case it isn’t really the ad-blocking that’s the issue, it’s business models that refuse to acknowledge human nature, and content providers who want to make money while pretending that’s not their motive. The hypochrisy of that is pretty insulting, in my view — if your content is free, make it free; if it’s not free then charge for it. But don’t make it superficially free and then try to make me feel guilty when it costs you something.

Anyway

That’s my opinion, but be that as it may, I do love to throw the odd cat among the pigeons! So I implemented the un-blocking script anyway just to throw some petrol on the debate, and remind anyone who’s forgotten of a simple truth: content control — and digital rights management in general — is an arms race, and nobody will ever win.

Here’s how it works

To begin with we need to establish that Adblock is actually running — we don’t want to run the script if it isn’t or we’ll end up duplicating all our ads. There used to be a Firefox bug that exposed information about installed extensions, but that’s long since fixed. So:

if(navigator.userAgent.toLowerCase().indexOf('firefox') == -1) { return; }

var self = this, tester = document.createElement('img');
document.getElementsByTagName('body').item(0).appendChild(tester);
tester.setAttribute('src', '/ads/draino.gif');
window.setTimeout(function()
{
	if(!tester || tester.style.display == 'none')
	{
		//adblock is running
	}
}, 10);

Here we’re assuming that Adblock’s filters will match a URI containing /ads, because if it does then we’ll never be able to load that image, and that’s how we know Adblock is running. One of the clever things about Adblock is the fact that it doesn’t actually remove things from the document at all — it never lets them get to the document in the first place. Adblock hooks into Firefox’s content policy API, which controls allowable content at the level of HTTP requests, which means that blocked content is never even requested. (The empty source element is also undisplayed with a dynamically-generated user stylesheet, which can’t be overriden by any styling on the document, because user stylesheets always have precedence).

Of course if the test image isn’t blocked, then the rest of the script won’t run, but given the obviousness of the target URI, it’s pretty likely that it will.

Once we’ve established that Adblock is running we can begin to look for blockable content. The script defines a subset of URI filters (similar to Adblock itself, but limited to the ads we know we’re running), and a list of elements to check (scripts, images and iframes). Each instance of a target element is tested against the filters to see if it might be blocked, and if so we remember its address.

Now we have a list of addresses, we need to be able to request that data manually, which we can do by calling a PHP script via XMLHttpRequest — the server can request data from any domain and then pass it back up to the client. However it isn’t quite that straightforward, because even XHR requests are affected by Adblock. We need to prevent our requests from being blocked, so to do that we encode the addresses so that the filters don’t pick them up. It doesn’t need to be fancy, so Rot13 encoding is quite sufficient, and we’ll end up making requests like this:

request.open('GET', 'draino.php?encuri=uggc://jjj.zlfvgr.pbz/nqf/cebzb.wf', true);

The PHP script decodes that URI and then grabs its contents using fopen() into a buffer (other approaches are possible, but this seemed like the most straightforward; note that the PHP script itself is validated to prevent injection attacks, and disallow requests that don’t come from specified hosts). Depending on what the ad code spits out, we might get JavaScript, HTML or image data in response.

If it’s HTML that would be because the original ad was an <iframe>, so in that case we simply change the iframe’s SRC to point to the PHP script, and we’re done:

<iframe src="draino.php?encuri=uggc://jjj.zlfvgr.pbz/nqf/cebzb.wf">

If it’s image data we base64 encode it, so that on the client side we can use the data to create a new image with a data URI, which has no external resource and therefore doesn’t get blocked:

var img = document.createElement('img');
img.setAttribute('src', 'data:image/gif;base64,' + request.responseText);
element.parentNode.insertBefore(img, element);

The element referred to in that code is the original ad — by inserting the new image directly before it, we ensure that it appears in the same place in the document as the original.

Finally, if the response is JavaScript, we need to parse and evaluate that code. Each ad will be different, so we basically need a different case for each provider. For example, if the provider returned HTML surrounded in a document.write statement, we would need to extract the output and direct it to a new element using innerHTML, then append that new element directly before the original ad script (just as with images, so that it appears in the same place):

var code = eval(request.responseText.replace('document.write', ''));
var span = document.createElement('span');
span.innerHTML = code;
element.parentNode.insertBefore(span, element);

And that’s how we roll!

Adblock revolves around URI filtering, so if we can fool it into letting us request disallowed data, we can simply add that data directly to the page, and then there’s nothing for Adblock to work with anymore. We’re not even distorting our ad statistics, because the original ad was never requested in the first place — ours is the only request that gets through!

The only thing we can’t handle is a script which generates an <iframe> which itself contains another blockable element. Theoretically we could drill down further and recursively unblock the contents of the iframe page … but that’s getting a bit ridiculous.

Blocking the unblocker

Yeah well, I never wanted to be evil (once you begin down the dark path, forever will it dominate your destiny). If you want to stop all of this from working you just have to block the unblocking script — simply add draino to your list of filters. And anyway, I’m going to remove it next week, it was just a bit of fun ;)

Unless I made that script unblockable too …

Free book: Jump Start HTML5 Basics

Grab a free copy of one our latest ebooks! Packed with hints and tips on HTML5's most powerful new features.

  • http://www.mikehealy.com.au cranial-bore

    Interesting read, I like the way you think, but as mentioned it is an arms race :)

  • anon

    I guess it doesn’t work if you have NoScript running?

  • http://thinkdrastic.net/ gnarly

    Clever stuff Mr. Cake.

    Of course, I was calling you every name under the sun up until I got to the last paragraph ;)

  • Dave

    Uh. Not working. Ads are still blocked for me with EasyList and EasyElement under FF3 beta.

  • hahaha not

    nice try.. would you like a boobie prize?

    I NEVER see ads ;p

  • Fox

    Here we’re assuming that Adblock’s filters will match a URI containing /ads, because if it does then we’ll never be able to load that image, and that’s how we know Adblock is running.

    You only know that ad was blocked.
    It can also be Kerio firewall or Opera content blocker, right?

    But anyway, i use Adblock Plus, but i don’t see any ads here:)

  • http://www.merhar.si/ jmerhar

    I find it quite funny that I don’t see the picture in this blog post, with my default AdBlock settings (no specific rules added for it). :)

    Besides, I think the ad provider would become pretty suspicious if the majority of the requests for the ads would suddenly start coming from one (i.e. the server’s) IP address.

  • Seantyr

    I still don’t see ads, and I’m running Firefox 2 with Ad Block Plus and with Javascript enabled. I still see the normal empty sponsored links places, but no ads.

    I do genuinely think SitePoint deserve my money from the advice I’ve been given here, so I buy books occassionally. To date I’ve never bought anything having clicked on an ad anyway, so the ‘reward’ loss to SitePoint will amount to a couple of dollars in lost pay-per-view clickthroughs, which you more than made up for in the cost of delivery of the books I bought, heh.

  • http://www.brothercake.com/ brothercake

    Let me just restate, in case anyone missed it – this is not a serious attempt to unblock ads, and the script will be gone tomorrow. It’s just a bit of fun.

  • Fox

    It never even worked:)

  • Hoq

    Never noticed. Most users running AdBlock also use NoScript.

    The reason why adblockers always win the arm race is because there are more of us than there are of you. I can simply blacklist your script in a fraction of the time it took for you to code and implement it, so the most I’ll ever see is one ad. I have this functionality because advertisers have pissed so many people off, a collective force of hundreds or thousands of coders consistently devote time to ensuring you don’t pester them (and everyone else).

  • http://www.igeek.info asp_funda

    @brothercake:
    Interesting theory & implementation, sounds very possible indeed. How about all these guys who say that they don’t see any ads despite the script? Is that script malfunction or just some possible exceptions as sometimes happens? I’d sure like to know your reply, this whole idea sure is interesting & food for (programmer’s)thought! :)

  • http://www.sitepoint.com AlexW

    Never noticed. Most users running AdBlock also use NoScript.

    I doubt very much that most Adblock users also run NoScript. In fact, I’d be surprised if as many as 1 in 20 did.

    The reason why adblockers always win the arm race is because there are more of us than there are of you. I can simply blacklist your script in a fraction of the time it took for you to code and implement it, so the most I’ll ever see is one ad. I have this functionality because advertisers have pissed so many people off, a collective force of hundreds or thousands of coders consistently devote time to ensuring you don’t pester them (and everyone else).

    I’d be careful about setting challenges like that. What if the script was randomly encoded each time the page loaded? It’s name and content are entirely different every time you reload and the inner working of the script are only the same once the script has decoded itself.

    Sure, you can manually block it again each time, but the programmer/user effort equation just veered back towards the programmer.

  • http://www.brothercake.com/ brothercake

    Interesting comments here.

    Firstly, if it wasn’t working for you there’s a few possibilities. The most likely is that in fact it is working, it’s just that you don’t know what you’re looking for; it doesn’t unblock *all* ads – as I said, some were too much trouble to bother with, and those tended to be the most obvious ones, like the banners at the top and the square in-article ads. Another possibility is that the test case didn’t match your filters, in which case sure, it won’t work, but if that’s the case then the original ads wouldn’t have been blocked either.

    Another possibility is that the filter list you’re using has already been updated to cater for this :)

    If you have Noscript, then yeah, the unblocker won’t work, and neither will the original ads. But again, this can be worked around on the server, by making requests for the JS content, parsing it and writing it to the page directly.

    The thing with blocking the unblocking script – the only reason you can block it is that I designed it to be blockable. I could have designed it to be unblockable, but I chose not to, because that was never the point of the excercise.

    Making it unblockable would be pretty simple. All I’d have to do is make all requests for ad content (including the unblocker script) point to root, and then use header information to discriminate what content is required. To block that the extension would need to be able to filter by header information .. but even that could be circumvented, by adding a degree of sessional or random data to the headers.

    Like I say, it’s an arms race – anything I can do, adblock can prevent; and anything it can prevent, I can circumvent. It goes on and on. The only guaranteed way of blocking content is to block an entire domain.

    Anyway the script is gone now – experiment over :)

  • http://www.igeek.info asp_funda

    @brothercake:
    Hmm, ok that can do well when ads are served by a site’s own ad program like phpAdsNew/OpenAds etc., the script can indeed be unblockable as Alex mentioned above by randomly encoding it & hiding it to fool the filters. But what if they are served by some 3rd party program like AdSense or TribalFusion etc. In that case the point raised by jmerhar seems to be valid that the ad network guys & advertiser will get suspicious as a lot of ad requests start coming from 1 IP & no clicks from there!! Won’t that become a point of concern?

  • RAMI7250

    can you simply give the code for unblocking adblock- for google adsense objects?
    cuz i didnt understand what to do for that issue.

    thanks on the help,
    rami