Beat Any Website into Shape with Greasemonkey

Greasemonkey is a Firefox extension by Aaron Boodman, which allows you to run personal DHTML scripts (known as “user scripts”) on any Website. User scripts are essentially bookmarklets that run automatically, but because they do, they’re a whole lot more useful.

Ever used a site that doesn’t work in Firefox, even though it could with just a few minor tweaks? Ever frustrated by a site you really like apart from one annoying thing it does, or felt the desire to customise sites you visit often? Just as user CSS allows you to change the way Websites appear, so user JS allows you change the way Websites behave.

This article looks at user scripting in general, and Greasemonkey in particular, to explore some possible applications, and discuss any issues that arise.
We’ll go from simple to increasingly complex examples, and I’ll be assuming you’re already familiar with JavaScript and the DOM; but all except one are working user scripts, so you can install and use them whatever your JavaScript skills. You will of course need Firefox and the Greasemonkey extension; if you don’t already have them, you can download them from the following links:

• Get Firefox
• Get Greasemonkey

You might also want to download the code archive for this article — it contains all the scripts we’ll use here.

Before we start, I want to take a brief overview of how to install and write user scripts for Greasemonkey; if you’re already familiar with the extension, you may want to skip straight to the section headed Putting it to use.

Installing User Scripts

Greasemonkey user scripts are .js files with the name convention “scriptname.user.js”. You can install the scripts in a couple of ways:

• Right-click on a link and select Install User Script… from the context menu.
• View the script in Firefox and select Install User Script… from the Tools menu.

Greasemonkey doesn’t have an interface that allows you to edit the source of user scripts directly, so if it has any optional settings or you want to change it at all, you will need to download and edit the script locally, before installing it in the normal way.

Writing User Scripts

Greasemonkey user scripts run on the document once the DOM is ready, but before the onload event. They have no special rules, but it’s considered best-practice to wrap them in anonymous functions, so they don’t interfere with other scripting:

(function() 

{ 

  ... scripting ... 

 

})();

Apart from that, I have one suggestion to make for writing user scripts: wherever possible, use only well-supported DOM scripting techniques.

Specifically, I try to avoid technologies like XPath, which are not very widely supported and, likewise, non-standard properties like document.body and innerHTML, which may not exist in XHTML mode (on pages served as application/xhtml+xml or equivalent).

These guidelines may seem pointless since we’re only writing for Firefox, but we’ll improve the chances of wider compatibility this way (Firefox is not the only browser in which user scripting is available; but we’ll talk more about that later).

Meta-data Comments

Greasemonkey has a simple comments syntax, which is used to define the sites that a script should or shouldn’t run on, the script’s name, it’s description and namespace (a URI namespace as with XML, in which the name of the script must be unique). Here’s a summary:

// ==UserScript== 

// @name            Script name 

// @namespace       https://www.sitepoint.com/userscripts/ 

// @description     Brief description of script 

// @include         http://google.com/* 

// @exclude         http://msn.com/* 

// ==/UserScript==

The syntax is pretty self-explanatory, but the Greasemonkey authoring guide goes into more detail.

Putting it to Use

The development of Greasemonkey was heavily inspired by Adrian Holovaty’s site-specific extension for All Music Guide, which is designed to fix what many people consider are serious problems with the new site design. The developer’s aim was to make it as easy to write a site-specific extension as it is to write DHTML. And so he has.

Simon Willison’s Fixing MSDN with Greasemonkey was among the first to make popular use of Greasemonkey, presenting a script that reveals information that would otherwise only be visible to IE, while Mihai Parparita’s Adding Persistent Searches to Gmail is by far the most sophisticated example I’ve seen to date.

But we’re going to dive in with a few simple site enhancements, to introduce the possibilities, as well as the practicalities of user scripting. I made a blank template for Greasemonkey scripts that you may find useful; it also includes some handy methods.

Customising Sites you Visit Often

Auto-complete Forms

Our first example auto-completes a single form element on a specific page. In this case, the page is the SitePoint Forum homepage, and the value is my username:

View or install the script auto-complete.user.js from the code archive.

Even though the forum remembers me with a cookie, I might not have it, since I clear all my cookies quite often. There are plenty of other times where this could come in handy: a cookie might have expired, or a site might have more basic functionality that doesn’t remember people.

Let’s extend the principle by adding another site: the Yahoo! mail login page. First we would add the @include path to the top of the comments section:

// @include         http://mail.yahoo.com/

We then add the appropriate scripting; in this case, the login field doesn’t have an id, so we’ll have to find it through document.forms:

//look for the yahoo mail login form 

var yahoo = typeof document.forms['login_form'] != 'undefined' 

  ? document.forms['login_form']['login'] : null; 

if(yahoo != null) 

{ 

  //write your username to the field 

  yahoo.value = 'brothercake'; 

}

You can add further sites as you like, defining for each an @include domain, and a chunk of code to find and complete its form field (changing the username to yours, obviously!).

Change the Layout

A user script can do much more than write a single element: it could rewrite the entire DOM of any page, making it look and behave exactly the way you want. In this next example, I’m going to modify the default Slashdot front page. I’ll remove the ad frame from the top and replace it with a duplicate of the site-search form, which is otherwise displayed only at the bottom:

View or install the script slashdot-restructure.user.js from the code archive.

As with the auto-complete example, or, for that matter, any user script that modifies a specific page, I only know which elements to change because I looked at the source code. But I don’t control it, and somewhere down the line it will probably change.

That’s an obvious point, but nonetheless, it’s important: anything we write that makes assumptions about the DOM will stop working one day, and we can’t do anything about that.

However, we can at least prepare for it by making sure we test for the existence of something before we modify it. That’s good practice anyway, but it becomes doubly-important here. In fact, it’s the purpose of the tests against null in the Slashdot example script; things like this:

//the first center element contains an ad frame 

var adframe = document.getElementsByTagName('center')[0]; 

 

//remove it 

if(adframe != null) { adframe.parentNode.removeChild(adframe); }

Now, if the script does fail, at least it won’t throw any errors.

Customise Search Results

I’ve often though it would be cool if search results could come back as a tree menu, with +/- icons to allow users to show and hide the summaries. Thankfully, Google results have a predictable (if soupy) structure that’s easy to iterate over, so it’s a fairly simple task to extract the relevant information and reformat it into a dynamic list:

View or install the script google-tree.user.js from the code archive.

The only serious complication we’ll face in parsing the HTML is that the format varies when the result includes a translate link (it has an additional containing <table>). But, provided we allow for that, we can reliably identify the summary of each result, for which we then add show/hide behaviours that are triggered from a dynamically-created icon.

The behaviours come from an onclick handler on the icon, which works with both mouse and keyboard navigation. The use of javascript:void(null) in the link href simply provides an href value; otherwise the link wouldn’t be keyboard navigable (because Mozilla can’t set focus on a link that has no href). It’s essentially the same trick as using href="#", but this method is cleaner: it doesn’t do anything else, so you don’t have to control its return value the same way.

Improving Web Usability

All the examples we’ve looked at so far are designed for specific Websites. But user scripts can be made to run on every Website, and this is perhaps where the idea becomes remarkable: you can write user scripts to change how the whole Web behaves. Effectively, you can customise your browser just by writing some simple JS.

The practical considerations are slightly different here, because, generally speaking, we won’t be looking for specific structures: we’ll be making something new, or modifying widely-used attributes. As such, the scripts in this example are generally more forwards-compatible than previous examples.

Change Link Targets

An obvious application, this script identifies links that have a "_blank" target attribute, and retargets them to "_self":

View or install the script target-changer.user.js from the code archive.

This script uses a single document onclick handler to change the link targets on-the-fly, rather than iterating through all links and changing them in advance. We do this partly because it’s more efficient, but mostly so that the script will still work if the target has been set by other scripting (remember that Greasemonkey user scripts run before the window.onload event).

With a more complex script you could go much further: change or remove onclick events, rewrite javascript: URIs, even override the open() method completely. But with such aggressive measures comes the risk of blocking out functionality you do want: if the open() method is overridden, some links might end up doing nothing. That’s the kind of thing you’d probably only want to do for specific sites, not for the Web in general.

Remove Ads

The difficulty with removing ads is that not all of them are unwanted. Some people don’t like any advertising at all, while for many, simple text-based ads are acceptable and only large or animated graphical ads are deemed intrusive. The source of the ads may also be relevant: maybe particular companies or sites are more likely to carry advertising you’re interested in, or are inclined to trust.

Any ad-blocking program will inevitably have limits to the degree of precision with which it can be customised, but with user scripting there are no such limitations: you can tailor the script exactly how you like, and avoid any unwanted interference.

In this example I’ve split the scripting into methods based on the type of advert — whether it’s an image, iframe, or flash ad; each object is further tested to see if it comes from a known ad server:

View or install the script remove-ads.user.js from the code archive.

If you don’t want to remove a certain type of object, simply comment-out its method call at the bottom of the script:

//instantiate and run   

var rem = new removers();  

rem.banners();  

rem.iframes();  

rem.flash();

If you want to extend the list of ad servers, you can add as many as you like to the domains array at the top of the object constructor:

//list the domains or subdomains from which ads might be coming  

this.domains = ['doubleclick.net','servedby.advertising.com'];

And, if you’d prefer to allow certain sites to show their advertising, whatever it may be, you can add those sites as @exclude comments:

// @exclude         https://www.sitepoint.com/*

The principle could be taken much further, to control to the nth degree whether an element is removed or not — you could differentiate by size, for example, images that are 468 x 60 pixels. Or you could allow Flash only if it’s embedded using <object>, not if it uses <embed>. This would filter out most advertising, while leaving compliant Websites alone!

Correct Language and Spelling

A study of commonly misspelled words, by Cornell Kimball, analysed Internet (Usenet) newsgroups to discover the most common misspellings. Using some of that data to construct regular expressions, this script makes text-replacements to correct common errors:

View or install the script language-corrector.user.js from the code archive.

You could extend the idea by adding profanity filters to the list: converting unacceptable language into “***” or any empty string. But do be careful with any words that might also be substrings of other words, such as “ham” is to “gingham”. (There’s a story of a local authority in the British town of Scunthorpe that went without email for an entire day because of a crucial omission in their mail filtering rules!) You may have to test for a word, plus a leading and trailing space or punctuation, to ensure you don’t have this kind of problem.

A more sophisticated example could scan individual pages for complicated terminology, or technical phrases which are not properly explained on the site, adding <abbr> or <dfn> where required. You could even construct links directly to an online dictionary or other reference source.

Google Site-search

A site that lacks a search facility can be frustrating to use, but Google allows you to search within individual sites simply by appending site:domain.com to the query. This script uses that syntax to create a site-specific search box on every page.

Or, at least … that was the plan:

View or install the script site-search.user.js from the code archive.

The scripting is solid, the idea sound, but it has quite a serious problem: it doesn’t work.

I wouldn’t usually publish a script if I couldn’t get it to work, but I think this is such a good idea it would be a shame not to include it, and anyway, I wanted to look at the questions it brings to light.

As you’ll see if you try it, the core scripting works but you can’t type into the textbox, even though you can copy and paste into it, and then press Enter to submit. Further investigation reveals that the "-moz-user-modify" property has a value of "read-only" (although explicitly setting it to "read-write" doesn’t help).

Now I did actually think of two ugly hacks to get around this, and they both work, but I’m very reluctant to use them, because I think they challenge a security restriction. The script creates a user-interface element, and perhaps Firefox extensions aren’t allowed to initiate user input in the DOM of the page you’re viewing. I’m making guesses here, because I haven’t been able to find the facts, but if that, or something like it, is the case, I suspect it will also be true for any form-based user scripting, making such things basically untenable.
If anyone can shed any light on this, I’m all ears!

Onfocus Tooltips

I’m often frustrated by the fact that title attribute tooltips don’t show up from keyboard navigation — they work only when you use the mouse. This script compensates for that by creating tooltips that are triggered by focus events:

View or install the script onfocus-tooltips.user.js from the code archive.

The scripting is quite straightforward; it creates a single element and writes in the title text (if any exists). The complications lie in the positioning of the tooltip: it has to position itself relative to the triggering element, then compensate for potentially being located outside the window, or below the fold.
It works on links, iframes, objects and form elements — those that can receive the focus — and creates tooltips that are styled using CSS2 System Colors.

Headings-navigation Bar

Most serial browsers (screenreaders like JAWS, and text-only browsers like Lynx) have a “headings mode” or something like it, where they list and link to all the headings on the page. It provides a greater degree of random access (on pages which use proper headings).

This script emulates that functionality, creating a small “H” icon at the top-right of the page, from which a menu drops down containing links to every heading:

View or install the script headings-navigation.user.js from the code archive.

The menu is built as a list, each item of which is a link populated with text or other HTML from the original heading. If the heading contains an image, it will be reproduced along with the rest, However, as this might not be ideal, an alternative approach could be to extract and use the ALT text, or perhaps remove extraneous markup altogether.

Persist Alternate Stylesheets

The final example in this section begins with a well-meaning script, but opens a can of worms along the way…

One issue with “alternate” stylesheets is that changes made using most browsers’ built-in switcher don’t persist between pages, making it little but a novelty feature unless the author intervenes with cookies. Perhaps, in future, more browsers will implement this behaviour, but until then, we can write a user script to do it for us.

This example looks for stylesheets that are included via <link> elements, and saves their disabled state to a cookie whenever it changes; the script then re-applies the last state to subsequent page views. This effectively adds domain-specific persistence to sites that use native stylesheet switching:

View or install the script persist-stylesheets.user.js from the code archive.

It works because Firefox updates the stylesheets’ disabled property with changes from the native switching mechanism, so we can implement persistence simply by continually getting and saving that data to a cookie.

This brings us to the first significant point, and the lid of our can of worms: the cookie we’re using is in a domain we don’t control. From a practical perspective, this means that if the site already uses cookies a lot, we run the risk of filling up the data limit (4K) and thereby overriding data that the site needs. We can’t avoid this possibility (short of not using cookies at all), but we can reduce the risk by keeping the data as small as possible. You could also reduce the impact by having a list of specific @include domains, where you know the feature is needed, rather than running it on every site that uses alternate stylesheets.

But suddenly there are security implications here, as well: it would be very easy to write a user script that steals the cookie and other data from every site it encounters, then sends it all somewhere else. Users could be exposing themselves to a whole new angle of exploitation; a recent cnet article has already brought this issue to the fore. It also asks more general questions around whether site owners might object to having their pages modified in this way.

Dean Edwards raises a similar point with his interesting take on how Greasemonkey broke his site; a lively debate follows. It certainly seems unfortunate, if, with more people using Greasemonkey, we can no longer predict the DOM of our Websites; but the question is really moot, because that’s already the case. Anyone can make any page appear any way they want in their own browsers; many browser add-ons, screenreaders and other user-agents will change, rewrite or add to the DOM of the page before any client-side scripting is run.

The capabilities Greasemonkey gives us are no different, apart from the larger number of people who might use it, and I suppose that’s really the rub — especially when it comes to things like the impact on advertising revenues. Just like TiVo (a digital TV system that can filter out ad-breaks from recordings), this becomes a threat only if lots of “ordinary” people use it.

But the security concerns are very serious, and the Greasemonkey blog takes it up by suggesting that browsers shouldn’t be able to make http requests outside the current domain without a user prompt. That could certainly reduce the problem of data stealing, though not without unwanted side-effects (what about the impact on legitimate remote syndication?). And it doesn’t eradicate the problem completely, as the author goes on to concede: nothing would stop a user script from rewriting page links via another server, adding query-string data along the way.

I think the implication is this: non-technical users must be able to trust the source of a user script they install. One possible solution might arise if an archive of user scripts was made available from a verifying source, as Firefox extensions are. This wouldn’t preclude developers from offering scripts directly, or users from downloading them, but it would give non-technical users a place where they could be sure that whatever they’re installing is safe. And in fact the ever-active GM development team are looking into ways of doing this, with a userscript.org Website planned to manage a catalogue of available scripts.

Developer Tools

User scripts might also represent a niche for the creation of developer tools — scripts that you can write and customise for specific projects, or keep as a collection of general tools that are enabled or disabled as required.
The obvious comparison with bookmarklets comes to mind again, as does the fact that user scripts run automatically, so we can do things that wouldn’t be possible with a run-once, manually initiated bookmarklet.

Word Count

Word counters are useful tools, but it’s not the kind of thing you’d expect in a browsing toolset, and it wouldn’t be much use for general surfing. But for development — and particularly for writing articles directly in HTML, as I always do — it’s handy to have a counter going as you write.

View or install the script word-count.user.js from the code archive.

This script uses a similar technique to the language corrector we looked at earlier, extracting the text from a page by iterating through elements and looking for their child text-nodes. It then splits and processes the text to count the number of words, and displays the result in the <title> attribute (adding to, rather than overwriting it).

Viewport Size

This short script simply reads the viewport (inner window) dimensions using Mozilla properties, and displays that data in the <title>:

View or install the script viewport-size.user.js from the code archive.

Fun with Greasemonkey

I was delighted to realise the comedic angle to user scripting … It reminds of a job I had once, where we used to amuse ourselves by writing bookmarklets that made subtle and not-so-subtle changes to the company Website. It made us laugh … and it’s all ultimately harmless, so if you want to make serious Websites look silly, I’m all for showing you how. Here are a couple of simple scripts that make the Web a more entertaining place!

Name that Celebrity

The names of politicians, industry leaders, and other well-known people are ripe for transformation into comical phrases … I’ll show you the basic script by making a few… er… adjustments to some names you’ll undoubtedly know:

View or install the script name-that-celebrity.user.js from the code archive.

This example is similar to the language corrector script, as it extracts the text from a page, and makes changes to the listed words on a semi-random basis.

Smurfify

This final script is cute and silly. It analyses text to look for words in particular patterns, and changes some of them to “smurf”:

View or install the script smurfify.user.js from the code archive.

User JS in Opera and Beyond

Opera has been planning user JavaScript for a while, and the feature is now available in Opera 8, having been introduced to the beta version and heralded by Hallvord Steen in a recent Journal entry. (Though, amusingly, this isn’t the first implementation to go public — the Opera 7 “Bork” edition loaded a JS file into MSN pages, to transform them into the language of the Muppets’ Swedish Chef!)

User scripting works similarly to Greasemonkey in Opera. The browser does implement a direct compatibility mode, but it also offers additional functionality in the form of special-purpose events from the window.opera object, and is really only intended for power-users, as some manual configuration is required. Rijk van Geijtenbeek has the most comprehensive resources for user scripting in Opera, and converting this article’s examples shouldn’t be difficult for the most part, especially since we planned for something like this from the beginning.

The workings and permissions of user scripting may have to change, or cause browsers to change, in light of the ways in which people use it. But, whatever else happens, it’s clear that user scripting is here to stay. Similar functionality is available in Safari via a browser plug-in called PithHelmet, and even the ubiquitous Internet Explorer will soon be getting in on the action, courtesy of Todd Ostermeier and the poetically-titled GreasemonkIE.

I think that technologies like this are good for the Web, because they give users more control over how the Web behaves, which improves overall access to information. But there are strong notes of dissension in several corners, and perhaps it won’t be long before we’re deluged with anti-greasemonkey coding techniques, followed by anti-techniques to those!

I’ll be watching this develop with great interest.

Further Resources

• Greasemonkey developer site
• Greasemonkey blog
• Greasemonkey Script Repository

Frequently Asked Questions (FAQs) about Greasemonkey