Ignore MSN Search at your Peril

Trying to catch up with Google in the search market is rather like trying to chase down a coast guard cutter in a rowing boat. Figures on the search giant’s market domination vary massively: sources say it has captured anywhere from 36.5% (according to Search Engine Watch, July 2005) to 52% (WebSideStory, June 2005) of all searches. Whatever the specific figures may be, there’s no doubt that it’s the market leader by a significant margin.

For any competitor, gaining on Google is certainly a massive task, but one thing that Microsoft will always have (for the foreseeable future, at least) is the ability to throw money at a project until at least some of it sticks. Financially and commercially, if anyone can challenge Google, Microsoft can. Indeed, it is, with its own MSN Search. This comparative new kid on the block has been out of Beta for over a year now, and MSN Search has matured into a service to which Microsoft is proud to give its name.

But is it any good?

Feature-rich Search

MSN Search certainly seems to be matching Google stride for stride in terms of features. For example, Google has a local search directory solution, Microsoft has a local search directory solution, Google has an images service, Microsoft has an images service, Google has the Earth mapping service, Microsoft has a mapping service. The list goes on and on!

Crucially though, there are already some things that Microsoft seems to be doing better than Google. Take the MSN Images Search, for example. At first glance, it looks like Google Images, but a second look reveals that Microsoft has exposed some very neat filtering options on the front page. (Okay, Google allows you to filter by "color" or "black & white" image files, and by image size, too, but these features are buried away in Advanced Options, and I’d speculate that most users don’t even realise that they exist.) MSN has also responded very quickly to the recent law suit that was successfully filed against Google Images (trial date pending) by adding a "Report this Image" feature in order to avoid being accused of ignoring copyright infringements. Not bad going.

What About the Search Engine?

The meat of MSN Search has got to be the web search engine and, looking at the numbers, it clearly has some work to do to catch up with Google in terms of the sheer volume of simple results. Google no longer boasts the exact size of its index is, but a quick search for "Alexander the Great" (the first thing that popped in to my head — don’t ask me why!) indicates that Google still has a much larger index than MSN. Google says it returns "about" 91,300,000 results whereas MSN claims to pull a positively miniscule 3,504,091 results (these figures were correct at the time of writing, though they may have changed by the time you read this).

Index Size — Does it Really Matter?

Sometimes, yes. Google’s index is much larger than MSN’s, which is vital according to some schools of thought. One subscriber to this view is the Director of Products at my work (an online media and search provider), who commented to me in a recent meeting:

People keep telling us they’re getting too much information — I don’t believe it is possible to have ‘too much’ information. You just need to know how to order it properly.

I take his point, and by that rationale, the bigger the index, the better the product is as long as you can serve that content up in a meaningful way. But we build focussed business search tools. Does this attitude apply to a straight search engine? In the case of general search, when the only output is a list of links, the absolute key is the quality of the first fifty-odd results. No one will go beyond that. I would suggest that there simply is no way to make all 91,000,000 results from a single query "useful" in a traditional web search context.

That said, there is one scenario in which Google’s much larger index is a distinct advantage — in searches for obscure information. Google’s ability to pull up some results even for the most bizarre of pastimes or strangest recipes or rarest guitar models is second to none. MSN is definitely playing catch-up, though, and seems to be gaining ground very rapidly. Possibly the biggest challenge for MSN Search will be to stay as targeted in its results as Google has, once it has a similarly monstrous archive to churn through.

Bringing the Cream to the Surface

Once a search engine can return millions and millions of results per query, the main problem is to make sure the top fifty are completely relevant to the user. This task is allotted to Google’s PageRank function, the mystical magic behind Google’s success. The PageRank function is Google’s means of ordering search results according to their importance, that is, by link popularity, the "honesty" of content, and various other factors, some secret, some not.

It shouldn’t come as any surprise that Microsoft has a very similar system for MSN Search; both approaches seem to work very effectively.

Getting the PageRank or equivalent mechanism to work correctly is crucial. It doesn’t only affect the content that’s returned when a user runs a query, but also decides the content indexed by search engine robots, and the order in which it’s indexed, when it’s indexed, and how regularly that information is updated. When you consider that most users don’t move past the first page of search results, clearly, presenting the most "important" content first is key — it’s how MSN can have a much smaller index and continue to serve results whose relevance equals those on Google.

I’ve used MSN Search in tandem with Google for quite a few weeks now and, as you’d expect, I’ve noticed that some searches in Google are more fruitful than those in its MSN counterpart (particularly for more obscure topics or those that are more prone to high volumes of fast-moving spam content), but the reverse is also true: sometimes, searches on MSN are more fruitful than on Google. Overall, in terms of getting me the information I need quickly and effectively, both score good marks.

Is there a Differentiator?

To my mind, it seems that the competitors are running this race almost neck and neck right now — if anything, Google has the edge with its larger index and greater experience in the search business. However, after mulling over the details, one thing was still eating at me: while carefully monitoring my own guitar reviews website, I noticed the site enjoyed much better rankings in MSN than in Google for my keywords and phrases. Moreover, I noticed that MSN showed quicker growth of its index, more swiftly reflected changes in the "terrain" of the Internet (the web of links that make up the WWW), and much, much more quickly responds to page edits and content updates, both from my searches and from server logs.

I started discussing this with a friend (and fellow SitePoint irregular) who has a penchant for SEO, and he’s of the opinion that Google’s filters have been made stronger in order to stop spamming and prevent upstart new keyword-optimised websites, such as our own, from achieving good rankings before they prove their worth.

He might be right, but I thought about this, and I don’t believe it. In the past, Google always claimed to bring fresh content to the top, yet right now it seems Google is taking months to respond to content changes that occur on normal web sites, while MSN is responding within days to the very same changes. I refuse to believe that there are any clever filters behind this. I think, for whatever reason, that, at the moment, Google simply is slower to respond to new content than MSN.

One factor may be that MSN is playing catch-up with Google — it’s trying harder because it has to, in order to be taken seriously. Another factor may be that Google has a lot more existing content to keep track of, and re-indexing its much larger index is simply too expensive for the company to handle any faster than it currently does.

As a consequence, and to touch on a point I made earlier, I do not believe that having so much more indexed content is helping Google to serve better results to most of its users, most of the time. In fact, I believe the opposite. I have already accepted that a large index yields benefits in certain scenarios, but most of Google’s (or that of any other search engine, for that matter) massive index is worthless junk at any one point in time.

Put simply, at this time, MSN Search seems to be much more nimble and relevant than Google’s search engine, which is a significant statement. Google seems too slow next to this newer, faster Microsoft offering, and this could be seen as a crucial differentiator between the two from a user perspective.

Should We Cry Foul?

I guess the many, many anti-Microsoft tub-thumpers out there will be shouting at their VDUs by now. "Typical Microsoft, stealing someone else’s hard work and ideas and passing them off as its own!"

Well, yes and no. Clearly MSN is following in Google’s footsteps, but it’s not passing off anything that belongs to Google as its own — it’s simply creating a rival service. This tactic of allowing Google to blaze the trail, then jumping straight onto the bandwagon (as appears to be the case to us Average Joe Users) must be infuriating for Google’s honchos … but what can they do?

All they can do is make Google’s products better — a task in which they’re currently experiencing varying degrees of success. I, for one, think the competition is healthy and would not discourage Microsoft from going after Google — much as I hope Google carries out its plans to release an operating system and continue with its other moves on Microsoft’s desktop tools space. This is exactly what our world needs.

The Proof of the Pudding

I admit that, on the technical side, there are more observational findings and user experiences driving this article than real, hard evidence, so all I can really say is give MSN Search a go, if you haven’t already, and see what you think — suck it and see, as they say.

My opinion (based purely on the observations detailed in this article) is that MSN Search is demonstrably quicker than Google at mapping changes in the very synapses of the Internet and responding to those changes, while still providing equally targeted search results. It has its weaknesses, of course. For example, Google can handle plurals — a search for "review" will also include "reviews" — whereas MSN can’t yet, but I’m sure it will soon. I’m wondering whether MSN Search should be my new search engine of choice, painful as it is to reset that "Home" button — especially for Firefox users.

All in all, MSN hasn’t done a bad job, and I expect the company will see an upturn in its search traffic before too long. I may be wrong, but I base this speculation on the fact that, over recent months, the fastest climber up the search competition ladder seems to have been AOL. This is significant because I happen to know that AOL is striving hard to drive more traffic to its online services (which were traditionally only used by their ISP customers) by marketing AOL.com as a portal and community web site — a tactic that seems to be working. Furthermore, according to the Alexa Top 10 English Language Sites, MSN (and Yahoo! for that matter) is more popular than Google, and has the same kind of focus as the AOL portal, while Google.com is still holding forth with a pure search interface.

The combination of a popular content portal that provides an effective search alternative to Google on-site will likely begin to reap reward for MSN sooner rather than later. And we mustn’t forget the new Windows Live Search Beta, which could also start to see the company take a larger market share. I await the next set of statistics from Search Engine Watch with interest.

In the meantime, I’m sure that no serious SEO guys and gals will ignore MSN (or any other decent search competitor), but for the rest of you, heads up! Here comes Microsoft, and it’s coming up fast!