Google’s Flash Indexing Disaster

On July 1st, Google announced that, using technology provided by Adobe, it had enhanced the Google Search Engine to index the text embedded within Flash movies. What followed was bad advice from Google, second-guessing by web developers, and finally a few straight answers.

Google’s initial announcement was so incredibly vague as to render it all but useless. Developers came away knowing that Google was doing something different with their Flash content, but that’s about it.

While Google’s Dion Almaer suggested that search engines have always been black boxes and that it was up to us to discover what had changed through testing, just about everyone else was crying foul.

Google’s credibility was immediately in question due to the obviously bad advice it contained:

"If you prefer Google to ignore your less informative content, such as a "copyright" or "loading" message, consider replacing the text within an image, which will make it effectively invisible to us."

For the record, replacing fast-loading, accessible text content with a bulky image simply to hide it from search engines is never a good idea.

Google’s list of caveats in the announcement were similarly perplexing:

"Googlebot does not execute some types of JavaScript. So if your web page loads a Flash file via JavaScript, Google may not be aware of that Flash file, in which case it will not be indexed."

What types of JavaScript? Established best practice for publishing Flash content is to use the SWFObject JavaScript library to overcome bugs in older browsers, so was Google saying that it would only index Flash content that was authored using broken/outdated HTML-only techniques?

"We currently do not attach content from external resources that are loaded by your Flash files. If your Flash file loads an HTML file, an XML file, another SWF file, etc., Google will separately index that resource, but it will not yet be considered to be part of the content in your Flash file."

Any experienced Flash developer knows that if you are going to have any significant amount of text in your Flash content, your best bet is to stick it in an XML file and load it on the fly, so you don’t have to rebuild your Flash movie whenever you change the content.

Apparently, not only will Google not see Flash content authored this way, but it will track down the XML file anyway and index it as a separate page on your site! That’s right, Google will helpfully direct people searching for your content to the raw XML file that contains it, rather than your slick, Flash front-end.

All this stuff made so little sense, that many developers questioned whether Google was actually able to index any Flash content of consequence. Within a few days, however, the Search Engine War blog was able to verify that Google was indeed indexing Flash content.

Finally, after several days of developer outcry, Google admitted it had left too many questions unanswered, and four days later, it posted a significant update that is well worth reading if you have any Flash content on your site.

Here’s a quick summary of what we now know:

  • The July 1st release didn’t index Flash content inserted with the SWFObject library‘s dynamic publishing method, which writes the Flash content into the page entirely with JavaScript. The recommended static publishing method (where two nested <object> tags are included in the page) was indexed. Google is now deploying an update that supports the dynamic publishing method as well.
     
  • Text content loaded on-the-fly from an XML file is not yet indexed, but Google is working on fixing this in the near term.
     
  • Google will do its best to detect when duplicate content is there to provide an HTML alternative to Flash content, and will only display one of the two versions in the search results. No penalty is applied to a site’s search ranking due to duplicate content.

There are still unknowns here, but that will always be the case with the Google search engine. Though it took a few days, Google is answering what questions it can, and responding to developer concerns with enhancements.

Before very long, most of the text within Flash-based web sites will make its way into the Google search index. Nevertheless, uncertainty will remain over how deeply Google is able to probe Flash content for a while yet. Providing non-Flash alternative content will remain an effective means of guaranteeing your most important content a place in the Google index. It also gives users of non-Flash-enabled browsers (like the iPhone) something to look at.

Though Google’s initial message was pretty half-baked, the follow-up has put most of my concerns to rest. How about yours?

Win an Annual Membership to Learnable,

SitePoint's Learning Platform

  • http://www.tyssendesign.com.au Tyssen

    The headline’s a bit misleading don’t you think? Wouldn’t it be more accurate to say “Google’s Flash indexing PR disaster?”

  • Peter

    Agreed- sounds more like a PR snafu around a lovely development than a real disaster. In other news, I found this article via your newsletter sent to my gmail. Unfortunately, the link wouldn’t open. Fortunately, I was able to find the article via an alternate route: Googling it.

  • FlashEng1

    Will be very interesting then to see how/if Google’s ability to crawl external .as files written in AS3 and taking a more OO approach will yield similar results.

  • aaroncampbell

    I would disagree with this just being a PR thing. It was PARTLY a PR problem, but the fact that it did NOT index flash that was included using best practices was a disaster. The fact that it STILL does NOT index external content properly is STILL a disaster. However, the progress is awesome and I see nothing but good coming from it in the future.

  • Darren

    @aaroncampbell

    A “disaster”? Did Google lose any customers because of this? Is it now less effective at indexing Flash content than it was 2 weeks ago before the announcement? I don’t think so.

  • Steve

    Kevin this is not a disaster this is a silly headline. Normally I really enjoy your articles and information but this sounded a tad hysterical. They will get it right and it will not be too long.

  • Fred Boulton

    I still don’t much see the relevance of using Flash on Web sites unless you’re on an ego trip of some kind.

    I, personally, cringe when I hit a site that’s using Flash! I want information, not animations and graphics going all over the place. Why put distractions on clients’ sites?

    Those Web developers using a lot of Flash content don’t seem to care if users are on a slow dial up or other slow connection ‘cos most of the files/pages are huge. Many people won’t bother to wait for those pages to load, they just click off somewhere else. Me included. They also don’t worry about users who are browsing without Flash installed or if people are browsing with images switched off.

    I wonder if those developers tell their business clients what’s at risk with Flash sites? I still don’t see many Web sites offering a non-Flash alternative for people who don’t want the Flash.

    Personally, for my clients I want the maximum number of people to be able to access their sites, which to me means fast-loading pages that are going to get indexed without having to worry about whether the S/Es are going to be able to index Flash pages or sites.

    I only see Google being talked about in regard to indexing. What are the other major S/Es doing about indexing Flash? 40% of users use other search engines.

    I love playing with Flash, but all but the tiniest bits, to me, are not suitable for using on Web sites.

  • Anon

    This is still a very misleading headline.

    Raised a lot of expectations – was left high and dry

    What a plonker!

  • Andy G

    Hello everyone.

    I agree wholeheartedly with Fred Boulton. I feel Flash is okay if you’re showcasing visuals of some form but if you have any written information to impart it should be in a simple, quick, accessible web page. For my own use of the web I prefer to utilise the faster download speeds available now to make the stuff I want to see open faster. Of course, if I’m looking at a photography site then I’m prepared to wait for larger files to download to get a fully detailed image – that’s the payoff for wanting to view images. If I’m looking for text then I want it now because as web speeds increase there’s no reason at all why a web page couldn’t change as quickly as the page of a book can be turned. Flash is a visual medium and should be used as such. However, having said all this, all content on the web should be indexed. I think the problem here is that inappropriate use of Flash has made the job harder than it should have been.

    As Steve said, they probably will get it right. But why didn’t they wait until it was right before launching? It’s very much a ‘software company’ thing to launch something first and check it works later – a company like Google is big enough and rich enough to develop and test properly and they should be ashamed that they didn’t.

    Regards.

    Andy.

  • Kevin Abs

    So basically flash is not a good idea to have on a website? Google needs to step its game and algorithms because it seems to be stuck in the past. Web 2.0 is already the wave of the future and companies need to keep up.

    sig..
    workouts ab

  • tonybone

    “Those Web developers using a lot of Flash content don’t seem to care if users are on a slow dial up or other slow connection ‘cos most of the files/pages are huge.”

    While I agree that the use of Flash on public-facing web sites should be judicious, someone pointed out to me (and I tend to agree) that these kinds of arguments remind him of back when people argued against switching from DOS to Windows because computers didn’t have enough memory to run it. Being overly concerned about those still using dialup will hamper the advancement of web technologies. Let’s get past the 20th century.

  • soly

    I think in this new changing world where the countries of resources have the benefit of having our dollar at a very low point, the outcome of that is, that there is a sales increase in our country. Therefore many see for very important to increase the photos and visual parts on their sites for the people shopping from the countries above. I saw many sites trying to focus on that issue. A good example of that is, a site I found, selling promotional items liftpromos.com where you can see them working closely to these important points. You can see it at the work they are putting in the pictures on the site. And many strongly support that issue.
    Thanks

  • Shan

    tonybone, whils i do agree with you, it does not mean that we should just kick them why not offer an alterative? html only, then when that day comes, at least you didn’t loose any cliente becasue you felt they didn’t deserver to be on your site becasue they don’t have a fast computer/internet or whatever.

  • Gel

    you are all idiots, especially this kevin person who is bitching about google screwing up their pr…indexable flash (with a few flaws) wins over non-indexable flash no matter how you slice it. do you think google isn’t aware of these problems? they recommended images to hide non-priority info instead of text because that’s the only option right now, at least until they find a solution with adobe…how stupid can this kevin guy be?

    kevin, i can’t believe you actually got a book published…this article is complete horse s***…i’d rather bet my marbles with google, you know, that search engine that everyone uses, the company thats growing exponentially ’cause they’re that good.

  • Elliot

    Why did this turn to a Flash bashing party? Is it hitting too close to home? The last stronghold of Web 1.0’s usefulness is it’s transparency to search engines. Luddites can bitch all they want about why not to learn new technologies including AJAX and Actionscript, but in the end you will all be replaced by people who embrace all available technologies no matter what they are.

    Get learnin’ and stop bitchin’ or find a new job.

  • simplo

    What people continue to not understand and complain is the fact that flash is not designed to be used for e-commerce and business purposes. Until Adobe designs an indexing engine for flash which can accurately direct visitors to indexed content flash will not be useful in respect to SE traffic. Kevin Yank doesn’t appear to understand some SEO principals and how Search Engines classify content. For example, the bit about replacing “less informative content” with images. This is done for a number of reasons, one is the fact that Google can’t divide a flash file into pages and direct people to a specific indexed section, therefore you should only use text for the most relevant and valuable content. This is a Flash problem, not Google’s or any other SE’s. If you use an optimized image you will have no trouble at all using text images. All your other complaints are moot points as well which are not Flash specific problems… This article is plain rubbish.

    It all comes down to knowing when to use flash. You should never use it for a money site, your competitors non-flash based site will consistently beat your Flash site in usability, conversion and search engines. Until Adobe develops a spidering protocol for content and it will require all flash apps to be optimized, either do a flash and non-flash site or non-flash only. Adobe should be bending over to make everything work with SE’s not the other way around. It’s not Google’s fault Flash was never designed to be Search Engine Friendly.

  • shareitall

    I think it speaks well of Google that they are advancing their indexing technology, and responding to webmaster’s concerns.

    That they communicate at all is a good thing, and unusual for a company so big. Ever try to get something out of Microsoft?

    Daniel
    entrepreneur blog

  • Greg

    I am a Flex / Flash AS3 developer at a creative agency who prides itself on rapid development of innovative and visually impressive ui. For us, there is no alternative to SWF content. Flex has only deepened our commitment to this technology, as the framework is robust and, most importantly, consistent across all browsers.
    These days, there is no excuse for slow-loading Flash content — all of the package libraries are locally installed with the Flash Player. Content that has not been properly staged and preloaded is the fault of the developer.
    I am rather surprised to see (considering the popularity of FLV streaming video these days) anyone bashing Flash. Really the only advantage markup languages still have is their crawlability.

    Which is what this article is about: While the title may be dramatic, it is true: My boss sent a company-wide email back when Google announced this, something to the effect of: “Our troubles are over!” But in actuality, nothing has changed: Google doesn’t see SWFs loaded with JavaScript (ie SWFObject, which is how all SWF content is loaded thanks to IE). Google doesn’t see externally loaded assets (ie, XML, which is how all good flash apps should manage their data). So really, Google’s announcement got our hopes up but delivered nothing.

  • Anonymous

    Flash is a UI and accessibility disaster in most instances…. Google was just trying to help.

  • corners

    Why the black and white??

    There is no reason why flash can’t be used sparingly around a site to add constructive interest. It doesn’t have to be fluff, there can be useful information delivered in this manner. If the developer incorporates the proper content replacement techniques, and keeps SEO related content within the html (or similar) areas of the site, flash inclusion can be more than ok.

    If you are still on a dialup in 2009, then quite honestly I don’t care if you hang around to load a flash file on my site or not. Get a job, make a few bucks, and get normal connection. For gods sake dsl can be had for $29 a month, and high-speed wi-fi is all over the place, coffee shops, etc. Why would you expect someone to downgrade their site for a dial up connection?? That’s like installing an outhouse just in case you have a visitor that doesn’t know how to work the flusher. My goodness it’s time to step out of the 90’s already!!

  • BGdesign

    I loved this discussion Ive been thinking about this a lot lately.