Skip to main content

Web Bots vs Humans: We're Losing

By Craig Buckler

Blogs

Share:

Free JavaScript Book!

Write powerful, clean and maintainable JavaScript.

RRP $11.95

51% of all web site traffic is non-human. If you’re reading this, you’re in the minority. Unless you’re a machine (check your ports — if they’re warm and moist, you’re probably OK).

The study was undertaken by Incapsula, a website security company. While we haven’t quite reached the point where Skynet becomes self-aware and dispatches Terminators to wipe us out, it does appear that the web would continue to operate happily without us.

Data was collected from a sample of 1,000 websites which use Incapsula services. The report concluded that web traffic is generated from:

  • 49%: people browsing the web
  • 20%: search engines bots indexing pages
  • 19%: spy-bots collecting competitive intelligence
  • 5%: website scrapers
  • 5%: automated hacking tools searching vulnerabilities
  • 2%: automated comment spammers

It’s easy to be cynical about these reports, especially when they’re conducted by a security company which could enjoy commercial benefits from scare-mongering. That said, it does indicate a significant volume of hacking activity even if the report is somewhat skewed or exaggerated.

If you or your clients aren’t concerned about security, perhaps it’s time to evaluate that policy. If we assume Incapsula report is over-estimated by a factor of 300%, malicious activity will still account for one in ten website requests. That’s equivalent to the average number of all IE6, IE7, Firefox 3.x and Opera users combined. Nasty.

Humans may be losing the web war, but at least we can win a few battles.

Do you believe Incapsula’s report? How does it compare with your website statistics? Have you successfully defeated a major hacking attempt?

Craig is a freelance UK web consultant who built his first page for IE2.0 in 1995. Since that time he's been advocating standards, accessibility, and best-practice HTML5 techniques. He's created enterprise specifications, websites and online applications for companies and organisations including the UK Parliament, the European Parliament, the Department of Energy & Climate Change, Microsoft, and more. He's written more than 1,000 articles for SitePoint and you can find him @craigbuckler.

New books out now!

Learn valuable skills with a practical introduction to Python programming!


Give yourself more options and write higher quality CSS with CSS Optimization Basics.