Anyone know how to tell where this mysterious traffic is coming from?

[Archbob]

So one of my sites that I kind of forgot about is suddenly getting like 30,000 uniques a day for no reason. This traffic has like a 98% bounce or something and its like 99% US traffic. Average stay time on site of like 9 seconds. I'm not sure where its coming from because when I check GA, it just says "direct". I thought someone iframed my site so I put this code:

<script type="text/javascript">
	<!--
		if (top.location!= self.location) {
			top.location = self.location.href
		}
	//-->
</script>

To pop out of frame.


But it made no difference.


Anyone have any idea what it is?

[John_Betong]

Have you got access to the log files. If you have then you the source URL should be available.

If you are using Apache and PHP then you could write the following to a file:

<?php
  $str      = $_SERVER['HTTP_REFERER'];
  $file     = getcwd() .'/LOG_FILE.TXT';
  $handle = fopen($file, 'a+');

  fwrite( $handle, "\n\r" .$str);
  fclose( $handle);
?>

Once you know the offending URL's then you could modify your .htaccess to ban the URL/

[Archbob]

Well, google adsense is going showing the traffic as "direct" so I think think that will work.
Do autosurf things leave a referral trail. Because if they don't, then I think its some sort of botnet traffic so all over. Not sure how to stop something like that.

[John_Betong]

I had gigabytes of Easter European robot traffic on a daily basis and stopped it using the following:

robots.txt

User-agent: *
Crawl-delay: 10

Google Webmaster Tools reports:

Line 2: Crawl-delay: 10 Rule ignored by Googlebot

[Archbob]

Mine is all from USA.
So that code does not effect google indexing the site?

[John_Betong]

Mine is all from USA.
So that code does not effect google indexing the site?

Give it a try and see if it makes any difference to the "30,000 uniques per day".

Can you supply a traffic source sample. Maybe there is some common theme.

[Archbob]

What do you mean by traffic source sample? I'm looking at some logs and mainly google analytics.

That code doesn't actually stop any search engines from crawling your site, right?

[TechnoBear]

Bots don't have to obey robots.txt files at all, and badbots are most unlikely to, so I'm surprised that approach worked for John_Betong (although I'm not doubting his word).

Is all this traffic to the same page or different pages, and is it a page that should actually be there? I ask because I first noticed one of my sites had been hacked when a couple of the most-requested pages were ones I didn't know existed.

[Archbob]

Its all being directed to index.php, which of course is the homepage. It has to be there. I'm not the only one to be targetted this year. There are many discussions about this topic:

http://netmeg.com/bot-attack/

I can't figure out how to get rid of it. I've pulled the adsense ads from the one site thats been hit the hardest. It doesn't seem to have caused much problems with adsense on my sites though since it doesn't generate fake clicks, just impressions.

Its drip, drip bot traffic so its not a DDOS attack, but its really annoying.

[TechnoBear]

If the user-string is the same in every instance, you can block it via .htaccess (assuming you're using Apache). Or you could look into something like Crawl Protect.

[Archbob]

Its not, all the traffic comes from the USA, but it comes from a variety of states and locations.

[TechnoBear]

That's not quite what I meant. In your logs, you should see a user-string something like:

Mozilla/5.0 (compatible; Ezooms/1.0; ezooms.bot@gmail.com)

or

Mozilla/5.0 (Windows; U; MSIE 9.0; WIndows NT 9.0; en-US)

[Archbob]

70.137.147.126 - - [13/Apr/2012:08:10:06 +0100] "GET / HTTP/1.1" 200 4705 "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)"85.164.115.226 - - [13/Apr/2012:08:10:07 +0100] "GET / HTTP/1.1" 200 4705 "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.0; Trident/5.0)"24.196.38.89 - - [13/Apr/2012:08:10:07 +0100] "GET / HTTP/1.1" 200 4705 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0;

I get logs like that, but you can't just block MSIE

[TechnoBear]

No, that's why I was asking if all these unwanted visitors have the same user-string, so that you could block that one and only that one.

[Archbob]

Well, I will keep trying and search to see if anyone else has found a solution. I have no idea where this botnet is coming from. I think this discussion should be moved to a sub-forum though. Thought since it had to do with traffic sources, this forum would be the right one but it seems that this is more technical.