Googlebot can't access your site (WordPress)

Marketing
1

Hi I received the following email from Google this morning, in relation to a Wordpress site:

Over the last 24 hours, Googlebot encountered 7 errors while attempting to connect to your site. Your site’s overall connection failure rate is 50.0%.

You can see more details about these errors in Webmaster Tools.

Recommended action

Verify that the web service software for your site is installed and running properly.
Verify that your firewall or server are not blocking googlebot's access to your site.
Verify that all scripts that run on your site have proper permissions to run.
Verify that your site has proper permissions to access the pages of your site.
Using Webmaster Tools, find a day with a high error rate and examine the logs for your web server for that day. Look for errors in the logs for that day and fix the causes of those errors.
Your site may be overloaded. Talk with your hosting provider about either reconfiguring your web server or allocating more resources to your web server.
If your site redirects to another hostname, another possible explanation is that a URL on your site is redirecting to a hostname that has one or more of the problems listed above.

After you think you’ve fixed the problem, use Fetch as Google to verify that Googlebot can properly access your site.

When I looked into it on Webmaster Tools it said

Google couldn’t access your site because of a server connectivity issue.

The site is up and running ok.
I’ve checked robots.txt and that seems fine.
I’ve chatted with my host (Fasthosts) about possible server connectivity issues or misconfiguration and the response was

I have heard of some instances of similar messages in the past, I believe it’s usually caused by the google bot wanting to query the server too many times in a short period of time. Changing the crawl rate will usually stop these messages from happening.

I can see why this would make the error messages less frequent! But I’m not sure it solves the problem.

My crawl rate is set to

Let Google optimize for my site (recommended)

I’d rather not post my site’s url publicly but will send it in a PM if necessary.

Can anyone suggest what’s causing this message to appear? And whether it’s harming my SEO?

Thanks

2

we would have deleted the url anyway, so no harm done there.[ot]We don’t allow self-promotion and posting your url is self-promotion (there are exceptions to the rule, of course)[/ot]

It may be a connectivity problem or not. The fact that you see your site up and running doesn’t mean that there may not be small delays at delivering the page. If these short delays are too long for the spider (and we’re talking miliseconds here) then the bot may think that your site is down.

The answer of your provided doesn’t mean that you don’t need more resources. Basically, he’s telling you “you’ve got what you contracted and it works fine on our side”. It may be true on a regular day but maybe your traffic has increased and you need more to keep on going. You should look at your traffic to see if you need more resources. Also, verify if when the bot was trying to crawl your site, you had a peak of visitors.

Your ISP is right on one thing: You want Google to crawl when you want to. That means that if you post an article every week, you want Google to crawl once a week. If you write everyday, then you want it to crawl everyday.

Leaving in on automatic can be good… but try to change it to something else, depending on the number of the updates you do, and see if this stops the messages

3

Qwaerstd,

Have you tried deleting your robots.txt file to see if that fixes the issue? Also, are you sure there are no other URL’s with an absolute path that your website may be referencing? Sometimes third party templates contain URL’s to other sites whose configurations prevent Googlebot from crawling your site.

Best,

Shawn

4

That recommended action list has appeared on many other forum posts, though there seems to be a different cause each time. Other discussions have focused on things like calendar add-ons for WP, which, when disabled, fix the issue … though most discussions seem to blame the server.