Hide test site from search results

(Not sure where to post this)

I have a domain that I host and mainly use it just for doing tests on sites that I’m developing.

Problem is, I ran a goole search on a site I just launched, and the test site came up a few times!

I would really like this test site to never show up in any search result.

Any recommendations or best practices that others use when using a development/test domain?

Add this to your meta tags:


<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">

Do I have to put this on every page? Or just the index.html?

You should use a robots.txt file:

http://www.robotstxt.org/

The meta method would have to go on every page indeed… the better solution as had been pointed out is the robots.txt

Both solutions require the spiders to come back to the site to chck it again, so may not disappear instantly if they have already been found… a better answer would be to password the new directory with a .htaccess/.htpassword setup, that way only people with the correct login details can get in.

If you want to do that with all page you’d better use robots.txt file
If you need to disable only part of your page I would recommend you warp it in nofollow tags.

You can use meta-nofollow as well as also can use the Robot.txt. both of them are beneficial from blocking the visiting of search engines.

you could use robots.txt to block spiders access certain url / part of your website. http://www.robotstxt.org/robotstxt.html

Have you considered password protecting the folder the test sites are in? Without authentication, the spiders cannot crawl the pages.