How to make Google not index a subdomain?

Hi forum,

I have got a question, but should be a piece of cake.

I just want to confirm with someone how do I block search engines (all of them) from a testing subdomain?

Lets say my domain is

My subdomain is

So, in my robots.txt file…

Should it be like this:

Agent: *

Is the above the right htaccess command and should this be placed in the main public_html folder?

My subdomain redirects to a subdomain not a subdirectory…it redirects to

Thanks for any clarifications

Yes, your subdomain points to a folder in the main sites document_root folder.


So, in your public_html folder create a robots.txt to include

User-agent: *
Disallow: /test/

Or, to specifically disallow a particular agent :

User-Agent: Googlebot
Disallow: /test/

Here is a good reference on robots.txt that you can bookmark.

Each subdomain should have it’s own robots.txt file in it’s own root directory. To block an entire subdomain you would use
User-Agent: *
Disallow: /

No htaccess involved. Just a plain-text file named robots.txt


I wonder why I was away for so long…you Sitepoint guys are so helpful :slight_smile:

Welcome to Sitepoint Datalife :wave: Thanks for posting this thread, I learned something from it too. :slight_smile: I should probably do the same for my multiple subdomains on my sites - though I’ve never bothered to create a robots.txt file as the instructions on google didn’t appear very clear to me - perhaps a second look is in order.

See my post above, and check out the link. It is about the clearest set of instructions on robots.txt files that I have found.

IMO there’s no need for a robots.txt if you don’t want to block anything. A lot of people throw up empty ones so they don’t get a bunch of 404s in their logs.

Your post is inaccurate. You can’t block access crawling of a subdomain by putting instructions in a robots.txt file in the top domain’s root.

Ah! Missed that on first glance… thanks, it should take some of the stress out of creating one -at first glance it looks very straightforward. :slight_smile: