How can i disallow a website address by robots file.
i need to disallow a site as [noparse]www.domain.com
if i use this code what will be
User-agent: *
Disallow: / www.domain.com[/noparse]
The robots.txt file needs to be placed in the top level directory for your site. So to disallow domain.com, you place the robots.txt file in its TLD and use
User-agent: *
Disallow: /
to disallow the entire site.
You can find more information at http://www.robotstxt.org/robotstxt.html