Results 1 to 4 of 4
Thread: Sitemap position in robots.txt
Sep 27, 2010, 23:16 #1
Sitemap position in robots.txt
Is there any importance in the position of sitemap keyword in robots.txt?
Specifically my query is, I have some disallow URL's in my robots.txt. But the same links are appearing in sitemap.xml and sitemap.xml is placed at the end of robots.txt file. Will the links specified in disallow section omitted when crawling? Or do i need to add the disallow links after sitemap declaration?
Also, If i add an old url to disallow section, will it remove from the google search index?
Sep 27, 2010, 23:30 #2
- Join Date
- Mar 2009
- Melbourne, AU
- 448 Post(s)
- 8 Thread(s)
Sep 27, 2010, 23:59 #3
Thanks for your reply. But still I am not clear about the following.
1.) Will the sitemap command override the disallow command in robots.txt?
2.) If a url is present in both site map and disallow section, which will get precedence?
Sep 28, 2010, 01:34 #4
You could test it in google masters! It doesnt change your robots.txt, just for test!