Do search engines look for sitemap.xml even if not declared in robots.txt?

Hi,
the site I am not working on has a sitemap and it is being used by google. But I am wondering if that is because it was submitted via webmaster tools rather than google finding it. There is no mention of the sitemap in the robots.txt file which is why I am wondering if this is the case. Or does google do a check for sitemap.xml anyway?

I’m assuming it would be beneficial to add the sitemap location to robots.txt to ensure other search engines will be able to find it.

thoughts?

Thanks

Making it generally available seems like a good idea to me.

Both Google and Bing advise either submitting the sitemap to them directly, or adding it to the robots.txt file, which would suggest that search engines do not routinely check for one.

1 Like

Thoughts: You put more effort into postulating the question than it would take to put the line into robots.txt regardless of whether or not it helps?

To be a little less snarky: The problem with answering the question is that “search engines” is not a singular entity. They haven’t all gotten together and said “Here’s how we will operate”. So the answer has to be “for some, it might help; for others, not at all. You’d have to investigate a specific search engine to find out the answer for that specific search engine.”

2 Likes

Well you say that… In my former position I was able to just get on and make changes to the site I built and managed but my role currently is more managing our relationship with an agency to do the work so everytime I want something as tiny as this I have to submit a support ticket and wait for them to make it live.
Previously as you say a 2min job for me, now more like a week whilst I wait for it to be questioned, done and checked and signed off and etc etc… :frowning:

Don’t worry I didn’t take it that way and I’m sure you’ve answered a lot of my questions over the years :+1:

oh also thanks both :slight_smile: super fast response to my question. Much appreciated.

2 Likes

This topic was automatically closed 91 days after the last reply. New replies are no longer allowed.