Is it true that if you require users to sign up or login before accessing certain web pages the search engines will be less likely to pick them up or does it matter? Trying to decide whether to have some blog content protected for clients and some not and for free.
Thanks in advance!
If users need to do anything other than click on links to get to a page, search engines won’t. How could they? Googlebot doesn’t have any special privileges that are available to a general unauthenticated user … if that was the case, anyone could spoof their headers to pretend to be Googlebot and have unfettered access to every password protected site on the web!
Googlebot is just a web user like you or me. It makes HTTP requests just like a browser. It can’t access anything a random surfer can’t. If you put things behind a login, they won’t be indexed.
That’s what I thought. Thanks for explaining and confirming!