SitePoint Sponsor

User Tag List

Results 1 to 4 of 4
  1. #1
    SitePoint Zealot
    Join Date
    May 2013
    Posts
    198
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)

    robots.txt can made private file not viewable by humans?

    1.
    robots.txt can made private file not viewable by humans?

    2.
    if i have 1st robots.txt in primary domain ggg.com
    www,ggg.com/robots.txt
    if folder1domain.com is a secondary domain...
    www,ggg.com/folder1/ === www.folder1domain.com, this latter root folder when accessed from www.folder1domain.com, should needed another robots.txt ???

  2. #2
    Programming Team silver trophybronze trophy
    Mittineague's Avatar
    Join Date
    Jul 2005
    Location
    West Springfield, Massachusetts
    Posts
    17,152
    Mentioned
    190 Post(s)
    Tagged
    2 Thread(s)
    The robots.txt file provides no security and in fact may lessen it.

    What it does is tell honest search engines your preferences.
    Granted, "Allow" and "Disallow" are misleading and better words could have been used (ignore?) but for better or worse that's what they are.

    You may find that "dishonest" search engines (or humans) use the file to find out what you don't want them to see. (eg. if you set up a "honeypot").

    If you really want to protect something either don't put it online, or if you must, use some other method like password protection.

  3. #3
    SitePoint Zealot
    Join Date
    May 2013
    Posts
    198
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    thks for the info...can you reply and second Q... in other words each hosted domain (each subfolder of primary domain) needs a robot.txt, in case use it... to really do its partly job?
    yes correct. / no just primary domain root folder needs it only.

  4. #4


Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •