SitePoint Sponsor

User Tag List

Results 1 to 13 of 13
  1. #1
    SitePoint Zealot alix's Avatar
    Join Date
    Jun 2006
    Posts
    119
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Question newbie question: how do I make google not show a page of my web site?

    In my website there's a pop up window and it appears in google search results.

    What can I do to undo this situation?

  2. #2
    SitePoint Wizard bronze trophy DaveWoods's Avatar
    Join Date
    Dec 2004
    Location
    Derbyshire - UK
    Posts
    2,651
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Place a file called robots.txt in the root of your website and include the following...

    Code:
    User-agent: * 
    Disallow: /pop-up.html/
    Just replace pop-up.html with the name of the file (and the path to this file).

    Obviously this won't take affect straight away but should remove the page from the results when Google re-indexes your site.

  3. #3
    SitePoint Zealot alix's Avatar
    Join Date
    Jun 2006
    Posts
    119
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    thanks man! I'll try that!

  4. #4
    SitePoint Addict Poiesis01's Avatar
    Join Date
    Jun 2007
    Location
    Cape Town
    Posts
    233
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    A more effective way would be to add the nofollow,noindex robots metatag to the page header.

  5. #5
    SitePoint Zealot alix's Avatar
    Join Date
    Jun 2006
    Posts
    119
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by Poiesis01 View Post
    A more effective way would be to add the nofollow,noindex robots metatag to the page header.
    That sounds like a simple and effective option. Thanks!

  6. #6
    SitePoint Guru marcel's Avatar
    Join Date
    Nov 2000
    Posts
    920
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Block it completely with scripting (PHP, ASP.NET)... if it's a google bot, ban it.

  7. #7
    SitePoint Wizard bronze trophy DaveWoods's Avatar
    Join Date
    Dec 2004
    Location
    Derbyshire - UK
    Posts
    2,651
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by marcel View Post
    Block it completely with scripting (PHP, ASP.NET)... if it's a google bot, ban it.
    Isn't that a bit overkill?

    Personally I use robots.txt as mentioned above but the meta should work just as well.

    I use the robots.txt method as it's easier to manage which pages you don't want to include if you store this information all in one place and is especially useful if you want to block google from whole directories.

  8. #8
    He's No Good To Me Dead silver trophybronze trophy stymiee's Avatar
    Join Date
    Feb 2003
    Location
    Slave I
    Posts
    23,424
    Mentioned
    2 Post(s)
    Tagged
    1 Thread(s)
    The robots.txt file is the best solution hands down. Every other method is either a convoluted method of it (the meta tag) or just overkill (serverside user-agent detection).

  9. #9
    SitePoint Zealot alix's Avatar
    Join Date
    Jun 2006
    Posts
    119
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    All right. I'll use the robots.txt file then.

    I've seen a lot of information about SEO and that's a big reason to ask questions here.

    Thanks!

  10. #10
    SitePoint Guru marcel's Avatar
    Join Date
    Nov 2000
    Posts
    920
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by csswiz View Post
    Isn't that a bit overkill?
    In this situation, yes it's overkill. But it's an option.

    Scrappers who copy website content usually ignore robot.txt.
    Google isn't the only search engine bot. Robot.txt is something search engines don't have to obey. Each situation is different.

  11. #11
    SitePoint Guru marcel's Avatar
    Join Date
    Nov 2000
    Posts
    920
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by csswiz View Post
    I use the robots.txt method as it's easier to manage which pages you don't want to include if you store this information all in one place and is especially useful if you want to block google from whole directories.

    What about the folks who ignore robot.txt ?

  12. #12
    SitePoint Wizard bronze trophy DaveWoods's Avatar
    Join Date
    Dec 2004
    Location
    Derbyshire - UK
    Posts
    2,651
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    If it was an issue then I'd probably look into another solution but as it is currently, I've got the following stats for the last month...

    1. google 2,638 - 88.46%
    2. yahoo 136 - 4.56%
    3. live 99 - 3.32%
    4. msn 43 - 1.44%
    5. aol 27 - 0.91%
    6. search 21 - 0.70%
    7. altavista 6 - 0.20%
    8. ask 5 - 0.17%
    netscape, seznam, alice, baidu - next to nothing (less than 3 visits).

    As far as I'm aware, the top 8 all support robots.txt so it's not really something I've considered.

    I completely agree that your solution is an option but such is the state of the search market that I personally find robots.txt sufficient for my needs

  13. #13
    He's No Good To Me Dead silver trophybronze trophy stymiee's Avatar
    Join Date
    Feb 2003
    Location
    Slave I
    Posts
    23,424
    Mentioned
    2 Post(s)
    Tagged
    1 Thread(s)
    Quote Originally Posted by marcel View Post
    In this situation, yes it's overkill. But it's an option.

    Scrappers who copy website content usually ignore robot.txt.
    Google isn't the only search engine bot. Robot.txt is something search engines don't have to obey. Each situation is different.
    In that case .htaccess would work much better. But still, that would be best for bad bots and robots.txt should be used for legit bots.


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •