SitePoint Sponsor

User Tag List

Results 1 to 6 of 6
  1. #1
    SitePoint Wizard
    Join Date
    May 2003
    Location
    Berlin, Germany
    Posts
    1,829
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    block from bots/spiders

    Is there a script (I mean no .htaccess) that'd block all search engine bots (including those which don't use robots.txt) to disallow access ?

  2. #2
    La la la la la bronze trophy lieut_data's Avatar
    Join Date
    Jun 2003
    Location
    Waterloo, ON
    Posts
    1,517
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by DarkAngelBGE
    Is there a script (I mean no .htaccess) that'd block all search engine bots (including those which don't use robots.txt) to disallow access ?
    Not that I'm aware of. PHP isn't usually run as a daemon or server of any sort, and thus has no control over who connects to what. If the files you don't want accessed are strictly PHP, or you can sucessfully hide them behind a PHP file, you could do some checking on $_SERVER['HTTP_USER_AGENT'], but this can easily be fooled...

    .htaccess effectively does the same thing, but without the hassle of hiding the files behind PHP. Is there something with .htacess that you can't achieve?
    My name is Steve, and I'm a super-villian.

  3. #3
    SitePoint Wizard
    Join Date
    May 2003
    Location
    Berlin, Germany
    Posts
    1,829
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Well I never really touched .htaccess and thus have no clue on how to access that file. It's even not on my server. Maybe this is the right time to get myself familiar with .htacess ?

  4. #4
    Apache Expert i_like_php's Avatar
    Join Date
    Nov 2001
    Location
    Dallas, Texas
    Posts
    1,342
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Thumbs up

    Quote Originally Posted by DarkAngelBGE
    Well I never really touched .htaccess and thus have no clue on how to access that file. It's even not on my server. Maybe this is the right time to get myself familiar with .htacess ? [img]images/smilies/smile.gif[/img]
    you should the .htaccess once mastered will be one of your best friends for apache
    i love php

  5. #5
    ********* Wizard silver trophy Cam's Avatar
    Join Date
    Aug 2002
    Location
    Burpengary, Australia
    Posts
    4,495
    Mentioned
    0 Post(s)
    Tagged
    1 Thread(s)
    .htaccess is a fantastic tool, and it's really not that hard to use.. really!

    Here is a link to an advanced .htaccess tutorial. I know you've never touched .htaccess before but you're pretty "advanced" in lots of things so you should be able to pick it up easily I also just found this, and contrary to the image you get from the URL, it's a very good page regarding spiders, what they do, where they come from, and how to stop them if you wish. A sure read and bookmark for me

  6. #6
    does not play well with others frezno's Avatar
    Join Date
    Jan 2003
    Location
    Munich, Germany
    Posts
    1,391
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    great links
    We are the Borg. Resistance is futile. Prepare to be assimilated.
    I'm Pentium of Borg.Division is futile.Prepare to be approximated.


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •