SitePoint Sponsor

User Tag List

Results 1 to 13 of 13
  1. #1
    SitePoint Member omega's Avatar
    Join Date
    Oct 2003
    Location
    Sweden
    Posts
    11
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Question Get through browser dectection?

    Hi, Im currently working on a contentgrabber. It works fine until I utilize it on sites that are proctected in some way. For example.
    http://www.newscientist.com/news/ , when Im trying download it through fopen another page downloads [img]images/smilies/frown.gif[/img]
    If its possible to download the page through a browser, shouldn't it be possible to download with PHP in someway?

  2. #2
    SitePoint Evangelist Daijoubu's Avatar
    Join Date
    Oct 2002
    Location
    Canada QC
    Posts
    454
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Just send the user-agent you want

    Some short of jsp running on Orion 1.5.2

  3. #3
    SitePoint Member omega's Avatar
    Join Date
    Oct 2003
    Location
    Sweden
    Posts
    11
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Yes.. I tried with :
    PHP Code:
    header('HTTP_USER_AGENT: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; MyIE2; .NET CLR 1.1.4322) '); 
    but it didn't worked..Im not sure if that is the right way to do send them though.
    If you can't get rid of the skeleton in your closet, you'd best
    teach it to dance.

  4. #4
    eschew sesquipedalians silver trophy sweatje's Avatar
    Join Date
    Jun 2003
    Location
    Iowa, USA
    Posts
    3,749
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    header() would send that HTTP head as part of the _reponse_ the php page generated. You need to send the header as part of your fetch request. I believe that the CURL extensions let you send whatever http headers you would like.
    Jason Sweat ZCE - jsweat_php@yahoo.com
    Book: PHP Patterns
    Good Stuff: SimpleTest PHPUnit FireFox ADOdb YUI
    Detestable (adjective): software that isn't testable.

  5. #5
    Non-Member
    Join Date
    Jan 2003
    Posts
    5,748
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    cURL comes with PHP5 yes ? Just curious that's all

  6. #6
    eschew sesquipedalians silver trophy sweatje's Avatar
    Join Date
    Jun 2003
    Location
    Iowa, USA
    Posts
    3,749
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by Dr Livingston
    cURL comes with PHP5 yes ? Just curious that's all
    Don't know for sure. It is an extension, so I assume you could still compile it in.

  7. #7
    SitePoint Member omega's Avatar
    Join Date
    Oct 2003
    Location
    Sweden
    Posts
    11
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    It in 4.33 already. If you running windows and want to to enable it just copy ssleay32.dll, libeay32.dll from php/dlls and php_curl.dll from php/extensions to your win32/system32 folder. Then uncomment ;extension=php_curl.dll in php.ini.
    To check if it's enabled you can do a <?php_info?> and look for curl.
    If you can't get rid of the skeleton in your closet, you'd best
    teach it to dance.

  8. #8
    PHP manual bot bronze trophy Gaheris's Avatar
    Join Date
    Oct 2003
    Location
    Germany
    Posts
    2,195
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Sockets

    You may want to use Sockets, or a quick'n dirty request with fsockopen.
    Don't know, could be easier if you have problems with a missing cURL extension.

  9. #9
    SitePoint Evangelist Daijoubu's Avatar
    Join Date
    Oct 2002
    Location
    Canada QC
    Posts
    454
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Try this?
    PHP Code:
    if ($fp = @fsockopen('www.newscientist.com'80$errno$errstr1))
    {
        
    fwrite ($fp"GET /news/ HTTP/1.1\r\nUser-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; MyIE2; .NET CLR 1.1.4322)\r\nHost: www.newscientist.com\r\nConnection: Close\r\n\r\n");
        while (!
    feof ($fp)) {
            
    $contents fgets($fp4096);
        }
        
    fclose($fp);
    }
    else
    {
        echo 
    $errstr'! ('$errno')';

    Speed & scalability in mind...
    If you find my reply helpful, fell free to give me a point

  10. #10
    SitePoint Member omega's Avatar
    Join Date
    Oct 2003
    Location
    Sweden
    Posts
    11
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Daijoubu: It didnt work.. I tried with:
    PHP Code:
    if ($fp = @fsockopen('www.newscientist.com'80$errno$errstr1)) 

        
    fwrite ($fp"GET /news/ HTTP/1.1\r\nUser-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; MyIE2; .NET CLR 1.1.4322)\r\nHost: [url]www.newscientist.com\r\nConnection:[/url] Close\r\n\r\n" ); 
        while (!
    feof ($fp)) { 
            
    $contents fgets($fp4096); 
        } 
        
    fclose($fp); 

    else 

        echo 
    $errstr'! ('$errno')'

    echo 
    $contents
    However I got it to work quite well with cURL .
    If you can't get rid of the skeleton in your closet, you'd best
    teach it to dance.

  11. #11
    eschew sesquipedalians silver trophy sweatje's Avatar
    Join Date
    Jun 2003
    Location
    Iowa, USA
    Posts
    3,749
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by omega
    Daijoubu: It didnt work.. I tried with:
    That is becuase you were mixing HTML in with HTTP. HTTP is the protocol controlling the request and response, HTML is only the data carried back from the request over HTTP.

  12. #12
    o_O O_o BlueFire2k5's Avatar
    Join Date
    Mar 2003
    Location
    Sioux Falls, SD
    Posts
    475
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by sweatje
    That is becuase you were mixing HTML in with HTTP. HTTP is the protocol controlling the request and response, HTML is only the data carried back from the request over HTTP.
    I think that was vB's 'Automatically parse URLs' option.

  13. #13
    SitePoint Member omega's Avatar
    Join Date
    Oct 2003
    Location
    Sweden
    Posts
    11
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Yes, it was the html auto parse. (doh)
    I just added this line to the end of the code, to see if it worked:
    echo $contents;
    If you can't get rid of the skeleton in your closet, you'd best
    teach it to dance.


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •