SitePoint Sponsor

User Tag List

Results 1 to 7 of 7
  1. #1
    ********* Ornithologist AtomicPenguin's Avatar
    Join Date
    May 2002
    Location
    Vancouver, BC
    Posts
    459
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Avoiding fatal errors on max execution time

    Hi folks,

    A quick question for somebody. I'm using file_get_contents() to scrape a webpage specified by a user. The problem is, it's entirely possible the user may enter an invalid page - or one running on an incredibly slow server. So, depending on the server timeout setting, it may die with a fatal error...
    "Fatal error: Maximum execution time of X seconds exceeded in..."
    Obviously I can just increase the max execution time that PHP uses, but I'd rather not. Instead, I'd like to give up on the request after (say) 60 seconds and inform the user rather than display the ugly error message.

    Oh, and one final catch: it needs to run on both PHP 4 and 5, so no try-catches...

    Any suggestions?
    A.P.

    generatedata.com - free JS/PHP/MySQL random test data generator.
    Form Tools - free PHP/MySQL form processor.

  2. #2
    SitePoint Evangelist
    Join Date
    Aug 2007
    Posts
    566
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Look upon the PHP streams functions, and more secifically on the streams timeout setting
    http://www.php.net/manual/en/functio...et-timeout.php

    You will need to change from file_get_contents() to stream_get_contents(), like this:
    PHP Code:
    if ($stream fopen('http://www.example.com''r')) {
        
    //set a 2 seconds timeout on $stream
        
    stream_set_timeout($stream2);
        
    //to detect timeout, we need to get stream meta data infos
        
    $info stream_get_meta_data($stream);
        
    // print all the page starting at the offset 10
        
    $html=stream_get_contents($stream, -110);

        if (isset(
    $info['timed_out']) {)
            echo 
    'Connection timed out!';
        } else {
            echo 
    $html;
        }
        
    fclose($stream);

    Not tested, but it should do the trick...

  3. #3
    ********* Ornithologist AtomicPenguin's Avatar
    Join Date
    May 2002
    Location
    Vancouver, BC
    Posts
    459
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Interesting approach! Unfortunately, stream_get_contents() is a PHP 5 function, so it won't work in my case. Any other ideas?
    A.P.

    generatedata.com - free JS/PHP/MySQL random test data generator.
    Form Tools - free PHP/MySQL form processor.

  4. #4
    Use The Cloud
    Join Date
    Jan 2006
    Location
    Boise, ID
    Posts
    556
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    stream_get_contents() is basically fread() for dummies.

    You should be able to write a drop in replacement using fread().
    Brad Hanson, Web Applications & Scalability Specialist
    ► Is your website outgrowing its current hosting solution?
    ► PM me for a FREE scalability consult!
    ► USA Based: Available by Phone, Skype, AIM, and E-mail.

  5. #5
    SitePoint Evangelist
    Join Date
    Aug 2007
    Posts
    566
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    You should be able to write a drop in replacement using fread().
    Not sure...
    Streams have been greatly reworked on php5, and I'm not sure the timeout set function exists for PHP4.

    But seriously AtomicPenguin, PHP4 is out of league.
    PHP6 is around the corner, and PHP5 is here since almost 5 years.
    It's time to let PHP4 rest in peace...

  6. #6
    ********* Ornithologist AtomicPenguin's Avatar
    Join Date
    May 2002
    Location
    Vancouver, BC
    Posts
    459
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I agree 100%. 100%. But I don't have any choice in the circumstances.
    A.P.

    generatedata.com - free JS/PHP/MySQL random test data generator.
    Form Tools - free PHP/MySQL form processor.

  7. #7
    SitePoint Wizard bronze trophy
    Join Date
    Jul 2008
    Posts
    5,757
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    You have pretty strong control if you do the whole process with fsockopen() and stream_set_timeout(). You have enough control to where you could even keep giving it a little longer if its making some gradual progress.

    fopen() tries to read immediately when used to open a url. This kinda prevents setting a timeout, since context support for fopen() wasnt introduced until php5.

    Also, curl supports CURLOPT_CONNECTTIMEOUT and CURLOPT_TIMEOUT
    http://www.php.net/manual/en/function.curl-setopt.php


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •