SitePoint Sponsor

User Tag List

Results 1 to 9 of 9
  1. #1
    SitePoint Evangelist
    Join Date
    Jan 2005
    Posts
    425
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    CURL / FOPEN help

    I've got the following code:

    PHP Code:
           $ch curl_init();
         
    curl_setopt($ch,CURLOPT_URL$flv_url);
         
    $data=curl_exec($ch);
        
        
    curl_close($ch);
        
        
    $output_dir $config['TEMP_IMP_DIR'] . "/test.flv";    
        
        
    $file fopen($output_dir,"w+");
        
    fputs($file$data);
        
    fclose($file); 
    All that is outputing in $output_dir file is the number 1.

    I know the input $flv_url exists, as I tested it manually in the browser.

    Also is there a better way (with PHP) to fetch big files remotely (5-100mb etc) and put them locally? One with minimum memory usage and one that won't grind the server to a halt, or is this the best method?

  2. #2
    SitePoint Evangelist
    Join Date
    Jan 2005
    Posts
    425
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Still having troubles with this.
    Its just outputting 1 in the $output_dir file, and not the content from the $flv_url source.

    Can anyone see any problems with this? Or suggest a better way of doing it?

    Thanks

  3. #3
    Twitter: @AnthonySterling silver trophy AnthonySterling's Avatar
    Join Date
    Apr 2008
    Location
    North-East, UK.
    Posts
    6,111
    Mentioned
    3 Post(s)
    Tagged
    0 Thread(s)
    What about this?

    Function
    PHP Code:
    <?php
    /**
     * Uses cURl to save a remote file locally.
     *
     * @param String $sRemoteFile
     * @param String $sSaveTo
     * @return Boolean
     */
    function saveRemoteData$sRemoteFile $sSaveTo )
    {
        
    $rcUrlHandle curl_init$sRemoteFile );
        
    curl_setopt$rcUrlHandle CURLOPT_RETURNTRANSFER true );
        
    $sData curl_exec$rcUrlHandle );
        return (
    file_put_contents$sSaveTo $sData )) ? true false ;
    }
    ?>
    Usage
    PHP Code:
    <?php
    if( saveRemoteData'http://www.google.co.uk' 'googlePage.html') )
    {
        echo 
    'Yay! We obtained the target file.';
    }
    else
    {
        echo 
    'Whoops, we could not obtain the data!';
    }
    ?>
    @AnthonySterling: I'm a PHP developer, a consultant for oopnorth.com and the organiser of @phpne, a PHP User Group covering the North-East of England.

  4. #4
    SitePoint Evangelist
    Join Date
    Jan 2005
    Posts
    425
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Thanks mate, that worked perfectly.

    Is curl memory/server intensive? Like downloading lots of 20 to 100mb files will cause problems for a normal dedicated server?

  5. #5
    Twitter: @AnthonySterling silver trophy AnthonySterling's Avatar
    Join Date
    Apr 2008
    Location
    North-East, UK.
    Posts
    6,111
    Mentioned
    3 Post(s)
    Tagged
    0 Thread(s)
    I wouldn't say so; although you'll have to remember that we store the complete file contents in memory before dumping it to the target file, so large files may affect performance somewhat.

    You could use the socket functions to read the target file a chunk at a time, with each iteration appending to the target file. This approach would be much more resource friendly, although I'm sure it could bring it's own nuances to the table.

    See how you get on with the supplied function, if indeed it appears to be hitting performance adversely, come back and we'll re-address.
    @AnthonySterling: I'm a PHP developer, a consultant for oopnorth.com and the organiser of @phpne, a PHP User Group covering the North-East of England.

  6. #6
    SitePoint Wizard bronze trophy
    Join Date
    Jul 2008
    Posts
    5,757
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Although I'm not positive, I bet copy() won't hog your memory.
    Using the CURLOPT_FILE option to curl would probably also be more memory efficient than buffering into a variable so you can write later write to a file using another php filesystem function. If you try to store the contents of a 100mb file in a php variable, you will need an extra 100mb of memory. Not good.

    You can test this stuff yourself with memory_get_peak_usage()

  7. #7
    SitePoint Evangelist
    Join Date
    Jan 2005
    Posts
    425
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Thanks, how would I change:
    PHP Code:
    <?php
    /**
     * Uses cURl to save a remote file locally.
     *
     * @param String $sRemoteFile
     * @param String $sSaveTo
     * @return Boolean
     */
    function saveRemoteData$sRemoteFile $sSaveTo )
    {
        
    $rcUrlHandle curl_init$sRemoteFile );
        
    curl_setopt$rcUrlHandle CURLOPT_RETURNTRANSFER true );
        
    $sData curl_exec$rcUrlHandle );
        return (
    file_put_contents$sSaveTo $sData )) ? true false ;
    }
    ?>
    ....to incorporate CURLOPT_FILE?

  8. #8
    Twitter: @AnthonySterling silver trophy AnthonySterling's Avatar
    Join Date
    Apr 2008
    Location
    North-East, UK.
    Posts
    6,111
    Mentioned
    3 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by LuckyB View Post
    Thanks, how would I change:
    PHP Code:
    <?php
    /**
     * Uses cURL to save a remote file locally.
     *
     * @param String $sRemoteFile
     * @param String $sSaveTo
     * @return Boolean
     */
    function saveRemoteData$sRemoteFile $sSaveTo )
    {
        
    $rcUrlHandle curl_init$sRemoteFile );
        
    curl_setopt$rcUrlHandle CURLOPT_RETURNTRANSFER true );
        
    $sData curl_exec$rcUrlHandle );
        return (
    file_put_contents$sSaveTo $sData )) ? true false ;
    }
    ?>
    ....to incorporate CURLOPT_FILE?
    PHP Code:
    <?php
    /**
     * Uses cURL to save a remote file locally.
     *
     * @param String $sRemoteFile
     * @param String $sSaveTo
     * @return Boolean
     */
    function saveRemoteData($sRemoteFile$sSaveTo)
    {
        
    $rcUrlHandle curl_init($sRemoteFile);
        
    $rFileHandle fopen($sSaveTo'w');
        
    curl_setopt($rcUrlHandleCURLOPT_FILE$rFileHandle);
        
    curl_exec($rcUrlHandle);
        
    curl_close($rcUrlHandle);
        
    fclose($rFileHandle);
        return (
    filesize($sSaveTo) > 0) ? true false ;
    }
    ?>
    @AnthonySterling: I'm a PHP developer, a consultant for oopnorth.com and the organiser of @phpne, a PHP User Group covering the North-East of England.

  9. #9
    SitePoint Evangelist
    Join Date
    Jan 2005
    Posts
    425
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Thanks that worked a charm!

    Now I'm going to post another question that I have (in a new topic)..

    Thanks again SilverB and CRMalibu.


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •