CURL / FOPEN help

[LuckyB]

I've got the following code:

       $ch = curl_init();
     curl_setopt($ch,CURLOPT_URL, $flv_url);
     $data=curl_exec($ch);
    
    curl_close($ch);
    
    $output_dir = $config['TEMP_IMP_DIR'] . "/test.flv";    
    
    $file = fopen($output_dir,"w+");
    fputs($file, $data);
    fclose($file); 

All that is outputing in $output_dir file is the number 1.

I know the input $flv_url exists, as I tested it manually in the browser.

Also is there a better way (with PHP) to fetch big files remotely (5-100mb etc) and put them locally? One with minimum memory usage and one that won't grind the server to a halt, or is this the best method?

[LuckyB]

Still having troubles with this.
Its just outputting 1 in the $output_dir file, and not the content from the $flv_url source.

Can anyone see any problems with this? Or suggest a better way of doing it?

Thanks

[AnthonySterling]

What about this?

Function

<?php
/**
 * Uses cURl to save a remote file locally.
 *
 * @param String $sRemoteFile
 * @param String $sSaveTo
 * @return Boolean
 */
function saveRemoteData( $sRemoteFile , $sSaveTo )
{
    $rcUrlHandle = curl_init( $sRemoteFile );
    curl_setopt( $rcUrlHandle , CURLOPT_RETURNTRANSFER , true );
    $sData = curl_exec( $rcUrlHandle );
    return (file_put_contents( $sSaveTo , $sData )) ? true : false ;
}
?>

Usage

<?php
if( saveRemoteData( 'http://www.google.co.uk' , 'googlePage.html') )
{
    echo 'Yay! We obtained the target file.';
}
else
{
    echo 'Whoops, we could not obtain the data!';
}
?>

[LuckyB]

Thanks mate, that worked perfectly.

Is curl memory/server intensive? Like downloading lots of 20 to 100mb files will cause problems for a normal dedicated server?

[AnthonySterling]

I wouldn't say so; although you'll have to remember that we store the complete file contents in memory before dumping it to the target file, so large files may affect performance somewhat.

You could use the socket functions to read the target file a chunk at a time, with each iteration appending to the target file. This approach would be much more resource friendly, although I'm sure it could bring it's own nuances to the table.

See how you get on with the supplied function, if indeed it appears to be hitting performance adversely, come back and we'll re-address.

[crmalibu]

Although I'm not positive, I bet copy() won't hog your memory.
Using the CURLOPT_FILE option to curl would probably also be more memory efficient than buffering into a variable so you can write later write to a file using another php filesystem function. If you try to store the contents of a 100mb file in a php variable, you will need an extra 100mb of memory. Not good.

You can test this stuff yourself with memory_get_peak_usage()

[LuckyB]

Thanks, how would I change:

<?php
/**
 * Uses cURl to save a remote file locally.
 *
 * @param String $sRemoteFile
 * @param String $sSaveTo
 * @return Boolean
 */
function saveRemoteData( $sRemoteFile , $sSaveTo )
{
    $rcUrlHandle = curl_init( $sRemoteFile );
    curl_setopt( $rcUrlHandle , CURLOPT_RETURNTRANSFER , true );
    $sData = curl_exec( $rcUrlHandle );
    return (file_put_contents( $sSaveTo , $sData )) ? true : false ;
}
?>

....to incorporate CURLOPT_FILE?

[AnthonySterling]

Thanks, how would I change:

<?php
/**
 * Uses cURL to save a remote file locally.
 *
 * @param String $sRemoteFile
 * @param String $sSaveTo
 * @return Boolean
 */
function saveRemoteData( $sRemoteFile , $sSaveTo )
{
    $rcUrlHandle = curl_init( $sRemoteFile );
    curl_setopt( $rcUrlHandle , CURLOPT_RETURNTRANSFER , true );
    $sData = curl_exec( $rcUrlHandle );
    return (file_put_contents( $sSaveTo , $sData )) ? true : false ;
}
?>

....to incorporate CURLOPT_FILE?

<?php
/**
 * Uses cURL to save a remote file locally.
 *
 * @param String $sRemoteFile
 * @param String $sSaveTo
 * @return Boolean
 */
function saveRemoteData($sRemoteFile, $sSaveTo)
{
    $rcUrlHandle = curl_init($sRemoteFile);
    $rFileHandle = fopen($sSaveTo, 'w');
    curl_setopt($rcUrlHandle, CURLOPT_FILE, $rFileHandle);
    curl_exec($rcUrlHandle);
    curl_close($rcUrlHandle);
    fclose($rFileHandle);
    return (filesize($sSaveTo) > 0) ? true : false ;
}
?>

[LuckyB]

Thanks that worked a charm!

Now I'm going to post another question that I have (in a new topic)..

Thanks again SilverB and CRMalibu.