Build a File Download Script in Perl

[Sally]

I think I agree with Paul :).

I used to use the print "" statement before I realised that I could use the print header(); to output HTML which by default return "Content-Type: text/html".

Anyway, this script looks great!

[sofy60]

that is funny.



I hadnt thought of that..

[Susie]

Hi,

For me, this script works fine on small files, but I'm having troubles downloading a large PDF (50 meg) that is in a secure area.

When I log in and click on the filename, Instead of being prompted to Save As, I get a 500 error. The error log shows "Premature end of script headers."

When I test with a small pdf, it works fine, but I get the above error with my large pdf. I tried it with both IE and Firefox, same thing.

I Googled and found some info about a problem with IE 6 and downloading a file in a protected area:
http://www.openrdf.org/issues/browse/SES-63

But in my case, the problem also happens in Firefox, and it doesn't happen with small files.

Any ideas? Thanks.

[Chris]

Suzie

That can be due to other issues such as your host not having certain globals activated or your script eceeding the maximum allowed permissions. Ie CHMOD set to 777 but your host supports a max of 755.

[Anonymous]

I have used this script for a couple years. It worked fine and I was basically satisfied. It was used to download MP3 files (averaging 7MB). Since our website's traffic is very light, I was not very concern about the momery usage issue. However our website was terminated by the host company due to over-use-resource reason. I still cannot image there would be so many simultanouse downloads that caused the problem. Could this caused by not releasing memory until long time timeout? I do include the Close line. Any body tested Comment#10's solution?

[redbone]

I was not very concern about the momery usage issue. However our website was terminated by the host company due to over-use-resource reason.

This script loads the whole file into memory before sending it to your site visitors. Since your mp3 files are 7mb, I can understand your host getting a bit grumpy. The alternative is to send out the file in small chunks. There is an example from Brian on page one of this thread that does this.

[m23]

What about a simple force download script? Would this work?

#!/usr/bin/perl -wT 
print "Content-Type:application/octet-stream\n\r\n\r";
open ( FILE, "<$file" );
print $_ while ( read( FILE, $_, 1024 ) );
close FILE;

[Remy]

A nice way of putting a download link in the HTML is:

<a href="/download.cgi/file.zip?ID=jan_file">download</a>

This will prompt the brower to see the link as a .zip file.

Also - remember to do some error checking to stop people from downloading ID=../../../private_file.txt etc.

[nick]

will this work as a file on my computer? coz its not working when i test it out

[KevinR]

will this work as a file on my computer? coz its not working when i test it out

Yes it will if you have an http server setup and running on your computer and it's configured to execute perl/cgi scripts.

[Tim Eveleigh]

Do you have any hints for downloading a very large file (100MB) using this sort of method. The problem is that the browser times out before the file has been copied into the array and so the 'save as...' dialog is never displayed!

Cheers

Tim

[praveen]

I faced the problem trying to download a 20M file !
I tried the following intead of reading the whole (huge) file into a single array ! (@fileholder = <DLFILE>;)

#open the file
#read the file in block of 1K and print to # stdout
while( sysread(DLFILE,$frm_data,1024) )
{
print $frm_data
}
#close the file.

should work

[pugface]

Hi there,

I'm trying to use this to download files stored on my server outside of the root.

Everything looks as though it should be working, but I always get "The server can't open the file: No such file or directory", which seems to indicate that I'm not setting the directory properly.

The directory is at /home/sites/my_domain/csv which is how I've got it set in the cgi.

Any advice?

Thanks.

[KevinR]

post your code.

[KevinR]

The code as posted in this article should not work. The -T switch on the shebang line compiles perl in Taint mode. This means any data that comes from outside the script has to be untainted before it is used in any way that could be insecure. The code in the script imports the parameter $ID from the form:

$ID = param('ID');

and then proceeds to use it to open a file:

open(DLFILE, "<$files_location/$ID") || Error('open', 'file');

opening a file is an insecure operation. The script should terminate with an error along the lines of:

Insecure dependency in blah blah.......

[KevinR]

A mischevious person could also use this script to try and download any file from the server that can be accessed from the root directory:

/usr50/home/webdata/public_html/

all they would have to do is send something like this to the script in the ID field:

cgi-bin/foo.pl

and if there were a foo.pl script on the server in the cgi-bin the script would download it. Hopefully there would be no sensitive data contained in the script, like names and passwords.

Although this might seem unlikely, it is very possible and is an example of how insecure scripts can be used to hack into websites, right through the front door.

I will post a better version of this script in this thread.

[KevinR]

Use this script instead of the one in the article unless and until the issues I have mentioned above are fixed.

#!/usr/bin/perl -T
 
use strict;
use warnings;
use CGI; 
# Uncomment the next line only for debugging the script.
#use CGI::Carp qw/fatalsToBrowser/;
 
# the path to where the downloadable files are. 
# prefereably this should be above the web root folder.
my $path_to_files = '/home/you/downloads';
 
my $q = CGI->new;
 
my $file = $q->param('file') or error('Error: No file selected.');
 
# here we untaint the filename and make sure there are no characters like / 
# in the name that could be used to download files from any folder on the website.
# personally I don't even recommend you pass filenames to the script, but that
# gets too complicated for a simple example of a download script. 
if ($file =~ /^(\w+[\w.-]+\.\w+)$/) {
   $file = $1;
}
else {
   error('Error: Unexpected characters in filename.');
}	
 
if ($file) {
  download($file) or error('Error: an unknown error has occured. Try again.');
}  
 
sub download {
 
   # Uncomment the next line only for debugging the script 
   #open(DLFILE, '<', "$path_to_files/$file") or die "Can't open file '$path_to_files/$file' : $!";
 
   # Comment the next line if you uncomment the above line 
   open(DLFILE, '<', "$path_to_files/$file") or return(0);
 
   # this prints the download headers with the file size included
   # so you get a progress bar in the dialog box that displays during file downlaods. 
   print $q->header(-type            => 'application/x-download',
                    -attachment      => $file,
                    'Content-length' => -s "$path_to_files/$file",
   );
 
   binmode DLFILE;
   print while <DLFILE>;
   close (DLFILE);
   return(1);
}
 
# This is a very generic error page. You should make a better one.
sub error {
   print $q->header(),
         $q->start_html(-title=>'Error'),
         $q->h1($_[0]),
         $q->end_html;
   exit(0);
}

Note that this does not have the download tracking log file like the script in the article. But that log file was nearly useless anyway (IMHO) as all it did was append the name of the file to the end of the log. This script should also log all errors to a file and record some data that would be helpful in tracking abuses (or attempted abuses) of the script.

[Dan]

Great Script, just what I was looking for. Thanks

[joec_]

Hi,

the new script i used keeps returning a 'bad file descriptor' error? on line 11, where the file is assigned to the $file variable?

Any ideas?
Thanks
Joe

[KevinR]

Do you mean this line?

my $path_to_files = '/home/you/downloads';

You have to determine what the correct value on the right should be for your server. '/home/you/downloads' is just a generic example.

Don't put a trailing slash "/" on the end:

'/home/you/downloads/'

although I don't think that would cause the error you are getting.

[joec_]

thanks for the reply. I know it was a generic example i changed it to my htdocs directory. still getting that error. whats interesting to note is that the script works at home, but not at work and yet the config is almost identical.

Joe

[KevinR]

Check the server error log and see if there is anymore information. There can be so many different setups on servers, and permissions problems, its very hard to say what your specific problem running the script at work could be.

[antiplex]

hi everyone!

i just came across the tutorial and the forum and found it quite helpful.
i think the (security)problems with the original script should almost be discussed more prominently; if one reads the tutorial he/she could easily understand this to be the state of the art. maybe some admin should place a well visible hint at the page of the tut.

anyways, i adopted many of the ideas to my own script which finally works very well.
still, there remains one thing i am not really sure about:
if someone clicks on a link to my download.cgi (with appropiate parameters of course), the transfer starts and a blank white page is displayed in the browser.
i tried adding a 'target="_blank" to the link, but result is the same...

is there a way to send a html-page additional to the file which would get displayed in the browser window?
how would i do that? can i just send another header (one of type 'application/x-download' and another 'text/html')?
is the order important?
i guess somehow the end of the first part has to be signalled, otherwise the second header might be seen as part of the first? i guess a simple 'sleep 1' or similar won't do it...

i'm pretty sure this issue has been brought up before. i just don't know what to search for, so if there is no simple answer to this, could anyone maybe point me to somewhere where this might be solved?

thanks a lot & greez - anti

[KevinR]

You can't mix the header content-types. I'm not sure how to do what you are trying.

[antiplex]

hmmm, thats what i almost expected, thanks for making this clear.

well, i still wonder how many (big) sites are doing this instead.
the pattern seems the same: you get on a page where downoadable files are listed, the links point to a dl.cgi, parameter is some form of id.

when you click on the link, a new page is loaded (download page) where usally something like "your download will start shortly, in not, please <do something>" and then the download dialog pops up on my side.

so if there is only one header sent in such a case, i guess its the one for the page so is the data follwing. but how is this next step triggered?
another thing coming to my mind here is that maybe a referer or refresh is used, but i don't know much about either. any idea how this could work?