SitePoint Sponsor

User Tag List

Results 1 to 5 of 5
  1. #1
    SitePoint Zealot
    Join Date
    May 2004
    Location
    London
    Posts
    155
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Accessing a .csv file on another server

    Hello

    Iím working on an idea that involves using PHP to grab a .csv file and parse it...but the thing is that the .csv in question is on a third party server. Obviously I would be asking for permission to use the file...itís nothing dodgy. My question is from the technical side of things on the server what are the issues involved ....being that itís a third party environment that Iím not involved with.

    Ie. Is it only possible if they donít have a firewall or other restriction in place...or do I need to have CURL installed or my host needs to allow fsocketopen()
    I thought I could use a code like this for example

    Code:
    <?php
    $fileTemp = "http://www.somethirdpartysite.com/folder/thecsvfile.csv";
    $fp = fopen($fileTemp,'r');
    $datas = array()
    while (($data = fgetcsv($fp)) !== FALSE)
    {
         $data['productName'] = trim($data[0]);
         $data['spec'] = trim($data[1]);
         $data['imageLocation'] = trim($data[2]);
         $datas[] = $data;
    }
    ?>
    Or perhaps I could use phpís ftp function or something.

    If anyone could advise Iíd be grateful.

    Thanks


    Juan

  2. #2
    SitePoint Wizard silver trophybronze trophy Cups's Avatar
    Join Date
    Oct 2006
    Location
    France, deep rural.
    Posts
    6,869
    Mentioned
    17 Post(s)
    Tagged
    1 Thread(s)
    How you grab the file is up to you. cURL, fopen and so on.

    What I advise is that you do it in two operations. 1) Grab the file, store it. 2) Read the file, parse it and the store the desired output ready to be used in your pages.

    That way if the first part goes wrong, latency issues for example, then you can at least revert back to using old data.

    That first part can be done say, hourly with a cron job.

    What you want to avoid is a situation where you call a script which has to do all of these things one after the other without failure:

    *connect to the external server
    *grab the file
    *parse it
    *display the output

  3. #3
    SitePoint Zealot
    Join Date
    May 2004
    Location
    London
    Posts
    155
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Thank you.

    I do appreciate the reply but what I trying to find out is what issues might there be with the fact the .csv file is on a thirdy party server.

  4. #4
    SitePoint Zealot Sogo7's Avatar
    Join Date
    May 2011
    Posts
    129
    Mentioned
    1 Post(s)
    Tagged
    0 Thread(s)
    Provided you have consent from the owners of the other site and they have no defences in place to prevent the the file being read such as an IP or referer block using htaccess or some server side scripting then there should be no significant problems. The size of the csv file could in theory cause problems if it takes longer than 30 seconds to move it between the two servers but it's rare and you can overide(depends on your hosting) the runtime limit of the retrieval script to compensate.

    As for parsing, flat file storage or uploading the processed results to a database these functions will of course consume part of your hosted web servers hourly resource allowence (%ram, %cpu and SQL calls/connections), how much of course depends upon the amount of raw data, what you are doing to process it and how often this is happening.

    Hosts policies vary but most will suspended any site hitting the limits for those parameters shutting the site down till the owner can revise the offending script. So always start any web-content retrieval system slowly, watch your cpanel useage figures then increase the frequency. Remember to give your site some spare capacity as visitors & Googlebot can be unpredictable and a surge in either coming to a site can send you over useage limits.
    Lovelogic.net Personal Projects Pit - Spammers welcome

  5. #5
    SitePoint Zealot
    Join Date
    May 2004
    Location
    London
    Posts
    155
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Thank you very much. Your answer was a great help.


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •