SitePoint Sponsor

User Tag List

Results 1 to 7 of 7
  1. #1
    SitePoint Evangelist
    Join Date
    Mar 2011
    Location
    Bellingham, WA
    Posts
    450
    Mentioned
    1 Post(s)
    Tagged
    0 Thread(s)

    Can I determine how much data was transferred using php?

    Hello,

    I use shared hosting with a limited number of bytes that can be transferred per month. I was hoping to replicate the figures using php's filegetsize() function; every time a user lands on a page, I can get the file size and then add it to my database, and then sum the per page transfers. However, after thinking about it, I'm wondering if the computation is that straightforward. In particular, where would external css and js files fit into this? Aren't both of these cached in a user's browser? And if so, would the data from these files then just be transferred once (or until the cache is cleared)?

    Thanks!

    -Eric

  2. #2
    Utopia, Inc. silver trophy
    ScallioXTX's Avatar
    Join Date
    Aug 2008
    Location
    The Netherlands
    Posts
    9,032
    Mentioned
    152 Post(s)
    Tagged
    2 Thread(s)
    Do you have access to the server's access logs? If so, the easiest way would be to just parse those.

    If you don't have access to those, indeed you can't predict how many assets will be downloaded by the client because you can't "see" from PHP whether the client already has it cached or not.

    OR you need to run everything through PHP, even all static assets, and use If-Modified-Headers to respond with 403 if the file hasn't changed. See an example here http://stackoverflow.com/questions/1...not-being-sent. This does put extra strain on PHP though and will probably slow down your site a bit as well. I would only use this if all else fails.

    Are the limits very tough that you need to know this? Doesn't the hoster have a panel where you can see your data traffic, like cacti, awstats, etc?
    Rémon - Hosting Advisor

    Minimal Bookmarks Tree
    My Google Chrome extension: browsing bookmarks made easy

  3. #3
    SitePoint Evangelist
    Join Date
    Mar 2011
    Location
    Bellingham, WA
    Posts
    450
    Mentioned
    1 Post(s)
    Tagged
    0 Thread(s)
    Thank you for the detailed response. Basically, I'm going to be running a site where User A and User B (representing different) will be assessed a fee based on how much traffic flows through their "section" of the site (there's a potential for high volume). I was considering on running the website through Amazon's cloud services to ensure scalability. Would the "parse the server logs" option be a possibility were I to run my website through them? (And, this would be the first time that I run a php application through Amazon, so this may be a super noob question...)

  4. #4
    SitePoint Enthusiast
    Join Date
    Feb 2007
    Posts
    51
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    If URL rewriting is not enabled on your website already, you could maybe rewrite everything (html, css, js, images, ... but not php files) to custom PHP script that would cause server to load everything through that page. For example, if yourwebsite.com/index.html is requested, rewrite will load yourwebsite.com/serve.php?path=/index.html, then index.html requests /style.css and rewrite loads /serve.php?path=/style.css and so on... Then, serve.php would:

    a) check if file exists and if so
    b) read it's size, add it to the database
    c) set correct headers (cache control, content type: text/html, text/css, application/x-javascript, image/jpeg, image/png, ...)
    d) readfile('./'.trim($_GET['path'], '/'));
    e) die;

    Be careful around d) though, as someone could easily request /serve.php?path=/serve.php or another critical file, so you should always (double) check what is in the ?path=

  5. #5
    SitePoint Evangelist
    Join Date
    Mar 2011
    Location
    Bellingham, WA
    Posts
    450
    Mentioned
    1 Post(s)
    Tagged
    0 Thread(s)

    Logs may be the better approach

    Your solution makes sense to me and seems pretty clever! However, wouldn't viewing the server logs be a lot easier as suggested earlier by ScallioXTX?

    Do you have access to the server's access logs? If so, the easiest way would be to just parse those.
    I now think that I can just use PHP to go through the log each night, finding which files were accessed, and sum up the bytes transferred directly from the log. I found this http://beginlinux.com/blog/2010/07/a...b-server-logs/ to be a helpful start.

  6. #6
    SitePoint Enthusiast
    Join Date
    Feb 2007
    Posts
    51
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    Of course you can do that, provided that you have permissions to read access log file. Note that this file can be huge sometimes, as it logs every server request

  7. #7
    SitePoint Evangelist
    Join Date
    Mar 2011
    Location
    Bellingham, WA
    Posts
    450
    Mentioned
    1 Post(s)
    Tagged
    0 Thread(s)
    I appreciate the additional input!


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •