SitePoint Sponsor

User Tag List

Page 1 of 3 123 LastLast
Results 1 to 25 of 66
  1. #1
    SitePoint Addict CrabbyX's Avatar
    Join Date
    Aug 2004
    Location
    England, UK
    Posts
    249
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Causes of High Server Load

    Hello,

    I've recently been informed by my host that my site is causing very high server load, although I don't really know how I can find out what is actually causing it. I tested to make sure and yes, its my site (and my coding).

    The speed of the scripts is fine, and I have optimised all of 20 or so mySQL queries to less than 0.001 seconds. I use mod_rewrite quite a lot too, but I don't see how this could cause such a rise in server load.

    Is there any way of finding out whether a certain function or operation is causing the high load?
    michael.Crabbe

  2. #2
    SitePoint Addict Synaesthesiac's Avatar
    Join Date
    Aug 2004
    Posts
    359
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Are you considering the fact that it could be lots of visitors?
    Danielion
    Imagine a world without hypothetical scenarios...

  3. #3
    SitePoint Addict CrabbyX's Avatar
    Join Date
    Aug 2004
    Location
    England, UK
    Posts
    249
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    My visitor count is around 1,000 visitors per day average whom make around 35,000 pageviews. As I've heard from my host there are other sites on the server with this many visitors and apparently they do not incur such load.
    michael.Crabbe

  4. #4
    <? echo "Kick me"; ?> petesmc's Avatar
    Join Date
    Nov 2000
    Location
    Hong Kong
    Posts
    1,508
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    IS that 20 queries on ONE page??

  5. #5
    SitePoint Addict CrabbyX's Avatar
    Join Date
    Aug 2004
    Location
    England, UK
    Posts
    249
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Err yes. I could probably reduce it to 15 but that would replaced by some extra include()s.
    michael.Crabbe

  6. #6
    <? echo "Kick me"; ?> petesmc's Avatar
    Join Date
    Nov 2000
    Location
    Hong Kong
    Posts
    1,508
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Maybe we can help you reduce that? Personally I have 2 queries on my pages (1 for articles and 1 for categories) and I have reduced that by Caching my pages as HTML pages. Caching should definately help you, unless your data has to be truly dynamic.

  7. #7
    SitePoint Addict CrabbyX's Avatar
    Join Date
    Aug 2004
    Location
    England, UK
    Posts
    249
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Yeah I'm caching a lot now, but I'm having to include() the cached components. Is this faster/lighter on the server?
    michael.Crabbe

  8. #8
    <? echo "Kick me"; ?> petesmc's Avatar
    Join Date
    Nov 2000
    Location
    Hong Kong
    Posts
    1,508
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Including isn't too intensive. Well if you have only a few it's ok. But it looks like that 20 query page could be the problem.

    Ask you host whether the server load is due to mysql or just in general (ie. because of many page views to php, etc).

  9. #9
    SitePoint Addict CrabbyX's Avatar
    Join Date
    Aug 2004
    Location
    England, UK
    Posts
    249
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I have quite a lot of includes--more than queries, however, my host has suggested the problem could be to do with mySQL.

    I've reduced the amount of queries to 9 average now, but most have been replaced with a new include.
    michael.Crabbe

  10. #10
    <? echo "Kick me"; ?> petesmc's Avatar
    Join Date
    Nov 2000
    Location
    Hong Kong
    Posts
    1,508
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Are you just moving the query into an include? If so, that defeats the purpose.

    Are you using Joins in your queries effectively and also re-using your data so that you don't need to constantly query the database. You might want to subject you table structure and queries to the MySQL forum here and see if they can help you reduce it.

  11. #11
    SitePoint Addict CrabbyX's Avatar
    Join Date
    Aug 2004
    Location
    England, UK
    Posts
    249
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    petesmc: Let me make this clear: I'm not stupid (moving the query to the include, ).

    I've already optimised the queries themselves quite a lot and thus do not take up much time and hopefully this means not much server load is generated.
    michael.Crabbe

  12. #12
    <? echo "Kick me"; ?> petesmc's Avatar
    Join Date
    Nov 2000
    Location
    Hong Kong
    Posts
    1,508
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Well then only thing I can think of is to move to a new server. You are serving 1 million page views a month, I think that warrants a dedicated server.

  13. #13
    SitePoint Evangelist CapitalWebHost's Avatar
    Join Date
    Apr 2003
    Location
    Albany, NY
    Posts
    417
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Joins are your friend. Learn them. Use them. See the Join. Feel the join. BE the join!!


  14. #14
    Massimiliano Bruno Giordano sid egg's Avatar
    Join Date
    Aug 2004
    Location
    Canada
    Posts
    1,280
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Image handling is also a HUGE {$server load}++'er... If you are using GD or IM anywhere ensure you cache EVERYTHING you make iwth it.
    GamesLib.com - the slickest, most complete and
    easily navigatible flash games site on the web.

  15. #15
    One website at a time mmj's Avatar
    Join Date
    Feb 2001
    Location
    Melbourne Australia
    Posts
    6,282
    Mentioned
    1 Post(s)
    Tagged
    0 Thread(s)
    If the queries all take less than a millisecond, then 20 queries on one page should not be a problem. I would guess that the problem would be somewhere else.

    If you have SSH or telnet access, you can use ab (Apache Benchmark) to test how much time each page request is taking on the server. Note that you must use the copy of ab that's on the same server as the site itself for the results to be meaningful. Usually ab is in /usr/local/apache/bin

    ab -n 100 http://www.example.com/

    That command will request http://www.example.com/ 100 times, and come up with an average load time per request.

    It is the median time that is most meaningful on a crowded server. I'd try to keep it below 100 ms (0.1 seconds) per page request.
    [mmj] My magic jigsaw
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    The Bit Depth Blog Twitter Contact me
    Neon Javascript Framework Jokes Android stuff

  16. #16
    SitePoint Addict myrdhrin's Avatar
    Join Date
    Jul 2004
    Location
    Montreal
    Posts
    211
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    mmj, and how do you do it if you don't have telnet access?
    Jean-Marc (aka Myrdhrin)
    M2i3 - blog - Protect your privacy with Zliki

  17. #17
    One website at a time mmj's Avatar
    Join Date
    Feb 2001
    Location
    Melbourne Australia
    Posts
    6,282
    Mentioned
    1 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by myrdhrin
    mmj, and how do you do it if you don't have telnet access?
    You would probably have to write a script to do it yourself. However you'd have to make sure that the benchmarking itself doesn't take up huge amounts of their server load too of course

    You might want to set up an Apache server on your local machine and test the whole thing on your local machine. It won't give you an accurate idea of how well it will run on a busy shared server but it will give you an idea of how long each page takes to load.
    [mmj] My magic jigsaw
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    The Bit Depth Blog Twitter Contact me
    Neon Javascript Framework Jokes Android stuff

  18. #18
    SitePoint Wizard realestate's Avatar
    Join Date
    May 2004
    Posts
    1,092
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    There is a big traffic jump on arcadexl.com according to alexa.

  19. #19
    Fully Sweet Car noddy's Avatar
    Join Date
    Aug 2002
    Location
    Perth, Western Australia
    Posts
    759
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Have you got a lop off loops ot repeative actions happening?

  20. #20
    SitePoint Addict CrabbyX's Avatar
    Join Date
    Aug 2004
    Location
    England, UK
    Posts
    249
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    CapitalWebHost: Yeah, I try to make use of JOINs in mySQL. I don't think there are any more oppurtunities to use JOINs on the home page, though.

    mmj: I tried to get my host to install APD though for some reason he couldn't make it compatible. I will be able to speak with my host about using ab.

    realestate: There sure is a big jump, going up by 150 uniques per day per week. Too bad I keep getting my site suspended for this load

    noddy: I have quite a few mysql_fetch loops, though this has been reduced now due to caching.

    There are still a substantial amount of queries on the page though they are mostly to do with sessions (not-cacheable) and the latest scores which cannot really be cached to provide the effect I need.
    michael.Crabbe

  21. #21
    Fully Sweet Car noddy's Avatar
    Join Date
    Aug 2002
    Location
    Perth, Western Australia
    Posts
    759
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Maybe its just your hosts system?

  22. #22
    SitePoint Zealot sanka69's Avatar
    Join Date
    Apr 2003
    Posts
    115
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Following up on noddy, my (former) host took my website down claiming that it was putting their servers under too much load. To cut a long story short, the host discovered faulty hardware was their problem.

  23. #23
    SitePoint Wizard cranial-bore's Avatar
    Join Date
    Jan 2002
    Location
    Australia
    Posts
    2,634
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Just to add to what petesmc said, if you are doing about 1M page views per month, with 15-20 queries each time, that is 15-20 million queries, or about 5 - 7 queries per second on average.
    while this is obviously less than MySQL is able to respond to it may be that your hosting (shared I assume) just considers 5 queries a second to be too many and too intensive for one site.

    If you've already done all the caching you can, and there are no actual mistakes in the code that cause more work than necessary to be done I don't really know what else you can do except get a dedicated server. I guess though certain PHP settings such as turning off register globals and magic quotes could give some performance boost if they are on now, but I imagine such tweaks would be marginal.

  24. #24
    SitePoint Wizard HarryR's Avatar
    Join Date
    Dec 2004
    Location
    London, UK
    Posts
    1,376
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Smile Causes of high server load

    Something you should consider when looking at the causes of high server load on shared hosting systems are the following:

    Do they use cpanel, webppliance, plesk or some other 'generic' hosting control panel. Many of these tend to run CGI scripts and PHP etc. in the most secure way possible, this means there is a large performance hit for every page visited (see cgiwrap, phpsuexec etc.).

    You could suggest to your webhost to install PHP a php bytecode cache (ionCube PHPA, Zend Performance Suite, mmcache etc.), and enable query caching in MySQL (a 5 or 6mb cache should be ok). Although this will decrease the security of all PHP applications running (they'll be required to run using mod_php or via fastcgi), the performance increase will be very noticable.

  25. #25
    SitePoint Enthusiast HeshamAmiri's Avatar
    Join Date
    Oct 2003
    Location
    Dubai
    Posts
    75
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    If you are on a virtual server, then the cause of the slowdown might not be your site!


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •