Apache2 settings for a VERY busy website

Server Config
1

Hello,

I have been searching the internet for a couple of days now trying to find Apache2 configuration files that will allow my blog to run.

Recently we moved WordPress and MySQL onto Linux from WordPress to take advantage of the plugins like W3 Total Cache.

Testing it with light traffic it seem to work swimmingly. The pages were cached and handled by Apache quite nicely. So we took a punt and moved the live site across.

The first couple of hours were fine but when the traffic started to increase, due to a recent posted link on Reddit, we started getting traffic of around 1000 visitors every 15 mins. This brought the site to a standstill. Thankfully I was able to point the DNS for the site back to the windows machine, leaving the database running on the new server. And things seem to be running ok, not the best though.

I was wondering if someone could help me set up Apache properly on our new server e.g. tell me what levels I should set the max/min clients etc. And possibly even some settings in MySQL incase we got those wrong as well.

It would be great to walk this through with someone as there is nothing really out there that isn’t either too old, wrong version of linux, etc. and turning it into a great post somewhere e.g. my blog http://you.arenot.me

Details on our new server from ServerBeach:

Options Type Date
5 TB bandwidth 2011-09-15
250 GB 7200 3 Gb/s 3.5 inch SATA hdd 2011-09-15
250 GB 7200 3 Gb/s 3.5 inch SATA hdd 2011-09-15
Intel Xeon Dual-Core 5130 processor 2011-09-15
Intel Xeon Dual-Core 5130 processor 2011-09-15
3Ware 8006-2LP RAID Card raid 2011-09-15
8 GB (X7DBN) ram 2011-09-15

Add-on Services Type Date
Ubuntu 10.04 64 bit os 2011-09-15

So as you can see it’s a chunky server. But being a Windows developer I made the wrong assumption that Apache would be set up properly to handle our traffic…it seems I need to tell it how to handle traffic.

So please let me know if you can help,

Colin

2

In this particular situation I think I wouldn’t bother with trying to tune Apache, but rather use ngix or lighttpd instead, and put a varnish caching server in front of it. That combination should be able to deal with a lot more traffic than Apache alone ever could.
Swapping the hard disks out for 15k disks, or (even better) SSD’s, will also do wonders.

3

Unforunately replacing 1 tech I don’t really understand with 2 bits of tech I don’t understand, on a server I help out voluntarily with isn’t the answer I was looking for. It sounds awesome and I definitely read up on it though. Cheers for responding.

4

Well in that case you might just try giving Apache another spin. In terms of the min/max servers from what I understand there aren’t any rules; you just fiddle around with them until it looks right.
The trick is to min so low that that amount is able to handle the regular amount of traffic. So if you have like 5 visitors every minute normally you’d have about 20 min servers (visitors can request up to 4 resources from a server concurrently, 4*5=20). Then for the max servers you’d set a value to handle peak traffic. Don’t set this too high though, at some point it just doesn’t make sense anymore. A value of like 20.000 will be too much for mostly any server. Just start it at 100 or 150, stress the server a bit with apache bench and see far you can get.

Also make sure you configure w3 total cache properly; you’d especially want to enable stuff like database caching. Not to disk (which I read is also an option), but to something like APC, XCache, or Memcache (the systems are comparable, in case you’re wondering my personal favorite is memcache), seeing how the bottleneck of most any site are database reads (or a plethora of assets).