Alright, I've been having a constant debate with my server tech about this.
As of right now, I have one web server that is a dual 5500 series (so about 2 generations back) with 64GB ram that has mysql disabled. For MySQL I have a primary update-only server, with two slave mysql servers for reading-only. So four servers total.
Even with this setup, I feel my site getting bogged down at times when traffic spikes. To top it off, we just launched our mobile apps and are expecting them to do a few million more mysql requests daily with a few million GD Library requests as well (so that portion will be on the web server).
I've been wanted to upgrade the web server to a dual e5-2620 series, but I'm told that I should instead remove cpanel/whm to free up resources and stick with the hardware I got. I feel like my server hardware is outdated, and that there must be traffic-handling advantages to moving up from 5500 series to e5-2600 series (2620 or 2640).
I think you would have better luck moving to a cloud hosting solution, Amazon, Microsoft Windows Azure, either or. Both will allow your site to scale to demand and in turn only pay for what you actually use. Your database for example, you won't need to fake load balancing (in PHP) with Amazon or Azure. You could also set it up to also offload your image manipulation to a second "instance". (Azure refers to instances as roles, web roles and worker roles. A Web role includes a web server pre-configured, a worker role is just a bare OS.)
Logic without the fatal effects.
All code snippets are licensed under WTFPL.