First, workflow-wise, you are typically better off having people develop locally then deploy to a central testing location. Everyone working on the same box is bad mojo for a few reasons. First, when someone messes up the box, no one can be working. Second, what if people are working on different parts of the project that aren’t working together well?
Insofar as the equipment goes, the main difference these days with server class chassis revolves around utter reliability through redundant systems and case designs for use in a rack. Ain’t much other difference beyond that.
You missed the point – a high configuration machine and a server have largely the same operational characteristics. They process the same amounts of bytes given similar configurations. Server machines tend to have beefier, more reliabile subsystems and better support.
Ultimately the question revolves around how much downtime costs you and what risk you are willing to take.