I have a question, given the following architecture of a server stack, where I have a centralized db, 2 webservers that can handle 1000 request each, along with server side caching and a load balancer installed. What happens if it gets 3000 request at a given time. How can we make it handle more traffic than it can take?
Thanks in advance for any suggestions, inputs.
Since you have a load balancer, why can’t you keep adding a new server?
If you can’t then I would try to limit HTTP Pool connections. Each server configures differently but I’m sure Google will help. I really doubt your 2 server can handle 2000 requests at the same time. It probably has default HTTP Pool connections. So, technically your 2 server can handle 3000 requests but your client may have to wait a bit or they get an error from browser timeout. If some of the requests take a long time then you may want to consider async request so that they can check the results later.
Can cloud be a solution to this?
There isn’t a straight, default answer to this – you really need to know what the bottleneck is and how things are breaking. IE, you could well get the web tier to handle more requests directly but that just puts more load on the data tier which might not be able to handle the increased load.