Server solution for file storage service


we are running a small file hosting service and we need some advice about how to upgrade our storage servers capacity.

What solution do you think is better -

  1. all servers with the same storage capacity working as both web server and a storage
  2. one (or how much is needed for our visitors) web server with no storage capacity and the rest will be storage servers(arrays) with much larger capacity just distributing requested files

Just info - right now we have about 50TB storage capacity and web server must handle up to 70 thousand unique visitors per day.

Which option do you think is better?
How other services may handle this?
Or any other ideas - suggestions about what we need to think through? please share anything, even if its very general or sources to learn from.

We do have a working solution but cause we started not long ago, we are still looking for the best way how to approach this issue.

thank you very much for your help

Everyone else quit rolling their own and started using Amazon S3 or Rackspace to grow their storage . . .

yes we know about that, but we are more of a local service (central Europe) and the pricing is out of our options for now (will see how it goes this year-)

Another option is to have a small amount of storage on the main web server, but only cache popular high demand files - you could run something like squid on this server. This server can be specced higher with fast disk, with cheaper low spec servers for long term storage.

I would go for keeping the storage and the web server separate myself - that way if a web front end dies, its easy to add / replace without affecting any file storage.

thank you for your reply, great idea about the squid cache, we will definitely look into it

TimIgoe> yes our solution works pretty much like you said, but we were looking for some other ideas but thanks anyway