SitePoint Sponsor

User Tag List

Results 1 to 9 of 9
  1. #1
    SitePoint Mentor NightStalker-DNS's Avatar
    Join Date
    Jul 2004
    Location
    Cape Town, South Africa
    Posts
    2,876
    Mentioned
    46 Post(s)
    Tagged
    0 Thread(s)

    Web farm enviroment

    Hi guys

    I have a bit an issue at the moment and just wana know what the best solution would be.

    We have a website, which is quite big. We are hosting it on one server. Now we are gona be moving to a web farm environment.

    So all the images/videos/files that are uploaded by users needs to be stored on one server while the the website files on other servers for load balancing.

    What would be the best/easiest way for me to change the way things work? Coz atm I save it normally to a folder on the server and call it the same way.

    How would I upload to the other servers now and move all my current files there? Whats the best solution, has anyone done this b4?

    Should i create a map network drive and upload to that or a web service for uploads or both?

    Any suggestions would be much appreciated.

    Thanks

  2. #2
    SitePoint Zealot
    Join Date
    Mar 2005
    Posts
    162
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    disclaimer: i have never done this before...


    BUT! i do know a little about it, and we will let someone correct me if i am wrong. load balancers should take care of everything for you pretty much. you just need to set your encryption keys in the machine.config files on every server to be the same, so that session always validates, no matter what server the request is set to. then just make sure the app has all the needed files on its own server, unless you have a special location where you want to store your vids all in 1 place. storing vids on 1 specific location kind of defeats the purpose of load balancing IMO, but it does make things easier. but since youd probably be storing copies of vids on each of your n servers, when a user uploads the vid, you need to make sure its copied to every server. im not sure what application you could use to do this syncing, but of course you could program your own tool to do it , maybe. -if youre good.

    with this architecture, you wouldnt need to change anything with your code, if i know wtf im talking about
    Web Application Development & Maintenance

  3. #3
    SitePoint Wizard
    Join Date
    Feb 2007
    Posts
    1,274
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I would look into using DFS for creating a automatic replication namespace amongst the servers for the media files. I also think you should go for a balanced approach. Serving all media files from one server and app from another is bound to create some bottleneck issue at some point.

    Now, you may run into some race condition if replication is slow. I'm not sure if DFS can pull the file from a remote server if requested and "not replicated yet".

  4. #4
    SitePoint Zealot
    Join Date
    Mar 2005
    Posts
    162
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by honeymonster View Post
    I would look into using DFS for creating a automatic replication namespace amongst the servers for the media files. I also think you should go for a balanced approach. Serving all media files from one server and app from another is bound to create some bottleneck issue at some point.

    Now, you may run into some race condition if replication is slow. I'm not sure if DFS can pull the file from a remote server if requested and "not replicated yet".
    ...and what if your vid server goes down??

    uh owe... =x
    Web Application Development & Maintenance

  5. #5
    SitePoint Author silver trophybronze trophy
    wwb_99's Avatar
    Join Date
    May 2003
    Location
    Washington, DC
    Posts
    10,629
    Mentioned
    4 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by honeymonster View Post
    I would look into using DFS for creating a automatic replication namespace amongst the servers for the media files. I also think you should go for a balanced approach. Serving all media files from one server and app from another is bound to create some bottleneck issue at some point.

    Now, you may run into some race condition if replication is slow. I'm not sure if DFS can pull the file from a remote server if requested and "not replicated yet".
    Check out the recent Hansleminutes podcast on scalability--it is very, very good.

    Anyhow, to paraphrase (and expand a bit) from that 'cast, the "one static server, one ASPX box" is a very good approach. Issue is optimization--image server needs big, fast disks and lots of RAM for caching. Moreover, you can strip IIS down to just handling static files (especially w/ IIS7). The ASPX box can be pure processor and ram with smallish disks and be tuned to serve ASPX content. Other simple advantage is that most browsers will only do 2 simultaneous downloads from a given domain. So splitting the request across multiple domains can create a very prominent perceived increase in responsiveness to the end user.

    Personally I would avoid reliance upon windows services outside of the basics (IIS). Makes things trickier to QA if you have to do alot of environmental dependencies.

    As for making it work, biggest thing IMHO is to make sure all paths are configurable so you can re-point things around with ease.

  6. #6
    SitePoint Mentor NightStalker-DNS's Avatar
    Join Date
    Jul 2004
    Location
    Cape Town, South Africa
    Posts
    2,876
    Mentioned
    46 Post(s)
    Tagged
    0 Thread(s)
    Hey guys

    Thanks for the replies.

    Yes, the reason I asked this is that we have 2 servers. 1 with SQL on has 600gb hdd and 4gb ram.

    While the other 1 has 2 gb ram and 140gb hdd.

    So for this reason I want to store all all images and videos on the SQL server box and serve all the aspx files from the other smaller box.

    I can run load balance if i want, but dnt think its necessary.

    Would this approach of mine cause any problems? And how would I handle the uploads to another box?

    Thanks

  7. #7
    SitePoint Mentor NightStalker-DNS's Avatar
    Join Date
    Jul 2004
    Location
    Cape Town, South Africa
    Posts
    2,876
    Mentioned
    46 Post(s)
    Tagged
    0 Thread(s)
    PS. They all have SCSI drives.

    thanks

  8. #8
    SitePoint Author silver trophybronze trophy
    wwb_99's Avatar
    Join Date
    May 2003
    Location
    Washington, DC
    Posts
    10,629
    Mentioned
    4 Post(s)
    Tagged
    0 Thread(s)
    Hmm, I would let the SQL box be a SQL box and get another cheap box to serve as the images server. Never pollute the database server.

    For uploading, you should make a "save location" configuration value. For production, it could be a network share on the other server rather than a local path.

  9. #9
    SitePoint Mentor NightStalker-DNS's Avatar
    Join Date
    Jul 2004
    Location
    Cape Town, South Africa
    Posts
    2,876
    Mentioned
    46 Post(s)
    Tagged
    0 Thread(s)
    i c... thanks a lot for the reply.

    I will see wat I can do


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •