SitePoint Sponsor

User Tag List

Results 1 to 5 of 5
  1. #1
    Bored One boredboi's Avatar
    Join Date
    Jul 2000
    Posts
    238
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    search engine database size

    hi...
    how much space do search engines use up?
    50 gb? more?
    how do they manage to pay for that amount of space?
    just curious...
    thanks.

  2. #2
    SitePoint Enthusiast bstrawse's Avatar
    Join Date
    Jul 2001
    Location
    Taunton, MA
    Posts
    81
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: search engine database size

    Originally posted by boredboi
    hi...
    how much space do search engines use up?
    50 gb? more?
    how do they manage to pay for that amount of space?
    just curious...
    thanks.
    It depends.

    The compressed XML listings for dmoz are about 750MB, once you expand them, it's several gigabytes.

    How do they pay for it? Revenue

    Bryan

  3. #3
    Serial Publisher silver trophy aspen's Avatar
    Join Date
    Aug 1999
    Location
    East Lansing, MI USA
    Posts
    12,939
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    hard drive space is dirt cheap nowadays. You can get a 100gb hard drive for like $300.

    But also consider google. Not only does google have it's index of over 1.3 billion pages, but it also has a saved copy of each one of those pages.

    If you think about how much they have stored its unbelievable.
    Chris Beasley - I publish content and ecommerce sites.
    Featured Article: Free Comprehensive SEO Guide
    My Guide to Building a Successful Website
    My Blog|My Webmaster Forums

  4. #4
    SitePoint Enthusiast Stallion's Avatar
    Join Date
    Jan 2001
    Location
    Cumberland, RI, US
    Posts
    97
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I believe Google is in the terabytes now. They have a 4000+ server farm to distribute the DB load.

    How do they support it? Google buys/builds lots of low-end machines, dirt cheap, and links them together for quite the network. Hard drive space is cheap, and a one-time purchase. What's killer, for most, is bandwidth. Google compresses their pages on the fly, and uses a very simplistic design to reduce network transfer. KISS has worked great for them, and they turned a profit. :-)
    /* Chris Lambert - chris@php.net
    WhiteCrown Networks, CTO - Web Application Security
    vBulletin, Security Programmer - Instant Community
    */

  5. #5
    SitePoint Enthusiast bstrawse's Avatar
    Join Date
    Jul 2001
    Location
    Taunton, MA
    Posts
    81
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Originally posted by Stallion
    I believe Google is in the terabytes now. They have a 4000+ server farm to distribute the DB load.

    How do they support it? Google buys/builds lots of low-end machines, dirt cheap, and links them together for quite the network. Hard drive space is cheap, and a one-time purchase. What's killer, for most, is bandwidth. Google compresses their pages on the fly, and uses a very simplistic design to reduce network transfer. KISS has worked great for them, and they turned a profit. :-)
    And they're using Linux... what more can I say
    Bryan


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •