SitePoint Sponsor

User Tag List

Results 1 to 3 of 3
  1. #1
    SitePoint Addict
    Join Date
    May 2003
    Location
    Lancaster
    Posts
    240
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Search Engine Result feeds (OpenSearch RSS 2.0)

    I've been working on mozdex.com for a while now and just introduced the capability to do RSS/XML queries and return an RSS 2.0 feed.

    http://www.mozdex.com/opensearch

    I'm looking for sites who are interested in using/testing this service to work out any kinks and potential issues as well as to give feedback that we can pass along to amazon.com/a9.com.

    Especially interested in making sure that all elements are clean, data is good, response is quick and such (summaries compiled from spidered pages are hard to cleanup sometimes, and that can be an issue with xml data..). We will be moving to a servlet and making some updates (requiring referr ip's for validation and such).

  2. #2
    SitePoint Member
    Join Date
    Mar 2005
    Posts
    15
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Hello,

    Are all your results coming from DMOZ?
    Is it a meta search engine?

  3. #3
    SitePoint Addict
    Join Date
    May 2003
    Location
    Lancaster
    Posts
    240
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    No, they're spidered by our robot.. We are working on indexing about a million sites a day right now. The index that is up right now will be refreshed over the weekend to fix some of the summary data as well as some tweaks we have been testing.

    http://www.mozdex.com/bot.html

    (check your referrs, you may see it coming)

    We seeded our database from dmoz.org data so we would have a good starting point to crawl from. We generally follow up to 100 outbound links per page so we are moving across sites quickly.


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •