SitePoint Sponsor

User Tag List

Results 1 to 17 of 17
  1. #1
    SitePoint Addict bbparis's Avatar
    Join Date
    Aug 2008
    Posts
    323
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Your opinion about sphider

    Hello,

    I want to know your opinion and advice for sure about Sphider as a search engine script ?? it's good to build a search engine with 10000 websites addresses ? ( I don't know, maybe it will take between 500 and 700 GB )

    what I need also to know, with large database, who will play the big rol, the server or the script ??

    Best regards to you all

  2. #2
    PHP Guru lampcms.com's Avatar
    Join Date
    Jan 2009
    Posts
    921
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I've never heard of it before. I just looked at its description breifly and I don't think I will be using it. I would rather use mnogosearch. Mnogosearch has been around for like 10 years, has extension of php and now has option to distribute the search index between up to 256 servers.

    It also has web spider, if you need this sort of thing as well as has all the features I see on sp hider

    www.mnogosearch.com
    My project: Open source Q&A
    (similar to StackOverflow)
    powered by php+MongoDB
    Source on github, collaborators welcome!

  3. #3
    SitePoint Addict bbparis's Avatar
    Join Date
    Aug 2008
    Posts
    323
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    ah nice, and what you mean when you said ( to distribute the search index between up to 256 servers ) can you explain more please ?

    thanks

  4. #4
    SitePoint Addict bbparis's Avatar
    Join Date
    Aug 2008
    Posts
    323
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    ohhh also it's not for free, I'm searching about something free or not to much expensive .

  5. #5
    PHP Guru lampcms.com's Avatar
    Join Date
    Jan 2009
    Posts
    921
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Mnogosearch is free and open source. They do have a non-free version for windows, but their main product is a free open source search engine based on mysql

    Distributed search index means it can spread the search index between several servers. this is helpful when your index is very large, like 100 million records. If you distribute 100 million records across 10 servers then each server holds only 10 million records, so the search will be performed faster. You should just read their manual, everything is explained there.
    My project: Open source Q&A
    (similar to StackOverflow)
    powered by php+MongoDB
    Source on github, collaborators welcome!

  6. #6
    SitePoint Addict bbparis's Avatar
    Join Date
    Aug 2008
    Posts
    323
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    woww it's nice if free, I will give it a try.

  7. #7
    Programming Team silver trophybronze trophy
    Mittineague's Avatar
    Join Date
    Jul 2005
    Location
    West Springfield, Massachusetts
    Posts
    17,035
    Mentioned
    187 Post(s)
    Tagged
    2 Thread(s)
    I've only tried phpSitemapNG and Sphider to crawl my own website for the purpose of creating sitemaps and looking for broken links.

    Both seemed a little slow even with the crawl depth set rather shallow.

    But they did provide a rudimentary sitemap for me to tweak. And I use robots.txt to remove redundant content from the picture.

  8. #8
    SitePoint Addict bbparis's Avatar
    Join Date
    Aug 2008
    Posts
    323
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    thanks for your opinion, and how you use robots.txt ? I don't know about it

  9. #9
    SitePoint Addict bbparis's Avatar
    Join Date
    Aug 2008
    Posts
    323
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Hello,

    I tested mnogosearch and it is so complicated to install, there is no video tutorial that show how install it ?

    regards

  10. #10
    PHP Guru lampcms.com's Avatar
    Join Date
    Jan 2009
    Posts
    921
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I agree, mnogosearch is complicated to install and to properly configure.
    Also installing the mnogosearch-php extension is not exactly easy. It's not for a beginner. But when you configure it correctly, it's a great search engine with many advanced features, much more advanced that any other database based search engine. And by the way, it's not limited to mysql, it supports Oracle, Postresql and other databases.
    My project: Open source Q&A
    (similar to StackOverflow)
    powered by php+MongoDB
    Source on github, collaborators welcome!

  11. #11
    SitePoint Wizard PHPycho's Avatar
    Join Date
    Dec 2005
    Posts
    1,201
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by lampcms.com View Post
    I agree, mnogosearch is complicated to install and to properly configure.
    Also installing the mnogosearch-php extension is not exactly easy. It's not for a beginner. But when you configure it correctly, it's a great search engine with many advanced features, much more advanced that any other database based search engine. And by the way, it's not limited to mysql, it supports Oracle, Postresql and other databases.
    Good to know such new thing.

  12. #12
    SitePoint Addict bbparis's Avatar
    Join Date
    Aug 2008
    Posts
    323
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by lampcms.com View Post
    I agree, mnogosearch is complicated to install and to properly configure.
    Also installing the mnogosearch-php extension is not exactly easy. It's not for a beginner. But when you configure it correctly, it's a great search engine with many advanced features, much more advanced that any other database based search engine. And by the way, it's not limited to mysql, it supports Oracle, Postresql and other databases.
    thanks a lot, there is no way to learn how to install it on my lunix dedicated server with PHP ??

  13. #13
    Programming Team silver trophybronze trophy
    Mittineague's Avatar
    Join Date
    Jul 2005
    Location
    West Springfield, Massachusetts
    Posts
    17,035
    Mentioned
    187 Post(s)
    Tagged
    2 Thread(s)
    Quote Originally Posted by bbparis View Post
    thanks for your opinion, and how you use robots.txt ? I don't know about it
    The robots.txt file let's crawlers know which "folders" (real or virtual) to not crawl. Because my blog and forum have multiple friendly URLs to get to the same content (i.e. topic/ thread/ archive/ etc.) it doesn't make sense to me to crawl them all. But I make sure the static pages and at least 1 URL for any post are all in the sitemap.

  14. #14
    SitePoint Wizard
    Join Date
    Mar 2008
    Posts
    1,149
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    There are also other dedicated search engine servers/libraries that may be of interest to you. Lucene, Solr, Sphinx, etc.

  15. #15
    SitePoint Addict bbparis's Avatar
    Join Date
    Aug 2008
    Posts
    323
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    thanks Mittineague for your answer.
    also thanks sk89q, I read about them, and I tested also Sphider-plus, it's nice but also has some problems with the setting, hemmm I don't know which one is good for me, I know installed the package, manage the things with database, but I'm not professional if the script is too hard to install like Mnogosearch for example

  16. #16
    SitePoint Wizard
    Join Date
    Mar 2008
    Posts
    1,149
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    You can also look into the other ones I noted. Though really, it doesn't get easier than Sphider to be honest.

  17. #17
    SitePoint Guru
    Join Date
    Jan 2005
    Location
    heaven
    Posts
    953
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by bbparis View Post
    thanks Mittineague for your answer.
    also thanks sk89q, I read about them, and I tested also Sphider-plus, it's nice but also has some problems with the setting, hemmm I don't know which one is good for me, I know installed the package, manage the things with database, but I'm not professional if the script is too hard to install like Mnogosearch for example
    Mnogosearch is fairly easy to install as far as I can tell. Some basic unix/linux knowlege is required. But the documentation seems to be pretty straight forward as to how to install it. I'd re-read chapter 2 of the documentation though if you choose to use it.
    Creativity knows no other restraint than the
    confines of a small mind.
    - Me
    Geekly Humor
    Oh baby! Check out the design patterns on that framework!


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •