SitePoint Sponsor

User Tag List

Results 1 to 8 of 8
  1. #1
    SitePoint Evangelist
    Join Date
    May 2005
    Location
    Houston, Texas, USA
    Posts
    418
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    I may be in too deep- url scraping

    I have a customer who wants to use a search script to search outside his site. How is this done? Am I talking about reinventing Google or what?
    Opportunity favors the prepared mind.

  2. #2
    SitePoint Addict
    Join Date
    Apr 2006
    Posts
    249
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Sort of depends on what they want to accomplish - would just using "site:" in a regular google search be good enough, or do they want to display results on their own site? If so, and if the other site has it's own internal search, it may be possible to use a middleman script that sends the query to the other site, grabs the results, and then parses them to display on the client's site.

  3. #3
    SitePoint Evangelist
    Join Date
    May 2005
    Location
    Houston, Texas, USA
    Posts
    418
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    They want to display on their own site. They want up-to-date results, daily I guess. They want to search within each link for keyword, not just titles or tags. It would be similar to asking someone to search sitepoint for the keyword, "php" and every record that contains php would be on the list.
    Opportunity favors the prepared mind.

  4. #4
    reads the ********* Crier silver trophybronze trophy longneck's Avatar
    Join Date
    Feb 2004
    Location
    Tampa, FL (US)
    Posts
    9,854
    Mentioned
    1 Post(s)
    Tagged
    0 Thread(s)
    is there a certain set of sites your customer wants to search? or do they want to search "all of the internet"?
    Check out our new Industry News forum!
    Keep up-to-date with the latest SP news in the Community Crier

    I edit the SitePoint Podcast

  5. #5
    SitePoint Addict
    Join Date
    Apr 2006
    Posts
    249
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Perhaps google's ajax search API would help?

    http://code.google.com/more/#products-search-ajaxsearch

  6. #6
    SitePoint Member
    Join Date
    Mar 2008
    Location
    Germany
    Posts
    24
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Yes, I also would recommend not to write a Perl script but just use one of the big search engines and integrate them into your client's website.

    Of course, using a CGI script, you could perform a Google search in the background, then with some regular expressions figure out the search results and display them in a custom format - but very likely this is against Google's use policy ;-) At least, unless you sign a contract with them paying for this service.
    STAR ENTERPRISE - Information Technology Is Our Universe !
    Consulting | Software Development | Services
    Internet: www.starenterprise.com - info@starenterprise.com

  7. #7
    SitePoint Member
    Join Date
    May 2008
    Posts
    15
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by Chri View Post
    I have a customer who wants to use a search script to search outside his site. How is this done? Am I talking about reinventing Google or what?
    Why not use Google? I.e. AdSense for Search. It will search outside the site, it will display results on his site, and the site owner can earn some money. And it only takes about 5 minutes to add it.
    Thousands of sites have been hacked recently.
    Unmask Parasites: Check if your site is still yours.

  8. #8
    SitePoint Member
    Join Date
    Mar 2008
    Location
    Germany
    Posts
    24
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Or does your client want something unlabelled, so used under their own name ? In this way, also Google can be your friend - they sell a search engine appliance. Very expensive, but may work. Check out their website for details.
    STAR ENTERPRISE - Information Technology Is Our Universe !
    Consulting | Software Development | Services
    Internet: www.starenterprise.com - info@starenterprise.com


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •