SitePoint Sponsor |
|
User Tag List
Results 1 to 8 of 8
-
Jun 17, 2008, 08:37 #1
- Join Date
- May 2005
- Location
- Houston, Texas, USA
- Posts
- 418
- Mentioned
- 0 Post(s)
- Tagged
- 0 Thread(s)
I may be in too deep- url scraping
I have a customer who wants to use a search script to search outside his site. How is this done? Am I talking about reinventing Google or what?
Opportunity favors the prepared mind.
-
Jun 17, 2008, 10:05 #2
- Join Date
- Apr 2006
- Posts
- 249
- Mentioned
- 0 Post(s)
- Tagged
- 0 Thread(s)
Sort of depends on what they want to accomplish - would just using "site:" in a regular google search be good enough, or do they want to display results on their own site? If so, and if the other site has it's own internal search, it may be possible to use a middleman script that sends the query to the other site, grabs the results, and then parses them to display on the client's site.
-
Jun 18, 2008, 04:14 #3
- Join Date
- May 2005
- Location
- Houston, Texas, USA
- Posts
- 418
- Mentioned
- 0 Post(s)
- Tagged
- 0 Thread(s)
They want to display on their own site. They want up-to-date results, daily I guess. They want to search within each link for keyword, not just titles or tags. It would be similar to asking someone to search sitepoint for the keyword, "php" and every record that contains php would be on the list.
Opportunity favors the prepared mind.
-
Jun 18, 2008, 07:40 #4
- Join Date
- Feb 2004
- Location
- Tampa, FL (US)
- Posts
- 9,854
- Mentioned
- 1 Post(s)
- Tagged
- 0 Thread(s)
is there a certain set of sites your customer wants to search? or do they want to search "all of the internet"?
Check out our new Industry News forum!
Keep up-to-date with the latest SP news in the Community Crier
I edit the SitePoint Podcast
-
Jun 18, 2008, 10:34 #5
- Join Date
- Apr 2006
- Posts
- 249
- Mentioned
- 0 Post(s)
- Tagged
- 0 Thread(s)
Perhaps google's ajax search API would help?
http://code.google.com/more/#products-search-ajaxsearch
-
Jun 19, 2008, 02:04 #6
- Join Date
- Mar 2008
- Location
- Germany
- Posts
- 24
- Mentioned
- 0 Post(s)
- Tagged
- 0 Thread(s)
Yes, I also would recommend not to write a Perl script but just use one of the big search engines and integrate them into your client's website.
Of course, using a CGI script, you could perform a Google search in the background, then with some regular expressions figure out the search results and display them in a custom format - but very likely this is against Google's use policy ;-) At least, unless you sign a contract with them paying for this service.STAR ENTERPRISE - Information Technology Is Our Universe !
Consulting | Software Development | Services
Internet: www.starenterprise.com - info@starenterprise.com
-
Jun 21, 2008, 00:45 #7
- Join Date
- May 2008
- Posts
- 15
- Mentioned
- 0 Post(s)
- Tagged
- 0 Thread(s)
Thousands of sites have been hacked recently.
Unmask Parasites: Check if your site is still yours.
-
Jun 21, 2008, 03:30 #8
- Join Date
- Mar 2008
- Location
- Germany
- Posts
- 24
- Mentioned
- 0 Post(s)
- Tagged
- 0 Thread(s)
Or does your client want something unlabelled, so used under their own name ? In this way, also Google can be your friend - they sell a search engine appliance. Very expensive, but may work. Check out their website for details.
STAR ENTERPRISE - Information Technology Is Our Universe !
Consulting | Software Development | Services
Internet: www.starenterprise.com - info@starenterprise.com
Bookmarks