SitePoint Sponsor

User Tag List

Results 1 to 6 of 6
  1. #1
    SitePoint Enthusiast mildfoam's Avatar
    Join Date
    May 2010
    Location
    Australia.
    Posts
    56
    Mentioned
    1 Post(s)
    Tagged
    0 Thread(s)

    Scrapping site search result with PHP (cUrl)

    hi all,


    i am going to create very basic web apps that fetch (scrape) all site search result. here is an example from me : click here. You can see that I've send any query to this site from the left menu (Rincian Pencarian Anda) then it return some results list.

    Here is my big problem, i want to fetch all search result from this site to my site (don't worry about copyright cause i have read any rules that allowed to fetch the site result).

    I am going to use PHP with cURl exts.

    Thanks mate,
    Object Oriented Programming Fans Boy ?

  2. #2
    Keeper of the SFL StarLion's Avatar
    Join Date
    Feb 2006
    Location
    Atlanta, GA, USA
    Posts
    3,748
    Mentioned
    71 Post(s)
    Tagged
    0 Thread(s)
    So... what's your specific question?
    Never grow up. The instant you do, you lose all ability to imagine great things, for fear of reality crashing in.

  3. #3
    SitePoint Enthusiast mildfoam's Avatar
    Join Date
    May 2010
    Location
    Australia.
    Posts
    56
    Mentioned
    1 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by StarLion View Post
    So... what's your specific question?
    i want to fetch the result content (html) from that site then show them in my own site.

    thanks
    Object Oriented Programming Fans Boy ?

  4. #4
    . shoooo... silver trophy logic_earth's Avatar
    Join Date
    Oct 2005
    Location
    CA
    Posts
    9,013
    Mentioned
    8 Post(s)
    Tagged
    0 Thread(s)
    Okay...and what exactly is stopping you from doing that?
    Btw, questions are followed by a "?" question mark, none of your post have one.
    Logic without the fatal effects.
    All code snippets are licensed under WTFPL.


  5. #5
    SitePoint Enthusiast mildfoam's Avatar
    Join Date
    May 2010
    Location
    Australia.
    Posts
    56
    Mentioned
    1 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by logic_earth View Post
    Okay...and what exactly is stopping you from doing that?
    Btw, questions are followed by a "?" question mark, none of your post have one.
    OK now i am builded a form that the visitor can choose the same option with the ticket.com site. So, i will need to send this option value to next page that will query to ticket.com and show the result to my site (i hope i can get the search result only not all website content)

    my site (form with some option list) -> send the value to ticket.com -> get the result and display to my site.

    is this need curl function? thank you sir.
    Object Oriented Programming Fans Boy ?

  6. #6
    Keeper of the SFL StarLion's Avatar
    Join Date
    Feb 2006
    Location
    Atlanta, GA, USA
    Posts
    3,748
    Mentioned
    71 Post(s)
    Tagged
    0 Thread(s)
    You can use CURL, or file_get_contents if your site allows remote URL references, and the site in question accepts all its parameters via GET.

    Incidentally, I read the rules that you claim to have read, and they state very specifically that you MAY NOT scrape their site without written permission of the company. I would suggest you go back and read again.
    Never grow up. The instant you do, you lose all ability to imagine great things, for fear of reality crashing in.


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •