Php to query multiple external searches and return results.

[padders]

I have got a links directory and one of the things we index is movie websites. Bascially, what i want is a category of movie reviews, about 15 or so for each film i list.

What i want to be able to do then is to find some movie review sites (no problem) and find their search script.

Then i want to put them all in a simple database, no problem and then when i come to add a new movie, i type in its title, it runs through each of the above searches and brings back the results. I want it somehow formatted so that i see the url of the page, the title of the page and also the first couple of paragraphs. Hopefully then i can decide if it is the right url without checking it but if i can't at least i can quickly click on it.

I can do most of this, the php and mysql stuff really but what i don't know how to do is tell a php script to run a script on another server (ie their php/cgi/asp search script) and then return the results to the current script (ie the one on my server), i also don't know the best way to deal with the results but i guess i can work that out, i would assume it will be a matter of a find type thing so look for "your search returned" and then get everything out until you hit "click here to search again" type thing.

Suggestions would trully be greatly appreciated.

[Nicky]

Aside from the technical issues of whether what you are wanting to do si technically possible or not, my first question would be - Is it ethical?

[aspen]

To make a script run on another server is easy... for it to return results to you isn't.

You can't just access someone elses script and pull data from it like that. A script will treat your script's request just like it would a web browser. So your script will get the contents of a webpage.

So what you then need to do is parse this webpage for the information you want. Which means you write a script that can go through and pick out the information you need from the rest of the page.

[Son Nguyen]

When having to request then parse each result page, it's extremely slow and so inefficient.

Moreover, as aspen said, the return from a script from another server (external in general) is only the result page (which browsers get)

[padders]

Nicky, what is any way is unethical about it? I am trying to link to people's websites reviews about certain films. I just want an easier way to get the correct url of the relevant review than tralling through 15 of the websites, this is the only way i came up with.

Ok, i understand it takes a while for it to do it, but it will be quicker than me. I have about 500 movies to find reviews of so the plan was to put them all in the database, press go and leave it for 24 hours going (a cron job probably) and come back and check through them.

So i get the results as a webpage, fine and now i just need to parse them. I can look that bit up i am sure.

So how do i get the webpage into a variable. I assume that is what i do? I have been trying to find this and the best bit i came up with is the chapter PHP connectivity in Professional PHP programming.

It is something like

$fd = fopen("http://www.moviereview.com/search.php?term=alien","r");

if ($fd) {

   fpassthru{$fd);
}

?>

is that it? does $fd contain the code of the webpage or am i completly not understanding it? will it return the html code which i can then parse through (i can work that bit out ok). or am i missing it?

There is a longer script given on page 438-440 but i don't really understand it,i think it is to do with proxy servers. Anyway, does it give the same result at the end of page 440, does $fp contain the html code of the page, is that what fpassthru does?

Thanks for help on this.