Login with curl

PHP
1

Hi guys -
I’ve been working on a recreational app that will allow me to log into Pinnacle Sportsbook and download the English Premier League odds. I’d use the publicly available feed but it doesn’t contain all their lines, so I’m resorting to scraping.

Basically I’ve got a problem after I log in and try to access a page. To be honest I’m not sure exactly where this is falling over. I think maybe I’m losing my login credential somewhere along the line.

The output I’m getting to my browser is:

Object Moved
This object may be found here.

Here is my php file I’m using to connect (with username and password obscured)




function curl_login($url,$data,$proxy,$proxystatus){
    $fp = fopen("cookie.txt", "w");
    fclose($fp);
    $login = curl_init();
	curl_setopt($login, CURLOPT_POSTFIELDS, $data);
    curl_setopt($login, CURLOPT_COOKIEJAR, "cookie.txt");
    curl_setopt($login, CURLOPT_COOKIEFILE, "cookie.txt");
	curl_setopt($login, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6");
    curl_setopt($login, CURLOPT_TIMEOUT, 40);
    curl_setopt($login, CURLOPT_RETURNTRANSFER, TRUE);
    if ($proxystatus == 'on') {
        curl_setopt($login, CURLOPT_SSL_VERIFYHOST, FALSE);
        curl_setopt($login, CURLOPT_HTTPPROXYTUNNEL, TRUE);
        curl_setopt($login, CURLOPT_PROXY, $proxy);
    }
    curl_setopt($login, CURLOPT_URL, $url);
    curl_setopt($login, CURLOPT_HEADER, TRUE);
    curl_setopt($login, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
    curl_setopt($login, CURLOPT_FOLLOWLOCATION, TRUE);
    curl_setopt($login, CURLOPT_POST, TRUE);
   
    ob_start();      // prevent any output
    return curl_exec ($login); // execute the curl command
    ob_end_clean();  // stop preventing output
    curl_close ($login);
    unset($login);    
}               

function curl_grab_page($site,$proxy,$proxystatus){
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
    if ($proxystatus == 'on') {
        curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE);
        curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, TRUE);
        curl_setopt($ch, CURLOPT_PROXY, $proxy);
    }
    curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
    curl_setopt($ch, CURLOPT_URL, $site);
    ob_start();      // prevent any output
    return curl_exec ($ch); // execute the curl command
    ob_end_clean();  // stop preventing output
    curl_close ($ch);
}

$loginurl = 'https://www.pinnaclesports.com/Secure/LoginPage.aspx?destination=sports';
$data = '__EVENTTARGET=ctl00%24LF%24LB&__EVENTARGUMENT=&__LASTFOCUS=&__VIEWSTATE=%2FwEPDwULLTE1MTczNzI3NjUPZBYCZg9kFgQCAQ9kFgQCBA8WAh4EVGV4dAVUDQo8bWV0YSBuYW1lPSJ2ZXJpZnktdjEiIGNvbnRlbnQ9InJvT3JjOUljZHlZVFgrVDZnWm1PN2VQejMwRWNLR2VXTkJqbmwzWnlaQUk9IiAvPg0KZAIKD2QWAmYPFgIfAAWJATxzdHlsZSB0eXBlPSJ0ZXh0L2NzcyI%2BKi5pIHtiYWNrZ3JvdW5kLWltYWdlOnVybChodHRwczovL2NvbnRlbnQucGlubmFjbGVzcG9ydHMuY29tL3VwbG9hZGVkSW1hZ2VzL0d1ZXN0U2VjdGlvbi9jb21iaW5lZDIzLnBuZyk7fTwvc3R5bGU%2BZAIDDxYEHgZvbmxvYWQFMmphdmFzY3JpcHQ6UDdfaW5pdFBNKDEsOCwwLC0yMCwyKTtQN19pbml0VFAoNCwgMCk7HghvbnVubG9hZAUWamF2YXNjcmlwdDpvblVubG9hZCgpOxYEAgEPZBYEAgMPZBYCAgEPZBYGZg8QZGQWAWZkAgEPEGRkFgECAWQCAg8QDxYCHgdWaXNpYmxlaGRkFgECAmQCBw9kFgQCBQ8WAh8DaGQCBw8WAh8DaGQCAw9kFgJmDxYEHgRNb2RlCyolU3lzdGVtLldlYi5VSS5XZWJDb250cm9scy5MaXRlcmFsTW9kZQEfAAWXFTwhLS0gU3RhcnQgb2YgSW50ZWxsaVRyYWNrZXIgUGFnZSBUYWcgLS0%2BDQo8c2NyaXB0IHR5cGU9InRleHQvamF2YXNjcmlwdCI%2BPCEtLSANCnZhciBwcXJ5PSJMYW5ndWFnZUlEJTNEMCUyNm9kZHNGb3JtYXQlM0RkZWNpbWFsIjsNCnZhciBycXJ5PSJpUkVHUXJ5IjsNCnZhciBzcXJ5PSJpU2FsZSI7DQp2YXIgaXRyTUlkID0gODI1Ow0KdmFyIGl0clJxc3RIID0gInRyYWNrZXIucGlubmFjbGVzcG9ydHMuY29tIjsNCnZhciBkdD13aW5kb3cuZG9jdW1lbnQsbnI9bmF2aWdhdG9yLGluYT1uci5hcHBOYW1lLHNyPSIwJjAiLHB4PTAsc3Y9MTMsamU9MDsNCnZhcglpbmF2PW5yLmFwcFZlcnNpb24saWllPWluYXYuaW5kZXhPZignTVNJRSAnKSxpbnRwPShpbmEuaW5kZXhPZignTmV0c2NhcGUnKT49MCk7DQppZihpaWU%2BMClpbmF2aT1wYXJzZUludChpbmF2LnN1YnN0cmluZyhpaWUrNSkpO2Vsc2UgaW5hdmk9cGFyc2VGbG9hdChpbmF2KTsNCmZ1bmN0aW9uIGlycyhzLGYscil7dmFyIHA9cy5pbmRleE9mKGYpO3doaWxlKHA%2BPTApe3M9cy5zdWJzdHJpbmcoMCxwKStyK3Muc3Vic3RyaW5nKHArZi5sZW5ndGgscy5sZW5ndGgpO3A9cy5pbmRleE9mKGYpfXJldHVybiBzfQ0KZnVuY3Rpb24gY2VzYyhzKXtpZihzLmxlbmd0aD4wKSByZXR1cm4gaXJzKGlycyhpcnMoaXJzKGlycyhzLCcrJywnJTJCJyksJy4nLCclMkUnKSwnLycsJyUyRicpLCc9JywnJTNEJyksJyYnLCclMjYnKSA7IGVsc2UgcmV0dXJuIHM7fQ0KZnVuY3Rpb24gaWVzYyhzKXtyZXR1cm4gY2VzYyhlc2NhcGUocykpO30gDQpmdW5jdGlvbiBncHIoKXsNCnZhciBwcj0nJywgaXB3PXdpbmRvdywgaXByPSd3aW5kb3cnLCBpd0w9JycsIGlwTD0nJzsNCndoaWxlIChpcEw9PWl3TCl7DQppdz1pcHc7IHByPWl3LmRvY3VtZW50LnJlZmVycmVyOw0KaWYoaW50cCkgYnJlYWs7aWYoKCcnK2l3LnBhcmVudC5sb2NhdGlvbik9PScnKWJyZWFrOw0KaXdMPShpdy5kb2N1bWVudC5sb2NhdGlvbi5wcm90b2NvbCsnXC9cLycraXcuZG9jdW1lbnQubG9jYXRpb24uaG9zdG5hbWUpLnRvTG93ZXJDYXNlKCk7DQppcEw9cHIuc3Vic3RyaW5nKDAsaXdMLmxlbmd0aCkudG9Mb3dlckNhc2UoKTsNCmlwcj1pcHIrJy5wYXJlbnQnOyBpcHc9ZXZhbChpcHIpOyBpZiAoaXc9PWlwdykgYnJlYWs7fXJldHVybiBwcjt9DQpmdW5jdGlvbiBpdHJjKCl7dmFyIG53PW5ldyBEYXRlKCksY2U9MixpdWw9Jyc7DQppZiAoZHQuY29va2llKSBjZT0xOw0KZWxzZSB7dmFyIGV4PW5ldyBEYXRlKG53LmdldFRpbWUoKSsxMDAwKTsgZHQuY29va2llPSJpdGM9MzsgRVhQSVJFUz0iK2V4LnRvR01UU3RyaW5nKCkrIjsgcGF0aD0vIjtpZiAoZHQuY29va2llKSBjZT0xO30JCQ0KaWYoaW5hdmk%2BPTQpIGl1bD1pZXNjKGlpZT4wJiZuci51c2VyTGFuZ3VhZ2U%2FbnIudXNlckxhbmd1YWdlOm5yLmxhbmd1YWdlKTsNCnZhciB1bj1NYXRoLnJvdW5kKE1hdGgucmFuZG9tKCkqMjEwMDAwMDAwMCk7DQppbD1pc2wrdW4rIiYiK2llc2MoZ3ByKCkpKyIlMjAmIitjZXNjKHBxcnkpKyIlMjAmIitjZXNjKHJxcnkpKyIlMjAmIg0KK2Nlc2Moc3FyeSkrIiUyMCYiK2NlKyImIitzcisiJiIrcHgrIiYiK2plKyImIitzdisiJiIraXVsKyIlMjAmIitudy5nZXRUaW1lem9uZU9mZnNldCgpKyImIitpZXNjKGlkbCkrIiUyMCI7DQppZihpaWU%2BMCAmJiBpbC5sZW5ndGg%2BMjA0NSlpbD1pbC5zdWJzdHJpbmcoMCwyMDQ1KTsNCnZhciBpaW49J2l0cjgyNScsIGl3cmk9dHJ1ZTsNCmlmKGR0LmltYWdlcyl7aWYoIWR0LmltYWdlc1tpaW5dKWR0LndyaXRlKCc8ZGl2IHN0eWxlPSJkaXNwbGF5Om5vbmUiPjxpJysnbWcgbmFtZT0iJytpaW4rJyIgaGVpZ2h0PSIxIiB3aWR0aD0iMSIgYWx0PSJJbnRlbGxpVHJhY2tlciIvPjwvZGl2PicpOw0KaWYoZHQuaW1hZ2VzW2lpbl0pe2R0LmltYWdlc1tpaW5dLnNyYz1pbCsnJjAnO2l3cmk9ZmFsc2U7fX0NCmlmKGl3cmkpZHQud3JpdGUoJzxpJysnbWcgc3InKydjPSInK2lsKycmMCIgaGVpZ2h0PSIxIiB3aWR0aD0iMSI%2BJyk7fQ0KdmFyIGlkbD13aW5kb3cubG9jYXRpb24uaHJlZjt2YXIgaXNsPSJodHRwIisoaWRsLmluZGV4T2YoJ2h0dHBzOicpPT0wPydzJzonJykrIjovL3RyYWNrZXIucGlubmFjbGVzcG9ydHMuY29tL2UvdDMuZGxsPzgyNSYiOw0KaXRyYygpOw0KLy8tLT48L3NjcmlwdD4NCjxzY3JpcHQgdHlwZT0idGV4dC9qYXZhc2NyaXB0Ij48IS0tDQppZihpaWU%2BMClkdC53cml0ZSgiXDxcIVwtXC0iKTsNCi8vLS0%2BPC9zY3JpcHQ%2BDQo8c2NyaXB0IHR5cGU9InRleHQvamF2YXNjcmlwdCIgc3JjPSJodHRwczovL3RyYWNrZXIucGlubmFjbGVzcG9ydHMuY29tL2UvY2xpY2tzLmpzIj48L3NjcmlwdD4NCjxub3NjcmlwdD4NCjxkaXYgc3R5bGU9ImRpc3BsYXk6bm9uZSI%2BPGltZyBzcmM9J2h0dHBzOi8vdHJhY2tlci5waW5uYWNsZXNwb3J0cy5jb20vZS90My5kbGw%2FODI1JmFtcDswJmFtcDslMjAmYW1wO0xhbmd1YWdlSUQlM0QwJTI2b2Rkc0Zvcm1hdCUzRGRlY2ltYWwmYW1wO2lSRUdRcnkmYW1wO2lTYWxlJmFtcDswJmFtcDswJmFtcDswJmFtcDswJmFtcDswJmFtcDswJmFtcDslMjAmYW1wOzE1MDAmYW1wOyUyMCZhbXA7MCcgaGVpZ2h0PSIxIiB3aWR0aD0iMSIgYWx0PSJJbnRlbGxpVHJhY2tlciIvPjwvZGl2Pg0KPC9ub3NjcmlwdD48IS0tLy8tLT4NCjwhLS0gRW5kIG9mIEludGVsbGlUcmFja2VyIFBhZ2UgVGFnIC0tPg0KZGQ%3D&__PREVIOUSPAGE=cC6v86rLgpzKauPW5R75VZ6MnouJHAk-4dX5bAgGIAb5u7LxzDThRsxab3Nvnz3jfDBRuieaF3sFEN_3HIHC-n835N01&ctl00%24LDDL=1&ctl00%24PSDDL=decimal&ctl00%24MCPH%24LF%24UserName=***&ctl00%24MCPH%24LF%24Password=***&ctl00%24MCPH%24LF%24LanguageID=0&ctl00%24MCPH%24LF%24PriceStyle=decimal&ctl00%24MCPH%24LF%24LinesTypeView=c&ctl00%24MCPH%24LF%24MemberServer=www39.pinnaclesports.com';

echo curl_login($loginurl,$data,'','off');
echo curl_grab_page('http://www39.pinnaclesports.com/Members/gameselection.asp?sportType=Soccer&sportSubType=Eng.+Premier&descr=1','','off');

Any help with this kind of this thing would be greatly appreciated.

Thanks.

2

Hi -

Don’t want to be rude. Just checked in and I see 50+ views but no replies.
Just wondering is my question too specific? Or too vague? Or is it just not a regular problem that people would encounter?

Cheers again.

3

Im sorry to hear that there are no replies. I have endured that with some threads and I know it can be quite frustrating. I apologise, but I do not have a solution.

However, I am interested in learning how to fix this :slight_smile:

Hopefully a reply may generate some traffic?

Regards,
Team 1504

4

Did you try to test your code on your localhost before trying to submit to an external site?
Also, if you pay attention to your curl_login - you have RETURN statement before ob_end_clean(); call - return will exit the function and anything below that won’t execute.

Also, are you sure you constructed your postdata string properly?
As I said, try to test your code on your localhost (post some data to test.php, then use print_r($_POST) to check whether it comes in expected format) and when you are sure you’re sending / receiving properly - move on to other websites.

5

Thanks for the reply.
To be honest, I followed a tutorial to get me this far.
The post data that I am sending is what is being sent when I submit the login form on the website - I got the information from LiveHeaders (firefox plugin that captures header information… http://www.youtube.com/watch?v=XcgQUsorF_8 for more info)

I’ll try your suggestion of posting to a test page. I’ll be back!

Thanks again.

6

Only in this instance showing us your curl will not help. curl is more or less straight forward. To help us help you we need to see some sort of API functions you are trying to invoke. As such can you show us what API functions you are invoking? if they dont have an API than you want read more about file_get_contents (http://php.net/manual/en/function.file-get-contents.php)

HTH

7

Ok - let me bring you up to speed with what I can do so far prior to my original request for some help.

Right now I can go to the website and login manually like a regular user and go to a list of prices the English Premier League.
I can save the list of odds (that is, the resulting html page in my browser) to an html file on my computer. I can then parse the page with a script I have already written that breaks up the file into the bits I need and puts them in a database. No trouble. I can do all that.

Now, what I cannot do is login via my script rather than having to go to Pinnacle and do it manually, and I cannot get the script to retrieve the odds automatically.

I’m a relative newbie - I was under the impression that I needed curl to do all that.

So all I wanted the script to do was get me as far as displaying the html page for English Premier League odds with me logged into my account. I guess I could use the help saving this file to my computer automatically, but that’s another thing. I can do the rest from there with the script I have.

So to go back to your kind reply, there is no api. Does file_get_contents still sound like the kind of thing I need?

Cheers pal - I’ll go and read about it now. Look forward to hearing from you again.

8

This is the deal - you do need curl, it’s way easier to do it with curl than without. file_get_contents gets contents of a page trough fopen wrapper, problem is that you aren’t recognized as logged on at that page since you haven’t sent any headers containing cookie information that mark you as logged on.

Now, where’s the problem with your initial script - you are not passing data in correct format, the data needed for the site to log you on.
Reason I told you to have a play on your local computer with sending the data was so you ensure you constructed the string containing your data right.
Another problem you had in your original script is that you made the function return something before you actually did any output - your curl_login NEVER sends any data because you end the function prematurely.

Now, you can either try to learn this the hard way so it stays with you forever or you can get the solution right away from me or someone else. If you need the solution instantly and you don’t want to think about this ever again - I can PM you the code so copypaste it and that’s it.

9

Furicane -

Thanks for the input once again.
It is probably a case of only needing this once and never having to look at it again, to be honest!

That said, I’d prefer to understand it so I’ll continue to try out the things you have suggested. Would you be able to PM the code, please? Maybe you would be kind enough to answer a few questions on that if they come up?

Thanks again.