Please help me with CURL

PHP
1

Hi,

I am trying to automated a boring, regular job that I have to do daily. I am trying to automated this job 100% at server end in php by using CURL:

  1. Login to a site https://www.tutorsteach.com/login.aspx automatically.
  2. then go to this page: https://www.tutorsteach.com/inquiry.aspx.
  3. Get the html source code of this page in variable and then scan it for new email ids.
  4. Then send them a mail.

This is my code:

<?php
// INIT CURL
$ch = curl_init();

// SET URL FOR THE POST FORM LOGIN
curl_setopt($ch, CURLOPT_URL, 'https://www.tutorsteach.com/Default.aspx');

// ENABLE HTTP POST
curl_setopt ($ch, CURLOPT_POST, 1);

// SET POST PARAMETERS : FORM VALUES FOR EACH FIELD
$EVENTTARGET = "left1$lnkLogin";
$EVENTARGUMENT = "";
$VIEWSTATE = "/wEPDwUKMTcwOTg5NzA3OA9kFgICAQ9kFgQCBQ9kFgICAQ9kFgICAQ8WAh4EVGV4dAWkKTxwPjxzdHJvbmc+V2VsY29tZSB0byBUdXRvcnNUZWFjaC5jb20gLS0gPGZvbnQgY29sb3I9ImJyb3duIj5UaGUgbGFyZ2VzdCB0dXRvciBwcm92aWRlciBpbiB0aGUgcmVnaW9uITwvZm9udD48L3N0cm9uZz48YnIvPjxpPlNpbmNlIDE5OTk8L2k+PC9wPg0KPHA+PGEgaHJlZj0iRmxhc2gvdG91ci5odG0iIG9uY2xpY2s9IndpbmRvdy5vcGVuKCdGbGFzaC90b3VyLmh0bScsJ1JJQycsJ3dpZHRoPTYwMCxoZWlnaHQ9NDM4LHJlc2l6YWJsZT1ubyxzY3JvbGxiYXJzPW5vLHRvb2xiYXI9bm8sbWVudWJhcj1ubyxsb2NhdGlvbj1ubyxzdGF0dXM9bm8sbGVmdD0xMDAsdG9wPTQwLHNjcmVlblg9MTAwLHNjcmVlblk9NDAnKTtyZXR1cm4gZmFsc2U7Ij48aW1nIHNyYz0iaW1hZ2VzL1RvdXIuanBnIiBhbHQ9IlRha2Ugb3VyIFNUVURFTlQgVE9VUiIgYWxpZ249InJpZ2h0IiBib3JkZXI9IjAiLz48L2E+PC9wPg0KPGRpdiBhbGlnbj0ibGVmdCI+PHN0cm9uZz5UdXRvcnNUZWFjaC5jb208L3N0cm9uZz48YnIvPg0KICAgICZuYnNwOy4uaXMgcHJvdWQgdG8gc3VwcG9ydCB0aGUgPGEgaHJlZj0iQ2hpbGRMZWZ0Lmh0bSIgb25jbGljaz0id2luZG93Lm9wZW4oJ0NoaWxkTGVmdC5odG0nLCdSSUMnLCd3aWR0aD00NTAsaGVpZ2h0PTI2MCxyZXNpemFibGU9bm8sc2Nyb2xsYmFycz15ZXMsdG9vbGJhcj1ubyxtZW51YmFyPW5vLGxvY2F0aW9uPW5vLHN0YXR1cz1ubyxsZWZ0PTE3MCx0b3A9NSxzY3JlZW5YPTE3MCxzY3JlZW5ZPTUnKTtyZXR1cm4gZmFsc2U7Ij5ObyBDaGlsZCBMZWZ0IEJlaGluZDwvYT4gcHJvZ3JhbSENCjwvZGl2Pg0KPHA+PGEgaHJlZj0iaGVsbG8uYXNweCI+PGltZyBzcmM9ImltYWdlcy9PbmxpbmUtVG9vbC5naWYiIGFsdD0iT25saW5lIFR1dG9yaW5nIGFuZCBXaGl0ZWJvYXJkIFRvb2wiIHdpZHRoPSIxMDUiIGFsaWduPSJyaWdodCIgYm9yZGVyPSIwIiBoZWlnaHQ9IjMwIi8+PC9hPjwvcD4NCjxwIGFsaWduPSJjZW50ZXIiPiAuLmFuZCA8c3Ryb25nPjxmb250IGNvbG9yPSJncmVlbiI+Y2VsZWJyYXRlczwvZm9udD4gPGEgaHJlZj0iTW9tRGF5Lmh0bSIgb25DbGljaz0id2luZG93Lm9wZW4oJ01vbURheS5odG0nLCdSSUMnLCd3aWR0aD0zNzUsaGVpZ2h0PTI2MSxyZXNpemFibGU9bm8sc2Nyb2xsYmFycz15ZXMsdG9vbGJhcj1ubyxtZW51YmFyPW5vLGxvY2F0aW9uPW5vLHN0YXR1cz1ubyxsZWZ0PTE1MCx0b3A9MzUsc2NyZWVuWD0xNTAsc2NyZWVuWT0zNScpO3JldHVybiBmYWxzZTsiPjxmb250IGNvbG9yPSJkZWVwcGluayI+PEk+TW90aGVyJ3MgRGF5PC9JPjwvYT48L2ZvbnQ+IDxmb250IGNvbG9yPSJkYXJrZ3JlZW4iPnRoaXMgbW9udGguPC9mb250Pjxici8+PGJyLz48Zm9udCBjb2xvcj0iYnJvd24iPkZSRUUgU2VhcmNoIEZvciBJTi1IT01FIGFuZCBPTkxJTkUgVHV0b3JzPGk+IEFsd2F5czwvaT4gLSBObyBuZWVkIHRvIHNpZ24tdXAhPC9mb250Pjwvc3Ryb25nPjwvcD4NCjx1bD4NCgk8bGk+PGEgaWQ9Imxua19rX2p1bmlvciIgaHJlZj0ibG9jYXRlVHV0b3IuYXNweD9EaXNjaXBsaW5lPTEiPjwvYT48L2xpPg0KCTxsaT48YSBpZD0ibG5rX2hpZ2hzY2hvb2wiIGhyZWY9ImxvY2F0ZVR1dG9yLmFzcHg/RGlzY2lwbGluZT0yIj48L2E+PC9saT4JCQkgICAgDQoJPGxpPjxhIGlkPSJsbmtfY29sbGVnZSIgaHJlZj0ibG9jYXRlVHV0b3IuYXNweD9EaXNjaXBsaW5lPTMiPjwvYT48L2xpPg0KCTxsaT48YSBpZD0ibG5rX3Rlc3RfcHJlcCIgaHJlZj0ibG9jYXRlVHV0b3IuYXNweD9EaXNjaXBsaW5lPTQiPjwvYT48L2xpPg0KCTxsaT48YSBpZD0ibG5rX2ZvcmVpZ25fbGFuZ3VhZ2UiIGhyZWY9ImxvY2F0ZVR1dG9yLmFzcHg/RGlzY2lwbGluZT05Ij48L2E+PC9saT4NCgk8bGk+PGEgaWQ9Imxua19jb21wdXRlciIgaHJlZj0ibG9jYXRlVHV0b3IuYXNweD9EaXNjaXBsaW5lPTgiPjwvYT48L2xpPg0KCTxsaT48YSBpZD0ibG5rX2J1c2luZXNzIiBocmVmPSJsb2NhdGVUdXRvci5hc3B4P0Rpc2NpcGxpbmU9MTAiPjwvYT48L2xpPg0KCTxsaT48YSBpZD0ibG5rX2FydCIgaHJlZj0ibG9jYXRlVHV0b3IuYXNweD9EaXNjaXBsaW5lPTUiPjwvYT48L2xpPg0KCTxsaT48YSBpZD0ibG5rX211c2ljIiBocmVmPSJsb2NhdGVUdXRvci5hc3B4P0Rpc2NpcGxpbmU9NiI+PC9hPjwvbGk+DQoJPGxpPjxhIGlkPSJsbmtfc3BvcnRzIiBocmVmPSJsb2NhdGVUdXRvci5hc3B4P0Rpc2NpcGxpbmU9NyI+PC9hPjwvbGk+DQo8L3VsPg0KPHA+R29vZCB3cml0ZXJzIGtub3cgdGhlcmUgaXMgYSBjb25zaWRlcmFibGUgZGlmZmVyZW5jZSBiZXR3ZWVuIHRoZSBhYmlsaXR5IHRvIHB1dCB3b3JkcyBvbiBhIHBhZ2UgYW5kIHRoZSBhYmlsaXR5IHRvIGV4cHJlc3MgeW91cnNlbGYgaW4gbHVjaWQsIGZsb3dpbmcgcGFyYWdyYXBocy4gIENvdW50bGVzcyBzdHVkZW50cyB0aGF0IGNhbiBkbyB0aGUgZm9ybWVyIHN0cnVnZ2xlIHdpdGggdGhlIGxhdHRlciwgYW5kIG9mdGVuIHByYWN0aWNlIGFsb25lIGNhbm5vdCBtYWtlIHRoZSBwcm9ibGVtIGdvIGF3YXkuICBRdWFsaWZpZWQgdHV0b3JzIHdobyBhcmUgZXhwZXJpZW5jZWQgaW4gYWRkcmVzc2luZyBzdWNoIHByb2JsZW1zIGNhbiBtYWtlIGEgY29uc2lkZXJhYmxlIGRpZmZlcmVuY2UuPC9wPjxkaXYgc3R5bGU9ImZsb2F0OnJpZ2h0O2ZvbnQtc2l6ZToxNHB4O2ZvbnQtd2VpZ2h0OiBib2xkO2xpbmUtaGVpZ2h0OjIwcHg7bWFyZ2luOiAwcHggMHB4IDIwcHggMjBweDtwYWRkaW5nOiAxMHB4IDEwcHggMTBweCAxMHB4O3RleHQtYWxpZ246IGNlbnRlcjt3aWR0aDoyMDBweDsiPjxhIHJlbD0ibm9mb2xsb3ciIG9ubW91c2VvdmVyPSJ3aW5kb3cuc3RhdHVzPScnO3JldHVybiB0cnVlOyIgb25tb3VzZW91dD0id2luZG93LnN0YXR1cz0nJztyZXR1cm4gdHJ1ZTsiIGhyZWY9ImxvY2F0ZVR1dG9yLmFzcHgiPkNsaWNrIEhlcmUgZm9yICBXcml0aW5nIFR1dG9ycyE8L2E+PC9kaXY+PHA+V3JpdGluZyBpcyBsaXR0bGUgbW9yZSB0aGFuIHRoZSBwcm9wZXIgb3JnYW5pemF0aW9uIG9mIHlvdXIgdGhvdWdodHMuICBNYW55IG9mIHRoZSBiZXN0IHR1dG9ycyBkZXNpZ24gZXhlcmNpc2VzIHdob3NlIHB1cnBvc2UgaXMgdG8gaGVscCB3aXRoIGdldHRpbmcgY29uY2VwdHMgaW4gdGhlIHByb3BlciBvcmRlciBhbmQgaW1wcm92aW5nIHRoZWlyIGV4ZWN1dGlvbi4gIFN0dWRlbnRzIHdobyBzdHJ1Z2dsZSB3aXRoIHN1Y2ggc2tpbGxzIG9mdGVuIGRvIHNvIGZvciBhIHZhcmlldHkgb2YgcmVhc29ucywgYW5kIGFuIGV4cGVydCBkaWFnbm9zaXMgY2FuIHNhdmUgeW91IGhvdXJzIG9mIGZydXN0cmF0aW9uLjwvcD48cD5UdXRvcnNUZWFjaC5jb20gd2FzIGRldmVsb3BlZCBhcyBhIGZyZWUgcmVzb3VyY2UgdG8gaGVscCBzdHVkZW50cyBjb25uZWN0IHdpdGggc3VwZXJpb3IgdHV0b3JzIGluIHRoZWlyIGFyZWFzLiAgTWFueSBzdWNoIHR1dG9ycyB3b3JrIHRocm91Z2ggdGVjaG5vbG9neSB0byBjcmVhdGUgb25saW5lIGxlc3NvbiBwbGFucywgZW1haWxzIGFuZCByZWFsLXRpbWUgY2hhdCB0dXRvcmlhbHMuICBXaGVuIHlvdSBmb3JlZ28gdGhvc2UgZXhwZW5zaXZlIGhvdXNlIHZpc2l0cywgeW91IGNhbiBsaXRlcmFsbHkgc2F2ZSBodW5kcmVkcyBvZiBkb2xsYXJzIG9uIHR1dG9yaW5nIGNvc3RzIG92ZXIgdGhlIGNvdXJzZSBvZiBhIHNjaG9vbCB5ZWFyITwvcD48cD5XaGF0ZXZlciB5b3VyIHdyaXRpbmcgbGV2ZWwsIHdlIGNhbiBoZWxwIGZpbmQgc29tZW9uZSB3aG9zZSBleHBlcnRpc2Ugc3VpdHMgeW91ciBuZWVkcy4gIFRoZSBzZWFyY2ggaXMgZnJlZSBmb3Igc3R1ZGVudHMgYW5kIG91ciB0dXRvcnMgcGF5IG9ubHkgYSBub21pbmFsIGZlZSB0byBnZXQgbGlzdGVkIGhlcmUsIHNvIHlvdSB3aWxsIG5ldmVyIGVuY291bnRlciBhbnkgdW5leHBlY3RlZCBmZWVzLiAgSWYgeW91IGFyZSB0aXJlZCBvZiB5b3VyIHBpZWNlcyBjb21pbmcgYmFjayBtYXJrZWQgdXAgd2l0aCByZWQgaW5rLCBhIHF1YWxpZmllZCB0dXRvciBtYXkgYmUgYSBncmVhdCBwbGFjZSB0byBzdGFydC48YnIgLz48L3A+DQo8cD48YSBocmVmPSJodHRwOi8vd3d3LmJiYm9ubGluZS5vcmcvY2tzLmFzcD9pZD0xMDUxMjE0MTExOTMzIiBvbmNsaWNrPSJ3aW5kb3cub3BlbignaHR0cDovL3d3dy5iYmJvbmxpbmUub3JnL2Nrcy5hc3A/aWQ9MTA1MTIxNDExMTkzMycsJ1JJQycsJ3dpZHRoPTYwMCxoZWlnaHQ9NDM4LHJlc2l6YWJsZT15ZXMsc2Nyb2xsYmFycz15ZXMsdG9vbGJhcj1ubyxtZW51YmFyPW5vLGxvY2F0aW9uPW5vLHN0YXR1cz1ubyxsZWZ0PTEwMCx0b3A9NDAsc2NyZWVuWD0xMDAsc2NyZWVuWT00MCcpO3JldHVybiBmYWxzZTsiPjxpbWcgc3JjPSJpbWFnZXMvQkJCU2VhbDMuZ2lmIiBhbHQ9IlRoZSBCZXR0ZXIgQnVzaW5lc3MgQnVyZWF1IFNlYWwiIHZzcGFjZT0iMTkiIGFsaWduPSJyaWdodCIgYm9yZGVyPSIwIi8+PC9hPjwvcD4NCjxwIGFsaWduPSJsZWZ0Ij48c3Ryb25nPlBhcmVudHMgYW5kIFN0dWRlbnRzOjwvc3Ryb25nPjwvcD4NCjxwIGFsaWduPSJsZWZ0Ij48YSBocmVmPSJsb2NhdGVUdXRvci5hc3B4Ij5DbGljayBoZXJlIHRvIEZpbmQgYSBUdXRvciBpbiB5b3VyIGFyZWEhPC9hPjxici8+RXZlcnkgcmVnaW9uLiBFdmVyeSBzdWJqZWN0LiBFdmVyeSBhY2FkZW1pYyBsZXZlbC4gQW5kIHRoZSBzZWFyY2ggZm9yIFR1dG9ycyBpcyBhbHdheXMgRlJFRSEgJm5ic3A7IDxhIGhyZWY9IkZsYXNoL3RvdXIuaHRtIiBvbmNsaWNrPSJ3aW5kb3cub3BlbignRmxhc2gvdG91ci5odG0nLCdSSUMnLCd3aWR0aD02MDAsaGVpZ2h0PTQzOCxyZXNpemFibGU9bm8sc2Nyb2xsYmFycz1ubyx0b29sYmFyPW5vLG1lbnViYXI9bm8sbG9jYXRpb249bm8sc3RhdHVzPW5vLGxlZnQ9MTAwLHRvcD00MCxzY3JlZW5YPTEwMCxzY3JlZW5ZPTQwJyk7cmV0dXJuIGZhbHNlOyI+Q2xpY2sgaGVyZSB0byBzZWUgYSBEZW1vPC9hPjxici8+PC9wPg0KPHAgYWxpZ249ImxlZnQiPjxzdHJvbmc+VHV0b3JzIGFuZCBFZHVjYXRvcnM6PC9zdHJvbmc+PC9wPg0KPHAgYWxpZ249ImxlZnQiPjxhIGhyZWY9Imh0dHBzOi8vd3d3LnR1dG9yc3RlYWNoLmNvbS9zaWduVXAuYXNweCI+Q2xpY2sgaGVyZSB0byBSZWdpc3RlciBhcyBhIFR1dG9yIT48L2E+PGJyLz5ObyBsb25nZXIgd2lsbCB5b3UsIHRoZSB0dXRvciwgbmVlZCB0byBmaW5kIG5ldyBzdHVkZW50cy4gTmV3IHN0dWRlbnRzIHdpbGwgZmluZCB5b3UuPGJyLz48YSBocmVmPSJmYXEuYXNweCI+PHNtYWxsPkxFQVJOIE1PUkUuLi48L3NtYWxsPjwvYT48L3A+DQpkAgcPZBYCAgEPDxYCHwAFBDIwMTBkZGR/VpP2kfOXZiQ6bnwjYk15PkTukw==";
$EVENTVALIDATION = "/wEWBALly9q5CQKMyrugBwKOhu66DgLz7ZDjD5DRTjhka66u5ECR4filq7BkSxxy";
curl_setopt ($ch, CURLOPT_POSTFIELDS, "left1_tbUser=myusername&left1_tbPassword=mypassword&__EVENTTARGET=".$EVENTTARGET."&__EVENTARGUMENT=".$EVENTARGUMENT."&__VIEWSTATE=".$VIEWSTATE."&__EVENTVALIDATION=".$EVENTVALIDATION);

// IMITATE CLASSIC BROWSER'S BEHAVIOUR : HANDLE COOKIES
curl_setopt ($ch, CURLOPT_COOKIEJAR, 'cookie.txt');

# Setting CURLOPT_RETURNTRANSFER variable to 1 will force cURL
# not to print out the results of its query.
# Instead, it will return the results as a string return value
# from curl_exec() instead of the usual true/false.
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);

// EXECUTE 1st REQUEST (FORM LOGIN)
$store = curl_exec ($ch);

//print $store;

// SET FILE TO DOWNLOAD
curl_setopt($ch, CURLOPT_URL, 'https://www.tutorsteach.com/inquiry.aspx');

// EXECUTE 2nd REQUEST (FILE DOWNLOAD)
$content = curl_exec ($ch);

// CLOSE CURL
curl_close ($ch);

print $content;

?>

Now this should display the page that comes after we are logged into the site and visit this page: https://www.tutorsteach.com/inquiry.aspx

But it is showing this error:
“Object Moved
This object may be found here.”

Can someone please help me? I am a novice in CURL.

thanks in advance.

FireFly

2

What it means is that the server is trying to redirect you, but your script is not set up to allow that.

Fortunately, CURL has such capabilities, which can be used by including the following line:

curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
3

thanks a ton Jake. I tried that line and now I am not getting that error and it is taking me to the redirect page.

But it is still showing me the page that comes without login which means that I am not logged in.

Can you please help?

This is the response of the server when used print_r(curl_getinfo($ch)); to see header:

Array ( => https://www.tutorsteach.com/inquiry.aspx [content_type] => text/html; charset=utf-8 [http_code] => 200 [header_size] => 281 [request_size] => 122 [filetime] => -1 [ssl_verify_result] => 0 [redirect_count] => 0 [total_time] => 0.348745 [namelookup_time] => 3.0E-5 [connect_time] => 3.1E-5 [pretransfer_time] => 3.3E-5 [size_upload] => 0 [size_download] => 77633 [speed_download] => 222606 [speed_upload] => 0 [download_content_length] => 77633 [upload_content_length] => 0 [starttransfer_time] => 0.276132 [redirect_time] => 0 )

here I can see that http_code is 200 which means that it is right. Then where am I wrong.

Please help.

4

I had used curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);

5

Your second curl command needs to send the cookie back.

curl_setopt($ch, CURLOPT_COOKIEFILE, ‘cookie.txt’);

6

Thanks StarLion. Can you please explain a bit more? Should I add this line somewhere?

7

You also need to set the COOKIEJAR:


curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie.txt');

See http://php.net/manual/en/function.curl-setopt.php, look at the explaination of the $value parameter.