SitePoint Sponsor

User Tag List

Results 1 to 5 of 5
  1. #1
    SitePoint Wizard
    Join Date
    Apr 2002
    Posts
    2,292
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)

    curl mangling url?

    Is there any reason why curl wouldn't be able to handle this URL? (I've removed the domain name -- not sure why, just have):
    Code:
    https://---mylocallibrary---/cgi-bin/spydus.exe/ENQ/OPAC/LOANRENQ/5237898?QRY=OND01%5C4728564%20%2B%20ONC01%5C0%20-%20(ONS01%5CCR%20/%20ONS01%5CLR%20/%20ONS01%5CL)&SQL=LEFT%20OUTER%20JOIN%20XAT%20ON%20XAT.IRN%20%3D%20%23TBLID.IRN%20WHERE%20%23TBLID.IRN%20NOT%20IN%20(SELECT%20XAT2.IRN%20FROM%20XAT%20AS%20XAT2%20WHERE%20XAT2.IRN%20%3D%20XAT.IRN%20AND%20XAT2.NAM%20%3D%20N'OND.EBK'%20AND%20XAT2.VAL%20%3D%20N'1')&QRYTEXT=Current%20Loans&FMT=CL&SETLVL=SET&NRECS=30&SEARCH_FORM=/cgi-bin/spydus.exe/FULL/OPAC/BRWENQ/5237898/4728564,1
    In my browser it works. Using curl, in a way that works fine for other URLS, it goes semi wrong. It's impossible to say exactly how it goes wrong because my local library server (the site I'm accessing) does stuff with it behind the scenes, or doesn't in the curl case. Basically, it does the right thing when I access that URL in my browser but not the right thing via curl.I know there's not a lot to go on there but maybe curl messes with URLs in some way?(separately, this form, for submitting posts, doesn't recognise new line/returns -- they get lost :/ Safari 6.0.5 on Mac)

  2. #2
    SitePoint Wizard
    Join Date
    Apr 2002
    Posts
    2,292
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    Just to add, that URL, that's how it is in the webpage in the href attribute of an anchor element -- verbatim. I've tried messing round with it in various encoding/decoding ways. No joy. Any ideas? It's a GET type URL obviously. It's obviously the query bit, after the ?, where something is going wrong. Do browsers do something with URLs like that when they're clicked on which isn't being done for curl?

  3. #3
    SitePoint Wizard
    Join Date
    Apr 2002
    Posts
    2,292
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    Aha, cracked it at last. Maybe predictably, it wasn't the URL getting mangled it was the cookies failing to get through because of http and https and the fact that the cookies were set to FALSE for the secure field which presumably stopped them getting through for the https requests.

    Haven't actually sussed out how to set curl cookies to secure TRUE (the fourth tabbed column in the text cookie file) yet -- I did it manually (find and replace in text editor) to see if that was the problem

  4. #4
    SitePoint Wizard
    Join Date
    Apr 2002
    Posts
    2,292
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by johnyboy View Post
    Aha, cracked it at last. Maybe predictably, it wasn't the URL getting mangled it was the cookies failing to get through because of http and https and the fact that the cookies were set to FALSE for the secure field which presumably stopped them getting through for the https requests.

    Haven't actually sussed out how to set curl cookies to secure TRUE (the fourth tabbed column in the text cookie file) yet -- I did it manually (find and replace in text editor) to see if that was the problem
    That turned out to be rubbish. Nothing to do with secure column. Anyway, it is fixed now. Thanks.

  5. #5
    SitePoint Wizard
    Join Date
    Apr 2002
    Posts
    2,292
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    Finally, finally properly sussed it. The url got from the webpage had & 's in. They needed converting to & 's before using with curl. Simple. Silly me.


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •