SitePoint Sponsor

User Tag List

Results 1 to 17 of 17
  1. #1
    SitePoint Addict
    Join Date
    Jun 2005
    Posts
    196
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Rotating Proxies and Lists

    I need to set up a rotating proxy for an application i am building. I have a cURL script that reads a page, but i need to seem like its not being called from the same IP address each time. I would like to have a list of proxies that i can rotate between each call i make.

    I have looked at Glype and PHPproxy but both i cannot get to work correctly. Does anybody have any suggestions, script examples or good info web pages they can provide. I would be interested in using Glype or PHPproxy if someone can point me in a good direction to use them correctly. Otherwise anything else would be perfect as well.

    Thanks much.

    Cesar

  2. #2
    Non-Member
    Join Date
    Oct 2009
    Posts
    1,852
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    It looks like you're trying to do something unwelcome.
    Consider not to build this kind of application.

  3. #3
    dooby dooby doo silver trophybronze trophy
    spikeZ's Avatar
    Join Date
    Aug 2004
    Location
    Manchester UK
    Posts
    13,806
    Mentioned
    158 Post(s)
    Tagged
    3 Thread(s)
    I'm with Shrapanel on this. Why would you need a rotating proxy to cover your tracks if you were doing something legit?
    Mike Swiffin - Community Team Advisor
    Only a woman can read between the lines of a one word answer.....

  4. #4
    SitePoint Addict
    Join Date
    Jun 2005
    Posts
    196
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I am scanning through Craigslist RSS. They have blocked me for using it too often. Why would this be not legit?

  5. #5
    SitePoint Addict
    Join Date
    Jun 2005
    Posts
    196
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    http://www.craigslist.org/about/help/faq
    http://www.craigslist.org/about/terms.of.use#conduct

    Nothing here says anything about RSS. I just think they see my IP a lot and think im doing something bad.. IM not! I'm just looking for the best deals first.

  6. #6
    SitePoint Wizard
    Join Date
    Nov 2005
    Posts
    1,191
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Because you're using it too often?
    It's their feed, their servers, their bandwidth ...

  7. #7
    Unobtrusively zen silver trophybronze trophy
    paul_wilkins's Avatar
    Join Date
    Jan 2007
    Location
    Christchurch, New Zealand
    Posts
    14,700
    Mentioned
    101 Post(s)
    Tagged
    4 Thread(s)
    Quote Originally Posted by cesarcesar View Post
    I am scanning through Craigslist RSS. They have blocked me for using it too often. Why would this be not legit?
    Because you are directly violating their terms of use?

    Additionally, you agree not to:
    ...
    u) use automated means, including spiders, robots, crawlers, data mining
    tools, or the like to download data from the Service - unless expressly
    permitted by craigslist;
    as well as

    12. ACCESS TO THE SERVICE

    craigslist grants you a limited, revocable, nonexclusive license to access
    the Service for your own personal use. This license does not include:
    (a) access to the Service by Posting Agents; or (b) any collection,
    aggregation, copying, duplication, display or derivative use of the Service
    nor any use of data mining, robots, spiders, or similar data gathering and
    extraction tools for any purpose unless expressly permitted by craigslist.
    Please stop it. Now.
    Programming Group Advisor
    Reference: JavaScript, Quirksmode Validate: HTML Validation, JSLint
    Car is to Carpet as Java is to JavaScript

  8. #8
    SitePoint Addict
    Join Date
    Jun 2005
    Posts
    196
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    do they have terms and conditions?
    i just posted a link to it which says nothing. If it says nothing that im not doing anything wrong as far as they have posted in the T&C.

    Because you're using it too often??????????
    I am assuming this because its worked for me for along time when i searched daily. I have been searching hourly for a week now and have seen my desired results jump greatly. Now the search wont work at all. I assume im being blocked.

  9. #9
    SitePoint Addict
    Join Date
    Jun 2005
    Posts
    196
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    they provide and RSS. WTF is it for if you cant read it?

  10. #10
    SitePoint Addict
    Join Date
    Jun 2005
    Posts
    196
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    use automated means
    Isnt an RSS reader just that?

  11. #11
    Unobtrusively zen silver trophybronze trophy
    paul_wilkins's Avatar
    Join Date
    Jan 2007
    Location
    Christchurch, New Zealand
    Posts
    14,700
    Mentioned
    101 Post(s)
    Tagged
    4 Thread(s)
    Quote Originally Posted by cesarcesar View Post
    Isnt an RSS reader just that?
    That is permitted by them in their terms of use

    it's worth a read.
    Last edited by paul_wilkins; Jan 15, 2010 at 05:06. Reason: minor rephrasing
    Programming Group Advisor
    Reference: JavaScript, Quirksmode Validate: HTML Validation, JSLint
    Car is to Carpet as Java is to JavaScript

  12. #12
    SitePoint Wizard
    Join Date
    Nov 2005
    Posts
    1,191
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    @cesar
    Heh, that was a quick round of posting, I edited when I saw your second post.

    You don't have a natural right to access craigslist. If they want to block you they can. If you believe they are blocking you for an inadequate reason, you can probably take it up with them, and perhaps find out how not to get blocked.

    Or you can just ignore their rules ... probably won't get alot of help here though :P

  13. #13
    SitePoint Addict
    Join Date
    Jun 2005
    Posts
    196
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Or you can just ignore their rules ... probably won't get alot of help here though :P
    i plan to.... i wont ask here again here.

    Moderator go ahead and pull this thread.

  14. #14
    Unobtrusively zen silver trophybronze trophy
    paul_wilkins's Avatar
    Join Date
    Jan 2007
    Location
    Christchurch, New Zealand
    Posts
    14,700
    Mentioned
    101 Post(s)
    Tagged
    4 Thread(s)
    Is there perhaps some way in which we can help the OP to use RSS to meet his needs?
    Programming Group Advisor
    Reference: JavaScript, Quirksmode Validate: HTML Validation, JSLint
    Car is to Carpet as Java is to JavaScript

  15. #15
    dooby dooby doo silver trophybronze trophy
    spikeZ's Avatar
    Join Date
    Aug 2004
    Location
    Manchester UK
    Posts
    13,806
    Mentioned
    158 Post(s)
    Tagged
    3 Thread(s)
    So you plan to ignore Craigslist rules and carry on attempting to make your RSS reader/mine look authentic by appearing to be many different people by switching IP address......


    Good luck with that and the resulting concequences.

    Paul - no.
    and no, this thread wont be being pulled.
    Mike Swiffin - Community Team Advisor
    Only a woman can read between the lines of a one word answer.....

  16. #16
    SitePoint Wizard
    Join Date
    Nov 2005
    Posts
    1,191
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by cesarcesar View Post
    I am assuming this because its worked for me for along time when i searched daily. I have been searching hourly for a week now and have seen my desired results jump greatly. Now the search wont work at all. I assume im being blocked.
    Quote Originally Posted by pmw57 View Post
    Is there perhaps some way in which we can help the OP to use RSS to meet his needs?
    Key phrase here "I assume I am being blocked", key intent of thread "Those @#$ can't block me!"

    Help away ...

  17. #17
    SitePoint Addict
    Join Date
    Jun 2005
    Posts
    196
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    So you plan to ignore Craigslist rules and carry on attempting to make your RSS reader/mine look authentic by appearing to be many different people by switching IP address......
    anonymous domain reg, anonymous hosting, anonymous private proxies should do the trick.

    NOw i will be a good sitepointer and never ask a less than squeaky clean question again.


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •