SitePoint Sponsor

User Tag List

Results 1 to 7 of 7
  1. #1
    SitePoint Addict
    Join Date
    Jun 2006
    Posts
    220
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    process curl requests in que

    Let me explain you the problem:

    I have a site with multiple buttons and on click of each button I have to pass some information in another server. Please keep in mind that I won't be able to use jQuery here. My approach is mentioned below:

    - On click of these button I will call a JS function which will trigger an ajax call to an internal php page. From that php page I will invoke curl to send data in other server.

    Now at a time there can be more than 1000 users who are clicking different buttons same time. I have two questions

    1. Can curl process hundreds of these requests without any problem? Do I need to make any special changes in the curl requests

    2. Considering very large no. of requests is there any other approach that I can follow?

    Thanks in advance.
    Nilanjan

  2. #2
    SitePoint Zealot txt3rob's Avatar
    Join Date
    Jul 2013
    Location
    Liverpool UK
    Posts
    171
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    i would set up a VPN between the two servers and work a way to post the data to the 2nd server with out curl cause i can imagine that many requests will lag your server.

  3. #3
    Keeper of the SFL StarLion's Avatar
    Join Date
    Feb 2006
    Location
    Atlanta, GA, USA
    Posts
    3,747
    Mentioned
    64 Post(s)
    Tagged
    0 Thread(s)
    Regarding another approach to follow; is there a need to have the button presses recorded by the second server instantaneously? You could stack the records up and send them across in batches every X minutes, for example.
    Never grow up. The instant you do, you lose all ability to imagine great things, for fear of reality crashing in.

  4. #4
    SitePoint Addict
    Join Date
    Jun 2006
    Posts
    220
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Thanks StarLion. But how to stack records and send them across in batches using php?

  5. #5
    SitePoint Guru bronze trophy
    Join Date
    Dec 2003
    Location
    Poland
    Posts
    925
    Mentioned
    7 Post(s)
    Tagged
    0 Thread(s)
    Make your ajax-invoked php scripts write data to a file or a database and then set up a cron job (every x minutes) to invoke another php script that will read your saved data and send it to the remote server in one go.

  6. #6
    SitePoint Addict
    Join Date
    Jun 2006
    Posts
    220
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    In that case it will increase load on DB also due to so much interaction

  7. #7
    SitePoint Guru bronze trophy
    Join Date
    Dec 2003
    Location
    Poland
    Posts
    925
    Mentioned
    7 Post(s)
    Tagged
    0 Thread(s)
    Well, if you want to store so much data from frequent user interactions then you will increase the load on something, anyway. An insert to the db will always be faster than sending the data by curl to a remote server - at least you can be sure (if all is working properly) that your script will finish fast whereas if there is network congestion your simultaneous curl requests can stack up resulting in many concurrent php scripts active and this way you are at risk of exceeding some server limits and in the worst case even bringing your site down when you reach the maximum allowed number of concurrent php requests.

    With the db you have some options of tuning it so as to minimize the overhead of your inserts (like using myisam, tuning innodb for faster but less safe inserts or using memory engine). Or simply save your data to a file in any format you choose (for example, each insert being a separate line of json-encoded data) - this should perform very well.

    With these methods you will not experience any problems if the remote server doesn't respond temporarily for any reason - the cron job will try sending the data again in a few minutes and that's it.


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •