SitePoint Sponsor

User Tag List

Results 1 to 9 of 9
  1. #1
    SitePoint Evangelist
    Join Date
    Jun 2005
    Posts
    436
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Sending POST Variable in a loop 3 Million Times

    I need to post one variable to a file, adding 1 to the variable after each POST, 3,000,000 times as fast as possible. What programming language should I use?

    Thanks,
    e39m5

  2. #2
    Programming Team silver trophybronze trophy
    Mittineague's Avatar
    Join Date
    Jul 2005
    Location
    West Springfield, Massachusetts
    Posts
    17,290
    Mentioned
    198 Post(s)
    Tagged
    3 Thread(s)

    Post

    Quote Originally Posted by Dr Evil
    3 million dollars times
    I don't think the "weak link" is going to be what programming language you use, but bandwidth.
    - Sorry, I couldn't resist.

  3. #3
    SitePoint Evangelist
    Join Date
    Jun 2005
    Posts
    436
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I have two instances of a flash application running a loop thats moving at a rate of about 2 posts per second on 2 computers. One is going from zero up and and the other from 3 million down, but at this rate its going to take a while.

    Would it be faster if the flash application was on the same server as the file I am posting to and not running from my computer?

    At this rate, if i find a better solution within a week I will probably benefit. I just hit 200,000 on the one that is moving up and it has been going since last night. I am only at 8000 on the one moving down

    e39m5

  4. #4
    Programming Team silver trophybronze trophy
    Mittineague's Avatar
    Join Date
    Jul 2005
    Location
    West Springfield, Massachusetts
    Posts
    17,290
    Mentioned
    198 Post(s)
    Tagged
    3 Thread(s)

    POST speed

    Is this something you'll have to do often? If it's just a one time deal maybe it would be easier to just let it run as it is until it's done. Otherwise I'm sure it would be faster to have everything on the same machine. I'm not familiar with flash so I can't really make any specific suggestions. Hopefully somebody that has done this will make a post.

  5. #5
    SitePoint Zealot
    Join Date
    Jul 2006
    Location
    Dundee, Scotland
    Posts
    179
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Mittineague it would take over 208 hours for it to run continuously.

    e39m5 it would be much quicker if you did have the calling program/script on the same pc/server as the program/script being called. At the moment you have the overhead of sending data over the network (presumably in this case the internet). Provided the pc/server was set up right having them on the same machine would completely eradicate this.

  6. #6
    SitePoint Evangelist
    Join Date
    Jun 2005
    Posts
    436
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I just got it on the server and I think its going faster. Thanks! Is there a better way to do this besides flash?

    e39m5

  7. #7
    SitePoint Zealot
    Join Date
    Jul 2006
    Location
    Dundee, Scotland
    Posts
    179
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Maybe if you let us know why and what you are doing this for it may help find a better way. I have personally not got any proper experience with flash only php.

  8. #8
    SitePoint Evangelist
    Join Date
    Jun 2005
    Posts
    436
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    The flash POSTs a number to a php file. The php file takes that number and puts it into a url. Than fsockopen is used to read the url, a piece of the contents is selected and saved into the database. I need to run it 3 million times to use every possible number

    e39m5

  9. #9
    Programming Team silver trophybronze trophy
    Mittineague's Avatar
    Join Date
    Jul 2005
    Location
    West Springfield, Massachusetts
    Posts
    17,290
    Mentioned
    198 Post(s)
    Tagged
    3 Thread(s)

    fsockopen

    The "weak link" is still the URL request. On the same server, you eliminate the flash to script traffic, but the script still has to fetch the URL content. I would simply have the PHP script (or another language, maybe Perl?) generate the number, and ration out the requests in portions over time. Depending on how much of the server's resources you want to devote to this and what the time constraints are.
    First I would have the script generate the test URLs and populate a database with them. This should take relatively little time. Then I would do the fetching during "off hours, low time" to either put the fetched info into the database, or delete the row, or flag it as a fail.


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •