SitePoint Sponsor

User Tag List

Page 2 of 3 FirstFirst 123 LastLast
Results 26 to 50 of 59
  1. #26
    SitePoint Enthusiast
    Join Date
    Feb 2005
    Posts
    38
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Hi,
    Below is some part of the plotter script, it reads data from the table and plot it.
    Plotting part is quite simple..I did it using GD.
    Here goes the script:

    <?php

    $hostname_mySQL = "10.8.144.133";
    $database_mySQL = "counter";
    $username_mySQL = "root";
    $password_mySQL = "root";
    $mySQL = mysql_pconnect($hostname_mySQL, $username_mySQL, $password_mySQL) or die(mysql_error());
    mysql_select_db($database_mySQL, $mySQL) or die("unable to select database 'db': " . mysql_error());

    $start = time() - 1;

    for($i=1;$i<6;$i++) {
    $sql = "select count(*) from counter where time='".$start."'";
    $fields = mysql_query($sql);
    if (!$fields) {
    die("query failed: " . mysql_error());
    }

    while($row = mysql_fetch_array($fields))
    {
    // Add values to the graph
    $val[]=$row[0];
    }
    }

    ....
    ....

    ?>

    The value i.e. count for each second comes in the array $val[] which is plotted against the corresponding time.

    Thanks..

    -Asim

  2. #27
    SitePoint Addict toggg's Avatar
    Join Date
    Jan 2005
    Location
    Auvergne/France
    Posts
    253
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Hi,
    so $val[] countains the counts for each seconds.
    Is it building it over a long period ? only 6 seconds? Your script is unclear.
    OK, I'm allready building some socket, should be UDP.
    It should count over each second, and then put the count if any in the database.
    So your plotter could quite stay the same.
    Difference is 0 or 1 access to SQL each second, more not.
    what kind of OS is it running under ?
    +
    bertrand Gugger toggg.com linux, PHP, Auvergne/France open source

  3. #28
    SitePoint Enthusiast
    Join Date
    Feb 2005
    Posts
    38
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    yes,
    I'm showing the graph for last 6 seconds only.
    That can be extended easily. But I kept it 6 for faster execution, as for each loop it has to perform count(*) for a timestamp.
    I'm running it under Linux (Apache , PHP, Mysql).

    -Asim

  4. #29
    SitePoint Addict toggg's Avatar
    Join Date
    Jan 2005
    Location
    Auvergne/France
    Posts
    253
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    OK,
    it's enough info for me, I think.
    Don't be nervous on the "post reply" button, I need perhaps 1 hour or 2.
    For the stuff, I think it's better allready to provide for several scripts in survey the same time.
    "Qui peut le plus, peut le moins." as we say here around.
    + (1 or 2 hours)
    bertrand Gugger toggg.com linux, PHP, Auvergne/France open source

  5. #30
    SitePoint Addict toggg's Avatar
    Join Date
    Jan 2005
    Location
    Auvergne/France
    Posts
    253
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Hi,
    You know how are the hours by computer people
    So that will be a little more as 3 hours.
    But I'm going on, I have a part working with unix sockets.
    +
    bertrand Gugger toggg.com linux, PHP, Auvergne/France open source

  6. #31
    SitePoint Addict toggg's Avatar
    Join Date
    Jan 2005
    Location
    Auvergne/France
    Posts
    253
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Talking Oh lala !

    Hi,
    Just some news to pull up the stuff as you know.
    Then you will also understand a few hours are necessary.
    The stratageme is quite simple but requires a few steps.

    You have script.php, this one (1 to simplify) you like to observe.
    I assume it can be several times connected at the same moment.
    For counting, this script.php will only deposit,
    without blocking and ignoring errors,
    4 bytes = packed time in a datagram unix socket.
    Lovely unix allready ensure the socket concurrent write.
    That's all for it.

    On the other side of the "pipe", and here again unix ensure a unique listener,
    you find the counter.php script.
    It's reading the socket and updating a cyclic array of hits/second,
    with some kind of autoshift/resync.
    Resync in case script.php does not talk for a moment, or counter.php itself restarts.
    For output when a "window" should be shifted, this counter.php will write it
    in a shared memory area.

    To the end, it's in this shared memory that the final counter client will read the data,
    and will display it, then with a few seconds delay.
    This final client will also have the privilege to start or stop the counter.

    Hope to shift to 10.000 pro second.
    I just miss the final client for now.
    Dig it
    bertrand Gugger toggg.com linux, PHP, Auvergne/France open source

  7. #32
    SitePoint Enthusiast
    Join Date
    Feb 2005
    Posts
    38
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Hi,
    As I understand, script.php when hit, will put some data(4 bytes) to the socket. Simultaneously, counter.php will read the data from the socket and put in an array so as to use it later (to draw the graph in this case).

    Please tell me if I'm wrong.
    Give me some more details so that I can move a step ahead.

    Also please tell me something about how resync will be implemented.

    I have a query :

    1. If a php script is hit simultaneously, how does it handle the situation? I mean it serves the requests one by one or is there any technique to handle it at the same time?

    -Asim

  8. #33
    SitePoint Addict toggg's Avatar
    Join Date
    Jan 2005
    Location
    Auvergne/France
    Posts
    253
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Hi,
    The socket acts as a pipe. All instances of script.php put their 4 bytes in it. Unix ensure that this operation is atomic, what means you can have many process writing the same time in this socket, you cannot loose any message and they will not mix.
    The central counter script is reading each pack of 4 bytes from this pipe and directly cumulate in an array of counters pro second. Simply it takes the timestamp send by the script.php and make +1 in the corresponding element of the array. In fact this script is permanently waiting the pipe so it will not consume any resource.
    The socket is playing the shock absorber, producing a queue to be treated by the counter script.
    As this counter script must permanently listen to the socket, it cannot do the output to the final client (you) directly. That's why the counter script just copy the current observation windows (around 10 seconds) when it's achieved in a shared memory area. In this way the counter script has no wait time and can actually proceed high level of hits.
    The final client read the results in this shared memory where it can also find the timestamp for each observation windows. These windows are stored in circular array. To avoid concurrence access with the counter script, 2 windows in it will be permanently reserved to the counter for it's current countings. In this way, even if it's right shifting the array, the client will get stable data.
    I was talking of resynch as the counter must start once and it can also be that the script will not be hit for a while. So when it is hit again, the counter must resynchronize its counter arrays (after copy out the rest old counters if any). In this version it's only the script.php which wake up the counter. Then if the counter detect it has not work since a while it copy and clear out all counters and restart to the timestamp.
    Hope that makes more sense, my english is so poor. I should have better taken more lessons in britains pubs
    I will rely on you, Asim, for the tests, so don't loose the thread please.
    What are you using in your load generator ?
    +
    bertrand Gugger toggg.com linux, PHP, Auvergne/France open source

  9. #34
    SitePoint Enthusiast
    Join Date
    Feb 2005
    Posts
    38
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Hi,
    That was very nice. The idea is great.
    It will take me some time to implement all these things.
    But if you have written something (I believe you have) and it is not protected by copyright laws , then you please give it to me. I will modify it according to my needs and see if it works perfectly (I believe it will)....!!
    I won't break the thread..you have helped me a lot..thanks for that..

    -Asim

  10. #35
    SitePoint Addict toggg's Avatar
    Join Date
    Jan 2005
    Location
    Auvergne/France
    Posts
    253
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by pintu_asim
    It will take me some time to implement all these things.
    I think I will deliver some ready to use/include scripts
    I don't know if it can fit in the forum, scripts are short but I have 6 pieces inclusive 2 common libraries (for socket and for shared memory)
    That could better do an article.
    Anyway I will put that on http://toggg.com/
    Quote Originally Posted by pintu_asim
    ... and it is not protected by copyright laws
    My english vocabulary is very limited and does not include such words
    I hope I can send you some first alpha this evening.
    Please send me a mail via SP interface, so I can send the copy back to you.
    +
    PS You did not show nothing from you load generator ?
    bertrand Gugger toggg.com linux, PHP, Auvergne/France open source

  11. #36
    SitePoint Enthusiast
    Join Date
    Feb 2005
    Posts
    38
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Hi,
    I have sent you the mail. Please inform me whether you have got it or not.
    The load generators I am using is developed by our team members only. I don't know much details about them.
    One of them is developed in JAVA and another one is developed in C.
    So we need some kind of tool that can test the accuracy of the load generators.
    Hey, can I use PHP for such broad purpose like developing load generators??
    Is there any concept in PHP for thread programming..interrupts..etc??

    -Asim

  12. #37
    SitePoint Addict toggg's Avatar
    Join Date
    Jan 2005
    Location
    Auvergne/France
    Posts
    253
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Hi,
    I got the mail. I will reply you soon with some unachieved code so you can better see.
    For the load generator, it will even be better test as we ignore what they are doing, blind test.
    I will first test with hit simulator from command line, simultaneously start more of it and counting back no hit were loosed
    Here is this test script, the @socket_sendto(...) will be the only stuff to add in script.php.
    (I will make an include for it)
    PHP Code:
    require_once 'skntSok.php';
    $usl = isset($_SERVER['argv'][1]) ? $_SERVER['argv'][1] + 1000// usleep
    $max = isset($_SERVER['argv'][2]) ? $_SERVER['argv'][2] + 100// count
    $fp socrea();
    while (
    $max--) {
        if (@
    socket_sendto($fppack('N',time()), SOCKOUNT_LTIM0x100SOCKOUNT_NAME)
                 == 
    false) {
            echo 
    "Error by send message ($max)\n";
        }
        
    usleep($usl);

    PHP Code:
    define ('SOCKOUNT_NAME''sock');
    define ('SOCKOUNT_LTIM'strlen(pack('L',time())));

    function 
    socrea($bind false) {
        if (
    false == ($fp = @socket_create(AF_UNIXSOCK_DGRAM0))) {
            die(
    "Couldn't create socket, error code is: " socket_last_error() .
                
    ",error message is: " socket_strerror(socket_last_error()));
        }
        if (
    $bind and (false == @socket_bind($fpSOCKOUNT_NAME))) {
            die(
    "Couldn't bind socket, error code is: " socket_last_error() .
                
    ",error message is: " socket_strerror(socket_last_error()));
        }
        return 
    $fp;

    With PHP you can fork and signaling with children the same as C so you can multithread.
    +
    bertrand Gugger toggg.com linux, PHP, Auvergne/France open source

  13. #38
    SitePoint Addict toggg's Avatar
    Join Date
    Jan 2005
    Location
    Auvergne/France
    Posts
    253
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Hi,
    This counting over socket and shared memory is becoming a real project !

    Please check http://toggg.com/tiki/tiki-index.php?page=Sockount

    If some people could try it, I would be happy to have some feedback
    Thanks.
    +
    bertrand Gugger toggg.com linux, PHP, Auvergne/France open source

  14. #39
    SitePoint Enthusiast
    Join Date
    Feb 2005
    Posts
    38
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Many Congratulations..
    It is successful to count 8000-9000 hits/second ..!!!!
    Hope it will be a wonderful project in days to come

    -Asim

  15. #40
    SitePoint Addict toggg's Avatar
    Join Date
    Jan 2005
    Location
    Auvergne/France
    Posts
    253
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Cool Counting 20.000 hits / second !

    Shwagata asim !
    I dig it really ! You know what Sockount can count now? 18.492 hits/second lol & lol & lol
    It's more as 1000 quicker as yours
    Take a look: http://toggg.com/tiki/tiki-index.php?page=Sockount
    He! He! Counting only in memory !
    And not one hit loosed
    Someone help me to go over 20.000 hits counting pro second ?

    Fight against nanoseconds !
    +
    bertrand Gugger toggg.com linux, PHP, Auvergne/France open source

  16. #41
    SitePoint Addict toggg's Avatar
    Join Date
    Jan 2005
    Location
    Auvergne/France
    Posts
    253
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Really want to count over 20.000 hits/second

    Bonsoir SP/computers/PHP fans,
    Sorry to pull up the stuff this way but
    It's a bet I made with pintu_asim.
    Counting hits/second of a script over 150 up to 500 what you achieve with disks means.
    I promised 15.000
    Now we reach 18.492 lol
    on a poor machine.

    can anybody help us ?

    I mean someone with a fight machine ?
    20.000 hits/second is my bet but could go up to 100.000
    material there --> http://toggg.com/tiki/tiki-index.php?page=Sockount

    We would be really to get your opinion.
    +
    shwagata
    bertrand Gugger toggg.com linux, PHP, Auvergne/France open source

  17. #42
    eschew sesquipedalians silver trophy sweatje's Avatar
    Join Date
    Jun 2003
    Location
    Iowa, USA
    Posts
    3,749
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Here was an output from our development web server:
    23:42:32 500 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 24948 25052 0 0 0 0
    50500

    I guess the 24,948 and 25,052 are counts per second then?
    Jason Sweat ZCE - jsweat_php@yahoo.com
    Book: PHP Patterns
    Good Stuff: SimpleTest PHPUnit FireFox ADOdb YUI
    Detestable (adjective): software that isn't testable.

  18. #43
    SitePoint Addict toggg's Avatar
    Join Date
    Jan 2005
    Location
    Auvergne/France
    Posts
    253
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    So good !
    sweatje ! You're a friend !
    Now we count more as 25000 ! Champagne !
    50.500 ? I don'tbelieve it. You kill
    Merci,
    +
    bertrand Gugger toggg.com linux, PHP, Auvergne/France open source

  19. #44
    eschew sesquipedalians silver trophy sweatje's Avatar
    Join Date
    Jun 2003
    Location
    Iowa, USA
    Posts
    3,749
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Our dev web server is nothing special, basically just a workstation retasked to server as a web server:

    sweatje@devweb proc $ uname -a
    Linux devweb 2.6.9-gentoo-r13 #2 Wed Jan 5 16:04:57 CST 2005 i686 Intel(R) Pentium(R) 4 CPU 2.66GHz GenuineIntel GNU/Linux

  20. #45
    SitePoint Addict toggg's Avatar
    Join Date
    Jan 2005
    Location
    Auvergne/France
    Posts
    253
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by sweatje
    I guess the 24,948 and 25,052 are counts per second then?
    Yes.
    So I guess you have sent 50,000 hits the first time,
    if yes nothing loosed, right ?
    What's that 50500 ? A real count ?
    Again many thanks,
    +
    bertrand Gugger toggg.com linux, PHP, Auvergne/France open source

  21. #46
    SitePoint Addict toggg's Avatar
    Join Date
    Jan 2005
    Location
    Auvergne/France
    Posts
    253
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    So,
    100,000 hits/seconds possible ?
    +
    bertrand Gugger toggg.com linux, PHP, Auvergne/France open source

  22. #47
    eschew sesquipedalians silver trophy sweatje's Avatar
    Join Date
    Jun 2003
    Location
    Iowa, USA
    Posts
    3,749
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    50500 is the total I think (I have not looked at what your scripts are outputting). I was having some problems with permissions on the socket and was sending 500 messages. Once I got it working I kicked it up to 50000. That looks like a mixture of the two reported as a total line?

  23. #48
    SitePoint Addict toggg's Avatar
    Join Date
    Jan 2005
    Location
    Auvergne/France
    Posts
    253
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Salut,
    Yes it's the total, I know right the guy who did that.
    Sorry sweatje, I forgot I put the sum in output.
    I did that to check no hits loosed.
    Quick but acurate, that's what we want
    So far, seems you get it 2.
    Dig it.
    +
    bertrand Gugger toggg.com linux, PHP, Auvergne/France open source

  24. #49
    eschew sesquipedalians silver trophy sweatje's Avatar
    Join Date
    Jun 2003
    Location
    Iowa, USA
    Posts
    3,749
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Would it make any difference on a system with an opcode cache? I would think not, as all of your logging is happening in scripts which are already running, not individual loads of scripts, but I guess it could be worth checking.

  25. #50
    eschew sesquipedalians silver trophy sweatje's Avatar
    Join Date
    Jun 2003
    Location
    Iowa, USA
    Posts
    3,749
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Using your loop, it looks like my system tops out at about 26200, but can do that reasonably reliably.

    07:33:35 280 30125 19595 0 0 0 0 19288 26232 26202 26217 26211 26261 26196 26179 26147 26185 26200 26223 26208 26254 26236 26198 26194 26216 26232 8921 0 0 0

    Your loop:
    PHP Code:
    while ($max--) {
        if (@
    socket_sendto(@socket_create(AF_UNIXSOCK_DGRAM0), pack('N',time()), SOCKOUNT_LTIM0x100SOCKOUNT_NAME)
                 == 
    false) {
            echo 
    "Error by send message ($max)\n";
        }
        if (
    $talk) {
            echo 
    $max.' ';
        }
        if (
    $usl) {
            
    usleep($usl);
        }

    When I squeeze the loop a bit to this:
    PHP Code:
    while (--$max) {
    @
    socket_sendto(@socket_create(AF_UNIXSOCK_DGRAM0), pack('N',time()), SOCKOUNT_LTIM0x100SOCKOUNT_NAME);

    I get about an extra 1000 counts per second:
    07:38:04 2388 27443 27490 27472 15207 0 0



    When I further removed the two error suppression operators I was able to get of 30000 consistantly, over 34000 at peak:

    07:47:01 12005 34816 30873 30821 31146 30999 31041 30875 30863 31181 31015 31015 30576 30477 30341 30356 21600 0 0

    If you want to maintain the flexibility with your test script, I would probably remove the conditional checks from your loop in the script, i.e.:

    PHP Code:
    if ($talk) {
        if (
    $usl) {
            while (--
    $max) {
                if (
    socket_sendto(socket_create(AF_UNIXSOCK_DGRAM0), pack('N',time()), SOCKOUNT_LTIM0x100SOCKOUNT_NAME)
                         == 
    false) {
                    echo 
    "Error by send message ($max)\n";
                }
                echo 
    $max.' ';
                
    usleep($usl);
            }
        } else {
            while (--
    $max) {
                if (
    socket_sendto(socket_create(AF_UNIXSOCK_DGRAM0), pack('N',time()), SOCKOUNT_LTIM0x100SOCKOUNT_NAME)
                         == 
    false) {
                    echo 
    "Error by send message ($max)\n";
                }
                echo 
    $max.' ';
            }
        }
    } else {
        if (
    $usl) {
            while (--
    $max) {
                if (
    socket_sendto(socket_create(AF_UNIXSOCK_DGRAM0), pack('N',time()), SOCKOUNT_LTIM0x100SOCKOUNT_NAME)
                         == 
    false) {
                    echo 
    "Error by send message ($max)\n";
                }
                
    usleep($usl);
            }
        } else {
            while (--
    $max) {
                if (
    socket_sendto(socket_create(AF_UNIXSOCK_DGRAM0), pack('N',time()), SOCKOUNT_LTIM0x100SOCKOUNT_NAME)
                         == 
    false) {
                    echo 
    "Error by send message ($max)\n";
                }
            }
        }

    It might look ugly, but it optimizes the inner loop.


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •