counting number of hits per second at run time

[toggg]

Hi sweatje,
Many thanks for your tests !
You pulled up the count to 34816 ! New best score ! Bravo !

In fact, as you could see, it's not the counting limiting but the sender
It's only test to ensure the counter will allways "follow"

You're right a production version should be optimized.
That with ugly is to forgot in fight against nanoseconds

Nevertheless the real sender code wich will be included in the script whose hits are to be counted *willl* keep the @. As side statitistic that should not block nor alter the production.
But it's a good idea for tests, just to pull up the stuff.
I don't know for opcode ? I tested under Apache 2.0 Filter without problem.

So, I can just say MERCI !
Next week with graphic output. You like that ?
à+

[pintu_asim]

Hi All,
I think we can further optimize the script (skntTst.php) by using
$fp = socrea();
and using $fp instead of using socket_create() each time in socket_sendto().
Please tell me what do you think of it?

-Asim

[toggg]

Hi asim,
Right! You save the time socket_create() uses.
It's a good idea, what sweatje even didn't used.
Shoulld be around 10% better.

OK for me, makes the test harder.
Just to recall: the target is here to get the counter out.
I mean, it's not only the hit simulator to enhance,
but should the counter never loose info.

Still the case
best,
à+

[Austin_Hastings]

Hi asim,
Right! You save the time socket_create() uses.

The only thing left is the pack() and the if statements. I'm not sure if caching the result at the PHP level would be faster than re-invoking pack each time. The if statements could probably be selectively eliminated using eval to build the function on the fly (much like conditional compilation in C):

  $last_time = time() - 100;
  $last_pack = 0;
  
  // ...
  
  $socket = @socket_create(AF_UNIX, SOCK_DGRAM, 0);
  $delay = $talk || $usl;
  
  while ($max--)
  {
      # Cache the pack() operation?
      $now = time();
      if ($now != $last_time)
      {
        $last_time = $now;
        $last_pack = pack('N', $now);
      }
  
      if (@socket_sendto($socket, $last_pack, SOCKOUNT_LTIM, 0x100, SOCKOUNT_NAME) == false)
      {
          echo "Error by send message ($max)\n";
      }
  
      if ($delay)
      {
          if ($talk)
          {
             echo $max . ' ';
          }
          if ($usl)
          {
               usleep($usl);
          }
       }
  } 

The 'eval' version looks like:

  $function_body = '
$last_time = time() - 100;
   $last_pack = 0;
   // ...
   $socket = @socket_create(AF_UNIX, SOCK_DGRAM, 0);
  
  while ($max--)
   {
       # Cache the pack() operation?
       $now = time();
       if ($now != $last_time)
       {
         $last_time = $now;
         $last_pack = pack("N", $now);
       }
   
       if (@socket_sendto($socket, $last_pack, SOCKOUNT_LTIM, 0x100, SOCKOUNT_NAME) == false)
       {
           echo "Error by send message ($max)\n";
       }
      ';
  
  if ($talk)
  {
      $function_body .= 'echo $max . " ";
      ';
  }
  if ($usl)
  {
      $function_body .= 'usleep(' . $usl . ');
      ';
  }
  
  $function = create_function('$max', $function_body);
  
  $function($max); 

=Austin

[toggg]

Hi,
Thanks Austin, it's a good idea
I applied it this way, even burning in the max value:

set_time_limit(0);
include 'skntSok.php';
$usl = isset($_GET['usleep']) ? $_GET['usleep'] + 0 : 1000; // usleep
$max = isset($_GET['count']) ? $_GET['count'] + 0 : 10000; // count
$talk = isset($_GET['talk']) ? true : false; // verbose
$function_body = '$max = '.$max.';
$fp = socrea();
while ($max--) {
    @socket_sendto($fp, pack(\'N1\',time()), SOCKOUNT_LTIM, 0x100, SOCKOUNT_NAME)
    or die("Error by send message ($max)\n");
';
if ($talk) {
    $function_body .= 'echo $max . " ";';
}
if ($usl) {
    $function_body .= 'usleep(' . $usl . ');';
}
$function_body .= '}';
$function = create_function('', $function_body);
 
$function(); 

I made some tests on a poor machine A and a better server B, we have a little better:
My old test script goes up to A: 3952 hits/second, B: 17332
Applying asim's single creation of the socket, A goes up 5436 so gain 37% , B: 24848 (43%).
Then applying your idea as above, we obtain A: 5600 (3%) and B: 25710 (3.5 %)
As you see, it's quite little 3% better, just the 2 if saved.
In the fight against nanoseconds 1% is 1%
OK, with some accelerator who would "compile" the function instead of interpret it again and again, (I don't know if that exist) your idea could give much more gain.

Thanks all to give a help
à+

[Austin_Hastings]

Hi,
Thanks Austin, it's a good idea

Bertrand,

Did you try caching the results of the pack() call? If you're getting 5- or 25- thousand calls per second, all of them except one will have an identical return value from time().


=Austin

[toggg]

Salut Austin & SP club !
Now, I get your point
It was unclear to me before, thanks.

How to implement that ? Obviously, it's useless to check the second is not changed before each sending.
The solution could be to run first some bench to evaluate the speed of this sending so the script knows how many it can send without checking the time.
The problem is: this velocity is never constant as the CPU is shared.
Example what you get over 5 seconds:
09:40:42 16759 22199 21678 15162 21642
Average: 19488 Standard deviation: 3277
So this number is quite difficult to guess

Nevertheless, some half way is certainly possible within the loop:
- send with synch on the seconds, checking time() each sending
- send bunches with checks of microtime() until say 80% the second, and loop again.
And if we consider the goal of this test sender, finding the limits of the counter itself on the other side of the socket, this sender does not need to be fully accurate. (just because it cannot be, see above .
From the counter's side it's the same.

That's what I will build in the next.
I will also try something else: build some bunch sending function with no loop but a sequence of say 1000 sendings. Not sure the interpreting cost will not decrease the performance, it's to be tested.

So, thanks again,
à+ for the results.

[Austin_Hastings]

Salut Austin & SP club !
Now, I get your point
It was unclear to me before, thanks.

How to implement that ? Obviously, it's useless to check the second is not changed before each sending.

Why? There's two functions being called: time() and pack().

Ignoring the time() function for a moment, you can save the result of pack() so long as time() keeps returning the same value.

I don't know if the call to pack is faster than the if (time == time) comparison, but I suspect it probably is.

My first example had a save-and-check part to it.


As far as inlining the packet sending code, that seems like a good use for the eval: build two functions, one to send 100 packets, the other to check whether to reset the time or not. You'll be accurate for the most part, which is what counts.

  // This should be generated using eval...
  function send100($soc, $buf)
  {
    @socket_sendto($soc, $buf, SOCKOUNT_LTIM, 0x100, SOCKOUNT_NAME); # 1
    @socket_sendto($soc, $buf, SOCKOUNT_LTIM, 0x100, SOCKOUNT_NAME); # 2
   // ...
    @socket_sendto($soc, $buf, SOCKOUNT_LTIM, 0x100, SOCKOUNT_NAME); # 100
  }
  
  function send_packets($max)
  {
    // ...
   
   $socket = @socket_create(AF_UNIX, SOCK_DGRAM, 0);
     $now = time();
  
    while ($max > 0)
   {
     # Cache the pack() operation?
     if ($now != $last_time)
     {
       $last_time = $now;
       $packed_time = pack('N', $now);
     }
   
      send100($socket, $packed_time);
      $max -= 100;
      $now = time();
   }
  } 

[toggg]

Hi Asim, Jason, Austin & nanoseconds Killers !
Revised goals for the tester:

• send the most quickly possible timestamps thru a socket,
• as far as possible respect the real time for these timestamps,
• allmost respect a total to be sent,
• but have an exact count reporting of what was actually sent !
  ( the counter other side the socket has to be tested, not the tester

Definive changes:

• Other tester scripts will take care of lower rythms so usleep() ripped out.
• No more talk outside errors as:
• Any error customized to die() so no more tests.
• Constants everywhere (not sure it uses)

Onliest parameter left: count now as "minimal to achieve".
create_function() just escaped the usleep an talk tests: out.
time() then pack() and counting "cached"
The scenario of the loop waiting enough has been sent:

• (re)synchroinze on whole second
• set initial bunch size to K=1000 hits
  [**] Send bunch
  [**] Evaluate speed and compute next bunch = half rest before full second
  [**] Loop to send bunch while 100 left
• loop to synchro while not enough sent

define ('K', 1000);    // first test bunch
define ('R', 100); // half number rest before synch
define ('N', 'sock'); // socket name
define ('L', strlen(pack('L',time()))); // timestamp length
// error handler function
function myDie($errno, $errstr) { die("<b>FATAL</b> [$errno] $errstr<br />\n"); }
}
error_reporting(E_ALL);
set_error_handler("myDie"); // didn't say mayday :)
$fp = socket_create(AF_UNIX, SOCK_DGRAM, 0);
set_time_limit(0);
$cnt = $max = isset($_GET['count']) ? $_GET['count'] + 0 : 10000; // count
$C = pack('N1', $last = time());
while ($max >= 0) {
    while (($now = time()) == $last) {
        socket_sendto($fp, $C, L, 0x100, N);
        $max--;
    }
    $C = pack('N1', $last = $now);
    $mm = ($ml = array_sum(explode(' ', microtime()))) + 1;
    $n = K;
    while ($n > R) {
        $nl = $n;
        while ($n--) {
            socket_sendto($fp, $C, L, 0x100, N);
        }
        $max -= $nl;
        $m = array_sum(explode(' ', microtime()));
        $n = floor($nl * ($mm - $m) / ($m - $ml) / 2);
        $ml = $m;
    }
}
echo "Sent: ".($cnt - $max)."\n"; 

The bunches send (excl. the 1000 test) within 4 seconds
14774 4758 2082 1036 512 254 126 62
11643 5485 2727 1335 667 338 168 84
11653 5460 2741 1367 680 339 168 83
11674 5503 2724 1362 674 336 168 82
Sent: 103304
And the counter result:
06:10:35 8062 24673 23538 23582 23449 0
103304 (24673) (nothing loosed

With it I could reach: 26961 (server B) to compare to 24848
what means we save 8.5 %
It's not so much, in fact, pack() does not pack but just move 4 bytes and time() quite only 20 bytes: it's not heavy compared to socket send it seems.

Thanks for the suggestions,
I will work out the counter side to get it smart
à+