SitePoint Sponsor

User Tag List

Page 1 of 2 12 LastLast
Results 1 to 25 of 29
  1. #1
    SitePoint Enthusiast
    Join Date
    Jan 2002
    Posts
    70
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Optimization Suggestions [Echo Long Strings]

    Hello,

    I am hoping someone will be able to offer a few suggestions in optimizing the methods of the script I am developing.

    I have broken the processing times down into segments, the major inefficiency comes in the last statement of the script, the echo of the data.

    •Segment 1: Processing time 0.00574 seconds.
    [Loads information from the SQL database regarding the page in question]

    •Content Processing: Processing time 0.00505 seconds.
    [Processes the content for the webpage called]

    •Other Modules: Processing time 0.00247 seconds.
    [Loads any other information into the webpage]

    •Segment 2: Processing time 0.00317 seconds.
    [Using STR_REPLACE, replaces the data into the template]

    •Template Output: Processing time 0.55253 seconds.
    Echo's one big string containing the page data with template, etc..

    As you can see, the last "segment" is horribly inefficient, it makes up the bulk of the page generation time. The last segment works like the following:

    PHP Code:
    //$output contains the entire page HTML
    echo $output
    Can anyone recommend another method of outputting the data, possibly another statement that is more efficient than using the echo statement as that seems very slow for longer statements.

    Kind Regards,
    Shawn
    Last edited by Shawn (GECom); Jun 24, 2002 at 09:19.

  2. #2
    SitePoint Wizard silver trophy Karl's Avatar
    Join Date
    Jul 1999
    Location
    Derbyshire, UK
    Posts
    4,411
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Hi,

    If you're using a templating system then there really isn't any other way of doing, except to incrementally output the data - but that may increase the time taken, then again it may not.

    You must be really echo'ing a lot of data for it to take that amount of time, I have a templating system and app the loads info from a pgSQL DB (Slower than mySQL in my experience - but then again, mySQL would be slower if I used it in this case), places the content into the template then renders the template through an XSL parser and outputs the data. To do all that only take about 0.11s and if it loads data from the cache it takes about 0.6s

    I would looks at using the Zend Optimizer if you're not already and possibly look at the hardware the script is running on and the loads other sites etc. are placing on them as this to me would appear to be your bottleneck.
    Karl Austin :: Profile :: KDA Web Services Ltd.
    Business Web Hosting :: Managed Dedicated Hosting
    Call 0800 542 9764 today and ask how we can help your business grow.

  3. #3
    SitePoint Enthusiast
    Join Date
    Jan 2002
    Posts
    70
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I just setup the script to output the number of characters in the $output variable before echoing it, one one of the larger pages the number of characters was 29535, yikes!

    I think the problem comes with this many characters in a single variable, echoing that is what takes the longest time so I think it may work better if I echo in "increments".

    Thanks,
    Shawn

  4. #4
    SitePoint Enthusiast
    Join Date
    Jan 2002
    Posts
    70
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I've been testing this script and trying to optimize it to the best of my abilities. I just ran a few more tests..

    The average time to process all the content and input it into the variables is 0.02 seconds. over a period of about 10 minutes, server loads around 0.2 on a shared server. The output variable contains roughly 25000 characters.

    To echo this variable after all the processing has occured, the script takes on avarage around 0.55 seconds to output these 25000 characters from a single variable.

    -Shawn

  5. #5
    SitePoint Columnist Skunk's Avatar
    Join Date
    Jan 2001
    Location
    Lawrence, Kansas
    Posts
    2,066
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    You could try using Smarty. It's a fully featured template engine which compiles templates in to PHP code when they are created. This gives fantastic performance when outputting HTML at the end as all of the "static" chunks are placed outside PHP tags where they will not be processed by the PHP engine. More info here:

    http://smarty.php.net/

  6. #6
    SitePoint Enthusiast
    Join Date
    Jan 2002
    Posts
    70
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Originally posted by Skunk
    You could try using Smarty. It's a fully featured template engine which compiles templates in to PHP code when they are created. This gives fantastic performance when outputting HTML at the end as all of the "static" chunks are placed outside PHP tags where they will not be processed by the PHP engine. More info here:

    http://smarty.php.net/
    Thanks for the suggestion. I have developed a custom template engine that seems to work very effectively, except for this final echo statement. I am looking into redeveloping a small portion of it to echo the statements as they are processed rather than storing it in a variable until the end and echoing it at once.

    Now, some may find this a little interesting with regards to the echo issue.

    Since it takes no time to echo small strings but problems with larger ones, I figured, ok, lets make the strings smaller. So, to test it out, I produced the following code:

    PHP Code:
    $output explode("\n"$output);
    while (list (
    $line_num$line) = each ($output)) {
    echo 
    $line"\n";

    Basically, this code creates an array, breaks apart the elements wherever there is a line break. Now, what is strange, this actually is MUCH faster than echoing the single variable, something is really not right here. In terms of comparison, this on an average string of 25000 characters and 1000 line breaks, is about 4x faster than simply echoing the string at one time.

    If anyone can offer any kind of explanation, it would be appreciated.

    Regards,
    Shawn

    [Edit: Fixed a few spelling mistakes]

  7. #7
    SitePoint Evangelist cyngon's Avatar
    Join Date
    Aug 2001
    Location
    Livonia, MI, USA
    Posts
    513
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Wow.

    That 4x speed gain by breaking up the string into lines really suprises me. I was expecting it would be slower to do it that way.

    Next time I have a script that needs to echo a huge string, I may try some benchmarks of my own. In the mean time, does anyone have any idea why it would be faster?

  8. #8
    SitePoint Enthusiast
    Join Date
    Jan 2002
    Posts
    70
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Originally posted by cyngon
    Wow.

    That 4x speed gain by breaking up the string into lines really suprises me. I was expecting it would be slower to do it that way.

    Next time I have a script that needs to echo a huge string, I may try some benchmarks of my own. In the mean time, does anyone have any idea why it would be faster?
    I know the feeling, it seems really strange, maybe PHP is haunted or something

    -Shawn

  9. #9
    SitePoint Evangelist cyngon's Avatar
    Join Date
    Aug 2001
    Location
    Livonia, MI, USA
    Posts
    513
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I would say a functon like echo going 4 times slower than it could in some cases almost qualifies as a bug.

    Heh, we could write a new echo function:

    PHP Code:
    function echo_fast($text) {
       if (
    strlen($text) > 1000) {
          
    $textexplode("\n"$text);
          while (list (
    $line_num$line) = each ($text)) {
             echo 
    $line"\n";
          }
       } else {
          echo 
    $text;
       }


  10. #10
    SitePoint Enthusiast Jack@dwd.com's Avatar
    Join Date
    Dec 2000
    Location
    UK
    Posts
    60
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    In my opinion, this is a bug, and maybe someone should submit it to www.php.net

    Could Shawn possibly try and replicate this problem in controlled conditions (after all, this might be a specific problem relating to his setup)? The usual way of doing this is to take out all the extraneous code like your template system so it just boils down to making a very large string and then echoing it, and then comparing that time with the explode() method of echo-ing.

  11. #11
    SitePoint Enthusiast
    Join Date
    Jan 2002
    Posts
    70
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I've been doing many different tests to determine where the problem lies.

    I have come to the conclusion that my script is not at fault. Special thanks to Matt, the developer of Invision Board, I have narrowed the problem down to the echo/print command by doing a series of tests and echoing a specific number of characters. Matt developed the code, I simply tested it. Here are the results...

    Code:
    Display Length: 14201 characters.
    Display Output: 0.00199 seconds.
    
    Display Length: 14251 characters.
    Display Output: 0.00159 seconds.
    
    Display Length: 14301 characters.
    Display Output: 0.12694 seconds.
    
    Display Length: 14401 characters.
    Display Output: 0.13424 seconds.
    It took some narrowing down, but the problem lies between the 14250-14300 character mark, as can be seen in the generation times. These times are SIMPLY to ECHO the data, nothing else. Any suggestions?

    Regards,
    Shawn

  12. #12
    SitePoint Evangelist cyngon's Avatar
    Join Date
    Aug 2001
    Location
    Livonia, MI, USA
    Posts
    513
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Could you post the code you used to test it? I would like to run some benchmarks on my machine and I'll post the results.

  13. #13
    SitePoint Enthusiast
    Join Date
    Jan 2002
    Posts
    70
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Here is the code...

    Simply replace the 20000 with any value you want to test...

    PHP Code:
    <?php
    for( $i 0$i <= 20000$i++)
    {
       
    $example_output .= '1';
    }

    echo 
    '<font color="FFFFFF">';

    $microtime explode(' 'microtime());
    $startTime $microtime[0] + $microtime[1];
    echo 
    $example_output;
    $microtime explode(' 'microtime());
    $endTime $microtime[0] + $microtime[1];
    echo 
    '</font><center><font face="Verdana, Arial, Helvetica, sans-serif" size="1" color="000000">Display Length: 'strlen($example_output), ' characters.</font></center>';
    echo 
    '<center><font face="Verdana, Arial, Helvetica, sans-serif" size="1" color="000000">Display Output: 'round($endTime $startTime5), ' seconds.</font></center><br>';
    ?>
    Regards,
    Shawn

  14. #14
    SitePoint Evangelist cyngon's Avatar
    Join Date
    Aug 2001
    Location
    Livonia, MI, USA
    Posts
    513
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Here are some results from my local install of PHP and Apache:

    Code:
    Display Length: 14201 characters.
    Display Output: 0.00118 seconds.
    
    Display Length: 14251 characters.
    Display Output: 0.00346 seconds.
    
    Display Length: 14301 characters.
    Display Output: 0.00291 seconds.
    
    Display Length: 14401 characters.
    Display Output: 0.00343 seconds.
    
    Display Length: 50001 characters.
    Display Output: 0.01155 seconds.
    
    Display Length: 100001 characters.
    Display Output: 0.01794 seconds.
    Interesting, my box (Dell Inspiron 2500 Laptop, Windows XP Pro) can echo() 100,000 characters in less than 1/10th of the time you can echo 14,300.

    What version of PHP are you running? I'm running 4.2.1 on Apache 1.3.

  15. #15
    SitePoint Enthusiast
    Join Date
    Jan 2002
    Posts
    70
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Here are the specs of the server that I am testing this script on.

    PHP Specs:
    http://dragon.nocdns.com/phpinfo/

    System Information
    http://dragon.nocdns.com

  16. #16
    SitePoint Evangelist cyngon's Avatar
    Join Date
    Aug 2001
    Location
    Livonia, MI, USA
    Posts
    513
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    You've deffinatly got quite a bit more power under the hood than I do with this laptop.

    Perhaps its time to file a bug report?

  17. #17
    SitePoint Enthusiast
    Join Date
    Jan 2002
    Posts
    70
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    It is a shared server from a hosting company, but I will look into filing that bug report this evening.

    Thanks for the support, its greatly appreciated.

    Regards,
    Shawn

  18. #18
    SitePoint Evangelist cyngon's Avatar
    Join Date
    Aug 2001
    Location
    Livonia, MI, USA
    Posts
    513
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Let me know how that goes. I'm not sure what system PHP uses for bug tracking, but if its something web-based, please post a link.

    Thanks.

  19. #19
    Mlle. Ledoyen silver trophy seanf's Avatar
    Join Date
    Jan 2001
    Location
    UK
    Posts
    7,168
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Copying your numbers cyngon (I just ran each one time):

    Display Length: 14201 characters.
    Display Output: 0.00476 seconds.

    Display Length: 14301 characters.
    Display Output: 0.00054 seconds.

    Display Length: 14401 characters.
    Display Output: 0.00496 seconds.

    Display Length: 50001 characters.
    Display Output: 0.00431 seconds.

    Display Length: 100001 characters.
    Display Output: 0.00632 seconds.

    Aletia Shawn!

    Sean
    Harry Potter

    -- You lived inside my world so softly
    -- Protected only by the kindness of your nature

  20. #20
    SitePoint Enthusiast
    Join Date
    Jan 2002
    Posts
    70
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Yup, Aletia Shawn! I'm submitting a support ticket now, it is a lengthy one

    -Shawn

  21. #21
    SitePoint Enthusiast
    Join Date
    Jan 2002
    Posts
    70
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Hey Guys, I just got a response back from Ron Kuris (one of the smartest people at the company), this was his response to the problem. If anyone can explain this to me and maybe offer a few suggestions, it would be appreciated.

    This is pretty common because of the way networking (in particular, the Nagle Algorithm and windowing) works. Packets can only be so big, and when you exceed the packet size being sent to the browser, it has to start another packet. When you exceed the window size, TCP/IP waits for acknowledgement.

    I can't see how any PHP configuration setting will change this. The cutoff point varies depending on the MSS, window value, and other TCP/IP parameters, which we should not change. However, if you have a suggestion as to what you would like to see changed, I would be happy to look into that for you.
    Thanks,
    Shawn

  22. #22
    SitePoint Evangelist cyngon's Avatar
    Join Date
    Aug 2001
    Location
    Livonia, MI, USA
    Posts
    513
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Sounds like there may be nothing you can do about it.

    Did you mention in your bug report that it goes much faster if you break it apart and echo it line by line?

  23. #23
    SitePoint Enthusiast
    Join Date
    Jan 2002
    Posts
    70
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Originally posted by cyngon
    Sounds like there may be nothing you can do about it.

    Did you mention in your bug report that it goes much faster if you break it apart and echo it line by line?
    Yes, I mentioned that in the bug report

  24. #24
    SitePoint Enthusiast
    Join Date
    Jan 2002
    Posts
    70
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    More information from Ron...

    This is our buddy Nagle. You might want to do a web search about the Nagle algorithm and how it affects TCP/IP transmissions of small packets done separately versus large packets. The explanation is pretty complex so I can't go into it here.

  25. #25
    SitePoint Guru okrogius's Avatar
    Join Date
    Mar 2002
    Location
    US
    Posts
    622
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Make sure you use single quotes when possible instead of double quotes for some optimization.


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •