SitePoint Sponsor

User Tag List

Page 1 of 5 12345 LastLast
Results 1 to 25 of 109
  1. #1
    ********* Victim lastcraft's Avatar
    Join Date
    Apr 2003
    Location
    London
    Posts
    2,423
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)

    Testing tool call for features

    Hi.

    I am trying to put together a testing tool that includes some features of the Java tools HTTPUnit and JWebUnit. I would like people's views on which features they would like to see and also the sort of interface they would expect.

    Features planned so far are...

    1. Header checks on reponses. Making sure certain headers are set, etc. and that a page is received at all.
    2. Link following, that is parsing the HTML response and being able to access the links by label. Useful for building site checks for valid links, forms and images.
    3. Form handling, including automatic submission of default values, cookie handling and post data possibly including mixed mime types.


    JavaScript support is not on the feature list for a version one. Frames, XML-RPC and SOAP testing could be added, but only if there was a very strong demand.

    There are two basic interface options: a browser view and a test case view. In the browser view you would do this...

    PHP Code:
    class MyPageTest extends UnitTestCase {
        function 
    MyPageTest("My page") {
            
    $this->UnitTestCase();
        }
        function 
    testPageExists() {
            
    $browser = &new TestBrowser($this);
            
    $browser->expectConnection("http://mypage.com");
            
    $browser->setExpectedResponseCodes(array(200));
            
    $browser->fetch();
        }

    In a test case view you do this...

    PHP Code:
    class MyPageTest extends WebTestCase {
        function 
    MyPageTest("My page") {
            
    $this->WebTestCase();
        }
        function 
    testPageExists() {
            
    $this->assertPageExists("http://mysite.com");
        }

    This is obviously simpler, but a lot less flexible, especially when dealing with multiple requests (e.g. images in a page). I have to do a browser view anyway in order to implement the thing, but also so that other unit testers can use these components. How much of the other functionality would people like to see implemented in the test case view?

    Feel free to post psuedo code examples of what you would like to be able to type. This is a wish list.

    yours, Marcus.

    p.s. http://sourceforge.net/projects/simpletest is the project, but only the CVS version has the HTTP functionality so far and the Alpha3 release doesn't quite match the tutorial :-(.

    Alpha4 is coming soon (I am writing some documentation for it).
    Marcus Baker
    Testing: SimpleTest, Cgreen, Fakemail
    Other: Phemto dependency injector
    Books: PHP in Action, 97 things

  2. #2
    SitePoint Zealot
    Join Date
    Aug 2002
    Posts
    178
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Just downloaded it and had a quick look and it doesn't look bad at all
    Your plans #2 & #3 above looks good for a first start if you can get it working. Infact: I was just looking into jWebUnit the other day and though: hmmm, should I start on this for PHP?? And here it pops up!
    I'm working on a large project for the moment, I might lend you a hand on this one in a bit!

  3. #3
    SitePoint Zealot
    Join Date
    Aug 2002
    Posts
    178
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    And while I'm still awake: how about implementing a renderer interface from day one so that we can easily change the look and feel for the output for those nice reports to the manager

  4. #4
    ********* Victim lastcraft's Avatar
    Join Date
    Apr 2003
    Location
    London
    Posts
    2,423
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)

    Unhappy

    Quote Originally Posted by nucleuz
    ... how about implementing a renderer interface from day one so that we can easily change the look and feel for the output for those nice reports to the manager
    This kind of highlights why I need to get some documentation in there! :-)

    The display is, well, minimal at present. I want to get all of the core stuff in before going to beta level and I haven't started learning JavaScript yet so I couldn't do a pretty display if you pointed a gun at my head.

    If you check out the tutorial that goes with it (eight pages, sorry) I mention how you can subclass the page display for your own purposes. You can subclass lower down (see the UML in CVS) for non-page based report generators too.

    Guess that's the best I can do at present, but tell me the sort of stuff you would want in the report so at least I can put the base class methods in that you need for the next release. I do plan a JUnit type display for teh final version 1.0 whenever that arrives. :-(

    yours, Marcus.

  5. #5
    ********* Victim lastcraft's Avatar
    Join Date
    Apr 2003
    Location
    London
    Posts
    2,423
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    Hi...

    Quote Originally Posted by nucleuz
    ..., I might lend you a hand on this one in a bit!
    If you are still interested please send me a mail.

    yours, Marcus.
    Marcus Baker
    Testing: SimpleTest, Cgreen, Fakemail
    Other: Phemto dependency injector
    Books: PHP in Action, 97 things

  6. #6
    SitePoint Wizard gold trophysilver trophy
    Join Date
    Nov 2000
    Location
    Switzerland
    Posts
    2,479
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Got a few features HTTP wise that I've finally thought of.

    1. Checks on HTTP 1.0 vs HTTP 1.1. I.e. if a client reports it only wants to use HTTP 1.0 then it should get back any HTTP 1.1 responses. This may be tricky though for your code - implementing an HTTP 1.1 client is PHP is hard work as the spec says you must be be capable of chunked encoding.

    2. Browser HTTP "cloning": this is probably a tall order but there are some issues with what some browsers do with HTTP headers and the like those described with Netscape (probably 4.x) here: http://www.zend.com/codex.php?id=435&single=1 - there's also a wierd one in how IE handles inline files which results in two HTTP requests - kind of described it here. Overall if there's a way to make this browser "cloning" easily extensible somehow then people could contribute.

    3. Perhaps an md5 hash of the content would be good although may be that's something a test script should implement; either way access to the body of a response would be handy.

    4. Perhaps HTTP caching support should be possible.

    Probably this is worth having if you're really determined - either that or slap it on an Amazon wishlist and release the core of SimpleTest without the web testing functionality via PEAR.

  7. #7
    ********* Victim lastcraft's Avatar
    Join Date
    Apr 2003
    Location
    London
    Posts
    2,423
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)

    Unhappy

    Hi Harry...

    Quote Originally Posted by HarryF
    1. Checks on HTTP 1.0 vs HTTP 1.1. I.e. if a client reports it only wants to use HTTP 1.0 then it should get back any HTTP 1.1 responses. This may be tricky though for your code - implementing an HTTP 1.1 client is PHP is hard work as the spec says you must be be capable of chunked encoding.
    The objective is not really to test the innards of HTTP beyond what can be set from within PHP. I am really out to test the web pages rather than the means of transport. In Java terms I really want to test the Servlets people write rather than the Servlet engine. Are there any issues from this perspective regarding HTTP 1.1 as a requirement? The browser class can already check the HTTP level, or can it? I really cannot remember, but I'll put it in now.

    (Please reach over the nappies to type a reply).

    My knowledge of HTTP was next to nothing until I started this project and although it has improved beyond nothing I don't think it is at a level I can implement all these requests yet. ;-(

    In particular...

    Quote Originally Posted by HarryF
    2. Browser HTTP "cloning": this is probably a tall order but there are some issues with what some browsers do with HTTP headers and the like those described with Netscape (probably 4.x) here: http://www.zend.com/codex.php?id=435&single=1 - there's also a wierd one in how IE handles inline files which results in two HTTP requests - kind of described it here. Overall if there's a way to make this browser "cloning" easily extensible somehow then people could contribute.
    Ok, you really are a task master. I had never even heard of this until you mentioned it. I'll pick through the HTTPUnit code and the like and if they implement it I'll investigate further. This one has a fat chance of happening though.

    Quote Originally Posted by HarryF
    3. Perhaps an md5 hash of the content would be good although may be that's something a test script should implement; either way access to the body of a response would be handy.
    That can be added. The content is really the whole point as it is a web page tester primarily. I am writing a crude HTML parser to extract form elements, images and links as well as to test for text. The content can be got at via TestBrowser::getContent(), but I don't recommend using it in any test cases yet as the interface on this side of things is very much "thrashing".

    Quote Originally Posted by HarryF
    4. Perhaps HTTP caching support should be possible.
    I'll add it to the wishlist for now. How does it work? :-)

    Quote Originally Posted by HarryF
    Probably this is worth having if you're really determined
    Thanks for that. I'll chase that up.

    Quote Originally Posted by HarryF
    - either that or slap it on an Amazon wishlist and release the core of SimpleTest without the web testing functionality via PEAR.
    If there is one thing I don't need, it's people buying me more books! People volunterring to come round and put up shelves is another matter. :-)

    Someone is about to start porting the mocks to PEAR. I am carrying on with SimpleTest as development is much more flexible for me that way.
    The package system of PEAR carries a lot of responsibility and I doubt it needs yet another Alpha package right now. Wolfram (of tree fame) was willing to offer some help as well on that side. For myself I am happy to beaver away in my little hovel churning out code as best I can...

    Besides, it clashes with Sebastian's module as is.

    yours, Marcus.

    p.s. Are you living in Switzerland now? I still owe you an article.

  8. #8
    SitePoint Wizard gold trophysilver trophy
    Join Date
    Nov 2000
    Location
    Switzerland
    Posts
    2,479
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    OK - starting to see where you're headed - guess many of my suggestions are something which really should be handled by a proper HTTP test tool. Perhaps all it needs is to provide access to the headers sent/received for running tests.

    The HTTP server to client Caching I'll have to get back to you on once I've got firm answers; something I've been working on recently hence the ideas here.

    That can be added. The content is really the whole point as it is a web page tester primarily. I am writing a crude HTML parser to extract form elements, images and links as well as to test for text. The content can be got at via TestBrowser::getContent(), but I don't recommend using it in any test cases yet as the interface on this side of things is very much "thrashing".
    Now there's where I can help PEAR::XML_HTMLSax: http://pear.php.net/package-info.php?pacid=203 - a Sax parser for HTML (i.e. badly formed XML). Using it should be alot like the native XML extension. When porting this to PEAR I actually used SimpleTest for the test scripts but right now the PEAR package system isn't quite ready for making the tests avaialble in releases.

    Glad you mock building code is making it into PEAR.

    p.s. Are you living in Switzerland now?
    Yeah for a couple of years now - from the UK originally.

    I still owe you an article.
    Thanks - whenever you like.

  9. #9
    ********* Victim lastcraft's Avatar
    Join Date
    Apr 2003
    Location
    London
    Posts
    2,423
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by HarryF
    Perhaps all it needs is to provide access to the headers sent/received for running tests.
    Which is pretty much what I have got. Cookie handling is important, for testing user management and sensible form handling (carrying defaults) cuts down the amount of explicit test code. I really haven't nailed it all down yet and won't until I can try it out.

    The HTTP server to client Caching I'll have to get back to you on once I've got firm answers; something I've been working on recently hence the ideas here.
    Cool. I brought the O'Reilly HTTP book and should have some free time coming up, so I'll finish this bit pretty quick I hope. I think I have got cookie handling correct at last.

    Now there's where I can help
    Darn, you got me! Is the character by character approach fast enough? I have found the Lexer approach (big regex) can manage a hundred pages a second (probably more) on a one GHz box. It would be interesting to compare them side by side, simpler shorter code versus using the regex engine.

    The SAX approach is a lot more flexible for SAX streams as you say. Hmmmm, I'll crack on to the first beta and then reassess as I fear being deflected by a redesign.

    When porting this to PEAR I actually used SimpleTest for the test scripts but right now the PEAR package system isn't quite ready for making the tests avaialble in releases.
    That is a bummer. This is probably not the thread to start griping about PEAR irritations and it would rehash a lot of old stuff I suspect, but not being able to include a self test is not going to do it's reputation much good. What were the difficulties?

    Glad you mock building code is making it into PEAR.
    You can use the mocks with PEAR already, but they lack methods like expectPearError() and the like. I am really waiting for PHP5 so I can put proper exception support in and generate the mocks from interfaces, all of which makes the PEAR base class pretty much redundant it seems.

    yours, Marcus.

  10. #10
    SitePoint Wizard gold trophysilver trophy
    Join Date
    Nov 2000
    Location
    Switzerland
    Posts
    2,479
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Darn, you got me! Is the character by character approach fast enough? I have found the Lexer approach (big regex) can manage a hundred pages a second (probably more) on a one GHz box. It would be interesting to compare them side by side, simpler shorter code versus using the regex engine.
    Guess that depends. I haven't done anything large with HTMLSax or really examined performance but for a single web page, if you keep the parsing down to a single run, should be OK. You may want to get in touch with Alan Knowles who's working on a PEAR package HTML Template Flexy using the PHP tokenizer extension. A by product of this should be a an HTML lexer (inspired by GNU Flex) - may be he can at least give you a rough idea of the API.

    That is a bummer. This is probably not the thread to start griping about PEAR irritations and it would rehash a lot of old stuff I suspect, but not being able to include a self test is not going to do it's reputation much good. What were the difficulties?
    Don't worry too much. I could have supplied the tests as part of the documentation - there's nothing stopping me but I'd read they're about to make an official procedure for publishing tests so thought I'd wait till their ready.

    all of which makes the PEAR base class pretty much redundant it seems.
    Let's hope so

  11. #11
    ********* Victim lastcraft's Avatar
    Join Date
    Apr 2003
    Location
    London
    Posts
    2,423
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    Hi...

    Finished.

    yours, Marcus
    Marcus Baker
    Testing: SimpleTest, Cgreen, Fakemail
    Other: Phemto dependency injector
    Books: PHP in Action, 97 things

  12. #12
    SitePoint Addict
    Join Date
    May 2003
    Location
    Calgary, Alberta, Canada
    Posts
    275
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by lastcraft
    Hi...

    Finished.

    yours, Marcus
    Congratulations on RC1 Marcus. I really appreciate all the hard work youve put into simpletest.

  13. #13
    SitePoint Addict been's Avatar
    Join Date
    May 2002
    Location
    Gent, Belgium
    Posts
    284
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by vBulletin Message
    You must spread some Reputation around before giving it to lastcraft again.
    Just downloaded RC1, swapped it for the beta6 that was on my dev server, reran all my tests (no webtests yet); no problem at all (even the green is the same green as before, amazing!).
    I hadn't expect anything less to tell you the thruth
    Thank you for a great and useful tool
    Per
    Everything
    works on a PowerPoint slide

  14. #14
    ********* Victim lastcraft's Avatar
    Join Date
    Apr 2003
    Location
    London
    Posts
    2,423
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    Hi.

    It's now stable (version 1.0) and so I can start to look at creating new versions. This means I am again on the look out for new features and improvements. The more mad the better .

    yours, Marcus
    Marcus Baker
    Testing: SimpleTest, Cgreen, Fakemail
    Other: Phemto dependency injector
    Books: PHP in Action, 97 things

  15. #15
    SitePoint Guru
    Join Date
    Nov 2002
    Posts
    841
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Congratulations. Downloading now.

    How about
    • XPath based assertions
    • Code Coverage reporting
    • Reporting the number of test methods as well as classes and assertions

  16. #16
    SitePoint Addict pachanga's Avatar
    Join Date
    Mar 2004
    Location
    Russia, Penza
    Posts
    265
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    SimpleTest rocks

    The only thing we're really missing is support for testing sequence of mock object methods calls....

  17. #17
    Non-Member
    Join Date
    Jan 2003
    Posts
    5,748
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    This looks cool Lastcraft,

    The display is, well, minimal at present.
    Don't worry about this, I use the TextReporter and I like the ouput, the simpler the better (for some of us at least )

  18. #18
    ********* Victim lastcraft's Avatar
    Join Date
    Apr 2003
    Location
    London
    Posts
    2,423
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    Hi...

    Quote Originally Posted by Selkirk
    XPath based assertions
    This is limited by the technology as PHP parsing is just too slow. I have pegged it as version 2 with HtmlTidy support, but it could be shoehorned in earlier for XHTML pages.

    Quote Originally Posted by Selkirk
    Code Coverage reporting
    You are the first person to suggest this . How useful is it?

    Quote Originally Posted by Selkirk
    Reporting the number of test methods as well as classes and assertions
    This is scheduled to change as soon as possible. Reporting the number of assertions was a brain dead quick fix. I am going to remove the reporting of pass messages in favour of a paintPass() call for the whole method. This means the stupid thing can stop doing a humungous load of string manipulations for messages it never shows .

    yours, Marcus
    Marcus Baker
    Testing: SimpleTest, Cgreen, Fakemail
    Other: Phemto dependency injector
    Books: PHP in Action, 97 things

  19. #19
    SitePoint Zealot
    Join Date
    Mar 2004
    Location
    Australia
    Posts
    101
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I think code coverage can be useful, it can at least tell you which part of your code was not tested. Full coverage just means that there are tests for the covered classes, doesn't mean that the tests are complete or extensive.

    The result of a code coverage that i have used before, xdebug required.

    http://xlab6.com/examples/PetShop/tests/index.php

    Wei.

  20. #20
    SitePoint Guru
    Join Date
    Nov 2002
    Posts
    841
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by lastcraft
    You are the first person to suggest this . How useful is it?
    I've found it useful in the past with Delphi programming.

    I was recently testing a moderately complicated set of methods and using die() statements to determine which of the code paths my test cases covered. I put a die() statement after every statement. When I had a test case that triggered a die(), I would remove the statement. I knew I was done testing when there were no more die() statements. This would be easier with coverage capability. Especially if someone else makes a change and doesn't repeat my process.

  21. #21
    SitePoint Guru
    Join Date
    Nov 2002
    Posts
    841
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Wouldn't it also be possible to determine coverage, although very slowly, without xdebug using declare(ticks=1) and debug_backtrace()?

  22. #22
    ********* Victim lastcraft's Avatar
    Join Date
    Apr 2003
    Location
    London
    Posts
    2,423
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    Hi...

    Quote Originally Posted by wei
    I think code coverage can be useful, it can at least tell you which part of your code was not tested.
    Yes, but why? I tend to find I deliberately leave parts untested in unit tests, either because they are trivial or because they are too close to resources and have to be tested as boundary/gateway tests or at the integration stage. Those parts should be covered rarely need just one test (although if there were more than about three I would refactor). I don't personally find it a useful metric.

    It sounds like a fear of imported code. Is this correct?

    I am not arguing here, but am genuinely fascinated. I haven't felt a need for it because it just tells me things that to me seem irrelevant, or it's stuff I already know in my heart of hearts. If code is not properly tested it feels not properly tested. Everyone knows the dodgy bits...don't they?

    Quote Originally Posted by wei
    The result of a code coverage that i have used before, xdebug required.
    You star! Could you post the code? I know PEAR::PHPUnit does code coverage with XDebug. Does it take a long time to run?

    yours, Marcus
    Marcus Baker
    Testing: SimpleTest, Cgreen, Fakemail
    Other: Phemto dependency injector
    Books: PHP in Action, 97 things

  23. #23
    eschew sesquipedalians silver trophy sweatje's Avatar
    Join Date
    Jun 2003
    Location
    Iowa, USA
    Posts
    3,749
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I attened Derik's session on xdebug at php|works in Toronto this fall. I saw him demo how to use xdebug for test coverage, so it is probably in the slides for that talk:

    Derik's site: http://www.derickrethans.nl/
    Slides (re-used at php|works, I believe): http://www.derickrethans.nl/pres-xde...2004/talk.html

    HTH


    P.S. Congratulations on 1.0 Marcus!!!
    Jason Sweat ZCE - jsweat_php@yahoo.com
    Book: PHP Patterns
    Good Stuff: SimpleTest PHPUnit FireFox ADOdb YUI
    Detestable (adjective): software that isn't testable.

  24. #24
    SitePoint Zealot
    Join Date
    Mar 2004
    Location
    Australia
    Posts
    101
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    "declare", never seem that function before, it could work, but it might be slow, have to test it out.

    Why code coverage might be useful? Sometime by laziness or foreign code, tests can not be written before the functional code. It would also help projects that do not have unit tests in place and wish to include tests in later stages. The code coverage would give a "hint" of test completeness.

    The code coverage for simpletest is a quick hack, thus looks pretty nasty. I have not conducted any performance tests, but it seems to be OK, so far only used it for < 20 test cases, with < 200 passes.

    SF CVS code

    Usage




    This is the doc for xdebug's code coverage

    http://xdebug.org/docs-functions.php#coverage

    However, there are undocumented information regarding the betas in regards to code coverage somewhere in some mailing list (can't find it anymore).

    Cheers, Wei.

  25. #25
    SitePoint Zealot ceefour's Avatar
    Join Date
    Feb 2005
    Location
    Bandung, Indonesia
    Posts
    138
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    BTW what is the main features of SimpleTest compared to PHPUnit[2] ?
    Why not make it an extension to PHPUnit?

    I've never used any. Too busy coding, who the hell would have time to develop tests?


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •