simpletest and website testing

[old_iron]

Hi guys,

Got a question regarding simpletest (by lastcraft). I'm loving unit testing at the moment, it's really making my life easier when testing/debugging things

Anyway, so far I have only tested individual objects and the interaction of objects. Since ultimately all these objects are going to work together to interpret a user request, do some processing maybe and then output some HTML - can we unit test this whole procedure?

obviously individual model/view/controller/etc classes can be tested by themselves or interacting with each other, but say if you start at index.php how did you check that, say, your application has produced a table of results? Just search for that pattern in the resultant output with assertPattern? What if we wanted to check a large, complex page with many elements that might be generated by PHP?

[Cups]

SimpleTest contains a webtester, else look at Selenium. This allows you to test, and record and playback tests from the outside in (from the GUI), unlike unit tests which test from the inside out, if I have understood it correctly.

Anyhow, its equally revolutionary if you have never come across it.

[old_iron]

Thanks for the Selenium link, I'll have a look at it.

Are there any web tutorials for unit testing website output? The simpletest site isn't bad but I'd like some more examples.

Like, if I submitted a form which was validated, updated a database and then wanted to access a page reflecting the database change by the form submission, how could I test each stage of this? If I submit a form then I get forwarded to the thank you page or whatever, I can't test the internal processes.

[auricle]

If I submit a form then I get forwarded to the thank you page or whatever, I can't test the internal processes.

What about querying the database directly to see if the change you expected occurred?

In any case, you might gain some confidence from the forwarded page. If you've designed the app to forward to a specific page only on success, and you have elsewhere tested that it happens only on success, then surely the forwarding to that page is indicative of success?

[old_iron]

What about querying the database directly to see if the change you expected occurred?

Yeah, I was gonna do that AND test for the forwarding.. seems like a lot of work in the test but it's worth it I hope!

[McGruff]

can we unit test this whole procedure?

Yes but you should only manipulate what you can reach out and touch. Don't try to "look under the bonnet". For example if I order a pizza, I just need to assertPizzaIsDelivered(). I don't need to know if it arrived by bicycle or by van or if the caterer's dough database is down by another pizza base.

OOP is all about encapsulating chunks of behaviour behind an interface and tests should respect this. It's very harmful if implementation details leak through since that makes code harder to refactor. Changes to low level classes will have a knock-on effect in higher level tests.

In each test you have a very specific point of view. In a unit test, this is the POV of clients of the tested class. These clients can only operate the public interface and so that's all you are allowed to do in the tests. When you turn on a tap to fill a cup you don't think how each drop has travelled halfway round the world to get there. You just want a cup of coffee. Tests always make the same kind of distinction between (a) a requirement of some kind expressed in the language of the domain at the point of testing and (b) the implementation details.

A web test takes the point of view of a web browser. Here, the only commands you're allowed are those which are available in a browser ie clicking links and submitting forms. Don't even *think* about databases at this level. Your site visitors certainly won't. Just describe the behaviour which you expect to observe when you interact with the app via the browser. These are acceptance tests ie top-level UI behaviours which an end-user could write down without any programming knowledge whatsoever.

What would be the requirement expressed in language that an end-user would understand?

[old_iron]

What would be the requirement expressed in language that an end-user would understand?

Well, the requirement would be that on submitting a form to the website to add a new topic to a forum, they are forwarded to a page which shows the new topic on screen, or if the submission was invalid, they'd be forwarded to an error page.

So you're saying I should only do something like:

<?php

$this->post('topic.php', $invalidParams);
$this->assertText('The following errors occurred:'); //part of InvalidSubmissionView

$this->post('topic.php', $validParams);
$this->assertText('Viewing topic :: '.$validParams['title']); //part of ShowTopicView

?>

Is that right?

[McGruff]

Yes. Something like:

class WhenPostingForumMessages extends WebTestCase {

    function testCanPostAMessage() {

        // fetch and assert the "submit post" page 
        $this->get(...);
        $this->assertResponse('200');
        $this->assertTitle(...);
        $this->assertField(...);
        $this->assertField(...);
        ...
        ...

        // enter data and submit the form
        $this->setField(...);
        $this->setField(...);
        $this->click(...);

        // assert the post is added to the forum
        $this->assertResponse('200');
        $this->assertTitle(...);
        $this->assertPattern(...);
        ...
        ...
    }
} 

In order to check a page has been served up I'd usually assert a response code, the page title, charset server header & html meta tag content type, and one or two key bits of body text - the posted message in this case. You don't need to assert every last piece of visible text: just whatever is significant for the current test.

http://www.lastcraft.com/web_tester_documentation.php
http://www.lastcraft.com/form_testing_documentation.php

[McGruff]

Excuse me there are actually two tests there:

class WhenPostingForumMessages extends WebTestCase {

    function testCanOpenSubmitPostPage() {
        $this->get(...);
        $this->assertResponse('200');
        $this->assertTitle(...);
        $this->assertField(...);
        $this->assertField(...);
        ...
        ...
    }
    function testCanSubmitPost() {

        // enter data and submit the form
        $this->get(...);
        $this->setField(...);
        $this->setField(...);
        $this->click(...);

        // assert the post is added to the forum
        $this->assertResponse('200');
        $this->assertTitle(...);
        $this->assertPattern(...);
        ...
        ...
    }
} 

You could save a page load by merging them together but I think it's best to refactor tests so that each makes a specific point.

[lastcraft]

Hi...

Like, if I submitted a form which was validated, updated a database and then wanted to access a page reflecting the database change by the form submission, how could I test each stage of this?

Individually, with a unit test...

class SignUpControllerTest extends UnitTestCase {
    function setUp() {
        $this->accounts()->deleteAllUsers();
    }

    function testSuccessfulSignUpRedirectsToWelcome() {
        $continuation = new MockContinuation();
        $continuation->expectOnce('redirect', array('welcome.html'));
        $request = new Request(array('u' => 'marcus', 'p' => 'secret'));
        new SignUp($request, $continuation);
    }

    function accounts() {
        return new AccountsFixture();
    }
} 

If I submit a form then I get forwarded to the thank you page or whatever, I can't test the internal processes.

class TestOfSignUp extends WebTestCase {
    function setUp() {
        $this->accounts()->deleteAllUsers();
    }

    function testSuccessfulSignUpGoesToWelcomePage() {
        $this->get($this->home());
        $this->click('sign up');
        $this->setField('Username:', 'marcus');
        $this->setField('Password:', 'secret');
        $this->click('Submit');
        $this->assertText('Welcome marcus');
    }

    function accounts() {
        return new AccountsFixture();
    }
} 

You want to limit your web tests to acceptance criteria and sections of the site that are difficult to unit test. The web tests are slow once you get a lot of them (about 30 times slower with Selenium).

When programming I tend to test all the detail in the unit tests (is this the bit you don't trust?) and just confirm the wiring with the web tests.

Before programming I try to capture the acceptance criteria pretty much as the stakeholders say it. They can then see the test being created in front of them.

I suspect you'll get quite a few different answers for this one. Your testing style will depend on your other development practices.

yours, Marcus

[dagfinn]

When programming I tend to test all the detail in the unit tests (is this the bit you don't trust?) and just confirm the wiring with the web tests.

It's also very slow to have too many separate test methods. So instead, to make it readable, I tend to use custom assertions.

[lastcraft]

Hi...

It's also very slow to have too many separate test methods. So instead, to make it readable, I tend to use custom assertions.

Do you have any examples?

yours, Marcus

[dagfinn]

Hi...



Do you have any examples?

yours, Marcus

Finally got around to answering this. I don't think I can use any actual examples. Let me just illustrate the principle with an example that may be a something of a caricature.

If performance and speed were not an issue, I might do something like this, keeping each test method short and sweet:

class ArticleListTest extends WebTestCase {

    function setUp() {
        // Some code to log in and go to the article list page
        //....
    }

    function testHttpStatusOk() {
        //...
    }

    function testHasCorrectTitle() {
        //...
    }

    function testHasNoDebugInfoAccidentallyLeftBehind() {
        //...
    }
} 

Unfortunately, it would be nauseatingly slow, since you have to traverse some web pages for every single test method. Instead, I can try to get similar small chunking by using custom assertions:

class VariousPageTests extends WebTestCase {

    function testLoginAndArticleList() {
        // Some code to log in
        //....

        $this->assertStartPage();

        // Some code to go to article list page
        //....

        $this->assertArticleListPage();
    }

    function assertArticleListPage() {
        $this->assertHttpStatusOk();
        $this->assertHasCorrectTitle();
        $this->assertHasNoDebugInfoAccidentallyLeftBehind();
    }

    function assertStartPage() {
        //
    }
}