Ruling the Swarm (of Tests) with Codeception

Tweet

What happens if you have more tests then zergs in the swarm? Really much more. You should find a way to control and manage them. In this article I’ll share some hints and best practices you may use writing acceptance tests. I’ll use Codeception testing framework to illustrate the best practices, but surely, they can be ported to any acceptance framework you use for testing.

Into Codeception

Designing and building a great web application means that our workflow and architecture allow for constant improvement to the product. In order to make our products stable and to have confidence that new commits have no adverse-effect on existing code, we use automated testing practices. And just as we need a proper architecture for our application, we need to design a proper architecture for our testing platform.

A testing platform is a complete infrastructure over our tests. And when I say tests, I don’t mean only unit tests. Think wider. The application also needs to be tested in a browser with Selenium (or an alternative). So we need unit tests, Selenium tests, probably API tests, and all of them should be executed together. Somehow. I’d like to introduce you Codeception – a modern PHP testing framework built on top of Symfony2 components, Mink, and PHPUnit.

Codeception includes more than 20 modules to test over popular PHP frameworks, APIs (REST, SOAP), data stores (MySQL, PostgreSQL, MongoDB, etc) and web with Selenium WebDriver. Check it out on the official web site: codeception.com

And what better way to run all of these tests than on an Continuous Integration server after each commit? I recommend using Codeception as the main block for your test automation platform; Codeception can execute Selenium tests, API tests, and PHPUnit tests all in one and generate complete HTML and XML reports for CI systems. It can even generate a complete code coverage report for all of the tests.

It’s pretty easy to start writing tests with Codeception. You can teach your QAs to write such tests since only a very basic knowledge of PHP is required to start. I’ll leave you to discover the basics of Codeception on your own, since what I want to tell you here is how Codeception can help you build a better test automation platform.

Hive of Tests

It’s not enough to say “We are writing tests” or “We practice TDD/BDD”. What you should be aiming for is the creation of a solid test automation platform. It doesn’t really matter if you’re writing your tests before code, or code before tests, or tests described as a story (in BDD-way), or tests as code. What really matters is how you manage all of these tests.

There may be more then 1,000 or 10,000 or 100,000 tests, and they may run for a three or four hours. You need keep the code for your tests clean and simple. Tests should be easy to refactor (as they constantly change with application) and they should be easy to extend. Basically, you should follow the same best practices you use for writing production code when writing testing code.

A Basic Test

Here is a sample acceptance test in Codeception. In this scenario I’m sending an invite to a friend from a website. To test this I need to be logged in, generate an invitation code, and then try to register a new user with the code. Here is how the scenario could look in Codeception:

<?php
$I = new WebGuy($scenario);
$I->wantTo('generate invitation code and sign up using it');
$I->amOnPage('/login');
$I->fillField('Username','tester1');
$I->fillField('Password','123456');
$I->click('Sign In');
$I->see('Hello, tester1');
$I->click('Invite');
$I->see('Copy this invitation code');
$code = $I->grabTextFrom('#invitation_code');
$I->click('Logout');
$I->amOnPage('/register');
$I->see('You can sign up if you have invitation code!', 'h1');
$I->fillField('Invitation Code', $code);
$I->fillField('Username','tester2');
$I->fillField('Password','654321');
$I->click('#invite_form input[type=submit]');
$I->see('Hello, tester2');

Even if you’re not familiar with Codeception, this test should be pretty readable; all actions are described from a user’s perspective and no special technical terms are used here. This test may look simple and clean, but if you have such tests then you should consider refactoring it. And here is why…

Design Pattern: Page Object

You should avoid hard-coded values directly in your tests whenever you can as this will make the tests more stable to changes. Suppose similar login and signup forms are used in other tests. If the username field was renamed to “Login” then you’d need to rewrite the tests that use login. We can use the Page Object pattern and CSS/XPath to locate elements on a page. The same goes for the Invite page.

A Page Object is an entity that represents a page (or several typical pages) of your site, a can be written as a class with static properties and methods.

<?php
class InvitePage
{
    // URL of a page
    public static $URL = '/register';
    // This properties define a UI map for Login Page
    public static $codeField = "Invitation Code";
    public static $usernameField = "Username";
    public static $passwordField = "Password";
    public static $submitButton = "#invite_form input[type=submit]";
}

Now part of the test can be rewritten in this manner:

<?php
$I->amOnPage(InvitePage::$URL);
$I->fillField(InvitePage::$codeField, $code);
$I->fillField(InvitePage::$usernameField,'tester2');
$I->fillField(InvitePage::$passwordField,'654321');
$I->click(InvitePage::$submitButton);

In this case, all tests visiting the Invite page use the same variables for locating elements on the page. If any element changes its position, you can update the InvitePage class as needed without touching the tests themselves.

Design Pattern: Controller

We’ve removed some hardcoded variables from the test, but what happens if you introduce two-factor authentication to the web application you’re developing. What does it mean for your tests? It means that you’ll need to update all of the tests that requires an authorized user to run. I will remind you: there may be thousands and of tests. It would be natural to move some scenario logic into a class so we can reuse the log in/out and signup scenarios in other tests.

I asked some automation engineer friends how they refer to such a class. They used the term Step Object (which is common in BDD), but I’d rather call it a Controller here because of the similarity it has with the controllers we use in web applications. There are no strict rules stating what page objects or step objects should look like; you are free to build them as you wish. Here is an example controller I would use for Codeception test:

<?php
class UserController
{
    protected $user;

    public function __construct(WebGuy $I) {
        $this->user = $I;
    }

    public function login($username, $password) {
        $this->user->amOnPage(LoginPage::$URL);
        $this->user->fillField(LoginPage::$usernameField, $username);
        $this->user->fillField(LoginPage::$passwordField, $password);
        $this->user->click(LoginPage::$submitButton);
        $this->user->see("Hello, $username");
    }

    public function logout() {
        $this->user->click("Logout");
    }

    public function registerWithInviteCode($code, $username, $password) {
        $this->user->amOnPage(InvitePage::$URL);
        $this->user->fillField(InvitePage::$codeField, $code);
        $this->user->fillField(InvitePage::$usernameField, $username);
        $this->user->fillField(InvitePage::$passwordField, $password);
        $this->user->click(InvitePage::$submitButton);
    }
}

Now the code has logical blocks and the scenario will look much cleaner:

<?php
$I = new WebGuy($scenario);
$U = new UserController($I);
$I->wantTo('generate invitation code and sign up using it');
$U->login('tester1', '123456');
$I->click('Invite');
$I->see('Copy this invitation code');
$code = $I->grabTextFrom('#invitation_code');
$U->logout();
$U->registerWithInviteCode($code, 'tester2', '654321');
$I->see('Hello, tester2');

By introducing controllers and page objects, we’ve gotten rid of most hardcoded values and common scenario steps. It can log users in from other tests, and that’s really useful as probably most of your application’s functionality is available only to authorized users. You can write 10 or 100 or even 1,000 tests, but using page objects and controllers they won’t make a mess and will be easy to support.

Report to the Commander

Because of the refactoring, the test code has probably lost some of its readability to non-developers. Luckily, Codeception can help with that, too. Codeception will show you all scenario steps that were executed when running the test:

$ ./codecept.phar run acceptance --steps
Codeception PHP Testing Framework v1.5.5
Powered by PHPUnit 3.7.14 by Sebastian Bergmann.

Suite acceptance started
Trying to generate invitation code and sign up using it (InviteCept.php)
Scenario:
* I am on page '/login'
* I fill field 'Username','tester1'
* I fill field 'Password','123456'
* I click 'Sign In'
* I see 'Hello, tester1'
* I click 'Invite'
* I see 'Copy this invitation code'
* I grab text from '#invitation_code'
* I click 'Logout'
* I am on page '/register'
* I see 'You can sign up if you have invitation code!', 'h1'
* I fill field 'Invitation Code', 'iuSia89ak'
* I fill field 'Username','tester2'
* I fill field 'Password','654321'
* I click '#invite_form input[type=submit]'
* I see 'Hello, tester2'

Time: 10 seconds, Memory: 26.00Mb
OK (1 test, 4 assertions)

In this case the output is readable. This scenario can even be shown to a non-technical person, to a manager, or a business analyst, and they’ll understand what is happening. Codeception allows you to generate HTML reports written in a similar manner. They can be shown to managers or business analysts to demonstrate what and how you you’re your application.

Conclusion

Nowadays test automation engineers prefer Java and Ruby, but as a PHP developer you’re undoubtedly interested in testing platforms that play well with your PHP code and is written in PHP itself. Codeception is a great tool to easily start acceptance testing, and it’s flexible enough to build a solid test automation platform.

But no matter what framework you use, keep in mind that it’s not enough just to write tests. To make tests clean, stable, and maintainable, use the KISS and DRY principles you already know. Design patterns like Page Object and Controller (Step Object) will help you in that.

Image via Fotolia

Free book: Jump Start HTML5 Basics

Grab a free copy of one our latest ebooks! Packed with hints and tips on HTML5's most powerful new features.

  • Khm

    Nice tricks, thanks for sharing

  • kanuj bhatnagar

    An excellent post. Just discovered Codeception and already it’s looking like it could take a lot of pain away from testing.

  • Beedge

    Sorry for such a Noob question, but it is unclear to me from this article. Where exactly would I put these new classes? do I save them in the acceptance folder?

    • http://codeception.com Michael Bodnarchuk

      Sorry, can’t login to my profile to answer.
      @Beedge
      Yes, you can save this classes anywhere you want. Looks like editor has cur the “require” statements in the begining of each file. This files can be required in _bootstrap file.

  • Sreejith M

    If we are using db interactions and function calls from models it will be better to check the scenario too, else the code will be executed _before().

    use
    if ($scenario->running()) {
    $U = new UsertestController($I);
    $I->wantTo(‘Do a Signin’);
    $U->signIn();
    }

  • http://whoami.vinu.io Vinu

    Nice one (y)
    One more addition to this:
    We can pass $scenario to the controller, just in case we need the current state of the scenario in the controller it wil help:)

  • Gav

    Keep getting a ‘cannot redeclare InvitePage’ error – any chance of having a download of the files?

    • Michael Bodnarchuk

      how are you loading the pageobject?
      i think if you use “require_once” in _bootstrap, everything will run ok.

      • Gav

        I’m not by my laptop but when I’m home I’ll try again and let you know how I get on using the _bootstrap. Thanks again

        • Gav

          Still having some problems.. here are my _bootstrap.php & InviteCept.php files
          http://phpfiddle.org/main/code/87c-n7r
          http://phpfiddle.org/main/code/whw-hbm

          • Gav

            The error I get is :-
            Suite acceptance started
            Running InviteCept.php
            Fatal error : Cannot redeclare class InvitePage in _bootstrap.php on line 5

          • Davert

            Oh, oh. You should not place classes in _bootstrap.
            Please create a separate file and “require_once” this file in _bootstrap.
            Bootstrap is loaded multiple times. So you can’t define anything there. But require_once will work.

  • Gav

    Thanks Davert! Such an awesome bit of kit.. puts the fun into testing (never thought i’d say that in my lifetime!)