SitePoint Sponsor

User Tag List

Page 1 of 2 12 LastLast
Results 1 to 25 of 28
  1. #1
    SitePoint Wizard dreamscape's Avatar
    Join Date
    Aug 2005
    Posts
    1,080
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    A Behavior Driven Development Framework Proposal

    I've been thinking some about Behavior Driven Development (BDD) and how it would be nice for PHP to have a simple and intuitive BDD framework. (Currently I don't think it has any, so actually something would be better than nothing)

    I've been sort of looking at both rSpec and JBehave to get some ideas. One thing I like about JBehave are the "Stories" that it uses for acceptance testing. And I like how rSpec sets up fine grained specifications. I was thinking that it would be nice to sort of combine the two, so you can have specifications that can be as finely grained or as broad as you want, which makes it easy to use the same framework for all specification related tests.

    One really nice thing about the Stories in JBehave is that the "tests" are split up into 3 different areas: Contexts, Events, and Outcomes ... which means that you can reuse them as needed without copying and having duplicate code. So I am using this idea too.

    And instead of Pass/Fail as traditional TDD does, to completely break away from the mentality, I was thinking of having 4 basic outcomes of a spec run:

    1. Implemented, means no errors, and no mocks in the world
    2. Implementing, means no errors, but the world still uses mocks
    3. Not Implemented, means has errors or is missing a context, event, or outcome
    4. Skipped, means skipped (e.g spec not relevant to this platform)

    Specs will also be allowed to be nested, and the spec run will output the results in a sort of outline format, which can serve a multitude of purposes.

    A report output might look something like this (example output in plain text for a string utility):
    Code:
        SAMPLE REPORT OUTPUT
    
    String Util Specs                               [... Implementing ...]
        - A string util should count words              [Implementing]
            - should count ascii words                      [Implemented]
            - should count accented words                   [Implemented]
            - should count chinese words                    [Implementing]
                - with spaces                                   [Implementing]
                - with no spaces                                [Implementing]
                                                   
    
    2 / 6 specs Implemented
    4 / 6 specs Implementing
    0 / 6 specs Not Implemented
    0 / 6 specs Skipped
    I've come up with the interface for the system, and would appreciate some feedback if you have any. The specs I used for the example are based on the specs from the rSpec tutorial. In addition to an easy to use interface, I've tried to make it as readable as possible, so I would appreciate any comments/suggestions.

    First setting up the specs in plain english:
    PHP Code:
    class Stack_Specs extends The_Specs {

        public function 
    defineSpecs() {

            
    $this->addSpec(
                
    'A new stack should be empty'$this
                
    -> Given    ('a new stack')
                -> 
    When     ('check if stack is empty')
                -> 
    Then     ('stack should be empty')
            );

            
    $this->addSpec(
                
    'A stack with one item should not be empty'$this
                
    -> Given    ('a stack with one item')
                -> 
    When     ('check if stack is empty')
                -> 
    Then     ('stack should not be empty')
            );

            
    $this->addSpec(
                
    'A stack with one item should return top item when you request top'$this
                
    -> Given    ('a stack with one item')
                -> 
    When     ('request top from stack')
                -> 
    Then     ('result should equal one item')
            );

        }


    The Contexts, Events, and Outcomes are derived from the Givens, Whens, and Thens, i.e, "Given these contexts, when these events, then these outcomes". The plain english specs match up to the Context, Event, and Outcome names.

    Now the 2 Contexts (both are using mocks to show that all specs should start out using mocks). A Context "sets up" The World (each spec is run in its own world):
    PHP Code:
    class A_New_Stack_Context extends A_Context {

        public function 
    setsUp(The_World $world) {
            
    $stack $world->mock('Stack');
            
    $stack->method('isEmpty')->returns(true);

            
    $world->stack $stack;
        }


    PHP Code:
    class A_Stack_With_One_Item_Context extends A_Context {

        public function 
    setsUp(The_World $world) {
            
    $stack $world->mock('Stack');
            
    $stack->method('isEmpty')->returns(false);
            
    $stack->method('top')->returns('one item');

            
    $world->stack $stack;
        }



    Then the Events. An Event "occurs in" The World:
    PHP Code:
    class Check_If_Stack_Is_Empty_Event extends An_Event {

        public function 
    occursIn(The_World $world) {
            
    $world->result $world->stack->isEmpty();
        }


    PHP Code:
    class Request_Top_From_Stack_Event extends An_Event {

        public function 
    occursIn(The_World $world) {
            
    $world->result $world->stack->top();
        }



    And finally the Outcomes. An Outcome "sets expectations in" The World:
    PHP Code:
    class Result_Should_Equal_One_Item_Outcome extends An_Outcome {

        public function 
    setsExpectationsIn(The_World $world) {
            
    $this->value($world->result)->should->equal('one item');
        }


    PHP Code:
    class Stack_Should_Be_Empty_Outcome extends An_Outcome {

        public function 
    setsExpectationsIn(The_World $world) {
            
    $this->value($world->result)->should->be->true;
        }


    PHP Code:
    class Stack_Should_Not_Be_Empty_Outcome extends An_Outcome {

        public function 
    setsExpectationsIn(The_World $world) {
            
    $this->value($world->result)->should->be->false;
        }




    A spec can also have multiple Contexts and multiple expected Outcomes. I'm debating on whether or not multiple Events should be allowed. To setup multiple ones, you'd just use And(). Here is an example:
    PHP Code:
    class My_Specs extends The_Specs {

        protected function 
    defineSpecs() {

            
    $this->addSpec(
                
    'spec description'$this
                
    -> Given    ('one context')
                    -> 
    And  ('another context')
                -> 
    When     ('an event')
                -> 
    Then     ('expect outcome 1')
                    -> 
    And  ('expect outcome 2')
            );

        }



    So there is the basic proposal. Any thoughts, ideas, feedback, criticisms, rants, or raves?
    <.smarter.web.development.>
    PHP Stuff: Plexus | Chocolate (BDD Framework... coming soon)
    Graphite

  2. #2
    simple tester McGruff's Avatar
    Join Date
    Sep 2003
    Location
    Glasgow
    Posts
    1,690
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    PHP Code:
      $this->assertTrue($mcgruff->hasDoublePosted()); 
    Last edited by McGruff; Nov 21, 2006 at 11:08. Reason: double post

  3. #3
    simple tester McGruff's Avatar
    Join Date
    Sep 2003
    Location
    Glasgow
    Posts
    1,690
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by dreamscape
    And instead of Pass/Fail as traditional TDD does, to completely break away from the mentality, I was thinking of having 4 basic outcomes of a spec run:

    1. Implemented, means no errors, and no mocks in the world
    2. Implementing, means no errors, but the world still uses mocks
    3. Not Implemented, means has errors or is missing a context, event, or outcome
    4. Skipped, means skipped (e.g spec not relevant to this platform)
    For an acceptance test, should I distinguish between (1) and (2)? Even if it's passing with mocks it's still not a functional app. I'd monitor progress with the component classes by running the unit tests.

    I know some test frameworks add skip and todo or whatever but personally I think that's a mistake. If it's not a requirement I should dump the test. If it is, I need to express it as a test and I need to know it's failing. I'm not sure why I'd want to skip.

    To my mind a simple boolean result is the best choice. I'd be concerned that other options weakens the idea of "test-first". To the test-infected, the tests are the main focus: code is just something that happens while you're testing. Turning tests on and off seems to be demoting them to second place. The strength of the simple red/green bar is that it forces you to make decisions. That's really what the tests are for, at least initially. They help to define your ideas clearly.

    The whole behaviour driven thing left me kind of cold since there didn't seem to be any new ideas just a new vocabulary. If it's easier to learn that way it could be a good thing. I don't know.

    PS: please have a look through Arbiter to see if our aims are similar. Don't let me put you off if some my views differ. It's Marcus' baby and I'm just there to (allegedly) do some rough spadework.

  4. #4
    SitePoint Wizard dreamscape's Avatar
    Join Date
    Aug 2005
    Posts
    1,080
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by McGruff
    For an acceptance test, should I distinguish between (1) and (2)? ...<<snipp>>... To my mind a simple boolean result is the best choice.
    I personally find a pass/fail to be too limiting in that not enough information is expressed. A test fails, so what? You have no clue why from the test results, and have to dig through the code to find out why. At least with a few different outcome scenarios, you have a general idea of why it "failed" and a better idea of what needs done.

    "Oh, I've still got this one mocked and need to finish implementing it. Better get on that."

    "Oh, I haven't even started on this one. Maybe I don't really need it?"

    "Great, this one is done."

    etc... IMO, the results should give a fairly clear picture of what is going on development wise and where work needs to focus. They should help serve as guides to help prioritizing tasks.


    Quote Originally Posted by McGruff
    I know some test frameworks add skip and todo or whatever but personally I think that's a mistake. If it's not a requirement I should dump the test. If it is, I need to express it as a test and I need to know it's failing. I'm not sure why I'd want to skip.
    Not everyone deploys to the same platforms all the time, and the different platforms sometimes have different requirements. There may be a spec that is only relevant to Windows platforms or BSD platforms or Linux platforms, for example. You still want to have the specs ready to go, but don't want to run them on irrelevant platforms.

    Another example is some developers like to setup certain tests in advance, such as regression bug tests, so that as bug reports come in you just have to add them and they are pre-setup to run. This is if you like to keep the bug tests separate from the other tests, which some do.

    Of course it would be open to abuse, but that is up to a developer to use their own discretion and self discipline to not abuse the use of skip.

    Quote Originally Posted by McGruff
    The whole behaviour driven thing left me kind of cold since there didn't seem to be any new ideas just a new vocabulary.
    I'm not sure how you can say that the proposal is just a change in vocabulary. It is quite clearly different from traditional TDD test cases.

    Quote Originally Posted by McGruff
    PS: please have a look through Arbiter to see if our aims are similar.
    No I don't believe so. There is some overlap in that what I am proposing can also be used as an acceptance testing framework, but it goes far beyond that and can be used for "testing" on all levels from broad down to as finely grained as you want.

    I'm not proposing something to use in conjunction with traditional unit tests, or something that merely changes the vocabulary. I'm proposing something that would replace the traditional unit tests, change the vocabulary, and change the way you write "tests" from the "test case" approach to discrete components of a "World" (contexts, events, and outcomes) that you combined together in different ways to define the specifications. (This idea comes from JBehave, though it uses it only in the broader sense, there is no reason it cannot be used on all levels) And "specifications" do not necessarily mean on the application level. Remember, units (whatever you consider a unit to be) also have specifications.

    Though I haven't thought too much about it, but also specs that cover more finely grained units should be able to also be reused in specs that cover broader [related] units.
    <.smarter.web.development.>
    PHP Stuff: Plexus | Chocolate (BDD Framework... coming soon)
    Graphite

  5. #5
    simple tester McGruff's Avatar
    Join Date
    Sep 2003
    Location
    Glasgow
    Posts
    1,690
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by dreamscape
    Not everyone deploys to the same platforms all the time, and the different platforms sometimes have different requirements.
    Yes I sometimes find myself doing this:
    PHP Code:
     function testFooOnLinux {
          if( !
    $this->isNix()) {
              return;
          }
        ...
      }
      function 
    testFooOnWindows {
          if( !
    $this->isWin()) {
              return;
          }
        ...
      } 
    I guess another reason to bail out of a test could be if some kind resource required for fixture set up isn't available. Skip has been added to the latest SimpleTest release.

    Quote Originally Posted by dreamscape
    I'm not sure how you can say that the proposal is just a change in vocabulary. It is quite clearly different from traditional TDD test cases.
    That was aimed at BDD rather than your ideas and even then was only meant to be a mild criticism (of BDD). If a language change were to make things easier to understand then fair enough.

  6. #6
    SitePoint Zealot johno's Avatar
    Join Date
    Sep 2003
    Location
    Bratislava, Slovakia
    Posts
    184
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Sorry but, it looks like a class explosion to me. I don't find myself using duplicated events/outcomes in my tests. Creating a class for a simple assert? I don't know. Maybe a method, but a class?

    As for BDD. Its just a different vocabulary. If you are doing TDD right, you will see no difference.

    I find myself already naming testcases like WhenUserIsLoggedIn, WhenUserIsNotLoggedIn with setUp() method just like what BDD calls context with setup, right?

    Context = TestCase, test method = specification, assert = should. It's a 1-to-1 bidirectional mapping.

    But as McGruff said, if it helps in understanding TDD right it probably is a good idea.
    Annotations support for PHP5
    TC/OPT™ Group Leader

  7. #7
    SitePoint Wizard dreamscape's Avatar
    Join Date
    Aug 2005
    Posts
    1,080
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by johno
    Context = TestCase, test method = specification, assert = should. It's a 1-to-1 bidirectional mapping.
    What the hell are you smoking? The only thing that has been renamed is assert to should. The rest of the concepts don't have a 1-to-1 mapping with TDD. I'm afraid you've seriously misunderstood if you think that, or maybe still need your coffee for the day.
    <.smarter.web.development.>
    PHP Stuff: Plexus | Chocolate (BDD Framework... coming soon)
    Graphite

  8. #8
    ********* Victim lastcraft's Avatar
    Join Date
    Apr 2003
    Location
    London
    Posts
    2,423
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    Hi...

    I've been following the BDD idea with interest, as you can imagine. I don't think it's a radical upgrade from TDD, not nearly as much as adding mock objects were, but I think it's significant. It's based on experience of how programmers have ended up using TDD, namely as an executable specification.

    I have a couple of problems with it and I think they are highlighted by your outline.

    1) The learning curve is low for TDD. I can see how it works with this...
    PHP Code:
    class MyTestCase extends UnitTestCase {
        function 
    testAnyOldThing() {
            
    $this->assertTrue(false);
        }

    What's the equivalent of "hello world" in BDD?

    2) It's difficult to see the whole test. It's a bit like using an Action based framework, with Actions, Views and Filters. The code thread passes through a large number of objects which can obfuscate the code path. Of course there is great value in an MVC framework with reuse. It's a bit contradictory when your aim is a readable specification.

    That said, I like the specification driven approach, and specifying methods kind of on the classes you are testing, rather nice.

    yours, Marcus
    Marcus Baker
    Testing: SimpleTest, Cgreen, Fakemail
    Other: Phemto dependency injector
    Books: PHP in Action, 97 things

  9. #9
    SitePoint Wizard dreamscape's Avatar
    Join Date
    Aug 2005
    Posts
    1,080
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Hi Marcus.

    There doesn't seem to be any set way of approaching BDD, other than to focus on specifications and behavior rather than "test" centric things. To some this means a straight up vocabulary change. But to others, it means other changes as well. I fall in the latter group.

    On the point of it not being readable because the spec concerns are split up across Contexts, Events, and Outcomes, that is true if you try to read the specs from those items. However, if you read the specs themselves where they are defined, then it is very readable. Given a context, when this event, then expect these outcomes.

    It doesn't get much more readable than that, except for plain text, and actually a parser could be written that takes plain text specs and turns them into the class that defines the specs and creates your base Context, Event, and Outcome classes. Then all of the "hard work" is done for you in getting the specs setup and ready to start implementing. JBehave has this option, and it is something I've considered, though I don't think it is necessary, it would be a nice touch. Or even a parser could be written to generate the other basic classes from the spec definition class if you didn't want to work with plain text.

    As for the learning curve, while a test case like your example can be learned and written quickly, the real question is, are people correctly learning?

    Marcus, assuming that a parser exists, then your TDD "hello world" could look like this in the type of BDD I'm proposing (in some BDD it would look the same as yours but just be a vocabulary change):
    PHP Code:
    class My_Specs extends The_Specs {

        public function 
    defineSpecs() {
            
    $this->addSpec(
                
    'any old thing should be true'$this
                
    -> Given    ('any old thing')
                -> 
    Then     ('result should be true')
            );
        }


    Mind you I don't think that it is a very good example, but then again sample test scenarios that are both ultra simple and make practical sense can be difficult to come up with. That's why I used the rSpec tutorial example in the original post to express my ideas, because it is probably the best simple example I've seen, and I certainly couldn't have come with with any better.

    Which leaves out a point I didn't mention, that I think Events should be optional, since there are a few times when they may not be necessary. So the actual mantra should be something like "Given a context [when an event] then an outcome".

    And then within the Outcomes, I think some syntax sugar can help a lot with readability and learning curve.

    For example, instead of "$this->assertTrue(false);" or "$this->shouldBeTrue(false);" I think something such as "$this->value(false)->should->be->true;" reads off a bit easier and better.

    I'm working on the basics of this framework, and have come up with this basic list of shoulds/assertions (I'm sure that the need for more will arise). Mind you I've chosen to use syntax sugar because I think it makes it read off easier:
    PHP Code:
    $this->value($actual)->should->be($expected);
    $this->value($actual)->should->not->be($expected);

    $this->value($actual)->should->be->greaterThan($expected);
    $this->value($actual)->should->not->be->greaterThan($expected);

    $this->value($actual)->should->be->greaterThan->orEqualTo($expected);
    $this->value($actual)->should->not->be->greaterThan->orEqualTo($expected);

    $this->value($actual)->should->be->lessThan($expected);
    $this->value($actual)->should->not->be->lessThan($expected);

    $this->value($actual)->should->be->lessThan->orEqualTo($expected);
    $this->value($actual)->should->not->be->lessThan->orEqualTo($expected);

    $this->value($actual)->should->be->identicalTo($expected);
    $this->value($actual)->should->not->be->identicalTo($expected);

    $this->value($actual)->should->contain($string);
    $this->value($actual)->should->not->contain($string);

    $this->value($actual)->should->match($pattern);
    $this->value($actual)->should->not->match($pattern);

    $this->object($actual)->should->be->instanceOf($class);
    $this->object($actual)->should->not->be->instanceOf($class);

    $this->value($actual)->should->be->a->string;
    $this->value($actual)->should->not->be->a->string;

    $this->value($actual)->should->be->an->array;
    $this->value($actual)->should->not->be->an->array;

    $this->value($actual)->should->be->an->integer;
    $this->value($actual)->should->not->be->an->integer;

    $this->value($actual)->should->be->a->float;
    $this->value($actual)->should->not->be->a->float;

    $this->value($actual)->should->be->boolean;
    $this->value($actual)->should->not->be->boolean;

    $this->value($actual)->should->be->null;
    $this->value($actual)->should->not->be->null;

    $this->value($actual)->should->be->an->object;
    $this->value($actual)->should->not->be->an->object;

    $this->value($actual)->should->be->true;
    $this->value($actual)->should->not->be->true;

    $this->value($actual)->should->be->false;
    $this->value($actual)->should->not->be->false
    But, JIMHO.
    <.smarter.web.development.>
    PHP Stuff: Plexus | Chocolate (BDD Framework... coming soon)
    Graphite

  10. #10
    SitePoint Zealot johno's Avatar
    Join Date
    Sep 2003
    Location
    Bratislava, Slovakia
    Posts
    184
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by dreamscape
    What the hell are you smoking? The only thing that has been renamed is assert to should. The rest of the concepts don't have a 1-to-1 mapping with TDD. I'm afraid you've seriously misunderstood if you think that, or maybe still need your coffee for the day.
    I don't smoke nor drink coffee, but thanks for asking.

    I was talking about BDD in general. Looking at http://rspec.rubyforge.org/tutorials/stack_04.html I just see a 1-1 mapping as I already said.

    As for your approach to BDD, JBehave or whatever it is. As I already said, it looks like a class explosion to me. I don't think that encapsulating simple asserts/shoulds in classes is a good idea. There might be some reuse, but you are making tests/specifications more scattered.

    But I like "should->be->equal" approach, thats much more readable and closer to human language. I must admit that.
    Annotations support for PHP5
    TC/OPT™ Group Leader

  11. #11
    SitePoint Wizard silver trophy kyberfabrikken's Avatar
    Join Date
    Jun 2004
    Location
    Copenhagen, Denmark
    Posts
    6,157
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by dreamscape
    Given a context, when this event, then expect these outcomes.
    As far as I understand it correctly, specs are a declarative extension to tests. This brings the traditional benefits/limitations of declarative vs. imperative.

    For example, the specs clearly define input and output from a given function. From one perspective, this is an improvement, but in practice I fear it may be more limiting than supporting.

    The biggest gain may be that it becomes possible to separate the process of writing specs/tests from implementing them. While this perspective is quite interesting in the large, I don't see how it immediately applies to smaller organizations.

    Quote Originally Posted by dreamscape
    For example, instead of "$this->assertTrue(false);" or "$this->shouldBeTrue(false);" I think something such as "$this->value(false)->should->be->true;" reads off a bit easier and better.
    I disagree. It would be true, if the people who work with this thing weren't programmers. But assuming they are, then they (we) already know the syntax of languages such as PHP. In the case of Ruby, things may be different, because the language itself has a different syntax. So I wouldn't call it more human readable, rather more Ruby alike.

  12. #12
    SitePoint Wizard dreamscape's Avatar
    Join Date
    Aug 2005
    Posts
    1,080
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by kyberfabrikken
    So I wouldn't call it more human readable
    I'm not sure how you can think that. Just read it, and it if reads off easier/better as written, than it is more human readable.

    Which sounds more like a human talking:

    "should be true value"

    OR

    "value should be true"

    We're all allowed our opinions, but I'm just not sure how you can rationalize to yourself that a syntax structure which specifically mimics natural language isn't more human readable than one which does not.
    <.smarter.web.development.>
    PHP Stuff: Plexus | Chocolate (BDD Framework... coming soon)
    Graphite

  13. #13
    SitePoint Wizard dreamscape's Avatar
    Join Date
    Aug 2005
    Posts
    1,080
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by johno
    I was talking about BDD in general.
    Ok, then can we stay on topic please, or at least if you want to talk about BDD out of the context of the thread to at least say so and make it clear.

    It gets very confusing when statements are posted that relate to the topic but only in a vague way.
    <.smarter.web.development.>
    PHP Stuff: Plexus | Chocolate (BDD Framework... coming soon)
    Graphite

  14. #14
    SitePoint Wizard dreamscape's Avatar
    Join Date
    Aug 2005
    Posts
    1,080
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Probably should have put all these replies in one post, but what are you gonna do?

    Quote Originally Posted by johno
    As for your approach to BDD, JBehave or whatever it is. As I already said, it looks like a class explosion to me.
    How do you approach TDD? Do you have just one test case for each class? Doesn't that then result in a method explosion? What is better having fewer classes with far more methods, or far more classes with fewer methods?

    I don't think the amount of coding is actually that much different. With an unrealistic and ridiculously simple test, then yeah what I'm proposing is going to be a lot more code, but let's not talk about unrealistic things.

    I think your main gripe thus far is that you're not sure how reusable Events and Outcomes will be, right? Well, if you are clever about how you build them, then you can get a lot more reuse than you might think, which means that as the application grows then the benefits of having those reusable components only grows as well. You just have to be clever about it

    Here is an example spec set from JBehave which shows how you can reuse items, and this is only within the same spec set. Broader specs or other specs might reuse some of the items as well, so reuse is not limited to the same "test case" as it is with TDD.

    PHP Code:
    $this->addSpec(
        
    'Happy scenario'$this
        
    -> Given    ('account is in credit')
        -> 
    When     ('user requests cash')
        -> 
    Then     ('atm should dispense cash')
            -> 
    And  ('atm should return bank card to customer')
            -> 
    And  ('account balance should be reduced')
    );


    $this->addSpec(
        
    'Happy scenario with overdraft'$this
        
    -> Given    ('account has overdraft permissions')
        -> 
    When     ('user requests cash')
        -> 
    Then     ('atm should dispense cash')
            -> 
    And  ('atm should return bank card to customer')
            -> 
    And  ('account balance should be reduced')
    );


    $this->addSpec(
        
    'Overdrawn without permission'$this
        
    -> Given    ('account has negative balance without permission')
        -> 
    When     ('user requests cash')
        -> 
    Then     ('atm should refuse cash')
            -> 
    And  ('atm should return bank card to customer')
    );


    $this->addSpec(
        
    'In lots of trouble'$this
        
    -> Given    ('account is overdrawn limit')
        -> 
    When     ('user requests cash')
        -> 
    Then     ('atm should refuse cash')
            -> 
    And  ('atm should retain bank card')
    ); 
    I'm really not sure how else to demonstrate reusability in a simple way. And actually here in this example, there is no context reuse, but there is quite a bit of event and outcome reuse.

    I'm fairly sure as well if you look through your current test code, that you will find a good deal of duplication. Maybe just a little in small projects, but when the application reaches a certain size, it is almost guaranteed that the test cases are going have good amounts of duplication in them.
    <.smarter.web.development.>
    PHP Stuff: Plexus | Chocolate (BDD Framework... coming soon)
    Graphite

  15. #15
    SitePoint Wizard silver trophy kyberfabrikken's Avatar
    Join Date
    Jun 2004
    Location
    Copenhagen, Denmark
    Posts
    6,157
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by dreamscape
    We're all allowed our opinions, but I'm just not sure how you can rationalize to yourself that a syntax structure which specifically mimics natural language isn't more human readable than one which does not.
    In german language, verbs are often placed at the end of a sentence, while in english, the verb is placed inside the sentence. Now, I'm not suggesting that we write out code in german, but this is to say that syntax isn't either human readable or not. It's highly subjective, and personally I don't have any trouble with the syntax that C-like languages use. In fact, I find it practical that the subjects are listed together, rather than randomly spread out over the whole sentence.

  16. #16
    SitePoint Wizard dreamscape's Avatar
    Join Date
    Aug 2005
    Posts
    1,080
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by kyberfabrikken
    but this is to say that syntax isn't either human readable or not It's highly subjective
    Alright fine, it is more "English readable". Since we are all sitting here talking in English with each other, I thought that was pretty much a given.

    PS. for the record, sentences typically have one subject (or a compound subject). In "value should be true", value is the only subject. "True" is an object of the verb "should be". So the subject is not "randomly spread out over the whole sentence". It is right where it is supposed to be, but let's not get into an English debate
    <.smarter.web.development.>
    PHP Stuff: Plexus | Chocolate (BDD Framework... coming soon)
    Graphite

  17. #17
    SitePoint Wizard silver trophy kyberfabrikken's Avatar
    Join Date
    Jun 2004
    Location
    Copenhagen, Denmark
    Posts
    6,157
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by dreamscape
    Alright fine, it is more "english readable".
    Alright fine, but we're not writing english essays. We're writing code.

  18. #18
    SitePoint Wizard dreamscape's Avatar
    Join Date
    Aug 2005
    Posts
    1,080
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by kyberfabrikken
    Alright fine, but we're not writing english essays. We're writing code.
    And where does it say that we can't strive to write code that is easily readable [to us, in our native languages] ?

    Why would the PHP developers be supporting code written in any characters in PHP 6 (instead of just ASCII characters), if writing code in your native language did not serve any purpose?
    <.smarter.web.development.>
    PHP Stuff: Plexus | Chocolate (BDD Framework... coming soon)
    Graphite

  19. #19
    ********* Victim lastcraft's Avatar
    Join Date
    Apr 2003
    Location
    London
    Posts
    2,423
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    Hi...

    Quote Originally Posted by dreamscape
    To some this means a straight up vocabulary change. But to others, it means other changes as well. I fall in the latter group.
    I think vocabulary changes are very powerful, so I'm not going to dismiss the idea just on those grounds. This...
    PHP Code:
    class TrueNoMatterWhat extends UseCase {
        function 
    MyFunctionShouldAlwaysBeTrue() {
            for (
    $i 0$i 10$i++) {
                
    $this->shouldBeTrue(my_function($i));
            }
        }

    ...sends a very different message than the usual asserts for developers who are not familiar with reading tests.

    Quote Originally Posted by dreamscape
    However, if you read the specs themselves where they are defined, then it is very readable. Given a context, when this event, then expect these outcomes.
    This only works if you have chosen very good names for these items, and you have already coded them. What happens in the heat of battle when both the outcomes and the spec are changing? Or are not clear yet? Rather than isolated test changes, you now have to edit several items.

    Quote Originally Posted by dreamscape
    It doesn't get much more readable than that, except for plain text, and actually a parser could be written that takes plain text specs and turns them into the class that defines the specs and creates your base Context, Event, and Outcome classes.
    As soon as you get into parsers and domain languages, rather than the programming language, then you are no longer doing unit testing. This is more like acceptance testing. This is a very different domain.

    With unit testing:
    1) You have a rapid cycle of test/code/refactor.
    2) You are concurrently designing. As you see what is possible, new ideas emerge and you change the interfaces.
    3) You are concurrently debugging. Occasionally you need a quick print statement.
    4) You are leaving a trail that someone else can pick up. It looks like work in progress. That may be a formal spec, but it may not.
    5) When the code gets tough, you add more tests. When things are easy, you strip them out.

    With acceptance testing:
    1) The tests are dictated by the client. They need to be readable by non-programmers.
    2) Developers cannot substantially change the test. Only the clients can.
    3) The test code cycle is in days and weeks.
    4) The size of the test bares no relation to the amount of code covered by the test.
    5) Refactoring and redesign should not result in test changes.

    Now it may be possible to marry up these conflicting demands, but it's a heck of a challenge. Which camp are you aiming at?

    Quote Originally Posted by dreamscape
    Or even a parser could be written to generate the other basic classes from the spec definition class if you didn't want to work with plain text.
    It's the psuedo-class generation that I really like about BDD.

    Quote Originally Posted by dreamscape
    As for the learning curve, while a test case like your example can be learned and written quickly, the real question is, are people correctly learning?
    The problem is that you are teaching a whole new way of learning. You won't get that from a manual, you have to play. For people to play, they need to be up and running in the smallest number of lines of code.

    Quote Originally Posted by dreamscape
    PHP Code:
    class My_Specs extends The_Specs {

        public function 
    defineSpecs() {
            
    $this->addSpec(
                
    'any old thing should be true'$this
                
    -> Given    ('any old thing')
                -> 
    Then     ('result should be true')
            );
        }


    Is that really all the code? And do you really have to pass strings to Given() and Then()? Why not an object? To me, this would be clearer...
    PHP Code:
    $this->given(new AnyContext())
                ->
    anyOldThing()
                ->
    then(new AlwaysTrue()); 
    There are fewer mystery mechanics here, and I have the call sequence back. I would expect a print statement in anyOldThing() to have th econtext available for example.

    It's also a little shorter than your example. I can make it shorter still...
    PHP Code:
    $this->given->anyContext()->then->anyOldThing()->gives->alwaysTrue(); 
    I think this can be made to work fairly easily.

    Assuming that the methods can optionally exist within the specification class, then you could get all the code into a short example.

    Quote Originally Posted by dreamscape
    I think some syntax sugar can help a lot with readability and learning curve.
    To me, that doesn't help. I would do a heck of a lot of usability testing before committing to the final language.

    yours, Marcus
    Marcus Baker
    Testing: SimpleTest, Cgreen, Fakemail
    Other: Phemto dependency injector
    Books: PHP in Action, 97 things

  20. #20
    SitePoint Wizard dreamscape's Avatar
    Join Date
    Aug 2005
    Posts
    1,080
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by lastcraft
    And do you really have to pass strings to Given() and Then()? Why not an object?
    Before I went with strings, I thought to use the class names as class properties.

    Nothing is set in stone yet. And I'm still very much working out the language.

    Here are some options I'm considering:

    PHP Code:
    $this->addSpec(
        
    'a secnario'$this
        
    -> Given    ('any context')
        -> 
    When     ('any old thing')
        -> 
    Then     ('result should be true')
    );


    $this->addSpec
        
    'a secnario'$this->
        
    Given->AnyContext->
        
    When->AnyOldThing->
        
    Then->ResultShouldBeTrue
    );


    $this->addSpec
        
    'a secnario'$this->
        
    Given->AnyContext()->
        
    When->AnyOldThing()->
        
    Then->ResultShouldBeTrue()
    );


    $this->addSpec
        
    'a secnario'$this->
        
    Given(new AnyContext)->
        
    When(new AnyOldThing)->
        
    Then(new ResultShouldBeTrue)
    ); 
    <.smarter.web.development.>
    PHP Stuff: Plexus | Chocolate (BDD Framework... coming soon)
    Graphite

  21. #21
    SitePoint Wizard silver trophy kyberfabrikken's Avatar
    Join Date
    Jun 2004
    Location
    Copenhagen, Denmark
    Posts
    6,157
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by dreamscape
    And where does it say that we can't strive to write code that is easily readable [to us, in our native languages] ?

    Why would the PHP developers be supporting code written in any characters in PHP 6 (instead of just ASCII characters), if writing code in your native language did not serve any purpose?
    No, I think you missed my point. Natural languages are good at expressing human thoughts and desires, and have evolved from this need. Computer languages has some resemblance to natural languages, but they are highly specialized in their focus. Just like mathematics is best expressed in a specialized language, so is imperative code structures best expressed in a language that is more strict than natural language. In programming it is generally not a good thing if an expression is vague, whereas this is an important feat of natural language. One of the consequences of this difference is that it is much more desirable to have a consistent syntax in programming than it is in natural language. Even smalltalk'ish languages such as Ruby, has a much stricter syntax than natural language. Grouping together the subjects in a parameter list as C'ish languages is an example of a strict syntax.

    Quote Originally Posted by dreamscape
    PS. for the record, sentences typically have one subject (or a compound subject). In "value should be true", value is the only subject. "True" is an object of the verb "should be". So the subject is not "randomly spread out over the whole sentence". It is right where it is supposed to be, but let's not get into an English debate
    God forbid, no. I probably missed some of the finer points of german language too. But I think we agree on the premises anyway, so let's just leave that be.

    Quote Originally Posted by dreamscape
    Why would the PHP developers be supporting code written in any characters in PHP 6 (instead of just ASCII characters), if writing code in your native language did not serve any purpose?
    On a sidenote, I write all code - even comments - in english, even though it's not my first language. I'm not sure how relevant that is for the discussion though.

  22. #22
    SitePoint Guru
    Join Date
    Nov 2002
    Posts
    841
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    You know, I've never liked the class name TestCase in the jUnit derivatives. The class isn't a test case, each individual test* method is. Context to me is such a better name than TestCase.

    Will renaming the class cause developers to write better test code, possibly with less setup code duplication? Probably.

    I agree that many unit tests are often written to test an implementation rather than an interface. Thus, the test is brittle and sensitive to legitimate implementation changes. I guess the idea with BDD is that you write the test in a way such that the implementation may vary, thus the test becomes more of a specification than a test. (This doesn't necessarily do away with the need for implementation tests.)

    This is harder to do. I have some test examples where I've been fully aware of how brittle and implementation dependent they are and I've never been able to come up with a way to perform the necessary test that wasn't.

    Can changing the name of assert to should encourage developers to factor out duplicate assertion code? I'm much less clear on that.

    Duplicate code and brittle tests are definitely the dark underside of unit testing. I'm just not sure BDD provides all that much light. Maybe there is more here than I understand?

    Interesting, tho.

  23. #23
    SitePoint Wizard dreamscape's Avatar
    Join Date
    Aug 2005
    Posts
    1,080
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by lastcraft
    Now it may be possible to marry up these conflicting demands, but it's a heck of a challenge. Which camp are you aiming at?
    Opps, missed this question.

    I guess I am attempting to aim at both camps.

    JBehave uses something similar to my proposal for acceptance testing (that's where I got the main idea of "Given <context> When <event> Then <outcome>" in the first place). And then for unit testing, JBehave uses something similar to TDD with a vocabulary change.

    But this left me wondering why JBehave didn't use the same framework on all testing levels. It seems, at least to me, that a straight up vocabulary change -- while perhaps powerful -- is an attempt to get current TDDers to buy into BDD since they'd only have to change the way they talk (and that would in theory change the way they think).

    It's much easier to just change the way you talk than to change they way you work and approach something, and so I realize that proposing to use something different for unit tests is going to be met with much head butting from long time TDDers. But just because some people are opposed to breaking their habits doesn't mean we shouldn't try to think in new or different ways.

    In my mind, it is very possible to use the same framework for both your acceptance tests and your unit tests. A unit has its own specs, so why not? The only fundamental difference between the two, as far as I can see and as far as specifications are concerned, is that with one specifications are dictated by the client (most likely in advance) and with the other, specifications are created by the developer as they work. But they both can be viewed as "specs" nonetheless, so I think it is very much possible for a spec driven framework to do both unit tests and acceptance tests.
    <.smarter.web.development.>
    PHP Stuff: Plexus | Chocolate (BDD Framework... coming soon)
    Graphite

  24. #24
    ********* Victim lastcraft's Avatar
    Join Date
    Apr 2003
    Location
    London
    Posts
    2,423
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    Hi...

    Quote Originally Posted by dreamscape
    In my mind, it is very possible to use the same framework for both your acceptance tests and your unit tests. A unit has its own specs, so why not?
    You are guessing.

    Quote Originally Posted by dreamscape
    The only fundamental difference between the two, as far as I can see and as far as specifications are concerned, is that with one specifications are dictated by the client (most likely in advance) and with the other, specifications are created by the developer as they work.
    That's a huge difference. As long as I hit my acceptance criteria (client plus any contracts with external components) I can refactor anything I want. As this makes lower level contracts (unit tests) pretty short term and nebulous, why are we putting them on a pedestal? Again, it's necessary for a developer to follow what I've done, but that doesn't dictate a specification. It's more important that the tests indicate intent.

    If you want a spec, you can just read the source code.

    My more fundamental problem with the BDD spec approach, including it's use in acceptance testing, is that specifications suck. Stories (use cases) are easier to capture, are much more the genuine raw material (specs are a construct from the stories even when a client writes them), are far less error prone and are far more independent of each other.

    This is why I would always mourn the loss of the time element. Stories are a sequence of conversations (with implied state changes). The intermediate contexts of each step are not important. I would like something like...
    PHP Code:
    $this->when->notLoggedIn->canLogIn()->
            
    then->canSelectBook()->
            
    then->areForcedToSeeRelatedItemsCosMarketingsaysSo()->
            
    then->canCheckOut()->
            
    whereupon->arePoorer(); 
    ...because that is closer to what the business would understand.

    If you want to improve developers working habits then act like an anthropologist. First immerse yourself in the culture (you are already a TDD'er, so you have a head start) and then use narrative capture (basically story gathering).

    Gather anecdotes about how they use the current tools. It may be that you get the same programmer stories for acceptance testing that you do for unit testing. I reckon you won't, and instead you'll get two distinct clusters (attractors).

    I am guessing.

    That said, JBehave has obviously been nudged in that direction. You should probably ask the authors to tell you the story of why this happened.

    yours, Marcus
    Marcus Baker
    Testing: SimpleTest, Cgreen, Fakemail
    Other: Phemto dependency injector
    Books: PHP in Action, 97 things

  25. #25
    SitePoint Wizard dreamscape's Avatar
    Join Date
    Aug 2005
    Posts
    1,080
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by lastcraft
    You are guessing.
    I'm making a hypothesis based on experience. All ideas are hypothetical until put into practice, but does that mean we shouldn't try them or come up with them? Hell no, IMO.
    <.smarter.web.development.>
    PHP Stuff: Plexus | Chocolate (BDD Framework... coming soon)
    Graphite


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •