TDD, I love you

[Chris Corbyn]

Are there any basic tutorials or introductions to TDD out there? I've been googling around a bit but haven't found one I liked yet. I want to get into it, but the things worrying me are code coverage (how do you know you are testing everything?), and the length of time I imagine it takes to come up with and program all the test cases.

Worrying about code coverage is more of a side-effect of testing afterwards. If you never write code without first justifying your reasons for writing that code with a test then you almost inevitably get very high code coverage. Literally, write a test then write code to make the test pass. If you're doing that is' difficult to have any gaps. Obviously there's the trivial things like accessors which probably don't need testing.

I don't have any links to tutorials sorry... you may find more if your search for JUnit tutorials but you'll have to read Java (looks very like PHP5 anyway) to grasp them.

I was actually thinking of writing on article on TDD and pretty much just talk about my reasoning and approaches to situations.... more of a readable article than a tutorial but educational nontheless. Finding time to write it is another matter :P

I'll speak to Matt in the office and see if we can publish an article

[Stormrider]

I know some Java anyway so it wouldn't be too much of a barrier, I will take a look around... cheers

An article would be fantastic :P

If I am half way through a project... is it too late to start writing tests? Do I need to have done that from the beginning?

[Chris Corbyn]

If I am half way through a project... is it too late to start writing tests? Do I need to have done that from the beginning?

It's not too late to start. You *can* unit test afterwards it just feels much more haphazard and it's often not clear how to go about testing things. You could certainly employ TDD on new components you write within your half-written code. Writing unit tests for the existing stuff would most likely be a waste of time if you've never unit tested and would perhaps even scare you away from the concept of TDD. Remember, TDD isn't really just about testing your code... it's a whole approach to producing code in the first place (in the same way some people start with UML diagrams, TDDers start with tests). It's not 100% clear cut though, nothing really is in programming. I often mock out a rough archicture with UML just to get a vague idea of where to start with my tests. It depends on the complexity of what I'm planning to do.

[BerislavLopac]

A quick response to nabeel (and others): in a MVC framework, you unit-test your models.

[McGruff]

I can say for sure that without the 77 test cases this library has I would never have even dared attempt the scale or refactoring I did.

It's a virus. You're infected for life now

[phpimpact]

I've been reading some crazy stuff lately about software testing. For example:

Over the last few months, I have developed the firm opinion that the developer that does not unit test when it is possible to unit test is a bad developer.

Yup, and so is the one that never debugs. Of course you have to test your code! Just like debugging, testing was not discovered in this millennium

I love writing tests as much as I love writing new code.

What's the difference between testing and writing code? If I'm not wrong, in order to test you need to write code.

Something I don't understand is why everyone now starts the sentence with: "I love testing". I don't get it, what where they doing before?

You could certainly employ TDD on new components you write within your half-written code. Writing unit tests for the existing stuff would most likely be a waste of time if you've never unit tested and would perhaps even scare you away from the concept of TDD. Remember, TDD isn't really just about testing your code... it's a whole approach to producing code in the first place.

Can you or can you not write a test of an existing block of code according to the TDD definition? If I'm not wrong, you can't.

in the same way some people start with UML diagrams, TDDers start with tests.

I think this might confuse people. UML diagrams and TDD are not related in any way. TDD is a technique used in the implementation stage of the process, UML is spec language for object modeling used in the design stage. In agile development you need the planning as much as you need the testing.

What's funny is that no one ever says: I love UML

[nabeel]

A quick response to nabeel (and others): in a MVC framework, you unit-test your models.

Yeah. Actually, I've been kinda doing that, without knowing/calling it "unit testing".
Now I'm writing routines to test the installer models, and other models. It's kind of a pain, but I suppose in the long run it'll show the benefits.

[Chris Corbyn]

What's the difference between testing and writing code? If I'm not wrong, in order to test you need to write code.

This is true. But it's a different sort of writing code. Test code isn't anywhere near as complex as the code you'll write to make the test pass. I wouldn't have much faith in my tests if they were unwieldy, complex bundles of code. Tests are very declarative... you're basically "using" your API, and making declarative assertions that "This should happen", "This should not happen", "This should look like this", "This should be invoked 5 times" etc.

Something I don't understand is why everyone now starts the sentence with: "I love testing". I don't get it, what where they doing before?

Testing probably. But just not loving it

Can you or can you not write a test of an existing block of code according to the TDD definition? If I'm not wrong, you can't. I think the definition of TDD has flaws, it has to be reviewed.

No you cannot. TDD is *test-driven* development. You can't write the tests after the code. But that's why my wording very carefully interchanges between "unit testing" and TDD. Unit testing alone is definitely not TDD.

I think this might confuse people. UML diagrams and TDD are not related in any way. TDD is a technique used in the implementation stage of the process, UML is spec language for object modeling used in the design stage. In agile development you need the planning as much as you need the testing.

I agree they are very different. I probably should not have gone down that route but it seemed like a good analogy at the time Basically I was trying to put across that the tests *are* specs in the same way UML is a spec.

What's funny is that no one ever says: I love UML

I do love UML

[webaddictz]

Hi Chris,

I was actually thinking of writing on article on TDD and pretty much just talk about my reasoning and approaches to situations.... more of a readable article than a tutorial but educational nontheless. Finding time to write it is another matter :P

I'll speak to Matt in the office and see if we can publish an article

Please do, I'm curious.

[dhtmlgod]

Can you or can you not write a test of an existing block of code according to the TDD definition? If I'm not wrong, you can't. I think the definition of TDD has flaws, it has to be reviewed.

What Chris said +1. When ever I introduce someone to TDD, I like to tell them that the Development might be better replaced with Design. I've found a few people get it a little bit easier after that.

Something I don't understand is why everyone now starts the sentence with: "I love testing". I don't get it, what where they doing before?

I did test, but it was a joyless task. TDD really makes testing fun. IMO anyway

I think this might confuse people. UML diagrams and TDD are not related in any way. TDD is a technique used in the implementation stage of the process, UML is spec language for object modeling used in the design stage. In agile development you need the planning as much as you need the testing.

UML is the work of the devil and be a huge hinderince! I used to use UML alot with Visio. Now I use it more for disposible sketches when I first start to tackle a problem, and as soon as the code either proves the UML correct or incorrect, it get disposed of.

A quick response to nabeel (and others): in a MVC framework, you unit-test your models.

I unit test everything with ASP.NET MVC: Models, Controllers, Routes, etc. The fact that is alot more testable is the reason I use it.

Here are some books that helped me get started:

TDD By Example
Pragmatic Unit Testing
Working Effectively with Legacy Code

[phpimpact]

Testing probably. But just not loving it

Wait, isn't that the slogan of... McDonald's??

Jokes aside, have you used PHPSpec before? I really like the idea behind it, because you are not only testing your code but also documenting it. I haven't used Yay Mock yet, but that's something I'm looking forward to.

Working Effectively with Legacy Code

Yes, great book.

[McGruff]

What's the difference between testing and writing code? If I'm not wrong, in order to test you need to write code.

In order to write code first you have to write a test

How do you know what code to write if you haven't defined the target to be hit?

[Chris Corbyn]

How do you know what code to write if you haven't defined the target to be hit?

Exactly. This is also another selling point for TDD. It helps prevent feature creep and "over engineering". You stop coding when you've made your test pass (unless you're just doing a bit of refactoring). Without the tests at what point do you say "Ok, I'm done now.."?

[phpimpact]

Well, all this is debatable. You can achieve the same thing with use cases. Basically, uses cases define the target to be hit and prevents a system from being over engineered. You can have a use case diagram or a use case analysis, or both, considering that modern software converts diagrams to code automatically.

IMO some TDD practitioners are trying to convert a technique into religion. Yes, testing is a fundamental part of software development, but TDD does not drive the development all by itself, so trying to alienate TDD from the software development process is wrong. And you are doing it by saying "do this first".

Only Testing defines the target to be hit? Nope. Only Testing ensures a well engineered system? No.

Keep in mind that testing is evolving and new techniques are emerging, like TDD, BDD and others, but also keep in mind that techniques evolve, while religions don't.

[phpimpact]

I just read my last post and I don't want people to think that I'm underestimating the value of TDD, because I'm not. I consider TDD to be a much better technique than the traditional unit testing.

But, when someone says "You have to test first", that's where the confusion begins. Because in TDD, you don't really test first, you don't skip other steps from the development process, such as writing use cases analysis. And according to some experts, use cases analysis can improve testing/design techniques, such as TDD. And to write a use case you need to code first.

So, you hear things such as: "Who needs UML". Well, I do. Or: "My first line of code is always a test". Mine is a use case for example.

That's why I consider TDD to be a technique that you can use in one or more steps of the development process. That's the beauty of agile development right, it lets you choose between different techniques and adjust them to your requirements. But these techniques can change, and they will change, and that's when we need to rapidly adapt.

[dagfinn]

And to write a use case you need to code first.

So, you hear things such as: "Who needs UML". Well, I do. Or: "My first line of code is always a test". Mine is a use case for example.

I don't think most of us would consider use cases code. That may be seen as just a quibble about wording, but naming is important.

I think the agile spirit is to produce only as much design and requirements documentation as is needed. How much that is might depend on the situation. In my experience, the complexity of the task is a crucial variable.

[phpimpact]

I completely agree with you Dagfinn, complex tasks involve more planning and design.

I don't think most of us would consider use cases code

What do you mean by "us", your company or forum users? Please allow me to disagree on this. While small teams of agile developers try to reduce the design and documentation as much as possible, large teams tend to do the opposite. The PHP project is a good example of this, or the Apache project or even the Zend Framework.

Lets take the Zend Framework for example, a project with a true agile spirit. They use TDD and other techniques to assure the quality of their product. The development team follows agile methodologies, but, as opposed to what you are saying, they try to write as much documentation as possible. And the design process involves writing use cases to produce a brainstorm of ideas and opinions, that will eventually have a huge influence on the design. All this happens before the TDD technique is used.

Maybe I'm missing something Dagfinn, I'd appreciate if you could explain this a little bit more.

Thanks

[dagfinn]

Maybe I'm missing something Dagfinn, I'd appreciate if you could explain this a little bit more.

I think this may be a misunderstanding. I understood you to say that use cases were "code", which seems like somewhat unusual terminology. It wasn't an important point anyway.

[McGruff]

Lets take the Zend Framework for example, a project with a true agile spirit.

I don't know if that's a good example for test-first programming. It's been a while since I've looked at it but I didn't get the impression that it was well-designed or written by test-infected programmers.

[sweatje]

Lets take the Zend Framework for example, a project with a true agile spirit. They use TDD and other techniques to assure the quality of their product.

I would say they use Unit Tests to ensure the quality of the product. Some of the overall design decisions lead me to believe that TDD itself is not being employed.

[phpimpact]

Sweatje, McGruff, Chris and Dagfinn, I know you are experts in TDD, and that's why I'm asking you all this, to learn more about TDD and software development in general.

Maybe the example I gave was debatable, but what about the point I was trying to make? I know that it's very difficult to find out if the developers wrote a test first or after, they are the only ones that know this.

In order to clarify some areas of the discussion, I'm going to write down some sentences. Feel free to write "true" or "false" next to them:

1. BDD is an evolution of TDD.
2. In order to write code, first you have to write a test.
3. In TDD you cannot write code before writing a test, that includes a use cases analysis.
4. The primary goal of a unit test is to design a system from the developer's perspective.
5. The primary goal of a use case analysis is to design a system from the user's perspective.
6. If you use TDD, creating an abstract model of a system is pointless.
7. Once you have written some code, you cannot use the TDD technique.
8. If you don't write a unit test first, you'll never know when to stop coding.
9. If you don't write a unit test firt, you'll never know what target to hit.
10. TDD is better than unit testing because it not only ensures the quality of a system, but helps design it.
11. By using TDD you ensure the quality of a software.
12. The quality of a software is good when it works and has no bugs.
13. TDD is also considered a design technique.
14. The agile spirit is to use TDD only if needed.

Thanks guys

[Chris Corbyn]

1. BDD is an evolution of TDD.
2. In order to write code, first you have to write a test.
3. In TDD you cannot write code before writing a test, that includes a use cases analysis.
4. The primary goal of a unit test is to design a system from the developer's perspective.
5. The primary goal of a use case analysis is to design a system from the user's perspective.
6. If you use TDD, creating an abstract model of a system is pointless.
7. Once you have written some code, you cannot use the TDD technique.
8. If you don't write a unit test first, you'll never know when to stop coding.
9. If you don't write a unit test firt, you'll never know what target to hit.
10. TDD is better than unit testing because it not only ensures the quality of a system, but helps design it.
11. By using TDD you ensure the quality of a software.
12. The quality of a software is good when it works and has no bugs.
13. TDD is also considered a design technique.
14. The agile spirit is to use TDD only if needed.

1. True (it's still a form of TDD though)
2. True for the mostpart. I don't write tests for simple accessors for example.
3. Mostly true. I'm not sure what your definition of "use case analysis" is though. I often open a blank file in the very early stages and sketch out how I'd like to use the API before I ever write it. Nothing final, just to get a rough feeling for it. This is NOT the same as writing code. It's the equivalent of grabbing a pad of paper and scribbling a few ideas down. My definition of "writing code" for the purposes of this discussion is "implementing the API".
4. False. It's just as much about designing from a user's perspective. This is the whole emphasis - to focus your thoughts on the interface, not on the implementation. You're documenting behaviour upfront and only then are you implementing the code to provide that behaviour.
5. True.
6. False. This comes with refactoring, for which your tests make a HUGE difference (hence my first post in this thread).
7. True, provided you're referring to only that specific component. You can have selected parts of a system developed under TDD and other parts developed without. Just because you didn't use TDD on component X does not mean you cannot use it when you come to develop component Y. Something Fowler likes to re-iterate is that "Having incomplete test coverage is massively better than having no test coverage at all". Even incomplete tests make your maintenance a whole lot easier.
8. False. It's just less clear-cut.
9. False. (As above)
10. True (ish). This is a bit like comparing chalk and cheese. Although unit testing is used in TDD they are not directly comparable. TDD does drive the design though (I agree it's not as clear-cut as many TDD proponents would have you believe). TDD does not "ensure the quality of a system" however if it's not done correctly. Employing TDD can be done very badly indeed. It falls on the developer(s) to put thought into the tests just as much as the design. What TDD ensures if done correctly is that "The system does what you think it should do". This doesn't mean "it does what it should do".
11. See above.
12. False. You can write software with no bugs and which does what it should do but is still a heap of impossible to maintain and undocumented spaghetti. Assessing the quality of software requires more than a "well, it works so it must be good" evaluation.
13. True. But as I said earlier, you'll often use other tools such as UML to help your design process. You can certainly use TDD to drive the design entirely though. I don't limit my toolkit to TDD alone however; I like to "sketch" out ideas first.
14. I can't answer this honestly sorry. I wouldn't use TDD to write a short command line script or a 20-line chunk of code. We're all capable of using our initiative and deciding when we're about to use a sledgehammer to crack a walnut.

[McGruff]

BDD is really just TDD repackaged with new labels. When I first encountered BDD I'd already been TDD-ing for a while and so it didn't make a big impact on me. In fact I was left feeling disappointed that I hadn't learned some crafty new testing process. Still, language is important and perhaps BDD terminology is easier to learn if you are starting out. I don't know. The word "test" has many meanings to the test-infected: a design process, documentation, and of course they are also "tests" too.

The agile manifesto emphasises customer collaboration. You always begin with some kind of end-user requirement. An acceptance test might describe a use case: this would be a "black box" test which describes the behaviour of the machine as a whole but doesn't tell you anything about the components you might use to build the machine. The component parts — classes — are described with unit tests.

TDD-ers talk about firing tracer bullets: "ready, fire, aim". It might not be clear how to solve the problem but it's better to take your best shot rather than get bogged down. Working top-down, in little steps, you can learn as you go. "Aiming" is done by pausing regularly to review the choices you have made, refactoring as necessary to do it better. I might have been a bit curt about the Zend Framework earlier and it's only fair to say that my own code, and probably a lot of TDD'd apps, stubbornly remain several refactorings away from design perfection.

Once you've got one use case working you can start to bring in others, one at a time, and see how your code stands up (probably it won't if you've been following the "code just enough to make the test pass"; that's good though since it stops you over-complicating things). TDD is an evolutionary process in lots of little steps and that's what makes it so powerful. Small problems are easier to solve than big ones. It's the ancient principle of divide and conquer.

Diagrams can be useful as developer documentation. I'll sometimes sketch out an object graph in Freehand (just a simple sketch showing client/service relationships) but only after the fact. I wouldn't try to sketch out the whole app before I start. I'd be committing myself to lots of choices before I have the information needed to make them.

In contrast, with TDD, you're only interested in an object's immediate neighbours as you work your way through the tests. At each step of the way you can choose what to do next. When the trail to be followed hasn't already been mapped out you have more freedom to discover new ways of solving a problem.

If you don't test-first, the code which you write will simultaneously be too much and not enough. Too much because you'll inevitably stick things in which you think you might need probably.. maybe.. instead of actually waiting until you really do (it might never happen). Not enough because the discipline and detail of testing often throws up things which you haven't thought of.

You can of course write unit tests afterwards. It's a bit of a chore when you do it the wrong way round but worth doing. However, although this is unit testing, it isn't TDD because you're not using unit tests to drive the design.

Test-first is a very focussed, satisfying way to work: describe a little piece of behaviour, code just enough to make the test pass, move on to something else. It sounds innocuous, simple common sense, but it really is a big deal. It's the most important thing I ever learned in programming. I'm running out of time now and I'm not sure if my explanation lives up to this level of hype. If not, keep pressing away.


PS: I'd never, ever call myself an expert on anything to do with programming. I don't have the years of professional experience of many other posters here but I've been very lucky to learn from them.

[phpimpact]

Great thanks, I think you've answered all my questions

My definition of "writing code" for the purposes of this discussion is "implementing the API"

Yes, you are right. It makes more sense now. I had a different definition of writing code.

I'd be committing myself to lots of choices before I have the information needed to make them.

I've spent a couple of minutes analysing this sentence and it's true: task, information, choice. One choice at a time. I cannot think of any other process, other than TDD, that would allow me to commit myself to one choice at a time.

Now, how do you explain this to a product manager, product owner or project manager? Supposing they don't know anything about white-box testing, and they think they are giving you all the information you need to make all the choices. TDD clearly benefits the developers, but how do you change the mentality of an organisation with traditional development processes, where the implementation comes before the testing and a developer writes a block of code and another one a test for it?

It's easy to infect a developer, but the question is, how do you infect your boss?

[Chris Corbyn]

It's a good question and one I've never asked at work here. I wonder how they convinced Mark and Matt... something for me to bring up in tomorrow's dev meeting. I suspect they didn't really need any convincing since we're pretty darn focused on doing things right even if it might take longer.

Now, my last boss... well, I couldn't convince him of it's worth. In the end he let me use TDD to shut me up but used to take the mick out of me. He used to take the mick out of me when I talked about design patterns too but having kept in contact I know that rubbed off on him at least :P