I’ve always considered Cucumber to be just another tool in my developer’s toolkit of testing tools. Like RSpec, Test::Unit or MiniTest::Spec I thought of Cucumber as just another way to test my code: I would use RSpec for my unit tests, and Cucumber for my integration tests. While Cucumber’s business oriented scenario language seemed fun and innovative to me, I never really gave much thought to why Cucumber tests were written in such a verbose, elaborate way.
Then last week I had the chance to speak with Matt Wynne, co-author of The Cucumber Book: Behaviour-Driven Development for Testers and Developers, and a Behaviour Driven Development (BDD) thought leader and coach. In a long and thoroughly entertaining conversation, Matt explained to me the fundamental ideas behind BDD and why Cucumber was designed the way it was. As you’ll see, Matt helped me to rethink how I’m using Cucumber in all of my projects. Here are the highlights of our conversation… possibly after reading this you’ll begin to reconsider the way you’re using Cucumber as well!
News
Q: Before we get started, is there any news we should know about in the world of Cucumber?
The big announcement in the Cucumber world is that Cucumber JVM just was released as 1.0 a couple of days ago. Aslak’s really been pushing to get that done: it’s a port of Cucumber to run on the Java VM as native Java code all the way through.
Q: Does that mean that Java developers can use Cucumber just as easily as Ruby developers can now?
They always have been able to, but it was a bit of a bodge which involved using JRuby. One of the clever things Aslak did a while back is that he extracted a library called Gherkin, which is the core parsing library around feature files. It uses Ragel to parse those files. Ragel is a code generator, so it generates code that parses feature files in Ruby, C, Java and JavaScript. The Java code can then be ported to C# as well; that’s how SpecFlow works for example. SpecFlow is the .NET port of Cucumber, built on top of basically the same core code as the Ruby Cucumber is. This JVM version is using that as well.
History
Q: How did you first get involved with Cucumber?
I went to the Software Practice Advancement conference in 2008, in Cambridge in the UK. It was probably the first conference I’d ever been to and I was really intimidated. There were people like Michael Feathers there; I’d just read his book. Dan North was there also. Dan and Joe Walnes did this talk called Awesome Acceptance Testing. It was all about the kind of acceptance tests they were writing – I think they were both working at ThoughtWorks at the time, and were talking about using examples and trying to write your acceptance tests in a way that was really, really readable so that you could actually share them with non-technical people on the team. And this just resonated with me deeply because I’d recognized for a long time that the thing that was so painful about software projects was where, you know, you couldn’t get that connection with the business people.
That evening, I ended up getting drunk in the bar with Dan and Chris Matts, and my BDD indoctrination was complete!
Q: Do you have any idea where “Cucumber” got its name? I heard a legend about how Aslak’s wife was cooking with cucumbers? Is this true?
She was about to bite into a Cucumber sandwich – that’s the story I heard. You’d have to ask them about that.
Q: We can put out another rumor; maybe the true story will emerge.
How Cucumber was intended to be used
Q: I really enjoyed reading The Cucumber Book. Since reading it I’ve come to think that I’m probably not using Cucumber the way I should be.
What do you mean?
Q: Well, I think generally my steps and my features and scenarios aren’t very well written. I write them the way I would write Ruby code. I don’t think about the intended audience of the scenarios.
I do a lot of work now training and coaching people that use Cucumber, so I get to see a lot of teams that are using it badly. I try to get a whole team together in a room, right from the product owner through to the project manager, the tester, the business analyst and the developers. I get everybody in a room together, and help them to find some consensus about how they want their features to read. Because the whole point is when these things work well is when the Cucumber scenarios become that single point of truth for the whole team.
It becomes something that everybody on the team feels like they own. So it’s not just the thing that the developers write – that thing that makes the browser flash around that developers do. If the business analyst is wondering about what the system does in this particular edge case they can go and search through the features and see. They trust what the features say.
Q: So do you view Cucumber as a pair or group programming tool?
I view it as a team communication or collaboration tool really. So by playing the game, forcing yourselves to try to describe what you want the software to do, together, what you end up doing is surfacing all the things that you haven’t actually figured out yet and all the things that you’re not really sure about. This is what Dan North calls deliberate discovery.
If I put a team in a room and I say: “Right OK – tell me about your user stories. Right. Tell me about the acceptance criteria for that user story. Right. Let’s write down the scenario together that describes how somebody might use that piece of functionality.” And they have a great, big argument! That’s what happens. They all have slightly different ways they want to describe it, slightly different words they want to use. And that argument is really, really important because what that is exposing is all of the different mental models all of the different people on the team have got. By exposing those differences and getting them out into the open, you can start to bridge the gaps between people’s different mental models. The earlier you do that on a project, the smoother the whole thing goes.
Q: Is that what you meant in the book by “Ubiquitous Language?”
Well, a ubiquitous language is a key part of building those bridges, yes. The term actually comes from Eric Evans’ book, Domain-Driven Design: Tackling Complexity in the Heart of Software. That’s a great book about TDD, and emergent architecture – the kind of architecture that emerges in an agile project where you keep refactoring as you go. You keep learning about the design as you go. It’s a tough read in many ways, but one really easy-to-grasp concept in that book is this concept of the “Ubiquitous Language”. It’s such a simple idea: that you just make a deliberate effort to make sure that everybody uses the same words for the things in and around your project, right from the business people to the database administrator.
Q: …because they may not be – or even worse they may be using the same words for different things?
Yeah that is even worse, isn’t it, when a term is overloaded? If different people are using different words to refer to the same things, then you’re just setting yourself up for a fall. Every single instance is an opportunity for a misunderstanding, and when you scale that up over the number of conversations that will be had over the course of a project, it’s scary.
Q: In your opinion, would you get most of the benefits that BDD provides if you wrote the scenarios but never executed them? If you never wrote step definitions?
I would say you’d get about half of the benefit. You wouldn’t get any of the regression good stuff. You haven’t got a safety net for refactoring future changes that you make and future iterations might break existing functionality. And you don’t have outside-in development driving which unit tests and code you do need to write – you miss out on all of that. But the benefits that you have as far as figuring out what the hell you are supposed to be building in the first place – I think that’s half the benefit, I really do.
Q: I think I’m making the huge mistake of thinking of Cucumber as just another tool in my testing toolkit. I would never want to bother my sponsors or business analysts with technical details.
It might be that you are wearing the business analyst’s hat as well as the developer’s hat. It might be that the client you are working for isn’t interested in the amount of detail that you have to put into those Cucumber specs. They are quite mind-numbingly tedious.
Q: Who should use Cucumber? Only developers? Or business people as well?
It should touch almost everybody on the team somehow. In my training I make a point of demonstrating the full-stack implementation of a single scenario in front of everyone on the team, just so they’re aware of what the pieces of the puzzle are: what a step definition is, what a scenario is, and so on. But the people who run Cucumber at the command line are generally going to be either your Jenkins build, or developers.
Q: But is it important to involve the entire team? At least to define what the scenarios are?
There’s always this tension between your cukes being good tests and good documentation. Imagine the scenarios are on a spectrum: on one end very easy for business people to read, great documentation but maybe not very good tests. And the other end of the spectrum is where they are really good thorough tests, but if you show them to the product owner they say “What’s that? I don’t understand that at all.” Every time a developer touches them they’re likely to be pulling them towards that technical end of the spectrum and making them a little bit less approachable and readable to the non-technical people on the team. Day to day, it does tend to be developers who will touch the features, because they’re the ones with access to git. So you need to keep making a deliberate effort to make them work well as documentation.
Granularity of Cucumber scenarios
Q: How granular should your Cucumber scenarios be? What level of detail should you write them in? I guess if they’re more high level and abstract then it would be more likely you could get clients and business people involved.
Yeah. And I find that they’re more useful in many ways like that. One thing is that they’re not tied to a particular implementation. So if you genuinely need to change the user experience, the journey the user has to take to achieve a goal, you’re probably talking about changing one Ruby method in your support layer. Also they will be more readable by non-technical people.
It’s something about scale also. Early on in a project, when you’ve only got a handful of features you might feel comfortable talking at quite a low level of abstraction – clicking things, etc. When you start to zoom that out, when you’ve got hundreds or thousands of scenarios, you stop being able to read things at a low level of detail.
One of the other things that happens when you use more of an imperative style, and you talk at that lower level of abstraction with a lot of detail, is you miss the opportunity to have to give names to things. Because if what you do every time you sign a user in is:
[gist id=”2314300″]
If you forget to give that little journey the name of “signing up” then you’re missing the chance to have a handle you can then use in a conversation to talk about that aspect of your system.
Q: That’s fascinating – I never thought of it that way.
Testing domain objects directly
Q: In one of the examples in your book, you started by running scenarios directly against model objects, for example “Bank.” Is this a good idea? Did you do this intentionally?
Yes, I did. We talked about the Ubiquitous Language earlier. When you’re first kicking off a project and you’re building a domain model, it’s really nice if you can see the relationship between the language you’ve used in your scenario and the way that’s being expressed in the domain model. Because you should see the same words.
So [testing the domain models directly with Cucumber] helps you to make sure you’re building the domain models with consistent terminology, not just in terms of the nouns but also the verbs. Do the operations, the things you’re doing to those models, do they map to the names of the methods that you’re using?
Even if just as a thought experiment you said to yourself: “Could I run this scenario just on the domain model?” It helps you to write the scenario at the level agnostic of the particular solution. So it’s very deliberate in the book that I write the scenario and make it pass the first time just to get to the domain model. And then I wrap the user interface around the domain model, but I never go back and change the scenario. The scenario is equally applicable to a user interface implementation or just to a domain model.
Q: You’re also debunking or invalidating another misconception that I’ve had for years: that Cucumber is a user interface testing tool.
It’s an acceptance testing tool. And yes, 90% of the time what the stakeholders care about is that a user can click around the user interface and do things. But really it’s about reflecting what the code does in a language that is accessible to people who can’t read the code. Usually the interesting bits of that code aren’t in the user interface.
I did a talk a while back on Mortgage-Driven Development (MDD) – I talked about how using very imperative, technical scenarios actively excludes the very people that Cucumber professes to reach out to. If you use a very technical style, then non-technical people will read them once and go “Oh that looks like something that I’ll never have to read,” and then they’ll never bother you again. Some developers would even like that.
Relish
Q: I noticed you built an app called Relish – what is that?
There’s one non-technical person that you nearly always have on a team who would really love to be able to read the Cucumber features. But the trouble is they probably don’t know how to use Vi, Git, etc. Actually Github is quite a useful interface for people like that to read features, but even then it’s still quite technical feeling. What you want to do is give them a way to browse around the features and use them as reference documentation, just the same way they would have been able to use their old Word specification documents.
That’s what Relish is for – in some way it’s just a glorified HTML formatter for Cucumber features. But it’s becoming quite a bit more than that. Your whole features directory is published online like an e-book, with each feature and scenario getting its own permanent URL. There’s a Solr/Lucene search index across all the features. We support markdown pages as well as feature files to add more editorial content. It’s dead easy to search for things and find scenarios that mention certain domain concepts, so it’s easy to navigate around and use it as reference documentation. And they look pretty! It’s surprising how much effect that can have in keeping non-technical people involved when they’re using a friendly medium like Relish to interact with the features.
Thanks!
Q: Sorry, Matt – I have to go… have to get back to my day job…
No problem. Have a good day – a good Friday!
Q: Likewise…
Nice to talk to you. Cheers.
Pat ShaughnessyPat Shaughnessy writes a blog about Ruby development and recently self-published an eBook called Ruby Under a Microscope. When he's not at the keyboard, Pat enjoys spending time with his wife and two kids. Pat is also a fluent Spanish speaker and travels frequently to Spain to visit his wife's family.
