Rails approach to testing
I posted this originally on the Rails mailing list but I thought I'd post it here as I know there are people here who also spend plenty of time in the PHP Application Design forum and this is the sort of thing that wouldn't feel out of place there.
Sorry if this comes across as a bunch of disjointed thoughts...I'm just trying to put my thoughts down and I'd like to know what other people's opinions on the subject are.
I've been working with Rails for 3 or 4 months now and I'm constantly trying to look at ways of improving my testing techniques, especially when it comes to TDD. I have a few concerns about the way Rails approaches tests with regards to controllers/models.
Rails has always seemed to apply that you unit test your models and functional test your controllers. For me however, this distinction breaks down in practice and leads to what are in my opinion very smelly tests.
Firstly, I don't like the way that by default Rails assumes you want to test an entire controller in a testcase. I've found that this leads to tests that are far to big. I believe better breakdown would be to have a test case for each action, or even a testcase for each action in different contexts. So, as an example, say you had an ArticleController with view, post and create actions. The post action simply displays the form which submits to the create action.
Now, instead of having one large ArticleControllerTest, it would make far more sense to break it down into ArticleViewTest, ArticlePostTest, FailedArticleCreate test and SuccessfulArticleCreate, because each of those represent different contexts, or fixtures.
I'm quite keen on Ben Griffiths' testing approach outlined here and have also been looking into Dave Astels' RSpec and have to agree with the idea of testcases being bound by fixtures. If you have test methods that require common setups, the code should be extracted to the setup method. If you have different groups of methods using two different types of setups, this is an indication that you should be creating a new testcase for the new fixture. Overall, I've found that trying to test a whole controller leads to multiple fixtures in one testcase and multiple violations of DRY.
This also leads me on to my other concern, and that is the practice of testing for the presence of markup in a view using assert_tag. I can't really see the value in this, and again it seems to violate DRY to me. Why would you want to assert that a tag exists in most cases? For example, given the ArticlePostTest example above, which wouldn't really do much in the controller, and simply pass to the view for the rendering of the form. Why would I want to test for the structure of the form? Ultimately my form will be laid out using CSS but there are still several ways I could markup my form, with all variations being "correct". By testing for one particular implementation in my test, I'm duplicating the markup in my test and my view, and binding the test to my implementation of the form. Testing that a particular HTML element is present in my view does nothing to tell me that my view is "correct", therefore I think this makes the test worthless.
Which leads me on to saying that views such as forms are the realm for proper functional testing using something like Watir or Selenium, not TestUnit and tag assertions. You want to check that you can fill in the form and submit it, not that the page contains a <form> tag.
Another difference this brings to mind is that functional tests (or Acceptance Tests as XP calls them) should not be written test-first, but are instead defined at the beginning of an iteration for a series of user stories. Acceptance tests do not have to run at 100% all the time (in fact they start at 0% and approach 100% as the iteration nears completion), wherease Unit Tests should always be running at 100%. Acceptance tests represent what the completed system should do when finished, whereas Unit Tests represent what your code actually does.
Its perfectly possible to design controller actions test-first, but this is still unit testing. Acceptance testing an action would be to load it in your browser, fill in the form, submit it, and check it worked. This is why I mention things like Watir or Selenium for acceptance testing over TestUnit. FIT would be another option.
This is just more evidence to me, that Rails' distinction of unit tests for models and functional tests for controllers, is wrong.
So what would I really like to see? No separation of unit and functional testing in terms of model and controller code. They should all be covered by unit tests. Functional testing should be the realm of something like Watir or Selenium, unless somebody can put together a good library that lets you do this kind of functional testing ( i.e., filling in forms, interacting with your web application as if it were in a browser) from within your TestUnit tests. I'd like to see less emphasis on the idea of testing your controllers as a whole and testing your different actions as smaller, focussed unit tests. I think this would also aid with TDD.
Dave Astels' has some good reading material here:
What do you think?