SitePoint Sponsor

User Tag List

Results 1 to 6 of 6
  1. #1
    SitePoint Guru silver trophy Luke Redpath's Avatar
    Join Date
    Mar 2003
    Location
    London
    Posts
    794
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Rails approach to testing

    I posted this originally on the Rails mailing list but I thought I'd post it here as I know there are people here who also spend plenty of time in the PHP Application Design forum and this is the sort of thing that wouldn't feel out of place there.

    Sorry if this comes across as a bunch of disjointed thoughts...I'm just trying to put my thoughts down and I'd like to know what other people's opinions on the subject are.

    I've been working with Rails for 3 or 4 months now and I'm constantly trying to look at ways of improving my testing techniques, especially when it comes to TDD. I have a few concerns about the way Rails approaches tests with regards to controllers/models.

    Rails has always seemed to apply that you unit test your models and functional test your controllers. For me however, this distinction breaks down in practice and leads to what are in my opinion very smelly tests.

    Firstly, I don't like the way that by default Rails assumes you want to test an entire controller in a testcase. I've found that this leads to tests that are far to big. I believe better breakdown would be to have a test case for each action, or even a testcase for each action in different contexts. So, as an example, say you had an ArticleController with view, post and create actions. The post action simply displays the form which submits to the create action.

    Now, instead of having one large ArticleControllerTest, it would make far more sense to break it down into ArticleViewTest, ArticlePostTest, FailedArticleCreate test and SuccessfulArticleCreate, because each of those represent different contexts, or fixtures.

    I'm quite keen on Ben Griffiths' testing approach outlined here and have also been looking into Dave Astels' RSpec and have to agree with the idea of testcases being bound by fixtures. If you have test methods that require common setups, the code should be extracted to the setup method. If you have different groups of methods using two different types of setups, this is an indication that you should be creating a new testcase for the new fixture. Overall, I've found that trying to test a whole controller leads to multiple fixtures in one testcase and multiple violations of DRY.

    This also leads me on to my other concern, and that is the practice of testing for the presence of markup in a view using assert_tag. I can't really see the value in this, and again it seems to violate DRY to me. Why would you want to assert that a tag exists in most cases? For example, given the ArticlePostTest example above, which wouldn't really do much in the controller, and simply pass to the view for the rendering of the form. Why would I want to test for the structure of the form? Ultimately my form will be laid out using CSS but there are still several ways I could markup my form, with all variations being "correct". By testing for one particular implementation in my test, I'm duplicating the markup in my test and my view, and binding the test to my implementation of the form. Testing that a particular HTML element is present in my view does nothing to tell me that my view is "correct", therefore I think this makes the test worthless.

    Which leads me on to saying that views such as forms are the realm for proper functional testing using something like Watir or Selenium, not TestUnit and tag assertions. You want to check that you can fill in the form and submit it, not that the page contains a <form> tag.

    Another difference this brings to mind is that functional tests (or Acceptance Tests as XP calls them) should not be written test-first, but are instead defined at the beginning of an iteration for a series of user stories. Acceptance tests do not have to run at 100% all the time (in fact they start at 0% and approach 100% as the iteration nears completion), wherease Unit Tests should always be running at 100%. Acceptance tests represent what the completed system should do when finished, whereas Unit Tests represent what your code actually does.

    Its perfectly possible to design controller actions test-first, but this is still unit testing. Acceptance testing an action would be to load it in your browser, fill in the form, submit it, and check it worked. This is why I mention things like Watir or Selenium for acceptance testing over TestUnit. FIT would be another option.

    This is just more evidence to me, that Rails' distinction of unit tests for models and functional tests for controllers, is wrong.

    So what would I really like to see? No separation of unit and functional testing in terms of model and controller code. They should all be covered by unit tests. Functional testing should be the realm of something like Watir or Selenium, unless somebody can put together a good library that lets you do this kind of functional testing ( i.e., filling in forms, interacting with your web application as if it were in a browser) from within your TestUnit tests. I'd like to see less emphasis on the idea of testing your controllers as a whole and testing your different actions as smaller, focussed unit tests. I think this would also aid with TDD.

    Dave Astels' has some good reading material here:

    http://blog.daveastels.com/?p=53

    What do you think?

  2. #2
    ☆★☆★ silver trophy vgarcia's Avatar
    Join Date
    Jan 2002
    Location
    in transition
    Posts
    21,236
    Mentioned
    1 Post(s)
    Tagged
    1 Thread(s)
    Good points in there, but as a newbie to test-driven development I'll say that some of what you wrote went way over my head

  3. #3
    SitePoint Guru silver trophy Luke Redpath's Avatar
    Join Date
    Mar 2003
    Location
    London
    Posts
    794
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    To be honest, I wouldn't consider myself an experienced TDDer by a long shot, and if what most of what I've written is b**locks then I welcome an experienced TDDer to correct me.

  4. #4
    SitePoint Addict Brak's Avatar
    Join Date
    Jul 2004
    Location
    Central Coast, CA
    Posts
    321
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I can answer one of your questions: assert_tag.

    I don't know what your background is, but personally I'm a designer (with moderate programming experience) that mostly works with programmers (not with rails, btw). You know what the most common response from a programmer is when something's wrong? "It works. Just try it again." 90% of the time they're wrong.

    assert_tag is a no-nonsense way to make sure your app is working right. Make sure that error message appeared. Really. Make sure the results table appeared. Really. Make sure the results table didn't appear if there's no results. Really.

    When you test variables' values it can often be misleading since that doesn't mean you're testing whether it appeared on the front-end or not. assert_tag is a test's eyes to the world.

    -----

    On the subject of functional tests, I think you might be mistaking what you should be testing in functional tests. Functional tests apply to controllers - but merely because they're the glue between the models and the view and provide the data for the view. What you're *really* testing is the view, but say that to a programmer and they'll snort their nose at you.

    Any kind of logic or data interaction should be done in the model anyway. I'd be confused what kind of unit tests you'd be doing on controllers... can you give some examples?
    Studio Rockstar's Blog - A journey to quitting the dayjob.

  5. #5
    SitePoint Guru silver trophy Luke Redpath's Avatar
    Join Date
    Mar 2003
    Location
    London
    Posts
    794
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by Brak
    Any kind of logic or data interaction should be done in the model anyway. I'd be confused what kind of unit tests you'd be doing on controllers... can you give some examples?
    I think you are mistakenly assuming that model == unit. The concept of what is a unit is often discussed but to me its a simple, self-contained unit of functionality, which doesn't nessecarily involve 1 class, or one method - it could be many.

    Say you have a create() action on your controller that takes input from the request parameters (say a POST) and creates a new record.

    So what would the unit test look like? Well there are at least two possible fixtures here...a post containing valid data (leading to a successful response) or a post containing invalid data (leading to a failed response). You might want to break the failed fixtures down further. The success fixture would probably be something along the lines of:

    Code:
    class SuccessfulArticleCreateTest < Test::Unit::TestCase
    
      def setup
        # set up the fixture here
        @request = ActionController::TestRequest.new
        @response = ActionController::TestResponse.new
        @controller = ArticleController.new
        @post_data = {
          :title => 'Some valid title',
          :body => 'Some valid body'
        }
        post :create, @post_data
      end
    
      def test_should_create_new_record
        assert_equal 'Some valid title', assigns('new_record').title
        assert_equal 'Some valid body', assigns('new_record').body
        assert_equal 'Article successfully created', flash[:success]
      end
    
      def test_should_redirect_to_article_display
        assert_redirected_to :controller => 'article', :action => 'view', :id => assigns('new_record').id
      end
    
      def test_should_store_new_article_in_database
        # etc...
      end
    end
    Its still a unit test to me. Also, if you combine the testcase name with the test methods (excluding the "text_" part) you'll see they read just like specifications...

    Successful Article Create should store new article in database
    Successful Article Create should redirect to article display
    etc...

  6. #6
    SitePoint Addict Brak's Avatar
    Join Date
    Jul 2004
    Location
    Central Coast, CA
    Posts
    321
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Ah yes Luke, I think what you're running into here is simply a differnece in opinion (Rails is opinionated software).

    Your approach seems completely valid (in fact, few fully developed test suite's unit tests only handle one model). There shouldn't be any restrictions to how tests are run. You're more than welcome to create custom unit test files and run them that way. There's just some pre-built magic that helps those who chose to do it the rails-way a bit faster.
    Studio Rockstar's Blog - A journey to quitting the dayjob.


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •