Tuesday, September 05, 2006

Ruby/Rails Unit Testing in less than 1 second

I previously wrote about removing the database dependency from Ruby on Rails Unit Tests. I'm pleased to say that several months later my team continues to write unit tests that do not require database access. As a result, our unit tests run quite quickly:
Loaded suite /opt/local/lib/ruby/gems/1.8/gems/rake-0.7.1/lib/rake/rake_test_loader
Finished in 0.555897 seconds.

132 tests, 219 assertions, 0 failures, 0 errors
However, there is another factor that contributes to our test time: Stubba and Mocha. Stubba and Mocha have become tools that I would not want to live without.

Mocha is my new favorite mocking framework. Mocha's syntax is intuative and the mocks are auto-verified.
def test_execute(command='sql string', connection=mock)
SqlExecutor.execute(command, connection)
Stubba is similar to Mocha except it adds mock and stub capabilities to concrete classes.
def test_execute(command='sql string')
Stubba can also be used for easy white box unit testing of specific methods. Stubba allows you to mock some methods of a concrete class while testing the actual implementations of the other methods.
def test_parse(text='any text')
Both Stubba and Mocha allow our tests to be concise and descriptive while increasing test performance.

For more information on removing the database dependency check out this entry and for more info on Stubba and Mocha check out the RubyForge page.


  1. Thanks for the positive feedback on Mocha. Your readers might find this article useful in getting started.

  2. I have found when I run ruby tests my tests complete very quickly, 3 secs, but actually getting the ruby instance up takes much longer (12 secs). My total test run time while reported as 3 secs actually is taking 15-20. Is there something I am missing about how to do ruby testing where you can keep a test instance up and keep using it, or improve ruby load time?


  3. The amount of time to run a test seems arbitrary when compaired to test readability.

    Mocking out something as simple as a model seems backwards if all that model is doing is holding state.

    Note: I agree with mocks for complex objects

  4. Ryan S.4:37 PM

    Jay, another great article. One stumbling block I see in your quest to remove database dependency from unit testing is with complex AR::Base#find() methods.

    Imagine we're providing the user a way to search through models. Also imagine we want to :join with other tables, :select on custom attributes, provide :limits and :offsets for pagination support, and other options that add complexity to the search.

    Now we could use Mocha to provide white-box testing to verify AR::Base#find is called in the method, and that the parameters are correct. But how can Mocha determine if we have a SQL syntax error in our :join or :select or :limit or :offset statements without actually running the query?

  5. Ryan,

    I'd probably test complex find methods in functional tests (where the AR subclasses hit the database)

    If you truly want to test the SQL that is generated by the AR find methods you can mock the "execute" method and expect the SQL that you think AR should create. I'm not suggesting that, but it is possible to test in that manner.

    Cheers, Jay

  6. Ryan S.8:41 PM

    That would be an interesting choice -- to write functional tests on find() methods, that would be encapsulated in the model. I think I respectfully disagree with you there. IMO, functional tests should be used to verify proper communication from the view to the model only. Also, if any method is encapsulated in the model, then there is a good chance it can and will be used in multiple controllers. I believe that testing model operations should be the job of unit tests alone.

    Now in the case of functional testing the actions that call these complex find() methods, Mocha would be very useful. Here, we're expecting the model the call this method, and we want it to return some set of AR subclassed objects. There is no need to go down to the database in this case.

    Either way, I agree with you 90% of the time. I'm sure MOST model operations, where rails would hit the database, can and should be mocked out. But there are some operations, like methods that build complex queries, explicitly constructed by the programmer, that ought to be relayed to the database and not mocked.

  7. thanks ..
    it looks cool.:O)

  8. We employed a powerful DB unit testing tool in our orgnization, it named as AnyDbTest. (www.anydbtest.com). By using this tool, we need NOT write Java/.Net/Ruby DB unit testing code any more. We can configure one xml file to tell what we want to test and what is our assertion. Then AnyDbTest can do the rest things.

    Additionally, we can versioned control the xml-style test case just like the code files. It is very handy.


Note: Only a member of this blog may post a comment.