Testing: Frameworks are evolving

Test-Driven Development was introduced to me by way of xUnit frameworks.

In the beginning, all xUnit frameworks looked very much the same. All tests lived within a class that inherited from an abstract test case class, and all tests were defined by creating a method that begin with the word test. However, as time progressed the xUnit frameworks began to take advantage of features specific to individual languages.

Martin Fowler's bliki entry Xunit contains a description of how NUnit matured.

The first version of NUnit even had a "isVAforJava" method which originated in special handling for Visual Age for Java. Others have been more sophisticated: NUnit 2.0 was praised by Anders Heljsberg for its use of attributes in C#.

Utilizing language specific features was the first step; however, as testing methods mature it's becoming more clear that the features of xUnit also need to mature.

The first framework I noticed that decided to start breaking rules was RSpec. The 1.0 version of RSpec provided a new syntax for defining test cases and for defining assertions. The following example shows how you would write the same test in Test::Unit (a traditional xUnit framework) and RSpec.

# RSpec 
describe Bowling do
  before(:each) do
    @bowling = Bowling.new
  end

  it "should score 0 for gutter game" do
    20.times { @bowling.hit(0) }
    @bowling.score.should == 0
  end
end

# Test::Unit
class BowlingTest < Test::Unit::TestCase
  def setup
    @bowling = Bowling.new
  end
  
  def testShouldScoreZeroForGutterGame
    20.times { @bowling.hit(0) }
    assert_equal 0, @bowling.score
  end
end

I do believe that the RSpec version is more semantically pleasant. However, given a programmer who practices TDD, when they maintain tests they use an xUnit clone, then I'm not sure the semantic benefit is worth learning a new framework. At least, I wasn't at first.

Another new xUnit framework is xUnit.net, announced recently on James Newkirk's blog. xUnit.net excites me more than any other piece of software that I'm likely to never use. I'll probably never use it since I don't plan on going back to .net development, but the features are exciting nonetheless. James et al have taken their test-driven development experiences and created a framework that guides them towards writing better tests. James' entry gives the full details (which I suggest reading even if you never plan on doing any .net work), but a few features I like are the removal of setup and teardown, and the aspect-like functionality. You may be less impressed since Ruby provides aspect-like functionality very easily, but I believe it's nice to see a xUnit framework built with extension points in mind.

These new frameworks are exciting because they are incorporating lessons learned from the past few years of practicing test-driven development. As our experience with TDD grows, so should our tools.