Friday, March 07, 2008

Ruby: expectations gem version 0.2.3

I'm very opinionated about unit testing. I believe that you should only have one assertion per test which includes having only one expectation per test. I also believe that tests should be as clear as possible, and doing things such as inlining setup and expecting literals will result in more maintainable tests.

I've been an Xunit fan for years. I can do everything I want using Xunit frameworks, but at times the framework gets in my way. For example, I generally don't need a test name, which is really nothing more than a glorified comment. I also dislike having different syntaxes for state based and behavior based tests. Most importantly it enables other people to make questionable decisions.

On Christmas 2007 I released my opinionated unit testing framework: expectations.

Expectations has one syntax for both state based and behavior based tests. Expectations has no test name, if you need a comment, you add a real comment, if you don't you aren't required to. Expected values are specified outside the scope of the test, which encourages you to use literals whenever possible. Expectations has no support for setup or teardown, it forces you to duplicate code or write more loosely coupled code that requires less setup -- which is a good thing.

Some tests aren't a good fit for expectations, that's where a functional test suite comes into play. I've used both test/unit and RSpec for functional testing, both have benefits and provide you the ability to break every suggestion expectations provides.

I've been using expectations on my current project and all my open source projects for about 3 months. The framework has grown in many ways in those 3 months, the rest of this post is about the features that are currently supported.

Expectations supports traditional state based tests. The result of executing the block is compared with the expected value and if they are equal the test passes. The resulting tests are very easy to follow. The expected value is obvious in the first line, all but the last line in the block are setting up the test and the last line is the actual value.

expect 2 do
1 + 1
end

Expectations also supports behavior based tests by setting expectations on objects. The object can be a concrete class, a stub or a mock, it doesn't matter, they all support behavior based expectations (with the same syntax).

# Behavior based test using a traditional mock
expect mock.to.receive(:dial).with("2125551212").times(2) do |phone|
phone.dial("2125551212")
phone.dial("2125551212")
end

# Behavior based test using a stub
expect stub.to.receive(:dial).with("2125551212").times(2) do |phone|
phone.dial("2125551212")
phone.dial("2125551212")
end

# Behavior based test using a stub_everything
expect stub_everything.to.receive(:dial).with("2125551212").times(2) do |phone|
phone.dial("2125551212")
phone.dial("2125551212")
end

# Behavior based test on a concrete mock
expect Object.to.receive(:deal) do
Object.deal
end

Expectations also supports asserting an exception will be thrown.

expect NoMethodError do
Object.no_method
end

Expectations generally uses == internally to test equality; however, expectations also supports case equality (===) for Ranges, Regexps, and Modules.

# State based test matching a Regexp
expect /a string/ do
"a string"
end

# State based test checking if actual is in the expected Range
expect 1..5 do
3
end

# State based test to determine if the object is an instance of the module
expect Enumerable do
[]
end

Every expectation that uses case equality also uses regular equality, so the following tests also pass.

# State based test matching a Regexp
expect /a string/ do
/a string/
end

# State based test checking if actual is in the expected Range
expect 1..5 do
1..5
end

# State based test to determine if the object is an instance of the module
expect Enumerable do
Enumerable
end

Expectations also introduces a fluent interface for asserting boolean values. The following tests verify that the attribute is set to true or false, based on the expectation definition.

# this is normally defined in the file specific to the class
klass = Class.new do
attr_accessor :started, :finished
end

# State based fluent interface boolean tests using to be
expect klass.new.not.to.have.started

expect klass.new.to.be.started do |process|
process.started = true
end

# State based fluent interface boolean test using to have
expect klass.new.not.to.have.finished

expect klass.new.to.have.finished do |process|
process.finished = true
end

The above example also shows expectations that exist without using a block. These are fully functional and can be run individually within TextMate (snippet for running individual expectations is in the README).

If you have opinions like mine, you may want to give expectations a shot. If you disagree, that's cool too, to each their own.

2 comments:

  1. I see some wisdom in your ideas here, but I have a slightly different take on this issue. I prefer to make one unit test per development task. So, when I pick a story card off the task board at the daily meeting I write a unit test, with multiple assertions, to cover that card and I name it after the card it tests. This maintains full requirements traceability with minimal process overhead.

    ReplyDelete
  2. Anonymous2:02 PM

    Hello Mark,
    I think that's a pretty cool approach. I don't think expectations goes against that approach, instead I think it augments it.

    I would start where you are starting, probably using RSpec, but I would consider that a functional test. Then I would work on making each assertion pass one at a time. However, I'm likely to need other code while I'm making those assertions pass, for those situations I'd probably use expectations to write very focused tests.

    Thanks for the comment.

    Cheers, Jay

    ReplyDelete

Note: Only a member of this blog may post a comment.