Sunday, February 03, 2008

Behavior Based Testing

Behavior based testing is when you expect specific interactions to occur between objects when certain methods are executed. Behavior based testers (Mockists) utilize mocks to verify expected interaction between objects of a system. Behavior based testing is about specifying how the system should behave rather than specifying the expected result of running the system.

Behavior based testing has been criticized by state based advocates because it can cause test failures when implementation changes are made. In it's worst form, behavior based testing can be fragile. However, behavior based tests that avoid high implementation specification can effectively verify behavior without sacrificing maintainability.

The following test verifies the interaction between a Hotel instance and the ReservationService without the need to specify any other implementation detail. A change to book_for method will only cause the test to fail if the interaction between the two objects is changed, in which case a failing test is appropriate.

require 'test/unit'
require 'rubygems'
require 'mocha'

class ReservationService
# implementation
end

class HotelRoom
attr_accessor :booked
def book_for(customer)
reservation = ReservationService.reserve_for(customer, self)
self.booked = true
MaidService.notify(reservation) if reservation.tomorrow?
VipService.notify(reservation) if reservation.for_vip?
reservation.confirmation_number
end
end

class HotelRoomTests < Test::Unit::TestCase
def test_book_for_reserves_via_ReservationService
room = HotelRoom.new
ReservationService.expects(:reserve_for).with(:customer, room).returns(stub_everything)
room.book_for(:customer)
end
end

Behavior based tests are easy to spot because they focus on mock verification to ensure a system behaves as expected.

Developers who generally rely on behavior based tests can be considered Mockists. Martin Fowler has a great article, Mocks Aren't Stubs, that expands greatly on the ideas that are briefly mentioned here.

State Based Testing

State based testing is when you exercise one or many methods of an object and then assert the expected state of the object. Many test-driven developers prefer state based tests because they specify as little implementation detail as possible. Tests that specify no implementation details should continue to pass even if the implementation of the methods being tested are changed. The following state based delegation test will continue to be valid for either of the following implementations.

require 'test/unit'

class ShowPresenterTests < Test::Unit::TestCase
def test_delegation_of_name_to_employee
employee = Employee.new("Shane Harvie")
assert_equal "Shane Harvie", ShowPresenter.new(employee).name
end
end

class Employee < Struct.new(:name); end

class ShowPresenter < Struct.new(:employee)
def name
employee.name
end
end

require 'forwardable'
class ShowPresenter < Struct.new(:employee)
extend Forwardable
def_delegator :employee, :name
end

State based tests can contain stubs, but they are generally avoided where possible. State based tests are easy to spot because they rely on assertions to verify the state of the application.

Developers who generally rely on state based tests can be considered Classicists. Martin Fowler has a great article, Mocks Aren't Stubs, that expands greatly on the ideas that are briefly mentioned here.

Saturday, February 02, 2008

Ruby: Did TDD make Ruby a viable option?

Some people ask if Test Driven Development (TDD) is what caused Ruby to become popular. Other people smarter than me claim that TDD is in fact the reason that dynamic languages are now viable options.

I'm sorry, but I disagree.

Can you write a non-trivial application in Ruby without tests and have confidence in it? No. Can you write a non-trivial application in Java without tests and have confidence in it? Again, no.

A compiler lets you know that some things are correct, but should not give you confidence that your application behaves as expected. A trivial example is, I need to know that the calculate_tax (that's calculateTax for you Java fans) method returns 56, not that the result is a Fixnum (or Integer in Java).

As much as it should be, TDD is not mainstream. There's no TDD Conference, but RailsConf sells out every year.

I would actually credit David Heinemeier Hansson with making Ruby viable by creating a framework that drove mass adoption. But, I bet of you looked at the majority of Rails applications you would find empty test folders (or only the generated tests, which are never run). I'm quite sure that's true because I expect the conferences to attract the best of the Ruby developers, and several of the people I talk to at those conferences "simply don't have time" to write tests.

Testing: Mocha's stub_everything method

The Mocha mocking framework provides a method, stub_everything, that returns an object that will respond 'nil' to any message sent to it. The stub_everything method is indispensable for creating an object that is needed for testing, but the interactions are completely unimportant.

For example, in the following test the ReservationService::reserve_for method needs to return a duck that responds to the same methods that are defined on the (not shown) Reservation class. However, the focus of the test has nothing to do with the behavior of the reservation duck, so stub_everything is the perfect duck because it requires no additional set up and responds to all the methods required without error.

require 'test/unit'
require 'rubygems'
require 'mocha'

class ReservationService
# implementation
end
class MaidService
# implementation
end
class VipService
# implementation
end

class HotelRoom
def book_for(customer)
reservation = ReservationService.reserve_for(customer, self)
MaidService.notify(reservation) if reservation.tomorrow?
VipService.notify(reservation) if reservation.for_vip?
reservation.confirmation_number
end
end

class HotelRoomTests < Test::Unit::TestCase
def test_book_for_reserves_via_ReservationService
room = HotelRoom.new
ReservationService.expects(:reserve_for).with(:customer, room).returns(stub_everything)
room.book_for(:customer)
end
end

The stub_everything method is also nice because it can take parameters. For example you may want to test that the book_for method returns a confirmation number. The following test specifies the value of the confirmation_number value, but it uses stub_everything so that each other method call will return nil (without causing an error).

require 'test/unit'
require 'rubygems'
require 'mocha'

class ReservationService
# implementation
end
class MaidService
# implementation
end
class VipService
# implementation
end

class HotelRoom
def book_for(customer)
reservation = ReservationService.reserve_for(customer, self)
MaidService.notify(reservation) if reservation.tomorrow?
VipService.notify(reservation) if reservation.for_vip?
reservation.confirmation_number
end
end

class HotelRoomTests < Test::Unit::TestCase
def test_book_for_reserves_via_ReservationService
reservation = stub_everything(:confirmation_number => "confirmation number")
ReservationService.stubs(:reserve_for).returns(reservation)
assert_equal "confirmation number", HotelRoom.new.book_for(:customer)
end
end

Another usage of stub_everything that might not be immediately apparent is that it can be used as both a stub and a mock. The following test creates a stub and then sets an expectation on it. As a result, the expectation is verified (which is the focus of the test) and all other calls to the same object simply do nothing.

require 'test/unit'
require 'rubygems'
require 'mocha'


class HotelTests < Test::Unit::TestCase
def test_name_is_set_from_options
hotel = stub_everything
hotel.expects(:name=).with "Marriott"
Hotel.stubs(:new).returns hotel
Hotel.parse("Marriott||")
end
end

class Hotel
def self.parse(attributes)
hotel = self.new
hotel.name, hotel.location, hotel.capacity = attributes.split("|")
hotel
end
end

Using the stub_everything method of mocha can remove noise from tests and also make them more robust by ignoring all messages that are unimportant to the current test.

Testing: High Implementation Specification Smell

At the end of my last blog entry, I touched briefly on the testing smell: High Implementation Specification. I got a fair amount of good feedback and based on several requests I decided it was worth creating a blog entry focused entirely on the smell.

Well written tests specify as little implementation detail as possible.

Developers new to mocking and stubbing tend to overuse the capabilities provided by mocking frameworks. I consider each call to a method of a mocking framework to be 1 implementation specification point. While using mocha I count a point for each of: stub, stub_everything, stubs, mock, expects, with, and returns. Any test that has an implementation specification score of 5 or more has a high implementation specification smell.

High Implementation Specification can stem from misusing mocking frameworks in various ways such as verifying parameters while stubbing methods, returning values that are never used, and specifying every interaction instead of using an object that will respond to any message.

Most mock frameworks provide the ability to verify the parameters sent to a mocked or stubbed method. When setting up a mock expectation, specifying the parameters can be useful for verifying interaction; however, when stubbing methods that are not the focus of the test, adding the additional parameter verification makes the test unnecessarily brittle.

The following code provides an example where the MainService notify method stub could verify the parameters, but the test is focused on testing the ReservationService. Given that the test is verifying the ReservationService there's no need for it to return a value and there's no need to verify the parameters sent to MaidService.notify. If the test did specify the parameters send to MaidService it would fail if those parameters changed. Since the test is verifying ReservationService, it would be disappointing if it failed due to a change in how MadService is called.

require 'test/unit'
require 'rubygems'
require 'mocha'

class ReservationService
# implementation
end
class MaidService
# implementation
end

class HotelRoom
attr_accessor :booked
def book_for(customer)
reservation = ReservationService.reserve_for(customer, self)
MaidService.notify(reservation)
end
end

class HotelRoomTests < Test::Unit::TestCase
def test_book_for_reserves_via_ReservationService
room = HotelRoom.new
ReservationService.expects(:reserve_for).with(:customer, room)
MaidService.stubs(:notify)
room.book_for(:customer)
end
end

Returning a value from a mocked method can be necessary, but should be avoided in general for two reasons: Returning a value signifies to someone reading a test that the value is necessary for some reason. Returning a value that is later verified with an assertion can cause false positives.

In the above example, the ReservationService does not provide a return value because that value isn't used anywhere within the test. If you read the above test and saw that a return value was specified you would probably want to spend time finding where and why it was used. Worse, if the above test returned a value and then later asserted equality the test would pass, but the implementation could change the return value in the future and the test would not break.

At times a mock or stub must be exercised in various ways within one test due to how the original method was written. However, specifying all the interactions within one test leads to a test that breaks with every implementation change. A better solution is to utilize an object that can respond to anything and only specify the behavior that the test focuses on.

The following example uses a stub_everything object to ensure that the location and capacity setting methods can be called without being specified, but it also expects that the name is set correctly. Using a stub_everything also ensures that if more attributes are set in the future the test will continue to pass without any intervention.

require 'test/unit'
require 'rubygems'
require 'mocha'


class HotelTests < Test::Unit::TestCase
def test_name_is_set_from_options
hotel = stub_everything
hotel.expects(:name=).with "Marriott"
Hotel.stubs(:new).returns hotel
Hotel.parse("Marriott||")
end
end

class Hotel
def self.parse(attributes)
hotel = self.new
hotel.name, hotel.location, hotel.capacity = attributes.split("|")
hotel
end
end