Showing posts with label behavior based. Show all posts
Showing posts with label behavior based. Show all posts

Tuesday, March 19, 2013

Clojure: Expectations Interaction Tests For Java Objects

I recently ran into some code that forced me to integrate with a Java library. While using the library I found myself wanting to do a bit of interaction testing, which I've historically done with Mockito. As a result, I added the ability to do interaction based tests on mock Java objects, directly in expectations.

Hopefully the code is what you'd expect.


The previous example creates a mock Runnable in an expect-let, expects the run method to be run, and then calls the run method of the mock. This test is worthless in a real world context, but it's the simplest way to demonstrate the syntax for creating a mock & specifying the interaction.

The mock function defined in erajure, a minimal wrapper around mockito. All of the "times" arguments are the same as what's available for function interaction tests, examples can be found here.

Tuesday, January 15, 2013

Clojure: Expectations Verify Interaction Args

The expectations framework provides the ability to create interaction (or behavior) based tests. I've previously written about adding interaction based testing to expectations; however, the examples from that blog entry focused exclusively on testing interactions where each argument is matched using equality. In this entry I'll give examples of how each argument can be also be verified using a class, regex, exception, or a custom function.

When writing state based tests using expectations the type of test you're writing is inferred from the expected value. If the expected value is a regex, expectations will test the actual value to see if it matches the regex. If you passed in a class, expectations will test the actual value to see if it's an instance of that class. If you passed in an exception... you get the idea. All of what I said above, is also true for arguments of an interaction.

Let's start with a simple interaction based test:


In the example above, we're calling the spit function with exactly the arguments that we've specified in our test. This test will pass; however, we've had to specify the exact file location and the exact data. If for some reason you can't specify exactly what the argument will be, it's nice to have a way to specify as much as you possibly can.

In the example below, we're still specifying the exact data, but we're only verifying that the file is somewhere in /tmp/.


As I previously mentioned, we can also get more general and only verify the class of an argument. For example, if we knew our data was going to be a String, but we didn't want to specify exactly what that string was, the following test would do the trick.


While expectations provides you with a lot of default options, there are times when you'll want to write your own argument "matcher". As a contrived example, let's pretend that we want to test that the last argument is true or nil.


One of the best features of expectations is it's error reporting, and the same error reporting logic is applied to arguments when an interaction based test fails. Given the example above, you'll get the following error message.
failure in (success_examples.clj:204) : success.success-examples
           expected: (spit #"/tmp/" String :append true-or-nil?) 
                got: 0 times 

           -- got: (spit "/tmp/somewhere-else" "nil")
           "nil", "/tmp/somewhere-else" are in actual, but not in expected
           true_or_nil_QMARK, #"/tmp/", :append, String are in expected, but not in actual
           expected is larger than actual 

           -- got: (spit "/tmp/hello-world" "some data" :append "s")
           - arg4: not true or nil
As you can see both calls are reported, and each argument has a detailed report (if it did not match).

Finally, expectations provides and additional function that can be used to verify that certain key/value pairs are in an argument. The following example doesn't really make sense, since you'd never want to pass a map as the last argument to spit, but it's easy to follow in the context of this blog entry.


In the above example, (contains-kvs) is used to verify that the final argument to spit contains the key/value pairs :a :b :c :d.

I hope that interaction arg matching follows the principle of least surprise, since it behaves the same as expectations state based tests. I also hope that the ability to use an arbitrary function for verification will provide any necessary flexibility. If you're using expectations, give it a try and let me know.

Thursday, November 01, 2012

Clojure: Interaction Based Testing Added To expectations

The vast majority of testing I do these days is state-based; however, there are times when I need to test an interaction (e.g. writing to a file or printing to standard out). The ability to test interactions has been in expectations.scenarios for quite awhile, but there isn't any reason that you need a scenario to test an interaction - so, as of version 1.4.16, you also have the ability to test interactions with bare expectations.

The following test shows how you can specify an expected interaction. This test passes.


Writing the test should be straightforward - expect the interaction and then call the code that causes the interaction to happen.

As I was adding this behavior I enhanced the error reporting. Below you can find a failing test and the output that is produced.


As you can see, all three calls to the 'one' function are reported. If the number of args used to call 'one' are of the same size as the expected args, each arg is compared in detail; otherwise the two lists are compared in detail (but the elements are not).

As you can see in this failure the first argument, "hello", matches.
 ;          got: (one "hello" {2 3, :a 1})
 ;                   arg1: matches
 ;          expected arg2: {:a :b, :c {:ff :gg, :dd :ee}}
 ;            actual arg2: {2 3, :a 1}
 ;          2 with val 3 is in actual, but not in expected
 ;          :c {:dd with val :ee is in expected, but not in actual
 ;          :c {:ff with val :gg is in expected, but not in actual
 ;          :a expected: :b
 ;                  was: 1
Anytime an argument matches expectations will simply print "matches". You can also specify :anything as an argument, to ignore that argument and always 'match'. The following test shows an example of matching the second argument, while the first argument is no longer matching.


That's it. Hopefully these interaction tests follow the principle of least surprise, and are easy for everyone to use.

Sunday, February 03, 2008

Behavior Based Testing

Behavior based testing is when you expect specific interactions to occur between objects when certain methods are executed. Behavior based testers (Mockists) utilize mocks to verify expected interaction between objects of a system. Behavior based testing is about specifying how the system should behave rather than specifying the expected result of running the system.

Behavior based testing has been criticized by state based advocates because it can cause test failures when implementation changes are made. In it's worst form, behavior based testing can be fragile. However, behavior based tests that avoid high implementation specification can effectively verify behavior without sacrificing maintainability.

The following test verifies the interaction between a Hotel instance and the ReservationService without the need to specify any other implementation detail. A change to book_for method will only cause the test to fail if the interaction between the two objects is changed, in which case a failing test is appropriate.

require 'test/unit'
require 'rubygems'
require 'mocha'

class ReservationService
# implementation
end

class HotelRoom
attr_accessor :booked
def book_for(customer)
reservation = ReservationService.reserve_for(customer, self)
self.booked = true
MaidService.notify(reservation) if reservation.tomorrow?
VipService.notify(reservation) if reservation.for_vip?
reservation.confirmation_number
end
end

class HotelRoomTests < Test::Unit::TestCase
def test_book_for_reserves_via_ReservationService
room = HotelRoom.new
ReservationService.expects(:reserve_for).with(:customer, room).returns(stub_everything)
room.book_for(:customer)
end
end

Behavior based tests are easy to spot because they focus on mock verification to ensure a system behaves as expected.

Developers who generally rely on behavior based tests can be considered Mockists. Martin Fowler has a great article, Mocks Aren't Stubs, that expands greatly on the ideas that are briefly mentioned here.