In January of 2006, Martin Fowler wrote a quick blog entry that included definitions for the various types of test doubles. I've found working with those definitions to be very helpful.
Test Doubles: Mocks and Stubs
Mockito gives you the ability to easily create mocks and stubs. More precisely, I use Mockito 'mocks' for both mocking and stubbing. Mockito's mocks are not strict: They do not throw an exception when an 'unexpected' method is called. Instead, Mockito records all method calls and allows you to verify at the end of your test. Any method calls that are not verified are simply ignored. The ability to ignore method calls you don't care about makes Mockito perfect for stubbing as well.
Test Doubles: Dummies
I didn't find many cases where I needed a dummy. A dummy is really nothing more than a traditional mock (one that throws exceptions on 'unexpected' method calls) with no expected method calls. In cases where I wanted to ensure that nothing was called on an object, I usually used the verifyZeroInteractions method on Mockito mocks. However, there are exceptional cases where I don't want to verify that no methods were called, but I would like to be alerted if a method is called. This is more of a warning scenario than an actual failure.
On my current project we ended up rolling our own dummies where necessary and overriding (or implementing) each method with code that would throw a new Exception. This was fairly easy to do in IntelliJ; however, I'm looking forward to the next version of Mockito, which should allow for easier dummy creation.
Test Doubles: Fakes
My current project does make use of Fakes. We define a fake as an instance that works for and is targeted towards a single problem. For example, if your class depended on a Jetlang fiber you could pass in a fake Jetlang fiber that synchronously executed a command as soon as it was given one. The fake fiber wouldn't allow you to schedule tasks, but that's okay, it's designed to process synchronous requests and that's it.
We don't have a large amount of fakes, but they can be a superior alternative when compared to creating multiple mocks that behave in the same way. If I need a CreditCardProcessingGateway to return a result once, it's good to use a mock. However, if I need a CreditCardProcessingGateway to consistently return true when given a luhn valid credit card number (or false otherwise) a fake can be a superior option.
Concrete Classes
I'm a big fan of Nat Pryce's Test Data Builders for creating concrete classes. I use test data builders to create the majority of dependencies and collaborators. Test data builders also allow me to easily drop in a test double where necessary. The following example code shows how I'll use a test data builder to easily create a car object.
aNew().car().The (contrived) example demonstrates 4 difference concepts:
with(mock(Engine.class)).
with(fake().radio().fm_only()).
with(aNew().powerWindows()).build();
- A car can easily be built with a mock engine
- A car can easily be built with a fake fm radio
- A car can easily be built with a power windows concrete implementation
- All other dependencies will be sensible defaults
The builders are easy to work with because you know the default dependencies are concrete, instead of having to look at the code to determine if the dependencies are concrete, mocks, or fakes. This convention makes writing and modifying tests much faster.
You may have also noticed the aNew() and fake() methods from the previous example. The aNew() method returns a DomainObjectBuilder class and the fake() method returns a Faker class. These methods are convenience methods that can be staticly imported. The implementations of these classes are very simple. Given a domain object Radio, the DomainObjectBuilder would have a method defined similar to the example below.
public RadioBuilder radio() {This allows you to import the aNew method and then have access to all the test data builders in a code completeable manner. Keeping the defaults in the create method of each builder ensures that any shared builders will come packaged with their defaults. You could also create a no-arg constructor, but I prefer each builder to have only one constructor that contains all the dependencies necessary for creating the actual concrete class.
RadioBuilder.create();
}
The following code shows how all these things work together.
// RadioTest.java
{
{
Radio radio = aNew().radio().build();
radio.turnOn();
assertTrue(true, radio.isOn());
}
// .. other tests...
}
// DomainObjectBuilder.java
{
{
return new DomainObjectBuilder();
}
{
return RadioBuilder.create();
}
}
// RadioBuilder.java
{
private int buttons;
private CDPlayer cdPlayer;
private MP3Player mp3Player;
{
return new RadioBuilder(4,
CDPlayerBuilder.create().build(),
MP3PlayerBuilder.create().build());
}
{
this.buttons = buttons;
this.cdPlayer = cdPlayer;
this.mp3Player = mp3Player;
}
{
return new RadioBuilder(buttons, cdPlayer, mp3Player);
}
{
return new RadioBuilder(buttons, cdPlayer, mp3Player);
}
{
return new RadioBuilder(buttons, cdPlayer, mp3Player);
}
{
return new Radio(buttons, cdPlayer, mp3Player);
}
}
As you can see from the example, it's easy to create a radio with sensible defaults, but you can also replace dependencies where necessary using the 'with' methods.
Since Java gives me method overloading based on types, I always name my methods 'with' when I can (like the example shows). This doesn't always work, if for example you have two different properties that are of the same type. This usually happens with built in types, and in those cases I create methods such as withButtons, withUpperLimit, or withLowerLimit.
The one other habit I've gotten into is using Builders to create all objects within my tests, even the Class Under Test. This results in more maintainable tests. If you use the the Class Under Test's constructor explicitly within your test and you add a dependency you'll end up having to change each line that creates a Class Under Test instance. However, if you use a builder you may not need to change anything, and if you do have to change anything it will probably only be for a subset of the tests.
Conclusion
I'm a big fan of Test Data Builders and Test Doubles. Combining the two concepts has resulted in being able to write tests faster and maintain them more easily. These ideas can be incrementally added as well, which is a nice bonus for someone looking to add this style to an existing codebase.
Great post.
ReplyDeleteIt gives a good overview of different types of mocking/handling dependencies. It also gives a good example and great arguments for the Test Data Builder.
Thanks!
Jay -- great post. It's a good case for builders, in testing and otherwise. I especially like the DSL-ish aspect of "aNew".
ReplyDeleteNice post, I'd going to have to try out the test data builder. When you use the test data builder, do you only use it for test code, or do you end up using it to build your real domain objects as well, or does it just depend on the situation? I also like the modified (?) builder pattern where you are always returning newly constructed object, it makes the build() method redundant (if I'm not mistaken).
ReplyDeleteI've found the xunitpattern test double definitions to be pretty helpful and as a more detail version of Fowlers.
Thanks,
Zach
@zdsbs: There are occasions where I'll use a builder in domain code, but the vast majority of usages are within tests.
ReplyDeleteLike you said, it depends on the situation. I tend to go with the code that I think is the most maintainable.
As far as making the build() method unnecessary, I'm not sure what you mean. Can you post a code sample on pastie.org?
Thanks for the comment.
Cheers, Jay
@Jay Yeah, about 5 minutes after I posted that about the builders, I was like, "what am I talking about". No way to get rid of the build method, at least as far as I can see. Sorry about that.
ReplyDelete-Zach
Jay - thanks for this. This can be a pretty confusing topic, at least to discuss with others, because there are a lot of overloaded terms.
ReplyDeleteI generally point people to the Fowler articles to clarify my definitions as well, as does Lasse Koskela in his book Test Driven. Good to see we're gaining some consensus on what to name everything.
great differential of dummies, stubs, mocks and fakes. i always have trouble expressing that....
ReplyDeleteHi Jay, another good post. Nice to see you tackling stuff in Java now.
ReplyDeleteA specific comment about your code snippet:
aNew().car().
with(mock(Engine.class)).
with(fake().radio().fm_only()).
with(aNew().powerWindows()).build();
Your implementation seems to suggest that the above would actually be:
aNew().car().
with(mock(Engine.class)).
with(fake().radio().fm_only().build()).
with(aNew().powerWindows().build()).build();
You need the build() calls in the with() arguments because the with() methods accept the actual class instances and not the builders for the same. This is the only ugly part of the Builder pattern which otherwise is very readable/maintainable. (I think you could get rid of the build() calls for arguments by adding more overloaded with() methods to your Builder which in turn would call the build() for you... or is that too much code for removing a little ugliness?).
Also, why have you kept your Builder immutable instead of having the with() methods modify the builder's instance variables and return this (the traditional builder style)?
A related question, Jay, that I keep asking myself when I'm creating objects while unit testing a Rails app (having used builders in my Java experience):
ReplyDeleteDoes the Test Data Builder pattern make sense in Ruby/Rails where (1) creating a readable hash is trivial, and (2) ActiveRecord classes take a hash in the constructor? I'm asking about the method chaining involved (of with() methods finally leading to a build() call).
Ruby example:
class CustomerBuilder
def self.build(params = {})
defaults = {:name => 'Aman', :email => 'blah@blah.com', :department => DepartmentBuilder.build}
Customer.new(defaults.merge(params))
end
end
And the usage would be like:
customer = CustomerBuilder.build(:name => 'King', :email => 'abc@test.com')
Isn't the above as good as:
customer = CustomerBuilder.create.
with_name('King').
with_email('abc@test.com').build
Thoughts please?
Earlier, I asked: Also, why have you kept your Builder immutable instead of having the with() methods modify the builder's instance variables and return this (the traditional builder style)?
ReplyDeleteI found the answer here: Tricks with Test Data Builders
So basically the builder is kept immutable to avoid side-effects if you reuse the builder to create multiple objects (and not as one-liner "use and throw" objects).
That's definitely something I'll keep in mind in the future, when dealing with builders.
The website above is a very good read. Thanks for linking to it, Jay.
Aman (response to first comment) - You can definitely add overloaded methods that also takes builders. I definitely do that, and that's how I achieve the shorter syntax that my example demonstrates.
ReplyDelete@Aman (response to 2nd comment)
ReplyDeleteIn Ruby I would probably do something more ruby-esque. Last time I worked on something similar was with Dan Manges. He wrote a blog post with examples from what we did: http://www.dcmanges.com/blog/38
for ruby, you should check out factory_girl, gem from thoughtbot that does this for you in a very slick way:
ReplyDeletehttp://www.thoughtbot.com/projects/factory_girl
I find one of the trickiest aspects of creating good unit tests is avoiding making them implementation dependent. They need to be implementation independent so that you can refactor the implementation and re-run the tests to make sure nothing is broken. For a given object-under-test, some other objects may be collaborators that you want to set expectations on, while others are internal interactions that you don't want to know about (because they could change). The latter objects should probably either be the real thing or fake objects that behave like the real thing without the test having to set them up.
ReplyDeleteHello Mike, I tend to use builders to create objects quickly and easily, and my builders always have concrete implementations as defaults. However, I tend to use fakes and stubs where it makes sense for each test.
ReplyDeleteLike you, I prefer tests that don't break when the implementation changes. So I tend to start with defaults, and switch to fakes or stubs when I notice that implementation changes are causing irrelevant failures.
Cheers, Jay
Great survey, I learned some new tricks, thanks!
ReplyDeleteFor yet another way of creating test objects, you might take a look at our example based extension of JUnit. The basic idea is that each test is responsible to create its own Unit under Test. These units are cached by the framework (hey, they are valid and well tested instances!) and can later be reused by other tests using an @Given annotation. Therefore we call these units "examples".
More see JExample
Hi Jay thanks for the informative post - I am using it to aid development of my unit tests.
ReplyDeleteOne question though:
How do you deal with collections? I am doing the same as you - creating new instances of the builder with every method, but each time the collections in the class contain references to the same objects. I foresee potential problems when changing these in multiple objects created from the same root builder?
Any thoughts appreciated!
Thanks again for the article,
Paul
Hi Paul,
ReplyDeleteI have run into problems where not creating a new collection every time caused unexpected results. In general I have a method that takes varargs or a List and I always pass in the full collection. It's a bit more verbose, but it beats having to track down a tricky bug.
Cheers, Jay