Jay Fields' Thoughts

Over two years ago I started working primarily with Ruby. Since those days I haven't written a line of C# or Java, and I've often wondered if I would be annoyed by the ceremony that both C# and Java generally require.

I've recently been working a bit with Flex.Overall, I've enjoyed working with something new, and I can definitely see the appeal of RIA in general. It's been fairly easy to get started with Flex because it feels very similar to working with HTML and Javascript.

MXML, an XML-based markup language, offers a way to build and lay out graphic user interfaces. Interactivity is achieved through the use of ActionScript, the core language of Flash Player that is based on the ECMAScript standard. --Wikipedia Flex Information

ActionScript might be based on the ECMAScript standard, but it feels a lot more like Java than Javascript.

[Disclaimer: My experience with ActionScript is very limited, so there may be a better way that I'm not aware of]

I was working with a few MXML views the other day and I ran into a situation that made me think about essence and ceremony. The following example code is a simplified example of what I was working on.

<?xml version="1.0"?>
<mx:HBox xmlns:mx="http://www.adobe.com/2006/mxml">
    <mx:ComboBox id="typeCombo"/>
    <mx:Box id="typeDetailsBox"/>
</mx:HBox>

<!-- ~/views/OptionOne.mxml -->
<mx:HBox xmlns:mx="http://www.adobe.com/2006/mxml">
    <mx:Label text="You chose option 1"/>
</mx:HBox>

<!-- ~/views/OptionTwo.mxml -->
<mx:HBox xmlns:mx="http://www.adobe.com/2006/mxml">
    <mx:Label text="You chose option 2"/>
</mx:HBox>

The intent of the simplified example is to change what's inside the typeDetailsBox based on what's selected in the typeCombo.

To solve this in ActionScript I wrote code similar to the following.

public static const Option1:String = "1"; 
public static const Option2:String = "2"; 

public function init():void {
    typeCombo.dataProvider = new ArrayCollection([{label:Option1, data:Option1}, {label:Option2, data:Option2}]);
    typeCombo.addEventListener("close", function(e:Event):void{ doTypeSelected(); });
    doTypeSelected();
}

public function doTypeSelected():void {
    if (typeDetailsBox.numChildren > 0) {
        typeDetailsBox.removeChildAt(0);
    }
    switch (typeCombo.selectedItem.data) {
    case Option1:
        typeDetailsBox.addChild(new OptionOne());
        break;
    case Option2:
        typeDetailsBox.addChild(new OptionTwo());
        break;
    }
}

The code is easy enough to follow, but it feels much more verbose than what I prefer. I like being able to pass classes around as objects, so if I could, I'd probably make the classes themselves the "data" for the drop down. If that wasn't an option, I'd prefer an eval method that let me do the following.

public function init():void {
    typeCombo.dataProvider = new ArrayCollection([{label:"1", data:"new OptionOne()"}, {label:"2", data:"new OptionTwo()"}]);
    typeCombo.addEventListener("close", function(e:Event):void{ doTypeSelected(); });
    doTypeSelected();
}

public function doTypeSelected():void {
    if (typeDetailsBox.numChildren > 0) {
        typeDetailsBox.removeChildAt(0);
    }
    typeDetailsBox.addChild(eval(typeCombo.selectedItem.data));
}

Unfortunately, ActionScript seems to be a language a bit more concerned with ceremony than I am.

Don't read this as a dismissal of Flex or ActionScript. Whether or not to use a language is a very complicated question and I wouldn't ever base my answer on a switch statement or the ability to eval. Like I said before, working with Flex has been a largely enjoyable experience, but, I must admit, after working 7 years with high ceremony languages and the past 2 with a low ceremony language, I know which I prefer.

Labels: ActionScript, essence

Luke Kanies recently left a comment on Using Stubs to Capture Test Essence:

Now if you would only see the wisdom of applying DRY to your whole code base, rather than just the functional bit. :)

Sadly, DRY has become a philosophy that is dogmatically and blindly applied to every aspect of programming. I'm a big fan of firing silver bullets but if you go that path your number one priority needs to be finding where a solution does not fit. In fact, those who have applied DRY excessively have found that there are situations when DRY may not be advantageous.

Three of the points from the Wikipedia entry relate to test setup code.

In some small-scale contexts, the effort required to design around DRY may be far greater than the effort to maintain two separate copies of the data.
There's no question that designing around DRY for tests adds additional effort. First of all, there's no reason that tests need to run in the context of an object; however, the most common way to use a setup is to create instance variables and reuse those instance variables within a test. The necessity of instance variables creates the requirement that a test must be run within a new object instance for each test.

Ensuring that a setup method only creates the objects that are necessary for each test adds complexity. If all tests don't require the same data, but you create a setup that creates everything for every test, digesting that code is painful and unnecessary. Conversely, if you strictly adhere to only using setup for objects used by all methods you quickly limit the number of tests you can put within one test case. You can break the test case into multiple test cases with different setup methods, but that adds complexity. Consider the situation where you need the foo instance variable for test 1 and 2 and the bar instance variable for test 2 and 3. There's no straightforward way to create foo and bar in a DRY way using a setup method.

Of course, there's also readability impacts to consider. When a test breaks that relies on setup and teardown I have to look in 3 places to get the full picture of what's going on. When a test is broken I want to fix it as quickly as possible, so I prefer looking in one place.

The crux of the issue is whether a test case is one context or the individual tests are small-scale multiple contexts. While a test case is conceptually one object, we rarely concern ourselves with a test cases as a whole. Instead testing is very much about many individual executable pieces of code. In fact, it's a testing anti-pattern to have any tests depend on other tests. We write tests in isolation, fix tests in isolation and often run tests in isolation; therefore, I consider them to be many small independent entities that do completely isolated jobs. The result of those jobs is aggregated in the end, but ultimately the different entities never (or should never) interact. Each test is and runs in it's own context.

Imposing standards aimed at strict adherence to DRY could stifle community involvement in contexts where it is highly valued, such as a wiki.
Tests are often written by one, but maintained by many. Any barrier to readable, reliable, and performant tests should be removed as quickly as possible. As I've previously discussed, abstracting behavior to 3 different places reduces readability, which can reduce collaboration. Additionally, setup methods can cause unexpected behavior for tests that are written by an author other than the creator of the setup method. This results in less reliable tests, another obvious negative.

Human-readable documentation (from code comments to printed manuals) are typically a restatement of something in the code with elaboration and explanation for those who do not have the ability or time to read and internalize the code...While tests aren't code comments or printed manuals they are very much (or should be) Human-readable documentation. They are also restatements of something in the code with elaboration and explanation. In fact the setup and teardown logic is usually nothing more than elaboration and explanation. The idea of setup and teardown is to put the system into a known state before executing an assertion. While you should have the ability to understand setup and teardown, you don't have the time. Sure, you can make the time, but you shouldn't need to.

The problem is that it's much easier to understand how to apply DRY than it is to understand why.

If your goal is to create a readable, reliable, and performant test suite (which it should be) then there are better ways to achieve that goal than by blindly applying DRY.

Where there's smoke, pour gasoline --Scott Conley

Duplicate code is a smell. Setup and teardown are deodorant, but they don't fix the underlying issue. Using a setup method is basically like hiding your junk in the closet. There are better ways.

Pull the data and the behavior out of the setup and teardown methods and move it to the tests, then take a hard look.

Unimportant behavior specification
Setup methods often mock various interactions; however, the ultimate goal of the interactions is a single result. For example, you may have designed a Gateway class that takes a message, builds a request, sends the request and parses the results of the request. A poorly designed Gateway class will require you to stub the request building and sending the request. However, a well designed Gateway class will allow you to stub two methods at most and verify the behavior of the parse method in isolation. The poorly designed Gateway class may require a setup method, the well designed Gateway class will have a simple test that stubs a method or two in a straightforward manner and then focuses on the essence of the behavior under test. If your domain model requires you to specify behavior which is has nothing to do with what you are testing, don't specify that behavior in a setup method, remove the need to specify the behavior.

Loosely couple objects
I've always disliked the ObjectMother pattern. ObjectMother is designed to build complex hierarchies of objects. If you need an ObjectMother to easily test, you probably actually need to take another look at your domain model. Loosely coupled objects can be created without complicated graphs of dependencies. Loosely coupled objects don't require setting up multiple collaborators. Fewer collaborators (or dependencies) result in tests that don't rely on setup methods to create various collaborators.

The idea is to write tests that only specify what's necessary and verify one thing at a time. If you find that you need to specify things that are unrelated to what you are trying to verify, change the domain model so that the unnecessary objects are no longer required.

Solve problems globally or locally, but not between
Sometimes there are things that need to happen before and after every test. For example, running each test in a transaction and rolling back after the test is run is a good thing. You could add this behavior at a test case level, but it's obviously beneficial to apply this behavior to every test in the entire test suite.

There are other cases where it can make sense to solve issues on a global level. For example, I once worked on a codebase that relied heavily on localization. Some tests needed to verify strings that were specified in the localization files. For those tests we originally specified what localization file to use (in each test), but as we used this feature more often we decided that it made sense to set the localization file for the entire test suite.

Putting everything within a test is the best solution for increasing understanding. However, applying behavior to an entire test suite also aides in comprehension. I would expect any team member to understand that reference data is loaded once before the test suite is run, each test is run in a transaction, and localization uses the test localization file.

Understanding the behavior of an entire test suite is valuable and understanding the behavior of a test is valuable, but understanding a test case provides almost no benefit. Rarely do you run a single test case, rarely does an entire test case break, and rarely do you edit an entire test case. Since your primary concerns are the test suite and the tests themselves, I prefer to work on those levels.

I wouldn't go too far with the global behavior approach, and I definitely wouldn't start moving conditional logic into global behavior, but applied judiciously this can be a very valuable technique.

Bringing it all together
The core of the idea is that it should be easy to understand a test. Specifying everything within a test makes understanding it easy; however, that solution isn't always feasible. Setup and teardown are not the solution to this problem. Luckily the techniques above can be used to make specifying everything within a test a realistic and valuable solution.

Labels: testing, Testing Refactorings

Back in January of 2006 I wrote about Using Stubs and Mocks to Convey Intent. In those days I was working mainly with C# and NMock and the simplest way to create a stub was to allow ReSharper generate a stub based on an Interface. This introduced a bit more effort when first generating the stub and added complexity around where the stubs should live and how they should be used. Despite the additional effort, I still felt that using them was a net gain.

Things are very different in the Ruby world. While using C#, Dependency Injection was something I used religiously; however, using Ruby I can simply stub the new method of a dependency. C# had hurdles that made stubbing controversial; however, using stubs in Ruby tests is basically seamless thanks to Mocha. Mocha makes it easy to Replace Collaborators with Stubs, but you shouldn't stop there--you should also Replace Mocks with Stubs.

Replacing Mocks with Stubs is beneficial for several reasons. Stubs are more concise. One of the largest problems with tests is that they can be overwhelming the first time you look at them. Reducing a 3 line mock definition to stub(:one => 1, :two => 2) is almost always a readability win.

Stubs are great, but you may need to use a mock to verify behavior. However, you don't need to verify several behaviors in one test. Using a single mock to verify specific behavior and stubbing all other collaborations is an easy way to create tests that focus on essence. Following the One Expectation per Test suggestion often results in using one or more stubs... and more robust tests.

Using stubs is a good first step towards concise and relevant test code; however, you can take stubs a step further. Mocha defines a stub_everything method that creates an object which will return nil to any message it doesn't understand. The stub_everything stubs are fantastic for dependencies that are used several times, but you are only interested in specific interactions. The interactions which are unimportant for the current test no longer need to be defined.

Stubs can be even more valuable if you are interested verifying a single interaction but are completely uninterested in all other interactions. In that case it's possible to define a stub_everything and set an expectation on it. Consider the following sample code.

class MessageService
  def self.deliver(message)
    message.from = current_user
    message.timestamp
    message.sent = Gateway.process(message)
  end

def test_the_message_is_sent
  Gateway.stubs(:process).returns(true)
  message = stub_everything
  message.expects(:sent=).with(true)
  MessageService.deliver(message)
end

Since the expects method is defined on Object you can set expectations on concrete objects, mocks, and stubs. This results in a lot of possible combinations for defining concise behavior based tests.

Lastly stubbing can help with test essence by allowing you to stub methods you don't care about. The following example defines a gateway that creates a message, sends the message, and parses the response.

class MessageGateway
  def process(message_text)
    response = post(create_request(message_text))
    parse_response(response)
  end
  
  def post(message)
    # ...
  end
  
  def create_request(message_text)
    # ...
  end
  
  def parse_response(response)
    # ...
  end

The Gateway#process method is the only method that would be used by clients. In fact, it probably makes sense to make post, create_request, and parse_response private. While it's possible to test private methods, I prefer to create tests that verify what I'm concerned with and stub what I don't care about. Using partial stubbing I can always test using the public interface.

def test_create_request
  gateway = MessageGateway.new
  gateway.expects(:post).with("<text>hello world</text>")
  gateway.stubs(:parse_response)
  gateway.process("hello world")
end

def test_post
  gateway = MessageGateway.new
  gateway.stubs(:create_request).returns("<text>hello world</text>")
  gateway.expects(:parse_response).with("<status>success</status>")
  gateway.process("")
end

def test_parse_response
  gateway = MessageGateway.new
  gateway.stubs(:create_request)
  gateway.stubs(:post).returns("<status>success</status>")
  assert_equal true, gateway.process("")
end

The combination of partial stubbing and defining small methods results in highly focused tests that can independently verify behavior and avoid cascading failures.

Mocha makes it as easy to define a mock as it is to define a stub, but that doesn't mean you should always prefer mocks. In fact, I generally prefer stubs and use mocks when necessary.

Labels: stub, Testing Refactorings