Tuesday, February 12, 2008

Testing: Expect literals

When writing a state based test you must specify an expected value and an actual value. Whether you are using RSpec, Test::Unit, or any other framework, there is always an expected and an actual value.

# RSpec
presenter.hotel.should == :the_hotel

# Test::Unit
assert_equal :the_hotel, presenter.hotel

Expected values (:the_hotel in our example) can be any object; however, using literals (strings, integers, symbols) for expected values is advantageous for readability and traceability. To be clear, the expected value should be the literal itself, not a variable holding a literal that was created earlier in the test.

Tests that use literals are easier to read because they allow you to focus on the actual value, since the expected value is simply the literal. The following 3 tests verify the same thing, but the 3rd example uses a literal for the expected value. Since the 3rd example uses a literal, it's easy to see what the value of the actual should be without scanning the body of the test.

require 'test/unit'
require 'rubygems'
require 'expectations'

class SuiteTests < Test::Unit::TestCase
EXPECTED = :expected

def test_expectations_for_should_only_run_the_test_on_the_specified_line_with_local_var
expected = :expected
suite = Expectations::Suite.new
suite.expect(1) { 2 }
suite.expect(expected) { 2 }
assert_equal expected, suite.expectations_for(__LINE__ - 1).first.expected
end

def test_expectations_for_should_only_run_the_test_on_the_specified_line_with_constant
suite = Expectations::Suite.new
suite.expect(1) { 2 }
suite.expect(EXPECTED) { 2 }
assert_equal EXPECTED, suite.expectations_for(__LINE__ - 1).first.expected
end

def test_expectations_for_should_only_run_the_test_on_the_specified_line_with_literal
suite = Expectations::Suite.new
suite.expect(1) { 2 }
suite.expect(:expected) { 2 }
assert_equal :expected, suite.expectations_for(__LINE__ - 1).first.expected
end
end

note: these tests are fairly clean by design, the value of expecting literals can be even greater as the tests become more complex, thus harder to follow.

Expecting literals also makes your life easier when a test breaks. If neither your expected or actual values are literals, you will need to determine the value of both of them when a test breaks. Conversely, if your expected is a literal, only the actual value will need to be traced when a test is broken. For example the following tests both verify the result of the popularity method; however, the 2nd version is explicit in the value in expects.

require 'test/unit'

WEIGHT = 1.14

class Topic
def initialize(votes)
@votes = votes
end

def popularity
WEIGHT * @votes * @votes
end
end

class CircleTest < Test::Unit::TestCase
def test_popularity_with_variables
votes = 2
assert_equal WEIGHT * votes * votes, Topic.new(votes).popularity
end

def test_popularity_with_literals
assert_equal 4.56, Topic.new(2).popularity
end
end

In the above example, if the second test breaks it's because the calculation changed in some way. When the calculation changes, it's valuable to get feedback that it changed (by way of a breaking test). The first test doesn't use any literals, but it will not break if the value of WEIGHT is changed. Some people might consider this valuable or perhaps even more maintainable, I'd consider it a bug waiting to happen.

In reality, it's not likely that the programmer is responsible for setting the value of WEIGHT. Most businesses would have a subject matter expert that would give the value of weight and express (via requirements) the formula for popularity. The programmer would write the actual Ruby code, but using literal numbers for the expected value would allow the programmer to verify the functionality with the business. If in the future the business wants to change the value of WEIGHT, the programmers and the business could first change the tests to represent the new expected values, and then they could change the implementation. When you change code, it's always valuable to have a breaking test. Even if the business didn't sit with the programmer, the programmer could request the expected values as acceptance criteria. The tests will be easier to read, but they will also protect the business from a mistaken change to WEIGHT.

There are times when it's not possible to use a literal as the expected value. For example, when you need to assert a time or date you'll need to use a Time or Date object respectively. You can handle these situations in various ways. I prefer to use expected values that are as close to literals as I can get. This means using a literal as an argument to a method or calling to_s on the actual and expecting a string. The following two examples illustrate both of these techniques.

require 'test/unit'
require 'date'

class DatesTest < Test::Unit::TestCase
def test_valentines_day_last_year
assert_equal Date.parse("2/14/2007"), ValentinesDay.last_year
end

def test_valentines_day_last_year_to_s
assert_equal "2007-02-14", ValentinesDay.last_year.to_s
end
end

It's not always possible to use literals for expected values, but if you strive to use literals whenever possible your tests will be more readable and traceable in the future.

Saturday, February 09, 2008

Designing a Domain Specific Language

I've worked on and with various Domain Specific Languages (DSLs), but I've found designing them to be the same process no matter what type of DSL is being created.

In March of 2005 I started one of the more technically interesting projects of my life. ThoughtWorks had been hired to design a system that would allow the director of finance for the client bank to edit, test, and deploy the business rules of the system without any assistance from IT. It was a bold step, but the return on investment was realized within weeks, not months (or worse, years).

One of the major challenges of designing this type of system is determining what the language should look like. Being new to the idea, we started out by telling the business what was possible. That was a mistake. The early versions of the language were easy for IT to work with, but were clunky for the business. The result was clearly suboptimal. As we continued evolving the language for new features we took a different approach, we asked the director what he wanted the language to look like. The more he shaped the language the easier it was for him to take ownership of the domain logic.

We figured out that we didn't need to write the domain logic in our language, we needed to make the domain language executable. The resulting project was a huge success, which led to me coming up with the term Business Natural Languages and presenting the idea at several conferences*.

Almost 3 years later, I decided I needed a new framework for unit testing. The primary motivation for designing a new framework was my desire for a more syntactically elegant way of defining tests. This time I knew I wanted to design what the language looked like and then worry about making it work at a later time. In fact, when writing expectations, I had a small suite of example tests before I had any way to execute them.

None of the languages I've helped design are flawless. But, I've seen significantly more success with the languages that were designed by a subject matter expert concerned with the language, instead of a language designer who was concerned with how to make the language execute.

* If you'd like to catch the Business Natural Language presentation you can see it at QCon London 2008 or rupy.eu 2008.

ThoughtWorks DSL Podcast (part 2)

Part 2 of the DSL podcast Martin Fowler, Rebecca Parsons, Neal Ford and I participated in is now available. The podcast is once again being released as part of the ThoughtWorks IT Matters series and is available at ThoughtWorks: What we say.

Friday, February 08, 2008

Subversion: Rolling back revisions

Earlier this week someone mistakenly checked in some experimental code and wanted to back that code out. If you've never needed to do this, you might think that Subversion gives you a command to make this happen easily, it doesn't. In fact, I've found that to be a fairly common misconception.

There may be easier ways, but the following steps are the ones I take to roll back a specific revision.

Find the revision that you wish to back out. This can be done with various tools, but 'svn log --limit 10' will probably do the trick. Once you have the offending revision you are ready to get started. (Revision X in the example is the revision that needs to be rolled back and revision Y is the revision that came immediately before X [i.e. X-1])
  1. svn diff -rX:Y > a.patch
  2. patch -p0 < a.patch
  3. svn commit
note: while it's possible to give patch parameters and do this pretty much anywhere in the tree, I'd stick to doing this in the root folder of your project, for simplicity.

Again, there is probably an easier way to do this (which is part of my motivation for writing this entry), and obviously this solution depends on having 'patch' available.

Monday, February 04, 2008

Static typing considered harmful

(Updated Info, 2014-10-29)
I still find almost no value in the static typing provided by Java. That said, I've done some ML lately and been very impressed with what types can provide. In addition, I've worked with some great programmers who do find value in Java's static typing.

If you take anything away from this blog entry, I would recommend this: No one can tell you which tools will make you the most effective, try them all and determine for yourself what helps you the most.

(Original Post, 2008-04-02)
Given a good test suite the return on investment simply does not justify the use of static typing.
In 5 years, we'll view compilation as the weakest form of unit testing. -- Stuart Halloway
Type verification provides very little confidence that an application works. The little confidence it does provide comes at the cost of being confined by the type system. Verifying types is better than testing nothing at all. But, only verifying types is insufficient for application development.

The obvious answer is to write more tests. As you write more tests, you can verify independent pieces of the application and integration of the different pieces of the application. Currently, you know that your code compiles; but that is to general to be considered sufficient for verifying system behavior. Specific tests can verify specific behavior. You may start out testing that 1 + 1 is a number, but eventually you will be better off testing that 1 + 1 is 2. The test that 1 + 1 is a number may have given you some confidence that your application was working, but once you add a test to verify that 1 + 1 is 2, the test that verifies that 1 + 1 is a number no longer provides you with any additional confidence. Since the test that verifies 1 + 1 is a number no longer provides any value, it can safely be removed.