Clojure: expectations Introduction

Clojure Unit Testing with Expectations Part One (this entry)
Clojure Unit Testing with Expectations Part Two
Clojure Unit Testing with Expectations Part Three
Clojure Unit Testing with Expectations Part Four
Clojure Unit Testing with Expectations Part Five
Clojure Unit Testing with Expectations Part Six

A bit of history
Over a year ago I blogged that I'd written a testing framework for Clojure - expectations. I wrote expectations to test my production code, but made it open source in case anyone else wanted to give it a shot. I've put zero effort into advertising expectations; however, I've been quietly adding features and expanding it's use on my own projects. At this point it's been stable for quite awhile, and I think it's worth looking at if you're currently using clojure.test.

Getting expectations
Setting up expectations is easy if you use lein. In your project you'll want to add:

:dev-dependencies [[lein-expectations "0.0.1"]
                   [expectations "1.1.0"]]

After adding both dependencies you can do a "lein deps" and then do a "lein expectations" and you should see the following output.

Ran 0 tests containing 0 assertions in 0 msecs
0 failures, 0 errors.

At this point, you're ready to start writing your tests using expectations.

Unit Testing using expectations
expectations is built with the idea that unit tests should contain one assertion per test. A result of this design choice is that expectations has very minimal syntax.

For example, if you want to verify the result of a function call, all you need to do is specify what return value you expect from the function call.

(expect 2 (+ 1 1)

note: you'll want to (:use expectations); however, no other setup is required for using expectations. I created a sample project for this blog post and the entire test file looks like this (at this point):

(ns sample.test.core
  (:use [expectations]))

(expect 2 (+ 1 1))

Again, we use lein to run our expectations.

jfields$ lein expectations
Ran 1 tests containing 1 assertions in 2 msecs
0 failures, 0 errors.

That's the simplest, and most often used expectation - an equality comparison. The equality comparison works across all Clojure types - vectors, sets, maps, etc and any Java instances that return true when given to Clojure's = function.

(ns sample.test.core
  (:use [expectations]))

(expect 2 (+ 1 1))
(expect [1 2] (conj [] 1 2))
(expect #{1 2} (conj #{} 1 2))
(expect {1 2} (assoc {} 1 2))

Running the previous expectations produces similar output as before.

jfields$ lein expectations
Ran 4 tests containing 4 assertions in 26 msecs
0 failures, 0 errors.

Successful equality comparison isn't very exciting; however, expectations really begins to prove it's worth with it's failure messages. When comparing two numbers there's not much additional information that expectations can provide. Therefore, the following output is what you would expect when your expectation fails.

(expect 2 (+ 1 3))

jfields$ lein expectations
failure in (core.clj:4) : sample.test.core
           (expect 2 (+ 1 3))
           expected: 2 
                was: 4

expectations gives you the namespace, file name, and line number along with the expectation you specified, the expected value, and the actual value. Again, nothing surprising. However, when you compare vectors, sets, and maps expectations does a bit of additional work to give you clues on what the problem might be.

The following 3 expectations using vectors will all fail, and expectations provides detailed information on what exactly failed.

(expect [1 2] (conj [] 1))
(expect [1 2] (conj [] 2 1))
(expect [1 2] (conj [] 1 3))

jfields$ lein expectations
failure in (core.clj:5) : sample.test.core
           (expect [1 2] (conj [] 1))
           expected: [1 2] 
                was: [1]
           2 are in expected, but not in actual
           expected is larger than actual
failure in (core.clj:6) : sample.test.core
           (expect [1 2] (conj [] 2 1))
           expected: [1 2] 
                was: [2 1]
           lists appears to contain the same items with different ordering
failure in (core.clj:7) : sample.test.core
           (expect [1 2] (conj [] 1 3))
           expected: [1 2] 
                was: [1 3]
           3 are in actual, but not in expected
           2 are in expected, but not in actual
Ran 3 tests containing 3 assertions in 22 msecs
3 failures, 0 errors.

In these simple examples it's easy to see what the issue is; however, when working with larger lists expectations can save you a lot of time by telling you which specific elements in the list are causing the equality to fail.

Failure reporting on sets looks very similar:

(expect #{1 2} (conj #{} 1))
(expect #{1 2} (conj #{} 1 3))

jfields$ lein expectations
failure in (core.clj:9) : sample.test.core
           (expect #{1 2} (conj #{} 1))
           expected: #{1 2} 
                was: #{1}
           2 are in expected, but not in actual
failure in (core.clj:10) : sample.test.core
           (expect #{1 2} (conj #{} 1 3))
           expected: #{1 2} 
                was: #{1 3}
           3 are in actual, but not in expected
           2 are in expected, but not in actual
Ran 2 tests containing 2 assertions in 15 msecs
2 failures, 0 errors.

expectations does this type of detailed failure reporting for maps as well, and this might be one if the biggest advantages expectations has over clojure.test - especially when dealing with nested maps.

(expect {:one 1 :many {:two 2}}
        (assoc {} :one 2 :many {:three 3}))

jfields$ lein expectations
failure in (core.clj:13) : sample.test.core
           (expect {:one 1, :many {:two 2}} (assoc {} :one 2 :many {:three 3}))
           expected: {:one 1, :many {:two 2}} 
                was: {:many {:three 3}, :one 2}
           :many {:three with val 3 is in actual, but not in expected
           :many {:two with val 2 is in expected, but not in actual
           :one expected: 1
                     was: 2
Ran 1 tests containing 1 assertions in 19 msecs
1 failures, 0 errors.

expectations also provides a bit of additional help when comparing the equality of strings.

(expect "in 1400 and 92" "in 14OO and 92")

jfields$ lein expectations
failure in (core.clj:17) : sample.test.core
           (expect in 1400 and 92 in 14OO and 92)
           expected: "in 1400 and 92" 
                was: "in 14OO and 92" 
            matches: "in 14" 
           diverges: "00 and 92" 
                  &: "OO and 92"
Ran 1 tests containing 1 assertions in 8 msecs
1 failures, 0 errors.

That's basically all you'll need to know for using expectations to equality test. I'll be following up this blog post with more examples of using expectations with regexs, expected exceptions and type checking; however, if you don't want to wait you can take a quick look at the success tests that are found within the framework.