Clojure: Expectations Verify Interaction Args

The expectations framework provides the ability to create interaction (or behavior) based tests. I've previously written about adding interaction based testing to expectations; however, the examples from that blog entry focused exclusively on testing interactions where each argument is matched using equality. In this entry I'll give examples of how each argument can be also be verified using a class, regex, exception, or a custom function.

When writing state based tests using expectations the type of test you're writing is inferred from the expected value. If the expected value is a regex, expectations will test the actual value to see if it matches the regex. If you passed in a class, expectations will test the actual value to see if it's an instance of that class. If you passed in an exception... you get the idea. All of what I said above, is also true for arguments of an interaction.

Let's start with a simple interaction based test:


In the example above, we're calling the spit function with exactly the arguments that we've specified in our test. This test will pass; however, we've had to specify the exact file location and the exact data. If for some reason you can't specify exactly what the argument will be, it's nice to have a way to specify as much as you possibly can.

In the example below, we're still specifying the exact data, but we're only verifying that the file is somewhere in /tmp/.


As I previously mentioned, we can also get more general and only verify the class of an argument. For example, if we knew our data was going to be a String, but we didn't want to specify exactly what that string was, the following test would do the trick.


While expectations provides you with a lot of default options, there are times when you'll want to write your own argument "matcher". As a contrived example, let's pretend that we want to test that the last argument is true or nil.


One of the best features of expectations is it's error reporting, and the same error reporting logic is applied to arguments when an interaction based test fails. Given the example above, you'll get the following error message.

failure in (success_examples.clj:204) : success.success-examples
           expected: (spit #"/tmp/" String :append true-or-nil?) 
                got: 0 times 

           -- got: (spit "/tmp/somewhere-else" "nil")
           "nil", "/tmp/somewhere-else" are in actual, but not in expected
           true_or_nil_QMARK, #"/tmp/", :append, String are in expected, but not in actual
           expected is larger than actual 

           -- got: (spit "/tmp/hello-world" "some data" :append "s")
           - arg4: not true or nil

As you can see both calls are reported, and each argument has a detailed report (if it did not match).

Finally, expectations provides and additional function that can be used to verify that certain key/value pairs are in an argument. The following example doesn't really make sense, since you'd never want to pass a map as the last argument to spit, but it's easy to follow in the context of this blog entry.


In the above example, (contains-kvs) is used to verify that the final argument to spit contains the key/value pairs :a :b :c :d.

I hope that interaction arg matching follows the principle of least surprise, since it behaves the same as expectations state based tests. I also hope that the ability to use an arbitrary function for verification will provide any necessary flexibility. If you're using expectations, give it a try and let me know.

Clojure: Expectations Interactions - Interactions Are Code, Interactions Are Data

If you read my blog you've probably heard "code is data, data is code" and at one time and you've looked up homoiconicity. You may have deeply understood the idea the first time you heard it; I definitely did not. However, a recent addition to expectations opened my eyes to how truly powerful this programming language property can be.

I'll start by admitting what I heard when I originally encountered homoiconicity. Stuart Halloway had begun promoting Clojure, and homoiconicity was one of the advantages he noted. I hit the wikipedia page, digested the words "code is data, data is code", and thought to myself: well, yeah, obviously. I'd spent plenty of time working with DSLs in Ruby, and I had plenty of experience evaluating code in various contexts. I thought something along the lines of: So you capture the code as data and evaluate it wherever it makes sense, I don't see the big deal. In short, I didn't get it.

Fast forward a few years and several hours of full time Clojure development and you'll find me adding interaction based testing to expectations. What I had in mind for testing interactions was simple, I want to write exactly the same thing for the test as what I write for the production code. Additionally, I want the format of the test to follow the same format that is used for state based testing: (expect expected actual)

Once I had a clear vision for my requirements, the format of the tests became easy to visualize. Assume I have a function that prints to standard out, and I want to test that this print occurs.

The above test looks great, but (println 5) will be evaluated, return nil, and use nil as the expected value. I needed some way for the programmer to tell the testing framework that this was an interaction test, and expectations needed to verify that the function was called with the specified parameters. After trying a few different formats, I settled on the following solution.

By wrapping the interaction I wanted to test with (interaction ...), I created an easy way to identify and capture the function and arguments that needed to be verified.

Once I'd decided on the syntax, I went about the task of adding support to expectations. If you dug into the implementation of expectations, you'd find that expect is a macro that delegates the handling of the "expected" and "actual" arguments to the doexpect macro. The first thing the doexpect macro does is check if expected is a list and (if so) if the first argument is the symbol "interaction" (source here). If the first argument is not a list that begins with 'interaction, then the data is passed to do-value-expect and expanded more or less as is. However, if the first argument is a list that begins with 'interaction, then the data is passed to do-interaction-expect, and do-interaction-expect then destructures the data, grabbing only the pieces of the list that it cares about (source here). When I wrote this code, I found it very interesting.

When I envisioned the interaction syntax, I assumed that (interaction ...) would be a call to a macro, and I would need to need to manipulate the data passed to interaction. However, once I got into the actual implementation, I found myself using the symbol "interaction", but never actually defining a macro or even a function. That's when homoiconicity really started to become clear to me. I'd written code that I was sure would need an implementation, yet it was used exclusively as data.

If you kept digging into this example you would find that anything found within (interaction ...) is never used as written, but is instead expanded in a way that allows expectations to rebind the specified function and use the expected arguments at verification time. As a result, you write the same code in the same way but within your test it's used exclusively as data and in your production code it's used exclusively as code. I'm a big fan of convention, and there's no better convention than 'use the exact same thing'.

I later added the ability to add interaction tests for calls to Java objects as well, which led to the following behavior for expectations.

• If your expected value is not an interaction, it will be expanded as is.
• If your expected value is an interaction with a Clojure function, it will be used as data exclusively and expanded to rebind the function, capture all calls to the function and verify that a call occurred with the arguments you specified.
• If your expected value is an interaction with a Java method, it wil be used as data exclusively and expanded to mockito setup and verification code.

Thus, an expected value is sometimes code, and sometimes data.

Clojure: Expectations Warn On State Change During Test Runs

While writing tests it can be easy to accidentally change any global state that exists in your application. I've previously written about Redefining State Within a Test; however, redef-state and with-redefs only help you if you redefine all of the affected state. The situation is even more problematic due to the fact that accidental state alteration often doesn't cause issues until a completely unrelated test suddenly fails. After being bitten by this issue a few times, I added (to expectations) the ability to warn when global state is modified by a test.

As of version 1.4.24 if you add (expectations/warn-on-iref-updates) anywhere, then expectations will provide you with a warning whenever any global state is altered.

While you can add that snippet anywhere, I prefer to add it to the expectations Before Run Hook. There's an example expectations_options.clj in the expectations codebase that shows all of the code you need to enable this feature. Simply add this file or add the function to your existing file and you should see a warning on any global state alteration. If you're not sure where to put this file, refer to the Before Run Hook blog post.

Here's an example warning (generated by running the expectations tests).

WARNING: success.success-examples:280 modified #'success.success-examples-src/an-atom from "atom" to "another atom"

The warning should let you know which test is doing an unexpected modification, and the to and from values should give you an idea of where in the source the alteration is occurring.

Clojure: Expectations Before Run Hook

The expectations library now supports calling an arbitrary number of custom functions before the test suite is run.

There are several reasons that you might want to call a custom function before the test suite is executed-

• load sample data to a database
• delete a temp directory
• replace logging with standard out or /dev/null

As of version 1.4.24, expectations will execute any function that includes the following metadata {:expectations-options :before-run}. The following example should serve as a decent reference on how to add your own 'before-run' functions.


These functions can be defined anywhere; however, expectations looks in a default location for a configuration file. If expectations finds this configuration file it removes (what should be) the namespace and requires it with :reload. As a result, this configuration namespace will be removed and redefined with each expectations suite run.

The name of this default configuration file is expectations_options.clj (thus the namespace is expectations-options). Expectations looks for this file in the root test directory. If you have :test-paths ["test/clojure"] in your project.clj, then you'll want to create test/clojure/expectations_options.clj.

If you place your expectations_options.clj file in the correct location and you add the {:expectations-options :before-run} metadata to a function in the expectations-options namespace, your function should be run automatically the next time your test suite runs. You may want to start with a (println) just to verify that things are going as you expect.