Since I've deprecated scenarios, I went through all of my projects and removed any usages of expectations.scenarios. For the most part the conversion was simple; however, I did run into one instance where the scenario contained interleaved expectations.
The following code is an example of a scenario with interleaved expectations.
In the previously linked blog entry I recommend using a clojure assert to replace the interleaved expectations. That solution works, but I found an additional approach that I wanted to share.
When I encountered code similar in structure to the code above, I immediately envisioned writing 3 expectations similar to what you find below.
note: for my contrived example the first two tests could have been written without the let; however, the tests from my codebase could not - and I believe the blog entry is easier to follow if the tests are written in the way above.
While these tests verify the same expectations, the way that they are written doesn't convey to a test maintainer that they relate to each other more than they are related to the other tests within the file. While pondering this complaint, I grouped the tests in the following way more as a joke than anything else.
I would never actually use given simply to group code; however, grouping the code together did cause me to notice that there was a usage of given that would not only keep the code grouped, but it would also allow me to test what I needed with less code.
The following example is very similar in structure to the finished product within my codebase.
The above example verifies everything that the original scenario verified, does not use a scenario, and conveys to a maintainer that related logic is being tested within all three tests - in short: this felt like the right solution.
Wednesday, November 14, 2012
Tuesday, November 13, 2012
Elisp: Duplicate Line
After switching to emacs I quickly noticed that I was missing a simple keystroke for duplicating a line. Yes, I know it's as easy as C-a, C-k, C-y, Enter, C-y. Still, I wanted a keystroke. So, I coded up the following snippet. It's similar to what you'd find elsewhere on the net, but it also moves the cursor to where I've gotten used to it ending up.
Thursday, November 08, 2012
Elisp: Automated Switching Between Clojure Test and Source Files
The majority of the work that I do in emacs is Clojure programming. The Clojure navigation support (M-.) is usually all I need, but one thing that I find myself doing manually fairly often is jumping between tests and source. After suffering manual navigation for a few days, I finally automated the task using the Elisp from this blog entry.
All of my Clojure projects use both Leiningen and expectations; therefore, my directory structures always look similar to what you see below.
Since my projects follow this convention, I'm able to make several assumptions about where the expectations and where the source will actually live. If you don't use expectations, or you follow a slightly different directory structure, you'll want to hack this a bit to follow your conventions.
If you're in a source file, find (and open) the expectations.
If you're in a test file, find (and open) the source.
All of my Clojure projects use both Leiningen and expectations; therefore, my directory structures always look similar to what you see below.
source - /Users/jfields/src/project-name/src/clojure/
tests - /Users/jfields/src/project-name/test/clojure/expectations/
Since my projects follow this convention, I'm able to make several assumptions about where the expectations and where the source will actually live. If you don't use expectations, or you follow a slightly different directory structure, you'll want to hack this a bit to follow your conventions.
If you're in a source file, find (and open) the expectations.
If you're in a test file, find (and open) the source.
Wednesday, November 07, 2012
Elisp: Grep in Clojure Project
Grep'ing within my current project is something I do frequently. It's not much to type, but I do it often enough that I was looking for a keystroke. Most of my projects are Clojure and use Leiningen, thus I'm able to make some pretty safe assumptions. The following snippets allow you to easily grep within your project.
note: both use expand-region to grab the clojure that the cursor is on or immediately after, and grep for whatever was selected.
grep recursively starting at the directory where the project.clj file lives
grep recursively, but allow the user to select the root directory (defaulting to the location of the project.clj file). I often use this one for selecting only the src or test directories of my project.
note: both use expand-region to grab the clojure that the cursor is on or immediately after, and grep for whatever was selected.
grep recursively starting at the directory where the project.clj file lives
grep recursively, but allow the user to select the root directory (defaulting to the location of the project.clj file). I often use this one for selecting only the src or test directories of my project.
Tuesday, November 06, 2012
Clojure: Deprecating expectations.scenarios
I previously mentioned:
Truthfully, I've never liked scenarios - I've always viewed them as a necessary evil. First of all, I hate that you can't mix them with bare expectations - this leads to having 2 files or 2 namespaces in 1 file (or you put everything in a scenario, meh). You either can't see all of your tests at the same time (2 files), or you run the risk of your tests not working correctly with other tools (expectations-mode doesn't like having both namespaces in 1 file). Secondly, I think they lead to sloppy tests.
The second complaint causes me to get on my soap-box about test writing, but never motivated me to do anything. However, as expectations-mode has become more integral to my workflow, the first issue caused me to make a change.
As of 1.4.17 you should be able to write anything that you would usually write in a scenario in a bare expect instead.
I've already published several blog entries that should help if you're interested in migrating your scenarios to bare expectations.
If you were previously using stubbing, your test can be converted in the following way.
There is one type of scenario that I haven't yet addressed, interleaved expectations. I found zero of these types of scenarios in my codebases; however, I'm addressing these types of scenarios here for completeness.
If you run into issues while converting your scenarios, please open an issue on github: https://github.com/jaycfields/expectations/issues?state=open
The functionality in expectations.scenarios was borne out of compromise. I found certain scenarios I wanted to test, but I wasn't sure how to easily test them using what was already available in (bare) expectations. The solution was to add expectations.scenarios, and experiment with various features that make testing as easy as possible.
Truthfully, I've never liked scenarios - I've always viewed them as a necessary evil. First of all, I hate that you can't mix them with bare expectations - this leads to having 2 files or 2 namespaces in 1 file (or you put everything in a scenario, meh). You either can't see all of your tests at the same time (2 files), or you run the risk of your tests not working correctly with other tools (expectations-mode doesn't like having both namespaces in 1 file). Secondly, I think they lead to sloppy tests.
The second complaint causes me to get on my soap-box about test writing, but never motivated me to do anything. However, as expectations-mode has become more integral to my workflow, the first issue caused me to make a change.
As of 1.4.17 you should be able to write anything that you would usually write in a scenario in a bare expect instead.
I've already published several blog entries that should help if you're interested in migrating your scenarios to bare expectations.
- Freezing Time Added To expectations
- Interaction Based Testing Added To expectations
- redef-state Added To expectations
- Using given & expect To Replace scenarios
If you were previously using stubbing, your test can be converted in the following way.
(stubbing [a-fn true] (do-work)) ;;; can now be written as (with-redefs [a-fn (constantly true)] (do-work))A nice side effect of removing stubbing is the reduction of indention if you are using both stubbing and with-redefs. This seems like the right trade-off for me (less indenting, relying on core functions that everyone should know); however, I'm not against adding stubbing again in the future if it becomes a painfully missing feature.
There is one type of scenario that I haven't yet addressed, interleaved expectations. I found zero of these types of scenarios in my codebases; however, I'm addressing these types of scenarios here for completeness.
(scenario (do-work) (expect a b) (do-more-work) (expect c d))Any scenario that has interleaved expectations can be converted in the following way:
(expect c
(do
(do-work)
(assert (= a b))
(do-more-work)
d))
expectations 1.4.17 still has support for scenarios, so you can upgrade and migrate at your own pace. I'll likely leave scenarios in until the point that I change some code that breaks them, then I'll remove them. Of course, if you prefer scenarios, you're welcome to never upgrade, or fork expectations.
If you run into issues while converting your scenarios, please open an issue on github: https://github.com/jaycfields/expectations/issues?state=open
Monday, November 05, 2012
Clojure: Using given & expect To Replace scenarios
The functionality in expectations.scenarios was borne out of compromise. I found certain scenarios I wanted to test, but I wasn't sure how to easily test them using what was already available in (bare) expectations. The solution was to add expectations.scenarios, and experiment with various features that make testing as easy as possible.
Two years later, the features that make sense have migrated back to expectations:
Below is an example of a scenario that ends with multiple expects.
Using given, these scenarios are actually very easy to convert. The given + bare expectation example below tests exactly the same logic.
The test coverage is the same in the second example, but it is important to note that the let will now be executed 3 times instead of 1. This isn't an issue if your tests run quickly, if they don't you may want to revisit the test to determine if it can be written in a different way.
An interesting side-effect occurred while I was converting my scenarios - I found that some of my scenarios could be broken into multiple expectations that were then easier to read and maintain.
For example, the above expectations could be written as the example below.
note: you could simplify even further and remove the given, but that's likely only due to how contrived the test is. Still, the possibility exists that some scenarios will be easily convertible to bare expectations.
Using the technique described here, I've created bare expectations for all of the scenarios in the codebase I'm currently working on - and deleted all references to expectations.scenarios.
Two years later, the features that make sense have migrated back to expectations:
- Freezing Time Added To expectations
- Interaction Based Testing Added To expectations
- redef-state Added To expectations
Below is an example of a scenario that ends with multiple expects.
Using given, these scenarios are actually very easy to convert. The given + bare expectation example below tests exactly the same logic.
The test coverage is the same in the second example, but it is important to note that the let will now be executed 3 times instead of 1. This isn't an issue if your tests run quickly, if they don't you may want to revisit the test to determine if it can be written in a different way.
An interesting side-effect occurred while I was converting my scenarios - I found that some of my scenarios could be broken into multiple expectations that were then easier to read and maintain.
For example, the above expectations could be written as the example below.
note: you could simplify even further and remove the given, but that's likely only due to how contrived the test is. Still, the possibility exists that some scenarios will be easily convertible to bare expectations.
Using the technique described here, I've created bare expectations for all of the scenarios in the codebase I'm currently working on - and deleted all references to expectations.scenarios.
Thursday, November 01, 2012
Clojure: Use expect-let To Share A Value Between expected And actual
Most of the time you can easily divorce the values needed in an expected form and an actual form of an expectation. In those cases, nothing needs to be shared and your test can use a simple bare expect. However, there are times when you need the same value in both the expected and actual forms - and a bare expect doesn't easily provide with a way to accomplish that.
In version 1.4.16 or higher of expectations, you can now use the expect-let macro to let one or more values and reference them in both the expected and actual forms.
Below is a simple example that makes use of expect-let to compare two maps that both have a DateTime.
If possible you should prefer expect, but expect-let gives you another option for the rare cases where you absolutely need to share a value.
In version 1.4.16 or higher of expectations, you can now use the expect-let macro to let one or more values and reference them in both the expected and actual forms.
Below is a simple example that makes use of expect-let to compare two maps that both have a DateTime.
If possible you should prefer expect, but expect-let gives you another option for the rare cases where you absolutely need to share a value.
Clojure: Freezing Time Added To expectations
If you're using expectations and Joda Time, you now have the ability to freeze time in bare expectations (version 1.4.16 and above). The following code demonstrates how you can use the freeze-time macro to set the time, verify anything you need, and allow time to be reset for you.
Under the covers freeze-time is setting the current millis using the DateTime you specify, running your code and resetting the current millis in a finally. As a result, after your code finishes executing, even if finishing involves throwing an exception, the millis of Joda Time will be set back to working as you'd expect.
The freeze-time macro can be used in both the expected and actual forms, and can be nested if you need to set the time multiple times within a single expectation.
Under the covers freeze-time is setting the current millis using the DateTime you specify, running your code and resetting the current millis in a finally. As a result, after your code finishes executing, even if finishing involves throwing an exception, the millis of Joda Time will be set back to working as you'd expect.
The freeze-time macro can be used in both the expected and actual forms, and can be nested if you need to set the time multiple times within a single expectation.
Clojure: Interaction Based Testing Added To expectations
The vast majority of testing I do these days is state-based; however, there are times when I need to test an interaction (e.g. writing to a file or printing to standard out). The ability to test interactions has been in expectations.scenarios for quite awhile, but there isn't any reason that you need a scenario to test an interaction - so, as of version 1.4.16, you also have the ability to test interactions with bare expectations.
The following test shows how you can specify an expected interaction. This test passes.
Writing the test should be straightforward - expect the interaction and then call the code that causes the interaction to happen.
As I was adding this behavior I enhanced the error reporting. Below you can find a failing test and the output that is produced.
As you can see, all three calls to the 'one' function are reported. If the number of args used to call 'one' are of the same size as the expected args, each arg is compared in detail; otherwise the two lists are compared in detail (but the elements are not).
As you can see in this failure the first argument, "hello", matches.
That's it. Hopefully these interaction tests follow the principle of least surprise, and are easy for everyone to use.
The following test shows how you can specify an expected interaction. This test passes.
Writing the test should be straightforward - expect the interaction and then call the code that causes the interaction to happen.
As I was adding this behavior I enhanced the error reporting. Below you can find a failing test and the output that is produced.
As you can see, all three calls to the 'one' function are reported. If the number of args used to call 'one' are of the same size as the expected args, each arg is compared in detail; otherwise the two lists are compared in detail (but the elements are not).
As you can see in this failure the first argument, "hello", matches.
; got: (one "hello" {2 3, :a 1})
; arg1: matches
; expected arg2: {:a :b, :c {:ff :gg, :dd :ee}}
; actual arg2: {2 3, :a 1}
; 2 with val 3 is in actual, but not in expected
; :c {:dd with val :ee is in expected, but not in actual
; :c {:ff with val :gg is in expected, but not in actual
; :a expected: :b
; was: 1
Anytime an argument matches expectations will simply print "matches". You can also specify :anything as an argument, to ignore that argument and always 'match'. The following test shows an example of matching the second argument, while the first argument is no longer matching.
That's it. Hopefully these interaction tests follow the principle of least surprise, and are easy for everyone to use.