Showing posts with label unit testing. Show all posts
Showing posts with label unit testing. Show all posts

Thursday, June 16, 2016

Maintainability and Expect Literals

Recently, Stephen Schaub asked the following on the wewut group:
Several of the unit test examples in the book verify the construction of both HTML and plain text strings. Jay recommends using literal strings in the assertions. However, this strikes me as not a particularly maintainable approach. If the requirements regarding the formatting of these strings changes (a very likely scenario), every single test that verifies one of these strings using a literal must be updated. Combined with the advice that each test should check only one thing, this leads to a large number of extremely brittle tests.

Am I missing something here? I can appreciate the reasons Jay recommends using literals in the tests. However, it seems that we pay a high maintainability price in exchange for the improved readability.
I responded to Stephen; however, I've seen similar questions asked a few times. Below are my extended thoughts regarding literals as expected values.

In general, given the option of having many similar strings (or any literal) vs a helper function, I would always prefer the literal. When a test is failing I only care about that single failing test. If I have to look at the helper function I no longer have the luxury of staying focused on the single test; now I need to consider what the helper function is giving me and what it's giving all other callers. Suddenly the scope of my work has shifted from one test to all of the tests coupled by this helper function. If this helper function wasn't written by me, this expansion in scope wasn't even my decision, it was forced upon me by the helper function creator. In the best case the helper function could return a single, constant string. The scope expansion becomes even worse when the helper function contains code branches.

As for alternatives, my solution would depend on the problem. If the strings were fairly consistent, I would likely simply duplicate everything knowing that any formatting changes can likely be addressed using a bulk edit via find and replace. If the strings were not consistent, I would look at breaking up the methods in a way that would allow me to verify the code branches using as little duplication as possible, e.g. if I wanted to test a string that dynamically changed based on a few variables, I would look to test those variables independently, and then only have a few tests for the formatting.

A concrete example will likely help here. Say I'm writing a trading system and I need to display messages such as

"paid 10 on 15 APPL. $7 Commission. spent: $157"
"paid 1 on 15 VTI. Commission free. spent: $15"
"sold 15 APPL at 20. $7 Commission. collected: $293"
"sold 15 VTI at 2. Commission free. collected: $30"

There's quite a bit of variation in those messages. You could have 1 function that creates the entire string:
confirmMsg(side, size, px, ticker)

However, I think you'd end up with quite a few verbose tests. Given this problem, I would look to break down those strings into smaller, more focused functions, for example:

describeOrder(side, size, px, ticker)
describeCommission(ticker)
describeTotal(side, size, px, ticker)

Now that you've broken down the function, you're free to test the code paths of the more focused functions, and the test for confirmMsg becomes trivial. Something along the lines of
assertEquals("paid 10 on 15 APPL",
  describeOrder("buy", 10, 15, {tickerName:"APPL",commission:"standard"}))
assertEquals("sell 15 APPL at 10",
  describeOrder("sell", 10, 15, {tickerName:"APPL",commission:"standard"}))

assertEquals("$7 Commission", 
  describeCommission({tickerName:"APPL",commission:"standard"}))
assertEquals("Commission free", 
  describeCommission({tickerName:"APPL",commission:"free"}))

assertEquals("spent: $157", 
  describeOrder("buy", 10, 15, {tickerName:"APPL",commission:"standard"}))
assertEquals("collected: $143", 
  describeOrder("sell", 10, 15, {tickerName:"APPL",commission:"standard"}))
assertEquals("spent: $150", 
  describeOrder("buy", 10, 15, {tickerName:"APPL",commission:"free"}))
assertEquals("collected: $150", 
  describeOrder("sell", 10, 15, {tickerName:"APPL",commission:"free"}))

assertEquals("order. commission. total", 
  confirmMsg("order", "commission", "total"))
I guess I could summarize it by saying, I should be able to easily find and replace my expected literals. If I cannot, then I have an opportunity to further break down a method and write more focused tests on the newly introduced, more granular tests.

Monday, April 06, 2015

Unit Testing Points of View, Probably

Michael Feathers, Brian Marick, and I are collaborating to create a new book: Unit Testing Points of View ... probably.

Origin

In 2014 Martin Fowler provided Technical Review for Working Effectively with Unit Tests. As part of his feedback he said something along the lines of: I'm glad you wrote this book, and I'll likely write a bliki entry noting what I agree with and detailing what I would do differently. I'm still looking forward to that entry, and I think the idea is worth extending beyond Martin.

Unit testing is now mainstream, has tons of entry level books, and has a great reference book. The result is a widely accepted idea that you can quickly and easily adopt; unfortunately, I've found little in the way of documenting pattern trade-offs. I believe that combination leads to a lot of waste. To help avoid some of that waste, I'd like to see more written about why you may want to choose one Unit Testing style over another. That was the inspiration and my goal for Working Effectively with Unit Tests. Growing Object-Oriented Software is another great example of a book that documents both the hows and whys of unit testing. After that, if you're looking for tradeoff guidance... good luck.

Is this a problem? I think so. Without discussion of tradeoffs, experience reports, and concrete direction you end up with hours wasted on bad tests and proclamations of TDD's death. The report of TDD's death was an exaggeration, and the largest clue was the implication that TDD and Unit Testing were synonymous. To this day, Unit Testing is still largely misunderstood.

What could be

Working Effectively with Unit Tests starts with one set of opinionated Unit Tests and evolves to the types of Unit Tests that I find more productive. There's no reason this evolution couldn't be extended by other authors. This is the vision we have for Unit Testing Points of View. Michael, Brian, and I began by selecting a common domain model. The first few chapters will detail the hows and whys of how I would test the common domain model. After I've expressed what motivates my tests, Brian or Michael will evolve the tests to a style they find superior. After whoever goes second (Brian or Michael) finishes, the other will continue the evolution. The book will note where we agree, but the majority of the discussion will occur around where our styles differ and what aspect of software development we've chosen emphasize by using an alternative approach.

Today, most teams fail repeatedly with Unit Tests, leading to (at best) significant wasted time and (at worst) abandoning the idea with almost nothing gained. We aim to provide tradeoff guidance that will help teams select a Unit Testing approach that best fits their context.

As I said above: There's no reason this evolution couldn't be extended by other authors. In fact, that's our long term hope. Ideally, Brian, Michael and I write volume one of Unit Testing Points of View. Volume two could be written by Kevlin Henney, Steve Freeman, and Roy Osherove - or anyone else who has interest in the series. Of course, given the original inspiration, we're all hoping Martin Fowler clears some time in his schedule to take part in the series.

Why "Probably"?

Writing a book is almost always a shot in the dark. Martin, Michael, Brian, and I all think this a book worth writing (and a series worth starting), but we have no idea if people actually desire such a book. In the past you took the leap expecting failure and praying for success. Michael, Brian, and I believe there's a better way: leanpub interest lists (here's Unit Testing Points of View's). We're looking for 15,000 people to express interest (by providing their email addresses). I'm writing the first 3 chapters, and they'll be made available when we cross the 15k mark. At that point, Michael and Brian will know that the interest is real, and their contributions need to be written. If we never cross the 15k mark then we know such a book isn't desired, and we shouldn't waste our time.

If you'd like to see this project happen, please do sign up on the interest list and encourage others to as well (here's a tweet to RT, if you'd like to help).

Wednesday, December 17, 2014

Working Effectively with Unit Tests Official Launch

Today marks the official release release of Working Effectively with Unit Tests. The book is available in various formats:
I’m very happy with the final version. Michael Feathers wrote a great foreword. I incorporated feedback from dozens of people - some that have been friends for years, and some that I’d never previously met. I can’t say enough great things about http://leanpub.com, and I highly recommend it for getting an idea out there and making it easy to get fast feedback. 

As far as the softcover edition, I had offers from a few major publishers, but in the end none of them would allow me to continue to sell on leanpub at the same time. I strongly considered caving to the demands of the major publishers, but ultimately the ability to create a high quality softcover and make it available on Amazon was too tempting to pass up.

The feedback has been almost universally positive - the reviews are quite solid on goodreads (http://review.wewut.com). I believe the book provides specific, concise direction for effective Unit Testing, and I hope it helps increase the quality of the unit tests found in the wild.

If you'd like to try before you buy, there's a sample available in pdf format or on the web.


Wednesday, May 21, 2014

Working Effectively with Unit Tests

Unit Testing has moved from fringe to mainstream, which is a great thing. Unfortunately, as a side effect developers are creating mountains of unmaintainable tests. I've been fighting the maintenance battle pretty aggressively for years, and I've decided to write a book that captures what I believe is the most effective way to test.

From the Preface

Over a dozen years ago I read Refactoring for the first time; it immediately became my bible. While Refactoring isn’t about testing, it explicitly states: If you want to refactor, the essential precondition is having solid tests. At that time, if Refactoring deemed it necessary, I unquestionably complied. That was the beginning of my quest to create productive unit tests.

Throughout the 12+ years that followed reading Refactoring I made many mistakes, learned countless lessons, and developed a set of guidelines that I believe make unit testing a productive use of programmer time. This book provides a single place to examine those mistakes, pass on the lessons learned, and provide direction for those that want to test in a way that I’ve found to be the most productive.
The book does touch on some theory and definition, but the main purpose is to show you how to take tests that are causing you pain and turn them into tests that you're happy to work with.

For example, the book demonstrates how to go from...

looping test with many (built elsewhere) collaborators
.. to individual tests that expect literals, limit scope, explicitly define collaborators, and focus on readability
.. to fine-grained tests that focus on testing a single responsibility, are resistant to cascading failures, and provide no friction for those practicing ruthless Refactoring.
As of right now, you can read the first 2 chapters for free at https://leanpub.com/wewut/read

I'm currently ~25% done with the book, and it's available now for $14.99. My plan is to raise the price to $19.99 when I'm 50% done, and $24.99 when I'm 75% done. Leanpub offers my book with 100% Happiness Guarantee: Within 45 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks. Therefore, if you find the above or the free sample interesting, you might want to buy it now and save a few bucks.

Buy Now here: https://leanpub.com/wewut

Monday, May 19, 2014

Weighing in on Long Live Testing

DHH recently wrote a provocative piece that gave some views into how he does and doesn't test these days. While I don't think I agree with him completely, I applaud his willingness to speak out against TDD dogma. I've written publicly about not buying the pair-programming dogma, but I hadn't previously been brave enough to admit that I no longer TDD the vast majority of the time.

The truth is, I haven't been dogmatic about TDD in quite some time. Over 6 years ago I was on a ThoughtWorks project where I couldn't think of a single good reason to TDD the code I was working on. To be honest, there weren't really any reasons that motivated me to write tests at all. We were working on a fairly simple, internal application. They wanted software as fast as they could possibly get it, and didn't care if it crashed fairly often. We kept everything simple, manually tested new features through the UI, and kept our customer's very happy.

There were plenty of reasons that we could have written tests. Reasons that I expect people will want to yell at me right now. To me, that's actually the interesting, and missing part, of the latest debate on TDD. I don't see people asking: Why are we writing this test? Is TDD good or bad? That depends; TDD is just a tool, and often the individual is the determining factor when it comes to how effective a tool is. If we start asking "Why?", it's possible to see how TDD could be good for some people, and bad for DHH.

I've been quietly writing a book on Working Effectively with Unit Tests, and I'll have to admit that it was really, really hard not to jump into the conversation with some of the content I've recently written. Specifically, I think this paragraph from the Preface could go a long way to helping people understand an opposing argument.

Why Test?

The answer was easy for me: Refactoring told me to. Unfortunately, doing something strictly because someone or something told you to is possibly the worst approach you could take. The more time I invested in testing, the more I found myself returning to the question: Why am I writing this test?

There are many motivators for creating a test or several tests:
  • validating the system
    • immediate feedback that things work as expected
    • prevent future regressions
  • increase code-coverage
  • enable refactoring of legacy codebase
  • document the behavior of the system
  • your manager told you to
  • Test Driven Development
    • improved design
    • breaking a problem up into smaller pieces
    • defining the "simplest thing that could possibly work"
  • customer approval
  • ping pong pair-programming
Some of the above motivators are healthy in the right context, others are indicators of larger problems. Before writing any test, I would recommend deciding which of the above are motivating you to write a test. If you first understand why you're writing a test, you'll have a much better chance of writing a test that is maintainable and will make you more productive in the long run.

Once you start looking at tests while considering the motivator, you may find you have tests that aren't actually making you more productive. For example, you may have a test that increases code-coverage, but provides no other value. If your team requires 100% code-coverage, then the test provides value. However, if you team has abandoned the (in my opinion harmful) goal of 100% code-coverage, then you're in a position to perform my favorite refactoring: delete.
I don't actually know what motivates DHH to test, but if we assumed he cares about validating the system, preventing future regressions, and enabling refactoring (exclusively) then there truly is no reason to TDD. That doesn't mean you shouldn't; it just means, given what he values and how he works, TDD isn't valuable to him. Of course, conversely, if you value immediate feedback, problems in small pieces, and tests as clients that shape design, TDD is probably invaluable to you.

I find myself doing both. Different development activities often require different tools; i.e. Depending on what I'm doing, different motivators apply, and what tests I write change (hopefully) appropriately.

To be honest, if you look at your tests in the context of the motivators above, that's probably all you need to help you determine whether or not your tests are making you more or less effective. However, if you want more info on what I'm describing, you can pick up the earliest version of my upcoming book. (cheaply, with a full refund guarantee)

Wednesday, August 11, 2010

clojure.test Introduction

I'll admit it, the first thing I like to do when learning a new language is fire up a REPL. However, I'm usually ready for the next step after typing in a few numbers, strings and defining a function or two.

What feels like centuries ago, Mike Clark wrote an article about using unit testing to learn a new language. Mike was ahead of his time. This blog entry should help you if you want to follow Mike's advice.

Luckily, Clojure has built in support for simple testing. (I'm currently using Clojure 1.2, you can download it from clojure.org)

Before we get started, let's make sure everything is working. Save a file with the following clojure in it and run* it with clojure.
(ns clojure.test.example
(:use clojure.test))

(run-all-tests)
If everything is okay, you should see something similar the following output.
Testing clojure.walk

Testing clojure.core

(a bunch of other namespaces tested)

Testing clojure.zip

Ran 0 tests containing 0 assertions.
0 failures, 0 errors.
If you've gotten this far, you are all set to start writing your own tests. If you are having any trouble, I suggest logging into the #clojure IRC chat room on Freenode.net

The syntax for defining tests is very simple. The following test verifies that 1 + 1 = 2. You'll want to add the test after the ns definition and before the (run-all-tests) in the file you just created.
(deftest add-1-to-1
(is (= 2 (+ 1 1))))
Running the test should produce something similar to the following output.
Testing clojure.walk

Testing clojure.test.example

(a bunch of other namespaces tested)

Ran 1 tests containing 1 assertions.
0 failures, 0 errors.
We see all of the standard clojure namespaces; however, we see our namespace (clojure.test.example) in the results as well. The output at the bottom also tells us that 1 test with 1 assertion was executed.

The following example shows testing a custom add function. (we will add additional tests from here, without ever deleting the old tests)
(defn add [x y] (+ x y))

(deftest add-x-to-y
(is (= 5 (add 2 3))))
If everything goes to plan, running your tests should now produce the following text towards the bottom of the output.
Ran 2 tests containing 2 assertions.
0 failures, 0 errors.
At this point you might want to pass in a few different numbers to verify that add works as expected.
(deftest add-x-to-y-a-few-times
(is (= 5 (add 2 3)))
(is (= 5 (add 1 4)))
(is (= 5 (add 3 2))))
Running the tests shows us our status
Ran 3 tests containing 5 assertions.
0 failures, 0 errors.
This works perfectly fine; however, clojure.test also provides are for verifying several values.

The following example tests the same conditions using are.
(deftest add-x-to-y-a-using-are
(are [x y] (= 5 (add x y))
2 3
1 4
3 2))
And, the unsurprising results.
Ran 4 tests containing 8 assertions.
That's a simple are; however, you can do whatever you need in the form. Let's grab the value out of a map as an additional example.
(deftest grab-map-values-using-are
(are [y z] (= y (:x z))
2 {:x 2}
1 {:x 1}
3 {:x 3 :y 4}))
Leaving us with
Ran 5 tests containing 11 assertions.
The is and are macros will be all that you need for 90% of all the tests you'll ever want to write. For additional assertions and more details you can check out the clojure.test documentation.

Advanced Topics (very unnecessary to get started)

I get annoyed with noise in my test results. Our results have been very noisy due to the namespace reporting. The run-all-tests function takes a regular expression (documented here). We can change our test running call to include a regular expression, as the following example shows.
(run-all-tests #"clojure.test.example")
Once we switch to providing a regular expression the results should be limited to the following output.

Testing clojure.test.example

Ran 5 tests containing 11 assertions.
0 failures, 0 errors.
This approach works fine for our current sample file; however, it seems like a better solution would be to stop reporting namespaces that do not contain any tests. The following snippet changes the report multimethod to ignore namespaces that don't contain any tests.
(defmethod report :begin-test-ns [m]
(with-test-out
(when (some #(:test (meta %)) (vals (ns-interns (:ns m))))
(println "\nTesting" (ns-name (:ns m))))))
If you're just getting started, don't worry you don't need to understand what's going on in that snippet. I've copied the original report method and made it conditional by adding the code in bold. As a result, the namespace is only printed if it contains any tests.

Now that our results are clean, let's talk about ways of getting those results.

Adding calls to the run-all-tests function isn't a big deal when working with one namespace; however, you'll need to get clever when you want to run a suite of tests. I've been told that leiningen and Maven have tasks that allow you to run all the tests. You might want to start there. I don't currently use either one, and I'm lazy. I don't want to set up either, especially since all I want to do is run all my tests.

It turns out it's very easy to add a shutdown hook in Java. So, as a simple solution, I run all my tests from the Java shutdown hook.
(.addShutdownHook
(Runtime/getRuntime)
(proxy [Thread] []
(run []
(run-all-tests))))
In general, I create a test_helper.clj with the following code.
(ns test-helper
(:use clojure.test))

(defmethod report :begin-test-ns [m]
(with-test-out
(if (some #(:test (meta %)) (vals (ns-interns (:ns m))))
(println "\nTesting" (ns-name (:ns m))))))

(.addShutdownHook
(Runtime/getRuntime)
(proxy [Thread] []
(run []
(run-all-tests))))
Once you've created a test_helper.clj you can use test-helper (just like you used clojure.test) (example below) and your tests will automatically be run on exit, and only namespaces with tests will be included in the output.

It's worth noting that some clojure.contrib namespaces seem to include tests, so in practice I end up using a regular expression that ignores all namespaces beginning with "clojure"** when running all tests. With all of those ideas combined, I find I can execute all my tests or only the tests in the current namespace very easily.

Below you can find all the code from this entry.

clojure.test.example.clj
(ns clojure.test.example
(:use clojure.test test-helper))

(deftest add-1-to-1
(is (= 2 (+ 1 1))))

(defn add [x y] (+ x y))

(deftest add-x-to-y
(is (= 5 (add 2 3))))

(deftest add-x-to-y-a-few-times
(is (= 5 (add 2 3)))
(is (= 5 (add 1 4)))
(is (= 5 (add 3 2))))

(deftest add-x-to-y-a-using-are
(are [x y] (= 5 (add x y))
2 3
1 4
3 2))

(deftest grab-map-values-using-are
(are [y z] (= y (:x z))
2 {:x 2}
1 {:x 1}
3 {:x 3 :y 4}))
test_helper.clj
(ns test-helper
(:use clojure.test))

(defmethod report :begin-test-ns [m]
(with-test-out
(if (some #(:test (meta %)) (vals (ns-interns (:ns m))))
(println "\nTesting" (ns-name (:ns m))))))

(.addShutdownHook
(Runtime/getRuntime)
(proxy [Thread] []
(run []
(run-all-tests))))

* Running a clojure file should be as easy as: java -cp /path/to/clojure.jar clojure.main -i file.to.run.clj

** (run-all-tests #"[^(clojure)].*") ; careful though, now your clojure.test.example tests will be ignored. Don't let that confuse you.

Wednesday, February 24, 2010

The Maintainability of Unit Tests

At speakerconf 2010 discussion repeatedly arose around the idea that unit tests hinder your ability to refactor and add new features. It's true that tests are invaluable when refactoring the internals of a class as long as the interface doesn't change. However, when the interface does change, updating the associated tests is often the vast majority of the effort. Additionally, if a refactoring changes the interaction between two or more classes, the vast majority of the time is spent fixing tests, for several classes.

In my experience, making the interface or interaction change often takes 15-20% of the time, while changing the associated tests take the other 80-85%. When the effort is split that drastically, people begin to ask questions.

Should I write Unit Tests? The answer at speakerconf was: Probably, but I'm interested in hearing other options.

Ayende proposed that scenario based testing was a better solution. His examples drove home the point that he was able to make large architectural refactorings without changing any tests. Unfortunately, his tests suffered from the same problems that Integration Test advocates have been dealing with for years: Long Running Tests (20 mins to run a suite!) and Poor Defect Localization (where did things go wrong?). However, despite these limitations, he's reporting success with this strategy.

In my opinion, Martin Fowler actually answered this question correctly in the original Refactoring book.
The key is to test the areas that you are most worried about going wrong. That way you get the most benefit for your testing effort.
It's a bit of a shame that sentence lives in Refactoring and not in every book written for developers beginning to test their applications. After years of trying to test everything, I stumbled upon that sentence while creating Refactoring: Ruby Edition. That one sentence changed my entire attitude on Unit Testing.

I still write Unit Tests, but I only focus on testing the parts that provide the most business value.

An example
you find yourself working on an insurance application for a company that stores it's policies by customer SSN. Your application is likely to have several validations for customer information.

The validation that ensures a SSN is 9 numeric digits is obviously very important.

The validation that the customer name is alpha-only is probably closer to the category of "nice to have". If the alpha-only name validation is broken or removed, the application will continue to function almost entirely normally. And, the most likely problem is a typo - probably not the end of the world.

It's usually easy enough to add validations, but you don't need to test every single validation. The value of each validation should be used to determine if a test is warranted.
How do I improve the maintainability of my tests? Make them more concise.

Once you've determined you should write a test, take the time to create a concise test that can be maintained. The longer the test, the more likely it is to be ignored or misunderstood by future readers.

There are several methods for creating more concise tests. My recent work is largely in Java, so my examples are Java related. I've previously written about my preferred method for creating objects in Java Unit Tests. You can also use frameworks that focus on simplicity, such as Mockito. But, the most important aspect of creating concise tests is taking a hard look at object modeling. Removing constructor and method arguments is often the easiest way to reduce the amount of noise within a test.

If you're not using Java, the advice is the same: Remove noise from your tests by improving object modeling and using frameworks that promote descriptive, concise syntax. Removing noise from tests always increases maintainability.

That's it? Yes. I find when I only test the important aspects of an application and I focus on removing noise from the tests that I do write, the maintainability issue is largely addressed. As a result the pendulum swings back towards a more even effort split between features & refactoring vs updating tests.

Tuesday, April 15, 2008

Story 9: Display Customer Support Number

I thought it might be beneficial to create an entry that shows my workflow when implementing a few feature.

I generally work from stories, so it seemed logical to create an example story.
As a consumer
I would like to see the customer support number
So that I can call if I have a problem
There are a few technical details that make this story a bit more complicated than expected.
  • The site is one rails application, but it needs to be branded differently for each affiliate.
  • Based on the affiliate name (which will come from the params) we will need to connect to the web service specific to that affiliate.
  • The customer support number will come from the affiliate web service.
  • The affiliate web service will require an authentication token in production, but it is optional in development mode.
Based on the story and the known technical details we come up with the following list of tasks.
  • task 1: Create a configuration file (yaml) that will be stored on the web server. This configuration file will map the affiliate name to their web service.
  • task 2: Retrieve the customer support number from the affiliate web service.
  • task 3: Add authentication since it will be required in production.
You may prefer more fine grained tasks (I usually do), but these should do for example purposes.

Task 1
The first thing I generally do is write a functional test. I want unit tests as well, and I'll probably write the unit tests and have them passing before I have the functional test passing, but starting with a functional test allows me to think at a higher level instead of getting bogged down in the details of the implementation. Therefore, the first test I write would probably look like this RSpec spec.

describe Configuration do
it "gets the support number from the client specific configuration" do
Configuration.for("localhost").support_number.should eql("212.646.9208")
end
end

[RSpec link]

Now, I know that this test isn't going to pass for awhile, but it gives me a good reminder of the direction that I want to continue to head in. At this point I would be looking to write a unit test that verifies that I can get the address from a stub configuration file. Having a unit test will allow me to get the Configuration class working without worrying about the configuration file. The unit test will also run significantly faster than the functional test, which benefits us in the long term.

Expectations do
expect "some url" do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.for("localhost")
end
end

[expectations link]

Ultimately I know that the Configuration::for method won't return the address, but my first task is to get the configuration file working, and this test is a good way to complete that task.

At this point I have failing tests, so I write the implementation of the Configuration class. Below is all of the code written at this point. (Of course, in a real application it would be broken out to appropriate files)

require 'yaml'
require 'rubygems'
require 'expectations'
require 'spec'

class Configuration < Struct.new(:support_number)
def self.for(environment)
YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
end
end

Expectations do
expect "some url" do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.for("localhost")
end
end
# >> Expectations .
# >> Finished in 0.001 seconds
# >>
# >> Success: 1 fulfilled

describe Configuration do
it "gets the support number from the client specific configuration" do
Configuration.for("localhost").support_number.should eql("212.646.9208")
end
end
# >> F
# >>
# >> 1)
# >> NoMethodError in 'Configuration gets the support number from the client specific configuration'
# >> undefined method `support_number' for "127.0.0.1/config":String
# >>
# >> Finished in 0.008434 seconds
# >>
# >> 1 example, 1 failure

The unit test is passing; therefore, assuming that we've stubbed the configuration file correctly we've completed our first task.

Task 2
Our functional test is sufficient for verifying that task two is done correctly, so the next step is to write another unit test. For the purpose of the example, assume that there's a ConfigurationGateway::retrieve_from method that we can use to get the configuration from the affiliate web service. The configuration information is going to be returned by the gateway as a hash, so we'll create an appropriate stub so our unit tests don't rely on the real service. Our next unit test would look something like the following example.

Expectations do
expect "support number" do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
ConfigurationGateway.stubs(:retrieve_from).returns(:support_number => "support number")
Configuration.for("localhost").support_number
end
end

A quick change to the Configuration class and we have our new unit test passing.

class Configuration < Struct.new(:support_number)
def self.for(environment)
configuration_url = YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
self.new(ConfigurationGateway.retrieve_from(configuration_url)[:support_number])
end
end

Unfortunately, this change has broken our previously created unit test.

Expectations do
expect "some url" do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.for("localhost")
end
end
# >> Expectations F
# >> Finished in 0.001 seconds
# >>
# >> Failure: 1 failed, 0 errors, 0 fulfilled
# >>
# >> --Failures--
# >> -:40:in `expect'
# >> file <->
# >> line <40>
# >> expected: <"some url"> got: <#<struct Configuration support_number="212.646.9208">>

Our first unit test no longer works because we are no longer returning the address from the Configuration::for method. In fact, we don't need to expose the address outside of the Configuration class at all, so a behavior based unit test is probably a good choice for replacing our first unit test. The example below represents the new method for verifying that the configuration file is correctly returning the address.

Expectations do
expect ConfigurationGateway.to.receive(:retrieve_from).with("some url").returns({}) do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.for("localhost")
end
end

At this point, if the ConfigurationGateway is properly working and if your configuration file is in place, your functional test should also be passing.

Here's all the code we've written so far, including a fake ConfigurationGateway to show our Configuration class is working.

require 'yaml'
require 'rubygems'
require 'expectations'
require 'spec'

class Configuration < Struct.new(:support_number)
def self.for(environment)
configuration_url = YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
self.new(ConfigurationGateway.retrieve_from(configuration_url)[:support_number])
end
end

class ConfigurationGateway
def self.retrieve_from(url)
# In actual code this would go to a external service...
{:support_number => "212.646.9208"}
end
end

Expectations do
expect ConfigurationGateway.to.receive(:retrieve_from).with("some url").returns({}) do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.for("localhost")
end

expect "support number" do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
ConfigurationGateway.stubs(:retrieve_from).returns(:support_number => "support number")
Configuration.for("localhost").support_number
end
end
# >> Expectations ..
# >> Finished in 0.001 seconds
# >>
# >> Success: 2 fulfilled

describe Configuration do
it "gets the support number from the client specific configuration" do
Configuration.for("localhost").support_number.should eql("212.646.9208")
end
end
# >> .
# >>
# >> Finished in 0.008254 seconds
# >>
# >> 1 example, 0 failures

And, here's the configuration file that you'll also need to get the tests passing.

# config.yaml
localhost:
configuration_url: 127.0.0.1/config

At this point both our functional and unit tests are passing, and we are done with task two.

Task 3
For the purposes of this example, we're going to change the ConfigurationGateway to require the authentication token. If this were real code, I would expect the ConfigurationGateway to have this ability already built in, since we'd need a way to make the development server behave like production.

class ConfigurationGateway
def self.retrieve_from(url, token)
# In actual code this would go to a external service...
{:support_number => "212.646.9208"}
end
end

This change causes the functional test to break, but the unit tests do not break. That's OK. In order to get the functional tests passing we'll need to change the implementation, which will cause the unit tests to break. In the end, all of our tests will be updated to represent the correct interactions.

To make the functional tests pass we'll need to add a call to the AuthenticaionGateway to get the next authentication token. Following this change, the code below represents the new Configuration::for method.

class Configuration < Struct.new(:support_number)
def self.for(environment)
configuration_url = YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
token = AuthenticationGateway.next_token
self.new(ConfigurationGateway.retrieve_from(configuration_url, token)[:support_number])
end
end

With this change in place the functional test now passes. To get it working locally, you'll need to add the fake AuthenticationGateway class that can be found below.

class AuthenticationGateway
def self.next_token
# In actual code this would go to an external service...
1979
end
end

At this point your functional test should be passing, but a unit test should be failing.

Expectations do
expect ConfigurationGateway.to.receive(:retrieve_from).with("some url").returns({}) do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.for("localhost")
end
end
# >> Expectations F.
# >> Finished in 0.00104 seconds
# >>
# >> Failure: 1 failed, 0 errors, 1 fulfilled
# >>
# >> --Failures--
# >> -:30:in `expect'
# >> file <->
# >> line <30>
# >> #<Mock:0x5168f0>.retrieve_from('some url', 1979) - expected calls: 0, actual calls: 1
# >> Similar expectations:
# >> #<Mock:0x5168f0>.retrieve_from('some url')

We need to update the ConfigurationGateway::for expectation, but we also need to remove the call to the external service from our unit tests. We do that by stubbing the AuthenticationGateway::next_token method.

Expectations do
expect ConfigurationGateway.to.receive(:retrieve_from).with("some url", "some token").returns({}) do
AuthenticationGateway.stubs(:next_token).returns("some token")
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.for("localhost")
end

expect "support number" do
AuthenticationGateway.stubs(:next_token).returns("some token")
File.stubs(:read).returns("localhost:\n configuration_url: some url")
ConfigurationGateway.stubs(:retrieve_from).returns(:support_number => "support number")
Configuration.for("localhost").support_number
end
end

At this point we are done with task three. The code below represents all the code necessary for completing all 3 of our tasks.

require 'yaml'
require 'rubygems'
require 'expectations'
require 'spec'

class Configuration < Struct.new(:support_number)
def self.for(environment)
configuration_url = YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
token = AuthenticationGateway.next_token
self.new(ConfigurationGateway.retrieve_from(configuration_url, token)[:support_number])
end
end

class ConfigurationGateway
def self.retrieve_from(url, token)
# In actual code this would go to a external service...
{:support_number => "212.646.9208"}
end
end

class AuthenticationGateway
def self.next_token
# In actual code this would go to an external service...
1979
end
end

Expectations do
expect ConfigurationGateway.to.receive(:retrieve_from).with("some url", "some token").returns({}) do
AuthenticationGateway.stubs(:next_token).returns("some token")
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.for("localhost")
end

expect "support number" do
AuthenticationGateway.stubs(:next_token).returns("some token")
File.stubs(:read).returns("localhost:\n configuration_url: some url")
ConfigurationGateway.stubs(:retrieve_from).returns(:support_number => "support number")
Configuration.for("localhost").support_number
end
end

describe Configuration do
it "gets the support number from the client specific configuration" do
Configuration.for("localhost").support_number.should eql("212.646.9208")
end
end

Refactoring
We are done with our tasks, but we aren't ready to move on to the next card yet. Our functional test is great, but our unit tests specify way too much. They use stubs instead of mocks, which helps with creating more robust tests, but they suffer from High Implementation Specification.

The solution is to create smaller methods that are more test friendly.

The first refactoring is to change the configuration_url and token local variables to be method calls instead.

class Configuration < Struct.new(:support_number)
def self.for(environment)
self.new(ConfigurationGateway.retrieve_from(configuration_url(environment), authentication_token)[:support_number])
end

def self.configuration_url(environment)
YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
end

def self.authentication_token
AuthenticationGateway.next_token
end
end

This change to the Configuration class makes our second unit test a bit more readable and robust.

Expectations do
expect "support number" do
Configuration.stubs(:authentication_token)
Configuration.stubs(:configuration_url)
ConfigurationGateway.stubs(:retrieve_from).returns(:support_number => "support number")
Configuration.for("localhost").support_number
end
end

At this point our second unit test looks pretty good, but the first one is still testing a bit too much. We can solve this by breaking the first test into 4 different tests. After breaking the first test up, our unit tests would look like the code below.

Expectations do
expect ConfigurationGateway.to.receive(:retrieve_from).with(nil, "token") do
Configuration.stubs(:authentication_token).returns("token")
Configuration.stubs(:configuration_url)
Configuration.configuration_results("localhost")
end

expect ConfigurationGateway.to.receive(:retrieve_from).with("configuration url", nil) do
Configuration.stubs(:authentication_token)
Configuration.stubs(:configuration_url).returns("configuration url")
Configuration.configuration_results("localhost")
end

expect AuthenticationGateway.to.receive(:next_token) do
Configuration.authentication_token
end

expect "some url" do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.configuration_url("localhost")
end

expect "support number" do
Configuration.stubs(:configuration_results).returns(:support_number => "support number")
Configuration.for("localhost").support_number
end
end

Whether or not to break the test apart would be a judgment call. The number of things I was specifying would drive me to go ahead and break it up, but if there had only been 5 values specified I might have left the test the way it was.

The Result
Below is all the code written for this entry. I hope you find it helpful.

require 'yaml'
require 'rubygems'
require 'expectations'
require 'spec'

class Configuration < Struct.new(:support_number)
def self.for(environment)
self.new(configuration_results(environment)[:support_number])
end

def self.configuration_results(environment)
ConfigurationGateway.retrieve_from(configuration_url(environment), authentication_token)
end

def self.configuration_url(environment)
YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
end

def self.authentication_token
AuthenticationGateway.next_token
end
end

class ConfigurationGateway
def self.retrieve_from(url, token)
{:support_number => "212.646.9208"}
end
end

class AuthenticationGateway
def self.next_token
1979
end
end

Expectations do
expect ConfigurationGateway.to.receive(:retrieve_from).with(nil, "token") do
Configuration.stubs(:authentication_token).returns("token")
Configuration.stubs(:configuration_url)
Configuration.configuration_results("localhost")
end

expect ConfigurationGateway.to.receive(:retrieve_from).with("configuration url", nil) do
Configuration.stubs(:authentication_token)
Configuration.stubs(:configuration_url).returns("configuration url")
Configuration.configuration_results("localhost")
end

expect AuthenticationGateway.to.receive(:next_token) do
Configuration.authentication_token
end

expect "some url" do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.configuration_url("localhost")
end

expect "support number" do
Configuration.stubs(:configuration_results).returns(:support_number => "support number")
Configuration.for("localhost").support_number
end
end

describe Configuration do
it "gets the support number from the client specific configuration" do
Configuration.for("localhost").support_number.should eql("212.646.9208")
end
end