Showing posts with label Testing Refactorings. Show all posts
Showing posts with label Testing Refactorings. Show all posts

Wednesday, December 17, 2014

Working Effectively with Unit Tests Official Launch

Today marks the official release release of Working Effectively with Unit Tests. The book is available in various formats:
I’m very happy with the final version. Michael Feathers wrote a great foreword. I incorporated feedback from dozens of people - some that have been friends for years, and some that I’d never previously met. I can’t say enough great things about http://leanpub.comand I highly recommend it for getting an idea out there and making it easy to get fast feedback. 

As far as the softcover edition, I had offers from a few major publishers, but in the end none of them would allow me to continue to sell on leanpub at the same time. I strongly considered caving to the demands of the major publishers, but ultimately the ability to create a high quality softcover and make it available on Amazon was too tempting to pass up.

The feedback has been almost universally positive - the reviews are quite solid on goodreads (http://review.wewut.com). I believe the book provides specific, concise direction for effective Unit Testing, and I hope it helps increase the quality of the unit tests found in the wild.

If you'd like to try before you buy, there's a sample available in pdf format or on the web.


Thursday, June 12, 2008

Developer Testing and the Importance of Context

How is it we keep falling for the same trick? Why is it so hard to remember: there is no silver bullet.

I've spent a significant amount of my time for the past 3 years focusing on testing. I've learned several lessons.Unfortunately, that list doesn't represent best practices or rules for better testing, it represents patterns for creating a readable, reliable, and performant test suite -- if and only if you work on the kind of projects I work on and you use tests the way I do.

And that's the killer. Context is still king.

I work on teams generally of size 6-16 developers. More often than not I fix tests that I've never seen before. I never read test files to understand class responsibilities. I never generate documentation based on my tests. I do my best to write perfect tests so that my application runs perfectly, but the best tests I write, I never look at again. My tests have a thankless task: guide my system design and ensure that I don't introduce regression bugs, and that's it. I run the entire test suite every minute on average.

I expect some of my readers work the way that I do, and in environments similar to mine; however, the vast majority probably don't. That means that a small minority can blindly follow my suggestions, but the majority of my readers will need to understand why I prefer those patterns and if they apply to their work environment.

Perhaps an example is appropriate. Which of these tests would you prefer to find when the build fails because of it.

test "when attribute is not nil or empty, then valid is true" do
validation = Validatable::ValidatesPresenceOf.new stub, :name
assert_equal true, validation.valid?(stub(:name=>"book"))
end

test "when attribute is not nil or empty, then valid is true" do
assert_equal true, @validation.valid?(@stub)
end

It's actually a trick question. The context is too important to ignore when composing an answer. If this test lived in a project where I was the sole author and expected maintainer then the latter is probably a better solution because I would know where to find the creation of @validation. However, on a large team where it's more likely that I'll never see this test until it's broken, there's a great argument for keeping all the necessary logic within the test itself.

The same test could be written with or without a test name.

test "when attribute is not nil or empty, then valid is true" do
validation = Validatable::ValidatesPresenceOf.new stub, :name
assert_equal true, validation.valid?(stub(:name=>"book"))
end

expect Validatable::ValidatesPresenceOf.new(stub, :name).to.be.valid?(stub(:name=>"book"))

Again, the context is critical in deciding which approach to use on your project. The second test is one line, but it provides very little understanding of why it exists. You can resolve this issue by adding a comment or (what I would prefer) by changing the class to have a more fluent interface that explains why as well as how. However, both of these solutions make it hard to easily scan the file for an understanding of ValidatesPresenceOf or generate documentation based on the tests. Are those things important to you?

Dan North and I agree more than we disagree, but we have very different styles of testing. We also use test in different ways, but we both have the same goal in mind -- use tests to create reliable, readable, and performant software. I believe striving for reliable, readable, and performant applications and tests is a good goal to have and there are several ways to get there. Your best bet is to understand the patterns that work for me, understand the patterns that work for Dan, and understand the patterns that work for anyone else who is passionate about developer testing. You'll find that some of their approaches are in direct conflict. This isn't because one pattern is superior to another in isolation, it's because one pattern is superior to another in context.

There's also another factor worth mentioning. Innovation around testing is still happening at a rapid pace. It seems as though there's a new testing or mocking framework appearing on a weekly basis. I suspect this is probably representative of the future of testing -- testing frameworks targeted at specific contexts. As of right now you may write your unit tests and functional tests using the same framework; however, in the future you may prefer Synthesized Testing when focusing on developer tests and RSpec Story Runner for acceptance tests. It's also possible that the newest features of XUnit.net, JMock or Mockito will give you a better way to model your domain. These, and the other testing frameworks, are evolving at a rapid pace because developer testing patterns are still immature and are being adapted to the contexts in which they are used. In the future you may not use the same tool for several different types of tasks -- and that's probably a good thing.

It all comes back to context. The best advice anyone can give you is to consider yours and take the patterns that should help the most... and then adapt as your context changes.

Tuesday, May 27, 2008

Testing: The Value of Test Names

Test names are glorified comments.

def test_address_service_recognizes_post_code_WC1H_0HT # Test::Unit
test "address service recognizes post code WC1H 0HT" # Dust
it "should recognize post code WC1H 0HT" # RSpec
should "recognize post code WC1H 0HT" # Shoulda

No matter what form is used a test name is a description, a required comment.

I used to think comments were a really good thing. I always tried to think about why when I was writing a comment instead of how. Having your comments describe why was a best practice.

Then, Martin Fowler and Kent Beck showed me the light.
Don't worry, we aren't saying that people shouldn't write comments. In our olfactory analogy, comments aren't a bad smell; indeed they are a sweet smell. The reason we mention comments here is that comments often are used as a deodorant. It's surprising how often you look at thickly commented code and notice that the comments are there because the code is bad. -- Fowler & Beck, Refactoring
Fast forward a few years...

I used to think test names were a really good thing. I always tried to think about why when I was writing a test name instead of how... I think you see where this is going.

Once I recognized that test names were comments I began to look at them differently. I wondered if all the things I disliked about comments could also be said about test names.

Just like comments, their accuracy seemed to degrade over time. What once explained why is now a distracting bit of history that no longer applies. Sure, better developers should have updated them. Whatever. The problem with test names is that they aren't executable or required. Making a mistake doesn't stop the test suite from running.

While test names can be greatly important, they are never essential to execution. There's no guard to ensure accuracy other than conscience and diligence. One persons essential comment is another persons annoying noise, which complicates things even more. Stale test names make me uneasy, but I wonder if test names have bigger problems. For example, it would be poor form to put the why in the name instead of in the code. [C]omments are there because the code is bad, could the same be said of your test names and their tests?

In fact, the entire section in Refactoring on comments may apply to test names also.
Comments lead us to bad code that has all the rotten whiffs we've discussed in [Chapter 3 of Refactoring]. Our first action is to remove the bad smells by refactoring. When we're finished, we often find that the comments are superfluous.
Comments lead us to bad code...remove the bad smells by refactoring. When we're finished, we often find that the comments are superfluous. With this in mind I began to toy with the idea that I could write tests that made test names unnecessary.

In fact, writing tests without names was fairly easy. I focused very heavily on creating a domain model that allowed me to write tests that required no explanation. Eighty percent of the time I found I could remove a test name by writing better code. Sometimes coming back to a test required a bit of time to see what was going on, but I didn't take this as a failure, instead I took the opportunity convert the code to something more maintainable.
If you need a comment to explain what a block of code does, try Extract Method. If the method is already extracted but you still need a comment to explain what it does, use Rename Method. If you need to state some rules about the required state of the system, use Introduce Assertion.
To be clear, I don't think you should be extracting or renaming methods in your tests, I think test names can indicate that those things are needed in the code you are testing. Conversely, Introduce Assertion could apply directly to your test. Instead of stating your assumptions in the test name, introduce an assertion that verifies an assumption. Or better yet, break the larger test up into several smaller tests.
A good time to use a comment is when you don't know what to do. In addition to describing what is going on, comments can indicate areas in which you aren't sure. A comment is a good place to say why you did something. This kind of information helps future modifiers, especially forgetful ones.
I agree with Fowler and Beck, comments are valuable when used correctly. Wrapping legacy code with a few tests may result in tests where you know the results, but you don't quite understand how they are generated. Also, higher level tests often benefit from a brief description of why the test exists. These are situations where comments are truly valuable. However, I found I could handle those situations with a comment, they didn't require a test name.

I started sharing the idea of making test names superfluous with friends and got very mixed reviews.

John Hume asks for a test suite with examples of how I put this idea into practice. The first draft of this entry didn't include examples because I think the idea isn't tied to a particular framework, and I didn't want to give the impression that the framework was the important bit. However, I think John is right, the entry is better if I point to examples for those interested in putting the idea into practice.

The expectations.rubyforge.org testing framework is designed with anonymous tests as a core concept. I have used expectations on my last 3 client projects and all the open source projects I still work on. You can browse the following tests for plenty of examples.It's probably worth mentioning that RSpec also supports testing without a description.

John also likes code folding at the test name level, which is obviously not possible without test names. Truthfully, I don't have an answer for this. If you use code folding, you'll need to take that into account.

Pat Farley asks: So when do method names get the ax?
Method names are actually an interesting comparison. The reality is that methods are already available nameless. Blocks, procs, lambdas, closures, anonymous functions, or whatever you prefer to call them are already essentially methods (or functions) that have no name.

Ola Bini probably had the best response when it came to method names.
A method selector is always necessary to refer to the method from other places. Not necessarily a name, but some kind of selector is needed. That's not true for test names, and to a degree it's interesting that the default of having test names comes from the implementation artifact of tests usually being represented as methods. Would we still have test names in all cases if we didn't start designing a test framework based around a language that cripples usage of abstractions other than methods for executable code?
Prasanna Pendse pointed out that if you use test names for anything you'll lose that ability. The only usage we could think of was creating documentation from the names (RSpec specdoc).

Dan North and I stumbled onto the idea that test names may be more or less valuable based on your workflow. I find that I never care near as much about the test names as I do about individual tests. Dan seems to use test names to get a high level understanding of how a class works. I'll probably explore this idea in a later post after I do a bit more to mature the thoughts. For now it's sufficient to say, if you rely heavily on test names for understanding, you might not be at a point where dropping them makes sense.

Mike Mason likes test names because they can act as a boundary for what the test should do. The basic rules I always followed were that a test can never contain a "and" or an "or". I think Mike is correct that there is a benefit to requiring people to stick to tests that can be explained in a short sentence. However, I like to think the same thing can be achieved with other self imposed constraints such as limiting each test to One Assertion per Test or One Expectation per Test.

Of course, now I'm suggesting one assertion per test and no test names, so if you aren't ready to adopt both, it might not be time to adopt nameless tests.

Ben Butler-Cole provided some anonymous tests written in Haskell that were quite readable. He made the observation that [T]he terser your language the more likely you will be able to make your tests sufficiently comprehensible. I actually find this to be a key in advocating anonymous tests. The test must be easily readable and maintainable, otherwise the effort necessary to work with a test would warrant a description.

A common thought (that I agree with) is that using anonymous tests is something you'll want to try only if you have a talented team. If you're new to testing or have a fair amount of junior programmers on your team, you should probably stick to conventional thinking. However, if you have a team of mostly masters and journeyman, you may find that anonymous tests result in better tests and a better domain model, which is clearly a good thing.

Thanks to Ben Butler-Cole, Chris Stevenson, Dan North, John Hume, Mike Mason, Ola Bini, Prasanna Pendse, Ricky Lui and anyone else who provided feedback on the rough draft.

Monday, May 12, 2008

Testing: Duplicate Code in Your Tests

Luke Kanies recently left a comment on Using Stubs to Capture Test Essence:
Now if you would only see the wisdom of applying DRY to your whole code base, rather than just the functional bit. :)
Sadly, DRY has become a philosophy that is dogmatically and blindly applied to every aspect of programming. I'm a big fan of firing silver bullets but if you go that path your number one priority needs to be finding where a solution does not fit. In fact, those who have applied DRY excessively have found that there are situations when DRY may not be advantageous.

Three of the points from the Wikipedia entry relate to test setup code.

In some small-scale contexts, the effort required to design around DRY may be far greater than the effort to maintain two separate copies of the data.
There's no question that designing around DRY for tests adds additional effort. First of all, there's no reason that tests need to run in the context of an object; however, the most common way to use a setup is to create instance variables and reuse those instance variables within a test. The necessity of instance variables creates the requirement that a test must be run within a new object instance for each test.

Ensuring that a setup method only creates the objects that are necessary for each test adds complexity. If all tests don't require the same data, but you create a setup that creates everything for every test, digesting that code is painful and unnecessary. Conversely, if you strictly adhere to only using setup for objects used by all methods you quickly limit the number of tests you can put within one test case. You can break the test case into multiple test cases with different setup methods, but that adds complexity. Consider the situation where you need the foo instance variable for test 1 and 2 and the bar instance variable for test 2 and 3. There's no straightforward way to create foo and bar in a DRY way using a setup method.

Of course, there's also readability impacts to consider. When a test breaks that relies on setup and teardown I have to look in 3 places to get the full picture of what's going on. When a test is broken I want to fix it as quickly as possible, so I prefer looking in one place.

The crux of the issue is whether a test case is one context or the individual tests are small-scale multiple contexts. While a test case is conceptually one object, we rarely concern ourselves with a test cases as a whole. Instead testing is very much about many individual executable pieces of code. In fact, it's a testing anti-pattern to have any tests depend on other tests. We write tests in isolation, fix tests in isolation and often run tests in isolation; therefore, I consider them to be many small independent entities that do completely isolated jobs. The result of those jobs is aggregated in the end, but ultimately the different entities never (or should never) interact. Each test is and runs in it's own context.

Imposing standards aimed at strict adherence to DRY could stifle community involvement in contexts where it is highly valued, such as a wiki.
Tests are often written by one, but maintained by many. Any barrier to readable, reliable, and performant tests should be removed as quickly as possible. As I've previously discussed, abstracting behavior to 3 different places reduces readability, which can reduce collaboration. Additionally, setup methods can cause unexpected behavior for tests that are written by an author other than the creator of the setup method. This results in less reliable tests, another obvious negative.

Human-readable documentation (from code comments to printed manuals) are typically a restatement of something in the code with elaboration and explanation for those who do not have the ability or time to read and internalize the code...While tests aren't code comments or printed manuals they are very much (or should be) Human-readable documentation. They are also restatements of something in the code with elaboration and explanation. In fact the setup and teardown logic is usually nothing more than elaboration and explanation. The idea of setup and teardown is to put the system into a known state before executing an assertion. While you should have the ability to understand setup and teardown, you don't have the time. Sure, you can make the time, but you shouldn't need to.

The problem is that it's much easier to understand how to apply DRY than it is to understand why.

If your goal is to create a readable, reliable, and performant test suite (which it should be) then there are better ways to achieve that goal than by blindly applying DRY.
Where there's smoke, pour gasoline --Scott Conley
Duplicate code is a smell. Setup and teardown are deodorant, but they don't fix the underlying issue. Using a setup method is basically like hiding your junk in the closet. There are better ways.

Pull the data and the behavior out of the setup and teardown methods and move it to the tests, then take a hard look.

Unimportant behavior specification
Setup methods often mock various interactions; however, the ultimate goal of the interactions is a single result. For example, you may have designed a Gateway class that takes a message, builds a request, sends the request and parses the results of the request. A poorly designed Gateway class will require you to stub the request building and sending the request. However, a well designed Gateway class will allow you to stub two methods at most and verify the behavior of the parse method in isolation. The poorly designed Gateway class may require a setup method, the well designed Gateway class will have a simple test that stubs a method or two in a straightforward manner and then focuses on the essence of the behavior under test. If your domain model requires you to specify behavior which is has nothing to do with what you are testing, don't specify that behavior in a setup method, remove the need to specify the behavior.

Loosely couple objects
I've always disliked the ObjectMother pattern. ObjectMother is designed to build complex hierarchies of objects. If you need an ObjectMother to easily test, you probably actually need to take another look at your domain model. Loosely coupled objects can be created without complicated graphs of dependencies. Loosely coupled objects don't require setting up multiple collaborators. Fewer collaborators (or dependencies) result in tests that don't rely on setup methods to create various collaborators.

The idea is to write tests that only specify what's necessary and verify one thing at a time. If you find that you need to specify things that are unrelated to what you are trying to verify, change the domain model so that the unnecessary objects are no longer required.

Solve problems globally or locally, but not between
Sometimes there are things that need to happen before and after every test. For example, running each test in a transaction and rolling back after the test is run is a good thing. You could add this behavior at a test case level, but it's obviously beneficial to apply this behavior to every test in the entire test suite.

There are other cases where it can make sense to solve issues on a global level. For example, I once worked on a codebase that relied heavily on localization. Some tests needed to verify strings that were specified in the localization files. For those tests we originally specified what localization file to use (in each test), but as we used this feature more often we decided that it made sense to set the localization file for the entire test suite.

Putting everything within a test is the best solution for increasing understanding. However, applying behavior to an entire test suite also aides in comprehension. I would expect any team member to understand that reference data is loaded once before the test suite is run, each test is run in a transaction, and localization uses the test localization file.

Understanding the behavior of an entire test suite is valuable and understanding the behavior of a test is valuable, but understanding a test case provides almost no benefit. Rarely do you run a single test case, rarely does an entire test case break, and rarely do you edit an entire test case. Since your primary concerns are the test suite and the tests themselves, I prefer to work on those levels.

I wouldn't go too far with the global behavior approach, and I definitely wouldn't start moving conditional logic into global behavior, but applied judiciously this can be a very valuable technique.

Bringing it all together
The core of the idea is that it should be easy to understand a test. Specifying everything within a test makes understanding it easy; however, that solution isn't always feasible. Setup and teardown are not the solution to this problem. Luckily the techniques above can be used to make specifying everything within a test a realistic and valuable solution.

Tuesday, May 06, 2008

Using Stubs to Capture Test Essence

Back in January of 2006 I wrote about Using Stubs and Mocks to Convey Intent. In those days I was working mainly with C# and NMock and the simplest way to create a stub was to allow ReSharper generate a stub based on an Interface. This introduced a bit more effort when first generating the stub and added complexity around where the stubs should live and how they should be used. Despite the additional effort, I still felt that using them was a net gain.

Things are very different in the Ruby world. While using C#, Dependency Injection was something I used religiously; however, using Ruby I can simply stub the new method of a dependency. C# had hurdles that made stubbing controversial; however, using stubs in Ruby tests is basically seamless thanks to Mocha. Mocha makes it easy to Replace Collaborators with Stubs, but you shouldn't stop there--you should also Replace Mocks with Stubs.

Replacing Mocks with Stubs is beneficial for several reasons. Stubs are more concise. One of the largest problems with tests is that they can be overwhelming the first time you look at them. Reducing a 3 line mock definition to stub(:one => 1, :two => 2) is almost always a readability win.

Stubs are great, but you may need to use a mock to verify behavior. However, you don't need to verify several behaviors in one test. Using a single mock to verify specific behavior and stubbing all other collaborations is an easy way to create tests that focus on essence. Following the One Expectation per Test suggestion often results in using one or more stubs... and more robust tests.

Using stubs is a good first step towards concise and relevant test code; however, you can take stubs a step further. Mocha defines a stub_everything method that creates an object which will return nil to any message it doesn't understand. The stub_everything stubs are fantastic for dependencies that are used several times, but you are only interested in specific interactions. The interactions which are unimportant for the current test no longer need to be defined.

Stubs can be even more valuable if you are interested verifying a single interaction but are completely uninterested in all other interactions. In that case it's possible to define a stub_everything and set an expectation on it. Consider the following sample code.

class MessageService
def self.deliver(message)
message.from = current_user
message.timestamp
message.sent = Gateway.process(message)
end

def test_the_message_is_sent
Gateway.stubs(:process).returns(true)
message = stub_everything
message.expects(:sent=).with(true)
MessageService.deliver(message)
end

Since the expects method is defined on Object you can set expectations on concrete objects, mocks, and stubs. This results in a lot of possible combinations for defining concise behavior based tests.

Lastly stubbing can help with test essence by allowing you to stub methods you don't care about. The following example defines a gateway that creates a message, sends the message, and parses the response.

class MessageGateway
def process(message_text)
response = post(create_request(message_text))
parse_response(response)
end

def post(message)
# ...
end

def create_request(message_text)
# ...
end

def parse_response(response)
# ...
end

The Gateway#process method is the only method that would be used by clients. In fact, it probably makes sense to make post, create_request, and parse_response private. While it's possible to test private methods, I prefer to create tests that verify what I'm concerned with and stub what I don't care about. Using partial stubbing I can always test using the public interface.

def test_create_request
gateway = MessageGateway.new
gateway.expects(:post).with("<text>hello world</text>")
gateway.stubs(:parse_response)
gateway.process("hello world")
end

def test_post
gateway = MessageGateway.new
gateway.stubs(:create_request).returns("<text>hello world</text>")
gateway.expects(:parse_response).with("<status>success</status>")
gateway.process("")
end

def test_parse_response
gateway = MessageGateway.new
gateway.stubs(:create_request)
gateway.stubs(:post).returns("<status>success</status>")
assert_equal true, gateway.process("")
end

The combination of partial stubbing and defining small methods results in highly focused tests that can independently verify behavior and avoid cascading failures.

Mocha makes it as easy to define a mock as it is to define a stub, but that doesn't mean you should always prefer mocks. In fact, I generally prefer stubs and use mocks when necessary.

Tuesday, April 15, 2008

Story 9: Display Customer Support Number

I thought it might be beneficial to create an entry that shows my workflow when implementing a few feature.

I generally work from stories, so it seemed logical to create an example story.
As a consumer
I would like to see the customer support number
So that I can call if I have a problem
There are a few technical details that make this story a bit more complicated than expected.
  • The site is one rails application, but it needs to be branded differently for each affiliate.
  • Based on the affiliate name (which will come from the params) we will need to connect to the web service specific to that affiliate.
  • The customer support number will come from the affiliate web service.
  • The affiliate web service will require an authentication token in production, but it is optional in development mode.
Based on the story and the known technical details we come up with the following list of tasks.
  • task 1: Create a configuration file (yaml) that will be stored on the web server. This configuration file will map the affiliate name to their web service.
  • task 2: Retrieve the customer support number from the affiliate web service.
  • task 3: Add authentication since it will be required in production.
You may prefer more fine grained tasks (I usually do), but these should do for example purposes.

Task 1
The first thing I generally do is write a functional test. I want unit tests as well, and I'll probably write the unit tests and have them passing before I have the functional test passing, but starting with a functional test allows me to think at a higher level instead of getting bogged down in the details of the implementation. Therefore, the first test I write would probably look like this RSpec spec.

describe Configuration do
it "gets the support number from the client specific configuration" do
Configuration.for("localhost").support_number.should eql("212.646.9208")
end
end

[RSpec link]

Now, I know that this test isn't going to pass for awhile, but it gives me a good reminder of the direction that I want to continue to head in. At this point I would be looking to write a unit test that verifies that I can get the address from a stub configuration file. Having a unit test will allow me to get the Configuration class working without worrying about the configuration file. The unit test will also run significantly faster than the functional test, which benefits us in the long term.

Expectations do
expect "some url" do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.for("localhost")
end
end

[expectations link]

Ultimately I know that the Configuration::for method won't return the address, but my first task is to get the configuration file working, and this test is a good way to complete that task.

At this point I have failing tests, so I write the implementation of the Configuration class. Below is all of the code written at this point. (Of course, in a real application it would be broken out to appropriate files)

require 'yaml'
require 'rubygems'
require 'expectations'
require 'spec'

class Configuration < Struct.new(:support_number)
def self.for(environment)
YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
end
end

Expectations do
expect "some url" do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.for("localhost")
end
end
# >> Expectations .
# >> Finished in 0.001 seconds
# >>
# >> Success: 1 fulfilled

describe Configuration do
it "gets the support number from the client specific configuration" do
Configuration.for("localhost").support_number.should eql("212.646.9208")
end
end
# >> F
# >>
# >> 1)
# >> NoMethodError in 'Configuration gets the support number from the client specific configuration'
# >> undefined method `support_number' for "127.0.0.1/config":String
# >>
# >> Finished in 0.008434 seconds
# >>
# >> 1 example, 1 failure

The unit test is passing; therefore, assuming that we've stubbed the configuration file correctly we've completed our first task.

Task 2
Our functional test is sufficient for verifying that task two is done correctly, so the next step is to write another unit test. For the purpose of the example, assume that there's a ConfigurationGateway::retrieve_from method that we can use to get the configuration from the affiliate web service. The configuration information is going to be returned by the gateway as a hash, so we'll create an appropriate stub so our unit tests don't rely on the real service. Our next unit test would look something like the following example.

Expectations do
expect "support number" do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
ConfigurationGateway.stubs(:retrieve_from).returns(:support_number => "support number")
Configuration.for("localhost").support_number
end
end

A quick change to the Configuration class and we have our new unit test passing.

class Configuration < Struct.new(:support_number)
def self.for(environment)
configuration_url = YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
self.new(ConfigurationGateway.retrieve_from(configuration_url)[:support_number])
end
end

Unfortunately, this change has broken our previously created unit test.

Expectations do
expect "some url" do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.for("localhost")
end
end
# >> Expectations F
# >> Finished in 0.001 seconds
# >>
# >> Failure: 1 failed, 0 errors, 0 fulfilled
# >>
# >> --Failures--
# >> -:40:in `expect'
# >> file <->
# >> line <40>
# >> expected: <"some url"> got: <#<struct Configuration support_number="212.646.9208">>

Our first unit test no longer works because we are no longer returning the address from the Configuration::for method. In fact, we don't need to expose the address outside of the Configuration class at all, so a behavior based unit test is probably a good choice for replacing our first unit test. The example below represents the new method for verifying that the configuration file is correctly returning the address.

Expectations do
expect ConfigurationGateway.to.receive(:retrieve_from).with("some url").returns({}) do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.for("localhost")
end
end

At this point, if the ConfigurationGateway is properly working and if your configuration file is in place, your functional test should also be passing.

Here's all the code we've written so far, including a fake ConfigurationGateway to show our Configuration class is working.

require 'yaml'
require 'rubygems'
require 'expectations'
require 'spec'

class Configuration < Struct.new(:support_number)
def self.for(environment)
configuration_url = YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
self.new(ConfigurationGateway.retrieve_from(configuration_url)[:support_number])
end
end

class ConfigurationGateway
def self.retrieve_from(url)
# In actual code this would go to a external service...
{:support_number => "212.646.9208"}
end
end

Expectations do
expect ConfigurationGateway.to.receive(:retrieve_from).with("some url").returns({}) do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.for("localhost")
end

expect "support number" do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
ConfigurationGateway.stubs(:retrieve_from).returns(:support_number => "support number")
Configuration.for("localhost").support_number
end
end
# >> Expectations ..
# >> Finished in 0.001 seconds
# >>
# >> Success: 2 fulfilled

describe Configuration do
it "gets the support number from the client specific configuration" do
Configuration.for("localhost").support_number.should eql("212.646.9208")
end
end
# >> .
# >>
# >> Finished in 0.008254 seconds
# >>
# >> 1 example, 0 failures

And, here's the configuration file that you'll also need to get the tests passing.

# config.yaml
localhost:
configuration_url: 127.0.0.1/config

At this point both our functional and unit tests are passing, and we are done with task two.

Task 3
For the purposes of this example, we're going to change the ConfigurationGateway to require the authentication token. If this were real code, I would expect the ConfigurationGateway to have this ability already built in, since we'd need a way to make the development server behave like production.

class ConfigurationGateway
def self.retrieve_from(url, token)
# In actual code this would go to a external service...
{:support_number => "212.646.9208"}
end
end

This change causes the functional test to break, but the unit tests do not break. That's OK. In order to get the functional tests passing we'll need to change the implementation, which will cause the unit tests to break. In the end, all of our tests will be updated to represent the correct interactions.

To make the functional tests pass we'll need to add a call to the AuthenticaionGateway to get the next authentication token. Following this change, the code below represents the new Configuration::for method.

class Configuration < Struct.new(:support_number)
def self.for(environment)
configuration_url = YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
token = AuthenticationGateway.next_token
self.new(ConfigurationGateway.retrieve_from(configuration_url, token)[:support_number])
end
end

With this change in place the functional test now passes. To get it working locally, you'll need to add the fake AuthenticationGateway class that can be found below.

class AuthenticationGateway
def self.next_token
# In actual code this would go to an external service...
1979
end
end

At this point your functional test should be passing, but a unit test should be failing.

Expectations do
expect ConfigurationGateway.to.receive(:retrieve_from).with("some url").returns({}) do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.for("localhost")
end
end
# >> Expectations F.
# >> Finished in 0.00104 seconds
# >>
# >> Failure: 1 failed, 0 errors, 1 fulfilled
# >>
# >> --Failures--
# >> -:30:in `expect'
# >> file <->
# >> line <30>
# >> #<Mock:0x5168f0>.retrieve_from('some url', 1979) - expected calls: 0, actual calls: 1
# >> Similar expectations:
# >> #<Mock:0x5168f0>.retrieve_from('some url')

We need to update the ConfigurationGateway::for expectation, but we also need to remove the call to the external service from our unit tests. We do that by stubbing the AuthenticationGateway::next_token method.

Expectations do
expect ConfigurationGateway.to.receive(:retrieve_from).with("some url", "some token").returns({}) do
AuthenticationGateway.stubs(:next_token).returns("some token")
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.for("localhost")
end

expect "support number" do
AuthenticationGateway.stubs(:next_token).returns("some token")
File.stubs(:read).returns("localhost:\n configuration_url: some url")
ConfigurationGateway.stubs(:retrieve_from).returns(:support_number => "support number")
Configuration.for("localhost").support_number
end
end

At this point we are done with task three. The code below represents all the code necessary for completing all 3 of our tasks.

require 'yaml'
require 'rubygems'
require 'expectations'
require 'spec'

class Configuration < Struct.new(:support_number)
def self.for(environment)
configuration_url = YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
token = AuthenticationGateway.next_token
self.new(ConfigurationGateway.retrieve_from(configuration_url, token)[:support_number])
end
end

class ConfigurationGateway
def self.retrieve_from(url, token)
# In actual code this would go to a external service...
{:support_number => "212.646.9208"}
end
end

class AuthenticationGateway
def self.next_token
# In actual code this would go to an external service...
1979
end
end

Expectations do
expect ConfigurationGateway.to.receive(:retrieve_from).with("some url", "some token").returns({}) do
AuthenticationGateway.stubs(:next_token).returns("some token")
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.for("localhost")
end

expect "support number" do
AuthenticationGateway.stubs(:next_token).returns("some token")
File.stubs(:read).returns("localhost:\n configuration_url: some url")
ConfigurationGateway.stubs(:retrieve_from).returns(:support_number => "support number")
Configuration.for("localhost").support_number
end
end

describe Configuration do
it "gets the support number from the client specific configuration" do
Configuration.for("localhost").support_number.should eql("212.646.9208")
end
end

Refactoring
We are done with our tasks, but we aren't ready to move on to the next card yet. Our functional test is great, but our unit tests specify way too much. They use stubs instead of mocks, which helps with creating more robust tests, but they suffer from High Implementation Specification.

The solution is to create smaller methods that are more test friendly.

The first refactoring is to change the configuration_url and token local variables to be method calls instead.

class Configuration < Struct.new(:support_number)
def self.for(environment)
self.new(ConfigurationGateway.retrieve_from(configuration_url(environment), authentication_token)[:support_number])
end

def self.configuration_url(environment)
YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
end

def self.authentication_token
AuthenticationGateway.next_token
end
end

This change to the Configuration class makes our second unit test a bit more readable and robust.

Expectations do
expect "support number" do
Configuration.stubs(:authentication_token)
Configuration.stubs(:configuration_url)
ConfigurationGateway.stubs(:retrieve_from).returns(:support_number => "support number")
Configuration.for("localhost").support_number
end
end

At this point our second unit test looks pretty good, but the first one is still testing a bit too much. We can solve this by breaking the first test into 4 different tests. After breaking the first test up, our unit tests would look like the code below.

Expectations do
expect ConfigurationGateway.to.receive(:retrieve_from).with(nil, "token") do
Configuration.stubs(:authentication_token).returns("token")
Configuration.stubs(:configuration_url)
Configuration.configuration_results("localhost")
end

expect ConfigurationGateway.to.receive(:retrieve_from).with("configuration url", nil) do
Configuration.stubs(:authentication_token)
Configuration.stubs(:configuration_url).returns("configuration url")
Configuration.configuration_results("localhost")
end

expect AuthenticationGateway.to.receive(:next_token) do
Configuration.authentication_token
end

expect "some url" do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.configuration_url("localhost")
end

expect "support number" do
Configuration.stubs(:configuration_results).returns(:support_number => "support number")
Configuration.for("localhost").support_number
end
end

Whether or not to break the test apart would be a judgment call. The number of things I was specifying would drive me to go ahead and break it up, but if there had only been 5 values specified I might have left the test the way it was.

The Result
Below is all the code written for this entry. I hope you find it helpful.

require 'yaml'
require 'rubygems'
require 'expectations'
require 'spec'

class Configuration < Struct.new(:support_number)
def self.for(environment)
self.new(configuration_results(environment)[:support_number])
end

def self.configuration_results(environment)
ConfigurationGateway.retrieve_from(configuration_url(environment), authentication_token)
end

def self.configuration_url(environment)
YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
end

def self.authentication_token
AuthenticationGateway.next_token
end
end

class ConfigurationGateway
def self.retrieve_from(url, token)
{:support_number => "212.646.9208"}
end
end

class AuthenticationGateway
def self.next_token
1979
end
end

Expectations do
expect ConfigurationGateway.to.receive(:retrieve_from).with(nil, "token") do
Configuration.stubs(:authentication_token).returns("token")
Configuration.stubs(:configuration_url)
Configuration.configuration_results("localhost")
end

expect ConfigurationGateway.to.receive(:retrieve_from).with("configuration url", nil) do
Configuration.stubs(:authentication_token)
Configuration.stubs(:configuration_url).returns("configuration url")
Configuration.configuration_results("localhost")
end

expect AuthenticationGateway.to.receive(:next_token) do
Configuration.authentication_token
end

expect "some url" do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.configuration_url("localhost")
end

expect "support number" do
Configuration.stubs(:configuration_results).returns(:support_number => "support number")
Configuration.for("localhost").support_number
end
end

describe Configuration do
it "gets the support number from the client specific configuration" do
Configuration.for("localhost").support_number.should eql("212.646.9208")
end
end