Showing posts with label TDD. Show all posts
Showing posts with label TDD. Show all posts

Wednesday, December 17, 2014

Working Effectively with Unit Tests Official Launch

Today marks the official release release of Working Effectively with Unit Tests. The book is available in various formats:
I’m very happy with the final version. Michael Feathers wrote a great foreword. I incorporated feedback from dozens of people - some that have been friends for years, and some that I’d never previously met. I can’t say enough great things about http://leanpub.comand I highly recommend it for getting an idea out there and making it easy to get fast feedback. 

As far as the softcover edition, I had offers from a few major publishers, but in the end none of them would allow me to continue to sell on leanpub at the same time. I strongly considered caving to the demands of the major publishers, but ultimately the ability to create a high quality softcover and make it available on Amazon was too tempting to pass up.

The feedback has been almost universally positive - the reviews are quite solid on goodreads (http://review.wewut.com). I believe the book provides specific, concise direction for effective Unit Testing, and I hope it helps increase the quality of the unit tests found in the wild.

If you'd like to try before you buy, there's a sample available in pdf format or on the web.


Monday, May 19, 2014

Weighing in on Long Live Testing

DHH recently wrote a provocative piece that gave some views into how he does and doesn't test these days. While I don't think I agree with him completely, I applaud his willingness to speak out against TDD dogma. I've written publicly about not buying the pair-programming dogma, but I hadn't previously been brave enough to admit that I no longer TDD the vast majority of the time.

The truth is, I haven't been dogmatic about TDD in quite some time. Over 6 years ago I was on a ThoughtWorks project where I couldn't think of a single good reason to TDD the code I was working on. To be honest, there weren't really any reasons that motivated me to write tests at all. We were working on a fairly simple, internal application. They wanted software as fast as they could possibly get it, and didn't care if it crashed fairly often. We kept everything simple, manually tested new features through the UI, and kept our customer's very happy.

There were plenty of reasons that we could have written tests. Reasons that I expect people will want to yell at me right now. To me, that's actually the interesting, and missing part, of the latest debate on TDD. I don't see people asking: Why are we writing this test? Is TDD good or bad? That depends; TDD is just a tool, and often the individual is the determining factor when it comes to how effective a tool is. If we start asking "Why?", it's possible to see how TDD could be good for some people, and bad for DHH.

I've been quietly writing a book on Working Effectively with Unit Tests, and I'll have to admit that it was really, really hard not to jump into the conversation with some of the content I've recently written. Specifically, I think this paragraph from the Preface could go a long way to helping people understand an opposing argument.

Why Test?

The answer was easy for me: Refactoring told me to. Unfortunately, doing something strictly because someone or something told you to is possibly the worst approach you could take. The more time I invested in testing, the more I found myself returning to the question: Why am I writing this test?

There are many motivators for creating a test or several tests:
  • validating the system
    • immediate feedback that things work as expected
    • prevent future regressions
  • increase code-coverage
  • enable refactoring of legacy codebase
  • document the behavior of the system
  • your manager told you to
  • Test Driven Development
    • improved design
    • breaking a problem up into smaller pieces
    • defining the "simplest thing that could possibly work"
  • customer approval
  • ping pong pair-programming
Some of the above motivators are healthy in the right context, others are indicators of larger problems. Before writing any test, I would recommend deciding which of the above are motivating you to write a test. If you first understand why you're writing a test, you'll have a much better chance of writing a test that is maintainable and will make you more productive in the long run.

Once you start looking at tests while considering the motivator, you may find you have tests that aren't actually making you more productive. For example, you may have a test that increases code-coverage, but provides no other value. If your team requires 100% code-coverage, then the test provides value. However, if you team has abandoned the (in my opinion harmful) goal of 100% code-coverage, then you're in a position to perform my favorite refactoring: delete.
I don't actually know what motivates DHH to test, but if we assumed he cares about validating the system, preventing future regressions, and enabling refactoring (exclusively) then there truly is no reason to TDD. That doesn't mean you shouldn't; it just means, given what he values and how he works, TDD isn't valuable to him. Of course, conversely, if you value immediate feedback, problems in small pieces, and tests as clients that shape design, TDD is probably invaluable to you.

I find myself doing both. Different development activities often require different tools; i.e. Depending on what I'm doing, different motivators apply, and what tests I write change (hopefully) appropriately.

To be honest, if you look at your tests in the context of the motivators above, that's probably all you need to help you determine whether or not your tests are making you more or less effective. However, if you want more info on what I'm describing, you can pick up the earliest version of my upcoming book. (cheaply, with a full refund guarantee)

Wednesday, September 01, 2010

Clojure: Mocking

An introduction to clojure.test is easy, but it doesn't take long before you feel like you need a mocking framework. As far as I know, you have 3 options.
  1. Take a look at Midje. I haven't gone down this path, but it looks like the most mature option if you're looking for a sophisticated solution.

  2. Go simple. Let's take an example where you want to call a function that computes a value and sends a response to a gateway. Your first implementation looks like the code below. (destructuring explained)
    (defn withdraw [& {:keys [balance withdrawal account-number]}]
    (gateway/process {:balance (- balance withdrawal)
    :withdrawal withdrawal
    :account-number account-number}))
    No, it's not pure. That's not the point. Let's pretend that this impure function is the right design and focus on how we would test it.

    You can change the code a bit and pass in the gateway/process function as an argument. Once you've changed how the code works you can test it by passing identity as the function argument in your tests. The full example is below.
    (ns gateway)

    (defn process [m] (println m))

    (ns controller
    (:use clojure.test))

    (defn withdraw [f & {:keys [balance withdrawal account-number]}]
    (f {:balance (- balance withdrawal)
    :withdrawal withdrawal
    :account-number account-number}))

    (withdraw gateway/process :balance 100 :withdrawal 22 :account-number 4)
    ;; => {:balance 78, :withdrawal 22, :account-number 4}

    (deftest withdraw-test
    (is (= {:balance 78, :withdrawal 22, :account-number 4}
    (withdraw identity :balance 100 :withdrawal 22 :account-number 4))))

    (run-all-tests #"controller")
    If you run the previous example you will see the println output and the clojure.test output, verifying that our code is working as we expected. This simple solution of passing in your side effect function and using identity in your tests can often obviate any need for a mock.

  3. Solution 2 works well, but has the limitations that only one side-effecty function can be passed in and it's result must be used as the return value.

    Let's extend our example and say that we want to log a message if the withdrawal would cause insufficient funds. (Our gateway/process and log/write functions will simply println since this is only an example, but in production code their behavior would differ and both would be required)
    (ns gateway)

    (defn process [m] (println "gateway: " m))

    (ns log)

    (defn write [m] (println "log: " m))

    (ns controller
    (:use clojure.test))

    (defn withdraw [& {:keys [balance withdrawal account-number]}]
    (let [new-balance (- balance withdrawal)]
    (if (> 0 new-balance)
    (log/write "insufficient funds")
    (gateway/process {:balance new-balance
    :withdrawal withdrawal
    :account-number account-number}))))

    (withdraw :balance 100 :withdrawal 22 :account-number 4)
    ;; => gateway: {:balance 78, :withdrawal 22, :account-number 4}

    (withdraw :balance 100 :withdrawal 220 :account-number 4)
    ;; => log: insufficient funds
    Our new withdraw implementation calls two functions that have side effects. We could pass in both functions, but that solution doesn't seem to scale very well as the number of passed functions grows. Also, passing in multiple functions tends to clutter the signature and make it hard to remember what is the valid order for the arguments. Finally, if we need withdraw to always return a map showing the balance and withdrawal amount, there would be no easy solution for verifying the string sent to log/write.

    Given our implementation of withdraw, writing a test that verifies that gateway/process and log/write are called correctly looks like a job for a mock. However, thanks to Clojure's binding function, it's very easy to redefine both of those functions to capture values that can later be tested.

    The following code rebinds both gateway/process and log/write to partial functions that capture whatever is passed to them in an atom that can easily be verified directly in the test.
    (ns gateway)

    (defn process [m] (println "gateway: " m))

    (ns log)

    (defn write [m] (println "log: " m))

    (ns controller
    (:use clojure.test))

    (defn withdraw [& {:keys [balance withdrawal account-number]}]
    (let [new-balance (- balance withdrawal)]
    (if (> 0 new-balance)
    (log/write "insufficient funds")
    (gateway/process {:balance new-balance
    :withdrawal withdrawal
    :account-number account-number}))))

    (deftest withdraw-test1
    (let [result (atom nil)]
    (binding [gateway/process (partial reset! result)]
    (withdraw :balance 100 :withdrawal 22 :account-number 4)
    (is (= {:balance 78, :withdrawal 22, :account-number 4} @result)))))

    (deftest withdraw-test2
    (let [result (atom nil)]
    (binding [log/write (partial reset! result)]
    (withdraw :balance 100 :withdrawal 220 :account-number 4)
    (is (= "insufficient funds" @result)))))

    (run-all-tests #"controller")
In general I use option 2 when I can get away with it, and option 3 where necessary. Option 3 adds enough additional code that I'd probably look into Midje quickly if I found myself writing a more than a few tests that way. However, I generally go out of my way to design pure functions, and I don't find myself needing either of these techniques very often.

Tuesday, April 15, 2008

Story 9: Display Customer Support Number

I thought it might be beneficial to create an entry that shows my workflow when implementing a few feature.

I generally work from stories, so it seemed logical to create an example story.
As a consumer
I would like to see the customer support number
So that I can call if I have a problem
There are a few technical details that make this story a bit more complicated than expected.
  • The site is one rails application, but it needs to be branded differently for each affiliate.
  • Based on the affiliate name (which will come from the params) we will need to connect to the web service specific to that affiliate.
  • The customer support number will come from the affiliate web service.
  • The affiliate web service will require an authentication token in production, but it is optional in development mode.
Based on the story and the known technical details we come up with the following list of tasks.
  • task 1: Create a configuration file (yaml) that will be stored on the web server. This configuration file will map the affiliate name to their web service.
  • task 2: Retrieve the customer support number from the affiliate web service.
  • task 3: Add authentication since it will be required in production.
You may prefer more fine grained tasks (I usually do), but these should do for example purposes.

Task 1
The first thing I generally do is write a functional test. I want unit tests as well, and I'll probably write the unit tests and have them passing before I have the functional test passing, but starting with a functional test allows me to think at a higher level instead of getting bogged down in the details of the implementation. Therefore, the first test I write would probably look like this RSpec spec.

describe Configuration do
it "gets the support number from the client specific configuration" do
Configuration.for("localhost").support_number.should eql("212.646.9208")
end
end

[RSpec link]

Now, I know that this test isn't going to pass for awhile, but it gives me a good reminder of the direction that I want to continue to head in. At this point I would be looking to write a unit test that verifies that I can get the address from a stub configuration file. Having a unit test will allow me to get the Configuration class working without worrying about the configuration file. The unit test will also run significantly faster than the functional test, which benefits us in the long term.

Expectations do
expect "some url" do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.for("localhost")
end
end

[expectations link]

Ultimately I know that the Configuration::for method won't return the address, but my first task is to get the configuration file working, and this test is a good way to complete that task.

At this point I have failing tests, so I write the implementation of the Configuration class. Below is all of the code written at this point. (Of course, in a real application it would be broken out to appropriate files)

require 'yaml'
require 'rubygems'
require 'expectations'
require 'spec'

class Configuration < Struct.new(:support_number)
def self.for(environment)
YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
end
end

Expectations do
expect "some url" do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.for("localhost")
end
end
# >> Expectations .
# >> Finished in 0.001 seconds
# >>
# >> Success: 1 fulfilled

describe Configuration do
it "gets the support number from the client specific configuration" do
Configuration.for("localhost").support_number.should eql("212.646.9208")
end
end
# >> F
# >>
# >> 1)
# >> NoMethodError in 'Configuration gets the support number from the client specific configuration'
# >> undefined method `support_number' for "127.0.0.1/config":String
# >>
# >> Finished in 0.008434 seconds
# >>
# >> 1 example, 1 failure

The unit test is passing; therefore, assuming that we've stubbed the configuration file correctly we've completed our first task.

Task 2
Our functional test is sufficient for verifying that task two is done correctly, so the next step is to write another unit test. For the purpose of the example, assume that there's a ConfigurationGateway::retrieve_from method that we can use to get the configuration from the affiliate web service. The configuration information is going to be returned by the gateway as a hash, so we'll create an appropriate stub so our unit tests don't rely on the real service. Our next unit test would look something like the following example.

Expectations do
expect "support number" do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
ConfigurationGateway.stubs(:retrieve_from).returns(:support_number => "support number")
Configuration.for("localhost").support_number
end
end

A quick change to the Configuration class and we have our new unit test passing.

class Configuration < Struct.new(:support_number)
def self.for(environment)
configuration_url = YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
self.new(ConfigurationGateway.retrieve_from(configuration_url)[:support_number])
end
end

Unfortunately, this change has broken our previously created unit test.

Expectations do
expect "some url" do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.for("localhost")
end
end
# >> Expectations F
# >> Finished in 0.001 seconds
# >>
# >> Failure: 1 failed, 0 errors, 0 fulfilled
# >>
# >> --Failures--
# >> -:40:in `expect'
# >> file <->
# >> line <40>
# >> expected: <"some url"> got: <#<struct Configuration support_number="212.646.9208">>

Our first unit test no longer works because we are no longer returning the address from the Configuration::for method. In fact, we don't need to expose the address outside of the Configuration class at all, so a behavior based unit test is probably a good choice for replacing our first unit test. The example below represents the new method for verifying that the configuration file is correctly returning the address.

Expectations do
expect ConfigurationGateway.to.receive(:retrieve_from).with("some url").returns({}) do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.for("localhost")
end
end

At this point, if the ConfigurationGateway is properly working and if your configuration file is in place, your functional test should also be passing.

Here's all the code we've written so far, including a fake ConfigurationGateway to show our Configuration class is working.

require 'yaml'
require 'rubygems'
require 'expectations'
require 'spec'

class Configuration < Struct.new(:support_number)
def self.for(environment)
configuration_url = YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
self.new(ConfigurationGateway.retrieve_from(configuration_url)[:support_number])
end
end

class ConfigurationGateway
def self.retrieve_from(url)
# In actual code this would go to a external service...
{:support_number => "212.646.9208"}
end
end

Expectations do
expect ConfigurationGateway.to.receive(:retrieve_from).with("some url").returns({}) do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.for("localhost")
end

expect "support number" do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
ConfigurationGateway.stubs(:retrieve_from).returns(:support_number => "support number")
Configuration.for("localhost").support_number
end
end
# >> Expectations ..
# >> Finished in 0.001 seconds
# >>
# >> Success: 2 fulfilled

describe Configuration do
it "gets the support number from the client specific configuration" do
Configuration.for("localhost").support_number.should eql("212.646.9208")
end
end
# >> .
# >>
# >> Finished in 0.008254 seconds
# >>
# >> 1 example, 0 failures

And, here's the configuration file that you'll also need to get the tests passing.

# config.yaml
localhost:
configuration_url: 127.0.0.1/config

At this point both our functional and unit tests are passing, and we are done with task two.

Task 3
For the purposes of this example, we're going to change the ConfigurationGateway to require the authentication token. If this were real code, I would expect the ConfigurationGateway to have this ability already built in, since we'd need a way to make the development server behave like production.

class ConfigurationGateway
def self.retrieve_from(url, token)
# In actual code this would go to a external service...
{:support_number => "212.646.9208"}
end
end

This change causes the functional test to break, but the unit tests do not break. That's OK. In order to get the functional tests passing we'll need to change the implementation, which will cause the unit tests to break. In the end, all of our tests will be updated to represent the correct interactions.

To make the functional tests pass we'll need to add a call to the AuthenticaionGateway to get the next authentication token. Following this change, the code below represents the new Configuration::for method.

class Configuration < Struct.new(:support_number)
def self.for(environment)
configuration_url = YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
token = AuthenticationGateway.next_token
self.new(ConfigurationGateway.retrieve_from(configuration_url, token)[:support_number])
end
end

With this change in place the functional test now passes. To get it working locally, you'll need to add the fake AuthenticationGateway class that can be found below.

class AuthenticationGateway
def self.next_token
# In actual code this would go to an external service...
1979
end
end

At this point your functional test should be passing, but a unit test should be failing.

Expectations do
expect ConfigurationGateway.to.receive(:retrieve_from).with("some url").returns({}) do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.for("localhost")
end
end
# >> Expectations F.
# >> Finished in 0.00104 seconds
# >>
# >> Failure: 1 failed, 0 errors, 1 fulfilled
# >>
# >> --Failures--
# >> -:30:in `expect'
# >> file <->
# >> line <30>
# >> #<Mock:0x5168f0>.retrieve_from('some url', 1979) - expected calls: 0, actual calls: 1
# >> Similar expectations:
# >> #<Mock:0x5168f0>.retrieve_from('some url')

We need to update the ConfigurationGateway::for expectation, but we also need to remove the call to the external service from our unit tests. We do that by stubbing the AuthenticationGateway::next_token method.

Expectations do
expect ConfigurationGateway.to.receive(:retrieve_from).with("some url", "some token").returns({}) do
AuthenticationGateway.stubs(:next_token).returns("some token")
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.for("localhost")
end

expect "support number" do
AuthenticationGateway.stubs(:next_token).returns("some token")
File.stubs(:read).returns("localhost:\n configuration_url: some url")
ConfigurationGateway.stubs(:retrieve_from).returns(:support_number => "support number")
Configuration.for("localhost").support_number
end
end

At this point we are done with task three. The code below represents all the code necessary for completing all 3 of our tasks.

require 'yaml'
require 'rubygems'
require 'expectations'
require 'spec'

class Configuration < Struct.new(:support_number)
def self.for(environment)
configuration_url = YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
token = AuthenticationGateway.next_token
self.new(ConfigurationGateway.retrieve_from(configuration_url, token)[:support_number])
end
end

class ConfigurationGateway
def self.retrieve_from(url, token)
# In actual code this would go to a external service...
{:support_number => "212.646.9208"}
end
end

class AuthenticationGateway
def self.next_token
# In actual code this would go to an external service...
1979
end
end

Expectations do
expect ConfigurationGateway.to.receive(:retrieve_from).with("some url", "some token").returns({}) do
AuthenticationGateway.stubs(:next_token).returns("some token")
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.for("localhost")
end

expect "support number" do
AuthenticationGateway.stubs(:next_token).returns("some token")
File.stubs(:read).returns("localhost:\n configuration_url: some url")
ConfigurationGateway.stubs(:retrieve_from).returns(:support_number => "support number")
Configuration.for("localhost").support_number
end
end

describe Configuration do
it "gets the support number from the client specific configuration" do
Configuration.for("localhost").support_number.should eql("212.646.9208")
end
end

Refactoring
We are done with our tasks, but we aren't ready to move on to the next card yet. Our functional test is great, but our unit tests specify way too much. They use stubs instead of mocks, which helps with creating more robust tests, but they suffer from High Implementation Specification.

The solution is to create smaller methods that are more test friendly.

The first refactoring is to change the configuration_url and token local variables to be method calls instead.

class Configuration < Struct.new(:support_number)
def self.for(environment)
self.new(ConfigurationGateway.retrieve_from(configuration_url(environment), authentication_token)[:support_number])
end

def self.configuration_url(environment)
YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
end

def self.authentication_token
AuthenticationGateway.next_token
end
end

This change to the Configuration class makes our second unit test a bit more readable and robust.

Expectations do
expect "support number" do
Configuration.stubs(:authentication_token)
Configuration.stubs(:configuration_url)
ConfigurationGateway.stubs(:retrieve_from).returns(:support_number => "support number")
Configuration.for("localhost").support_number
end
end

At this point our second unit test looks pretty good, but the first one is still testing a bit too much. We can solve this by breaking the first test into 4 different tests. After breaking the first test up, our unit tests would look like the code below.

Expectations do
expect ConfigurationGateway.to.receive(:retrieve_from).with(nil, "token") do
Configuration.stubs(:authentication_token).returns("token")
Configuration.stubs(:configuration_url)
Configuration.configuration_results("localhost")
end

expect ConfigurationGateway.to.receive(:retrieve_from).with("configuration url", nil) do
Configuration.stubs(:authentication_token)
Configuration.stubs(:configuration_url).returns("configuration url")
Configuration.configuration_results("localhost")
end

expect AuthenticationGateway.to.receive(:next_token) do
Configuration.authentication_token
end

expect "some url" do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.configuration_url("localhost")
end

expect "support number" do
Configuration.stubs(:configuration_results).returns(:support_number => "support number")
Configuration.for("localhost").support_number
end
end

Whether or not to break the test apart would be a judgment call. The number of things I was specifying would drive me to go ahead and break it up, but if there had only been 5 values specified I might have left the test the way it was.

The Result
Below is all the code written for this entry. I hope you find it helpful.

require 'yaml'
require 'rubygems'
require 'expectations'
require 'spec'

class Configuration < Struct.new(:support_number)
def self.for(environment)
self.new(configuration_results(environment)[:support_number])
end

def self.configuration_results(environment)
ConfigurationGateway.retrieve_from(configuration_url(environment), authentication_token)
end

def self.configuration_url(environment)
YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
end

def self.authentication_token
AuthenticationGateway.next_token
end
end

class ConfigurationGateway
def self.retrieve_from(url, token)
{:support_number => "212.646.9208"}
end
end

class AuthenticationGateway
def self.next_token
1979
end
end

Expectations do
expect ConfigurationGateway.to.receive(:retrieve_from).with(nil, "token") do
Configuration.stubs(:authentication_token).returns("token")
Configuration.stubs(:configuration_url)
Configuration.configuration_results("localhost")
end

expect ConfigurationGateway.to.receive(:retrieve_from).with("configuration url", nil) do
Configuration.stubs(:authentication_token)
Configuration.stubs(:configuration_url).returns("configuration url")
Configuration.configuration_results("localhost")
end

expect AuthenticationGateway.to.receive(:next_token) do
Configuration.authentication_token
end

expect "some url" do
File.stubs(:read).returns("localhost:\n configuration_url: some url")
Configuration.configuration_url("localhost")
end

expect "support number" do
Configuration.stubs(:configuration_results).returns(:support_number => "support number")
Configuration.for("localhost").support_number
end
end

describe Configuration do
it "gets the support number from the client specific configuration" do
Configuration.for("localhost").support_number.should eql("212.646.9208")
end
end