Software Development Lessons Learned From Poker

I recently wrote an article for InfoQ

I wasn't always a software developer. The two years before I joined ThoughtWorks I lived primarily off playing poker. Of course, if you've ever asked me about the tattoo on my forearm, you've already heard the story. If you haven't, feel free to ask me next time we get a drink together.

I've never regretted spending so much time playing poker. I believe it taught me quite a few lessons that apply widely to other topics. In fact the more I develop software the more I'm convinced that the two jobs are incredibly similar.

Learning
I approached learning poker the same way I approached learning software development: read as many books as possible. Over the course of 2 years I read every book on poker I could find. I stopped counting at 39. Of course, the same can be true for programming. I have 5 books in front of me right now that are queued up to read next, and I have a large collection of books that I've burned through in the past 3 years with ThoughtWorks.

I consider reading books, blogs, and magazines to be essential for both programming and playing poker, but in both professions reading books isn't good enough. You may be able to keep all the knowledge in your head, but knowing when to apply which rules is the mark of a true professional.

Reading material is great for learning, but it's almost always the context that determines when to apply a certain technique. Since books cannot be specific enough to provide all possible contexts, only experience can give you the ability to be able to make a quick decision that could end up costing you or your employer thousands or even millions of dollars.

...Read the entire article on InfoQ...

Understanding Why

Why is the most important question a software developer can ask. It is your responsibility as a developer and failing to do so is negligent and unprofessional.

I think Fred George would agree with me. Why is important enough to Fred that he uses it to determine if you are an apprentice or a journeyman.

An apprentice is a developer who is still learning the requisite skills... Techniques are being learned, but the why of the technique is still a bit vague. --Fred George

Unfortunately, I don't think why is asked nearly enough.

A great example is best practices. Best practices are defined by masters to help journeyman and apprentices learn and complete their tasks. But, best practices are rarely silver bullets, in fact the vast majority of best practices are heavily influenced by context. And, this is where the problem lies, the vast majority of software developers don't ask why a practice is considered best. Instead the common behavior is to blindly follow contextual advise without any thought given to the current situation.

Most test suites are also great examples of people failing to ask why. Programmers have gotten quite good at using OO concepts and patterns, but do they truly apply to testing? Largely, no. Of course, that's not surprising since tests are not object oriented.

Tests are often written using an object oriented language, but the tests themselves are not object oriented. In fact, the large majority of tests are purely procedural. Tests have one entry point and (hopefully) one exit point. Tests generally set up some state, then verify a result, that's it. Since tests are not object oriented, concepts such as DRY, inheritance, and polymorphism often lead to less readable code.

The problem is, rarely does anyone ask why a test is written and what purpose does it serve.

I write tests that help me write better applications. Tests must be readable, reliable, and performant. I want tests that focus on one thing, so when the test breaks I can quickly figure out what has gone wrong. I want tests that only break when something went wrong, so I don't waste time investigating side effects. And, I want tests that run quickly, so I can run them hundreds of times a day. When I write a test, I ask those questions, not whether my code follows good OO patterns.

Not asking why leads to poorly designed applications. Conversely, asking why often leads to innovation. Someone asking why led to C, Java, Smalltalk, Ruby, and every other language. Why also led to evolved frameworks and various other productivity boosting tools.

Asking why is almost always a good thing, and doing so is your professional responsibility.

Story 9: Display Customer Support Number

I thought it might be beneficial to create an entry that shows my workflow when implementing a few feature.

I generally work from stories, so it seemed logical to create an example story.

As a consumer
I would like to see the customer support number
So that I can call if I have a problem

There are a few technical details that make this story a bit more complicated than expected.

• The site is one rails application, but it needs to be branded differently for each affiliate.
• Based on the affiliate name (which will come from the params) we will need to connect to the web service specific to that affiliate.
• The customer support number will come from the affiliate web service.
• The affiliate web service will require an authentication token in production, but it is optional in development mode.

Based on the story and the known technical details we come up with the following list of tasks.

• task 1: Create a configuration file (yaml) that will be stored on the web server. This configuration file will map the affiliate name to their web service.
• task 2: Retrieve the customer support number from the affiliate web service.
• task 3: Add authentication since it will be required in production.

You may prefer more fine grained tasks (I usually do), but these should do for example purposes.

Task 1
The first thing I generally do is write a functional test. I want unit tests as well, and I'll probably write the unit tests and have them passing before I have the functional test passing, but starting with a functional test allows me to think at a higher level instead of getting bogged down in the details of the implementation. Therefore, the first test I write would probably look like this RSpec spec.

describe Configuration do
  it "gets the support number from the client specific configuration" do
    Configuration.for("localhost").support_number.should eql("212.646.9208")
  end
end

[RSpec link]

Now, I know that this test isn't going to pass for awhile, but it gives me a good reminder of the direction that I want to continue to head in. At this point I would be looking to write a unit test that verifies that I can get the address from a stub configuration file. Having a unit test will allow me to get the Configuration class working without worrying about the configuration file. The unit test will also run significantly faster than the functional test, which benefits us in the long term.

Expectations do
  expect "some url" do
    File.stubs(:read).returns("localhost:\n  configuration_url: some url")
    Configuration.for("localhost")
  end
end

[expectations link]

Ultimately I know that the Configuration::for method won't return the address, but my first task is to get the configuration file working, and this test is a good way to complete that task.

At this point I have failing tests, so I write the implementation of the Configuration class. Below is all of the code written at this point. (Of course, in a real application it would be broken out to appropriate files)

require 'yaml'
require 'rubygems'
require 'expectations'
require 'spec'

class Configuration < Struct.new(:support_number)
  def self.for(environment)
    YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
  end
end

Expectations do
  expect "some url" do
    File.stubs(:read).returns("localhost:\n  configuration_url: some url")
    Configuration.for("localhost")
  end
end
# >> Expectations .
# >> Finished in 0.001 seconds
# >> 
# >> Success: 1 fulfilled

describe Configuration do
  it "gets the support number from the client specific configuration" do
    Configuration.for("localhost").support_number.should eql("212.646.9208")
  end
end
# >> F
# >> 
# >> 1)
# >> NoMethodError in 'Configuration gets the support number from the client specific configuration'
# >> undefined method `support_number' for "127.0.0.1/config":String
# >> 
# >> Finished in 0.008434 seconds
# >> 
# >> 1 example, 1 failure

The unit test is passing; therefore, assuming that we've stubbed the configuration file correctly we've completed our first task.

Task 2
Our functional test is sufficient for verifying that task two is done correctly, so the next step is to write another unit test. For the purpose of the example, assume that there's a ConfigurationGateway::retrieve_from method that we can use to get the configuration from the affiliate web service. The configuration information is going to be returned by the gateway as a hash, so we'll create an appropriate stub so our unit tests don't rely on the real service. Our next unit test would look something like the following example.

Expectations do
  expect "support number" do
    File.stubs(:read).returns("localhost:\n  configuration_url: some url")
    ConfigurationGateway.stubs(:retrieve_from).returns(:support_number => "support number")
    Configuration.for("localhost").support_number
  end
end

A quick change to the Configuration class and we have our new unit test passing.

class Configuration < Struct.new(:support_number)
  def self.for(environment)
    configuration_url = YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
    self.new(ConfigurationGateway.retrieve_from(configuration_url)[:support_number])
  end
end

Unfortunately, this change has broken our previously created unit test.

Expectations do
  expect "some url" do
    File.stubs(:read).returns("localhost:\n  configuration_url: some url")
    Configuration.for("localhost")
  end
end
# >> Expectations F
# >> Finished in 0.001 seconds
# >> 
# >> Failure: 1 failed, 0 errors, 0 fulfilled
# >> 
# >> --Failures--
# >>  -:40:in `expect'
# >> file <->
# >> line <40>
# >> expected: <"some url"> got: <#<struct Configuration support_number="212.646.9208">>

Our first unit test no longer works because we are no longer returning the address from the Configuration::for method. In fact, we don't need to expose the address outside of the Configuration class at all, so a behavior based unit test is probably a good choice for replacing our first unit test. The example below represents the new method for verifying that the configuration file is correctly returning the address.

Expectations do
  expect ConfigurationGateway.to.receive(:retrieve_from).with("some url").returns({}) do
    File.stubs(:read).returns("localhost:\n  configuration_url: some url")
    Configuration.for("localhost")
  end
end

At this point, if the ConfigurationGateway is properly working and if your configuration file is in place, your functional test should also be passing.

Here's all the code we've written so far, including a fake ConfigurationGateway to show our Configuration class is working.

require 'yaml'
require 'rubygems'
require 'expectations'
require 'spec'

class Configuration < Struct.new(:support_number)
  def self.for(environment)
    configuration_url = YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
    self.new(ConfigurationGateway.retrieve_from(configuration_url)[:support_number])
  end
end

class ConfigurationGateway
  def self.retrieve_from(url)
    # In actual code this would go to a external service...
    {:support_number => "212.646.9208"}
  end
end

Expectations do
  expect ConfigurationGateway.to.receive(:retrieve_from).with("some url").returns({}) do
    File.stubs(:read).returns("localhost:\n  configuration_url: some url")
    Configuration.for("localhost")
  end
  
  expect "support number" do
    File.stubs(:read).returns("localhost:\n  configuration_url: some url")
    ConfigurationGateway.stubs(:retrieve_from).returns(:support_number => "support number")
    Configuration.for("localhost").support_number
  end
end
# >> Expectations ..
# >> Finished in 0.001 seconds
# >> 
# >> Success: 2 fulfilled

describe Configuration do
  it "gets the support number from the client specific configuration" do
    Configuration.for("localhost").support_number.should eql("212.646.9208")
  end
end
# >> .
# >> 
# >> Finished in 0.008254 seconds
# >> 
# >> 1 example, 0 failures

And, here's the configuration file that you'll also need to get the tests passing.

# config.yaml
localhost:
  configuration_url: 127.0.0.1/config

At this point both our functional and unit tests are passing, and we are done with task two.

Task 3
For the purposes of this example, we're going to change the ConfigurationGateway to require the authentication token. If this were real code, I would expect the ConfigurationGateway to have this ability already built in, since we'd need a way to make the development server behave like production.

class ConfigurationGateway
  def self.retrieve_from(url, token)
    # In actual code this would go to a external service...
    {:support_number => "212.646.9208"}
  end
end

This change causes the functional test to break, but the unit tests do not break. That's OK. In order to get the functional tests passing we'll need to change the implementation, which will cause the unit tests to break. In the end, all of our tests will be updated to represent the correct interactions.

To make the functional tests pass we'll need to add a call to the AuthenticaionGateway to get the next authentication token. Following this change, the code below represents the new Configuration::for method.

class Configuration < Struct.new(:support_number)
  def self.for(environment)
    configuration_url = YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
    token = AuthenticationGateway.next_token
    self.new(ConfigurationGateway.retrieve_from(configuration_url, token)[:support_number])
  end
end

With this change in place the functional test now passes. To get it working locally, you'll need to add the fake AuthenticationGateway class that can be found below.

class AuthenticationGateway
  def self.next_token
    # In actual code this would go to an external service...
    1979
  end
end

At this point your functional test should be passing, but a unit test should be failing.

Expectations do
  expect ConfigurationGateway.to.receive(:retrieve_from).with("some url").returns({}) do
    File.stubs(:read).returns("localhost:\n  configuration_url: some url")
    Configuration.for("localhost")
  end  
end
# >> Expectations F.
# >> Finished in 0.00104 seconds
# >> 
# >> Failure: 1 failed, 0 errors, 1 fulfilled
# >> 
# >> --Failures--
# >>  -:30:in `expect'
# >> file <->
# >> line <30>
# >> #<Mock:0x5168f0>.retrieve_from('some url', 1979) - expected calls: 0, actual calls: 1
# >> Similar expectations:
# >> #<Mock:0x5168f0>.retrieve_from('some url')

We need to update the ConfigurationGateway::for expectation, but we also need to remove the call to the external service from our unit tests. We do that by stubbing the AuthenticationGateway::next_token method.

Expectations do
  expect ConfigurationGateway.to.receive(:retrieve_from).with("some url", "some token").returns({}) do
    AuthenticationGateway.stubs(:next_token).returns("some token")
    File.stubs(:read).returns("localhost:\n  configuration_url: some url")
    Configuration.for("localhost")
  end
  
  expect "support number" do
    AuthenticationGateway.stubs(:next_token).returns("some token")
    File.stubs(:read).returns("localhost:\n  configuration_url: some url")
    ConfigurationGateway.stubs(:retrieve_from).returns(:support_number => "support number")
    Configuration.for("localhost").support_number
  end
end

At this point we are done with task three. The code below represents all the code necessary for completing all 3 of our tasks.

require 'yaml'
require 'rubygems'
require 'expectations'
require 'spec'

class Configuration < Struct.new(:support_number)
  def self.for(environment)
    configuration_url = YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
    token = AuthenticationGateway.next_token
    self.new(ConfigurationGateway.retrieve_from(configuration_url, token)[:support_number])
  end
end

class ConfigurationGateway
  def self.retrieve_from(url, token)
    # In actual code this would go to a external service...
    {:support_number => "212.646.9208"}
  end
end

class AuthenticationGateway
  def self.next_token
    # In actual code this would go to an external service...
    1979
  end
end

Expectations do
  expect ConfigurationGateway.to.receive(:retrieve_from).with("some url", "some token").returns({}) do
    AuthenticationGateway.stubs(:next_token).returns("some token")
    File.stubs(:read).returns("localhost:\n  configuration_url: some url")
    Configuration.for("localhost")
  end
  
  expect "support number" do
    AuthenticationGateway.stubs(:next_token).returns("some token")
    File.stubs(:read).returns("localhost:\n  configuration_url: some url")
    ConfigurationGateway.stubs(:retrieve_from).returns(:support_number => "support number")
    Configuration.for("localhost").support_number
  end
end

describe Configuration do
  it "gets the support number from the client specific configuration" do
    Configuration.for("localhost").support_number.should eql("212.646.9208")
  end
end

Refactoring
We are done with our tasks, but we aren't ready to move on to the next card yet. Our functional test is great, but our unit tests specify way too much. They use stubs instead of mocks, which helps with creating more robust tests, but they suffer from High Implementation Specification.

The solution is to create smaller methods that are more test friendly.

The first refactoring is to change the configuration_url and token local variables to be method calls instead.

class Configuration < Struct.new(:support_number)
  def self.for(environment)
    self.new(ConfigurationGateway.retrieve_from(configuration_url(environment), authentication_token)[:support_number])
  end
  
  def self.configuration_url(environment)
    YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
  end
  
  def self.authentication_token
    AuthenticationGateway.next_token
  end
end

This change to the Configuration class makes our second unit test a bit more readable and robust.

Expectations do
  expect "support number" do
    Configuration.stubs(:authentication_token)
    Configuration.stubs(:configuration_url)
    ConfigurationGateway.stubs(:retrieve_from).returns(:support_number => "support number")
    Configuration.for("localhost").support_number
  end
end

At this point our second unit test looks pretty good, but the first one is still testing a bit too much. We can solve this by breaking the first test into 4 different tests. After breaking the first test up, our unit tests would look like the code below.

Expectations do
  expect ConfigurationGateway.to.receive(:retrieve_from).with(nil, "token") do
    Configuration.stubs(:authentication_token).returns("token")
    Configuration.stubs(:configuration_url)
    Configuration.configuration_results("localhost")
  end
  
  expect ConfigurationGateway.to.receive(:retrieve_from).with("configuration url", nil) do
    Configuration.stubs(:authentication_token)
    Configuration.stubs(:configuration_url).returns("configuration url")
    Configuration.configuration_results("localhost")
  end

  expect AuthenticationGateway.to.receive(:next_token) do
    Configuration.authentication_token
  end
  
  expect "some url" do
    File.stubs(:read).returns("localhost:\n  configuration_url: some url")
    Configuration.configuration_url("localhost")
  end

  expect "support number" do
    Configuration.stubs(:configuration_results).returns(:support_number => "support number")
    Configuration.for("localhost").support_number
  end  
end

Whether or not to break the test apart would be a judgment call. The number of things I was specifying would drive me to go ahead and break it up, but if there had only been 5 values specified I might have left the test the way it was.

The Result
Below is all the code written for this entry. I hope you find it helpful.

require 'yaml'
require 'rubygems'
require 'expectations'
require 'spec'

class Configuration < Struct.new(:support_number)
  def self.for(environment)
    self.new(configuration_results(environment)[:support_number])
  end
  
  def self.configuration_results(environment)
    ConfigurationGateway.retrieve_from(configuration_url(environment), authentication_token)
  end
  
  def self.configuration_url(environment)
    YAML.load(File.read("config.yaml"))[environment]["configuration_url"]
  end
  
  def self.authentication_token
    AuthenticationGateway.next_token
  end
end

class ConfigurationGateway
  def self.retrieve_from(url, token)
    {:support_number => "212.646.9208"}
  end
end

class AuthenticationGateway
  def self.next_token
    1979
  end
end

Expectations do
  expect ConfigurationGateway.to.receive(:retrieve_from).with(nil, "token") do
    Configuration.stubs(:authentication_token).returns("token")
    Configuration.stubs(:configuration_url)
    Configuration.configuration_results("localhost")
  end
  
  expect ConfigurationGateway.to.receive(:retrieve_from).with("configuration url", nil) do
    Configuration.stubs(:authentication_token)
    Configuration.stubs(:configuration_url).returns("configuration url")
    Configuration.configuration_results("localhost")
  end

  expect AuthenticationGateway.to.receive(:next_token) do
    Configuration.authentication_token
  end
  
  expect "some url" do
    File.stubs(:read).returns("localhost:\n  configuration_url: some url")
    Configuration.configuration_url("localhost")
  end

  expect "support number" do
    Configuration.stubs(:configuration_results).returns(:support_number => "support number")
    Configuration.for("localhost").support_number
  end  
end

describe Configuration do
  it "gets the support number from the client specific configuration" do
    Configuration.for("localhost").support_number.should eql("212.646.9208")
  end
end