Agile: Estimating

Following a previous entry, Stories are Placeholders not Buckets, Aman King states:

A blog entry from you on "estimating" and how you go about it (for various types of stories/buckets) would be a big help.

On the projects I'm generally involved in each story is estimated twice.

High level estimate
A high level estimate is usually helpful when a story is created. The high level estimate can be used to determine where it belongs in the current release (or in a subsequent release). I prefer that high level estimates be done by 3 or 4 developers. I also prefer the estimation scale be the same as the scale that will ultimately be used for the iteration estimate (i.e. don't use small, medium, large for the high level estimate and 1, 2, 3 for an iteration estimate). A high level estimate usually begins with a subject matter expert (often an analyst) explaining the details of the story. These details aren't the full details of the story, but the key aspects that effect the estimates. Following the explanation and any questions, the devs will independently determine their own estimate for the story. If all the devs agree (or are close) then you move on; however, if the estimates are wildly off then a conversation is had and the independent estimation process will occur again.

Iteration estimate
An iteration estimate is helpful for getting the entire team on the same page and determining which stories will be played in the next iteration. At ThoughtWorks we generally have an Iteration Planning Meeting (IPM) before each iteration. During the IPM the subject matter experts (again, usually analysts) explain all the details of the story. Following the explanation and any questions the entire* dev team will independently estimate the story. If the estimates are consistent, you move on, otherwise more discussion needs to occur and the independent estimates need to happen again (just like high level estimates).

Having 2 estimates is helpful for a few reasons. While the original estimate is good for planning purposes, the iteration estimate is better for determining what goes in the upcoming iteration. The iteration estimates should be more accurate because more people have the chance to participate in the estimation process, and (it's likely that) more information is known then when the high level estimate was taken. Given more accurate estimations and a known team velocity, predicting what can be done in each iteration should be a fairly easy task. Also, it's valuable to track if the high level estimates are generally lower or higher than the iteration estimates. Knowing this statistic can help make the high level estimates more accurate for release planning.

*Generally teams contain between 4-8 developers. For teams of this size having the entire team estimate seems to work well. However, if your team has more than 8 devs, I would suggest breaking into two estimation groups. Splitting the team clearly has consequences. At a minimum you will want each story to be started by at least one person who estimated it. Also, you'll need to find a way to communicate the details of each story to the half of the team not involved in estimation. I've seen this handled 2 ways.

• The details of the story are presented to the entire team, then the team is broken to estimate.
• Team Split 1 presents their stories to Team Split 2 following estimation and vice-versa.

I've used both, and they both seem to work fine.

Despite the consequences, splitting large teams for estimation has been much more effective in my experience.

Ruby: Multiple ifs (unless)

I recently ran into some code similar to the snippet below.

item.currency if item.currency != :usd unless item.nil?

Based on reading the code I assumed it worked as expected; however, having never actually tried it myself I decided to hit irb for a moment with the following code.

p 3 unless p 2 unless p 1

Sure, I'm not using if, but if and unless have the same precedence, so I thought the example was good enough.

p 3 unless p 2 unless p 1
# >> 1
# >> 2
# >> 3

The output shows the execution order: The rightmost if (or unless) is evaluated first and then it moves to the next conditional immediately to it's left.

Of course, the statement could be rewritten simply using ||.

item.currency unless item.nil? || item.currency == :usd

Due to short circuit evaluation neither statement executes item.currency != :usd if item.nil?.

Stories are Placeholders not Buckets

Agile methodologies often use stories as requirement documents. Stories are placeholders that signify conversations that will need to occur between the customer and the developers before features are implemented. A story gives a high level idea of the required functionality; however, the details of the functionality live in the mind of the customer until they are expressed to the development team.

Stories are generally concise and purposefully vague; however, they do contain enough detail for the developers to estimate the level of effort required to implement the feature(s). Accurate estimates are crucial because they are used to plan what is in and out of scope for a release. Clearly, inaccurate estimates create volatile scope.

The importance of estimates has the following implication: Stories cannot be used as Buckets.

An example of a bucket would be:

"Create Reports" or "Deploy the application to a production environment."

If the 'production environment' is known this is a perfectly valid story. However, the story cannot be accurately estimated if the 'production environment' is a box that will be ordered later in the project and the platform is still undetermined. A story in which the details are unknown at this point is nothing more than a bucket.

The largest problem with buckets is that they generally end up being far too small. Traditionally, buckets represent the more complicated parts of the system because they aren't yet understood or defined. Since they aren't fully understood, often buckets are pushed off until near the end of the project. Because buckets are generally addressed at the end you are generally already working with a mostly written system that hasn't taken in to account the effects of the bucket. While everyone should be striving to write systems that are flexible and maintainable, having any system at all often makes things more complicated than when the bucket was originally envisioned and estimated (which is another reason buckets are often underestimated). Another problem with buckets is that they often end up with things that don't belong in them. This is also a product of defining a bucket toward the end of a release. Since the release is nearing it only makes sense that the business would like to put as much in a bucket as possible. Of course, overfilling buckets doesn't work either. As Martin already noted:

You can't put ten pounds of shit into a five pound bag.
--Anyone who has tried

Unfortunately, over-estimating buckets is also problematic for the planning process because it breeds distrust between the developers and the business.

This does not mean that buckets do not have their place in agile projects. It is always important to capture required functionality whether or not it is well defined at this point. However, buckets simply cannot be stories or the planning process will break down. Any bucket that must be in the current release likely represents the largest risk for the current release. Therefore, the current release should be limited to stories and buckets should live only in subsequent releases.

Ruby: Metaprogrammatically defining methods

Neal Ford and I were recently looking at some code that used metaprogramming to define similar methods. The example below isn't the code, nor would I suggest managing state in this way; however, it does provide an example of metaprogrammatically defined methods.

class Ticket
  attr_accessor :status
  
  [:planned, :pre_sale, :on_sale].each do |method_name|
    define_method :"#{method_name}?" do
      status == method_name
    end
  end
end

The above code works, but I find it painful to maintain. When I begin to read [:planned, :pre_sale, :on_sale].each do |method_name| I'm not expecting to find methods being defined. Often, I define class methods that give me a more descriptive way to generate the similar methods. In this case I would consider creating a create_status_boolean_methods method. If I went down that path the code could look like the example found below.

class Ticket
  attr_accessor :status
  
  def self.create_status_boolean_methods(*methods)
    methods.each do |method_name|
      define_method :"#{method_name}?" do
        status == method_name
      end
    end
  end
  
  create_status_boolean_methods :planned, :pre_sale, :on_sale
end

This solution is decent, but there are a few things I don't like about it. Firstly, it's usually not important to understand how the create_status_boolean_methods method does it's job. Second, the definition of create_status_boolean_methods is often not near the actual usage of create_status_boolean_methods; therefore, on the rare occasion that you do need to understand how the methods are being defined you'll have to do a little searching within the class. Third, and perhaps the most obvious, it would take less code to define each method on it's own.

The above option is a good solution in some cases; however, Neal and I were looking for something simpler that didn't suffer from the above list of limitations. We came up with the following generalized solution to defining similar methods.

class Class
  def def_each(*method_names, &block)
    method_names.each do |method_name|
      define_method method_name do
        instance_exec method_name, &block
      end
    end
  end
end

Given the def_each method defined on Class the Ticket class becomes easier to read in my opinion. The reason it's easier (in my opinion) is because when I'm scanning the file and I see def_each :planned?, :pre_sale?, :on_sale? do |method_name| I know that similar methods are being defined. Below is the full code now required to define the Ticket class.

class Ticket
  attr_accessor :status
  
  def_each :planned?, :pre_sale?, :on_sale? do |method_name|
    :"#{status}?" == method_name
  end
end

You may have noticed the instance_exec method call in the implementation of the def_each method. I've previously written about instance_exec, but I'll include the implementation here for completeness.

class Object
  module InstanceExecHelper; end
  include InstanceExecHelper
  def instance_exec(*args, &block)
    begin
      old_critical, Thread.critical = Thread.critical, true
      n = 0
      n += 1 while respond_to?(mname="__instance_exec#{n}")
      InstanceExecHelper.module_eval{ define_method(mname, &block) }
    ensure
      Thread.critical = old_critical
    end
    begin
      ret = send(mname, *args)
    ensure
      InstanceExecHelper.module_eval{ remove_method(mname) } rescue nil
    end
    ret
  end
end

My Job Went to India

A few months ago I read My Job Went to India. My conclusion: It's an essential book for any software developer. The book contains a list for:

managing and developing your career as a programmer --Dave Thomas

I actually think reading this book is as important for your career as reading The Pragmatic Programmer is for your skills.

Unfortunately, the title isn't great.

In fact the title is so bad that I've wanted to recommend it for awhile, but I couldn't think of what to write that would convince people to forgive the title and read it anyway.

Then, today, Dave Thomas wrote about the importance of book titles. In his entry he mentions that it was a "Big mistake" to go with the title My Job Went to India.

That's when I knew what to say about the title, my honest opinion.

So, forgive the title and give the book a shot. You'll be glad that you did.

Team Size

3 projects ago I worked on a team of 4 developers. We were highly effective and everyone knew the entire codebase. Following that project I was convinced that small, talented teams was 'the way' to build software.

Then I joined my next team: 16 developers.

I was skeptical at first. The team seemed huge. The idea was that the time to market was critical, but there's a reason that The Mythical Man Month was written, right? No one on the team seemed keen on the idea, but we gave it our best effort. The result: A big success.

Of course, there are a million reasons that a project succeeds and a million ways that it could fail, but in the end we did succeed. Not only did we succeed, but I think everyone on the team was surprised at how well the team worked together towards the delivery.

My current project size? 16 developers again.

While some people remain skeptical of large teams, the time to market conditions we are fighting (again) didn't leave us another choice. While there are inefficiencies that are inevitable, I think a lot of us were surprised by the positives that we found.

A large team provides some benefits that I hadn't previously experienced. The Presenter pattern was born on the smaller project, but it experienced trial by fire on the large project. 16 developers all pushing a pattern in every direction can do a lot to see where it fits and where it falls down.

Also, a large team taught me plenty of lessons about testing. On a large team the test suite grows quickly. Keeping all of those tests performing well requires creativity. The test performance lessons that you learn can be reused for the remainder of your career. Additionally, when working on a small team you generally work on tests where you were the original author. However, on a large team it's much more likely that you'll find broken tests that you've never even seen before. Looking at tests from the perspective of a maintainer can make you a much better test author.

Of course, there are plenty of cons with working on a large team. On my last project I was interested in performance of the application; however, I always ended up working on some other aspect of the application. By the time the project ended some time had been devoted to performance, but I hadn't gained any of the experience that the other team members picked up. Some of that is my fault. At ThoughtWorks we often get to pick what tasks we are involved in. But, when there are several interesting things going on at once, you end up having to choose the one that's most important to you.

I still find that smaller teams are more effective when time to market allows a slower pace. However, try not to miss the lessons available from working with a large team if you find yourself on one. Large teams are good for finding language, framework, and developer limitations quickly. Those limitations should be embraced and fixed for everyone's benefit.

iPhone: Layout View Email and AirPort Music

Two other (granted, lower priority) features that I'd like to see the iPhone include: The ability to view email in layout view and to play music through an AirPort.

Layout view would be nice for email since some emails come in HTML form and expect a wider width.

As far as playing music: My speakers are connected to the same AirPort that my iPhone uses, so it would be nice if I could play music using my iPhone. It's not a huge deal, but it is more convenient to play music without needing to open my laptop

iPhone: Messaging

A few other missing features on the iPhone: MMS messaging and easily sending contact information.

Yes, I can email photos, but not everyone has an iPhone that can check email. MMS has been on the phones I've owned for awhile, so how did the iPhone miss the boat?

As far as sending contact information, I'd be happy if the iPhone could automatically convert contact information into text and send it via SMS. It doesn't seem that you can email contact information either.