Tuesday, February 19, 2008

Rake: Task Overwriting

Update at bottom

By default Rake Tasks append behavior every time they are defined. The following example shows that both definitions are executed.

require 'rubygems'
require 'rake'

task :the_task do
p "one"
end

task :the_task do
p "two"
end

Rake::Task[:the_task].invoke
# >> "one"
# >> "two"

I like this behavior, but sometimes you want to overwrite a task instead of appending to what's already there.

When you no longer want the existing behavior the overwrite method can come in handy.

require 'rubygems'
require 'rake'

class Rake::Task
def overwrite(&block)
@actions.clear
enhance(&block)
end
end

task :the_task do
p "one"
end

Rake::Task[:the_task].overwrite do
p "two"
end


Rake::Task[:the_task].invoke
# >> "two"

The overwrite method is good, but sometimes you want to redefine the task using one of the specialized Rake tasks. I recently wanted to redefine the test task, so I created the abandon method to remove the existing definition.

require 'rubygems'
require 'rake'
require 'rake/testtask'

class Rake::Task
def abandon
@actions.clear
end
end

task :the_task do
p "one"
end

Rake::Task[:the_task].abandon

Rake::TestTask.new(:the_task) do |t|
t.libs << "test"
t.pattern = 'test/**/*_test.rb'
t.verbose = true
end

Rake::Task[:the_task].invoke
# >> Expectations ..................
# >> Finished in 0.00442 seconds
# >>
# >> Success: 18 fulfilled

Hopefully you wont need this type of thing very often, but it can be handy when you want to overwrite a task that has been previously by a framework you've included.

Update below...
Ola Bini pointed out that I'm not clearing the prerequisites. Clearing the prerequisites is something you can do without any modifications to rake.

Rake::Task[:task_name].prerequisites.clear

Also, if you want prerequisites cleared as part of overwrite or abandon, you can easily add it to both tasks.

class Rake::Task
def overwrite(&block)
@actions.clear
prerequisites.clear
enhance(&block)
end
def abandon
prerequisites.clear
@actions.clear
end
end

Sunday, February 17, 2008

Shared Ruby Code

It's no secret that I think plugins are unnecessary. Unfortunately, they are also popular, largely due to to how easy it is to create a plugin and put it in your Rails project.

It's also easy to create a new gem. In 2006 Dr. Nic released the newgem gem. This was a great step in 2006, but it's 2008 now and it doesn't feel like we've moved much in the way of shared ruby code.

We're all to blame.

The vast majority of Ruby work is working with Rails. I do believe the next step for RubyGems is to play seamlessly with Rails. Since RubyGems are great for any Ruby code, it doesn't make sense to add Rails specifics to RubyGems. However, RubyGems could be more opinionated about how gems are structured. Right now, RubyGems library code can be structured in almost any way that you'd like. If the structure followed a stricter convention then auto gem loading code could be written for Rails.

I like newgem, but I think the features provided by newgem should actually be part of RubyGems itself. If the gem command could create a project skeleton it could be opinionated about the file structure without creating a larger barrier to entry.

There is another change that needs to happen. Everyone knows that you should vendor everything for your Rails projects. But, there's no single step to go from uninstalled gem to vendored gem. The path is generally to install it locally and them unpack it in whatever folder you've created that gets autoloaded.

I'm pretty sure a remote unpack is fairly simple, which is why we are all to blame. RubyGems is an open source project. Anyone can contribute patches to make it better.

Lastly, if gems become the standard way to share Ruby/Rails code then we'll need a few rake tasks added to Rails. For example, who wouldn't want a rake command to upgrade to a later version of a gem.

I'm hoping 2008 is the year that shared ruby code becomes synonymous with RubyGems and the year Rails Plugins become unnecessary.

Friday, February 15, 2008

Implementing an internal DSL in Ruby

On Christmas Day 2007 I released the 0.0.1 version of expectations. Expectations is the result of several months of designing what I wanted the Domain Specific Language (DSL) to look like, and a few days of making it executable. To design the DSL, I created what I was looking for without worrying about implementation. This entry is going to focus on what it took to take the DSL from looking good to actually executing.

The following code is an example of what I decided I wanted the DSL to look like (for a state based test, expectations also has support for behavior based testing).

Expectations do
expect :expected do
:expected
end
end

When making a DSL executable I generally begin by building up objects with the values expressed in the DSL. For example, the above code creates a test suite containing 1 expectation where the expected value is :expected and the actual value can be determined by evaluating the block. While designing expectations I tested it using expectations, but that's probably more complex than is necessary for this example, so I'll simply print values instead.

The first step is ensuring that an Expectations::Suite is created when Expectations is called with a block. The following code demonstrates that an Expectations::Suite is created successfully. The code executes since the block given to Expectations is never executed.

require 'singleton'
module Expectations; end

class Expectations::Suite
include Singleton
end

def Expectations(&block)
Expectations::Suite.instance
end

Expectations do
expect :expected do
:expected
end
end

Expectations::Suite.instance # => #<Expectations::Suite:0x1d4fc>

note: Expectations::Suite is a singleton so that it can be appended to multiple times and later used for execution without being stored within another object.

The next step is to verify the number of expectations the suite stores for later execution. The following code verifies that one object is stored in the expectations array.

require 'singleton'
module Expectations; end

class Expectations::Suite
include Singleton
attr_accessor :expectations

def initialize
self.expectations = []
end

def expect(expected, &actual)
expectations << :expectation
end

end

def Expectations(&block)
Expectations::Suite.instance.instance_eval(&block)
end

Expectations do
expect :expected do
:expected
end
end

Expectations::Suite.instance # => #<Expectations::Suite:0x1c82c @expectations=[:expectation]>
Expectations::Suite.instance.expectations.size # => 1

The final step is to create an object for each expectation and verify the values of both the expected and the actual.

require 'singleton'
module Expectations; end

class Expectations::Suite
include Singleton
attr_accessor :expectations

def initialize
self.expectations = []
end

def expect(expected, &actual)
expectations << Expectations::Expectation.new(expected, actual)
end

end

class Expectations::Expectation
attr_accessor :expected, :actual

def initialize(expected, actual)
self.expected, self.actual = expected, actual.call
end
end

def Expectations(&block)
Expectations::Suite.instance.instance_eval(&block)
end

Expectations do
expect :expected do
:expected
end
end

Expectations::Suite.instance # => #<Expectations::Suite:0x1b184 @expectations=[#<Expectations::Expectation:0x1b0e4 @expected=:expected, @actual=:expected>]>
Expectations::Suite.instance.expectations.size # => 1
Expectations::Suite.instance.expectations.first.expected # => :expected
Expectations::Suite.instance.expectations.first.actual # => :expected

note: expectations actually defers determining the value of actual until the suite is executed, but that's outside the scope of this example.

The bottom two lines of the last example show that the Expectations::Expectation instance is being populated successfully.

That's it, at this point the DSL can successfully be parsed. Of course, making the expectations run to verify that the values match still needs to be done, but that's no different than working with any plain old ruby code. The DSL can be evaluated and the framework can work with the objects that have been created.

As the example demonstrates, designing an internal DSL in Ruby is very easy largely due to open classes, class methods, and the ability to eval. The above code for evaluating the DSL is less than 30 lines of code.

Thursday, February 14, 2008

Pair Programming thoughts

It's very rare that I go to a client and pair programming is not an issue. Developers love their desks, personal keyboard configurations, and music. I like that stuff too; unfortunately, any personal space benefits or keyboard configurations cannot make you more productive than a team of effectively pairing developers of the same skill level. I can't prove that, but you can't prove that I'm wrong either. Of course, to begin (a futile) attempt to prove which is more effective we would need to decide on what we were measuring. Building software isn't only about the number of features delivered, it's also about the quality of the software delivered. It's also about having a team that can maintain an application as (inevitable) turn over occurs.

That's what's so great about Pair Programming, it improves the quality of software while spreading knowledge and best practices at the same time. Everyone benefits from the sharing of knowledge from small things such as efficiency gaining key short cuts to large things such as understanding the big picture of the project and spreading common patterns throughout the system.

Adopting pair programming isn't as hard as you might think. In May of 2005 I wrote that setup is important. I was wrong, setup is not important, it's the most important step towards adoption. Anyone looking to bring pair programming into their organization doesn't need to do much. Get tables large enough for several team members to sit in the same area comfortably. Get the latest and greatest desktops for the pairing area. (My current client got me a beautiful Mac Pro). Get a desktop with dual video out if possible, or get a video splitter. Buy 2 keyboards, 2 mice, and 2 (large) monitors for each pairing station. If the pairing stations are the best systems to work on, developers will naturally migrate to those machines.

70% of ensuring adoption success is work station set up.

I suggest not mandating pairing, let it happen naturally. I also suggest allowing the developers to keep their desks. Everyone needs an area to escape to and read slashdot when they need a break. If you just bought everyone 30 inch monitors you may need to move those to the pairing stations and give everyone small monitors for their personal desks. The idea is to allow people to have their space, but encourage them to use the pairing stations by providing a superior set up.

20% of ensuring adoption is ping pong pair programming.

Ping pong pair programming is fantastic at keeping everyone involved and given the opportunity to drive.

The remaining 10% will vary by organization, but if you handle the first 90% the rest should just fall into place.

Tests reflect code quality

As a consultant for ThoughtWorks I get to see a lot of different codebases. A pattern that I've seen in almost every codebase is that you can usually tell how well a codebase is written by the tests that cover the codebase.

The first indicator of quality is how covered the code is. In my experience the coverage percentage is generally representative of the confidence the business has in the software. Unfortunately, a low coverage percentage can show probable lack of quality, but a high coverage percentage does not ensure high quality. If you have less than 80% coverage, you probably have software that is broken in places. However, if you have better than 80% coverage it only means that you have better than 80% coverage. Again, coverage does not ensure quality.

The second indicator of quality is how easily new tests can be added to the existing suite. Working with someone for 10 minutes is usually enough time to see how much effort is required to add new tests (and features). If the developers believe in TDD, but are consistently frustrated with the level of effort it takes to write a test, you probably have a problem. Of course, if the developers do not believe in TDD, raising the barrier to entry with code that is hard to test only compounds the problem.

New tests can also be fairly easy to write, but very hard for new team members to understand. For example, special test superclasses designed to minimize the pain are also warning signs that the system may be unnecessarily complex.

Every project has a ratio of lines of application code and lines of test code. This ratio is effected by the language decision. The ratio does not need to be 1:1. In fact, language isn't the only factor in determining what the correct ratio is. For example, an internal application for a cafe that is not mission critical should not be as thoroughly tested as the command system for NASA's shuttles. The point is not to set some language standard ratio, but every project should recognize that if their desired ratio is 1:4 and it takes 8 times longer to write the tests than it takes to write the feature, something is wrong. The time to implement a feature versus the time to write the tests should be roughly equivalent to the application loc to test loc ratio.

While you do not need to strive for a 1:1 ratio, the farther you stray from 1:1 the more you should be looking at the reason for doing so. If your language requires a larger number, so be it. If your project demands more than the normal amount of tests, fair enough. But, you should always be on the look out for your ratio growing unjustifiably. It's always good to have at least one skeptic who believes that 1:1 is what you should be striving for. If you need 8+ lines of setup code to test a one line method you have a problem, and it pays to have a guy who is not afraid to point that out.

Conversely, I've rarely found a codebase that was easy to test that wasn't well written. I believe this is true largely because systems that are easily testable are generally very loosely coupled. Generally, loosely coupled applications are written fairly well to begin with. However, even if a component is poorly written, it will be easy to refactor or rewrite since it can be done independently of the rest of the system.

Along the same lines, when people ask me how to improve a legacy, monster codebase the first thing I recommend is breaking as many dependencies as possible. Once an application is broken into components it's possible to make educated decisions on what to refactor and what to redo from scratch.

Getting tests around a legacy codebase is always painful; however, it can also give you direction on where the application can be logically broken up. While writing tests for a legacy codebase you should keep track of dependencies that need to be setup, mocked or stubbed but have nothing to do with the current functionality you are focusing on. In general, these are the pieces that should be broken out into components that are easily stubbed (ideally in 1 or 0 lines).

Well written tests generally require loosely coupled, single responsibility code. The same type of code generally increases the quality of the codebase as a whole. Thus, tests that are readable and maintainable generally ensure the same type of codebase.