Saturday, October 27, 2007

Rails: Unit Test without Rails

I've written in the past about How We Test without hitting the database in our unit tests. I recently joined a project where the team decided to unit test without loading the environment at all (Kudos to George Malamidis for pushing the team in that direction).

I like to test classes in isolation (when unit testing) so this was an easy transition for me. It also solved a problem that had been annoying me for a few projects: even when the unit tests are disconnected from the database, running a focused test (in TextMate) still takes several seconds to load the entire Rails environment.

On my last project we looked at a benchmark once and the test ran in .002 seconds, but the real time was 5.002 seconds. We generally put all of our require statements in environment.rb, which is great for maintainability; however, it lead to our environment taking up to 5 seconds to load. You may not care about 5 seconds, but taking a test that runs in .002 seconds and waiting 5.002 seconds several times over the life of a year long project, on a team of 16 developers, adds up to a bit of annoyance and a fair amount of wasted time

While I really liked the idea, I did have one complaint: Without Rails our codebase (and tests) needed to be littered by require statements. This was fairly easily fixed by requiring activesupport and calling a few methods on Rails::Initializer. After requiring activesupport I was able to remove all the require statements, follow standard Rails naming conventions, and get autoloading for free.

At that point we could test all of our POROs very quickly. We ran a few benchmarks and never got a real time of more than .05 when running an individual test class. I also benchmarked changing the unit_test_helper to require environment.rb. When requiring environment.rb it took 10-12 times longer to run a unit test.

We were happy with the results, but without environment.rb ActiveRecord wasn't defined. Since we also wanted to unit test our AR::Base subclasses (where appropriate) we created AR::Base ducks that do nothing when AR::Base methods are called. Essentially they are struct objects that can read and write to their attributes, but also contain any behavior that you define in the subclasses. This code led to a new gem: arbs (ActiveRecord::Base Struct). The current release is the first release of arbs. This release only contains support for the methods that have been necessary for our current project. The README (the index page for arbs.rubyforge.org) contains a list of methods from ActiveRecord::Base that are supported. Patches are welcome.

I think an example is probably the best way to bring it all together. Image a project that has the following migration that creates a phone_numbers table.

class ReleaseMigration < ActiveRecord::Migration
def self.up
create_table :phone_numbers do |t|
t.column :value, :string
t.column :country, :string
end
end

def self.down
drop_table :phone_numbers
end
end

The following PhoneNumber class is the object representation of the phone_numbers class.

class PhoneNumber < ActiveRecord::Base
validates_presence_of :value
belongs_to :user

def value=(number)
write_attribute :value, number.gsub(/\D/,"")
end

def to_formatted_s
case country.to_sym
when :us then "(#{value[0,3]}) #{value[3,3]}-#{value[6,4]}"
# other formats
end
end
end

Testing the PhoneNumber class to_formatted_s and value methods can be done without hitting the database or utilizing any of ActiveRecord's magic.

require File.dirname(__FILE__) + '/unit_test_helper'

class PhoneNumberTest < Test::Unit::TestCase
test "to formatted s" do
assert_equal "(212) 555-1212", PhoneNumber.new(:value => "2125551212", :country => "us").to_formatted_s
end

test "strip non numbers" do
assert_equal "2125551212", PhoneNumber.new(:value => "(212) 555-1212").value
end
end

All of the above code is fairly boilerplate; however, unit_test_helper.rb is where the magic lies.

require 'rubygems'
require 'test/unit'
require 'dust'
require 'active_support'
require 'initializer'
require 'arbs'
RAILS_ROOT = File.expand_path(File.dirname(__FILE__) + "/../../")
Rails::Initializer.run(:set_load_path)
Rails::Initializer.run(:set_autoload_paths)
ArbsGenerator.run(RAILS_ROOT + "/db/schema.rb")

That's all there is to it. The PhoneNumber tests provide the following output on my box.
focus:~/work/example/test/unit jay$ time ruby phone_number_test.rb 
Loaded suite phone_number_test
Started
..
Finished in 0.000501 seconds.

2 tests, 2 assertions, 0 failures, 0 errors

real 0m0.396s
user 0m0.286s
sys 0m0.106s
Changing to requiring environment.rb yields the following results.
focus:~/work/example/test/unit jay$ time ruby phone_number_test.rb 
Loaded suite phone_number_test
Started
..
Finished in 0.043915 seconds.

2 tests, 2 assertions, 0 failures, 0 errors

real 0m1.862s
user 0m1.336s
sys 0m0.519s
The results are significantly higher, but not 10 or 12 times. This is because the output comes from a newly created Rails application. I expect most Rails applications that load multiple gems and plugins would see results similar to those that my current project produces.

6 comments:

  1. Interesting approach... Have you done something similar for when testing your controllers?

    ReplyDelete
  2. Anonymous12:28 PM

    Nope, I've given up on unit testing controllers. I stick to functional testing them for the time being.

    ReplyDelete
  3. Anonymous3:05 PM

    On Windows startup time is horrendous so this would really help. However Rails 2 generates columns like
    t.date "created_on"
    and so it chokes on the new syntax for defining columns.

    ReplyDelete
  4. Anonymous7:53 PM

    Managed to get it working by stubbing quite a few more AR class methods (such before_save. mmm) and with this code added just after the arbs require it works with Rails 2!

    class BehaviorAppender
    def method_missing(type, method, *arguments, &block)
    column(method, type, arguments)
    end
    end

    Thanks.

    ReplyDelete
  5. Anonymous10:30 PM

    Phil,

    That's fantastic. If you send me a patch I'll be glad to update the library and add you as a contributor.

    Cheers, Jay

    ReplyDelete
  6. Anonymous6:33 PM

    Could you give me a hint, how you test associations like has_many in a unit test. In particular if the tested behaviour depends the 'master'.

    ReplyDelete

Note: Only a member of this blog may post a comment.