Jay Fields' Thoughts

I'm going to be presenting on Business Natural Languages (Domain Specific Languages for Subject Matter Experts) at QCon San Francisco in November.

Labels: upcoming presentations

Before I joined ThoughtWorks I worked for Nelnet. Nelnet is a student loan products and services organization. While employed at Nelnet (around Sept 2004), I was sent to an education conference where one of the keynote speakers presented data showing that students who were born after 1981 learn differently than those born prior. The reason this data was important was because it demonstrated that students born after 1981 were far more effective at learning through watching, listening, and doing. The speakers conclusion was that the educational system needed to react by creating classes which didn't rely so heavily on simply reading a textbook. At the time I thought the information was interesting, but not particularly relevant to me.

Today, some of the more popular educational materials can be found in audio (podcasts) or audiovisual (screencasts) form. Additionally, sites like Try Ruby encourage you to give kinesthetic learning a try. After seeing the growth in each of these areas over the past few years, I think it's fair to say that the conference speaker was on to something.

I sent a draft of this entry to Geoffrey Grosenbach for review. He sent back a few more reasons why he believes in screencasts as good teaching tools.

• Efficiency of time: It takes dozens of hours to read a 300 page book straight through. But a screencast can pack the most relevant information into an hour.
• Passive/Active: You can sit back and pick up whatever elements are interesting to you. Some people say the most valuable part is learning auxiliary shell shortcuts or workflow tips.
• Graphical/Textual: I don't know any tech publisher who pays an artist to draw helpful diagrams. Screencasts are inherently graphical and it makes sense to include a few diagrams that explain the topic better than paragraphs of text would.

Here's a few links for those intersted in trying out some alternative learning tools.

• PeepCode
• Info Q (Presentations and Interviews)
• Rails Podcasts
• Railscasts

Labels: learning, screencasts

There's plenty of hype and momentum around Ruby on Rails. The screencasts give quick demos of how easy it is to craft a solution using Ruby on Rails. There's even a humorous picture implying how simple it is to learn and use Rails to build web applications.

The screencasts and pictures don't lie. It is easy to learn Rails and quickly become productive. However, the dirty little secret is that it's very hard to be come an expert at utilizing Rails.

Anatomy of a Rails Web 2.0 Application
Rails provides many helpers that make your life easy. But, you can't entirely hide the fact that you'll need to be proficient with Ruby, JavaScript, YAML, and SQL. Just like Rails, getting started with any of the above languages is easy, it's mastering them that takes time and effort.

Becoming an expert with each of the above languages involves gaining knowledge and becoming proficient with several aspects of the language.

Development Environment
There's no IDE that can provide all the features you'd like for Ruby, JavaScript, YAML, and SQL. Some are (hopefully) on the way, but they still suffer from speed, feature, and stability issues. That generally means is you'll have two options:

• Find a productive development environment that specializes in one or two of them and use a different IDE/Text Editor for each language
• Find an editor that provides decent support for all of them.

As of today, I use TextMate, which I find acceptable for all 4 languages. If I couldn't use TextMate, I'd be using IntelliJ. Those are my preferences, but I suggest you try a few different options and see what works.

Language Features
Look for documentation and try out the examples for each language you are looking to master. Ruby has decent documentation on http://www.ruby-doc.org/. YAML documentation can be found on http://www.yaml.org/. JavaScript and SQL have seemingly endless documentation resources. I've always been a fan of the O'Reilly books, so I'd recommend Safari Books Online, but there's plenty of other options if you'd like.

But, don't stop with reading. Reading documentation and trying examples isn't going to bring you to the expert level. Another great way to get some insight into a language is to look at the successful frameworks and see how they are implemented. Browsing the Rails codebase should provide examples of almost every feature available in Ruby. Likewise, understanding the Prototype and Script.aculo.us libraries should greatly improve your JavaScript skills.

Testing
Testing is valuable in any language. Ruby provides a few different testing solutions, but the most popular are still Test::Unit and RSpec. JavaScript also provides several options for testing. I can't say there's a clear winner, but JsUnit and Crosscheck seem to be gaining momentum. As far as testing SQL and YAML, I generally functionally test them through Ruby. Additionally, Selenium is a great resource for acceptance testing Rails applications.

Frameworks
Both Ruby and JavaScript provide a plethora of framework options. Ruby provides RubyGems for distributing shared Ruby code. RubyGems are frameworks that can provide all kinds of building blocks for creating applications. In the past I've used the Prototype and Scriptaculous JavaScript libraries, but my current team is enjoying using JQuery.

Debugging
Debugging Ruby is a bit primitive these days. The simplest form of debugging is printing to standard out. You can roll your own simple debugging solution also. Additionally, ruby-debug looks promising. None of the solutions are great, so it's good to have all 3 tools at your disposal. For JavaScript debugging I suggest getting proficient with FireBug. SQL debugging solutions are generally vendor specific, check the docs of the database that you choose for a good suggestion. YAML doesn't generally require a debugger since most errors simply result in parsing problems.

What else makes Rails difficult to master?
Mastering languages is important, but it's not enough to run a high traffic Rails application. You'll still need to understand performance optimization and deployment strategies. There's an abundance of Rails production heuristics available, but little of it is well documented. I'd suggest reading the following blogs: Brainspl.at, Err the Blog, and RailsExpress. Beyond those blogs the web provides several other resources. A Google search should provide more than enough reference material.

That's a ton to learn
It's true, that is a lot of information to digest. But, experts aren't made overnight. The good news is, you can get started very easily and expand your knowledge as needed. There is a large amount of information available, but don't feel overwhelmed: The majority of the information is very straightforward. Becoming an expert with Rails takes time more than anything else.

Labels: javascript, rails, ruby, sql, yaml

A common question you receive as a Ruby developer is: Don't you miss robust debugging support? The short answer is No, but that's not entirely true. The reality is that Ruby developers are generally willing to make trade-offs in order to work with a language that they find more pleasing.

Do I wish I had a great IDE with fantastic debugging support? Of course I do, and I'd also like refactoring support while we are daydreaming. Until that day comes, I'll continue to look for other ways to be effective.

To mitigate the debugging issue the Ruby developers I work with end up relying very heavily on their tests; therefore, debugging becomes less of an issue. Unfortunately, even the best testers can't always isolate complicated issues. In these circumstances the best Ruby developers are often forced to use the age old debugging strategy of printing values. I was recently in one of these situations and I used the following statement in lieu of the typical p statement.

loop do
  p eval(gets)
end

I do miss mature tools and infrastructure. However, the ease in which I can create adequate solutions in Ruby makes it hard to consider doing anything else, currently.

Labels: debugging, ruby

Over the past two and a half years I've interviewed several ThoughtWorks candidates. These days I tend to put people into 3 categories when interviewing them: Gross Exaggerator, WYSIWYG, and the Semi-bluffer.

The Gross Exaggerator
Being a Gross exaggerator is never a good idea. Your resume is the first clue. I've yet to meet anyone who's an expert* with .net, Java, Ruby, Smalltalk, and Lisp. There's just not enough hours in the day to actually be an expert in each language. Anyone with several buzzwords on their resume signals two things to me: They likely have no idea what they don't know, and they aren't picky enough about technologies they are willing to work with.

I took C in college also, but one semester of Data Structures does not an expert make. I did Java at my first job, but I have no desire to program in it now, so what's the point of listing it as one of my skills? I'm not saying that my resume doesn't contain the word Java. In fact, it does state in my first job description that we used Java, but my skills list is not a summary of what I've done in my past. My skills list is the list of skills that I've kept up with and am interested in continuing to keep up with. A bloated skills list indicates that you think you know more than you do, or you haven't spent the time to be and expert in anything; both situations look bad.

Being a Gross Exaggerator in an interview is painful. You generally have a bit of knowledge of several things, but almost no in depth knowledge of anything. Most questions quickly expose this. Gross Exaggerators are fairly easy to spot, and I always pass on hiring them.

WYSIWYG (What You See Is What You Get)
It's very easy to be a WYSIWYG candidate, because you are very clear about what you know and what you don't. A WYSIWYG candidate never gives a partial explanation; they either tell you every painful detail, or claim to know little or nothing about the topic.

Hiring a WYSIWYG is an easy choice, but I don't think being a WYSIWYG is the best option. The problem with being a WYSIWYG is that there are several Gross Exaggerators and some Semi-bluffers that will be competing with you. Someone who claims to have the knowledge you do, plus more, will always be an attractive choice for a potential employer.

The Semi-bluffer

In [poker] games with multiple betting rounds, to bluff on one round with an inferior or drawing hand that might improve in a later round is called a semi-bluff. A player making a semi-bluff can win the pot two different ways: by all opponents folding immediately or by catching a card to improve the player's hand. --Wikipedia

Semi-bluff is a term that I picked up playing poker. A few of the things I love about poker is that you don't know what the competition has, you don't know what will be needed in the end to win, and you probably don't know if you are currently winning or losing. Of course, each of those things can be said about almost any interview.

I think Semi-bluffing interviewees generally do the best because they demonstrate what they know, suggest that they know more, but admit what they don't know when directly challenged. In fact, I believe the above quote could be rewritten as the following to describe Semi-bluffing in an interview.

Almost all interviews involve a series of questions, to give a vague answer to one question when you don't know enough details can be called a semi-bluff. An interviewee making a semi-bluff can pass that round of questioning two different ways: if the interviewer doesn't request more information or if the follow up question happens to hit one aspect where you do understand the details. A semi-bluff can also be a success if the interviewee admits to not knowing the answer at the moment, but later in the interview process demonstrates that the desired knowledge has been acquired.

An example of a Semi-bluff could be demonstrated as the following.
Interviewer: Do you have experience with deploying Rails applications.
Interviewee: On my current project we use Capistrano and deploy to RHEL. The Webserver is Nginx fronting a Mongrel cluster.

Notice that the interviewee hasn't stated that he did any of that work, instead he's simply listed his current project's deployment plan. The questions may end right there, but if they don't you can easily follow up with: Truthfully, I've been focused on other parts of the application so I only have a high level view of our deployment. However, I believe that it would be very easy for me to get into the details of your specific deployment if I am hired. Following the interview, the interviewee should then dig into the details of their current deployment and send answers via email to the interviewer. Obviously, the faster the email is sent, the better.

A Semi-bluff in poker, and while interviewing, takes a great deal of finesse. Coming off as a WYSIWYG while Semi-bluffing is safe, but coming off as a Gross Exaggerator can cost you a job. Therefore, it's obvious which way you should lean if you find Semi-bluffing isn't working out for you.

Try Semi-bluffing a question in your next interview. It can be a great rush, but be sure to admit when you don't know something. Also be careful about choosing which question to Semi-bluff. A direct and detailed question isn't a good candidate for a Semi-bluff. When someone knows they have "the nuts" a bluff is an (often embarrassing) waste.

* I define expert as someone who is knowledgeable concerning: syntax, development environment, advanced language features, how and what to test, framework choices, details of several frameworks of the language, debugging, performance optimization, deployment strategies, and pattern implementation specific to the language.

Labels: interview

Test-Driven Development was introduced to me by way of xUnit frameworks.

In the beginning, all xUnit frameworks looked very much the same. All tests lived within a class that inherited from an abstract test case class, and all tests were defined by creating a method that begin with the word test. However, as time progressed the xUnit frameworks began to take advantage of features specific to individual languages.

Martin Fowler's bliki entry Xunit contains a description of how NUnit matured.

The first version of NUnit even had a "isVAforJava" method which originated in special handling for Visual Age for Java. Others have been more sophisticated: NUnit 2.0 was praised by Anders Heljsberg for its use of attributes in C#.

Utilizing language specific features was the first step; however, as testing methods mature it's becoming more clear that the features of xUnit also need to mature.

The first framework I noticed that decided to start breaking rules was RSpec. The 1.0 version of RSpec provided a new syntax for defining test cases and for defining assertions. The following example shows how you would write the same test in Test::Unit (a traditional xUnit framework) and RSpec.

# RSpec 
describe Bowling do
  before(:each) do
    @bowling = Bowling.new
  end

  it "should score 0 for gutter game" do
    20.times { @bowling.hit(0) }
    @bowling.score.should == 0
  end
end

# Test::Unit
class BowlingTest < Test::Unit::TestCase
  def setup
    @bowling = Bowling.new
  end
  
  def testShouldScoreZeroForGutterGame
    20.times { @bowling.hit(0) }
    assert_equal 0, @bowling.score
  end
end

I do believe that the RSpec version is more semantically pleasant. However, given a programmer who practices TDD, when they maintain tests they use an xUnit clone, then I'm not sure the semantic benefit is worth learning a new framework. At least, I wasn't at first.

Another new xUnit framework is xUnit.net, announced recently on James Newkirk's blog. xUnit.net excites me more than any other piece of software that I'm likely to never use. I'll probably never use it since I don't plan on going back to .net development, but the features are exciting nonetheless. James et al have taken their test-driven development experiences and created a framework that guides them towards writing better tests. James' entry gives the full details (which I suggest reading even if you never plan on doing any .net work), but a few features I like are the removal of setup and teardown, and the aspect-like functionality. You may be less impressed since Ruby provides aspect-like functionality very easily, but I believe it's nice to see a xUnit framework built with extension points in mind.

These new frameworks are exciting because they are incorporating lessons learned from the past few years of practicing test-driven development. As our experience with TDD grows, so should our tools.

Labels: TDD, testing, xUnit

I was recently looking over a codebase and noticed that several different files contained require statements. I'm not talking about the require statement that each test file has, I'm taking about seeing require 'ostruct', require 'enumerator' or require 'soap4r' in numerous class definitions.

For the last few projects I've been following the convention of putting all the require statements in environment.rb. There are more efficient strategies, but I find this to be the most maintainable of the solutions I've tried.

While following the above strategy I haven't run into any issues with two libraries defining behavior on differing classes that are named the same, but I have heard concerns from team members on the time it takes to require each library. I think that's a valid concern and I was planning on addressing it by writing a method that takes an array of strings and requires them one at a time. The new method would also benchmark each require and print a warning if the require is taking an unacceptable amount of time.

Please drop me a line (or blog it and leave a link) in the comments if you have a strategy that's worked well for you.

Labels: library loading, ruby

Here are the resources relevant to my RailsConf Europe 07 presentation: Extending Rails to use the Presenter Pattern.

The Presentation: http://www.jayfields.com/images/RailsConfEurope2007.pdf

My entries (in order)

• Model View Controller + Presenter?
• Another Rails Presenter Example
• Presenters - An additional layer alternative
• Rails Presenter Pattern

Jamis Buck

• Skinny Controller, Fat Model
• Moving Associated Creations to the Model

Courtenay

• Simple Presenters

Labels: railsconfeurope07

Sometimes your pair is uninterested in the task at hand. It's never fun for either person when working with an Uninterested Pair.

There are several reasons that a pair can be uninterested, but that isn't going to be the focus of this entry. Addressing why a pair is uninterested is a people question that I'm not interested in addressing in a general way.

A benefit of pair programming is getting the best solution when two developers collaborate. Clearly, an uninterested developer is not collaborating, thus the solution is likely inferior. At a minimum, the solution is as good as it would have been and 1 person's time was completely wasted.

The problem with uninterested pairs is that they can spend an entire day paying little attention and contributing almost nothing. If you find yourself stuck with an uninterested pair there are a few things that can help. Dealing with an uninterested pair is very similar to dealing with a Distracted Pair. Ping Pong Pair programming helps turn an uninterested pair into a contributing pair; however, often their level of contribution isn't up to their full potential. I've found that when dealing with an uninterested pair it's actually more beneficial to feign confusion.

Feigning confusion can be boring, really boring. You generally have a grasp on what needs to be done. You also generally have the solution in your head. You can implement this solution on your own and bring the uninterested pair along for the ride; unfortunately, in my experience that's exactly what happens. But, simply taking the uninterested pair along for the ride isn't the best long term solution. That pair may someday be asked to improve the existing solution. If they can't, because they weren't paying attention, you may be taken away from your task. Worse, you may be off the team and the context of the code is permanently lost.

So, feigning confusion is boring, but it is also in the best interest of the team. Instead of implementing the solution, ask your pair: How we are going to get this done? Don't accept their first answer. Their first answer is likely going to be fast, but incredibly ugly. Provide your own input, but don't let on that you have a solution. Instead, stick to asking questions. Chances are, you are going to understand their solution. It doesn't matter if you understand their solution, pretend that you don't. Force the uninterested pair to write the first 3 tests and implement the code that causes the tests to pass. At that point, take over and write a test, but force them to implement the solution. Then slowly proceed into Ping Pong Pair programming, but don't jump straight into it. Instead, force the uninterested pair to do about 65% of the coding. It shouldn't take long for the uninterested pair to be a fully contributing pair, which benefits everyone.

Labels: pair programming

Ever since I wrote Spellchecking Pair I've been noticing other patterns concerning pairing. Distracted Pair, as the name implies, is a pair who is easily distracted by their surroundings.

Distracted pairs tend to spend less time typing, because they are always busy helping everyone around them. Distracted pairs are not useless; in fact they generally contribute significantly to the code the pair is working on. However, distracted pairs are rarely contributing at their full potential because they are generally attempting to solve the problems of multiple pairs at the same time. Some distracted pairs can also spend too much time checking email and doing other ancillary tasks.

Joel believes that open space working areas are fun but not productive. While I don't agree with him, I can see how a few distracted pairs could easily make a manager draw that conclusion.

I've been a distracted pair. I've also worked with several distracted pairs. I believe that it happens to all of us from time to time. In the past I've seen distracted pairs managed by creating rules. For example, a "no personal laptops in the development room" rule existed on a previous project. This rule can help address the problem, but it creates other problems.

My preferred method of handling a distracted pair is to practice Ping Pong Pairing. Ping Pong Pairing forces both pairs to be paying attention at level one. Another nice attribute of this suggestion is that it doesn't require any additional rules that could have problematic side effects.

I'm led to believe that Joel doesn't believe in Pair Programming either, so my solution may be worthless to him. However, if you are working in an environment where Pair Programming is embraced, I suggest using Ping Pong Pairing to get the most out of working with a Distracted Pair.

Labels: pair programming

While looking through our core extensions I found the Array#collect_with_remaining method. The method was designed to yield each element with the other elements of an array and collect the results.

This method (wasn't, but) could be used to find duplicates within an Array*. I would suggest not using it for finding duplicates (since there are other easier ways), but the example should help in understanding what collect_with_remaining can do. If you wanted to implement Array#duplicates using collect_with_remaining you could write the following method.

class Array  
  def duplicates
    block = lambda { |element, remaining| element if remaining.include?(element) }
    self.collect_with_remaining(&block).uniq.compact
  end
end

The duplicates method shows that Array#collect_with_remaining yields each element and the remaining elements of the array, then collects the results. The result is a list of the duplicate values when the remaining did contain the element, and nil for each element that was not a duplicate. After the results have been returned the array is uniq(ed) and compact(ed) to remove the nil and duplicate entries. The result is all the duplicate values from the array. The following test verifies that the duplicates method works as expected.

unit_tests do
  test "return duplicates" do
    assert_equal [1, 3], [1, 2, 3, 1, 4, 3].duplicates
  end
end

On to the actual collect_with_remaining implementation, right after we write the tests.

unit_tests do
  test "head and tail are collected first" do
    assert_equal [1, [2, 3]], [1, 2, 3].collect_with_remaining { |element, remaining| [element, remaining] }.first
  end

  test "elements and remaining elements are collected" do
    expected = [[1, [2, 3]], [2, [1, 3]], [3, [1, 2]]]
    assert_equal expected, [1, 2, 3].collect_with_remaining { |element, remaining| [element, remaining] }
  end
end

The implementation I chose is straightforward, assuming you are comfortable with Enumerable#enum_with_index.

class Array
  def collect_with_remaining
    self.enum_with_index.inject([]) do |result, (element, index)|
      remaining = self.dup
      remaining.delete_at(index)
      result << yield(element, remaining)
    end
  end
end

And, here's all the code if you're interested.

require 'enumerator'
require 'rubygems'
require 'test/unit'
require 'dust'

class Array
  def collect_with_remaining
    self.enum_with_index.inject([]) do |result, (element, index)|
      remaining = self.dup
      remaining.delete_at(index)
      result << yield(element, remaining)
    end
  end

  def duplicates
    block = lambda { |element, remaining| element if remaining.include?(element) }
    self.collect_with_remaining(&block).uniq.compact
  end
end

unit_tests do
  test "head and tail are collected first" do
    assert_equal [1, [2, 3]], [1, 2, 3].collect_with_remaining { |element, remaining| [element, remaining] }.first
  end

  test "elements and remaining elements are collected" do
    expected = [[1, [2, 3]], [2, [1, 3]], [3, [1, 2]]]
    assert_equal expected, [1, 2, 3].collect_with_remaining { |element, remaining| [element, remaining] }
  end

  test "return duplicates" do
    assert_equal [1, 3], [1, 2, 3, 1, 4, 3].duplicates
  end
end

As always, please post alternate suggestions in comments, or better yet, blog them and leave a comment with the link.

* If I actually wanted to check for duplicates I would use Array#duplicates, which is defined in Facets.

Labels: array, ruby

Yesterday I was working on some code that needed to split an array into 3 different chunks. My pair, Philippe Hanrigou, wrote tests similar to the following.

unit_tests do
  test "chunk evenly" do
    assert_equal [[1], [2], [3]], [1, 2, 3].chunk(3)
  end

  test "divide evenly" do
    assert_equal [[1], [2], [3]], [1, 2, 3] / 3
  end
  
  test "chunk unevenly" do
    assert_equal [[1, 3], [2]], [1, 2, 3].chunk(2)
  end
  
  test "chunk creates empty array when there aren't enough elements in the array" do
    assert_equal [[1], [], []], [1].chunk(3)
  end
end

To make the above tests pass, I wrote the following code.

class Array
  def chunk(number_of_chunks)
    chunks = (1..number_of_chunks).collect { [] }
    while self.any?
      chunks.each do |a_chunk|
        a_chunk << self.shift if self.any?
      end
    end
    chunks
  end
  alias / chunk
end

The above code works, but I'm curious to see if there is a better solution. How would you make the tests pass?

Here's the code in it's entirety.

require 'rubygems'
require 'test/unit'
require 'dust'

class Array
  def chunk(number_of_chunks)
    chunks = (1..number_of_chunks).collect { [] }
    while self.any?
      chunks.each do |a_chunk|
        a_chunk << self.shift if self.any?
      end
    end
    chunks
  end
  alias / chunk
end

unit_tests do
  test "chunk evenly" do
    assert_equal [[1], [2], [3]], [1, 2, 3].chunk(3)
  end

  test "divide evenly" do
    assert_equal [[1], [2], [3]], [1, 2, 3] / 3
  end
  
  test "chunk unevenly" do
    assert_equal [[1, 3], [2]], [1, 2, 3].chunk(2)
  end
  
  test "chunk creates empty array when there aren't enough elements in the array" do
    assert_equal [[1], [], []], [1].chunk(3)
  end
end

Yesterday, Mike Clark posted a blog entry asking "How Would You Test This?" The topic of the entry is how to test controllers. Mike's article is good, and you should start there for context.

I think Mike has the Functional tests well covered in his entry. However, I've never been able to accept that Controllers can only be functionally tested. In my earlier Rails days I would probably have written something similar to the example below and considered it a Unit Test. (Sorry Mike, I prefer Mocha to Flexmock)

require File.dirname(__FILE__) + '/../test_helper'

class MenuItemsControllerTest < Test::Unit::TestCase
  
  def test_create_with_valid_menu_item
    controller = MenuItemsController.new
    controller.expects(:params).returns(:menu_item => {:title => 'Classic'})
    controller.expects(:flash).returns(flash_mock = mock)
    controller.expects(:menu_items_url).returns(:menu_items_url)
    controller.expects(:redirect_to).with(:menu_items_url)
    flash_mock.expects(:[]=).with(:notice, "MenuItem was successfully created.")
    controller.create
    assert_not_nil controller.instance_eval { @menu_item }
  end
  
  def test_create_with_invalid_menu_item
    controller = MenuItemsController.new
    controller.expects(:params).returns(:menu_item => {})
    controller.expects(:render).with(:action => :new)
    controller.create
    assert_not_nil controller.instance_eval { @menu_item }
  end

end

There's a few concerns with the above example. Probably the largest concern is that a lot of mocking almost always results in brittle tests. Another concern is that there's so much noise in the test that it's hard to determine what the intent of the test is. Furthermore, the test is clearly specifying not what should be done, but how it should be done.

On my current project, David Vollbracht created something called an IsolatedControllerTest. The IsolatedControllerTest class mocks the render method and does a few other things to give you the ability to test your controllers in isolation. While David had the right idea, it simply didn't catch on. We have a few IsolatedControllerTests; however, the majority of controller tests are all Functional.

As Mike points out, the current situation often leads to a lack of testing controllers. I'm no more okay with that option than Mike is, but we've had other more interesting problems to solve so I put the thought on a back burner. Luckily, Mike isn't willing to settle.

I think part of the problem is that controllers are not Good Citizens. Controllers violate the first rule of Good Citizenship. Upon creation (Controller.new), controllers are not in a valid state. Instead, controllers depend on being initialized within the framework and having their state set post construction time. That ends up being a problem for unit testing since it requires each test to set additional state on newly created controllers.

Another problem is that methods are marked as protected. For example the *_url and *_path methods are all only accessible from within a controller. I'm sure this was done with the best intentions; however, I subscribe to the philosophy that making something public to increase testability is a good idea.

One of the larger shortcomings that controllers have is that they don't behave like POROs (Plain Old Ruby Objects). Ideally, testing controllers should be as easy as writing the following tests.

require File.dirname(__FILE__) + '/../test_helper'

class MenuItemsControllerTest < Test::Unit::TestCase
  
  def test_redirection_to_menu_items_on_success
    controller = MenuItemsController.new(:params => { :menu_item => {:title => 'Classic'} })
    controller.expects(:redirect_to).with(controller.menu_items_url)
    controller.create
  end
  
  def test_flash_notice_on_success
    controller = MenuItemsController.new(:params => { :menu_item => {:title => 'Classic'} })
    controller.create
    assert_equal "MenuItem was successfully created.", controller.flash[:notice]
  end
  
  def test_menu_item_is_available_for_view
    controller = MenuItemsController.new(:params => { :menu_item => {:title => 'Classic'} })
    controller.create
    assert_instance_variable(:menu_item).exists_in(controller)
  end
  
end

I look forward to the day when I can create controllers (via Controller.new) and I will have a valid object that I can easily test (with tests similar to the ones above).

Labels: controller, rails, testing

Update begins with the quote

Several years ago I was working on a web application that had a login screen. I created separate error messages based on whether the user could not be found or the password was invalid. It wasn't a requirement, but I thought it was a nice to have (and I hadn't begun doing Agile, yet). When I demoed the feature to my boss he asked "Isn't that a security concern? Now hackers will know what are valid usernames." At the time I thought his observation was fair and I removed the feature.

Fast-forward a few years. These days, Several of my logins are my email address. Actually, my logins are usually an email address I set up for individual sites. For example, I might create americanairlines@jayfieldsthoughts.com if I were going to give American Airlines my email address (Don't bother emailing me at that address, it's not real). However, sometimes I don't bother to create an address for a site; I'll use something generic such as throwaway1@jayfieldsthoughts.com. Of course, this creates a problem when I go to a site that I use about once a year. Did I sign up with a specific address or did I use a throwaway one? The usual workflow from that point is to try a specific email address, and click the "forgot password" link if it fails. In forgot password I can try my specific email and a few throwaways if necessary. I know when I find a match, because the site tells me that "an email has been sent."

Here's where I have an issue. Maybe I can't find out from the login screen what is a valid username and what isn't, but it only takes me a click to get to a screen that tells me what a valid username is. Do we really believe that a hacker is going to give up on the login screen and not just hit the "forgot password" link like I do? I don't believe that, which brings me to the question: Why not just show me on the login screen that the email address is invalid.

Of course, this doesn't apply to sites that use non-email usernames. But, those sites that do, please improve my user experience and save me the extra click. You aren't providing me any extra protection. In fact, the only people you are slowing down are your users.

If the goal is to stop attackers enumerating valid account names then the forgotten password screen should not indicate the difference between a hit on username and a miss.

That's not a problem for the valid user as they'll get an e-mail to the valid account, but it stops the attacker from getting that information. -- Rory McCune

I forgot to address this. I considered the idea of changing the forgot password screen to display "an email may or may not have been sent" message following a submit. While it's an option, I basically dismissed the idea as a user experience so poor it wasn't worth the additional security. The message alone isn't horrible, but the problem is that a speedy email isn't always guaranteed. So, I can imagine a scenario where I misspell my email address, submit and get the "no confirmation" confirmation, and never receive an email. Or, I submit various email addresses waiting for an email to show up eventually. Later, I'm disappointed to find that I got it right the first time, but the server wasn't sending out emails very quickly and I wasted 15 minutes trying to guess all possible email combinations.

If security were such an issue that I had to display a meaningless message, I believe a superior solution would be to create unique usernames instead.

Good observation though and thanks for the comment Rory.

Labels: login error, security

I've previously written about creating custom assertions for delegation and Validatable validations. Both solutions followed the same pattern, which is very similar to the example below.

Model.verify do
  fluent.interface
  fluent.interface
  ...
end

I do like the readability provided by the pattern; however, it isn't a practical solution.

My colleague and friend, John Hume, also created a solution for testing delegation: Handoff. The thing I really like about Handoff is that it follows the traditional test definition pattern.

def test_description
  assert_handoff....
end

At first glance this might not seem like a big deal; however, following the traditional pattern is crucial when you want to run a test in isolation.

A large problem with my delegation custom assertion is that you must run all the tests in a file if you want the delegation tests to run. It's possible to create ways to run the individual lines; however, with each custom assertion a new way to run it in isolation must be devised. A far simpler solution is to stick to the common test definition pattern and leverage existing tools that know how to run one test at a time.

The next release of Validatable will include custom assertions that follow the traditional test definition pattern.

Labels: custom assertions, ruby

Last weekend, at The Rails Edge (which was fantastic), Mike Clark (jokingly) told me that there was enough buzz around "The Jay Fields Way of Testing" that I should trademark it. I took that as a huge compliment; however, it's also not fair to the teams I've worked with. On all the project teams I am apart of, "How we test" is a collaborative decision. We don't test "The Jay Fields Way", we test our way.

So, what is "our way" as of Monday, September 3rd, 2007?

File Structure
We use RAILS_ROOT/test/unit & RAILS_ROOT/test/functional, which Rails provides by default. Under unit and functional we mirror the structure of the RAILS_ROOT/app folder.

For example, a user model that lives in

RAILS_ROOT/app/models/user.rb

would have unit tests in

RAILS_ROOT/test/unit/models/user_test.rb

and functional tests in

RAILS_ROOT/test/functional/models/user_test.rb

Tests In General
We use dust to define our tests, and we use the disallow_setup! method to ensure that setup methods aren't being added to the codebase.

Unit Tests
Unit tests are where we test classes in isolation. Within unit tests, dependencies are mocked or stubbed to ensure that a breaking test is broken because a feature of the class under test has changed. We defer testing object interactions to the functional tests. As a result, there are rarely cascading failures.

In general, there are far more unit tests than functional tests. This is a result of testing permutations, edge cases, etc within the unit test suite. If a format class contains 10 logical code paths, there should be at least 10 unit tests to verify that it works correctly; however, you may only need 1 functional test to verify that the format class works correctly with the other objects it interacts with.

Unit tests utilize a test helper specific to unit tests (RAILS_ROOT/test/unit/unit_test_helper.rb), which is the one and only require statement at the top of each unit test. In the unit_test_helper.rb we require mocha, UnitRecord and any other library that we need for unit testing. We require mocha only in unit_test_helper.rb to ensure that mocking/stubbing via mocha is only done in the unit tests. UnitRecord is a gem by Dan Manges that disables access to the database and provides the ability to easily unit test ActiveRecord::Base subclasses.

Functional Tests
Functional tests are where we verify that all the pieces of the application interact seamlessly. While stubs do exist, they are hand-coded and only stub external systems. Ideally, no stubs would exist; however, a trade-off is necessary when hitting external systems that significantly decrease your ability to quickly run the functional tests.

Functional tests often need small graphs of objects. Generally, this type of code is put in a setup method or created by fixtures. Both setup and fixtures provide a solution; however, we've found a more maintainable solution is to create a factory. Dan Manges has a great entry on our Functional Test Factory. A Factory has been very helpful; however, one thing to stress is that the Factory contains one create method per model. What's implied in that statement is that the methods are not defined on a scenario basis. If you need an Apartment that has a Renter that has a Job, you'll need to create each model within the test. Adding a method for a specific scenario (to the Factory) is the road to a completely unmaintainable Factory. We toyed with the idea of creating a fluent interface builder; however, Rails associations mostly remove the need.

For example, the following snippet can create an Apartment that has a Renter that has a job.

Factory.create_apartment(:renter => Factory.create_user(:job => Factory.create_job))

External Tests
In functional tests we stub external services; however, it's important to verify that the API of an external service hasn't changed. To perform that verification we write external services tests that interact with the external services. Since these tests are slower to run, they are generally only run on the continuous integration server or while debugging a contract change.

Why Bother?
There are several reasons for each decision we made; however, the context is the most important factor to consider. We work on large teams where the tests are run as often as possible and need to execute quickly. Those same tests also need to be as readable as possible, because it's likely that you'll be looking at tests you didn't write more often than not. While I think the above ideas are also good for small teams, I haven't personally put them to use on a small team.

Integration Tests, Selenium Tests, View Tests, aren't you missing something?
Yes, the above discussion does leave off the area of Acceptance Testing. We've yet to come up with something that works from project to project. On one project we were very successful with creating a DSL that executed as Rails integration tests locally and ran as Selenium on the build. While this might be a great solution, I haven't seen it used enough to recommend it.

View tests, in my experience, aren't necessary unless you are putting logic in your view. I prefer to keep the logic out of the view and ignore view tests entirely.

Look for updates on this entry, I'm sure I've forgotten a few things.

Labels: rails, testing