Speaking: QCon San Francisco

I'm going to be presenting on Business Natural Languages (Domain Specific Languages for Subject Matter Experts) at QCon San Francisco in November.

Are Screencasts the Future?

Before I joined ThoughtWorks I worked for Nelnet. Nelnet is a student loan products and services organization. While employed at Nelnet (around Sept 2004), I was sent to an education conference where one of the keynote speakers presented data showing that students who were born after 1981 learn differently than those born prior. The reason this data was important was because it demonstrated that students born after 1981 were far more effective at learning through watching, listening, and doing. The speakers conclusion was that the educational system needed to react by creating classes which didn't rely so heavily on simply reading a textbook. At the time I thought the information was interesting, but not particularly relevant to me.

Today, some of the more popular educational materials can be found in audio (podcasts) or audiovisual (screencasts) form. Additionally, sites like Try Ruby encourage you to give kinesthetic learning a try. After seeing the growth in each of these areas over the past few years, I think it's fair to say that the conference speaker was on to something.

I sent a draft of this entry to Geoffrey Grosenbach for review. He sent back a few more reasons why he believes in screencasts as good teaching tools.

• Efficiency of time: It takes dozens of hours to read a 300 page book straight through. But a screencast can pack the most relevant information into an hour.
• Passive/Active: You can sit back and pick up whatever elements are interesting to you. Some people say the most valuable part is learning auxiliary shell shortcuts or workflow tips.
• Graphical/Textual: I don't know any tech publisher who pays an artist to draw helpful diagrams. Screencasts are inherently graphical and it makes sense to include a few diagrams that explain the topic better than paragraphs of text would.

Here's a few links for those intersted in trying out some alternative learning tools.

• PeepCode
• Info Q (Presentations and Interviews)
• Rails Podcasts
• Railscasts

Learning Rails is Easy, Mastering Rails is Hard

There's plenty of hype and momentum around Ruby on Rails. The screencasts give quick demos of how easy it is to craft a solution using Ruby on Rails. There's even a humorous picture implying how simple it is to learn and use Rails to build web applications.

The screencasts and pictures don't lie. It is easy to learn Rails and quickly become productive. However, the dirty little secret is that it's very hard to be come an expert at utilizing Rails.

Anatomy of a Rails Web 2.0 Application
Rails provides many helpers that make your life easy. But, you can't entirely hide the fact that you'll need to be proficient with Ruby, JavaScript, YAML, and SQL. Just like Rails, getting started with any of the above languages is easy, it's mastering them that takes time and effort.

Becoming an expert with each of the above languages involves gaining knowledge and becoming proficient with several aspects of the language.

Development Environment
There's no IDE that can provide all the features you'd like for Ruby, JavaScript, YAML, and SQL. Some are (hopefully) on the way, but they still suffer from speed, feature, and stability issues. That generally means is you'll have two options:

• Find a productive development environment that specializes in one or two of them and use a different IDE/Text Editor for each language
• Find an editor that provides decent support for all of them.

As of today, I use TextMate, which I find acceptable for all 4 languages. If I couldn't use TextMate, I'd be using IntelliJ. Those are my preferences, but I suggest you try a few different options and see what works.

Language Features
Look for documentation and try out the examples for each language you are looking to master. Ruby has decent documentation on http://www.ruby-doc.org/. YAML documentation can be found on http://www.yaml.org/. JavaScript and SQL have seemingly endless documentation resources. I've always been a fan of the O'Reilly books, so I'd recommend Safari Books Online, but there's plenty of other options if you'd like.

But, don't stop with reading. Reading documentation and trying examples isn't going to bring you to the expert level. Another great way to get some insight into a language is to look at the successful frameworks and see how they are implemented. Browsing the Rails codebase should provide examples of almost every feature available in Ruby. Likewise, understanding the Prototype and Script.aculo.us libraries should greatly improve your JavaScript skills.

Testing
Testing is valuable in any language. Ruby provides a few different testing solutions, but the most popular are still Test::Unit and RSpec. JavaScript also provides several options for testing. I can't say there's a clear winner, but JsUnit and Crosscheck seem to be gaining momentum. As far as testing SQL and YAML, I generally functionally test them through Ruby. Additionally, Selenium is a great resource for acceptance testing Rails applications.

Frameworks
Both Ruby and JavaScript provide a plethora of framework options. Ruby provides RubyGems for distributing shared Ruby code. RubyGems are frameworks that can provide all kinds of building blocks for creating applications. In the past I've used the Prototype and Scriptaculous JavaScript libraries, but my current team is enjoying using JQuery.

Debugging
Debugging Ruby is a bit primitive these days. The simplest form of debugging is printing to standard out. You can roll your own simple debugging solution also. Additionally, ruby-debug looks promising. None of the solutions are great, so it's good to have all 3 tools at your disposal. For JavaScript debugging I suggest getting proficient with FireBug. SQL debugging solutions are generally vendor specific, check the docs of the database that you choose for a good suggestion. YAML doesn't generally require a debugger since most errors simply result in parsing problems.

What else makes Rails difficult to master?
Mastering languages is important, but it's not enough to run a high traffic Rails application. You'll still need to understand performance optimization and deployment strategies. There's an abundance of Rails production heuristics available, but little of it is well documented. I'd suggest reading the following blogs: Brainspl.at, Err the Blog, and RailsExpress. Beyond those blogs the web provides several other resources. A Google search should provide more than enough reference material.

That's a ton to learn
It's true, that is a lot of information to digest. But, experts aren't made overnight. The good news is, you can get started very easily and expand your knowledge as needed. There is a large amount of information available, but don't feel overwhelmed: The majority of the information is very straightforward. Becoming an expert with Rails takes time more than anything else.

Ruby: Trivial Debugger Implementation

A common question you receive as a Ruby developer is: Don't you miss robust debugging support? The short answer is No, but that's not entirely true. The reality is that Ruby developers are generally willing to make trade-offs in order to work with a language that they find more pleasing.

Do I wish I had a great IDE with fantastic debugging support? Of course I do, and I'd also like refactoring support while we are daydreaming. Until that day comes, I'll continue to look for other ways to be effective.

To mitigate the debugging issue the Ruby developers I work with end up relying very heavily on their tests; therefore, debugging becomes less of an issue. Unfortunately, even the best testers can't always isolate complicated issues. In these circumstances the best Ruby developers are often forced to use the age old debugging strategy of printing values. I was recently in one of these situations and I used the following statement in lieu of the typical p statement.

loop do
  p eval(gets)
end

I do miss mature tools and infrastructure. However, the ease in which I can create adequate solutions in Ruby makes it hard to consider doing anything else, currently.

Semi-bluffing your interview

Over the past two and a half years I've interviewed several ThoughtWorks candidates. These days I tend to put people into 3 categories when interviewing them: Gross Exaggerator, WYSIWYG, and the Semi-bluffer.

The Gross Exaggerator
Being a Gross exaggerator is never a good idea. Your resume is the first clue. I've yet to meet anyone who's an expert* with .net, Java, Ruby, Smalltalk, and Lisp. There's just not enough hours in the day to actually be an expert in each language. Anyone with several buzzwords on their resume signals two things to me: They likely have no idea what they don't know, and they aren't picky enough about technologies they are willing to work with.

I took C in college also, but one semester of Data Structures does not an expert make. I did Java at my first job, but I have no desire to program in it now, so what's the point of listing it as one of my skills? I'm not saying that my resume doesn't contain the word Java. In fact, it does state in my first job description that we used Java, but my skills list is not a summary of what I've done in my past. My skills list is the list of skills that I've kept up with and am interested in continuing to keep up with. A bloated skills list indicates that you think you know more than you do, or you haven't spent the time to be and expert in anything; both situations look bad.

Being a Gross Exaggerator in an interview is painful. You generally have a bit of knowledge of several things, but almost no in depth knowledge of anything. Most questions quickly expose this. Gross Exaggerators are fairly easy to spot, and I always pass on hiring them.

WYSIWYG (What You See Is What You Get)
It's very easy to be a WYSIWYG candidate, because you are very clear about what you know and what you don't. A WYSIWYG candidate never gives a partial explanation; they either tell you every painful detail, or claim to know little or nothing about the topic.

Hiring a WYSIWYG is an easy choice, but I don't think being a WYSIWYG is the best option. The problem with being a WYSIWYG is that there are several Gross Exaggerators and some Semi-bluffers that will be competing with you. Someone who claims to have the knowledge you do, plus more, will always be an attractive choice for a potential employer.

The Semi-bluffer

In [poker] games with multiple betting rounds, to bluff on one round with an inferior or drawing hand that might improve in a later round is called a semi-bluff. A player making a semi-bluff can win the pot two different ways: by all opponents folding immediately or by catching a card to improve the player's hand. --Wikipedia

Semi-bluff is a term that I picked up playing poker. A few of the things I love about poker is that you don't know what the competition has, you don't know what will be needed in the end to win, and you probably don't know if you are currently winning or losing. Of course, each of those things can be said about almost any interview.

I think Semi-bluffing interviewees generally do the best because they demonstrate what they know, suggest that they know more, but admit what they don't know when directly challenged. In fact, I believe the above quote could be rewritten as the following to describe Semi-bluffing in an interview.

Almost all interviews involve a series of questions, to give a vague answer to one question when you don't know enough details can be called a semi-bluff. An interviewee making a semi-bluff can pass that round of questioning two different ways: if the interviewer doesn't request more information or if the follow up question happens to hit one aspect where you do understand the details. A semi-bluff can also be a success if the interviewee admits to not knowing the answer at the moment, but later in the interview process demonstrates that the desired knowledge has been acquired.

An example of a Semi-bluff could be demonstrated as the following.
Interviewer: Do you have experience with deploying Rails applications.
Interviewee: On my current project we use Capistrano and deploy to RHEL. The Webserver is Nginx fronting a Mongrel cluster.

Notice that the interviewee hasn't stated that he did any of that work, instead he's simply listed his current project's deployment plan. The questions may end right there, but if they don't you can easily follow up with: Truthfully, I've been focused on other parts of the application so I only have a high level view of our deployment. However, I believe that it would be very easy for me to get into the details of your specific deployment if I am hired. Following the interview, the interviewee should then dig into the details of their current deployment and send answers via email to the interviewer. Obviously, the faster the email is sent, the better.

A Semi-bluff in poker, and while interviewing, takes a great deal of finesse. Coming off as a WYSIWYG while Semi-bluffing is safe, but coming off as a Gross Exaggerator can cost you a job. Therefore, it's obvious which way you should lean if you find Semi-bluffing isn't working out for you.

Try Semi-bluffing a question in your next interview. It can be a great rush, but be sure to admit when you don't know something. Also be careful about choosing which question to Semi-bluff. A direct and detailed question isn't a good candidate for a Semi-bluff. When someone knows they have "the nuts" a bluff is an (often embarrassing) waste.

* I define expert as someone who is knowledgeable concerning: syntax, development environment, advanced language features, how and what to test, framework choices, details of several frameworks of the language, debugging, performance optimization, deployment strategies, and pattern implementation specific to the language.

Testing: Frameworks are evolving

Test-Driven Development was introduced to me by way of xUnit frameworks.

In the beginning, all xUnit frameworks looked very much the same. All tests lived within a class that inherited from an abstract test case class, and all tests were defined by creating a method that begin with the word test. However, as time progressed the xUnit frameworks began to take advantage of features specific to individual languages.

Martin Fowler's bliki entry Xunit contains a description of how NUnit matured.

The first version of NUnit even had a "isVAforJava" method which originated in special handling for Visual Age for Java. Others have been more sophisticated: NUnit 2.0 was praised by Anders Heljsberg for its use of attributes in C#.

Utilizing language specific features was the first step; however, as testing methods mature it's becoming more clear that the features of xUnit also need to mature.

The first framework I noticed that decided to start breaking rules was RSpec. The 1.0 version of RSpec provided a new syntax for defining test cases and for defining assertions. The following example shows how you would write the same test in Test::Unit (a traditional xUnit framework) and RSpec.

# RSpec 
describe Bowling do
  before(:each) do
    @bowling = Bowling.new
  end

  it "should score 0 for gutter game" do
    20.times { @bowling.hit(0) }
    @bowling.score.should == 0
  end
end

# Test::Unit
class BowlingTest < Test::Unit::TestCase
  def setup
    @bowling = Bowling.new
  end
  
  def testShouldScoreZeroForGutterGame
    20.times { @bowling.hit(0) }
    assert_equal 0, @bowling.score
  end
end

I do believe that the RSpec version is more semantically pleasant. However, given a programmer who practices TDD, when they maintain tests they use an xUnit clone, then I'm not sure the semantic benefit is worth learning a new framework. At least, I wasn't at first.

Another new xUnit framework is xUnit.net, announced recently on James Newkirk's blog. xUnit.net excites me more than any other piece of software that I'm likely to never use. I'll probably never use it since I don't plan on going back to .net development, but the features are exciting nonetheless. James et al have taken their test-driven development experiences and created a framework that guides them towards writing better tests. James' entry gives the full details (which I suggest reading even if you never plan on doing any .net work), but a few features I like are the removal of setup and teardown, and the aspect-like functionality. You may be less impressed since Ruby provides aspect-like functionality very easily, but I believe it's nice to see a xUnit framework built with extension points in mind.

These new frameworks are exciting because they are incorporating lessons learned from the past few years of practicing test-driven development. As our experience with TDD grows, so should our tools.

Rails: Where do you require?

I was recently looking over a codebase and noticed that several different files contained require statements. I'm not talking about the require statement that each test file has, I'm taking about seeing require 'ostruct', require 'enumerator' or require 'soap4r' in numerous class definitions.

For the last few projects I've been following the convention of putting all the require statements in environment.rb. There are more efficient strategies, but I find this to be the most maintainable of the solutions I've tried.

While following the above strategy I haven't run into any issues with two libraries defining behavior on differing classes that are named the same, but I have heard concerns from team members on the time it takes to require each library. I think that's a valid concern and I was planning on addressing it by writing a method that takes an array of strings and requires them one at a time. The new method would also benchmark each require and print a warning if the require is taking an unacceptable amount of time.

Please drop me a line (or blog it and leave a link) in the comments if you have a strategy that's worked well for you.

RailsConf Europe 07: Presenter Links

Here are the resources relevant to my RailsConf Europe 07 presentation: Extending Rails to use the Presenter Pattern.

The Presentation: http://www.jayfields.com/images/RailsConfEurope2007.pdf

My entries (in order)

• Model View Controller + Presenter?
• Another Rails Presenter Example
• Presenters - An additional layer alternative
• Rails Presenter Pattern

Jamis Buck

• Skinny Controller, Fat Model
• Moving Associated Creations to the Model

Courtenay

• Simple Presenters

Uninterested Pair

Sometimes your pair is uninterested in the task at hand. It's never fun for either person when working with an Uninterested Pair.

There are several reasons that a pair can be uninterested, but that isn't going to be the focus of this entry. Addressing why a pair is uninterested is a people question that I'm not interested in addressing in a general way.

A benefit of pair programming is getting the best solution when two developers collaborate. Clearly, an uninterested developer is not collaborating, thus the solution is likely inferior. At a minimum, the solution is as good as it would have been and 1 person's time was completely wasted.

The problem with uninterested pairs is that they can spend an entire day paying little attention and contributing almost nothing. If you find yourself stuck with an uninterested pair there are a few things that can help. Dealing with an uninterested pair is very similar to dealing with a Distracted Pair. Ping Pong Pair programming helps turn an uninterested pair into a contributing pair; however, often their level of contribution isn't up to their full potential. I've found that when dealing with an uninterested pair it's actually more beneficial to feign confusion.

Feigning confusion can be boring, really boring. You generally have a grasp on what needs to be done. You also generally have the solution in your head. You can implement this solution on your own and bring the uninterested pair along for the ride; unfortunately, in my experience that's exactly what happens. But, simply taking the uninterested pair along for the ride isn't the best long term solution. That pair may someday be asked to improve the existing solution. If they can't, because they weren't paying attention, you may be taken away from your task. Worse, you may be off the team and the context of the code is permanently lost.

So, feigning confusion is boring, but it is also in the best interest of the team. Instead of implementing the solution, ask your pair: How we are going to get this done? Don't accept their first answer. Their first answer is likely going to be fast, but incredibly ugly. Provide your own input, but don't let on that you have a solution. Instead, stick to asking questions. Chances are, you are going to understand their solution. It doesn't matter if you understand their solution, pretend that you don't. Force the uninterested pair to write the first 3 tests and implement the code that causes the tests to pass. At that point, take over and write a test, but force them to implement the solution. Then slowly proceed into Ping Pong Pair programming, but don't jump straight into it. Instead, force the uninterested pair to do about 65% of the coding. It shouldn't take long for the uninterested pair to be a fully contributing pair, which benefits everyone.

Distracted Pair

Ever since I wrote Spellchecking Pair I've been noticing other patterns concerning pairing. Distracted Pair, as the name implies, is a pair who is easily distracted by their surroundings.

Distracted pairs tend to spend less time typing, because they are always busy helping everyone around them. Distracted pairs are not useless; in fact they generally contribute significantly to the code the pair is working on. However, distracted pairs are rarely contributing at their full potential because they are generally attempting to solve the problems of multiple pairs at the same time. Some distracted pairs can also spend too much time checking email and doing other ancillary tasks.

Joel believes that open space working areas are fun but not productive. While I don't agree with him, I can see how a few distracted pairs could easily make a manager draw that conclusion.

I've been a distracted pair. I've also worked with several distracted pairs. I believe that it happens to all of us from time to time. In the past I've seen distracted pairs managed by creating rules. For example, a "no personal laptops in the development room" rule existed on a previous project. This rule can help address the problem, but it creates other problems.

My preferred method of handling a distracted pair is to practice Ping Pong Pairing. Ping Pong Pairing forces both pairs to be paying attention at level one. Another nice attribute of this suggestion is that it doesn't require any additional rules that could have problematic side effects.

I'm led to believe that Joel doesn't believe in Pair Programming either, so my solution may be worthless to him. However, if you are working in an environment where Pair Programming is embraced, I suggest using Ping Pong Pairing to get the most out of working with a Distracted Pair.

Ruby: Array#collect_with_remaining

While looking through our core extensions I found the Array#collect_with_remaining method. The method was designed to yield each element with the other elements of an array and collect the results.

This method (wasn't, but) could be used to find duplicates within an Array*. I would suggest not using it for finding duplicates (since there are other easier ways), but the example should help in understanding what collect_with_remaining can do. If you wanted to implement Array#duplicates using collect_with_remaining you could write the following method.

class Array  
  def duplicates
    block = lambda { |element, remaining| element if remaining.include?(element) }
    self.collect_with_remaining(&block).uniq.compact
  end
end

The duplicates method shows that Array#collect_with_remaining yields each element and the remaining elements of the array, then collects the results. The result is a list of the duplicate values when the remaining did contain the element, and nil for each element that was not a duplicate. After the results have been returned the array is uniq(ed) and compact(ed) to remove the nil and duplicate entries. The result is all the duplicate values from the array. The following test verifies that the duplicates method works as expected.

unit_tests do
  test "return duplicates" do
    assert_equal [1, 3], [1, 2, 3, 1, 4, 3].duplicates
  end
end

On to the actual collect_with_remaining implementation, right after we write the tests.

unit_tests do
  test "head and tail are collected first" do
    assert_equal [1, [2, 3]], [1, 2, 3].collect_with_remaining { |element, remaining| [element, remaining] }.first
  end

  test "elements and remaining elements are collected" do
    expected = [[1, [2, 3]], [2, [1, 3]], [3, [1, 2]]]
    assert_equal expected, [1, 2, 3].collect_with_remaining { |element, remaining| [element, remaining] }
  end
end

The implementation I chose is straightforward, assuming you are comfortable with Enumerable#enum_with_index.

class Array
  def collect_with_remaining
    self.enum_with_index.inject([]) do |result, (element, index)|
      remaining = self.dup
      remaining.delete_at(index)
      result << yield(element, remaining)
    end
  end
end

And, here's all the code if you're interested.

require 'enumerator'
require 'rubygems'
require 'test/unit'
require 'dust'

class Array
  def collect_with_remaining
    self.enum_with_index.inject([]) do |result, (element, index)|
      remaining = self.dup
      remaining.delete_at(index)
      result << yield(element, remaining)
    end
  end

  def duplicates
    block = lambda { |element, remaining| element if remaining.include?(element) }
    self.collect_with_remaining(&block).uniq.compact
  end
end

unit_tests do
  test "head and tail are collected first" do
    assert_equal [1, [2, 3]], [1, 2, 3].collect_with_remaining { |element, remaining| [element, remaining] }.first
  end

  test "elements and remaining elements are collected" do
    expected = [[1, [2, 3]], [2, [1, 3]], [3, [1, 2]]]
    assert_equal expected, [1, 2, 3].collect_with_remaining { |element, remaining| [element, remaining] }
  end

  test "return duplicates" do
    assert_equal [1, 3], [1, 2, 3, 1, 4, 3].duplicates
  end
end

As always, please post alternate suggestions in comments, or better yet, blog them and leave a comment with the link.

* If I actually wanted to check for duplicates I would use Array#duplicates, which is defined in Facets.

Ruby: Array#chunk

Yesterday I was working on some code that needed to split an array into 3 different chunks. My pair, Philippe Hanrigou, wrote tests similar to the following.

unit_tests do
  test "chunk evenly" do
    assert_equal [[1], [2], [3]], [1, 2, 3].chunk(3)
  end

  test "divide evenly" do
    assert_equal [[1], [2], [3]], [1, 2, 3] / 3
  end
  
  test "chunk unevenly" do
    assert_equal [[1, 3], [2]], [1, 2, 3].chunk(2)
  end
  
  test "chunk creates empty array when there aren't enough elements in the array" do
    assert_equal [[1], [], []], [1].chunk(3)
  end
end

To make the above tests pass, I wrote the following code.

class Array
  def chunk(number_of_chunks)
    chunks = (1..number_of_chunks).collect { [] }
    while self.any?
      chunks.each do |a_chunk|
        a_chunk << self.shift if self.any?
      end
    end
    chunks
  end
  alias / chunk
end

The above code works, but I'm curious to see if there is a better solution. How would you make the tests pass?

Here's the code in it's entirety.

require 'rubygems'
require 'test/unit'
require 'dust'

class Array
  def chunk(number_of_chunks)
    chunks = (1..number_of_chunks).collect { [] }
    while self.any?
      chunks.each do |a_chunk|
        a_chunk << self.shift if self.any?
      end
    end
    chunks
  end
  alias / chunk
end

unit_tests do
  test "chunk evenly" do
    assert_equal [[1], [2], [3]], [1, 2, 3].chunk(3)
  end

  test "divide evenly" do
    assert_equal [[1], [2], [3]], [1, 2, 3] / 3
  end
  
  test "chunk unevenly" do
    assert_equal [[1, 3], [2]], [1, 2, 3].chunk(2)
  end
  
  test "chunk creates empty array when there aren't enough elements in the array" do
    assert_equal [[1], [], []], [1].chunk(3)
  end
end