Life After Pair Programming

When I first joined DRW I noticed that the vast majority of developers were not pair-programming. I was surprised that a company that employed so many smart people would just simply disregard a practice that I considered to be so obviously beneficial.

note: it's so easy to judge people who don't pair-program, isn't it? You can write-off their behavior for so many reasons:

• They simply haven't done it enough to see how much value it provides.
  
• They've used the wrong setup, and turned a positive ROI into a negative ROI situation in the past.
  
• They must be anti-social or not team players.
  
• They must guard their code because they are too proud or believe it's job security.

Never once did I consider that their behavior might be correct based on their context. There was no context in which pair-programming wasn't the right choice, right?

So, I did what any responsible employee who wants to help would do: I gave a presentation. I took pictures of a traditional (read: broken) pairing station setup - the setup where one employee looked over the shoulder of the person doing all the work. Then I took a picture of a dual monitor, dual keyboard, and dual mouse setup. I made the distinction between pairing and watching someone work. I explained how ping-pong-pair-programming worked. I think I said the right things, and the presentation was well received. Management was on board and offered to setup pairing stations for any developers in the company that wanted to give it a try.

A few people embraced it, but most went back to their traditional habits. I did manage to convert the guys on my team, and we paired the vast majority of the time.

Almost 3 years later, and on a different team, I basically never pair. In fact, when my lone teammate (and boss) asks if we should pair on something my initial reaction is always 'no'. The 2 of us are the only people working on the applications we support, and when we pair I often find myself very distracted and not really participating. Sometimes we ping-pong-pair and it keeps us both involved, but I don't feel like we're going faster than if I'd been working solo. Don't get me wrong, we sit directly next to each-other, talk constantly, and even look at code together at times when we're just starting to talk about a new problem. That level of collaboration feels beneficial, but actually coding the implementation together feels wasteful.

I know how I felt 3 years ago (when I joined DRW), and I know how I feel now. The fact that I've changed my tune so drastically made me think it was worth putting a few ideas on the web. I was also motivated a bit by the recent publication of Pair Programming Benefits. The Pair Programming Benefits article actually gives me a perfect starting place, by examining the benefits of pairing and how they apply to my context.

I feel like I can safely summarize the Team and System Benefits as: transfer of knowledge, superior quality, and collaboration. The Programmer Benefits can be summarized as: expansion of knowledge, truck factor reduction, superior quality, and team support. The Management/Project Management Benefits can be summarized as: aiding addition and removal of employees, knowledge transfer, and the ability to hire any skilled generalist.

So, our full list of benefits is: expansion and transfer of knowledge, superior quality, collaboration, truck factor reduction, team support, eased hiring and firing. I would agree with these benefits of pair-programming, which explains why I was such an avid believer in pairing in the past. However, if those pros don't apply to my situation then pair-programming becomes nothing more than using twice as many man hours to get the same amount of work done.

I'll start by addressing the easiest "benefits" that are effectively irrelevant for my team: eased hiring and firing and truck factor. Like I said before, my team size is 2. Mike and I are both very happy and there's almost no chance that either of us leave anytime soon. Even if we did choose to leave, we would give enough notice that someone could easily come in and replace us - and either one of us could support the applications on our own if necessary. The only real risk is that we both are killed at the same time - at which point it wouldn't matter if we paired or not. Statistically it's much more likely that only one of us would suffer something causing us to not be able to return to work, and as Chris Zuehlke (manager at DRW) once pointed out to me: if you build small, well written processes the time to understand or rewrite any software you know nothing about is small - often smaller than the cost of keeping 2 people informed on the details of the process. If I asked the stakeholders if we should use 2x as many man hours to ensure that we're covered in case one of us is suddenly killed they'd laugh at me - and I'd deserve it.

We're also not looking to add to our team, and if we decided we did want to grow I expect we wouldn't have any trouble finding someone who was looking for a well-paying job in manhattan using very cool technology.

The previous discussion was meant to address team size, but it also touches on "transfer of knowledge". Transfer of knowledge is valuable for teaching new techniques and enabling people to do maintenance. However, if little teaching is going on (often the case as Mike and I add features to an app we know fairly well, using technologies we know fairly well) and we both tend to do the maintenance on the code we previously wrote then "transfer of knowledge" is either not occurring anyway or is unnecessary. Also, on more than one occasion I've had to take a first look at some code Mike had recently written. It's always been straightforward and taken no more than an hour to dig-in, make, and test a change. Had we paired on everything he worked on I would have unquestionably spent far more than an hour here or there. Again, the ROI on knowledge transfer doesn't look good for our situation.

The "superior quality" benefit is an interesting one. I consider myself to be a good engineer who on occasion writes something beautiful. I like to share what's beautiful with everyone. This is something that's going to happen whether or not I pair (evidenced by my blogging). I know Mike does the same. I expect we're all proud of our most elegant solutions. However, more often than not I write fairly good software that isn't excitement provoking, but solves the problem at hand. I'm not sure anyone is benefiting from working with me at those moments. And, while it's possible that we'd get a slightly better solution if Mike & I paired, spending twice as many man hours for a 10% better solution doesn't seem like a great choice in the short term. In the long term those 10%s could add up, but what if I see the superior solution on my 2nd pass through and end up there while expanding a feature anyway? If I have a massive solution that might be a big cost. However, with small, focused processes I can tend to make a massive change to a process in ~4 hours. Delete is still my favorite refactoring and I do it as often as possible on my quests for superior solutions. Therefore, I don't believe those 10%s add up, I believe the 90% superior solution is often good enough or deleted.

I believe a few additional factors allow us to write code that is "good enough" on our own.

• The application is fairly mature and many of the core design decisions have already been made.
• Mike and I have Compatible Opinions on Software.
• Mike and I are at a similar skill level.

In my current situation I find that many of the tough decisions have already been made and I can follow the existing path. And, any decisions that I do need to make I usually end up choosing what Mike would choose and solving in a way that we both think is appropriate.

While I believe "expansion of knowledge" is often undervalued and under-practiced in our profession, I don't think pair-programming is the most effective way to expand your skills. Blogs, books, and presentations allow you to work at your own pace and dig in to your desired level. My experience pairing with people using tools or languages I don't know have been painful. I did learn things, but I couldn't help but feel like there were much more effective ways for me to get up to speed.

The final 2 benefits that I need to consider are "collaboration" and "team support". As I previously stated, Mike and I collaborate quite closely. We're always both happy to stop what we're doing to provide an opinion or direction. I honestly can't imagine a more collaborative environment. Basically the same thing can be said of "team support" - we constantly watch each-other's back. We're both looking to succeed and we know we need each-other to get to that goal, so it only makes sense that we look out for each other. We don't need pairing to force us to support each-other, the desire to succeed is more than enough.

When considering my context and the "benefits" of (or, in my situation, lack of benefits of) pair-programming, I'm no longer surprised that I basically never want to pair these days. Also, looking back at the other teams I was previously judging - their context back then looks a lot like my current context. With a bit more experience under my belt I have to say I'm honestly a bit embarrassed for never even considering that they already "got it" and were already making the right choice. Oh well, live and learn.

While it's easy for me to examine my situation and make a determination I think there is a more general context where these same ideas apply. The aspects of an environment where pairing may not be beneficial seem to look something like this:

• small team (< 4 people)
• small software (components/processes can be rewritten in < 2 days)
• motivated teammates
• talented and well-seasoned teammates (similar skill level, and skilled overall)
• the design of the application is fairly mature and stable
• all team members have compatible options on software

Thanks to Michael Feathers, Alexey Verkhovsky, Pat Kua, Jeremy Lightsmith, Lance Walton, Dan Worthington-Bodart, James Richardson, and John Hume for providing feedback on the initial draft of this entry.

Clojure: Check For nil In a List

The every? function in Clojure is very helpful for determining if every element of a list passes a predicate. From the docs:

Usage: (every? pred coll)

Returns true if (pred x) is logical true for every x in coll, else false.

The usage of every? is very straightforward, but a quick REPL session is always nice to verify our assumptions.

Clojure 1.2.0

user=> (every? nil? [nil nil nil])

true

user=> (every? nil? [nil 1])      

false

As expected, every? works well when you know exactly what predicate you need to use.

Yesterday, I was working on some code that included checking for nil - similar to the example below.

user=> (def front 1)

#'user/front

user=> (def back 2)

#'user/back

user=> (when (and front back) [front back])        

[1 2]

This code works perfectly if you have the individual elements "front" and "back", but as the code evolved I ended up representing "front" and "back" simply as a list of elements. Changing to a list required a way to verify that each entry in the list was not nil.

I was 99% sure that "and" was a macro; therefore, combining it with apply wasn't an option. A quick REPL reference verified my suspicion.

user=> (def legs [front back])               

#'user/legs

user=> (when (apply and legs) legs)      

java.lang.Exception: Can't take value of a macro: #'clojure.core/and (NO_SOURCE_FILE:8)

Several other options came to mind, an anonymous function that checked for (not (nil? %)), map the values to (not (nil? %)) and use every? with true?; however, because of Clojure's truthiness the identity function is really all you need. The following REPL session shows how identity works perfectly as our predicate for this example.

(when (every? identity legs) legs)

[1 2]

For a few more looks at behavior, here's a few examples that include nil and false.

user=> (every? identity [1 2 3 4])

true

user=> (every? identity [1 2 nil 4])

false

user=> (every? identity [1 false 4])

false

As you can see, using identity will cause every? to fail if any element is falsey (nil or false). In my case the elements are integers or nil, so this works perfectly; however, it's worth noting so you don't see unexpected results if booleans ever end up in your list.

Clojure: partition-by, split-with, group-by, and juxt

Today I ran into a common situation: I needed to split a list into 2 sublists - elements that passed a predicate and elements that failed a predicate. I'm sure I've run into this problem several times, but it's been awhile and I'd forgotten what options were available to me. A quick look at http://clojure.github.com/clojure/ reveals several potential functions: partition-by, split-with, and group-by.

partition-by
From the docs:

Usage: (partition-by f coll)

Applies f to each value in coll, splitting it each time f returns
a new value. Returns a lazy seq of partitions.

Let's assume we have a collection of ints and we want to split them into a list of evens and a list of odds. The following REPL session shows the result of calling partition-by with our list of ints.

user=> (partition-by even? [1 2 4 3 5 6])

((1) (2 4) (3 5) (6))

The partition-by function works as described; unfortunately, it's not exactly what I'm looking for. I need a function that returns ((1 3 5) (2 4 6)).

split-with
From the docs:

Usage: (split-with pred coll)

Returns a vector of [(take-while pred coll) (drop-while pred coll)]

The split-with function sounds promising, but a quick REPL session shows it's not what we're looking for.

user=> (split-with even? [1 2 4 3 5 6])

[() (1 2 4 3 5 6)]

As the docs state, the collection is split on the first item that fails the predicate - (even? 1).

group-by
From the docs:

Usage: (group-by f coll)

Returns a map of the elements of coll keyed by the result of f on each element. The value at each key will be a vector of the corresponding elements, in the order they appeared in coll.

The group-by function works, but it gives us a bit more than we're looking for.

user=> (group-by even? [1 2 4 3 5 6])

{false [1 3 5], true [2 4 6]}

The result as a map isn't exactly what we desire, but using a bit of destructuring allows us to grab the values we're looking for.

user=> (let [{evens true odds false} (group-by even? [1 2 4 3 5 6])]

[evens odds])

[[2 4 6] [1 3 5]]

The group-by results mixed with destructuring do the trick, but there's another option.

juxt
From the docs:

Usage: (juxt f)
              (juxt f g)
              (juxt f g h)
              (juxt f g h & fs)

Alpha - name subject to change.
Takes a set of functions and returns a fn that is the juxtaposition
of those fns. The returned fn takes a variable number of args, and
returns a vector containing the result of applying each fn to the
args (left-to-right).
((juxt a b c) x) => [(a x) (b x) (c x)]

The first time I ran into juxt I found it a bit intimidating. I couldn't tell you why, but if you feel the same way - don't feel bad. It turns out, juxt is exactly what we're looking for. The following REPL session shows how to combine juxt with filter and remove to produce the desired results.

user=> ((juxt filter remove) even? [1 2 4 3 5 6])

[(2 4 6) (1 3 5)]

There's one catch to using juxt in this way, the entire list is processed with filter and remove. In general this is acceptable; however, it's something worth considering when writing performance sensitive code.

Clojure: Apply a Function To Each Value of a Map

I recently needed to update all of the values of map, and couldn't find an individual function in clojure.core that did the trick. Luckily, implementing an update-values function is actually very straightforward. The following function reduces the original map to a new map with the same elements, except the values are all the result of applying the specified function and any additional args.

(defn update-values [m f & args]

 (reduce (fn [r [k v]] (assoc r k (apply f v args))) {} m))

The code is concise, but perhaps a bit terse. Still, it does the trick, as the REPL session below demonstrates.

Clojure 1.2.0

user=> (defn update-values [m f & args]

 (reduce (fn [r [k v]] (assoc r k (apply f v args))) {} m))

#'user/update-values

user=> (update-values {:a 1 :b 2 :c 3} inc)

{:c 4, :b 3, :a 2}

user=> (update-values {:a 1 :b 2 :c 3} + 10)

{:c 13, :b 12, :a 11}

user=> (update-values {:a {:z 1} :b {:z 1} :c {:z 1}} dissoc :z)

{:c {}, :b {}, :a {}}

The last example is the specific problem I was looking to solve: remove a key-value pair from all the values of a map. However, the map I was working with actually had a bit of nesting. Let's define an example map that is similar to what I was actually working with.

user=> (def data {:boxes {"jay" {:linux 2 :win 1} "mike" {:linux 2 :win 2}}})

#'user/data

Easy enough, I have 2 linux boxes and 1 windows box, and Mike has 2 linux and 2 windows. But, now the company decides to discard all of our windows boxes, and we'll need to update each user. A quick combo of update-in with update-values does the trick.

user=> (update-in data [:boxes] update-values dissoc :win)

{:boxes {"mike" {:linux 2}, "jay" {:linux 2}}}

As you can see, the update-values function plays nice with existing Clojure functions as well.

Disclosure: I am long AAPL and own every device Apple puts out. Don't take my example numbers to literally. Also, much to my dismay, DRW has yet to discard all of our windows boxes.

Clojure: memfn

The other day I stumbled upon Clojure's memfn macro.

The memfn macro expands into code that creates a fn that expects to be passed an object and any args and calls the named instance method on the object passing the args. Use when you want to treat a Java method as a first-class fn.

(map (memfn charAt i) ["fred" "ethel" "lucy"] [1 2 3])
-> (\r \h \y)

-- clojure.org

At first glance it appeared to be something nice, but even the documentation states that "...it is almost always preferable to do this directly now..." - with an anonymous function.

(map #(.charAt %1 %2) ["fred" "ethel" "lucy"] [1 2 3])
-> (\r \h \y)

-- clojure.org, again

I pondered memfn. If it's almost always preferable to use an anonymous function, when is it preferable to use memfn? Nothing came to mind, so I moved on and never really gave memfn another thought.

Then the day came where I needed to test some Clojure code that called some very ugly and complex Java.

In production we have an object that is created in Java and passed directly to Clojure. Interacting with this object is easy (in production); however, creating an instance of that class (while testing) is an entirely different task. My interaction with the instance is minimal, only one method call, but it's an important method call. It needs to work perfectly today and every day forward.

I tried to construct the object myself. I wanted to test my interaction with this object from Clojure, but creating an instance turned out to be quite a significant task. After failing to easily create an instance after 15 minutes I decided to see if memfn could provide a solution. I'd never actually used memfn, but the documentation seemed promising.

In order to verify the behavior I was looking for, all I'll I needed was a function that I could rebind to return an expected value. The memfn macro provided exactly what I needed.

As a (contrived) example, let's assume you want to create a new order with a sequence id generated by incrementAndGet on AtomicLong. In production you'll use an actual AtomicLong and you might see something like the example below.

(def sequence-generator (AtomicLong.))
(defn new-order []
  (hash-map :id (.incrementAndGet sequence-generator)))

(println (new-order)) ; => {:id 1}
(println (new-order)) ; => {:id 2}

While that might be exactly what you need in production, it's generally preferable to use something more explicit while testing. I haven't found an easy way to rebind a Java method (.incrementAndGet in our example); however, if I use memfn I can create a first-class function that is easily rebound.

(def sequence-generator (AtomicLong.))
(def inc&get (memfn incrementAndGet))
(defn new-order []
  (hash-map :id (inc&get sequence-generator)))

(println (new-order)) ; => {:id 1}
(println (new-order)) ; => {:id 2}

At this point we can see that memfn is calling our AtomicLong and our results haven't been altered in anyway. The final example shows a version that uses binding to ensure that inc&get always returns 10.

(def sequence-generator (AtomicLong.))
(def inc&get (memfn incrementAndGet))
(defn new-order []
  (hash-map :id (inc&get sequence-generator)))

(println (new-order)) ; => 1
(println (new-order)) ; => 2
(binding [inc&get (fn [_] 10)]
  (println (new-order)) ; => 10
  (println (new-order))) ; => 10

With inc&get being constant, we can now easily test our new-order function.